Prepare Observation Data
Choose source: Manual CSV, GRQA database extraction, or Copernicus synthetic generation
OpenWQ
Model Calibration
A comprehensive guide to calibrating OpenWQ models
β sensitivity analysis, optimization algorithms, and best practices
β sensitivity analysis, optimization algorithms, and best practices
Contents
β’ Calibration Framework Overview
β’ Parameter Categories (106+)
Section 01: Sensitivity Analysis
β’ Morris Screening
β’ Sobol Sensitivity Analysis
Section 02: DDS Optimization
β’ DDS Algorithm
β’ Objective Functions
β’ Temporal Resolution
Section 03: Calibration Workflow
β’ Step-by-Step Workflow
β’ Observation Data Sources
β’ GRQA Database Integration
β’ CSV Format
Section 04: Implementation
β’ Priority-Based Calibration
β’ Running Calibration
β’ Calibration Output
β’ Best Practices
Section 05: Post-Calibration Reporting
β’ Interactive HTML Reports
β’ Basin Maps & Spatial Analysis
β’ Multi-Variant Basin Reports
OpenWQ
2
Calibration Framework Overview
OpenWQ includes a comprehensive calibration framework with:
- 106+ calibratable parameters
- DDS optimization algorithm
- Morris screening for sensitivity
- Sobol analysis for detailed SA
- Docker/Apptainer deployment
- Checkpoint/restart capability
βββββββββββββββββββββββββββββββββββββββ
β Calibration Workflow β
βββββββββββββββββββββββββββββββββββββββ€
β β
β βββββββββββββββββββββββββββββββ β
β β 1. Morris Screening (~200) β β
β β β Identify sensitive β β
β ββββββββββββββββ¬βββββββββββββββ β
β βΌ β
β βββββββββββββββββββββββββββββββ β
β β 2. Sobol Analysis (~1000) β β
β β β Rank top parameters β β
β ββββββββββββββββ¬βββββββββββββββ β
β βΌ β
β βββββββββββββββββββββββββββββββ β
β β 3. DDS Optimization (~300) β β
β β β Optimize top 10-15 β β
β ββββββββββββββββ¬βββββββββββββββ β
β βΌ β
β βββββββββββββββββββββββββββββββ β
β β 4. Validation β β
β β β Independent period β β
β βββββββββββββββββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββ
OpenWQ
3
Parameter Categories (106+ total)
| Category | Count | Examples | Typical Range |
|---|---|---|---|
| BGC (NATIVE_BGC_FLEX) | 26 | k_nitrification, k_denitrification, theta | 0.001 - 1.0 /day |
| PHREEQC | 22 | Initial concentrations, pCO2, SI | Varies |
| Sorption | 16 | Kfr, qmax, KL, bulk_density | 0.01 - 100 L/kg |
| Sediment Transport | 14 | erosion_index, erodibility, cohesion | 0.1 - 10 |
| Transport | 5 | dispersion_x/y/z, characteristic_length | 0.1 - 100 mΒ²/s |
| Lateral Exchange | 4 | k_exchange (river-soil, soil-GW) | 0.0001 - 0.1 /s |
| Source/Sink | 21 | Load scaling, export coefficients | 0.1 - 5.0Γ |
Key Insight: Source/sink parameters are typically THE MOST SENSITIVE β start calibration there!
OpenWQ
4
01
Sensitivity Analysis
Morris Screening (Elementary Effects)
Efficient method to identify influential parameters with minimal model runs.
Cost: O(r Γ (k+1))
r = 10-20 trajectories, k = parameters
Example: 30 params Γ 15 trajectories = ~465 runs
Metrics
| ΞΌ* (mu-star) | Mean absolute effect β Overall influence |
| Ο (sigma) | Standard deviation β Non-linearity |
Interpretation
| ΞΌ* | Ο | Meaning |
|---|---|---|
| High | Low | Linear influence β Calibrate |
| High | High | Non-linear β Calibrate carefully |
| Low | Low | Non-influential β Fix at default |
| Low | High | Interactive β Investigate |
Typical result: Reduces 100+ params to 30-40 for detailed analysis
OpenWQ
6
Sobol Sensitivity Analysis
Variance-based method for detailed sensitivity ranking.
Cost: O(N Γ (2k+2))
N = 1000-5000 samples, k = parameters
Example: 30 params Γ 2000 samples = ~124,000 runs
Indices
| S1 | First-order: Direct parameter effect |
| ST | Total-order: Including interactions |
Interpretation
- S1 β ST: Parameter acts independently
- ST >> S1: Strong interactions
- Sum(S1) < 1: Significant interactions exist
Use after Morris: Apply Sobol only to the ~30 parameters identified as potentially influential
OpenWQ
7
02
DDS Optimization
Dynamically Dimensioned Search (DDS)
Derivative-free optimization designed for expensive models.
Key Features
- Reduces search dimensionality over time
- Early: Explore many parameters
- Late: Focus on few promising ones
- No gradient computation needed
Typical budget: 200-500 evaluations for 10-15 parameters
DDS Behavior Over Iterations
Early (exploration)
ββββββββββββββββββββββββββββββββ
β ββββββββββββββββββββββββββββ β Many params
β ββββββββββββββββββ β perturbed
β ββββββββββββ β
ββββββββββββββββββββββββββββββββ
Late (exploitation)
ββββββββββββββββββββββββββββββββ
β βββ β Few params
β ββ β refined
β β β
ββββββββββββββββββββββββββββββββ
Probability of perturbation:
P(i) = 1 - ln(i)/ln(max_iter)
OpenWQ
9
Objective Functions
RMSE
β(Ξ£(obs-sim)Β²/n)
Root Mean Square Error
0 β β (lower better)
NSE
1-Ξ£(o-s)Β²/Ξ£(o-Ε)Β²
Nash-Sutcliffe Efficiency
-β β 1 (1 perfect)
KGE
1-β((r-1)Β²+(Ξ±-1)Β²+(Ξ²-1)Β²)
Kling-Gupta Efficiency
Recommended!
PBIAS
100ΓΞ£(s-o)/Ξ£(o)
Percent Bias
-β β +β (0 perfect)
KGE advantages: Balances correlation (r), variability ratio (Ξ±), and bias ratio (Ξ²) β better for multi-species calibration
OpenWQ
10
Temporal Resolution
Aggregate observations and model outputs to a common temporal scale before computing objective functions.
Available Resolutions
| Option | Use Case |
|---|---|
| native | Original timestamps (no aggregation) |
| daily | Daily patterns matter |
| weekly | Smooth day-to-day noise |
| monthly | Seasonal patterns, sparse obs |
| yearly | Long-term trends, budgets |
Aggregation Methods
mean (default) | sum | median | min | max
Use mean for concentrations, sum for loads/fluxes
# Configuration in calibration file temporal_resolution = "monthly" aggregation_method = "mean" # How it works: # 1. Group obs by reach, species, month # 2. Extract model outputs for same periods # 3. Aggregate both to monthly means # 4. Compute KGE/NSE/RMSE on aggregated data
Tip: Use
monthly for sparse grab samples β ensures fair comparison with continuous model output
Output: Performance plots generated at specified resolution (time series, scatter, residuals)
OpenWQ
11
03
Calibration Workflow
Step-by-Step Workflow
1
2
Morris Screening (~200 runs)
Test all 100+ parameters β identify ~30-40 influential ones
3
Sobol Analysis (~1000 runs)
Detailed SA on 30 params β rank and select top 10-15
4
DDS Optimization (~300-500 runs)
Optimize top 10-15 parameters, fix others at defaults
5
Validation
Test calibrated parameters on independent time period
OpenWQ
13
Observation Data Sources
Two options for preparing observation data:
π Option 1: Manual CSV
observation_data_source = "csv"
- Prepare data manually
- Full control over format
- Use any data source (USGS, EPA, local monitoring, etc.)
Best for: Custom datasets, local monitoring networks
π Option 2: GRQA Database
observation_data_source = "grqa"
- 43 water quality parameters
- ~100 million observations worldwide
- Auto station-to-reach matching
- Local data or Zenodo download
Best for: Large-scale studies, data-rich regions
Local Data Support: GRQA supports pointing to local data folders if already downloaded from Zenodo
OpenWQ
14
GRQA Database Integration
Global River Water Quality Archive
- 43 water quality parameters
- ~100 million observations worldwide
- Automatic Zenodo download
- Spatial station-reach matching
grqa_config = {
# Local or download from Zenodo
"local_data_path": "/data/GRQA",
"river_network_shapefile": "rivers.shp",
"max_station_distance_m": 500,
"species_mapping": {
"NO3": "NO3-N",
"NH4": "NH4-N"
}
}
Species Mapping (GRQA β Model)
| GRQA | Model Species |
|---|---|
| NO3 | NO3-N |
| NH4 | NH4-N |
| TN | TN |
| PO4 | PO4-P |
| TP | TP |
| TSS | TSS |
| DOC | DOC |
Run extraction:
python my_calibration.py --prepare-obs-only
OpenWQ
15
Observation Data CSV Format
# observations.csv (all sources produce this format)
datetime,reach_id,species,value,units,source,uncertainty,quality_flag
2018-01-15 10:00:00,1200014181,NO3-N,2.50,mg/l,USGS_station_A,0.25,GOOD
2018-01-15 10:00:00,1200014181,NH4-N,0.15,mg/l,USGS_station_A,0.02,GOOD
2018-02-01 10:00:00,1200014181,NO3-N,3.10,mg/l,USGS_station_A,0.31,GOOD
2018-02-01 10:00:00,1200014181,TP,0.08,mg/l,USGS_station_A,0.01,GOOD
...
Required Columns
| datetime | YYYY-MM-DD HH:MM:SS |
| reach_id | Matching model output |
| species | Case-sensitive name |
| value | Measured concentration |
| units | mg/l, ug/l, etc. |
Optional Columns
| source | Data provider ID |
| uncertainty | Measurement error |
| quality_flag | GOOD, SUSPECT, BAD |
OpenWQ
16
Priority-Based Calibration
When resources are limited, focus on the most influential parameters:
TIER 1 Must Calibrate
- Source/sink scaling factors
- k_nitrification
- k_denitrification
- Kfr_PO4 or qmax_PO4
- dispersion_x
5-8 parameters
TIER 2 Important
- Temperature coefficients (ΞΈ)
- k_mineralization
- k_P_adsorption
- erosion_index
- Secondary sorption params
Next 5-8 parameters
TIER 3 Refinement
- Half-saturation constants
- Lateral exchange rates
- Volatilization rates
- Langmuir exponents
Remaining parameters
OpenWQ
17
Running Calibration
Copy the template to your working directory, configure parameters, then run:
# 1) Copy and edit the template cp calibration_config_template.py my_calibration.py # 2) Run in different modes: python my_calibration.py # Full calibration python my_calibration.py --sensitivity-only # SA only python my_calibration.py --prepare-obs-only # Obs data only python my_calibration.py --dry-run # Validate config python my_calibration.py --resume # Resume from checkpoint
Template pattern: Copy β Edit β Run β no need to modify library code
Command-Line Flags
| --sensitivity-only | Run Morris/Sobol only |
| --prepare-obs-only | Prepare observations (GRQA/CSV) |
| --dry-run | Validate without running |
| --resume | Continue from checkpoint |
In-File Options
run_sensitivity_first = True
β Auto SA β Calibration pipeline
OpenWQ
18
Calibration Output
Output Files
best_parameters.json | Optimal values |
calibration_history.json | All evaluations |
parameter_definitions.json | Parameter metadata & bounds |
matched_data.csv | Obs-model matched pairs |
calibration_report.html | Interactive HTML report |
basin_report.html | Per-basin multi-variant report |
sensitivity_results.json | SA results (if run) |
Runtime Estimates
| Model Runtime | 300 Evals |
|---|---|
| 5 min | ~25 hours |
| 15 min | ~75 hours |
| 30 min | ~150 hours |
HPC Tip: Use job arrays to run evaluations in parallel on cluster
OpenWQ
19
Best Practices
β Do
- Start with Morris screening to reduce parameters
- Use log transform for rate constants
- Split data: calibration + validation periods
- Use KGE for multi-species calibration
- Document parameter choices
- Check physical plausibility of results
β Don't
- Calibrate all 100+ parameters at once
- Ignore parameter correlations
- Use entire dataset for calibration
- Accept physically unrealistic values
- Skip sensitivity analysis
- Overfit to noisy observations
Golden Rule: Fewer well-chosen parameters beats many poorly constrained ones
OpenWQ
20
05
Post-Calibration Reporting
Interactive HTML Reports
Calibration automatically generates self-contained HTML reports with interactive Plotly.js charts.
6 Diagnostic Charts
- Convergence curve (objective vs iteration)
- Parameter evolution trajectories
- Time series: observed vs simulated
- Scatter plot with 1:1 line
- Residual analysis
- Parameter sensitivity ranking
Self-contained: Single HTML file with embedded data β share via email, no server needed
Report Features
| Plotly.js | Zoom, pan, hover tooltips |
| Dark/Light mode | Theme toggle included |
| Parameter table | Best values + bounds + position bar |
| Species metrics | Per-species KGE/NSE/RMSE |
| Sidebar nav | Jump to any section |
Auto-Generated
Reports are created automatically when calibration completes β no extra steps needed.
OpenWQ
22
Interactive Basin Maps
Calibration reports include interactive Leaflet.js maps showing the basin spatial context.
Map Layers
- HRU polygons β colored by area quintiles
- River network β styled by Strahler order
- Observation stations β red markers with popup info
Basemaps
3 selectable basemaps: CARTO Light, OpenTopoMap, Esri Satellite
Data source: GeoPackage files (
*_basinHru.gpkg, *_riverNetwork.gpkg)
Basin Info Grid
| Total HRUs | Count from GeoPackage |
| River reaches | Network segments |
| Total area | kmΒ² from HRU polygons |
| Network length | km of river network |
| Max Strahler order | Stream hierarchy |
| Obs stations | Matched monitoring sites |
Interactive Controls
Layer toggle, zoom, scale bar, legend, click-for-info popups
OpenWQ
23
Per-Basin Multi-Variant Reports
When running multiple variants (A/B/C/D), a consolidated basin report is auto-generated comparing all variants.
Basin Report Contents
- Overview KPIs β best KGE per variant
- Basin map β shared spatial context
- Variant comparison table β side-by-side metrics
- Detail cards β per-variant parameter tables
- Links β to detailed per-variant reports
Auto-discovery: Sibling workspaces (
workspace_{basin}_{variant}) are detected automatically
Post-Calibration Pipeline
Calibration completes β βΌ parameter_definitions.json β saved matched_data.csv β saved β βΌ calibration_report.html β per-variant β βΌ Detect sibling workspaces β βΌ basin_report.html β multi-variant
OpenWQ
24
Summary
π Screen
Morris screening to identify influential parameters (106 β 30)
π Rank
Sobol analysis for detailed sensitivity ranking (30 β 10-15)
π― Optimize
DDS calibration of top parameters (200-500 evaluations)
π Report
Auto-generated interactive HTML reports with maps & charts
Key Message: Hierarchical approach (Screen β Rank β Optimize β Report) with
automated post-calibration diagnostics for every basin and variant
automated post-calibration diagnostics for every basin and variant
OpenWQ
25
Thank You
Questions?
Calibration scripts: supporting_scripts/Calibration/
Observation data: CSV | GRQA Database | Reports: HTML with interactive maps & charts