Supporting Scripts

Python Tools & Post-Processing
Diogo Costa
University of Évora, Portugal

Contents

• Python Environment Setup

• Tools Overview

Section 01: Preprocessing

1_Model_Config/ — copy, edit, run

• What goes inside model_config_<project>.py

• The HTML configuration report

Section 02: Post-Processing

2_Read_Outputs/read_output_template.py

• The HTML flood-statistics report

fluxos_viewer.py (KML / WebGL / MP4)

• Summary

Python Environment Setup

Create the venv once after clone:

# 1. Create the venv
python3 -m venv .venv

# 2. Activate (the conda/unset lines are harmless
#    w/o Conda; they stop a stale PROJ_LIB from
#    hanging `import rasterio`)
conda deactivate 2>/dev/null || true
unset PROJ_LIB PROJ_DATA
source .venv/bin/activate

# 3. Install deps
pip install --upgrade pip
pip install -r supporting_scripts/requirements.txt

# 4. Verify — call the venv's own python directly with a clean
#    environment. This is the "it just works" form: no timeout,
#    no retries, no chance of a Conda shim interfering.
env -u PROJ_LIB -u PROJ_DATA -u GDAL_DATA \
    -u CONDA_PREFIX -u CONDA_DEFAULT_ENV \
    ./.venv/bin/python -c "import rasterio, pygmsh, pyvista; print('ok')"
# → if it hangs > 10 s, see the troubleshooting note on the right

requirements.txt

rasterioGeoTIFF I/O
numpy / scipyarrays, resampling
pygmsh, gmshtriangular meshing
pyvistaread .vtu output

gmsh wheel fails? brew install gmsh (macOS) / apt install gmsh (Linux), then re-run pip.

Verify still hangs? Conda is polluting the child process with DYLD/LD_LIBRARY_PATH. Last resort: open a fresh terminal (no auto-Conda), source .venv/bin/activate and re-run step 4. Or permanently disable Conda auto-activation: conda config --set auto_activate_base false.

Windows PowerShell:  if ($env:CONDA_DEFAULT_ENV){conda deactivate}; Remove-Item Env:PROJ_LIB,Env:PROJ_DATA -EA SilentlyContinue; .venv\Scripts\Activate.ps1; then .venv\Scripts\python -c "import rasterio, pygmsh, pyvista; print('ok')".

Tools Overview

Two editable templates drive the full workflow — the user only edits these two files. Everything else is internal library code.

🔧 Preprocessing

1_Model_Config/
model_config_template.py

One editable template that builds the DEM (.asc), Gmsh triangular mesh (.msh), and modset.json in one run — and emits an HTML configuration report with copy-paste Docker commands.

📊 Post-Processing

2_Read_Outputs/
read_output_template.py

Streams FLUXOS output files and emits an HTML flood-statistics report with time-series (volume, flooded area, depth, velocity), max-inundation map, hazard classification, depth histogram, and first-inundation map.

🌎 Results Viewer

2_Read_Outputs/output_supporting_lib/
fluxos_viewer.py

Standalone CLI that exports the same results as animated KML (Google Earth), MP4 video, or an interactive 3D WebGL page.

Folder layout: supporting_scripts/1_Model_Config/  +  supporting_scripts/2_Read_Outputs/
01

Preprocessing

1_Model_Config/ — copy, edit, run

The preprocessing entry point is a single Python file you copy, edit, and run. The template never changes — each project gets its own copy so configurations are reproducible and diff-able under version control.

# 1. Move into the 1_Model_Config folder
# If you are in FLUXOS_cpp, you can run
cd supporting_scripts/1_Model_Config/

# 2. Copy the template to a project-specific name
#    (anything like model_config_<project>.py)
cp model_config_template.py \
   model_config_myTest.py

# 3. Open the copy and edit the _config dict
#    (see next slide for the fields you need to set)
#    You can also just double-click the file and edit
#    it in any text editor — VS Code, Sublime, PyCharm,
#    TextEdit, Notepad++, etc. — no terminal needed.
nano model_config_myTest.py

# 4. Run it — from this folder OR anywhere else,
#    just point python at your edited copy
python model_config_myTest.py

# (run from repo root instead, if you prefer):
# python supporting_scripts/1_Model_Config/\
#        model_config_myTest.py

What the 4 steps do

  • 1. cd — the templates, their support lib and the generated reports all live in 1_Model_Config/
  • 2. cp — never edit model_config_template.py directly; keep it pristine
  • 3. edit — fill in project_name, DEM source, bbox, mesh size, inflow point, simulation dates (details on the next slide)
  • 4. run — downloads/reads the DEM, builds the mesh, writes the modset JSON, opens an HTML report with ready-to-paste Docker commands
Activate your venv first: source .venv/bin/activate (see slide 3). The template imports rasterio / pygmsh — they have to be installed.
Reproducibility: the copy lives alongside the template under git, so reviewers / future-you can re-run the exact same setup by git checkout + python model_config_<project>.py.

What goes inside model_config_<project>.py

Example — this is the default template content (model_config_template.py, Rosa Creek case study). It runs out-of-the-box on the GeoTIFF shipped at Working_example/Rosa_2m.tif — no edits needed.
_config = dict(
    project_name    = "Rosa Creek",
    authors         = ["Diogo Costa"],
    date            = "2026-04-21",

    output_bin_dir  = "Working_example",
    modset_name     = "modset_rosa.json",

    # --- DEM: local GeoTIFF ---
    dem_source_mode         = "file",
    dem_source_geotiff      = "Working_example/Rosa_2m.tif",
    dem_target_resolution_m = 2.0,
    dem_output_asc          = "Rosa_2m.asc",

    # --- Mesh ---
    mesh_type               = "regular",
    trimesh_min_size        = 2.0,
    trimesh_max_size        = 30.0,
    trimesh_slope_factor    = 2.0,
    mesh_output_msh         = "Rosa_trimesh.msh",

    # --- Forcing ---
    meteo_file              = None,  # no rain; hydrograph-driven
    inflow_file             = dict(
        path = "Working_example/Flow_river_30h.fluxos",
        lon  = -124.08154, lat = 52.21373,  # Rosa inlet, UTM 10N
    ),

    # --- Simulation + output ---
    sim_datetime_start      = "2009-01-01 00:00:00",
    roughness_height        = 0.005,
    output_folder           = "Results/",
    print_step_s            = 1800,
    h_min_to_print_m        = 0.001,

    # --- Optional modules ---
    ade_transport           = dict(enabled=False, d_coef=0.5),
    soil_infiltration       = dict(enabled=False, default_ks_mm_hr=10.0),

    # --- Run options ---
    fluxos_executable       = "bin/fluxos",
    generate_report         = True,
)

What the run produces

  • Working_example/<dem>.asc — ESRI-ASCII DEM
  • Working_example/<mesh>.msh — Gmsh triangular mesh (only for mesh_type="triangular")
  • Working_example/<modset>.json — solver input
  • HTML report under 1_Model_Config/reports/ with copy-paste Docker compile + run commands

DEM sources & resolutions

  • "file" — your own GeoTIFF (must be in a projected CRS) — default
  • "download" — auto-fetch by bbox:
      No key: COP30_AWS 30 m (global) ★, USGS 3DEP 1-10 m (USA)
      OpenTopography (free key): COP30, SRTM, AW3D30, NASADEM (30 m), EU-DTM (10 m, Europe)
Projected CRS required! DEMs in degrees give wrong slopes. The "download" mode reprojects to UTM automatically (e.g. EPSG:32629 for Portugal).

The HTML Configuration Report

Each run of model_config_template.py opens a self-contained HTML report in your browser. One page, no further navigation needed to build and run the model.

Sections

  • Summary — KPI tiles (mesh type, cell count, DEM resolution, active modules)
  • Domain — DEM bounds, CRS, rows×cols, interactive 2D/3D terrain map (Plotly)
  • Mesh — vertex/cell/edge counts, overlaid on the DEM map
  • Configuration — table of non-default modset values + full JSON
  • Modules — ADE transport / soil infiltration status pills
  • Next Steps — copy-paste shell snippets

"Next Steps" snippets

  1. cd <repo>
  2. docker compose ... build (or Apptainer equivalent)
  3. Open a shell in the container
  4. Compile FLUXOS into /work/bin and run the modset
  5. List & open the Results/ directory
  6. Visualise: WebGL viewer & the statistics template
One editable template drives everything. No separate CLI commands — python model_config_template.py and follow the links in the report.
02

Post-Processing

2_Read_Outputs/read_output_template.py

Don't edit this template by hand if you don't have to. The HTML configuration report generated by the preprocessing step already contains ready-to-run snippets for every post-processing tool — the statistics report and the WebGL / KML / MP4 viewer — pre-filled with the correct results folder, modset path, DEM, and UTM zone. Use those if you just want results now. The template + CLI flags shown below are the manual escape hatch for batch runs, CI, or calling the scripts from a different project.
# Edit the _config dict:

_config = dict(
    project_name = "Rosa Creek",

    # Folder FLUXOS wrote to
    results_dir = "Results_river_30h",

    # The modset that drove the run
    modset_file = "Working_example/modset_river_30h.json",

    # Flooded = cells with h > this (metres)
    h_threshold_flooded_m = 0.01,

    # Analyse every Nth timestep
    stride = 1,

    generate_report       = True,
    export_csv_timeseries = True,
)

# Then run it:
cd supporting_scripts/2_Read_Outputs
python read_output_template.py

What the report contains

  1. Time-series panel — volume, flooded area, max depth, max velocity vs. time
  2. Max-inundation map — per-cell peak depth over the entire run
  3. ARR-2019 hazard mapD = h·(v+0.5) with standard thresholds
  4. Depth histogram — area distribution across depth bins
  5. First-inundation map — time at which each cell first became wet
  6. Chemical transport panel — if ADE was on, max / mean conc_SW vs. time
Streaming: snapshots are read one at a time — a 30 h run with ∼10 GB of .txt files analyses fine on a laptop.

fluxos_viewer.py — WebGL / KML / MP4

Pre-filled for you: step 6a of the configuration report is already the WebGL command with the correct DEM, mesh type, UTM zone and results path. Paste it and skip the block below. These CLI examples are for when you want KML / MP4 instead, or need to drive the viewer from a script.
# Lives at 2_Read_Outputs/output_supporting_lib/fluxos_viewer.py
# Add --mesh-type triangular for .vtu output (regular = .txt default)

# Interactive 3D WebGL viewer (starts a local HTTP server
# on --webgl-port and opens http://localhost:8080)
python supporting_scripts/2_Read_Outputs/\
output_supporting_lib/fluxos_viewer.py \
    --results-dir Results/ \
    --dem Working_example/Rosa_2m.asc \
    --utm-zone 10 \
    --export-webgl \
    --webgl-output fluxos_web_rosa \
    --webgl-step 1 \
    --webgl-port 8080

# Animated KML/KMZ for Google Earth
# (default action if neither --export-webgl nor --export-video given)
python fluxos_viewer.py \
    --results-dir Results/ \
    --dem Working_example/Rosa_2m.asc \
    --utm-zone 10 \
    --output-dir fluxos_kml_rosa

# MP4 animation (requires ffmpeg in PATH)
python fluxos_viewer.py \
    --results-dir Results/ \
    --dem Working_example/Rosa_2m.asc \
    --utm-zone 10 \
    --export-video mp4 \
    --video-output flood.mp4 \
    --video-fps 15

Features

  • Reads both regular (.txt) and triangular (.vtu) output — pass --mesh-type
  • UTM → WGS84 on the fly; UTM zone auto-detected from DEM if omitted
  • Variables: --variable h (default), velocity, conc_SW
  • WebGL folder is self-contained (index.html + data); a local HTTP server auto-opens in the browser
  • Satellite tiles cached under ~/.fluxos_tile_cache/ so re-runs skip the network

Useful flags

--webgl-step Nkeep every Nth snapshot (smaller bundle)
--webgl-no-serveexport only, don't start the server
--video-scale NNx DEM resolution for MP4 frames
--velocity-arrowsoverlay flow arrows on KML / MP4

Summary

Template / Tool Role User edits Output
1_Model_Config/
model_config_template.py
Preprocessing copy → edit _config dict of the copy DEM .asc, mesh .msh, modset.json, HTML configuration report (with ready-to-paste Docker, sim-run, and visualisation snippets)
2_Read_Outputs/
read_output_template.py
Post-processing (statistics) flags on the CLI, or _config if running the template HTML flood-statistics report: time-series, max-inundation & first-inundation maps, ARR-2019 hazard map, depth histogram
2_Read_Outputs/output_supporting_lib/
fluxos_viewer.py
Post-processing (visualisation) flags only Interactive WebGL viewer folder, animated KML/KMZ for Google Earth, or MP4 video
End-to-end workflow: python3 -m venv .venv & install requirements → cp model_config_template.py model_config_<project>.py → edit the copy → python model_config_<project>.pyopen the HTML report and paste its Next-Steps snippets (compile FLUXOS, run the sim, visualise results).
The configuration report is the single source of truth for running your simulation — DEM path, modset name, UTM zone, and visualisation flags are all pre-filled. You rarely need to type a command by hand after python model_config_<project>.py.

Thank You

Questions & Resources

GitHub: github.com/ue-hydro/FLUXOS_cpp

Documentation: fluxos-cpp.readthedocs.io

Contact: diogo.costa@uevora.pt