symfluence-openhydronet

An external, pip-installable SYMFLUENCE plugin for Google's OpenHydroNet riverine flood forecasting framework. Installing this package registers OPENHYDRONET without modifying SYMFLUENCE source code.

Current scope

The initial integration runs inference from an existing OpenHydroNet run directory (configuration, checkpoint, and normalization metadata) in an isolated Conda environment. It standardizes forecast output for SYMFLUENCE and includes a CFIF to OpenHydroNet forcing adapter. Fully automatic construction of OpenHydroNet training datasets from arbitrary SYMFLUENCE domains is intentionally not claimed yet: pretrained models require the exact static attributes, feature order, meteorological products, and normalization used during training.

Install

pip install -e ../symfluence-openhydronet

Install OpenHydroNet in its own environment following its upstream instructions. The default environment name used by this plugin is googlehydrology.

SYMFLUENCE configuration

model:
  hydrological_model: OPENHYDRONET
  openhydronet:
    run_dir: /absolute/path/to/openhydronet/run
    conda_env: googlehydrology
    mode: infer
    output_glob: "test/model_epoch*/test_results.zarr"
    frequency: 1D
    lead_time: 0

A complete configuration is provided at examples/openhydronet_tutorial.yaml. After updating its absolute paths, run the normal framework workflow:

symfluence workflow run --config examples/openhydronet_tutorial.yaml

The runner executes:

conda run -n googlehydrology run infer --run-dir <run_dir>

Set executable to an absolute path if the OpenHydroNet CLI is available directly and Conda isolation is not wanted.

OpenHydroNet inference writes a Zarr result with dimensions (basin, freq, date, time_step). The plugin converts this to forecast-native NetCDF while preserving selected basins and lead times. The standardized result contains basin, issue_time, lead_time, valid_time, and streamflow. If SYMFLUENCE itself uses Zarr 2, the postprocessor automatically reads Zarr 3 through the isolated OpenHydroNet environment; no framework environment upgrade is required.

Selection and execution controls include:

openhydronet:
  period: test
  epoch: 15            # null selects the latest checkpoint
  device: cpu          # config, cpu, or mps
  basin_ids: null      # null retains all basins
  lead_times: null     # null retains all lead times
  reuse_results: true

The NetCDF stores checkpoint/config hashes, upstream commit, MultiMet source, Caravan dataset, and plugin/SYMFLUENCE versions as global provenance attributes.

Data preflight

Validate the complete inference contract before starting an expensive run:

symfluence-openhydronet doctor \
  --run-dir /tmp/openhydronet-tutorial-run

The doctor checks the checkpoint/config/scaler bundle, test basin list, Caravan target files and variables, Caravan static attributes and gauge IDs, scaler feature coverage, and anonymous access to every required Caravan-MultiMet Zarr product. It reports missing basins or features by name and exits nonzero if any check fails. Use --skip-cloud for an offline local-only check or --json for automation.

Postprocessing produces:

results/
├── tutorial_inference_openhydronet_forecast.nc
├── tutorial_inference_openhydronet_skill_by_basin_lead.csv
├── tutorial_inference_openhydronet_skill_by_lead.csv
└── plots/
    ├── tutorial_inference_openhydronet_hydrograph_by_lead.png
    ├── tutorial_inference_openhydronet_forecast_fan.png
    ├── tutorial_inference_openhydronet_skill_by_lead.png
    └── tutorial_inference_openhydronet_basin_lead_heatmap.png

The evaluator calculates NSE, KGE, and RMSE independently for each basin and lead time, then reports mean and median skill by lead.

Upstream tutorial contract test

Clone OpenHydroNet and point the optional integration test at it:

OPENHYDRONET_UPSTREAM=/path/to/flood-forecasting pytest -m integration

This checks the plugin against the real tutorial config, Caravan sample, checkpoint, scaler, and Zarr metadata without copying those large/upstream-owned artifacts here.

For Apple Silicon, create the CPU/MPS-compatible environment and prepare a portable copy of the tutorial run (the committed upstream config contains Google-internal absolute paths):

mamba env create -f environments/openhydronet-macos.yml
conda run -n googlehydrology pip install -e /path/to/flood-forecasting
python scripts/prepare_upstream_tutorial.py \
  --upstream /path/to/flood-forecasting \
  --destination /tmp/openhydronet-tutorial-run

PYTHONPATH=src:/path/to/SYMFLUENCE/src python scripts/run_upstream_e2e.py \
  --run-dir /tmp/openhydronet-tutorial-run \
  --executable ~/.local/share/mamba/envs/googlehydrology/bin/run \
  --data-dir /tmp/symfluence-openhydronet-e2e

Run the opt-in genuine inference test with:

OPENHYDRONET_E2E=1 \
OPENHYDRONET_UPSTREAM=/path/to/flood-forecasting \
OPENHYDRONET_EXECUTABLE=/path/to/googlehydrology/bin/run \
pytest -m e2e

Development

pip install -e '.[dev]'
pytest
ruff check .

The CPU Linux image runs the unit suite during its build and again by default:

docker build -t symfluence-openhydronet:test .
docker run --rm symfluence-openhydronet:test

Dockerfile.cuda verifies installation against the CUDA 12.4 PyTorch runtime. The GitHub workflow builds that image without executing GPU inference because hosted runners do not provide a GPU; genuine CUDA inference can be run on a GPU-backed self-hosted runner with docker run --gpus all .... Linux CPU tests run normally on every push and pull request.