PyStorm

A modular Python and C++ framework for probabilistic modeling of coastal storm hazards.

---

Overview

The PyStorm framework supports coastal hazards quantification, probabilistic modeling, stochastic storm simulation, and extreme-value analysis. Python handles orchestration, configuration, I/O, and plotting; C++ engines carry the inner numerical kernels that dominate runtime. Workflows stay script-driven and reproducible.

Following CyHAN v2.2, a C++ engine is shipped where a module has performance-critical computation, and is omitted where it does not. Compute-heavy modules (POT, PST, RSS) ship a C++ kernel with a pure-Python fallback so each workflow runs whether or not the extension is built; pure-Python modules (SCA, CSH, LCS) ship no engine. AHD ships an optional C++ accelerator with a NumPy fallback.

Modules

PyStorm has eight modules: a tropical-cyclone chain (AHD -> SCA -> LCS / JDM), a response chain (POT -> PST), and two independent modules (RSS, CSH):

Module	Purpose
augmented_hurricane_database (AHD)	Build an augmented HURDAT2 best-track: parse NHC HURDAT2, derive storm motion, optionally backfill Rmax from EBTRK and impute missing Cp/Rmax with a Gaussian-process metamodel. The best-track foundation for downstream TC analyses.
storm_climatology_analysis (SCA)	Per-CRL tropical-cyclone storm recurrence rates (SRR/DSRR) from the augmented HURDAT2, annual and monthly, via the Gaussian Kernel Function.
life_cycle_simulation (LCS)	Monte-Carlo synthetic tropical-cyclone life cycles for a CRL: a Poisson number of TCs per year (rate from the SCA SRR and a radius of influence), each stratified by intensity and placed on a calendar day from the seasonal SRR. Consumes the SCA SRR tables.
joint_distribution_model (JDM)	Per-CRL JPM joint distribution of TC parameters [heading, Dp, Rmax, Vt]: distance-weighted marginal distributions and a meta-Gaussian copula. Consumes the SCA selection + DSRR outputs.
peaks_over_threshold (POT)	Extract independent storm peaks from a continuous water-level or NTR time series.
probabilistic_simulation_technique (PST)	Turn a POT peak sample into a hazard curve (response magnitude versus AER) with a confidence band.
reduced_storm_suite (RSS)	Select a small, representative Reduced Storm Suite that reproduces the full synthetic suite's hazard.
coastal_storm_hydrograph (CSH)	Reduce an ensemble of synthetic-TC surge series to one dimensionless surge shape per save point, scalable by a peak elevation and an equivalent width.

Data flow: AHD writes the augmented best-track that SCA reads; SCA writes the per-CRL SRR that LCS reads and the selection + DSRR that JDM reads; POT writes per-station peak files that PST reads. RSS and CSH are independent.

Architecture

Every module is a self-contained vertical (CyHAN v2.2 §5) with the same core shape: a user-facing launcher at the module root that calls a non-user-facing orchestrator under backend/python/.

Path	Language	Role
run_<module>.py	Python	Launcher at the module root. User options + a thin CLI; assembles config and calls the orchestrator's run(config).
backend/python/api_<module>.py	Python	Orchestrator / module API: the Python Orchestration role (data flow, I/O, config, diagnostics, plotting), exposed as run(config).
backend/engines/	C++	Compute kernel exposed through pybind11 (only where a module has heavy compute).
tests/	Python	Smoke and integration tests covering the Python path and, where present, the C++ binding.

The run(config) convention

Every module exposes a single programmatic entry point in backend/python/api_<module>.py:

run(config) -> <Module>Result

config is a plain dict (some modules also accept their Pydantic config model). The launcher is just a thin front-end: it builds config from its USER OPTIONS block (and any CLI overrides) and calls run. To drive a module from your own code, a notebook, or another module, put its backend on the path and call run directly:

import sys
sys.path.insert(0, "modules/<module>/backend/python")
from api_<module> import run

result = run(config)

run returns a typed <Module>Result (e.g. AHDResult, POTResult, SCAResult, RSSResult). Batch-capable modules (POT, PST, RSS) return a dict[str, <Module>Result] when given multiple inputs (stations, paths, or datasets), collapsing to a single result when given one. Each module README's Programmatic API section documents that module's exact config and result.

What each module ships varies with its workload:

Module	C++ engine	Ancillary scripts
AHD	optional _gpm accelerator (NumPy fallback)	—
SCA	none (pure Python)	analysis/ (kernel sensitivity)
LCS	none (pure Python)	—
JDM	optional _jdm bootstrap kernel (NumPy fallback)	—
POT	_pot kernel	—
PST	_pst kernel	scripts/ (method testbed)
RSS	_rss kernel	scripts/ (preprocess, DSW)
CSH	none (pure Python)	analysis/ (shape/timescale studies)

Install

All modules target Python >= 3.10. The fastest path installs every module and builds the C++ kernels in one step:

./install.sh            # Linux / macOS / Git Bash
.\install.ps1           # Windows PowerShell

This installs the dependencies, registers each module (editable) so the pystorm-<acro> commands and orchestrators are importable from anywhere, and builds the kernels (a failed build is non-fatal: that module runs on its pure-Python fallback). Then check the environment:

python check_env.py

The doctor reports the Python version, every required package, and whether a C++ toolchain is present, so you know up front whether the fast kernels will build.

If you only want the dependencies (no editable install), the root requirements.txt is a convenience superset of every module's needs:

pip install -r requirements.txt

Each module also declares its own exact dependencies in modules/<module>/pyproject.toml, so to install a single module in isolation use that instead (pip install -e modules/<module>). Installing the packages is the only manual step: the C++ kernels then build automatically on first run (see Building the C++ engines).

Task runner

A Makefile (Linux / macOS) and tasks.ps1 (Windows) wrap the common verbs:

make install            # or:  .\tasks.ps1 install
make doctor             #      .\tasks.ps1 doctor
make build              #      .\tasks.ps1 build      (build all C++ kernels)
make test               #      .\tasks.ps1 test       (every module's tests)
make run M=rss ARGS="--mode optimal --scope regional"
#                              .\tasks.ps1 run rss --mode optimal --scope regional

After an editable install, each launcher is also exposed as a console command: pystorm-ahd, pystorm-sca, pystorm-lcs, pystorm-jdm, pystorm-pot, pystorm-pst, pystorm-rss, pystorm-csh.

Quickstart

Run a module from its directory. Where a module has a C++ kernel it builds automatically on the first run; if no compiler is available the pure-Python fallback runs instead.

# AHD: build the augmented HURDAT2 best-track
cd modules/augmented_hurricane_database
python run_augmented_hurricane_database.py

# SCA: per-CRL storm recurrence rates from the augmented best-track
cd modules/storm_climatology_analysis
python run_storm_climatology_analysis.py

# LCS: Monte-Carlo synthetic TC life cycles for a CRL from the SCA SRR
cd modules/life_cycle_simulation
python run_life_cycle_simulation.py

# JDM: per-CRL joint distribution of TC parameters (marginals + copula) from SCA
cd modules/joint_distribution_model
python run_joint_distribution_model.py

# POT: extract peaks from a water-level / NTR series
cd modules/peaks_over_threshold
python run_peaks_over_threshold.py

# PST: hazard curves from the POT peaks
cd modules/probabilistic_simulation_technique
python run_probabilistic_simulation_technique.py

# RSS: reduced storm suite selection
cd modules/reduced_storm_suite
python run_reduced_storm_suite.py

# CSH: unit storm-surge hydrographs
cd modules/coastal_storm_hydrograph
python run_coastal_storm_hydrograph.py

Each launcher has a USER OPTIONS block at the top and --help for command-line overrides. Per-module data lives under modules/<module>/data/ (inputs/ and outputs/, both gitignored).

Building the C++ engines

The compiled kernels are not required for correctness; they accelerate the inner loops by roughly one to two orders of magnitude on large problems. Each kernel builds on first run, or build it manually:

python modules/peaks_over_threshold/backend/engines/cpp/build.py             # _pot
python modules/probabilistic_simulation_technique/backend/engines/build.py   # _pst
python modules/reduced_storm_suite/backend/engines/cpp/build.py                 # _rss
python modules/joint_distribution_model/backend/engines/cpp/build.py         # _jdm
python modules/augmented_hurricane_database/backend/engines/cpp/build.py     # _gpm (optional)

build.py tries setuptools, then CMake, then a direct compiler call. It needs pybind11 (pip install pybind11) and a C++17 toolchain (MSVC or MinGW on Windows, gcc or clang elsewhere). SCA, LCS, and CSH have no engine to build.

Repository layout

PyStorm/
│
├── modules/                                  eight self-contained capability verticals
│   ├── augmented_hurricane_database/         AHD  augmented HURDAT2 best-track
│   ├── storm_climatology_analysis/           SCA  per-CRL SRR/DSRR (consumes AHD)
│   ├── life_cycle_simulation/                LCS  synthetic TC life cycles (consumes SCA)
│   ├── joint_distribution_model/             JDM  TC-parameter joint distribution (consumes SCA)
│   ├── peaks_over_threshold/                 POT  storm-peak extraction
│   ├── probabilistic_simulation_technique/   PST  hazard curves (consumes POT)
│   ├── reduced_storm_suite/                     RSS representative suite selection
│   └── coastal_storm_hydrograph/               CSH  unit storm-surge hydrographs
│
│   Each module:
│     run_<module>.py            launcher (user options)
│     README.md                  module reference (methods, workflow, outputs)
│     ENGINE_MANIFEST.toml       structured module manifest
│     pyproject.toml             installable orchestrator package
│     backend/
│       engines/                 C++ kernel + pybind11 binding (compute-heavy modules)
│       python/
│         api_<module>.py        orchestrator / module API: run(config)
│         <module>/              expanded orchestration package
│     tests/                     smoke + integration tests
│     data/                      inputs/{raw,processed}/ & outputs/ (gitignored)
│
├── docs/
│   ├── CyHAN-Standard-v2.2.md                architecture standard
│   └── CyHAN-Comment-Standard-v0.5.2.md        comment and docstring conventions
│
├── backend/   (planned, CyHAN §6.1)          shared API surface above the modules
├── common/    (CyHAN §5.2 / §16.10)          shared library: pystorm_common
│                                             (palette, style_ax, save_figure) - used by all modules
└── archive/                                  pre-refactor snapshot

The root-level integration tier (backend/api/, frontend/) and the shared common/ library are permitted by CyHAN v2.2 but are not required for any module to build or run; each module remains independently operable through its launcher. See Shared common library.

Shared common library

common/python/pystorm_common/ is the shared library that holds cross-module presentation helpers, so there is one source of truth instead of per-module copies. All six modules write their figures through it.

• Contents: the Wave Maker design palette, the style_ax axes-styling helper, and the save_figure writer (which fixes the PyStorm figure DPI standard at DEFAULT_DPI = 150 and creates parent dirs). The palette and style_ax were previously duplicated in POT and PST (the palette byte-for-byte, _style_ax already drifting); every module's figure write now goes through save_figure at the 150 DPI standard. AHD and CSH still pass their fast PNG settings (compress_level, no tight bbox) through save_figure. The one exception is SCA's per-CRL map renderer (a blit/PIL fast-path for ~1000+ maps that does not use savefig), which stays at 110 dpi by design.
• Scope (presentation and pure utilities only): it must not hold module domain logic, numerical kernels, or orchestration - that would couple modules through their compute, not just their look.
• How modules find it: each launcher adds common/python/ to sys.path alongside backend/python/ (tests do the same via conftest.py); it is also pip-installable (pip install -e common/). CyHAN §5.2 permits a module to depend on a shared common library and §16.10 lists it as an optional extension.
• Self-containment caveat: importing from common/ makes it part of a module's dependency set, so vendoring a module standalone means vendoring common/ too. Modules still run in isolation through their launcher; common/ is an integration-tier dependency, not a sibling-module dependency (CyHAN §5.2 forbids the latter).

Acronyms

Acronym	Expansion
AER	Annual Exceedance Rate
AHD	Augmented Hurricane Database
API	Application Programming Interface
CLI	Command-Line Interface
CRL	Coastal Reference Location
CyHAN	C++/Python Hybrid Architecture Network
DSRR	Directional Storm Recurrence Rate
DSW	Discrete Storm Weight
EBTRK	Extended Best Track
GKF	Gaussian Kernel Function
GP	Gaussian Process
GPD	Generalized Pareto Distribution
HC	Hazard Curve
HURDAT2	HURricane DATabase, 2nd generation (NHC best-track)
JDM	Joint Distribution Model
JPM	Joint Probability Method
LCS	Life Cycle Simulation
MRI	Mean Return Interval (MRI = 1 / AER)
MSVC	Microsoft Visual C++ (compiler)
NTR	Non-Tidal Residual
PAM	Partitioning Around Medoids (k-medoids)
POT	Peaks Over Threshold
PST	Probabilistic Simulation Technique
RSS	Reduced Storm Suite
SCA	Storm Climatology Analysis
SRR	Storm Recurrence Rate
CSH	Coastal Storm Hydrograph
TC	Tropical Cyclone

Standards

The architecture follows CyHAN Standard v2.2: each capability is a self-contained module with its own Python orchestration and, where it has performance-critical computation, a C++ engine. See docs/CyHAN-Standard-v2.2.md. Source comments and docstrings follow the Comment and Docstring Standard.