Reference implementation of RAME — a two-stage stacking ensemble that forecasts ground-level ozone 24 hours ahead, station by station, across an urban monitoring network.
Seven diverse deep learners are aggregated with weights inversely proportional to their validation error, and a Bi-LSTM is then trained to predict what that aggregate got wrong. Separating the two stages is the point: the static blend handles calm nocturnal hours where it is already close to correct, and all the learned capacity goes into the part that actually varies — the systematic under-prediction of afternoon photochemical peaks.
Built for the Tehran network (8 stations, hourly, July 2020 – December 2024). Station files are discovered automatically. For another network, configure the column names, units, study periods and quality requirements as described in the data schema.
| If you want to… | Read |
|---|---|
| understand what the code does, stage by stage, in plain text | PROJECT_GUIDE.txt |
| run it | the Install and Quick start sections below |
| find the function behind an equation in the paper | docs/PAPER_MAPPING.md |
| know why each stage is built the way it is | docs/METHODS.md |
| point it at your own data | docs/DATA_SCHEMA.md |
o3-rame-forecasting/
├── PROJECT_GUIDE.txt plain-text walkthrough of the whole pipeline
├── README.md this file — commands and output layout
├── configs/ default.yaml is the published method
├── docs/ paper mapping, methods rationale, data schema
├── src/o3forecast/ the package: one module per concern
│ ├── data/ loading, quality screening, imputation, segments
│ ├── models/ the seven base learners, and RAME
│ └── pipeline/ one module per stage, in run order
└── tests/ paper-fidelity, windowing, and end-to-end tests
raw CSVs
│
├─ 1. PREPROCESS ── hourly grid → 4-criterion Z-score screen → row screening
│ → KNN impute → wind components + meteorological lead
│ → segments → min-max scaler (then verified)
│
├─ 2. BASE MODELS ─ LSTM │ GRU │ Bi-LSTM │ Bi-GRU │ TCN │ TCN-LSTM │ TCN-GRU
│ trained on the first 60% of development,
│ early-stopped on the last 40%
│
├─ 3. RAME ──────── Eq. 8 z̄ = Σ ωₘ zₘ, ωₘ ∝ 1 / RMSE_val(m)
│ Eq. 9 r = y − z̄
│ Eq. 10 r̂ = BiLSTM(z) 256 → 128 → 128 → 64 → 1
│ Eq. 11 ŷ = z̄ + r̂
│
├─ 4. EVALUATE ──── Jul–Dec 2024, read exactly once
│ overall │ 8-hour regulatory windows │ per-hour error
│
└─ 5. SENSITIVITY ─ leave-one-member-out, the ensemble refitted each time
Why seven members. The family is chosen for diversity of inductive bias, which is the precondition for a stack to help at all: gated recurrence (LSTM/GRU), bidirectional encoding of the observed window, and dilated causal convolutions that widen the receptive field without recurrence. Seven models that forget in the same way would have correlated errors, and a meta-learner over correlated members can recover almost nothing.
Why the weights come from validation. Deriving them from test error would let the ensemble tune itself on the data it is about to be scored on — and would guarantee an apparent advantage over the members by construction. A guard recomputes each member's validation RMSE from cached predictions and refuses to run if the stored values disagree.
git clone https://github.com/sinadadras021/O3-RAME-forecasting.git
cd O3-RAME-forecasting
python -m venv .venv
source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1
pip install -e . # or: pip install -e ".[cpu]" for tensorflow-cpu
pip install keras-tcn # required by the three TCN members
pip install -e ".[dev]" # to run the testsRequires Python ≥ 3.10. A GPU is optional. The smoke configuration uses two small GRU members; elapsed time depends on the archive size and hardware.
# 1. Check your configuration before spending any compute
o3forecast validate-config --config configs/default.yaml --raw-dir /path/to/csvs --strict-paths
# 2. Run a small configuration on your own local archive
o3forecast run-all --config configs/quick_test.yaml --raw-dir /path/to/csvs
# 3. The real run
o3forecast run-all --config configs/default.yaml --raw-dir /path/to/csvsStages also run individually — each reads the previous stage's artefacts from disk:
o3forecast preprocess --config configs/default.yaml --raw-dir /path/to/csvs
o3forecast train-base --config configs/default.yaml
o3forecast train-meta --config configs/default.yaml
o3forecast evaluate --config configs/default.yaml
o3forecast sensitivity --config configs/default.yamlo3forecast run-all --raw-dir /data/o3 --work-dir /scratch/run17 # CLI flags win
RAME_PATHS__RAW_DIR=/data/o3 o3forecast run-all # environment
# ...or edit paths.raw_dir in your YAML configPrecedence: defaults → YAML → environment → CLI flags. No interactive prompts anywhere, by design: prompts cannot be scripted, cannot run in CI, and cannot be recorded in a provenance file.
| file | purpose |
|---|---|
configs/default.yaml |
the published method, exactly |
configs/extended.yaml |
optional robustness settings, all beyond the paper |
configs/quick_test.yaml |
Two GRU members: 1 base epoch and 2 meta epochs; a smoke run, not the research configuration |
Every key is documented in src/o3forecast/config.py, which is the schema. The
YAML only overrides fields that already exist there, and a typo raises rather than
being silently ignored.
artifacts/
├── resolved_config.yaml # exactly what ran, for provenance
├── run.log
├── 01_cleaned/ # per-station parquet + fitted scalers
├── 02_base_models/<station>/ # *.keras, meta_model.keras, rame_meta.json
├── 04_results/
│ ├── station_quality_report.csv # who was admitted, and why not
│ ├── metrics_all_stations.csv
│ ├── metrics_city_mean.csv
│ ├── window8h_<station>.csv # per 8-hour regulatory window
│ ├── hourly_rmse_<station>.csv # per lead hour
│ ├── critical_days_<station>.csv # exceedance classification, per day
│ ├── sensitivity_all_stations.csv
│ └── predictions_<station>.parquet # redraw any figure without retraining
└── 05_figures/
rame_meta.json records the Eq. 8 weights, the residual statistics, and the
before/after RMSE of the residual correction — so the paper's central claim can
be checked, not just read.
Enforced at runtime, not left to reviewer trust:
- Derive the ensemble weights from test error. The weights come from validation RMSE, and the values are recomputed from cached predictions and compared before the meta-learner is fitted.
- Select a model on test metrics. The meta-learner is fitted once, as the paper describes; if restarts are enabled, the winner is chosen on internal validation loss and the config emits a warning that it departs from the paper.
- Clip out-of-bound readings into range. Configured plausibility limits
replace these readings with
NaNbefore imputation. Check those limits against the station instruments and site conditions before using a new archive. - Flag a value on magnitude alone. All four outlier criteria must fire simultaneously. The rule is a screening heuristic, not verified sensor-fault ground truth.
- Let a window cross a data gap or a split boundary. Segments break on gaps in
the timestamp index as well as on
NaN; window intervals are reconstructed and compared across splits. - Score a window that is not one calendar day. Test windows are anchored at midnight, which is what makes the 8-hour regulatory windows and the critical-day classification well defined.
For every station: each of the seven members, the Eq. 8 weighted baseline on its own, and full RAME.
Reporting the baseline separately is what makes the paper's claim falsifiable — without it, "RAME beats the base models" could be explained entirely by averaging.
Three views, following the paper:
- Overall — RMSE, MAE, R², RRMSE (range-normalised, and the mean-normalised variant alongside it, each named explicitly because the same forecast can be "6.6 % error" or "23 % error" depending on the denominator)
- 8-hour regulatory windows — the quantity the ozone standard is written against, plus critical-day classification at 70 ppb with POD, FAR, CSI and HSS
- Per-hour error distribution — which is what reveals a systematic afternoon under-prediction that an aggregate RMSE hides completely
pytest -m "not slow" # unit and paper-fidelity tests
pytest -m slow # end-to-end run on synthetic stations
pytest --cov=o3forecast # with coveragetests/test_paper_fidelity.py pins the implementation to the manuscript: each
test names the section or equation it protects, so if a default changes later,
one of them fails and says which published claim it just broke.
The synthetic stations in tests/conftest.py use illustrative diurnal, seasonal,
titration and ventilation terms —
then deliberately damaged with outages, instrument spikes and duplicated
timestamps, plus one station engineered to fail the quality gate.
This is a retrospective implementation. Meteorological inputs are shifted from the observed archive under a perfect-prognosis assumption; they are not a live weather-forecast feed. Read the method limitations before interpreting operational forecast skill. No new research experiment was performed as part of this documentation update.
No data is distributed with this repository. The observations used in the associated study belong to the Tehran Air Quality Control Company (pollutants) and the Iran Meteorological Organization (meteorology), and are available from them on request.
docs/DATA_SCHEMA.md specifies exactly what the code
expects.
docs/PAPER_MAPPING.md— every equation and table of the paper mapped to the function that implements itdocs/METHODS.md— the rationale for each stage and an explicit list of assumptions and limitationsdocs/DATA_SCHEMA.md— input format, units, adaptation
If this code contributes to your work, please cite it via
CITATION.cff.
MIT — see LICENSE.
Author: Sina Dadras · CV