diff --git a/README.md b/README.md index 1cafef6..271c5f2 100644 --- a/README.md +++ b/README.md @@ -284,4 +284,4 @@ LDSFL-Meander is named after and authored by: - Sergio Lopez-Dubon - Alessandro Sgarabotto - Alessandro Frascati -- Stefano Lanzoni +- Stefano Lanzoni\n\n## Documentation\n\n- [Theory-to-code mapping](docs/theory_code_mapping.md)\n- [Validation strategy and known limitations](docs/validation_strategy.md)\n\n \ No newline at end of file diff --git a/docs/theory_code_mapping.md b/docs/theory_code_mapping.md new file mode 100644 index 0000000..e438712 --- /dev/null +++ b/docs/theory_code_mapping.md @@ -0,0 +1,162 @@ +# Theory-to-code mapping + +This document maps the main LDSFL-Meander modelling concepts to the repository implementation. It is intended to make the code easier to audit, test, and cite as research software. + +## Purpose + +The repository implements a reduced morphodynamic meander model. The code is organised around a repeatable pipeline: + +1. read dimensionless model parameters and an initial centreline; +2. preprocess the centreline geometry; +3. evaluate the hydrodynamic/free-flow response; +4. advance the centreline; +5. update geometry and morphodynamic parameters; +6. save geometry, variable history, sinuosity history, and diagnostics. + +The mapping below is deliberately implementation-oriented. It does not replace the paper or original derivation; it helps readers find where each model component lives in the code. + +## Pipeline map + +| Model step | Implementation | Notes | +|---|---|---| +| Input parameters | `ldsfl.inputs.read_parameter_table`, `ldsfl.inputs.dimensionless_input_table` | Reads `Input/Parameter.csv` and converts each row into the dimensionless parameters used by the solver. | +| Initial centreline | `ldsfl.inputs.read_xy`, `ldsfl.profile.preprof_3` | Reads `Input/xy.csv`, computes arclength, resampled coordinates, tangent angle, wavelength, valley length, and initial sinuosity. | +| Initial curvature | `ldsfl.main.initial_curvature` | Uses the same tangent-angle convention as the geometry update, avoiding a first-step sign flip. | +| Resistance terms | `ldsfl.resistance.resistance_function_flagbed` | Computes resistance-related coefficients for plane-bed and dune-bed cases. | +| Flow-field response | `ldsfl.flowfield.parall_u_free` | Main free-boundary flow response used by default runs. Supports NumPy and optional Numba backends. | +| Periodic flow response | `ldsfl.flowfield_periodic.parall_u_periodic` | Alternative periodic path. Treat as experimental unless specifically validated for the target case. | +| Centreline migration | `ldsfl.evolution.dxdy2` | Converts flow response into centreline displacement increments with a timestep stability coefficient. | +| Geometry update | `ldsfl.geometry.geometry4` | Resamples, smooths, detects cutoffs, updates curvature and sinuosity, and writes cutoff geometries. | +| Parameter update | `ldsfl.evolution.update_parameters` | Updates evolving parameters after geometry changes. | +| Output writing | `ldsfl.outputs` | Writes geometry snapshots, variable-history CSVs, sinuosity history, and figures. | +| Stability diagnostics | `ldsfl.main._sinuosity_stability_metrics`, `ldsfl.stability.sinuosity_equivalence_stability` | Moving-window diagnostic is lightweight. Equivalence/HAC diagnostic is explicit because it can be expensive for long histories. | +| CLI entry point | `run_ldsfl.py` | Provides a command-line interface around `ldsfl.main.run_project`. | +| GUI entry point | `gui_ldsfl.py` | Interactive interface for running and inspecting simulations. | + +## Main runtime call graph + +The high-level runtime flow is: + +```text +run_ldsfl.py + -> ldsfl.main.run_project() + -> ldsfl.main.run_case() + -> read inputs + -> preprof_3() + -> initial_curvature() + -> resistance_function_flagbed() + -> loop: + -> parall_u_free() or parall_u_periodic() + -> save_xystcu() / save_variables() at output intervals + -> dxdy2() + -> geometry4() + -> resistance_function_flagbed() + -> update_parameters() + -> sinuosity diagnostics + -> final snapshot and histories +``` + +## Boundary-condition status + +The default and currently best-supported path is the free-boundary flow solver: + +```text +flow_bc="free" +``` + +The periodic path exists for experimentation and comparison: + +```text +flow_bc="periodic" +``` + +The periodic path should not be presented as equally validated until it has dedicated regression tests and physical comparison cases. For publication-quality runs, use the free-boundary path unless there is a specific reason and validation record for the periodic path. + +## Backend status + +The default backend is NumPy: + +```text +backend="numpy" +``` + +The optional Numba backend is intended as an acceleration path. It is an optional extra and should be tested against NumPy for the exact solver branch being used. + +Current testing distinguishes between: + +- default NumPy execution; +- NumPy serial versus threaded mode accumulation; +- optional Numba parity where validated; +- known or suspected backend discrepancies marked as expected failures until investigated. + +## Stability diagnostics + +Two sinuosity diagnostics are available. + +### Lightweight moving-window diagnostic + +Implemented in: + +```text +ldsfl.main._sinuosity_stability_metrics +``` + +This diagnostic is cheap and suitable for default return values and GUI display. + +### Equivalence/HAC diagnostic + +Implemented in: + +```text +ldsfl.stability.sinuosity_equivalence_stability +``` + +This diagnostic estimates total fitted drift over a post-transient analysis window and checks whether the confidence interval lies within a practical drift tolerance. It is more statistically meaningful, but it is more expensive because it uses a HAC/Newey-West-style covariance calculation. + +For that reason, it should be computed only when explicitly needed, for example when: + +```text +stop_on_sinuosity_stability=True +``` + +or when the CLI/user explicitly requests it: + +```text +--return-equivalence-stability 1 +``` + +## Output map + +| Output type | Location | Producer | +|---|---|---| +| Geometry snapshots | `Output//files/xyu_*.csv` | `ldsfl.outputs.save_xystcu` | +| Variable history | `Output//files/var_*.csv` | `ldsfl.outputs.save_variables` | +| Sinuosity history | `Output//files/sinuosity_history_*.csv` | `ldsfl.outputs.save_sinuosity_history` | +| Planform figures | `Output//figures/` | `ldsfl.outputs.plot_it` | +| Cutoff files | `Output//files/` | `ldsfl.geometry.save_xy_cut` | +| Cutoff figures | `Output//figures/` | `ldsfl.geometry.plot_cut` | + +## Testing map + +| Test topic | Typical files | +|---|---| +| CLI parsing and validation | `tests/test_cli_config.py`, CLI-focused tests | +| Output bookkeeping | `tests/test_variable_history_complete.py` | +| Initial curvature sign | `tests/test_curvature_sign_consistency.py` | +| Plot disabling / no-plots behaviour | `tests/test_cutoff_no_plots.py` | +| Stop criteria and stability | `tests/test_sinuosity_stability_stop.py`, `tests/test_sinuosity_equivalence_performance_guard.py` | +| Flow-field physics/regression | `tests/test_flowfield_physics_regression.py`, `tests/test_flowfield_backend_guardrails.py` | + +## Maintenance notes + +When changing the solver, update this document if the change affects: + +- model inputs; +- boundary-condition behaviour; +- backend behaviour; +- geometry conventions; +- timestep/stability logic; +- output file semantics; +- validation evidence. + +Small refactors that do not change behaviour should still preserve the mapping between the model concepts and the public functions listed here. diff --git a/docs/validation_strategy.md b/docs/validation_strategy.md new file mode 100644 index 0000000..5c61291 --- /dev/null +++ b/docs/validation_strategy.md @@ -0,0 +1,92 @@ +# Validation strategy and known limitations + +This document summarises the current validation strategy for LDSFL-Meander and records known limitations that should be considered when using the model for research or portfolio demonstrations. + +## Validation philosophy + +The repository uses a layered validation approach: + +1. **Unit-level checks** for input parsing, CLI behaviour, and small helper functions. +2. **Regression tests** for solver-output bookkeeping and numerical conventions. +3. **Physics-oriented tests** for invariants that should hold independently of a specific benchmark dataset. +4. **Integration tests** for short reproducible runs. +5. **Documented expected failures** for paths that are useful to keep visible but are not yet validated. + +This is intentionally different from claiming full physical validation for all possible settings. The tests provide guardrails against implementation errors and regressions. + +## Current validated behaviours + +| Area | Evidence | +|---|---| +| CLI case parsing | Tests cover empty cases, invalid case IDs, repeated cases, and ranges. | +| Initial curvature convention | Test checks that the first curvature calculation matches the geometry update convention. | +| Variable-history output | Test checks that every solver step is recorded exactly once, including final partial save blocks. | +| Plot disabling | Test checks that cutoff CSVs can still be written while plotting is disabled. | +| Stop-on-sinuosity-stability | Test checks that the stability stop can fire before `max_steps`. | +| Equivalence/HAC performance guard | Test checks that expensive equivalence diagnostics are not computed by default. | +| Flow-field invariants | Tests check zero-curvature response, linearity in curvature amplitude, and serial/threaded NumPy agreement. | + +## Known limitations + +### Periodic flow boundary condition + +The periodic flow path is present but should be treated as experimental until it has dedicated validation against a trusted reference. + +Recommended wording: + +```text +The free-boundary flow solver is the default validated path. The periodic boundary-condition path is retained for experimental comparison and should be validated for the target study before use. +``` + +### Optional Numba backend + +The Numba backend is optional. It should be used only when installed and when the relevant solver path has been compared against the NumPy backend. + +Recommended wording: + +```text +NumPy is the reference backend. Numba is an optional acceleration backend and should be checked against NumPy for the selected solver branch. +``` + +### Statistical stability + +The moving-window stability diagnostic is lightweight and useful for monitoring. The equivalence/HAC diagnostic is more rigorous but more expensive. It is therefore opt-in unless used for a stopping criterion. + +Recommended wording: + +```text +Equivalence-style sinuosity stability is available for publication-style analysis, but it is not computed by default for long runs unless requested. +``` + +### Demonstration examples + +Short example runs are designed for reproducibility and continuous integration. They are not a substitute for full scientific calibration or site-specific validation. + +## Suggested validation roadmap + +The following additions would strengthen the repository further: + +1. Add one or more external benchmark cases with expected summary outputs. +2. Add a documented periodic-boundary comparison test if the periodic path is retained. +3. Add a small analytic transfer-function style test for the free-flow solver if the assumptions can be clearly documented. +4. Add reproducibility artefacts for a paper figure or reference run. +5. Add a versioned `docs/validation_matrix.md` table linking each claim in the README to code/tests. + +## What to avoid + +Avoid mixing unrelated changes into validation PRs. In particular: + +- do not combine solver changes with documentation-only changes; +- do not regenerate large fixture files in the same PR as code refactors; +- do not enable strict lint gates until the repository is fully lint-clean; +- do not mark optional accelerated backends as equivalent unless the tested path supports that claim. + +## Review checklist + +Before merging a validation-related PR, check: + +- Does the PR state whether it changes behaviour or only adds tests/docs? +- Are new tests focused and deterministic? +- Do tests run without optional dependencies unless explicitly skipped? +- Does the PR avoid committing generated output files? +- Does the README avoid overclaiming beyond the available validation evidence? diff --git a/ldsfl/main.py b/ldsfl/main.py index 2a0f93b..ced0311 100644 --- a/ldsfl/main.py +++ b/ldsfl/main.py @@ -306,7 +306,7 @@ def run_case( sinuo_hist: list[float] = [float(sinuo)] stability_interval = max(1, int(sinuo_stability_interval)) use_equivalence_stability = bool(stop_on_sinuosity_stability) or bool(return_equivalence_stability) - if use_equivalence_stability: + if bool(stop_on_sinuosity_stability): stability_info = _combined_sinuosity_stability_metrics( step_hist, sinuo_hist, @@ -595,7 +595,7 @@ def run_case( steps += 1 step_hist.append(int(steps)) sinuo_hist.append(float(sinuo)) - if use_equivalence_stability and (steps % stability_interval) == 0: + if bool(stop_on_sinuosity_stability) and (steps % stability_interval) == 0: stability_info = _combined_sinuosity_stability_metrics( step_hist, sinuo_hist, @@ -615,7 +615,7 @@ def run_case( window=sinuo_window, rel_tol=sinuo_rel_tol, ) - if use_equivalence_stability and previous_equivalence is not None: + if bool(stop_on_sinuosity_stability) and previous_equivalence is not None: stability_info["equivalence"] = previous_equivalence jt += 1 diff --git a/tests/test_sinuosity_equivalence_performance_guard.py b/tests/test_sinuosity_equivalence_performance_guard.py index de071a9..e01aeec 100644 --- a/tests/test_sinuosity_equivalence_performance_guard.py +++ b/tests/test_sinuosity_equivalence_performance_guard.py @@ -64,5 +64,5 @@ def fake_equivalence(*args, **kwargs): do_plots=False, ) - assert calls["n"] >= 1 + assert calls["n"] == 1 assert result["sinuosity_stability"]["equivalence"]["stable"] is True