NephroQ

A mechanistic digital twin for Type 2 Diabetes → Chronic Kidney Disease

A calibratable, nonlinear mechanistic model to predict renal function (eGFR) progression in type 2 diabetic patients and estimate time to dialysis, with hierarchical Bayesian inference and validation against published data.

Why "NephroQ": the model's central, physically meaningful parameter is q — the hyperfiltration feedback exponent that quantifies how abrupt the terminal collapse of renal function is. It is estimated from clinical trajectories, not just fitted as a black box.

Author: Daniel Pérez-Calixto — INMEGEN / UNAM (Facultad de Ciencias)

---

What this is

A low-dimensional dynamical system describing renal function decline (N = functional nephron mass fraction) through two physical mechanisms:

• Hyperfiltration — positive feedback: as nephrons are lost, the remaining ones become overloaded and are damaged faster (power law, exponent q).
• Compensation — observed eGFR is buffered while there is functional reserve and collapses at the end (weak power law, exponent α).

dN/dt = −N · [ k0 + k_hf·(1/N)^q + I(HbA1c, UACR, blood pressure) ]
eGFR  = G_max · N^α

The central parameter, q (the feedback / "how abrupt is the terminal collapse" exponent), is estimated from clinical trajectories using three inference methods depending on data availability: per-patient fitting, an amortized AI estimator (for scarce data), and a full hierarchical Bayesian model (for cohorts with uneven follow-up).

Full documentation: docs/MODEL_DOCUMENTATION.md.

---

Repository structure

├── src/                          # All the code, ready to run
│   ├── model_core.py             # SINGLE source of truth for the model (imported by app + calibration)
│   ├── mechanistic_twin.py       # Nonlinear core (hyperfiltration + compensation)
│   ├── hybrid_twin.py            # Matrix variant: state-space + absorbing Markov + Kalman
│   ├── egfr_measurement.py       # CKD-EPI 2021 equations (creatinine / cystatin / combined)
│   ├── inverse_fit.py            # Inverse problem: parameter recovery (synthetic data)
│   ├── noise_identifiability.py  # Effect of the lab assay on the identifiability of q
│   ├── amortized_ai.py           # Amortized AI estimator (network ensemble) for scarce data
│   ├── hierarchical_model.py     # Mixed effects with partial pooling (EM)
│   ├── bayesian_model.py         # Full hierarchical Bayesian inference (adaptive Metropolis)
│   ├── forecast_comparison.py    # Compares forecasting with biased vs Bayesian q
│   ├── real_data_validity.py     # Face validity against real profiles (Al-Shamsi 2018)
│   ├── mimic_loader.py           # MIMIC-IV (PhysioNet) loader -> pipeline schema
│   ├── calibrate_mimic.py        # Local calibration with MIMIC-IV -> calibration/mimic_calibration.json
│   └── mvp_calibration.py        # MVP: calibrates and validates with real or synthetic data
├── docs/                         # Documentation and protocol
│   ├── MODEL_DOCUMENTATION.md               # Mathematical spec + implementation guide + publication analysis
│   ├── digital_twin_protocol.md
│   ├── validation_report_example.md
│   ├── WEB_DEPLOYMENT.md
│   ├── MIMIC_COMPLIANCE.md                  # Summary: MIMIC-IV license compliance
│   ├── CHANGELOG.md                         # History of fixes (code review rounds)
│   └── KNOWN_ISSUES.md                      # Currently open limitations
├── results/                      # Figures generated by each script
├── calibration/                  # mimic_calibration.json (NOT in git) + README.md (handling policy)
├── data/                         # Place your real data here (not versioned, see .gitignore)
├── tests/                        # Unit tests
├── app_web.py                    # Streamlit web interface
├── requirements.txt
└── README.md

---

Maturity status: TRL 4

The system is integrated, with automated tests and auditable evidence per run. Each system_twin.py run produces a manifest.json recording what ran, how long it took, and whether the system passed or failed overall.

cd src
python system_twin.py --skip-slow --skip-bayes   # fast full-system run (~1 min)
cd .. && python -m pytest tests/ -v                # 23 unit tests

Each system_twin.py run generates results/system_run_<timestamp>/ with a log per stage and a manifest.json recording what ran, how long it took, and whether the system passed or failed overall.

---

Calibrate with your local copy of MIMIC-IV

cd src
python calibrate_mimic.py --mimic-dir /path/to/your/mimic-iv/hosp

Runs 100% on your machine — uploads nothing. Generates calibration/mimic_calibration.json (excluded from git) with the calibrated aggregate parameters; the intermediate per-patient CSV is deleted automatically. See calibration/README.md for the handling policy of this file (shareable "upon reasonable request" in the publication) and docs/MIMIC_COMPLIANCE.md for license detail.

Covariates (HbA1c/UACR/SBP) are time-varying by default — each visit uses its own nearest measurement, falling back to a patient baseline and then population imputation only where needed (see docs/CHANGELOG.md Round 5). A patient-level bootstrap (--n-bootstrap, default 15) also runs after the fit, so the app can show a 90% prediction interval instead of a bare point estimate; use --n-bootstrap 0 to skip it for a faster run.

Reproducible environment (Docker)

docker build -f docker/Dockerfile -t nephroq .
docker run --rm nephroq

The build only succeeds if the unit tests pass (quality gate). Build context is the repository root — the -f flag just points to the Dockerfile's location, . at the end keeps the context correct.

Web interface

app_web.py — an interactive (Streamlit) app to enter a patient's markers and see the risk projection, designed to share with physicians.

streamlit run app_web.py

The active calibration is resolved automatically across three tiers (highest to lowest priority):

1. Private (st.secrets) — for a future real clinical cohort, not yet active.
2. Local MIMIC-IV (calibration/mimic_calibration.json) — this repository's research calibration, if you've already generated it.
3. Public fallback (synthetic + Al-Shamsi 2018 validation) — so the demo works even if someone clones the repo without having run calibrate_mimic.py.

Deploys for free and automatically from this same GitHub repo (Streamlit Community Cloud or Hugging Face Spaces) — see docs/WEB_DEPLOYMENT.md for the step-by-step guide and the evolution path toward a backend/frontend architecture once the project matures beyond TRL5.

---

Installation

git clone https://github.com/Danpc11/nephroq.git
cd nephroq
python -m venv venv && source venv/bin/activate    # optional but recommended
pip install -r requirements.txt

Requires Python ≥ 3.10.

---

How to run it — guided path

Each script is self-contained, runs in seconds/minutes, and prints its own numerical verification results. Run from src/:

cd src

# 1. Nonlinear mechanistic core
python mechanistic_twin.py

# 2. (Optional) Analytic matrix variant: state-space + Markov + exact Kalman
python hybrid_twin.py

# 3. eGFR measurement model (CKD-EPI 2021)
python egfr_measurement.py

# 4. Inverse problem: verify parameter recovery on synthetic data
python inverse_fit.py

# 5. Identifiability of q depending on the lab assay used
python noise_identifiability.py

# 6. Amortized AI estimator (for patients with few visits)
python amortized_ai.py

# 7. Hierarchical model with partial pooling
python hierarchical_model.py

# 8. Full hierarchical Bayesian inference (corrects the bias in q)
python bayesian_model.py

# 9. Compares forecasting with biased q (EM) vs Bayesian q
python forecast_comparison.py

# 10. Face validity against real published profiles (Al-Shamsi 2018)
python real_data_validity.py

# 11. MVP: full calibration + validation (real or synthetic data)
python mvp_calibration.py

Key checkpoint at step 4: chi²/n at theta_true must be ≈ 1. If not, there is a numerical issue to fix before touching real data.

With real data

mvp_calibration.py accepts a CSV with columns patient_id, time_years, egfr, hba1c, uacr, sbp:

CKD_CSV=../data/my_data.csv python mvp_calibration.py

Generates results/mvp_validation.png and results/validation_report.md — the one-page report designed to share with physicians/clinical collaborators.

Real-data sources evaluated for future calibration: HCHS/SOL, CRIC, AASK (via NIDDK Central Repository, request-based access), Synthea (open synthetic data), and MIMIC-IV (PhysioNet, see docs/MIMIC_COMPLIANCE.md).

---

This repository incorporates fixes from several rounds of detailed code review — see docs/CHANGELOG.md for the full history (including two critical bugs that affected the app's projections) and docs/KNOWN_ISSUES.md for what's still open today.

Main results (synthetic data, verified)

Validation	Result
Parameter recovery (synthetic)	chi²/n ≈ 1.0; q recovered within ±1σ
Identifiability of q by assay	creatinine only: ±0.15 → creatinine+cystatin: ±0.03
Amortized AI vs per-patient fit (3-4 visits)	AI is more stable and does not diverge
Hierarchical model (partial pooling), patients with 3-5 visits	error in η_i ~2.7× lower than without pooling
Hierarchical Bayesian, bias in q	EM: q=1.06 (biased) → Bayesian: q=1.52 [1.45,1.60] (truth 1.6)
Forecast improvement from correcting q	RMSE −6% overall, −7% in progressors
Face validity (real profiles, Al-Shamsi 2018)	progressors: 4.0 years to stage 3; non-progressors: 13.4 years

---

Project status and what's needed for publication

See section 6 of docs/MODEL_DOCUMENTATION.md for the full analysis. In summary: the model is verified on synthetic data and has a first face-validity check against real published data (Al-Shamsi et al. 2018, PLOS One, S1 Dataset, CC BY 4.0). Missing: calibration and external validation with a real longitudinal cohort (eGFR repeated per visit), in-silico replication of at least one clinical trial (DAPA-CKD/CREDENCE/FLOW), and comparison against the standard risk equation (KFRE).

License and data

Code under the MIT license (see LICENSE). This repository does not include patient data. The data/ folder is only a local mount point for personal use (excluded from git); any real data used with this code must be handled per its own data use agreement and ethics approval.

How to cite

If this code is useful in your work, please cite the repository. See CITATION.cff.