From eb08706526606bcd228a225023c4d84ea872fcb8 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 00:19:38 +0200 Subject: [PATCH 01/18] docs: establish canonical mission, architecture, model and source-of-truth baseline --- docs/DOCUMENTATION_INVENTORY.yaml | 173 ++++++++++++++ docs/architecture/CURRENT_SYSTEM.md | 145 +++++++++++ docs/architecture/TARGET_VIRTUAL_LAB.md | 151 ++++++++++++ docs/governance/DOCUMENTATION_POLICY.md | 99 ++++++++ docs/governance/SOURCE_OF_TRUTH.md | 78 ++++++ docs/models/REGISTRY.md | 238 +++++++++++++++++++ docs/product/CAPABILITIES_AND_LIMITATIONS.md | 89 +++++++ docs/product/CURRENT_STATE.md | 95 ++++++++ docs/product/MISSION_AND_NORTH_STAR.md | 133 +++++++++++ docs/product/ROADMAP.md | 54 +++++ 10 files changed, 1255 insertions(+) create mode 100644 docs/DOCUMENTATION_INVENTORY.yaml create mode 100644 docs/architecture/CURRENT_SYSTEM.md create mode 100644 docs/architecture/TARGET_VIRTUAL_LAB.md create mode 100644 docs/governance/DOCUMENTATION_POLICY.md create mode 100644 docs/governance/SOURCE_OF_TRUTH.md create mode 100644 docs/models/REGISTRY.md create mode 100644 docs/product/CAPABILITIES_AND_LIMITATIONS.md create mode 100644 docs/product/CURRENT_STATE.md create mode 100644 docs/product/MISSION_AND_NORTH_STAR.md create mode 100644 docs/product/ROADMAP.md diff --git a/docs/DOCUMENTATION_INVENTORY.yaml b/docs/DOCUMENTATION_INVENTORY.yaml new file mode 100644 index 0000000..900a22e --- /dev/null +++ b/docs/DOCUMENTATION_INVENTORY.yaml @@ -0,0 +1,173 @@ +inventory_version: documentation-inventory-v1 +verified_at: 2026-08-24 +verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +owner: Andrea Zedda +classification_resolution: first_matching_rule +unmatched_classification: UNCLASSIFIED_REQUIRES_REVIEW +navigation_policy: + allowed: + - CANONICAL + - CURRENT_SUPPORTING + - TARGET_SUPPORTING + forbidden: + - HISTORICAL_ARCHIVE + - DUPLICATE + - DELETE_CANDIDATE + - UNCLASSIFIED_REQUIRES_REVIEW +explicit_entries: + - path: README.md + classification: CANONICAL + audience: all + replacement_or_parent: docs/index.md + reason: repository landing page + - path: CHANGELOG.md + classification: CANONICAL + audience: maintainers + replacement_or_parent: null + reason: current release history governed by claims policy + - path: SECURITY.md + classification: CANONICAL + audience: security reporters and deployers + replacement_or_parent: docs/operations/AUTHENTICATION_AND_ACCESS.md + reason: public security boundary and reporting path + - path: CONTRIBUTING.md + classification: CURRENT_SUPPORTING + audience: contributors + replacement_or_parent: docs/governance/DOCUMENTATION_POLICY.md + reason: contributor workflow; CI details updated under issue 13 + - path: RELEASE_PREPARATION.md + classification: HISTORICAL_ARCHIVE + audience: historical + replacement_or_parent: docs/archive/releases/RELEASE_PREPARATION_2024.md + reason: obsolete 2024 release and coverage claims + - path: docs/index.md + classification: CANONICAL + audience: all + replacement_or_parent: null + reason: documentation landing page + - path: docs/guides/architecture.md + classification: CURRENT_SUPPORTING + audience: existing links + replacement_or_parent: docs/architecture/CURRENT_SYSTEM.md + reason: compatibility pointer, not separate architecture truth +rules: + - glob: docs/product/** + classification: CANONICAL + reason: mission, current state, capabilities and roadmap + - glob: docs/architecture/CURRENT_SYSTEM.md + classification: CANONICAL + reason: verified current architecture + - glob: docs/architecture/TARGET_VIRTUAL_LAB.md + classification: TARGET_SUPPORTING + reason: approved target architecture, explicitly non-current + - glob: docs/models/** + classification: CANONICAL + reason: current model mapping and status + - glob: docs/governance/INTENDED_USE.md + classification: CANONICAL + reason: intended-use authority + - glob: docs/governance/CLAIMS_POLICY.md + classification: CANONICAL + reason: claims authority + - glob: docs/governance/EPISTEMIC_LABELS.md + classification: CANONICAL + reason: epistemic label authority + - glob: docs/governance/MODEL_OUTPUT_LANGUAGE.md + classification: CANONICAL + reason: output-language authority + - glob: docs/governance/SOURCE_OF_TRUTH.md + classification: CANONICAL + reason: conflict-resolution authority + - glob: docs/governance/DOCUMENTATION_POLICY.md + classification: CANONICAL + reason: documentation governance + - glob: docs/research/STRUCTURED_DATASET_CONTRACT_V0_1.md + classification: CANONICAL + reason: dataset contract + - glob: docs/research/RUN_MANIFESTS.md + classification: CANONICAL + reason: run identity and artifact contract + - glob: docs/research/NUMERICAL_REPRODUCIBILITY.md + classification: CANONICAL + reason: numerical policy + - glob: docs/research/VALIDATION_UNCERTAINTY_PROTOCOL.md + classification: CURRENT_SUPPORTING + reason: current validation protocol; maturity limits apply + - glob: docs/research/REAL_PATIENT_WHATIF_ARCHITECTURE.md + classification: CURRENT_SUPPORTING + reason: research workflow design with current/target distinctions + - glob: docs/research/** + classification: CURRENT_SUPPORTING + reason: research documentation requiring status metadata review + - glob: docs/operations/DEPENDENCIES.md + classification: CANONICAL + reason: dependency operations + - glob: docs/operations/DEPENDENCY_AUDIT_TRIAGE.md + classification: CURRENT_SUPPORTING + reason: current audit record + - glob: docs/operations/** + classification: CURRENT_SUPPORTING + reason: current operations documentation + - glob: docs/reference/** + classification: CURRENT_SUPPORTING + reason: generated or manually maintained reference; code consistency required + - glob: docs/guides/mathematical_models*.md + classification: CURRENT_SUPPORTING + reason: current runtime formula guide; registry governs status + - glob: docs/guides/glossary.md + classification: CURRENT_SUPPORTING + reason: learning support + - glob: docs/guides/learning_path.md + classification: CURRENT_SUPPORTING + reason: learning support + - glob: docs/guides/simulator_kpis.md + classification: CURRENT_SUPPORTING + reason: KPI interpretation support + - glob: docs/guides/decision_algorithm.md + classification: CURRENT_SUPPORTING + reason: model-relative thresholds only + - glob: docs/guides/** + classification: UNCLASSIFIED_REQUIRES_REVIEW + reason: mixed generations; excluded from canonical navigation until reviewed + - glob: docs/development/DEVELOPMENT.md + classification: CURRENT_SUPPORTING + reason: development setup + - glob: docs/development/TROUBLESHOOTING.md + classification: CURRENT_SUPPORTING + reason: troubleshooting + - glob: docs/development/**SUMMARY*.md + classification: HISTORICAL_ARCHIVE + reason: implementation history + - glob: docs/development/**LOG*.md + classification: HISTORICAL_ARCHIVE + reason: implementation history + - glob: docs/development/** + classification: UNCLASSIFIED_REQUIRES_REVIEW + reason: mixed current and historical content + - glob: docs/features/** + classification: HISTORICAL_ARCHIVE + reason: feature implementation history, not current source of truth + - glob: docs/archive/** + classification: HISTORICAL_ARCHIVE + reason: explicit archive + - glob: docs/en/** + classification: UNCLASSIFIED_REQUIRES_REVIEW + reason: translation/content consistency review required + - glob: docs/it/** + classification: UNCLASSIFIED_REQUIRES_REVIEW + reason: translation/content consistency review required + - glob: docs/API_*.md + classification: HISTORICAL_ARCHIVE + reason: historical API implementation narratives + - glob: docs/*SUMMARY*.md + classification: HISTORICAL_ARCHIVE + reason: historical implementation summary + - glob: docs/*ROADMAP*.md + classification: HISTORICAL_ARCHIVE + reason: superseded by product roadmap and GitHub issue 31 + - glob: docs/*.md + classification: UNCLASSIFIED_REQUIRES_REVIEW + reason: root documentation requires explicit review + - glob: '*.md' + classification: HISTORICAL_ARCHIVE + reason: root implementation/debug narrative unless explicitly overridden diff --git a/docs/architecture/CURRENT_SYSTEM.md b/docs/architecture/CURRENT_SYSTEM.md new file mode 100644 index 0000000..9cb61af --- /dev/null +++ b/docs/architecture/CURRENT_SYSTEM.md @@ -0,0 +1,145 @@ +--- +title: Current System Architecture +status: CURRENT_VERIFIED +owner: Andrea Zedda +audience: engineers, researchers and reviewers +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +source_of_truth: repository tree, Django configuration, model registry and current contracts +--- + +# Current system architecture + +> **CURRENT architecture.** This page describes the repository at the verified SHA. It is not the target virtual-laboratory architecture. + +## System context + +```mermaid +flowchart LR + U["Research user"] --> W["Django web application"] + W --> DB[("Relational database")] + W --> FS[("Static, media and research artifacts")] + W --> R["Redis / Celery when enabled"] + W --> DOCS["MkDocs and in-app documentation"] +``` + +The primary UI is server-rendered Django with incremental HTMX interactions. Development uses SQLite by default. PostgreSQL and a production-grade runtime are target M0-S concerns unless separately verified in a deployment. + +## Current application containers + +```mermaid +flowchart TB + subgraph APP["Django project: mmportal"] + CL["clinic\nstructured records and navigation"] + SI["simulator\nmechanistic scenarios and design prototypes"] + TW["twin_engine\nstate, lineage, runs and research diagnostics"] + CH["chemtools\noptional molecular utilities"] + DV["docs_viewer\nin-app documentation"] + GV["governance\nmachine-readable release claims"] + end + + CL --> TW + TW --> SI + SI --> CH + GV --> CL + GV --> SI + GV --> TW +``` + +## Bounded responsibilities + +### `clinic` + +- structured patient, assessment and treatment entities; +- patient list/detail and clinical-history surfaces; +- ownership fields and selected API filtering; +- current product-authentication entry points; +- temporary governed smoke-user command. + +Current limitation: object-level authorization is not yet consistently centralized across all web views and aggregate queries. + +### `simulator` + +- `MathematicalModel` ODE execution; +- scenario and `SimulationAttempt` workflows; +- PK/PD presets and time-resolved dose functions; +- model-relative KPI summaries; +- experimental therapy-design code. + +Current limitation: core biological parameters and several endpoint definitions are heuristic research abstractions. + +### `twin_engine` + +- persistent `PatientTwinState` and parent/child lineage; +- computational-input identity; +- longitudinal labs, interruptions and adverse events; +- observation model and residuals; +- calibration infrastructure; +- mechanistic counterfactual runs; +- uncertainty, sensitivity, robustness and backtesting prototypes; +- immutable run manifests, artifact hashes, comparability and invalidation; +- Simple Research View, Research Cockpit and Developer Console. + +Current limitation: infrastructure does not establish identifiable or externally validated patient-specific prediction. + +### `chemtools` + +- optional RDKit-dependent and molecular-analysis utilities; +- asynchronous jobs where configured; +- generated artifacts. + +Current limitation: molecular utilities are exploratory and are not evidence of anti-myeloma efficacy. + +## Current scientific run flow + +```mermaid +sequenceDiagram + autonumber + actor U as Research user + participant C as clinic/twin data + participant T as twin_engine + participant S as simulator + participant A as artifact store + + U->>T: Select governed Twin and intervention + T->>C: Validate patient access, input identity and readiness + T->>S: Execute mechanistic baseline/alternative simulation + S-->>T: Trajectory, KPIs and predicted biomarker projection + T->>T: Add toxicity, uncertainty and comparison diagnostics + T->>A: Write hashed trajectory/report artifacts + T->>C: Persist immutable run manifest and artifact records + T-->>U: Render research-only result and limitations +``` + +## Current data and identity flow + +```mermaid +flowchart LR + S["Source assertion / private evidence"] --> D["Versioned structured dataset"] + D --> I["Idempotent import"] + I --> CI["Content-addressed computational input"] + CI --> TS["Lineage-bound Twin state"] + TS --> RM["Immutable research run manifest"] + RM --> AR["Hashed artifacts"] +``` + +Private source documents and direct identifiers stay outside Git. The repository stores contracts and de-identified/synthetic material only. + +## Current authentication boundary + +```text +public: documentation and selected simulator surfaces +protected: clinic, research, simulator-management and API surfaces +configured login target: Django admin login +normal product login/role contract: incomplete +``` + +The target least-privilege role architecture is tracked in M0-S issue `#8`; it is not current behavior. + +## Current deployment and CI boundary + +- deterministic dependency lock exists; +- several GitHub Actions workflows exist; +- image-build and deployment automation exist; +- branch-protected aggregate scientific CI is not yet verified; +- production security, PostgreSQL, storage, quotas, observability and recovery must be evidenced separately under M0-S. diff --git a/docs/architecture/TARGET_VIRTUAL_LAB.md b/docs/architecture/TARGET_VIRTUAL_LAB.md new file mode 100644 index 0000000..4462406 --- /dev/null +++ b/docs/architecture/TARGET_VIRTUAL_LAB.md @@ -0,0 +1,151 @@ +--- +title: Target Virtual Laboratory Architecture +status: TARGET_APPROVED +owner: Andrea Zedda +audience: architects, researchers and collaborators +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +source_of_truth: GitHub roadmap issue 31 and milestone trackers +--- + +# Target virtual laboratory architecture + +> **TARGET architecture.** Components in this document are not current capabilities unless explicitly mapped to [Current system architecture](CURRENT_SYSTEM.md) and the [model registry](../models/REGISTRY.md). + +## Target capability map + +```mermaid +flowchart TB + K["Knowledge Commons"] --> E["Evidence and measurement layer"] + E --> V["Virtual Patient Factory"] + V --> X["Experiment Builder"] + X --> S["Simulation and inference engines"] + S --> C["Comparison and falsification laboratory"] + C --> P["Reproducible research protocol"] + P --> K +``` + +## Target layers + +### 1. Knowledge Commons + +- disease biology, cell lineage, targets, pathways and mechanisms; +- current and emerging therapies; +- resistance, toxicity and measurement semantics; +- evidence levels, contradictions and valid domains; +- learning paths by professional role; +- direct links from claims to model parameters and experiments. + +### 2. Evidence and measurement layer + +- atomic source assertions; +- immutable datasets and data cards; +- biomarker, MRD, imaging, genomic and functional-outcome contracts; +- uncertainty, missingness and measurement-error models; +- external dataset adapters and frozen benchmark tasks. + +### 3. Virtual Patient Factory + +- lineage-bound patient state; +- synthetic archetypes and governed cohort generation; +- state estimation and observation models; +- calibration, identifiability and posterior/ensemble uncertainty; +- explicit healthy lineage, malignant compartments, organ and immune states as evidence permits. + +### 4. Experiment Builder + +- current regimens and alternative schedules; +- drug combinations and sequences; +- innovative molecules and target/modality definitions; +- CAR-T, bispecific, ADC and logic-gated hypotheses; +- comparator, endpoint, horizon and falsification definitions; +- no experiment can execute without versioned inputs and allowed-claim boundaries. + +### 5. Simulation and inference engines + +Candidate engines include: + +- multi-drug PK/PD; +- state-space estimation; +- clonal evolution; +- competing residual-disease/reservoir hypotheses; +- marrow niche and immune recovery; +- bone and organ dynamics; +- mechanistic toxicity; +- causal protocols only when identification assumptions are explicit; +- adaptive therapy and optimal-control research. + +These are target candidates, not validated current models. + +### 6. Comparison and falsification laboratory + +- simple and mechanistic baselines; +- rolling-origin temporal backtesting; +- sensitivity, uncertainty and identifiability diagnostics; +- robust Pareto fronts and decision stability; +- negative-result preservation; +- automatic rejection when data, evidence or compatibility gates fail. + +### 7. Reproducible protocol and artifact layer + +- immutable run manifest and version vector; +- source/data/model/configuration lineage; +- hashed artifacts and comparability decisions; +- model, dataset and evidence cards; +- machine-readable experiment protocol; +- export for wet-lab, retrospective or external-validation follow-up. + +## Target virtual-patient state + +A future multiscale state may be represented abstractly as: + +$$ +X_t = [B_t, R_t, C_{1:k,t}, N_t, I_t, O_t]^T +$$ + +where: + +- $B_t$ is malignant bulk burden; +- $R_t$ is a residual/dormant compartment under a declared hypothesis; +- $C_{1:k,t}$ are subclone states; +- $N_t$ is marrow-niche state; +- $I_t$ is immune/normal-lineage state; +- $O_t$ is organ and functional state. + +This is a target state contract, not a claim that every component is observable or identifiable. + +### Falsification requirement + +A proposed state component is rejected or deferred if it has no measurable proxy, no identifiable effect, no evidence-linked parameterization or no improvement over a simpler comparator. + +## Target experiment contract + +Every experiment must specify: + +```text +research question +virtual patient/cohort identity +source and dataset versions +model versions +intervention and comparator +endpoints and units +assumptions +uncertainty model +falsification criteria +allowed and forbidden conclusions +artifact and report versions +``` + +## Current-to-target dependency + +```text +M0-R canonical baseline +→ M1 reproducible longitudinal loop +→ M2 measurement/evidence semantics +→ M3 external benchmark +→ M4 multiscale competing models +→ M5 therapy design and control +→ M6 E2 reproducible-research release +``` + +A target capability cannot bypass the evidence and validation layers merely because it is technically implementable. diff --git a/docs/governance/DOCUMENTATION_POLICY.md b/docs/governance/DOCUMENTATION_POLICY.md new file mode 100644 index 0000000..85640bc --- /dev/null +++ b/docs/governance/DOCUMENTATION_POLICY.md @@ -0,0 +1,99 @@ +--- +title: Documentation Policy +status: CANONICAL +owner: Andrea Zedda +audience: document authors and reviewers +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + +# Documentation policy + +## Required document statuses + +Every governed document must use one status: + +```text +CURRENT_VERIFIED +CURRENT_PARTIAL +CURRENT_HEURISTIC_PROTOTYPE +CURRENT_HYPOTHETICAL_PROTOTYPE +TARGET_APPROVED +CANDIDATE_COMPARATOR +RESEARCH_HYPOTHESIS +DEFERRED +REJECTED +HISTORICAL_ARCHIVE +``` + +## Required metadata + +Current and target documents must declare: + +```yaml +title: +status: +owner: +audience: +last_verified_at: +last_verified_git_sha: +source_of_truth: +supersedes: +superseded_by: +``` + +Fields that do not apply may be omitted, but `status`, `owner`, `last_verified_at` and `last_verified_git_sha` are mandatory for current documents. + +## Current versus target + +- `CURRENT_*` content must map to an existing code/schema/registry entry point. +- `TARGET_APPROVED` content must state prerequisites and acceptance evidence. +- `RESEARCH_HYPOTHESIS` content must include competing hypotheses and a falsification path. +- historical content must begin with an archive banner and must not appear in current navigation. + +## Formula standard + +Every formula that affects project interpretation must include: + +1. scientific or computational question; +2. equation; +3. definition and unit of every variable; +4. parameter provenance/evidence status; +5. source-code entry point and model version; +6. worked numerical example; +7. valid domain and assumptions; +8. sensitivity/uncertainty note; +9. falsification or rejection criterion; +10. allowed and forbidden conclusions. + +A formula in a DOCX must use native Word equation objects. The canonical repository representation remains Markdown/LaTeX plus code mapping. + +## Diagram standard + +- label every diagram `CURRENT` or `TARGET`; +- version the source; +- prefer several small diagrams over one dense diagram; +- use no more than approximately eight primary nodes per operational view; +- avoid crossing connectors where practical; +- use landscape/wider layout for complex flows; +- keep labels readable at normal page zoom; +- render Mermaid during validation; +- provide a caption, scope and interpretation note; +- do not place floating overlays over text in exported office documents. + +## Claims standard + +Current documents inherit `INTENDED_USE.md` and `CLAIMS_POLICY.md`. Archived claims cannot be reintroduced through examples, screenshots, translations or generated API documentation. + +## Review cadence + +- verify current-state pages whenever a release or model/schema change affects them; +- verify at least every 90 days while the project is active; +- update the verified Git SHA after review; +- create an issue when a document cannot be verified rather than asserting currency. + +## Inventory and navigation + +`docs/DOCUMENTATION_INVENTORY.yaml` classifies the documentation corpus. Only `CANONICAL`, `CURRENT_SUPPORTING` and explicitly selected `TARGET_SUPPORTING` documents may appear in current MkDocs navigation. + +Unmatched files resolve to `UNCLASSIFIED_REQUIRES_REVIEW` and must stay outside current navigation until reviewed. diff --git a/docs/governance/SOURCE_OF_TRUTH.md b/docs/governance/SOURCE_OF_TRUTH.md new file mode 100644 index 0000000..5b8dc7e --- /dev/null +++ b/docs/governance/SOURCE_OF_TRUTH.md @@ -0,0 +1,78 @@ +--- +title: Source of Truth +status: CANONICAL +owner: Andrea Zedda +audience: all contributors +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + +# Source of truth + +## Hierarchy + +When two records disagree, use this precedence order: + +1. **Code, schemas, migrations, registries and machine-readable governance** at the canonical Git commit. +2. **Current GitHub documentation** explicitly verified against that commit. +3. **GitHub issues, pull requests and validation evidence**. +4. **Notion current project dashboard and decision records**. +5. **Architecture DOCX and other external design proposals**. +6. **Historical archives and implementation logs**. + +A lower layer must be reconciled; it must not silently override a higher layer. + +## GitHub boundary + +GitHub owns: + +- code and migrations; +- model/data/input/run schemas; +- model registry and versions; +- current technical documentation; +- issues, milestones, PRs and acceptance evidence; +- numerical and migration evidence; +- release gates and artifacts. + +## Notion boundary + +Notion owns: + +- current project summary; +- strategic rationale; +- decision records; +- cross-project planning; +- human-readable status and risk summaries. + +Notion must link to canonical GitHub issues and a verified Git SHA. It must not duplicate mutable file-level acceptance criteria or retain a superseded SHA as current state. + +## DOCX boundary + +The architecture DOCX is a **target architecture proposal**. Every revision must state: + +```yaml +document_version: +document_status: TARGET_ARCHITECTURE_PROPOSAL +verified_against_git_sha: +current_intended_use: E1_research_prototype +target_intended_use: E2_reproducible_research +``` + +Every major component/model must be marked `CURRENT`, `PARTIAL`, `TARGET`, `HYPOTHESIS` or `DEFERRED`. + +## Conflict protocol + +1. Freeze the disputed claim. +2. Identify the highest-precedence evidence. +3. Record the discrepancy in issue `#14` or the relevant issue. +4. Update lower-precedence records. +5. Re-run applicable documentation, semantic, migration or numerical checks. +6. Record the new verified SHA/date. + +## Current canonical baseline + +```text +repository = andreazedda/bmyCure4MM +branch = master +verified_head = bf097810b337dc6b766cda04497005670cd96513 +``` diff --git a/docs/models/REGISTRY.md b/docs/models/REGISTRY.md new file mode 100644 index 0000000..5b963a3 --- /dev/null +++ b/docs/models/REGISTRY.md @@ -0,0 +1,238 @@ +--- +title: Current Model Registry and Status +status: CURRENT_VERIFIED +owner: Andrea Zedda +audience: model developers, reviewers and research users +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +machine_readable_source: twin_engine/model_registry.json +--- + +# Current model registry and status + +The machine-readable registry is authoritative for IDs, versions, schemas and entry points. This page adds formulas, examples and interpretation boundaries. + +## Status summary + +| Model ID | Version | Entry point | Status | +|---|---|---|---| +| `patient_twin_state_model` | `research-twin-v1` | `twin_engine.simulation_bridge.run_patient_simulation` | CURRENT_HEURISTIC_PROTOTYPE | +| `observation_model` | `observation-model-v1` | `twin_engine.observation_model.predict_biomarkers` | CURRENT_HEURISTIC_PROTOTYPE | +| `lenalidomide_exposure_model` | `exposure-bridge-v1` | `twin_engine.exposure_bridge.build_exposure_dose_function` | CURRENT_EXECUTABLE | +| `hepatic_signal_model` | `toxicity-prototype-v1` | `twin_engine.toxicity_dynamics.compute_toxicity_dynamics` | CURRENT_HEURISTIC_PROTOTYPE | +| `neutropenia_signal_model` | `toxicity-prototype-v1` | `twin_engine.toxicity_dynamics.compute_toxicity_dynamics` | CURRENT_HEURISTIC_PROTOTYPE | +| `counterfactual_model` | `research-twin-v1` | `twin_engine.counterfactual.run_counterfactual` | CURRENT_HEURISTIC_PROTOTYPE | +| `therapy_design_toy_model` | `therapy-design-toy-v1` | `simulator.design.reporting.run_design_report` | CURRENT_HYPOTHETICAL_PROTOTYPE | + +## 1. Patient Twin state / mechanistic runtime + +### Question + +How does the current abstract tumour/healthy system evolve under a configured therapy schedule? + +### Core equations + +$$ +\frac{dT}{dt} = r_T T\left(1-\frac{T}{K_T}\right)-\left(\sum_i E_i(C_i)\right)T +$$ + +$$ +\frac{dH}{dt} = r_H H\left(1-\frac{H}{K_H}\right)-I\,\overline{E(C)}\,H +$$ + +$$ +\frac{dC_i}{dt}=-k_{elim,i}C_i+u_i(t) +$$ + +$$ +E_i(C_i)=E_{max,i}\frac{C_i}{EC50_i+C_i} +$$ + +Variables: + +- $T$: abstract tumour-cell burden, cells; +- $H$: abstract healthy-cell compartment, cells; +- $C_i$: model concentration for drug $i$, model units; +- $r_T,r_H$: growth rates, day$^{-1}$; +- $K_T,K_H$: carrying capacities, cells; +- $I$: dimensionless immune-compromise/toxicity multiplier; +- $u_i(t)$: schedule-resolved model input; +- $k_{elim,i}=\ln(2)/t_{1/2,i}$, day$^{-1}$; +- $E_{max,i}$: maximum model effect; +- $EC50_i$: model concentration at half maximum effect. + +Parameters in the current Twin registry are primarily `HEURISTIC`. The equations are mechanistic research abstractions, not a validated MM natural-history model. + +### Worked example + +With $T=10^9$, $K_T=10^{12}$ and $r_T=0.02$ day$^{-1}$, untreated logistic growth is approximately: + +$$ +0.02\times10^9\left(1-10^{-3}\right)\approx1.998\times10^7\ \text{cells/day} +$$ + +If the total model drug effect is $0.03$ day$^{-1}$, the kill term is $3.0\times10^7$ cells/day and the instantaneous derivative becomes negative. This illustrates model behavior; it does not predict a patient's tumour-cell count. + +### Falsification + +Reject or revise the current model for a task when it fails declared numerical invariants, cannot improve on simple temporal baselines, produces non-identifiable parameters or does not explain held-out observations within the validation protocol. + +## 2. Observation model + +### Question + +How are latent $T$ and $H$ mapped to three currently supported observed biomarkers? + +$$ +\widehat{M}=\alpha_M T+\beta_M +$$ + +$$ +\widehat{F}=\alpha_F\left(\frac{T}{T_{ref}}\right)^{\gamma_F} +$$ + +$$ +\widehat{Hb}=Hb_0\left(\frac{H}{H_{ref}}\right)^{\eta_H} +$$ + +- $\widehat{M}$: predicted M-protein, g/dL; +- $\widehat{F}$: predicted FLC ratio, dimensionless; +- $\widehat{Hb}$: predicted hemoglobin, g/dL; +- $\alpha_M,\beta_M,\alpha_F,\gamma_F,Hb_0,T_{ref},H_{ref},\eta_H$: heuristic or calibration parameters. + +Default values are defined in `twin_engine/observation_model.py` and can be initialized from a baseline assessment. + +### Worked example + +If $\alpha_M=10^{-9}$ g/dL/cell, $\beta_M=0$ and $T=8\times10^8$ cells, then: + +$$ +\widehat{M}=10^{-9}\times8\times10^8=0.8\ \text{g/dL} +$$ + +This result is model-relative. FLC ratio is not a universal direct tumour-burden scalar, and hemoglobin is affected by factors not represented by $H$ alone. + +### Falsification + +The mapping is rejected for patient-specific prediction if residuals show systematic bias, parameters are non-identifiable or a simple baseline performs as well or better on future observations. + +## 3. Exposure bridge + +### Question + +Does the system preserve the temporal identity of a treatment schedule? + +For a horizon of $N$ days with administered dose $d_j$ on day $j$: + +$$ +D_{cum}(n)=\sum_{j=0}^{n} d_j +$$ + +$$ +\bar d=\frac{1}{N}\sum_{j=0}^{N-1}d_j +$$ + +The profile stores the daily vector, cumulative vector, peak dose, interruption days, schedule type and a canonical profile hash. + +### Worked example + +```text +5 mg daily: [5, 5, 5, 5] +10 mg alternate: [10, 0, 10, 0] +``` + +Both average 5 mg/day, but mean absolute temporal distance is: + +$$ +\frac{|5-10|+|5-0|+|5-10|+|5-0|}{4}=5\ \text{mg/day} +$$ + +Therefore `same_average_exposure=true` and `different_temporal_profile=true`. + +### Limitation + +This is administered-dose exposure identity, not patient-specific plasma concentration or PK validation. + +## 4. Hepatic and neutropenia signal models + +### Question + +Can exposure and prior history generate normalized prototype risk signals for stress testing? + +The current implementation uses clipped heuristic combinations. A simplified day-$t$ representation is: + +$$ +S_{AST,t}=clip_{[0,1]}(b_{AST}+\alpha_{AST}d_t^*+\beta_s S+\gamma_{AST}P_{AST}) +$$ + +$$ +S_{ALT,t}=clip_{[0,1]}(b_{ALT}+\alpha_{ALT}d_t^*+\beta_s S+\gamma_{ALT}P_{ALT}) +$$ + +$$ +S_{liver,t}=\max(S_{AST,t},S_{ALT,t}) +$$ + +$$ +S_{neu,t}=clip_{[0,1]}(b_{neu}+\alpha_{neu}D_{cum,t}^*+\beta_{prior}N_{prior}) +$$ + +where normalized daily and cumulative dose are referenced to 10 mg and 100 mg, respectively. Coefficients are defaults or limited single-patient estimates. + +### Worked example + +Using $b_{neu}=0.15$, $\alpha_{neu}=0.45$, normalized cumulative exposure $0.6$, $\beta_{prior}=0.25$ and prior-neutropenia flag $1$: + +$$ +S_{neu}=clip(0.15+0.45\times0.6+0.25)=0.67 +$$ + +The value `0.67` is a normalized prototype signal, not a predicted ANC or probability of clinical neutropenia. + +### Falsification + +Do not promote this model to attribution or prediction unless it is source-verified, compared against competing explanations, calibrated and validated prospectively or temporally under a governed protocol. + +## 5. Counterfactual model + +### Question + +How do a baseline and alternative mechanistic configuration differ under the same governed Twin state? + +The current utility is heuristic: + +$$ +U_1=E_{tumour}+(1-L_{healthy})+D_{durability}-P_{constraint} +$$ + +A second version adds prototype toxicity penalties: + +$$ +U_2=U_1-\lambda_L S_{liver}-\lambda_N S_{neu} +$$ + +Current default weights are $\lambda_L=\lambda_N=0.5$. + +### Worked example + +If tumour reduction is $0.70$, healthy loss $0.10$, durability $0.80$, constraint penalty $0.15$, liver signal $0.30$ and neutropenia signal $0.40$: + +$$ +U_1=0.70+0.90+0.80-0.15=2.25 +$$ + +$$ +U_2=2.25-0.5(0.30)-0.5(0.40)=1.90 +$$ + +This score ranks model configurations only. It does not identify clinical benefit or a causal effect. + +## 6. Therapy-design toy model + +This experimental model exercises target, modality, logic, bulk/reservoir/subclone and Pareto software abstractions. Its antigens, coefficients, escape and relapse proxies are hypothetical/heuristic. It is useful for architecture tests and falsifiable research design, not for selecting or claiming efficacy of ADC, CAR-T, bispecific or logic-gated therapies. + +The target M4/M5 models require evidence-linked parameterization, competing hypotheses, external comparators, sensitivity analysis and validation before their status can be upgraded. + +## Version and invalidation rule + +Any equation, endpoint, unit, parameter default or dependency change that alters numerical output requires a model-version decision and an output-diff report. Prior runs are never silently reinterpreted. diff --git a/docs/product/CAPABILITIES_AND_LIMITATIONS.md b/docs/product/CAPABILITIES_AND_LIMITATIONS.md new file mode 100644 index 0000000..93dbb6f --- /dev/null +++ b/docs/product/CAPABILITIES_AND_LIMITATIONS.md @@ -0,0 +1,89 @@ +--- +title: Capabilities and Limitations +status: CURRENT_VERIFIED +owner: Andrea Zedda +audience: users, reviewers and contributors +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + +# Capabilities and limitations + +## Interpretation rule + +A capability is usable only inside its stated evidence and intended-use boundary. + +```text +computation completed +≠ model verified +≠ model validated +≠ externally validated +≠ clinically useful +``` + +## Current capability matrix + +| Capability | What exists | What it supports | What it does not establish | +|---|---|---|---| +| Structured research data | Versioned schema, content hashes, idempotent import infrastructure | Reproducible data snapshots and semantic identity | Source-level verification of every private-case value | +| Patient Twin state | Persistent state, parameters, uncertainty fields and lineage | Versioned research state and parent/child history | A validated physiological digital twin | +| Observation residuals | Predicted/observed payloads, RMSE and MAE | Calibration diagnostics | Identifiability or future predictive validity | +| Mechanistic simulation | Logistic tumour/healthy dynamics, PK/PD and schedule inputs | Exploratory trajectories and model-relative comparisons | Clinical outcome prediction | +| Exposure bridge | Day-resolved administered-dose vectors, cumulative dose and hashes | Distinguishing schedules with equal average dose | Patient-specific pharmacokinetics or bioavailability | +| Toxicity signals | Heuristic normalized liver and neutropenia signals | Stress testing and research ranking | AST, ALT or ANC prediction; causal attribution | +| Counterfactual research | Baseline/alternative mechanistic runs with immutable identity | Hypothetical configuration comparison | Identified causal treatment effects | +| Therapy-design toy model | Experimental target/modality/logic and Pareto scaffolding | Testing software abstractions and hypotheses | Validated ADC, CAR-T, bispecific or optimal-treatment design | +| Uncertainty/sensitivity | Prototype sampling and diagnostics | Inspecting model fragility | Calibrated population uncertainty | +| Provenance and run identity | Version vector, hashes, comparability and invalidation | Reproduction and audit | Correctness of the biological assumptions | +| Knowledge/documentation | Governance, research contracts and technical guides | Learning and source navigation | A complete curated Multiple Myeloma evidence graph | + +## Allowed current conclusions + +- A run was executed using a stated code/model/data/configuration identity. +- Two runs are or are not directly comparable under declared compatibility rules. +- A scenario produces a particular trajectory under the current model and assumptions. +- A schedule differs temporally from another schedule. +- Data or provenance are missing for a requested research operation. +- A calibration attempt improved or failed a declared metric, with all other validity questions still open. +- A hypothesis is unsupported, non-identifiable or unstable under sensitivity analysis. + +## Forbidden current conclusions + +- one regimen is clinically best or superior for a patient; +- a specific dose or schedule should be used; +- a patient will benefit from a simulated intervention; +- a model-relative trajectory is an identified causal effect; +- MRD negativity or a negative imaging result proves zero disease; +- a B-like or stem-like reservoir is an established universal mechanism; +- heuristic toxicity signals predict clinical laboratory values; +- successful optimization validates the selected strategy. + +## Main scientific limitations + +1. **Observation model:** current mappings compress heterogeneous biomarkers into simplified functions of tumour and healthy-cell states. +2. **Latent-state interpretation:** current Twin states are research abstractions, not directly measured physiological states. +3. **Identifiability:** multiple parameter sets may explain the same sparse longitudinal observations. +4. **External validity:** current models are not externally benchmarked for patient-level prediction. +5. **Disease representation:** the runtime core does not yet represent validated subclonal, spatial, niche, immune, bone and organ dynamics. +6. **Toxicity attribution:** current risk signals do not separate drug effect from disease, infection, organ reserve, co-medications and measurement error. +7. **Causality:** mechanistic counterfactuals remain distinct from identified causal estimands. + +## Main engineering and platform limitations + +- normal product authentication and least-privilege roles are incomplete; +- object-level authorization remains inconsistent in some patient-derived web/aggregate paths; +- CI is present but not yet a single protected required gate; +- documentation has historical debt and is being reconciled under issue `#14`; +- production runtime, PostgreSQL, storage, observability, quotas and backup/restore remain governed by M0-S rather than assumed complete. + +## Falsification triggers + +A current capability claim must be downgraded if: + +- the referenced entry point no longer exists; +- the registered model version does not match runtime output; +- a documented command or route fails under the supported environment; +- numerical identity changes without a version/output-diff decision; +- an artifact hash cannot be verified; +- a current document describes a target model as implemented; +- source-level evidence does not support a patient-derived input. diff --git a/docs/product/CURRENT_STATE.md b/docs/product/CURRENT_STATE.md new file mode 100644 index 0000000..df8b7d0 --- /dev/null +++ b/docs/product/CURRENT_STATE.md @@ -0,0 +1,95 @@ +--- +title: Current Verified State +status: CURRENT_VERIFIED +owner: Andrea Zedda +audience: all contributors +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +source_of_truth: code, registries, schemas, migrations and current validation evidence +--- + +# Current verified state + +## Canonical identity + +```yaml +repository: andreazedda/bmyCure4MM +branch: master +verified_head: bf097810b337dc6b766cda04497005670cd96513 +intended_use_level: E1_research_prototype +clinical_decision_support: false +patient_specific_prediction_validated: false +causal_effect_identified: false +``` + +This page describes the repository state at the SHA above. Later changes require a new verification record. + +## Current components + +| Component | Current responsibility | Status | +|---|---|---| +| `clinic` | Structured patients, assessments, therapies, longitudinal navigation and current authentication entry points | CURRENT_PARTIAL | +| `simulator` | Mechanistic tumour/healthy-cell and PK/PD simulation, schedules, scenarios and experimental therapy-design code | CURRENT_HEURISTIC_PROTOTYPE | +| `twin_engine` | Twin state/lineage, observation residuals, calibration infrastructure, counterfactual research, run identity, provenance and research views | CURRENT_PARTIAL | +| `chemtools` | Optional exploratory cheminformatics and molecular utilities | CURRENT_PARTIAL | +| `docs_viewer` | In-application documentation rendering and feedback | CURRENT_EXECUTABLE | +| `governance` | Machine-readable release claims and intended-use controls | CURRENT_EXECUTABLE | +| `scripts` | Repository hygiene, numerical baseline, dependency audit and research safety tooling | CURRENT_EXECUTABLE | + +## Completed M0-R foundations + +- Structured Research Dataset Contract v0.1 and generic idempotent importer. +- Content-addressed computational-input and Twin-lineage contracts. +- Canonical E1 intended use and non-prescriptive model-output language. +- Deterministic `uv.lock` dependency graph and supported Python 3.11/3.12 tooling. +- Immutable research run manifests and version vectors. +- Governed model registry and versioned scientific JSON contracts. +- Artifact hashing, direct-comparability rules and append-only invalidation semantics. +- Numerical baseline and repository/dependency checks. + +## Current registered models + +The machine-readable source is `twin_engine/model_registry.json`. + +| Model ID | Version | Current status | +|---|---|---| +| `patient_twin_state_model` | `research-twin-v1` | CURRENT_HEURISTIC_PROTOTYPE | +| `observation_model` | `observation-model-v1` | CURRENT_HEURISTIC_PROTOTYPE | +| `lenalidomide_exposure_model` | `exposure-bridge-v1` | CURRENT_EXECUTABLE with research-only interpretation | +| `hepatic_signal_model` | `toxicity-prototype-v1` | CURRENT_HEURISTIC_PROTOTYPE | +| `neutropenia_signal_model` | `toxicity-prototype-v1` | CURRENT_HEURISTIC_PROTOTYPE | +| `counterfactual_model` | `research-twin-v1` | CURRENT_HEURISTIC_PROTOTYPE | +| `therapy_design_toy_model` | `therapy-design-toy-v1` | CURRENT_HYPOTHETICAL_PROTOTYPE | + +Detailed status and formula mappings are in [Current model registry](../models/REGISTRY.md). + +## Current research loop capability + +The repository can persist structured longitudinal entities, construct lineage-bound Twin states, execute mechanistic simulations, create immutable run identity and compare hypothetical configurations. It also has residual/calibration, uncertainty, sensitivity, robustness and temporal-diagnostic infrastructure. + +It has **not** yet demonstrated a source-verified, leakage-free, identifiable and out-of-sample-valid patient-specific Research Loop. M1 remains blocked until the M0-R documentation, CI and minimum QA baseline closes. + +## Current authentication and authorization boundary + +Verified current behavior includes public documentation and public simulator surfaces plus protected clinic, research, simulator-management and API routes. The configured login target is the Django admin login. A reviewed normal product login and least-privilege product-role contract is not yet complete. + +Known authorization gaps are tracked in GitHub issue `#8`. Admin access or temporary smoke-user creation must not be interpreted as successful normal-user product authentication. + +## Current CI state + +GitHub Actions already exist for dependency/reproducibility checks, documentation deployment, secret scanning, image build/publish, UI screenshots and automatic redeployment. They are not yet consolidated into one stable mandatory aggregate gate, and external Actions are not yet uniformly pinned to immutable commit SHAs. That work is tracked in `#13` after documentation paths stabilize. + +## Privacy boundary + +Private patient documents, direct identifiers, source excerpts and private dataset payloads must remain outside Git under ignored `local_private/` storage. Git may contain schemas, generic import logic, synthetic fixtures and explicitly validated de-identified artifacts. + +## Immediate execution path + +```text +#14 documentation/source-of-truth baseline +→ #13 mandatory CI aggregate gate and PR governance +→ #23 minimum M0-R scientific/privacy invariants +→ #26 close M0-R +``` + +Private read-only source verification for `#32` may prepare in parallel. New canonical patient-specific calibration or published what-if results may not be promoted until M0-R closes. diff --git a/docs/product/MISSION_AND_NORTH_STAR.md b/docs/product/MISSION_AND_NORTH_STAR.md new file mode 100644 index 0000000..8c63a22 --- /dev/null +++ b/docs/product/MISSION_AND_NORTH_STAR.md @@ -0,0 +1,133 @@ +--- +title: Mission and North Star +status: CANONICAL_STRATEGY +owner: Andrea Zedda +audience: researchers, clinicians, biologists, pharmacologists, bioengineers, data scientists, contributors +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +source_of_truth: docs/product/MISSION_AND_NORTH_STAR.md +--- + +# Mission and North Star + +## Mission + +bmyCure4MM is intended to become a **virtual and reproducible Multiple Myeloma research laboratory**. + +Its purpose is to make Multiple Myeloma research questions computable, inspectable and falsifiable. The platform should allow interdisciplinary users to: + +1. acquire structured knowledge about disease biology, measurement, pharmacology, current treatments, innovative modalities, uncertainty and evidence; +2. construct lineage-bound virtual patients and synthetic cohorts; +3. define experiments using established regimens, alternative schedules, combinations, sequences, novel molecules and experimental modalities; +4. execute mechanistic simulations under explicit assumptions; +5. compare efficacy signals, toxicity, residual disease, resistance, antigen escape, uncertainty and model failure; +6. reject weak or non-identifiable hypotheses before committing to more expensive experimental work; +7. translate computational results into source-linked, reproducible and testable research protocols. + +The platform is not currently a treatment-selection system. It does not provide validated patient-specific benefit prediction, identified causal effects or clinical decision support. + +## Why this architecture is required + +The architecture follows from the research loop: + +```text +Knowledge +→ source assertions +→ versioned data +→ virtual patient +→ experiment definition +→ simulation +→ comparison +→ falsification +→ research protocol +→ new evidence +``` + +A database alone cannot execute this loop. A simulator without provenance cannot make results reproducible. A Patient Twin without observation, calibration and validation contracts cannot support credible patient-specific research. An optimizer without uncertainty and evidence penalties can rank invalid hypotheses. A knowledge portal without links to executable models cannot close the loop from literature to experiment. + +## Research objective + +The general decision problem is: + +$$ +a^* = \arg\max_{a \in \mathcal{A}} U(a \mid D, M, C) +$$ + +where: + +- $a$ is a research action: regimen, schedule, sequence, combination, molecule, modality or experiment; +- $\mathcal{A}$ is the set of admissible research actions; +- $D$ is the versioned evidence and dataset available for the experiment; +- $M$ is the explicit model set and its assumptions; +- $C$ is the set of biological, privacy, security, feasibility and intended-use constraints; +- $U$ is a declared research utility, not a clinical recommendation. + +A representative future utility may combine: + +$$ +U(a) = +w_B E_{bulk}(a) ++ w_R E_{reservoir}(a) ++ w_I E_{immune}(a) ++ w_O R_{organ}(a) +- w_T T_{tox}(a) +- w_L P_{relapse}(a) +- w_U U_{uncertainty}(a) +- w_F C_{infeasible}(a) +$$ + +Every term must have a versioned definition, measurable proxy, uncertainty statement and falsification condition before it can be used as evidence. + +### Example + +Suppose two hypothetical schedules have the same average dose. Schedule A administers 5 mg every day; schedule B administers 10 mg every other day. Their average daily dose may be equal, but their time-resolved exposure vectors differ. The platform must therefore preserve schedule identity, simulate both profiles and report whether the result depends on temporal exposure rather than collapse them into one scalar. + +## Knowledge Commons + +The Knowledge Commons is a first-class target capability. It should connect: + +```text +claim +↔ source +↔ evidence level +↔ biological entity +↔ model parameter +↔ executable experiment +↔ result +↔ limitation +``` + +It must support progressive entry into the field without lowering epistemic standards. A new contributor should be able to learn the relevant terminology, inspect the evidence behind a claim, see how that claim affects a model, and understand which conclusions remain forbidden. + +## Primary users + +- **Hematology researchers:** formulate and inspect longitudinal and mechanistic hypotheses. +- **Clinicians in research roles:** examine data readiness, assumptions, model limitations and candidate experiments without receiving prescriptive output. +- **Biologists and pharmacologists:** connect targets, pathways, resistance and toxicity mechanisms to experiments. +- **Bioengineers and quantitative scientists:** implement state estimation, PK/PD, uncertainty, control and validation methods. +- **Data scientists:** build governed adapters, benchmarks, predictive comparators and diagnostics. +- **Students and new contributors:** follow a structured learning path into Multiple Myeloma research. + +## Scientific success criteria + +A successful result is not necessarily a positive result. Valid outcomes include: + +```text +INSUFFICIENT_DATA +CALIBRATION_FAILED +MODEL_NOT_IDENTIFIABLE +NO_OUT_OF_SAMPLE_GAIN +BACKTEST_UNACCEPTABLE +HYPOTHESIS_NOT_SUPPORTED +``` + +The platform creates value when it reduces unsupported confidence, exposes missing data, distinguishes competing mechanisms and produces reproducible next experiments. + +## Non-goals for the current roadmap + +- automated treatment choice; +- patient-specific dosing instructions; +- claims of clinical superiority; +- causal treatment-effect claims without identification; +- regulated or SaMD use; +- representation of a target model as implemented merely because it appears in a design document. diff --git a/docs/product/ROADMAP.md b/docs/product/ROADMAP.md new file mode 100644 index 0000000..d5928c5 --- /dev/null +++ b/docs/product/ROADMAP.md @@ -0,0 +1,54 @@ +--- +title: Project Roadmap +status: CANONICAL_STRATEGY +owner: Andrea Zedda +audience: maintainers, contributors and collaborators +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +canonical_execution_tracker: https://github.com/andreazedda/bmyCure4MM/issues/31 +--- + +# Project roadmap + +GitHub issues and milestones are the execution control plane. This page explains the scientific sequence; issue `#31` is the mutable canonical tracker. + +## Current path + +```text +COMPLETED: +#11 → #12 → #15 → #19 + +CURRENT: +#14 documentation/source of truth +→ #13 mandatory CI and PR governance +→ #23 minimum QA invariants +→ #26 close M0-R + +NEXT: +#32 → #33 → #34 → #35 → #36 → #18 +``` + +## Milestones + +| Milestone | Purpose | Exit emphasis | +|---|---|---| +| M0-R — Canonical Scientific Baseline | One technical and scientific source of truth | Documentation, protected CI, minimum contracts and privacy invariants | +| M0-S — Safe Shared Research Platform | Safe multi-user/external research operation | Product auth, object authorization, RBAC, security, quotas, runtime, API and observability | +| M1 — Real-Patient Research Loop v0.1 | First source-verified lineage-bound longitudinal loop | Observation, calibration decision, exposure, toxicity attribution, temporal backtest and immutable report | +| M2 — Measurement & Evidence Layer v0.2 | Define what inputs and outputs mean | Evidence registry, biomarker semantics, genomic risk, MRD, imaging, functional outcomes and cards | +| M3 — External Validation & Benchmark v0.3 | Test beyond the private case | Governed datasets, frozen tasks, simple baselines and external evidence | +| M4 — Multiscale MM Model v0.4 | Represent competing disease mechanisms | Clones, residual-disease hypotheses, niche, normal lineage, immunity, bone and organ states | +| M5 — Therapy Design & Optimization v0.5 | Research therapeutic strategies | Resistance, escape, multi-drug PK/PD, modalities, robust Pareto, causal protocol and control | +| M6 — Research v1 — E2 | Auditable reproducible-research release | Complete evidence bundle, release artifacts and explicit E2 intended use | + +## Controlled parallelism + +Private read-only source verification for M1 issue `#32` may prepare while M0-R closes. It may not promote a new canonical dataset, calibration, what-if result or project claim before protected documentation/CI/QA gates exist. + +## Decision rule + +Priorities maximize scientific validity, falsifiability, evidence, reproducibility and risk reduction while penalizing implementation cost, migration risk, model complexity, validation burden, non-identifiability and maintenance cost. + +## Release boundary + +M6 targets `E2_reproducible_research` with `clinical_decision_support: false`. Clinical pilot, SaMD, regulated use, automated treatment selection and prescriptive dosing require a separate roadmap and evidence base. From cb5be8a253ccb8a1933c01e40d4975b5d2261543 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 00:22:33 +0200 Subject: [PATCH 02/18] docs: reconcile landing pages, security, research contracts and archive --- CHANGELOG.md | 49 ++- README.md | 137 +++---- RELEASE_PREPARATION.md | 371 ------------------ SECURITY.md | 176 +++------ .../releases/RELEASE_PREPARATION_2024.md | 35 ++ docs/guides/architecture.md | 269 +------------ docs/index.md | 68 ++-- docs/operations/AUTHENTICATION_AND_ACCESS.md | 66 ++++ docs/operations/TESTING.md | 75 ++++ .../COMPARABILITY_AND_INVALIDATION.md | 59 +++ docs/research/RESEARCH_LOOP.md | 74 ++++ docs/research/TWIN_LINEAGE.md | 61 +++ mkdocs.yml | 125 +++--- 13 files changed, 628 insertions(+), 937 deletions(-) delete mode 100644 RELEASE_PREPARATION.md create mode 100644 docs/archive/releases/RELEASE_PREPARATION_2024.md create mode 100644 docs/operations/AUTHENTICATION_AND_ACCESS.md create mode 100644 docs/operations/TESTING.md create mode 100644 docs/research/COMPARABILITY_AND_INVALIDATION.md create mode 100644 docs/research/RESEARCH_LOOP.md create mode 100644 docs/research/TWIN_LINEAGE.md diff --git a/CHANGELOG.md b/CHANGELOG.md index f1125a8..6b3f3a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,25 +1,40 @@ # Changelog -All current release claims are governed by -[`docs/governance/CLAIMS_POLICY.md`](docs/governance/CLAIMS_POLICY.md). +All current release claims are governed by [`docs/governance/CLAIMS_POLICY.md`](docs/governance/CLAIMS_POLICY.md). A changelog entry records implementation history; it does not establish scientific validity. -## Unreleased — canonical E1 governance +## Unreleased — M0-R Canonical Scientific Baseline -- Declared `E1_research_prototype` as the intended-use level. -- Declared clinical decision support, validated patient-specific prediction, - and identified causal effects as false. -- Replaced prescriptive simulation interpretation with model-relative - diagnostic flags carrying epistemic and model-version context. +### Documentation and planning + +- Added a canonical mission: virtual patients, current and experimental therapy hypotheses, falsification, reproducible protocols and an interdisciplinary Knowledge Commons. +- Separated current verified architecture from the target virtual-laboratory architecture. +- Added a current model-status catalog with formulas, units, examples, code mappings and falsification boundaries. +- Added source-of-truth and documentation-governance policies plus a corpus inventory. +- Reconciled GitHub roadmap/tracker state and Notion project state at canonical `master` `bf097810...`. + +### Scientific identity and reproducibility + +- Added the Structured Research Dataset Contract v0.1 and generic idempotent importer. +- Added content-addressed computational-input and Patient Twin lineage contracts. +- Added immutable research run manifests and a governed model registry. +- Added typed scientific payload contracts, artifact integrity checks, direct-comparability rules and append-only invalidation. +- Locked the dependency graph with `uv.lock`, Python 3.11/3.12 support and numerical/repository/dependency checks. + +### Intended use and language + +- Declared `E1_research_prototype` as the current intended-use level. +- Declared clinical decision support, validated patient-specific prediction and identified causal effects as false. +- Replaced prescriptive simulation interpretation with model-relative diagnostic flags carrying epistemic and model-version context. - Added machine-readable release claims and semantic regression tests. -- Separated learning, research, and administration navigation. -- Invalidated pre-policy chemistry efficacy, survival, toxicity/risk-benefit, - and patient-prognosis outputs for scientific interpretation; numerical model - equations and parameters are unchanged. +- Separated learning, research and administration navigation. +- Invalidated pre-policy chemistry efficacy, survival, toxicity/risk-benefit and patient-prognosis outputs for scientific interpretation; numerical equations and parameters were not changed by that language migration. + +### Operational identity + +- Added a governed temporary least-privilege M0 smoke-user command. It does not constitute a normal product login or reviewed RBAC contract. -## 1.0.0 — 2024-11 +## Historical 1.0.0 claim — 2024-11 -- Initial public research-prototype release. +A 2024 document described the repository as ready for a public `1.0.0` release and incorrectly equated passing 30 of 32 tests with 94% code coverage. Those statements are historical and are not current release, coverage or security evidence. -Pre-governance descriptions are preserved in Git history and summarized in -[`docs/archive/LEGACY_PRESCRIPTIVE_OUTPUTS.md`](docs/archive/LEGACY_PRESCRIPTIVE_OUTPUTS.md). -They are not current product capabilities or authorized claims. +The archived record is retained at [`docs/archive/releases/RELEASE_PREPARATION_2024.md`](docs/archive/releases/RELEASE_PREPARATION_2024.md). Legacy prescriptive claims are summarized in [`docs/archive/LEGACY_PRESCRIPTIVE_OUTPUTS.md`](docs/archive/LEGACY_PRESCRIPTIVE_OUTPUTS.md). diff --git a/README.md b/README.md index 3add9fa..2e02c19 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,6 @@ # bmyCure4MM -bmyCure4MM is an **E1 research prototype** for reproducible Multiple Myeloma -computational research. +bmyCure4MM is an **E1 research prototype** whose north-star is a virtual and reproducible Multiple Myeloma research laboratory. ```yaml intended_use_level: E1_research_prototype @@ -10,118 +9,102 @@ patient_specific_prediction_validated: false causal_effect_identified: false ``` -The platform supports mechanistic simulation, structured research data, -lineage-bound patient-twin experiments, exploratory scenario comparison, and -scientific QA. It does not choose treatment, issue patient-specific dose or -schedule instructions, predict patient benefit, or identify causal treatment -effects. +The platform currently provides governed structured research data, lineage-bound Patient Twin infrastructure, mechanistic PK/PD and schedule-aware simulation, model-relative counterfactual research, immutable run identity, artifact integrity and scientific QA foundations. -The canonical policy is [Canonical Intended Use](docs/governance/INTENDED_USE.md). +It does **not** choose treatment, issue patient-specific dose or schedule instructions, predict validated patient benefit or identify causal treatment effects. + +## Start here + +- [Mission and north star](docs/product/MISSION_AND_NORTH_STAR.md) +- [Current verified state](docs/product/CURRENT_STATE.md) +- [Capabilities and limitations](docs/product/CAPABILITIES_AND_LIMITATIONS.md) +- [Canonical intended use](docs/governance/INTENDED_USE.md) +- [Source-of-truth policy](docs/governance/SOURCE_OF_TRUTH.md) +- [Current architecture](docs/architecture/CURRENT_SYSTEM.md) +- [Target virtual laboratory](docs/architecture/TARGET_VIRTUAL_LAB.md) +- [Current model registry and formulas](docs/models/REGISTRY.md) +- [Project roadmap](docs/product/ROADMAP.md) + +## Current verified baseline + +```text +repository = andreazedda/bmyCure4MM +branch = master +verified_head = bf097810b337dc6b766cda04497005670cd96513 +``` + +Current M0-R path: + +```text +#14 documentation/source of truth +→ #13 mandatory CI aggregate gate and PR governance +→ #23 minimum scientific/privacy invariants +→ #26 close M0-R +``` ## Current capabilities -- versioned structured research-dataset import with content hashes; -- mechanistic PK/PD and tumor/healthy-cell simulation; -- lineage-bound research twin initialization and calibration infrastructure; -- model-relative diagnostic flags with explicit epistemic labels; -- time-respecting research diagnostics, uncertainty, sensitivity, robustness, - and backtesting prototypes; -- molecular research utilities, with RDKit-dependent features optional; -- separate educational (`/learn/`), research (`/research/`), and administrative - (`/admin/`) surfaces. +- versioned Structured Research Dataset Contract and idempotent import infrastructure; +- content-addressed computational-input and Twin lineage; +- persistent Twin states, residuals and calibration infrastructure; +- logistic tumour/healthy-cell and PK/PD research simulation; +- day-resolved administered-dose profiles and schedule identity; +- heuristic hepatic and neutropenia risk signals; +- mechanistic baseline/alternative counterfactual runs; +- immutable run manifests, model registry, artifact hashes, comparability and invalidation; +- uncertainty, sensitivity, robustness and backtesting prototypes; +- optional molecular research utilities; +- separate learning, research and administration surfaces. -All scientific results require interpretation in model and data context. -Negative and inconclusive results are valid research outcomes. +See [Capabilities and limitations](docs/product/CAPABILITIES_AND_LIMITATIONS.md) before interpreting any result. ## Local setup -Python 3.11 and 3.12 are supported. `uv.lock` is the sole dependency lock; -`uv` 0.12.3 is required and refuses a stale lock. +Python 3.11 and 3.12 are supported. `uv.lock` is the sole dependency lock and `uv` 0.12.3 is required. ```bash git clone https://github.com/andreazedda/bmyCure4MM.git cd bmyCure4MM -python3.11 -m pip install uv==0.12.3 # one-time bootstrap +python3.11 -m pip install uv==0.12.3 uv sync --frozen --extra chemistry uv run python manage.py migrate uv run python manage.py check uv run python manage.py runserver ``` -Omit `--extra chemistry` for core work that does not use RDKit or the molecular -pipelines. Dependency groups, lock updates, audits, and deterministic numerical -checks are documented in [Dependency Operations](docs/operations/DEPENDENCIES.md). +Omit `--extra chemistry` for core work that does not require RDKit. Dependency operations are documented in [Dependency Operations](docs/operations/DEPENDENCIES.md). -Do not use production secrets or private patient payloads in development -commands, fixtures, tests, or Git. - -## Tests and safety gate +## Verification and safety ```bash +uv lock --check +uv sync --frozen --extra chemistry uv run python manage.py check uv run python manage.py makemigrations --check --dry-run uv run python manage.py test uv run ruff check . uv run ruff format --check . uv run mypy +uv run python -m scripts.check_repository_hygiene uv run python -m scripts.check_numerical_baseline uv run python -m scripts.audit_dependencies -scripts/pre_push_research_safety_check.sh +bash scripts/pre_push_research_safety_check.sh ``` -The safety script checks ignored private paths, staged sensitive files, -PHI-like markers, Django configuration, twin-engine tests, and selected -authorization/simulation tests. - -## Scientific identity - -The current research model identifier is `research-twin-v1`. Twin states may -bind model/config/Git identity, dataset identity, and a content-addressed -computational-input hash. Historical states without lineage remain explicitly -unbound; they are not silently reinterpreted. - -Every persisted scientific execution now creates an immutable run manifest -containing the mandatory model/data/code/configuration version vector and -artifact hashes. The governed model registry, direct-comparison rules, and -append-only invalidation semantics are documented in -[Research run manifests](docs/research/RUN_MANIFESTS.md). A successful process -exit still does not imply calibration, validation, or scientific success. - -## Epistemic labels - -Current canonical labels are: - -`OBSERVED`, `DERIVED`, `USER_PROVIDED`, `SIMULATED`, `HEURISTIC`, -`LITERATURE_INFORMED`, `HYPOTHETICAL`, and `VALIDATED_EXTERNAL`. - -Definitions and restrictions are in -[Epistemic Labels](docs/governance/EPISTEMIC_LABELS.md) and -[Model Output Language](docs/governance/MODEL_OUTPUT_LANGUAGE.md). - -## Repository areas - -```text -clinic/ structured clinical-record workspace and data navigation -simulator/ mechanistic simulation and synthetic learning scenarios -twin_engine/ research twin, lineage, diagnostics, and artifacts -chemtools/ molecular research utilities -docs/ current and archived documentation -governance/ machine-readable release claims -local_private/ ignored private research material (never commit) -``` +Current test boundaries and CI limitations are documented in [Testing](docs/operations/TESTING.md). ## Privacy -Names, medical-record numbers, dates of birth, clinical PDFs, source excerpts, -direct identifiers, private dataset payloads, and private artifacts must never -enter Git. Keep private research material under ignored `local_private/` paths -and export only explicitly validated, de-identified research artifacts. +Names, medical-record numbers, dates of birth, clinical PDFs, source excerpts, direct identifiers, private dataset payloads and private artifacts must never enter Git. Keep private research material under ignored `local_private/` paths and export only explicitly validated, de-identified research artifacts. ## Documentation status -Current governance documents are authoritative. Historical feature narratives -live under `docs/archive/` and must not be interpreted as current behavior. A -full documentation source-of-truth rebuild is tracked in GitHub issue #14. +The documentation corpus is governed by: + +- [Documentation policy](docs/governance/DOCUMENTATION_POLICY.md) +- [Documentation inventory](docs/DOCUMENTATION_INVENTORY.yaml) + +Historical feature and release narratives are retained only as archives and are not current product claims. ## License and security diff --git a/RELEASE_PREPARATION.md b/RELEASE_PREPARATION.md deleted file mode 100644 index 514396e..0000000 --- a/RELEASE_PREPARATION.md +++ /dev/null @@ -1,371 +0,0 @@ -# Repository Preparation for Public Release - Summary - -## Overview -This document summarizes all changes made to prepare the bmyCure4MM repository for public release. - -**Date**: November 14, 2024 -**Status**: ✅ Complete - Ready for Public Release - ---- - -## ✅ Completed Tasks - -### 1. Documentation Organization -**Status**: Complete - -Reorganized scattered documentation files into a logical structure: - -``` -docs/ -├── README.md (new navigation hub) -├── guides/ -│ └── architecture.md (new system overview) -├── development/ -│ ├── DEVELOPMENT.md -│ ├── TROUBLESHOOTING.md -│ ├── IMPLEMENTATION_LOG.md -│ └── IMPLEMENTATION_SUMMARY.md -├── features/ -│ ├── FORM_ENHANCEMENTS_SUMMARY.md -│ ├── MATHEMATICAL_MODELS_DOCUMENTATION.md -│ ├── CHEMTOOLS_TESTING_SUMMARY.md -│ ├── DOCS_VIEWER_COMPLETE.md -│ └── TESTING_REPORT.md -└── api/ - └── (existing API docs) -``` - -**Impact**: Easier navigation for new contributors and users. - -### 2. Comprehensive Onboarding Documentation -**Status**: Complete - -Created new documentation: -- **CONTRIBUTING.md** (300+ lines) - Complete contributor guide with: - - Development setup instructions - - Code style guidelines - - Testing requirements - - PR submission process - - Bug reporting templates - - Feature request templates - - Code of conduct - -- **Architecture Guide** (300+ lines) - System architecture including: - - High-level architecture diagram - - Component descriptions (Simulator, ChemTools, Clinic, DocsViewer) - - Data flow diagrams - - Technology stack details - - Database schema - - Configuration guide - - Deployment checklist - -- **docs/README.md** - Central navigation hub for all documentation - -**Impact**: New developers can start contributing within hours instead of days. - -### 3. Sensitive Path Removal -**Status**: Complete - -Removed all absolute filesystem paths: - -**Files Modified**: -- `pipelines/configs/general_settings.json` - Changed to relative paths (`./ instead of /Volumes/nvme/Github/bmyCure4MM/`) -- `modules/binding_visualizer/sources/5LF3_structure_report.tex` - Removed absolute path from LaTeX includegraphics - -**Files Deleted**: -- LaTeX auxiliary files containing absolute paths: - - `5LF3_structure_report.fdb_latexmk` - - `5LF3_structure_report.fls` - - `5LF3_structure_report.log` - - `5LF3_structure_report.aux` - -**Verification**: -```bash -# No absolute paths remaining (tested passwords are for tests only) -grep -r "/Volumes/nvme/Github" . --exclude-dir=venv --exclude-dir=.git -# Returns: No matches (except this file) -``` - -**Impact**: No exposure of developer's local filesystem structure. - -### 4. Environment Variable Template -**Status**: Complete - -Created `.env.example` (150+ lines) with: -- All environment variables documented -- Default values provided -- Security notes and best practices -- Comments explaining each setting -- Production security recommendations - -**Key Variables Documented**: -- `DJANGO_SECRET_KEY` (REQUIRED - with generation command and security validation) -- `DJANGO_DEBUG` -- `ALLOWED_HOSTS` -- `CSRF_TRUSTED_ORIGINS` -- `CELERY_BROKER_URL` -- `CELERY_RESULT_BACKEND` -- Email configuration -- Security settings (HTTPS, HSTS, cookies) -- Static/media file paths - -**Security Enhancement (Dec 2025)**: -- Application now **requires** `DJANGO_SECRET_KEY` to be set (no default) -- Validates that key is not insecure or using default values -- Fails fast with clear error message if SECRET_KEY is missing or weak -- See Issue #7 for implementation details - -**Impact**: Users can quickly configure the application without hunting for required settings. Security is enforced at startup. - -### 5. Enhanced .gitignore -**Status**: Complete - -Updated `.gitignore` to prevent committing sensitive/generated files: - -**Added Protections**: -- Environment files (`.env`, `.env.*` except `.env.example`) -- Secrets (`.pem`, `.key`, `.cert`, `secrets.json`) -- Database files (`*.sqlite3`, `*.db`) -- Logs (`*.log`, `logs/`) -- Media uploads (`media/`, `uploads/`) -- LaTeX generated files (`*.aux`, `*.fls`, `*.fdb_latexmk`) -- Cache directories (`.pytest_cache/`, `.ruff_cache/`) -- Generated reports (`*.pdf` except in docs/) -- Data outputs (`pipelines/data/outputs/`, `pipelines/data/logs/`) - -**Impact**: Prevents accidental commit of sensitive information. - -### 6. Production-Ready README -**Status**: Complete - -Completely rewrote `README.md` (400+ lines) with: - -**New Sections**: -- Clear project description and value proposition -- Feature highlights with icons -- Multiple installation options -- Quick start guide (3 different methods) -- Comprehensive documentation links -- Project structure diagram -- Use cases for different audiences (researchers, clinicians, educators, students) -- Testing instructions with coverage stats -- Contribution guidelines -- Technology stack details -- Security & privacy notes -- License information -- Author contact (prepared for public email) -- Roadmap and project status - -**Key Improvements**: -- Professional badges (License, Python version, Django version) -- Clear value proposition -- Multiple quick-start options -- Comprehensive feature listing -- Use case examples -- Security best practices section - -**Impact**: Professional first impression, clear onboarding path for all user types. - -### 7. Additional Security Documentation -**Status**: Complete - -Created `SECURITY.md` with: -- Vulnerability reporting process -- Supported versions table -- Security best practices for deployment -- Known security considerations -- Disclosure timeline -- Attribution policy - -Created `LICENSE` (MIT License): -- Standard MIT license text -- Copyright attribution to Andrea Zedda - -**Impact**: Clear security reporting process and legal clarity. - ---- - -## 🔐 Security Audit Results - -### ✅ No Sensitive Information Exposed - -**Checked For**: -- ❌ Absolute filesystem paths (all removed) -- ❌ API keys or tokens (none found - only test passwords in test files) -- ❌ Database credentials (configured via environment variables) -- ❌ Email addresses (only placeholder andreazedda@example.com) -- ✅ CSRF tokens (Django framework - expected) -- ✅ Test passwords (test files only - acceptable) - -**Remaining References** (All Safe): -- `csrftoken` in templates (Django standard) -- `password` in test files (test data only) -- `email="test@example.com"` in tests (test data only) -- `User.objects.create_user(..., password="testpass123")` in tests (acceptable) - ---- - -## 📊 Repository Statistics - -**Before Cleanup**: -- 15 markdown files scattered in root -- No contributor guidelines -- Absolute paths in config files -- No .env.example -- Basic README (100 lines) - -**After Cleanup**: -- Organized documentation structure -- Comprehensive contributor guide (300+ lines) -- Security policy (140+ lines) -- Architecture documentation (300+ lines) -- Environment template (150+ lines) -- Professional README (400+ lines) -- Enhanced .gitignore -- All sensitive paths removed - -**Test Coverage**: 94% (30/32 tests passing) - ---- - -## 🚀 Ready for Public Release Checklist - -### Repository Configuration -- ✅ All documentation organized -- ✅ CONTRIBUTING.md created -- ✅ SECURITY.md created -- ✅ LICENSE added (MIT) -- ✅ README.md rewritten -- ✅ .env.example created -- ✅ .gitignore enhanced - -### Security -- ✅ No absolute paths -- ✅ No API keys or secrets -- ✅ No real credentials -- ✅ Environment variable configuration -- ✅ Security best practices documented - -### Documentation -- ✅ Installation guide -- ✅ Architecture overview -- ✅ Development guide -- ✅ API documentation -- ✅ Troubleshooting guide -- ✅ Feature documentation - -### Code Quality -- ✅ 94% test coverage -- ✅ All tests passing (30/32) -- ✅ No hardcoded secrets -- ✅ Django security best practices - ---- - -## 📝 Pre-Release Steps (Manual) - -Before making the repository public, complete these steps: - -### 1. Update Email References -Replace placeholder email in: -- `README.md` (line 327, 340) -- `SECURITY.md` (line 12) - -Change `andreazedda@example.com` to real email address. - -### 2. Update GitHub Links -In `README.md`, update: -- Repository URL (currently `https://github.com/andreazedda/bmyCure4MM`) -- Issues link -- Discussions link - -### 3. Create GitHub Repository Settings - -**Set up**: -- Repository description -- Topics/tags: `django`, `multiple-myeloma`, `drug-discovery`, `pkpd-modeling`, `python`, `bioinformatics` -- About section with website (if any) -- Enable Issues -- Enable Discussions -- Set up branch protection for `main`/`master` - -### 4. Optional Enhancements - -**Consider adding**: -- GitHub Actions for CI/CD (`.github/workflows/`) -- Code coverage badge from Codecov -- Code quality badge from CodeClimate -- Documentation hosting (ReadTheDocs) -- Demo/preview environment link - -### 5. Verify Database - -Before first commit: -```bash -# Make sure no production data in db.sqlite3 -rm db.sqlite3 -python manage.py migrate -# Create fresh test database -``` - -### 6. Final Security Scan - -Run security scan before going public: -```bash -# Install security scanner -pip install pip-audit - -# Scan for vulnerabilities -pip-audit - -# Check for secrets (optional) -# Install: pip install detect-secrets -detect-secrets scan -``` - ---- - -## 📞 Support Resources - -### For Users -- README.md - Getting started -- docs/ - Comprehensive guides -- GitHub Issues - Bug reports and feature requests -- GitHub Discussions - Community support - -### For Contributors -- CONTRIBUTING.md - How to contribute -- docs/development/DEVELOPMENT.md - Development setup -- docs/guides/architecture.md - System architecture - -### For Security Researchers -- SECURITY.md - Vulnerability reporting - ---- - -## 🎯 Success Metrics - -This preparation enables: -- ✅ New users can install in <15 minutes -- ✅ New developers can contribute in <1 hour -- ✅ Security researchers know how to report issues -- ✅ No sensitive information exposed -- ✅ Professional, welcoming first impression -- ✅ Clear value proposition for different audiences - ---- - -## 🙏 Next Steps After Public Release - -1. **Monitor initial feedback** - Watch for issues in first week -2. **Engage with community** - Respond to issues and discussions -3. **Create first release** - Tag v1.0.0 with release notes -4. **Set up CI/CD** - Automate testing and deployment -5. **Promote project** - Share in relevant communities -6. **Add contributors** - Welcome first external contributions - ---- - -**✨ The repository is now ready for public release! ✨** - -All sensitive information has been removed, comprehensive documentation has been added, and the repository follows open-source best practices. diff --git a/SECURITY.md b/SECURITY.md index 749b647..44938e3 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,141 +1,93 @@ -# Security Policy +# Security policy -## Reporting a Vulnerability +## Current support state -We take security seriously. If you discover a security vulnerability in bmyCure4MM, please report it responsibly. +bmyCure4MM is an `E1_research_prototype`. -### How to Report +```yaml +production_security_certified: false +clinical_use_supported: false +product_auth_model_complete: false +object_authorization_complete: false +``` -**DO NOT** create a public GitHub issue for security vulnerabilities. +No current version is represented as production-certified or suitable for clinical deployment. Security fixes are handled on the active `master` research line, subject to maintainer capacity and explicit verification. -Instead, please send an email to: -- **andreazedda@example.com** +## Reporting a vulnerability -Include in your report: -- Description of the vulnerability -- Steps to reproduce the issue -- Potential impact -- Suggested fix (if any) +Do **not** disclose vulnerabilities or patient-derived information in a public issue. -### What to Expect +Preferred reporting path: -- **Acknowledgment**: We'll acknowledge receipt within 48 hours -- **Assessment**: We'll assess the vulnerability and determine severity -- **Fix**: We'll work on a fix and keep you updated on progress -- **Disclosure**: Once fixed, we'll coordinate public disclosure +1. use the repository's private GitHub Security Advisory reporting flow when available; +2. otherwise email the repository owner at **andrea.zedda@outlook.it** with the subject `bmyCure4MM security report`. -### Supported Versions +Do not attach real patient records, clinical PDFs, direct identifiers, credentials or production secrets. Use a minimal synthetic reproducer or offer the material through an agreed private channel. -| Version | Supported | -| ------- | ------------------ | -| 1.0.x | :white_check_mark: | -| < 1.0 | :x: | +Include: -## Security Best Practices +- affected commit/version; +- vulnerability class; +- minimal reproduction steps; +- potential impact; +- whether patient-derived data, authentication, authorization or artifact access may be involved; +- suggested mitigation, if known. -### For Deployment +Acknowledgment and remediation timing are targets, not a contractual SLA. Disclosure will be coordinated according to severity, exploitability and available mitigation. -1. **Environment Variables** - - Never commit `.env` files - - Use strong, random `DJANGO_SECRET_KEY` - - Set `DJANGO_DEBUG=0` in production +## Verified current controls -2. **Database** - - Use PostgreSQL in production - - Enable SSL connections - - Regular backups +- environment-based secret-key configuration and weak/default-key rejection; +- Django password hashing and CSRF protections; +- selected authenticated views and global DRF `IsAuthenticated` default; +- secret-scanning workflow; +- repository hygiene and dependency-audit tooling; +- research safety checks for staged sensitive files and PHI-like markers; +- ignored `local_private/` boundary for private research material; +- temporary M0 smoke identities are constrained to non-staff, non-superuser accounts with no groups or direct permissions. -3. **HTTPS** - - Always use HTTPS in production - - Enable HSTS headers - - Set secure cookie flags +These controls do not prove complete application security. -4. **Dependencies** - - Regularly update dependencies - - Monitor security advisories - - Use `pip-audit` or similar tools +## Known open security gaps -5. **Access Control** - - Implement proper authentication - - Use role-based permissions - - Enable two-factor authentication if possible +### Product authentication -### For Development +Documentation and selected simulator surfaces are public, while clinic/research/simulator-management/API surfaces are protected. `LOGIN_URL` currently targets the Django admin login and there is no reviewed normal-user product-login and role contract. -1. **Local Setup** - - Keep local `.env` files secure - - Don't share credentials - - Use test databases for development +A temporary smoke identity cannot authenticate through the admin login by design and must not be promoted to staff merely to make a smoke test pass. -2. **Code Review** - - Review all security-related changes - - Check for SQL injection vulnerabilities - - Validate all user inputs +### Object authorization and RBAC -3. **Testing** - - Write security tests - - Test authentication flows - - Verify permission checks +GitHub issue `#8` tracks inconsistent patient-derived queryset scoping, dashboard aggregation, duplicated policy logic, least-privilege roles and privacy-safe access auditing. -## Known Security Considerations +### Production security -### Input Validation -- All forms use Django's built-in validation -- Additional validation in model `clean()` methods -- XSS prevention through template auto-escaping +GitHub issue `#9` tracks fail-closed production settings, proxy/TLS behavior, HSTS, CSP, cookies, deployment checks and resource-specific framing policy. -### SQL Injection -- Protected by Django ORM -- No raw SQL queries without parameterization -- Queryset filtering uses safe methods +### Rate and resource controls -### CSRF Protection -- Enabled by default in Django -- All forms include CSRF tokens -- API endpoints use token authentication +GitHub issue `#10` tracks request throttling, job quotas, concurrency, timeout, deduplication and cost controls. -### Authentication -- Built on Django's authentication system -- Passwords hashed with PBKDF2 -- Session security configured +### CI and branch protection -## Security Features +GitHub issue `#13` tracks consolidation of existing workflows into one mandatory protected scientific gate, exact-PR safety behavior, immutable Action pinning and PR governance. -### Current Implementation -- ✅ CSRF protection -- ✅ XSS prevention -- ✅ SQL injection protection -- ✅ Secure password hashing -- ✅ Session security -- ✅ Environment-based secrets - -### Recommended Additions -- [ ] Rate limiting -- [ ] Two-factor authentication -- [ ] Security headers (django-csp) -- [ ] IP whitelisting for admin -- [ ] Audit logging -- [ ] Intrusion detection +## Data-safety rules -## Vulnerability Disclosure Timeline - -1. **Day 0**: Vulnerability reported -2. **Day 1-2**: Acknowledgment sent -3. **Day 3-7**: Assessment and severity determination -4. **Day 7-30**: Fix development and testing -5. **Day 30**: Coordinated public disclosure - -## Attribution - -Security researchers who responsibly disclose vulnerabilities will be credited in: -- Release notes -- Security advisories -- Project documentation (with permission) - -## Questions? - -For security questions that aren't vulnerabilities, please: -- Open a GitHub Discussion -- Contact maintainers via email - -Thank you for helping keep bmyCure4MM secure! +Never commit: + +- names, medical-record numbers or dates of birth; +- clinical PDFs or source excerpts; +- private longitudinal dataset payloads; +- database files, media uploads or generated private artifacts; +- `.env`, keys, credentials or tokens. + +Use synthetic/demo fixtures in tests and CI. Security evidence must identify findings without unnecessarily printing sensitive matched content. + +## Deployment boundary + +The repository may contain deployment automation, but a successful image build or running deployment does not establish production security. A deployment must separately prove configuration, database, storage, backup/restore, observability, access control and incident-response requirements. + +## Security-related issues + +Use public issues only for non-sensitive hardening tasks after removing exploit details and private data. Use private reporting for vulnerabilities. diff --git a/docs/archive/releases/RELEASE_PREPARATION_2024.md b/docs/archive/releases/RELEASE_PREPARATION_2024.md new file mode 100644 index 0000000..9bd042d --- /dev/null +++ b/docs/archive/releases/RELEASE_PREPARATION_2024.md @@ -0,0 +1,35 @@ +--- +title: Repository Preparation for Public Release — 2024 +status: HISTORICAL_ARCHIVE +owner: Andrea Zedda +audience: historical reviewers +archived_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +original_path: RELEASE_PREPARATION.md +--- + +# Historical archive: repository preparation for public release — 2024 + +> **HISTORICAL_ARCHIVE — NOT A CURRENT RELEASE OR SECURITY STATEMENT.** + +The original document, created on 14 November 2024 and later edited in 2025, declared the repository ready for a public `1.0.0` release. It also described 30 of 32 passing tests as “94% test coverage,” which is not a valid coverage measurement. + +The original bytes remain recoverable from Git history at `RELEASE_PREPARATION.md` before the issue-14 documentation migration. They are not duplicated here to prevent obsolete claims from being indexed as current guidance. + +## Historical claims superseded by current governance + +- “Ready for Public Release” is superseded by the current `E1_research_prototype` intended use. +- “Production-ready README” is superseded by the current capability and limitation contract. +- `1.0.x` support claims are superseded by the current security policy. +- passing-test counts are not code or branch coverage. +- authentication, authorization, CI, branch protection and production runtime require current evidence. +- the placeholder security email was not a valid reporting path. + +## Current references + +- [Mission and north star](../../product/MISSION_AND_NORTH_STAR.md) +- [Current verified state](../../product/CURRENT_STATE.md) +- [Capabilities and limitations](../../product/CAPABILITIES_AND_LIMITATIONS.md) +- [Security policy](../../../SECURITY.md) +- [Changelog](../../../CHANGELOG.md) +- [Documentation policy](../../governance/DOCUMENTATION_POLICY.md) diff --git a/docs/guides/architecture.md b/docs/guides/architecture.md index 48207a6..d6031ce 100644 --- a/docs/guides/architecture.md +++ b/docs/guides/architecture.md @@ -1,258 +1,23 @@ -# Architecture - -> **Governance boundary:** this is an `E1_research_prototype`. Components and -> data flows described below do not constitute clinical decision support, -> validated patient-specific prediction, or identified causal inference. - -=== "IT" - ## Cos’è bmyCure4MM - - bmyCure4MM è un’applicazione web **Django** orientata a: - - - **Clinic**: gestione pazienti/assessment/terapie (CRUD + timeline) - - **Simulator**: scenari clinici e simulazioni (PK/PD, patient twin, reportistica) - - **ChemTools**: tool di chem-informatics (job asincroni + output in media) - - **Docs Viewer**: viewer interno per documentazione Markdown (analytics + feedback) - - La root app Django è `mmportal/`. - - ## Se cerchi… - - | Voglio… | Vai a… | - | --- | --- | - | schema DB e tabelle | `Guides → Database` | - | mapping Model↔Table | `Reference → Database Objects` | - | parametri simulazione | `Reference → Simulator Parameters` | - | teoria modelli (ODE/PK/PD) | `Guides → Mathematical Models` | - - ## Terminologia - - - **App**: modulo Django (es. `clinic`, `simulator`). - - **Artifact**: file generato (CSV/HTML/JSON) salvato in `media/`. - - **HTMX**: aggiornamenti parziali della UI senza SPA. - - **Attempt**: tentativo simulazione persistito (`SimulationAttempt`). - - ## Componenti (vista di insieme) - - ```mermaid - flowchart LR - subgraph Client - B[Browser] - end - - subgraph DjangoApp["Django (mmportal)"] - URLS[URL routing] - VIEWS[Views / DRF ViewSets] - TPL[Templates] - ORM[Django ORM] - end - - subgraph Apps - CLINIC[clinic] - SIM[simulator] - CHEM[chemtools] - DV[docs_viewer] - end - - subgraph Infra - DB[(SQLite / Postgres)] - REDIS[(Redis)] - CELERY[Celery worker] - FS[(media/ + static/)] - end - - B -->|HTTP| URLS --> VIEWS - VIEWS --> TPL - VIEWS -->|ORM| ORM --> DB - - VIEWS --> CLINIC - VIEWS --> SIM - VIEWS --> CHEM - VIEWS --> DV - - CHEM -->|enqueue| REDIS --> CELERY - CELERY -->|read/write| DB - CELERY -->|artifacts| FS - SIM -->|plots/artifacts| FS - TPL -->|static| FS - ``` - - !!! note "Frontend" - La UI è principalmente server-rendered (Django templates) con interazioni incremental-update tramite **HTMX**. - - ## Apps (responsabilità) - - | App | Scopo | Model principali | Folder template | - | --- | --- | --- | --- | - | `clinic` | pazienti/assessment/terapie | `Patient`, `Assessment`, `PatientTherapy` | `clinic/templates/clinic/` | - | `simulator` | scenari + simulazioni + twin | `Scenario`, `SimulationAttempt` | `simulator/templates/simulator/` | - | `chemtools` | job chem-informatics | `ChemJob` | `chemtools/templates/chemtools/` | - | `docs_viewer` | viewer doc interno | `DocumentView`, `DocumentFeedback` | `docs_viewer/templates/docs_viewer/` | - - ## Flussi principali - - ### Simulazione (Scenario → Attempt → Results) - - ```mermaid - sequenceDiagram - autonumber - actor U as Utente - participant W as Django view - participant DB as DB - participant M as Modello matematico - participant FS as media/ - - U->>W: Submit form (scenario + parametri) - W->>DB: Crea/aggiorna SimulationAttempt - W->>M: run_model(parameters) - M-->>W: trajectory + summary + artifacts - W->>DB: Salva results/results_summary/artifacts (JSON) - W->>FS: (opzionale) salva HTML/plot in media/ - W-->>U: Render pagina risultati (HTMX o full page) - ``` - - ### ChemTools (job asincrono) +--- +title: Architecture Compatibility Page +status: CURRENT_SUPPORTING +owner: Andrea Zedda +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +superseded_by: docs/architecture/CURRENT_SYSTEM.md +--- - ```mermaid - sequenceDiagram - autonumber - actor U as Utente - participant W as Django view - participant DB as DB - participant R as Redis (broker) - participant C as Celery worker - participant FS as media/ - - U->>W: Avvia tool (es. similarity) - W->>DB: Crea ChemJob (Queued) - W->>R: enqueue task(job_id) - C->>R: fetch task - C->>DB: update_progress(...) - C->>FS: scrive output (html/csv) - C->>DB: aggiorna ChemJob (Completed/Failed) - W-->>U: UI poll/refresh stato (HTMX) - ``` - - ## Dati e persistenza - - - Default database: `db.sqlite3` (sviluppo) - - In produzione: consigliato Postgres - - ## Punti operativi - - - Static: `mmportal/static/` → `collectstatic` in produzione - - Media: `media/` (artifact e upload) - - Logs: `logs/` (Django + activity + Celery) - -=== "EN" - ## What is bmyCure4MM - - bmyCure4MM is a **Django** web application focused on: - - - **Clinic**: patient records, assessments, therapies - - **Simulator**: scenarios + PK/PD simulations + patient twin - - **ChemTools**: cheminformatics tools as background jobs - - **Docs Viewer**: in-app Markdown documentation viewer - - Root Django project: `mmportal/`. - - ## Quick find - - | I want… | Go to… | - | --- | --- | - | DB schema and tables | `Guides → Database` | - | Model↔Table mapping | `Reference → Database Objects` | - | simulation parameters | `Reference → Simulator Parameters` | - | ODE/PK/PD theory | `Guides → Mathematical Models` | - - ## Terminology - - - **App**: Django module (e.g., `clinic`, `simulator`). - - **Artifact**: generated file (CSV/HTML/JSON) stored under `media/`. - - **HTMX**: partial UI updates without a full SPA. - - **Attempt**: persisted simulation attempt (`SimulationAttempt`). - - ## High-level components - - ```mermaid - flowchart LR - subgraph Client - B[Browser] - end - - subgraph DjangoApp["Django (mmportal)"] - URLS[URL routing] - VIEWS[Views / DRF ViewSets] - TPL[Templates] - ORM[Django ORM] - end - - subgraph Apps - CLINIC[clinic] - SIM[simulator] - CHEM[chemtools] - DV[docs_viewer] - end - - subgraph Infra - DB[(SQLite / Postgres)] - REDIS[(Redis)] - CELERY[Celery worker] - FS[(media/ + static/)] - end - - B -->|HTTP| URLS --> VIEWS - VIEWS --> TPL - VIEWS -->|ORM| ORM --> DB - - VIEWS --> CLINIC - VIEWS --> SIM - VIEWS --> CHEM - VIEWS --> DV - - CHEM -->|enqueue| REDIS --> CELERY - CELERY -->|read/write| DB - CELERY -->|artifacts| FS - SIM -->|plots/artifacts| FS - TPL -->|static| FS - ``` - - !!! note "Frontend" - Mostly server-rendered templates with incremental updates via **HTMX**. - - ## Apps overview - - | App | Purpose | Main models | Templates | - | --- | --- | --- | --- | - | `clinic` | patients/assessments/therapies | `Patient`, `Assessment`, `PatientTherapy` | `clinic/templates/clinic/` | - | `simulator` | scenarios + simulations + twin | `Scenario`, `SimulationAttempt` | `simulator/templates/simulator/` | - | `chemtools` | background chem jobs | `ChemJob` | `chemtools/templates/chemtools/` | - | `docs_viewer` | in-app docs viewer | `DocumentView`, `DocumentFeedback` | `docs_viewer/templates/docs_viewer/` | - - ## Core flows +# Architecture - ```mermaid - sequenceDiagram - autonumber - actor U as User - participant W as Django view - participant DB as DB - participant M as Mathematical model - participant FS as media/ +This path is retained for existing links. - U->>W: Submit form (scenario + params) - W->>DB: Create/update SimulationAttempt - W->>M: run_model(parameters) - M-->>W: trajectory + summary + artifacts - W->>DB: Persist results/results_summary/artifacts (JSON) - W->>FS: (optional) save HTML/plots in media/ - W-->>U: Render results (HTMX or full page) - ``` +The former page described an earlier four-application architecture and omitted central `twin_engine`, dataset-lineage, model-registry and immutable-run infrastructure. It is no longer an independent source of architectural truth. - ## Persistence and ops +Use: - - Default DB: SQLite (`db.sqlite3`) - - Production: typically Postgres + `collectstatic`, background workers, logs. +- [Current verified system architecture](../architecture/CURRENT_SYSTEM.md) +- [Target virtual-laboratory architecture](../architecture/TARGET_VIRTUAL_LAB.md) +- [Current model registry and formulas](../models/REGISTRY.md) +- [Source-of-truth policy](../governance/SOURCE_OF_TRUTH.md) - !!! tip "More details" - For deeper engineering notes (patterns, security, performance, deployment), see `Guides → Architecture Deep Dive`. +Current and target architecture must not be merged into one diagram or capability list. diff --git a/docs/index.md b/docs/index.md index f1ac08e..fdccc8f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,40 +1,50 @@ +--- +title: bmyCure4MM Documentation +status: CANONICAL +owner: Andrea Zedda +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + # bmyCure4MM documentation -bmyCure4MM is an `E1_research_prototype` for reproducible computational -multiple-myeloma research. It stores structured research records, constructs -versioned model state, and runs mechanistic hypothetical scenarios. +bmyCure4MM is an `E1_research_prototype` for Multiple Myeloma computational research. Its target is a virtual and reproducible laboratory for virtual patients, current and experimental therapy hypotheses, falsification, reproducible protocols and an interdisciplinary Knowledge Commons. + +It is not clinical decision support. Patient-specific prediction is not validated and causal treatment effects are not identified. + +## First reading -It is not clinical decision support. Patient-specific prediction is not -validated, and causal treatment effects are not identified. +1. [Mission and north star](product/MISSION_AND_NORTH_STAR.md) +2. [Current verified state](product/CURRENT_STATE.md) +3. [Capabilities and limitations](product/CAPABILITIES_AND_LIMITATIONS.md) +4. [Canonical intended use](governance/INTENDED_USE.md) +5. [Source-of-truth hierarchy](governance/SOURCE_OF_TRUTH.md) -## Start with governance +## Architecture -1. [Canonical intended use](governance/INTENDED_USE.md) -2. [Claims policy](governance/CLAIMS_POLICY.md) -3. [Epistemic labels](governance/EPISTEMIC_LABELS.md) -4. [Model output language](governance/MODEL_OUTPUT_LANGUAGE.md) +- [Current system](architecture/CURRENT_SYSTEM.md) +- [Target virtual laboratory](architecture/TARGET_VIRTUAL_LAB.md) +- [Current model registry and formulas](models/REGISTRY.md) -## Current research surfaces +Current and target architecture are deliberately separated. A target component is not an implemented capability. -- **Clinic** records pseudonymized structured inputs and dated observations. -- **Research Twin** stores model state and lineage to its input snapshot. -- **Simulator** evaluates hypothetical mechanistic configurations. -- **Scientific Cockpit** exposes diagnostics, assumptions, and provenance. -- **ChemTools** is an optional exploratory cheminformatics surface. +## Research contracts -Every displayed simulation must be read as `SIMULATED` or `HYPOTHETICAL` and -must retain its stated model version and limitations. A successful process exit -means computation completed; it does not establish scientific or clinical -validity. +- [Structured dataset contract](research/STRUCTURED_DATASET_CONTRACT_V0_1.md) +- [Twin lineage](research/TWIN_LINEAGE.md) +- [Immutable run manifests](research/RUN_MANIFESTS.md) +- [Comparability and invalidation](research/COMPARABILITY_AND_INVALIDATION.md) +- [Real-patient Research Loop](research/RESEARCH_LOOP.md) +- [Validation and uncertainty](research/VALIDATION_UNCERTAINTY_PROTOCOL.md) -## Contributor path +## Governance and operations -- [Architecture](guides/architecture.md) -- [Database](guides/database.md) -- [Operations](guides/operations.md) -- [Development](development/DEVELOPMENT.md) -- [Research simulator](en/simulator.md) +- [Claims policy](governance/CLAIMS_POLICY.md) +- [Epistemic labels](governance/EPISTEMIC_LABELS.md) +- [Model output language](governance/MODEL_OUTPUT_LANGUAGE.md) +- [Documentation policy](governance/DOCUMENTATION_POLICY.md) +- [Authentication and access](operations/AUTHENTICATION_AND_ACCESS.md) +- [Testing](operations/TESTING.md) +- [Dependency operations](operations/DEPENDENCIES.md) -Historical material with superseded claims is isolated under -[`archive/`](archive/LEGACY_PRESCRIPTIVE_OUTPUTS.md) and is not a source of -current product claims. +A successful process exit means computation completed. It does not establish scientific validity, external validity or clinical utility. diff --git a/docs/operations/AUTHENTICATION_AND_ACCESS.md b/docs/operations/AUTHENTICATION_AND_ACCESS.md new file mode 100644 index 0000000..28ffac3 --- /dev/null +++ b/docs/operations/AUTHENTICATION_AND_ACCESS.md @@ -0,0 +1,66 @@ +--- +title: Authentication and Access +status: CURRENT_PARTIAL +owner: Andrea Zedda +audience: maintainers, security reviewers and operators +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + +# Authentication and access + +## Current verified boundary + +```text +public: documentation and selected simulator surfaces +protected: clinic, research, simulator-management and DRF/API surfaces +configured login target: Django admin login +normal product login: absent/incomplete +``` + +The repository uses Django authentication and selected `login_required` controls; DRF defaults to `IsAuthenticated`. This is not a complete product authentication architecture. + +## Temporary smoke identity + +The `ensure_m0_smoke_user` management command creates, rotates or disables a reserved temporary identity with: + +```text +staff = false +superuser = false +groups = 0 +direct_permissions = 0 +``` + +It is an operational smoke identity, not a product role. It cannot use the Django admin login and must not be promoted to staff to bypass the missing product-login contract. + +## Current authorization risk + +Authorization logic exists in several locations, but patient-derived list/detail/dashboard and aggregate behavior is not yet proven to use one central selector/policy layer. Issue `#8` is release-blocking for shared use. + +## Target role architecture — not current + +Candidate least-privilege roles include: + +```text +Research Viewer +Data Curator +Clinical Reviewer +Simulator Editor +Administrator +``` + +Permissions must distinguish view, edit, import, validate, simulate, export and administer. Role names do not become current until migrations, policies, selectors, tests and audit evidence exist. + +## Required access invariant + +Patient-derived objects should be resolved from a user-scoped selector before rendering or serialization: + +```python +patient = get_object_or_404(accessible_patients(request.user), pk=pk) +``` + +The exact implementation is governed by issue `#8` and may differ, but post-load authorization after disclosure is not acceptable. + +## Audit boundary + +Future access events should record actor ID/role snapshot, action, pseudonymous resource reference, allowed/denied decision, policy reason, request ID and status without patient names, raw query strings or clinical payloads. diff --git a/docs/operations/TESTING.md b/docs/operations/TESTING.md new file mode 100644 index 0000000..a3dfaf0 --- /dev/null +++ b/docs/operations/TESTING.md @@ -0,0 +1,75 @@ +--- +title: Testing and Verification +status: CURRENT_PARTIAL +owner: Andrea Zedda +audience: contributors and reviewers +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + +# Testing and verification + +## Epistemic boundary + +```text +software verification +≠ model verification +≠ calibration evidence +≠ external validation +≠ clinical utility +``` + +A green test suite proves only the tested contracts under the tested environment. + +## Canonical current commands + +```bash +uv lock --check +uv sync --frozen --extra chemistry +uv run python manage.py check +uv run python manage.py makemigrations --check --dry-run +uv run python manage.py test +uv run ruff check . +uv run ruff format --check . +uv run mypy +uv run python -m scripts.check_repository_hygiene +uv run python -m scripts.check_numerical_baseline +uv run python -m scripts.audit_dependencies +bash scripts/pre_push_research_safety_check.sh +uv run --frozen --no-default-groups --group docs mkdocs build --strict +``` + +Use the dependency groups documented in `DEPENDENCIES.md`; core-only and optional-chemistry environments have different purposes. + +## Current GitHub Actions + +Current workflows cover dependency/reproducibility checks, docs deployment, secret scanning, image workflows, UI screenshots and redeployment. Issue `#13` will rationalize them into one stable protected aggregate gate and add exact pull-request safety behavior. + +## Research safety gate + +The current local safety script is designed around staged changes. Until issue `#13` adds a verified base/head CI mode, do not assume that merely invoking the script in a pull-request runner checks the exact proposed diff. + +## Numerical baseline + +The numerical baseline detects unreviewed output drift. A baseline update requires: + +- an output-diff report; +- a model/version decision; +- an invalidation decision for prior runs; +- updated documentation when endpoint interpretation changes. + +## Test data + +Tests and CI use synthetic or explicitly governed demo data only. Never copy private patient payloads, source documents or direct identifiers into fixtures, logs, snapshots or uploaded artifacts. + +## M0-R minimum QA + +Issue `#23` Phase A covers: + +- dataset, input, Twin-lineage, run-manifest and artifact contracts; +- canonical hashing and semantic idempotency; +- schedule identity and deterministic numerical invariants; +- current high-risk migration preservation; +- core privacy, authorization and forbidden-claim regressions. + +PostgreSQL, extensive property testing, browser E2E, coverage ratchets and asynchronous lifecycle testing remain extended scope unless explicitly completed or transferred to linked issues. diff --git a/docs/research/COMPARABILITY_AND_INVALIDATION.md b/docs/research/COMPARABILITY_AND_INVALIDATION.md new file mode 100644 index 0000000..b6d022d --- /dev/null +++ b/docs/research/COMPARABILITY_AND_INVALIDATION.md @@ -0,0 +1,59 @@ +--- +title: Run Comparability and Invalidation +status: CURRENT_VERIFIED +owner: Andrea Zedda +audience: research users and model maintainers +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + +# Run comparability and invalidation + +## Immutable runs + +A completed research run is not edited. A change in data, code, model, configuration, evidence, endpoint definition or seed creates a new run and a new manifest. + +## Version vector + +A run identity contains the governed application/API/schema, dataset and subset identity, model and schema versions, configuration hash, evidence/validation/report versions, Git SHA, container identity where available, random seed, intended-use level and epistemic label. + +## Direct comparability + +Direct comparison is computed, not assumed. At minimum the system evaluates compatibility of: + +```text +model major version +input/output schema +endpoint definitions +units +intended-use class +data compatibility +``` + +When these conditions fail, results are marked: + +```text +NOT_DIRECTLY_COMPARABLE_WITHOUT_RERUN +``` + +## Example + +Two runs with identical data and code but different definitions of `toxicity_score` are not directly comparable, even if both values are between zero and one. The endpoint-semantic change requires a model/endpoint version decision and usually a rerun. + +## Invalidation + +Corrections or changes that may invalidate prior runs include: + +- a corrected clinical value or unit; +- a changed dataset subset; +- a model equation or parameter default; +- changed endpoint semantics; +- a numerical dependency upgrade that alters output; +- an artifact hash mismatch; +- an intended-use change. + +Invalidation is append-only and records the reason; prior artifacts are not silently overwritten. + +## Boundary + +Comparability and integrity prove whether outputs may be contrasted under the contract. They do not prove that either model is scientifically valid. diff --git a/docs/research/RESEARCH_LOOP.md b/docs/research/RESEARCH_LOOP.md new file mode 100644 index 0000000..49dde41 --- /dev/null +++ b/docs/research/RESEARCH_LOOP.md @@ -0,0 +1,74 @@ +--- +title: Real-Patient Research Loop v0.1 +status: TARGET_APPROVED +owner: Andrea Zedda +audience: M1 contributors and reviewers +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +canonical_tracker: https://github.com/andreazedda/bmyCure4MM/issues/28 +--- + +# Real-Patient Research Loop v0.1 + +> **TARGET M1 workflow.** The repository has many prerequisites, but this complete loop has not yet been demonstrated. + +## Flow + +```text +source-verified evidence +→ immutable dataset +→ frozen computational input +→ baseline Twin +→ pre-calibration residuals +→ calibration accepted or rejected +→ exposure-aware scenarios +→ toxicity attribution with competing explanations +→ temporal backtest against baselines +→ uncertainty and identifiability report +→ immutable research report and manifest +``` + +## Completed prerequisites + +- Twin and computational-input lineage; +- E1 intended use and epistemic output language; +- deterministic dependency graph; +- immutable run manifests, model registry, artifact integrity and comparability. + +## Remaining milestone prerequisite + +M0-R must close through documentation, protected CI and minimum QA issues `#14 → #13 → #23`. + +Private read-only source verification may prepare in parallel. New canonical calibration and patient-specific what-if claims cannot be promoted before M0-R closes. + +## Scientific work order + +```text +#32 source assertions and dataset +→ #33 observation model +→ #34 day-resolved lenalidomide exposure +→ #35 toxicity attribution +→ #36 calibration, identifiability and temporal validation +→ #18 immutable end-to-end loop +``` + +## Acceptance is not positivity + +Valid outcomes include: + +```text +CALIBRATION_ACCEPTED +INSUFFICIENT_DATA +CALIBRATION_FAILED +MODEL_NOT_IDENTIFIABLE +NO_OUT_OF_SAMPLE_GAIN +BACKTEST_UNACCEPTABLE +``` + +## Forbidden conclusions + +- an alternative regimen would have produced a superior clinical outcome; +- a mechanistic trajectory is an identified causal effect; +- optimizer completion proves validity; +- unresolved source ambiguity may be silently imputed; +- in-sample RMSE improvement alone establishes patient-specific prediction. diff --git a/docs/research/TWIN_LINEAGE.md b/docs/research/TWIN_LINEAGE.md new file mode 100644 index 0000000..9967423 --- /dev/null +++ b/docs/research/TWIN_LINEAGE.md @@ -0,0 +1,61 @@ +--- +title: Twin and Computational-Input Lineage +status: CURRENT_VERIFIED +owner: Andrea Zedda +audience: model developers and research reviewers +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + +# Twin and computational-input lineage + +## Purpose + +A Twin state is interpretable only when its data subset, model, configuration, code identity and parent state are explicit. + +## Computational-input identity + +The current contract binds a governed structured subset to model/configuration/code identity and produces content-addressed input metadata. + +A change to a governed input must produce a new computational-input identity. Historical states without the current lineage contract remain explicitly legacy/unbound; they are not silently reinterpreted. + +## Twin lineage + +A lineage-bound state records at least: + +```text +lineage schema/version +computational-input identity +model version +configuration hash +code/Git identity +dataset identity/subset hash +parent Twin state when applicable +calibration relationship +``` + +Calibration creates a child state; it does not mutate the parent state into a different scientific object. + +## Readiness invariant + +Residuals or calibration artifacts from another lineage cannot satisfy readiness for the current Twin. + +```text +same patient +≠ same computational input +≠ same Twin lineage +``` + +## Historical data + +The lineage schema migration preserved historical states with an empty/legacy lineage representation. Such states remain readable but cannot be treated as current lineage-bound evidence without an explicit new state. + +## Scientific boundary + +Valid lineage proves identity and traceability. It does not prove: + +- that the state variables are biologically correct; +- that parameters are identifiable; +- that calibration is accepted; +- that future prediction improves over a baseline; +- that a counterfactual is causal. diff --git a/mkdocs.yml b/mkdocs.yml index ceab5a2..d7947c8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,5 +1,5 @@ site_name: bmyCure4MM Docs -site_description: E1 reproducible Multiple Myeloma computational research prototype +site_description: E1 Multiple Myeloma computational research prototype; target E2 reproducible research site_url: https://andreazedda.github.io/bmyCure4MM/ repo_url: https://github.com/andreazedda/bmyCure4MM repo_name: andreazedda/bmyCure4MM @@ -8,101 +8,68 @@ edit_uri: edit/master/docs/ nav: - Home: index.md - Start Here: - - Canonical Intended Use: governance/INTENDED_USE.md + - Mission and North Star: product/MISSION_AND_NORTH_STAR.md + - Current Verified State: product/CURRENT_STATE.md + - Capabilities and Limitations: product/CAPABILITIES_AND_LIMITATIONS.md + - Intended Use: governance/INTENDED_USE.md + - Source of Truth: governance/SOURCE_OF_TRUTH.md + - Roadmap: product/ROADMAP.md + - Architecture: + - Current System: architecture/CURRENT_SYSTEM.md + - Target Virtual Laboratory: architecture/TARGET_VIRTUAL_LAB.md + - Compatibility Architecture Link: guides/architecture.md + - Models: + - Current Registry and Formulas: models/REGISTRY.md + - Runtime Mathematical Models — IT: guides/mathematical_models.md + - Runtime Mathematical Models — EN: guides/mathematical_models_en.md + - Numerical Reproducibility: research/NUMERICAL_REPRODUCIBILITY.md + - Research Contracts: + - Structured Dataset Contract: research/STRUCTURED_DATASET_CONTRACT_V0_1.md + - Twin Lineage: research/TWIN_LINEAGE.md + - Run Manifests: research/RUN_MANIFESTS.md + - Comparability and Invalidation: research/COMPARABILITY_AND_INVALIDATION.md + - Real-Patient Research Loop: research/RESEARCH_LOOP.md + - Validation and Uncertainty: research/VALIDATION_UNCERTAINTY_PROTOCOL.md + - Real-Patient What-if Architecture: research/REAL_PATIENT_WHATIF_ARCHITECTURE.md + - Research Cockpit Guide: research/RESEARCH_COCKPIT_USER_GUIDE.md + - Operations: + - Authentication and Access: operations/AUTHENTICATION_AND_ACCESS.md + - Testing: operations/TESTING.md + - Dependencies: operations/DEPENDENCIES.md + - Dependency Audit Triage: operations/DEPENDENCY_AUDIT_TRIAGE.md + - Development: development/DEVELOPMENT.md + - Troubleshooting: development/TROUBLESHOOTING.md + - Governance: - Claims Policy: governance/CLAIMS_POLICY.md - Epistemic Labels: governance/EPISTEMIC_LABELS.md - Model Output Language: governance/MODEL_OUTPUT_LANGUAGE.md - - Learning Path: guides/learning_path.md - - Tutorial (EN): en/TUTORIAL_FIRST_RUN.md - - Tutorial (IT): it/TUTORIAL_PRIMO_USO.md - - Guides: - - Core: - - Architecture: guides/architecture.md - - Architecture Deep Dive: guides/architecture_deep_dive.md - - Database: guides/database.md - - Operations: guides/operations.md - - Modeling: - - Mathematical Models: - - Italiano: guides/mathematical_models.md - - English: guides/mathematical_models_en.md - - Optimization Theory: guides/optimization_theory.md - - Virtual Patients: guides/virtual_patients.md - - Exploratory Research Context: - - Regimen Context: guides/regimen_suggester.md - - Model-relative Thresholds: guides/decision_algorithm.md - - Docs: - - Glossary: guides/glossary.md - - Simulations Gallery: guides/simulations_gallery.md - - Simulator KPIs (How to Read): guides/simulator_kpis.md - - Custom drug (experimental): guides/custom_drug_experimental.md - - Doc Coverage (Gaps): guides/gaps.md - - Documentation Standard: guides/documentation_standard.md - - Documentation Roadmap: guides/documentation_roadmap.md - - Development: development/DEVELOPMENT.md - - Troubleshooting: development/TROUBLESHOOTING.md - - Reproducibility: - - Dependency Operations: operations/DEPENDENCIES.md - - Dependency Audit Triage: operations/DEPENDENCY_AUDIT_TRIAGE.md - - Numerical Policy: research/NUMERICAL_REPRODUCIBILITY.md + - Documentation Policy: governance/DOCUMENTATION_POLICY.md + - Documentation Inventory: DOCUMENTATION_INVENTORY.yaml - Reference: - Configuration: reference/configuration.md - Database Objects: reference/database_objects.md - Simulator Parameters: reference/simulator_parameters.md - Endpoints: reference/endpoints.md - Scripts: reference/scripts.md - - Repo Docs README: repo_readme.md - Apps: - Clinic: reference/apps/clinic.md - Simulator: reference/apps/simulator.md - ChemTools: reference/apps/chemtools.md - Docs Viewer: reference/apps/docs_viewer.md - - Deep Dives: - - Data: - - Dynamic Data Architecture: dynamic_data_architecture.md - - API: - - API Integration Summary: api_integration_summary.md - - API Integration Guide: API_INTEGRATION_GUIDE.md - - API Data Sources Quickref: API_DATA_SOURCES_QUICKREF.md - - API Enhancement Summary: API_ENHANCEMENT_SUMMARY.md - - MM API Selection Guide: MM_API_SELECTION_GUIDE.md - - Simulator: - - Twin In-App Guidance: IN_APP_GUIDANCE_TWIN_SIMULATOR_REGIMENS_THERAPIES.md - - Patient Twin (As Built): development/PATIENT_TWIN_AS_BUILT.md - - ChemTools: - - 3D Viewer Styling: 3D_VIEWER_STYLING.md - - Product: - - Before/After Comparison: BEFORE_AFTER_COMPARISON.md - - Development Logs: - - Embed Debug Guide: development/EMBED_DEBUG_GUIDE.md - - Implementation Summary: development/IMPLEMENTATION_SUMMARY.md - - Features: - - Docs Viewer (Complete): features/DOCS_VIEWER_COMPLETE.md - - ChemTools Testing Summary: features/CHEMTOOLS_TESTING_SUMMARY.md - - Testing Report: features/TESTING_REPORT.md - - Archive: - - Legacy Prescriptive Outputs: archive/LEGACY_PRESCRIPTIVE_OUTPUTS.md - - Legacy Unvalidated Model Outputs: archive/LEGACY_UNVALIDATED_MODEL_OUTPUTS.md - - Legacy Twin Architecture: archive/PATIENT_TWIN_TARGET_ARCHITECTURE_PRE_E1.md - - EN: - - Quickstart: en/quickstart.md - - Simulator: en/simulator.md - - Optimization Lab: en/optimization_lab.md - - Patient Twin: en/patient_twin.md - - PK/PD Presets: en/pkpd_presets.md - - Pipelines: en/pipelines.md - - API: en/api.md - - IT: - - Avvio rapido: it/avvio_rapido.md - - Simulatore: it/simulatore.md - - Optimization Lab: it/optimization_lab.md - - Gemello Paziente: it/gemello_paziente.md - - Preset PK/PD: it/pkpd_preset.md - - Pipeline: it/pipeline.md - - API: it/api.md + - Learning: + - Learning Path: guides/learning_path.md + - Glossary: guides/glossary.md + - Simulator KPIs: guides/simulator_kpis.md + - Model-relative Thresholds: guides/decision_algorithm.md + - Archive: + - Legacy Prescriptive Outputs: archive/LEGACY_PRESCRIPTIVE_OUTPUTS.md + - Legacy Unvalidated Outputs: archive/LEGACY_UNVALIDATED_MODEL_OUTPUTS.md + - Legacy Twin Architecture: archive/PATIENT_TWIN_TARGET_ARCHITECTURE_PRE_E1.md + - 2024 Release Preparation: archive/releases/RELEASE_PREPARATION_2024.md theme: name: material - language: it + language: en features: - navigation.tabs - navigation.sections From e18f3cbcddb5ca85fa38299ca9991a2adc6c3228 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 00:25:10 +0200 Subject: [PATCH 03/18] docs: fix documentation status taxonomy and navigable inventory --- docs/governance/DOCUMENTATION_POLICY.md | 14 ++++++-- docs/product/DOCUMENTATION_INVENTORY.md | 43 +++++++++++++++++++++++++ mkdocs.yml | 2 +- 3 files changed, 56 insertions(+), 3 deletions(-) create mode 100644 docs/product/DOCUMENTATION_INVENTORY.md diff --git a/docs/governance/DOCUMENTATION_POLICY.md b/docs/governance/DOCUMENTATION_POLICY.md index 85640bc..c5284fd 100644 --- a/docs/governance/DOCUMENTATION_POLICY.md +++ b/docs/governance/DOCUMENTATION_POLICY.md @@ -14,6 +14,8 @@ last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 Every governed document must use one status: ```text +CANONICAL +CANONICAL_STRATEGY CURRENT_VERIFIED CURRENT_PARTIAL CURRENT_HEURISTIC_PROTOTYPE @@ -26,6 +28,12 @@ REJECTED HISTORICAL_ARCHIVE ``` +- `CANONICAL` governs a current policy or source-of-truth contract. +- `CANONICAL_STRATEGY` governs the approved mission or roadmap while keeping target capabilities separate from current implementation. +- `CURRENT_*` content maps to present repository behavior at a verified SHA. +- `TARGET_APPROVED` content is approved design, not current implementation. +- `RESEARCH_HYPOTHESIS` content requires competing hypotheses and a falsification path. + ## Required metadata Current and target documents must declare: @@ -49,7 +57,7 @@ Fields that do not apply may be omitted, but `status`, `owner`, `last_verified_a - `CURRENT_*` content must map to an existing code/schema/registry entry point. - `TARGET_APPROVED` content must state prerequisites and acceptance evidence. - `RESEARCH_HYPOTHESIS` content must include competing hypotheses and a falsification path. -- historical content must begin with an archive banner and must not appear in current navigation. +- historical content must begin with an archive banner and must not appear in current navigation except inside the explicit Archive section. ## Formula standard @@ -94,6 +102,8 @@ Current documents inherit `INTENDED_USE.md` and `CLAIMS_POLICY.md`. Archived cla ## Inventory and navigation -`docs/DOCUMENTATION_INVENTORY.yaml` classifies the documentation corpus. Only `CANONICAL`, `CURRENT_SUPPORTING` and explicitly selected `TARGET_SUPPORTING` documents may appear in current MkDocs navigation. +`docs/DOCUMENTATION_INVENTORY.yaml` classifies the documentation corpus. [Documentation Inventory](../product/DOCUMENTATION_INVENTORY.md) explains how to read the machine-readable file. + +Only `CANONICAL`, `CURRENT_SUPPORTING` and explicitly selected `TARGET_SUPPORTING` inventory classifications may appear in current MkDocs navigation. Historical files may appear only in the explicit Archive section. Unmatched files resolve to `UNCLASSIFIED_REQUIRES_REVIEW` and must stay outside current navigation until reviewed. diff --git a/docs/product/DOCUMENTATION_INVENTORY.md b/docs/product/DOCUMENTATION_INVENTORY.md new file mode 100644 index 0000000..8ae58a0 --- /dev/null +++ b/docs/product/DOCUMENTATION_INVENTORY.md @@ -0,0 +1,43 @@ +--- +title: Documentation Inventory +status: CANONICAL +owner: Andrea Zedda +audience: maintainers and documentation reviewers +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +machine_readable_source: ../DOCUMENTATION_INVENTORY.yaml +--- + +# Documentation inventory + +The machine-readable inventory is [`docs/DOCUMENTATION_INVENTORY.yaml`](../DOCUMENTATION_INVENTORY.yaml). + +It classifies documentation by explicit path and ordered glob rules. The first matching rule wins. Any unmatched file is: + +```text +UNCLASSIFIED_REQUIRES_REVIEW +``` + +and must remain outside current navigation until reviewed. + +## Inventory classifications + +| Classification | Meaning | +|---|---| +| `CANONICAL` | Current authority for project state, governance, architecture or contracts | +| `CURRENT_SUPPORTING` | Current support material governed by a canonical parent | +| `TARGET_SUPPORTING` | Approved target design, explicitly not current implementation | +| `HISTORICAL_ARCHIVE` | Retained history; not a current product claim | +| `DUPLICATE` | Redundant with a canonical document | +| `DELETE_CANDIDATE` | No continuing value after review | +| `UNCLASSIFIED_REQUIRES_REVIEW` | Currency and ownership not yet proven | + +## Review rule + +A document may be moved into current navigation only when: + +1. its status and owner are explicit; +2. its claims match the canonical intended use; +3. current/target/hypothesis content is separated; +4. referenced paths, routes, commands, formulas and model versions are verified; +5. its `last_verified_git_sha` is recorded. diff --git a/mkdocs.yml b/mkdocs.yml index d7947c8..9998ec1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -44,7 +44,7 @@ nav: - Epistemic Labels: governance/EPISTEMIC_LABELS.md - Model Output Language: governance/MODEL_OUTPUT_LANGUAGE.md - Documentation Policy: governance/DOCUMENTATION_POLICY.md - - Documentation Inventory: DOCUMENTATION_INVENTORY.yaml + - Documentation Inventory: product/DOCUMENTATION_INVENTORY.md - Reference: - Configuration: reference/configuration.md - Database Objects: reference/database_objects.md From 69f4b93df3df73d8edb4fc60bb0bb0a566b80e81 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 00:27:35 +0200 Subject: [PATCH 04/18] ci(docs): validate strict MkDocs builds on pull requests --- .github/workflows/docs.yml | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 820ed4a..e5d59f9 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -1,8 +1,27 @@ -name: Deploy Docs (MkDocs) +name: Docs Build and Deploy (MkDocs) on: + pull_request: + paths: + - "docs/**" + - "README.md" + - "CHANGELOG.md" + - "SECURITY.md" + - "mkdocs.yml" + - "pyproject.toml" + - "uv.lock" + - ".github/workflows/docs.yml" push: branches: [master, main] + paths: + - "docs/**" + - "README.md" + - "CHANGELOG.md" + - "SECURITY.md" + - "mkdocs.yml" + - "pyproject.toml" + - "uv.lock" + - ".github/workflows/docs.yml" workflow_dispatch: permissions: @@ -11,12 +30,14 @@ permissions: id-token: write concurrency: - group: pages - cancel-in-progress: false + group: docs-${{ github.ref }} + cancel-in-progress: true jobs: build: + name: strict-build runs-on: ubuntu-latest + timeout-minutes: 15 steps: - uses: actions/checkout@v4 @@ -32,15 +53,17 @@ jobs: - name: Install docs dependencies run: uv sync --frozen --no-default-groups --group docs - - name: Build site + - name: Build site in strict mode run: uv run --frozen --no-default-groups --group docs mkdocs build --strict - name: Upload Pages artifact + if: github.event_name == 'push' uses: actions/upload-pages-artifact@v3 with: path: site deploy: + if: github.event_name == 'push' runs-on: ubuntu-latest needs: build environment: From b8bc4abcd9c242ce28d2c6711c6eb95c8012e0ba Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 00:28:51 +0200 Subject: [PATCH 05/18] docs: repair strict-build links in historical release archive --- docs/archive/releases/RELEASE_PREPARATION_2024.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/archive/releases/RELEASE_PREPARATION_2024.md b/docs/archive/releases/RELEASE_PREPARATION_2024.md index 9bd042d..669448d 100644 --- a/docs/archive/releases/RELEASE_PREPARATION_2024.md +++ b/docs/archive/releases/RELEASE_PREPARATION_2024.md @@ -30,6 +30,6 @@ The original bytes remain recoverable from Git history at `RELEASE_PREPARATION.m - [Mission and north star](../../product/MISSION_AND_NORTH_STAR.md) - [Current verified state](../../product/CURRENT_STATE.md) - [Capabilities and limitations](../../product/CAPABILITIES_AND_LIMITATIONS.md) -- [Security policy](../../../SECURITY.md) -- [Changelog](../../../CHANGELOG.md) +- [Security policy on GitHub](https://github.com/andreazedda/bmyCure4MM/blob/master/SECURITY.md) +- [Changelog on GitHub](https://github.com/andreazedda/bmyCure4MM/blob/master/CHANGELOG.md) - [Documentation policy](../../governance/DOCUMENTATION_POLICY.md) From 3bc54788f8a17143a0cd6fe21fdf66d35f6e81d2 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 00:46:33 +0200 Subject: [PATCH 06/18] docs: record active dependency and framework security blockers --- CHANGELOG.md | 2 + README.md | 8 +- SECURITY.md | 12 +- docs/operations/DEPENDENCY_AUDIT_TRIAGE.md | 129 ++++++++++++++----- docs/operations/TESTING.md | 32 +++++ docs/product/CAPABILITIES_AND_LIMITATIONS.md | 8 +- docs/product/CURRENT_STATE.md | 29 ++++- docs/product/ROADMAP.md | 10 +- 8 files changed, 188 insertions(+), 42 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b3f3a0..c4edccd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ All current release claims are governed by [`docs/governance/CLAIMS_POLICY.md`]( - Added a current model-status catalog with formulas, units, examples, code mappings and falsification boundaries. - Added source-of-truth and documentation-governance policies plus a corpus inventory. - Reconciled GitHub roadmap/tracker state and Notion project state at canonical `master` `bf097810...`. +- Added strict MkDocs validation to documentation pull requests without deploying PR content. ### Scientific identity and reproducibility @@ -19,6 +20,7 @@ All current release claims are governed by [`docs/governance/CLAIMS_POLICY.md`]( - Added immutable research run manifests and a governed model registry. - Added typed scientific payload contracts, artifact integrity checks, direct-comparability rules and append-only invalidation. - Locked the dependency graph with `uv.lock`, Python 3.11/3.12 support and numerical/repository/dependency checks. +- Recorded newly disclosed Django/sqlparse advisories as a blocking current condition under issues `#69` and `#70`; no dependency fix or audit bypass is claimed by this documentation change. ### Intended use and language diff --git a/README.md b/README.md index 2e02c19..d11fa8d 100644 --- a/README.md +++ b/README.md @@ -37,11 +37,15 @@ Current M0-R path: ```text #14 documentation/source of truth ++ #69 immediate dependency-advisory remediation +→ #70 supported Django 5.2 LTS baseline → #13 mandatory CI aggregate gate and PR governance → #23 minimum scientific/privacy invariants → #26 close M0-R ``` +The current lock remains deterministic, but the security audit now detects newly disclosed advisories in Django 4.2.30 and sqlparse 0.5.4. The durable Django migration is tracked in `#70`; no red dependency gate may be relabelled green through blanket suppression. + ## Current capabilities - versioned Structured Research Dataset Contract and idempotent import infrastructure; @@ -72,7 +76,7 @@ uv run python manage.py check uv run python manage.py runserver ``` -Omit `--extra chemistry` for core work that does not require RDKit. Dependency operations are documented in [Dependency Operations](docs/operations/DEPENDENCIES.md). +Omit `--extra chemistry` for core work that does not require RDKit. Dependency operations are documented in [Dependency Operations](docs/operations/DEPENDENCIES.md). The current Django 4.2 lock is unsupported and must not be promoted to a shared/production baseline before issues `#69` and `#70` are resolved. ## Verification and safety @@ -91,7 +95,7 @@ uv run python -m scripts.audit_dependencies bash scripts/pre_push_research_safety_check.sh ``` -Current test boundaries and CI limitations are documented in [Testing](docs/operations/TESTING.md). +Current test boundaries, the active dependency blocker and CI limitations are documented in [Testing](docs/operations/TESTING.md) and [Dependency Audit Triage](docs/operations/DEPENDENCY_AUDIT_TRIAGE.md). ## Privacy diff --git a/SECURITY.md b/SECURITY.md index 44938e3..e3ada23 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -9,6 +9,7 @@ production_security_certified: false clinical_use_supported: false product_auth_model_complete: false object_authorization_complete: false +dependency_security_green: false ``` No current version is represented as production-certified or suitable for clinical deployment. Security fixes are handled on the active `master` research line, subject to maintainer capacity and explicit verification. @@ -50,6 +51,15 @@ These controls do not prove complete application security. ## Known open security gaps +### Dependency and framework support + +The deterministic lock currently pins unsupported Django 4.2.30 and sqlparse 0.5.4. Newly disclosed advisories make the dependency audit red. + +- issue `#69` tracks immediate sqlparse upgrade and exact advisory triage; +- issue `#70` tracks durable migration to Django 5.2 LTS; +- shared/production promotion is prohibited while the framework baseline is unsupported or security findings remain untriaged; +- blanket audit suppression is prohibited. + ### Product authentication Documentation and selected simulator surfaces are public, while clinic/research/simulator-management/API surfaces are protected. `LOGIN_URL` currently targets the Django admin login and there is no reviewed normal-user product-login and role contract. @@ -86,7 +96,7 @@ Use synthetic/demo fixtures in tests and CI. Security evidence must identify fin ## Deployment boundary -The repository may contain deployment automation, but a successful image build or running deployment does not establish production security. A deployment must separately prove configuration, database, storage, backup/restore, observability, access control and incident-response requirements. +The repository may contain deployment automation, but a successful image build or running deployment does not establish production security. A deployment must separately prove configuration, supported dependencies, database, storage, backup/restore, observability, access control and incident-response requirements. ## Security-related issues diff --git a/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md b/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md index fbe6eb3..17598a5 100644 --- a/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md +++ b/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md @@ -1,31 +1,98 @@ -# Dependency Audit Triage - -Audit date: 2026-08-13 - -The broad legacy environment had 161 findings in 25 packages. The canonical -locked environment removes 155 findings. `pip-audit` reports six remaining -Django advisories; all are low or medium severity and none has a Django 4.2 -patch. Issue #15 requires one consistent Django 4.2 series, so Django is pinned -to the final 4.2.30 release and each exception is narrowly allowlisted by ID. - -| Advisory | Severity | Affected surface | Repository evidence / mitigation | -| --- | --- | --- | --- | -| [GHSA-923m-gv2p-w5qp](https://github.com/advisories/GHSA-923m-gv2p-w5qp) | low | `has_vary_header` whitespace caching | No project call and no Django cache middleware. | -| [GHSA-h7pc-vwp9-298g](https://github.com/advisories/GHSA-h7pc-vwp9-298g) | low | signed-cookie salt collision | Database sessions are used; no signed-cookie session backend. | -| [GHSA-8cjm-8mp7-r2xf](https://github.com/advisories/GHSA-8cjm-8mp7-r2xf) | low | `UpdateCacheMiddleware` header handling | Cache update middleware is absent. | -| [GHSA-3h9f-r86x-qvjx](https://github.com/advisories/GHSA-3h9f-r86x-qvjx) | low | cache middleware private responses | Django cache middleware is absent. | -| [GHSA-crhf-3pfg-w68w](https://github.com/advisories/GHSA-crhf-3pfg-w68w) | medium | `GDALRaster` byte over-read | GeoDjango/GDAL are neither declared nor imported. | -| [GHSA-8qcx-xf44-272x](https://github.com/advisories/GHSA-8qcx-xf44-272x) | medium | `DomainNameValidator` newline acceptance | The validator is not imported; host configuration is parsed and validated separately. | - -One additional medium development-only advisory, -[GHSA-6w46-j5rx-g56g](https://github.com/advisories/GHSA-6w46-j5rx-g56g), -affects pytest temporary-directory handling on shared UNIX hosts. The compatible -`pytest-playwright==0.7.1` release requires pytest below 9. Tests run on isolated -CI runners or single-user development machines; untrusted users must not share -that host. Remove the exception as soon as pytest-playwright supports patched -pytest 9. - -There are no ignored critical or high findings. `scripts/audit_dependencies.py` -ignores only these exact identifiers, so any new advisory fails CI. A future -Django 5.2 migration must remove the Django exceptions and rerun authorization, -migration, template, numerical, and output-language regression suites. +--- +title: Dependency Audit Triage +status: CURRENT_PARTIAL +owner: Andrea Zedda +audience: maintainers and security reviewers +last_verified_at: 2026-08-24 +last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +--- + +# Dependency audit triage + +## Current status — 24 August 2026 + +```text +LOCK_DETERMINISTIC = true +DEPENDENCY_AUDIT_GREEN = false +UNTRIAGED_HIGH_FINDINGS = true +SHARED_OR_PRODUCTION_PROMOTION = prohibited +``` + +The canonical lock was not changed by documentation PR `#68`. Current advisory data nevertheless makes the repository-native audit fail. This is a newly current supply-chain condition. + +## Current findings + +### Django + +| Package | Advisory | Severity | Affected component | Current decision | +|---|---|---:|---|---| +| Django 4.2.30 | `PYSEC-2026-3717` / `CVE-2026-15830` | medium | GeoDjango `GEOSGeometry` and spatial geometry parsing | Applicability must be reproduced. Django 4.2 is unsupported; migrate under `#70`. | + +Initial repository code search found no direct `django.contrib.gis`, `GEOSGeometry` or GIS-field use. This is not sufficient for a permanent exception. A clean-checkout scan and runtime/settings review are required. + +Django 4.2.30 reached end of extended support on 7 April 2026. The durable target is a current Django 5.2 LTS security patch under issue `#70`. + +### sqlparse + +| Package | Advisory | Severity | Patched version | Current decision | +|---|---|---:|---:|---| +| sqlparse 0.5.4 | `PYSEC-2026-3696` / `CVE-2026-59894` | moderate | 0.6.0 | Upgrade required | +| sqlparse 0.5.4 | `PYSEC-2026-3697` / `CVE-2026-71491` | high | 0.6.0 | Upgrade required | +| sqlparse 0.5.4 | `PYSEC-2026-3698` / `CVE-2026-59893` | high | 0.6.0 | Upgrade required | +| sqlparse 0.5.4 | `PYSEC-2026-3699` / `CVE-2026-54284` | high | 0.6.0 | Upgrade required | + +Repository search finds no direct sqlparse call outside dependency/lock declarations. Django and tooling may still invoke parsing paths; absence of direct calls is not a risk-acceptance basis. Upgrade and regenerate the lock under `#69`. + +## Temporary Django exception policy + +A temporary exact allowlist for `PYSEC-2026-3717` is allowed only when: + +```text +no GIS import/app/model/form/exposed parsing path is proven +owner and rationale are recorded +expiry <= 2026-09-30 +issue #70 remains open and P0 +shared/production promotion remains prohibited +``` + +No blanket ignore is permitted. The exception must be removed when `#70` merges. + +## Previous 13 August triage + +The earlier locked baseline removed 155 of 161 broad-environment findings and narrowly triaged six then-current Django advisories plus one development-only pytest advisory. Those decisions remain historical evidence but do not cover newly published advisory IDs. + +Previously triaged exact IDs: + +```text +GHSA-923m-gv2p-w5qp +GHSA-h7pc-vwp9-298g +GHSA-8cjm-8mp7-r2xf +GHSA-3h9f-r86x-qvjx +GHSA-crhf-3pfg-w68w +GHSA-8qcx-xf44-272x +GHSA-6w46-j5rx-g56g +``` + +## Required record fields + +Every exception or fix records: + +```text +advisory ID +package/version +severity +applicability status +technical rationale +evidence +owner +expiry/review date +fix issue/PR/commit +``` + +## Current execution + +```text +#69 sqlparse upgrade and immediate exact advisory decision +→ #70 Django 5.2 LTS migration and removal of Django 4.2 exceptions +→ #13 protected aggregate CI +``` diff --git a/docs/operations/TESTING.md b/docs/operations/TESTING.md index a3dfaf0..6955895 100644 --- a/docs/operations/TESTING.md +++ b/docs/operations/TESTING.md @@ -45,6 +45,28 @@ Use the dependency groups documented in `DEPENDENCIES.md`; core-only and optiona Current workflows cover dependency/reproducibility checks, docs deployment, secret scanning, image workflows, UI screenshots and redeployment. Issue `#13` will rationalize them into one stable protected aggregate gate and add exact pull-request safety behavior. +Documentation PR `#68` added strict MkDocs validation on pull requests without deploying PR content. This is a bounded prerequisite for `#13`, not the final aggregate CI design. + +## Current recorded PR evidence + +At PR `#68` head `b8bc4abc...`: + +```text +strict MkDocs: PASS +secret scan: PASS +Ruff: PASS +format: PASS +mypy: PASS +repository hygiene: PASS +numerical baseline: PASS / unchanged +Python 3.11 core check/tests: PASS +Python 3.12 core check/tests: PASS +Docker locked build: PASS +dependency audit: FAIL +``` + +The audit failure is not caused by the documentation diff. It identifies newly disclosed advisories in the unchanged canonical dependency graph and is governed by `#69` and `#70`. + ## Research safety gate The current local safety script is designed around staged changes. Until issue `#13` adds a verified base/head CI mode, do not assume that merely invoking the script in a pull-request runner checks the exact proposed diff. @@ -58,6 +80,16 @@ The numerical baseline detects unreviewed output drift. A baseline update requir - an invalidation decision for prior runs; - updated documentation when endpoint interpretation changes. +The current documentation PR produced no numerical differences. + +## Dependency security + +- `#69` owns immediate sqlparse remediation and exact current advisory triage. +- `#70` owns migration from unsupported Django 4.2 to Django 5.2 LTS. +- a temporary advisory exception must be exact, evidence-backed, owned and expiring; +- high findings cannot be globally ignored; +- shared/production promotion remains prohibited on an unsupported or untriaged framework baseline. + ## Test data Tests and CI use synthetic or explicitly governed demo data only. Never copy private patient payloads, source documents or direct identifiers into fixtures, logs, snapshots or uploaded artifacts. diff --git a/docs/product/CAPABILITIES_AND_LIMITATIONS.md b/docs/product/CAPABILITIES_AND_LIMITATIONS.md index 93dbb6f..5e04fab 100644 --- a/docs/product/CAPABILITIES_AND_LIMITATIONS.md +++ b/docs/product/CAPABILITIES_AND_LIMITATIONS.md @@ -73,7 +73,10 @@ computation completed - normal product authentication and least-privilege roles are incomplete; - object-level authorization remains inconsistent in some patient-derived web/aggregate paths; - CI is present but not yet a single protected required gate; -- documentation has historical debt and is being reconciled under issue `#14`; +- documentation baseline exists in draft PR `#68` and is not yet canonical on `master`; +- Django 4.2.30 is outside upstream extended support; +- the current dependency audit is red for newly disclosed Django/sqlparse advisories under `#69`; +- durable migration to Django 5.2 LTS is required under `#70`; - production runtime, PostgreSQL, storage, observability, quotas and backup/restore remain governed by M0-S rather than assumed complete. ## Falsification triggers @@ -86,4 +89,5 @@ A current capability claim must be downgraded if: - numerical identity changes without a version/output-diff decision; - an artifact hash cannot be verified; - a current document describes a target model as implemented; -- source-level evidence does not support a patient-derived input. +- source-level evidence does not support a patient-derived input; +- a required security/dependency gate is red or covered only by an expired risk decision. diff --git a/docs/product/CURRENT_STATE.md b/docs/product/CURRENT_STATE.md index df8b7d0..92cdc7d 100644 --- a/docs/product/CURRENT_STATE.md +++ b/docs/product/CURRENT_STATE.md @@ -45,7 +45,9 @@ This page describes the repository state at the SHA above. Later changes require - Immutable research run manifests and version vectors. - Governed model registry and versioned scientific JSON contracts. - Artifact hashing, direct-comparability rules and append-only invalidation semantics. -- Numerical baseline and repository/dependency checks. +- Numerical baseline and repository/dependency-check infrastructure. + +The lock is deterministic but is **not currently security-green**: newly disclosed advisories affect the pinned Django 4.2.30 and sqlparse 0.5.4 graph. Django 4.2 is outside upstream extended support. Immediate triage is tracked in `#69`; migration to Django 5.2 LTS is tracked in `#70`. ## Current registered models @@ -67,7 +69,7 @@ Detailed status and formula mappings are in [Current model registry](../models/R The repository can persist structured longitudinal entities, construct lineage-bound Twin states, execute mechanistic simulations, create immutable run identity and compare hypothetical configurations. It also has residual/calibration, uncertainty, sensitivity, robustness and temporal-diagnostic infrastructure. -It has **not** yet demonstrated a source-verified, leakage-free, identifiable and out-of-sample-valid patient-specific Research Loop. M1 remains blocked until the M0-R documentation, CI and minimum QA baseline closes. +It has **not** yet demonstrated a source-verified, leakage-free, identifiable and out-of-sample-valid patient-specific Research Loop. M1 remains blocked until the M0-R documentation, supported dependency, CI and minimum QA baseline closes. ## Current authentication and authorization boundary @@ -75,9 +77,26 @@ Verified current behavior includes public documentation and public simulator sur Known authorization gaps are tracked in GitHub issue `#8`. Admin access or temporary smoke-user creation must not be interpreted as successful normal-user product authentication. -## Current CI state +## Current CI evidence + +GitHub Actions already exist for dependency/reproducibility checks, documentation deployment, secret scanning, image build/publish, UI screenshots and automatic redeployment. + +On documentation PR `#68`, current evidence at head `b8bc4abc...` is: + +```text +strict MkDocs build: PASS +secret scan: PASS +Ruff and format: PASS +mypy: PASS +repository hygiene: PASS +numerical baseline: PASS / unchanged +Python 3.11 core check/tests: PASS +Python 3.12 core check/tests: PASS +Docker locked build: PASS +dependency audit: FAIL on newly disclosed advisories +``` -GitHub Actions already exist for dependency/reproducibility checks, documentation deployment, secret scanning, image build/publish, UI screenshots and automatic redeployment. They are not yet consolidated into one stable mandatory aggregate gate, and external Actions are not yet uniformly pinned to immutable commit SHAs. That work is tracked in `#13` after documentation paths stabilize. +The workflows are not yet consolidated into one stable mandatory aggregate gate, and external Actions are not yet uniformly pinned to immutable commit SHAs. That work is tracked in `#13` after `#14`, `#69` and `#70` establish canonical paths and a supported dependency baseline. ## Privacy boundary @@ -87,6 +106,8 @@ Private patient documents, direct identifiers, source excerpts and private datas ```text #14 documentation/source-of-truth baseline ++ #69 immediate dependency-advisory remediation +→ #70 supported Django 5.2 LTS baseline → #13 mandatory CI aggregate gate and PR governance → #23 minimum M0-R scientific/privacy invariants → #26 close M0-R diff --git a/docs/product/ROADMAP.md b/docs/product/ROADMAP.md index d5928c5..270dcdc 100644 --- a/docs/product/ROADMAP.md +++ b/docs/product/ROADMAP.md @@ -20,6 +20,8 @@ COMPLETED: CURRENT: #14 documentation/source of truth ++ #69 immediate dependency-advisory remediation +→ #70 supported Django 5.2 LTS baseline → #13 mandatory CI and PR governance → #23 minimum QA invariants → #26 close M0-R @@ -28,11 +30,15 @@ NEXT: #32 → #33 → #34 → #35 → #36 → #18 ``` +## Current supply-chain condition + +The deterministic lock currently pins unsupported Django 4.2.30 and sqlparse 0.5.4. Newly disclosed advisory data makes the dependency audit red. Issue `#69` owns the immediate sqlparse upgrade and exact triage; issue `#70` owns the durable Django 5.2 LTS migration. This condition is independent of documentation PR `#68`, which did not modify dependency files. + ## Milestones | Milestone | Purpose | Exit emphasis | |---|---|---| -| M0-R — Canonical Scientific Baseline | One technical and scientific source of truth | Documentation, protected CI, minimum contracts and privacy invariants | +| M0-R — Canonical Scientific Baseline | One technical and scientific source of truth | Documentation, supported/security-triaged dependencies, protected CI, minimum contracts and privacy invariants | | M0-S — Safe Shared Research Platform | Safe multi-user/external research operation | Product auth, object authorization, RBAC, security, quotas, runtime, API and observability | | M1 — Real-Patient Research Loop v0.1 | First source-verified lineage-bound longitudinal loop | Observation, calibration decision, exposure, toxicity attribution, temporal backtest and immutable report | | M2 — Measurement & Evidence Layer v0.2 | Define what inputs and outputs mean | Evidence registry, biomarker semantics, genomic risk, MRD, imaging, functional outcomes and cards | @@ -43,7 +49,7 @@ NEXT: ## Controlled parallelism -Private read-only source verification for M1 issue `#32` may prepare while M0-R closes. It may not promote a new canonical dataset, calibration, what-if result or project claim before protected documentation/CI/QA gates exist. +Private read-only source verification for M1 issue `#32` may prepare while M0-R closes. It may not promote a new canonical dataset, calibration, what-if result or project claim before protected documentation, dependency, CI and QA gates exist. ## Decision rule From 6e81fe93e693d10b92738568b88d523e78dbcf91 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 00:55:19 +0200 Subject: [PATCH 07/18] docs: refresh current PR validation identity --- docs/product/CURRENT_STATE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/product/CURRENT_STATE.md b/docs/product/CURRENT_STATE.md index 92cdc7d..2279127 100644 --- a/docs/product/CURRENT_STATE.md +++ b/docs/product/CURRENT_STATE.md @@ -81,7 +81,7 @@ Known authorization gaps are tracked in GitHub issue `#8`. Admin access or tempo GitHub Actions already exist for dependency/reproducibility checks, documentation deployment, secret scanning, image build/publish, UI screenshots and automatic redeployment. -On documentation PR `#68`, current evidence at head `b8bc4abc...` is: +On documentation PR `#68`, current evidence at head `3bc54788...` is: ```text strict MkDocs build: PASS From 4c2f5f02beb06472db7bd9aa6cb01ac2d9cde794 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 00:55:52 +0200 Subject: [PATCH 08/18] docs: align testing evidence with current PR head --- docs/operations/TESTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/operations/TESTING.md b/docs/operations/TESTING.md index 6955895..4629030 100644 --- a/docs/operations/TESTING.md +++ b/docs/operations/TESTING.md @@ -49,7 +49,7 @@ Documentation PR `#68` added strict MkDocs validation on pull requests without d ## Current recorded PR evidence -At PR `#68` head `b8bc4abc...`: +At PR `#68` head `3bc54788...`: ```text strict MkDocs: PASS From 085154e7d4aab3b09075e9161cb414c9bea56337 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:22:50 +0200 Subject: [PATCH 09/18] chore(docs): align dependency triage before branch synchronization --- docs/operations/DEPENDENCY_AUDIT_TRIAGE.md | 97 ++++++++++++---------- 1 file changed, 53 insertions(+), 44 deletions(-) diff --git a/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md b/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md index 17598a5..7a2bf6e 100644 --- a/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md +++ b/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md @@ -9,73 +9,82 @@ last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 # Dependency audit triage -## Current status — 24 August 2026 +## Issue #70 migration candidate ```text -LOCK_DETERMINISTIC = true -DEPENDENCY_AUDIT_GREEN = false -UNTRIAGED_HIGH_FINDINGS = true -SHARED_OR_PRODUCTION_PROMOTION = prohibited +DJANGO_TARGET = 5.2.17 LTS +SQLPARSE = 0.6.0 +DJANGO_ADVISORY_EXCEPTIONS = zero +TEST_TOOL_EXCEPTION = GHSA-6w46-j5rx-g56g only +SHARED_OR_PRODUCTION_PROMOTION = pending full #70 validation ``` -The canonical lock was not changed by documentation PR `#68`. Current advisory data nevertheless makes the repository-native audit fail. This is a newly current supply-chain condition. +This branch migrates from unsupported Django 4.2.30 to Django 5.2.17 LTS and removes all Django 4.2 advisory exceptions from `scripts/audit_dependencies.py`. -## Current findings +The candidate is not canonical until its dedicated pull request passes the full compatibility, migration, authorization, numerical and container evidence required by #70. -### Django - -| Package | Advisory | Severity | Affected component | Current decision | -|---|---|---:|---|---| -| Django 4.2.30 | `PYSEC-2026-3717` / `CVE-2026-15830` | medium | GeoDjango `GEOSGeometry` and spatial geometry parsing | Applicability must be reproduced. Django 4.2 is unsupported; migrate under `#70`. | - -Initial repository code search found no direct `django.contrib.gis`, `GEOSGeometry` or GIS-field use. This is not sufficient for a permanent exception. A clean-checkout scan and runtime/settings review are required. - -Django 4.2.30 reached end of extended support on 7 April 2026. The durable target is a current Django 5.2 LTS security patch under issue `#70`. +## Resolved immediate findings ### sqlparse -| Package | Advisory | Severity | Patched version | Current decision | -|---|---|---:|---:|---| -| sqlparse 0.5.4 | `PYSEC-2026-3696` / `CVE-2026-59894` | moderate | 0.6.0 | Upgrade required | -| sqlparse 0.5.4 | `PYSEC-2026-3697` / `CVE-2026-71491` | high | 0.6.0 | Upgrade required | -| sqlparse 0.5.4 | `PYSEC-2026-3698` / `CVE-2026-59893` | high | 0.6.0 | Upgrade required | -| sqlparse 0.5.4 | `PYSEC-2026-3699` / `CVE-2026-54284` | high | 0.6.0 | Upgrade required | +Issue #69 upgraded the direct and locked security floor to `sqlparse 0.6.0`, addressing: -Repository search finds no direct sqlparse call outside dependency/lock declarations. Django and tooling may still invoke parsing paths; absence of direct calls is not a risk-acceptance basis. Upgrade and regenerate the lock under `#69`. +```text +PYSEC-2026-3696 / CVE-2026-59894 +PYSEC-2026-3697 / CVE-2026-71491 +PYSEC-2026-3698 / CVE-2026-59893 +PYSEC-2026-3699 / CVE-2026-54284 +``` -## Temporary Django exception policy +### Django -A temporary exact allowlist for `PYSEC-2026-3717` is allowed only when: +The previous candidate used an exact temporary exception for: ```text -no GIS import/app/model/form/exposed parsing path is proven -owner and rationale are recorded -expiry <= 2026-09-30 -issue #70 remains open and P0 -shared/production promotion remains prohibited +PYSEC-2026-3717 / CVE-2026-15830 ``` -No blanket ignore is permitted. The exception must be removed when `#70` merges. - -## Previous 13 August triage +because the affected GeoDjango path was not found in repository code. That exception was never a supported-framework claim and carried an expiry of 2026-09-30. -The earlier locked baseline removed 155 of 161 broad-environment findings and narrowly triaged six then-current Django advisories plus one development-only pytest advisory. Those decisions remain historical evidence but do not cover newly published advisory IDs. +The #70 candidate removes the exception entirely by moving to Django 5.2.17 LTS, the patched supported line identified for the advisory at implementation time. -Previously triaged exact IDs: +## Remaining exact development-tool decision ```text -GHSA-923m-gv2p-w5qp -GHSA-h7pc-vwp9-298g -GHSA-8cjm-8mp7-r2xf -GHSA-3h9f-r86x-qvjx -GHSA-crhf-3pfg-w68w -GHSA-8qcx-xf44-272x GHSA-6w46-j5rx-g56g ``` +This pytest temporary-directory advisory requires a malicious local user on a shared UNIX host. The compatible `pytest-playwright==0.7.1` release still constrains pytest below the patched major line. Tests run on isolated CI runners or single-user development machines. This exception remains exact and must be removed when the toolchain supports the patched pytest line. + +## Required #70 evidence + +```text +Django 5.2.17 and sqlparse 0.6.0 in pyproject and uv.lock +uv lock --check +pip-audit through scripts.audit_dependencies +Django check +synthetic manage.py check --deploy +migration drift +migration graph and disposable forward evidence +Python 3.11 and 3.12 full core suites +authentication and authorization regressions +numerical baseline unchanged +run/artifact/comparability contracts +optional chemistry environment +strict MkDocs +secret and PHI safety +Docker locked build +``` + +## Historical decisions + +Django 4.2.30 was the final 4.2 release and reached end of extended support on 7 April 2026. It is not an acceptable durable M0-R framework baseline. + +The earlier exact Django exceptions remain visible in Git history and issue #69 evidence; they are deliberately absent from the #70 audit script. + ## Required record fields -Every exception or fix records: +Every future exception or fix records: ```text advisory ID @@ -92,7 +101,7 @@ fix issue/PR/commit ## Current execution ```text -#69 sqlparse upgrade and immediate exact advisory decision -→ #70 Django 5.2 LTS migration and removal of Django 4.2 exceptions +#69 immediate sqlparse/advisory remediation +→ #70 Django 5.2 LTS migration candidate → #13 protected aggregate CI ``` From e50517fb61f93d4c14a7cf7aa202a9c8c1aa8721 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:24:24 +0200 Subject: [PATCH 10/18] docs: align README with the supported Django baseline --- README.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index d11fa8d..c440fca 100644 --- a/README.md +++ b/README.md @@ -30,21 +30,19 @@ It does **not** choose treatment, issue patient-specific dose or schedule instru ```text repository = andreazedda/bmyCure4MM branch = master -verified_head = bf097810b337dc6b766cda04497005670cd96513 +verified_head = a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 ``` Current M0-R path: ```text #14 documentation/source of truth -+ #69 immediate dependency-advisory remediation -→ #70 supported Django 5.2 LTS baseline → #13 mandatory CI aggregate gate and PR governance → #23 minimum scientific/privacy invariants → #26 close M0-R ``` -The current lock remains deterministic, but the security audit now detects newly disclosed advisories in Django 4.2.30 and sqlparse 0.5.4. The durable Django migration is tracked in `#70`; no red dependency gate may be relabelled green through blanket suppression. +The canonical dependency baseline now uses Django 5.2.17 LTS and sqlparse 0.6.0. Dependency audit, Django compatibility/deploy checks, disposable migrations, Python 3.11/3.12 suites, numerical identity, Secret Scan, Docker build and the protected `required` gate passed in PRs #71 and #72. ## Current capabilities @@ -76,7 +74,7 @@ uv run python manage.py check uv run python manage.py runserver ``` -Omit `--extra chemistry` for core work that does not require RDKit. Dependency operations are documented in [Dependency Operations](docs/operations/DEPENDENCIES.md). The current Django 4.2 lock is unsupported and must not be promoted to a shared/production baseline before issues `#69` and `#70` are resolved. +Omit `--extra chemistry` for core work that does not require RDKit. Dependency operations are documented in [Dependency Operations](docs/operations/DEPENDENCIES.md). ## Verification and safety @@ -95,7 +93,7 @@ uv run python -m scripts.audit_dependencies bash scripts/pre_push_research_safety_check.sh ``` -Current test boundaries, the active dependency blocker and CI limitations are documented in [Testing](docs/operations/TESTING.md) and [Dependency Audit Triage](docs/operations/DEPENDENCY_AUDIT_TRIAGE.md). +Current test boundaries and CI limitations are documented in [Testing](docs/operations/TESTING.md) and [Dependency Audit Triage](docs/operations/DEPENDENCY_AUDIT_TRIAGE.md). ## Privacy From 6bdfc25e49bcd19b9f8f2b125f364d8f58f0e1a9 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:24:47 +0200 Subject: [PATCH 11/18] docs: align current state with merged dependency and Django migrations --- docs/product/CURRENT_STATE.md | 36 +++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/product/CURRENT_STATE.md b/docs/product/CURRENT_STATE.md index 2279127..5a62d54 100644 --- a/docs/product/CURRENT_STATE.md +++ b/docs/product/CURRENT_STATE.md @@ -4,7 +4,7 @@ status: CURRENT_VERIFIED owner: Andrea Zedda audience: all contributors last_verified_at: 2026-08-24 -last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +last_verified_git_sha: a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 source_of_truth: code, registries, schemas, migrations and current validation evidence --- @@ -15,7 +15,7 @@ source_of_truth: code, registries, schemas, migrations and current validation ev ```yaml repository: andreazedda/bmyCure4MM branch: master -verified_head: bf097810b337dc6b766cda04497005670cd96513 +verified_head: a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 intended_use_level: E1_research_prototype clinical_decision_support: false patient_specific_prediction_validated: false @@ -41,13 +41,13 @@ This page describes the repository state at the SHA above. Later changes require - Structured Research Dataset Contract v0.1 and generic idempotent importer. - Content-addressed computational-input and Twin-lineage contracts. - Canonical E1 intended use and non-prescriptive model-output language. -- Deterministic `uv.lock` dependency graph and supported Python 3.11/3.12 tooling. +- Deterministic `uv.lock` dependency graph using Django 5.2.17 LTS, sqlparse 0.6.0 and supported Python 3.11/3.12 tooling. - Immutable research run manifests and version vectors. - Governed model registry and versioned scientific JSON contracts. - Artifact hashing, direct-comparability rules and append-only invalidation semantics. -- Numerical baseline and repository/dependency-check infrastructure. - -The lock is deterministic but is **not currently security-green**: newly disclosed advisories affect the pinned Django 4.2.30 and sqlparse 0.5.4 graph. Django 4.2 is outside upstream extended support. Immediate triage is tracked in `#69`; migration to Django 5.2 LTS is tracked in `#70`. +- Numerical baseline, repository hygiene and dependency-audit infrastructure. +- Synthetic Django deployment checks, migration-drift checks and disposable migration evidence. +- A stable protected `required` result over locked quality, Python matrix and Docker build. ## Current registered models @@ -69,7 +69,7 @@ Detailed status and formula mappings are in [Current model registry](../models/R The repository can persist structured longitudinal entities, construct lineage-bound Twin states, execute mechanistic simulations, create immutable run identity and compare hypothetical configurations. It also has residual/calibration, uncertainty, sensitivity, robustness and temporal-diagnostic infrastructure. -It has **not** yet demonstrated a source-verified, leakage-free, identifiable and out-of-sample-valid patient-specific Research Loop. M1 remains blocked until the M0-R documentation, supported dependency, CI and minimum QA baseline closes. +It has **not** yet demonstrated a source-verified, leakage-free, identifiable and out-of-sample-valid patient-specific Research Loop. M1 remains blocked until the M0-R documentation, final CI-governance and minimum QA baseline closes. ## Current authentication and authorization boundary @@ -79,24 +79,26 @@ Known authorization gaps are tracked in GitHub issue `#8`. Admin access or tempo ## Current CI evidence -GitHub Actions already exist for dependency/reproducibility checks, documentation deployment, secret scanning, image build/publish, UI screenshots and automatic redeployment. +GitHub Actions cover dependency/reproducibility checks, documentation, secret scanning, image workflows, Django 5.2 compatibility, UI screenshots and redeployment. -On documentation PR `#68`, current evidence at head `3bc54788...` is: +Merged PRs `#71` and `#72` established: ```text -strict MkDocs build: PASS -secret scan: PASS -Ruff and format: PASS -mypy: PASS -repository hygiene: PASS +Django 5.2.17 LTS and sqlparse 0.6.0 identity: PASS +dependency audit: PASS +ordinary and synthetic deploy checks: PASS +migration drift and disposable migrations: PASS +Ruff, format, mypy and repository hygiene: PASS numerical baseline: PASS / unchanged Python 3.11 core check/tests: PASS Python 3.12 core check/tests: PASS +authorization-sensitive compatibility subset: PASS +Secret Scan: PASS Docker locked build: PASS -dependency audit: FAIL on newly disclosed advisories +protected required result: PASS ``` -The workflows are not yet consolidated into one stable mandatory aggregate gate, and external Actions are not yet uniformly pinned to immutable commit SHAs. That work is tracked in `#13` after `#14`, `#69` and `#70` establish canonical paths and a supported dependency baseline. +The workflows are not yet fully rationalized: external Actions remain tag-pinned rather than immutable-SHA-pinned, the research safety script lacks verified base/head PR mode, and PR templates/CODEOWNERS/final branch-protection evidence remain under issue `#13`. ## Privacy boundary @@ -106,8 +108,6 @@ Private patient documents, direct identifiers, source excerpts and private datas ```text #14 documentation/source-of-truth baseline -+ #69 immediate dependency-advisory remediation -→ #70 supported Django 5.2 LTS baseline → #13 mandatory CI aggregate gate and PR governance → #23 minimum M0-R scientific/privacy invariants → #26 close M0-R From 4da83231a5f93ebc935d7d47cb6ddef6ac291798 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:25:10 +0200 Subject: [PATCH 12/18] docs: align roadmap after dependency and Django migrations --- docs/product/ROADMAP.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/product/ROADMAP.md b/docs/product/ROADMAP.md index 270dcdc..04b4455 100644 --- a/docs/product/ROADMAP.md +++ b/docs/product/ROADMAP.md @@ -4,7 +4,7 @@ status: CANONICAL_STRATEGY owner: Andrea Zedda audience: maintainers, contributors and collaborators last_verified_at: 2026-08-24 -last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +last_verified_git_sha: a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 canonical_execution_tracker: https://github.com/andreazedda/bmyCure4MM/issues/31 --- @@ -16,12 +16,10 @@ GitHub issues and milestones are the execution control plane. This page explains ```text COMPLETED: -#11 → #12 → #15 → #19 +#11 → #12 → #15 → #19 → #69 → #70 CURRENT: #14 documentation/source of truth -+ #69 immediate dependency-advisory remediation -→ #70 supported Django 5.2 LTS baseline → #13 mandatory CI and PR governance → #23 minimum QA invariants → #26 close M0-R @@ -30,9 +28,9 @@ NEXT: #32 → #33 → #34 → #35 → #36 → #18 ``` -## Current supply-chain condition +## Current dependency and framework baseline -The deterministic lock currently pins unsupported Django 4.2.30 and sqlparse 0.5.4. Newly disclosed advisory data makes the dependency audit red. Issue `#69` owns the immediate sqlparse upgrade and exact triage; issue `#70` owns the durable Django 5.2 LTS migration. This condition is independent of documentation PR `#68`, which did not modify dependency files. +The deterministic lock now uses Django 5.2.17 LTS and sqlparse 0.6.0. Merged PRs `#71` and `#72` removed the current sqlparse findings and all Django 4.2 advisory exceptions while preserving Python 3.11/3.12, numerical and container evidence. The remaining exact development-tool advisory decision is documented in the dependency triage record. ## Milestones @@ -49,7 +47,7 @@ The deterministic lock currently pins unsupported Django 4.2.30 and sqlparse 0.5 ## Controlled parallelism -Private read-only source verification for M1 issue `#32` may prepare while M0-R closes. It may not promote a new canonical dataset, calibration, what-if result or project claim before protected documentation, dependency, CI and QA gates exist. +Private read-only source verification for M1 issue `#32` may prepare while M0-R closes. It may not promote a new canonical dataset, calibration, what-if result or project claim before protected documentation, CI and QA gates exist. ## Decision rule From cacc20c74ada2efc2ee8792b99027743d0a0b9bd Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:25:38 +0200 Subject: [PATCH 13/18] docs: align capabilities with the supported framework baseline --- docs/product/CAPABILITIES_AND_LIMITATIONS.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/product/CAPABILITIES_AND_LIMITATIONS.md b/docs/product/CAPABILITIES_AND_LIMITATIONS.md index 5e04fab..a4634ea 100644 --- a/docs/product/CAPABILITIES_AND_LIMITATIONS.md +++ b/docs/product/CAPABILITIES_AND_LIMITATIONS.md @@ -4,7 +4,7 @@ status: CURRENT_VERIFIED owner: Andrea Zedda audience: users, reviewers and contributors last_verified_at: 2026-08-24 -last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +last_verified_git_sha: a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 --- # Capabilities and limitations @@ -72,13 +72,12 @@ computation completed - normal product authentication and least-privilege roles are incomplete; - object-level authorization remains inconsistent in some patient-derived web/aggregate paths; -- CI is present but not yet a single protected required gate; +- a protected `required` dependency/reproducibility gate exists, but the complete #13 workflow rationalization, immutable Action pinning, exact base/head safety mode, PR templates and CODEOWNERS remain open; - documentation baseline exists in draft PR `#68` and is not yet canonical on `master`; -- Django 4.2.30 is outside upstream extended support; -- the current dependency audit is red for newly disclosed Django/sqlparse advisories under `#69`; -- durable migration to Django 5.2 LTS is required under `#70`; - production runtime, PostgreSQL, storage, observability, quotas and backup/restore remain governed by M0-S rather than assumed complete. +The supported dependency baseline is now Django 5.2.17 LTS with sqlparse 0.6.0. Merged PRs `#71` and `#72` passed the repository dependency audit, Python 3.11/3.12 suites, numerical baseline, Secret Scan and Docker build. This does not by itself establish production security. + ## Falsification triggers A current capability claim must be downgraded if: From ab9b8cb5f96d61c8ca5e3fd44f1cf06aec2fd3d7 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:26:06 +0200 Subject: [PATCH 14/18] docs(deps): mark the supported dependency baseline canonical --- docs/operations/DEPENDENCY_AUDIT_TRIAGE.md | 80 ++++++++++------------ 1 file changed, 35 insertions(+), 45 deletions(-) diff --git a/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md b/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md index 7a2bf6e..ea81056 100644 --- a/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md +++ b/docs/operations/DEPENDENCY_AUDIT_TRIAGE.md @@ -1,33 +1,48 @@ --- title: Dependency Audit Triage -status: CURRENT_PARTIAL +status: CURRENT_VERIFIED owner: Andrea Zedda audience: maintainers and security reviewers last_verified_at: 2026-08-24 -last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +last_verified_git_sha: a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 --- # Dependency audit triage -## Issue #70 migration candidate +## Canonical status ```text -DJANGO_TARGET = 5.2.17 LTS +DJANGO = 5.2.17 LTS SQLPARSE = 0.6.0 DJANGO_ADVISORY_EXCEPTIONS = zero +DEPENDENCY_AUDIT = pass TEST_TOOL_EXCEPTION = GHSA-6w46-j5rx-g56g only -SHARED_OR_PRODUCTION_PROMOTION = pending full #70 validation ``` -This branch migrates from unsupported Django 4.2.30 to Django 5.2.17 LTS and removes all Django 4.2 advisory exceptions from `scripts/audit_dependencies.py`. +Merged PR `#71` upgraded sqlparse and established exact temporary Django 4.2 triage. Merged PR `#72` then migrated to Django 5.2.17 LTS and removed every Django 4.2 advisory exception. -The candidate is not canonical until its dedicated pull request passes the full compatibility, migration, authorization, numerical and container evidence required by #70. +The canonical dependency evidence includes: -## Resolved immediate findings +```text +uv lock --check: PASS +frozen chemistry sync: PASS +pip-audit through scripts.audit_dependencies: PASS +ordinary Django check: PASS +synthetic manage.py check --deploy: PASS +migration drift: zero +disposable migrations: PASS +Python 3.11 core check/tests: PASS +Python 3.12 core check/tests: PASS +authorization-sensitive compatibility subset: PASS +numerical baseline: PASS / unchanged +Secret Scan: PASS +Docker locked build: PASS +protected required result: PASS +``` -### sqlparse +## Resolved sqlparse findings -Issue #69 upgraded the direct and locked security floor to `sqlparse 0.6.0`, addressing: +The direct and locked version is `sqlparse 0.6.0`, addressing: ```text PYSEC-2026-3696 / CVE-2026-59894 @@ -36,17 +51,15 @@ PYSEC-2026-3698 / CVE-2026-59893 PYSEC-2026-3699 / CVE-2026-54284 ``` -### Django +## Resolved Django finding and support state -The previous candidate used an exact temporary exception for: +Django 4.2.30 was the final 4.2 release and was outside upstream extended support. The former exact temporary exception for: ```text PYSEC-2026-3717 / CVE-2026-15830 ``` -because the affected GeoDjango path was not found in repository code. That exception was never a supported-framework claim and carried an expiry of 2026-09-30. - -The #70 candidate removes the exception entirely by moving to Django 5.2.17 LTS, the patched supported line identified for the advisory at implementation time. +was removed when Django 5.2.17 LTS became canonical. The audit wrapper contains no Django advisory exception. ## Remaining exact development-tool decision @@ -54,35 +67,11 @@ The #70 candidate removes the exception entirely by moving to Django 5.2.17 LTS, GHSA-6w46-j5rx-g56g ``` -This pytest temporary-directory advisory requires a malicious local user on a shared UNIX host. The compatible `pytest-playwright==0.7.1` release still constrains pytest below the patched major line. Tests run on isolated CI runners or single-user development machines. This exception remains exact and must be removed when the toolchain supports the patched pytest line. - -## Required #70 evidence - -```text -Django 5.2.17 and sqlparse 0.6.0 in pyproject and uv.lock -uv lock --check -pip-audit through scripts.audit_dependencies -Django check -synthetic manage.py check --deploy -migration drift -migration graph and disposable forward evidence -Python 3.11 and 3.12 full core suites -authentication and authorization regressions -numerical baseline unchanged -run/artifact/comparability contracts -optional chemistry environment -strict MkDocs -secret and PHI safety -Docker locked build -``` - -## Historical decisions - -Django 4.2.30 was the final 4.2 release and reached end of extended support on 7 April 2026. It is not an acceptable durable M0-R framework baseline. +This pytest temporary-directory advisory requires a malicious local user on a shared UNIX host. The compatible `pytest-playwright==0.7.1` release still constrains pytest below the patched major line. Tests run on isolated CI runners or single-user development machines. -The earlier exact Django exceptions remain visible in Git history and issue #69 evidence; they are deliberately absent from the #70 audit script. +This is a narrow development-only decision. It does not authorize private data in test artifacts, shared hostile-host execution or blanket advisory suppression. Remove it when the browser-test toolchain supports the patched pytest line. -## Required record fields +## Required record fields for future findings Every future exception or fix records: @@ -101,7 +90,8 @@ fix issue/PR/commit ## Current execution ```text -#69 immediate sqlparse/advisory remediation -→ #70 Django 5.2 LTS migration candidate -→ #13 protected aggregate CI +supported dependency baseline complete +→ #14 documentation source of truth +→ #13 protected aggregate CI hardening +→ #23 minimum scientific QA ``` From 18573dd1528180f313edb3e62911204584dbf21b Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:26:34 +0200 Subject: [PATCH 15/18] docs: align testing evidence with merged dependency and framework baselines --- docs/operations/TESTING.md | 50 ++++++++++++++++++++++++-------------- 1 file changed, 32 insertions(+), 18 deletions(-) diff --git a/docs/operations/TESTING.md b/docs/operations/TESTING.md index 4629030..6127904 100644 --- a/docs/operations/TESTING.md +++ b/docs/operations/TESTING.md @@ -4,7 +4,7 @@ status: CURRENT_PARTIAL owner: Andrea Zedda audience: contributors and reviewers last_verified_at: 2026-08-24 -last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +last_verified_git_sha: a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 --- # Testing and verification @@ -43,29 +43,43 @@ Use the dependency groups documented in `DEPENDENCIES.md`; core-only and optiona ## Current GitHub Actions -Current workflows cover dependency/reproducibility checks, docs deployment, secret scanning, image workflows, UI screenshots and redeployment. Issue `#13` will rationalize them into one stable protected aggregate gate and add exact pull-request safety behavior. +Current workflows cover dependency/reproducibility checks, strict documentation validation, secret scanning, Django 5.2 compatibility, image workflows, UI screenshots and redeployment. -Documentation PR `#68` added strict MkDocs validation on pull requests without deploying PR content. This is a bounded prerequisite for `#13`, not the final aggregate CI design. +Merged PR `#71` added the stable protected `required` result over locked quality, the Python 3.11/3.12 core matrix and Docker build. This repaired a verified mismatch between branch protection and workflow status names. -## Current recorded PR evidence +Issue `#13` remains open because the final CI contract still requires: -At PR `#68` head `3bc54788...`: +- immutable commit-SHA pinning for external Actions; +- exact base/head pull-request safety mode; +- documentation status/path checks; +- PR and issue templates; +- CODEOWNERS; +- privacy-safe artifact retention; +- final workflow rationalization and branch-protection re-read. + +## Current recorded baseline evidence + +Merged PRs `#71` and `#72` recorded: ```text -strict MkDocs: PASS -secret scan: PASS -Ruff: PASS -format: PASS -mypy: PASS +Django 5.2.17 and sqlparse 0.6.0 identity: PASS +uv lock --check and frozen sync: PASS +dependency audit: PASS +Ruff, format and mypy: PASS repository hygiene: PASS -numerical baseline: PASS / unchanged +ordinary and synthetic deploy checks: PASS +migration drift: zero +disposable migrations: PASS Python 3.11 core check/tests: PASS Python 3.12 core check/tests: PASS +authorization-sensitive compatibility subset: PASS +numerical baseline: PASS / unchanged +Secret Scan: PASS Docker locked build: PASS -dependency audit: FAIL +protected required result: PASS ``` -The audit failure is not caused by the documentation diff. It identifies newly disclosed advisories in the unchanged canonical dependency graph and is governed by `#69` and `#70`. +Documentation PR `#68` separately established strict MkDocs validation on pull requests. It must be revalidated on the current master baseline before merge. ## Research safety gate @@ -80,15 +94,15 @@ The numerical baseline detects unreviewed output drift. A baseline update requir - an invalidation decision for prior runs; - updated documentation when endpoint interpretation changes. -The current documentation PR produced no numerical differences. +The dependency and Django 5.2 migrations produced no numerical differences. ## Dependency security -- `#69` owns immediate sqlparse remediation and exact current advisory triage. -- `#70` owns migration from unsupported Django 4.2 to Django 5.2 LTS. -- a temporary advisory exception must be exact, evidence-backed, owned and expiring; +- canonical runtime: Django 5.2.17 LTS and sqlparse 0.6.0; +- no Django advisory exception remains in the audit wrapper; +- one exact development-only pytest advisory decision remains documented; - high findings cannot be globally ignored; -- shared/production promotion remains prohibited on an unsupported or untriaged framework baseline. +- a green dependency audit does not by itself establish production security. ## Test data From 3f54d20d22f9a4bee44bbc7e5d5b5012cf00aa96 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:27:01 +0200 Subject: [PATCH 16/18] docs(security): align policy with the supported dependency baseline --- SECURITY.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/SECURITY.md b/SECURITY.md index e3ada23..bfff335 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -9,10 +9,12 @@ production_security_certified: false clinical_use_supported: false product_auth_model_complete: false object_authorization_complete: false -dependency_security_green: false +dependency_security_green: true ``` -No current version is represented as production-certified or suitable for clinical deployment. Security fixes are handled on the active `master` research line, subject to maintainer capacity and explicit verification. +The canonical research dependency baseline uses Django 5.2.17 LTS and sqlparse 0.6.0. Dependency audit, synthetic deployment checks, migration evidence, Python 3.11/3.12 suites, numerical identity, Secret Scan, Docker build and the protected `required` result passed in merged PRs `#71` and `#72`. + +This does not make the application production-certified or suitable for clinical deployment. Production security also depends on authentication, object authorization, settings, TLS/proxy behavior, storage, backup/restore, quotas, observability and incident response. ## Reporting a vulnerability @@ -38,28 +40,22 @@ Acknowledgment and remediation timing are targets, not a contractual SLA. Disclo ## Verified current controls +- supported Django 5.2 LTS dependency baseline and deterministic `uv.lock`; +- repository-native dependency audit with exact documented development-tool triage only; - environment-based secret-key configuration and weak/default-key rejection; - Django password hashing and CSRF protections; - selected authenticated views and global DRF `IsAuthenticated` default; - secret-scanning workflow; -- repository hygiene and dependency-audit tooling; -- research safety checks for staged sensitive files and PHI-like markers; +- repository hygiene and research safety tooling; - ignored `local_private/` boundary for private research material; -- temporary M0 smoke identities are constrained to non-staff, non-superuser accounts with no groups or direct permissions. +- temporary M0 smoke identities constrained to non-staff, non-superuser accounts with no groups or direct permissions; +- synthetic CI deployment check and disposable migration evidence; +- protected `required` dependency/reproducibility result. These controls do not prove complete application security. ## Known open security gaps -### Dependency and framework support - -The deterministic lock currently pins unsupported Django 4.2.30 and sqlparse 0.5.4. Newly disclosed advisories make the dependency audit red. - -- issue `#69` tracks immediate sqlparse upgrade and exact advisory triage; -- issue `#70` tracks durable migration to Django 5.2 LTS; -- shared/production promotion is prohibited while the framework baseline is unsupported or security findings remain untriaged; -- blanket audit suppression is prohibited. - ### Product authentication Documentation and selected simulator surfaces are public, while clinic/research/simulator-management/API surfaces are protected. `LOGIN_URL` currently targets the Django admin login and there is no reviewed normal-user product-login and role contract. @@ -80,7 +76,11 @@ GitHub issue `#10` tracks request throttling, job quotas, concurrency, timeout, ### CI and branch protection -GitHub issue `#13` tracks consolidation of existing workflows into one mandatory protected scientific gate, exact-PR safety behavior, immutable Action pinning and PR governance. +A protected `required` status exists. GitHub issue `#13` still tracks immutable Action pinning, exact base/head PR safety mode, documentation contracts, PR templates, CODEOWNERS, privacy-safe artifact retention, final workflow rationalization and branch-protection re-read. + +### Remaining development-tool advisory decision + +The dependency audit contains one exact development-only pytest advisory decision, documented in `docs/operations/DEPENDENCY_AUDIT_TRIAGE.md`. It is not a runtime-framework exception and must be removed when the browser-test toolchain supports the patched pytest line. ## Data-safety rules From 7ac9a45fa454ce21b3104ee3b0fe52e6cb85f9e9 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:27:21 +0200 Subject: [PATCH 17/18] docs: record merged dependency and Django baseline work --- CHANGELOG.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c4edccd..cd9af15 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,7 @@ All current release claims are governed by [`docs/governance/CLAIMS_POLICY.md`]( - Separated current verified architecture from the target virtual-laboratory architecture. - Added a current model-status catalog with formulas, units, examples, code mappings and falsification boundaries. - Added source-of-truth and documentation-governance policies plus a corpus inventory. -- Reconciled GitHub roadmap/tracker state and Notion project state at canonical `master` `bf097810...`. +- Reconciled GitHub roadmap/tracker state and Notion project state against canonical `master` `a33418fb...`. - Added strict MkDocs validation to documentation pull requests without deploying PR content. ### Scientific identity and reproducibility @@ -20,7 +20,10 @@ All current release claims are governed by [`docs/governance/CLAIMS_POLICY.md`]( - Added immutable research run manifests and a governed model registry. - Added typed scientific payload contracts, artifact integrity checks, direct-comparability rules and append-only invalidation. - Locked the dependency graph with `uv.lock`, Python 3.11/3.12 support and numerical/repository/dependency checks. -- Recorded newly disclosed Django/sqlparse advisories as a blocking current condition under issues `#69` and `#70`; no dependency fix or audit bypass is claimed by this documentation change. +- Upgraded sqlparse to 0.6.0 and recorded exact dependency-audit decisions through merged PR `#71`. +- Migrated the unsupported Django 4.2 baseline to Django 5.2.17 LTS through merged PR `#72`. +- Removed all Django 4.2 advisory exceptions; ordinary/deploy checks, disposable migrations, authorization-sensitive regressions, dependency audit, numerical baseline and Docker evidence passed. +- Added a stable protected `required` dependency/reproducibility result after detecting a mismatch between branch protection and workflow status names. ### Intended use and language From affdd6f717ba85b725b61438bfd762c3c1054b55 Mon Sep 17 00:00:00 2001 From: Andrea Date: Mon, 24 Aug 2026 01:27:43 +0200 Subject: [PATCH 18/18] docs: update source-of-truth baseline after supported dependency migration --- docs/governance/SOURCE_OF_TRUTH.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/governance/SOURCE_OF_TRUTH.md b/docs/governance/SOURCE_OF_TRUTH.md index 5b8dc7e..0e6d130 100644 --- a/docs/governance/SOURCE_OF_TRUTH.md +++ b/docs/governance/SOURCE_OF_TRUTH.md @@ -4,7 +4,7 @@ status: CANONICAL owner: Andrea Zedda audience: all contributors last_verified_at: 2026-08-24 -last_verified_git_sha: bf097810b337dc6b766cda04497005670cd96513 +last_verified_git_sha: a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 --- # Source of truth @@ -74,5 +74,5 @@ Every major component/model must be marked `CURRENT`, `PARTIAL`, `TARGET`, `HYPO ```text repository = andreazedda/bmyCure4MM branch = master -verified_head = bf097810b337dc6b766cda04497005670cd96513 +verified_head = a33418fb8ae9cb9fd05832dd9bc1cb0778e08533 ```