Repository navigation
Import SBML reaction networks as verified graphs (Track E) - #4
Conversation
Add a from-scratch reader for SBML Level 2/3 core (vh/graph/sbml.py): compartments, species, global and local parameters, function definitions, initial assignments, assignment rules and reactions with MathML kinetic laws. Unsupported constructs (events, rate and algebraic rules, delays, fast reactions, stoichiometryMath, conversion factors, variable compartments, required packages) are refused by name. Documents with a DOCTYPE are refused and MathML compiles to closures over a fixed operator table. Models simulate, solve for steady states on the conservation-reduced system and report reduced eigenvalues. vh/graph/stoichiometry.py does exact rational algebra: canonical conservation laws (RREF of the left nullspace) and positive-mass stoichiometric consistency (Gevorgyan et al. 2008) with exactly checked certificates. SBML graphs carry species, reaction, compartment and moiety nodes; the schema recomputes N from edges and checks stoichiometry, location, exact complete conservation and consistency. Validation (scripts/validate_sbml.py, auto-research report): three CC0 BioModels vendored with SHA-256 provenance agree with libRoadRunner, an independent SBML engine used only as a test referee, to <=6.9e-10 on trajectories, <=1.7e-12 on steady states and <=1.3e-9 on reduced eigenvalues; periods match to ~1e-11. Across BIOMD0000000001-060, 49 models are read and 11 refused by name; trajectories of all 49 agree with the referee (median 1.7e-9). The consistency invariant flags 8 models with exact certificates, including a stoichiometry used as a unit conversion in Chassagnole 2002, as its own notes describe. libroadrunner joins the dev extras as a test-only referee. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CoFhtFtE7wJgvHKAzNB3xD
8f3f193 to
46fc3f9
Compare
orda-aska
left a comment
There was a problem hiding this comment.
REQUEST CHANGES at 46fc3f9. One blocker: the DOCTYPE/entity refusal can be bypassed with UTF-16 input, which falsifies the PR's security claim.
Merge conflict: at the earlier head 8f3f193, the PR conflicted with main (88bc75a1, after #2 and #3) in README.md and auto-research/README.md, both credit/ledger rows. The live head 46fc3f9 is already rebased with both sets of rows kept, the SBML code is byte-identical, and it's MERGEABLE. Please confirm that rebase is intentional. Any further rebase is yours to do; reviewers won't rewrite auto/E-sbml-import.
Blocker
- A UTF-16 document with an internal entity loads and gets expanded.
read_sbmlrefuses DOCTYPE and ENTITY declarations with an ASCII byte scan (b"<!doctype" in data[:4096].lower(),b"<!entity" in data.lower()), then callsET.fromstring. In UTF-16 those markers are interleaved with NUL bytes, so the scan never matches. Repro at46fc3f9:External entities still fail in ElementTree, but internal-entity expansion, and with it nested-entity expansion attacks, gets through.doc = '''<?xml version="1.0" encoding="UTF-16"?> <!DOCTYPE sbml [<!ENTITY x "EXPANDED_VALUE">]> <sbml xmlns="http://www.sbml.org/sbml/level2/version4" level="2" version="4"><model id="m"><listOfCompartments><compartment id="c" name="&x;" size="1"/></listOfCompartments></model></sbml>''' sbml.read_sbml(doc.encode("utf-16")) # LOADED; compartment name == "EXPANDED_VALUE" sbml.read_sbml(doc.replace("UTF-16","UTF-8").encode()) # SBMLError: ... refused (ASCII path works)
- Fix: refuse any input that isn't UTF-8 or ASCII before scanning, or decode before scanning, or parse with DTDs disabled (e.g.
defusedxml). - Add a regression test that feeds a UTF-16 DOCTYPE+entity document and expects
SBMLError.
- Fix: refuse any input that isn't UTF-8 or ASCII before scanning, or decode before scanning, or parse with DTDs disabled (e.g.
Verified
- CI: at
46fc3f9,pytest -qgives 201 passed in a clean Python 3.12 venv, and the JS engine checks pass. Actions run 42061180139 is green. The pre-rebase head gave 158, matching the PR body; the 47 SBML tests are all present. - Maths: I checked the stoichiometry independently on all three CC0 fixtures.
l·N = 0holds exactly,rank(N) + n_laws == n_dyn, numpy/SVD rank agrees with the exact counts, and every positive-mass certificate has masses > 0. Kholodenko's moiety totals are exactly 100/300/300. validate_sbml.py: reproduces the PR table. The Kholodenko period differs from libRoadRunner by 8.9e-12 (claimed ≤ 9e-12) and the repressilator period by 1.8e-12. Conserved-total drift is ≤ 1.1e-13.- Mutations:
- Rejected at load: a dropped referenced species, an unknown species, an unknown compartment, malformed or empty XML, a non-
<sbml>root, and ASCII DOCTYPE/ENTITY. - A flipped stoichiometric coefficient loads, but the schema check catches it with an exact certificate (
stoichiometric-consistency, net production by -2·J0, -3·J1).
- Rejected at load: a dropped referenced species, an unknown species, an unknown compartment, malformed or empty XML, a non-
- Other claims: no
evalon MathML, unsupported constructs are refused by name, no network calls in the SBML tests, and scope matches the Limits section.
Not re-run: the "8 of 49" BioModels survey count, since that corpus isn't in the PR.
The DOCTYPE/entity refusal scanned raw bytes for "<!doctype" and "<!entity". In UTF-16 those markers are interleaved with NUL bytes, so a UTF-16 document with an internal entity passed the scan and ElementTree expanded it (reported in review on #4). read_sbml now pre-parses with expat, which decodes the declared encoding, and refuses on any DOCTYPE or entity declaration. Input expat cannot decode (for example UTF-32), empty input and truncated XML raise SBMLError instead of passing through. New tests cover UTF-16 with and without a byte-order mark in both byte orders, a UTF-16 document without a DOCTYPE that must still load, and unreadable input; six of the seven fail on the previous code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CoFhtFtE7wJgvHKAzNB3xD
|
Thanks @orda-aska, the UTF-16 bypass was real, and I reproduced it in UTF-16, UTF-16-LE and UTF-16-BE. It's fixed in Fix. New tests, six of which fail on the previous code:
Checks: 208 passed locally, and the JS engine checks pass. Rebase: yes, it was intentional. After #2 and #3 merged I rebased my own branch onto Generated by Claude Code |
orda-aska
left a comment
There was a problem hiding this comment.
Approve at 05dd317. My one blocker at 46fc3f9 was UTF-16 input getting past the DOCTYPE and entity refusal. Commit 05dd317 fixes it by refusing DOCTYPE and entity declarations through expat handlers in whatever encoding the document declares, instead of scanning raw bytes, and it adds tests. CI test is green. Merging at the owner's request.
This PR adds Track E's SBML import. It is a from-scratch reader that turns curated BioModels into executable reaction networks and typed graphs. Their structural claims are proven in exact rational arithmetic, and their dynamics are checked against libRoadRunner, an independent SBML engine.
What's added
vh/graph/sbml.pyreads SBML Level 2 (versions 1–5) and Level 3 (versions 1–2) core. It supports:eval.vh/graph/stoichiometry.pydoes the exact algebra withFractions:SBMLModel.to_graph()buildsspecies,reaction,compartmentandmoietynodes.sbml_schema()recomputes the stoichiometry from the edges alone and checks four invariants: stoichiometry, located, conservation and stoichiometric consistency.data/sbml/holds three BioModels whose notes state CC0, vendored unmodified, with their SHA-256 values and PubMed IDs insources.json: Huang & Ferrell 1996, Kholodenko 2000 and the Elowitz & Leibler 2000 repressilator. I left out Markevich 2004 because its file carries an older BioModels copyright notice rather than a CC0 statement.scripts/validate_sbml.py,tests/test_sbml.py(47 tests), the report inauto-research/2026-11-05-E-sbml-import/, doc and roadmap updates, and renderer styles for the new node types.libroadrunneris added to thedevextras as a test-only referee.Validation
The authorities are exact rational algebra and closed-form solutions for the structure, and libRoadRunner 2.11.0 (libSBML + CVODE) for the dynamics. No parameter was fitted.
l·N = 0; rank(N) + laws = number of dynamic species; same moiety space as libRoadRunner. Kholodenko totals are 100, 300 and 300.rtol1e-10: median 1.7e-9, 47 of 49 within 1e-6, and the two calcium oscillators fall below 1e-6 atrtol1e-12.The new invariant catches real inconsistencies. Positive-mass consistency fails, with exact certificates, in 8 of the 49 surveyed models that were read.
cglcex + 65 cpep → 65 cg6p + 65 cpyr. The model's notes say its stoichiometries were adjusted to encode a biomass-to-volume ratio, so a unit conversion is hiding in the stoichiometry. A second certificate runs throughvPGDH, which is written without CO₂.The report is explicit that a flag proves a property of the network as written, not a modelling error.
Checked locally. GitHub Actions is not enabled on this fork, so there is no CI run. I ran:
pytest -q: 158 passed, no skipsnode web/test_engines.mjs: passedscripts/validate_sbml.py --survey: reproducesresults.jsonThe environment was Python 3.13, NumPy 2.5.3, SciPy 1.18.1, libRoadRunner 2.11.0, OpenMM 8.6.1 and PySCF 2.15.0.
Limits
vh.bridgeor patient data.Relationship to other open PRs
auto-research/README.mdandREADME.md, and I'll rebase whichever PR merges second.vh/graph/reactions.pycould reusevh/graph/stoichiometry.py.chemical_species/elementary_reactionnodes with integer coefficients, and this PR usesspecies/reactionnodes with exact rational strings.https://claude.ai/code/session_01CoFhtFtE7wJgvHKAzNB3xD
Generated by Claude Code