From 9e3678801fb1936b6435fe649346ef01f45c9a82 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:39:44 +0000 Subject: [PATCH 1/4] Five ways a claim-graph firewall failed open, and the controls that kill them `tools/claims_check.py` passed. That was the problem: five of its guards could not refuse the thing they existed to refuse. 1. `certifying is True` is an identity test, so the JSON string "true" walked past it. `certifying` must now be a real boolean, and the rule is inverted: a record that claims certification must SHOW exact arithmetic. Float, not-applicable, unrecognised and ambiguous are each refused by name. 2. `is_float_arithmetic` was a bool over two token lists, and an unlisted word ("binary64"), a missing field and a non-string all answered False, which the caller read as nothing to refuse. It is now `classify_arithmetic`, returning one of five classes, and the two unreadable ones are counted in the summary. 3. "An exact token anywhere wins" is defeated by a negation. `engine/rn_engine/BINDING.json` describes RNENG-01 as "no fractions.Fraction, no decimal.Decimal and no interval arithmetic occurs anywhere" -- genuinely float code whose own denial mentions three exact tokens, and the old rule classified it exact. Hence AMBIGUOUS: a string carrying tokens from both vocabularies is prose, not a classification. 4. The carrier override read only `engine/carriers/MANIFEST.json`. The graph names exactly one `carrier_id`, RNENG-01, which lives in BINDING.json, so the override resolved nothing: one lookup, one miss, every run. Both indexes are read now, and a refusal names which file it came from. Activating this naively would have WEAKENED the float firewall, via (3) above. 5. FW-UNCONDITIONAL asked whether ONE of the two status columns held one of two open words. So `NAMED_HYPOTHESIS` -- carried by two premises, and by definition not a discharge -- passed as a discharge, and the register-note column, which exists because it disagrees with the frozen column, was never read. Both columns are read now, against closed vocabularies, and a status word in neither is refused rather than assumed harmless. `RESTATED` and `REFINEMENT` are deliberately undischarged: whether a restatement discharges a premise is a status decision, and this file does not make status decisions. Two more, found while writing the controls: a node's `track` is read by three firewalls as a membership test against a literal set, so an unrecognised value dropped the node out of all three at once and failed nothing; and three lists of firewalls -- declared in graph.json, enforced here, documented in claims/README.md -- had nothing reconciling them, so the prose documented eight where nine were enforced and twenty-four claims over a graph of twenty-six. All three are compared on every run, and a missing claims/README.md is a refusal rather than a skip. `tests/test_claims_firewalls.py` is 34 controls, all CLI-invoked. Each fix was reverted in a copy of the checker outside the tree, one at a time, in a subprocess: sixteen mutants, sixteen killed, none surviving. The matrix is in claims/README.md, including the two arms the first harness did not reach because `if False: ... elif ...` still evaluates the elif. No status moved. The conversion was measured against the committed graph before it was adopted and adds zero refusals to it: the two `certifying: true` records both classify EXACT, nine say not_applicable with certifying false, all five track values are known, and the strict two-column premise rule refuses nothing present. A firewall that now fails closed catches a future edit; it certifies nothing about this one. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01Cz7WZybv8znP64SpPj6sWY --- claims/README.md | 136 +++++++++-- tests/test_claims_firewalls.py | 415 +++++++++++++++++++++++++++++++++ tools/claims_check.py | 269 ++++++++++++++++++--- 3 files changed, 769 insertions(+), 51 deletions(-) create mode 100644 tests/test_claims_firewalls.py diff --git a/claims/README.md b/claims/README.md index 60bf695e6..48b1d658e 100644 --- a/claims/README.md +++ b/claims/README.md @@ -1,9 +1,21 @@ # Claim / premise dependency graph -`graph.json` is the program's claim structure as data: 24 claims, 13 named -premises, 8 firewalls. `tools/claims_check.py` turns the firewalls into -assertions; `tests/test_claims.py` proves the checker actually rejects each -violation it is supposed to reject. +`graph.json` is the program's claim structure as data: 26 claims, 13 named +premises, 9 firewalls. `tools/claims_check.py` turns the firewalls into +assertions; `tests/test_claims.py` and `tests/test_claims_firewalls.py` prove the +checker actually rejects each violation it is supposed to reject. + +Those counts are checked rather than asserted. `tools/claims_check.py` reconciles +three lists that had no reason to agree and did not: the firewalls **declared** +in `graph.json`, the firewalls it **enforces** (`ENFORCED_FIREWALLS`), and the +firewalls named in this file. A run where any of the three disagrees is refused, +and so is a run where this file's claim count or firewall count has drifted from +the graph. All three had drifted: this file documented eight firewalls over nine +enforced, and twenty-four claims over a graph of twenty-six. An undocumented +firewall is not a safeguard a reader can check, and a stale count is how a reader +learns to stop trusting the rest. A **missing** `claims/README.md` is a refusal +too, not a skip: guarding the reconciliation with "if the file exists" would have +made deleting this file the way to switch it off. This encodes what the Drive sources state in prose. It asserts no mathematics of its own, and it never changes a status. @@ -26,8 +38,22 @@ Two status fields, deliberately. The frozen `D1_ASSEMBLY_v2_2` body and the register note plus addenda disagree about four of the five premises, and that disagreement is *correct*: the note's deltas take effect only at the next issuance. Collapsing them would promote the v2.3 draft by accident. The -firewalls read the **frozen** column, except `FW-NO-RECEIPT-PROMOTION`, which -reads **both columns separately** — a receipt may not move either one. +firewalls read **both columns separately**, and neither column may be moved: +`FW-NO-RECEIPT-PROMOTION` because a receipt may not move either one, and +`FW-UNCONDITIONAL` because a premise open in its register note is open. Reading +only the frozen column, which is what this checker used to do, made a +disagreement in the direction that matters invisible. + +Both columns are also read against a **closed** vocabulary. A premise counts as +discharged only when a column says so in a word listed in `PREMISE_DISCHARGED` +(`CLOSED`, `DISCHARGED`, `PROMOTED`, `SATISFIED`, `CERTIFIED`); a word in +`PREMISE_UNDISCHARGED` is a refusal; a word in neither is a refusal too, because +a status this checker cannot read is not evidence that anything was discharged. +The old test asked only whether the frozen column read `OPEN` or `NOT_CLOSED`, +which let `NAMED_HYPOTHESIS` — carried by two premises, and by definition not a +discharge — pass as though it were one. `RESTATED` and `REFINEMENT` are +deliberately in the undischarged set: whether a restatement discharges a premise +is a status decision, and this file does not make status decisions. A premise outside the D1 assembly (`H-B3`, `LM013-JOINT-STACK`) carries the same transcribed status in both columns and says so in `status_columns_note`. That is @@ -59,21 +85,22 @@ the registers' own technical statuses `NEEDS_RECONCILIATION` and | field | vocabulary | |---|---| | `kind` | `proof_body`, `frozen_certificate_body`, `certificate_set`, `register_row`, `review_record`, `exact_rational_module`, and the four that establish nothing: `receipt`, `test`, `carrier_binding`, `reproduction` | -| `arithmetic` | `exact_rational`, `interval_*`, `not_applicable`, `float`, `mpmath_float`, … — or, when it comes from a carrier manifest, that manifest's own sentence | -| `certifying` | what the artifact's own record says about itself. It is used **only to refuse**; it never grants anything | +| `arithmetic` | `exact_rational`, `interval_*`, `not_applicable`, `float`, `mpmath_float`, … — or, when it comes from a carrier index, that index's own sentence. `classify_arithmetic` sorts it into exactly one of `exact`, `float`, `not_applicable`, `unrecognised` and `ambiguous`; the last two are counted in the summary line so an unreadable record is visible rather than inert | +| `certifying` | what the artifact's own record says about itself. It must be a real JSON boolean — a string `"true"` is refused, because the guard is an identity test that such a string walks straight past. It is used **only to refuse**; it never grants anything | -## The eight firewalls +## The nine firewalls | ID | Rule | Source | |---|---|---| -| `FW-UNCONDITIONAL` | a claim graded `LIVE_ROOT_THEOREM`, `FROZEN_CERTIFICATE` or `RATIFIED_3D_ONLY` may not rest, transitively, on a premise whose **frozen** status is OPEN or NOT_CLOSED | `HOLD_OPEN_VALIDITY_PREMISES.md`; OP-GDN-002 §6 | +| `FW-UNCONDITIONAL` | a claim graded `LIVE_ROOT_THEOREM`, `FROZEN_CERTIFICATE` or `RATIFIED_3D_ONLY` may not rest, transitively, on a premise **either** of whose status columns is anything other than a word in the closed discharged vocabulary | `HOLD_OPEN_VALIDITY_PREMISES.md`; OP-GDN-002 §6 | | `FW-2D-3D-COMPOSITION` | no claim may depend on both the 2D tracks and the 3D lifetime track | `ERRATA_AND_CLARIFICATIONS_2026-09-13.md` §1 | | `FW-PRIZE-ISOLATION` | the prize track and the q0/3D tracks may not depend on each other | `LANE_MATH_MAP.md` firewall | | `FW-NO-PRIZE-CLOSURE` | every prize claim must carry `original_prize_closed: false` | `CLAIM_REGISTRY_VERIFIED_INTAKE.json` | | `FW-DECIMAL-KILL` | the qualitative rate must forbid a finite decimal `C_Q0`; Theorem B must forbid a numerical `C*` | `Q0_MASTER.md` Part I; C092 §12.3 | | `FW-LM011-PRECONDITION` | the RV-LM011 synthesis route may not be marked satisfiable while any named prerequisite is unsatisfied, and a technical pass at **zero** organizational independence credit does not discharge an independence-requiring gate | `review_queue.json` RV-LM011-MAIN; `easy_closure_queue.json` P02-LM-011; `docs/OPEN_PROBLEMS.md` §D | | `FW-NO-RECEIPT-PROMOTION` | a receipt, a green test run, a reproduction or a carrier binding may never raise a grade or move a status, on a claim or on a premise | `engine/README.md`; OP-PROT-012 §4(c); OP-GDN-002 §6 | -| `FW-FLOAT-NOT-CERTIFIED` | high precision is not certification: no evidence may be `certifying` while its arithmetic is float, and no certified/enclosed claim may rest solely on float evidence | `engine/README.md`; `engine/rn_engine/BINDING.json`; `README.md` status discipline | +| `FW-FLOAT-NOT-CERTIFIED` | high precision is not certification: evidence that declares itself `certifying` must **show** exact arithmetic — float, not-applicable, unrecognised and ambiguous are each refused by name — and no certified/enclosed claim may rest solely on float evidence | `engine/README.md`; `engine/rn_engine/BINDING.json`; `README.md` status discipline | +| `FW-RETRACTED-NOT-UNCONDITIONAL` | a claim carrying a retraction record, or whose transcribed `register_status` says RETRACTED, may not carry an unconditional grade | `ERRATA_AND_CLARIFICATIONS_2026-09-13.md`; `CLAIM_REGISTRY_VERIFIED_INTAKE.json` | Plus referential integrity, acyclicity, "a CONDITIONAL claim must name at least one premise", "`technical_status` and `grade` may not drift apart", and "an @@ -117,13 +144,34 @@ carriers RNENG-01..08) and the receipts of runs against it. Evidence of kind `carrier_binding` cannot carry a discharging status in either column, so the premise cannot be closed here by any amount of green CI. -`FW-FLOAT-NOT-CERTIFIED` reads `engine/carriers/MANIFEST.json` when it exists: -for evidence naming a `carrier_id` the manifest lists, that carrier's own -`arithmetic` and `certifying` fields win over the graph's copy, because the -carrier's record is what the run actually used. When the manifest is absent, -unreadable, or does not list the carrier, the graph's own evidence record is -used and the manifest is skipped cleanly. Skipping can only lose a refusal that -the graph's own record would have to state anyway; it cannot manufacture a pass. +`FW-FLOAT-NOT-CERTIFIED` reads **both** carrier indexes — +`engine/carriers/MANIFEST.json` and `engine/rn_engine/BINDING.json` — because the +graph names exactly one `carrier_id`, `RNENG-01`, and it lives in the second one. +Reading only the first meant the override resolved nothing: one lookup, one miss, +every run. For evidence naming a `carrier_id` either index lists, that carrier's +own `arithmetic` and `certifying` fields win over the graph's copy, because the +carrier's record is what the run actually used, and the refusal names which file +it came from. When both indexes are absent, unreadable, or do not list the +carrier, the graph's own evidence record is used and the lookup is skipped +cleanly. Skipping can only lose a refusal that the graph's own record would have +to state anyway; it cannot manufacture a pass. + +Activating that override naively would have *weakened* the firewall, which is +worth recording. `BINDING.json` describes RNENG-01 in a sentence that reads "no +`fractions.Fraction`, no `decimal.Decimal` and no interval arithmetic occurs +anywhere" — genuinely float code, whose own **denial** mentions three exact +tokens. Under the old rule that an exact token anywhere wins, that sentence +classified as exact. Hence `ambiguous`: a string carrying tokens from both +vocabularies is prose, not a classification, and the checker says so instead of +picking a side. A record that claims no certification may describe its arithmetic +in prose freely, because nothing rests on it. + +A node's `track` is read against a closed set for the same reason. Three +firewalls — `FW-2D-3D-COMPOSITION`, `FW-PRIZE-ISOLATION` and +`FW-NO-PRIZE-CLOSURE` — test `track` by membership against a literal set, so an +unrecognised value drops the node out of all three at once and fails nothing. A +missing `track`, or one outside `TRACKS`, is now a refusal: a typo or a rename +was a silent opt-out from rule 4. `GRADE_STRENGTH` in the checker orders grades for **one** purpose: refusing a record that claims more than its evidence can carry. It is not a mathematical @@ -136,7 +184,9 @@ rather than a default — a new grade has to be placed explicitly. python3 tools/claims_check.py # check the committed graph python3 tools/claims_check.py --graph X.json # check a candidate graph python3 tools/claims_check.py --manifest M.json # a different carrier manifest -python3 -m pytest tests/test_claims.py -q # firewalls + negative controls +python3 tools/claims_check.py --binding B.json # a different rn_engine carrier index +python3 tools/claims_check.py --readme R.md # reconcile against different prose +python3 -m pytest tests/test_claims.py tests/test_claims_firewalls.py -q ``` `tests/test_claims.py` runs the checker on the committed graph and then breaks a @@ -150,6 +200,49 @@ from something incidental. Every mutation is passed to the checker with an explicit `--graph`: a default-argument bug once made these tests silently re-check the good graph and pass regardless. +`tests/test_claims_firewalls.py` is the second control file, and it covers the +fail-open cases rather than the firewalls' subject matter: a `certifying` string +that is not a boolean, a certifying record whose arithmetic is not-applicable or +unrecognised or the both-vocabularies sentence, a premise status word in neither +vocabulary, a `NAMED_HYPOTHESIS` premise under an unconditional claim, a premise +open only in its register-note column, an unknown and a missing `track`, a +firewall declared but not enforced, a firewall enforced but not documented, and a +drifted count in this file. Each is checked through the CLI against a mutated +**copy**, and each is paired with the assertion that the committed tree passes. + +### The mutants these controls kill + +A control that passes against the fixed checker has not been shown to test +anything. Each fix below was reverted in a **copy** of `tools/claims_check.py` +outside the tree, one at a time, in a subprocess with `PYTHONDONTWRITEBYTECODE=1` +— an in-process harness once gave a false result here, because same-size mutants +written in quick succession reused a stale `__pycache__`. Sixteen reverted fixes, +sixteen killed, none surviving: + +| reverted fix | control that dies | +|---|---| +| `certifying` need not be a boolean | `..._a_certifying_string_that_is_not_a_boolean` | +| an unlisted arithmetic word reads as exact | `..._an_unrecognised_arithmetic_word` | +| a missing or non-string arithmetic reads as exact | `..._a_missing_arithmetic_field` | +| "an exact token anywhere wins" | `..._a_sentence_from_both_vocabularies` | +| `not_applicable` may certify | `..._not_applicable_arithmetic` | +| one carrier index instead of two | `..._the_graph_s_only_carrier_id_resolves`, `..._overrides_the_graph_and_names_its_source` | +| one status column instead of two | `..._open_only_in_its_register_note_column` | +| the old `OPEN`/`NOT_CLOSED` whitelist | `..._a_named_hypothesis_premise_...`, `..._restated_and_refinement_are_not_discharges` | +| no closed status vocabulary | `..._a_premise_status_word_in_neither_vocabulary` | +| no `track` presence check | `..._a_missing_track` | +| no `track` vocabulary check | `..._an_unrecognised_track` | +| a missing prose document is skipped | `..._an_absent_prose_document` | +| declared-not-enforced unchecked | `..._declared_in_the_graph_and_not_enforced` | +| enforced-not-declared unchecked | `..._enforced_and_not_declared` | +| prose need not name an enforced firewall | `..._does_not_name_an_enforced_firewall` | +| the prose counts are not compared | `..._a_drifted_claim_count...`, `..._a_drifted_firewall_count...` | + +Two of the sixteen exist only because the first harness did not reach them: `if +False: ... elif t not in TRACKS:` still evaluates the `elif`, so disabling the +missing-`track` arm left the vocabulary arm live and the matrix looked complete +when it was not. Both arms now have their own mutant. + ## Updating it When a source status changes, edit the matching `status_frozen_v2_2`, @@ -175,3 +268,8 @@ Nothing mathematical. The graph is a transcription with a checker attached. graph is 0. * The 2D upper/lower tracks and the 3D lifetime track are never composed, and no original prize problem is solved. +* Closing the fail-open holes moved no status and discharged nothing. Every + refusal added here is a refusal the committed graph already survives: the + conversion was measured against the tree before it was adopted, and it adds + zero refusals to it. A firewall that now fails closed is a firewall that will + catch a *future* edit — it certifies nothing about the present one. diff --git a/tests/test_claims_firewalls.py b/tests/test_claims_firewalls.py new file mode 100644 index 000000000..05edb8bab --- /dev/null +++ b/tests/test_claims_firewalls.py @@ -0,0 +1,415 @@ +"""Negative controls for the ways a claim-graph firewall failed *open*. + +`tests/test_claims.py` covers the firewalls' subject matter: it promotes a +conditional theorem, composes 2D with 3D, leaks the prize track, and asserts each +is rejected. Those tests pass against a checker whose guards are fail-open, +because they mutate the data in the one shape the guard happens to recognise. + +This file covers the guards themselves. Each test drives the checker into a +region where it used to answer "nothing to refuse" — an unlisted arithmetic word, +a `certifying` that is a string rather than a boolean, a premise status in neither +vocabulary, a status column nobody read, a `track` that matches no firewall's +membership test, and a firewall list nothing reconciled — and asserts the checker +now refuses. Every one is paired with a positive control: the same mutation with +the offending field put back, or the committed tree, so that a refusal is known to +come from the mutated field and not from the harness. + +As in `tests/test_claims.py`, the checker is always invoked through its CLI with +an explicit `--graph`. A default-argument bug once made every mutation test in +this repository silently re-check the committed graph. +""" +import copy +import json +import os +import subprocess +import sys +import tempfile + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +CHECKER = os.path.join(ROOT, "tools", "claims_check.py") +sys.path.insert(0, os.path.join(ROOT, "tools")) + +import claims_check # noqa: E402 + +# The unconditional claims in the committed graph all carry `depends_on: []`, so +# FW-UNCONDITIONAL's transitive closure over them is empty and no premise status +# is reached at all. Controls below therefore wire a premise underneath one on +# purpose. That wiring is a hypothetical about the checker; it records nothing +# about the mathematics, and nothing in this repository says any of these claims +# rests on any of these premises. +UNCONDITIONAL_CLAIM = "H3-BAND-FLOOR" +CARRIER_EVIDENCE = ("D3-LEMMA-RN-UNIF", 0) # the one record naming a carrier_id +CERTIFYING_EVIDENCE = ("RN3-FAR", 0) # certifying: true, interval arithmetic + +# From engine/rn_engine/BINDING.json, RNENG-01. Genuinely float code whose own +# sentence denies exact arithmetic, and therefore mentions three exact tokens. +BOTH_VOCABULARIES = ("mpmath binary floating point at mp.dps = 100; no " + "fractions.Fraction, no decimal.Decimal and no interval " + "arithmetic occurs anywhere") + + +def graph(): + return claims_check.load() + + +def run(mutated=None, readme=None, binding=None, manifest=None): + """Run the checker through its CLI. Returns (returncode, stdout). + + `mutated` is written to a temporary file and passed with `--graph`; passing + None checks the committed graph. `readme`/`binding`/`manifest` take either a + string of file contents (or a dict, JSON-dumped) to write out, or a path, + which is passed through unchanged so a deliberately absent path is testable. + """ + with tempfile.TemporaryDirectory() as d: + argv = [sys.executable, CHECKER] + + def add(flag, value, name): + if value is None: + return + if isinstance(value, str) and (os.path.isabs(value) or value.startswith("/")): + argv.extend([flag, value]) + return + path = os.path.join(d, name) + with open(path, "w", encoding="utf-8") as handle: + handle.write(value if isinstance(value, str) else json.dumps(value)) + argv.extend([flag, path]) + + if mutated is not None: + add("--graph", json.dumps(mutated), "graph.json") + add("--readme", readme, "README.md") + add("--binding", binding, "BINDING.json") + add("--manifest", manifest, "MANIFEST.json") + p = subprocess.run(argv, capture_output=True, text=True) + return p.returncode, p.stdout + + +def evidence(g, ref): + name, i = ref + node = g["claims"].get(name) or g["premises"][name] + return node["evidence"][i] + + +def prose(): + with open(os.path.join(ROOT, "claims", "README.md"), encoding="utf-8") as h: + return h.read() + + +# ------------------------------------------------------------------ baseline -- + +def test_the_committed_tree_passes(): + code, out = run() + assert code == 0, out + assert "problems=0" in out + + +def test_the_summary_reports_enforced_and_unreadable_counts(): + """A count that is not printed cannot be noticed drifting.""" + code, out = run() + assert code == 0, out + assert "enforced=9" in out, out + # One record's arithmetic is the BINDING sentence, which is ambiguous by + # construction. Reporting it is the point: it is unreadable, not inert. + assert "evidence_arithmetic_unreadable=1" in out, out + + +# ------------------------------- FW-FLOAT-NOT-CERTIFIED, the certifying field -- + +def test_rejects_a_certifying_string_that_is_not_a_boolean(): + """`certifying is True` is an identity test. "true" walked straight past it.""" + g = graph() + evidence(g, CERTIFYING_EVIDENCE)["certifying"] = "true" + code, out = run(g) + assert code == 1, out + assert "is not a boolean" in out, out + + +def test_the_same_record_passes_with_a_real_boolean(): + g = graph() + evidence(g, CERTIFYING_EVIDENCE)["certifying"] = True + assert run(g)[0] == 0 + + +def test_rejects_certifying_with_an_unrecognised_arithmetic_word(): + """"binary64" is float. The old token list did not contain it, and an + unlisted word answered "not float", which the caller read as nothing to + refuse.""" + g = graph() + ev = evidence(g, CERTIFYING_EVIDENCE) + ev["arithmetic"], ev["certifying"] = "binary64", True + code, out = run(g) + assert code == 1, out + assert "unrecognised" in out, out + + +def test_rejects_certifying_with_a_missing_arithmetic_field(): + g = graph() + ev = evidence(g, CERTIFYING_EVIDENCE) + ev.pop("arithmetic", None) + ev["certifying"] = True + code, out = run(g) + assert code == 1, out + assert "FW-FLOAT-NOT-CERTIFIED" in out, out + + +def test_rejects_certifying_with_not_applicable_arithmetic(): + """A record that performs no arithmetic certifies nothing. Nine records in + the graph say `not_applicable`, and the old guard let any of them claim + certification.""" + g = graph() + ev = evidence(g, CERTIFYING_EVIDENCE) + ev["arithmetic"], ev["certifying"] = "not_applicable", True + code, out = run(g) + assert code == 1, out + assert "certifies nothing" in out, out + + +def test_rejects_certifying_with_a_sentence_from_both_vocabularies(): + """The trap: a denial of exact arithmetic mentions exact tokens, and "an + exact token anywhere wins" classified float code as exact.""" + g = graph() + ev = evidence(g, CERTIFYING_EVIDENCE) + ev["arithmetic"], ev["certifying"] = BOTH_VOCABULARIES, True + code, out = run(g) + assert code == 1, out + assert "ambiguous" in out, out + + +def test_a_non_certifying_record_may_describe_its_arithmetic_in_prose(): + """The other direction has to stay open, or the BINDING sentence itself + becomes a failure. Nothing rests on a record that certifies nothing.""" + g = graph() + ev = evidence(g, CERTIFYING_EVIDENCE) + ev["arithmetic"], ev["certifying"] = BOTH_VOCABULARIES, False + assert run(g)[0] == 0 + + +def test_the_classifier_refuses_to_call_the_binding_sentence_exact(): + assert claims_check.classify_arithmetic(BOTH_VOCABULARIES) == claims_check.AMBIGUOUS + assert claims_check.classify_arithmetic("exact_rational") == claims_check.EXACT + assert claims_check.classify_arithmetic("mpmath_float") == claims_check.FLOAT + assert claims_check.classify_arithmetic("not_applicable") == claims_check.NOT_APPLICABLE + for unreadable in ("binary64", "", " ", None, 3, True, ["float"]): + assert claims_check.classify_arithmetic(unreadable) == claims_check.UNRECOGNISED + + +def test_is_float_arithmetic_still_answers_only_the_float_question(): + """Kept as a wrapper, so a caller that only wants the float question is not + broken. It must not start answering True for the fail-open classes.""" + assert claims_check.is_float_arithmetic("mpmath_float") is True + assert claims_check.is_float_arithmetic("exact_rational") is False + assert claims_check.is_float_arithmetic("binary64") is False + assert claims_check.is_float_arithmetic(BOTH_VOCABULARIES) is False + + +# ----------------------------------------------- the second carrier index -- + +def test_the_graph_s_only_carrier_id_resolves(): + """It did not. The graph names exactly one carrier_id and the checker read + only the index that does not contain it, so the override that says "the + carrier's own record wins" resolved nothing on every run.""" + idx = claims_check.carrier_indexes() + name, i = CARRIER_EVIDENCE + node = graph()["premises"][name] + cid = node["evidence"][i]["carrier_id"] + assert cid in idx, cid + _, label = idx[cid] + assert label == "engine/rn_engine/BINDING.json", label + + +def test_a_carrier_index_overrides_the_graph_and_names_its_source(): + """Point --binding at an index whose record certifies a float, and the + refusal must quote that file rather than the graph.""" + g = graph() + name, i = CARRIER_EVIDENCE + ev = g["premises"][name]["evidence"][i] + ev["arithmetic"], ev["certifying"] = "exact_rational", False + fake = {"carriers": [{"carrier_id": ev["carrier_id"], + "arithmetic": "mpmath_float", "certifying": True}]} + code, out = run(g, binding=fake) + assert code == 1, out + assert "FW-FLOAT-NOT-CERTIFIED" in out and "BINDING.json" in out, out + + +def test_both_carrier_indexes_absent_is_skipped_cleanly(): + """Skipping can only lose a refusal the graph's own record would have to + state anyway. It must not crash and must not manufacture one.""" + missing = "/nonexistent/carrier-index.json" + code, out = run(binding=missing, manifest=missing) + assert code == 0, out + + +# ---------------------------------------- FW-UNCONDITIONAL, both columns -- + +def test_rejects_a_named_hypothesis_premise_under_an_unconditional_claim(): + """A named hypothesis is by definition not discharged. The old test asked + only whether the frozen column read OPEN or NOT_CLOSED, so NAMED_HYPOTHESIS + — carried by two premises — passed as though it were a discharge.""" + g = graph() + g["claims"][UNCONDITIONAL_CLAIM]["depends_on"] = ["H5-RIM"] + code, out = run(g) + assert code == 1, out + assert "FW-UNCONDITIONAL" in out and "NAMED_HYPOTHESIS" in out, out + + +def test_rejects_a_premise_open_only_in_its_register_note_column(): + """The two-column shape exists because the columns disagree. Reading one of + them made a disagreement in the unsafe direction invisible.""" + g = graph() + g["premises"]["H5-RIM"]["status_frozen_v2_2"] = "CLOSED" + g["premises"]["H5-RIM"]["status_register_note"] = "OPEN" + g["claims"][UNCONDITIONAL_CLAIM]["depends_on"] = ["H5-RIM"] + code, out = run(g) + assert code == 1, out + assert "status_register_note" in out, out + + +def test_the_same_premise_passes_when_both_columns_are_discharged(): + """The positive control for the two above: the refusal comes from the status + columns, not from the wiring the controls add.""" + g = graph() + g["premises"]["H5-RIM"]["status_frozen_v2_2"] = "CLOSED" + g["premises"]["H5-RIM"]["status_register_note"] = "CLOSED" + g["claims"][UNCONDITIONAL_CLAIM]["depends_on"] = ["H5-RIM"] + assert run(g)[0] == 0 + + +def test_rejects_a_premise_status_word_in_neither_vocabulary(): + """An unrecognised status is not evidence of a discharge, and it is not + evidence of anything else either. It has to be placed explicitly.""" + g = graph() + g["premises"]["H5-RIM"]["status_frozen_v2_2"] = "UNDER_REVIEW" + code, out = run(g) + assert code == 1, out + assert "PREMISE_DISCHARGED" in out, out + + +def test_restated_and_refinement_are_not_discharges(): + """Two premises carry these words today. Whether a restatement discharges a + premise is a status decision; this checker transcribes and does not decide, + so the safe reading is the only one available to it.""" + for word in ("RESTATED", "REFINEMENT"): + assert word in claims_check.PREMISE_UNDISCHARGED + assert word not in claims_check.PREMISE_DISCHARGED + g = graph() + g["premises"]["H5-RIM"]["status_frozen_v2_2"] = "RESTATED" + g["premises"]["H5-RIM"]["status_register_note"] = "RESTATED" + g["claims"][UNCONDITIONAL_CLAIM]["depends_on"] = ["H5-RIM"] + code, out = run(g) + assert code == 1, out + assert "FW-UNCONDITIONAL" in out, out + + +def test_the_two_status_vocabularies_do_not_overlap(): + assert not (claims_check.PREMISE_DISCHARGED & claims_check.PREMISE_UNDISCHARGED) + + +# ------------------------------------------------- the track vocabulary -- + +def test_rejects_an_unrecognised_track(): + """Three firewalls test `track` by membership against a literal set, so an + unrecognised value drops the node out of all three at once and fails + nothing. A rename was a silent opt-out from the 2D/3D rule.""" + g = graph() + g["premises"]["H5-RIM"]["track"] = "UPPER_2D" # a plausible typo + code, out = run(g) + assert code == 1, out + assert "is not one of" in out, out + + +def test_rejects_a_missing_track(): + g = graph() + del g["premises"]["H5-RIM"]["track"] + code, out = run(g) + assert code == 1, out + assert "carries no `track`" in out, out + + +def test_every_track_in_the_committed_graph_is_known(): + g = graph() + for node in list(g["claims"].values()) + list(g["premises"].values()): + assert node.get("track") in claims_check.TRACKS, node.get("track") + + +def test_the_q0_tracks_are_a_subset_of_the_track_vocabulary(): + assert claims_check.Q0_TRACKS <= claims_check.TRACKS + + +# ------------------------------------- declared / enforced / documented -- + +def test_rejects_a_firewall_declared_in_the_graph_and_not_enforced(): + g = graph() + g["firewalls"].append({"id": "FW-INVENTED", "rule": "nothing applies this", + "source": "none"}) + code, out = run(g) + assert code == 1, out + assert "FW-INVENTED" in out and "ENFORCED_FIREWALLS" in out, out + + +def test_rejects_a_firewall_enforced_and_not_declared(): + g = graph() + g["firewalls"] = [f for f in g["firewalls"] + if f["id"] != "FW-RETRACTED-NOT-UNCONDITIONAL"] + code, out = run(g) + assert code == 1, out + assert "not declared in claims/graph.json" in out, out + + +def test_rejects_prose_that_does_not_name_an_enforced_firewall(): + """The finding this reconciliation came from: the checker enforced nine + firewalls and claims/README.md documented eight.""" + stripped = prose().replace("FW-RETRACTED-NOT-UNCONDITIONAL", "FW-REDACTED") + code, out = run(readme=stripped) + assert code == 1, out + assert "FW-RETRACTED-NOT-UNCONDITIONAL is enforced" in out, out + + +def test_rejects_a_drifted_claim_count_in_the_prose(): + code, out = run(readme=prose().replace("26 claims", "24 claims")) + assert code == 1, out + assert "claim count has drifted" in out, out + + +def test_rejects_a_drifted_firewall_count_in_the_prose(): + code, out = run(readme=prose().replace("9 firewalls", "8 firewalls")) + assert code == 1, out + assert "firewall count has drifted" in out, out + + +def test_rejects_an_absent_prose_document(): + """Guarding the reconciliation with "if the file exists" would have made + deleting claims/README.md the way to switch it off.""" + code, out = run(readme="/nonexistent/claims/README.md") + assert code == 1, out + assert "is not a pass" in out, out + + +def test_the_committed_prose_states_both_counts(): + g = graph() + text = prose() + assert f"{len(g['claims'])} claims" in text + assert f"{len(claims_check.ENFORCED_FIREWALLS)} firewalls" in text + + +def test_enforced_matches_declared_on_the_committed_graph(): + declared = {f["id"] for f in graph()["firewalls"]} + assert declared == set(claims_check.ENFORCED_FIREWALLS), ( + declared ^ set(claims_check.ENFORCED_FIREWALLS)) + + +# --------------------------------------------------------------- the CLI -- + +def test_an_unknown_flag_is_an_error_rather_than_a_silent_default(): + code, out = run() + assert code == 0, out + p = subprocess.run([sys.executable, CHECKER, "--graf", "x.json"], + capture_output=True, text=True) + assert p.returncode == 2, p.stdout + assert "unknown argument" in p.stdout + + +def test_every_path_flag_needs_a_path(): + for flag in ("--graph", "--manifest", "--binding", "--readme"): + p = subprocess.run([sys.executable, CHECKER, flag], + capture_output=True, text=True) + assert p.returncode == 2, (flag, p.stdout) + assert "needs a path" in p.stdout, (flag, p.stdout) diff --git a/tools/claims_check.py b/tools/claims_check.py index 74dc369d3..40c5c32d4 100644 --- a/tools/claims_check.py +++ b/tools/claims_check.py @@ -34,7 +34,12 @@ ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) GRAPH = os.path.join(ROOT, "claims", "graph.json") +CLAIMS_README = os.path.join(ROOT, "claims", "README.md") MANIFEST = os.path.join(ROOT, "engine", "carriers", "MANIFEST.json") +# The second carrier index, and the one the graph actually names. Until this +# existed the override below resolved nothing: the graph names exactly one +# carrier_id, RNENG-01, which is in BINDING and not in MANIFEST. +BINDING = os.path.join(ROOT, "engine", "rn_engine", "BINDING.json") OPEN_STATUSES = {"OPEN", "NOT_CLOSED"} UNCONDITIONAL_GRADES = {"LIVE_ROOT_THEOREM", "FROZEN_CERTIFICATE", "RATIFIED_3D_ONLY"} @@ -42,6 +47,47 @@ # tracks here so that the 2D/3D and prize firewalls cover it too. Grouping only # ever adds refusals. Q0_TRACKS = {"UPPER2D", "LOWER2D", "P02_ADJACENCY"} +# Every track a node may carry. Closed, because three firewalls read `track` as a +# membership test against a literal set -- FW-2D-3D-COMPOSITION, +# FW-PRIZE-ISOLATION and FW-NO-PRIZE-CLOSURE -- so an unrecognised value matches +# none of them and drops the node out of all three at once, silently. Nothing +# validated it: the graph happens to use exactly these five, and a typo or a +# rename would have been a quiet opt-out from rule 4. +TRACKS = Q0_TRACKS | {"LIFETIME3D", "NUMBER_THEORY"} + +# Premise status vocabulary, closed for the same reason. FW-UNCONDITIONAL used to +# ask `status_frozen_v2_2 in {"OPEN", "NOT_CLOSED"}`, which is an open-word +# whitelist over ONE of the two columns, and it let two things through: +# +# * NAMED_HYPOTHESIS read as discharged. A named hypothesis is by definition +# not discharged, and two premises carry exactly that word. +# * the register-note column was never read at all -- and the two-column shape +# exists precisely because the columns disagree about four of five D1 +# premises. A premise open in its note and something else when frozen was +# invisible. +# +# So the test is inverted: a premise is treated as discharged only when a column +# says so in a word listed here, and an unrecognised word is refused rather than +# assumed harmless. RESTATED and REFINEMENT are deliberately NOT in +# PREMISE_DISCHARGED: whether a restatement discharges the premise is a status +# question, and this file transcribes statuses rather than deciding them. +# The firewalls this file actually enforces. Three lists used to exist -- declared +# in graph.json, enforced here, documented in claims/README.md -- and nothing +# reconciled them, so claims/README.md documented eight where nine are enforced +# and said "24 claims" over a graph of twenty-six. A checker that cannot say +# which rules it applies cannot be audited, so the list is written down and +# compared against both the graph and the prose on every run. +ENFORCED_FIREWALLS = frozenset({ + "FW-UNCONDITIONAL", "FW-2D-3D-COMPOSITION", "FW-PRIZE-ISOLATION", + "FW-NO-PRIZE-CLOSURE", "FW-DECIMAL-KILL", "FW-LM011-PRECONDITION", + "FW-NO-RECEIPT-PROMOTION", "FW-FLOAT-NOT-CERTIFIED", + "FW-RETRACTED-NOT-UNCONDITIONAL", +}) + +PREMISE_STATUS_COLUMNS = ("status_frozen_v2_2", "status_register_note") +PREMISE_DISCHARGED = {"CLOSED", "DISCHARGED", "PROMOTED", "SATISFIED", "CERTIFIED"} +PREMISE_UNDISCHARGED = {"OPEN", "NOT_CLOSED", "NAMED_HYPOTHESIS", "RESTATED", + "REFINEMENT", "PARTIAL", "REFUTED", "RETRACTED"} # ---------------------------------------------------------------- FW-LM011 -- # Technical statuses the registers use for a route that has passed. Transcribed @@ -86,23 +132,55 @@ # way as the graph's token ("mpmath_float"). FLOAT_ARITHMETIC_TOKENS = ("float", "mpmath", "numpy", "double") EXACT_ARITHMETIC_TOKENS = ("fraction", "rational", "interval", "arb", "decimal", "exact") +# The third case, and it must be spelled rather than left blank: a review record +# or a register row performs no arithmetic at all. Nine evidence records say so. +NOT_APPLICABLE_ARITHMETIC_TOKENS = ("not_applicable", "not applicable") +FLOAT, EXACT, NOT_APPLICABLE = "float", "exact", "not_applicable" +UNRECOGNISED, AMBIGUOUS = "unrecognised", "ambiguous" -def is_float_arithmetic(arithmetic) -> bool: - """True when the declared arithmetic is binary floating point. - An exact/interval token anywhere wins: `engine/rn_engine/BINDING.json` - records arithmetic as a sentence, and a sentence that mentions intervals or - `fractions.Fraction` is not being described as plain float. Unknown or - missing arithmetic is NOT reported as float — it is simply unchecked, which - is why an unrecorded arithmetic is never a pass either. +def classify_arithmetic(arithmetic) -> str: + """One of FLOAT, EXACT, NOT_APPLICABLE, UNRECOGNISED, AMBIGUOUS. + + This was `is_float_arithmetic`, returning a bool, and the bool was a + fail-open: an unlisted word ("IEEE 754", "binary64"), a missing field and a + non-string all returned False, which the caller read as "not float" and so + as nothing to refuse. The old docstring claimed "an unrecorded arithmetic is + never a pass either". It was. + + AMBIGUOUS is the second fix and the less obvious one. The old rule was that + an exact token anywhere wins, which a NEGATION defeats. + `engine/rn_engine/BINDING.json` records RNENG-01 as + + "mpmath binary floating point at mp.dps = 100 ... no fractions.Fraction, + no decimal.Decimal and no interval arithmetic occurs anywhere" + + -- genuinely float code, whose own denial mentions `fraction`, `decimal` and + `interval`, so the old rule classified it EXACT. A string carrying tokens + from both vocabularies is prose, not a classification, and the checker says + so rather than picking one. """ - if not isinstance(arithmetic, str): - return False + if not isinstance(arithmetic, str) or not arithmetic.strip(): + return UNRECOGNISED s = arithmetic.lower() - if any(t in s for t in EXACT_ARITHMETIC_TOKENS): - return False - return any(t in s for t in FLOAT_ARITHMETIC_TOKENS) + float_hit = any(t in s for t in FLOAT_ARITHMETIC_TOKENS) + exact_hit = any(t in s for t in EXACT_ARITHMETIC_TOKENS) + if float_hit and exact_hit: + return AMBIGUOUS + if float_hit: + return FLOAT + if exact_hit: + return EXACT + if any(t in s for t in NOT_APPLICABLE_ARITHMETIC_TOKENS): + return NOT_APPLICABLE + return UNRECOGNISED + + +def is_float_arithmetic(arithmetic) -> bool: + """Only the float question. Prefer the classifier: this still answers False + for UNRECOGNISED and AMBIGUOUS, which is the fail-open it exists to close.""" + return classify_arithmetic(arithmetic) == FLOAT def load(path: str | None = None) -> dict: @@ -153,20 +231,46 @@ def load_carrier_manifest(path: str | None = None) -> dict: return out +def carrier_indexes(manifest_path: str | None = None, + binding_path: str | None = None) -> dict: + """{carrier_id: (record, source label)} over BOTH carrier indexes. + + `tools/quarantine_check.py` and `tools/lanes_check.py` already resolve a + carrier against both; this one read only `engine/carriers/MANIFEST.json`, + and the graph names exactly one carrier_id, which lives in the other file. + So the override that says "the carrier's own record is what the run actually + used" resolved nothing at all: one lookup, one miss, every run. + + MANIFEST wins a duplicate id, and a duplicate that disagrees is reported by + the caller rather than silently resolved. + """ + out: dict[str, tuple[dict, str]] = {} + for path, label in ((binding_path or BINDING, "engine/rn_engine/BINDING.json"), + (manifest_path or MANIFEST, "engine/carriers/MANIFEST.json")): + for cid, rec in load_carrier_manifest(path).items(): + out[cid] = (rec, label) + return out + + def evidence_arithmetic(ev: dict, manifest: dict) -> tuple[object, object, str]: """(arithmetic, certifying, where) for one evidence record. - The carrier manifest wins over the graph for any evidence naming a - `carrier_id` it lists: the carrier's own record is what the run actually - used. + A carrier index wins over the graph for any evidence naming a `carrier_id` + it lists: the carrier's own record is what the run actually used. `manifest` + maps a carrier_id to either a bare record or a (record, label) pair, so a + caller that has only one index still works. """ arithmetic, certifying, where = ev.get("arithmetic"), ev.get("certifying"), "graph" - carrier = manifest.get(ev.get("carrier_id")) if ev.get("carrier_id") else None + found = manifest.get(ev.get("carrier_id")) if ev.get("carrier_id") else None + if isinstance(found, tuple): + carrier, label = found + else: + carrier, label = found, "engine/carriers/MANIFEST.json" if carrier: if carrier.get("arithmetic") is not None: - arithmetic, where = carrier["arithmetic"], "engine/carriers/MANIFEST.json" + arithmetic, where = carrier["arithmetic"], label if carrier.get("certifying") is not None: - certifying, where = carrier["certifying"], "engine/carriers/MANIFEST.json" + certifying, where = carrier["certifying"], label return arithmetic, certifying, where @@ -185,7 +289,7 @@ def closure(g: dict, name: str, seen: set[str] | None = None) -> set[str]: def main(argv: list[str] | None = None) -> int: - global GRAPH, MANIFEST + global GRAPH, MANIFEST, BINDING, CLAIMS_README argv = argv if argv is not None else sys.argv[1:] # Parsed by hand and strictly: an unrecognised flag is an error rather than # a silent fall-back to the committed graph. A default-argument bug once @@ -194,23 +298,61 @@ def main(argv: list[str] | None = None) -> int: rest = list(argv) while rest: flag = rest.pop(0) - if flag in ("--graph", "--manifest"): + if flag in ("--graph", "--manifest", "--binding", "--readme"): if not rest: print(f"{flag} needs a path") return 2 if flag == "--graph": GRAPH = rest.pop(0) - else: + elif flag == "--manifest": MANIFEST = rest.pop(0) + elif flag == "--binding": + BINDING = rest.pop(0) + else: + CLAIMS_README = rest.pop(0) else: print(f"unknown argument {flag!r}; usage: claims_check.py " - f"[--graph PATH] [--manifest PATH]") + f"[--graph PATH] [--manifest PATH] [--binding PATH] " + f"[--readme PATH]") return 2 g = load() - manifest = load_carrier_manifest() + manifest = carrier_indexes() claims, premises = g["claims"], g["premises"] known = set(claims) | set(premises) problems: list[str] = [] + unreadable_arithmetic = 0 + + # Declared / enforced / documented, reconciled three ways. + declared = {f["id"] if isinstance(f, dict) else f for f in g.get("firewalls", [])} + for fid in sorted(declared - ENFORCED_FIREWALLS): + problems.append( + f"{fid} is declared in claims/graph.json and is not in ENFORCED_FIREWALLS, " + f"so nothing in this file applies it") + for fid in sorted(ENFORCED_FIREWALLS - declared): + problems.append( + f"{fid} is enforced here and is not declared in claims/graph.json") + # A missing prose document is a refusal, not a skip. `if os.path.isfile(...)` + # would have been the next fail-open in this file: delete claims/README.md + # and the entire reconciliation below disappears without a word. + if not os.path.isfile(CLAIMS_README): + problems.append( + f"{os.path.relpath(CLAIMS_README, ROOT)} is absent, so the firewalls this " + f"file enforces cannot be reconciled against the prose that documents them. " + f"A missing document is not a pass.") + else: + with open(CLAIMS_README, encoding="utf-8") as handle: + prose = handle.read() + for fid in sorted(ENFORCED_FIREWALLS): + if fid not in prose: + problems.append( + f"{fid} is enforced and claims/README.md does not name it; a firewall " + f"nobody can read about is not a safeguard a reader can check") + for want, what in ((f"{len(claims)} claims", "claim count"), + (f"{len(ENFORCED_FIREWALLS)} firewalls", "firewall count")): + if want not in prose: + problems.append( + f"claims/README.md does not state {want!r}; its {what} has drifted " + f"from the graph this checker reads") # 0. referential integrity and acyclicity for name, node in list(claims.items()) + list(premises.items()): @@ -235,16 +377,49 @@ def find_cycle(name, stack): problems.append(f"dependency cycle: {' -> '.join(c)}") break + # Every premise status word must be in the closed vocabulary, on both + # columns. An unrecognised word is not evidence of discharge and must not be + # read as one. + for name, p in premises.items(): + for col in PREMISE_STATUS_COLUMNS: + v = p.get(col) + if v is None: + continue + if v not in PREMISE_DISCHARGED and v not in PREMISE_UNDISCHARGED: + problems.append( + f"FW-UNCONDITIONAL: {name} carries {col} {v!r}, which is in neither " + f"PREMISE_DISCHARGED nor PREMISE_UNDISCHARGED. A status word this file " + f"does not know is not a discharge; place it explicitly, transcribing " + f"the register rather than deciding anything.") + # FW-UNCONDITIONAL for name, claim in claims.items(): if claim.get("grade") not in UNCONDITIONAL_GRADES: continue - for node in closure(g, name) - {name}: + for node in sorted(closure(g, name) - {name}): p = premises.get(node) - if p and p.get("status_frozen_v2_2") in OPEN_STATUSES: - problems.append( - f"FW-UNCONDITIONAL: {name} is graded {claim['grade']} but rests on " - f"{node} (frozen status {p['status_frozen_v2_2']})") + if not p: + continue + for col in PREMISE_STATUS_COLUMNS: + v = p.get(col) + if v is not None and v not in PREMISE_DISCHARGED: + problems.append( + f"FW-UNCONDITIONAL: {name} is graded {claim['grade']} but rests on " + f"{node}, whose {col} is {v!r} and is not a discharge") + + # A node's `track` is read by three firewalls as a membership test, so an + # unrecognised value is an opt-out from all three rather than an error. + for name, node in list(claims.items()) + list(premises.items()): + t = node.get("track") + if t is None: + problems.append( + f"{name}: carries no `track`, so FW-2D-3D-COMPOSITION, " + f"FW-PRIZE-ISOLATION and FW-NO-PRIZE-CLOSURE all skip it") + elif t not in TRACKS: + problems.append( + f"{name}: track {t!r} is not one of {sorted(TRACKS)}. Three firewalls " + f"test `track` by membership, so an unrecognised value drops this node " + f"out of all three at once rather than failing anything") # FW-2D-3D-COMPOSITION for name, claim in claims.items(): @@ -391,11 +566,39 @@ def unsatisfied_reason(dep: str) -> str | None: f"{name}: evidence[{i}] kind {ev.get('kind')!r} is not in the evidence " f"vocabulary {sorted(EVIDENCE_KINDS)}") arithmetic, certifying, where = evidence_arithmetic(ev, manifest) - if certifying is True and is_float_arithmetic(arithmetic): + kind = classify_arithmetic(arithmetic) + if kind in (UNRECOGNISED, AMBIGUOUS): + unreadable_arithmetic += 1 + # The old test was `certifying is True and is_float_arithmetic(...)`, + # and it was a fail-open three ways over: `is True` is an identity + # test the JSON string "true" walks past; an unlisted word and a + # missing field both answered "not float"; and a sentence denying + # exact arithmetic classified as exact. So the rule is inverted. + # A record that claims certification must SHOW exactness; every + # other class is refused, by name. A record that claims none is + # free to describe its arithmetic in prose, because nothing rests + # on it. + if not isinstance(certifying, bool): + problems.append( + f"FW-FLOAT-NOT-CERTIFIED: {name} evidence[{i}] declares certifying " + f"{certifying!r} (from {where}), which is not a boolean. The test that " + f'guards this is an identity test, so the string "true" reads as ' + f"not-certifying and refuses nothing.") + elif certifying is True and kind != EXACT: + why = { + FLOAT: "High precision is not certification.", + NOT_APPLICABLE: "A record that performs no arithmetic certifies nothing.", + UNRECOGNISED: ("It is in neither vocabulary, so it is not evidence of " + "exactness; say which it is."), + AMBIGUOUS: ("It carries tokens from both vocabularies, which is prose " + "rather than a classification -- a sentence saying 'no " + "fractions.Fraction ... no interval arithmetic' describes " + "float code in words that used to classify it exact."), + }[kind] problems.append( f"FW-FLOAT-NOT-CERTIFIED: {name} evidence[{i}] declares certifying: true " - f"with arithmetic {arithmetic!r} (from {where}). High precision is not " - f"certification.") + f"with arithmetic {arithmetic!r} (from {where}), which classifies as " + f"{kind}. {why}") kinds = {ev.get("kind") for ev in evidence} if kinds and kinds <= NON_ESTABLISHING_EVIDENCE_KINDS: @@ -432,7 +635,9 @@ def unsatisfied_reason(dep: str) -> str | None: for p in problems: print(p) print(f"claims={len(claims)} premises={len(premises)} " - f"firewalls={len(g['firewalls'])} problems={len(problems)}") + f"firewalls={len(g['firewalls'])} enforced={len(ENFORCED_FIREWALLS)} " + f"evidence_arithmetic_unreadable={unreadable_arithmetic} " + f"problems={len(problems)}") return 1 if problems else 0 From d4ab3618a07563e4c0da2703b0ab3856b1ff7fd2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 13:01:33 +0000 Subject: [PATCH 2/4] Date the firewall-count drift, since the base moved it mid-branch The reconciliation was written against eight documented and nine enforced. FW-RUNG-OPEN-PREMISE arrived from the base while this branch was in flight and made it ten, which is the drift the check exists to catch rather than a contradiction of the sentence describing it. Says so. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01Cz7WZybv8znP64SpPj6sWY --- claims/README.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/claims/README.md b/claims/README.md index 64753ce8c..0d94225ff 100644 --- a/claims/README.md +++ b/claims/README.md @@ -10,8 +10,11 @@ three lists that had no reason to agree and did not: the firewalls **declared** in `graph.json`, the firewalls it **enforces** (`ENFORCED_FIREWALLS`), and the firewalls named in this file. A run where any of the three disagrees is refused, and so is a run where this file's claim count or firewall count has drifted from -the graph. All three had drifted: this file documented eight firewalls over nine -enforced, and twenty-four claims over a graph of twenty-six. An undocumented +the graph. All three had drifted when the reconciliation was written: this file +documented eight firewalls where nine were enforced — `FW-RUNG-OPEN-PREMISE` +brought it to ten while that work was in flight, which is exactly the drift the +check now catches — and twenty-four claims over a graph of twenty-six. An +undocumented firewall is not a safeguard a reader can check, and a stale count is how a reader learns to stop trusting the rest. A **missing** `claims/README.md` is a refusal too, not a skip: guarding the reconciliation with "if the file exists" would have From 497ceffb10ff4f44cc1880d58c26396b870636e9 Mon Sep 17 00:00:00 2001 From: Dylan Roy Date: Sat, 26 Sep 2026 15:44:22 +0000 Subject: [PATCH 3/4] Four more ways the float firewall failed open, one of them introduced by this branch All four found by a nonauthor engineering review of this branch's own first revision, each reproduced before being fixed and each now carrying a control that fails against that revision. 1. REGRESSION, and the worst of the four, because this branch caused it. Making the checker read both carrier indexes let a carrier's prose RESCUE a claim. The claim-level guard asked `all(is_float_arithmetic(...))`, and that wrapper answers False for ambiguous and unrecognised, so resolving a float evidence record against BINDING's mixed-vocabulary sentence turned float into ambiguous, "not every record is float" became true, and the refusal vanished. Measured: RN3-FAR with one float proof_body naming RNENG-01 exits 0 on this branch's previous head and exits 1 on the base at cd66a65, naming FW-FLOAT-NOT-CERTIFIED. The guard now asks the opposite question -- does ANY record declare exact arithmetic -- so an unreadable declaration can no longer substitute for an exact one. What it still does not refuse, now said in the code and in claims/README.md: a certified claim citing one exact record alongside float ones. That is the published semantics of this firewall and narrowing it belongs to the register. 2. Substring matching admitted explicit denials of exactness. "inexact" contains "exact"; "no interval arithmetic" contains "interval"; "arbitrary precision" contains "arb". All three classified EXACT, so a record could claim certification while saying in words that it is not exact. Declarations are now matched whole against closed sets, after normalising case, underscores, hyphens and whitespace. Prose classifies nothing; the both-vocabularies scan survives only to NAME prose in a message, never to grant exactness. The vocabulary is closed in the same spirit as GRADE_STRENGTH: an unplaced declaration is unrecognised, which refuses wherever certification is claimed. A control asserts the vocabulary covers every declaration in the committed graph, so the strictness is not paid for by a refusal nobody intended. "binary64" is now classified as float rather than unreadable, which is a strictly better message for the same refusal. 3. Absence was read as a discharge. A premise with neither status_frozen_v2_2 nor status_register_note skipped the closed vocabulary AND both grade guards, so deleting two fields let an unconditional or CERTIFIED_RUNG claim rest on it with nothing refused. A missing column is now a refusal. All thirteen committed premises carry both columns -- a control asserts that -- so this costs the committed graph nothing. 4. carrier_indexes promised in its own docstring that a disagreeing duplicate would be reported by the caller, and no caller reported anything: MANIFEST overwrote BINDING unconditionally, so a manifest entry declaring RNENG-01 exact_rational/certifying masked the live float declaration and exited 0. The conflicts are now a return value rather than a claim in prose, and the checker refuses rather than choosing between two files. An agreeing duplicate is not a conflict, which has its own positive control. Eleven controls added, nine of them behavioural negative controls verified to fail against the pre-fix checker in a real recursive copy outside the tree. Five existing controls were updated where the fixes changed what they should assert, and one fixture was split: putting prose on RN3-FAR's only exact record now removes the last exact declaration under a certifying grade, which is a refusal in its own right, so the prose control moved to the premise record that actually carries that sentence. A fixture that collapses two facts tests neither. claims=26 premises=13 firewalls=10 enforced=10 evidence_arithmetic_unreadable=1 problems=0, unchanged. 98 tests in the two claims files. claims_check, quarantine_check, lanes_check, carriers_verify, noncertifying_check, consumers_check and registers_check all clean. Scientific effect: NONE. No grade, premise status, claim status or firewall disposition is changed. Every fix only ADDS refusals; none removes one. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01Cz7WZybv8znP64SpPj6sWY --- claims/README.md | 54 ++++++++-- tests/test_claims_firewalls.py | 186 +++++++++++++++++++++++++++++++-- tools/claims_check.py | 136 +++++++++++++++++++----- 3 files changed, 335 insertions(+), 41 deletions(-) diff --git a/claims/README.md b/claims/README.md index 0d94225ff..27b652c4d 100644 --- a/claims/README.md +++ b/claims/README.md @@ -101,7 +101,7 @@ the registers' own technical statuses `NEEDS_RECONCILIATION` and | field | vocabulary | |---|---| | `kind` | `proof_body`, `frozen_certificate_body`, `certificate_set`, `register_row`, `review_record`, `exact_rational_module`, and the four that establish nothing: `receipt`, `test`, `carrier_binding`, `reproduction` | -| `arithmetic` | `exact_rational`, `interval_*`, `not_applicable`, `float`, `mpmath_float`, … — or, when it comes from a carrier index, that index's own sentence. `classify_arithmetic` sorts it into exactly one of `exact`, `float`, `not_applicable`, `unrecognised` and `ambiguous`; the last two are counted in the summary line so an unreadable record is visible rather than inert | +| `arithmetic` | a declaration from a **closed** vocabulary — `exact_rational`, `interval_384bit`, `interval_arb_384bit`, `not_applicable`, `float`, `mpmath_float`, `binary64`, … — or, when it comes from a carrier index, that index's own sentence. Matching is on the whole declaration after normalising case, `_`, `-` and whitespace; it is **not** substring matching, which accepted `inexact`, `no exact arithmetic` and `arbitrary precision` as exact. `classify_arithmetic` sorts it into exactly one of `exact`, `float`, `not_applicable`, `unrecognised` and `ambiguous`; the last two are counted in the summary line so an unreadable record is visible rather than inert. A declaration not in the vocabulary is `unrecognised`, which refuses wherever certification is claimed — a new token must be placed here explicitly, as with `GRADE_STRENGTH` | | `certifying` | what the artifact's own record says about itself. It must be a real JSON boolean — a string `"true"` is refused, because the guard is an identity test that such a string walks straight past. It is used **only to refuse**; it never grants anything | ## The ten firewalls @@ -116,7 +116,7 @@ the registers' own technical statuses `NEEDS_RECONCILIATION` and | `FW-DECIMAL-KILL` | the qualitative rate must forbid a finite decimal `C_Q0`; Theorem B must forbid a numerical `C*` | `Q0_MASTER.md` Part I; C092 §12.3 | | `FW-LM011-PRECONDITION` | the RV-LM011 synthesis route may not be marked satisfiable while any named prerequisite is unsatisfied, and a technical pass at **zero** organizational independence credit does not discharge an independence-requiring gate | `review_queue.json` RV-LM011-MAIN; `easy_closure_queue.json` P02-LM-011; `docs/OPEN_PROBLEMS.md` §D | | `FW-NO-RECEIPT-PROMOTION` | a receipt, a green test run, a reproduction or a carrier binding may never raise a grade or move a status, on a claim or on a premise | `engine/README.md`; OP-PROT-012 §4(c); OP-GDN-002 §6 | -| `FW-FLOAT-NOT-CERTIFIED` | high precision is not certification: evidence that declares itself `certifying` must **show** exact arithmetic — float, not-applicable, unrecognised and ambiguous are each refused by name — and no certified/enclosed claim may rest solely on float evidence | `engine/README.md`; `engine/rn_engine/BINDING.json`; `README.md` status discipline | +| `FW-FLOAT-NOT-CERTIFIED` | high precision is not certification: evidence that declares itself `certifying` must **show** exact arithmetic — float, not-applicable, unrecognised and ambiguous are each refused by name — a certified/enclosed claim must cite **at least one** record declaring exact arithmetic, and the two carrier indexes may not disagree about a carrier without being refused. What it still does **not** refuse, said plainly: a certified claim citing one exact record alongside float ones. Narrowing that is a decision for the register, not for a checker | `engine/README.md`; `engine/rn_engine/BINDING.json`; `README.md` status discipline | | `FW-RETRACTED-NOT-UNCONDITIONAL` | a claim carrying a retraction record, or whose transcribed `register_status` says RETRACTED, may not carry an unconditional grade | `ERRATA_AND_CLARIFICATIONS_2026-09-13.md`; `CLAIM_REGISTRY_VERIFIED_INTAKE.json` | Plus referential integrity, acyclicity, "a CONDITIONAL claim must name at least @@ -183,6 +183,37 @@ vocabularies is prose, not a classification, and the checker says so instead of picking a side. A record that claims no certification may describe its arithmetic in prose freely, because nothing rests on it. +That first repair was not enough, and a nonauthor engineering review found three +further holes plus one **regression** it had introduced. All four are recorded +here because a checker's history of failing open is the most useful thing to know +about it. + +- **The regression.** Reading both carrier indexes let a carrier's prose *rescue* + a claim. The claim-level guard asked `all(is_float_arithmetic(...))`, and that + wrapper answers `False` for `ambiguous`, so resolving a float evidence record + against the `BINDING` sentence turned `float` into `ambiguous`, "not every + record is float" became true, and a refusal that the previous revision made + disappeared. The guard now asks the opposite question — does **any** record + declare exact arithmetic — so an unreadable declaration can no longer stand in + for an exact one. +- **Substring matching admitted denials of exactness.** `inexact` contains + `exact`; `no interval arithmetic` contains `interval`; `arbitrary precision` + contains `arb`. Each classified as exact, so a record could claim certification + while saying in words that it is not exact. Declarations are now matched whole, + against closed sets; prose classifies nothing, and the both-vocabularies scan + survives only to *name* prose in a message. +- **Absence was read as a discharge.** A premise with no `status_frozen_v2_2` and + no `status_register_note` skipped the closed vocabulary and both grade guards, + so deleting two fields let an unconditional or `CERTIFIED_RUNG` claim rest on it + with nothing refused. A missing column is now a refusal. All thirteen committed + premises carry both, so this costs the committed graph nothing. +- **A conflicting carrier index was silently overwritten.** `carrier_indexes` + promised in its own docstring that a disagreeing duplicate would be reported, + and no caller reported anything: `MANIFEST` overwrote `BINDING` unconditionally, + so a manifest entry could mask a live float declaration. The conflicts are now a + return value rather than a claim in prose, and the checker refuses rather than + choosing between the two files. + A node's `track` is read against a closed set for the same reason. Three firewalls — `FW-2D-3D-COMPOSITION`, `FW-PRIZE-ISOLATION` and `FW-NO-PRIZE-CLOSURE` — test `track` by membership against a literal set, so an @@ -224,9 +255,12 @@ unrecognised or the both-vocabularies sentence, a premise status word in neither vocabulary, a `NAMED_HYPOTHESIS` premise under an unconditional claim, a premise open only in its register-note column, an unknown and a missing `track`, a firewall declared but not enforced, a firewall enforced but not documented, and a -drifted count in this file, and a `CERTIFIED_RUNG` grade over a register note -that is not a discharge. Each is checked through the CLI against a mutated -**copy**, and each is paired with the assertion that the committed tree passes. +drifted count in this file, a `CERTIFIED_RUNG` grade over a register note that is +not a discharge, four declarations that deny the exactness they were read as +asserting, a premise with no status columns at all, a certifying grade whose only +readable declaration is prose, and two carrier indexes that disagree. Each is +checked through the CLI against a mutated **copy**, and each is paired with the +assertion that the committed tree passes. ### The mutants these controls kill @@ -234,8 +268,8 @@ A control that passes against the fixed checker has not been shown to test anything. Each fix below was reverted in a **copy** of `tools/claims_check.py` outside the tree, one at a time, in a subprocess with `PYTHONDONTWRITEBYTECODE=1` — an in-process harness once gave a false result here, because same-size mutants -written in quick succession reused a stale `__pycache__`. Seventeen reverted -fixes, seventeen killed, none surviving: +written in quick succession reused a stale `__pycache__`. Twenty-one reverted +fixes, twenty-one killed, none surviving: | reverted fix | control that dies | |---|---| @@ -256,8 +290,12 @@ fixes, seventeen killed, none surviving: | enforced-not-declared unchecked | `..._enforced_and_not_declared` | | prose need not name an enforced firewall | `..._does_not_name_an_enforced_firewall` | | the prose counts are not compared | `..._a_drifted_claim_count...`, `..._a_drifted_firewall_count...` | +| substring matching for the arithmetic vocabulary | `..._a_declaration_denying_exactness_is_never_classified_exact`, `..._certifying_with_a_declaration_denying_exactness_is_refused` | +| the claim-level guard back to `all(is_float_arithmetic(...))` | `..._a_certifying_grade_needs_one_exact_record_not_merely_a_non_float_one` | +| a missing premise status column is skipped | `..._a_premise_with_no_status_columns_is_refused`, `..._a_certified_rung_on_a_premise_with_no_status_columns_is_refused` | +| a conflicting carrier id is overwritten silently | `..._a_conflicting_carrier_index_is_refused_not_silently_overwritten` | -Two of the sixteen exist only because the first harness did not reach them: `if +Two of the twenty-one exist only because the first harness did not reach them: `if False: ... elif t not in TRACKS:` still evaluates the `elif`, so disabling the missing-`track` arm left the vocabulary arm live and the matrix looked complete when it was not. Both arms now have their own mutant. diff --git a/tests/test_claims_firewalls.py b/tests/test_claims_firewalls.py index 8f6af7a7d..257cfe00e 100644 --- a/tests/test_claims_firewalls.py +++ b/tests/test_claims_firewalls.py @@ -129,15 +129,27 @@ def test_the_same_record_passes_with_a_real_boolean(): assert run(g)[0] == 0 -def test_rejects_certifying_with_an_unrecognised_arithmetic_word(): - """"binary64" is float. The old token list did not contain it, and an +def test_rejects_certifying_with_a_float_word_the_token_list_lacked(): + """"binary64" is float. The original token list did not contain it, and an unlisted word answered "not float", which the caller read as nothing to - refuse.""" + refuse. It is now in FLOAT_ARITHMETIC_DECLARATIONS, so it is refused *as + float* rather than merely as unreadable -- a strictly better message for the + same refusal.""" g = graph() ev = evidence(g, CERTIFYING_EVIDENCE) ev["arithmetic"], ev["certifying"] = "binary64", True code, out = run(g) assert code == 1, out + assert "classifies as float" in out, out + assert claims_check.classify_arithmetic("binary64") == claims_check.FLOAT + + +def test_rejects_certifying_with_a_word_in_no_vocabulary(): + g = graph() + ev = evidence(g, CERTIFYING_EVIDENCE) + ev["arithmetic"], ev["certifying"] = "quaternionic abacus", True + code, out = run(g) + assert code == 1, out assert "unrecognised" in out, out @@ -176,11 +188,20 @@ def test_rejects_certifying_with_a_sentence_from_both_vocabularies(): def test_a_non_certifying_record_may_describe_its_arithmetic_in_prose(): """The other direction has to stay open, or the BINDING sentence itself - becomes a failure. Nothing rests on a record that certifies nothing.""" + becomes a failure. Nothing rests on a record that certifies nothing. + + Deliberately mutated on the PREMISE record that really does carry that + sentence, not on RN3-FAR. Putting prose on RN3-FAR's only exact record + removes the last exact declaration under a certifying grade, which is now a + refusal in its own right -- see + test_a_certifying_grade_needs_one_exact_record_not_merely_a_non_float_one. + Two separate facts; a fixture that collapses them tests neither.""" g = graph() - ev = evidence(g, CERTIFYING_EVIDENCE) + ev = evidence(g, CARRIER_EVIDENCE) ev["arithmetic"], ev["certifying"] = BOTH_VOCABULARIES, False assert run(g)[0] == 0 + # And the committed tree, where that sentence is live via the carrier index. + assert run()[0] == 0 def test_the_classifier_refuses_to_call_the_binding_sentence_exact(): @@ -188,8 +209,11 @@ def test_the_classifier_refuses_to_call_the_binding_sentence_exact(): assert claims_check.classify_arithmetic("exact_rational") == claims_check.EXACT assert claims_check.classify_arithmetic("mpmath_float") == claims_check.FLOAT assert claims_check.classify_arithmetic("not_applicable") == claims_check.NOT_APPLICABLE - for unreadable in ("binary64", "", " ", None, 3, True, ["float"]): + for unreadable in ("", " ", None, 3, True, ["float"], "mixed", "unknown"): assert claims_check.classify_arithmetic(unreadable) == claims_check.UNRECOGNISED + # Normalisation only: underscores, hyphens, case and runs of whitespace. + for same in ("exact_rational", "EXACT-RATIONAL", " exact rational "): + assert claims_check.classify_arithmetic(same) == claims_check.EXACT def test_is_float_arithmetic_still_answers_only_the_float_question(): @@ -197,8 +221,14 @@ def test_is_float_arithmetic_still_answers_only_the_float_question(): broken. It must not start answering True for the fail-open classes.""" assert claims_check.is_float_arithmetic("mpmath_float") is True assert claims_check.is_float_arithmetic("exact_rational") is False - assert claims_check.is_float_arithmetic("binary64") is False + # "binary64" now answers True, because it is float and is in the vocabulary. + # It answered False while the token list lacked it, and that False was the + # fail-open: the caller read it as "not float, nothing to refuse". + assert claims_check.is_float_arithmetic("binary64") is True + # These two still answer False, so no caller may use this wrapper to decide + # that a record is SAFE. That is why the guards ask the classifier instead. assert claims_check.is_float_arithmetic(BOTH_VOCABULARIES) is False + assert claims_check.is_float_arithmetic("quaternionic abacus") is False # ----------------------------------------------- the second carrier index -- @@ -207,7 +237,8 @@ def test_the_graph_s_only_carrier_id_resolves(): """It did not. The graph names exactly one carrier_id and the checker read only the index that does not contain it, so the override that says "the carrier's own record wins" resolved nothing on every run.""" - idx = claims_check.carrier_indexes() + idx, conflicts = claims_check.carrier_indexes() + assert conflicts == [], conflicts name, i = CARRIER_EVIDENCE node = graph()["premises"][name] cid = node["evidence"][i]["carrier_id"] @@ -436,3 +467,142 @@ def test_every_path_flag_needs_a_path(): capture_output=True, text=True) assert p.returncode == 2, (flag, p.stdout) assert "needs a path" in p.stdout, (flag, p.stdout) + + +# --------------------------------------------- four ways it still failed open -- +# All four were found by a nonauthor engineering review of this file's own first +# revision, reproduced here before being fixed. Three are holes the original +# change left; the first is a REGRESSION the original change introduced -- making +# this checker read both carrier indexes let a carrier's prose rescue a claim that +# the previous revision refused. Each control below fails against that revision. + +def test_a_certifying_grade_needs_one_exact_record_not_merely_a_non_float_one(): + """The regression, in one mutation. + + `all(is_float_arithmetic(...))` asks "is every record float?", and the + wrapper answers False for AMBIGUOUS and UNRECOGNISED. So resolving a float + evidence record against BINDING's mixed-vocabulary sentence turned FLOAT into + AMBIGUOUS, "not all float" became true, and the refusal disappeared. The same + graph is refused by the revision before carrier resolution was added, which is + what makes this a regression rather than a pre-existing gap.""" + g = graph() + g["claims"]["RN3-FAR"]["evidence"] = [ + {"kind": "proof_body", "carrier_id": "RNENG-01", + "arithmetic": "mpmath_float", "certifying": False}] + code, out = run(g) + assert code == 1, out + assert "not one evidence record it cites declares exact arithmetic" in out, out + + +def test_the_certifying_grade_guard_has_a_positive_control(): + """RN3-FAR passes on the committed graph, and it passes because it really + does cite an exact record -- not because the guard is inert.""" + assert run()[0] == 0 + idx, _conflicts = claims_check.carrier_indexes() + classes = [claims_check.classify_arithmetic(claims_check.evidence_arithmetic(ev, idx)[0]) + for ev in graph()["claims"]["RN3-FAR"]["evidence"]] + assert claims_check.EXACT in classes, classes + assert graph()["claims"]["RN3-FAR"]["grade"] in claims_check.CERTIFYING_GRADES + + +DENIALS_OF_EXACTNESS = ("inexact", "no exact arithmetic", + "binary64; no interval arithmetic", "arbitrary precision") + + +def test_a_declaration_denying_exactness_is_never_classified_exact(): + """Substring matching accepted the denial of the property it looked for. + "inexact" contains "exact"; "no interval arithmetic" contains "interval"; + "arbitrary precision" contains "arb". Each classified EXACT, so a record could + claim certification while saying in words that it is not exact.""" + for word in DENIALS_OF_EXACTNESS: + assert claims_check.classify_arithmetic(word) != claims_check.EXACT, word + + +def test_certifying_with_a_declaration_denying_exactness_is_refused(): + for word in DENIALS_OF_EXACTNESS: + g = graph() + ev = evidence(g, CERTIFYING_EVIDENCE) + ev["arithmetic"], ev["certifying"] = word, True + ev.pop("carrier_id", None) + code, out = run(g) + assert code == 1, (word, out) + assert "FW-FLOAT-NOT-CERTIFIED" in out, (word, out) + + +def test_the_declaration_vocabularies_are_disjoint(): + assert not (claims_check.FLOAT_ARITHMETIC_DECLARATIONS + & claims_check.EXACT_ARITHMETIC_DECLARATIONS) + assert not (claims_check.FLOAT_ARITHMETIC_DECLARATIONS + & claims_check.NOT_APPLICABLE_ARITHMETIC_DECLARATIONS) + assert not (claims_check.EXACT_ARITHMETIC_DECLARATIONS + & claims_check.NOT_APPLICABLE_ARITHMETIC_DECLARATIONS) + + +def test_every_arithmetic_declaration_in_the_committed_graph_is_readable(): + """The closed vocabulary must actually cover the graph it ships with, or the + strictness is paid for by a refusal nobody intended.""" + g = graph() + for bucket in ("claims", "premises"): + for name, node in g[bucket].items(): + for i, ev in enumerate(node.get("evidence") or []): + if "carrier_id" in ev: + continue # resolved against an index; prose is expected there + kind = claims_check.classify_arithmetic(ev.get("arithmetic")) + assert kind != claims_check.UNRECOGNISED, (name, i, ev.get("arithmetic")) + + +def test_a_premise_with_no_status_columns_is_refused(): + """Absence is not a discharge. Deleting both columns was a way round the + closed vocabulary: the loop skipped None, and so did both grade guards.""" + g = graph() + g["claims"][UNCONDITIONAL_CLAIM]["depends_on"] = ["H5-RIM"] + for col in claims_check.PREMISE_STATUS_COLUMNS: + g["premises"]["H5-RIM"].pop(col, None) + code, out = run(g) + assert code == 1, out + assert "carries no status_frozen_v2_2" in out, out + assert "is not a discharge" in out, out + + +def test_a_certified_rung_on_a_premise_with_no_status_columns_is_refused(): + g = graph() + g["claims"]["D1-v2.2(1)"]["grade"] = "CERTIFIED_RUNG" + g["claims"]["D1-v2.2(1)"]["depends_on"] = ["H5-RIM"] + for col in claims_check.PREMISE_STATUS_COLUMNS: + g["premises"]["H5-RIM"].pop(col, None) + code, out = run(g) + assert code == 1, out + assert "FW-RUNG-OPEN-PREMISE" in out, out + + +def test_every_committed_premise_carries_both_status_columns(): + """So requiring them costs the committed graph nothing, which is why the + refusal above can be unconditional.""" + for name, p in graph()["premises"].items(): + for col in claims_check.PREMISE_STATUS_COLUMNS: + assert p.get(col) is not None, (name, col) + + +def test_a_conflicting_carrier_index_is_refused_not_silently_overwritten(): + """The docstring promised a disagreeing duplicate would be reported. Nothing + reported it, and MANIFEST overwrote BINDING unconditionally, so a manifest + entry could mask a live float declaration.""" + mask = {"carriers": [{"carrier_id": "RNENG-01", "arithmetic": "exact_rational", + "certifying": True}]} + code, out = run(manifest=mask) + assert code == 1, out + assert "declared in both indexes with different arithmetic" in out, out + assert "will not choose between them" in out, out + + +def test_an_agreeing_duplicate_is_not_a_conflict(): + """Or the check would fire on any harmless restatement, and the refusal above + would prove nothing about disagreement.""" + idx, _ = claims_check.carrier_indexes() + rec, label = idx["RNENG-01"] + assert label == "engine/rn_engine/BINDING.json", label + same = {"carriers": [{"carrier_id": "RNENG-01", + "arithmetic": rec["arithmetic"], + "certifying": rec["certifying"]}]} + code, out = run(manifest=same) + assert code == 0, out diff --git a/tools/claims_check.py b/tools/claims_check.py index 59227db2f..73c6accdc 100644 --- a/tools/claims_check.py +++ b/tools/claims_check.py @@ -127,19 +127,54 @@ } CONDITIONAL_STRENGTH = GRADE_STRENGTH["CONDITIONAL"] -# Arithmetic vocabulary. Matching is on substrings so that a carrier manifest's -# own prose ("mpmath binary floating point at mp.dps = 100") classifies the same -# way as the graph's token ("mpmath_float"). -FLOAT_ARITHMETIC_TOKENS = ("float", "mpmath", "numpy", "double") -EXACT_ARITHMETIC_TOKENS = ("fraction", "rational", "interval", "arb", "decimal", "exact") +# Arithmetic vocabulary. A declaration classifies a record only when it IS one +# of these, after normalising case, underscores, hyphens and whitespace. The +# previous rule matched SUBSTRINGS, and substring matching accepts the denial of +# the very property it looks for: "inexact" contains "exact", "no interval +# arithmetic" contains "interval", "arbitrary precision" contains "arb". All +# three classified as EXACT, so a record could claim certification while saying +# in words that it is not exact. Prose is not a classification. +# +# These sets are closed on purpose, in the same spirit as GRADE_STRENGTH: a new +# declaration must be placed here explicitly rather than slipping in on a +# coincidence of letters. An unplaced declaration is UNRECOGNISED, which is +# refused wherever certification is claimed -- the safe direction. +FLOAT_ARITHMETIC_DECLARATIONS = frozenset({ + "float", "floats", "floating point", "binary floating point", + "binary64", "double", "double precision", "ieee754", "ieee 754", + "mpmath", "mpmath float", "numpy", "numpy float", +}) +EXACT_ARITHMETIC_DECLARATIONS = frozenset({ + "exact", "exact rational", "rational", "fraction", "fractions", + "fractions.fraction", "decimal", "decimal.decimal", + "interval", "interval arithmetic", "arb", + # The precision-suffixed forms actually used by the committed graph. A new + # suffix is UNRECOGNISED until it is added here, which refuses rather than + # admits. + "interval 384bit", "interval arb 384bit", +}) # The third case, and it must be spelled rather than left blank: a review record # or a register row performs no arithmetic at all. Nine evidence records say so. -NOT_APPLICABLE_ARITHMETIC_TOKENS = ("not_applicable", "not applicable") +NOT_APPLICABLE_ARITHMETIC_DECLARATIONS = frozenset({"not applicable", "none", "n/a"}) + +# Kept ONLY to describe prose in a message. These never grant a classification; +# see classify_arithmetic. Without that restriction they are the substring rule +# the strict sets above replaced. +_FLOAT_PROSE_HINTS = ("float", "mpmath", "numpy", "double") +_EXACT_PROSE_HINTS = ("fraction", "rational", "interval", "arb", "decimal", "exact") + +assert not (FLOAT_ARITHMETIC_DECLARATIONS & EXACT_ARITHMETIC_DECLARATIONS), ( + "a declaration cannot be both float and exact") FLOAT, EXACT, NOT_APPLICABLE = "float", "exact", "not_applicable" UNRECOGNISED, AMBIGUOUS = "unrecognised", "ambiguous" +def normalise_arithmetic(arithmetic: str) -> str: + """Lowercase, and treat `_`, `-` and runs of whitespace as one space.""" + return " ".join(arithmetic.lower().replace("_", " ").replace("-", " ").split()) + + def classify_arithmetic(arithmetic) -> str: """One of FLOAT, EXACT, NOT_APPLICABLE, UNRECOGNISED, AMBIGUOUS. @@ -163,17 +198,21 @@ def classify_arithmetic(arithmetic) -> str: """ if not isinstance(arithmetic, str) or not arithmetic.strip(): return UNRECOGNISED - s = arithmetic.lower() - float_hit = any(t in s for t in FLOAT_ARITHMETIC_TOKENS) - exact_hit = any(t in s for t in EXACT_ARITHMETIC_TOKENS) - if float_hit and exact_hit: - return AMBIGUOUS - if float_hit: - return FLOAT - if exact_hit: + n = normalise_arithmetic(arithmetic) + if n in EXACT_ARITHMETIC_DECLARATIONS: return EXACT - if any(t in s for t in NOT_APPLICABLE_ARITHMETIC_TOKENS): + if n in FLOAT_ARITHMETIC_DECLARATIONS: + return FLOAT + if n in NOT_APPLICABLE_ARITHMETIC_DECLARATIONS: return NOT_APPLICABLE + # Not a declaration. It is prose, and prose classifies nothing. The one + # shape worth naming separately is prose drawing on both vocabularies, + # because that is what the live BINDING sentence does and what the old + # substring rule read as EXACT. AMBIGUOUS and UNRECOGNISED are both + # non-EXACT, so neither can carry a certification either way. + if (any(t in n for t in _FLOAT_PROSE_HINTS) + and any(t in n for t in _EXACT_PROSE_HINTS)): + return AMBIGUOUS return UNRECOGNISED @@ -241,15 +280,32 @@ def carrier_indexes(manifest_path: str | None = None, So the override that says "the carrier's own record is what the run actually used" resolved nothing at all: one lookup, one miss, every run. - MANIFEST wins a duplicate id, and a duplicate that disagrees is reported by - the caller rather than silently resolved. + Returns `(index, conflicts)`. MANIFEST wins a duplicate id, and a duplicate + whose arithmetic or certifying DISAGREES is returned in `conflicts` for the + caller to refuse. An earlier revision of this docstring promised exactly + that while the code overwrote unconditionally and no caller reported + anything, so a manifest entry could mask a live BINDING declaration + silently. The promise is now structural: the conflicts are a return value, + not a claim in prose. """ out: dict[str, tuple[dict, str]] = {} + conflicts: list[str] = [] for path, label in ((binding_path or BINDING, "engine/rn_engine/BINDING.json"), (manifest_path or MANIFEST, "engine/carriers/MANIFEST.json")): for cid, rec in load_carrier_manifest(path).items(): + prior = out.get(cid) + if prior is not None: + old_rec, old_label = prior + for field in ("arithmetic", "certifying"): + if old_rec.get(field) != rec.get(field): + conflicts.append( + f"carrier {cid} is declared in both indexes with different " + f"{field}: {old_label} says {old_rec.get(field)!r}, {label} says " + f"{rec.get(field)!r}. The second wins silently, so a manifest " + f"entry can mask a live declaration. Reconcile the two indexes; " + f"this checker will not choose between them.") out[cid] = (rec, label) - return out + return out, conflicts def evidence_arithmetic(ev: dict, manifest: dict) -> tuple[object, object, str]: @@ -316,11 +372,14 @@ def main(argv: list[str] | None = None) -> int: f"[--readme PATH]") return 2 g = load() - manifest = carrier_indexes() + manifest, carrier_conflicts = carrier_indexes() claims, premises = g["claims"], g["premises"] known = set(claims) | set(premises) problems: list[str] = [] unreadable_arithmetic = 0 + # A duplicate carrier id whose declaration disagrees between the two indexes + # is refused here rather than resolved by whichever file is read second. + problems.extend(f"FW-FLOAT-NOT-CERTIFIED: {c}" for c in carrier_conflicts) # Declared / enforced / documented, reconciled three ways. declared = {f["id"] if isinstance(f, dict) else f for f in g.get("firewalls", [])} @@ -384,6 +443,15 @@ def find_cycle(name, stack): for col in PREMISE_STATUS_COLUMNS: v = p.get(col) if v is None: + # Absence is not a status, and skipping it was a way around the + # closed vocabulary: deleting both columns from a premise let an + # unconditional or CERTIFIED_RUNG claim rest on it with nothing + # refused. All thirteen committed premises carry both columns, so + # requiring them costs the committed graph nothing. + problems.append( + f"FW-UNCONDITIONAL: {name} carries no {col}. A premise with no status " + f"is not a discharged premise; transcribe the register's word, or " + f"transcribe that the register has none.") continue if v not in PREMISE_DISCHARGED and v not in PREMISE_UNDISCHARGED: problems.append( @@ -402,7 +470,9 @@ def find_cycle(name, stack): continue for col in PREMISE_STATUS_COLUMNS: v = p.get(col) - if v is not None and v not in PREMISE_DISCHARGED: + # `is not None and` was the fail-open: a missing column read as + # nothing to refuse. Absence is not a discharge. + if v not in PREMISE_DISCHARGED: problems.append( f"FW-UNCONDITIONAL: {name} is graded {claim['grade']} but rests on " f"{node}, whose {col} is {v!r} and is not a discharge") @@ -440,7 +510,7 @@ def find_cycle(name, stack): # refusals, because no claim is graded CERTIFIED_RUNG there at all. for col in PREMISE_STATUS_COLUMNS: v = p.get(col) - if v is not None and v not in PREMISE_DISCHARGED: + if v not in PREMISE_DISCHARGED: # absence included, deliberately problems.append( f"FW-RUNG-OPEN-PREMISE: {name} is graded CERTIFIED_RUNG but rests " f"on {node}, whose {col} is {v!r} and is not a discharge") @@ -649,12 +719,28 @@ def unsatisfied_reason(dep: str) -> str | None: if is_claim and node.get("grade") in CERTIFYING_GRADES: views = [evidence_arithmetic(ev, manifest) for ev in evidence] - if views and all(is_float_arithmetic(a) for a, _c, _w in views): + classes = [classify_arithmetic(a) for a, _c, _w in views] + # This asked `all(is_float_arithmetic(...))`, and `is_float_arithmetic` + # answers False for UNRECOGNISED and AMBIGUOUS. So a carrier index whose + # record is prose RESCUED the claim: resolving a float evidence record + # against BINDING's mixed-vocabulary sentence turned FLOAT into AMBIGUOUS, + # the guard read "not all float", and the refusal vanished. Making this + # checker read both carrier indexes is what opened that path, so the fix + # belongs here: a certifying grade must SHOW exactness somewhere, and an + # unreadable declaration is not a way past the requirement. + # + # What this still does NOT refuse, said plainly: a certified claim citing + # one exact record alongside float ones passes. That is the published + # semantics of this firewall and narrowing it is a decision for the + # register, not for a checker. + if views and EXACT not in classes: problems.append( - f"FW-FLOAT-NOT-CERTIFIED: {name} is graded {node['grade']} but every " - f"evidence record it cites is floating point " + f"FW-FLOAT-NOT-CERTIFIED: {name} is graded {node['grade']} but not one " + f"evidence record it cites declares exact arithmetic. The records " + f"classify as {sorted(set(classes))} " f"({sorted({str(a) for a, _c, _w in views})}). A high-precision float " - f"computation is not a certified bound.") + f"computation is not a certified bound, and an arithmetic declaration " + f"this checker cannot read is not one either.") for p in problems: print(p) From 72ccf9f96be4e52e1fe92d10dc3082ee5b180193 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Sun, 27 Sep 2026 18:21:38 +0000 Subject: [PATCH 4/4] Close the carrier-index fail-open and match the README to the checker Review 5848479354: 1. The README said a certified claim 'must cite at least one' exact record; the guard runs only over claims that cite evidence (H3-BAND-FLOOR is FROZEN_CERTIFICATE with evidence: []). The sentence is narrowed to claims that cite evidence, the unexamined case is counted in the summary line as certifying_without_evidence=, and a control pins the count. 2. A carrier index that exists but does not parse is refused (CarrierIndexUnreadable), and an evidence record naming a carrier_id that resolves in neither index is refused; the docstring and README paragraph that said skipping 'can only lose a refusal' are corrected, and test_both_carrier_indexes_absent_is_skipped_cleanly is replaced by controls for the nonexistent-path probe, the unparsable index, an index that omits the carrier, and a graph without carrier_ids. 3. A list-valued status column is refused by name (FW-UNCONDITIONAL) instead of raising TypeError; the two downstream discharge guards tolerate it too. Committed graph: claims=26 premises=13 firewalls=10 enforced=10 evidence_arithmetic_unreadable=1 certifying_without_evidence=1 problems=0. Ran: pytest tests/test_claims.py tests/test_claims_firewalls.py (102 passed); closure_pipeline.py check-plan PASS; consumers_check PASS. Co-authored-by: Dylan Roy --- claims/README.md | 15 +++++--- tests/test_claims_firewalls.py | 69 +++++++++++++++++++++++++++++++-- tools/claims_check.py | 70 +++++++++++++++++++++++++++------- 3 files changed, 132 insertions(+), 22 deletions(-) diff --git a/claims/README.md b/claims/README.md index 27b652c4d..2e4a263f8 100644 --- a/claims/README.md +++ b/claims/README.md @@ -116,7 +116,7 @@ the registers' own technical statuses `NEEDS_RECONCILIATION` and | `FW-DECIMAL-KILL` | the qualitative rate must forbid a finite decimal `C_Q0`; Theorem B must forbid a numerical `C*` | `Q0_MASTER.md` Part I; C092 §12.3 | | `FW-LM011-PRECONDITION` | the RV-LM011 synthesis route may not be marked satisfiable while any named prerequisite is unsatisfied, and a technical pass at **zero** organizational independence credit does not discharge an independence-requiring gate | `review_queue.json` RV-LM011-MAIN; `easy_closure_queue.json` P02-LM-011; `docs/OPEN_PROBLEMS.md` §D | | `FW-NO-RECEIPT-PROMOTION` | a receipt, a green test run, a reproduction or a carrier binding may never raise a grade or move a status, on a claim or on a premise | `engine/README.md`; OP-PROT-012 §4(c); OP-GDN-002 §6 | -| `FW-FLOAT-NOT-CERTIFIED` | high precision is not certification: evidence that declares itself `certifying` must **show** exact arithmetic — float, not-applicable, unrecognised and ambiguous are each refused by name — a certified/enclosed claim must cite **at least one** record declaring exact arithmetic, and the two carrier indexes may not disagree about a carrier without being refused. What it still does **not** refuse, said plainly: a certified claim citing one exact record alongside float ones. Narrowing that is a decision for the register, not for a checker | `engine/README.md`; `engine/rn_engine/BINDING.json`; `README.md` status discipline | +| `FW-FLOAT-NOT-CERTIFIED` | high precision is not certification: evidence that declares itself `certifying` must **show** exact arithmetic — float, not-applicable, unrecognised and ambiguous are each refused by name — a certified/enclosed claim **that cites evidence** must cite at least one record declaring exact arithmetic; an evidence record naming a `carrier_id` must resolve in one of the two carrier indexes, an index that exists but does not parse is refused, and the two indexes may not disagree about a carrier. What it still does **not** refuse, said plainly: a certified claim citing one exact record alongside float ones, and a certifying-grade claim citing **no** evidence at all — that case is not examined, only counted (`certifying_without_evidence=` in the summary line; `H3-BAND-FLOOR` is one today). Narrowing either is a decision for the register, not for a checker | `engine/README.md`; `engine/rn_engine/BINDING.json`; `README.md` status discipline | | `FW-RETRACTED-NOT-UNCONDITIONAL` | a claim carrying a retraction record, or whose transcribed `register_status` says RETRACTED, may not carry an unconditional grade | `ERRATA_AND_CLARIFICATIONS_2026-09-13.md`; `CLAIM_REGISTRY_VERIFIED_INTAKE.json` | Plus referential integrity, acyclicity, "a CONDITIONAL claim must name at least @@ -168,10 +168,15 @@ Reading only the first meant the override resolved nothing: one lookup, one miss every run. For evidence naming a `carrier_id` either index lists, that carrier's own `arithmetic` and `certifying` fields win over the graph's copy, because the carrier's record is what the run actually used, and the refusal names which file -it came from. When both indexes are absent, unreadable, or do not list the -carrier, the graph's own evidence record is used and the lookup is skipped -cleanly. Skipping can only lose a refusal that the graph's own record would have -to state anyway; it cannot manufacture a pass. +it came from. Because the index **overrides** the graph's copy, a lookup that +fails is not harmless: a record whose graph copy said `exact_rational` / +`certifying: true` while the live `BINDING.json` said `mpmath_float` was refused +with the index present and passed with the index pointed at a nonexistent path. +An earlier version of this paragraph said skipping "can only lose a refusal that +the graph's own record would have to state anyway"; that was false. Now an index +file that exists but does not parse is refused, and an evidence record whose +`carrier_id` resolves in neither index is refused. An index file that is simply +absent is tolerated only because the other index may hold the carrier. Activating that override naively would have *weakened* the firewall, which is worth recording. `BINDING.json` describes RNENG-01 in a sentence that reads "no diff --git a/tests/test_claims_firewalls.py b/tests/test_claims_firewalls.py index 257cfe00e..04ffc9b46 100644 --- a/tests/test_claims_firewalls.py +++ b/tests/test_claims_firewalls.py @@ -110,6 +110,7 @@ def test_the_summary_reports_enforced_and_unreadable_counts(): # One record's arithmetic is the BINDING sentence, which is ambiguous by # construction. Reporting it is the point: it is unreadable, not inert. assert "evidence_arithmetic_unreadable=1" in out, out + assert "certifying_without_evidence=1" in out, out # ------------------------------- FW-FLOAT-NOT-CERTIFIED, the certifying field -- @@ -261,12 +262,74 @@ def test_a_carrier_index_overrides_the_graph_and_names_its_source(): assert "FW-FLOAT-NOT-CERTIFIED" in out and "BINDING.json" in out, out -def test_both_carrier_indexes_absent_is_skipped_cleanly(): - """Skipping can only lose a refusal the graph's own record would have to - state anyway. It must not crash and must not manufacture one.""" +def test_a_carrier_id_that_resolves_in_neither_index_is_refused(): + """The old control here asserted the opposite: both indexes absent, exit 0, + "skipping can only lose a refusal". It could lose exactly the refusal that + matters. A carrier index OVERRIDES the graph's copy, so a record whose graph + copy says exact/certifying while the live BINDING says mpmath_float was + refused with the index present and passed with `--binding /nonexistent`.""" missing = "/nonexistent/carrier-index.json" code, out = run(binding=missing, manifest=missing) + assert code == 1, out + assert "FW-FLOAT-NOT-CERTIFIED" in out and "RNENG-01" in out, out + assert "neither engine/rn_engine/BINDING.json nor engine/carriers/MANIFEST.json" in out + # The probe from the review: graph copy upgraded to exact/certifying, index gone. + g = graph() + name, i = CARRIER_EVIDENCE + ev = g["premises"][name]["evidence"][i] + ev["arithmetic"], ev["certifying"] = "exact_rational", True + code, out = run(g, binding=missing, manifest=missing) + assert code == 1, out + assert "resolves nowhere" in out or "neither" in out, out + + +def test_an_index_that_exists_but_does_not_parse_is_refused(): + """Present-and-unparsable is not absent. Returning {} for it handed every + lookup back to the graph's own copy, which the index exists to override.""" + code, out = run(binding="{not json") + assert code == 1, out + assert "cannot be read as JSON" in out, out + # A well-formed index that omits the carrier is the same hole by another door. + code, out = run(binding={"carriers": []}) + assert code == 1, out + assert "neither" in out and "RNENG-01" in out, out + + +def test_a_carrier_absent_from_the_graph_needs_no_index(): + """Evidence that names no carrier_id is judged on the graph's own fields; + the carrier requirement attaches to records that name one.""" + g = graph() + for node in list(g["claims"].values()) + list(g["premises"].values()): + for ev in node.get("evidence") or []: + ev.pop("carrier_id", None) + missing = "/nonexistent/carrier-index.json" + code, out = run(g, binding=missing, manifest=missing) + assert code == 0, out + + +def test_a_list_valued_status_column_is_refused_by_name(): + """This raised TypeError (`list not in set`), fail-closed by traceback.""" + g = graph() + g["premises"]["H5-RIM"]["status_frozen_v2_2"] = ["CLOSED"] + code, out = run(g) + assert code == 1, out + assert "FW-UNCONDITIONAL" in out and "not a string" in out, out + assert "Traceback" not in out + + +def test_certifying_claims_with_no_evidence_are_counted_not_refused(): + """The README once said a certified claim "must cite at least one" exact + record; the guard runs only over claims that cite evidence. H3-BAND-FLOOR is + FROZEN_CERTIFICATE with evidence: [] in the committed graph, so refusing it + is a register decision. The gap is printed instead of hidden.""" + code, out = run() + assert code == 0, out + assert "certifying_without_evidence=1" in out, out + g = graph() + g["claims"]["RN3-FAR"]["evidence"] = [] + code, out = run(g) assert code == 0, out + assert "certifying_without_evidence=2" in out, out # ---------------------------------------- FW-UNCONDITIONAL, both columns -- diff --git a/tools/claims_check.py b/tools/claims_check.py index 73c6accdc..b68c03624 100644 --- a/tools/claims_check.py +++ b/tools/claims_check.py @@ -229,16 +229,25 @@ def load(path: str | None = None) -> dict: return json.load(f) +class CarrierIndexUnreadable(ValueError): + """An index file that exists but cannot be read or parsed.""" + + def load_carrier_manifest(path: str | None = None) -> dict: - """{carrier_id: record} from `engine/carriers/MANIFEST.json`, or {}. - - That manifest is owned by another part of this repository and may not exist. - When it is absent, unreadable or shaped in a way this function does not - recognise, the FW-FLOAT-NOT-CERTIFIED check falls back to the graph's own - evidence records and skips the manifest cleanly. Skipping it can only lose a - refusal that the graph's own record would have to state anyway; it can never - manufacture a pass, because an evidence record that declares float - arithmetic is refused on the graph's own fields. + """{carrier_id: record} from one carrier index, or {} when the file is absent. + + An index owned by another part of this repository may legitimately not exist. + An index that EXISTS and does not parse is a different thing: the earlier + version of this function returned {} for both and its docstring said skipping + "can only lose a refusal that the graph's own record would have to state + anyway". That was false. A carrier index OVERRIDES the graph's copy of + `arithmetic`/`certifying`, so an evidence record whose graph copy says + `exact_rational`/`certifying: true` while the live BINDING says + `mpmath_float` was refused with the index present and passed with it + unparsable. An unreadable index is therefore refused (CarrierIndexUnreadable), + and an evidence record naming a `carrier_id` that resolves in NEITHER index is + refused by the caller. Absence of a file is tolerated only because the other + index may hold the carrier; absence of the carrier is not. """ p = path or MANIFEST if not os.path.exists(p): @@ -246,8 +255,11 @@ def load_carrier_manifest(path: str | None = None) -> dict: try: with open(p, encoding="utf-8") as f: man = json.load(f) - except (OSError, ValueError): - return {} + except (OSError, ValueError) as exc: + raise CarrierIndexUnreadable( + f"carrier index {p} exists but cannot be read as JSON ({exc}). An index that " + f"is present and unparsable would silently hand every carrier lookup back to " + f"the graph's own copy, which the index exists to override.") from exc entries = man if isinstance(man, dict): for field in ("carriers", "entries", "members", "items"): @@ -292,7 +304,12 @@ def carrier_indexes(manifest_path: str | None = None, conflicts: list[str] = [] for path, label in ((binding_path or BINDING, "engine/rn_engine/BINDING.json"), (manifest_path or MANIFEST, "engine/carriers/MANIFEST.json")): - for cid, rec in load_carrier_manifest(path).items(): + try: + loaded = load_carrier_manifest(path) + except CarrierIndexUnreadable as exc: + conflicts.append(f"FW-FLOAT-NOT-CERTIFIED: {exc}") + continue + for cid, rec in loaded.items(): prior = out.get(cid) if prior is not None: old_rec, old_label = prior @@ -377,6 +394,11 @@ def main(argv: list[str] | None = None) -> int: known = set(claims) | set(premises) problems: list[str] = [] unreadable_arithmetic = 0 + # Certifying-grade claims that cite NO evidence are not examined by the + # exact-arithmetic guard below. Refusing them would refuse the committed + # graph (H3-BAND-FLOOR is FROZEN_CERTIFICATE with evidence: []), which is a + # register decision; counting them keeps the gap visible in the summary line. + certifying_without_evidence = 0 # A duplicate carrier id whose declaration disagrees between the two indexes # is refused here rather than resolved by whichever file is read second. problems.extend(f"FW-FLOAT-NOT-CERTIFIED: {c}" for c in carrier_conflicts) @@ -442,6 +464,14 @@ def find_cycle(name, stack): for name, p in premises.items(): for col in PREMISE_STATUS_COLUMNS: v = p.get(col) + if v is not None and not isinstance(v, str): + # A list or object here raised TypeError on the set test below -- + # fail-closed by accident, and with a traceback instead of a + # firewall name. Refuse it by name. + problems.append( + f"FW-UNCONDITIONAL: {name} carries {col} {v!r}, which is not a string. " + f"A status column holds one register word, not a structure.") + continue if v is None: # Absence is not a status, and skipping it was a way around the # closed vocabulary: deleting both columns from a premise let an @@ -472,7 +502,7 @@ def find_cycle(name, stack): v = p.get(col) # `is not None and` was the fail-open: a missing column read as # nothing to refuse. Absence is not a discharge. - if v not in PREMISE_DISCHARGED: + if not isinstance(v, str) or v not in PREMISE_DISCHARGED: problems.append( f"FW-UNCONDITIONAL: {name} is graded {claim['grade']} but rests on " f"{node}, whose {col} is {v!r} and is not a discharge") @@ -510,7 +540,7 @@ def find_cycle(name, stack): # refusals, because no claim is graded CERTIFIED_RUNG there at all. for col in PREMISE_STATUS_COLUMNS: v = p.get(col) - if v not in PREMISE_DISCHARGED: # absence included, deliberately + if not isinstance(v, str) or v not in PREMISE_DISCHARGED: # absence included, deliberately problems.append( f"FW-RUNG-OPEN-PREMISE: {name} is graded CERTIFIED_RUNG but rests " f"on {node}, whose {col} is {v!r} and is not a discharge") @@ -652,6 +682,8 @@ def unsatisfied_reason(dep: str) -> str | None: for name, node in list(claims.items()) + list(premises.items()): evidence = node.get("evidence") if not evidence: + if name in claims and node.get("grade") in CERTIFYING_GRADES: + certifying_without_evidence += 1 continue is_claim = name in claims for i, ev in enumerate(evidence): @@ -659,6 +691,15 @@ def unsatisfied_reason(dep: str) -> str | None: problems.append( f"{name}: evidence[{i}] kind {ev.get('kind')!r} is not in the evidence " f"vocabulary {sorted(EVIDENCE_KINDS)}") + cid = ev.get("carrier_id") + if cid is not None and cid not in manifest: + problems.append( + f"FW-FLOAT-NOT-CERTIFIED: {name} evidence[{i}] names carrier_id {cid!r}, " + f"which neither engine/rn_engine/BINDING.json nor " + f"engine/carriers/MANIFEST.json lists. A carrier record is what the run " + f"actually used and overrides the graph's copy; a carrier that resolves " + f"nowhere leaves the graph's copy unchecked, which is the fail-open this " + f"firewall exists to close.") arithmetic, certifying, where = evidence_arithmetic(ev, manifest) kind = classify_arithmetic(arithmetic) if kind in (UNRECOGNISED, AMBIGUOUS): @@ -747,6 +788,7 @@ def unsatisfied_reason(dep: str) -> str | None: print(f"claims={len(claims)} premises={len(premises)} " f"firewalls={len(g['firewalls'])} enforced={len(ENFORCED_FIREWALLS)} " f"evidence_arithmetic_unreadable={unreadable_arithmetic} " + f"certifying_without_evidence={certifying_without_evidence} " f"problems={len(problems)}") return 1 if problems else 0