Repository navigation
fix(cbz): repair G11 mutation harness target matching - #720
Conversation
|
FINITE PERSONAL ENGINEERING REVIEW PICKUP — GTW-MAIN720-G11-MUTATION-TARGET-REVIEW-GPT6-20261030T0527CDT Actual reviewer: OpenAI / ChatGPT GPT-6, GitHub Task Watch scheduled automation invocation 11. GitHub exposes only shared principal Canonical object and exact source cut
Finite review questions / outputs
Dependencies and contention
Exclusions Authority carried: current Main AGENTS.md · current CBZ-0 governance · complete owner mandate · standing delegation · current workflow · 25 October review-family decision · owner master #686/6511339492. Link delivery is not acknowledgement. |
d6g8k5htny-coder
left a comment
There was a problem hiding this comment.
AMEND_SCOPED_ENGINEERING — GTW-MAIN720-G11-MUTATION-TARGET-REVIEW-GPT6-20261030T0527CDT
Disposition: AMEND_SCOPED_ENGINEERING / P1_STALE_MUTATION_INVENTORY_DIGEST / HOSTED_BASELINE_RED / MUTATION_CAMPAIGN_NOT_REACHED. Exact reviewed base/head: 11e28815163e15223e8827fe6ff47a1faaf36ebd / 2aa6b26e1bb0964f4bd9b1a4b640ecd617ef9c95; tree ccbeab55149e8062713102c078e72e08ec0d1f60; tested synthetic merge d468d760433922c6572f7ffeb3592bbd4409204f (parents exactly base + head). This is a source-exposed same-family engineering review only; organizational-independence credit 0; scientific effect NONE. No security/workflow certification, Layer-A, proof, theorem, tool-admission or CBZ-0 conclusion is made.
Finding — P1: test-side mutation inventory digest does not bind the final source
At exact candidate head:
tools/cbz/g11_mutation_selftest.pyblobc7f74bb270e7dafa2b538ab8f550a72cae5a2f2f.- The exact byte slice used by the test, from
MUTATIONS = [up to but excluding\n\n\ndef _build_cases, hashes toec3a19f1e5213a937e09119f29d63fec6d6bdda2aa9415965201d44587a0ec5e. tests/test_g11_census.pyblobfb400aca9c58e10cf1043bdc8d25f443ff46bd84still declaresf4e4e5226705746d6ae7625c59efab54410d9fabf7622e11bd99889b2d7aee7aat line 20.- The test compares those exact values at lines 520–528. The mismatch is not speculative: hosted run 41040319012, job 124011220547, checked out the current synthetic merge and failed
test_mutation_inventory_is_sealedat line 524 with the same actual/expected digests.
This is a hard baseline failure. In the same exact hosted run, 38 tests ran with one failure; optimized census controls and both mutation campaigns were skipped. Therefore the current run establishes no 75-case mutation execution, no intended kills and no G11 auditor pass.
Scoped checks that passed at source level
- The two names
log_message context ignoredandlog_message context too broadeach occur once in the 75-entry literal inventory. - Their shared preimage is the exact two-line multiline census condition at
tools/cbz/g11_census.pylines 316–317 and occurs once in that source. - The two replacements remain distinct:
if False:disables the context; the other removes thefunc.name == "log_message"guard and broadens the context. The harness'ssource.count(old) == 1check enforces one target occurrence before replacement. - Native compare from frozen r2
6a91249d77b4e7e3470cfa38566576f9ba2efb8cto r3 confirms exactly two changed paths and two ordinary commits, zero behind. The inherited census, documentation and workflow blobs remain unchanged in r3. - These are static source observations only, not execution evidence.
Required author repair / acceptance boundary
Recompute the expected digest from the final exact literal inventory slice after the final source correction, update only the appropriate test constant, and rerun the exact current PR workflow. Required next evidence remains: 38-test normal and optimized baselines with zero failures/errors/skips, followed by the complete 75-case mutation campaign in both modes with all nine harness-target mutations actually applied and every intended kill reason-bound. Preserve the current RED and all per-case results; no skipped job or author replay is a pass.
This review is a COMMENT, not an approval. No Ready, merge, G11 gate, Layer-A, tool-admission, mathematical, scientific or CBZ-0 promotion is authorized. The PR remains Draft. This closes only reviewer pickup #720/6520544270; it does not release the writer's #684/6520283287 scope or Root's canonical registration/integration.
Authority carried: current Main AGENTS.md · current CBZ-0 governance · complete owner mandate · standing delegation · current workflow · 25 October review-family decision · owner master #686/6511339492. Link delivery is not acknowledgement.
|
RESULT / ONLY-OWN REVIEWER RELEASE — Canonical native exact-head COMMENT review: review 5711070475, read back as Disposition: Fresh hosted evidence is exact current PR run 41040319012, job I release only reviewer pickup #720/6520544270. The Research Task Loop writer pickup #684/6520283287 and its sequence amendment #684/6520398531 remain untouched; Root's canonical registration/integration remains separate. PR #720 remains Draft and unmerged. No source, branch, check, workflow, protection, Ready, proof, theorem, scientific or Layer-A state was changed. Independence credit 0; scientific effect NONE. Authority carried: current Main AGENTS.md · current CBZ-0 governance · complete owner mandate · standing delegation · current workflow · 25 October review-family decision · owner master #686/6511339492. Link delivery is not acknowledgement. |
|
RESULT / BLOCKED SOURCE CORRECTION / ONLY-OWN SUCCESSOR RELEASE — Actual performer: OpenAI / ChatGPT GPT-6, Research Task Loop scheduled automation invocation 11. GitHub exposes only the shared principal Canonical task and custody
Exact current candidate and finding
Fresh hosted check readback (not a rerun)
Stop and release Disposition: I release only this finite successor pickup #684/6520283287. The predecessor's UNKNOWN closeout, all five-path history, reviewer finding, failed/skipped hosted outcomes, original hypotheses and negative controls remain unchanged. No G11, Layer-A, proof, theorem or CBZ-0 status promotion. Next admissible step is the same two-path correction only after supported clearance of the reported platform denial, followed by the unchanged 38-test normal/optimized baselines, full 75-case normal/optimized campaign, nine harness-target application checks and fresh exact-head engineering review. The separate navigation failure must also be resolved on its own evidence. Authority: current Main AGENTS.md · current Math- AGENTS.md · complete owner mandate · standing delegation · OP-WORKFLOW · review-family decision. Link delivery is not acknowledgement. |
|
PICKUP and CHECKPOINT on #720 (G11 harness), patch-only. 01SHCrxv (Anthropic Claude), a different family from the GPT-6 author lanes. Credit 0. Scientific effect NONE. Scope. I will offer a patch in a comment: no push, no merge, no Ready. Files: Why the hosted check has never been green (all 26 runs of this workflow failed). Run locally at
State of the working patch: 77 cases, which are the 75 plus the two controls. 38 tests OK in each mode. The campaign executes 77 of 77 and kills 64, with the same set in both modes. The 13 survivors, by kind.
Next. I will work through the 13 and post the full patch, with a reason for each mutant: re-anchor, re-designate, or a fixture line. Any change to what the auditor accepts will be listed separately for the author lane to decide. The separate Generated by Claude Code |
|
RESULT on #720: a patch that turns the G11 campaign green, with 75 of 75 mutants killed in each mode. My pickup (6541676227) is released. 01SHCrxv (Anthropic Claude), a different family from the GPT-6 author lanes. Credit 0. Scientific effect NONE. Nothing was pushed, and the PR stays Draft. The author lane, or any lane with push rights on this branch, can apply the patch below with Verified locally. The patch was applied with
Python 3.12 gives the same 75 of 75; that is a portability check only. Part A: harness mechanics. These do not change what the auditor accepts. Findings F1–F5 are in my checkpoint.
Part B: the 13 mutants that survived once the harness ran.
For the author lane to decide. Everything else in the patch is mechanical.
Observation, not fixed (P3). Apply. Save the block below as p720.patch (3 files, +74 −46)From 0c9049df04c25ad308047bc798cc1908065c2320 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Sat, 31 Oct 2026 09:35:08 +0000
Subject: [PATCH] fix(cbz): make the G11 mutation campaign run and credit kills
under Python 3.11
No hosted run of this workflow reached a mutant:
- reseal the mutation inventory digest;
- re-anchor eleven stale targets: four census targets changed at
9dbc9760, and seven one-line harness targets also matched their own
declaration literal; update one stale designated fragment;
- parse Python 3.11+ failure headers, "method (module.Class.method)",
with or without a subTest suffix. The old parser repeated the method,
so no kill could be credited;
- swap the old and new strings of "FAIL header mistaken for exception",
which edited its own declaration instead of the regex.
With the harness running, 13 mutants survived. Each now fails its
designated assertion. Three are re-designated to the assertion that
detects them first. Eight get a fixture line, a two-line anchor or a
name-based mutation, so the designated test can tell them apart. Two
equivalent mutants are dropped, because exception bases confer no stub
exemption. Two header controls are added: 75 cases, all 75 killed in
normal and optimized mode.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHCrxvZuVwzVNC7SDC9TAj
---
docs/cbz/G11_AUDITOR.md | 2 +-
tests/test_g11_census.py | 32 ++++++++---
tools/cbz/g11_mutation_selftest.py | 86 +++++++++++++++++-------------
3 files changed, 74 insertions(+), 46 deletions(-)
diff --git a/docs/cbz/G11_AUDITOR.md b/docs/cbz/G11_AUDITOR.md
index 1b9bf76a..1d527e25 100644
--- a/docs/cbz/G11_AUDITOR.md
+++ b/docs/cbz/G11_AUDITOR.md
@@ -23,7 +23,7 @@ The census exits 0 for no detected production finding, 1 for a detected producti
- Stub exemptions are bound to source-resolved imports, not decorator or base names. An imported `abc.abstractmethod` exempts only a method in an immediate class with a trusted imported `abc.ABC` base. Imported `typing.overload`/`typing_extensions.overload` exempts only declarations in one contiguous same-name overload group followed by a concrete, non-stub implementation. Immediate imported `typing.Protocol`/`typing_extensions.Protocol` interface methods and the named `http.server` request-handler `log_message` override remain contextual. ABC inheritance alone and exception-class inheritance do not exempt a method. Nested outer-class context, unpaired overloads, name suffixes, local look-alikes, and lexical bindings shadowing an imported decorator/base do not grant exemptions.
- Python reference evidence is limited to module-level statements and classes whose base resolves to the imported `unittest.TestCase` identity (including explicit aliases), following only locally reachable helper calls and literal branches; a local class merely named `TestCase` is not credited. Uncalled helpers, `if False`, runtime-dependent branches, and loop bodies are not credited. Workflow references ignore here-document bodies and are read from run commands, not YAML comments. Python/shell `-c` payloads and JavaScript template-literal text are not credited. These are static references only, never proof of runtime execution, and cannot satisfy a runtime-exercise requirement.
- YAML comments are scanned for marker words. Empty and comment-only `run` blocks are reported as no-op steps.
-- The mutation self-test preserves all 74 original cases and adds one F4 decoy-`TestCase` control (75 total). It seals the exact source slice containing `MUTATIONS`, `TARGETS`, `NEW_MUTATIONS` and `EXPECTED_MUTATION_NAMES` with a test-side SHA-256, then compares complete literal case tuples—not names alone—against the harness source. A kill is admitted only when the clean full 38-test baseline succeeds and the exact designated module/class/method reports `FAILED` with one AssertionError at its exact designated assertion line, zero errors, and zero skips. The harness propagates the parent's optimized mode to child tests and applies each mutation to its declared source, including all harness-target cases.
+- The mutation self-test has 75 cases: 72 of the 74 original cases, one F4 decoy-`TestCase` control, and two controls for the Python 3.11+ unittest failure header `method (module.Class.method)`, with and without a `subTest` suffix. Two original cases were dropped as equivalent mutants: exception bases confer no stub exemption, so neither a decoy `…Error` base name nor an ignored shadow of a builtin exception name can change a report. It seals the exact source slice containing `MUTATIONS`, `TARGETS`, `NEW_MUTATIONS` and `EXPECTED_MUTATION_NAMES` with a test-side SHA-256, then compares complete literal case tuples—not names alone—against the harness source. A kill is admitted only when the clean full 38-test baseline succeeds and the exact designated module/class/method reports `FAILED` with one AssertionError at its exact designated assertion line, zero errors, and zero skips. The harness propagates the parent's optimized mode to child tests and applies each mutation to its declared source, including all harness-target cases.
## Limits
diff --git a/tests/test_g11_census.py b/tests/test_g11_census.py
index fb400aca..e025ed88 100644
--- a/tests/test_g11_census.py
+++ b/tests/test_g11_census.py
@@ -17,7 +17,7 @@ sys.path.insert(0, str(ROOT / "tools" / "cbz"))
import g11_census as g11 # noqa: E402
import g11_mutation_selftest as g11m # noqa: E402
-EXPECTED_MUTATION_INVENTORY_SHA256 = "f4e4e5226705746d6ae7625c59efab54410d9fabf7622e11bd99889b2d7aee7a"
+EXPECTED_MUTATION_INVENTORY_SHA256 = "333f981944fbd08389c9fb384d6cd0dbf89fff4e40514ca67b123a0057be5d77"
DOCUMENTED_TESTED = {
"tools/README.md": b"Run `tools/good.py` to check things.\n",
@@ -179,7 +179,7 @@ class ToolChecks(unittest.TestCase):
def test_documented_and_exercised_by_path_import_and_basename(self):
tree = {"tools/a.py": "x = 1\n", "tools/b.py": "x = 1\n", "tools/c.mjs": "export const x = 1;\n", "tools/d.py": "x = 1\n",
- "docs/USE.md": "See tools/a.py, b.py, c.mjs and d.py.\n",
+ "docs/USE.md": "See tools/a.py, tools/b.py, tools/c.mjs and d.py.\n",
"tests/test_a.py": "from tools import helper_x, a\n",
"tests/test_d.py": "import d\n",
".github/workflows/ci.yml": " - run: python tools/b.py\n",
@@ -289,7 +289,7 @@ class CommandLine(unittest.TestCase):
class AdversarialControls(unittest.TestCase):
def test_decoy_names_do_not_exempt_stubs(self):
text = (
- "from abc import abstractmethod as am\nimport typing as t\n"
+ "from abc import ABC as RealABC, abstractmethod as am\nimport typing as t\n"
"def abstractmethod(fn): return fn\n"
"class Fake:\n @staticmethod\n def abstractmethod(fn): return fn\n"
"class Protocol: pass\nclass ABC: pass\nclass ValueError: pass\n"
@@ -298,29 +298,39 @@ class AdversarialControls(unittest.TestCase):
"class NotReallyError:\n def fail(self):\n pass\n"
"class NotReallyProtocol:\n def fail(self):\n ...\n"
"class NotReallyABC:\n def fail(self):\n pass\n"
- "class NotReallyRequestHandler:\n def log_message(self, *args):\n pass\n"
+ "class StreamRequestHandler: pass\n"
+ "class NotReallyRequestHandler(StreamRequestHandler):\n def log_message(self, *args):\n pass\n"
"class DecoyProtocol(Protocol):\n def proto(self):\n pass\n"
"class DecoyABC(ABC):\n def abc(self):\n pass\n"
"class ShadowedBuiltin(ValueError):\n def builtin(self):\n pass\n"
"class Genuine:\n @am\n def abstract(self):\n pass\n"
"class Shape(t.Protocol):\n def area(self):\n ...\n"
+ "class RealBase(RealABC):\n @abstractmethod\n def decoy_method(self):\n pass\n"
)
report = run({"tools/good.py": text, **DOCUMENTED_TESTED})
self.assertEqual(sorted(f["detail"].split(":")[0] for f in report["failing"]),
- ["abc", "abstract", "builtin", "decoy_decorator", "decoy_qualified",
+ ["abc", "abstract", "builtin", "decoy_decorator", "decoy_method", "decoy_qualified",
"fail", "fail", "fail", "log_message", "proto"])
self.assertEqual(report["status"], "FAIL")
local_shadow = (
"from abc import abstractmethod\n"
+ "from typing import overload\n"
"def factory():\n"
" abstractmethod = lambda fn: fn\n"
" @abstractmethod\n"
" def local_decoy():\n"
" pass\n"
+ "def overloads():\n"
+ " overload = lambda fn: fn\n"
+ " @overload\n"
+ " def local_overload(x: int): ...\n"
+ " def local_overload(x):\n"
+ " return x\n"
)
local_report = run({"tools/good.py": local_shadow, **DOCUMENTED_TESTED})
self.assertEqual([(f["class"], f["detail"]) for f in local_report["findings"]
- if f["class"].startswith("stub")], [("stub", "local_decoy: pass")])
+ if f["class"].startswith("stub")],
+ [("stub", "local_decoy: pass"), ("stub", "local_overload: ellipsis")])
def test_comment_and_string_mentions_do_not_exercise_tools(self):
tree = {
@@ -339,7 +349,8 @@ class AdversarialControls(unittest.TestCase):
})
self.assertFalse(python_cmd["tools"]["tools/runner.py"]["referenced"])
- workflow_text = "jobs:\n test:\n steps:\n - run: \"sh -c 'echo tools/runner.py'\"\n"
+ workflow_text = ("jobs:\n test:\n steps:\n - run: \"sh -c 'echo tools/runner.py'\"\n"
+ " - run: python3 -c tools/runner.py\n")
shell_cmd = run({
"tools/runner.py": "def main(): return 0\n",
"tools/README.md": "Use tools/runner.py.\n",
@@ -404,6 +415,12 @@ class MutationAttribution(unittest.TestCase):
parsed = g11m._parse_run(proc)
self.assertEqual(parsed["test_id"], expected_parser)
self.assertTrue(g11m.qualifies_kill(parsed, expected_parser, 123))
+ current = output.replace("(tests.test_g11_census.MutationAttribution)", "(%s)" % expected_parser)
+ parsed_current = g11m._parse_run(subprocess.CompletedProcess(["python"], 1, current, ""))
+ self.assertEqual(parsed_current["test_id"], expected_parser)
+ subtest = current.replace("(%s)\n" % expected_parser, "(%s) (tool='tools/d.py')\n" % expected_parser)
+ parsed_subtest = g11m._parse_run(subprocess.CompletedProcess(["python"], 1, subtest, ""))
+ self.assertEqual(parsed_subtest["test_id"], expected_parser)
@@ -429,6 +446,7 @@ class ReviewResidualControls(unittest.TestCase):
" def exception_method(self): ...\n"
"def build(ABC, Exception, BaseHTTPRequestHandler):\n"
" class ShadowedABC(ABC):\n"
+ " @abstractmethod\n"
" def shadowed_abc(self): ...\n"
" class ShadowedError(Exception):\n"
" def shadowed_exception(self): ...\n"
diff --git a/tools/cbz/g11_mutation_selftest.py b/tools/cbz/g11_mutation_selftest.py
index c7f74bb2..27688671 100644
--- a/tools/cbz/g11_mutation_selftest.py
+++ b/tools/cbz/g11_mutation_selftest.py
@@ -54,8 +54,8 @@ MUTATIONS = [
' if _http_request_handler_base(base, imports, scope_shadowed):'),
("module docstring not recorded", " docstring = bool(ast.get_docstring(ast.parse(files[path])))",
" docstring = True"),
- ("class bodies not visited", " visit(child, classes + [child], scope_shadowed | _scope_bindings(child))", " pass"),
- ("nested functions not visited", " visit(child, [], scope_shadowed | _scope_bindings(child))\n else:",
+ ("class bodies not visited", " visit(child, classes + [child], scope_shadowed | _scope_bindings(child), child.body)", " pass"),
+ ("nested functions not visited", " visit(child, [], scope_shadowed | _scope_bindings(child), child.body)\n else:",
" pass\n else:"),
("comments read as data", " if tok.type == tokenize.COMMENT:\n target = markers",
" if tok.type == tokenize.COMMENT:\n target = mentions"),
@@ -79,8 +79,9 @@ MUTATIONS = [
("block scalar not read", ' if not value or value[0] in "|>":', " if False:"),
("block scalar ends late", " if nxt.strip() and len(nxt) - len(nxt.lstrip()) <= key_column:",
" if nxt.strip() and len(nxt) - len(nxt.lstrip()) < key_column:"),
- ("inline comment kept", " commands = [_strip_shell_comment(value).strip()]",
- " commands = [value.strip()]"),
+ ("inline comment kept",
+ " commands = [_strip_shell_comment(value).strip()]\n commands = [_strip_shell_comment(c).strip() for c in commands]",
+ " commands = [value.strip()]\n commands = [c.strip() for c in commands]"),
("tests counted as tools", ' and area(p) == "tooling" and not p.endswith("__init__.py"))',
' and True and not p.endswith("__init__.py"))'),
("__init__ counted as tool", ' and area(p) == "tooling" and not p.endswith("__init__.py"))',
@@ -116,7 +117,7 @@ MUTATIONS = [
RESULT = re.compile(r"^(FAILED|OK)\b(?: \(([^)]*)\))?\s*$", re.M)
SUMMARY = re.compile(r"Ran (\d+) tests? in [^\n]+")
-FAIL_HEADER = re.compile(r"^FAIL: ([^(]+) \(([^)]+)\)$", re.M)
+FAIL_HEADER = re.compile(r"^FAIL: ([^(]+) \(([^)]+)\)(?: \[.*\])?(?: \(.*\))?$", re.M)
TRACE_FRAME = re.compile(r'File "([^"]*test_g11_census\.py)", line (\d+), in ([A-Za-z_]\w*)')
EXCEPTION = re.compile(r"(?m)^([A-Za-z_][A-Za-z0-9_]*(?:Error|Exception|Warning|Interrupt|Exit|StopIteration)):")
@@ -156,7 +157,7 @@ TARGETS = {
"inline comment kept": ("tests.test_g11_census.Workflows.test_noop_steps", "self.assertEqual(found,"),
"tests counted as tools": ("tests.test_g11_census.ToolChecks.test_tests_and_init_files_are_not_tools", "self.assertEqual(sorted(report),"),
"__init__ counted as tool": ("tests.test_g11_census.ToolChecks.test_tests_and_init_files_are_not_tools", "self.assertEqual(sorted(report),"),
- "basename ambiguity ignored": ("tests.test_g11_census.ToolChecks.test_ambiguous_basename_needs_the_full_path", 'self.assertFalse(report["tools/run.py"]["documented"])'),
+ "basename ambiguity ignored": ("tests.test_g11_census.ToolChecks.test_ambiguous_basename_needs_the_full_path", 'self.assertFalse(ambiguous["tools/run.py"]["documented"])'),
"path reference not recognised": ("tests.test_g11_census.ToolChecks.test_full_path_counts_even_when_basename_is_ambiguous", 'self.assertTrue(report["tools/run.py"]["documented"])'),
"import not recognised": ("tests.test_g11_census.ToolChecks.test_documented_and_exercised_by_path_import_and_basename", 'self.assertTrue(report[path]["referenced"])'),
"from-import list not recognised": ("tests.test_g11_census.ToolChecks.test_documented_and_exercised_by_path_import_and_basename", 'self.assertTrue(report[path]["referenced"])'),
@@ -165,9 +166,9 @@ TARGETS = {
"site not production": ("tests.test_g11_census.Markers.test_site_marker_fails_and_vendor_marker_does_not", 'self.assertEqual(run({"docs/site/app.mjs":'),
"research counted as production": ("tests.test_g11_census.PythonStubs.test_production_stub_fails_but_research_stub_does_not", 'self.assertEqual(research["status"],'),
"vendor counted as site": ("tests.test_g11_census.Markers.test_site_marker_fails_and_vendor_marker_does_not", 'self.assertEqual(run({"docs/site/vendor/lib.js":'),
- "undecodable decoded leniently": ("tests.test_g11_census.FailClosed.test_undecodable_and_unparseable_production_files_fail", 'self.assertEqual(report["status"], "FAIL")'),
+ "undecodable decoded leniently": ("tests.test_g11_census.FailClosed.test_undecodable_and_unparseable_production_files_fail", 'self.assertEqual([f["class"] for f in report["failing"]], [expected])'),
"unparseable not a failure class": ("tests.test_g11_census.FailClosed.test_undecodable_and_unparseable_production_files_fail", 'self.assertEqual(report["status"], "FAIL")'),
- "legitimate context still fails": ("tests.test_g11_census.PythonStubs.test_legitimate_contexts_are_reported_but_do_not_fail", 'self.assertEqual(sorted(f["detail"] for f in report["failing"]), ["__init__: pass", "m: ellipsis"])'),
+ "legitimate context still fails": ("tests.test_g11_census.PythonStubs.test_legitimate_contexts_are_reported_but_do_not_fail", "self.assertEqual(contexts,"),
"markers do not fail": ("tests.test_g11_census.Markers.test_production_marker_fails", 'self.assertEqual(report["status"], "FAIL")'),
"noop steps do not fail": ("tests.test_g11_census.Workflows.test_production_noop_fails", 'self.assertEqual(run({".github/workflows/w.yml": " - run: true'),
"tool findings do not fail": ("tests.test_g11_census.ToolChecks.test_undocumented_and_unexercised_tools_fail", 'self.assertEqual(result["status"], "FAIL")'),
@@ -177,14 +178,13 @@ TARGETS = {
NEW_MUTATIONS = [
- ("decoy decorator names accepted", "census", " if _trusted_decorator(dec, imports, scope_shadowed):", " if _decorator_name(dec) in LEGIT_DECORATORS:",
+ ("decoy decorator names accepted", "census",
+ " if _trusted_decorator(dec, imports, scope_shadowed):\n resolved = _resolve_scoped_import(dec, imports, scope_shadowed)",
+ ' if _decorator_name(dec) in LEGIT_DECORATORS:\n resolved = ("abc", "abstractmethod") if _decorator_name(dec) == "abstractmethod" else ("typing", "overload")',
"tests.test_g11_census.AdversarialControls.test_decoy_names_do_not_exempt_stubs", 'self.assertEqual(sorted(f["detail"].split(":")[0] for f in report["failing"]),'),
("decoy abstract-base names accepted", "census", ' if base_kind == "Protocol":', " if _base_name(base) in LEGIT_BASES:",
"tests.test_g11_census.AdversarialControls.test_decoy_names_do_not_exempt_stubs", 'self.assertEqual(sorted(f["detail"].split(":")[0] for f in report["failing"]),'),
- ("decoy exception suffix accepted", "census", " if _builtin_exception_base(base, imports, scope_shadowed):",
- ' if _base_name(base).endswith(("Error", "Exception", "Warning")):',
- "tests.test_g11_census.AdversarialControls.test_decoy_names_do_not_exempt_stubs", 'self.assertEqual(sorted(f["detail"].split(":")[0] for f in report["failing"]),'),
- ("decoy request-handler suffix accepted", "census", ' if _http_request_handler_base(base, imports) and func.name == "log_message":',
+ ("decoy request-handler suffix accepted", "census", ' if (_http_request_handler_base(base, imports, scope_shadowed)\n and func.name == "log_message"):',
' if _base_name(base).endswith("RequestHandler") and func.name == "log_message":',
"tests.test_g11_census.AdversarialControls.test_decoy_names_do_not_exempt_stubs", 'self.assertEqual(sorted(f["detail"].split(":")[0] for f in report["failing"]),'),
("comment and string mentions count as exercise", "census",
@@ -196,12 +196,8 @@ NEW_MUTATIONS = [
"tests.test_g11_census.AdversarialControls.test_empty_and_comment_only_run_blocks_fail", "self.assertEqual(found,"),
("YAML comments omitted", "census", ' elif path.endswith((".yml", ".yaml")):', " elif False:",
"tests.test_g11_census.AdversarialControls.test_yaml_comments_are_scanned_for_markers", "self.assertEqual(classes(report),"),
- ("shadowed builtin exception name accepted", "census",
- 'elif (isinstance(node, ast.Name) and node.id in builtins.__dict__\n and node.id not in imports["shadowed"] and node.id not in scope_shadowed):',
- 'elif isinstance(node, ast.Name) and node.id in builtins.__dict__:',
- "tests.test_g11_census.AdversarialControls.test_decoy_names_do_not_exempt_stubs",
- 'self.assertEqual(sorted(f["detail"].split(":")[0] for f in report["failing"]),'),
- ("mutation attribution accepts wrong assertion", "harness", ' and outcome.get("traceback_line") == expected_line', " and True",
+ ("mutation attribution accepts wrong assertion", "harness", ' and outcome.get("traceback_line") == expected_line\n and outcome.get("exception_type")',
+ ' and True\n and outcome.get("exception_type")',
"tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
"self.assertFalse(g11m.qualifies_kill(wrong_line, expected, 17))"),
("function-local decorator shadow accepted", "census",
@@ -215,8 +211,8 @@ NEW_MUTATIONS = [
"tests.test_g11_census.AdversarialControls.test_comment_and_string_mentions_do_not_exercise_tools",
'self.assertFalse(python_cmd["tools"]["tools/runner.py"]["referenced"])'),
("shell -c path mention counts as execution", "census",
- ' if "-c" in tokens[1:]:\n continue',
- " if False:\n continue",
+ ' if "-c" not in tokens[1:]:',
+ " if True:",
"tests.test_g11_census.AdversarialControls.test_comment_and_string_mentions_do_not_exercise_tools",
'self.assertFalse(shell_cmd["tools"]["tools/runner.py"]["referenced"])'),
("template literal import text counts as static import", "census",
@@ -225,28 +221,38 @@ NEW_MUTATIONS = [
"tests.test_g11_census.AdversarialControls.test_comment_and_string_mentions_do_not_exercise_tools",
'self.assertFalse(js_cmd["tools"]["tools/runner.mjs"]["referenced"])'),
("parser drops method identity", "harness",
- ' result["test_id"] = header.group(2) + "." + header.group(1)',
- ' result["test_id"] = header.group(2)',
+ ' test_id += "." + method\n result["test_id"] = test_id',
+ ' pass\n result["test_id"] = test_id',
"tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
'self.assertEqual(parsed["test_id"], expected_parser)'),
+ ("parser repeats qualified method identity", "harness",
+ ' if not test_id.endswith("." + method):\n test_id += "." + method',
+ ' if True:\n test_id += "." + method',
+ "tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
+ 'self.assertEqual(parsed_current["test_id"], expected_parser)'),
+ ("subTest failure header not parsed", "harness",
+ 'FAIL_HEADER = re.compile(r"^FAIL: ([^(]+) \\(([^)]+)\\)(?: \\[.*\\])?(?: \\(.*\\))?$", re.M)\nTRACE_FRAME',
+ 'FAIL_HEADER = re.compile(r"^FAIL: ([^(]+) \\(([^)]+)\\)$", re.M)\nTRACE_FRAME',
+ "tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
+ 'self.assertEqual(parsed_subtest["test_id"], expected_parser)'),
("wrong test id credited", "harness",
- ' and outcome.get("test_id") == expected_test',
- " and True",
+ ' and outcome.get("test_id") == expected_test\n and outcome.get("traceback_method")',
+ ' and True\n and outcome.get("traceback_method")',
"tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
"self.assertFalse(g11m.qualifies_kill(wrong_test, expected, 17))"),
("nonfailure status credited", "harness",
- ' and outcome.get("status") == "FAILED"',
- " and True",
+ ' and outcome.get("status") == "FAILED"\n and outcome.get("exit")',
+ ' and True\n and outcome.get("exit")',
"tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
"self.assertFalse(g11m.qualifies_kill(not_failed, expected, 17))"),
("error outcome credited", "harness",
- ' and outcome.get("errors") == 0',
- " and True",
+ ' and outcome.get("failures") == 1\n and outcome.get("errors") == 0',
+ ' and outcome.get("failures") == 1\n and True',
"tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
"self.assertFalse(g11m.qualifies_kill(has_errors, expected, 17))"),
("skipped outcome credited", "harness",
- ' and outcome.get("skipped") == 0',
- " and True",
+ ' and outcome.get("skipped") == 0\n and outcome.get("test_id")',
+ ' and True\n and outcome.get("test_id")',
"tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
"self.assertFalse(g11m.qualifies_kill(has_skips, expected, 17))"),
("quoted workflow command boundary ignored", "census",
@@ -255,14 +261,14 @@ NEW_MUTATIONS = [
"tests.test_g11_census.AdversarialControls.test_comment_and_string_mentions_do_not_exercise_tools",
"self.assertEqual(g11._workflow_commands(workflow_text)[0][1], [\"sh -c 'echo tools/runner.py'\"])"),
("FAIL header mistaken for exception", "harness",
- 'EXCEPTION = re.compile(r"(?m)^([A-Za-z_][A-Za-z0-9_]*):")',
- 'EXCEPTION = re.compile(r"(?m)^([A-Za-z_][A-Za-z0-9_]*(?:Error|Exception|Warning|Interrupt|Exit|StopIteration)):")',
+ 'EXCEPTION = re.compile(r"(?m)^([A-Za-z_][A-Za-z0-9_]*(?:Error|Exception|Warning|Interrupt|Exit|StopIteration)):")\n\n\nclass HarnessError',
+ 'EXCEPTION = re.compile(r"(?m)^([A-Za-z_][A-Za-z0-9_]*):")\n\n\nclass HarnessError',
"tests.test_g11_census.MutationAttribution.test_mutation_kill_requires_exact_target_assertion",
"self.assertTrue(g11m.qualifies_kill(parsed, expected_parser, 123))"),
("outer class exemption leaks", "census",
" for cls in classes[-1:]:", " for cls in reversed(classes):",
"tests.test_g11_census.ReviewResidualControls.test_stub_exemptions_are_immediate_and_scope_safe",
- 'self.assertIn("inner_abc", stub_names)'),
+ "self.assertIn(name, stub_names)"),
("local base shadow accepted", "census",
" if binding and binding in scope_shadowed:\n return None",
" if False:\n return None",
@@ -282,7 +288,7 @@ NEW_MUTATIONS = [
"tests.test_g11_census.ReviewResidualControls.test_mutation_inventory_is_sealed",
'self.assertRaisesRegex(g11m.HarnessError, "mutation inventory")'),
("hostile Python environment survives", "harness",
- " for name in PYTHON_ENVIRONMENT_KEYS:", " for name in ():",
+ " for name in PYTHON_ENVIRONMENT_KEYS:\n env.pop(name, None)", " for name in ():\n env.pop(name, None)",
"tests.test_g11_census.ReviewResidualControls.test_mutation_child_environment_is_sanitized",
'self.assertEqual(outcome.get("status"), "OK")'),
("in-scope tracked symlink silently skipped", "census",
@@ -347,18 +353,18 @@ EXPECTED_MUTATION_NAMES = (
"unreadable revision exits zero",
"decoy decorator names accepted",
"decoy abstract-base names accepted",
- "decoy exception suffix accepted",
"decoy request-handler suffix accepted",
"comment and string mentions count as exercise",
"empty workflow run block not detected",
"YAML comments omitted",
- "shadowed builtin exception name accepted",
"mutation attribution accepts wrong assertion",
"function-local decorator shadow accepted",
"python -c path mention counts as execution",
"shell -c path mention counts as execution",
"template literal import text counts as static import",
"parser drops method identity",
+ "parser repeats qualified method identity",
+ "subTest failure header not parsed",
"wrong test id credited",
"nonfailure status credited",
"error outcome credited",
@@ -462,7 +468,11 @@ def _parse_run(proc):
result["skipped"] = int(counts.get("skipped", 0))
header = FAIL_HEADER.search(output)
if header:
- result["test_id"] = header.group(2) + "." + header.group(1)
+ method, test_id = header.group(1), header.group(2)
+ # Python 3.11+ prints "method (module.Class.method)"; earlier versions omit the method.
+ if not test_id.endswith("." + method):
+ test_id += "." + method
+ result["test_id"] = test_id
frames = TRACE_FRAME.findall(output)
if frames:
_path, line, method = frames[-1]
--
2.43.0
Generated by Claude Code |
|
Independent local verification of 01SHCrxv's patch (720/6541823925) on #720 head
🤖 Generated with Claude Code Generated by Claude Code |
Reconstruct the five-path #720 component on current Main without rewriting its history. Apply the reviewed digest/anchor/header repair, preserve the sealed 75-case inventory, and retain source-bound limits.
Current-Main reconstruction published — fresh qualification pendingThe reviewed #720 repair has been reconstructed non-rewriting on current Main and pushed by compare-and-swap to this existing branch.
Exact final blobs match the independently reviewed patch record: Local test-first reconstruction on CPython 3.12.14 observed the unpatched RED exactly at
Every published blob was recomputed and matched before the ref update. This is engineering qualification only; it does not establish full G11, Layer A, proof closure, scientific acceptance or CBZ-0. Fresh exact-head hosted checks and an exact-head source review still control integration. |
|
Integration readback on exact Ready head |
Scope
Current-base repair and qualification of the existing five-path G11 auditor component:
.github/workflows/cbz-g11-auditor.ymldocs/cbz/G11_AUDITOR.mdtests/test_g11_census.pytools/cbz/g11_census.pytools/cbz/g11_mutation_selftest.pyNo mathematical statement, scientific alignment, federation-wide Layer A result, or CBZ-0 state is changed by this pull request.
Exact current identity
165e99b9f2f0f6b8885dec2f73295bfd1c5604cbf3da535251108b99cb3e59638c8a7e3dd0c0fb0c77e81a4db3b90297c1cb8eeabdecbf58a9575395b99ff9690b97ef2d7faec0c39e5b054afd40a02fExact final blobs:
0d3fe0c6493af5c9a0c5f9c0ce7cc51722b176391d527e2524d9d34577286425d71fbd441adcbdb0e025ed88d53c0f82df9b3ef02573722a06e8c999d2952d559e49de54a8a5a8a8995d812eec5cbaaf27688671ce4e7003f251c047895de6aba0e4f7dfThe sealed 75-entry mutation inventory digest is
333f981944fbd08389c9fb384d6cd0dbf89fff4e40514ca67b123a0057be5d77.Preserved historical failures
The prior r2 head
6a91249d77b4e7e3470cfa38566576f9ba2efb8cpassed 38/38 tests in both modes, then failed because thelog_message context ignoredmutation matched zero targets; optimized mutation execution did not run.The initial r3 correction
1b96460453304b8db631a3dcee0a0c2a5bd4b60aaccidentally included quote characters in two target literals. Ordinary commit2aa6b26e1bb0964f4bd9b1a4b640ecd617ef9c95corrected that source, but its test-side digest was stale. Hosted run 41040319012 consequently ran 38 tests with one failure attest_mutation_inventory_is_sealed; optimized controls and both mutation campaigns were skipped. ReviewPRR_kwDOQpeC2s8AAAABVGgFCwand its P1 finding remain historical evidence.Current commit
f3da535251108b99cb3e59638c8a7e3dd0c0fb0cpreserves those commits and failures, merges the current Main base without rewriting history, and applies the exact reviewed three-file repair.Observed verification
Local CPython 3.12.14 before the final patch reproduced the sealed-digest RED: 38 tests, one failure, actual
ec3a19f1…versus stalef4e4e522….After the exact patch, local normal and optimized test runs each passed 38/38. The normal and optimized mutation self-tests each killed 75/75 mutations, 75 executed.
Hosted exact-head run 42134673355 / job 127619673193 on CPython 3.11.18 observed:
G11_SELFTEST_PASS: baseline 38 tests, 75/75 mutations killed (75 executed)normally;All fresh exact-head workflows completed successfully: federation drift run 42134673098; G11 run 42134673355; public-shop/browser run 42134672963; navigation run 42134673122; and workspace-landing/formal/architecture run 42134673213. Every job in those runs completed successfully. The PR is Ready on the unchanged head/base; no cached or skipped run is used as a substitute.
The stale digest review thread was replied to on the exact current head and resolved only after the hosted G11 log was observed. Organizational-independence credit remains 0; no scientific review is inferred.
Authority and limits
Complete owner mandate · standing delegation · OP-WORKFLOW · owner master #686/6511339492 · canonical pickup #684/6520283287.
This repair qualifies only the exact G11 component at the stated identities and domains. It does not close separate active #697, #698, #721, or #752 work; does not alter proof boundaries; and does not establish CBZ-0.