Skip to content

Harden C6 hard-direction mutant verdicts - #560

Merged
d6g8k5htny-coder merged 7 commits into
mainfrom
codex/cbz-hard-direction-contract-20261112
Nov 13, 2026
Merged

d6g8k5htny-coder merged 7 commits into
mainfrom
codex/cbz-hard-direction-contract-20261112

Conversation

@d6g8k5htny-coder

@d6g8k5htny-coder d6g8k5htny-coder commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

The C6 hard-direction workflow previously counted any exit 1 as a rejected mutant, including an unrelated crash. This change requires the named failure reason, the exact JSON verdict, the expected exit code, and empty stderr; it rejects duplicate keys and non-finite JSON values. Contract tests cover baseline and mutant failures in both child modes. The numerical checker and mathematical statements are unchanged.

Tested source: Math head 2cbde50c80844b3ba428bb6004acab42fe89bd36; base 9bdfc54b76173b57912503bfcf901a5f9d43b09c; hosted merge 52b5e20387d5d97faa19539c0ebe2c5e7fd63083, tree 380ef32db41c7f2476bda546f27e416aadf88016.

Observed validation:

  • Hosted C6 job 131481300857: eight focused tests in each Python mode and the workflow replay passed; replay reported certified: false.
  • Hosted full regression job 131481302805: 1,152 tests passed in each Python mode, with no skips listed; the unchanged-commit step passed. Downstream run 43303710579 and organization-only hypergraph run 43303710205 reported success.
  • Six individual mutant omissions and the boolean-as-number mutation were rerun with original/mutant controls in both runner modes using the scoped three-chain origin predicate: 28 executions, originals passed and mutant failures matched the predicate. Four earlier workflow mutations were likewise rerun: 16 valid executions. Seven-mutation evidence; four-mutation evidence.
  • The full workflow shell was replayed against the real packet and 12 disposable synthetic Git fixtures. The adverse fixtures checked formatted traceback frames and lines, not live code-object provenance. Full-shell evidence.

Limits and preserved history: The original external mutation grader accepted an unrelated helper assertion when the selected test method appeared anywhere in the traceback. Its results are retained as observations, not qualified kills. The replacement is limited to three specific unittest assertion code-object chains, has its own adverse and mutation checks, and cannot see exceptions swallowed by unittest. Earlier failed and interrupted local runs and all prior evidence links are preserved in the description history snapshot.

These checks neither certify floating-point mathematical bounds nor establish scientific acceptance, a full A8/G6 audit, or CBZ-0.

Additional full-shell adverse cases: Eighteen disposable Git fixtures exercised unexpected stderr, duplicate JSON keys and malformed JSON at each of three mutant positions in each child mode. Each exited 1 with empty stdout and the expected formatted workflow traceback frame/line and terminal exception. The candidate workflow blob matched this head. This remains synthetic formatted-traceback evidence, not live code-object provenance or mathematical certification. Evidence libfile_fde3_f-b73b8549539c8191abd94c18327309ad_59a0f9f234e48191bc19fddcc0f80648, SHA256 86f8df7b7e728cfcb91c6baee8c4bddbb250391206bbbf90292ba65e7bdf987e.

Additional full-shell baseline adverse cases: Twelve disposable Git fixtures exercised wrong schema, boolean numeric values, NaN, Infinity, unexpected stderr, and child crash in both Python modes. All exited 1 at the expected formatted workflow traceback frame and terminal exception. This synthetic evidence does not establish live code-object provenance, a full audit, or mathematical certification. Baseline probe evidence, SHA256 ed86c784907fa259eb5a73a33c90be37c22444f5cf865f329f1f5cc219deb948.

@d6g8k5htny-coder
d6g8k5htny-coder marked this pull request as ready for review November 13, 2026 04:34
@d6g8k5htny-coder
d6g8k5htny-coder merged commit dee08f4 into main Nov 13, 2026
6 checks passed
@d6g8k5htny-coder
d6g8k5htny-coder deleted the codex/cbz-hard-direction-contract-20261112 branch November 13, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant