Repair qualification process contracts, recovery isolation and completion gates

.github/workflows/plananvil-codex-qualification.yml

@@ -13,6 +13,7 @@ on:
           - c13
           - diagnostics
           - precision
+          - recovery
           - full
 
 permissions:
@@ -247,7 +248,7 @@ jobs:
     if: >-
       github.event_name == 'workflow_dispatch' &&
       github.ref == 'refs/heads/main' &&
-      (inputs.mode == 'full' || inputs.mode == 'c13')
+      (inputs.mode == 'full' || inputs.mode == 'c13' || inputs.mode == 'recovery')
     environment: plananvil-codex
     runs-on:
       - self-hosted
@@ -318,20 +319,28 @@ jobs:
         run: |
           set -euo pipefail
           cd "${QUALIFICATION_REPO}"
-          # Baseline 2.3 permits the narrowly gated C13 fallback in both full and
-          # C13-only modes. The harness still tries ephemeral first and ignores
-          # this permission unless the recognized parent-thread failure occurs.
+          # All selected modes retain ephemeral-first C13. Permission to retry
+          # never applies unless the recognized parent-thread failure occurs.
           qualification_args=(--allow-c13-non-ephemeral-fallback)
           if [ "${{ inputs.mode }}" = "c13" ]; then
             qualification_args+=(--only C13)
           fi
           set +e
-          python3 tools/live_codex_qualification_harness_v7.py \
-            --root "${QUALIFICATION_REPO}" \
-            --source-commit "${GITHUB_SHA}" \
-            --run-id "${GITHUB_RUN_ID}" \
-            --output "${QUALIFICATION_ARTIFACT}" \
-            "${qualification_args[@]}"
+          if [ "${{ inputs.mode }}" = "recovery" ]; then
+            python3 tools/live_codex_qualification_recovery.py \
+              --root "${QUALIFICATION_REPO}" \
+              --source-commit "${GITHUB_SHA}" \
+              --run-id "${GITHUB_RUN_ID}" \
+              --output "${QUALIFICATION_ARTIFACT}" \
+              "${qualification_args[@]}"
+          else
+            python3 tools/live_codex_qualification_harness_v7.py \
+              --root "${QUALIFICATION_REPO}" \
+              --source-commit "${GITHUB_SHA}" \
+              --run-id "${GITHUB_RUN_ID}" \
+              --output "${QUALIFICATION_ARTIFACT}" \
+              "${qualification_args[@]}"
+          fi
           rc=$?
           set -e
           echo "exit_code=${rc}" >> "${GITHUB_OUTPUT}"

.github/workflows/plananvil-tests.yml

@@ -48,6 +48,8 @@ jobs:
         run: python -m compileall -q .agents/skills/plan-anvil .codex/hooks tools tests
       - name: Run core unit and integration tests
         run: python -m unittest discover -s .agents/skills/plan-anvil/tests -v
+      - name: Execute qualification boundary regressions without Codex
+        run: python -m unittest discover -s tests -p "test_qualification_*.py" -v
       - name: Check patch whitespace
         run: git diff --check
 

docs/CODEX_QUALIFICATION_EXECUTION_AUDIT_2026-09-05.md

@@ -0,0 +1,58 @@
+# Qualification execution-boundary audit — 2026-09-05
+
+## Scope and authority
+
+Audited PlanAnvil source: `7b6322e4c2b11d4b78b5be714c94b88644fb6147`.
+Observed live run: [qualification #21](https://github.com/KeyffMS/PlanAnvil/actions/runs/33968031850), Codex CLI `0.153.4`, explicit model `gpt-5.6-sol`.
+
+The implementation specification, generator/executor separation, product hooks, approval policy, sandbox, source immutability, baseline 2.3 and all `expected.json` assertions remain unchanged. This patch repairs qualification machinery, not the meaning of the product capabilities. Offline tests are not live evidence.
+
+## Source-verified contracts
+
+- [Official hooks documentation](https://developers.openai.com/codex/hooks): project/user hook sources, context output and event-specific control effects.
+- [Pinned config loader](https://github.com/openai/codex/blob/rust-v0.153.4/codex-rs/config/src/loader/mod.rs): `root_checkout_hooks_folder_for_dir` and `merge_root_checkout_project_hooks` select root-checkout hook declarations for linked worktrees. Ordinary worktree-local config is distinct from hook-declaration provenance.
+- [Pinned hook discovery](https://github.com/openai/codex/blob/rust-v0.153.4/codex-rs/hooks/src/engine/discovery.rs): uses each layer's `hooks_config_folder()`; hook sources can be additive.
+- [Pinned hook execution](https://github.com/openai/codex/blob/rust-v0.153.4/codex-rs/hooks/src/engine/command_runner.rs): executes commands with the event cwd, which is not necessarily the directory containing the declarations.
+- [Pinned startup semantics](https://github.com/openai/codex/blob/rust-v0.153.4/codex-rs/hooks/src/events/session_start.rs): SubagentStart matches `agent_type`, injects context and does not use `continue:false` as a startup stop.
+
+## Defect, repair and executable regression
+
+### C13 process contract
+
+The active compatibility context substituted a generic proxy requiring event and script arguments while the project fixture generated a command with neither argument. The proxy raised `IndexError` before executing the actual hook or recording telemetry. Zero recorder entries did not prove missing project-hook discovery.
+
+Repair: pass `SubagentStart subagent-start-fixture.py` to the existing generated command. The agent and hook remain project-scoped, with explicit `fixture_agent`, and the recognized-error-only ephemeral fallback is unchanged.
+
+Regression: enter the real compatibility context, seed the same declared project fixture as v7, execute the actual configured command in a repository with spaces, and assert exit code, context JSON, continue=false, one telemetry record and unchanged Git state. No Codex process or model is involved in this process-contract test.
+
+### C10 configuration provenance
+
+The previous PostCompact probe removed SessionStart only from a linked planning worktree. Codex still loaded the primary checkout's declarations. The same proof could reach the model through SessionStart, invalidating attribution to PostCompact.
+
+Repair: independent source repositories, planning worktrees and random proofs for the two trials. Prepare the second root checkout's hook selection before the fixture commit, product bootstrap and checkpoint. Verify root/local declarations agree and preserve both source and planning state during the probe. Keep the actual product recovery and compaction scripts; never synthesize a live hook event in the qualification runtime.
+
+Regression: the offline driver substitutes only Codex. It executes the real installer, Git operations, product start/checkpoint/validator and generated hook processes, using root-checkout declarations and planning cwd. It checks independent roots/proofs, valid checkpoints, context delivery and no proof retention. The driver models the pinned loader rule; it is not a substitute for live confirmation of that rule.
+
+### C09 successful completion
+
+Run #21 recorded 31 PreCompact and 30 PostCompact events and a Codex invocation timeout. The old evaluator could nevertheless classify the trial REPRODUCED.
+
+Repair: require successful invocation completion and a positive structured C09 result in addition to the existing two-cycle, checkpoint and continuation assertions. A timeout or missing result is BLOCKED, never a pass. Partial lifecycle observations remain visible. C08's intentional negative stop trial is not changed.
+
+Regression: call the actual C09 evaluator with controlled observations; two cycles cannot conceal a timeout or absent completion payload. Completed positive evidence still passes; a missing continuation or observed stop still prevents reproduction.
+
+This patch does not guess a new auto-compaction threshold. C09 may still time out live; that would now be accurately reported rather than converted into a green result. Repeated compaction under the artificially low fixture limit is not, by itself, proof of a product loop.
+
+## Red-to-green verification
+
+The test-only commit `f25fb15e4664a7afab371f696eae71dedd8876fe` added executable regressions before repairs. [Hosted CI #102](https://github.com/KeyffMS/PlanAnvil/actions/runs/33976076211) ran 134 distribution/harness tests and reported exactly four failures: C13 argv, C10 root-source isolation, C09 timeout and C09 missing output. Existing tests did not detect those defects.
+
+The same regressions remain in the patched suite. CI additionally executes qualification boundary tests on Linux, macOS and Windows with Python 3.11 and 3.14. The POSIX platforms execute the exact generated shell command; Windows checks the equivalent generated Python argv contract. The actual controlled live runner is Linux.
+
+## Targeted runner handoff
+
+Use the existing allowed workflow `PlanAnvil Codex qualification`, branch `main`, mode `recovery`. It selects C09, C10 and C13 using the same v7 capability runtimes as `full`; it does not use the separate precision fixtures.
+
+The targeted summary explicitly declares `scope`, `diagnostic_only=true`, `selected_gate_passed` and `release_gate_passed=false`. The full index still considers every required capability. A targeted success cannot release the product or substitute for a final C01–C16 run.
+
+No hosted test needs runner credentials, and this change does not dispatch a self-hosted run. No claim is made that the upstream ephemeral parent-thread failure is fixed. Live qualification remains necessary after offline verification.

tests/test_live_codex_qualification_c10.py

@@ -1,8 +1,10 @@
 from __future__ import annotations
 
 from contextlib import nullcontext
+import json
 from pathlib import Path
 import sys
+import tempfile
 import types
 import unittest
 from unittest import mock
@@ -26,17 +28,28 @@ def test_c10_fixture_is_outer_harness_owned_and_product_validated(self) -> None:
         source = C10_SOURCE.read_text(encoding="utf-8")
         self.assertIn("v4._start_active_run(", source)
         self.assertIn("v4._create_checkpoint(planning=planning, run_root=run_root)", source)
-        self.assertGreaterEqual(source.count("v4._checkpoint_validation(planning)"), 4)
+        self.assertEqual(source.count("v4._checkpoint_validation(planning)"), 2)
+        self.assertEqual(source.count("v4._checkpoint_validation(compact_planning)"), 2)
         self.assertIn('"fixture_prepared_by_outer_harness=true"', source)
         self.assertNotIn("planner_prompt", source)
 
     def test_postcompact_is_independent_of_session_start_context(self) -> None:
-        source = C10_SOURCE.read_text(encoding="utf-8")
-        self.assertIn('configured.pop("SessionStart", None)', source)
-        self.assertIn('bool(configured.get("PreCompact"))', source)
-        self.assertIn('bool(configured.get("PostCompact"))', source)
-        self.assertIn("not compact_session_start", source)
-        self.assertIn("postcompact_isolated", source)
+        with tempfile.TemporaryDirectory() as tmp:
+            repo = Path(tmp)
+            hooks = {"hooks": {name: [{"hooks": [{"type": "command", "command": "fixture"}]}]
+                               for name in ("SessionStart", "PreCompact", "PostCompact")}}
+            c10.base.json_dump(repo / ".codex/hooks.json", hooks)
+            self.assertTrue(c10._disable_session_start_for_postcompact(repo))
+            remaining = c10.base.load_json(repo / ".codex/hooks.json")["hooks"]
+            self.assertNotIn("SessionStart", remaining)
+            self.assertEqual(remaining["PreCompact"], hooks["hooks"]["PreCompact"])
+            self.assertEqual(remaining["PostCompact"], hooks["hooks"]["PostCompact"])
+
+    def test_missing_compaction_handler_fails_isolation(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            repo = Path(tmp)
+            c10.base.json_dump(repo / ".codex/hooks.json", {"hooks": {"SessionStart": []}})
+            self.assertFalse(c10._disable_session_start_for_postcompact(repo))
 
     def test_compaction_trigger_is_qualification_only(self) -> None:
         source = C10_SOURCE.read_text(encoding="utf-8")
@@ -46,12 +59,14 @@ def test_compaction_trigger_is_qualification_only(self) -> None:
         self.assertIn("C10_COMPACT_LIMIT = 200", source)
 
     def test_opaque_recovery_value_is_not_persisted_in_evidence(self) -> None:
-        source = C10_SOURCE.read_text(encoding="utf-8")
-        self.assertIn("proof = secrets.token_hex(16)", source)
-        self.assertIn("def _payload_summary", source)
-        self.assertIn('"opaque_recovery_value_persisted=false"', source)
-        self.assertNotIn('"model_payload": session_payload', source)
-        self.assertNotIn('"model_payload": compact_payload', source)
+        secret_a, secret_b = "a" * 32, "b" * 32
+        raw = {"error": "failure " + secret_a, "trials": [{"detail": "target=" + secret_b}]}
+        redacted = c10._redact_proofs(raw, (secret_a, secret_b))
+        self.assertNotIn(secret_a, json.dumps(redacted))
+        self.assertNotIn(secret_b, json.dumps(redacted))
+        self.assertIn(secret_a, raw["error"])
+        self.assertFalse(c10._exact_echo({"observations": ["C10_RECOVERY_ECHO=" + secret_b]}, secret_a))
+        self.assertTrue(c10._exact_echo({"observations": ["C10_RECOVERY_ECHO=" + secret_a]}, secret_a))
 
     def test_install_routes_only_c10(self) -> None:
         calls: list[dict[str, object]] = []

tests/test_prepare_capabilities_overlay.py

@@ -44,13 +44,14 @@ def test_c13_baseline23_overlay_materializes_and_rehashes(self) -> None:
             self.assertIn("agent_type", readme)
             self.assertIn("fixture_agent", readme)
             self.assertIn(
-                "live_codex_qualification_harness_v6.py",
+                "live_codex_qualification_harness_v7.py",
                 (c13 / "run-command.txt").read_text(encoding="utf-8"),
             )
             config = (c13 / "config" / "README.md").read_text(encoding="utf-8")
-            self.assertIn("home-scoped", config)
+            self.assertIn("No home-scoped synthetic agent or hook substitutes", config)
             self.assertIn("project-scoped", config)
-            self.assertIn("fixture_agent.toml", config)
+            self.assertIn("[agents.fixture_agent]", config)
+            self.assertIn('config_file = "./agents/fixture_agent.toml"', config)
             self.assertIn("agent_type=fixture_agent", config)
             prompt = (c13 / "prompt.txt").read_text(encoding="utf-8")
             self.assertIn("agent_type` exactly `fixture_agent", prompt)
@@ -68,6 +69,27 @@ def test_c13_baseline23_overlay_materializes_and_rehashes(self) -> None:
             self.assertEqual(index["baseline_version"], "2.3")
             self.assertEqual(validate_capabilities.validate_all(target), [])
 
+    def test_recovery_overlays_do_not_change_expected_assertions(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            target = Path(tmp)
+            prepare_capabilities.materialize(ROOT, target, force=True)
+            expected = {
+                "C09": ["Valid checkpoint allows compaction.",
+                        "Recovery reconciles canonical files/Git after compaction.",
+                        "A second valid compaction path is not permanently blocked."],
+                "C10": ["Recovery hook injects a pointer/context, not hidden mutable state.",
+                        "Session continuation can reconstruct from canonical files and Git."],
+            }
+            for cid, assertions in expected.items():
+                directory = target / "capabilities" / cid
+                package = json.loads((directory / "expected.json").read_text(encoding="utf-8"))
+                self.assertEqual(package["assertions"], assertions)
+                self.assertIn("live_codex_qualification_recovery.py",
+                              (directory / "run-command.txt").read_text(encoding="utf-8"))
+            self.assertIn("BEFORE", (target / "capabilities/C10/fixture/README.md").read_text(encoding="utf-8"))
+            self.assertIn("timeout", (target / "capabilities/C09/fixture/README.md").read_text(encoding="utf-8"))
+            self.assertEqual(validate_capabilities.validate_all(target), [])
+
     def test_overlays_do_not_remove_other_capabilities(self) -> None:
         with tempfile.TemporaryDirectory() as tmp:
             target = Path(tmp) / "materialized"