Skip to content

Fix C02 C09 C11 C13 C14 C16 qualification harness - #15

Merged
KeyffMS merged 5 commits into
mainfrom
fix/codex-capability-harness-v2
Sep 2, 2026
Merged

Fix C02 C09 C11 C13 C14 C16 qualification harness#15
KeyffMS merged 5 commits into
mainfrom
fix/codex-capability-harness-v2

Conversation

@KeyffMS

@KeyffMS KeyffMS commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Summary

Targets only the six run #6 outcomes attributable to incomplete or unstable qualification setup. C12 remains untouched for independent runtime verification.

C02 — explicit-only skill activation

  • installs a real repository skill with agents/openai.yaml and policy.allow_implicit_invocation: false;
  • runs separate semantic implicit and explicit $fixture-capability trials;
  • records the exact policy hash as outer structural evidence.

C09 / C14 — bytecode-clean PlanAnvil bootstrap

  • disables Python bytecode generation for the complete controller + Codex subprocess lifetime;
  • preserves the existing genuine auto-compaction and writable auxiliary Git isolation trials;
  • prevents fixture-owned __pycache__ artifacts from tripping PlanAnvil clean-source preflight.

C11 — current AGENTS precedence evidence

  • seeds root AGENTS.md plus nested AGENTS.override.md and a deliberately ignored nested AGENTS.md;
  • exercises Codex from nested CWD without reading instruction files through tools;
  • runs real PlanAnvil map_instructions.py outside the model and records full-read hashes/order;
  • includes a dated structural snapshot of the current official Codex AGENTS precedence rules.

C13 — project-scoped SubagentStart semantics

  • installs a real configured fixture_agent and project .codex/hooks.json;
  • the SubagentStart hook emits both additionalContext and continue=false;
  • the trial requires a real child startup with the injected context, demonstrating that continue=false is not relied upon as a startup blocker.

C16 — real Git diagnostic classification

  • exercises the real reversible PlanAnvil Git probe in three separate workspace-write trials;
  • verifies GIT_READY in the clean case;
  • forces a real signing failure with a deterministic fake GPG program and requires GIT_SIGNING_BLOCKED;
  • forces a real executable pre-commit rejection and requires GIT_HOOK_BLOCKED;
  • each trial verifies cleanup and source snapshot preservation.

Safety / scope

  • no danger-full-access or sandbox bypass;
  • model-generated command network remains disabled;
  • self-hosted qualification stays workflow_dispatch-only on main;
  • this PR itself uses only the ordinary hosted CI matrix;
  • C12 is intentionally not changed.

@KeyffMS
KeyffMS merged commit c56ad5d into main Sep 2, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant