Skip to content

Add Codex runner diagnostic variant matrix - #23

Merged
KeyffMS merged 3 commits into
mainfrom
diag/codex-runner-variant-matrix
Sep 3, 2026
Merged

Add Codex runner diagnostic variant matrix#23
KeyffMS merged 3 commits into
mainfrom
diag/codex-runner-variant-matrix

Conversation

@KeyffMS

@KeyffMS KeyffMS commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a diagnostic-only self-hosted runner matrix to observe Codex CLI 0.152 behavior across multiple runtime variants without changing C01-C16 capability results or the release gate.

Hook variants

  • project hooks.json, canonical ^Bash$, hook-trust bypass;
  • negative matcher control ^exec_command$;
  • project TOML hook representation;
  • trusted project without hook-trust bypass;
  • hooks feature disabled negative control;
  • external sidecar environment propagation plus repository-local marker.

Compaction variants

  • body_after_prefix, TokenBudget disabled, one-tool control;
  • body_after_prefix, TokenBudget disabled, forced second tool call;
  • same with TokenBudget enabled;
  • total scope with TokenBudget disabled and forced follow-up.

Subagent variants

  • multi-agent v1 + explicit agent_type=fixture_agent;
  • multi-agent v2 + explicit agent_type;
  • v1 default/omitted agent_type control;
  • isolated CODEX_HOME non-ephemeral home-agent fallback.

Each variant records only structural event counts, sanitized error strings, diagnostic markers, hook records, and repository/auth/cleanup invariants. Raw transcripts are not persisted.

A separate PlanAnvil Codex runner diagnostics workflow runs only by workflow_dispatch on main using the existing controlled self-hosted labels. Variant outcomes are intentionally non-gating; only diagnostic harness execution and artifact creation are enforced.

@KeyffMS
KeyffMS merged commit dee662c into main Sep 3, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant