[christso]

Summary

• replaces the stale evals/self layout with plugin-aligned evals/agentv-self and evals/agentv-dev suites
• removes large stale skill transcript fixtures in favor of live repo/plugin inputs
• updates self-eval docs and workspace setup so agentv-self copies the current AGENTS.md, .agents, and tracked repo content via before_all
• clarifies that the agentv-dev entrypoint should return none when no bundled AgentV skill applies

Verification

• bun apps/cli/src/cli.ts validate evals/agentv-self evals/agentv-dev/skills
• bunx biome check evals/agentv-self/scripts/setup.mjs
• Codex target configured as /tmp/agentv-1435-codex-gpt55-low.targets.yaml with provider: codex, model: gpt-5.5, and model_reasoning_effort: low plus explicit Codex CLI args --model gpt-5.5 -c model_reasoning_effort="low"
• Smoke: repo-change-preflight from evals/agentv-self/agentv-self.eval.yaml passed 100% with codex-gpt55-low
• Smoke: select-bench-for-eval-run from evals/agentv-dev/skills/skill-selection.eval.yaml passed 100% with codex-gpt55-low
• Full live run across the five PR suites completed 27/28; the only failure was select-no-false-positive, where the old prompt forced “skill name(s) only” and Codex selected agentv-bench for a Kubernetes deploy task
• After fixing the negative-selection contract, reran evals/agentv-dev/skills/skill-selection.eval.yaml with codex-gpt55-low: 8/8 passed, mean 100%, artifact index /tmp/agentv-1435-codex-skill-selection-rerun/index.jsonl

Bead: av-kss

[cloudflare-workers-and-pages]

Deploying agentv with    Cloudflare Pages

Latest commit:	df90eea
Status:	✅  Deploy successful!
Preview URL:	https://6d13ee01.agentv.pages.dev
Branch Preview URL:	https://chore-agentv-self-evals-spli.agentv.pages.dev

View logs

[christso]

Pushed df90eeab with the live-eval fixes.

What changed:

• agentv-self setup now materializes the tracked repo snapshot with git archive HEAD instead of recursively copying ignored build/install artifacts. This fixed the before_all timeout seen in the Codex smoke run.
• agentv-dev skill selection now has an explicit none contract for unrelated tasks, and the negative selection eval asserts that deterministic output.

Codex GPT-5.5 low verification:

• Full run over the five PR suites: 27/28 initially; only select-no-false-positive failed because the old prompt forced “skill name(s) only”.
• After the fix, reran evals/agentv-dev/skills/skill-selection.eval.yaml: 8/8 passed, mean 100%.
• Rerun artifact index: /tmp/agentv-1435-codex-skill-selection-rerun/index.jsonl.

CI is now running on the pushed commit.