Skip to content
This repository was archived by the owner on Oct 16, 2026. It is now read-only.

feat(cli): bundle eval suites for portable reruns - #1421

Merged
christso merged 1 commit into
mainfrom
feat/eval-bundle-portability
Jun 19, 2026
Merged

christso merged 1 commit into
mainfrom
feat/eval-bundle-portability

Conversation

@christso

Copy link
Copy Markdown
Collaborator

Summary

agentv eval bundle <eval> --out <dir> now creates a self-contained eval directory that can be moved and rerun without the source tree's inherited targets or shared fixture directories. Before this branch, eval bundle fell through to the old eval-run path and rejected --out, so there was no way to package a portable rerun artifact from the CLI.

What Changed

  • Reused the existing eval task-bundle materialization path to build whole-suite bundles instead of introducing a second resolver or execution engine.
  • Added the eval bundle subcommand and taught argv preprocessing that bundle is a real eval subcommand instead of an implicit run invocation.
  • Bundle output now includes a rewritten eval YAML, selected target definitions, copied relative file dependencies, workspace templates, workspace hook scripts, and a snake_case agentv_bundle.json manifest.
  • Bundle-time failures now identify the specific eval reference that could not be copied so portability issues fail clearly without mutating the source eval.

Validation

  • bun test apps/cli/test/commands/eval/bundle.test.ts apps/cli/test/unit/preprocess-argv.test.ts
  • bun run typecheck
  • bun run lint
  • Red: in a clean origin/main worktree, bun --no-env-file apps/cli/dist/cli.js eval bundle <eval> --out <dir> failed with --out was removed from agentv eval, showing that bundle still routed into the legacy eval run path.
  • Green: in this branch, bun --no-env-file apps/cli/src/cli.ts eval bundle <eval> --out <dir> wrote the bundle, and rerunning bun --no-env-file .../apps/cli/src/cli.ts eval evals/demo.eval.yaml --output <dir> from the bundled directory completed with RESULT: PASS.

Compound Engineering
Codex

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 58ba943
Status: ✅  Deploy successful!
Preview URL: https://a9c91fe2.agentv.pages.dev
Branch Preview URL: https://feat-eval-bundle-portability.agentv.pages.dev

View logs

@christso
christso marked this pull request as ready for review June 19, 2026 04:00
@christso
christso merged commit 4a246d9 into main Jun 19, 2026
8 checks passed
@christso
christso deleted the feat/eval-bundle-portability branch June 19, 2026 04:00
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant