chore(e2e): coverage - split model and workflow world suites

[AndrewBarba]

Summary

• split e2e discovery into a Local live-model suite and deterministic Local, Vercel, and Postgres world suites
• run 8 meaningful live-model legs and 15 fixtures in each world, reducing focused coverage from 120 to 53 jobs while retaining every fixture
• extend the internal authored-model adapter for deterministic subagents, repeated tools, Workflow fan-out, silent completion, and schema-derived inputs
• preserve Vercel redeploy and Postgres persistence coverage, with stable aggregate checks for all three worlds
• document suite ownership and add a patch changeset for the internal eve adapter update

Validation

• pnpm fmt
• pnpm lint
• pnpm typecheck
• pnpm check:deps
• pnpm guard:invariants
• pnpm guard:fixtures
• pnpm docs:check
• pnpm exec oxfmt --check .
• pnpm exec oxlint .
• pnpm --filter eve exec vitest run --config vitest.unit.config.ts src/runtime/agent/mock-model-adapter.test.ts
• discovery validation: 8 model legs, 15 world fixtures, 53 focused jobs
• git diff --check

CI is the official e2e validation.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
eve-docs	Ready	Preview	Jul 30, 2026 1:08am
eve-docs-4759	Ready	Preview, v0	Jul 30, 2026 1:08am

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/​@​workflow/​world-postgres@​5.0.0-beta.29

View full report

[github-actions]

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

• No notable deltas vs main (98d17c7).

Delta vs main (98d17c7)

Area	Metric	Baseline	Current	Delta
Package	Packed tarball	7.62 MB	7.62 MB	+3.5 kB ⚠️
Package	Unpacked publish size	28.79 MB	28.80 MB	+18.0 kB ⚠️
Package	Installed footprint	91.09 MB	91.11 MB	+18.0 kB ⚠️
Package	Published files	2875	2879	+4
Package	Installed files	6662	6666	+4
Runtime	Unique function payloads	2	2	0
Runtime	Total function bytes	16.79 MB	16.83 MB	+34.9 kB ⚠️
Runtime	Public routes	11	11	0

Changed function payloads vs main (98d17c7) (2)

Function	Status	Baseline	Current	Delta	Route changes
functions/__server.func	changed	8.40 MB	8.41 MB	+17.4 kB ⚠️	none
functions/.well-known/workflow/v1/flow.func	changed	8.40 MB	8.41 MB	+17.4 kB ⚠️	none

eve init install

Metric	Baseline	Current	Delta
Installed footprint	129.49 MB	129.51 MB	+18.0 kB ⚠️
Installed packages	130	130	0
dependencies	4	4	0
devDependencies	2	2	0
Dependency package bytes	43.12 MB	43.14 MB	+18.0 kB ⚠️
devDependency package bytes	5.04 MB	5.04 MB	0 B ➖

Build Metadata

• Preset: vercel
• Nitro: nitro@3.0.260610-beta
• Output directory: apps/fixtures/weather-agent/.vercel/output
• Build metadata timestamp: 2026-07-30T01:09:20.873Z
• Route aliases: 11 public, 1 internal (12 total aliases)
• Vercel routes in config: 14
• Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small

Package Drill-Down

Package Details

• Package: eve@0.27.13
• Package directory: packages/eve
• Tarball: 7.62 MB (eve-0.27.13.tgz)
• Unpacked payload: 28.80 MB across 2879 published files
• Installed footprint: 91.11 MB across 6666 installed files
• Installed root package: 27.45 MB
• Installed dependencies: 63.66 MB
• Runtime dependencies: 2
• Peer dependencies: 5 (4 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.

Heavy installed dependencies

• eve: 27.45 MB (30.1%)
• @rolldown/binding-linux-x64-gnu: 19.28 MB (21.2%)
• @rolldown/binding-wasm32-wasi: 10.66 MB (11.7%)
• ai: 6.53 MB (7.2%)
• @napi-rs/wasm-runtime: 6.39 MB (7.0%)

Publish payload breakdown

Published file size
🔴 dist/src/compiled/shadcn-registry/index.js       [########################] 13.15 MB 45.7%
🟠 dist/src/compiled/experimental-ai-sdk-code-mo... [###.....................] 1.51 MB 5.2%
🟡 dist/src/compiled/@vercel/sandbox/index.js       [#.......................] 632.5 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-DWL... [#.......................] 502.4 kB 1.7%
🟡 dist/src/compiled/@chat-adapter/slack/index.js   [#.......................] 440.5 kB 1.5%
🔴 Other published files                            [#######################.] 12.57 MB 43.6%

Installed footprint breakdown

Installed package size
🔴 eve                             [########################] 27.45 MB 30.1%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 19.28 MB 21.2%
🔴 @rolldown/binding-wasm32-wasi   [#########...............] 10.66 MB 11.7%
🔴 ai                              [######..................] 6.53 MB 7.2%
🔴 @napi-rs/wasm-runtime           [######..................] 6.39 MB 7.0%
🔴 zod                             [####....................] 5.07 MB 5.6%
🔴 Other installed packages        [##############..........] 15.73 MB 17.3%

Runtime dependencies (2)

Package	Range	Notes
nitro	3.0.260610-beta
undici	8.9.0

Peer dependencies (5)

Package	Range	Notes
@opentelemetry/api	^1.0.0	optional peer
ai	catalog:
braintrust	^3.0.0	optional peer
just-bash	^3.0.0	optional peer
microsandbox	^0.5.0	optional peer

eve init install drill-down

eve init install details

• Command: eve init my-agent
• Package manager: npm
• Installed footprint: 129.51 MB across 8534 installed files
• Installed packages: 130 total (124 transitive-only)
• dependencies: 4 direct packages totaling 43.14 MB
• devDependencies: 2 direct packages totaling 5.04 MB
• Other transitive package files: 81.33 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

• @typescript/typescript-linux-x64: 27.95 MB (21.6%)
• eve: 27.45 MB (21.2%)
• @rolldown/binding-linux-x64-gnu: 19.28 MB (14.9%)
• @rolldown/binding-wasm32-wasi: 10.66 MB (8.2%)
• zod: 9.02 MB (7.0%)

Installed footprint breakdown

Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 21.6%
🔴 eve                              [########################] 27.45 MB 21.2%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.28 MB 14.9%
🔴 @rolldown/binding-wasm32-wasi    [#########...............] 10.66 MB 8.2%
🔴 zod                              [########................] 9.02 MB 7.0%
🔴 ai                               [######..................] 6.53 MB 5.0%
🔴 @napi-rs/wasm-runtime            [#####...................] 6.39 MB 4.9%
🔴 Other installed packages         [###################.....] 22.23 MB 17.2%

dependencies (4)

Package	Range	Installed size	Share
@vercel/connect	0.4.2	135.8 kB	0.1%
ai	^7.0.38	6.53 MB	5.0%
eve	file:eve-0.27.13.tgz	27.45 MB	21.2%
zod	4.4.3	9.02 MB	7.0%

devDependencies (2)

Package	Range	Installed size	Share
@types/node	24.x	2.54 MB	2.0%
typescript	7.0.2	2.50 MB	1.9%

Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 8.41 MB 50.0%
🔴 functions/__server.func                         [########################] 8.41 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 8.41 MB

Metric	Value
Public routes	/.well-known/workflow/v1/flow
Runtime	nodejs24.x
Handler	index.mjs
Payload	8.41 MB
Function files	8.41 MB across 44 files
Traced dependencies	0 B
Signal	🟠 Bundled file index.mjs is 2.32 MB (27.5%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                       [########################] 2.32 MB 27.5%
🟠 _chunks/runtime-artifacts.mjs   [################........] 1.59 MB 18.9%
🟡 _libs/undici.mjs                [##########..............] 980.5 kB 11.7%
🟡 _chunks/sandbox.mjs             [########................] 768.8 kB 9.1%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.1%
🟠 Other bundled files             [########################] 2.33 MB 27.6%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

---

🟠 functions/__server.func • 10 public routes, 1 internal alias • 8.41 MB

Metric	Value
Public routes	/ /eve/v1/callback/[token] /eve/v1/connections/[name]/callback/[token] /eve/v1/health /eve/v1/info /eve/v1/session /eve/v1/session/[sessionId] /eve/v1/session/[sessionId]/cancel /eve/v1/session/[sessionId]/stream /eve/v1/session/reset
Internal aliases	/__server
Runtime	nodejs24.x
Handler	index.mjs
Payload	8.41 MB
Function files	8.41 MB across 44 files
Traced dependencies	0 B
Signal	🟠 Bundled file index.mjs is 2.32 MB (27.5%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                       [########################] 2.32 MB 27.5%
🟠 _chunks/runtime-artifacts.mjs   [################........] 1.59 MB 18.9%
🟡 _libs/undici.mjs                [##########..............] 980.5 kB 11.7%
🟡 _chunks/sandbox.mjs             [########................] 768.8 kB 9.1%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.1%
🟠 Other bundled files             [########################] 2.32 MB 27.6%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

• Build pipeline: 2.02 s -> 2.02 s (+0.1 ms) vs main (98d17c7).
• Timing is informational: shared GitHub runners are too variable for a hard timing budget.

Detailed phase timings vs `main (98d17c7)`

Phase	Baseline	Current	Delta
extension.check	10.4 ms	1.2 ms	-9.2 ms
project.resolve	1.1 ms	0.5 ms	-0.6 ms
workspace.create	4.3 ms	0.6 ms	-3.7 ms
host.prepare	295.0 ms	200.5 ms	-94.5 ms
vercel.service-prefix.resolve	1.6 ms	2.0 ms	+0.4 ms
nitro.create	180.5 ms	197.0 ms	+16.5 ms
sandbox.prewarm	211.5 ms	259.7 ms	+48.2 ms
nitro.cache.prepare	0.3 ms	0.2 ms	-0.1 ms
nitro.prepare	0.6 ms	0.7 ms	+0.1 ms
nitro.public-assets	0.7 ms	0.8 ms	+0.1 ms
nitro.prerender	0.5 ms	0.4 ms	-0.1 ms
nitro.bundle	1.25 s	1.29 s	+38.3 ms
nitro.cache.write	0.3 ms	0.3 ms	0.0 ms
vercel.workflow-function.materialize	34.4 ms	54.6 ms	+20.2 ms
agent-summary.emit	0.4 ms	0.5 ms	+0.1 ms
nitro.close	0.1 ms	0.1 ms	0.0 ms
output.publish	18.1 ms	2.7 ms	-15.4 ms
workspace.remove	1.7 ms	1.7 ms	0.0 ms

[AndrewBarba]

Closing in favor of #1387, which lands the suite split with a different architecture:

• Every fixture stays in both the model suite and every world suite; membership is expressed with eval-level real-model tags (eve eval --exclude-tag) rather than an exclusive fixture partition, so no fixture loses live-model or world coverage.
• World suites use fixture-authored mockModel() responders (via the shared @eve-e2e/config package this PR introduced — thanks, that design carried over directly) instead of expanding the internal runtime adapter, keeping eval-prompt-specific logic out of packages/eve.
• Aggregate check names stay e2e-local / e2e-vercel, plus the new e2e-postgres (this PR's workflow, adapted to the mock convention, including the Postgres run-count durability assertion).

Salvaged from here:

• The turn-cancellation race fix + test → extracted to fix(eve): settle turn as cancelled when abort races step completion #1389 so the runtime fix gets independent review.
• The image-attachment, cancel-subagent, and agent-subagents eval determinism tweaks → cherry-picked into test(e2e): split model and world suites #1387 (image-attachment now runs mocked in all world suites).
• The mock directive programs (subagent delegation, Workflow fan-out, repeated tools, schema-derived inputs) remain the reference recipes for untagging agent-subagents / agent-schedules / agent-skills as fixture-side responders in follow-ups.