fix(eve): settle turn as cancelled when abort races step completion - #1389
fix(eve): settle turn as cancelled when abort races step completion#1389AndrewBarba wants to merge 4 commits into
Conversation
A cancellation observed while a durable turn step was already returning lost the race to an ordinary completion: the step body could miss the abort signal and finish done, leaving the session completed after the user had cancelled. Honor the claimed cancel whenever the abort signal fired and the step produced no pending runtime-action batch (those resolve in-line and observe the signal in their own wait). Surfaced by deterministic-model timing in the e2e world suites, where turns reach completion fast enough to expose the race (originally found in #1367). Signed-off-by: Andrew Barba <barba@hey.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Bundle + Package Summary:
|
| Area | Metric | Baseline | Current | Delta |
|---|---|---|---|---|
| Package | Packed tarball | 7.62 MB | 7.62 MB | +1.9 kB |
| Package | Unpacked publish size | 28.79 MB | 28.80 MB | +10.9 kB |
| Package | Installed footprint | 91.09 MB | 91.10 MB | +10.9 kB |
| Package | Published files | 2875 | 2879 | +4 |
| Package | Installed files | 6662 | 6666 | +4 |
| Runtime | Unique function payloads | 2 | 2 | 0 |
| Runtime | Total function bytes | 16.80 MB | 16.82 MB | +19.3 kB |
| Runtime | Public routes | 11 | 11 | 0 |
Changed function payloads vs main (46c6ce3) (2)
| Function | Status | Baseline | Current | Delta | Route changes |
|---|---|---|---|---|---|
functions/__server.func |
changed | 8.40 MB | 8.41 MB | +9.6 kB |
none |
functions/.well-known/workflow/v1/flow.func |
changed | 8.40 MB | 8.41 MB | +9.6 kB |
none |
eve init install
| Metric | Baseline | Current | Delta |
|---|---|---|---|
| Installed footprint | 129.49 MB | 129.50 MB | +10.9 kB |
| Installed packages | 130 | 130 | 0 |
| dependencies | 4 | 4 | 0 |
| devDependencies | 2 | 2 | 0 |
| Dependency package bytes | 43.12 MB | 43.13 MB | +10.9 kB |
| devDependency package bytes | 5.04 MB | 5.04 MB | 0 B ➖ |
Build Metadata
- Preset:
vercel - Nitro:
nitro@3.0.260610-beta - Output directory:
apps/fixtures/weather-agent/.vercel/output - Build metadata timestamp: 2026-07-30T03:12:18.204Z
- Route aliases: 11 public, 1 internal (12 total aliases)
- Vercel routes in config: 14
- Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down
Package Details
- Package:
eve@0.27.13 - Package directory:
packages/eve - Tarball: 7.62 MB (
eve-0.27.13.tgz) - Unpacked payload: 28.80 MB across 2879 published files
- Installed footprint: 91.10 MB across 6666 installed files
- Installed root package: 27.44 MB
- Installed dependencies: 63.66 MB
- Runtime dependencies: 2
- Peer dependencies: 5 (4 optional)
Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Heavy installed dependencies
eve: 27.44 MB (30.1%)@rolldown/binding-linux-x64-gnu: 19.28 MB (21.2%)@rolldown/binding-wasm32-wasi: 10.66 MB (11.7%)ai: 6.53 MB (7.2%)@napi-rs/wasm-runtime: 6.39 MB (7.0%)
Publish payload breakdown
Published file size
🔴 dist/src/compiled/shadcn-registry/index.js [########################] 13.15 MB 45.7%
🟠 dist/src/compiled/experimental-ai-sdk-code-mo... [###.....................] 1.51 MB 5.2%
🟡 dist/src/compiled/@vercel/sandbox/index.js [#.......................] 632.5 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-DWL... [#.......................] 502.4 kB 1.7%
🟡 dist/src/compiled/@chat-adapter/slack/index.js [#.......................] 440.5 kB 1.5%
🔴 Other published files [#######################.] 12.56 MB 43.6%
Installed footprint breakdown
Installed package size
🔴 eve [########################] 27.44 MB 30.1%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 19.28 MB 21.2%
🔴 @rolldown/binding-wasm32-wasi [#########...............] 10.66 MB 11.7%
🔴 ai [######..................] 6.53 MB 7.2%
🔴 @napi-rs/wasm-runtime [######..................] 6.39 MB 7.0%
🔴 zod [####....................] 5.07 MB 5.6%
🔴 Other installed packages [##############..........] 15.73 MB 17.3%
Runtime dependencies (2)
| Package | Range | Notes |
|---|---|---|
nitro |
3.0.260610-beta |
|
undici |
8.9.0 |
Peer dependencies (5)
| Package | Range | Notes |
|---|---|---|
@opentelemetry/api |
^1.0.0 |
optional peer |
ai |
catalog: |
|
braintrust |
^3.0.0 |
optional peer |
just-bash |
^3.0.0 |
optional peer |
microsandbox |
^0.5.0 |
optional peer |
eve init install drill-down
eve init install details
- Command:
eve init my-agent - Package manager:
npm - Installed footprint: 129.50 MB across 8534 installed files
- Installed packages: 130 total (124 transitive-only)
- dependencies: 4 direct packages totaling 43.13 MB
- devDependencies: 2 direct packages totaling 5.04 MB
- Other transitive package files: 81.33 MB
Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.
Heavy installed dependencies
@typescript/typescript-linux-x64: 27.95 MB (21.6%)eve: 27.44 MB (21.2%)@rolldown/binding-linux-x64-gnu: 19.28 MB (14.9%)@rolldown/binding-wasm32-wasi: 10.66 MB (8.2%)zod: 9.02 MB (7.0%)
Installed footprint breakdown
Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 21.6%
🔴 eve [########################] 27.44 MB 21.2%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 19.28 MB 14.9%
🔴 @rolldown/binding-wasm32-wasi [#########...............] 10.66 MB 8.2%
🔴 zod [########................] 9.02 MB 7.0%
🔴 ai [######..................] 6.53 MB 5.0%
🔴 @napi-rs/wasm-runtime [#####...................] 6.39 MB 4.9%
🔴 Other installed packages [###################.....] 22.23 MB 17.2%
dependencies (4)
| Package | Range | Installed size | Share |
|---|---|---|---|
@vercel/connect |
0.4.2 |
135.8 kB | 0.1% |
ai |
^7.0.38 |
6.53 MB | 5.0% |
eve |
file:eve-0.27.13.tgz |
27.44 MB | 21.2% |
zod |
4.4.3 |
9.02 MB | 7.0% |
devDependencies (2)
| Package | Range | Installed size | Share |
|---|---|---|---|
@types/node |
24.x |
2.54 MB | 2.0% |
typescript |
7.0.2 |
2.50 MB | 1.9% |
Function Drill-Down
Payload Size Graph
Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func [########################] 8.41 MB 50.0%
🔴 functions/__server.func [########################] 8.41 MB 50.0%
Top Function Payloads
🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 8.41 MB
| Metric | Value |
|---|---|
| Public routes | /.well-known/workflow/v1/flow |
| Runtime | nodejs24.x |
| Handler | index.mjs |
| Payload | 8.41 MB |
| Function files | 8.41 MB across 44 files |
| Traced dependencies | 0 B |
| Signal | 🟠 Bundled file index.mjs is 2.31 MB (27.5%) |
🟠 🔎 Dependency Analysis
📦 Bundled files:
Bundled file size
🟠 index.mjs [########################] 2.31 MB 27.5%
🟠 _chunks/runtime-artifacts.mjs [################........] 1.59 MB 18.9%
🟡 _libs/undici.mjs [##########..............] 980.5 kB 11.7%
🟡 _chunks/sandbox.mjs [########................] 768.8 kB 9.1%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.1%
🟠 Other bundled files [########################] 2.33 MB 27.6%
🧾 Vercel Config
{
"handler": "index.mjs",
"launcherType": "Nodejs",
"shouldAddHelpers": false,
"supportsResponseStreaming": true,
"runtime": "nodejs24.x",
"maxDuration": "max",
"experimentalTriggers": [
{
"type": "queue/v2beta",
"topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
"consumer": "default",
"retryAfterSeconds": 5,
"initialDelaySeconds": 0
}
],
"environment": {
"WORKFLOW_PRECONDITION_GUARD": "1"
}
}🟠 functions/__server.func • 10 public routes, 1 internal alias • 8.41 MB
| Metric | Value |
|---|---|
| Public routes | //eve/v1/callback/[token]/eve/v1/connections/[name]/callback/[token]/eve/v1/health/eve/v1/info/eve/v1/session/eve/v1/session/[sessionId]/eve/v1/session/[sessionId]/cancel/eve/v1/session/[sessionId]/stream/eve/v1/session/reset |
| Internal aliases | /__server |
| Runtime | nodejs24.x |
| Handler | index.mjs |
| Payload | 8.41 MB |
| Function files | 8.41 MB across 44 files |
| Traced dependencies | 0 B |
| Signal | 🟠 Bundled file index.mjs is 2.31 MB (27.5%) |
🟠 🔎 Dependency Analysis
📦 Bundled files:
Bundled file size
🟠 index.mjs [########################] 2.31 MB 27.5%
🟠 _chunks/runtime-artifacts.mjs [################........] 1.59 MB 18.9%
🟡 _libs/undici.mjs [##########..............] 980.5 kB 11.7%
🟡 _chunks/sandbox.mjs [########................] 768.8 kB 9.1%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.1%
🟠 Other bundled files [########################] 2.32 MB 27.6%
🧾 Vercel Config
{
"handler": "index.mjs",
"launcherType": "Nodejs",
"shouldAddHelpers": false,
"supportsResponseStreaming": true,
"runtime": "nodejs24.x"
}Build Timing: e2e/fixtures/agent-tools-sandbox
This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.
Build mode: deployable Vercel build with sandbox template prewarm included.
- Build pipeline: 2.21 s -> 1.85 s (-360.1 ms) vs
main (46c6ce3). - Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `main (46c6ce3)`
| Phase | Baseline | Current | Delta |
|---|---|---|---|
extension.check |
9.7 ms | 7.1 ms | -2.6 ms |
project.resolve |
3.5 ms | 2.5 ms | -1.0 ms |
workspace.create |
1.2 ms | 1.2 ms | 0.0 ms |
host.prepare |
518.2 ms | 160.9 ms | -357.3 ms |
vercel.service-prefix.resolve |
1.7 ms | 1.7 ms | 0.0 ms |
nitro.create |
193.0 ms | 191.3 ms | -1.7 ms |
sandbox.prewarm |
187.8 ms | 193.9 ms | +6.1 ms |
nitro.cache.prepare |
0.3 ms | 0.3 ms | 0.0 ms |
nitro.prepare |
0.8 ms | 0.8 ms | 0.0 ms |
nitro.public-assets |
0.7 ms | 0.7 ms | 0.0 ms |
nitro.prerender |
0.5 ms | 0.5 ms | 0.0 ms |
nitro.bundle |
1.26 s | 1.26 s | -3.4 ms |
nitro.cache.write |
0.3 ms | 0.3 ms | 0.0 ms |
vercel.workflow-function.materialize |
21.4 ms | 21.1 ms | -0.3 ms |
agent-summary.emit |
0.4 ms | 0.5 ms | +0.1 ms |
nitro.close |
0.1 ms | 0.1 ms | 0.0 ms |
output.publish |
3.2 ms | 3.2 ms | 0.0 ms |
workspace.remove |
1.8 ms | 1.9 ms | +0.1 ms |
The e2e cancellation flake has a second face beyond the settle race: on the Vercel world, a cancelled parent's cascade to its subagent child can drop entirely — the child's wait-for-cancellation tool rides out its fallback timeout and the child runs to completion with no turn.cancelled. Every drop path in cancelDescendantTurnsStep was silent: a missing pending batch, missing child session ids, and retry exhaustion all returned as if the cancel had been delivered. Retry descendant cancels with exponential backoff (~8s budget instead of a flat 3s), classify no_active_turn results with the error that marked the target inactive (HookNotFoundError vs EntityConflictError etc.), and warn on every path that gives up so an uncancelled child is at least observable in logs while a durable delivery design is worked out. Signed-off-by: Andrew Barba <barba@hey.com>
|
Added a second commit after digging into this PR's own e2e failures: the two Failure forensics ( Every drop path in The hardening commit:
This makes the drop far less likely and diagnosable when it still happens. A fully durable delivery (child consults durable cancel state at settle time, instead of best-effort hook resume) is a design change worth its own research doc — this keeps the fix reviewable. The remaining e2e failures on the first run ( |
A wake that replays a journaled cancel alongside a completed turn step drives both continuations in one microtask drain, payload first. The abort was chained on consumeMatchingCancel with .then, so it ran one hop behind the turn loop's continuation: the loop checked signal.aborted before the abort fired, took the done arm, and disposed the pending cancel. Fire the abort inside the matching read's own continuation so the signal is already flipped when any same-drain continuation checks it. Observed on the Vercel world as a subagent child completing normally after its parent's cascade cancel had been accepted. Signed-off-by: Andrew Barba <barba@hey.com>
|
The single The microtask race: when the child run wakes and replays a journaled cancel alongside a completed turn step, both continuations resolve in the same drain, payload first. But the abort was chained on Fix: fire the abort inside the matching read's own continuation, so the signal is already flipped when any continuation from the same drain checks it. The new unit test reproduces the drain ordering (payload resolves before the step result) and fails against the chained- Residual risk, for the record: the run→step abort bridge (the durable-stream side channel workflow-core uses to propagate an abort into an in-flight step invocation) swallows write failures; if that hop is lost, the child's tool still rides out its fallback timeout before the (now-reliable) run-side settle path catches the cancel at the next step boundary — slow, but no longer droppable by eve. Making descendant cancellation fully durable (child consults durable cancel state at settle) is follow-up design work. |
The descendant cascade previously lived only inside the parent's workflow (cancelDescendantTurnsStep), which cannot execute while the parent run is suspended — including while it is parked awaiting the very child it needs to cancel. When the workflow world deferred the parent's cancel wake until the child's completion delivery, the cascade was dead on arrival: the child ran its full fallback timeout and completed uncancelled (observed repeatedly in the cancel-subagent e2e eval). runtime.cancelTurn now cancels the target and immediately fans out to every dispatched-but-unresolved descendant, recursively, without any run having to wake. Descendants are derived from the session's own durable event stream — subagent.called records the dispatch and action.result marks it resolved — read up to the tail observed on entry so a live stream cannot block the request. Remote children are re-resolved from the registry by dispatch URL and cancelled over the standard remote cancel route, which fans out on its own deployment. The settle-time cascade remains as the durable backstop, but no longer burns its 8-attempt retry budget on children that were already cancelled at request time: terminal no_active_turn reasons (hook gone, run gone, run expired) return after a single attempt, and only hook-claim contention is retried. Signed-off-by: Andrew Barba <barba@hey.com>
|
Root cause found and fixed structurally in The smoking gun (from the run-3 artifact timestamps): the cancel was accepted at The fix: Deliberately avoided: extending the public Unit (5,670) + integration (568) green, guard/docs checks pass. The dropped/deferred hook-resume wake in the workflow world is still worth an upstream look — with this change eve no longer depends on it for cancellation correctness, but it cost this parent 92 seconds of suspension with an accepted cancel pending. |
Why
A turn cancellation observed while a durable turn step is already returning loses the race to an ordinary completion: the step body can miss the abort signal and finish
done, leaving the session in a completed state after the user's cancel was claimed.What
In
runTurnOwnedWorkflow, honor the claimed cancel whenever the cancellation signal has fired and the step produced no pending runtime-action batch — pending batches are exempt because their in-line wait observes the signal itself. The turn then settles through the existing cancelled-park path (finishCancelledTurn), whose parkability was already established when the cancel hook was claimed.Includes a unit test that returns a
doneresult only after the step's abort signal fires, asserting the turn parks as cancelled and never resumes asdone.Provenance
Extracted from #1367, where deterministic-model timing in the e2e world suites made turns complete fast enough to expose the race. Split out here so the runtime fix can be reviewed independently of the e2e suite changes (#1387).
Testing
pnpm --filter eve exec vitest run --config vitest.unit.config.ts src/execution/turn-workflow.test.ts(23 passed)