Investigate the anonymous "internal error" stream from the field surface

[schmug]

Task

A dev-server run of dewpt emitted a continuous stream of opaque Error: internal error; reference = <id> lines from workerd, with no attributable HTTP request and no stack. It was isolated to sessions in which the field surface (/) had been loaded, and did not reproduce on a server exercising only /board. After this issue, we know what throws, whether it costs metered Workers AI calls, and either it is fixed or it is a documented known-benign condition. Today it is an unexplained recurring error in the default surface of the app.

Context

Observed 2026-08-08 while browser-verifying #61. Deliberately not attributed to that PR, and here is the isolation evidence:

• Every /api/board/* response in the same window was 2xx.
• None of the errors carried BoardDO's own signatures — board pump failed, hop failed, station prepare failed — which are the strings the board's own catch blocks emit as JSON.
• A freshly restarted dev server that loaded only /board, ran a full lineage lifecycle to the edge with real Workers AI generation, and evaporated it, produced zero errors.
• The errors appeared only in the earlier server instance, which had also loaded / (the field surface) — which creates a SessionDO, preseeds, and runs its own pool pump on an alarm loop, independently of whether any client is still connected.

That last point is the leading hypothesis and nothing more: the shape (recurring, no request line, no stack) is consistent with a throw inside an alarm handler or a remote-binding transport fault, and env.AI is bound remote in local dev. It has not been confirmed, and I did not diagnose it.

Why it matters beyond noise: if the source is SessionDO's pool pump throwing and retrying, each cycle may be a metered Workers AI call against a live account, burning quota for a tab nobody has open. That would make this a cost bug rather than a logging annoyance. It may equally be a benign local-dev artifact of the remote-binding proxy. Both are plausible; neither is established.

Reproduction

1. npm run dev (or preview_start with the dewpt config in .claude/launch.json).
2. Load http://localhost:8787/ — the field surface — and let it initialise.
3. Navigate away to http://localhost:8787/board/, or simply leave the server running.
4. Watch the wrangler output.

Expected: no errors, or errors that name what failed.

Actual: a repeating stream of ✘ [ERROR] Error: internal error; reference = <random id> with no request line, no stack, and no JSON payload. Interleaved with successful 200 OK request lines for unrelated routes.

Environment: wrangler 4.111.0, macOS (darwin 25.5.0), local dev with env.AI bound remote, Node via the repo's toolchain. Not observed in production — nobody has looked.

Pointers

• src/session-do.ts — the field's Durable Object; its alarm() and pool pump are the leading suspect
• src/board/board-do.ts:558 — the board's alarm(), whose catch logs board pump failed; the pattern to compare against, and the reason we can say the board is not the source
• src/generation.ts — embedTexts / expandPole; these throw on transient AI faults, and where those throws land is the question
• src/index.ts — /api/debug/ai, the existing probe (returned ok, 635ms, 1024 dims throughout, so the binding was healthy)
• public/app.js, public/preseed.js, public/preseed-pool.js — what the field kicks off on load
• wrangler.jsonc — the AI binding's remote mode
• CLAUDE.md — the WARP and Cloudflare Access traps, both of which produce different symptoms and should be ruled out first
• feat: board speed presets and a pause control #61 — where this was observed and isolated

Constraints

• Do not silence it with a broad catch. An unexplained error that stops printing is strictly worse than one that prints.
• If the fix is a catch, it must log something that names the operation, matching the JSON.stringify({ level, message, ... }) shape already used in board-do.ts and session-do.ts.
• Reproduce before changing anything — this may be a local-dev artifact of the remote-binding proxy and not a code defect at all, in which case the deliverable is documentation, not a patch.
• Rule out the two known local traps first (WARP blocking workerd egress; Cloudflare Access on the workers.dev domain). Both are in CLAUDE.md and both masquerade as app bugs.
• Any AI calls made while investigating are metered: state and cap the expected number before running a loop.

Acceptance criteria

• The throwing code path is named, with a file and line — or it is established that the error originates outside our code, with the evidence for that.
• It is stated whether each occurrence costs a Workers AI call, and roughly at what rate.
• Either the error no longer appears in a normal dev session, or CLAUDE.md gains an entry describing it as known-benign alongside the WARP and Access traps, so the next person does not re-investigate it.
• No new broad catch that hides an error without logging it.

Out of scope

• Anything on the /board surface — it was shown clean in isolation.
• The 7 Dependabot advisories on the default branch; unrelated, separate track.
• Refactoring SessionDO beyond what diagnosing this requires.

Reasoning guidance

Think carefully and step-by-step; this is harder than it looks. The error is anonymous by construction, so the work is mostly building an attribution method — bisecting which surface triggers it, and instrumenting the alarm paths — rather than reading code until the answer appears.