Skip to content

bake: consolidate the open robobun dev-server, HMR runtime, production build and router fixes - #39488

Open
Jarred-Sumner wants to merge 178 commits into
mainfrom
claude/bake-consolidation
Open

Jarred-Sumner wants to merge 178 commits into
mainfrom
claude/bake-consolidation

Conversation

@Jarred-Sumner

@Jarred-Sumner Jarred-Sumner commented Aug 18, 2026 •

Copy link
Copy Markdown
Collaborator

Consolidates the open robobun Bake PRs from Aug 12–17 (71 of them) into one branch, re-thought per theme so that each root cause is fixed once instead of once per symptom. Every PR was checked against current main, its review threads read, and its fix either taken, reworked, or deferred; the details per PR are in the commit that carries its number.

What changed, by theme

HMR runtime (hmr-module.ts) — one module-loading state machine: Pending/Stale/Loaded/Error plus a per-module generation and an in-flight loading {promise, asyncId}; beginEvaluation is the only Stale→Pending point; every failure path goes through throwLoadFailure; require() throws a fresh Error instead of mutating the loader's; dynamicImport is async and the import(id, opts) fallback returns its promise; replaceModules re-evaluates the modules between a changed module and its accepting boundary (Vite semantics), runs dispose/accept once per module per update, lets failed roots load again, and settles all in-flight reloads before it rejects; bun:wrap exports the whole runtime.bun helper namespace (decorators work in dev); vite:* event names are aliased. The server runtime observes the update promise instead of dying on an unhandled rejection.

Dev server incremental graph / failure bookkeeping — three invariants implemented once: (1) File.failed ⇔ bundling_failures contains it, and every flip is published (IncrementalGraph::retract_failure, used on demotion, deletion and re-bundle); (2) a file's parts are emptied only when a failure is recorded on a node reachable from its importers, and a failed file never gets a chunk (single CSS-root discovery loop; failed-root removals are final; plugin fall-through inside try/catch keeps the module bundled; an onLoad plugin that does not answer fails the route; a "use client" file whose import fails keeps its route failed); (3) check_route_failures traces into a scratch list and re-bundles a failing route once per file change instead of on every request. Also: CSS roots get edges to every import and stylesheets are served in source order; a framework route's meta.styles follows its imports; a changed file is re-bundled with the loader it was bundled with (text-imported HTML no longer panics); the first hot-reload event of a batch is processed once; server.reload() reuses an HTML file's route bundle; relative_path handles a root of /; parse-opened fds the watcher declines are closed; a failed onLoad releases its request before giving the scan unit back.

Dev server lifetime — Box<DevServer> is fully initialized before anything fallible runs and Drop is the single teardown path (init failures no longer leak, and a never-started watcher closes its fd); the app plugin cell is an RAII OwnedPlugin; a pending [serve.static] plugin load holds a pending request so stop() cannot free the server under it; deferred requests are unlinked in Drop; JS-thread bundle allocations go into the bundle's arena through an RAII scope guard. The memory-visualizer timer is re-armed from its tick handler (it corrupted the timer heap on canary); the HMR socket unsubscribes topics dropped by a re-subscribe; the incremental/memory visualizer pages and graph feed are restored (bitset, one serializer), with .bake-debug source dumps behind BUN_BAKE_DUMP_SOURCES=1. The error-overlay wire format handles line-only and column-less locations.

bun build --app failure/exit model — build_with_vm never exits the process; it returns Ok, JSError, or the new Error::BakeBuildFailed ("already printed"); build_command has one tail that sets the exit code and exits through on_exit() + global_exit() for success too, so process.on("exit") and process.exitCode work; a bundler BuildFailed is converted at the bundle call (log printed, exit 1, no "internal error"); an unwritable output directory or file fails the build before prerendering; route scan errors (aliases, unsupported files) fail the build; all Global::crash() sites are gone. The production transpilers and framework projection are dropped (TranspilerSlot); the C++ module helpers check exceptions, take JSC::JSGlobalObject*, and release the strings production.rs hands them; BakeProdResolve handles Windows disk paths.

Production linker — AstBuilder modules get a real exports_ref/wrapper_ref (namespace/require() of a "use client" proxy or bun:bake/server works); server-component boundaries are redirected per importer side, so client-side imports of a "use client" file bind to its client build; route classification runs before import() records are cleared and uses one reverse BFS from the client proxies (cycle-safe; skipped in dev), so import() of a "use client" module works in production; routes with layouts prerender and stay static only if every layout is.

Router / config — one route-matching contract: [x] matches exactly one non-empty segment, [...x] one or more, [[...x]] zero or more, and among matching dynamic patterns the winner is by Next.js precedence (text < param < catch-all < optional catch-all; same rule #39291 gives Bun.FileSystemRouter). Scan/config errors are reported instead of panicking (function style, over-long roots and entries, aliased/unsupported route files, ignoreDirs actually honored, plugins must be arrays); app.root and router roots go through one checked directory resolver; the local get_optional_value shims are replaced by JSValue::get_optional::<JSValue>.

Module ids / paths — dev-server module ids are relative to the dev server root (app.root or the cwd at creation), so app.root != cwd and process.chdir() after Bun.serve work; over-long unresolvable imports and root-absolute <script src> are reported instead of panicking.

Found while consolidating — Path::dupe_alloc put text in the per-bundle arena when pretty was disjoint from it (every ssr: path); the watcher borrows that slice, so a hot reload after the bundle was freed read freed memory (intermittent ASAN BUS on the watcher thread, present on main).

Tests

  • Harness: the error overlay is checked once instead of polling 5×200 ms on every connect/write; page load is an IPC ack; every client request rejects when the client exits; multi-HTML route keys are posix; new helpers countBundles, watchSynchronizationMessagesFor, expectErrorPage, expectElemText. bundle.test.ts and css.test.ts share dev servers and assert served output.
  • Every fix carries the test from its PR (all fail on the unfixed build), folded onto shared servers where the fixtures allow. test/bake/framework-router.test.ts covers matching via FrameworkRouter.match() from bun:internal-for-testing; test/bake/app-options.test.ts and test/bake/dev/route-matching.test.ts are new; production.test.ts now covers failure and exit paths and runs under BUN_JSC_validateExceptionChecks.
  • LeakSanitizer is re-enabled for the eight bake files that no longer leak; bundle, css, production stay listed for one CI cycle (css needs watcher: free the owned path of evicted watchlist entries #39197).
  • Locally (debug ASAN, macOS): all of test/bake + bun-serve-html + BunFrontendDevServer + related bundler files: 758 pass, 0 fail apart from the pre-existing --compile timeouts in cli.test.ts and an occasional flake of the pre-existing deinitialization fixture (stop() then fetch races a 250 ms timer; unchanged here); rust:check-all clean.

Behaviour changes

  • A successful bun build --app runs process.on("exit") handlers and exits with process.exitCode; an unwritable output or a route scan error exits 1 without prerendering.
  • Routing: pages/[user]/posts.tsx now matches /joe/posts (1.3 only matched /joe/posts/); one trailing / is ignored for every route (static ones too, like Bun.FileSystemRouter); pages/[...slug].tsx stops matching / and docs/[...rest].tsx stops matching /docs; missing trailing params (/a/b against [c]/[y]/[s]) stop matching instead of binding empty strings; // inside a catch-all no longer matches; overlapping dynamic routes resolve by Next.js precedence, not directory-scan order; requests are matched on the pathname (query string ignored); fileSystemRouterTypes[n].root may live outside app.root; ignoreDirs actually applies (main iterated the wrong array) and replaces the defaults; colliding routes fail bun build --app; non-array plugins is an error.
  • Dev server: a route left in a build-error state is re-bundled once per watcher batch or server.reload() instead of on every request; a file's import attribute (with { type }) is honoured on rebundle.
  • HMR now re-executes the modules between the changed file and its accepting boundary (Vite semantics; what the import.meta.hot docs promise). Concretely for bun ./index.html React apps: editing a leaf component re-runs the modules that import it up to the nearest import.meta.hot.accept boundary (in React apps that is the Fast Refresh boundary, so component modules only), and module-level state in intermediate non-component files (store.ts, context.ts style singletons that are not accepted themselves) is re-created on such an edit where 1.3 kept the old instance and only patched bindings. Modules that call import.meta.hot.accept() / hot.data keep their state as before. An unaccepted client update reloads the page without running dispose first. This wants a docs note.
  • server.reload() with an unchanged HTML route serves the cached bundle.
  • style: () => … in fileSystemRouterTypes is rejected with ERR_INVALID_ARG_TYPE (it always panicked); app.root, plugins[n].name, serverRuntimeImportSource, serverRegisterClientReferenceExport must be strings.
  • .bake-debug dumps require BUN_BAKE_DUMP_SOURCES=1.

Deferred (own PRs)

#38714 (UTF-8 module text; part of a stack — rebases in one line), #38617 (crash-dump; touches four non-bake crates), #37872 (overlay underline; needs #37848), #39197 (watcher eviction leak).

Supersedes

#37120 #37727 #37734 #37742 #37759 #37765 #37773 #37779 #37785 #37786 #37794 #37795 #37827 #37833 #37837 #37838 #37844 #37845 #37850 #37859 #37862 #37863 #37866 #37867 #37868 #37871 #37878 #37885 #37886 #37888 #37890 #37892 #37897 #37899 #37902 #37929 #37936 #38004 #38041 #38227 #38233 #38234 #38238 #38241 #38252 #38323 #38336 #38340 #38664 #38737 #38902 #38949 #38960 #38973 #39050 #39092 #39150 #39174 #39188 #39196 #39202 #39203 #39209 #39293 #39295 #39301 #39302 #39315 #39344 #39432 #39447 (and #38462)

Fixes #22533
Fixes #23299
Fixes #24893
Fixes #28117
Fixes #30488

…h the VM (#39302, #38241)

Replace the Global::crash() and fail_with_build_error exits in production.rs with a
BakeBuildFailed error the command tail turns into exit code 1, name the output
directory in the open failure, and route the success path through on_exit +
global_exit so 'exit' handlers and process.exitCode apply to a finished build.
#38233)

TranspilerSlot lives in bake/mod.rs so the dev server can share it.
Rust calls BakeLoadModuleByKey/BakeGetModuleNamespace/BakeGetDefaultExportFromModule/
BakeGetOnModuleNamespace through from_js_host_call; the module loader hooks release
their throw scopes; BakeRegisterProductionChunk (no callers) is deleted.
…Path (#39092)

Also drops the Windows skip on the out-of-bundle import exception-check test and
gives the transpiler-leak tests the bake harness's production build budget.
…#37862, #37888)

Adds IncrementalGraph::retract_failure so every path that stops a node from
failing without going through insert_failure publishes the removal.
…lugin-namespace module and release the failed load before giving its scan unit back (#37897, #37871)
robobun added a commit that referenced this pull request Oct 28, 2026
…a client file

The gate is the one from #39488: a production build only, `Loader::Css`
only, the file namespace only. The dev server tracks a stylesheet per
importing graph, and a plugin namespace can turn the same path text into
anything.

The shared stylesheet is always parsed as a client file. Before, the graph
that registered it first parsed it, so the class names of a CSS module and
the export condition of its `@import` differed between identical builds
(2 outputs in 32 builds of one fixture).

`css_target()` reads `framework`, which every bake transpiler has. The
ParseTask condition is main's again: it picked the transpiler that
`transpiler_for_target(task.known_target)` had already picked.

Co-authored-by: Alistair Smith <hi@alistair.sh>
@robobun

robobun commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

The output write item of this PR (from #39295) is re-cut on current main as #47194. It has the same OutputDir recorder at the three write_to_disk sites and the same gate position. The gate in #47194 is Global::crash(), because main has no Error::BakeBuildFailed. With the exit model of this PR, only that statement changes. #47194 asks in its notes which exit form is wanted until then.

@robobun

robobun commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

The CSS target item of this PR (b1e8210, with the sharing gate of 6a23d31 and 5451190) is on current main in two PRs.

Neither PR has the shared source index. In #37051 that sharing failed two builds that pass without it: two routes that share a server component whose stylesheet has a local url(), and a CSS module with a non-ASCII class name. The description of #37051 has the fixtures and the numbers.

robobun added a commit that referenced this pull request Oct 30, 2026
React takes any other return value of `write()` as a full buffer. It then
waits for a "drain" event before it writes the next completed Suspense
boundary. The two streams of the React framework never emit that event.

- `StaticRscInjectionStream` (`bun build --app`): with react-dom
  0.0.0-experimental-dc9aed36-20261027, a page with several large Suspense
  boundaries never gave a result.
- `RscInjectionStream` (the dev server): a streamed response could stop after
  one Suspense boundary and never send the next one.

The two lines and the dev server test are from #39488.

In `production.test.ts`:
- The React cases run one at a time, and `build()` kills a build that does
  not exit in 60 s.
- The `getStaticPaths` case that throws runs on a react-dom that does not call
  `onAllReady` after a shell error.
- A new case builds a page with large Suspense boundaries on a react-dom that
  waits for "drain".
- The case for a server component that throws inside `<Suspense>` is gone.
  That page keeps its fallback, so a test must not call it a recovery.

Co-authored-by: Alistair Smith <hi@alistair.sh>
@robobun

robobun commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

The prerender failure item of this PR (80892c6, aeed1f2, 97db92b, and the return true lines of 9d19e58) is re-cut on current main as #47356.

#47356 has the { pages } await, the reject in onShellError, and return true in the two write() methods with the dev server test of this branch. Two things differ on purpose:

  • The flight render of prerender has no onError. So a server component that throws inside <Suspense> still builds as on main. This branch fails that page.
  • prerender still returns undefined for a page with no HTML. So the build error stays Route "..." cannot be pre-rendered to a static page., after the errors that React logs. Three cases of bake: fail bun build --app when a page fails to prerender #47356 expect that line, and production.test.ts:92 of this branch expects that it is gone.

The notes of #47356 ask which result a maintainer wants for each of the two. After #47356, the { pages } hunk and the two write() hunks of this branch are already on main.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants