Skip to content

test(aot): share one program between three tests of entry-points-that-are-not-text.test.ts and compare embedded files by SHA-256 - #47541

Open
robobun wants to merge 2 commits into
mainfrom
robobun/ca8d50a5/faster-entry-points-that-are-not-text
Open

robobun wants to merge 2 commits into
mainfrom
robobun/ca8d50a5/faster-entry-points-that-are-not-text

Conversation

@robobun

@robobun robobun commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Behaviour change: none

Problem

  • test/aot/cli/entry-points-that-are-not-text.test.ts takes 45 s on darwin x64 (build 135961). Linux takes 3 s.
  • Its 4 tests make 8 builds and start 8 new executables. On that macOS agent a first start waits about 3.3 s (Notes).
  • Runs compare only stdout and the exit code.

Fix

  • Three tests share one program, built with --aot and without: 4 builds and 4 executables, all at the same time.
  • Each run compares stdout, stderr, exit code and signal. An embedded file is compared by name, size and SHA-256.
  • The 4 tests keep their names and lanes.
  • Verified: bun bd test test/aot/cli/entry-points-that-are-not-text.test.ts takes 35 s, was 47 s. Release build, 2 CPUs: 1.5 s, was 2.8 s. macOS is not measured. Also test-files-self-skip, every-test-runs-in-ci.

Background

  • bun build --compile a.ts b.bin writes a copy of Bun that starts at a.ts and holds b.bin, a further entry point, as a file.
  • The shared program prints a line for each file it holds and what require("./tool.unknown") answers. Each test reads its lines.
  • Considered 8 builds at once: 8 first starts still wait on macOS. Considered --aot-separate-programs: only canary builds start its programs.

Downsides

  • A build error in the shared program fails three tests.
  • No build has one further entry point alone now. The program that requires tool.unknown has them too. build-carries-files-that-are-no-code.test.ts covers that require without them.
  • On 12 idle CPUs the gain is small: 1.9 s, was 2.6 s.
Notes

Where the time goes. Release build of 34cbc70, Linux x64. Each build ran alone, best of 3. CPU is user plus system time. The functions and the engine runs are from --aot-report.

Build Before CPU before After CPU after Functions compiled Engine runs
--aot, the program and a further entry point that is no code 2 3.54 s 1 1.77 s 438, now 444 2
the same without --aot 2 0.28 s 1 0.13 s
--aot tool.unknown 1 0.23 s 1 0.23 s 1 1
a program that requires tool.unknown, without --aot 1 0.12 s in the shared build
the same with --aot 1 0.22 s in the shared build 2 1
--aot tool 1 0.23 s 1 0.23 s 1 1
total 8 4.63 s 4 2.36 s
  • A preload that wraps Bun.spawn lists the child processes of the file. Before: 18. That is 8 builds, 8 starts of an executable, bun app.ts, and the probe of test/aot/harness/compiler.ts. After: 10. That is 4 builds, 4 starts, and the same two.
  • Windows skips the link test. It made 6 builds and now makes 4.
  • Each build writes a copy of Bun: 138 MB with the stripped release build, 1.6 GB with the debug build.

Which build became which.

Before After
--aot app.ts table.unknown, --aot app.ts linked.node, --aot app.ts (requires ./tool.unknown) --aot app.ts table.unknown linked.node
app.ts table.unknown, app.ts linked.node, app.ts (requires ./tool.unknown) app.ts table.unknown linked.node
--aot tool.unknown the same
--aot tool the same
bun app.ts the same

All local times. Linux x64, 12 vCPUs, a shared VM. Main's file and this PR's file ran in turns. A time is the wall time of the command.

Bun CPUs Before After
debug with ASAN (bun bd test <file>), 5 rounds 12 43.1 to 51.1 s, median 46.9 s 29.5 to 44.1 s, median 35.2 s
release, 15 rounds 12 1.75 to 3.35 s, median 2.55 s 1.65 to 2.09 s, median 1.85 s
release, 15 rounds 4 (taskset -c 0-3) 1.68 to 2.83 s, median 2.11 s 1.14 to 1.92 s, median 1.58 s
release, 11 rounds 2 (taskset -c 0-1) 2.31 to 3.38 s, median 2.76 s 1.14 to 1.74 s, median 1.45 s
release, 7 rounds 1 (taskset -c 0) 3.62 to 4.18 s, median 3.95 s 1.84 to 2.16 s, median 1.91 s
release, CPU time of all processes 12 median 5.57 s median 2.98 s
release, CPU time of all processes 2 median 4.04 s median 2.15 s
  • Release is the stripped bun of bun run build:release at 34cbc70, with the variables that scripts/runner.node.ts sets: BUN_GARBAGE_COLLECTOR_LEVEL=1, BUN_JSC_randomIntegrityAuditRate=1.0, BUN_RUNTIME_TRANSPILER_CACHE_PATH=0, BUN_FEATURE_FLAG_NO_ORPHANS=1.
  • The Linux lanes of CI run two files that build executables side by side (7 files that build many executables (2-wide) in build 135961). The row with 2 CPUs stands for that: 2.76 s here, 3.4 s on debian 13 x64 in that build.
  • In the debug build the --aot build of the shared program takes 28 s alone, and each of the 4 builds writes 1.6 GB. The runs vary with the state of the disk.
  • The ASAN lane of CI tests a release build with ASAN. The debug row is the nearest that this machine has.
  • Every run above has 4 pass. expect() calls: 17 before, 20 after.
  • The times are of the first commit. The second commit changes what the program prints for a name and where the folder is made. It adds no process.

CI on main. Seconds of the file in build 135961, then in build 135905: darwin x64 45, 46. debian 13 x64-asan 21, 21. darwin aarch64 13, 1. windows 11 aarch64 5, 5. windows 2019 x64 4, 6. alpine x64 4, 4. ubuntu x64 4, 3. debian x64, debian aarch64, ubuntu aarch64 and alpine aarch64 3, 3.

macOS. Not measured: this machine is Linux x64. The numbers here are from the logs of CI.

  • In build 135961 the file ran on the agent darwin-x64-15.7.9-1, in a group of 12 files that build many executables (4-wide). It took 44.99 s.
  • In the same job run-parallel.ts reports 40 builds: 159.0 s, 162.4 s of CPU and 273 executables: 145.2 s, 13.2 s of CPU. The builds do not wait. The runs wait 132 s. That is 3.3 s for each new executable.
  • Build 135905, agent darwin-x64-14.8.9-1: the file took 46.01 s, and 273 executables: 150.5 s, 12.4 s of CPU.
  • debian 13 x64 in build 135961: 24 executables: 0.6 s, 0.6 s of CPU.
  • darwin aarch64: 13.22 s on darwin-aarch64-26.6.1-1, which waits too (347 executables: 134.5 s, 21.1 s of CPU). 0.98 s on darwin-arm64-sourdough-tart-15-1 in build 135905, which waits little (347 executables: 36.1 s, 19.7 s of CPU).
  • test(aot): share one build and run the tests of entry-point-names.test.ts concurrently #47463 reports that the first starts do not overlap on a darwin x64 agent. If that holds, 8 builds at the same time leave 8 starts in a row. The file now has 4.

CI times are not like for like. On main the file runs beside other files that build executables (buildingFilesWidth in scripts/runner.node.ts). The runner puts a test file that a PR changes in front and runs it without them (modifiedSerialTests). So the time of the file in this PR's own build is the time of the file alone.

Build 136064 ran the first commit and build 136077 ran the head. The file passed on every lane in both. One run each:

Lane main, beside other files (135961) first commit, alone (136064) head, alone (136077)
darwin x64 45.0 s 5.3 s (darwin-x64-14.8.9-1) 5.4 s (darwin-x64-14.8.9-1)
debian 13 x64-asan 20.7 s 3.4 s 3.6 s
darwin aarch64 13.2 s (darwin-aarch64-26.6.1-1) 0.9 s (darwin-arm64-baguette-tart-26-2) 1.0 s (darwin-arm64-sourdough-tart-15-1)
windows 11 aarch64 5.1 s 1.9 s 2.3 s
windows 2019 x64 4.3 s 2.6 s 2.9 s
alpine 3.23 x64 3.6 s 2.0 s 1.4 s
ubuntu 25.04 x64 3.5 s 1.4 s 2.1 s
debian 13 x64 3.4 s 2.8 s 3.7 s
debian 13 aarch64 3.3 s 4.3 s 2.9 s
ubuntu 25.04 aarch64 3.3 s 1.2 s 1.4 s
alpine 3.23 aarch64 2.8 s 1.7 s 1.0 s

Assertions, before and after.

  • A run of an executable. Before: stdout and the exit code. After: also an empty stderr and no signal.
  • bun app.ts. Before: toMatchObject on stdout and the exit code. After: toEqual on stdout, an empty stderr, exit code 0 and no signal.
  • A further entry point that is embedded. Before: its name and size (table.unknown 3000). After: its name, its size, and the SHA-256 of its bytes, against the bytes that the test wrote. The two files now differ (3000 and 2000 bytes), so one cannot pass for the other.
  • The list of embedded files. Before: one name, exact. After: blob., table.unknown, tool.unknown, exact, without blob. on Windows. Another name, or a name twice, fails. A name is compared as Bun.embeddedFiles has it, but for the hash, as before. So a folder in front of a name fails.
  • A failure of the shared executable shows the names that it printed, stderr, the exit code and the signal in one object.
  • The file that the program requires. Before: required undefined, which any answer without a member n prints. After: the answer is a string whose last part is tool.unknown, and the executable holds tool.unknown with the size and the SHA-256 of the source text. An executable that reads the file as code prints ran 42 and fails.
  • With --aot and without: the lines of the two executables are compared, as before.
  • No inline snapshot. The expected lines hold sizes and digests that the test computes from the bytes it writes, and the list differs on Windows.

The assertions fail when they must. Each line is one change to this PR's file, run with the release build:

  • table.unknown written with 3001 bytes: the first test fails, the others pass.
  • store/blob written with the bytes of table.unknown: the link test fails, the others pass.
  • tool.unknown written with one byte more: the require test fails, the others pass.
  • linked.node left out of the entry points: the three tests of the shared program fail at the list of names.
  • A console.error in the program: the three tests fail at stderr.
  • A line ran 42 printed by the program: the three tests fail at the list of names.
  • store/blob. printed for the link: the three tests fail at the list of names.
  • process.exitCode = 3 in the program: the three tests fail. The failure shows the five names beside exitCode: 3.

Seen in bun, not changed here. With --aot, a further entry point that is no code makes the build compile much more of Bun's library.

Entry points (release build) Functions compiled Engine runs CPU of the build
main's app.ts alone 30 2 0.63 s
main's app.ts and a .ts file 31 2 0.72 s
main's app.ts and table.unknown 438 2 1.73 s
console.log("x") alone 1 1
console.log("x") and a .ts file 2 1
console.log("x") and table.unknown, a .txt, a .json or a .png 437 2
a program that requires table.unknown 2 1
  • --explain=trim prints 240 lines for console.log("x") with a .ts further entry point, and 2098 lines with table.unknown. The second list has uses Bun serve.
  • Debug build, main's app.ts: 26 s of CPU alone, 48 s with table.unknown.
  • One such build is left in this file. On an idle machine the file ends when that build and its run end.

Other checks

  • 30 runs in a row with the release build and the first commit: 30 pass. 20 runs with the second commit: 20 pass.
  • Debug build with the variables of the ASAN lane (BUN_JSC_validateExceptionChecks=1, BUN_JSC_dumpSimulatedThrows=1, BUN_DESTRUCT_VM_ON_EXIT=1, and the ASAN_OPTIONS and LSAN_OPTIONS of the runner): 4 pass.
  • In one run with other files, as a bucket of the runner starts them: bun test --parallel=3 --max-concurrency=2 with image-rejection, carrier-identity, extensionless-entry-points and entry-points-behind-links: 27 pass.
  • --max-concurrency=1, AOT_PROGRAMS_JOBS=1, and one CPU: 4 pass. A build and the run of its executable take one slot of test/aot/harness/commands.ts. A test holds no slot while it waits for a build.
  • -t "not UTF-8" builds tool only.
  • A run that fails leaves no folder. afterAll waits for the builds that are under way, then removes the folder.
  • A run with -t that selects no test of the file makes no folder. bun:test runs no afterAll in a file where a name filter leaves no test, so the folder is made in beforeAll.
  • test/aot/test-files-self-skip.test.ts and test/aot/every-test-runs-in-ci.test.ts pass.
  • prettier --check and oxlint report nothing. tsc reports nothing in the file.
  • test/expected-durations.json and test/parallel-allowlist.json come from CI runs. They are not changed.

Not done.

  • A build for each case, all at the same time. It keeps 8 first starts.
  • withoutDebugInformation() of the harness. On the Linux lanes the runner already hands this file a Bun without debug information, and the helper does nothing on macOS and Windows. Debug build with ASAN: the copy has 1.26 GB where the build has 1.60 GB, and bun bd test takes 38.4 s without the helper and 37.5 s with it (medians of 4 rounds in turns). A local bun-profile: 2.26 s without, 1.88 s with (medians of 6).

no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/aot/cli/entry-points-that-are-not-text.test.ts

…re-not-text

Three of the four tests built a program of their own with --aot and
without: 8 builds, 8 new executables. They now read their lines from
one program that lists the files it holds and requires tool.unknown.
The file makes 4 builds and starts 4 executables, and the builds run
at the same time.

Each run compares stdout, stderr, the exit code and the signal. An
embedded file is compared by its name, its size and its SHA-256. The
require test checks that the answer is the path of the carried file.
@robobun

robobun commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 6:49 AM PT - Oct 31st, 2026

⏳ @robobun, your commit 4dfcf21 is still building in Build #136077, but has 2 failures so far (All Failures):

@robobun

robobun commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: ready for review. The file passes on all 11 lanes with the head commit (build 136077).

Two tests are red in that build, and this PR touches neither. test/aot/engine/ir-6.test.ts fails on main too. test/aot/parallel/workers/spinning-workers-are-terminated-at-once.ts ran over its limit of 90 s on Windows 2019 x64.

How the time was found. A preload that wraps Bun.spawn lists every child process of the file. On main (34cbc70) the file starts 18: 8 builds, 8 executables, bun app.ts, and the probe of the harness. With this change it starts 10: 4 builds, 4 executables, and the same two. The log of build 135961 shows that a first start of a new executable waits about 3.3 s on the darwin x64 agent (273 executables: 145.2 s, 13.2 s of CPU for 40 builds).

Runs. bun bd test test/aot/cli/entry-points-that-are-not-text.test.ts on a debug ASAN build of main, Linux x64: 46.9 s before, 35.2 s after (medians of 5 rounds in turns). Release build on 2 CPUs: 2.76 s before, 1.45 s after (medians of 11 rounds). No macOS machine was available, so the darwin x64 lane is not measured before and after. In CI the file took 45 s there on main, beside three other files that build executables. With this change it took 5.3 s and 5.4 s there (one run for each commit), first and alone in its shard. The numbers are not like for like.

How the assertions were checked. Eight changes to the new file, one at a time, each fail the tests that they must. For example, table.unknown written with 3001 bytes fails only the first test, and a console.error in the shared program fails its three tests. The Notes of the description hold the list and the tables.

@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 23fc93b and 4dfcf21.

Files selected for processing: 1
  • test/aot/cli/entry-points-that-are-not-text.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

Walkthrough

The AOT entry-point tests share fixtures and cached executable runs. They compare embedded-file names, sizes, and SHA-256 hashes across AOT and non-AOT builds. The cases cover unknown extensions, symlinks, TypeScript content, and a non-UTF-8 comment.

Changes

Entry-point file embedding

Layer / File(s) Summary
Shared fixtures and executable harness
test/aot/cli/entry-points-that-are-not-text.test.ts
The tests use shared fixtures and cache builds by mode and entry points. The app helper checks successful execution and expected output.
Entry-point embedding assertions
test/aot/cli/entry-points-that-are-not-text.test.ts
Tests compare embedded-file names, sizes, and hashes across build modes. The TypeScript case checks entry-point execution, embedding, and require resolution.

Possibly related PRs

  • oven-sh/bun#47296: Changes AOT root selection; this PR tests embedded names and require paths with multiple entry points.
  • oven-sh/bun#47535: Shares AOT builds and executable launches for related test cases.
  • oven-sh/bun#47463: Shares AOT builds and temporary-directory setup in another AOT test suite.

Suggested reviewers: jarred-sumner

Merge Risk: Minimal · up to 4dfcf

The tests reduce duplicate builds while retaining the checked embedding behavior. No merge-blocking issue was established; merge after normal checks pass.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title names the main change: the test file now shares one program between three tests and compares embedded files by SHA-256. The changeset matches this. The title is a single line and specific. I…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description follows the repository template. It has a 'What does this PR do?' section that explains the problem, the fix, the background, and the downsides. It has a 'How did you verify your code …
Full details: Title check

Explanation

The title names the main change: the test file now shares one program between three tests and compares embedded files by SHA-256. The changeset matches this. The title is a single line and specific. It is a bit long at 128 characters, but it is accurate and not vague.

Full details: Description check

Explanation

The description follows the repository template. It has a 'What does this PR do?' section that explains the problem, the fix, the background, and the downsides. It has a 'How did you verify your code works?' section that lists local timings, CI timings, assertion-failure checks, and other test runs. The main text is long, and most of the detail sits in a collapsed details block. The description is substantive and specific, so the check passes. The only gap is that macOS timing was not measured locally. The author states this openly and cites CI logs instead.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

…e-not-text

The program took the last part of a path for each embedded name too, so
a name with a folder in front of it passed. It now removes only the hash
from a name, and takes the last part only of the path that require
answers.

The names, stderr, the exit code and the signal of the shared executable
are compared in one object, so a failure shows what was printed.

The folder is made in beforeAll. A run with -t that selects no test of
the file runs no afterAll, and it left the folder behind.
@robobun

robobun commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

The four review notes, after 4dfcf21:

  • A folder in front of an embedded name (line 23). Handled. The program removes only the hash from file.name, as the test on main does. It takes the last part of a path only from what require answers. A change to the program that prints store/blob. now fails the three tests of the shared program.
  • What the shared executable printed (line 70). Handled. One object holds the names that were printed, stderr, the exit code and the signal.
  • The folder under -t (line 51). Handled. The folder is made in beforeAll. A run that selects no test of the file makes nothing. My first check of this case looked in /tmp, and the tests of this folder use /tmp/aot.
  • withoutDebugInformation() (line 58). Not taken. It does nothing in CI, because the runner already hands this file a Bun without debug information on Linux. bun bd test of the file takes 38.4 s without it and 37.5 s with it (medians of 4 rounds in turns). The thread has the other numbers.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

LGTM — 4dfcf21 addresses the three threads from the earlier review (fixtures now created in beforeAll, embedded names printed without a directory strip, and app() asserts one combined {names, stderr, exitCode, signal} object), and the remaining inline comment is a nit. For weighing it: ../harness/shared-builds is imported by one file under test/aot/trimmed-is-whole, while every file in test/aot/cli uses ../harness/commands (slot/spawned/built) as this one does, and sharedBuilds takes a single entry and makes its folder at module load, so reusing it here would reopen the lazy-folder point.

Extended reasoning...

The PR rewrites one test file, test/aot/cli/entry-points-that-are-not-text.test.ts, so that three tests share two memoised bun build --compile executables (with and without --aot) instead of six, and compares embedded further entry points by name, size and SHA-256 plus stderr and signal; no production code, no security-sensitive surface, and no CODEOWNERS path is touched. I traced the embedded program's output against the expected ordered list on both the Windows and non-Windows branches, the hash-strip regex and the require-answer basename against /$bunfs/root/tool-<hash>.unknown, the slot usage (no nested slots, so no deadlock between the four concurrent spawns), and the beforeAll/afterAll pairing with Promise.allSettled so a failed build does not skip disposal. The second commit resolved all three prior inline threads from the code itself, the hunt ran dry, and the single surviving finding is a harness-reuse nit that the directory's own conventions argue against, which decided approve.

@robobun

robobun commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

About the note on sharedBuilds (line 68): not taken in this PR. The helper takes one entry point, texts as files, and makes its folder when it is called. This test needs the entry points in order, files of bytes, and a folder that is made in beforeAll. The helper would have to change, and 23 files import its module. The review thread has the details.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant