Skip to content

perf(preinit): bake callback declarations into the snapshot - #423

Merged
sd2k merged 1 commit into
mainfrom
perf/preinit-baked-callbacks
Sep 11, 2026
Merged

sd2k merged 1 commit into
mainfrom
perf/preinit-baked-callbacks

Conversation

@sd2k

@sd2k sd2k commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

#416 made the guest reinstall callback wrappers only when the host's declarations differ from the installed set, which fixed sessions but not fresh sandboxes: every fresh instance starts from the pre-init snapshot, and the snapshot was taken with the empty callback set (the preinit list-callbacks stub answered []), so the first execute() on each instance still ran the full setup script — ~3.4 ms of Python for three callbacks, more than the instantiation itself.

This lets the callback declarations be baked into the snapshot, so a fresh instance whose host registers the same set (by name, description and parameter schema, in any order) skips the setup entirely. It is the "bake the declarations into the snapshot" follow-up named in #416 and the preferred alternative to keying the warm pool by callback set (#411), which moves CPU around rather than removing it.

No guest change: #416's INSTALLED_CALLBACKS fingerprint and the generated wrappers already live in linear memory and survive the snapshot; only the host side needed to supply the declarations at pre-init time and present them in a stable order.

Changes

  • eryx-runtime: PreInitOptions (builder) and pre_initialize_with_options(); pre_initialize() is kept as a thin wrapper. PreInitOptions::callbacks(Vec<CallbackDeclaration>) makes the preinit list-callbacks stub return the declarations, and a no-op execute("pass") runs when nothing else would have, so the installation lands in the snapshot. Declarations are sorted by name before use.
  • eryx: re-exports plus eryx::preinit::callback_declaration(&dyn Callback). ExecutorState::list_callbacks now returns the host's callbacks sorted by name, so registration order can never cause a mismatch with the baked set (the guest compares a serialized list). The list_callbacks() seen from Python is therefore name-sorted; existing tests already sorted() it.
  • eryx-precompile: --callbacks <file.json> ([{"name", "description", "parameters"}], parameters defaults to {}), requires --preinit.
  • pyeryx: SandboxFactory(callbacks=...) bakes the declarations and keeps the callbacks; create_sandbox() / create_session() register them unless given their own (an explicit set still overrides, and installs itself as before). SandboxFactory.load(path, callbacks=...) takes the same callbacks back, since the file holds the declarations but not the Python callables.
  • Docs: guide/precompile.md (Callbacks section) and guide/packages.md (Baking Callbacks into the Factory); _eryx.pyi updated.

Stacked on #416 (perf/guest-callback-setup-cache); independent of #418.

Benchmarks

Criterion (cargo bench --package eryx --features embedded,preinit --bench execution -- stateless_execution), ERYX_WARM_INSTANCES=0, stock wasmtime 48.0.1, Ryzen 9 7950X. Both groups register the same three callbacks (noop, echo, work); the baked group runs from a snapshot built with their declarations (PreInitOptions::callbacks), the other from the embedded runtime, which installs them on every fresh instance.

per-instance callback setup (today) baked declarations (this PR) Δ
stateless_execution/pass 4.703 ms 1.302 ms (stateless_execution_baked/pass) −72%

For reference, a cold pass with no callbacks on this build is ~1.28 ms, so the remaining gap to "callbacks are free" is within noise. The new stateless_execution_baked group needs the preinit feature and is skipped without it.

Testing

  • cargo nextest run --workspace --features embedded,preinit --cargo-profile release: 631 passed, 0 failed. New: preinit_baked_callbacks (same set works; a different set is installed instead; no registration → empty introspection) and preinit_baked_callbacks_ignore_registration_order.
  • pyeryx: tests/test_factory_callbacks.py (7 tests: default registration, introspection, repeated sandboxes, explicit override, sessions, save/load with and without callbacks, with setup code) plus existing factory/callback tests — 74 passed.
  • cargo clippy --workspace --all-targets --all-features -- -D warnings and cargo fmt --all --check: clean.

Notes

  • Setup code cannot invoke the baked callbacks (the preinit invoke stub still errors); documented.
  • eryx-server only benefits when the callback set is fixed per deployment; per-request callback sets still pay the setup once per fresh instance.
  • A loaded factory cannot verify that the callbacks it is handed match the baked declarations (the artifact is a plain cwasm); a mismatch just falls back to per-sandbox installation.

🤖 Generated with Claude Code

Base automatically changed from perf/guest-callback-setup-cache to main September 11, 2026 08:28
A fresh sandbox starts from the pre-init snapshot, which was taken with
the empty callback set, so its first execute() always reinstalled the
callback wrappers (~3.4 ms of Python for three callbacks) even after #416
taught the guest to skip unchanged sets.

Let the declarations be supplied at pre-init time: PreInitOptions /
pre_initialize_with_options() in eryx-runtime (the list-callbacks stub
answers with them and a no-op execute installs them into the snapshot),
eryx::preinit::callback_declaration(), `eryx-precompile --callbacks
<json>`, and SandboxFactory(callbacks=...) in pyeryx, whose sandboxes and
sessions register those callbacks by default. The host now presents its
callbacks to the guest sorted by name so registration order cannot cause
a mismatch with the baked set.

stateless_execution/pass with three callbacks: 4.70 ms -> 1.30 ms.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sd2k
sd2k force-pushed the perf/preinit-baked-callbacks branch from 1247ecb to fbd2dc3 Compare September 11, 2026 09:11
@sd2k

sd2k commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main (dropping the now-merged #410/#416 commits, then onto #418). Conflict resolutions, both from overlapping work on main:

Re-measured on top of #418 (ERYX_PROFILE_TRACE=0, three callbacks, Ryzen 9 7950X):

Benchmark time vs. pre-#418 run
stateless_execution/pass 4.48 ms −5%
stateless_execution_baked/pass 966 µs −26%

So the two compose: baked declarations are still the big win (4.6x on this path), and #418's direct execute path takes another quarter off it.

Verified locally: 632 tests (--workspace --all-features), clippy -D warnings, fmt, and the pyeryx suite (340 tests, including the 7 new test_factory_callbacks.py cases).

@github-actions

Copy link
Copy Markdown
Contributor

@sd2k
sd2k merged commit 7929442 into main Sep 11, 2026
31 checks passed
@sd2k
sd2k deleted the perf/preinit-baked-callbacks branch September 11, 2026 09:29
This was referenced Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant