perf(preinit): bake callback declarations into the snapshot - #423
Merged
Merged
Conversation
A fresh sandbox starts from the pre-init snapshot, which was taken with the empty callback set, so its first execute() always reinstalled the callback wrappers (~3.4 ms of Python for three callbacks) even after #416 taught the guest to skip unchanged sets. Let the declarations be supplied at pre-init time: PreInitOptions / pre_initialize_with_options() in eryx-runtime (the list-callbacks stub answers with them and a no-op execute installs them into the snapshot), eryx::preinit::callback_declaration(), `eryx-precompile --callbacks <json>`, and SandboxFactory(callbacks=...) in pyeryx, whose sandboxes and sessions register those callbacks by default. The host now presents its callbacks to the guest sorted by name so registration order cannot cause a mismatch with the baked set. stateless_execution/pass with three callbacks: 4.70 ms -> 1.30 ms. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
sd2k
force-pushed
the
perf/preinit-baked-callbacks
branch
from
September 11, 2026 09:11
1247ecb to
fbd2dc3
Compare
Collaborator
Author
|
Rebased onto
Re-measured on top of #418 (
So the two compose: baked declarations are still the big win (4.6x on this path), and #418's direct execute path takes another quarter off it. Verified locally: 632 tests ( |
Contributor
|
🌐 Demo preview: https://perf-preinit-baked-callbacks.eryx-bvy.pages.dev |
This was referenced Sep 11, 2026
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#416 made the guest reinstall callback wrappers only when the host's declarations differ from the installed set, which fixed sessions but not fresh sandboxes: every fresh instance starts from the pre-init snapshot, and the snapshot was taken with the empty callback set (the preinit
list-callbacksstub answered[]), so the firstexecute()on each instance still ran the full setup script — ~3.4 ms of Python for three callbacks, more than the instantiation itself.This lets the callback declarations be baked into the snapshot, so a fresh instance whose host registers the same set (by name, description and parameter schema, in any order) skips the setup entirely. It is the "bake the declarations into the snapshot" follow-up named in #416 and the preferred alternative to keying the warm pool by callback set (#411), which moves CPU around rather than removing it.
No guest change: #416's
INSTALLED_CALLBACKSfingerprint and the generated wrappers already live in linear memory and survive the snapshot; only the host side needed to supply the declarations at pre-init time and present them in a stable order.Changes
eryx-runtime:PreInitOptions(builder) andpre_initialize_with_options();pre_initialize()is kept as a thin wrapper.PreInitOptions::callbacks(Vec<CallbackDeclaration>)makes the preinitlist-callbacksstub return the declarations, and a no-opexecute("pass")runs when nothing else would have, so the installation lands in the snapshot. Declarations are sorted by name before use.eryx: re-exports pluseryx::preinit::callback_declaration(&dyn Callback).ExecutorState::list_callbacksnow returns the host's callbacks sorted by name, so registration order can never cause a mismatch with the baked set (the guest compares a serialized list). Thelist_callbacks()seen from Python is therefore name-sorted; existing tests alreadysorted()it.eryx-precompile:--callbacks <file.json>([{"name", "description", "parameters"}],parametersdefaults to{}), requires--preinit.SandboxFactory(callbacks=...)bakes the declarations and keeps the callbacks;create_sandbox()/create_session()register them unless given their own (an explicit set still overrides, and installs itself as before).SandboxFactory.load(path, callbacks=...)takes the same callbacks back, since the file holds the declarations but not the Python callables.guide/precompile.md(Callbacks section) andguide/packages.md(Baking Callbacks into the Factory);_eryx.pyiupdated.Stacked on #416 (
perf/guest-callback-setup-cache); independent of #418.Benchmarks
Criterion (
cargo bench --package eryx --features embedded,preinit --bench execution -- stateless_execution),ERYX_WARM_INSTANCES=0, stock wasmtime 48.0.1, Ryzen 9 7950X. Both groups register the same three callbacks (noop, echo, work); the baked group runs from a snapshot built with their declarations (PreInitOptions::callbacks), the other from the embedded runtime, which installs them on every fresh instance.stateless_execution/passstateless_execution_baked/pass)For reference, a cold
passwith no callbacks on this build is ~1.28 ms, so the remaining gap to "callbacks are free" is within noise. The newstateless_execution_bakedgroup needs thepreinitfeature and is skipped without it.Testing
cargo nextest run --workspace --features embedded,preinit --cargo-profile release: 631 passed, 0 failed. New:preinit_baked_callbacks(same set works; a different set is installed instead; no registration → empty introspection) andpreinit_baked_callbacks_ignore_registration_order.tests/test_factory_callbacks.py(7 tests: default registration, introspection, repeated sandboxes, explicit override, sessions, save/load with and without callbacks, with setup code) plus existing factory/callback tests — 74 passed.cargo clippy --workspace --all-targets --all-features -- -D warningsandcargo fmt --all --check: clean.Notes
invokestub still errors); documented.eryx-serveronly benefits when the callback set is fixed per deployment; per-request callback sets still pay the setup once per fresh instance.🤖 Generated with Claude Code