Conversation
Prior to this commit, mise.toml asked for rust "1.98", which rustup had installed as 1.98.0, while rust-toolchain.toml and the workspace rust-version both require 1.98.1. Every cargo invocation through the mise shim failed with "rustc 1.98.0 is not supported" until the toolchain was overridden by hand. This commit pins mise to the same 1.98.1 so all three agree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…xecution Prior to this commit, every stateless `Sandbox::execute()` paid for creating and destroying its WebAssembly instance on the request path. Profiling `pass` on a fresh instance (~1.3 ms) showed only ~15% running Python: ~30% was wasmtime eagerly building a VMFuncRef for each of the runtime's ~6k table entries (the dynamically linked modules' element segments are offset by an imported `__table_base` into an imported table, so lazy table init does not apply), and ~45% was mapping and then tearing down the 16 MB pre-initialized heap image, taking ~430 page faults per execution. This commit moves that work off the request path in two layers: - The wasmtime engine now uses the pooling allocator. A released memory slot is reused by the next instance with its pages still mapped, and on Linux only the dirtied pages are reset. `ERYX_ALLOCATOR`, `ERYX_POOL_INSTANCES`, `ERYX_POOL_KEEP_RESIDENT_MB` and `ERYX_POOL_PAGEMAP_SCAN` tune it; if the pool's address-space reservation is refused, engine creation falls back to on-demand. - A process-wide warm-instance pool (`crates/eryx/src/warm.rs`) keeps an instantiated store ready per runtime component. `execute()` takes it, swaps in the per-execution state, and hands the used store to a background task that drops it and instantiates the replacement. The pool is keyed by component so short-lived sandboxes (the `SandboxFactory` pattern) share it, only runs on a multi-threaded Tokio runtime, and leaves executions whose memory limit is below the snapshot baseline to the cold path so they fail as before. `ERYX_WARM_INSTANCES` sizes it. With a 2 ms gap between executions (a request-like pattern), `pass` on a fresh sandbox drops from 1.44 ms to 0.81 ms on a 7950X; in a tight loop from 1.28 ms to 0.94 ms. Isolation is unchanged: every execution still runs on an instance that has never run user code. Also adds `examples/profile_stateless.rs` for profiling this path and a Performance Tuning page to the book. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
|
🌐 Demo preview: https://perf-instantiation-overhead.eryx-bvy.pages.dev |
Prior to this commit, a fresh instantiation set the execution's fuel before instantiating, so `fuel_consumed` included the instructions of the modules' table-initialising start functions, while an execution on a pre-instantiated store did not. The same code therefore reported different fuel depending on whether it happened to get a warm instance, which the book's determinism doctest caught. This commit instantiates on an unlimited tank and sets the requested fuel afterwards, so fuel measures only the user's code on both paths. Fuel limits now bound user code rather than user code plus instantiation, which is what the limit was meant to express. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 10, 2026
This was referenced Sep 18, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Takes WebAssembly instance creation and teardown off the request path of stateless
Sandbox::execute(), the path everySandboxFactory.create_sandbox().execute()render takes.Profiling
passon a fresh instance (~1.3 ms locally) showed only ~15% of the time running Python:VMFuncReffor each of the runtime's ~6k function-table entries at instantiation. The dynamically linked modules'elemsegments are offset by an imported__table_baseinto an imported table, and wasmtime's lazy table init (try_func_table_init) bails on both conditions, so it synthesizes a module-start function that callsref.func~6k times.Changes
Pooling allocator (
AllocatorSettingsinwasm.rs). The shared engine now uses wasmtime's pooling allocator, so a released memory slot is reused by the next instance with its pages still mapped, and on Linux only dirtied pages are reset. Configured byERYX_ALLOCATOR,ERYX_POOL_INSTANCES(default 1000),ERYX_POOL_KEEP_RESIDENT_MB(64),ERYX_POOL_PAGEMAP_SCAN(1). If the pool's address-space reservation is refused, engine creation logs a warning and falls back to on-demand. Both engine constructors now share onebase_engine_config(). This does not affect.cwasmcompatibility.Warm-instance pool (
crates/eryx/src/warm.rs). After each stateless execution, a background task drops the used store and instantiates a replacement; the nextexecute()takes it and swaps in the per-executionExecutorState(WASI ctx, VFS, channels, memory tracker) before any export is called. Keyed by component so short-lived sandboxes share it; only active on a multi-threaded Tokio runtime (on current-thread it would just add to the next request's latency); executions whose memory limit is below the snapshot baseline take the cold path so they fail exactly as before.ERYX_WARM_INSTANCES(default 1,0disables) sizes it;PythonExecutor::warm_instances_ready()reports it. Per-execution channel senders are detached synchronously before the hand-off, otherwiserun_inner's handler-task joins would wait on the background drop.Also:
examples/profile_stateless.rs(the harness used for these measurements — supports a gap between executions so replenishment gets the same chance it has between real requests), a Performance Tuning book page, README/guide cross-links, and a stale bench comment fixed.Measurements (7950X, release,
pass, no callbacks)ERYX_WARM_INSTANCES=1)ERYX_WARM_INSTANCES=2Inside
execute_internal, cold is acquire 0.62 ms + run 0.66 ms + drop 0.16 ms; warm is acquire 0.001 ms + run 0.76 ms. Page faults per execution: ~430 (on-demand) → ~100 (pooling; the remainder is glibc growing/trimming the brk heap for wasmtime's ~450 KB per-instanceVMContext, which wasmtime 48 heap-allocates).Page faults in cloud VMs cost several times what they do here, so the production win should be larger than the local one.
Not in this PR (follow-ups)
setup_callbacksruns on every execute when callbacks are registered (and always for sessions): it recompiles ~200 lines of Python each time. That is ~3.2 ms of the criterion bench's 4.7 msstateless_execution/pass. Guest-side change (python.rs), needs a runtime rebuild — separate PR._eryx_exec("""…""")wrapper +compile()) plus two morePyRun_SimpleStringper execute; also guest-side.PAGEMAP_SCANreset only handles 32 dirty regions (MAX_REGIONSinsys/unix/pagemap.rs), which is whylinear_memory_keep_residentbarely helps CPython's scattered refcount writes here — worth an upstream issue.Testing
cargo nextest run --workspace --features embedded: 637 passed (new:tests/warm_instances.rs— pool fills after an execution, is shared across sandboxes, isolation between executions, callbacks/result variable/VFS/fuel limit on a warm instance, below-baseline memory limit still rejected without consuming a warm instance, inactive on a current-thread runtime; unit tests forAllocatorSettingsparsing and that the pooling config is accepted by wasmtime).cargo clippy --workspace --all-targets --all-features -- -D warningsclean.Checklist
🤖 Generated with Claude Code