Skip to content

perf(runtime): call the guest execute helpers directly instead of compiling snippets - #418

Merged
sd2k merged 1 commit into
mainfrom
perf/guest-direct-execute
Sep 11, 2026
Merged

sd2k merged 1 commit into
mainfrom
perf/guest-direct-execute

Conversation

@sd2k

@sd2k sd2k commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

The guest's execute path built a _eryx_exec("""<escaped user code>""", False) string and ran it through PyRun_SimpleString, then compiled three more fixed snippets (_eryx_output, _eryx_errors = _eryx_get_output(), _eryx_capture_result(), _eryx_discard_result()) and re-resolved __main__ four times to read the results back. So CPython tokenised an escaped copy of the user's source inside the wrapper and compiled it again inside _eryx_exec, on every request.

This resolves __main__.__dict__ and the four _eryx_* helper function objects once, at the end of initialize_python() (owned references in a OnceLock; they live in linear memory and survive the pre-init snapshot like everything else), and execute_python now:

  • calls _eryx_exec(code, trace_enabled) with PyObject_Call, passing the source as a str object — no wrapper string, no escaping, no second parse;
  • reads (stdout, stderr) from the tuple _eryx_get_output() returns;
  • calls _eryx_capture_result() / _eryx_discard_result() directly and reads _eryx_result / _eryx_result_error with two dict lookups.

Error semantics are unchanged: on an exception PyErr_Print() runs before the streams are restored, so the traceback lands in the captured stderr exactly as PyRun_SimpleString produced it (the contract from #239). The pending/async path is untouched. If the helpers are missing from __main__ for any reason the old PyRun_SimpleString path is used, so a broken infrastructure snippet degrades rather than fails.

Stacked on #416 (perf/guest-callback-setup-cache) because that is where the session path stopped being dominated by callback re-setup; the python.rs changes are disjoint from it. prebuilt/liberyx_runtime.so.zst is rebuilt from this branch, so the usual conflict-on-merge applies to that file.

Benchmarks

Criterion (cargo bench --package eryx --features embedded --bench execution), stock wasmtime 48.0.1, Ryzen 9 7950X, this branch vs its base #416:

Benchmark #416 this PR Δ
session_execution/pass 187.3 µs 85.9 µs −54%
stateless_execution/pass (3 callbacks) 4.90 ms 4.79 ms −2% (dominated by per-instance callback setup)
session_creation/new 449 µs 441 µs — (not touched)

Spike harness (profile_stateless, 2000 executions, median of 3; measured on top of the wasmtime pagemap/funcref fixes from the cold-start spike so instantiation cost does not hide the guest cost):

Path before after Δ
warm instance, 2 ms gap (request-like), trace collection on 282 µs 145 µs −48%
warm instance, trace collection off 196 µs 82 µs −58%
cold instance 497 µs 400 µs −19%
cold render workload (json + string.Template), trace off 903 µs 815 µs −10%

In the warm-path profile the parser/compiler symbols (_PyPegen_*, _PyCfg_*, _PyAssemble_*, tok_get_*) drop from 6% to 1.8%; what remains is compile() of the user's own source.

Testing

  • cargo nextest run --workspace --features embedded --cargo-profile release: 629 passed, 0 failed.
  • cargo clippy --workspace --all-targets --all-features -- -D warnings: clean.

Not included

  • Skipping the list-callbacks host call on the execute path (still made every execute; cheap next to the above).

🤖 Generated with Claude Code

@sd2k
sd2k force-pushed the perf/guest-direct-execute branch from 7895c8e to 3b4313c Compare September 10, 2026 22:47
Base automatically changed from perf/guest-callback-setup-cache to main September 11, 2026 08:28
… snippets

execute() built a `_eryx_exec("""<escaped code>""", False)` string and ran it
through PyRun_SimpleString, then compiled three more fixed snippets and
re-resolved __main__ four times to read the results back, so CPython parsed
an escaped copy of the user's source in the wrapper and compiled it again
inside _eryx_exec on every request.

Resolve __main__.__dict__ and the _eryx_exec / _eryx_get_output /
_eryx_capture_result / _eryx_discard_result function objects once at the
end of initialize_python() (owned references that survive the pre-init
snapshot) and call them with PyObject_Call, passing the source as a str.
PyErr_Print() still runs before the streams are restored, so a traceback
lands in the captured stderr exactly as before. Falls back to the
PyRun_SimpleString path if the helpers are missing.

session_execution/pass: 187 µs -> 86 µs on stock wasmtime 48.0.1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sd2k
sd2k force-pushed the perf/guest-direct-execute branch from 3b4313c to 616d7c1 Compare September 11, 2026 08:48
@sd2k

sd2k commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main (dropping the now-merged #410/#416 commits) and rebuilt prebuilt/liberyx_{runtime,bindings}.so.zst against the wasm-tools 0.259 toolchain from #414, so the committed guest artifact matches what the current tree builds. Verified the rebuilt .so actually carries the new code (the fallback path means a stale artifact would pass tests silently) and that the fast path is live: session_execution/pass 48.7 µs with ERYX_PROFILE_TRACE=0. 629 tests, clippy -D warnings and fmt all clean locally.

@github-actions

Copy link
Copy Markdown
Contributor

@sd2k
sd2k merged commit 39fd350 into main Sep 11, 2026
31 checks passed
@sd2k
sd2k deleted the perf/guest-direct-execute branch September 11, 2026 09:08
This was referenced Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant