Skip to content

feat: run benches in a libkrun microVM for predictable numbers - #15

Merged
enekos merged 1 commit into
masterfrom
feat/microvm-bench-isolation
Jul 8, 2026
Merged

enekos merged 1 commit into
masterfrom
feat/microvm-bench-isolation

Conversation

@enekos

@enekos enekos commented Jul 8, 2026

Copy link
Copy Markdown
Owner

What

Opt-in --isolation microvm runs a bench inside an ephemeral libkrun microVM (via the krunvm CLI) instead of on the host — pinned vCPUs, fixed RAM, clean rootfs — so numbers don't drift with host load. One code path on macOS (Hypervisor.framework) and Linux (KVM). No libkrun FFI, no new build-time dependency; we just shell out to krunvm.

Default stays local — nothing changes unless you ask for the VM.

Design

  • New sandbox module — Isolation::{Local, Microvm}, resolves VmOpts, boots an ephemeral guest, mounts the workdir 1:1 + a persistent build cache, execs through /bin/sh with a normalized PATH, and deletes the VM on drop (no leaks on error/panic).

  • Wired into both bench entry points through their single choke points:

    • aatxe perf-vs — both A/B sides run in matching guests, so the comparison stays fair.
    • aatxe run — the shared run_runner all three language adapters (TS/Go/Rust) funnel through, so they all get it for free.
  • Only the knobs that move numbers are surfaced (everything else = libkrun defaults):

    flag (env) default
    --isolation local
    --vm-cpus (AATXE_VM_CPUS) 2
    --vm-mem (AATXE_VM_MEM) 2048
    --vm-image (AATXE_VM_IMAGE) rust:1 / golang:1 / node:22

Plug-and-play

make microvm-setup     # one-time: install krunvm (Homebrew on macOS)
make microvm-doctor    # readiness check
make perf-vs-vm        # = make perf-vs ISOLATION=microvm

Missing krunvm fails with an install hint, not a cryptic spawn error. Build caches (cargo/go/npm) persist under ~/.cache/aatxe/vm, isolated from the host's own target/ (different arch), so only the first run pays the image-pull + cold-build cost.

Testing

  • cargo fmt --check, clippy -D warnings, full suite green (112 tests, incl. 9 new sandbox unit tests on argv construction / shell-quoting / config resolution).
  • Verified end-to-end on macOS that the flag is exposed on both commands and the microVM path surfaces the actionable error through the full stack; the refactored local run path still captures/parses/writes and propagates env+cwd.

Caveat

The live VM boot path (krunvm create/start/delete) is unit-tested at the argv level but not exercised live — krunvm isn't installed on my box. All krunvm invocation is centralized in sandbox.rs (create_argv / guest_argv), so if the installed version names a flag differently it's a one-line fix. Validate with make microvm-setup → make perf-vs-vm.

Host noise (background load, thermal throttling, neighbouring processes)
makes bench numbers drift run-to-run. Add an opt-in `--isolation microvm`
that runs the bench inside an ephemeral libkrun microVM — pinned vCPUs,
fixed RAM, clean rootfs — driven by the `krunvm` CLI. One code path on
macOS (Hypervisor.framework) and Linux (KVM); no libkrun FFI, no new
build-time dependency.

- New `sandbox` module: Isolation::{Local,Microvm}, resolves VmOpts, boots
  an ephemeral guest, mounts the workdir 1:1 + a persistent build cache,
  execs through /bin/sh with a normalized PATH, and deletes the VM on drop.
- Wired into both bench entry points via their single choke points:
  `perf-vs` (both A/B sides run in matching guests) and `aatxe run` (the
  shared run_runner all three language adapters funnel through).
- Surfaces only the knobs that move numbers: --isolation / --vm-cpus /
  --vm-mem / --vm-image (env-backed, sane per-language image defaults).
- Plug-and-play: `make microvm-setup` installs krunvm, `make microvm-doctor`
  checks readiness, `make perf-vs-vm` runs the A/B loop in a guest. Missing
  krunvm fails with an install hint, not a cryptic spawn error.

Build caches (cargo/go/npm) persist under ~/.cache/aatxe/vm, isolated from
the host's own target/ (different arch). Unit-tested at the argv/quoting
level; the live boot path needs krunvm installed.
@enekos
enekos merged commit 35a7dd9 into master Jul 8, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant