Refactor

[khanhnd61-vr]

Refactor report - layer/module/model split

• Generated: 2026-08-14
• Branch: refactor, commits 87a7e8f..04df961; llama.cpp pinned at b10331
• Hardware: 1x RTX 3090 (24 GB), Intel i7-14700F (20C/28T)
• Success rate: libero_object, 10 tasks x 10 episodes = 100 episodes per model,
  via eval/run_libero.sh -n 10
• Latency: tests/predict_check with fixed images / language / state / noise,
  VLA_BENCH_ITERS=20, minimum of 20 timed predict() calls after 3 warmups

The refactor's contract was that no model's output moves by a bit. That was
enforced continuously rather than checked at the end: eval/refactor_verify.sh
runs every arch against its real checkpoint at two precisions and diffs the
action chunk against the pre-refactor bytes, and it was re-run after each of the
eleven commits. Every one passed.

Results

Model	SR (100 eps)	Latency, default	Latency, fastest	Peak VRAM
bitvla	99/100	50.3 ms	48.0 ms	1,464 MiB
gr00t_n1_5	99/100	67.9 ms	67.9 ms	5,064 MiB
gr00t_n1_7	98/100	55.4 ms	55.4 ms	6,468 MiB
openvla_oft	97/100	219.5 ms	219.5 ms	14,886 MiB
pi05	96/100	112.3 ms	112.3 ms	6,126 MiB
vla_adapter	96/100	69.7 ms	69.7 ms	3,098 MiB
evo1	91/100	157.5 ms	114.1 ms	1,794 MiB
smolvla	90/100	73.6 ms	50.5 ms	1,578 MiB
gr00t_n1_6	84/100	55.5 ms	55.5 ms	6,114 MiB
pi0	81/100	105.5 ms	94.1 ms	5,666 MiB
vla_jepa	not evaluated	44.0 ms	44.0 ms	-

No episode was skipped mid-step on any model (0/100 across the board).

vla_jepa has no success rate because the LIBERO client cannot drive it.

The SR column is measured at each model's shipping defaults, not at the
flags in the "fastest" column.

How to run each model at its fastest

Every performance switch is now a vla-server flag. The per-arch environment
variables that used to carry them (VLA_GR00T_BF16_WEIGHTS, VLA_PI0_FA,
VLA_SMOLVLA_FA, VLA_MM_PREC, ...) are retired, and setting one is a hard
load failure with a message naming its replacement - a silently-ignored switch
would let a benchmark measure a configuration nobody asked for.

Model	Flags for the fastest configuration	Same output as default?
smolvla	--flash-attn --mm-prec default	no - both change numerics
pi0	--act-dtype bf16 --flash-attn	bf16 yes, FA no
evo1	--act-dtype bf16 --flash-attn	bf16 yes, FA no
gr00t_n1_5	(none) + VLA_GR00T_EMBODIMENT=new_embodiment	-
gr00t_n1_6	(none) + VLA_GR00T_EMBODIMENT=libero_panda	-
gr00t_n1_7	(none)	-
pi05	(none)	-
bitvla	--weight-dtype bf16	no - 2.3 ms for a numerics change
vla_adapter	(none)	-
openvla_oft	(none)	-
vla_jepa	(none)	-

./build/vla-server --bind tcp://*:5555 \
    --act-dtype bf16 --flash-attn \
    /path/to/pi0-libero-finetuned-v044.gguf

Seven of eleven models need no flags at all: their fastest configuration is now
the default. The full option list is in vla-server --help; the same keys are
accepted in the "runtime" object of the config JSON passed to --config.

VLA_GR00T_EMBODIMENT selects normalization statistics, not speed - the wrong
value produces wrong actions rather than slower ones. It is also available as
--embodiment.

Refactoring the source tree

src/models/ held eleven self-contained files that each re-derived the same
pieces. The three levels are now:

Level	Path	Contents
layer	src/layers/	attention, linear, norms, rope, FFN, host-side embedding tables
module	src/modules/	SigLIP / Qwen3-VL / dual towers, Qwen3 + Gemma stacks, DiT head, action expert, prompt assembly
model	src/models/	per-arch config, composition, predict

Deduplicated: build_dit_block (4 copies → 1), SigLipLayerW (5 → EncBlockW),
Qwen3LayerW (4 → Qwen3LM), ViTLayerW/MergerW (2 → Qwen3VLTower), the
DINOv2+SigLIP declaration (2 → DualTower), and all eleven copies of the
mk/mk_mm/mk_f32 + ok &= a&&b&&c + upload-loop pattern (→ WeightLoader).

src/models/ went from ~9,600 to ~8,400 lines while gaining ~1,150 lines of
shared src/layers/ + src/modules/.

Reproducing

# bit-exactness and latency, all archs, both precisions
BENCH=20 eval/refactor_verify.sh outputs/refactor/check

# success rate, 100 episodes per model
SKIP_BUILD=1 eval/run_libero.sh -i /mnt/data/hf_data/vrfai \
    -o outputs/libero_sr_100 -n 10 -m all

Two traps worth knowing. Rebuilding while a sweep runs relinks
libvla_core.so underneath it and every remaining arch fails to load with
"file too short" - which reads like a regression and is not one. And a failed
build leaves the previous binary in place, so a verify run can pass against
stale code; check the compiler's error count before trusting a result.