feat: add compiled VM renderer

[Mido-sys]

This PR adds an opt-in compiled VM renderer for Plush while keeping the existing interpreter path as the default. The new renderer lives under VM/ and supports the same user-facing Plush behavior as the interpreter: expressions, variables, functions, closures, helpers, struct and map access, nested chains, loops, conditionals, partials, punch-hole rendering, render budgets, escaping, trim tags, optional if parentheses, and mixed Go numeric comparisons.

The root package can now switch render engines with plush.SetRenderMode(plush.RenderModeVM) after importing github.com/gobuffalo/plush/v5/VM/plush. Applications that want direct compiled reuse can call VM/plush.Compile and render the returned template repeatedly.

Major Changes

• Added VM packages for bytecode, objects, compiler, runtime execution, and VM-backed Plush rendering.
• Added compiler support for Plush template nodes, script expressions, closures, recursion, globals, locals, free variables, arrays, hashes, indexes, infix expressions, helper calls, method calls, block helpers, loops, partials, holes, budgets, line metadata, and render output opcodes.
• Added root render-mode support:
  • RenderModeInterpreter remains the default.
  • RenderModeVM routes root plush.Render through the compiled renderer when the VM package is imported.
• Added compiled-template reuse through VM/plush.Compile.
• Added shared template-cache support for VM bytecode.
• Added parity coverage across root Plush behavior and VM rendering.
• Added focused compiler, VM, object, code, cache, fast path, concurrency, budget, partial, and parity tests.
• Added benchmark coverage for one-shot renders, compiled-template reuse, cached filenames, reused contexts, and render-mode comparisons.

Compatibility

This is opt-in. Existing applications continue using the interpreter unless they explicitly enable VM render mode or call the VM package directly.

import (
  plush "github.com/gobuffalo/plush/v5"
  _ "github.com/gobuffalo/plush/v5/VM/plush"
)

func init() {
  plush.SetRenderMode(plush.RenderModeVM)
}

For repeated direct renders:

import vmplush "github.com/gobuffalo/plush/v5/VM/plush"

tmpl, err := vmplush.Compile(`<h1><%= title %></h1>`)
if err != nil {
  return err
}

out, err := tmpl.Render(ctx)

VM Bytecode Cache

The VM bytecode cache reuses compiled bytecode for file-backed templates. The interpreter already caches parsed ASTs through the template cache. This PR extends the same cache entry so the VM can store compiled bytecode beside the AST.

The first render of a cached file-backed template does this:

1. Parse template source.
2. Compile the AST into VM bytecode.
3. Store the bytecode on the shared template-cache entry.
4. Execute the bytecode.

Later renders of the same filename can skip parse and compile:

1. Look up the template-cache entry by filename cache key.
2. Load the cached VM bytecode.
3. Execute it with the current request context.

The cache stores bytecode and execution plans, not request data. It does not cache context values, helper return values, struct fields, branch results, rendered HTML, or partial output. Each render still reads fresh values from the provided context.

The cache also respects Plush safety rules:

• File-backed .plush, .plush.html, and compatible .html templates can use the filename bytecode cache.
• Punch-hole renders avoid unsafe full-template bytecode reuse when the hole context requires separate behavior.
• Partial bytecode links are keyed by partial identity and source hash so updated partial source does not reuse stale bytecode.
• Source-unknown plain .html partials still consult the current partial feeder when needed, so cached bytecode does not silently reuse stale partial source.

Fast Paths

The VM includes automatic fast paths for common Plush shapes. These require no template changes:

• static text and raw output opcodes
• direct output writing through strings.Builder
• simple variable interpolation
• top-level property chains like <%= item.Name %>
• nested property/index chains like <%= robots[0].Name.Echo() %>
• typed map access such as labels["status"]
• top-level infix output such as <%= count + 1 %>
• safe mixed numeric comparisons such as uint32 == 0 and float32 == 3
• simple conditionals and infix conditions
• static and mixed templates with reusable execution plans
• loop-local slot specialization
• typed loop field getters for slices of structs and pointers
• helper calls in hot output positions
• method calls in hot output positions
• optional custom fast helpers for application-specific hotspots
• inline partial rendering when the partial bytecode can be safely linked
• partial data maps such as partial("row", {name: item.Name})
• partial data maps with helper-call values such as partial("row", {label: label(item.Name, prefix)})

Fast paths cache lookup plans and call plans, not the current values. For example, a loop over items may cache how to read Name from the item type, but it still reads each item's current Name value on every render.

New Language And Runtime Behavior

This PR also brings the following behavior into both the interpreter and VM paths where applicable:

• optional parentheses around if conditions #102
• trim-tag whitespace handling #89
• partial rendering with explicit data maps
• safe mixed numeric equality and ordering across common Go numeric types
• safer comparisons for backend values such as int32, uint32, uint64, float32, and float64
• render budgets with work counters, helper cost overrides, object traversal cost, and sub-render accounting
• struct, pointer, map, method, and nested chain access parity

Benchmarks

Latest local checkpoint, comparing VM bytecode cache with interpreter AST cache:

Scenario	Speed	B/op	allocs/op
Full render-mode matrix, 26 scenarios	61.2% faster	59.6% less	71.8% fewer
Realistic mixed template	59.6% faster	73.3% less	81.3% fewer
Simple conditional access output	45.0% faster	32.8% less	65.6% fewer
Nested access/simple access output	55.4% faster	56.7% less	61.8% fewer
Partial simple access bodies	52.5% faster	49.6% less	70.6% fewer
Partial rendering with data maps	53.6% faster	55.6% less	68.9% fewer
Partial data maps with helper-call values	63.4% faster	53.5% less	68.0% fewer
Typed map access	35.6% faster	24.8% less	56.8% fewer
Loop helper direct string calls	69.6% faster	50.6% less	74.5% fewer
Loop helper direct int calls	58.9% faster	51.1% less	70.3% fewer
Mixed numeric operator output	46.4% faster	28.3% less	61.5% fewer

These numbers come from local benchmark medians using count=3 and 500ms benchtime. Exact results will vary by machine and template shape. One-shot VM rendering is intentionally not the headline number because parse and compile time are included; the main performance path is cached bytecode or compiled-template reuse.

Integrated application validation was also run against a live local Buffalo application using response headers from the render engine. These measurements include real application helpers, nested partials, large generated output, request context setup, and application-specific template work, so they are intentionally more conservative than the VM microbenchmarks:

Live scenario	Interpreter avg	VM avg	VM avg gain	Interpreter p95	VM p95	VM p95 gain
Light template scenario	69.737ms	47.214ms	32.30% faster	111.960ms	85.337ms	23.78% faster
Mixed template scenario	103.631ms	71.615ms	30.89% faster	149.934ms	104.843ms	30.07% faster
Heavy helper/partial scenario	333.565ms	279.348ms	16.25% faster	442.746ms	344.368ms	22.22% faster

The live scenario tests were run by toggling the application render mode, warming each page, then collecting 100 HTTPS requests at concurrency 10 with curl -k. The comparison uses the X-Plush-Render-Engine-Time-Ms response header, not total ApacheBench wall time, so the numbers are scoped to the Plush render engine and the helpers/partials executed during render.

These X-Plush-* headers are new render diagnostics added for the VM work. They are intended for validation, profiling, and A/B testing; they are not nginx, Buffalo, or browser-provided timings.

For each scenario:

1. Start in interpreter mode with PLUSH_RENDER_MODE=interpreter.
2. Warm the page and collect the interpreter header CSV.
3. Switch to compiled mode with PLUSH_RENDER_MODE=vm.
4. Warm until X-Plush-Vm-Bytecode-Cache: hit and X-Plush-Fast-Path: fast.
5. Collect the VM header CSV.
6. Compare averages and p95 values from the saved CSV files.

Example live benchmark commands:

make headers-interpreter AB_URL="https://example.test/path" HEADER_RESULTS_DIR=tmp/render-headers-scenario HEADER_REQUESTS=100 HEADER_CONCURRENCY=10
make headers-vm AB_URL="https://example.test/path" HEADER_RESULTS_DIR=tmp/render-headers-scenario HEADER_REQUESTS=100 HEADER_CONCURRENCY=10
make headers-compare HEADER_RESULTS_DIR=tmp/render-headers-scenario

The generated response body sizes were also measured from the live scenarios with curl using the downloaded body byte count. These are uncompressed body bytes as seen by curl during local HTTPS requests:

Live scenario	Generated body bytes	Generated body size
Light template scenario	45,999 bytes	44.92 KiB
Mixed template scenario	628,002 bytes	613.28 KiB
Heavy helper/partial scenario	22,536,853 bytes	21.49 MiB

The live benchmark headers mean:

Metric	Meaning
X-Plush-Render-Engine-Time-Ms	Time spent inside Plush rendering for this request. This includes Plush execution plus helpers and partials called while rendering, but not full network/nginx/client time.
X-Plush-Render-Mode	Which Plush engine handled the render: interpreter or vm.
X-Plush-Vm-Bytecode-Cache	VM bytecode cache state. A warmed VM benchmark should show hit; a first render may show miss-store; interpreter mode reports disabled.
X-Plush-Fast-Path	Whether the VM used its optimized render path. fast means the compiled fast plan handled the render; generic means first compile/generic execution/fallback behavior.
X-Plush-Fast-Plan-*	Static complexity counters from the compiled plan, such as bindings, segments, property reads, helper calls, conditionals, loops, partials, and max nesting depth. These describe the template shape, not elapsed time.
X-Plush-VM-Helper-Time-Ms	Optional hotspot diagnostic: total time spent inside VM fast helper calls for the render. It is populated only when VM hotspot diagnostics are enabled.
X-Plush-VM-Helper-Calls	Optional hotspot diagnostic: number of VM fast helper calls observed during the render.
X-Plush-VM-Helper-Hotspots	Optional hotspot diagnostic in name:calls:time_ms form, for example format_label:700:3465.659 in an aggregated CSV sample.
X-Plush-VM-Partial-Time-Ms	Optional hotspot diagnostic: total time spent rendering VM partials.
X-Plush-VM-Partial-Calls	Optional hotspot diagnostic: number of partial renders observed during the render.
X-Plush-VM-Partial-Hotspots	Optional hotspot diagnostic in partial:calls:time_ms form, useful for identifying expensive nested partials.

The tested pages had very different template shapes:

Live scenario	Bindings	Segments	Static segments	Property reads	Value writes	Helper calls	Conditionals	Loops	Loop parts	Partials	Max depth
Light template scenario	1	3	2	0	0	1	0	0	0	0	3
Mixed template scenario	26	80	193	162	41	27	41	7	116	1	12
Heavy helper/partial scenario	14	13	63	39	2	18	12	2	20	6	12

The light scenario is a compact template with one helper call, so it mostly measures the fixed overhead of entering the render engine and executing a simple plan. The mixed scenario exercises a wider template shape with many property reads, conditionals, loops, helper calls, and one partial. The heavy helper/partial scenario has fewer top-level compiled-plan operations than the mixed scenario, but it performs much more work at runtime because its partials and helpers expand into additional nested rendering and data-preparation work.

The heavy helper/partial scenario is intentionally a large stress case rather than a small synthetic template. It includes many nested partial renders and real template helpers, such as formatting helpers, asset/tag helpers, URL/path helpers, content helpers, and helpers that prepare structured data for output. In one VM hotspot sample, the render spent about 36.461ms/request inside helpers and about 157.198ms/request inside partial rendering. That means a large part of the remaining render time was not VM bytecode execution itself; it was work the VM correctly had to call into. This makes the scenario a useful ceiling check: the VM still wins, but once parse, compile, lookup, and opcode dispatch overhead are reduced, the next bottlenecks become helper logic, data formatting, and nested partial rendering.

Benchmark command:

go test ./VM/vm -run '^$' -bench '^BenchmarkRenderModeMatrix/.*/(interpreter_ast_cache|vm_bytecode_cache)$' -benchmem -count=3 -benchtime=500ms

Additional benchmark entry points:

go test ./VM/vm -run '^$' -bench '^BenchmarkRenderOneShot$' -benchmem
go test ./VM/vm -run '^$' -bench '^BenchmarkRenderCompiledReuse$' -benchmem
go test ./VM/vm -run '^$' -bench '^BenchmarkRenderReusedContextMatrix$' -benchmem
go test ./VM/vm -run '^$' -bench '^BenchmarkRenderCachedFilename$' -benchmem

Test Plan

Full package test suite:

go test ./...

VM-specific package tests:

go test ./VM/...

[Mido-sys]

@paganotoni Can you check why the tests failed to run?

[paganotoni]

@Mido-sys it's time to update our CI here.
Root cause from CI logs:

dyld[5414]: missing LC_UUID load command in .../plush.test
signal: abort trap
FAIL github.com/gobuffalo/plush/v5

Same missing LC_UUID load command aborts every package built with -race on macOS ARM64 + Go 1.22.12. dyld kills test binary before any test runs. Other matrix jobs cancel because fail-fast strategy sees first macOS failure.

This is Go 1.22 race-detector / macOS runner incompatibility, not a test assertion failure in PR code.

[paganotoni]

@Mido-sys can you try pulling from main into your PR branch?

[Mido-sys]

@paganotoni Done. But some tests timed out.

[Mido-sys]

@paganotoni ,

It's green now. Thanks.

[paganotoni]

Great! Thanks for this. I learned something new given how fast the rendering is going. Apparently Windows Tick is 15ms and this is, of course, faster.