Proposal: low-overhead stack-keyed sampled heap profiling

[cpunion]

Summary

Implement Go-compatible sampled heap profiles on native LLGo targets while keeping profiling work out of ordinary executables and minimizing the enabled allocator fast path.

There are two target paths:

• native targets with physical stack walking use stack-and-size keyed samples;
• wasm and bare-metal targets retain the existing size-class fallback until they have an equivalent stack source.

Allocator replacement and sampled-object death tracking are separate work because they depend on allocator and GC lifecycle details.

Goals

• preserve the public runtime.MemProfile, runtime.MemProfileRate, and runtime/pprof contracts;
• attribute native samples to user allocation stacks and source lines;
• use randomized exponential thresholds and expose raw samples for Go's normal Poisson correction;
• give ordinary executable builds no profiler work in AllocZ or AllocU when profiling is provably unused;
• keep the enabled common path to a rate load, one TLS lookup, and a countdown update;
• avoid global lock contention for existing buckets and snapshots;
• preserve correctness under concurrent allocation and profile collection;
• make native, wasm, bare-metal, LTO, and library behavior explicit.

Design

1. Native and fallback recorders

Native LLGo builds that retain profiling install a physical-stack capture hook during runtime initialization; proven non-consumer executables omit this setup entirely. Their allocator recorder has no fallback-mode dispatch. Wasm, bare-metal, and host-tool builds compile a separate size-class recorder.

A native allocation checks MemProfileRate before touching TLS. Rate zero returns without a TLS access. Otherwise, one physical-thread-local state holds the sampling countdown and RNG state. Threshold generation, recursion suppression, stack walking, bucket creation, and random-number work remain in a noinline sampled slow path. This matches LLGo's current 1:1 native-thread execution model and avoids shared countdown races.

2. Whole-program disabled-path selection

After all Go SSA is available but before package backends start, the build coordinator conservatively detects whether non-runtime code consumes runtime.MemProfile, runtime/pprof, or runtime.MemProfileRate.

• For a proven non-consumer executable, code generation omits the direct recordMemProfileAlloc calls from the internal runtime allocator and the public runtime profile-setup call. Frame-table initialization, hook installation, capture, and recorder bodies then become unreachable. AllocZ and AllocU keep their existing names, signatures, and ABI; there are no alternate allocator entries or runtime mode branches.
• Any uncertain executable keeps recording. False positives cost performance but never correctness.
• c-archive and c-shared always keep recording because future external calls are not visible during whole-program analysis.
• Full-LTO and non-LTO builds use the same pre-backend decision. Linker dead stripping then removes the unused profiler implementation from non-consumer executables.

The decision is copied to isolated backend programs and included only in the internal runtime package's cache fingerprint. Switching between profiling and non-profiling executables cannot reuse an incompatible allocator archive and does not make the setting part of user-package ABI.

3. Bucket key and concurrency

A native bucket is keyed by (stack, allocation size), matching Go's correction model for mixed-size allocations at one call site. Bucket nodes and next links are immutable after publication.

Lookup traverses an atomically loaded head without a lock. On a miss, the slow path allocates a node, takes the insertion lock, rechecks, initializes counters, and atomically publishes it. Existing counters use atomic updates. A snapshot captures each table head once and traverses immutable chains without stopping allocators.

Profile materialization temporarily suppresses sampling on its physical thread. This prevents rate-1 profiling from recursively profiling symbolization and buffer growth while leaving other threads active.

4. Stack attribution

Allocator and capture plumbing is trimmed from exposed records. Profile-enabled compilation preserves physical frames on compiler-classified allocation paths across the whole static call graph, including cross-package wrappers, rather than disabling inlining for unrelated helpers. A general inline-call-site representation remains the preferred future replacement for retaining physical frames.

5. Compatibility

Changing MemProfileRate at runtime remains supported in builds that retain profiling. Rate 1 samples every allocation; rate 0 disables recording immediately. Default rates store raw counts so existing Go consumers apply their normal correction.

Wasm and bare-metal keep the size-class fallback. Moving either target to stack-keyed recording later does not require a Go API or allocator ABI change.

Validation

Required coverage includes:

• native Darwin/arm64 and Linux/amd64 stack attribution;
• rate 0, default rate, rate 1, and runtime rate changes;
• same-stack mixed-size allocations producing distinct buckets;
• concurrent native sampling and snapshots;
• runtime/pprof heap output and Go GOROOT heapsampling.go;
• wasm and bare-metal fallback compilation;
• executable, c-archive, c-shared, non-LTO, and full-LTO selection;
• final-symbol and disassembly checks proving a non-consumer executable has no allocator hook;
• allocation microbenchmarks separating allocator cost from disabled-hook, enabled-countdown, and sampled slow-path cost.

Prototype results

PR #2027 follows the design above without an allocator ABI or target-policy change. On native non-consumer executables, recordMemProfileAlloc, stack capture, and bucket symbols are absent, and AllocZ contains only allocation and zeroing. The enabled path resolves TLS once and enters the noinline slow path only at a sampling boundary.

On an Apple M4 Max (Darwin/arm64, Go 1.26.5), a GC-disabled allocation microbenchmark used 21 rotated rounds; each process ran 5 warm-up and 61 measured batches of 250,000 escaping 16-byte allocations. The table reports medians of process medians; signed differences subtract those displayed medians, while percentages are paired within each round.

Toolchain and path	ns/op	Signed difference / paired change
LLGo PR base (0f480253): unconditional size-class recorder	19.11	baseline
PR #2027: no profile consumer	18.04	-1.07 ns/op / -6.5% vs PR base
PR #2027: consumer, rate 0	18.99	+0.95 ns/op / +6.1% vs omitted; -0.12 ns/op / -2.4% vs PR base
PR #2027: consumer, default rate	20.03	+1.99 ns/op / +11.1% vs omitted; +0.92 ns/op / +5.8% vs PR base
Go 1.26.5: rate 0	12.69	Go baseline
Go 1.26.5: default rate	12.84	+0.15 ns/op / +0.3% vs Go rate 0

The standard Go toolchain has no no-consumer specialization, so rate 0 is its closest disabled path. Against Go's corresponding rate-0/rate-0/default paths, LLGo's omitted/rate-0/default medians differ by +5.35/+6.30/+7.19 ns/op and the paired changes are +39.9%/+48.7%/+53.0%. This gap includes allocator/runtime differences; profiling overhead is represented by the within-toolchain comparisons. LLGo used GC_disable; Go used debug.SetGCPercent(-1).

Expected benefits

• no profiling calls, eager setup, or reachable profiler bodies in ordinary executables where profiling is unused;
• no per-allocation function call, repeated TLS resolution, RNG work, or fallback dispatch in profiled native programs;
• no global serialization for repeated samples or snapshots;
• correct Poisson correction for mixed allocation sizes;
• unchanged public allocator ABI and isolated target fallbacks.

Minimal Linux cprintf remains +256 B on disk because 16 additional pre-DCE funcinfo symbol-index records (16 B each) remain in .rodata; .relro_padding shrinks by the same amount, so .text, loaded-section total, and page count are unchanged. This is a metadata-filtering limitation, not retained profiler machine code.

Non-goals and rollout

PR #2027 is the implementation vehicle for stack-keyed sampling, the native hot-path split, whole-program hook omission, correct bucket keys, lock-free lookup/snapshot, attribution narrowing, and selection tests. Allocator-specific tuning, sampled-object free accounting, improved wasm/bare-metal stack capture, and dead funcinfo filtering remain follow-up work.