Stage M6 — Static decode plan on Apple (old 20, Apple half)
Goal: make the Apple hot decode path boring.
- After warm-up: weights resident, KV preallocated to context bound,
scratch arena fixed, pipelines compiled and cached, zero heap
allocation per token — asserted by instrumentation, not claimed.
- Bounded in-flight work; predictable encode/submit structure per token;
ICB/replay decision from M3 finalized here.
- Memory report: weights / KV / scratch / peak RSS, per model and
context length.
Gate: an allocation-tracking assertion passes over a long generation;
per-token latency variance is characterized (p50/p95/p99).
Tutorial: docs/tutorials/20-static-decode-plan.md.
Stage M6 — Static decode plan on Apple (old 20, Apple half)
Goal: make the Apple hot decode path boring.
scratch arena fixed, pipelines compiled and cached, zero heap
allocation per token — asserted by instrumentation, not claimed.
ICB/replay decision from M3 finalized here.
context length.
Gate: an allocation-tracking assertion passes over a long generation;
per-token latency variance is characterized (p50/p95/p99).
Tutorial:
docs/tutorials/20-static-decode-plan.md.