docs(adr-0015): reframe north star — product-grade engine to REPLACE vLLM, Kakeya Attention native

Per direction: ADR 0015's prior 'add chunked-prefill+paged+CUDA-graph' roadmap
read as copying vLLM and treated the research bench as the measuring stick. Both
wrong. New highest goal recorded in ADR: the Kakeya Inference Engine is a
product-grade engine whose goal is to REPLACE vLLM, with Kakeya Attention as its
native first-class algorithm. The engine is bounded-KV-native (full history never
resident; admission by peak window; restoration fused into prefill/decode), NOT a
port of vLLM's full-KV pipeline. The eager-transformers bench is a feasibility
probe only, never reported as 'Kakeya performance'. README + reports aligned.

Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>

Author: cursoragent

README.md

@@ -175,13 +175,19 @@ restoration compute (a proposer forward), quantified below (recall 1.0;
 ~AR-parity / 1.79–2.06× on CUDA; ~4× more concurrent agents per GB,
 [ADR 0014 §3.4](docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md)).
 
-**Engine substrate (target).** Kakeya Attention is the algorithm; the
-**Kakeya Inference Engine** combines it with **CUDA graphs + fused-MoE kernels**
-to compete with vLLM on absolute throughput. This substrate is a **build target**
-([ADR 0015](docs/adr/0015-kakeya-attention-and-engine-substrate.md)): today's
-research path runs the algorithm on an **eager HF-transformers** substrate, which
-is correct and recall-preserving but ~8–14× slower than vLLM and OOMs on long
-prefills — see [the same-H200 comparison](docs/reports/kakeya-vs-vllm-multitenant-h200.md).
+**North star — a product-grade engine that replaces vLLM.** Kakeya Attention is
+the native algorithm of a **product-grade inference engine whose goal is to
+replace vLLM** — not a technique bolted onto HuggingFace transformers, and not
+"vLLM with a different cache". The engine is designed **bounded-KV-native**: the
+full history is never resident, admission/scheduling sizes sessions by their
+**peak window** (not total tokens), and restoration is fused into prefill/decode.
+Graph-captured decode, fused-MoE and efficient masking are table stakes built *in
+service of* that design, not a port of vLLM's full-KV pipeline
+([ADR 0015](docs/adr/0015-kakeya-attention-and-engine-substrate.md)). The
+eager-transformers numbers in the comparison reports are **feasibility probes**,
+not "Kakeya performance"; the vLLM-beating demonstration runs on a
+**full-attention** verifier, where restoration is load-bearing (on gemma-4 its
+native sliding window already bounds 25/30 layers, so it is not the showcase).
 
 **Where the bounded-KV win is large (and where it isn't).** The advantage scales
 with the model's **full-attention fraction**. On **gemma-4-26B-A4B** only 5 of 30

docs/adr/0015-kakeya-attention-and-engine-substrate.md

@@ -1,98 +1,108 @@
-# ADR 0015 — Kakeya Attention as an attention algorithm + engine substrate roadmap
+# ADR 0015 — Kakeya Inference Engine: a product-grade vLLM replacement, Kakeya Attention native
 
-- Status: Accepted (definition); substrate = roadmap
-- Date: 2026-06-15
+- Status: Accepted (north star + algorithm definition); engine = in design
+- Date: 2026-06-15 (rev. 2026-06-15)
 - Supersedes/extends: the "Kakeya Attention vs PagedAttention/RadixAttention"
-  framing in README and ADR 0014 §3.4 (MLX serial-only / CUDA batched).
+  framing in README; ADR 0014 §3.4.
+
+## North star (highest goal — this governs everything below)
+
+The **Kakeya Inference Engine is a product-grade inference engine whose goal is
+to replace vLLM.** It is **not** a research script, **not** a technique bolted
+onto HuggingFace transformers, and **not** "vLLM with a different cache". Its
+**native, first-class attention algorithm is Kakeya Attention** — sink+window
+bound + f_θ KV-projection + dLLM-proposer restoration, as **one primitive**. The
+whole engine (prefill, KV management, admission/scheduling, kernels) is designed
+**around bounded-KV + on-demand restoration as the default invariant**, exactly
+the way vLLM is designed around full-KV PagedAttention.
+
+Everything else in this ADR serves that objective. Explicitly:
+
+- The product target is measured against vLLM by the **engine**, never by the
+  research bench. The bench (`k3_cuda_multitenant_parallel_bench.py`, eager HF
+  transformers) is **only a correctness/feasibility probe** and is not a thing we
+  ship or benchmark as "Kakeya".
+- "Parity → win" with vLLM is an engine deliverable, not a roadmap of vLLM
+  features to copy.
+
+## Why "borrow vLLM's pipeline" is the wrong plan (rejected)
+
+vLLM's architecture is **full-KV-centric**: paged blocks store the *whole*
+history; chunked prefill and flash masking are optimizations for *processing and
+storing the full KV*. Porting Kakeya's bounded cache onto that pipeline inherits
+a full-KV-shaped engine and caps the advantage at whatever the cache layout
+saves — i.e. it makes Kakeya a vLLM feature, not a replacement.
+
+A product Kakeya engine instead makes **bounded-KV the native invariant**:
+
+- the **full history is never resident**; evicted context is reconstructed by
+  the proposer on demand (Kakeya Attention);
+- **prefill** produces the bounded resident set **+** the restoration path in one
+  pass — it must never materialize the O(N·T²) attention mask or full-vocab
+  logits that sink the research bench;
+- **admission/scheduling** sizes sessions by their **peak window**, not their
+  total token count — this is the structural source of the concurrency win;
+- graph-captured decode, fused-MoE, efficient masking are **table stakes** any
+  product engine needs, implemented *in service of* the Kakeya-native design —
+  not as a port of vLLM's full-KV pipeline.
+
+## Kakeya Attention — the native algorithm
+
+**Kakeya Attention** = sink+window bound + f_θ KV-projection + dLLM-proposer
+restoration, taken as one primitive. Peer of, and replacement for, the attention
+layer in current engines:
 
-## Context
+| Algorithm | Replaces | Keeps full KV? |
+| --- | --- | --- |
+| eager / **FlashAttention** | the **compute** layer | yes |
+| vLLM **PagedAttention** / SGLang **RadixAttention** | the **storage** layer | yes |
+| **Kakeya Attention** | **compute + storage** | **no — bounded; evicted KV reconstructed on demand** |
 
-Kakeya's KV-restoration work (AR verifier + dLLM proposer + f_θ + S5 sink/window)
-has been described as a *technique inside* the engine. A same-H200 benchmark vs
-vLLM (`docs/reports/kakeya-vs-vllm-multitenant-h200.md`) and a long-context probe
-(`docs/reports/kakeya-vs-vllm-longcontext-h200.md`) forced two clarifications:
+FlashAttention makes attention compute cheaper; Paged/Radix make the same *total*
+KV cheaper to allocate/share. **Kakeya Attention makes the total itself bounded.**
 
-1. The throughput gap to vLLM (~8–14×) is a **substrate** gap (eager HF
-   transformers decode loop, O(T²) eager prefill), **not** an algorithmic one.
-2. The bounded-KV memory advantage is **architecture-dependent** and is a
-   **decode-time** property that the eager prefill currently masks (OOM at 32k).
+## Where the win is real (model architecture matters)
 
-This ADR fixes the vocabulary and the roadmap.
+Resident KV is dominated by the **full-attention** layers (they hold full context
+in any engine). So the engine's advantage over vLLM scales with the model's
+full-attention fraction:
 
-## Decision
+- **gemma-4-26B-A4B** is 25/30 **natively sliding** → vLLM already bounds those
+  layers, and gemma-4 keeps recall 1.0 at `sliding_window=68` *with no
+  restoration at all* → **no Kakeya moat on gemma-4** (probed; see below). It is
+  the wrong showcase model.
+- **Full-attention models** (Qwen/Llama, no native sliding): shrinking the window
+  without restoration **destroys recall**, so f_θ+proposer restoration is the
+  *only* way to bound memory at full recall — and vLLM, having no restoration,
+  **must keep full KV and cannot match it**. This is the engine's target regime.
 
-### 1. Kakeya Attention is a first-class attention algorithm
+## Feasibility probes so far (informed the design — NOT the product)
 
-Define **Kakeya Attention** = **sink+window bound + f_θ KV-projection +
-dLLM-proposer restoration, as one primitive**. It is a peer of, and a drop-in
-replacement for, the attention layer in current engines:
+These ran on the eager-transformers research bench; they validate correctness and
+locate the engine's required invariants. They are not the product engine.
 
-| Algorithm | Replaces | Keeps full KV? |
-| --- | --- | --- |
-| eager / **FlashAttention** | the **compute** layer | yes |
-| vLLM **PagedAttention** / SGLang **RadixAttention** | the **storage** layer | yes |
-| **Kakeya Attention** | **compute + storage** | **no — bounded; evicted KV reconstructed on demand** |
+- **Restoration prefill, memory-efficient (SDPA + chunked logits + bf16 K/V)** —
+  unblocked long-context execution at recall 1.0 (16k N=1-only→N=4, 32k OOM→N=2,
+  62k OOM→N=1). Confirms the engine must avoid O(N·T²) masks / full-vocab logits.
+- **gemma-4 bounded decode** — recall 1.0 at `sliding_window=68` natively (no
+  restoration); native bounded decode still fits only N=2 @62k vs vLLM's 15.5
+  because the bench does non-chunked prefill. Confirms (a) gemma-4 is the wrong
+  showcase, (b) the engine — not bench retrofits — is what must beat vLLM.
 
-FlashAttention makes attention compute cheaper; Paged/Radix make the same *total*
-KV cheaper to allocate/share. **Kakeya Attention makes the total itself bounded**,
-and is **composable** with all of them (a flash kernel can compute a Kakeya
-window; a paged/radix store can hold it).
-
-### 2. The engine substrate is Kakeya Attention + CUDA graphs + fused MoE
-
-The **Kakeya Inference Engine** = Kakeya Attention on a production substrate
-(**CUDA graphs + fused-MoE kernels + memory-efficient prefill**), targeting vLLM
-on absolute throughput. This is a **build target**, not yet shipped. The current
-research path runs the algorithm on **eager HF transformers**, which is correct
-and recall-preserving but (a) ~8–14× slower than vLLM at ctx-1238 and (b) OOMs on
-long prefills (N=1-only at 16k, OOM at 32k) due to O(T²) eager scores +
-full-vocab logits + a redundant `capture_verifier_own_kv` forward.
-
-**Substrate work items (ordered):**
-1. ✅ **Memory-efficient restoration prefill** — patched forward routes to
-   **SDPA** (`--attn-impl sdpa`), LM-head logits chunked (`logits_to_keep=1` on
-   the restored forward + `capture_verifier_own_kv`), restored K/V held in bf16.
-   **Unblocked long context** (recall 1.0): 16k went N=1-only→N=4, 32k OOM→N=2,
-   62k OOM→N=1; 16k N=1 peak 138.8→74.5 GB. (`docs/reports/kakeya-vs-vllm-longcontext-h200.md`.)
-2. **Bounded decode cache as the native KV layout** — resident sink+window +
-   5 exact full-attention layers; no Python per-step `DynamicCache`. **Probed on
-   gemma-4 and found NOT to beat vLLM there (by architecture):** (a) gemma-4
-   keeps recall 1.0 at `sliding_window=68` *natively, with no restoration* (5/30
-   full-attn layers carry recall) — so bounded decode is just a native window
-   shrink that vLLM can match; (b) even native bounded decode fits only N=2 @62k
-   vs vLLM's 15.5, because the bottleneck is **non-chunked prefill activations**,
-   not the KV bound — vLLM wins on **chunked-prefill + paged-KV engineering**.
-   **Conclusion: the Kakeya bounded-KV win must be demonstrated on a
-   full-attention model** (Qwen/Llama), where shrinking the window without
-   restoration destroys recall and only f_θ+proposer restoration bounds memory at
-   full recall — vLLM, lacking restoration, must keep full KV.
-   (`docs/reports/kakeya-vs-vllm-longcontext-h200.md` Update 2.)
-3. **CUDA graphs** for the decode step; **fused-MoE** kernels for the verifier.
-4. (Optional) integrate Kakeya Attention as a **vLLM attention backend** so the
-   bounded window rides vLLM's paged store + scheduler.
-
-### 3. The bounded-KV win is architecture-dependent
-
-Resident KV is dominated by the **full-attention** layers (they hold full
-context in any engine). gemma-4-26B-A4B is 25/30 **natively sliding** → vLLM
-already bounds 25 layers → Kakeya's resident-KV edge is **~7 % at 62k**. On a
-**full-attention** model (no native sliding) the edge is **~6×**. Long-context
-concurrency "sweet spots" must be claimed **per model architecture**, not
-universally. (Consistent with the recorded "S5 free-lunch" caveat: on gemma-4 the
-5-exact-layer S5 shortcut already covers recall, so the proposer/f_θ restoration
-is *replaced* by S5 on this model.)
+(`docs/reports/kakeya-vs-vllm-longcontext-h200.md`.)
 
 ## Consequences
 
-- README and docs now present Kakeya Attention as an algorithm peer to
-  Flash/Paged/Radix, with the substrate explicitly a roadmap.
-- Throughput parity with vLLM is gated on the substrate work items above, not on
-  the algorithm.
-- Memory/sweet-spot claims are scoped by full-attention fraction; the strong
-  long-context demonstration should be run on a full-attention verifier.
+- The repo's highest engineering goal is the **product Kakeya Inference Engine
+  replacing vLLM**, with Kakeya Attention as its native algorithm; the eager
+  research bench is a probe only and is never reported as "Kakeya performance".
+- The engine is designed bounded-KV-native (admission by peak window; restoration
+  fused into prefill/decode), not as a port of vLLM's full-KV pipeline.
+- The vLLM-beating demonstration is to be run on a **full-attention** verifier,
+  where restoration is load-bearing.
 
 ## Evidence
 
 - `docs/reports/kakeya-vs-vllm-multitenant-h200.md` (ctx-1238, same H200)
-- `docs/reports/kakeya-vs-vllm-longcontext-h200.md` (16k–62k, OOM ceiling + KV model)
-- `results/research/{k3_cuda_multitenant_parallel,vllm_multitenant_parallel}_h200nvl_*.json`
+- `docs/reports/kakeya-vs-vllm-longcontext-h200.md` (16k–62k probes + KV model)
+- `results/research/{k3_cuda_multitenant_parallel,vllm_multitenant_parallel,gemma_bounded_decode}_h200nvl_*.{json,log}`

docs/adr/README.md

@@ -41,7 +41,7 @@ reader what was *not* chosen.
 | 0007 | [Cross-request KV cache reuse for long sessions](0007-cross-request-kv-reuse.md) | Superseded by 0008 |
 | 0008 | [Session-bound runtime + gRPC protocol](0008-session-bound-runtime-and-grpc-protocol.md) | Accepted |
 | 0014 | [Agent-connection capacity & cross-host proposer/verifier topology: test plan & results](0014-agent-connection-capacity-and-cross-host-topology-tests.md) | Accepted |
-| 0015 | [Kakeya Attention as an attention algorithm + engine substrate roadmap](0015-kakeya-attention-and-engine-substrate.md) | Accepted |
+| 0015 | [Kakeya Inference Engine: a product-grade vLLM replacement, Kakeya Attention native](0015-kakeya-attention-and-engine-substrate.md) | Accepted |
 
 Note: ADR numbering is monotonically increasing; in-flight or
 planned numbers (0005) appear in the index so readers can

docs/reports/kakeya-vs-vllm-longcontext-h200.md

@@ -91,15 +91,14 @@ patched forward routes to `all_attention_functions["sdpa"]`) + **chunked logits*
 (16k N=1 peak dropped 138.8 → 74.5 GB.) This is the item-#1 win: the restoration
 prefill now executes at 32k/62k where eager could not.
 
-**Remaining bottleneck (→ items #2–3).** Per-session memory still grows
-~linearly with context (~17 GB/session @16k, ~37 GB @32k) because **this bench
-captures the full-T K/V into the decode cache** and holds the f_θ projection
-intermediates — i.e. it does **not** yet exercise the *bounded* resident cache
-(SinkWindowKVCache). So the restored path here is still effectively full-KV at
-decode; beating vLLM's **15.5-way** 62k concurrency requires item #2 (bounded
-native decode cache: store only sink+window + 5 exact layers, restore on demand)
-+ freeing the f_θ intermediates. Item #1 was the prerequisite (it makes long
-prefills run at all); the concurrency-ceiling win is gated on #2.
+**Remaining bottleneck (this is a bench probe, not the engine).** Per-session
+memory still grows ~linearly with context (~17 GB/session @16k, ~37 GB @32k)
+because **this research bench captures the full-T K/V into the decode cache** and
+holds the f_θ projection intermediates — it does **not** exercise a bounded
+resident cache. That is precisely what the **product-grade Kakeya Inference
+Engine** (ADR 0015) owns natively: bounded-KV as the resident layout (sink+window
++ 5 exact layers, restore on demand), admission by peak window, restoration fused
+into prefill/decode — none of which a transformers research bench provides.
 
 Evidence: `results/research/k3_cuda_mt_longctx_sdpa_ceiling_h200nvl.log`.
 
@@ -136,9 +135,11 @@ engineering the research bench lacks.
 (a) there is no algorithmic lever — vLLM already exploits gemma-4's native hybrid
 attention and can shrink the window too; (b) vLLM's long-context concurrency is
 **prefill/KV engineering** (chunked prefill, paged KV, CUDA graphs, fused MoE),
-not the resident-KV bound. Matching it needs that engine substrate (ADR 0015
-items #2–4, essentially a vLLM-class build), which is orthogonal to the Kakeya
-algorithm. **The Kakeya advantage requires a full-attention model** (Qwen/Llama):
+not the resident-KV bound. Matching/beating it is the job of the **product-grade
+Kakeya Inference Engine** (ADR 0015) — a bounded-KV-native engine (admission by
+peak window, restoration fused into prefill/decode), **not** a port of vLLM's
+full-KV pipeline. **The Kakeya advantage requires a full-attention model**
+(Qwen/Llama):
 there, shrinking the window without restoration destroys recall, so f_θ+proposer
 restoration is the *only* way to bound memory at full recall — and vLLM, having
 no restoration, must keep full KV and cannot match it.