Skip to content

Commit 5efa19d

Browse files
docs(adr-0015): vLLM prefill techniques through the Kakeya lens (adopt/wrap/drop)
Record the precise stance: chunked prefill = our native chunked restoration (adopt, reinterpreted); FlashAttention = a kernel we wrap (table stakes); paged KV = structurally NOT needed (paging manages a growing full KV that a Kakeya-native on-demand-restoration engine never holds). The engine needs chunked restoration + wrapped flash + native bounded-KV mgmt, not PagedAttention - the concrete sense in which it replaces vLLM's design rather than extending it. Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
1 parent 9420e49 commit 5efa19d

1 file changed

Lines changed: 17 additions & 0 deletions

File tree

docs/adr/0015-kakeya-attention-and-engine-substrate.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,23 @@ A product Kakeya engine instead makes **bounded-KV the native invariant**:
4646
product engine needs, implemented *in service of* the Kakeya-native design —
4747
not as a port of vLLM's full-KV pipeline.
4848

49+
### vLLM's three prefill techniques, through the Kakeya lens (adopt / wrap / drop)
50+
51+
vLLM gets its 62k concurrency from three prefill-engineering pieces. They are
52+
**not** copied wholesale; each is reinterpreted against the bounded-KV-native
53+
design — two are adopted (one reinterpreted, one wrapped), the third is
54+
structurally **unnecessary**:
55+
56+
| vLLM technique | what it solves for vLLM | Kakeya engine stance |
57+
| --- | --- | --- |
58+
| **Chunked prefill** | process a long prompt in fixed token blocks so mask/activation memory is O(N·chunk·d), not O(N·T²) | **Adopt — it *is* our chunked restoration.** Restoration is inherently incremental: consume the prompt in fixed blocks, emitting the bounded resident set + restoration path per block. Chunking is native to how restoration works, not a bolt-on. |
59+
| **FlashAttention** (native causal + sliding-window kernel) | compute attention without materializing a `[.,.,T,T]` mask/score tensor | **Wrap and use directly.** Window is a kernel parameter; we call the flash kernel over the Kakeya window. It is a table-stakes kernel, not an architecture — no reason to reinvent it. |
60+
| **Paged KV** | store the **whole growing KV** in non-contiguous pages so a large, ever-growing cache fits and shares | **Not needed — structurally.** Paging is a solution to the problem of *storing a growing full KV*. Kakeya is **on-demand KV restoration**: the resident KV is bounded (sink+window + exact layers) and the full history is never stored, so there is no growing full-KV to page. The problem paging solves **does not exist** in a Kakeya-native engine. |
61+
62+
So the engineering Kakeya needs is **chunked restoration + a wrapped flash kernel
63+
+ native bounded-KV management***not* PagedAttention. This is the concrete
64+
sense in which the engine **replaces** vLLM's design rather than extending it.
65+
4966
## Kakeya Attention — the native algorithm
5067

5168
**Kakeya Attention** = sink+window bound + f_θ KV-projection + dLLM-proposer

0 commit comments

Comments
 (0)