Production paged + tiered KV cache for the agent army (MAD-126) - #12

Merged

kmbandy merged 20 commits into

masterfrom

feat/MAD-126-army-goal

May 11, 2026

Commits on May 10, 2026

mt:: paged-attn — paged-block semantic prefetch infra (MAD-122, partial)

kmbandy
and
claude
committed
b5f029b
mt:: server — paged-block semantic fingerprinting + restore (MAD-122)

kmbandy
and
claude
committed
d432230
mt:: paged-attn — relocate semantic prefetch API to llama_kv_cache_paged (MAD-125 follow-up)

kmbandy
and
claude
committed
e16916d
mt:: ARCH-CLEANUP — remove legacy tiered paths; thin the wrapper (MAD-127)

kmbandy
and
claude
committed
7bf8a5b
mt:: paged-attn — block-aligned partial seq_rm (MAD-128, part 1)

kmbandy
and
claude
committed
d23d327
mt:: server — clearer paged ctx-shift fallback messaging (MAD-128, part 2)

kmbandy
and
claude
committed
db75156
mt:: paged-attn — seq_cp CoW via BlockPool refcounting (MAD-128, part 3)

kmbandy
and
claude
committed
f3ec51c
mt:: semantic prefetch — prefill-time write trigger on llama_kv_cache_paged (MAD-129)

kmbandy
and
claude
committed
9299eab
mt:: paged-attn — state persistence + cold-tier resume + fingerprint save/load (MAD-130)

kmbandy
and
claude
committed
e2f9e8f
mt:: multi-instance — per-instance cold subdir + lockfile + budget cap (MAD-131)

kmbandy
and
claude
committed
4f667ab
mt:: paged-attn — concurrency hardening (MAD-132)

kmbandy
and
claude
committed
50e1ce6
mt:: observability — tier counters + /metrics/tier + /slots tier extension + per-batch log (MAD-133)

kmbandy
and
claude
committed
de644a5
mt:: ergonomics — paged-default-on for tiered + bge warmup + config validation (MAD-134)

kmbandy
and
claude
committed
93796b6
mt:: paged-attn — F16 cold-tier int4 compression + round-trip tests (MAD-135)

kmbandy
and
claude
committed
6358dcc
mt:: build — multi-target HIP (gfx1201;gfx1030) + comment fix (MAD-136 partial)

kmbandy
and
claude
committed
7d520e4
mt:: tests — unit tests for tier primitives (MAD-137 partial)

kmbandy
and
claude
committed
d788f7a
mt:: tests — integration + stress + matrix runner + CI (MAD-137)

kmbandy
and
claude
committed
b9ad370
server: fail-fast on paged admission deadlock instead of GGML_ABORT (MAD-141)

kmbandy
and
claude
committed
0d66d8a
ci: strip self-hosted matrix from army workflow

kmbandy
and
claude
committed
1871f4f
docs: tiered KV cache user guide + operator runbook + architecture (MAD-138)

kmbandy
and
claude
committed
9c16a2f

Navigation Menu

Production paged + tiered KV cache for the agent army (MAD-126) - #12

Production paged + tiered KV cache for the agent army (MAD-126)#12
kmbandy merged 20 commits into
masterfrom
feat/MAD-126-army-goal

Commits on May 10, 2026

mt:: paged-attn — paged-block semantic prefetch infra (MAD-122, partial)

mt:: server — paged-block semantic fingerprinting + restore (MAD-122)

mt:: paged-attn — relocate semantic prefetch API to llama_kv_cache_paged (MAD-125 follow-up)

mt:: ARCH-CLEANUP — remove legacy tiered paths; thin the wrapper (MAD-127)

mt:: paged-attn — block-aligned partial seq_rm (MAD-128, part 1)

mt:: server — clearer paged ctx-shift fallback messaging (MAD-128, part 2)

mt:: paged-attn — seq_cp CoW via BlockPool refcounting (MAD-128, part 3)

mt:: semantic prefetch — prefill-time write trigger on llama_kv_cache_paged (MAD-129)

mt:: paged-attn — state persistence + cold-tier resume + fingerprint save/load (MAD-130)

mt:: multi-instance — per-instance cold subdir + lockfile + budget cap (MAD-131)

mt:: paged-attn — concurrency hardening (MAD-132)

mt:: observability — tier counters + /metrics/tier + /slots tier extension + per-batch log (MAD-133)

mt:: ergonomics — paged-default-on for tiered + bge warmup + config validation (MAD-134)

mt:: paged-attn — F16 cold-tier int4 compression + round-trip tests (MAD-135)

mt:: build — multi-target HIP (gfx1201;gfx1030) + comment fix (MAD-136 partial)

mt:: tests — unit tests for tier primitives (MAD-137 partial)

mt:: tests — integration + stress + matrix runner + CI (MAD-137)

server: fail-fast on paged admission deadlock instead of GGML_ABORT (MAD-141)

ci: strip self-hosted matrix from army workflow

docs: tiered KV cache user guide + operator runbook + architecture (MAD-138)