Add H3_ATTENTION_CACHE: stream QKV/attention-out (and optionally MLP)… - #55

Open

JunzoKamahara wants to merge 3 commits into

antirez:mainfrom

JunzoKamahara:attention-int8-cache-stream

Commits on Aug 25, 2026

Add H3_ATTENTION_CACHE: stream QKV/attention-out (and optionally MLP) int8 from a pre-quantized cache
JunzoKamahara
committed
f7166a2

Commits on Sep 3, 2026

Ignore H3_ATTENTION_CACHE when --ssd-streaming is passed explicitly

JunzoKamahara
and
claude
committed
3ec6ac6

Commits on Sep 5, 2026

Add model_kind/model_id to the attention cache, and H3_ATTENTION_CACHE_DIR

JunzoKamahara
and
claude
committed
8b51e19