feat(prefetch): async expert prefetcher v4 -- 94.3% cache hit rate, 2.8x speedup, zero RAM overhead - #362

Merged

JustVugg merged 11 commits into

JustVugg:devfrom

EgonRuiter:feature/async-expert-prefetcher-v4

Jul 19, 2026

Commits on Jul 15, 2026

feat(prefetch): implement prefetcher v2.1 with lookahead-2, hot pinning, adaptive cache, RMSNorm scaling, and routing EMA
EgonRuiter
committed
feat(io): implement Consolidated Expert I/O to reduce expert disk reads by 3x and accelerate prefetching
EgonRuiter
committed
feat(prefetch): implement stale request pruning and queue de-duplication to break 90% hit rate barrier
EgonRuiter
committed
feat(prefetch): add opt-in lookahead-3 prefetching option for PILOT=3
EgonRuiter
committed
feat(pinning): implement dynamic asymmetric expert pinning and layer 0 EMA update to reach 90.9% hit rate and 3.25 tok/s
EgonRuiter
committed
feat(prefetch): implement persistent hot pinning and pre-warmup wait loop to break 94% hit rate barrier
EgonRuiter
committed