glm52: session-affinity / cache-aware decode routing for wide EP fleets

[xiaguan]

Spun out of #799 (its item 4, traffic-side mitigation) before #804 closes it.

On a wide EP decode fleet, a conversation's next turn lands on the rank already holding its KV ~1/N of the time (N = fleet width), so nearly every turn full-history-refetches over the pegaflow P2P plane. Measured on the 4-node EP16 fleet (multi-turn 1024+256/turn × 5 turns): per-turn TTFT p50 grows roughly linearly with history while EP4 stays flat; at c64 this compounds with the single-P queue into multi-second late-turn TTFT (see docs/models/glm52/pd-native-mtp-handoff.md, EP16 execution-log entries).

The engine-side blocking half was fixed (#801/#804: restores park-and-poll, whole ITL distribution flat). What remains is traffic shaping:

• Router-side session affinity: route a conversation's turns to the same decode endpoint (the vLLM router's session/prefix-aware policies, or hash-on-conversation), turning most restores into rank-local GPU prefix hits.
• Complementary code-side lever: restore pipelining (deferred in glm52: unify admission restore/save into one pollable OffloadTask abstraction #802's addendum) — prefetch the next queued request's restore while the front decodes, hiding the refetch that affinity misses.

Correctness does not depend on placement (pegaflow P2P fetches from whichever node owns the blocks; the EP16 runs completed 640/640 with zero rejects at c64) — this is purely a latency/TTFT optimization.

🤖 Generated with Claude Code

[github-actions]

This issue has been inactive for 14 days. It will be closed after another 30 days unless there is new activity.