[jvmenen]

Why

The SSD session cache reports one number per restore, ssd_restore_s. That cannot tell whether a slow restore waits on the disk, on copying into arrays or on the GPU. It also cannot tell, on hybrid models where recurrent layers resume only at a stored checkpoint, how much of a matched prompt is computed again. And it does not show how much the cache writes compared with its hourly write budget. Those are the numbers that decide what to improve next.

How it works, in plain terms

• Restore time, split. While a restore runs, the codec adds its time to three counters: reading blobs, copying them into arrays, and the GPU work that joins them. The remainder (manifest, payload, refused rows) is reported separately, so the parts add up to restore_s. Outside a restore nothing is timed.
• Where restores land. For every restore from RAM or SSD, the request records how many prompt tokens matched, at which checkpoint the state resumed, the gap between the two, and whether the entry came from the same conversation or another one with the same prompt head. This covers the regular lanes, the Gemma 4 assistant backend and the batched MTP lane; on non-hybrid models the restore point is the matched length and the gap is 0.
• Write volume. The writer counts the bytes it writes and the bytes it skips because the blob is already on disk, split into recurrent checkpoints and the rest, over a rolling hour.

What changes

• Request log: ssd_restore_read_s, ssd_restore_copy_s, ssd_restore_eval_s, ssd_restore_other_s, ssd_restore_bytes, and prefix_restore_matched_tokens, prefix_restore_point_tokens, prefix_restore_checkpoint_gap_tokens, prefix_restore_partial, prefix_restore_origin.
• The restore log line gains read_s, copy_s, eval_s and bytes.
• /health and /admin/cache/ssd:
  • restore totals and the last split;
  • write_volume, with written and deduplicated bytes per hour and since start, and the share of the write budget used;
  • session_bank.restore_landings counters per source.
• New MLX-free module mtplx/cache_bank/telemetry.py; small changes in the codec, cold tier, session bank, generation, the Gemma 4 assistant path and the server. No restore or write decision changes.

Cost

On an M5 Pro: 3.6 ms on a restore of 20,000 blobs (one second or more), and 1.3 ms per written entry on the writer thread.

Tests

tests/test_cold_tier_telemetry.py and tests/test_restore_landings_other_lanes.py (23 tests, temporary directories only) check that:

• the split adds up to restore_s and its bytes match the blobs on disk;
• a partial restore counts only what it reads;
• the rolling hour drops old buckets;
• landings are counted per source and origin, also on the Gemma 4 and batched MTP paths.

Full suite: 11,258 passed, 0 failed.