[BobbyZhouZijian]

Closes #207. Stacked on #206 — review that one first; this branch contains its
two commits.

Motivation

A disjoint engine pauses in_place and deliberately preserves each in-flight
request's KV across a weight update. That is what makes it disjoint, and it is
also why it could not share prefixes: an entry built under one version stays
matchable by a request running under the next, and the resulting rollout would
claim a provenance it does not have.

#207 proposed giving entries a version. Implementing it surfaced the part that
matters: a version on the entry is only meaningful while no request carries KV
across the boundary. in_place guarantees the opposite.

Changes

• Sharing and retracting become one decision. configure_sglang_runtime
  derives the pause mode and the radix cache from the same predicate, so a
  deployment cannot end up sharing entries it cannot scope.
• --disjoint-prefix-sharing (default off) takes the trade for a disjoint
  deployment: publication retracts in-flight requests instead of preserving
  them, and prefix reuse turns on.
• The pause clears the cache. pause_generation_for_update flushes every
  engine after a retracting pause, which is the point every publication path
  already goes through. Nothing the previous weights built survives into the
  next ones — the same guarantee a colocated engine gets for free, since
  SGLang's release path flushes the tree with the KV allocation.

Colocated deployments are unchanged: they retract already, because their KV
has to leave the GPU regardless.

Why the flush has to follow a retracting pause

SGLang refuses to flush while requests are pending, and the refusal is not
visible over HTTP: the 200 response's own body says so — "(When there are
running or waiting requests, the operation will not be performed.)" — while
the scheduler only logs "Cache not flushed because there are pending requests". A flush issued against an engine that kept its in-flight KV would
therefore be a no-op that reads as a success, which is the worst possible
outcome here.

A retracting pause is what makes it real: it releases every in-flight
request's KV allocation, and Reef's own install_colocated_retract_offload
relaxes is_fully_idle so the flush can proceed with the retracted queue in
place. That is the same premise colocate has been relying on for its
release-path flush.

Flushing is also the right instrument only once nothing is mid-flight — it
would otherwise discard entries belonging to requests that legitimately hold
them.

Compatibility and operational impact

• Default behavior unchanged for both layouts. A disjoint deployment opts in
  explicitly.
• Opting in changes what a publication costs: whatever is in flight is
  re-prefilled rather than resumed from its own KV. Worth it when requests
  share long prefixes and publications are not constant; not worth it when
  publications are frequent relative to traffic. That balance is the
  scheduling question in [Bug] Wake the colocated actor once per training job instead of four times #202 and the discussion behind it.
• The config validator now rejects a re-enabling override only for a
  deployment that preserves in-flight KV.
• install_colocated_retract_offload becomes live for an opted-in disjoint
  engine. It is dormant unless generation is paused in retract mode and
  already excludes health checks, but it is no longer colocate-only in
  practice.

Verification

Run on macOS/CPU, where slime, Megatron, and CUDA are not installed:

• pytest tests/slime_backend (excluding the two modules importing slime at
  module scope) → 100 passed, 2 skipped, 26 failed, against 99 passed and 25
  failed on this branch's parent (perf(training): share prefix caches wherever they are cleared at publication #206). The new pass is the disjoint opt-in
  wiring in test_sglang_engine.py; the new failure is
  test_retracting_pause_clears_the_shared_cache_before_a_publication, which
  lives in test_rollout_recovery.py where every manager test needs the real
  slime package and fails the same way here. That test has therefore not
  been executed — it will first run in CI.
• pytest tests/reef_service --ignore=tests/reef_service/test_slime_bridge.py
  → 1411 passed, 12 failed, 4 skipped (all slime, same on main).
• pre-commit run --all-files → all hooks pass.
• python -m mypy → 4 errors from local torch stubs, unchanged from main.

Not verified, and worth confirming on a live disjoint run: that the flush
actually takes effect there. HTTP 200 does not prove it, so the guarantee
rests on the retracted engine passing is_fully_idle — Reef's own premise in
install_colocated_retract_offload, which colocate depends on today, but
which has never been exercised on a disjoint engine. A scheduler log line
("Cache not flushed because there are pending requests") is the thing to
watch for on the first opted-in run.

AI assistance

Claude Code (Opus 5) traced the invalidation and pause paths through Reef,
slime, and SGLang, wrote this change and its tests, and ran the checks above.

Checklist

• The change is focused and contains no unrelated cleanup.
• Tests cover behavior changes, or this pull request does not change behavior.
• Public interface changes include contract tests, or no public interface changes are present.
• Affected user and developer documentation is updated, or no documentation update is required.
• An accepted RFC is linked, or this change does not require an RFC.
• Relevant pre-commit and test checks pass.
• Non-trivial AI assistance is disclosed above, or none was used.

[github-actions]

Merge Oncall: @BobbyZhouZijian

The merge oncall will route reviews and keep this pull request moving toward a clear decision.

[BobbyZhouZijian]

Folded into #206: both halves turned out to be the same invariant — prefix reuse is safe exactly where the cache is cleared at publication, which is only possible once no request holds KV. Keeping them in one pull request rather than stacking a disjoint opt-in on a colocate enable. Branch left in place; #206 carries the combined commit.