docs(scheduler): spike recompute preemption parity - #19
Open
localai-org-maint-bot wants to merge 1 commit into
Open
docs(scheduler): spike recompute preemption parity#19localai-org-maint-bot wants to merge 1 commit into
localai-org-maint-bot wants to merge 1 commit into
Conversation
Inventory the pinned-vLLM recompute-preemption chain, reconcile the existing bounded CPU implementation, and record the missing spec-token, prefix-cache, and encoder cleanup gates without advancing runtime support. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Outcome
Moves
ENG-PREEMPT-RECOMPUTEfromANCHOR-BACKFILLtoSPIKEwith an accepted, CPU-only parity contract for the already-shipped bounded scheduler path. No executable file or runtime support claim changes.The pinned-source audit confirms that vllm.cpp already implements KV-pressure victim selection, KV release, computed-progress reset, front-of-queue retry, event/reset-id reporting, and MRV2 resumed-as-new output. It also finds three concrete closure gaps: preemption does not clear stale
spec_token_ids, prefix-cache on/off recomputation and sampled-output preservation need focused parity cases, and encoder-cache/in-flight cleanup waits on the separately inventoried encoder scheduler surface.Design selected
I evaluated three scopes:
This PR chooses option 3. The full port map, upstream tests, gates, dependencies, risks, and W1-W4 breakdown are in
.agents/specs/preemption.md.Verification
All CPU/record gates pass on
be9b35f2:python3 scripts/check-agent-record.py(ENGINE=131, MODEL=327, QUANT=82, KERNEL=45, BACKEND=68)python3 tests/scripts/test_agent_record.py(13 mutation tests)python3 scripts/check-doc-checkpoint.py --commit HEADplus 5 mutation testsgit diff-tree --check HEADNo C/C++, test, CMake, model, fixture, kernel, README, GPU, model download, external execution host, or benchmark changed. This host has no CMake; no build is claimed or required for this records-only W0 checkpoint. Benchmark disposition is
NOT APPLICABLEwithbenchmark_binding=false.Slack
The required selection notification was attempted through the bundled secret-safe sender. No channel is configured;
#generalreturnedchannel_not_found. No credential was read or exposed.FOLLOWING_AGENTS_PROTOCOL