Skip to content

docs(scheduler): spike recompute preemption parity - #19

Open
localai-org-maint-bot wants to merge 1 commit into
mudler:mainfrom
localai-org-maint-bot:codex/eng-preempt-recompute-spike
Open

docs(scheduler): spike recompute preemption parity#19
localai-org-maint-bot wants to merge 1 commit into
mudler:mainfrom
localai-org-maint-bot:codex/eng-preempt-recompute-spike

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Contributor

Outcome

Moves ENG-PREEMPT-RECOMPUTE from ANCHOR-BACKFILL to SPIKE with an accepted, CPU-only parity contract for the already-shipped bounded scheduler path. No executable file or runtime support claim changes.

The pinned-source audit confirms that vllm.cpp already implements KV-pressure victim selection, KV release, computed-progress reset, front-of-queue retry, event/reset-id reporting, and MRV2 resumed-as-new output. It also finds three concrete closure gaps: preemption does not clear stale spec_token_ids, prefix-cache on/off recomputation and sampled-output preservation need focused parity cases, and encoder-cache/in-flight cleanup waits on the separately inventoried encoder scheduler surface.

Design selected

I evaluated three scopes:

  1. close only the existing FCFS subset, which would hide cleanup shared by priority preemption;
  2. split FCFS and priority into separate lifecycle rows, which duplicates one state transition and obscures cross-policy gaps;
  3. retain one upstream-semantic row and decompose closure into CPU W1/W2, dependency-gated encoder W3, and final every-axis W4.

This PR chooses option 3. The full port map, upstream tests, gates, dependencies, risks, and W1-W4 breakdown are in .agents/specs/preemption.md.

Verification

All CPU/record gates pass on be9b35f2:

  • python3 scripts/check-agent-record.py (ENGINE=131, MODEL=327, QUANT=82, KERNEL=45, BACKEND=68)
  • python3 tests/scripts/test_agent_record.py (13 mutation tests)
  • python3 scripts/check-doc-checkpoint.py --commit HEAD plus 5 mutation tests
  • README structure plus 18 mutation tests
  • model checklist plus 10 mutation tests
  • fusion consistency plus 18 mutation tests
  • device leakage plus 24 mutation tests
  • environment documentation plus 7 mutation tests
  • git diff-tree --check HEAD

No C/C++, test, CMake, model, fixture, kernel, README, GPU, model download, external execution host, or benchmark changed. This host has no CMake; no build is claimed or required for this records-only W0 checkpoint. Benchmark disposition is NOT APPLICABLE with benchmark_binding=false.

Slack

The required selection notification was attempted through the bundled secret-safe sender. No channel is configured; #general returned channel_not_found. No credential was read or exposed.

FOLLOWING_AGENTS_PROTOCOL

Inventory the pinned-vLLM recompute-preemption chain, reconcile the existing bounded CPU implementation, and record the missing spec-token, prefix-cache, and encoder cleanup gates without advancing runtime support.

FOLLOWING_AGENTS_PROTOCOL

Assisted-by: Codex:gpt-5 [Codex]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant