fix(scheduler): back-fill deferred-output placeholders before overwrite (IndexError crash)

[yhl-amd]

Symptom

EngineCore-DP0 dies with:

atom/model_engine/scheduler.py, in postprocess
    seq.output_tokens[-num_placeholder - offset + i] = el
IndexError: list assignment index out of range

The uvicorn front-end survives, so the container stays Up and /v1/models
answers, but requests just queue (nothing decodes). Reproduced twice on a
GLM-5.2 MXFP4 deploy (MTP off, tp8, chunked prefill on, chunk=16384, 1M ctx).

Root cause

The deferred-output write overwrites the last num_placeholder (+offset) slots
of token_ids / output_tokens with the real sampled tokens, assuming the
previous step's tail (if need_placeholder: loop) already appended those
eos placeholder slots.

That holds for a normally chunk-prefilled seq: on the step its prefill
completes it is still partial at step start, gets continue'd, receives its
placeholder in the tail loop, and only writes on the next step.

It breaks for a seq whose prefill completes in a single chunk — e.g. a
prefix-cache-heavy prompt that only forwards a few new tokens (observed:
cached=33584, new=4). Such a seq is never partial at step start, so it is
not continue'd, and reaches the deferred write on its first appearance
with output_tokens still empty. The negative index then:

• raises IndexError on output_tokens → engine core dies; and
• would otherwise silently overwrite the last PROMPT token in token_ids
  (token_ids[-1] points at the prompt when no completion slot exists yet).

Fix

Back-fill the missing eos placeholder slots via append_token (keeping
token_ids / output_tokens / num_tokens in lockstep) immediately before the
overwrite loop. Self-healing (any step arriving short fixes exactly what it
needs, regardless of whether the tail loop ran) and a no-op on the normal
path (slots already reserved → guard never fires).

Scope / risk

• One file, +20 lines, guarded by while len(output_tokens) < num_placeholder + offset
  so the normal decode path is untouched.
• Not GLM-specific; affects any deferred-output deploy with prefix caching /
  single-chunk prefill completion.

Validation status

Root-caused from two captured crashes + code-path analysis (preempt and
abort_request were ruled out as red herrings — the aborted seq is continue'd
before the write). NOT yet run under a repro load. Reviewer note: repro plan
is to resend a long prompt so the second request hits the prefix cache and
completes prefill in one chunk, confirming crash on main and survival here,
and verifying the first decoded token is correct (not the back-filled eos).
Draft pending that validation.

Supersedes #1703 (that PR chased a preempt() placeholder-strip hypothesis which
the second crash log refuted — no preempt occurred; the trigger is
single-chunk prefill completion, not preemption).

🤖 Generated with Claude Code

[github-actions]

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

• ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

• ✅ Run after the PR is approved and Pre Checkin passes
• ✅ Run immediately when an approval review is submitted
• ✅ Can be requested before approval with labels

Label	Tests
ci:full	Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom	Run native ATOM model accuracy tests
ci:vllm	Run ATOM vLLM OOT model accuracy tests
ci:sglang	Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 1707 --add-label <label>