fix(qwen3): preserve explicit stop-token causes

[RicardoMin]

Related to #865

Qwen3: preserve explicit stop-token causes in the stepped contract

Why this is a separate PR

This is the scoped follow-up requested by the maintainers during review of
#865. They asked
that the broad stop-contract change be split so that the shared boundary,
frontend bridges, and one model can be reviewed and validated independently.
This PR therefore extracts the Qwen3 migration from that work; Qwen3.5 and the
other model schedulers remain on their existing contract for now.

Summary

The previous vLLM dependency update fixed several frontend compatibility issues,
but the stepped Qwen3 path still collapsed two independent controls into the
single legacy ignore_eos flag. That made an explicit stop_token_ids request
indistinguishable from a model-EOS request and forced the bridge to guess a
synthetic stop token after the scheduler had already discarded the real one.

This PR gives Qwen3 a typed stop contract. It preserves the sampled trigger
token and its logprob, carries the concrete StopCause through the scheduler
and stepped bridge, and keeps EOS handling independent from request-provided
stop IDs.

What was wrong

The old contract exposed only FinishReason::Stop or FinishReason::Length.
When a request stopped, the bridge could not tell whether the model emitted EOS
or an explicit request stop token. It therefore reconstructed a sentinel (EOS
first, otherwise the first configured stop ID). That reconstruction can report
the wrong token, loses the token's logprob, and is incorrect for a speculative
span where the first terminal token is followed by additional accepted tokens.

The old boolean also could not express the valid combination "ignore model EOS,
but still stop on these explicit request token IDs".

Contract change

Event	Legacy behavior	Qwen3 stepped behavior
Model EOS	FinishReason::Stop; trigger may be suppressed or reconstructed	FinishReason::Stop + StopCause::Eos(id); token is retained internally; wire stop_reason is absent
Explicit stop_token_ids match	Often treated as ordinary output or replaced by a guessed sentinel	FinishReason::Stop + StopCause::Token(id); actual ID is reported as wire stop_reason
ignore_eos=true	Could inadvertently disable explicit stops	Disables only model EOS; explicit request stops remain active
Length limit	FinishReason::Length	FinishReason::Length + no stop cause; final sampled token is retained
Speculative span	Trigger/suffix ownership was ambiguous	Commit the prefix through the first trigger and discard only the suffix after it

EOS has precedence when the same ID is both the active EOS token and an explicit
request stop. Completion-token accounting is incremented once, including the
trigger token.

Scope and compatibility

Following the maintainer's scope request on #865, this PR intentionally
migrates Qwen3 only. The shared request/step types
accept an optional typed cause, while the existing legacy event path remains
available for models that have not been audited. The legacy bridge keeps its
synthetic-sentinel fallback only when an old producer supplies no typed cause.
Therefore Qwen3.5 and other model schedulers are not changed in this PR and do
not need to adopt the new resolver contract yet. If the maintainers agree with
the semantics, the remaining model lines can be migrated one at a time with
their own lifecycle tests.

Implementation

• Added StopPolicy, EosPolicy, and StopCause at the frontend engine
  boundary.
• Converted wire EOS and explicit stop fields without collapsing them into one
  boolean for the stepped path.
• Propagated the policy through Qwen3 request state, ledger updates, prefill,
  ordinary decode, and speculative verification.
• Emitted the trigger token before terminal metadata and retained its logprob.
• Mapped StopCause::Token(id) to the vLLM-compatible wire stop_reason.
• Left the legacy bridge fallback and un-migrated model implementations intact.
• Updated two existing test fixtures (K3 and simulator) only to supply the new
  shared default field; no legacy model runtime behavior is changed.

Automated verification

Check	Result
cargo test --release -p pegainfer-frontend --lib	73 passed, 0 failed
cargo test --release -p pegainfer-qwen3 --lib	93 passed, 0 failed
Qwen3 request-stop focused tests	4 passed, 0 failed
Qwen3 speculative-stop focused tests	2 passed, 0 failed
cargo test --release -p pegainfer-sim --tests -- --test-threads=1	6 passed, 0 failed
cargo check --release -p pegainfer-qwen35 --features qwen35	Passed (control build only)
cargo build --release -p pegainfer-server --bin pegainfer	Passed
cargo fmt --all -- --check	Passed
git diff --check	Passed

HTTP A/B verification

The comparison used two already-running OpenAI-compatible endpoints on the
same validation host. The explicit stop set covered the complete vocabulary,
so the first generated token was guaranteed to exercise the request-stop path.
This is a deterministic contract probe, not a generation-quality benchmark.

Results are shown as finish_reason / stop_reason / completion_tokens:

Target	Control (ignore_eos=true)	Explicit stop + EOS ignored	Explicit stop + EOS enabled
Qwen3-0.6B (adapted)	length / null / 8	stop / 12095 / 1	stop / 12095 / 1
Qwen3.5-0.8B (legacy control)	length / null / 8	length / null / 8	length / null / 8

Contract check	Qwen3 adapted	Qwen3.5 legacy control
Baseline control	pass	pass
Explicit stop with EOS ignored	pass	fail
Explicit stop with EOS enabled	pass	fail
Stop-set order invariant	pass	not satisfied (no typed stop)
Trigger logprob present	pass	fail
Streaming typed stop	pass	fail
Mixed controls (3/3)	3/3	3/3
Mixed explicit stops (3/3)	3/3	0/3
Overall new-contract checks	8/8	2/8

The Qwen3.5 rows are an intentional legacy comparison: ordinary generation
still works, but its un-migrated scheduler does not yet satisfy the new typed
explicit-stop contract. They are not a claim that every legacy model fails in
all workloads.

Reproduction

Build and start each server independently. Qwen3.5 requires its feature-gated
Triton build environment; it does not support or require a
--gpu-memory-utilization CLI argument.

# Qwen3 stepped path
cargo run --release -p pegainfer-server -- \
  --model-path "$QWEN3_MODEL" \
  --served-model-name qwen3-adapted \
  --port 18081

# Qwen3.5 legacy control path (set PEGAINFER_TRITON_PYTHON if needed)
cargo run --release -p pegainfer-server --features qwen35 -- \
  --model-path "$QWEN35_MODEL" \
  --served-model-name qwen35-legacy \
  --port 18082

Then run the attached script (Python standard library only):

python3 pr865_qwen3_stop_contract_ab.py \
  --qwen3-url http://127.0.0.1:18081 \
  --qwen3-model qwen3-adapted \
  --qwen35-url http://127.0.0.1:18082 \
  --qwen35-model qwen35-legacy \
  --out stop-contract-ab.json

The script prints a compact comparison table and writes machine-readable JSON.
Use --stop-token-id ID to replace the full-vocabulary deterministic set with
a single known token when reproducing on a different prompt/model pair.

Follow-up

As requested during review of #865, this PR deliberately stops at the Qwen3
migration boundary. After the
maintainers confirm that the independent EOS/request-stop semantics are wanted,
the same policy propagation and resolver audit can be applied to Qwen3.5 and the
other legacy model lines in separate, model-scoped changes.

pr865_qwen3_stop_contract_ab.py

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a4d324ec8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[FeathBow]

Thanks. Please first rebase this PR onto the current main and handle CI error :)

[RicardoMin]

Hi @FeathBow, I have rebased this PR onto the latest main and addressed the CI error you mentioned. Could you please take another look when you have a chance? Thank you!

[FeathBow]

Thanks for separating the Qwen3 stop contract and for preserving the trigger token, logprob, completion count, EOS priority, and legacy fallback.

First, a successful hedged DFlash verify returns before terminal truncation. It selects and copies a winner, advances DFlash hidden context, updates acceptance accounting, and ticks the hedge controller from the untruncated span. The later executor truncation protects the final returned/KV length, but cannot undo those worker-side decisions. Apply each request's stop policy to every A/B candidate before winner comparison and add a hedge + mid-span terminal gate.

Second, nonzero min_tokens is silently accepted even though the new policy starts EOS/explicit-stop classification at token one. Restore the fail-early rejection until sampler-side masking exists; one assertion in the shared validator test is sufficient because both bridges invoke that validator before submission.

Please also revert the frontend architecture text that describes nonexistent ActiveRequest/StepEmitter code. Finally, normalize and share large stop sets: the current Vec::contains is linear in a per-token path and the full vector is deep-copied three times per speculative step. A shared sorted slice with binary search removes the request-size-linear scan and bulk copies; verify both the common zero/one-ID and full-vocabulary shapes before choosing anything more elaborate.

Keep the patch narrow: remove the unconstructed EosPolicy::Token branch, collapse the two identical resolver wrappers, and replace the one-off fake speculative executor/test with the missing production hedge gate. The existing bridge mapping and prefill/decode logprob tests cover distinct local contracts and should remain.

[github-actions]

All commit attributions previously flagged on this pull request are resolved.

[RicardoMin]

Summary

This update narrows the stop-contract change to Qwen3 while retaining the legacy frontend path for models that have not migrated yet.

The previous implementation could classify a terminal token after speculative candidate selection and KV-state preparation. When a stop token appeared in the middle of a verify span, the accepted suffix could therefore remain visible in the candidate state or be copied back before truncation.

This update applies terminal truncation before hedge winner selection and before speculative KV/state commit.

Stop Contract

Condition	finish_reason	Internal cause	Wire stop_reason
Model EOS	stop	Eos(token_id)	null
Explicit stop_token_ids match	stop	Token(token_id)	Actual token ID
Output limit reached	length	None	null

The contract preserves the triggering token, its logprob, and completion-token accounting. ignore_eos=true disables only model EOS termination; explicit request stop tokens remain active.

Review Fixes

• Classify and truncate every speculative candidate at its first terminal token.
• Select the hedge winner using the retained prefix length.
• Use the retained prefix for KV-page copy-back, hidden-state compaction, DFlash context, and counters.
• Keep an idempotent truncation check immediately before the final speculative commit.
• Fix the accepted-draft-token boundary so a terminal token inside the accepted draft prefix is counted correctly.
• Restore early rejection for non-zero min_tokens, since the current scheduler does not yet implement EOS/stop-token masking.
• Remove fake speculative scaffolding and duplicate helper tests.
• Keep legacy bridge behavior for un-migrated model lines.

Only Qwen3 consumes the typed stop policy in this change. The legacy frontend conversion path remains compatible with other models.

Verification

Test	Result
cargo fmt --all -- --check	Passed
git diff --check	Passed
cargo test --release -p pegainfer-frontend --lib -- --nocapture --test-threads=1	74 passed, 0 failed
cargo check --release -p pegainfer-qwen3 --lib	Passed
cargo check --release -p pegainfer-qwen3 --tests	Passed
Strict DSpark hedge gate with Qwen3-4B and dspark_qwen3_4b_block7	Passed
Qwen3 HTTP request with ignore_eos=true, max_tokens=8	finish_reason=length, completion_tokens=8
Qwen3 HTTP request with explicit stop token coverage	finish_reason=stop, numeric stop_reason, trigger token preserved

The change does not modify model mathematics, attention kernels, sampling kernels, or CUDA Graph shapes.

[RicardoMin]

Hi @FeathBow, your requested changes have been implemented in the latest commits.

Please take another look when you get a chance. Thank you for the detailed review!

[xiaguan]

Thank you for the careful revision. The production ordering is much closer to the required contract, and the min_tokens, stop-set lookup, and documentation changes address several parts of the previous review. I still do not think the new hedge test protects the worker-side bug, and there is one invariant leak in the new public type.

1. Please make the hedge regression fail against the previous implementation.

   dflash_hedged_midspan_stop_retains_trigger only asserts the final emitted tokens and Terminal. Before the latest worker-side fix, execute_speculative_verify_impl already called truncate_after_terminal after run_step and before RequestKv::apply_speculative, so those assertions still pass even if the worker ranks A/B candidates, copies KV/hidden state, and records DFlash context from the untruncated result.

   The test also derives its stop ID from a baseline run on the same hedged engine. Because the stop policy does not affect sampling, the old worker reproduces the same candidates and winner; the later executor truncation then produces exactly the expected external prefix. The parent gate requires each child to execute a hedge span, but total_wins > 0 is aggregated across all three children. The stop child is not required to have a B win, and it never demonstrates a case where terminal truncation changes the A/B ordering.

   Please add a real hedge case that observes the request-local raw and retained candidate lengths and the selected winner, with a fixture where truncation changes the winner decision. It should also verify that the retained winner is what feeds the KV/hidden context and committed/controller accounting. As a practical mutation check, this gate must fail when the new pre-selection truncation in try_execute_hedged_verify is reverted while the executor-side safety truncation remains.

2. Please keep the sorted stop-set invariant inside StopPolicy.

   StopPolicy::classify relies on binary_search, while both eos and token_ids are public. Any caller can therefore construct StopPolicy { token_ids: Arc::from([7, 3]), ... } and get a silently incorrect classification, bypassing the normalization promised by the type. The production callers only need new, default, and classify; please make the fields private and add a narrow read-only accessor only if a real caller needs one.

3. Please reduce the low-value tests and refresh the comments to match the final flow.

   normalizes_stop_sets_across_common_sizes repeats the one-ID behavior already covered above it and allocates an entire 151,936-ID set while primarily retesting sort_unstable, dedup, and binary_search. It does not prove that per-step clones share the allocation or that the production wire conversion preserves the contract. A focused dedup/membership test plus the meaningful wire/hedge gates is enough.

   Several comments now overstate or contradict the implementation: the hedge test claims it proves pre-winner selection and commit ordering although it only checks the final stream; the parent gate still says it runs “two” losslessness tests and “one child per lossless suite” after adding a third, non-losslessness child; and execute_speculative_verify_impl says the worker returns the mathematically accepted span even though the worker now applies the stop policy and the executor only rechecks the invariant. Please remove or update these while narrowing the tests.

The main implementation direction looks reasonable, but the current regression can stay green with the original worker-side bug restored, so I cannot approve this head yet. Thank you.

[RicardoMin]

Summary

This update addresses the remaining review feedback for the Qwen3 stop-contract migration.

Review Fixes

• Apply the stop policy to every speculative hedge candidate before winner selection.
• Ensure a terminal suffix cannot affect the selected hedge winner, hidden-state compaction, DFlash context, KV commit, or accounting.
• Strengthen the production hedge gate to require a real raw_winner=B -> retained_winner=A -> selected=A transition.
• Verify that the retained length matches the context append and KV commit length.
• Keep StopPolicy fields private and preserve its normalized sorted-set invariant.
• Remove redundant helper coverage and align comments with the actual execution flow.
• Preserve the legacy contract for models that have not migrated to the typed stop policy.

Verification

Check	Result
Strict DSpark hedge gate (hedged_ladder_passes_the_lossless_gates)	Passed
Mid-span explicit-stop regression (dflash_hedged_midspan_stop_retains_trigger)	Passed
Mutation check (reverting pre-selection truncation in try_execute_hedged_verify)	Failed as expected (retained_winner=B, gate caught bug)
cargo fmt --all -- --check	Passed
git diff --check	Passed

The strict hedge test now fails if terminal truncation is moved after winner selection, even when the final output is later corrected by the legacy safety truncation.
This PR remains scoped to Qwen3. Other model lines and their legacy scheduler contracts are unchanged.

[RicardoMin]

Hi! @xiaguan ,The PR description has been updated with the full verification details. Please take another look when you have a moment. Thank you!

[FeathBow]

Thanks. Two things I would still fix before merge. First, the parent ladder now parses the hedge win count and discards it: total_wins > 0 and total_spans > total_wins are gone, and the stop child's qualifying case is raw_winner=B with selected=A, which is exactly the case where copy-back does not run. So no test proves the copy-back branch executes any more; please restore both assertions for the two losslessness children. Second, the trace that makes the new gate mutation-sensitive is larger than the two facts it needs (raw B win flipped to A with retained < raw, and selected_len == commit_len): retained_winner equals selected by construction and context_len equals selected_len by construction, and the hedged flag on VerifyResult and record_verify_dflash_context exists only to pair log lines. btw please remove outdated docs and descriptions.

[FeathBow]

The previous gate asserted total_wins > 0 (the page copy-back / hidden compaction branch executed at least once) and total_spans > total_wins (the discard branch executed too). This head parses the win count and drops it, replacing both with total_spans > 0. The stop child cannot supply that coverage: its qualifying trace is raw_winner=B with selected=A, which is precisely the case where copy-back does not run. So after this change no test shows the hedge copy-back branch ever executes. I recommend restoring both assertions over the two losslessness children (per child or aggregated as before) and leaving the stop child to its own worker-trace check.

[FeathBow]

I recommend keeping raw A/B lengths (captured before truncation) plus selected_is_b, dropping the retained_* / best recomputation, and dropping the hedged field on VerifyResult and the hedged parameter on record_verify_dflash_context: if the commit log line is emitted for every verify round (not only hedged ones) the parent can pair detail and commit lines per request by order without the flag, and the context line is not needed.

[FeathBow]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 0a25c14f91

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[RicardoMin]

Sorry, I do not have my computer on hand at the moment. I will resolve the CL problem tomorrow.

[FeathBow]

Thank you for addressing the earlier worker-ordering, stop-set ownership, and win/discard coverage feedback. The production path now applies terminal truncation before hedge selection and context recording.

I found four remaining issues. The strict hedge gate still requires an appended= field that is no longer emitted, so it cannot pass when executed. The wire policy also treats secondary model EOS IDs as primary EOS, losing the stop reason required by the pinned vLLM contract. In addition, the HTTP probe accepts several invalid outcomes, and the detail/commit pairing can compare different verify rounds.

Please address these issues and refresh the validation evidence for the resulting head. Handle redundant compatibility and test/comment descriptions.

[RicardoMin]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 8def99771b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[RicardoMin]

Summary

This head resolves all four findings from FeathBow's Sep 8 review and refreshes
the validation evidence end-to-end on a fresh 32 GB instance (Qwen3-4B +
deepseek-ai/dspark_qwen3_4b_block7).

Review Fixes

• Gate fields now match production output. The strict gate no longer
  requires the removed appended= field. It parses the production
  Qwen3 DFlash hedge detail round=/request=/raw_a=/raw_b_lens=/selected=/selected_len=
  line and pairs it with the Qwen3 DFlash context context_len= and
  Qwen3 DFlash commit accepted_len= lines, asserting
  selected_len == context_len == commit_len.
• Same-round pairing. Hedge-detail, context, and commit records are keyed
  by (round, request), so an ordinary verify round before the first hedged
  round can no longer be compared against a different round's commit.
• Primary vs secondary EOS. convert_stop_policy keeps the protocol's
  primary eos_token_id as EosPolicy::Token(primary) instead of
  ModelDefault. Secondary model EOS IDs — which the pinned vLLM lowering
  already merges into stop_token_ids — now classify as
  StopCause::Token(id) and report the actual stop_reason; only the primary
  EOS yields StopCause::Eos with a null wire stop_reason. Covered by
  convert_stop_policy_keeps_eos_and_explicit_stops_independent and
  primary_eos_keeps_secondary_model_eos_as_token_stop.
• Copy-back / discard coverage restored. The parent ladder re-asserts
  total_wins > 0 and total_spans > total_wins over the two losslessness
  children, and the stop child still requires a worker-side
  raw_winner=B -> selected=A -> selected_len < raw_b_max case.
• Stale descriptions cleaned. The nonexistent ActiveRequest/StepEmitter
  architecture text was dropped, and production comments now describe the
  actual pre-selection truncation flow.

Verification

Environment: RTX 4080 SUPER 32 GB, driver 580.142 / CUDA 12.8, Rust
nightly-2026-07-10; target Qwen/Qwen3-4B, drafter
deepseek-ai/dspark_qwen3_4b_block7 (block_size=7, markov_rank=256).

Check	Result
Strict DSpark hedge gate hedged_ladder_passes_the_lossless_gates (PEGAINFER_REQUIRE_HEDGE_GATE=1, PEGAINFER_SPEC_HEDGE=8, positions 0,1,2)	Passed (36.4 s; greedy + heterogeneous losslessness + mid-span stop children)
Mutation check: revert pre-selection truncation in try_execute_hedged_verify (executor-side safety truncation kept)	Failed as expected — hedge/commit mismatch round=28: selected 2 vs commit 1
cargo test --release -p pegainfer-frontend --lib	80 passed, 0 failed
cargo fmt --all -- --check	Passed
git diff --check	Passed

This PR remains scoped to Qwen3. Other model lines and their legacy scheduler
contracts are unchanged.

[xiaguan]

Thanks for the updates! There are still a couple of gaps in the validation:

• In the hedge test, continue skips any result whose last token isn't the stop token, including a result that emitted more tokens after the stop. I appended an extra token to one stopped result locally, and the full gate still passed. Could you check the stop position, finish reason and count before filtering candidates, and validate the no-stop outcomes too?
• The attached HTTP probe can also report a false pass. I ran the unmodified script against a deliberately invalid local HTTP service: a string stop_reason, eight completion tokens with a full-vocabulary stop set, [null] logprobs, and streamed text after the finish event. It still passed all eight checks and exited 0, even with a model-name mismatch. Could you make these checks reject malformed responses and verify the actual trigger ID, its logprob, token count and stream termination? Please commit the corrected probe so the reported HTTP results are reproducible. This checks the probe's assertions; it doesn't mean the real server produced those bad responses.

There is also some duplication we can remove: merge the two StepCollector collection loops, consolidate the overlapping stop-policy cases in stop.rs and wire.rs, and carry the policy in VerifyStepItem instead of maintaining a parallel array. The goal here is fewer wrappers and duplicated responsibilities; there's no need to split files just to meet a line count.

[RicardoMin]

Summary

This update completes the final review round for the Qwen3 typed stop contract.
It closes the hedge-gate filter hole the reviewer reproduced, replaces the
attached HTTP probe with a strict, repository-committed version, and removes
the remaining duplication around request stop policies.

Review fixes

1. The hedge gate no longer filters candidates out.
   dflash_hedged_midspan_stop_retains_trigger used to continue past any
   result whose last token was not the stop ID, so a run that emitted extra
   tokens after a mid-span stop, and every no-stop run, went unvalidated. It
   now validates every candidate: if the stop token appears it must be the
   final emitted token (position, finish_reason, and completion_tokens are
   all asserted); if it never appears, the run must terminate cleanly at the
   length limit. Appending a token after the stop now fails the gate
   (stop N was followed by 1 more token(s)).

2. The HTTP probe is strict and reproducible.
   The previous attached script accepted a string stop_reason, eight
   completion tokens under a full-vocabulary stop set, [null] logprobs,
   streamed text after the finish event, and a model-name mismatch. The new
   scripts/qwen3_stop_contract_probe.py rejects all of them and verifies the
   real trigger ID, its logprob, the token count, and stream termination. A
   built-in --self-check replays those five malformed shapes (plus a valid
   control) against a local mock server, so the reviewer's regression is
   reproducible with no GPU.

3. Duplication removed.
   StopPolicy is now carried in VerifyStepItem instead of a parallel array
   threaded through VerifyPlan and StepCommand; the two StepCollector
   collection loops are merged; and the overlapping stop-policy test cases are
   consolidated so stop.rs owns classification semantics while wire.rs
   only tests the wire-to-policy conversion.

What the probe verifies

Against each OpenAI-compatible server it drives 7 /v1/completions shapes and
one /v1/models probe, then scores 9 checks:

Check	Passes only when
model_present	/v1/models lists the requested model name
baseline_control	ignore_eos, no stop set → length with exactly max_tokens tokens and no stop_reason
explicit_stop_ignore_eos	stop, integer stop_reason from the requested set, one token under a full-vocabulary set
explicit_stop_eos_enabled	stop; stop_reason is a legal integer or absent (model EOS may win)
stop_set_order_invariant	ascending and reversed stop sets yield the same ID and token count
trigger_logprob_preserved	the last emitted token carries a non-null finite logprob
stream_reports_typed_stop	stream ends with [DONE], typed stop_reason, no content after the finish event, valid trigger logprob
mixed_controls_pass_3_of_3	three concurrent ignore-EOS requests all end at the length limit
mixed_explicit_stops_pass_3_of_3	three concurrent explicit-stop requests all end with a typed stop

--self-check spins up a local mock server and asserts that a valid service
passes while five malformed responses (string stop_reason, extra tokens under
a full-vocabulary set, [null] logprobs, post-finish stream content,
model-name mismatch) are each rejected. --require-legacy-gap additionally
asserts the expected adapted-passes/legacy-fails A/B outcome as an exit code.

Running it

Self-check (no GPU, stdlib only):

python3 scripts/qwen3_stop_contract_probe.py --self-check

Start the adapted Qwen3 server:

cargo run --release -p pegainfer-server -- \
  --model-path "$QWEN3_MODEL" \
  --served-model-name qwen3-adapted \
  --port 18081

Start the legacy Qwen3.5 server (its Triton AOT kernels need a Python with
Triton at build time):

export PEGAINFER_TRITON_PYTHON=/path/to/python-with-triton
cargo run --release -p pegainfer-server --features qwen35 -- \
  --model-path "$QWEN35_MODEL" \
  --served-model-name qwen35-legacy \
  --port 18082

Then run the A/B probe:

python3 scripts/qwen3_stop_contract_probe.py \
  --qwen3-url http://127.0.0.1:18081 --qwen3-model qwen3-adapted \
  --qwen35-url http://127.0.0.1:18082 --qwen35-model qwen35-legacy \
  --require-legacy-gap --out stop-contract-ab.json

Validation

• cargo test --release -p pegainfer-frontend --lib: 80 passed
• cargo test --release -p pegainfer-qwen3 --lib: 104 passed
• cargo clippy --release --locked -p pegainfer-frontend -p pegainfer-qwen3 --all-targets -- -D warnings: passed
• Strict DSpark hedge gate (PEGAINFER_REQUIRE_HEDGE_GATE=1, Qwen3-4B +
  dspark_qwen3_4b_block7): passed
• Mutation checks: reverting worker-side truncation, the executor-side safety
  truncation, or appending a token after a stop each make the gate fail
• Probe self-check: 6/6; live A/B (Qwen3-4B vs Qwen3.5-0.8B): adapted 9/9,
  legacy 2/9 with the typed-stop checks failing exactly as expected

[xiaguan]

Thanks for the updates! The hedge test now rejects the injected token after a stop, and the policy-array and collection-loop cleanup looks good.

I rechecked 9b91ad31 and found three cases where the new HTTP probe still reports 9/9 PASS:

• An extra token in the same SSE frame as the finish metadata is skipped by request_stream, so a full-vocabulary stop response with two tokens is counted as one.
• With a single explicit stop ID, a missing logprob on the final stop token inherits the previous token's finite logprob and passes the trigger check.
• A response containing token ID 12095 but reporting stop_reason=17 passes when both IDs are in the stop set. Membership alone doesn't verify the actual trigger.

These were deliberately constructed local HTTP responses, not errors observed from the inference server. Could you request the returned token IDs and check the exact trigger, its corresponding logprob, and the complete token sequence? The parser should also account for or reject content in a terminal frame instead of silently skipping it.

One other point: removing only the executor-side second truncation loop, with worker truncation intact, still passed the real hedge gate locally. That differs from the reported mutation result. Could you clarify what that second pass needs to protect and update the validation note? Both worker paths already normalize the span before recording context; if a guard is needed at commit, an explicit invariant check would make a broken worker result visible instead of silently truncating it again.

[RicardoMin]

Summary

This update strengthens the committed HTTP probe's assertions and consolidates the remaining stop-policy validation loops. The production scope remains Qwen3; the legacy compatibility bridge and Qwen3.5 scheduler are unchanged.

Review fixes

• Complete token-sequence validation. The probe requests return_token_ids and return_tokens_as_token_ids. A shared validator checks that the first applicable stop is the final returned token, the sequence stays within max_tokens, and completion usage matches the returned IDs. stop_reason must identify the actual trigger, with primary EOS taking precedence when enabled.
• Matching trigger logprobs. Logprob arrays must align with the returned IDs, and logprobs.tokens must contain the corresponding token_id:<id> values. A final token without its own logprob cannot inherit an earlier value.
• Complete SSE handling. Content is processed before finish metadata. Inconsistent token/logprob fields, extra choices, output after finish, events after [DONE], and server errors are rejected. The probe requires final usage and reads through [DONE] to HTTP EOF. Legitimate decoder text flushes before finish remain supported; text is never used to estimate token counts.
• Reliable legacy comparison. --require-legacy-gap requires healthy ordinary-generation controls and an observed explicit stop that the legacy path ignored. Unavailable services, wrong models, and incorrect trigger reasons cannot produce a successful comparison.
• Targeted self-checks. Negative cases assert specific diagnostics. The cross-frame missing-logprob case includes a valid preceding logprob, and the terminal-extra-token case includes valid logprobs and consistent usage, preventing unrelated failures from masking the intended assertion.

Worker candidate normalization now uses one traversal with each candidate's own policy. Request-ID and commit-invariant validation also share one traversal, retaining rollback before any KV commit. The existing hedge baseline now checks Length, no stop cause, and the full completion count without adding another GPU request. Related comments and documentation were corrected.

Correction to the previous mutation result

My previous statement that removing only the executor-side second truncation must fail the real hedge gate was incorrect.

Workers already normalize candidates before winner selection, KV/hidden copying, and DFlash context recording. The executor's invariant check exposes an invalid worker result before KV commit; it does not replace worker normalization.

I repeated isolated mutations, preserving the exact diff for each experiment and restoring the original source bytes between runs:

Mutation	Strict real hedge gate
Remove only hedged worker normalization	Fails with an untruncated-span error
Remove only the commit invariant, retaining request-ID validation	Passes
Append a token to a stopped child result	Fails on output after the stop
Restore final source	Passes

The gate retains its real hedge win/discard assertions and same-round worker/context/commit checks.

Validation

Validation used the CUDA 12.8 / SM89 instance with Qwen3-4B, dspark_qwen3_4b_block7, and Qwen3.5-0.8B. The HTTP server was rebuilt from the final Rust sources before testing.

• Rust formatting and locked Cargo metadata: passed.
• Clippy across the CI CPU and default Qwen3 package sets, all targets, -D warnings: passed.
• Frontend library tests: 80 passed.
• Qwen3 library tests: 104 passed.
• Simulated frontend E2E: 18 passed.
• Strict DSpark hedge gate, including its three child gates: passed.
• Probe self-check: 45 expected outcomes passed.
• Disabling first-stop-position, missing-logprob, exact-reason, or logprob-identity assertions each causes self-check failure.
• CLI negative controls: wrong trigger, wrong model, and unreachable legacy service each exit 1 with --require-legacy-gap.
• Real full-vocabulary HTTP A/B: Qwen3 9/9; healthy Qwen3.5 3/9 with the expected explicit-stop gap.
• Real Qwen3 single-stop-ID probe (576): 9/9.

The single-ID test returned [12095, 13, 576], stop_reason=576, and completion count 3. JSON and SSE both returned matching logprobs:
[-0.35472107, -0.20951462, -0.20934486].

Qwen3.5 passed model identity, baseline generation, and mixed ordinary-generation controls. Under the full-vocabulary stop set it continued to length / null / 8, demonstrating the expected legacy behavior.

The constructed malformed responses validate the probe's assertions; they are not errors observed from the inference server.

Running the probe

Self-check, using only the Python standard library:

python3 scripts/qwen3_stop_contract_probe.py --self-check

With the Qwen3 and Qwen3.5 servers running:

python3 scripts/qwen3_stop_contract_probe.py \
  --qwen3-url http://127.0.0.1:18081 \
  --qwen3-model qwen3-adapted \
  --qwen3-eos-token-id 151645 \
  --qwen35-url http://127.0.0.1:18082 \
  --qwen35-model qwen35-legacy \
  --qwen35-eos-token-id 248046 \
  --require-legacy-gap \
  --out stop-contract-ab.json

Live validation requires each server's actual primary EOS ID. For these checkpoints, the pinned vLLM backend resolves primary EOS from the tokenizer; Qwen3.5 uses 248046, despite its model configuration also containing 248044.

[RicardoMin]

Hi @xiaguan ,
I've addressed the previous feedback and updated the PR accordingly. Could you please take another look when you have a moment?
Thanks!