Repository navigation
Preserve chat content and recognize packed Qwen projections - #2
Merged
data-angel merged 3 commits intoOct 13, 2026
Merged
Conversation
Keep the full blocking assistant answer when the heuristic refusal classifier matches, consistent with streaming. Retain refusal annotations, tool-call gating, and finish reasons. Add production assembly/serialization and streaming regressions; update the compatibility expectation. Reported in Atlas-Inf#172.
…pace probe has_linear() recognised only .weight (dense/NVFP4) and .trellis (EXL3), so a compressed-tensors NVFP4 checkpoint - whose projections are stored as .weight_packed - probed 0 routed-expert layers and the build was refused with "qwen4_exp: 0 of 48 layers have routed experts", despite the format being detected and all tensors loading. Attention probes passed because those weights are FP8 with a plain .weight; only the NVFP4 experts are packed. Verified on 2x DGX Spark (GB10, sm_121) at TP2 with a compressed-tensors NVFP4 Flash-Next checkpoint: probe reports 48/48 and the model serves. ModelOpt and EXL3 paths are unaffected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Thanks for the contribution. Before we can merge, please sign our Contributor License Agreement by replying to this comment with:
I have read the CLA Document and I hereby sign the CLA 0 out of 2 committers have signed the CLA. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preserve full blocking chat content when the refusal-prefix classifier matches, alongside the existing refusal metadata. Streaming already preserves this text; this fixes the response loss reported in Atlas-Inf#172 without changing the classifier.
Include the packed-projection namespace fix from #1, preserving contributor attribution, and add real WeightStore regressions for packed, dense, EXL3, missing-expert and quantization-sidecar cases.
Validation: parent reran 7 namespace tests, 32 refusal-related tests and 43 streaming tests; formatting and diff checks passed. The packed tests reproduce 4 failures without the fix. Native GB10 image build passed, including 32 refusal and 43 streaming regressions plus the existing model-config, lifecycle, admission, cancellation and tool-parser gates. Changes are based directly on the published RC16 engine and contain no unrelated optimization work.
CI limitations inherited from the RC16 base: file-cap and typo findings are in unchanged files; the workspace-test, coverage and benchmark jobs fail against incomplete cuBLASLt stubs (
cublasLtMatrixLayoutSetAttribute,cublasLtMatmulAlgoCheck) in unchanged code. The fork also lacks thecla-signaturesbranch expected by its inherited CLA workflow. These are not passing checks. Targeted parent tests and the native GB10 release-image gates pass.Live qualification: the published SparkQwen image
b151fbe840a1ran on a TP2 GB10 pair at utilization 0.70. A harmless two-sentence classifier match retained complete content and refusal metadata in both blocking and streaming Chat Completions; the LLooM gateway also retained complete content. Both ranks used the pinned engine commit and image digest, with zero container restarts.