Skip to content

[Bugfix][HCU] Restore DeepSeek V4 compressor writes for all cache layouts - #317

Open
lungy666 wants to merge 3 commits into
HYGON-AI:mainfrom
lungy666:fix/hcu-dsv4-compressor-cache-write
Open

lungy666 wants to merge 3 commits into
HYGON-AI:mainfrom
lungy666:fix/hcu-dsv4-compressor-cache-write

Conversation

@lungy666

@lungy666 lungy666 commented Sep 5, 2026

Copy link
Copy Markdown

Motivation

CompressorBackendMixin.forward_unified() selects an indexer cache or a unified KV cache, but its compression/store call is nested inside the ordinary compressed-cache else branch. Both earlier branches therefore return without writing the compressed data. The ordinary branch also contains duplicate, unreachable indexer/unified selection.

This is a correctness fix for skipped cache writes, not a quantization or performance change. The bug is present in the current HYGON main baseline (0457572a). It was found while investigating long-history retrieval failures in a DeepSeek-V4-Flash-0731 W4A8 deployment.

Modifications

  • Select the cache layout once, then execute the shared compression/store call for every non-idle path.
  • Preserve indexer precedence, output locations, page sizes, FP4/BF16 flags, HiSparse location translation, and the following online C128 prefix-state update.
  • Add registered dispatch regressions and explicitly include them in required HCU Stage A.

The functional fix and tests are isolated in commit 52c2fa05. This branch also carries two independent CI prerequisite commits already submitted in #314/#316: restore the required sampling registration, install the shared pinned sgl-eval, update the FP8 reference for the selected format, and repair the serving-chat configuration mock. Without these prerequisites, the current base fails required Stage B for unrelated reasons. No performance/operator changes from #314/#316 are included. No reasoning-profile or API compatibility changes are included.

Accuracy Tests

Run against the selected checkout with its python directory on PYTHONPATH:

python test/registered/hcu/kernels/test_dsv4_compressor_dispatch_hcu.py -v
  • Before the compressor fix: 16 failing subcases, including indexer/unified writes called zero times and prefix-state updates occurring without a preceding cache write.
  • After the fix: all 6 test methods pass, covering all three layouts; C4/C128 where applicable; extend/decode/target-verify/draft-extend dispatch; indexer precedence; FP4/BF16 flags; HiSparse translation; idle; and cache-before-prefix ordering.
  • These are mocked dispatch tests with CPU tensors in an HCU environment, not numerical kernel equivalence tests.
  • Earlier local deployment validation with this same runtime fix recorded successful 4K/32K/128K retrieval on four BW1000 GPUs and recovery of the original long-history failures on eight GPUs. Those runs also used local deployment settings and optimizations; no new full-model accuracy or near-256K run was performed for this PR.
  • CI prerequisites: FP8 numerical test passed all four shapes on gfx936; serving-chat tests passed (17 methods, one existing skip). The pinned evaluation CLI installed successfully in a disposable deployment-image container and both CLI/MMLU help commands succeeded. Installer control flow was checked for regular, skipped-dependency, wheel, and failure paths. This does not constitute a full MMLU accuracy run.

Speed Tests and Profiling

No new throughput claim. Skipping a required write is not a valid performance baseline. Existing kernels, quantization and GEMM backends are unchanged.

Checklist

  • Added registered regression tests and required workflow selection.
  • Checked formatting, imports, lint, spelling, shell syntax and whitespace.
  • Kept the functional fix separate from shared CI prerequisites.
  • Full remote HCU CI result (pending).

CI States

Latest PR Test (Base): ❌ Missing run-ci label -- add it to run CI tests.
Latest PR Test (Extra): ❌ Blocked -- run-ci is required first.

@github-actions github-actions Bot added the quant label Sep 5, 2026
Restore the HCU Stage B registration dropped during the upstream sync. The required workflow still includes this file, so its missing registration aborted all four partitions before test execution. Verified all 46 required files resolve across the four partitions with run_suite.py --list.
Install the shared pinned sgl-eval CLI in regular and image/wheel modes. Derive FP8 reference scales and one-ULP rounding bounds from the output format. Add the config_value interface to the serving-chat mock. Validated FP8 numeric cases on gfx936, 17 chat tests (one existing skip), installer branch/failure behavior, and actual pinned CLI installation in a disposable container.
Move the shared compression/store call outside cache selection so indexer and unified KV paths cannot skip it. Remove unreachable duplicate selection, preserving idle, BF16/FP4, HiSparse translation and online prefix-state behavior. Add six regression tests with prefill/decode/speculative coverage and run them in required HCU Stage A. Tests reproduce 16 failing subcases before the fix and pass after it.

@aivictor0901-tech aivictor0901-tech left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

该问题已经在下面commit中进行了修复,非常感谢您的提交
4c367db

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants