Conversation
(cherry picked from commit 412dd6a638f1d1cc6ed162d40cfc4dfbb89915ea)
Backport the final functional diff of sgl-project/sglang#31466 at c96e68b62 onto deepseek-v4-hcu while preserving HCU DCP, staging, CP, and graph-replay changes. PD runtime validation is pending dedicated prefill/decode resources. Static compilation and test_pd_hidden_state.py (12 tests) pass.
…greation prefill phase
The PD hidden-state hooks added to DecodePreallocQueue.pop_preallocated and DecodeTransferQueue.pop_transferred dereferenced collaborators that are wired up after construction (transfer_queue, kv_manager) and per-request state that only exists once a request enters the PD hidden path (pd_hidden_state). Any caller holding a partially constructed queue -- including main's registered decode-queue cleanup tests -- hit AttributeError on the abort/failure paths. Guard the three sites: getattr for transfer_queue, a class-level kv_manager default on DecodeTransferQueue, and a None-tolerant pd_hidden_state read in _drain_pd_hidden_ready_chunks. Production wiring is unchanged (all three are always set by scheduler init), so this only affects the unwired case. Fixes the 3 test_decode_queue_cleanup failures that predate this rebase.
…ation
SchedulerDisaggregationInitMixin.__init_subclass__ swapped a full copy of
init_disaggregation (SchedulerDisaggregationPrefillMixin, 175 lines) over the
one defined in the Scheduler class body. That copy was forked from an older
main and has since drifted: it lost the PD Decode DP-sync Gloo group, the
rust-server ascend config store, the unified-memory disagg move gate, the
get_disagg()/get_parallel() accessors, and -- the crash reported here --
`self.disagg_prefill_pending_chunk_rids`, which main's send_kv_chunk now
maintains:
File ".../disaggregation/prefill.py", line 2305, in send_kv_chunk
self.disagg_prefill_pending_chunk_rids.discard(req.rid)
AttributeError: 'Scheduler' object has no attribute
'disagg_prefill_pending_chunk_rids'
Whole-method duplication cannot survive rebases: every main-side addition to
init_disaggregation silently disappears on the PD path. The branch's actual
delta is three lines wide -- resolve_disagg_metadata_config() widening the
hidden-state geometry, plus **metadata_buffer_kwargs on the two MetadataBuffers
constructions -- so fold exactly that into the single implementation, guarded
on disaggregation_mode != NULL (a non-PD server must not inspect speculative
workers), and delete the copy along with the installer mixin.
Repro: P HCU_NUM=4 PD_OPEN=1 PD_MODE=prefill MTP_MODE=dspark
DSPARK_MOE_MODE=deepep + D HCU_NUM=4 PD_OPEN=1 MTP_MODE=dspark
DSPARK_MOE_MODE=deepep + router.
The PD Decode single-clock setup in init_disaggregation still reads
self.pp_size / self.attn_tp_size / self.attn_cp_size / self.tp_rank, which no
longer exist on Scheduler after the ParallelState (self.ps) refactor. The block
only runs for disaggregation_mode == DECODE with dp attention, so nothing
upstream exercises it and the stale names never surfaced:
File ".../managers/scheduler.py", line 1315, in init_disaggregation
if self.pp_size != 1:
AttributeError: 'Scheduler' object has no attribute 'pp_size'
Re-express the four reads on self.ps. Verified no bare self.{pp_size,
attn_tp_size, attn_cp_size, tp_rank} remains in scheduler.py; the rest of the
file already uses self.ps for all four.
Inherited verbatim from the rebase base d20a475 -- it was reachable only once
the duplicated init_disaggregation copy stopped shadowing the real one.
Repro: D side HCU_NUM=4 PD_OPEN=1 PD_MODE=decode MTP_MODE=dspark
DSPARK_MOE_MODE=deepep.
add a mutual exclusion warning for the startup parameters --disable-radix-cache and --enable-unified-cache-external-linker.
…se_tmp_lc Rye 20260814 deepseek v4 open rebase tmp lc
…rs, and add a new switch mooncake-enable-page-wise-load to control page‑wise data reading.
…se_tmp_lc When writing data to mooncake, set dfs_replica_num to 1 to avoid erro…
…rebase_metrics Keep direct L4 token metrics in SGLang only
…FT, including queue wait, token pre‑processing, Mooncake‑to‑L1 data transfer, inference compute, etc., to pinpoint performance bottlenecks. Page‑wise load batching removal – modifies MooncakeDirectLinker._load_page_wise to fetch all required pages in a single batch instead of chunking them by page_wise_load_batch_size, reducing I/O overhead and improving throughput. Graceful error handling for L3 load failures – when the direct‑path call to batch_get_into_multi_buffer_ranges fails with error 707, the request is marked as failed (or raised) but does not crash the whole SGLang service; the service continues operating.
… errors. Modify default page_wise_load_threshold value.
…vironment variable to distinguish DFS mode.
feat: Calculate the hit rates of L1, L3 and L4
github-actions
Bot
added
documentation
AI Review审查尚未完整完成，无法给出整体结论；当前没有可发布的已确认问题，不代表代码没有问题。
|
AI CI 失败分析质量门禁未通过。
建议处理由有权限的维护者在 CI 原始报告中定位并处理门禁发现的问题，再验证对应检查。若涉及凭据，应撤销或轮换受影响凭据，而不只是删除文本。
|
AI CI 失败分析工作流：PR Test (HCU)
CI 汇总结论已定位直接异常的任务：Stage A HCU smoke、Stage B HCU required (stage-b-required-2)。
仍待定位或尚未处理：Stage B HCU required (stage-b-required-3)、Stage B HCU required (stage-b-required-0)、Stage B HCU required (stage-b-required-1)、PR Test (HCU) finish。
Stage A HCU smoke失败原因run_suite 收集测试阶段直接抛出 ValueError：No CI registry found in /sglang-checkout/test/registered/unit/disaggregation/test_pd_hidden_state.py（ci_register.py:428），随后以退出码1失败。pip 依赖冲突告警（L296-L313）同批出现，但仅为安装期提示，未证明导致本次失败。 问题分类：代码 与本次改动的关系相关：报错文件正是本 PR diff 中新增的 test/registered/unit/disaggregation/test_pd_hidden_state.py，新增测试缺少 ci_register 要求的注册信息导致收集失败，日志与 diff 形成可核对的触发路径。 建议处理参照仓库中已注册同类测试的写法，为该新增测试文件补充 CI 注册声明（具体注册机制需对照 ci_register.py 确认），或按注册规则调整文件位置；方案需人工核验后实施。 验证方式：补充注册后重跑 Stage A HCU smoke，确认 collect_tests 不再抛出该 ValueError、用例被正常收集且任务退出码为0。
证据批次：1
查看逐批证据与处理进度（1/1）证据批次 1直接异常证据
Stage B HCU required (stage-b-required-3)失败原因现有已处理证据未定位直接异常。启动器终止信号、退出码和失败统计只能说明失败，不能单独确定根因。 建议：优先处理尚未完成的证据批次；若处理完成后仍无直接异常，再获取失败进程的原始异常输出。不能把其它批次未出现在某一片段中称为日志下载截断。
查看逐批证据与处理进度（0/1）证据批次 1分析未完成：模型未返回完整、有效的审查格式。可续审；不代表没有异常。 Stage B HCU required (stage-b-required-2)失败原因直接报错：git fetch 传输层中断，curl 56 OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 110（L62），fetch-pack 意外断开、index-pack 输出无效（L64-L66），checkout 以 exit code 128 失败；随后清理脚本因检出目录未生成而报 No such file或目录并 exit 127（L86-L87），属连带失败，非独立根因。 问题分类：基础设施 与本次改动的关系无关：失败发生在按 SHA（与 PR SHA 一致）检出代码阶段，错误类型为网络传输超时/断连；PR 代码尚未编译或执行，不存在日志与代码可核对的触发路径，与 59 个改动文件无关联证据。 建议处理最小处理：重试本 checkout 任务确认是否偶发网络抖动；若复现，排查 runner 到远端仓库的网络链路/代理稳定性，并可为 fetch 增加重试或减小传输量。以上方向未执行，需人工核验。 验证方式：重跑 checkout 步骤，观察 git fetch 是否完整完成并正常进入后续阶段；若再次出现 curl 56 SSL_read 或 errno 110，需在网络层（DNS、代理、TLS 链路）进一步定位。
证据批次：1
查看逐批证据与处理进度（1/1）证据批次 1直接异常证据
Stage B HCU required (stage-b-required-0)失败原因现有已处理证据未定位直接异常。启动器终止信号、退出码和失败统计只能说明失败，不能单独确定根因。 建议：优先处理尚未完成的证据批次；若处理完成后仍无直接异常，再获取失败进程的原始异常输出。不能把其它批次未出现在某一片段中称为日志下载截断。
查看逐批证据与处理进度（0/1）证据批次 1分析未完成：模型未返回完整、有效的审查格式。可续审；不代表没有异常。 Stage B HCU required (stage-b-required-1)失败原因现有已处理证据未定位直接异常。启动器终止信号、退出码和失败统计只能说明失败，不能单独确定根因。 建议：优先处理尚未完成的证据批次；若处理完成后仍无直接异常，再获取失败进程的原始异常输出。不能把其它批次未出现在某一片段中称为日志下载截断。
查看逐批证据与处理进度（0/1）证据批次 1分析未完成：模型未返回完整、有效的审查格式。可续审；不代表没有异常。 PR Test (HCU) finish失败原因现有已处理证据未定位直接异常。启动器终止信号、退出码和失败统计只能说明失败，不能单独确定根因。 建议：优先处理尚未完成的证据批次；若处理完成后仍无直接异常，再获取失败进程的原始异常输出。不能把其它批次未出现在某一片段中称为日志下载截断。
查看逐批证据与处理进度（1/1）证据批次 1本批未发现可核对的直接异常，不单独推断根因
|
3090303 to
80c98e7
Compare
AI CI 失败分析工作流：PR Test (HCU) 总结
与本次改动的关系无法确定：不同失败原因与本次改动的关联判断不一致，需要人工核验。 建议处理
|
7 participants
Motivation
The Mooncake direct external-linker path currently lacks several capabilities
needed for production deployment and performance analysis:
batch write.
preprocessing, Mooncake lookup, direct loading, and inference bottlenecks.
metrics or rank-aware Mooncake client metrics endpoints.
arguments are not sufficiently visible during startup.
contained at the request/load level instead of terminating the service.
This PR improves the Mooncake direct-linker I/O path and adds the observability
needed to diagnose its behavior in production.
Modifications
Mooncake direct I/O
--mooncake-enable-page-wise-loadto enable the page-wise path.--mooncake-page-wise-load-thresholdto select the minimum number ofMooncake object keys required to switch from layer-wise to page-wise loading.
single Mooncake range-get operation.
SGLANG_MOONCAKE_LOCAL_BUFFER_SIZEto configure the Mooncake client localbuffer size.
Reliability
terminating the SGLang service.
Request timing observability
process boundaries.
Direct-storage metrics
dfsorlocal_disksource reported byMooncake.
source-specific token metrics in SGLang.
port.
MOONCAKE_CLIENT_METRICS_PORT_BASE, defaulting to9301.mooncake_dfsormooncake_local_diskin cached-token details.DeepSeek-V4 and startup diagnostics
--disable-radix-cacheis used together with--enable-unified-cache-external-linker.Compatibility
existing session-start API; their storage source remains unattributed rather
than being guessed.
Accuracy Tests
N/A. This PR does not modify model forward computation, sampling, or numerical
kernels.
The following regression tests should be run before merge:
Checklist
Code Quality
pre-commit run --all-filesand ensure all checks pass.git diff --check origin/main...HEAD.Tests