Repository navigation
Add a MUSA backend, the vllm_musa repository pack and a vllm-musa skill - #518
yeahdongcn wants to merge 5 commits into
Conversation
Skill evaluationPLANNED: not executed; there are no task outcomes yet.
Author verification steps (nonpaid; not yet run)
python tools/build_skill_index.py --source-revision dd76c988f0faa1ba54c64432fcd2df50587b72e0 --checkThe declared source pin matches this head's canonical library bytes. The command has not run: retain its exit code and full drift/success output. It checks the generated projection, not claim truth or skill efficacy. Evidence and logs · Retention: 30 days. Plan bindings, budget, frozen exam and advisory detailsAdvisory: obtain missing evidenceRetained evidence availability: planned / unexecuted. No approval or merge authorization is issued.
Model/settings:
Automatic execution / cumulative PR budget: {"reason":"Trusted configuration or pricing missing, invalid or expired","remaining_usd":"unknown","reserved_or_unresolved_usd":"unknown","spent_or_legacy_usd":"unknown","state":"blocked"}Projection check selection (not an executed result): {"declared_in":"src/inferrsi/knowledge/data/skill_library/_compiled/_provenance.json","source_revision":"dd76c988f0faa1ba54c64432fcd2df50587b72e0","status":"ready"}Provider/settings: Uncovered / unsupported paths (49)
|
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
InferRSI #518 — reviewed the full diff from base 8dcf5adee04abef97435edb54648a1f01ea2c33e to head 45b5ed8c66a42a3b1ef243cca75b7bd924399cfd. The review record lists no formal approval on this head and no outstanding CHANGES_REQUESTED. The blockers below are from this review.
- The MUSA reservation check treats an unknown device state as idle. In
src/inferrsi/infra/platforms/musa.py(BUSY_CODE,qualify_devices), a missing process-query API or a failed MTML initialization returns[]with exit code 0. The caller also treats empty stdout as[]. Once discovery succeeds, any of these paths can qualify a device without checking whether it already has a compute process. Return a distinct unavailable/error result and fail qualification unless the query succeeds for every selected device. Add a fixture that executesBUSY_CODEwith the API missing or initialization failing; the existing mocked subprocess tests only establish how the caller handles a prebuilt busy or idle list. - UUID selections pass discovery but fail the busy check. In the same module,
qualify_devicesmatches visibility entries against index or UUID, then forwards the original entries toBUSY_CODE, which callsint(index). A matching UUID therefore fails before the process query. Pass the resolved physical indices to the subprocess (or explicitly reject UUID input and document that restriction), and cover that path in a regression test. - The manifest overwrites historical architecture pins.
docs/legacy-knowledge-manifest.jsonreplaces the K02 result hash and both sides of the historical C2 byte-for-byte relocation with the new MUSA registry hash. This makes the historical entries describe bytes introduced in this PR. Keep the K02 and C2 pins intact, record the current hash and registry addition as a separate revision, and updatetests/test_knowledge_compatibility_shells.pyto check that history separately from the current leaf.
Static diff review only: I did not run tests, verify CI, or exercise MUSA hardware. The pack and knowledge records explicitly remain unverified; lack of a hardware result is not itself a blocker for those scoped claims. I'll re-read after the author pushes.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model gpt-6.2-sol).
david6666666
left a comment
There was a problem hiding this comment.
One execution-identity gap beyond the process-query checks: discovery can verify a different physical card than the trial uses.
|
|
||
| device = parsed_tool_facts( | ||
| command([python or sys.executable, "-c", PROBE_CODE]), | ||
| device_ids=device_ids, |
There was a problem hiding this comment.
With MUSA_VISIBLE_DEVICES=1 and --device-ids 0, the unmasked MTML probe can mark physical device 0 as verified while the trial inherits the visibility mask and logical device 0 is physical device 1. Please bind or validate the physical selection against the child process's visibility, or report the execution device as unverified when the caller controls it. A CLI-to-trial regression with a mismatched mask would cover the distinction.
45b5ed8 to
c784e08
Compare
|
Rebased onto current 1. Unknown device state is no longer treated as idle
Covered by two new tests:
2. UUID selections are rejected instead of failing inside the probe
The busy query is also built from the resolved ids rather than the caller's raw entries, so no path forwards a UUID into 3. The historical pins are preserved
4. Execution identity (@david6666666)The root cause is that
Remaining gap, stated plainly: the MUSA Evidence
Untested on hardware: the new fail-closed probe paths and the UUID rejection are exercised with stubs and executed probe fixtures only. The earlier MUSA run ( |
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
首次独立检视:9c7b7b05042f54ca71d13b2897a7505d818d3429 → c784e0810789edde7b435de6c50035933b6c597a。
批准前必做:MUSA 的实际设备映射不能在 Dashboard 比较身份中丢失。 _environment_signature 新允许 MUSA 生成可比较签名,但使用的是配置声明的物理设备 UUID,没有核对运行进程可见设备和实际逻辑映射。探测报告的物理选择有效不等于进程逻辑设备映射有效,运行路径也不会应用 qualify_devices() 的结果。因此,物理声明相同但可见设备映射不同的两条记录可能得到相同签名并进入收益聚合。
建议只在有证据把逻辑设备绑定到实际 UUID 时才返回 MUSA 比较签名,否则保留未验证状态。补一个声明设备相同、可见映射不同的回归,确认不会被聚到同一可比较组。这不要求为本次代码检视额外补硬件性能跑分。非阻塞:文档可一并澄清 qualify 当前支持显式 logical index。
已静态核对注册、设备探测、运行环境传递、签名和聚合路径;未运行仓库代码、测试或 CI。
|
Head is now What changedDiscovery refuses a remapping mask (
An identity mask ( The dashboard signature now requires that binding ( Regressions added
On your non-blocking doc point
Evidence
Untested on hardware, as before: the new mismatch path is exercised with stub probes ( |
david6666666
left a comment
There was a problem hiding this comment.
The explicit mismatch is fixed; the no-mask case still needs a comparable-device binding.
| if mask is not None and not isinstance(mask, str): | ||
| return None | ||
| declared = [str(item) for item in devices] | ||
| if mask and [item.strip() for item in mask.split(",") if item.strip()] != declared: |
There was a problem hiding this comment.
An unset or empty mask passes this check without binding the child to a card. With --device-ids 1 and no mask, a caller workload using logical musa:0 can execute on physical card 0 while this signature records physical card 1's UUID. Please leave the result incomparable unless the mask matches or a separate receipt proves the execution device. The existing test removes visible_devices entirely; add the normal snapshot shape with MUSA_VISIBLE_DEVICES: null and a nonzero requested card.
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
应本轮请求增量复核:c784e0810789edde7b435de6c50035933b6c597a → e2be20109f8185193a4a59ac2831015ab4a8a714,上轮意见。
已修复:物理声明为卡 0、显式 mask 却指向卡 1 时,现在探测和 Dashboard 都会拒绝。但 Dashboard 签名 仍只检查非空 mask。物理声明同为卡 1 时,有 mask 1 和无 mask 仍可能得到相同签名,但同一逻辑设备 musa:0 不一定对应同一卡。缺少环境键也会被接受。请只在能确认实际逻辑→物理绑定时给出可比签名,否则保留未验证;回归应覆盖非零卡的有/无 mask 和缺键。
已静态核对增量、探测与签名路径。未运行代码、测试或查 CI。
e2be201 to
5350056
Compare
|
Rebased onto The rule, in one placeBoth of you were pointing at the same hole: a declared id only names the card that produced the tokens when the inherited visibility mask pins logical index
Because the function is backend-neutral and mask-name-driven, CUDA and Ascend can adopt the same check in a follow-up; today their signature branches do not test their visibility variable at all, which is the same latent gap this thread found on the MUSA side. Say the word and I will file that separately rather than widen this PR. Regressions for the cases you named
EvidenceUntested on hardware, unchanged from last round: these paths are exercised with stub probes and synthetic environment snapshots. The claim is only that a record no longer receives a comparable device identity it cannot prove. |
david6666666
left a comment
There was a problem hiding this comment.
A vendor binding can report an empty list without implementing process accounting.
| busy = [] | ||
| for index in sys.argv[1].split(","): | ||
| handle = pynvml.nvmlDeviceGetHandleByIndex(int(index)) | ||
| for process in query(handle) or (): |
There was a problem hiding this comment.
The official pymtml binding can expose this symbol without supporting process accounting: the pinned implementation unconditionally returns []. The absent-symbol and exception checks therefore still let this binding qualify a potentially busy device as idle. Please use a verified process source or report qualification unavailable with this stub, and add a fixture for a present but unsupported query.
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量检视 e2be20109f8185193a4a59ac2831015ab4a8a714 → 53500563711ed6eb4338f9f1c9738efe60c1ce00。上轮检视 的设备绑定问题已解决。
Dashboard 现在要求运行证据明确包含 MUSA_VISIBLE_DEVICES 键,并与 MUSA discovery 共用设备绑定的判断。声明物理设备 1 但未设置 mask 时,不再判定为已验证;显式 mask 1 则继续通过。缺键、非零设备未设置 mask 的回归用例均已覆盖。分别对照 rebase 前后各自的 base→head diff:路径迁移归一化后 43 个文件内容一致,实质改动的 6 个文件均已静态核对。
批准前必做:无。非阻塞:可同步 docs/musa.md 中对 logical_index 的旧描述。未运行测试或 MUSA 实机验证。
|
Fixed at
The rationale, stated plainly: an empty list from a binding that never implemented accounting is indistinguishable from a genuinely idle card at the API level, so the check needs an independent signal. Used memory is that signal, and the message names the number so an operator can see why. The residual limit is honest: a binding that fabricates both the process list and the memory figure cannot be caught from inside the probe — that needs a positive control (start a known workload and observe it), which is a campaign-level check rather than a device probe. I did not fake one. Regressions: Also picked up @zengchuang-hw's non-blocking note:
|
|
Real-MTML evidence for the two fail-closed paths, collected on an MTT S5000 host ( Discovery and the mask binding
Discovery-only ( The blind process binding is real on this hardwareThis is the important one: the host's card 0 was in use and the binding still reported zero processes.
So @david6666666's scenario is not hypothetical: on this driver the process query answers "none" while a card holds 78 GiB, and before Note on
|
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量 53500563 → f3e86601(完整 head f3e8660124de8d2ac605a7c48fd0399543cefb5e),上轮检视。实质变化是给空进程表加独立的逐设备显存校验:忙进程仍拒绝,显存未知、缺失或超过 256 MiB 也拒绝。文档已对齐逻辑绑定和两类探针;新增测试覆盖未知显存、缺字段和空进程表但占用 4096 MiB。未发现批准前必修项。
非阻塞:可显式拒绝负数显存值;此校验是采样时刻的筛选,不是独占预留。已静态核对三文件差异、仓库约定和测试定义;未运行测试、实机或查询 CI。
|
Controlled follow-up to the earlier probe-level note: same branch head EnvironmentMask binding, in-container
Dashboard signature, fed the real The blind accounting, now controlledI started my own process on an otherwise idle card (
Before This is not a one-card accident: on the same host three other cards were serving 74–80 GiB each and all three also reported zero processes. Still not claimedNo engine, throughput, TTFT or quality number is behind this comment, and nothing here exercises |
|
Runtime-path check on the same head SetupStartup completed: The engine ran on the card that was declared…and the process binding was still blind for a full serverWhile that engine was serving: So the memory cross-check is not protecting against an edge case: on this host the process query reports nothing even for a live Device identity asked mid-serveThe first is the case that matters for the reviewers' point: the card the adapter verifies is the card the engine serves on, because the identity mask pins logical LimitsOne functional run, eager mode, a 2B model, no concurrent load, and no campaign record behind it — this is identity and guard behaviour, not a measurement of the engine. |
|
#516 has been merged into |
Introduce Moore Threads MUSA as a first-class backend: discovery and qualification through pymtml, a mask-binding rule shared with the dashboard signature, and fail-closed idleness so a binding that cannot report either processes or used memory is treated as busy instead of idle. Add the vllm_musa repository pack, the vllm-musa skill with its references, and three source-derived records covering device identity, the torchada dispatch boundary and the Omni image lane. The skill and records live in the unified skill_library layout; the pack keeps only its pack.json. Signed-off-by: xiaodongye <xiaodong.ye@mthreads.com>
The skill moved to systems/engines/vllm-musa and its three records now live in skill_library/records, so the compiled inputs, the registry pin and the coverage table are regenerated together. Expected counts follow the library: 89 registered skills, 152 SKILL.md documents, 140 records. Signed-off-by: xiaodongye <xiaodong.ye@mthreads.com>
The unified library keeps skill metadata in SKILL.md instead of a sidecar metadata.json, and the docs point at the new systems/engines path. Signed-off-by: xiaodongye <xiaodong.ye@mthreads.com>
Signed-off-by: xiaodongye <xiaodong.ye@mthreads.com>
…table The library now registers 89 skills, 152 SKILL.md documents and 140 records; a campaign search therefore considers 132 documents. Signed-off-by: xiaodongye <xiaodong.ye@mthreads.com>
f3e8660 to
e78d0dd
Compare
|
Rebased onto What the rebase changed beyond mechanics:
Verification on this head: the full test suite has exactly the same failure set as The knowledge-authorship declaration and the approval text are refreshed for this head. Nothing else changed in substance: the platform adapter, the probe, the tests and |
There was a problem hiding this comment.
Independent third-party tip COMMENT (cursor[bot]; not Approve / not Request changes).
Head e78d0dd9e41c67ad93951116d7193266514936e8 is still noncompliant on required declarations / knowledge-model-policy:
- Ready declarations job: https://github.com/InferMatrixLab/InferRSI/actions/runs/44666264257/job/136056205381
- Evaluator errors:
Exception requires explicit independent owner approval;Material AI author 1: model is unapproved or suspended - Peer tip Approve from zengchuang-hw is present, but policy failure blocks Ready/admission.
Deferring formal cursor vote until declarations are compliant (and any Dong1017 tip findings on older heads are addressed or re-checked on this tip). Technical verify only — permission none; not write+ / dual / merge-ready.
Why
InferRSI can drive a controlled comparison on any engine, but it currently has no way to
recognise a Moore Threads (MUSA) device. On a MUSA host
doctorcannot identify the vendor,runcannot map--device-ids, and the skill library has novllm_musarepository pack, sothe whole evidence loop (frozen workload → A/B → verdict) is unavailable where vLLM-MUSA runs.
This PR adds MUSA as a first-class backend next to GPU and NPU, and adds a
vllm_musarepository pack so knowledge lookups do not quietly borrow CUDA advice.
What is added
Platform adapter —
src/inferrsi/infra/platforms/musa.pypymtml(the MUSA analogue of NVML). Physical ids are reportedas-is;
MUSA_VISIBLE_DEVICESis a separate, logical namespace thattorch.musaremaps,so the adapter deliberately does not apply
MUSA_VISIBLE_DEVICESto the probe the way aCUDA adapter applies
CUDA_VISIBLE_DEVICES. Reading the variable would hide the verydevices the caller asked about.
MUSA_VISIBLE_DEVICESis registered as a visibility variable and interpreted as aninteger index list.
dist_backendismccl, and the environment fingerprint records the MUSA toolchainvariables that change numerics or performance.
bytesandstrreturns frompymtml(it returnsbyteson somedriver versions), so a driver difference cannot masquerade as "no device".
Repository pack —
repo_packs/vllm_musa/with avllm_musascenario entry sorun --repo-type vllm_musaresolves, pluslinear_attention-aware guidance for thehybrid Qwen3.5/Qwen3.8 GDN families (Mamba/attention block budgets and the
max_num_seqscap they impose).Skill —
skill_library/repositories/vllm-musa/SKILL.mdand the regenerated catalog.DeepSeek Harness plugin — MUSA is offered as a backend option in the form, named in the
generated backend instruction, and covered by
tests/check_dsh_form.cjs.Validation
tests/test_musa_platform.py(282 lines) — probe parsing (bytes/str), physical-vs-logicalid handling,
mccldefault, visibility-variable semantics.tests/test_dsh_plugin.py,tests/test_cli_catalog.py,tests/test_knowledge_find.py,tests/test_repository_packs_facade.pypass.inferrsi doctor --backend musareportsdevice.status: available,selection_verified: true,n=8, and the repository packresolves for
--repo-type vllm_musa.vllm-musav0.32.0-devservesQwen3.8-27B-FP8on one S5000 andruns the frozen protocol (warmups, alternating arms, quality gate, medians, verdict).
Known limits (deliberately not hidden)
behaviour is claimed.
compatibilitystaysunverifiedfor an unseen model.bit-exact for every model: in our campaign, enabling graph capture changed greedy
decoding (token agreement 0.66 against the eager baseline, max |Δlogprob| 2.27), so it
is inadmissible under a bit-exact gate regardless of the throughput it gains. The adapter
does not paper over this; a campaign records it as a FAIL.
tests/test_skill_index.pyhas four tests that shell out togit rev-parse --no-lazy-fetch, which needs Git ≥ 2.46; they fail on older Git independent of this PR.Commits
All commits are DCO signed off (
Signed-off-by: xiaodongye <xiaodong.ye@mthreads.com>).The branch is rebased onto current
main(1b826b5), whose knowledge change moved thelibrary to the unified layout: the skill now lives in
systems/engines/vllm-musa, itsmetadata is embedded in the SKILL.md frontmatter instead of a sidecar
metadata.json,and the three records live in
skill_library/records.Knowledge authorship declaration