Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,24 @@

## [Unreleased]

### Added

- A `musa` platform backend and a `vllm_musa` repository pack cover Moore
Threads MUSA text-LLM and Omni serving. `doctor --backend musa` runs a
`pymtml` probe inside the target Python, because MUSA publishes no discovery
CLI to require, and records `MUSA_VISIBLE_DEVICES`, `torch_musa`, `torchada`,
`mate`, `mthreads-ml-py` and `vllm-musa` with the run. `--repo-type vllm_musa`
admits `llm.decode`, `llm.serving`, `diffusion.image_generation`,
`diffusion.video_generation` and `audio.tts`. Three source-derived records and
one skill state the device-identity boundary (MTML answers with physical ids
and is unaffected by `MUSA_VISIBLE_DEVICES` while `torch.musa` remaps), the
`torchada` dispatch boundary, and the Omni image's pinned dependency set.
Every descriptor stays unverified, the records carry
`locally_verified: false`, and this change claims no MUSA result.
- The DeepSeek Harness plugin's test-environment form offers `MUSA` beside GPU
and NPU, so a session can start a campaign on the backend the CLI now accepts
instead of only naming a generic GPU.

### Changed

- Apps workflow implementations now live in `apps/workflows/`: campaign capture,
Expand Down
6 changes: 3 additions & 3 deletions docs/dsh-plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ In DSH Web, open a target repository and its conversation, then click
1. **Optimization target (优化对象).** Enter the repository, model, framework,
SIAS plane and objective. The form fills the local repository path from the
current session when available.
2. **Test environment (测试环境).** Choose GPU or NPU and local or remote
2. **Test environment (测试环境).** Choose GPU, NPU or MUSA and local or remote
execution. After selecting remote execution, enter the remote repository path
in the first section, then return here to test the SSH connection. Set the
maximum optimization rounds and total time budget in minutes. The prepared
Expand Down Expand Up @@ -119,7 +119,7 @@ Click **Test connection**. For a new server, compare the displayed SHA-256 host
key fingerprint against a trusted value, then choose **Trust this fingerprint
and test**. A supplied fingerprint must match exactly. Successful testing checks
SSH authentication, Linux command execution and access to the selected repository.
GPU/NPU availability and benchmark readiness are checked by the Agent afterward.
GPU/NPU/MUSA availability and benchmark readiness are checked by the Agent afterward.

After a successful test, **Start iteration** becomes available. Changing the
host, authentication, repository, designated scripts or execution location
Expand Down Expand Up @@ -201,7 +201,7 @@ flowchart LR

| Owner | Responsibility |
| --- | --- |
| Form and prompt builder | Collect GPU/NPU, execution location, goal, SIAS plane, user budgets and optional acceptance inputs |
| Form and prompt builder | Collect GPU/NPU/MUSA, execution location, goal, SIAS plane, user budgets and optional acceptance inputs |
| DSH conversation | Deliver the prompt once to the selected session, preserving its model and permissions |
| SSH connection bridge | Authenticate the owner request, verify host key and login, retain a session-bound connection and return an opaque ID |
| `inferrsi_remote` | Carry explicit repository operations and cancellation through DSH tools without including authentication material |
Expand Down
19 changes: 16 additions & 3 deletions docs/legacy-knowledge-manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -338,10 +338,10 @@
"source": "REPO_TYPES = (\"vllm\", \"vllm_omni\", \"afd_plugin\", \"vllm_rlt\", \"strata\", \"sglang\", \"sglang_omni\")\n\n\ndef repository_pack(repo_type: str) -> dict[str, Any]:\n if repo_type not in REPO_TYPES:\n raise ValueError(f\"Unknown repository type: {repo_type}\")\n path = files(\"inferrsi\").joinpath(\"data\", \"repo_packs\", repo_type, \"pack.json\")\n return json.loads(path.read_text(encoding=\"utf-8\"))"
},
"current_fragment": {
"description": "REPO_TYPES and repository_pack as they stand after the K02 resource-path revision",
"description": "REPO_TYPES and repository_pack as they stand after the MUSA registry revision",
"hash_convention": "fragment_sha256 is the SHA-256 of source plus one trailing newline",
"fragment_sha256": "b6e49fdd67f7029e080e5e224cf3bf417c7dbc1e337644c3e1cb2b783cae2e4c",
"source": "REPO_TYPES = (\"vllm\", \"vllm_omni\", \"afd_plugin\", \"vllm_rlt\", \"strata\", \"sglang\", \"sglang_omni\")\n\n\ndef repository_pack(repo_type: str) -> dict[str, Any]:\n if repo_type not in REPO_TYPES:\n raise ValueError(f\"Unknown repository type: {repo_type}\")\n path = files(\"inferrsi\").joinpath(\"harness\", \"data\", \"repo_packs\", repo_type, \"pack.json\")\n return json.loads(path.read_text(encoding=\"utf-8\"))"
"fragment_sha256": "bddf8d146d14db887fcbc81d650a2101785303b3ea6f398d7f63af1cbd9c9371",
"source": "# Neutral raw repository-pack data leaf (D020). It holds only the original\n# REPO_TYPES tuple and pack.json reader, and imports no inferrsi module.\n# K02 moved pack.json to the Harness resource root (D007). The anchor stays the\n# root package and joins the physical path, so reading a Harness-owned resource\n# never imports inferrsi.harness and never gives infra an edge into Harness.\nfrom __future__ import annotations\n\nimport json\nfrom importlib.resources import files\nfrom typing import Any\n\nREPO_TYPES = (\n \"vllm\", \"vllm_omni\", \"vllm_musa\", \"afd_plugin\", \"vllm_rlt\", \"strata\", \"sglang\", \"sglang_omni\"\n)\n\n\ndef repository_pack(repo_type: str) -> dict[str, Any]:\n if repo_type not in REPO_TYPES:\n raise ValueError(f\"Unknown repository type: {repo_type}\")\n path = files(\"inferrsi\").joinpath(\"harness\", \"data\", \"repo_packs\", repo_type, \"pack.json\")\n return json.loads(path.read_text(encoding=\"utf-8\"))"
},
"resource_path_revision": {
"stage": "K02",
Expand Down Expand Up @@ -392,6 +392,19 @@
"to": "inferrsi.apps.host.inspection",
"review": "The move changes which module the rule names, so it requires the same formal architecture review as the rule itself. Passing tests are not that review."
}
},
"registry_revision": {
"stage": "MUSA-R1",
"issue": null,
"pull_request": "MUSA backend and vllm_musa repository pack",
"reason": "The vllm_musa repository pack needs a REPO_TYPES entry, so the leaf's registry grew while the reader body stayed byte-identical.",
"previous_file_sha256": "751d7bf663318962a2e73cf9d75369e0f8f8c2bda27964c56e44021b9f86c992",
"file_sha256": "bddf8d146d14db887fcbc81d650a2101785303b3ea6f398d7f63af1cbd9c9371",
"added_repo_types": [
"vllm_musa"
],
"unchanged": "repository_pack(), allowed_imports, two_symbols, top_level_shape and both permitted importers are untouched; the K02 path expression and the C2 relocation identity remain historical records of the bytes that existed then and are not rewritten by this revision.",
"review": "Reviewable as an additive registry entry; unlike the K02 path revision it changes no expression inside the reader and self-approves nothing."
}
},
"infra_inspection_rewrite": {
Expand Down
83 changes: 83 additions & 0 deletions docs/musa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# MUSA (Moore Threads) backend

`--backend musa` targets the Moore Threads MUSA plugin
([`MooreThreads/vllm-musa`](https://github.com/MooreThreads/vllm-musa)). It is a
discovery and measurement lane, not a validation claim: the adapter reports what
the host says, and every number still has to come from your own benchmark and
quality commands.

## What the adapter does

- Probes devices through `pymtml` (the pynvml-compatible MTML binding shipped by
`mthreads-ml-py`), in the interpreter you pass with `--python`, in a bounded
read-only subprocess. There is no `nvidia-smi`/`npu-smi` equivalent to require
and no MUSA context is initialized by the probe.
- Records `index`, `uuid`, `name`, driver version and total memory per device,
plus `MUSA_VISIBLE_DEVICES` in the environment snapshot.
- Registers `torch_musa`, `vllm-musa`, `mthreads-ml-py`, `torchada` and `mate` in
the dependency fingerprint, so a run records the compat layer it measured.
- Qualifies a campaign selection (`inferrsi.apps.host.platforms.musa.qualify_devices`)
by device count, SKU substring and an already-running compute process. An empty
process list only counts as idle when the device's used memory agrees
(`IDLE_USED_MIB`), because a vendor binding can implement the query without
implementing process accounting. A binding that reports neither fails.

## Device identity caveat

`pymtml` answers with **physical** device ids and is documented by the plugin as
unaffected by `MUSA_VISIBLE_DEVICES`, while `torch.musa` applies the logical
remapping. `--device-ids` therefore means physical ids.

```bash
# Devices as the plugin sees them, then pin the same physical ids.
inferrsi doctor --repo ~/ws/vllm-musa --repo-type vllm_musa --backend musa \
--python /path/to/musa/python --device-ids 0,1 --device-model "S5000"
```

A declared id only names the card that produced the tokens when the inherited
mask pins logical index `L` to that id. An unset mask is the runtime's implicit
identity view, so it binds `--device-ids 0` but not `--device-ids 1`. Discovery
and the dashboard signature share `logical_binding()`
(`inferrsi.apps.host.platforms`): a selection whose mask does not bind is
reported unverified and gets no comparable device identity. `qualify_devices`
returns each record with the `logical_index` the trial will use, so evidence can
state "verified physical N == logical L" explicitly.

## First experiment

```bash
inferrsi plan \
--repo ~/ws/vllm-musa --repo-type vllm_musa --backend musa \
--python /path/to/musa/python --device-ids 0 --device-model "S5000" \
--baseline-command "bash bench.sh baseline {result}" \
--candidate-command "bash bench.sh candidate {result}" \
--validation-command "bash check_outputs.sh" \
--metric-key latency_ms --direction minimize --repetitions 3 \
--min-improvement-pct 3 --output-dir ~/campaigns/musa-first
inferrsi run ... # same flags, then read report.html
```

Commands are executed as `argv` (no shell). Wrap pipelines in a script and pass
`{result}` as the path your script must write the metric JSON to.

## Lanes

- **Text LLM**: `--repo-type vllm_musa`, scenarios `llm.decode` and
`llm.serving`.
- **Omni (image/video/audio)**: same repo type; scenarios
`diffusion.image_generation`, `diffusion.video_generation`, `audio.tts`. Build
from the MUSA Omni image so the pinned dependency set is part of the record.

Knowledge: `skill_library/records/` for the three `vllm_musa.*` records and
`systems/engines/vllm-musa/` for the operating procedure
([`SKILL.md`](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-musa/SKILL.md)).

## Bounds

- The probe was written against the plugin's `pymtml` usage and tested with
fixtures; it has not been executed against MUSA hardware in this repository.
- A `musa` descriptor is deliberately `unverified` in `pack.json`; device
discovery is not a compatibility or performance certification.
- Records under `skill_library/records/` are source-derived
(`locally_verified: false`). CUDA and Ascend records remain scoped to their own
hardware.
21 changes: 21 additions & 0 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,27 @@ are distinct from `knowledge search`, whose package catalog filters
backend, engine or kernel path is compatible. compatibility stays unverified.
- **Example:** inferrsi doctor --repo . --repo-type vllm_omni --backend ascend --device-ids 0 --device-model 910B3

### Discover Moore Threads MUSA devices — `doctor --backend musa`

- **Inputs:** a prepared Git checkout and its existing repo type, target Python
(`--python`), optional repeatable `--device-ids` and one case-insensitive
`--device-model`.
- **Effect:** read-only. The bounded command runner starts a probe in the target
Python, imports `pymtml` (the pynvml-compatible MTML binding) and reads the
index, UUID, name, driver version and total memory of every device. MUSA
publishes no discovery CLI to require, and the probe creates no MUSA context.
- **Output:** those device records, plus `MUSA_VISIBLE_DEVICES` and the
`torch_musa`, `vllm-musa`, `mthreads-ml-py`, `torchada` and `mate` versions in
the environment snapshot. A requested ID and the requested model must match the
same record. A reached probe whose selection or record list is unusable is
reported `unavailable`, not `tool_available`.
- **Boundary:** the probe answers with physical ids. MTML is unaffected by
`MUSA_VISIBLE_DEVICES` while `torch.musa` applies the logical remapping, so a
matched record does not prove which logical device a trial measured; record the
mapping with the evidence. Discovery is not compatibility or performance
qualification, and the descriptor stays unverified.
- **Example:** `inferrsi doctor --repo . --repo-type vllm_musa --backend musa --python /opt/musa/bin/python --device-ids 0 --device-model S5000`

### Search packaged records — `knowledge search` / `knowledge show`

Use `find` for ordinary discovery. With no query, `knowledge search` lists
Expand Down
2 changes: 1 addition & 1 deletion docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ inferrsi plan \

Replace `plan` with `run` using the same flags after the plan matches your intended experiment. Script paths are templates that you supply; InferRSI does not create them. Start with one-variable, same-quality configuration optimization. Quantization, TeaCache, generation-step changes and adaptive exit require a quality-changing protocol and an explicit validation script.

On ROCm use `--backend rocm`; on Ascend use `--backend ascend`. The doctor retains raw backend output but does not certify device selection, model support or full topology. Record the actual driver/runtime/communication versions and topology with your experiment evidence.
On ROCm use `--backend rocm`; on Ascend use `--backend ascend`; on Moore Threads MUSA use `--backend musa` with `--repo-type vllm_musa` and `--python` pointing at the interpreter that has `pymtml`, because the probe runs there. The doctor retains raw backend output but does not certify device selection, model support or full topology. Record the actual driver/runtime/communication versions and topology with your experiment evidence.

## Native vLLM-Omni benchmark inside a trial script

Expand Down
Loading
Loading