Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/knowledge-base.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,20 +282,20 @@ Complete procedures, references and cases are packaged together. Browse by
system, engineering task, workflow or prior outcome. Backend is recorded independently. The 21 engine and research-router skills
use the same ownership layout under `systems/engines/<engine>/` and
`workflows/research/inference-engine-research/`, including their sub-skills and references.
The library contains 151 `SKILL.md` files, all in English. The registered
The library contains 152 `SKILL.md` files, all in English. The registered
[Lithos Metal candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/SKILL.md)
adds three owner-local subskills for compiler/fusion, generation state and
serving/cache contracts; it adds no executable repository target.
The collection overview, import notes and source manifest belong to the research
workflow's references; Strata design-space analysis belongs to Strata. The library root holds the registry, configuration, compiled inputs, authored findings and skill directories.

The [runtime catalog](../src/inferrsi/knowledge/data/skill_library/README.md)
projects the 88 registered skills into the contribution catalog. Its records
projects the 89 registered skills into the contribution catalog. Its records
use `inferrsi.<backend>.skills.<name_with_underscores>` IDs, with `ascend` and
`cuda` for scopes that depend on those backends and `common` for the remaining skills.
Each `skill_documents.original` points to the full packaged procedure. The
`skills` CLI retains the original hyphenated skill IDs and defaults to this
88-ID registry; engine procedures remain available through library navigation
89-ID registry; engine procedures remain available through library navigation
and catalog document pointers.

The registered [PyTorch](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/SKILL.md)
Expand Down
1 change: 1 addition & 0 deletions docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,7 @@ SSD/OS systems columns.
| [Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/6cd3ee98dfe32746b345a6abe521fa4cb9bdc187) @ `6cd3ee98dfe32746b345a6abe521fa4cb9bdc187`; [arXiv:2605.26118v1](https://arxiv.org/html/2605.26118v1) (repository Apache-2.0; paper CC BY 4.0) | evaluation | Bounded source inspection, 2026-11-02: 18-file trial identity/origin, external outcome, selection/finalization/reporting and baseline-oracle readset; complete paper HTML/appendix/tables/algorithm/listing/captions, unused graphic values unverified; [exact hashes, rights and exclusions](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-source.json) | [`inference-benchmark-design` reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-trial-evidence.md); candidate/not_run, efficacy not_evaluated; two measured meanings, named-verdict update limits and manual versus measured-only defaults retained; no runtime integration or whole-model/backend promotion |
| [Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc98043fcf89f3589aa5addb3742777a) @ `1882c6adbc98043fcf89f3589aa5addb3742777a` (selected software/docs Apache-2.0) | evaluation | Bounded source inspection, 2026-11-03: Qwen3.5 GDN/MTP eager record/fold and checkpoint/adoption/retirement contracts, unexecuted fold-test definitions and historical completion denominators; [27-file readset and rights](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-source.json) | [`inference-benchmark-design` reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md); candidate/not_run, efficacy not_evaluated. Historical author results retain their separate execution revision; no new implementation, backend or measured resource coverage |
| [shenh10/HuggingArch](https://github.com/shenh10/HuggingArch/tree/facbbbc96856e558b9084a94a5070b2e2c86cf0b) @ `facbbbc96856e558b9084a94a5070b2e2c86cf0b` (MIT); companion post dated 2026-10-08, live, not attributable to the pin | evaluation | Bounded source inspection, 2026-11-11: checkpoint join and weight ledger code, validator and parallelism documents, upload speed-of-light check and estimate-time anomaly flag; [12-file readset with digests and scope](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-source-identity.json). Marked `[code]`/`[doc]`/`[tree]`/`[blog]`; post figures absent from the repository recorded as unverified | [`performance-ceiling-analysis` reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md): two-gate weight anchor and two-threshold measurement rule; candidate/not_run. Anchors stored weights only; KV, collectives and E2E performance unanchored; no code, specs or measured corpus vendored, and HuggingArch was not run |
| **shenh10/HuggingArch** @ `e69d20a028e5b17524aa48b53bddf8b9aad5d15b` (MIT) | reference; mined for a model-level speed-of-light procedure | Per-phase binding with causal pair counts; fused-graph pricing with per-fusion source anchors; per-rank geometry with declared non-divisibility policies and MoE busiest-rank skew; analytical collective terms; per-node KV storage laws and boundedness gates; publication gates where unjudged means failed; measured envelope kept beside the roofline. Inspected 2026-11-10 at the default-branch head: 15 documents read (3 more hashed only) and 26 source files; the source's per-op SOL sum is relabelled a serial estimate because it assumes no kernel overlap. **No predicted-against-measured comparison exists in the inspected scope**; collectives are unmeasured upstream | `model-sol-analysis`; [document](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/model-sol-analysis/references/huggingarch-model-sol.md) + [identity](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/model-sol-analysis/references/huggingarch-source-identity.json); `performance-ceiling-analysis` keeps its earlier pin for the checkpoint anchor |
| Dong1017/OmniRSI @ `codex/skill-eval-bootstrap` (82218b4) | evaluation | skill_distill / skill_eval harness, no_skill/human/distilled contract, Tier A pilot | `npu-skill-factory` (evaluation contract) |
| InferMatrixLab/InferRSI PR #9 @ 4ad551a9 | reference | `skill_library/` layout + knowledge records | `npu-skill-factory` (bundle layout) |
| RRSI — arXiv 2609.24972v3 | search; mined via reviewed Pack F | bounded harness proposals; calibrated noise/cost/novelty gates; grouped-ablation limits, replacing the earlier generic constraint summary | `optimization-search-policy`; [exact ledger](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/references/source-notes.md#f1--rrsi) |
Expand Down
2 changes: 2 additions & 0 deletions docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ The primary compute and HBM resource. Most current work lives here.
| **Diagnose / Configure / Optimize / Verify** | Device-HBM stream-use lifetime, fragmentation/counter interpretation and gated allocator controls | `torchnpu-device-memory-lifecycle` | Lifetime recording does not insert data dependencies; expandable segments change counters/control legality; GC synchronization/comment conflict retained | TorchNPU @ 67511c53cd4c, Dot bundle; device allocator ledger | 🟢 (source-supported procedure; NPU execution not run) |
| **Diagnose / Verify** | Serving model: operator/context dependencies, calibration/load domain, fidelity provenance and joint-SLO prediction uncertainty | [calibrate-serving-simulation](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/SKILL.md), `performance-ceiling-analysis` | Vidur near-capacity CPU-overhead counterexample; simulator output is not a bound. [Infera source reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/README.md) separates scalar tuner scoring from DES percentiles and retains missing-SLO/stale-document limits; no service guarantee | Vidur v2 and bounded Infera Git readset; Reference Matrix systems sources | 🟢 (prediction guidance; calibration not run) |
| **Diagnose / Verify** | Reusable-prefix compute work versus host/network KV loading cost | [performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md), [Mooncake condition](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md) | Check bytes, FLOPs and effective rate at one GPU-group scope; favorable Equation 2 still requires exposed-cost/SLO measurements | Mooncake FAST '25 §2.2, Table 1, Eqs. 1-2; separate public-paper source, not recovered foundation archive | 🟢 (analytical source condition; current feasibility and E2E benefit unverified; no SSD credit) |
| **Diagnose / Verify** | Model-level prefill/decode speed-of-light under a declared layout: anchored counts, fused-graph pricing, per-rank geometry, KV growth laws, resource-total bound versus serial per-op estimate | [model-sol-analysis](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/model-sol-analysis/SKILL.md), `performance-ceiling-analysis` | Serial per-op sum assumes no kernel overlap, so it may only flag a faster result; unfused graphs overstate launches and activation round-trips; default MoE skew is optimistic. [Helper contract](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/model-sol-analysis/references/model-sol-contract.md) | HuggingArch @ `e69d20a028e5b17524aa48b53bddf8b9aad5d15b`; [source reading](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/model-sol-analysis/references/huggingarch-model-sol.md) | 🟢 (source-derived candidate; synthetic helper tests only; no model, device or measured comparison) |
| **Diagnose / Verify** | Offline offload feasibility across GPU, CPU RAM and disk | [performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md), [FlexGen condition](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/flexgen-offload-feasibility.md) | Joint weights/KV/activation placement; RAM resident plus staging/working peaks, SSD state capacity and disk↔CPU cost, effective batch versus GPU batch, exposed CPU/transfer cost and block-latency objective | FlexGen official ICML 2023, §§3–4.3 and selected A.3; separate public source, foundation archive still unavailable | Source-only candidate; empirical not_run, fitted/perfect-overlap model and relaxed OOM limits; no blanket RAM/SSD, Network/OS or current-engine qualification |
| **Diagnose / Configure / Optimize / Verify** | Serving service order versus HBM admission and suspended-KV recovery; replica migration versus elastic sequence groups | [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/SKILL.md), [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) | [Hypothetical pressure/migration cases](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md); cohort/fairness and destination-reservation costs | Orca proceedings, FastServe v3, Llumnix v1, LoongServe v2; implementation pins remain separate | 🟢 (conditional paper-derived plans) |
| **Diagnose / Optimize / Verify** | Training activation lifetime, rematerialization and reserved-address packing | [plan-activation-residency](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/SKILL.md) | Dependency repair and capacity feasibility checked separately; no recovery-checkpoint coverage | Checkmate v3, OLLA v2; [C readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md) | 🟢 (planning, not solver/runtime validation) |
Expand Down Expand Up @@ -154,6 +155,7 @@ Inter-device and inter-node communication: HCCL (Ascend), NCCL (CUDA), InfiniBan
| **Observe / Profile** | Communication time in step-trace four-tuple, HCCL per-collective timing, `communication.json` | `ascend-performance-analysis` (Layer 4) | HCCL single-collective latency varies ~2× across devices | MindIE-SD profiling-analyze @ 8481d39b | 🟢 |
| **Diagnose** | Exposed communication: `effective_comm = Communication - Overlapped`; communication-bottleneck class; wait-aware diagnosis (wait pollution, transport/rank metrics, device imbalance) | `ascend-performance-analysis` (L1 step-trace derivation), `cann-attribution-overlap` (wait-aware) | Overlapping vs exposed comm in step_trace_time.csv; zero/missing bandwidth = unavailable evidence, single-rank data cannot establish cross-rank balance | MindIE-SD; cann/oam-tools @ 8440935f (pack-mediated) | 🟢 |
| **Diagnose** | Topology-aware card group validation: intra-domain (UB/HCCS) vs inter-domain (PCIe/SYS) | `ascend-distributed-parallel`, `ascend-device-health` | npu-smi topo output; legal card-group rules; Health=OK not sufficient | MindIE-SD dit-parallel-opt @ 8481d39b | 🟢 |
| **Diagnose / Verify** | Analytical collective terms in a model-level floor: collectives at sharding-state changes, ring-factor bytes per rank, log-depth hops, intra- versus inter-node links and hierarchical all-reduce | [model-sol-analysis](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/model-sol-analysis/SKILL.md) | Unmeasured link latency enters the serial estimate only, never the bound; one latency for every fabric is a source simplification | HuggingArch @ `e69d20a028e5b17524aa48b53bddf8b9aad5d15b`; [source reading](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/model-sol-analysis/references/huggingarch-model-sol.md#5-communication) | 🟢 (analytical source method; collectives not measured; no Network qualification) |
| **Configure** | Collective-in-graph capture (HCCL graph mode), comm-stream masking; masking formula `1-1/n` when F≥C | `ascend-distributed-parallel` | Masking upper bound derivation; AlltoAllV defect workaround | MindIE-SD @ 8481d39b | 🟢 |
| **Optimize** | USP activation requirement (ParallelConfig wrapping), strategy selection, AlltoAllV workaround; evidence-gated compute/communication overlap of independently-ready work | `ascend-distributed-parallel`, `cann-attribution-overlap` | USP must set ParallelConfig; verify allToAll appears not just allGather; overlap preserves ordering and joins, no comm recommendation without comm evidence | MindIE-SD dit-parallel-opt @ 8481d39b (9 ref files); cann/oam-tools @ 8440935f | 🟢 |
| **Verify** | Masking ratio measurement, before/after comm-fraction comparison; strategy activation attribution | `ascend-distributed-parallel` (attribution method), `ascend-performance-analysis`, `dense-performance-evidence`, `cann-attribution-overlap` (overlap acceptance) | Step-trace four-tuple as ground truth; scope-effectiveness-check.md | MindIE-SD perf-gate + dit-parallel-opt; cann/oam-tools @ 8440935f | 🟢 |
Expand Down
Loading
Loading