diff --git a/.gitattributes b/.gitattributes index 238e95715..154103b64 100644 --- a/.gitattributes +++ b/.gitattributes @@ -9,7 +9,7 @@ /evidence/_template/** text eol=lf # Retain this upstream license's original CRLF Git blob bytes. -/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-LICENSE.txt -text +/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-LICENSE.txt -text # Generated third-party bundles, with source references and notices beside them. /plugins/inferrsi-dsh/vendor/ssh2.cjs linguist-generated=true linguist-vendored=true diff --git a/AGENTS.md b/AGENTS.md index bc8d25bb0..d61d4312c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -67,7 +67,7 @@ Parameterized or manual acceptance runs are issue-scoped; they are not the defau ## RSI contract -- **Resolve acceptance criteria** — preserve user-supplied criteria; when absent, use [performance-ceiling-analysis](src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/acceptance-criteria.md) to derive evidence-backed campaign targets. Retain partial requirements and derive only missing fields; unresolved evidence stays explicit. +- **Resolve acceptance criteria** — preserve user-supplied criteria; when absent, use [performance-ceiling-analysis](src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md) to derive evidence-backed campaign targets. Retain partial requirements and derive only missing fields; unresolved evidence stays explicit. - **Freeze gates before search** — checklist and `validator/` locked in `plan/` before Phase 3 candidate experiments ([validation levels](docs/contributing.md#validation-levels)). - **One plane per campaign step** — change only the declared SIAS plane per step: `harness` | `model` | `infra` (under `src/inferrsi/`). - **Emit RunSpec snapshots** — record repo × revision × backend × device × model × scenario (`contracts/runspec.py`, campaign `plan/`). @@ -83,7 +83,7 @@ Parameterized or manual acceptance runs are issue-scoped; they are not the defau - Cite **acceptance issues + pinned SHAs** in downstream text; do not invent gain or quality bars. - **Public wording only after `PASS`** — follow [evidence and claim rules](docs/contributing.md#evidence-and-claims); until then, label unverified or `INCONCLUSIVE`. -- Use [cite-check-downstream-pr](src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md) and [competitor-scan](src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/SKILL.md) as **operators**, not campaign scripts (no pins in skill bodies). +- Use [cite-check-downstream-pr](src/inferrsi/knowledge/data/skill_library/workflows/governance/cite-check-downstream-pr/SKILL.md) and [competitor-scan](src/inferrsi/knowledge/data/skill_library/workflows/research/competitor-scan/SKILL.md) as **operators**, not campaign scripts (no pins in skill bodies). ## Evidence and clearing the bar @@ -104,7 +104,9 @@ Agents and human reviewers **clear** work only with artifacts—not vibes. Green ## Skills -Portable operator guidance and candidate procedures live in [src/inferrsi/knowledge/data/skill_library/](src/inferrsi/knowledge/data/skill_library/README.md), with hardware-specific guidance under `hardware//`, framework/repository guidance under its owner, and generic `methods/` and `workflows/`. The library owns complete procedures, metadata, references and cases. Keep library content in English; supporting documents stay in owner-local resource directories. The [knowledge-base contract](docs/knowledge-base.md) and two matrices define ownership and coverage. Keep **AGENTS.md** short; follow skill bodies for procedures—skills must remain free of campaign SHAs. +For an inference task, first select one of the [seven required primary skills by deliverable](plugins/inferrsi/skills/inferrsi/SKILL.md#load-the-primary-inference-task). Run `inferrsi skills enter ` and read the returned instructions before the first evidence search. For an accuracy regression, use `inferrsi skills enter task-accuracy`. If the required skill cannot load, report the blocker. Frozen evaluations follow their assigned entry and knowledge condition. + +Portable operator guidance and candidate procedures live in the [skill library](src/inferrsi/knowledge/data/skill_library/README.md). Keep complete English instructions, metadata, references and cases with the owner. The [knowledge-base contract](docs/knowledge-base.md) and two matrices define ownership and coverage. Keep campaign SHAs out of skill bodies. ## Optimization requests diff --git a/README.md b/README.md index 61b9756ce..3eb933355 100644 --- a/README.md +++ b/README.md @@ -74,6 +74,7 @@ real-model gains remain unmeasured. [Flow, limits and reproduction →](docs/har | You want to… | Start at | | --- | --- | +| Choose a starting point for a new model, performance, debug, accuracy, upgrade or engine task | Choose the [primary skill for the requested outcome](docs/knowledge-layout.md#seven-deliverables) and read its procedure with `inferrsi skills enter ` before the first evidence search. Follow its decision points using the actual run and acceptance criteria. | | Modify InferRSI itself | [Contributor guide](docs/contributing.md) and [agent collaboration](docs/agent-collaboration.md), under an accepted Issue scope | | Use the Workbench to run experiments | [Quickstart](docs/quickstart.md): prepared target environment, benchmark/quality contracts, authorized experiment, recorded evidence | | Complete an optimization benchmark in the current Agent | [Iterative task sessions](docs/optimization-benchmarks.md), with frozen rounds, independent feedback and final retesting | @@ -298,12 +299,12 @@ performance evidence or an autonomous model-backed optimizer. | [Qwen Ascend](docs/qwen-image-2.1-turbo-ascend.md) | NPU results, rejected rounds and verification scope | | [h3-metal lessons](docs/h3-metal-apple-silicon-lessons.md) | MiniMax-H3 on Apple silicon: measurement pitfalls, gains, rejected directions and search-loop lessons | | [Apple silicon playbook](docs/apple-silicon-playbook.md) | Measured, engine-agnostic lessons for Metal/MLX engines: unified memory, Metal execution and benchmarking a shared Mac | -| [Knowledge base](src/inferrsi/knowledge/data/knowledge_packs/README.md) | Three learning tiers: contribution, dependency impact and peer techniques | -| [Skill catalog](src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/README.md) | Searchable records linking categorized skills to full packaged procedures and resources | +| [Knowledge base](src/inferrsi/knowledge/data/skill_library/records/README.md) | Three learning tiers: contribution, dependency impact and peer techniques | +| [Skill catalog](src/inferrsi/knowledge/data/skill_library/README.md) | Searchable records linking categorized skills to full packaged procedures and resources | | [Source audit](docs/reviews/source-audit-ac70272.md) | 34 review points with proposed remedies, qualifications and validation criteria | | [Daily learning prompt](docs/daily-learning-prompt.md) | 06:00 Asia/Shanghai L2C + performance workflow and prerequisites | -| [Complete skill library](src/inferrsi/knowledge/data/skill_library/README.md) | Registered skills and English engine/workflow documents, organized by owner | -| [vLLM-Omni experience records](src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/README.md) | Backend-scoped tuning methods, sources, and applicability | +| [Complete skill library](src/inferrsi/knowledge/data/skill_library/README.md) | Task entrypoint, system knowledge, playbooks, workflows and casebooks | +| [vLLM-Omni experience records](src/inferrsi/knowledge/data/skill_library/records/README.md) | Backend-scoped tuning methods, sources, and applicability | | [Contributor guide](docs/contributing.md) | Development workflow, evidence contracts, and CI | | [Engine delivery template](docs/template/README.md) | Copyable engine scaffold, branding, evidence and schema-v1 catalog recipe | diff --git a/docs/apple-silicon-playbook.md b/docs/apple-silicon-playbook.md index 3ba35a801..b26994961 100644 --- a/docs/apple-silicon-playbook.md +++ b/docs/apple-silicon-playbook.md @@ -23,9 +23,9 @@ acceptance result. | Existing source-derived guidance | What it leaves open | Measured observation here | | --- | --- | --- | -| [h3.c runtime audit](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md), [64 GiB worked preparation](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md) | Target execution and peak memory unverified; tensor counters are not physical footprint | Process peaks for three configurations, the component that sets the memory floor, and cold-load cost ([Unified memory](#unified-memory)) | -| [MLX lazy evaluation and stream lifetimes](../src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) | Submission versus completion stated from source; no device timing | CPU encoding and GPU wait measured separately; a large encode cost that did not move wall time ([Metal execution](#metal-execution)) | -| [vLLM Metal cache lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/SKILL.md) | One physical unified pool stated from source; memory pressure not measured | Resident weights evicting other weights from the page cache, with the cold-start cost | +| [h3.c runtime audit](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md), [64 GiB worked preparation](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md) | Target execution and peak memory unverified; tensor counters are not physical footprint | Process peaks for three configurations, the component that sets the memory floor, and cold-load cost ([Unified memory](#unified-memory)) | +| [MLX lazy evaluation and stream lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) | Submission versus completion stated from source; no device timing | CPU encoding and GPU wait measured separately; a large encode cost that did not move wall time ([Metal execution](#metal-execution)) | +| [vLLM Metal cache lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/SKILL.md) | One physical unified pool stated from source; memory pressure not measured | Resident weights evicting other weights from the page cache, with the cold-start cost | | [Quickstart A/B experiments](quickstart.md) | Generic; no Apple-specific interference sources | Interference seen on a shared Mac and how each was detected ([Benchmarking a shared Mac](#benchmarking-a-shared-mac)) | ## Unified memory diff --git a/docs/architecture.md b/docs/architecture.md index 0ea517ba0..ebc5a5724 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -82,7 +82,7 @@ Reviewed paper comparisons, readsets and cases live under their owning skill's Paper knowledge does not supply repository-only schema fields such as a Git SHA, so it remains linked Markdown rather than a parallel registry or fabricated `repo_packs` record. The [knowledge-base contract](knowledge-base.md) defines -local sidecar ownership, typed source examples, shared checkout references and +skill-local metadata ownership, typed source examples, shared checkout references and legacy package compatibility. Normalized reads and bounded Collect ([knowledge-operations.md](knowledge-operations.md)) and the authorized local store ([knowledge-store.md](knowledge-store.md)) are implemented Python @@ -281,10 +281,8 @@ src/inferrsi/ image.py OpenAI-style image payload and PNG response protocol infra/ artifacts.py Hashing and evidence writes - data/ - repo_packs/ Contribution descriptors and versioned tuning knowledge - knowledge_packs/ Dependency and peer knowledge, preserving existing tiers - skill_library/ Complete categorized English skills and resources + knowledge/data/skill_library/ Registered skills, engineering records, configuration and compiled inputs + harness/data/repo_packs/ Execution target descriptors examples/ Current Qwen runner, image validator and v4 adapter tests/fixtures/legacy/ Frozen historical validation dependencies for regressions ``` @@ -437,26 +435,17 @@ operations or the control CLI to Linux. Package data is the canonical source for noneditable installations: ```text -inferrsi/knowledge/data/repo_packs/REPO/knowledge/ - README.md Organization and navigation only - BACKEND/COMPONENT/ENTRY.json One canonical reusable experience -inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ - Generated from data/skill_library/, never hand-edited -inferrsi/knowledge/data/knowledge_packs/ - README.md Three-tier source navigation - dependency/REPO/BACKEND/COMPONENT/ENTRY.json - peer/REPO/BACKEND/COMPONENT/ENTRY.json +inferrsi/knowledge/data/skill_library/ + tasks/ systems/ playbooks/ workflows/ casebooks/ Registered owners + records/.json One authored engineering finding + config/ Aliases and research profiles + _compiled/ Build-time YAML headers and Git fingerprints + skill-index.json Stable ID, physical path and backend ``` JSON owns applicability, claims, recommendations, validation and limitations; `knowledge search`, `knowledge show` and human-readable `context` use that same -record. Per-entry duplicate READMEs are removed. The loader validates that -repository, backend, component and stable ID agree with the path. -The learning tier is an independent acquisition policy: contribution packs -retain their paths, while external dependency/peer packs add source-owned -candidates. Tier/source/backend/scenario filters intersect before ranking and -limits; no downstream repository inherits external knowledge implicitly. -Knowledge-only sources do not extend the executable repository registry. +record. The tier identifies each source relationship. The reader validates the repository, backend and component encoded in the stable filename. A backend filter includes that backend and repository-local `common` methods; scenario filters select exact workloads or component wildcards. Model, SKU, @@ -469,25 +458,26 @@ guidance or reported measurements independently from a source's merge status. Imported skills are source material, not executable tool authority. CPU tests do not upgrade imported knowledge to a validated claim. -The optional `skills` CLI defaults to the installed `data/skill_library/` and -its `skill-index.json` registry of stable skill IDs. Hardware-specific content -lives under `hardware/ascend`, `hardware/cuda` or `hardware/host`; other skills -are grouped by framework, repository, method or workflow. Full procedures, -sidecars, references and cases ship with the package. Engine entry skills live -under `repositories//`, the collection router under -`workflows/inference-engine-research/`. All library content is in English. -Supporting documents are owner-local; only `README.md` and `skill-index.json` -remain at the library root. An explicit -`--root` can still select a custom flat skills tree or a registry-based library. The CLI uses PyYAML from the `skills` extra, -lists metadata, reads a named body or resolves a named resource for an external -file reader. It never executes, installs or promotes skills. Core knowledge -commands read catalog JSON and packaged documents without the optional parser; -the two matrices retain their authority. +The bundled skill CLI uses `data/skill_library/` and its `skill-index.json` registry. +The main agent selects one primary procedure from `tasks/` and reads it with +`skills enter ` before searching for supporting evidence. The base +runtime loads the complete packaged primary procedure and its host header without PyYAML. +The structured InferRSI annotation stays in the pinned source file. + +Supporting procedures live in `systems/`, `playbooks/`, `workflows/` and +`casebooks/`. Engine owners live under `systems/engines//`. Each owner +packages its full English procedure, embedded metadata, references and cases. The library +also owns authored findings in `records/`, search configuration in `config/` +and generated build inputs in `_compiled/`. An explicit `--root` selects a +custom flat or registry-based library. Readers that parse skill frontmatter use +PyYAML from the `skills` extra. Core knowledge commands read catalog views and +packaged documents with the base interpreter. The two matrices retain their +authority. [Directory and search diagrams](knowledge-layout.md) Run evidence remains in the caller-selected run directory; frozen campaign plans/results/patches/validators remain under `evidence/`. -See the [knowledge guide](../src/inferrsi/knowledge/data/knowledge_packs/README.md) +See the [knowledge guide](../src/inferrsi/knowledge/data/skill_library/records/README.md) for all entries, retrieval rules and the maintenance workflow. The earlier [component design](assets/component-architecture-en.svg) and @@ -503,8 +493,8 @@ automatic promotion into the knowledge catalog or RunSpec. The original diagram is preserved; the [source audit](reviews/source-audit-ac70272.md) tracks cleanup portability and other implementation gaps. -The migrated skills have [generated catalog records](../src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/README.md) -in `data/repo_packs/inferrsi/knowledge/`, with backend-scoped IDs and +The migrated skills have [generated catalog records](../src/inferrsi/knowledge/data/skill_library/README.md) +assembled at read time, with backend-scoped IDs and `skill_documents.original` pointing to the complete packaged procedures. Normalized reads retain the library's dependency resources. The library is the single content owner. diff --git a/docs/assets/knowledge-layout/01-task-layers.png b/docs/assets/knowledge-layout/01-task-layers.png new file mode 100644 index 000000000..c795b93e8 Binary files /dev/null and b/docs/assets/knowledge-layout/01-task-layers.png differ diff --git a/docs/assets/knowledge-layout/02-physical-layout.png b/docs/assets/knowledge-layout/02-physical-layout.png new file mode 100644 index 000000000..9820cf419 Binary files /dev/null and b/docs/assets/knowledge-layout/02-physical-layout.png differ diff --git a/docs/assets/knowledge-layout/03-search-and-delivery.png b/docs/assets/knowledge-layout/03-search-and-delivery.png new file mode 100644 index 000000000..a63cc9bc0 Binary files /dev/null and b/docs/assets/knowledge-layout/03-search-and-delivery.png differ diff --git a/docs/assets/knowledge-layout/04-before-after.png b/docs/assets/knowledge-layout/04-before-after.png new file mode 100644 index 000000000..dfe10b77e Binary files /dev/null and b/docs/assets/knowledge-layout/04-before-after.png differ diff --git a/docs/assets/modular-directory-map-en.svg b/docs/assets/modular-directory-map-en.svg index eb9c2b92d..908040f19 100644 --- a/docs/assets/modular-directory-map-en.svg +++ b/docs/assets/modular-directory-map-en.svg @@ -37,8 +37,8 @@ cli.py, arguments.py CLI entry runspec.py Immutable RunSpec catalog.py Search + cited context -data/repo_packs/ Repository knowledge -data/knowledge_packs/ Dependency + peer +skill_library/records/ Findings +skill_library/config/ Discovery config examples/ Campaign composition tests/fixtures/legacy/ Frozen compatibility tests diff --git a/docs/codex-plugin.md b/docs/codex-plugin.md index 370a3a4df..927137fef 100644 --- a/docs/codex-plugin.md +++ b/docs/codex-plugin.md @@ -70,8 +70,10 @@ Or: > Use InferRSI to inspect the existing run I provide and explain its verdict, > missing evidence and next action. -The plugin checks its bundle/interpreter, reads relevant knowledge progressively, -and uses the current CLI's help to prepare operations. Its optional onboarding +The plugin checks its bundle and interpreter. For an inference engineering task, +it selects one of the seven primary skills +and reads the procedure returned by `skills enter ` before its first +supporting evidence search. It uses the current CLI's help to prepare operations. Its optional onboarding action invokes the same entry skill. Setup does not start benchmarks. Use the script resolved from the installed skill for manual diagnosis: @@ -81,9 +83,10 @@ python -X utf8 "/skills/inferrsi/scripts/inferrsi.py" --plugin-info python -X utf8 "/skills/inferrsi/scripts/inferrsi.py" --help ``` -Core `knowledge`, packaged `collect/read`, `doctor`, `plan/run` and -`status/report` delegate to the bundled runtime. The optional `skills` commands -and checkout-skill sources need PyYAML in the selected Python interpreter. +The bundled runtime serves `skills enter `, `knowledge`, packaged +`collect/read`, `doctor`, `plan/run` and `status/report` with the base +interpreter. Commands that parse skill frontmatter, including direct reads of +checkout skills, require PyYAML in the selected interpreter. Image validators and target engines retain their own optional dependencies. ## MindIE knowledge @@ -242,7 +245,7 @@ registers no MCP server, hooks, daemon or scheduled task. | --- | --- | | Python cannot be found or is too old | Select/install Python 3.10+ and use its executable consistently | | Incomplete bundle or version mismatch | Rebuild/reinstall the distribution; do not substitute an unrelated installed runtime | -| Optional skills dependency missing | Use packaged Collect/Read, or install PyYAML in the chosen interpreter | +| PyYAML is missing | Load a primary procedure with `skills enter ` or use packaged Collect/Read. Install PyYAML in the chosen interpreter for checkout/frontmatter operations. | | No knowledge match | Inspect the query/source selection; no match does not establish lack of an optimization | | Target backend unavailable | Fix the target environment; knowledge research remains usable | | Interrupted experiment | Inspect its existing status/evidence before deciding on a new run | diff --git a/docs/contracts/current-design.md b/docs/contracts/current-design.md index caad6eb3c..50b30e585 100644 --- a/docs/contracts/current-design.md +++ b/docs/contracts/current-design.md @@ -98,7 +98,7 @@ Acceptance: T01–T14, T22–T24, including the same-key conflict, finished repl ## D007 — target resource tree -Decision: `knowledge/data/{repo_packs//knowledge,skill_library,knowledge_packs,research_profiles}`; `pack.json` separately at `harness/data/repo_packs//pack.json`; the dashboard template at `Apps/views/templates/dashboard.html`. K01 migrates seven code modules only and moves no resources; K02 performs the three precise migration classes (importlib, checkout `Path`, plugin physical layout) with old logical pins/hashes preserved and without assuming a single loader. +Decision: `knowledge/data/skill_library (records, configuration, compiled provenance and skill owners)`; `pack.json` separately at `harness/data/repo_packs//pack.json`; the dashboard template at `Apps/views/templates/dashboard.html`. K01 migrates seven code modules only and moves no resources; K02 performs the three precise migration classes (importlib, checkout `Path`, plugin physical layout) with old logical pins/hashes preserved and without assuming a single loader. ## D008 — function ownership diff --git a/docs/contracts/diagrams/knowledge-read-and-internal-schema.mmd b/docs/contracts/diagrams/knowledge-read-and-internal-schema.mmd index 17b321719..6f46f06a8 100644 --- a/docs/contracts/diagrams/knowledge-read-and-internal-schema.mmd +++ b/docs/contracts/diagrams/knowledge-read-and-internal-schema.mmd @@ -1,11 +1,13 @@ flowchart TB subgraph Storage["storage layer: not the public read / collect delivery"] - S2["generated record schema 2
own fields + source_files resource hashes
no shared sources / registry hash"] - PROV["_provenance.json
schema_version, repository
source_revision, registry.path / sha256
path: repo_packs/inferrsi/knowledge/_provenance.json"] + S2["skill_library/_compiled/inputs.json
YAML headers + owner-local resource hashes"] + PROV["_provenance.json
schema_version, repository
source_revision, registry.path / sha256
path: skill_library/_compiled/_provenance.json"] end - S2 -->|"stored record"| CAT["_source_snapshot.resolve
called by inferrsi.catalog when reading the directory"] + S2 -->|"compiled inputs"| CAT["catalog.skill_views
reads registry and canonical metadata"] + OWN["skill_library/skill-index.json + owner metadata"] --> CAT PROV -->|"snapshot"| CAT CAT -->|"public schema 1: schema_version=1
sources[0].type / commit / blob URL
source_files with registry hash"| S1["parsed logical catalog record"] + AUTH["skill_library/records/<stable-id>.json"] --> S1 S1 -->|"KnowledgeReader.read / collect"| PUB["public delivery: KnowledgeBundle / Collection
entry record.json; revision is the Bundle content revision"] subgraph Outside["outside the ports"] PR["repository change in package / checkout"] diff --git a/docs/contracts/figures.md b/docs/contracts/figures.md index c21d614f6..eb82aa9cf 100644 --- a/docs/contracts/figures.md +++ b/docs/contracts/figures.md @@ -89,12 +89,14 @@ sequenceDiagram ```mermaid flowchart TB subgraph Storage["storage layer: not the public read / collect delivery"] - S2["generated record schema 2
own fields + source_files resource hashes
no shared sources / registry hash"] - PROV["_provenance.json
schema_version, repository
source_revision, registry.path / sha256
path: repo_packs/inferrsi/knowledge/_provenance.json"] + S2["skill_library/_compiled/inputs.json
YAML headers + owner-local resource hashes"] + PROV["_provenance.json
schema_version, repository
source_revision, registry.path / sha256
path: skill_library/_compiled/_provenance.json"] end - S2 -->|"stored record"| CAT["_source_snapshot.resolve
called by inferrsi.catalog when reading the directory"] + S2 -->|"compiled inputs"| CAT["catalog.skill_views
reads registry and canonical metadata"] + OWN["skill_library/skill-index.json + owner metadata"] --> CAT PROV -->|"snapshot"| CAT CAT -->|"public schema 1: schema_version=1
sources[0].type / commit / blob URL
source_files with registry hash"| S1["parsed logical catalog record"] + AUTH["skill_library/records/<stable-id>.json"] --> S1 S1 -->|"KnowledgeReader.read / collect"| PUB["public delivery: KnowledgeBundle / Collection
entry record.json; revision is the Bundle content revision"] subgraph Outside["outside the ports"] PR["repository change in package / checkout"] diff --git a/docs/contributing.md b/docs/contributing.md index 70ba27723..636fbde40 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -43,23 +43,7 @@ implementing module under `apps/host/platforms/`. That module owns the command name and the discovery boundary. Reuse the common AB/BA loop, cleanup and adjudication. See [extension contracts](architecture.md#where-to-extend). -For knowledge, edit one named JSON record under -`data/repo_packs//knowledge///` for contribution sources, -or `data/knowledge_packs/////` for -external sources, and add its navigation link to the owning README. See the -[three learning tiers](../src/inferrsi/knowledge/data/knowledge_packs/README.md). -`data/repo_packs/inferrsi/knowledge/` is the exception: it is generated from -`data/skill_library/` and must be refreshed, never edited. Change the skill body, -sidecar or registry, commit, then run -`python tools/build_skill_index.py --source-revision `; a -hand edit there is reverted by the next run and only `--check` reports it. -A one-skill edit regenerates only that skill's record, the shared -`_provenance.json` snapshot and the coverage view; see the -[knowledge contribution workflow](knowledge-contribution-workflow.md). -Do not maintain a second per-entry Markdown summary. -Keep IDs stable, omit PR links, retain source attribution and negative results, -and qualify new backends independently. See the -[knowledge maintenance guide](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/README.md#maintaining-the-pack). +For a finding, edit one [record](../src/inferrsi/knowledge/data/skill_library/records/README.md). For a procedure, edit its registered owner. Commit changed skill sources, then regenerate the two build inputs with `python tools/build_skill_index.py --source-revision `. Verify with the same command plus `--check`. The catalog constructs schema-1 views when requested. See the [knowledge contribution workflow](knowledge-contribution-workflow.md). Protected changes also require a current-head [knowledge authorship declaration](knowledge-model-policy.md). Every material AI @@ -70,19 +54,24 @@ is a required status on `main`, so a failing declaration blocks merging; the check establishes declaration compliance only, not execution settings or results. The [skill library](../src/inferrsi/knowledge/data/skill_library/README.md) owns complete -procedures, metadata, references and cases. Place platform-specific skills under -`hardware/ascend`, `hardware/cuda` or `hardware/host`; use `frameworks/`, -`repositories/`, `methods` or `workflows` for other primary owners. Maintain +procedures, metadata, references and cases. The seven primary inference skills live in +`tasks/`. The main agent loads one before searching for evidence. The historical router +supports explicit reads. Categorize supporting knowledge by its primary purpose as +system-specific implementation, engineering playbook, cross-stage workflow or casebook. Follow the library navigation for its subcategory. Maintain the registered stable IDs and their categorized paths in `skill-index.json`, then refresh -their generated abstracts with `tools/build_skill_index.py`. Keep all skill +the compiled build inputs with `tools/build_skill_index.py`. Keep all skill content in the canonical library. Preserve source state and `UNKNOWN` license declarations; moving content does not resolve licensing. Keep all library content in English. Engine procedures belong under -`repositories//` and the collection router under -`workflows/inference-engine-research/`, with their sub-skills and references. -Keep supporting Markdown and JSON with their owner; the library -root contains only `README.md` and `skill-index.json`. +`systems/engines//` and the collection router under +`workflows/research/inference-engine-research/`, with their sub-skills and references. +Keep a skill's supporting Markdown and JSON in its owner directory. At the +library root, `README.md` describes the library and `skill-index.json` maps +stable skill IDs to their owners. Place independent findings in `records/` +and shared search configuration in `config/`. The index builder writes the +two generated catalog inputs to `_compiled/`. See the +[directory contract](knowledge-layout.md#directory-contract). Catalog abstracts link to packaged bodies via `skill_documents.original` for migrated skills or `skill_documents.en` for engine procedures. Update affected source content and catalog records together. Source snapshot revisions @@ -93,7 +82,7 @@ licenses, local links and reads from a noneditable wheel installation. Use user-provided acceptance criteria, including those in a referenced accepted plan. When none are supplied, use -[performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/acceptance-criteria.md) +[performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md) to formulate evidence-backed targets. Retain partial requirements and label derived fields; record their source, calibration, uncertainty and freeze authority. Missing evidence stays unresolved, and estimates never lower an @@ -302,5 +291,5 @@ verification remains explicit. Workbench experiments use the the four CI lanes retain their current roles. For feedback-coupled models, stochastic audio and streaming/graph experiments, -use the [Omni experiment contract](../src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/omni-experiments.md) to select +use the [Omni experiment contract](../src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/omni-experiments.md) to select reference, preflight and executable per-trial qualification checks. diff --git a/docs/daily-learning-prompt.md b/docs/daily-learning-prompt.md index 70103b890..1e4d902bf 100644 --- a/docs/daily-learning-prompt.md +++ b/docs/daily-learning-prompt.md @@ -122,7 +122,7 @@ E. 检查、候选 PR 与晋升 - src/inferrsi/knowledge/data/skill_library/ 下按主要归属放置的候选技能: hardware//、frameworks//、repositories//、methods/ 或 workflows/; 技能库内容统一使用英文,参考资料放所属技能的 references/; - - src/inferrsi/knowledge/data/repo_packs//knowledge/ 的知识记录和导航; + - src/inferrsi/knowledge/data/skill_library/records/ 的知识记录和导航; - 本次 acquisition/learning manifest、评测摘要与性能候选队列的可追溯归档。 新增 vllm 或 vllm_ascend 记录时创建该 pack 的知识导航;知识身份不表示运行支持。 仓库级通用 SKILL.md 保持索引性质;细节进支持文件,体积预算读冻结配置。 diff --git a/docs/dashboard.md b/docs/dashboard.md index 54e1c167b..b79f38635 100644 --- a/docs/dashboard.md +++ b/docs/dashboard.md @@ -369,7 +369,7 @@ hits alone provide no passage for a reviewed decision. }, "decisions": [{ "use": "used", - "ref": "methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md", + "ref": "casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md", "event_line": 73, "passage_line": 18, "decision_line": 108, @@ -439,7 +439,7 @@ for the generated evidence and its interpretation. To follow one material across events and then see its reviewed uses, set `record` to the portable JSON: ```sh -ref=methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md +ref=casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md jq --arg ref "$ref" '{ returns: [.events[] | {event: .id, action, query, passages: [.delivery.passages[]? | select(.ref == $ref)]} diff --git a/docs/drama3-npu-readiness.md b/docs/drama3-npu-readiness.md index db6c84a6c..d06fca21a 100644 --- a/docs/drama3-npu-readiness.md +++ b/docs/drama3-npu-readiness.md @@ -31,9 +31,9 @@ These records describe the same GPU design family. They share lineage and do | Source | Pin | Role here | Status | |---|---|---|---| | [david6666666/drama-3-single-b300-inference](https://github.com/david6666666/drama-3-single-b300-inference/tree/58a1882a4d941f8d21bfce6be37d3d5e7c9fde80) | `58a1882a` (RFC-pinned) | GPU reference engine motivating the NPU questions; Python engine with persistent request state, CUDA Graphs, incremental audio decoding, native FP8 routed experts | external source; inspected only at the RFC-selected commits below | -| [`vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap`](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/audio/concurrency-scaling-state-and-overlap.json) | vllm-omni `fish_omni@d6ed4f4` | Existing imported record for the same audio design family (stage/state ownership, mixed prefill, codec overlap, workspace isolation, prefix reuse, FP8 MoE) | `imported`, `locally_verified=false`, CUDA | +| [`vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap`](../src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap.json) | vllm-omni `fish_omni@d6ed4f4` | Existing imported record for the same audio design family (stage/state ownership, mixed prefill, codec overlap, workspace isolation, prefix reuse, FP8 MoE) | `imported`, `locally_verified=false`, CUDA | | [`evidence/drama3-b300-scale150/`](../evidence/drama3-b300-scale150/) | issue #10 campaign | Frozen GPU plan + validator + diagnostic lessons from the GPU campaign | campaign **INCONCLUSIVE**; GPU-only | -| [`case.drama3-rejected-directions`](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md) | InferRSI `d51c941` diagnostic ledger | Authoritative casebook entry for the campaign's rejected directions and invalid controls (async overlap/shared capture workspace, production prefix reuse, padding-history harness mismatch); reused here, not restated | `historical_observation`; derived from the campaign above, so same lineage | +| [`case.drama3-rejected-directions`](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md) | InferRSI `d51c941` diagnostic ledger | Authoritative casebook entry for the campaign's rejected directions and invalid controls (async overlap/shared capture workspace, production prefix reuse, padding-history harness mismatch); reused here, not restated | `historical_observation`; derived from the campaign above, so same lineage | RFC-selected commit chain inspected for this supplement (direct inspection limited to concrete questions; no broad history scan, no readset expansion): @@ -54,8 +54,8 @@ TorchNPU allocator skills (`torchnpu-device-memory-lifecycle`, `torchnpu-pinned-host-memory-lifecycle`), measurement/evidence skills (`inference-benchmark-design`, `dense-performance-evidence`, `rsi-evidence`), and the omni/audio records -([`stage_ownership_and_search`](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/omni/stage-ownership-and-search.json), -[`streaming_serving_measurement`](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/audio/streaming-serving-measurement.json)). +([`stage_ownership_and_search`](../src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.omni.stage_ownership_and_search.json), +[`streaming_serving_measurement`](../src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.audio.streaming_serving_measurement.json)). --- @@ -140,7 +140,7 @@ Grounded in commit [`a6e660cc`](https://github.com/david6666666/drama-3-single-b - **Control discipline:** before attributing a target mismatch to production math, reset complete physical input rows and run an A/A control; the GPU campaign's initial prefix-reuse comparison differed only in private padding - history ([casebook row](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md)). + history ([casebook row](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md)). ### Representative path: one decode step @@ -157,7 +157,7 @@ Grounded in commit [`a6e660cc`](https://github.com/david6666666/drama-3-single-b parity against a higher-precision reference (`6ff8d808`). The GPU dedicated- capture finding and its limits (one reproducer, not every historical WER failure) stay in the - [casebook](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md); + [casebook](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md); NPU workspace ownership must be established on the target runtime. Reuse the imported record's L0–L4 `validation_plan` for the broader matrix; CPU diff --git a/docs/dsh-plugin.md b/docs/dsh-plugin.md index d8f90237d..8b76b2a66 100644 --- a/docs/dsh-plugin.md +++ b/docs/dsh-plugin.md @@ -37,9 +37,9 @@ Put `pnpm` on `PATH` for plugin management. For DSH and Node.js setup, see the [DeepSeek Harness bundle guide](https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/user/develop/basic/publish.md). Python 3.10+ must be available to the session's terminal. Core InferRSI -operations run from the bundled runtime. The optional -`skills` CLI and explicit checkout-skill sources require PyYAML. Git and the -prepared target environment are needed for comparisons. +operations and `skills enter ` run from the bundled runtime with the base +interpreter. Install PyYAML for commands that parse skill frontmatter, including +explicit checkout-skill sources. Comparisons also require Git and a prepared target. ## Start an iteration from the form @@ -232,7 +232,11 @@ the launcher is `scripts/inferrsi.py`. - Use `--plugin-info` to check the selected interpreter, bundled runtime version, and content SHA-256. An unbuilt source checkout can report `0+unknown` as the runtime version. -- Use `find ""` to search Cases, Skills and Records. Follow the +- For an inference engineering task, select the primary skill by the requested + outcome. For example, load `skills enter task-performance` for a performance + goal and read its returned procedure. Record the actual workload, device, + engine and benchmark before choosing a supporting query. +- Use `find ""` to search supporting Cases, Skills and Records. Follow the emitted `read` command to inspect the selected material and its revision. On a remote connection, use `inferrsi_remote` with `operation: "cli"` and `args: ["find", "", "--json"]`. Send the hit's `read_args` as diff --git a/docs/knowledge-base.md b/docs/knowledge-base.md index d3511c1aa..94ff09bb1 100644 --- a/docs/knowledge-base.md +++ b/docs/knowledge-base.md @@ -11,25 +11,84 @@ separate work, not an implemented capability of this knowledge base. ## Directory and responsibility map +The [illustrated layout and search guide](knowledge-layout.md) shows the task +entry, physical directories, evidence lookup and old-to-new bundle counts. + +The main agent selects one of the [seven primary skills](knowledge-layout.md#seven-deliverables) by the requested outcome and uses `inferrsi skills enter ` to load its procedure before evidence search. Its decision points use the observed run to choose supporting knowledge. For direct subject browsing, the first library directory answers the reader's question. The second +identifies a subsystem, an engineering activity, a workflow stage group or +the scope of a case collection. The registry maps each stable ID to a single +bundle; backend and evidence state remain separate metadata. + +```mermaid +flowchart LR + T["tasks · seven inference deliverables"] --> L["Supporting knowledge"] + L --> S["systems · How does it work?"] + L --> P["playbooks · How do I perform a task?"] + L --> W["workflows · How do the stages fit together?"] + L --> C["casebooks · What happened before?"] + S --> S1["platforms / frameworks / engines"] + P --> P1["measurement / planning / optimization"] + W --> W1["research / governance / maintenance"] + C --> C1["inference / ascend"] +``` + +A platform-specific operation such as Ascend graph capture belongs in +`systems/platforms/ascend/`. The Ascend optimization casebook belongs in +`casebooks/ascend/`. A cross-platform benchmarking method belongs in +`playbooks/measurement/`. Engine-specific examples can still ship with their +parent procedure. Frozen run artifacts remain under `evidence//`. + +For existing automation, resolve an ID through `skill-index.json` or use +`inferrsi skills show `. Raw paths containing the old category need +the new location. A reference pinned to an earlier commit uses that +commit's original path; regenerate resource pins when adopting a new bundle. + +The seven primaries and the compatibility router declare the task-entry discovery role. +The [legacy router](../src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/SKILL.md) +still supports explicit reads of historical route resources. Default Find excludes these +eight entries before computing IDF and ranks the technical evidence owners. +Use [the migration manifest](skill-layout-migration.csv) to resolve raw paths. +Retained material can be replayed from its saved bytes or matching source +snapshot. A digest-only reference requires its original package, checkout or +retained store revision. Never resolve an old digest to the new bundle. + +The manifest records the cutover from main at `9b418d7` and is historical. +All 79 readable original checkout skills changed bundle revision when moved. +The seven case identities retained their previous checkout revisions even +though their resource locations moved. The legacy `husky` skill failed the +same source-identity validation on both snapshots, marked +`unavailable_at_cutover`. Its registry path can still be resolved. +Run `python tools/check_skill_layout_migration.py --before --after ` +against these snapshots to verify every path and pin and reject old skill +digests at the new checkout. Validate privately retained campaign captures +and store revisions against their own saved material before rollout. + +On Windows hosts enforcing the 260-character path limit, use a short checkout +and virtual-environment root for this layout. A wheel under a long temporary +installation path reproduced a failed read of a nested reference on such a host. + + ```text src/inferrsi/knowledge/data/skill_library/ README.md Library navigation - skill-index.json Stable IDs → categorized paths/backend (79 skills) - hardware/{ascend,cuda,host}// Hardware/platform-specific complete skills - frameworks/// Framework-owned complete skills - repositories// Engine entry, sub-skills and focused complete skills - methods// General optimization/analysis complete skills - workflows// Cross-repository research, evidence and skill lifecycle + skill-index.json Stable IDs → current bundle path and backend + tasks/task-*/ Seven registered primary procedures + tasks/inference-task-router/ Compatibility owner for historical route resources + systems/platforms/{ascend,cuda,host}/ Device and host implementation guidance + systems/frameworks// Framework-specific behavior and procedures + systems/engines// Engine entry, sub-skills and resources + playbooks/{measurement,planning,optimization}/ Reusable engineering tasks + workflows/{research,governance,maintenance}/ Cross-stage processes + casebooks/{inference,ascend}/ Prior decisions, outcomes and retry conditions /references/ Owner-local supporting documents and source inventory - //SKILL.md Name, description, substantive procedure - //metadata.json Provenance, state and relationships + //SKILL.md Procedure; frontmatter holds provenance, state and relationships //references/ Owner's mechanisms, readsets and comparisons //cases/ Owner's scoped cases (optional) src/inferrsi/knowledge/catalog.py Installed knowledge reader src/inferrsi/knowledge/skill_catalog.py Packaged or custom-root skill reader -src/inferrsi/knowledge/data/repo_packs/ Contribution records and existing public IDs -src/inferrsi/knowledge/data/repo_packs/inferrsi/ Generated records linking full packaged skills -src/inferrsi/knowledge/data/knowledge_packs/ Dependency and peer records +src/inferrsi/knowledge/data/skill_library/records/.json Authored engineering findings +src/inferrsi/knowledge/data/skill_library/config/ Aliases and research profiles +src/inferrsi/knowledge/data/skill_library/_compiled/ Generated YAML headers and Git provenance src/inferrsi/harness/data/repo_packs//pack.json Target descriptor, read by repository_pack evidence// Frozen plans, results, patches and validators docs/reference-matrix.md Sources, inspected scope and extraction @@ -61,18 +120,18 @@ metadata authority or maintained matrix. | Item | Authority | Compatibility rule | |---|---|---| -| New local-skill profile | `SKILL.md`: name, description and procedure; `metadata.json`: `profile: local-skill-v1`, kind, origin, state, provenance, applicability and relationships | Name stays the existing directory ID; listing combines metadata without reading procedure/resource bodies | +| New local-skill profile | `SKILL.md` contains the procedure and frontmatter. The `metadata.inferrsi` string contains structured JSON for the local profile, kind, origin, state, provenance, applicability and relationships. | Name stays the existing directory ID; listing combines metadata without reading procedure/resource bodies | | Legacy local frontmatter | Existing frontmatter when no new profile is declared | Five inherited skills stay unchanged; no invented candidate state | -| Legacy sidecars | Existing keys retained unless explicitly normalized | Conflicting duplicate fields fail visibly; matching historical fields can be read | -| Catalog JSON | Existing record owns ID, tier, scope, recommendations, attribution and limitations | Do not migrate it into local sidecars or relabel imported records | +| Historical/custom checkout sidecars | The reader still accepts adjacent `metadata.json` | Conflicting duplicate fields fail visibly; matching historical fields can be read | +| Catalog JSON | Existing record owns ID, tier, scope, recommendations, attribution and limitations | Authored findings retain their own identity and attribution | | Engine procedures/resources | Imported English library and source manifest | Preserve licenses, source relationships and installed access; catalog abstracts link the complete English procedures | | Case | One owner-local case document plus structured identity/source record | Raw evidence stays frozen; case interpretation may evolve without rewriting results. Source identities and digests live only in the record; the document links it and cites the same source URLs. The record lists its own basename in `required_resources`, so that link resolves inside the read bundle | The 43 branch-added candidate headers were normalized without replacing their procedures. All structured frontmatter fields, including tags, synchronization -lineage, unknown license and source-pack notes, are retained in sidecars. Existing +lineage, unknown license and source-pack notes, are retained in their owner documents. Historical sidecar name/status duplication agreed with the frontmatter; normalization found -no contradictory values. Sidecars omit fields equal to the reader defaults: +no contradictory values. Local profiles omit fields equal to the reader defaults: empty `sources` and `references`, and the former shared placeholder `applicability`; absent sources still read as explicit unknown provenance. Abbreviated historical repository pins remain explicitly @@ -105,13 +164,13 @@ has no specific advice and does not fall back; only an absent field produces the generic navigation defaults. `limitations` is not owner-only. The generator emits four sentences of its own and -then appends the sidecar's `limitations` entries. `failure_modes` is therefore a +then appends the skill's `limitations` entries. `failure_modes` is therefore a separate field rather than more `limitations` entries. No mechanical allowlist restricts `applicability` keys. The packaged library uses nine distinct keys: the seven above plus `architectures` and `excluded_mechanisms`, -which appear only in `hardware/cuda/reason-hopper-kernels`. A closed key set would -reject that sidecar or have to grandfather it. Revisiting this is proposed for +which appear only in `systems/platforms/cuda/reason-hopper-kernels`. A closed key set would +reject that metadata or have to grandfather it. Revisiting this is proposed for batch 4 of issue #91 and is not agreed. ## Sources and revisions @@ -123,7 +182,7 @@ Its supplementary `Resource` tables describe guidance without claiming numbered layer coverage. A missing mapping means unknown, not that a procedure applies everywhere or nowhere. -A sidecar may declare `searchspace_not_applicable` as a non-empty reason when its +A skill may declare `searchspace_not_applicable` as a non-empty reason when its procedure does not address any of the five resource layers. The reason identifies the procedure's actual scope and its source section. Omit the field for positive or unanalyzed mappings; it is not a second mapping table. Device-independent @@ -187,16 +246,16 @@ the K0/K1 readers above do not enforce them by themselves. ## K1 integration and shared resources -- [Repository-derived procedure](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md): preserve +- [Repository-derived procedure](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md): preserve pinned scheduler/connector contracts and untested concrete-backend limits. -- [Reviewed paper mechanism](../src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/SKILL.md): +- [Reviewed paper mechanism](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/SKILL.md): empirical calibration, joint-SLO shortlist and near-capacity queueing risk; source paper is not reread or reproduced by this organization work. -- [Internal admission/drain case](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.md): +- [Internal admission/drain case](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md): observation, original action, interpretation, counter-signals and original evidence pointers remain separate. Campaign INCONCLUSIVE is retained. -- [Benchmark design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) and - [evidence classification](../src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md) consume +- [Benchmark design](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) and + [evidence classification](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/dense-performance-evidence/SKILL.md) consume that same case. No `shared/` namespace or duplicate case database is necessary. Local `resource` access remains restricted to the selected skill. Declared @@ -219,37 +278,35 @@ standalone export: retain the declared dependencies or the complete library. All 48 former root skills now have one canonical home under [`src/inferrsi/knowledge/data/skill_library/`](../src/inferrsi/knowledge/data/skill_library/README.md). -Complete procedures, sidecars, references and cases are packaged, with 14 Ascend, -3 CUDA and 5 host skills under `hardware/`; the rest are grouped as frameworks, -repositories, methods or workflows. The 21 engine and research-router skills -use the same ownership layout under `repositories//` and -`workflows/inference-engine-research/`, including their sub-skills and references. -The library contains 142 `SKILL.md` files, all in English. The registered -[Lithos Metal candidate](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/SKILL.md) +Complete procedures, references and cases are packaged together. Browse by +system, engineering task, workflow or prior outcome. Backend is recorded independently. The 21 engine and research-router skills +use the same ownership layout under `systems/engines//` and +`workflows/research/inference-engine-research/`, including their sub-skills and references. +The library contains 151 `SKILL.md` files, all in English. The registered +[Lithos Metal candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/SKILL.md) adds three owner-local subskills for compiler/fusion, generation state and serving/cache contracts; it adds no executable repository target. The collection overview, import notes and source manifest belong to the research -workflow's references; Strata design-space analysis belongs to Strata. The library -root contains only navigation and the registry. +workflow's references; Strata design-space analysis belongs to Strata. The library root holds the registry, configuration, compiled inputs, authored findings and skill directories. -The [generated catalog](../src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/README.md) -projects the 79 registered skills into the contribution catalog. Its records +The [runtime catalog](../src/inferrsi/knowledge/data/skill_library/README.md) +projects the 88 registered skills into the contribution catalog. Its records use `inferrsi..skills.` IDs, with `ascend` and `cuda` for scopes that depend on those backends and `common` for the remaining skills. Each `skill_documents.original` points to the full packaged procedure. The `skills` CLI retains the original hyphenated skill IDs and defaults to this -79-ID registry; engine procedures remain available through library navigation +88-ID registry; engine procedures remain available through library navigation and catalog document pointers. -The registered [PyTorch](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/SKILL.md) -and [TorchNPU](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md) -operating graphs each retain eight child procedures, their sidecars, references, +The registered [PyTorch](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/SKILL.md) +and [TorchNPU](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/SKILL.md) +operating graphs each retain eight child procedures with metadata, references, scripts and source notices under one root. The children are available through root resource reads and complete normalized bundles; they add no separate registry IDs or executable repository targets. Earlier compiler, DCP and TorchNPU allocator owners retain their independent source revisions. All new profiles use `local-skill-v1`: only name and description are in YAML; -the AREX license, invocation and operating-role fields remain in sidecars. +the AREX license, invocation and operating-role fields remain in the skill documents. An upstream AREX license check of a temporary frontmatter projection is distinct from final InferRSI format checks, backend execution and managed DisCo import. Required accelerator/distributed gaps remain `BLOCKED_REQUIRED_BACKEND`; @@ -264,7 +321,7 @@ backend gaps and verification scope. Repackaging the unchanged runtime bytes does not rerun those checks or establish new backend, quality or performance results. The process fixture trees are not part of this delivery. -The registered [h3.c runtime audit](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md) +The registered [h3.c runtime audit](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md) owns MiniMax-H3 component/lifetime, AdaLN, precision/SSD mechanisms and the retained Apple feasibility/header-inventory research. Its source-derived candidate state and imported header observations remain separate from native @@ -286,29 +343,24 @@ record, not promotion or verification of the source skill. Backend placement describes content scope; `common` and `skills.discovery` imply no cross-hardware qualification. `inferrsi` is a knowledge source, not an executable target. -The CUDA-owned [Hopper kernel reasoning candidate](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md) +The CUDA-owned [Hopper kernel reasoning candidate](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/SKILL.md) adds pinned KDA/KernelWiki source screening, selected primary-code qualification and architecture-specific synchronization/resource/FP8 procedures. Its coverage inventory distinguishes inspected bodies from source headers and excluded SM100 recipes; native execution and efficacy remain not_run. -The complete library is the single content owner. Maintain stable paths and -backend classifications in `skill-index.json`, then refresh generated records -and navigation with `tools/build_skill_index.py`; do not hand-edit generated -cards. Stored cards keep only their own fields and resource hashes, and the shared -`_provenance.json` snapshot holds the source revision and registry hash; the -catalog resolves them into the public records -([contribution workflow](knowledge-contribution-workflow.md)). Its source-revision and consistency checks prevent attributing edited -metadata to an older snapshot. Preserve existing states, provenance, notices, +Maintain stable skill IDs, paths and backends in `skill-index.json`. After committing changed skill sources, run `tools/build_skill_index.py` to refresh the two compiled inputs. The catalog assembles public records from the registry, canonical metadata, YAML headers and source fingerprints. Indexed material selection verifies the recorded hashes against the resources it reads. [Contribution workflow](knowledge-contribution-workflow.md) + +Preserve existing states, provenance, notices, `UNKNOWN` licenses and the two matrices' authority. Migration and installed-package checks do not establish licensing, empirical verification or automatic execution. ## Negative results as diagnostic options The existing Case authority now also holds scoped rejected directions for -[Qwen B300](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md), -[Qwen Ascend](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md) -and [Drama3 B300](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md). +[Qwen B300](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md), +[Qwen Ascend](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md) +and [Drama3 B300](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md). The records cite exact source summaries; they add no hardware validation. Observation, original disposition and newly suggested revisit checks stay separate. Raw evidence remains at its existing frozen owner. @@ -348,17 +400,17 @@ perf-loop, fixed skill count, global numeric policies and separate case registry are not imported. Exactly two primary matrices remain; HBM is not host RAM and H2D/IPC/team coordination do not establish Network coverage. -The registered [vllm-metal candidate](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/SKILL.md) adds selected platform/ABI, KV/offload lifecycle and speculative/deferred/dispatch procedures. Installed pure API/parser evidence is partial; required Apple native/model/quality/service evidence remains blocked. It is portable AREX-informed repository integration, not DisCo CLI execution or managed import. +The registered [vllm-metal candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/SKILL.md) adds selected platform/ABI, KV/offload lifecycle and speculative/deferred/dispatch procedures. Installed pure API/parser evidence is partial; required Apple native/model/quality/service evidence remains blocked. It is portable AREX-informed repository integration, not DisCo CLI execution or managed import. -The registered [mistral-rs candidate](../src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/SKILL.md) adds selected Metal paged-attention layout, metadata/workspace and preparation/evidence contracts. Rust tooling and required Apple native/model evidence are unavailable; offline helpers and source test definitions are not native execution or candidate promotion. +The registered [mistral-rs candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/SKILL.md) adds selected Metal paged-attention layout, metadata/workspace and preparation/evidence contracts. Rust tooling and required Apple native/model evidence are unavailable; offline helpers and source test definitions are not native execution or candidate promotion. -The [mlx-swift-lm candidate](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/SKILL.md) owns selected Swift generation-lifetime and recurrent native/ops contracts, with source/observation limits preserved. No Swift compile, original native/host-control test, actual Metal/model/quality/service run is implied by its helpers or packaged discovery. +The [mlx-swift-lm candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/SKILL.md) owns selected Swift generation-lifetime and recurrent native/ops contracts, with source/observation limits preserved. No Swift compile, original native/host-control test, actual Metal/model/quality/service run is implied by its helpers or packaged discovery. -The [mlc-llm candidate](../src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/SKILL.md) separates artifact metadata, supported consumers, Apple targets, pinned TVM runnable/runtime boundaries and compiled/runtime KV configuration. Canonical hashes, serializable fallback or estimated limits are not native/model/quality/fit/service evidence. +The [mlc-llm candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/SKILL.md) separates artifact metadata, supported consumers, Apple targets, pinned TVM runnable/runtime boundaries and compiled/runtime KV configuration. Canonical hashes, serializable fallback or estimated limits are not native/model/quality/fit/service evidence. -The registered [h3-metal FastH3 audit](../src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) supplements earlier h3.c residency with fork-specific effective configuration, tensor/schedule semantics and VSA eligibility/fallback. Its offline declaration arithmetic and label probes do not qualify native conversion, sparse attention, media, memory fit or performance; required Apple/model evidence remains blocked/not_run. +The registered [h3-metal FastH3 audit](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) supplements earlier h3.c residency with fork-specific effective configuration, tensor/schedule semantics and VSA eligibility/fallback. Its offline declaration arithmetic and label probes do not qualify native conversion, sparse attention, media, memory fit or performance; required Apple/model evidence remains blocked/not_run. -The registered [local-dream candidate](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/SKILL.md) +The registered [local-dream candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/SKILL.md) owns selected Android desired/serving/readiness and unauthenticated-host boundaries, native diffusion dispatch/cancellation/cache and DiT/QNN packaging contracts. Its helper compares declarations only; required Android/Qualcomm, diff --git a/docs/knowledge-contribution-workflow.md b/docs/knowledge-contribution-workflow.md index 3671d5283..8cd2c667b 100644 --- a/docs/knowledge-contribution-workflow.md +++ b/docs/knowledge-contribution-workflow.md @@ -1,20 +1,37 @@ # Knowledge contribution workflow This is the short path for a change to one skill in `src/inferrsi/knowledge/data/skill_library/`. -It implements the representation accepted in +It builds on the earlier projection format discussed in [issue #355](https://github.com/InferMatrixLab/InferRSI/issues/355#issuecomment-6730172305). ## What you write and what tools derive | You write | Tools derive | | --- | --- | -| Skill body, references, cases, `metadata.json`, registry entry | Per-skill catalog records, resource hashes, `_provenance.json`, navigation | +| Skill document (including frontmatter), references, cases, registry entry | Runtime catalog views, compiled YAML headers and resource hashes | | Matrix rows in `docs/reference-matrix.md` and `docs/searchspace-matrix.md` | `docs/skill-coverage.md` | | The PR's knowledge authorship declaration: every material AI author's actual tool, model and effort, or `human_only` | The head SHA and the exact protected, policy and generated path lists | Tooling never infers authorship, model, effort, human contribution or an exception decision. +A skill owner edits one `SKILL.md`. Put structured InferRSI fields in the +`metadata.inferrsi` literal string. The host sees a string-valued annotation, +and InferRSI reads its JSON object. For example, a candidate owner can declare: + +```yaml +--- +name: example-skill +description: "Inspect an inference configuration." +metadata: + inferrsi: | + {"status": "candidate", "tags": ["configuration"]} +--- +``` + +Write the procedure after the closing delimiter. Add references or cases only +when the procedure needs those resources. + ## Steps 1. Edit the canonical skill files and the matrices, then commit them. This commit is @@ -30,7 +47,7 @@ decision. The recorded prefix is `src/inferrsi/knowledge/data/skill_library/`, the same path as the current checkout. Commit the skill and registry edits first, then - regenerate from that commit. Do not hand-edit generated records. + regenerate from that commit. The generator writes both files under `skill_library/_compiled/`. 3. Commit the generated changes. List the declaration paths for the final head: @@ -43,73 +60,10 @@ decision. 4. Write the declaration in the PR body with the reported `head_sha`. Update it after every new commit. -## Stored format +## Packaged format -Stored skill records are schema 2. They keep each skill's own fields and resource -hashes. The shared `knowledge/_provenance.json` holds the full source revision and -the registry hash: +The only authored engineering records are `skill_library/records/.json`. Their source tier, provenance and validation state travel with the record. For registered skills the catalog assembles the public schema-1 view from the registry, the canonical metadata and two generated inputs. `_compiled/inputs.json` stores each owner's original YAML header and Git file hashes keyed by its local resource names. `_compiled/_provenance.json` stores the source revision and registry hash. The body remains in SKILL.md. The base wheel can reconstruct views without Git or PyYAML. Indexed material selection checks actual resource hashes against the declaration. -```json -{ - "schema_version": 1, - "repository": "https://github.com/InferMatrixLab/InferRSI", - "source_revision": "<40-hex commit>", - "registry": {"path": "src/inferrsi/knowledge/data/skill_library/skill-index.json", "sha256": "<64-hex>"} -} -``` +A concurrent edit to two owners can conflict in the compiled files. Resolve the canonical edits first, commit their combined sources and regenerate. The deterministic checker compares the generated files with their committed contents. Old raw projection paths were removed. Consumers can look up the same stable ID with `knowledge show`. -`registry.path` and the `source_files` keys name the library at the pinned -`source_revision`. After the explicit re-pin that path is -`src/inferrsi/knowledge/data/skill_library/`, matching the current checkout. -`skill_documents.original` and installed bundle keys stay at the package-relative -prefix `data/skill_library/`. A later move again needs an explicit new revision -that regenerates every record; do not hand-edit the generated files. - -`inferrsi.knowledge.catalog` resolves each stored record into a public schema-1 record with -the same fields and field order as before: the `sources` entry with the full commit -URL, and `source_files` including the registry hash. Search, `knowledge show`, -Collect and installed wheels keep reading schema 1. The generator refuses to write a -record that does not resolve back to the record it rendered. A pack with -`_provenance.json` holds only schema-2 records: the catalog rejects any other JSON -file in it, including a nested `_provenance.json`. Packs without a snapshot keep -hand-written schema-1 records. - -The one-time migration regenerated all 80 records at the previous pin -`d648f25732dee1c974da62a94c46a642ad52cef8`; each resolved record was byte-identical -to the previous stored record. It then moved the pin to -`56c500367d5495fdd9f0e35030c9cd957f70eac5`, the squash commit on `main` with the -same skill-library bytes, because the previous pin is a pre-squash branch commit -and not an ancestor of `main`. Resolved records therefore differ from the previous -records only in `sources[0].commit` and `sources[0].url`; every content hash and -the registry hash are unchanged. To check the stored hashes, run -`python tools/build_skill_index.py --source-revision --check` with -the `source_revision` from `_provenance.json`. - -## What changes per contribution - -Before the shared snapshot, every skill record repeated the repository-wide source -commit and registry hash, so a one-skill edit rewrote all 80 records. Measured on -`main` at `56c50036` with one appended line in `performance-ceiling-analysis`, the -old generator changed 82 generated files (80 records, the README and the coverage -view). The new generator changes 3: that skill's record, `_provenance.json` and the -coverage view. Editing the registry or a declared status also changes the README. - -Two PRs that change different skills share only `_provenance.json` and the coverage -view's input digest among generated files, so those are the expected conflicts. -Integrate the second on top of the first, keep both canonical changes, commit them -as the new source revision and regenerate; regeneration rewrites both shared files -and no unrelated record is hand-edited. Divergent edits to the same skill, or to the -same matrix row, remain ordinary content conflicts that must be resolved by hand. - -## Limits - -- After a squash merge, the recorded source revision is a branch commit that is not - an ancestor of `main`. The packaged content still resolves, and `--check` still - works when that commit is available. Git-backed SkillEval planning reports the pin - as unavailable. This already happened with the previous format: the projections - on `main` before this change pinned a squashed branch commit. -- Raw stored records changed from schema 1 to schema 2. Code that reads the JSON - files directly must use `inferrsi.knowledge.catalog` or - `inferrsi.knowledge._source_snapshot.resolve`. -- Generated files are protected knowledge paths. Regenerating them is material - authorship work and needs a valid declaration. +After a squash merge, the recorded source revision can fall outside the new main branch history. The installed wheel still contains its inputs. Git-backed SkillEval planning requires a reachable source pin and reports an unavailable check otherwise. diff --git a/docs/knowledge-layout.md b/docs/knowledge-layout.md new file mode 100644 index 000000000..8fb7f865b --- /dev/null +++ b/docs/knowledge-layout.md @@ -0,0 +1,183 @@ +# Knowledge layout and retrieval + +Start from the work an engineer needs to finish. The library has one physical root, one source location per registered skill and one authored file per engineering finding. Search and the catalog are two views over those sources. The migration manifest maps historical checkout paths and revisions. [Migration manifest](skill-layout-migration.csv) + +To check the effect of this layout on search, compare unmodified main with the PR on the same machine. The [search comparison](../tools/compare_find_layouts.py) checks ordered results across both trees and times the end-to-end CLI. The [task queries and acceptance criteria](../tests/fixtures/find/layout-queries.json) define the workload. + +## 1. Task layers and evidence + +```mermaid +flowchart TB + Q["Inference goal at the main agent entrypoint"] --> P["Select and load ONE primary SKILL.md before evidence search"] + P --> A["Day-0 model · performance · debugging · accuracy"] + P --> B["Version alignment · derived engine · independent engine"] + A & B --> R["Record actual vendor / device / backend / OS, model and stages, engine and revision,
script and flags, benchmark and gates, diagnosis and allowed search space"] + R --> D["Primary routine: observation → supporting skill → evidence to collect → next test"] + D --> S["systems · 53
platforms / frameworks / engines"] + D --> M["playbooks · 17
measurement / planning / optimization"] + D --> W["workflows · 8
research / governance / maintenance"] + D --> K["casebooks · 2
previous outcomes and retry conditions"] + S & M & W & K --> V["Match the evidence to the actual run and test the goal"] + V --> O["Gates met: cite artifacts and scope"] + V --> N["Open question: run the smallest discriminating test or record the budget blocker"] +``` + +![Task layers and evidence (Chinese)](assets/knowledge-layout/01-task-layers.png) + +## 2. One physical knowledge root + +```mermaid +flowchart LR + D["src/inferrsi/knowledge/data/"] --> L["skill_library/ · sole child"] + L --> T["tasks/"] + T --> P["7 primary owners: task-day-0, task-performance, task-debug, task-accuracy,
task-version-alignment, task-derived-engine, task-independent-engine"] + T --> H["inference-task-router/ · legacy card paths"] + L --> S["80 supporting owners in systems/, playbooks/, workflows/, casebooks/"] + P & H & S --> O["One SKILL.md owns procedure and typed fields · 88 registered IDs"] + L --> F["records/<stable-id>.json · 48 authored findings"] + L --> CFG["config/ · aliases and research profiles"] + L --> REG["skill-index.json · stable ID → owner and backend"] + L --> BUILD["_compiled/ · inputs.json and _provenance.json"] + O & REG -.->|fingerprint resources and compile host headers at the source commit| BUILD +``` + +![One physical knowledge root (Chinese)](assets/knowledge-layout/02-physical-layout.png) + +## 3. Search paths and evidence delivery + +```mermaid +flowchart TB + Q["Main agent receives inference task"] --> P["Choose primary by outcome; skills enter loads its full procedure"] + P --> R["Capture actual run, acceptance gates and search budget"] + R --> D["Observation triggers a supporting question"] + D -->|Known owner| I["Explicit read / resource / capture by ID and revision"] + D -->|Unknown owner| F["Find lexical ranker · ranked Collect / campaign capture share selector"] + D -->|Need catalog scope| C["knowledge search · filter by repo, backend, scenario or tier"] + C -->|Query| F + C -->|No query| B["Catalog by ID · 48 findings + 88 skill views
default limit 10 · use --limit 136 for all"] + F --> A["Rank the existing searchable fields and procedure text
Exclude 7 primary skills and legacy router before IDF and limits
default 135 evidence owners · 80 skills + 48 findings + 7 cases"] + I -->|Authored finding| E["Validate selected record and its dependencies
retain linked-library bundle and pin checks when skill documents are declared"] + I -->|Other exact owner| V["Check revision, device, model, flags and measurement against this run"] + A --> H["Find card · limited caveats and withheld count
next_read_content_bytes measures content at Find time"] + E & H & B --> V + V --> Z["Read the procedure and structured metadata, or Collect within budget
Pinned resource read returns original SKILL.md bytes"] + Z --> X["Meets gates → cite reproducible evidence and scope"] + V --> G["Gap → name missing evidence; test the next distinction or record exhausted budget"] + G -.->|budget remains| D +``` + +![Search paths and evidence delivery (Chinese)](assets/knowledge-layout/03-search-and-delivery.png) + +## 4. Before and after + +```mermaid +flowchart LR + subgraph OLD["main · before #516"] + A["skill_library/
hardware / frameworks / repositories / methods / workflows"] + B["repo_packs + knowledge_packs
28 contribution + 9 dependency + 11 peer findings"] + C["A separate metadata.json beside skill procedures
Generated per-skill JSON records, navigation and shared provenance"] + D["data/query_aliases.json + research_profiles/"] + end + subgraph NEW["PR · knowledge/data/skill_library/"] + N["tasks/ · 7 first-class primary owners + 1 compatibility router
systems / playbooks / workflows / casebooks · 53 + 17 + 8 + 2"] + R["records/ · 48 authored findings
IDs, tiers, sources and verification retained"] + V["Procedure and metadata in one SKILL.md
88 runtime views · 1 registry + 2 generated inputs"] + P["config/ · aliases + profiles"] + end + A -->|move 80 supporting owners and add primary skills| N + B -->|retain IDs and migrate linked paths| R + C -->|generate views on demand| V + D --> P +``` + +![Before and after (Chinese)](assets/knowledge-layout/04-before-after.png) + +## Directory contract + +```text +src/inferrsi/knowledge/data/ +└── skill_library/ + ├── tasks/ + │ ├── task-day-0/ SKILL.md contains the procedure and its structured fields + │ ├── task-performance/ + │ ├── task-debug/ + │ ├── task-accuracy/ + │ ├── task-version-alignment/ + │ ├── task-derived-engine/ + │ ├── task-independent-engine/ + │ └── inference-task-router/ compatibility entry and original route resources + ├── systems/ platforms/{ascend,cuda,host}, frameworks/, engines/ + ├── playbooks/ measurement/, planning/, optimization/ + ├── workflows/ research/, governance/, maintenance/ + ├── casebooks/ inference/, ascend/ + ├── records/.json 28 contribution + 9 dependency + 11 peer + ├── config/ query_aliases.json, research_profiles/ + ├── skill-index.json stable ID → directory and backend + └── _compiled/ inputs.json, _provenance.json +``` + +The README documents the library. A procedure, including its resources, belongs in one registered directory. An engineering record carries its own sources, applicability and verification status. Its filename must match its stable ID. The registry locates each skill. The existing protected `knowledge/catalog.py` owns the public view fields, including source links, recommendations and verification status. Builds compile YAML headers and pinned Git fingerprints required by the base wheel. Authors edit structured fields in the same SKILL.md as the procedure. The catalog combines that embedded object with the compiled host header and resource fingerprints to emit the public schema-1 record on demand. Publishing the same skill under multiple engine or accelerator trees would require synchronized edits. Capture these dimensions for the actual run and compare them with owner metadata. For a new environment with no applicable local evidence, retain the unknown and the first check that could resolve it. [Record authoring guide](../src/inferrsi/knowledge/data/skill_library/records/README.md) + +Budgeted Collect can deliver a `skill-procedure` view. Its `sha256` covers the delivered bytes. Its `source_sha256`, byte ranges and revision identify the full source and allow the receiver to reconstruct that view exactly. The full file is available through an exact resource read. [Context and byte ranges](knowledge-operations.md) + +For a readable Find hit, `next_read_content_bytes` counts the UTF-8 bytes in the advertised read command's `content` field at Find time. An unpinned material read follows the current checkout, so a subsequent edit can change its size. To replay the same bytes, retain the snapshot and use the bundle revision for the resource read. The envelope and other resources have separate costs. A skill material read returns its procedure and host header with the embedded InferRSI machine annotation removed. A raw resource read retains that annotation. The Find preview shows up to three complete limitations within a 400-byte UTF-8 JSON list. `limitations_withheld` counts the rest. Open the owner to inspect its qualifications in full. + +Claude reads `disable-model-invocation` from the native YAML header. Across this migration, the 33 procedures that previously exposed that flag and `metadata.disco-role` still expose them. For 22 other procedures (nine PyTorch, nine TorchNPU and four Lithos), the old sidecar alone contained those values. Readers of imported metadata can decode the same Boolean and role from `metadata.inferrsi`; promoting them into the native header would change host invocation behavior. [Claude skill invocation](https://code.claude.com/docs/en/skills#control-who-invokes-a-skill) + +### What to edit + +| When you change | Edit | How the change reaches readers | +| --- | --- | --- | +| A procedure, its applicability, provenance, tags or declared resources | The owner's `SKILL.md`. Ordinary agent fields such as `name` and `description` stay in the YAML header. `metadata.inferrsi` carries the structured InferRSI fields as a JSON string in a YAML block scalar. | The build compiles the header and resource hashes. Find uses the existing searchable fields and procedure text. | +| A supporting example, script or case | Its owner-local resource under `references/`, `scripts/` or `cases/` | Open it through the owner's ID and revision. | +| An independent engineering finding | `records/.json` | Its source, applicability and verification status travel with that evidence item. | +| A new registered procedure | Add its owner directory and record the ID, path and backend in `skill-index.json` | Listing and exact reads resolve the new ID. Regenerate the two files under `_compiled/` from the committed sources. | + +This migration folds the fields from 107 paired files into their procedures, including 19 nested procedures. The 19 remain resources of their registered parents. The 48 JSON findings each represent independently addressable evidence. Query aliases, route cards, case records and test inputs retain their own file formats. + +## Seven deliverables + +| Required primary skill | Inputs to distinguish | Acceptance evidence | +| --- | --- | --- | +| [Day-0 model](../src/inferrsi/knowledge/data/skill_library/tasks/task-day-0/SKILL.md) | Model revision, stages, reference implementation, backend | Stage outputs, first end-to-end result, regression fixture | +| [Performance](../src/inferrsi/knowledge/data/skill_library/tasks/task-performance/SKILL.md) | Target metric, workload, bottleneck, quality floor | Paired baseline/candidate runs, quality check and variability | +| [Debug](../src/inferrsi/knowledge/data/skill_library/tasks/task-debug/SKILL.md) | Symptom, last-good configuration, first failing stage | Minimal reproduction, implicated boundary, regression check | +| [Accuracy](../src/inferrsi/knowledge/data/skill_library/tasks/task-accuracy/SKILL.md) | Reference outputs, dtype, preprocessing, seeds | First divergence, agreed tolerance and end-to-end result | +| [Version alignment](../src/inferrsi/knowledge/data/skill_library/tasks/task-version-alignment/SKILL.md) | Versions of vLLM, vLLM-Omni, vLLM-Ascend and afd-plugin as applicable | Compatibility matrix, paired versions and rollback point | +| [Derived engine](../src/inferrsi/knowledge/data/skill_library/tasks/task-derived-engine/SKILL.md) | Parent revision, inherited APIs and changed paths | Upstream behavior checks and tests of the fork delta | +| [Independent engine](../src/inferrsi/knowledge/data/skill_library/tasks/task-independent-engine/SKILL.md) | Runtime, model format, device, OS and operator availability | Device-native first result, capability gaps and resource bounds | + +The main entrypoint loads the primary that owns the user's acceptance criterion. Record hardware vendor, device, backend and OS, model specification and stages, engine and version, run script and flags, benchmark, acceptance criteria, current diagnosis and allowed search space. State unknowns and obtain the smallest missing input that affects the decision. + +For example, when a Windows TTS engine still runs after an upgrade but audio quality falls, load **task-accuracy** before the first search. Fix the prompt, reference audio or codec tokens, seed, preprocessing, dtype and tolerance. Compare the last-good and target stacks on that input. When only the target stack fails, consult version alignment to isolate the changed component. When both stacks diverge, trace the model's earliest mismatching stage. An Ascend 910B1 result can suggest what to measure; verifying a Windows repair requires the original target reproducer and held-out quality checks. If the reference is missing or the allowed budget runs out, record the missing comparison and its impact. + +LLM, Omni, TTS, Diffusion and RLT describe workload conditions. CUDA, ROCm, Ascend and XPU describe backends. Mobile devices and Windows add deployment constraints. Check these dimensions against evidence individually. Use only the filters supported by the chosen CLI reader. + +## Search and read examples + +```bash +inferrsi skills enter task-accuracy +inferrsi find "TTS quality regression after engine upgrade" +inferrsi knowledge search --tier peer --repo-type qwen3_tts_engine --backend ascend --json +inferrsi knowledge show flashinfer.cuda.llm.paged_decode_dispatch +``` + +For an inference task, run `inferrsi skills enter ` and read its returned `instructions` before choosing the first supporting query. Its decision points specify the observation that warrants opening a supporting skill, the evidence to retrieve and the next action. A direct evidence lookup outside the main agent workflow can still use an exact ID or free text. The compatibility router retains its historical resource paths. + +For an exact read of an authored finding, the reader validates the selected record and the inputs of its bundle. A stale metadata edit in an unrelated skill cannot block a standalone finding such as FlashInfer paged decode. A record that declares linked skill documents retains its library dependencies and bundle revision checks. Changes to those dependencies can invalidate a pinned read. + +Find skips incidental system files such as `.DS_Store` beside the records and still validates record JSON. It ranks each eligible evidence owner once. A scoped catalog query reuses Find; a catalog listing without a query includes the eight task entries. Ranked Collect and managed campaign capture use the same evidence selector, and explicit IDs can still capture a task owner. Check revision, applicability and verification before using a result. Propose a reusable finding for human review with its source, scoped conclusion and retry condition. + +## Design choices and benefits + +Agent V describes Git-backed shared memory with human guidance, hard-to-rediscover facts, indexed workflows and session experience. InferRSI adopts the distinction between a procedure, a scoped finding and a run. Accepted findings enter through review. Frozen experiment materials stay in `evidence/`. [Agent V](https://inferact.ai/news/agent-v-connecting-people-and-agents-for-autonomous-inference) + +MindIE Agent separates agent capabilities, knowledge and execution plugins. KDA specializes in kernel development. Cannbot Skills packages Ascend procedures for reuse. InferRSI needs cross-engine, cross-model and cross-platform retrieval as well as evidence qualification. It combines a task front door, technical owners, stable IDs and scoped evidence. A claim that it retrieves better than any of these projects requires a common task benchmark. [MindIE Agent](https://github.com/mindie-agent/mindie-agent) · [KDA](https://github.com/NVlabs/kda) · [Cannbot Skills](https://github.com/hicann/cannbot-skills) + +For the separate question of how much the task entry helps engineers, measure three arms on the same corpus and tool budget. A uses the prior layout and search. B runs direct search on the unified layout as an evaluation-only ablation. C uses the normal main entrypoint on that same layout and loads the selected primary before evidence search. Use held-out tasks across the seven types, blinded reviewers and paired cases with an applicable answer and a similar-looking incompatible target. Use four held-out tasks for each of the seven task types. Run each arm in an isolated session and balance the order. The primary metric is the fraction of answerable tasks with an applicable, cited owner in the top three. Report top-five applicable-source recall separately. For missing evidence, score unjustified transfer, correct abstention and mistaken abstention separately. Record time to first useful evidence, files opened and maintenance effort. Compare A→B and B→C separately using the gates in the acceptance issue. Approve broader rollout only when the team meets its predeclared quality and cost targets. + +The authoring benefit is directly inspectable: one finding file or one skill owner is edited; no per-pack navigation list or projected record is maintained. Two compiled files are reproducible build products. Their shared merge point is a cost: resolve source edits, then regenerate. The build-time Git pin identifies source provenance. Installed material selection checks the resource hashes. Whole-library dependency revisions, checkout revisions, and hashes of individual findings have different scopes. Historical checkout pins must be replayed on the corresponding snapshot. Compared with main after #507, the full layout migration rejects 79 old checkout skill pins, retains the seven case pins from that baseline and preserves the pre-existing unavailable `husky` read. Direct callers that open old pack filenames must move to the record ID or new path. + +A provenance-only refresh preserves an authored record’s bundle pin when its linked skill resources stay identical. Each registered casebook is a skill package; the seven rankable cases are individual evidence items. + +The consolidation step moved all 48 files byte-for-byte from the preceding PR layout. Compared directly with main before the layout migration, 30 are byte-identical and 18 update only the path in `skill_documents`. All 48 retain their IDs, tiers, source pins and verification state. An installed package bundle for an authored record that declares linked skill documents also includes library dependencies. Editing a referenced library file can change that bundle revision. Archive a known-good snapshot for pinned replay. diff --git a/docs/knowledge-operations.md b/docs/knowledge-operations.md index cabe382c4..ae5733aea 100644 --- a/docs/knowledge-operations.md +++ b/docs/knowledge-operations.md @@ -13,7 +13,7 @@ from inferrsi.knowledge.knowledge_operations import KnowledgeReader reader = KnowledgeReader(skill_root=Path("src/inferrsi/knowledge/data/skill_library")) result = reader.collect(ids=["calibrate-serving-simulation"], material=True, limit=1) item = result["items"][0] -source = reader.resource(item["ref"], "methods/calibrate-serving-simulation/references/source-notes.md", +source = reader.resource(item["ref"], "playbooks/measurement/calibrate-serving-simulation/references/source-notes.md", revision=item["revision"]) ``` @@ -132,18 +132,21 @@ the resource rules below. `material=True` on a package Skill record returns its | `context_omissions` | Owner context unavailable: `unsupported_context`, `index_unavailable` or `context_limit` | | `context_bytes`, `context_budget` | Used body bytes, requested ceiling and accounting method | -Each context resource carries its complete UTF-8 `content`, canonical resource -`path`, exact-byte `sha256`, `role` (`core` or `reference`) and -`locator=whole-resource`. The parent view supplies its owning ref and revision; -`resource` can read exactly those bytes. The full canonical `SKILL.md`, declared -references, provenance readsets and owner licenses (when present) are mandatory. +Each context resource reports its canonical `path`, `role`, UTF-8 `content` and +the SHA-256 of those delivered bytes. A `whole-resource` locator covers the +entire file. A `skill-procedure` locator delivers the complete procedure and host +frontmatter, omitting the structured `metadata.inferrsi` block from prompt +context. Its `source_sha256` covers the entire file, and `source_byte_ranges` +lists the half-open byte ranges whose concatenation reproduces the projection. Use the parent ref, +revision and path with `resource` to reproduce the source bytes. The skill +procedure, declared references, provenance readsets and owner licenses are mandatory. Licenses are retained along each required resource's skill/repository/framework directory ancestry inside the packaged library, including cross-skill prerequisites. Owner-local named license/notice files (including retained `*-LICENSE.md` and `THIRD_PARTY_NOTICES.md`) and explicit `skill_metadata.sources[].third_party_notices` paths are mandatory as well. Declared notice paths use the same boundary and existence checks as declared references. These resources never depend on a -relevance match; their complete bytes count toward the mandatory set. +relevance match; their delivered bytes count toward the mandatory set. If that set does not fit, the owner's entire context is omitted, with required and available bytes, and a later smaller context can still fit. The owning item can still be returned. Mandatory instructions are never truncated. @@ -152,7 +155,7 @@ Remaining owner-local Markdown, JSON, text and CSV resources are optional. With a cue, Find ranks their filenames and text within the owner and only positive matches are eligible. An empty cue uses resource path order. The context's `omissions` list nonmatching, oversized, non-UTF-8 and ineligible resources -such as `metadata.json` and scripts. Follow links to further material through +such as scripts. Follow links to further material through the owning revision. Use exact `read` for the complete Bundle. The context budget sums exact UTF-8 resource-body bytes across owners in selected @@ -184,15 +187,20 @@ Selected resource reads require the exact returned revision. A changed checkout or installation rejects an old revision; it never substitutes new bytes. This reader does not itself retain history or claim standalone exports. -`read --material --omit-resources` returns the full entrypoint, provenance, -applicability and revision. Use `read --resource ` to display a specific +`read --material --omit-resources` returns the entrypoint, provenance, +applicability and revision. For skills with embedded InferRSI metadata, +`content` carries the procedure and its host header. `content_view` supplies +the delivered SHA-256, source SHA-256 and half-open byte ranges in the source. +The explicit resource read at the same revision returns the original bytes, +including the annotation. Older sidecar-backed procedures retain their +original entrypoint content. Use `read --resource ` to display a specific UTF-8 resource in `content` and its key in `content_resource`. Remove `--omit-resources` to list resource paths and digests. These views use the same full-bundle revision. - Local skills include their complete owner folder (except incidental `__pycache__`, `*.pyc` and `.DS_Store`) and declared cross-skill references, - whether declared in the sidecar or in legacy frontmatter. Other links remain navigation in the required full checkout. + whether declared in skill frontmatter or a legacy checkout sidecar. Other links remain navigation in the required full checkout. - Local structured cases keep their own identity, observation and source limits. A case bundle holds its record, content and any `required_resources` declared relative to the record; missing, unsafe or escaping resources fail. Case diff --git a/docs/operations.md b/docs/operations.md index 1e1842786..ec3a1222e 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -37,7 +37,7 @@ Effect classes used below: | Find relevant cases, skills and records | CLI `find` | Read-only | Default discovery across the registered library and package records. Follow the emitted command to read a hit and its revision | | Inspect the package catalog | CLI `knowledge search` / `knowledge show`; catalog API | Read-only | Filtered inventory and existing catalog scripts. Uses the installed package's tier/repo/backend/scenario filters and IDs | | Prepare model-aware engine research | CLI `knowledge research`; Python `research_knowledge` | Read-only | Independent model/framework/task/modality/mechanism queries over the same package catalog; source-grounded known profiles, deduplicated matches and explicit target annotations; no engine/hardware qualification | -| Discover/read packaged or custom skills | CLI `skills list` / `show` / `resource` / `reference` | Read-only | Metadata-first listing; `show` loads one named instruction body; `resource`/`reference` resolve paths for your file reader without loading or executing them | +| Enter an inference task or discover/read skills | CLI `skills enter` / `list` / `show` / `resource` / `reference` | Read-only | `enter` loads one primary owner's complete procedure before evidence search. `list`/ `show` inspect skill metadata and instructions. `resource`/`reference` resolve paths for your file reader. | | Collect selected knowledge | CLI `collect`; Python `KnowledgeReader.collect` | Read-only | Batch selection and budgeted assembly by query or exact IDs, with revisions and omissions. The only accepted explicit `--relevance-method` value is `find-lexical-v1` | | Read exact material/resources | CLI `read` / `resource`; Python `KnowledgeReader.read` / `KnowledgeReader.resource` | Read-only | Returns the requested revision or an explicit unavailable/stale error; never substitutes newer bytes | | Export legacy cited material | CLI `context` | Read-only | Cited package export for manual use with a coding agent; not a conversation store and not a full Collect equivalent | @@ -121,7 +121,7 @@ the filtered package catalog and still returns complete catalog records. The membership and order. Use `--explain` for the recorded ranking and counts. - **Inputs:** optional query words and `--tier`, `--repo-type`, `--backend`, `--scenario`, `--limit`, `--json`, `--explain`; `show` takes one record ID. -- **Scope:** the installed package catalog (`repo_packs`, `knowledge_packs`) only. +- **Scope:** the installed package catalog (authored records and registered skill views in `skill_library/`) only. - **Output:** tab-separated record lines (id, status, backend, title), or a JSON list with `--json`; `--explain` returns a JSON envelope with `items` and `retrieval` (entry, ranking, scores and counts). Python uses `search_knowledge(..., explain=True)` for that envelope. `show` prints one full record as JSON. - **Side effects:** none. - **Failure behavior:** no match prints nothing and exits 0 — an empty result is not an error (an empty JSON list with `--json`, or empty `items` in the `--explain` envelope); an unknown `show` ID and unsupported filter values exit 2. @@ -129,7 +129,7 @@ membership and order. Use `--explain` for the recorded ranking and counts. The tier/repo/backend/scenario filters are package-only: an empty filtered result does not mean that no checkout or store knowledge exists. See the -[three-tier guide](../src/inferrsi/knowledge/data/knowledge_packs/README.md) for all sources. +[three-tier guide](../src/inferrsi/knowledge/data/skill_library/records/README.md) for all sources. ### Prepare model-aware research — `knowledge research` @@ -145,17 +145,17 @@ Each research query uses Find over the same filtered package catalog. Querying separate dimensions broadens discovery while preserving the active package filters. Keep those filters explicit when transferring mechanisms from other backends; target intent does not qualify their applicability. Read the canonical -[preparation procedure](../src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/model-aware-preparation.md) -and the repository-owned [h3.c residency audit](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md) +[preparation procedure](../src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/model-aware-preparation.md) +and the repository-owned [h3.c residency audit](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md) for native-engine reference selection, full encoder/DiT/VAE accounting, quantized-package compatibility and target hardware decisions. Returned resource paths are relative to `importlib.resources.files("inferrsi")` and ship in the wheel. The small model profile links that skill's full inventory and worked example; those resources retain their repository owner and source attribution. -### Discover/read packaged or custom skills — `skills list` / `show` / `resource` / `reference` +### Discover/read packaged or custom skills — `skills list` / `enter` / `show` / `resource` / `reference` -- **Inputs:** `list` reads frontmatter and sidecar metadata; `show ` selects one instruction body; `resource ` and `reference ` resolve one path. The default is the installed library's registered skills; an explicit `--root` selects a custom flat tree or a categorized library with `skill-index.json`. The optional `skills` extra supplies the YAML parser. +- **Inputs:** The agent selects a primary skill by deliverable and runs `inferrsi skills enter task-accuracy`, for example. It returns the primary's complete procedure and host header before supporting evidence search. The base installation can omit the embedded search metadata from this delivery. A pinned `read --resource` exposes the complete source file, including its metadata. An unknown, unavailable or non-primary owner is an error. `list` reads skill frontmatter (and legacy sidecars in historical/custom checkouts); `show ` selects one instruction body; `resource ` and `reference ` resolve one path. The default is the installed library's registered skills; an explicit `--root` selects a custom flat tree or a categorized library with `skill-index.json`. The optional `skills` extra supplies the YAML parser for frontmatter operations. - **Scope:** complete packaged skills under `data/skill_library/`, or the explicitly selected custom root. The 21 English engine and research-router skills remain accessible through catalog records and library navigation, separately from the registry. - **Output:** metadata listing, one instruction body, or a resolved path. - **Side effects:** none. `resource`/`reference` do not load or execute the target. @@ -210,7 +210,7 @@ including `昇腾` for Ascend. When the query names exactly one supported backen results declared for a different backend receive half their lexical score. `--explain` reports the adjusted contribution of each matched term. -Chinese query terms expand through `data/query_aliases.json` before ranking. +Chinese query terms expand through `data/skill_library/config/query_aliases.json` before ranking. The packaged library is English by policy while its readers work in Chinese, and the corpus holds too few Chinese characters for a Chinese query to score on its subject. Expansion is one-way and query-side: 显存 reaches `device memory`, not @@ -244,7 +244,10 @@ question calls for prior evidence. `checkout:` read reference uses `--installed-library` to resolve the source from the reader's installation. A custom root uses `--skill-root`. Where recorded, items carry source, summary, applicability, limitations and - local verification status. Case findings are ordered by query overlap, + local verification status. The limitations preview contains up to three + complete entries within a 400-byte UTF-8 JSON list. `limitations_withheld` + counts omitted entries. Open the owner for the complete list. + Case findings are ordered by query overlap, with `findings_total` and `findings_withheld` describing the available preview. Each finding carries its outcome, reason, revisit condition and any assessed scope and bounds. `bounds_source` identifies whether bounds apply to the @@ -255,7 +258,13 @@ question calls for prior evidence. full subprocess arguments as `open_argv` and InferRSI CLI arguments as `read_args`. The arguments include the source-qualified ID and any needed source flag. Most hits use `read --material --omit-resources` to return the - full entrypoint, provenance and revision. Remove + entrypoint content, provenance and revision. For a skill, the procedure + omits the embedded machine annotation. A readable hit's + `next_read_content_bytes` counts the UTF-8 bytes in that command's `content` + on the snapshot inspected by Find. For a future unpinned read this field + is a size estimate. A later edit can change the content and its size. + Use Collect's context budget for bounded delivery. The count excludes the + envelope and additional resources. Remove `--omit-resources` to list the bundle's resources. Hits that fail the reader's material check carry `unreadable` with the reason. - **Base installation.** When PyYAML is absent, an installed Skill with a diff --git a/docs/plans/2026-10-19-campaign-plan-rfc.md b/docs/plans/2026-10-19-campaign-plan-rfc.md index 35355fc82..ae01625da 100644 --- a/docs/plans/2026-10-19-campaign-plan-rfc.md +++ b/docs/plans/2026-10-19-campaign-plan-rfc.md @@ -92,8 +92,8 @@ levels must improve, and which must not regress beyond the rollback line. **Row 4 —** one variable per round and one plane per step per [AGENTS.md](../../AGENTS.md). Justify conditional headroom against the campaign target and locally measured noise using -[performance-ceiling-analysis](../../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md) -and its [bounds guidance](../../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md). +[performance-ceiling-analysis](../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md) +and its [bounds guidance](../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md). Record conditionally removable exposed critical-path time, assumptions and uncertainty, including overlap, synchronization, capacity, queueing, batching and bottleneck migration. A non-dominant stage is not by itself a rejection reason; @@ -134,7 +134,7 @@ tampering but do not prevent it. | 1 | The delivery scaffold requires a "campaign contract" that no file defines | [engine-task.md](../template/engine-repo/.github/ISSUE_TEMPLATE/engine-task.md) and [reproducibility.md](../template/engine-repo/docs/reproducibility.md) point at a contract, while [plan/README.md](../../evidence/_template/plan/README.md) is three bullets and [campaign-acceptance.yml](../../.github/ISSUE_TEMPLATE/campaign-acceptance.yml) carries a free-text checklist | | 2 | Model and hardware identity are not a required set | A RunSpec has no SKU, IDs, topology, driver, container digest or weight hash; [drama3-npu-readiness.md](../drama3-npu-readiness.md) had to invent a TBD tuple | | 3 | Search intent has no home and no version | Directions are reconstructed from the H1–H19 ledger ([qwen-image-2.1-turbo-b300.md](../qwen-image-2.1-turbo-b300.md)) or casebook rejections; "one plane per step" has nothing to check against | -| 4 | The test plan is split and partly contradictory | L0–L4 in [contributing.md](../contributing.md) versus L0–L5 with different L2/L4 in [dense-performance-evidence](../../src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md); [inference-benchmark-design](../../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) proposes plan files that do not exist | +| 4 | The test plan is split and partly contradictory | L0–L4 in [contributing.md](../contributing.md) versus L0–L5 with different L2/L4 in [dense-performance-evidence](../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/dense-performance-evidence/SKILL.md); [inference-benchmark-design](../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) proposes plan files that do not exist | | 5 | Imported evidence cannot be checked against a plan | The Ascend 910B1 cohort is `INCONCLUSIVE` for exactly these fields ([qwen-image-2.1-turbo-ascend.md](../qwen-image-2.1-turbo-ascend.md)) | | 6 | No budget, spend accounting or stop condition | `experiment_protocol` in [plan.json](../../evidence/drama3-b300-scale150/plan/plan.json) states a stop rule in prose, but no cap, no planned-versus-spent account and no acceptance reserve | | 7 | Freezing detects tampering, it does not prevent it | [freeze.json](../../evidence/drama3-b300-scale150/plan/freeze.json) is a hash manifest; nothing names who owns the device, the validator or the freeze, or how the generator is kept away from them | @@ -192,7 +192,7 @@ nothing about that cohort's `INCONCLUSIVE` verdict. | 7 | Control and authority | Author-reported and imported: no local device owner, no local evaluator, and no generator/evaluator separation; the cohort is owned by the importing campaign | The `D-04` to `D-06` entries are the casebook's recorded directions -([qwen-ascend-rejected-directions.md](../../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md)); +([qwen-ascend-rejected-directions.md](../../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md)); their revisit conditions are diagnostic suggestions, not reproduced causes. Filling the table mechanically reports the missing fields — model revision, weight hash, corpus hash, validator, driver stack, service timing boundary and AB/BA @@ -225,7 +225,7 @@ explicitly as UTF-8 is a one-line existing fix. ## 8. Open questions 1. **Ladder authority** — [contributing.md](../contributing.md) defines L0–L4; - [dense-performance-evidence](../../src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md) + [dense-performance-evidence](../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/dense-performance-evidence/SKILL.md) defines L0–L5 with different L2/L4. Row 3 must cite one by name. 2. **File and format** — keep the existing `plan.json` name and JSON encoding so `freeze.json` and current readers keep working, or move to `campaign-plan.yaml`? @@ -234,7 +234,7 @@ explicitly as UTF-8 is a one-line existing fix. 4. **Enforcement** — template and tests only, or also a required acceptance-form field and a reviewer checklist item before a `PASS`? 5. **Reconcile the benchmark-design plan set** — fold the - [inference-benchmark-design](../../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) + [inference-benchmark-design](../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) files (`hypothesis.md`, `runspec.json`, `slo.yaml`, `decision.yaml`) into this table, keep them as optional sub-artifacts, or drop them? 6. **Change approval** — does a row-4 revision need an approver, or is appending @@ -250,4 +250,4 @@ explicitly as UTF-8 is a one-line existing fix. No runtime or CLI capability, no threshold, no pin, no campaign, no hardware or performance result; no recorded verdict changes. Conventions followed: [agent collaboration](../agent-collaboration.md), -[rsi-evidence](../../src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/SKILL.md), [AGENTS.md](../../AGENTS.md). +[rsi-evidence](../../src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/SKILL.md), [AGENTS.md](../../AGENTS.md). diff --git a/docs/quickstart.md b/docs/quickstart.md index 8630cde08..636a06320 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -14,9 +14,10 @@ inferrsi --help inferrsi doctor --repo . --repo-type vllm_omni --backend cpu ``` -This base install supports `find`, Case reads and package-only knowledge -operations. The `skills` command and direct reads of checkout Skill bodies -require `python -m pip install '.[skills]'` (PyYAML). +The base install supports `skills enter `, `find`, Case reads and +package-only knowledge operations. Install the `skills` extra with +`python -m pip install '.[skills]'` for commands that parse skill frontmatter, +including direct reads from a checkout or a custom skill root. For the maintained model examples, see [Qwen B300](qwen-image-2.1-turbo-b300.md) and [Qwen Ascend](qwen-image-2.1-turbo-ascend.md). Continue below for the generic @@ -45,9 +46,12 @@ Its common catalog ID and the separate MLX-LM procedure remain unchanged. ## Discover knowledge and skills -Start from the problem. `find` ranks packaged skills, imported records and -casebook entries together and reports what each one's evidence covers, so a -direction already tried and dropped surfaces beside the procedure for trying it: +For an inference engineering task, select the [primary skill for the user's +goal](knowledge-layout.md#seven-deliverables), run `inferrsi skills enter ` +and read the returned procedure before searching. For example, when inference +succeeds but output quality regresses, load `inferrsi skills enter task-accuracy`. +Use the observed run to choose supporting evidence. Standalone `find` ranks +supporting skills, imported records and casebook entries together: ```bash inferrsi find 'cache per-step masks ran out of device memory' @@ -115,7 +119,7 @@ the agent uses the packaged ceiling-analysis skill to formulate targets: inferrsi read inferrsi.common.skills.performance_ceiling_analysis --material ``` -Follow its [acceptance-criteria procedure](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/acceptance-criteria.md). +Follow its [acceptance-criteria procedure](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md). Preserve partial requirements; record derived targets with their baseline, calibration, uncertainty and authority in the acceptance issue and frozen plan. Missing evidence requires calibration, not a guessed threshold. This is agent @@ -214,8 +218,8 @@ or `--tier peer`. Export one source explicitly, for example `inferrsi context --repo-type xdit --backend cuda --tier peer`. `--repo-type` names the source owner; vllm_omni queries do not automatically include its dependencies or peers. See the -[three-tier guide](../src/inferrsi/knowledge/data/knowledge_packs/README.md) for all sources. +[three-tier guide](../src/inferrsi/knowledge/data/skill_library/records/README.md) for all sources. For per-trial completeness, lifecycle and graph qualification, see the -[Omni experiment contract](../src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/omni-experiments.md). These optional checks +[Omni experiment contract](../src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/omni-experiments.md). These optional checks are frozen in RunSpec and do not replace the quality command. diff --git a/docs/reference-matrix.md b/docs/reference-matrix.md index fcca4bc13..62f9f2d39 100644 --- a/docs/reference-matrix.md +++ b/docs/reference-matrix.md @@ -7,7 +7,7 @@ Current skill links resolve to the complete categorized procedures live under their hardware owner; this relocation does not change the source-inspection or maturity claims below. -The [22-owner hardware source inventory](../src/inferrsi/knowledge/data/skill_library/hardware/README.md) +The [22-owner hardware source inventory](../src/inferrsi/knowledge/data/skill_library/systems/platforms/README.md) separates direct implementation reads, retained supplier reads, skill/document imports and mirror/pack lineage. It is source navigation, not a third matrix or a validation score; native and service results remain separately qualified. @@ -47,37 +47,37 @@ Reading rules: | Reference (origin @ revision) | Inspected scope | Accelerator | RAM / Host | SSD / Storage | Communication | OS / System | |---|---|---|---|---|---|---| -| cann/hccl @ `96fab0a2acb761e7f8ab0f94aeb5138f200a345b` (CANN Open Software License Agreement Version 2.0) | Direct canonical GitCode implementation read, 2026-10-30; [17-file readset](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce-readset.json), [multi-rank path](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce.md); simulator test definitions inspected, not run | mined (narrow) — AICPU mesh scratch/chunking and local reduction ownership; no general accelerator or graph qualification | out-of-scope — HCCL scratch is not host RAM | out-of-scope | mined (one native path) — positive-count, multi-rank OPBASE AICPU mesh one-shot AllReduce selection, peer writes, joins and local reduction -> [ascend-distributed-parallel](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md); HCOMM/ACL physical transport, legacy/other engines and numerical/performance behavior unqualified | out-of-scope — collective threads are not OS placement | -| pytorch/pytorch @ `ba56102387ef21a3b04b357e5b183d48f0afefc7` (v2.8.0, BSD-3-Clause) | Direct bounded implementation/document read, 2026-10-30; CPU/CUDA copy, caching host allocator and stream/device-storage contract; [nine-file readset](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-host-copy-readset.json) | mined (narrow) — copy-stream ordering and device allocator lifetime; no graph/kernel qualification | mined — host mutation versus allocator release, completion-gated pinned-block reuse and D2H readiness → [host-memory-management](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/SKILL.md); candidate, CUDA/native/service not_run | out-of-scope — pinned DRAM is not SSD coverage | out-of-scope — local H2D/D2H is not network transport | out-of-scope — allocator lifetime is not OS placement | -| python/cpython @ `60403a5409ff2c3f3b07dd2ca91a7a3e096839c7` (v3.13.0, PSF-2.0) | Direct official documentation read, 2026-10-30; [free-threading readset](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/cpython-free-threading-readset.json); no interpreter implementation read | out-of-scope | mined (document only) — build/runtime GIL checks and extension compatibility boundaries → `host-memory-management`; no interpreter/inference-stack execution | out-of-scope | out-of-scope | out-of-scope — free threading does not establish affinity/NUMA controls | -| Ascend/torchair @ `e1118e8b0fc1b95190ab53b732c0872989550e20` (BSD-3-Clause) | Direct bounded Python implementation/document/test-definition read, 2026-10-30; [12-file readset](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-readset.json), [call-chain map](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-path.md) | mined — configuration versus backend callable, generated ACL runtime, capture key/recapture, persistent eager fallback, input staging and output lifetime → [ascend-graph-mode](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md); TorchNPU/CANN native graph/allocator and GE executor uninspected; NPU/model/service not_run | out-of-scope — host launch attribution is not host allocator coverage | out-of-scope — graph startup is not a storage implementation | out-of-scope — graph call sites do not establish native collectives | out-of-scope — graph scheduling is not OS scheduling | -| vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root Apache-2.0; in-file Tri Dao adaptation notice and separate vendored FLA MIT notice) | Direct bounded source inspection, 2026-10-27; Qwen3-Next caller, in-tree conv wrappers, GDN metadata, state shapes, row-resolution helper, selector defaults and two upstream test intents; [source ledger](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md) | mined — effective prefill/decode selection, mixed/speculative phase guards, state-row initialization and accepted-token offset → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md); [four source cases](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.md); native execution not_run, agent efficacy not_evaluated | out-of-scope — device state is not host-RAM evidence | out-of-scope | out-of-scope — packed state metadata is not network evidence | out-of-scope — metadata phase classification is not OS scheduling | -| huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` (v5.10.2) and `856157a2f3e9594954310df18fdccc31ffddebe9` (v5.17.0), Apache-2.0 | Bounded source inspection, 2026-10-29: imported Qwen3.5 module, original-package/Hub binding, forward/cache/padding and local numeric contracts; 13 files matched to server Git blobs, [exact readset](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-source.json); two existing worker seams are secondary context only | mined (narrow) — constructor versus global binding, independent conv/GDN fallback, first/cached/chunk phases and cache-free classification conditions → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md), [contract](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-binding.md) and five source cases; candidate, native/model/numerical/performance not_run, efficacy not_evaluated | out-of-scope — device states do not establish host-memory optimization | out-of-scope | out-of-scope | out-of-scope — layer state is not OS scheduling | -| sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` (Apache-2.0; conv adaptation notices credit vLLM and Tri Dao) | Bounded source inspection, 2026-10-29; Qwen3.5/Qwen3-Next caller, GDN phase/architecture policy, separate conv wrappers, state shapes/row mapping and selected upstream test intent; [readset and file identities](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md#source-ledger-and-attribution) | mined (narrow) — per-runner prefill/decode, CUDA/Triton conv stride/alias branches, state write-back and conditional one-shot prefill transfer → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md), [four source cases](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.md); [HF-worker/engine cross-map](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md#map-the-inspected-hf-workers-to-vllm-and-sglang) adds separate [PPLX caller context](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/pplx-gdn-caller-context.json) (worker license UNKNOWN); candidate/not_run, agent efficacy not_evaluated | out-of-scope — device state is not host RAM | out-of-scope | out-of-scope — row metadata is not network evidence | out-of-scope — engine phase classification is not OS scheduling | -| NVlabs/kda @ `acdd79343c5e9d0c7cb96289f26fd3f1051ce785` → mit-han-lab/KernelWiki gitlink @ `76d27b56f804e7e7295d4c570e1e5d7eef4b0a75` | Direct bounded distillation, 2026-10-23; cutoff 2026-04-27. All 48 synthesis headers and 2,217 source headers screened; 171 SM90 source headers retained, selected bodies and independently matched DeepGEMM/CUTLASS code plus versioned NVIDIA sections inspected. [Readset/coverage/license qualifications](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/sources.md); remaining PR bodies/algorithms unmined; KW pin has no root LICENSE, KDA MIT/sanitization assertions qualified | mined (conditional) — Hopper TMA/mbarrier/WGMMA ownership, resource/layout/pipeline/software scheduling and selected FP8 scaling → [reason-hopper-kernels](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md); SM100 recipes and unsafe snippet generalizations excluded; native/precision/performance not_run | out-of-scope — device registers/SMEM/HBM are not host RAM | out-of-scope — no storage implementation extracted | out-of-scope — CTA/cluster synchronization is not network transport | out-of-scope — tile scheduling is not OS scheduling | -| antirez/h3.c @ `8974cc055ea9c02fcd14cc27dfda3e1027c05153` (MIT; retained ccv BSD-3-Clause notice) | Direct bounded source distillation, 2026-10-23; 34 selected file bytes matched to pinned Git blobs; [readset/architecture](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/source-notes.md). Loader, generation/cache, TE/vision/input-output VAEs, AdaLN, selected Metal/int8/SSD paths and native test assertions; full kernels/driver/media internals unread. Prior model-header observations reused, not rerun | mined — joint audio/video schedule and row-map dependencies, BF16 rounding/fusion versus numerical approximation, actual TensorOps dispatch → [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md); native/quality/performance not_run | mined — shared/file-backed weights, Qwen prefetch ring, phase/cache/preview overlap and complete host output; tensor counters are not physical footprint or 64 GiB qualification | mined — exact raw BF16/F32 schema, direct/mapped reads and two-slot BF16 matrix streaming with completion/join boundaries; uncached-read request is not measured cache bypass or useful overlap | out-of-scope — local GPU/I/O ordering is not network transport | out-of-scope — no OS scheduling/control implementation extracted | -| lithos-ai/lithos-metal @ `6653b7df5f2c4392ec7b038960f2796b1499923f` (Apache-2.0; notices retained) | Bounded compiler/Program/StepState/DSpark, backend identity and serving/pack/prefix/idle paths; selected contract/runtime assertions; [readset](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/source-notes.md). Candidate; software/CPU/synthetic Metal checks do not qualify full-model numerics, DSpark or service performance | mined — chip-qualified task fusion, normalized controls, hazards/scratch and committed generation state → [lithos-metal](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/SKILL.md); empirical efficacy not_run | mined — program/pipeline versus GPU allocation and exact host-prefix ownership; unified-memory residency/performance unqualified | mined — derived pack validity, exact-prefix disk identity/integrity, bounded persistence and quota; weight-byte authentication and crash durability not established | out-of-scope — HTTP adapters are not distributed transport | out-of-scope — idle lifecycle is not OS scheduling | -| zuiho-kai/h3-metal @ `4ba47a3eabdbebf801ac6664b31f8ec168bc57a8` (inspected MIT; retained BSD-3-Clause notice; machine resolver unavailable) | Bounded source distillation, 2026-10-28; 21 identity-checked files with selected ranges; [scope/readset](../src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/source-notes.md). Fork-specific FastH3 params, tensor/schedule/RoPE and VSA dispatch, test/script limits; full engine/external FastVideo/model surface not mined | mined — requested versus effective controls, ambient conflicts, head-interleaved QKV/FC1 swap/BF16 schedule, real partial VSA tiles/double top-k and dense-plus-coarse fallback → [audit-h3-metal-fasth3-semantics](../src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md); native/numerical/quality/performance not_run | out-of-scope — tile arithmetic is not measured host footprint or memory-fit qualification | out-of-scope — FastH3 disables requested SSD parameter; no new storage optimization or payload validation | out-of-scope — local attention/dispatch is not network transport | out-of-scope — runtime selection is not OS scheduling | -| Comfy-Org/ComfyUI @ `b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c` (GNU GPL v3) | Source-reading archive, inspected 2026-10-21; bounded execution/cache/graph/model-management/static-dynamic-patcher/sampler paths and selected tests; [readset](../src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/repo-provenance.md); not full-model/custom-node/provider-native coverage | mined — managed weight/patch identity, partial unload versus detach and dynamic-backend boundary → [comfy-ui](../src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/SKILL.md); device behavior unverified | mined — output versus object identity, ancestor fingerprints, lazy consumer references and retention versus validity; host/pin ownership | out-of-scope — no storage implementation extracted | out-of-scope — no transport implementation extracted | out-of-scope — workflow scheduling is not OS scheduling | -| leejet/stable-diffusion.cpp @ `b13a48219d5cdf408edcb93da6cd69b612ed03d3` (MIT) | Supplied archive, inspected 2026-10-22: 18 bounded paths plus separately identity-checked README (19 blobs); historical receipt preserved. 2026-10-23 extension: 12 selected C++ blobs plus retained MIT text identity-checked, with line-scoped catalog/non-ZIP safetensors read/staging/mapping and immediate consumer cleanup; [readsets](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md). File identity is not full-file review; archive/submodules, model kernels and other-format implementations remain outside the bounded audit | mined — native weight/runtime capacity, protected segment/prefetch lifetime → [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md); native/backend behavior unverified | mined — parameter storage versus compute staging and retained runtime ownership; loader host read/conversion buffers versus mapped owners; host peaks unmeasured | mined — bounded catalog stamps versus content identity, range admission versus actual reads, mapping/shared-owner and newly created block cleanup → [loader/storage boundaries](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md); full other-format parsing, native failure/SSD/performance qualification remain open | out-of-scope — device copies are not network transport | out-of-scope — runner scheduling is not OS scheduling | +| cann/hccl @ `96fab0a2acb761e7f8ab0f94aeb5138f200a345b` (CANN Open Software License Agreement Version 2.0) | Direct canonical GitCode implementation read, 2026-10-30; [17-file readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-allreduce-readset.json), [multi-rank path](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-allreduce.md); simulator test definitions inspected, not run | mined (narrow) — AICPU mesh scratch/chunking and local reduction ownership; no general accelerator or graph qualification | out-of-scope — HCCL scratch is not host RAM | out-of-scope | mined (one native path) — positive-count, multi-rank OPBASE AICPU mesh one-shot AllReduce selection, peer writes, joins and local reduction -> [ascend-distributed-parallel](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/SKILL.md); HCOMM/ACL physical transport, legacy/other engines and numerical/performance behavior unqualified | out-of-scope — collective threads are not OS placement | +| pytorch/pytorch @ `ba56102387ef21a3b04b357e5b183d48f0afefc7` (v2.8.0, BSD-3-Clause) | Direct bounded implementation/document read, 2026-10-30; CPU/CUDA copy, caching host allocator and stream/device-storage contract; [nine-file readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/pytorch-host-copy-readset.json) | mined (narrow) — copy-stream ordering and device allocator lifetime; no graph/kernel qualification | mined — host mutation versus allocator release, completion-gated pinned-block reuse and D2H readiness → [host-memory-management](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/SKILL.md); candidate, CUDA/native/service not_run | out-of-scope — pinned DRAM is not SSD coverage | out-of-scope — local H2D/D2H is not network transport | out-of-scope — allocator lifetime is not OS placement | +| python/cpython @ `60403a5409ff2c3f3b07dd2ca91a7a3e096839c7` (v3.13.0, PSF-2.0) | Direct official documentation read, 2026-10-30; [free-threading readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/cpython-free-threading-readset.json); no interpreter implementation read | out-of-scope | mined (document only) — build/runtime GIL checks and extension compatibility boundaries → `host-memory-management`; no interpreter/inference-stack execution | out-of-scope | out-of-scope | out-of-scope — free threading does not establish affinity/NUMA controls | +| Ascend/torchair @ `e1118e8b0fc1b95190ab53b732c0872989550e20` (BSD-3-Clause) | Direct bounded Python implementation/document/test-definition read, 2026-10-30; [12-file readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-graph-readset.json), [call-chain map](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-graph-path.md) | mined — configuration versus backend callable, generated ACL runtime, capture key/recapture, persistent eager fallback, input staging and output lifetime → [ascend-graph-mode](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/SKILL.md); TorchNPU/CANN native graph/allocator and GE executor uninspected; NPU/model/service not_run | out-of-scope — host launch attribution is not host allocator coverage | out-of-scope — graph startup is not a storage implementation | out-of-scope — graph call sites do not establish native collectives | out-of-scope — graph scheduling is not OS scheduling | +| vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root Apache-2.0; in-file Tri Dao adaptation notice and separate vendored FLA MIT notice) | Direct bounded source inspection, 2026-10-27; Qwen3-Next caller, in-tree conv wrappers, GDN metadata, state shapes, row-resolution helper, selector defaults and two upstream test intents; [source ledger](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md) | mined — effective prefill/decode selection, mixed/speculative phase guards, state-row initialization and accepted-token offset → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md); [four source cases](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-source-contract.md); native execution not_run, agent efficacy not_evaluated | out-of-scope — device state is not host-RAM evidence | out-of-scope | out-of-scope — packed state metadata is not network evidence | out-of-scope — metadata phase classification is not OS scheduling | +| huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` (v5.10.2) and `856157a2f3e9594954310df18fdccc31ffddebe9` (v5.17.0), Apache-2.0 | Bounded source inspection, 2026-10-29: imported Qwen3.5 module, original-package/Hub binding, forward/cache/padding and local numeric contracts; 13 files matched to server Git blobs, [exact readset](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/hf-qwen35-gdn-source.json); two existing worker seams are secondary context only | mined (narrow) — constructor versus global binding, independent conv/GDN fallback, first/cached/chunk phases and cache-free classification conditions → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md), [contract](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/hf-qwen35-gdn-binding.md) and five source cases; candidate, native/model/numerical/performance not_run, efficacy not_evaluated | out-of-scope — device states do not establish host-memory optimization | out-of-scope | out-of-scope | out-of-scope — layer state is not OS scheduling | +| sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` (Apache-2.0; conv adaptation notices credit vLLM and Tri Dao) | Bounded source inspection, 2026-10-29; Qwen3.5/Qwen3-Next caller, GDN phase/architecture policy, separate conv wrappers, state shapes/row mapping and selected upstream test intent; [readset and file identities](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md#source-ledger-and-attribution) | mined (narrow) — per-runner prefill/decode, CUDA/Triton conv stride/alias branches, state write-back and conditional one-shot prefill transfer → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md), [four source cases](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/sglang-gdn-source-contract.md); [HF-worker/engine cross-map](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md#map-the-inspected-hf-workers-to-vllm-and-sglang) adds separate [PPLX caller context](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/pplx-gdn-caller-context.json) (worker license UNKNOWN); candidate/not_run, agent efficacy not_evaluated | out-of-scope — device state is not host RAM | out-of-scope | out-of-scope — row metadata is not network evidence | out-of-scope — engine phase classification is not OS scheduling | +| NVlabs/kda @ `acdd79343c5e9d0c7cb96289f26fd3f1051ce785` → mit-han-lab/KernelWiki gitlink @ `76d27b56f804e7e7295d4c570e1e5d7eef4b0a75` | Direct bounded distillation, 2026-10-23; cutoff 2026-04-27. All 48 synthesis headers and 2,217 source headers screened; 171 SM90 source headers retained, selected bodies and independently matched DeepGEMM/CUTLASS code plus versioned NVIDIA sections inspected. [Readset/coverage/license qualifications](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/sources.md); remaining PR bodies/algorithms unmined; KW pin has no root LICENSE, KDA MIT/sanitization assertions qualified | mined (conditional) — Hopper TMA/mbarrier/WGMMA ownership, resource/layout/pipeline/software scheduling and selected FP8 scaling → [reason-hopper-kernels](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/SKILL.md); SM100 recipes and unsafe snippet generalizations excluded; native/precision/performance not_run | out-of-scope — device registers/SMEM/HBM are not host RAM | out-of-scope — no storage implementation extracted | out-of-scope — CTA/cluster synchronization is not network transport | out-of-scope — tile scheduling is not OS scheduling | +| antirez/h3.c @ `8974cc055ea9c02fcd14cc27dfda3e1027c05153` (MIT; retained ccv BSD-3-Clause notice) | Direct bounded source distillation, 2026-10-23; 34 selected file bytes matched to pinned Git blobs; [readset/architecture](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/source-notes.md). Loader, generation/cache, TE/vision/input-output VAEs, AdaLN, selected Metal/int8/SSD paths and native test assertions; full kernels/driver/media internals unread. Prior model-header observations reused, not rerun | mined — joint audio/video schedule and row-map dependencies, BF16 rounding/fusion versus numerical approximation, actual TensorOps dispatch → [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md); native/quality/performance not_run | mined — shared/file-backed weights, Qwen prefetch ring, phase/cache/preview overlap and complete host output; tensor counters are not physical footprint or 64 GiB qualification | mined — exact raw BF16/F32 schema, direct/mapped reads and two-slot BF16 matrix streaming with completion/join boundaries; uncached-read request is not measured cache bypass or useful overlap | out-of-scope — local GPU/I/O ordering is not network transport | out-of-scope — no OS scheduling/control implementation extracted | +| lithos-ai/lithos-metal @ `6653b7df5f2c4392ec7b038960f2796b1499923f` (Apache-2.0; notices retained) | Bounded compiler/Program/StepState/DSpark, backend identity and serving/pack/prefix/idle paths; selected contract/runtime assertions; [readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/source-notes.md). Candidate; software/CPU/synthetic Metal checks do not qualify full-model numerics, DSpark or service performance | mined — chip-qualified task fusion, normalized controls, hazards/scratch and committed generation state → [lithos-metal](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/SKILL.md); empirical efficacy not_run | mined — program/pipeline versus GPU allocation and exact host-prefix ownership; unified-memory residency/performance unqualified | mined — derived pack validity, exact-prefix disk identity/integrity, bounded persistence and quota; weight-byte authentication and crash durability not established | out-of-scope — HTTP adapters are not distributed transport | out-of-scope — idle lifecycle is not OS scheduling | +| zuiho-kai/h3-metal @ `4ba47a3eabdbebf801ac6664b31f8ec168bc57a8` (inspected MIT; retained BSD-3-Clause notice; machine resolver unavailable) | Bounded source distillation, 2026-10-28; 21 identity-checked files with selected ranges; [scope/readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/source-notes.md). Fork-specific FastH3 params, tensor/schedule/RoPE and VSA dispatch, test/script limits; full engine/external FastVideo/model surface not mined | mined — requested versus effective controls, ambient conflicts, head-interleaved QKV/FC1 swap/BF16 schedule, real partial VSA tiles/double top-k and dense-plus-coarse fallback → [audit-h3-metal-fasth3-semantics](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md); native/numerical/quality/performance not_run | out-of-scope — tile arithmetic is not measured host footprint or memory-fit qualification | out-of-scope — FastH3 disables requested SSD parameter; no new storage optimization or payload validation | out-of-scope — local attention/dispatch is not network transport | out-of-scope — runtime selection is not OS scheduling | +| Comfy-Org/ComfyUI @ `b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c` (GNU GPL v3) | Source-reading archive, inspected 2026-10-21; bounded execution/cache/graph/model-management/static-dynamic-patcher/sampler paths and selected tests; [readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/repo-provenance.md); not full-model/custom-node/provider-native coverage | mined — managed weight/patch identity, partial unload versus detach and dynamic-backend boundary → [comfy-ui](../src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/SKILL.md); device behavior unverified | mined — output versus object identity, ancestor fingerprints, lazy consumer references and retention versus validity; host/pin ownership | out-of-scope — no storage implementation extracted | out-of-scope — no transport implementation extracted | out-of-scope — workflow scheduling is not OS scheduling | +| leejet/stable-diffusion.cpp @ `b13a48219d5cdf408edcb93da6cd69b612ed03d3` (MIT) | Supplied archive, inspected 2026-10-22: 18 bounded paths plus separately identity-checked README (19 blobs); historical receipt preserved. 2026-10-23 extension: 12 selected C++ blobs plus retained MIT text identity-checked, with line-scoped catalog/non-ZIP safetensors read/staging/mapping and immediate consumer cleanup; [readsets](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md). File identity is not full-file review; archive/submodules, model kernels and other-format implementations remain outside the bounded audit | mined — native weight/runtime capacity, protected segment/prefetch lifetime → [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md); native/backend behavior unverified | mined — parameter storage versus compute staging and retained runtime ownership; loader host read/conversion buffers versus mapped owners; host peaks unmeasured | mined — bounded catalog stamps versus content identity, range admission versus actual reads, mapping/shared-owner and newly created block cleanup → [loader/storage boundaries](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md); full other-format parsing, native failure/SSD/performance qualification remain open | out-of-scope — device copies are not network transport | out-of-scope — runner scheduling is not OS scheduling | | NVIDIA/TensorRT-LLM @ f33e0e5fd | skill-doc-only (`.claude/skills/`) | mined — evidence taxonomy, kernel writing, casebook, roofline refs → `dense-performance-evidence`, `kernel-writing-guide`, `inference-optimization-casebook`, `performance-ceiling-analysis` | mined — host overhead verdict + line-profiling loop → `host-overhead-profiling`, `host-overhead-optimization` | mined — weight-movement/overlap + KV host-cache patterns → `weight-loading-optimization`; disk-level: negative (see below) | out-of-scope — host/device weight-movement overlap remains in RAM/Storage, not network coverage | mined — CPU affinity rule + env preflight → `os-process-placement` | | Ascend/MindIE-SD @ 8481d39 | skill-doc-only (`.agents/skills/`) | mined — 5-layer analysis, bottleneck casebook → `ascend-performance-analysis`, `ascend-optimization-casebook` | mined — host-opt async/pattern sections → `weight-loading-optimization` | mined — load-path judgment, fixed-overhead accounting, offload tiers, weights-prep → `weight-loading-optimization` | mined — USP strategy, masking formula, topology rules → `ascend-distributed-parallel` | mined — device health, card-group contracts, measurement discipline → `ascend-device-health` | -| ascend-ai-coding/awesome-ascend-skills @ 75ec89c | Historical skill/document import (`skills/`, `external/cannbot/`), not whole native repository mining; separate communication-wrapper source slice pinned to `75ec89ce9d401be8ac95980e1c3f42f8d4315057` | mined — graph mode, operator profiling/tuning, tiling design → `ascend-graph-mode`, `ascend-operator-profiling`, `ascend-operator-tuning`, `ascendc-tiling-design` | assessed/deferred — KV overview and async-stream recipe lack a complete transfer/completion implementation and retain cannbot license UNKNOWN; [boundary/readset](reviews/issue91-awesome-ascend.md#host-kv-boundary); no new RAM capability | mined — device-side prefetch + diffusers CPU-offload playbook → `weight-loading-optimization`; disk-level: negative | mined (wrapper-source only) — native/PyTorch test selection, tensor-byte meaning, timing/oracle coverage and rank-map counterexamples → `ascend-distributed-parallel`; [full-pin readset](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md); no native-collective or transport audit | mined — env checks, device-busy preflight, thread/env tables → `os-process-placement` | -| cann/cannbot @ `c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110` (selected official plugin: CANN-2.0) | Direct canonical Git source inspection, 2026-10-23; fixed-stage model-inference workflow, three role instructions, registered callbacks and archive helper; [readset/trace/license](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md); dependent skills/recipes and host execution unread/unrun | mined (one source-derived rule) — current-stage execution, actual quality result and comparable baseline versus lexical callback completion → [ascend-performance-analysis](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md); no numerical/backend/performance qualification | out-of-scope — no host-memory mechanism extracted | out-of-scope — progress-file I/O is not storage optimization | out-of-scope — workflow instructions do not establish collective/transport implementation | out-of-scope — agent workflow is not OS scheduling | +| ascend-ai-coding/awesome-ascend-skills @ 75ec89c | Historical skill/document import (`skills/`, `external/cannbot/`), not whole native repository mining; separate communication-wrapper source slice pinned to `75ec89ce9d401be8ac95980e1c3f42f8d4315057` | mined — graph mode, operator profiling/tuning, tiling design → `ascend-graph-mode`, `ascend-operator-profiling`, `ascend-operator-tuning`, `ascendc-tiling-design` | assessed/deferred — KV overview and async-stream recipe lack a complete transfer/completion implementation and retain cannbot license UNKNOWN; [boundary/readset](reviews/issue91-awesome-ascend.md#host-kv-boundary); no new RAM capability | mined — device-side prefetch + diffusers CPU-offload playbook → `weight-loading-optimization`; disk-level: negative | mined (wrapper-source only) — native/PyTorch test selection, tensor-byte meaning, timing/oracle coverage and rank-map counterexamples → `ascend-distributed-parallel`; [full-pin readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md); no native-collective or transport audit | mined — env checks, device-busy preflight, thread/env tables → `os-process-placement` | +| cann/cannbot @ `c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110` (selected official plugin: CANN-2.0) | Direct canonical Git source inspection, 2026-10-23; fixed-stage model-inference workflow, three role instructions, registered callbacks and archive helper; [readset/trace/license](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md); dependent skills/recipes and host execution unread/unrun | mined (one source-derived rule) — current-stage execution, actual quality result and comparable baseline versus lexical callback completion → [ascend-performance-analysis](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/SKILL.md); no numerical/backend/performance qualification | out-of-scope — no host-memory mechanism extracted | out-of-scope — progress-file I/O is not storage optimization | out-of-scope — workflow instructions do not establish collective/transport implementation | out-of-scope — agent workflow is not OS scheduling | | vllm-project/vllm-ascend @ bc8115ed | skill-doc-only (`.agents/`); [version/responsibility routing](../src/inferrsi/knowledge/data/skill_library/README.md#vllm-ascend-version-and-responsibility-routing) | mined — FP8/KV-replication/multimodal adaptation → `ascend-model-adaptation` | unknown — runtime host logic not in sparse checkout | mined — FP8 paired-tensor loading pattern → `ascend-model-adaptation` (Configure cell); runtime loader not in checkout | unknown | mined — Dockerfile thread/env ship defaults → `os-process-placement` (historical defaults only) | | vllm-project/vllm @ `7f730b4cea9a66520a9200265928ebe016b48a39` (Apache-2.0) | Dot bundle-mediated; `scheduler.py` admission/budgets/preemption/deferred frees and connector `v1/base.py` contracts/consumers; concrete connector, worker, cache-manager internals, config schema and tests unread | mined — independent admission budgets and async-KV readiness → `diagnose-vllm-admission-pressure`; completion/retention/failure ownership → `audit-vllm-kv-lifetimes`; conditional proposals and unrun verification only | out-of-scope — device KV is not host RAM | out-of-scope — external-hit API establishes no physical tier | out-of-scope — connector contract is not transport implementation | out-of-scope — engine scheduling is not OS priority control | -| vllm-project/vllm-ascend @ `8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf` (Apache-2.0; separate runtime pin) | Dot bundle-mediated; `compilation/acl_graph.py`, pinned graph-mode guide and selected `test_acl_graph.py` cases; tests inspected by supplier, not run; caller/ENPU ordering and helpers untraced; native single-tier spec/worker, registration and DMA binding read directly ([seven-file ledger](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload-readset.json)); external vLLM dependencies unpinned; [separate model-adapter routing](../src/inferrsi/knowledge/data/skill_library/README.md#vllm-ascend-version-and-responsibility-routing) | mined — effective dispatch, caller-owned stable inputs, branch-specific replay/update order and capture-resource failures → `ascend-aclgraph-capture-replay`; separate from the older skill-doc pin | mined — normal event-query-gated CPU descriptor-pool ownership and wait/recycle distinction → [host-memory-management](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/SKILL.md); source candidate, native tests not run; shared/tiering and allocator behavior excluded | out-of-scope | out-of-scope — offloader synchronization is not network evidence | out-of-scope — ACLGraph runtime supplies no new OS-control evidence; older Dockerfile/thread-env defaults remain scoped to bc8115ed | +| vllm-project/vllm-ascend @ `8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf` (Apache-2.0; separate runtime pin) | Dot bundle-mediated; `compilation/acl_graph.py`, pinned graph-mode guide and selected `test_acl_graph.py` cases; tests inspected by supplier, not run; caller/ENPU ordering and helpers untraced; native single-tier spec/worker, registration and DMA binding read directly ([seven-file ledger](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-native-offload-readset.json)); external vLLM dependencies unpinned; [separate model-adapter routing](../src/inferrsi/knowledge/data/skill_library/README.md#vllm-ascend-version-and-responsibility-routing) | mined — effective dispatch, caller-owned stable inputs, branch-specific replay/update order and capture-resource failures → `ascend-aclgraph-capture-replay`; separate from the older skill-doc pin | mined — normal event-query-gated CPU descriptor-pool ownership and wait/recycle distinction → [host-memory-management](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/SKILL.md); source candidate, native tests not run; shared/tiering and allocator behavior excluded | out-of-scope | out-of-scope — offloader synchronization is not network evidence | out-of-scope — ACLGraph runtime supplies no new OS-control evidence; older Dockerfile/thread-env defaults remain scoped to bc8115ed | | Ascend/pytorch (TorchNPU) @ `67511c53cd4c88014f6123de7097677e60e85faa` (upstream attribution notices retained in source notes) | Dot bundle-mediated; device/host allocators and bounded copy/record-stream/config/stats/binding/test paths in linked ledgers; underlying ATen host allocator and broader capture/IPC/driver behavior unread | mined — stream-use lifetime versus ordering, HBM counters/fragmentation and feature-gated controls → `torchnpu-device-memory-lifecycle`; tests unrun | mined — owned/external pinned buffers, host-specific counters, default/expandable paths and unresolved registration-version warning → `torchnpu-pinned-host-memory-lifecycle`; tests unrun | out-of-scope | out-of-scope — H2D/D2H, registration and IPC are not network coverage | out-of-scope — no OS-priority implementation | -| Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` (BSD-3-Clause) | Bounded source distillation, 2026-10-27; 10 retained files / 119,136 bytes matched to Git blobs; forward/update wrappers, compiled gates, selected packed/state branches, CUDA build/binding paths and test intent; [readset/notice](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/causal-conv1d-contract.md#source-ledger) | mined (narrow) — forward versus in-place update, dtype/layout/packed history, gathered state rows and circular cursors; build targets versus observed dispatch → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md); native/integration/numerical/performance not_run; sm_89 compatibility unqualified | out-of-scope — device convolution state is not host RAM | out-of-scope — wheel retrieval is not storage optimization | out-of-scope — no transport implementation extracted | out-of-scope — request-state ownership is not OS scheduling | -| mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` (Apache-2.0) | Earlier discovery retained; 2026-10-23 bounded AKG-MLIR CUDA Affine tiling pass/options/pipeline/driver and one static elementwise FileCheck fixture; [readset](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/akg-affine-tail-guards.md#source-ledger) | mined (narrow) — target/option/loop gates, full point-loop bounds and guarded body placement → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md); compiler/device execution and gains unverified; no Ascend qualification | unknown — host implementation not extracted | negative — scanned 2026-10-15, no LLM storage material (carried finding, not rescanned) | unknown — transport not extracted | negative — 2026-10-15 OS scan: identifier false-positives only (carried finding, not rescanned) | +| Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` (BSD-3-Clause) | Bounded source distillation, 2026-10-27; 10 retained files / 119,136 bytes matched to Git blobs; forward/update wrappers, compiled gates, selected packed/state branches, CUDA build/binding paths and test intent; [readset/notice](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/causal-conv1d-contract.md#source-ledger) | mined (narrow) — forward versus in-place update, dtype/layout/packed history, gathered state rows and circular cursors; build targets versus observed dispatch → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md); native/integration/numerical/performance not_run; sm_89 compatibility unqualified | out-of-scope — device convolution state is not host RAM | out-of-scope — wheel retrieval is not storage optimization | out-of-scope — no transport implementation extracted | out-of-scope — request-state ownership is not OS scheduling | +| mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` (Apache-2.0) | Earlier discovery retained; 2026-10-23 bounded AKG-MLIR CUDA Affine tiling pass/options/pipeline/driver and one static elementwise FileCheck fixture; [readset](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/akg-affine-tail-guards.md#source-ledger) | mined (narrow) — target/option/loop gates, full point-loop bounds and guarded body placement → [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md); compiler/device execution and gains unverified; no Ascend qualification | unknown — host implementation not extracted | negative — scanned 2026-10-15, no LLM storage material (carried finding, not rescanned) | unknown — transport not extracted | negative — 2026-10-15 OS scan: identifier false-positives only (carried finding, not rescanned) | | mindie-agent/mindie-agent @ 6a81b41 | full | out-of-scope — agent tooling, not serving performance | out-of-scope | mined (narrow) — logical-vs-physical I/O + cache-flush measurement caveat → `weight-loading-optimization` | out-of-scope | out-of-scope | | hahnyuan/LLM-Viewer @ 1893e4b | full | mined — roofline model + hardware params → `performance-ceiling-analysis` (L1) | out-of-scope | out-of-scope | out-of-scope | out-of-scope | | ebugger/Empirical-Roofline-Toolkit @ 0771fa9 | full | mined — empirical roofline method → `performance-ceiling-analysis` (L2) | out-of-scope | out-of-scope | out-of-scope | out-of-scope | -| foundation-model-stack/fastsafetensors @ 219e256 | full (shallow pin, wave 5) | out-of-scope — loading/collective delivery only | mined (partial) — pinned bounce-buffer staging, reader NUMA binding → `weight-loading-optimization` §7 | mined — loading I/O mechanics (no-mmap design, bounce reader, GDS, readiness overlap, partial reads, static planner) + measurement protocol → `weight-loading-optimization` §7; wave-6 approved `saver.py` semantics → §8 (carried evidence; not re-mined this wave) | mined (source-only) — file owner → whole-tensor broadcast / TP scatter; PyTorch coalesced-run selection, readiness, storage ownership and early close → `weight-loading-optimization` §9; [full-pin readset](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/fastsafetensors-transfer-readset.md); local copies excluded, transport/performance unverified | mined (narrow) — reader worker NUMA binding + O_DIRECT per-filesystem policy → `weight-loading-optimization` §7 | +| foundation-model-stack/fastsafetensors @ 219e256 | full (shallow pin, wave 5) | out-of-scope — loading/collective delivery only | mined (partial) — pinned bounce-buffer staging, reader NUMA binding → `weight-loading-optimization` §7 | mined — loading I/O mechanics (no-mmap design, bounce reader, GDS, readiness overlap, partial reads, static planner) + measurement protocol → `weight-loading-optimization` §7; wave-6 approved `saver.py` semantics → §8 (carried evidence; not re-mined this wave) | mined (source-only) — file owner → whole-tensor broadcast / TP scatter; PyTorch coalesced-run selection, readiness, storage ownership and early close → `weight-loading-optimization` §9; [full-pin readset](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/fastsafetensors-transfer-readset.md); local copies excluded, transport/performance unverified | mined (narrow) — reader worker NUMA binding + O_DIRECT per-filesystem policy → `weight-loading-optimization` §7 | | LMCache/LMCache @ c3b875a | full (shallow pin, wave 5) | out-of-scope — approved scope was CPU/local-storage KV management | mined — L1 DRAM tier, admission dial, watermark eviction, multi-L1 overflow → `kv-tier-management` | mined — L2 local adapters (fs/nixl/raw_block/dax), serde capacity lever, persist/recover → `kv-tier-management` | out-of-scope — remote adapters noted only | mined (narrow) — MP process/thread concurrency model (isolation, client-affinity pools, TTL locks) → `kv-tier-management` | -| sgl-project/sglang @ `df13d5dce31abfeff0e701a62e8c39a136758e6c` (Apache-2.0; independent diffusion supplement) | Bounded multimodal_gen monolithic client/scheduler/worker, composed pipeline and stage/component-use call chain; [readset](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md); supplier checked one selected blob, not whole tree | mined — stage/use readiness, retention and executor rank boundaries → [audit-sglang-diffusion-lifetimes](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md); native tests not run | mined — host request/job/scheduler-object and finalizer lifetimes only; device memory is not RAM | out-of-scope — output-file API is not storage-I/O intelligence | out-of-scope — H2D/IPC and barrier declarations do not qualify network transport | out-of-scope — engine queues are not OS scheduling | -| ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT; loading/host snapshot) | full (shallow pin, wave 5); [owner scope/pin map](../src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope); static inspection 2026-10-22 carried | out-of-scope — approved scope was loading modes and host placement | mined — mmap/mlock residency, lazy tensors, host weight placement (`--cpu-moe` family), KV host offload → `weight-loading-optimization` §6 | mined — loading modes (mmap/mlock/dio/lazy), page-cache interactions, fit-probe bypass → `weight-loading-optimization` §6 | out-of-scope | mined — NUMA modes (distribute/isolate/numactl), gen-vs-batch thread/affinity splits, numa_balancing rule → `os-process-placement` | -| ggml-org/llama.cpp @ `707616feb5f379e99ff7442680a55b4a1704c7c9` (MIT; separate runtime snapshot, static inspection 2026-10-21 carried) | [Owner scope/pin map](../src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope); reading bundle; bounded context/memory/graph/scheduler and server task/slot call path; native assertions inspected, not run; all model builders, memory variants and backend kernels/capture caches not audited | mined — partial-ubatch progress, topology/input graph-reuse gates and backend copy/event obligations → `audit-llama-runtime-state`; actual backend execution unverified | mined (narrow) — host-owned task/result waiters and slot prompt-token bookkeeping; stateful retention versus cancellation; device KV is not host RAM | out-of-scope — older loading owner/pin retained | out-of-scope — scheduler copies do not establish network/RPC coverage | out-of-scope — no new placement or OS controls | +| sgl-project/sglang @ `df13d5dce31abfeff0e701a62e8c39a136758e6c` (Apache-2.0; independent diffusion supplement) | Bounded multimodal_gen monolithic client/scheduler/worker, composed pipeline and stage/component-use call chain; [readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md); supplier checked one selected blob, not whole tree | mined — stage/use readiness, retention and executor rank boundaries → [audit-sglang-diffusion-lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md); native tests not run | mined — host request/job/scheduler-object and finalizer lifetimes only; device memory is not RAM | out-of-scope — output-file API is not storage-I/O intelligence | out-of-scope — H2D/IPC and barrier declarations do not qualify network transport | out-of-scope — engine queues are not OS scheduling | +| ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT; loading/host snapshot) | full (shallow pin, wave 5); [owner scope/pin map](../src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope); static inspection 2026-10-22 carried | out-of-scope — approved scope was loading modes and host placement | mined — mmap/mlock residency, lazy tensors, host weight placement (`--cpu-moe` family), KV host offload → `weight-loading-optimization` §6 | mined — loading modes (mmap/mlock/dio/lazy), page-cache interactions, fit-probe bypass → `weight-loading-optimization` §6 | out-of-scope | mined — NUMA modes (distribute/isolate/numactl), gen-vs-batch thread/affinity splits, numa_balancing rule → `os-process-placement` | +| ggml-org/llama.cpp @ `707616feb5f379e99ff7442680a55b4a1704c7c9` (MIT; separate runtime snapshot, static inspection 2026-10-21 carried) | [Owner scope/pin map](../src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope); reading bundle; bounded context/memory/graph/scheduler and server task/slot call path; native assertions inspected, not run; all model builders, memory variants and backend kernels/capture caches not audited | mined — partial-ubatch progress, topology/input graph-reuse gates and backend copy/event obligations → `audit-llama-runtime-state`; actual backend execution unverified | mined (narrow) — host-owned task/result waiters and slot prompt-token bookkeeping; stateful retention versus cancellation; device KV is not host RAM | out-of-scope — older loading owner/pin retained | out-of-scope — scheduler copies do not establish network/RPC coverage | out-of-scope — no new placement or OS controls | | axboe/fio @ acb7a302 | pack-mediated (wave 6; HOWTO.rst targeted sections + `examples/aio-read.fio`) | out-of-scope | located-not-mined — cache-state and run-shape material partially feeds the ledger/run-classification rules | mined — workload matching, requested-vs-achieved depth, cache-state ledger, run-shape classification, output schema/units → `storage-io-measurement` | out-of-scope | out-of-scope | | sysstat/sysstat @ efd0c445 | pack-mediated (wave 6; `man/iostat.in`, `man/pidstat.1`) | out-of-scope | mined (narrow) — per-process I/O/fault attribution caveats → `storage-io-measurement` §8 | mined — per-device attribution, saturation guards (%util/%iowait) → `storage-io-measurement` §6-7 | out-of-scope | mined (narrow) — CPU-side scheduling-pressure vs disk-hypothesis rules (%wait, context switches) → `storage-io-measurement` §8 | | Linux cgroup v2 docs — kernel.org live 2026-10-15 (revision unidentified) + v6.12 `cgroup-v2.rst` (reproducible baseline) | pack-mediated (wave 6; core/CPU/Memory/IO/Cpuset interface sections) | out-of-scope | mined — memory pressure/events triage, high/low/max semantics → `cgroup-resource-controls` | mined (conditional) — io.max caps gated on confirmed block-backed traffic → `cgroup-resource-controls` | out-of-scope | mined — cpu.max/weight quota-vs-contention (cpu.stat vs .local), requested/effective cpuset, hierarchy/delegation preflight → `cgroup-resource-controls` | @@ -86,19 +86,19 @@ Reading rules: | **B1 / NVIDIA DALI** @ `6fe47c48d3b0ca335f5e1afd3c77c750b2c7e041` (Apache-2.0) | `pipeline.py` constructor/API, `executor_statistics`, queue deserialization; Python surface only | mined — CPU/GPU stage overlap and executor-mode gates → `training-input-pipeline` | mined — queue/buffer sizing and per-output real/reserved memory statistics; legacy stats require `exec_dynamic=False` | out-of-scope | out-of-scope — host/device copies and local buffers are not network evidence | out-of-scope | | **B2 / Mosaic Streaming** @ `d99bf9c7cdf2dd4ed62a4960ed35270b93337a5f` (Apache-2.0) | `dataset.py` configuration, cache admission/eviction, prepare/ready/yield loops | out-of-scope | mined — per-worker sample look-ahead versus memory/cache budget → `training-input-pipeline` | mined — raw+compressed shard admission, cache minimum and LRU/redownload trade-off | mined — remote dataset download look-ahead/cache-churn decisions only; no collective optimization | out-of-scope | | **B3 / FFCV** @ `3a12966b3afe3a81733a732e633317d747bfaac7` (Apache-2.0; historical 2024 pin) | `loader.py` configuration/manager selection, complete `os_cache.py`; process-cache algorithm not read | out-of-scope | mined — dataset-fits-RAM versus process cache, batches-ahead buffers → `training-input-pipeline` | mined — read-only mmap and documented quasi-random read-locality choice; `.beton` required | out-of-scope | mined — cache ownership and traversal configuration; distributed `QUASI_RANDOM` excluded, current compatibility unverified | -| **C3 / DLRover FlashCheckpoint** @ `e35b1ad90e2928ca6bacf5d22117f713566c5cb0` (Apache-2.0) | DDP/full engine → common-directory saver → POSIX storage; full/sharded CPU/Gloo replicas and restore consumer; [readset](../src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md#dlrover-extension-inspected-2026-10-22) | located-not-mined — HCCL device branch observed; NPU compatibility not validated | mined — memory readiness/step gates and replica byte/metadata ownership → `checkpoint-recovery-consistency`; restored values unverified | mined — queued save versus shard/done/tracker completion; serializer/synchronization boundaries, no durability proof | mined — actual full/sharded Gloo byte/metadata paths and world-position groups; gather/consumer correctness and throughput unverified | mined — worker versus helper/host lifetime; fallback daemon cannot persist after parent exits | +| **C3 / DLRover FlashCheckpoint** @ `e35b1ad90e2928ca6bacf5d22117f713566c5cb0` (Apache-2.0) | DDP/full engine → common-directory saver → POSIX storage; full/sharded CPU/Gloo replicas and restore consumer; [readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/sources.md#dlrover-extension-inspected-2026-10-22) | located-not-mined — HCCL device branch observed; NPU compatibility not validated | mined — memory readiness/step gates and replica byte/metadata ownership → `checkpoint-recovery-consistency`; restored values unverified | mined — queued save versus shard/done/tracker completion; serializer/synchronization boundaries, no durability proof | mined — actual full/sharded Gloo byte/metadata paths and world-position groups; gather/consumer correctness and throughput unverified | mined — worker versus helper/host lifetime; fallback daemon cannot persist after parent exits | | **C4 / NVIDIA Resiliency Extension** @ `49af0b03f14550d36a60dcfebd573848b5fd8435` (Apache-2.0 source headers; Megatron-Core lineage) | local `base_manager.py` complete-iteration selection/save/finalizer and `local_manager.py` dirty/final file lifecycle | mined (contract only) — CUDA D2H synchronized before async request returns; no NPU qualification | mined — snapshot/replica lifetime constraints → `checkpoint-recovery-consistency` | mined — clean/dirty publication, complete-rank generation selection and incomplete-save rejection; no power-loss proof | mined — all-rank availability/finalization and replica failure domains, not transport throughput | out-of-scope | | **CR1 / DeepSpeed ZeRO-Infinity** @ `5f266e26d216a705d9c64cac88fb813cefdbac4b` (Apache-2.0) | complete short `offload_config.py`, official ZeRO-3 live offload-field docs; no runtime/checkpoint audit | mined — parameter stage-3 and optimizer stage-1/2/3 boundaries → `training-resource-budget` | mined — CPU optimizer budget, pinned RAM, element-count residency and gated core share | mined — NVMe buffer pools and tiled read/compute/write overlap intent; buffer-size units unresolved in approved excerpt | out-of-scope — CPU/device transfer is not network coverage | mined — CPU resource-budget qualification; no privileged changes | -| **CR5 / Ascend MindSpeed adaptive memory** — historical `core_r0.8.0` document @ `53193db8bf5def407c4041a9beb3f4c9e9773daf` (root default BSD-3-Clause terms; identified file exceptions) | 2026-10-15 document inspection retained; [2026-10-30 pinned readset](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/mindspeed-source-readset.md): feature guard, allocator/callback, saved-storage rescue, wrapper and affinity branch | mined (historical source candidate) — spare-memory/recompute/swap decision and effective selection → `training-resource-budget`; changed-layer comparison rejected as controlled gain | mined — eligible shared-storage CPU rescue/return and allocator prerequisites; no whole-model validation | out-of-scope | out-of-scope — NPU/CPU copies not network evidence | mined (historical conditions) — validation versus dispatch exclusions; 910A/910B binding, topology/tools and CPU-share assumptions; no current compatibility claim | -| InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ `4a350344027960a48384626aa646f1765fda6d4f` (root Apache-2.0; model attribution in NOTICE) | Bounded manager/runner/scheduler/Ouro/host-metadata/Torch-consumer chain; [candidate](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md), [source map](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md), [construction/cases](reviews/issue91-vllm-rlt-finalization.md); historical [intake](reviews/issue91-vllm-rlt-intake.md) retained | source-guidance — synchronous LAST_EXITED copying versus SHARED, written-prefix validity, later deeper attention and normal release/wait/free; candidate, native runtime and efficacy not_run | out-of-scope — host bookkeeping does not establish host-RAM coverage | out-of-scope — no storage mechanism extracted | out-of-scope — local tensor copies; no transport mechanism extracted | out-of-scope — no OS mechanism extracted | -| vllm-project/afd-plugin @ `df8d6a0c403c3ca1b4ed9cf1687e7204908bebec` (Apache-2.0 selected Python readset; retained MIT CAM subset notice) | Fixed-pin topology/factory/runner/metadata/DeepSeek dense proxy and P2pNccl reads; [full ledger](../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md); [historical intake](reviews/issue91-afd-plugin-intake.md) retained | mined (bounded source-only candidate) — role/world/subgroup rank ledger and eager dense-FFN per-stage/per-peer payload ownership → [audit-afd-p2p-exchanges](../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md); native/backend/model validation not_run | out-of-scope — no host-memory mechanism extracted | out-of-scope — no storage mechanism extracted | source call chain mined — actual NCCL-backed send/recv consumer identified; underlying completion, graph and multi-host/physical transport unqualified; no Network coverage credit | out-of-scope — no OS mechanism extracted | +| **CR5 / Ascend MindSpeed adaptive memory** — historical `core_r0.8.0` document @ `53193db8bf5def407c4041a9beb3f4c9e9773daf` (root default BSD-3-Clause terms; identified file exceptions) | 2026-10-15 document inspection retained; [2026-10-30 pinned readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/mindspeed-source-readset.md): feature guard, allocator/callback, saved-storage rescue, wrapper and affinity branch | mined (historical source candidate) — spare-memory/recompute/swap decision and effective selection → `training-resource-budget`; changed-layer comparison rejected as controlled gain | mined — eligible shared-storage CPU rescue/return and allocator prerequisites; no whole-model validation | out-of-scope | out-of-scope — NPU/CPU copies not network evidence | mined (historical conditions) — validation versus dispatch exclusions; 910A/910B binding, topology/tools and CPU-share assumptions; no current compatibility claim | +| InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ `4a350344027960a48384626aa646f1765fda6d4f` (root Apache-2.0; model attribution in NOTICE) | Bounded manager/runner/scheduler/Ouro/host-metadata/Torch-consumer chain; [candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md), [source map](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md), [construction/cases](reviews/issue91-vllm-rlt-finalization.md); historical [intake](reviews/issue91-vllm-rlt-intake.md) retained | source-guidance — synchronous LAST_EXITED copying versus SHARED, written-prefix validity, later deeper attention and normal release/wait/free; candidate, native runtime and efficacy not_run | out-of-scope — host bookkeeping does not establish host-RAM coverage | out-of-scope — no storage mechanism extracted | out-of-scope — local tensor copies; no transport mechanism extracted | out-of-scope — no OS mechanism extracted | +| vllm-project/afd-plugin @ `df8d6a0c403c3ca1b4ed9cf1687e7204908bebec` (Apache-2.0 selected Python readset; retained MIT CAM subset notice) | Fixed-pin topology/factory/runner/metadata/DeepSeek dense proxy and P2pNccl reads; [full ledger](../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md); [historical intake](reviews/issue91-afd-plugin-intake.md) retained | mined (bounded source-only candidate) — role/world/subgroup rank ledger and eager dense-FFN per-stage/per-peer payload ownership → [audit-afd-p2p-exchanges](../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md); native/backend/model validation not_run | out-of-scope — no host-memory mechanism extracted | out-of-scope — no storage mechanism extracted | source call chain mined — actual NCCL-backed send/recv consumer identified; underlying completion, graph and multi-host/physical transport unqualified; no Network coverage credit | out-of-scope — no OS mechanism extracted | Wave-8 implementation provenance (Dot bundles integrated 2026-10-15): -[admission](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/source-notes.md), -[KV lifetimes](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/references/source-notes.md), -[ACLGraph](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/references/source-notes.md), -[device HBM](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/references/source-notes.md), -[pinned host RAM](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/references/source-notes.md). +[admission](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/source-notes.md), +[KV lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/references/source-notes.md), +[ACLGraph](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-aclgraph-capture-replay/references/source-notes.md), +[device HBM](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-device-memory-lifecycle/references/source-notes.md), +[pinned host RAM](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle/references/source-notes.md). These preserve the supplier's 2026-10-15 code/docs inspection; upstream reads were not repeated locally. F denotes pinned behavior or an identified contract; P denotes an unvalidated procedure. The three source pins are independent, @@ -132,7 +132,7 @@ attribution, owner readsets and candidate/imported limits remain unchanged. ### vLLM retained source responsibilities -Use the existing [engine/scheduler/KV responsibility map](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/SKILL.md#choose-the-source-and-responsibility) +Use the existing [engine/scheduler/KV responsibility map](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/SKILL.md#choose-the-source-and-responsibility) to distinguish the `2521446ac8c92937b5a32b7dc2a84ef7fa8e99c5` engine guide (inspection 2026-10-14) from the `7f730b4cea9a66520a9200265928ebe016b48a39` admission and connector-lifetime ledgers (supplier inspection 2026-10-15). @@ -150,23 +150,23 @@ not fabricated implementation SHAs or a version-matched runtime stack. | Original source / version | Inspected scope and local evidence | Mechanism / consumer | Evidence limits | |---|---|---|---| -| Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2) | [A readset](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md); §§3–7, A.1–A.2 | mined — operator/queue model calibration and joint-SLO shortlist → [calibrate-serving-simulation](../src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), ceiling analysis | Empirical prediction, not physical bound; near-capacity CPU overhead and 12.65% maximum error in the specified appendix case; no current runtime calibration | -| [AMD Infera @ ff1136e4](https://github.com/AMD-AGI/Infera/tree/ff1136e4a51998eba2933ed63cfb9c10120f86ea) | Direct bounded source inspection, 2026-11-03: serving CLI/evaluator/history, seed/objective parser, launcher/DES countercheck and named unexecuted test definitions; [16-path readset and rights](../src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/source-record.json) | mined — `calibrate-serving-simulation`: one [calibration-owned provenance reference](../src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md); search-policy links its seed comparator and evaluated-trial ledger | Source-derived candidate; empirical not_run. Scalar tuner scoring and DES percentiles are separate; missing SLO fields and stale no-P/D-queues text are qualified. Selected original paths MIT; no runtime, ranking accuracy, quality, service or agent-benefit result. | -| Mooncake — FAST '25 proceedings, pp. 155-170 | Direct public-paper read, retained qualification 2026-10-31; [source condition and readset](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md), §2.2, Table 1, Eqs. 1-2 | mined — conditional prefix-KV transfer versus saved-prefill-work comparison → `performance-ceiling-analysis` | Analytical gate only; current inputs and E2E benefit unverified. SSD mechanics excluded. Original transfer-gates extraction history remains positive; missing synthesis novelty UNKNOWN. Raw-paper redistribution permission UNKNOWN. | -| FlexGen — [ICML 2023 / PMLR 202:31094–31116](https://proceedings.mlr.press/v202/sheng23a.html), official ten-author edition | Direct scoped public-paper qualification, 2026-10-31; [condition, exact digest and readset](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/flexgen-offload-feasibility.md), §§3–4.3 and selected Appendix A.3 formulas; Figures 1–3, Algorithm 1, Eq. 1, Tables 2–3 | mined — joint weights/KV/activation placement, resident plus working-storage peaks and offline latency/throughput conditions → `performance-ceiling-analysis` | Source-derived candidate; empirical not_run. Perfect-overlap fitted model, relaxed-placement OOM and unresolved `bls < 20` versus `32 × 8` wording retained; no current runtime, quality or gain qualification. Attributed CC BY 4.0 synthesis; missing-archive novelty UNKNOWN. | -| Orca — OSDI 2022 proceedings | [Exact readset](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md); §§3–6, Algorithm 1 | mined — iteration eligibility, selective batching, FCFS/reservation → [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/SKILL.md) | Historical engine/harness and normalized-median denominators; not present vLLM behavior or fairness proof | -| Sarathi-Serve — [OSDI 2024 proceedings, pp. 117–134](https://www.usenix.org/conference/osdi24/presentation/agrawal) | Fresh official-edition grounding, 2026-10-30; full text and artifact appendix; §§4.1–4.3, §5 setup, Figures 9/14 and Table 4 checked for the [case record](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json) | mined — decode/partial-prefill/new-request token accounting and chunk-cost tradeoffs → [one optimization-search-policy case](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md) | Source-derived candidate; empirical not_run. Combined design has worse P50 TTFT than hybrid-only in both Table 4 traces; no current-engine, universal-budget or fairness claim. Raw source reuse rights UNKNOWN; no archive recovery or novelty over unavailable supplier synthesis claimed. | -| FastServe — [2305.05920v3](https://arxiv.org/abs/2305.05920v3) | [Exact readset](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md) | mined — service priority, promotion and suspended-state host residency → serving-preemption candidate | Aggregate tail improvement is not tenant fairness; same-engine contrast distinct from historical-vLLM comparison | -| Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1) | [Exact readset](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/llumnix-readset.md) | mined — replica migration, virtual headroom, pre-copy/downtime and host staging → [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | Virtual headroom is not physical capacity; normal-class cost and evaluated-artifact fault-tolerance limit retained | -| LoongServe — [2404.09526v2](https://arxiv.org/abs/2404.09526v2) | [Exact readset](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md) | mined — elastic sequence groups, phase-dependent state placement and ongoing attention communication → elastic-placement candidate | Distributed device KV is not host RAM; group transitions have different migration costs; no current engine/NPU claim | -| ZeRO-Offload — [2101.06840v1](https://arxiv.org/abs/2101.06840v1) | [C readset](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md), ZO1–ZO3 [mechanisms](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/notes/training-memory-scheduling.md) | mined — CPU optimizer ownership, update/return pipeline and separately gated delayed updates → [training-resource-budget](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/SKILL.md) | Restricted graph-cut model; Table 1 discrepancy; delayed updates change semantics; not the DeepSpeed config pin | -| Checkmate — [1910.02653v3](https://arxiv.org/abs/1910.02653v3) | [C readset](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md), CM1–CM2 | mined — event-level retain/recompute and dependency versus capacity feasibility → [plan-activation-residency](../src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/SKILL.md) | Static-cost model and rounded-plan limits; activation rematerialization is not recovery checkpointing | -| OLLA — [2210.12924v2](https://arxiv.org/abs/2210.12924v2) | [C readset](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md), OL1–OL2 | mined — lifetime shortening and static address packing → activation-residency candidate | Device address placement is not host/SSD tiering; runtime amortization estimate is not measured E2E gain | -| Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3) | [C readset](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md), AL1–AL2 | mined — stage/mesh/sharding and cross-mesh resharding → [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | Conditional hierarchical plan, not global optimum; intra-op compute simplification does not describe all stage costs | -| Zero Bubble — [2401.10241v1](https://arxiv.org/abs/2401.10241v1) | [C readset](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md), ZB1–ZB3 | mined — split input/weight gradients, activation caps, V schedules and optimizer rollback → [assess-split-backward-schedule](../src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/SKILL.md) | Calculated bubbles versus measured throughput; activation cap versus total HBM; rollback is distinct from stale updates | -| FlashAttention — [2205.14135v2](https://arxiv.org/abs/2205.14135v2) | [D readset](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/sources.md) and [mechanisms](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/mechanisms.md) | mined — online statistics, tiled IO and backward recomputation → [diagnose-attention-bottlenecks](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md) | Scoped dense exact-attention/fast-memory assumptions; operation mix can change after IO reduction | -| FlashAttention-2 — [2307.08691v1](https://arxiv.org/abs/2307.08691v1) | Same [D readset](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/sources.md) | mined — non-matmul work, output ownership and independent query tiles → attention diagnosis | Combined interventions are not isolated causal ablations; Accelerator only | -| FlashAttention-3 — [2407.08608v2](https://arxiv.org/abs/2407.08608v2) | Same [D readset](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/sources.md); [author-reported cases](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md) | mined — async dependencies, compiler/register trade-offs and FP8 layout/accuracy → attention diagnosis | Hopper primitives establish no Ascend support; synthetic accuracy is not model-quality validation | +| Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2) | [A readset](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/READSET.md); §§3–7, A.1–A.2 | mined — operator/queue model calibration and joint-SLO shortlist → [calibrate-serving-simulation](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/SKILL.md), ceiling analysis | Empirical prediction, not physical bound; near-capacity CPU overhead and 12.65% maximum error in the specified appendix case; no current runtime calibration | +| [AMD Infera @ ff1136e4](https://github.com/AMD-AGI/Infera/tree/ff1136e4a51998eba2933ed63cfb9c10120f86ea) | Direct bounded source inspection, 2026-11-03: serving CLI/evaluator/history, seed/objective parser, launcher/DES countercheck and named unexecuted test definitions; [16-path readset and rights](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/source-record.json) | mined — `calibrate-serving-simulation`: one [calibration-owned provenance reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/README.md); search-policy links its seed comparator and evaluated-trial ledger | Source-derived candidate; empirical not_run. Scalar tuner scoring and DES percentiles are separate; missing SLO fields and stale no-P/D-queues text are qualified. Selected original paths MIT; no runtime, ranking accuracy, quality, service or agent-benefit result. | +| Mooncake — FAST '25 proceedings, pp. 155-170 | Direct public-paper read, retained qualification 2026-10-31; [source condition and readset](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md), §2.2, Table 1, Eqs. 1-2 | mined — conditional prefix-KV transfer versus saved-prefill-work comparison → `performance-ceiling-analysis` | Analytical gate only; current inputs and E2E benefit unverified. SSD mechanics excluded. Original transfer-gates extraction history remains positive; missing synthesis novelty UNKNOWN. Raw-paper redistribution permission UNKNOWN. | +| FlexGen — [ICML 2023 / PMLR 202:31094–31116](https://proceedings.mlr.press/v202/sheng23a.html), official ten-author edition | Direct scoped public-paper qualification, 2026-10-31; [condition, exact digest and readset](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/flexgen-offload-feasibility.md), §§3–4.3 and selected Appendix A.3 formulas; Figures 1–3, Algorithm 1, Eq. 1, Tables 2–3 | mined — joint weights/KV/activation placement, resident plus working-storage peaks and offline latency/throughput conditions → `performance-ceiling-analysis` | Source-derived candidate; empirical not_run. Perfect-overlap fitted model, relaxed-placement OOM and unresolved `bls < 20` versus `32 × 8` wording retained; no current runtime, quality or gain qualification. Attributed CC BY 4.0 synthesis; missing-archive novelty UNKNOWN. | +| Orca — OSDI 2022 proceedings | [Exact readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md); §§3–6, Algorithm 1 | mined — iteration eligibility, selective batching, FCFS/reservation → [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/SKILL.md) | Historical engine/harness and normalized-median denominators; not present vLLM behavior or fairness proof | +| Sarathi-Serve — [OSDI 2024 proceedings, pp. 117–134](https://www.usenix.org/conference/osdi24/presentation/agrawal) | Fresh official-edition grounding, 2026-10-30; full text and artifact appendix; §§4.1–4.3, §5 setup, Figures 9/14 and Table 4 checked for the [case record](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/cases/sarathi-chunk-budget.json) | mined — decode/partial-prefill/new-request token accounting and chunk-cost tradeoffs → [one optimization-search-policy case](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/cases/sarathi-chunk-budget.md) | Source-derived candidate; empirical not_run. Combined design has worse P50 TTFT than hybrid-only in both Table 4 traces; no current-engine, universal-budget or fairness claim. Raw source reuse rights UNKNOWN; no archive recovery or novelty over unavailable supplier synthesis claimed. | +| FastServe — [2305.05920v3](https://arxiv.org/abs/2305.05920v3) | [Exact readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md) | mined — service priority, promotion and suspended-state host residency → serving-preemption candidate | Aggregate tail improvement is not tenant fairness; same-engine contrast distinct from historical-vLLM comparison | +| Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1) | [Exact readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/llumnix-readset.md) | mined — replica migration, virtual headroom, pre-copy/downtime and host staging → [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) | Virtual headroom is not physical capacity; normal-class cost and evaluated-artifact fault-tolerance limit retained | +| LoongServe — [2404.09526v2](https://arxiv.org/abs/2404.09526v2) | [Exact readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md) | mined — elastic sequence groups, phase-dependent state placement and ongoing attention communication → elastic-placement candidate | Distributed device KV is not host RAM; group transitions have different migration costs; no current engine/NPU claim | +| ZeRO-Offload — [2101.06840v1](https://arxiv.org/abs/2101.06840v1) | [C readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md), ZO1–ZO3 [mechanisms](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/notes/training-memory-scheduling.md) | mined — CPU optimizer ownership, update/return pipeline and separately gated delayed updates → [training-resource-budget](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/SKILL.md) | Restricted graph-cut model; Table 1 discrepancy; delayed updates change semantics; not the DeepSpeed config pin | +| Checkmate — [1910.02653v3](https://arxiv.org/abs/1910.02653v3) | [C readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md), CM1–CM2 | mined — event-level retain/recompute and dependency versus capacity feasibility → [plan-activation-residency](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/SKILL.md) | Static-cost model and rounded-plan limits; activation rematerialization is not recovery checkpointing | +| OLLA — [2210.12924v2](https://arxiv.org/abs/2210.12924v2) | [C readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md), OL1–OL2 | mined — lifetime shortening and static address packing → activation-residency candidate | Device address placement is not host/SSD tiering; runtime amortization estimate is not measured E2E gain | +| Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3) | [C readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md), AL1–AL2 | mined — stage/mesh/sharding and cross-mesh resharding → [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/SKILL.md) | Conditional hierarchical plan, not global optimum; intra-op compute simplification does not describe all stage costs | +| Zero Bubble — [2401.10241v1](https://arxiv.org/abs/2401.10241v1) | [C readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md), ZB1–ZB3 | mined — split input/weight gradients, activation caps, V schedules and optimizer rollback → [assess-split-backward-schedule](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-split-backward-schedule/SKILL.md) | Calculated bubbles versus measured throughput; activation cap versus total HBM; rollback is distinct from stale updates | +| FlashAttention — [2205.14135v2](https://arxiv.org/abs/2205.14135v2) | [D readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/sources.md) and [mechanisms](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/mechanisms.md) | mined — online statistics, tiled IO and backward recomputation → [diagnose-attention-bottlenecks](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/SKILL.md) | Scoped dense exact-attention/fast-memory assumptions; operation mix can change after IO reduction | +| FlashAttention-2 — [2307.08691v1](https://arxiv.org/abs/2307.08691v1) | Same [D readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/sources.md) | mined — non-matmul work, output ownership and independent query tiles → attention diagnosis | Combined interventions are not isolated causal ablations; Accelerator only | +| FlashAttention-3 — [2407.08608v2](https://arxiv.org/abs/2407.08608v2) | Same [D readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/sources.md); [author-reported cases](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md) | mined — async dependencies, compiler/register trade-offs and FP8 layout/accuracy → attention diagnosis | Hopper primitives establish no Ascend support; synthetic accuracy is not model-quality validation | FlashVector's host materialization/assembly and Agentic Kernel Optimization's regime-specific kernel cases are cross-links to the **same** source identities in @@ -189,11 +189,11 @@ The separate Sarathi-Serve case above uses a freshly pinned official edition; it does not recover or import that archive or its supplier-authored synthesis. Separately, the fresh official OSDI 24 DistServe edition is retained in the -[phase-placement reference](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/README.md) +[phase-placement reference](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/distserve-phase-placement/README.md) under `optimization-search-policy` (`candidate`, empirical validation `not_run`). The 2026-10-30 full-paper qualification and selected visual checks support the independent-versus-paired-stage feasible sets, joint-SLO per-GPU denominator and -simulation limits; the [source record](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/source-record.json) +simulation limits; the [source record](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/distserve-phase-placement/source-record.json) retains the exact edition, readset and UNKNOWN rights. This source requalification does not recover the archive, establish novelty over its unseen synthesis or validate a current serving engine. @@ -208,7 +208,7 @@ validate a current serving engine. sparse checkouts). Queries used: safetensors, mmap, memory_map, pinned, O_DIRECT, posix_fadvise, readahead, page cache, fio, iostat, nvme, startup time, load latency, async save. Recorded 2026-10-15; also embedded in - `src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md`. + `src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md`. - **cgroups / cpuset / isolcpus / nice / ionice / SCHED_FIFO**: zero hits in the same 8 repos and scope. `os-process-placement` covers placement, affinity, thread env, and container flags/ulimits only. @@ -247,14 +247,14 @@ SSD/OS systems columns. | microsoft/SkillOpt @ 343db22 | evolution | bounded-edit skill evolution | `npu-skill-factory`, `skill-evolution-playbook`, `dense-performance-evidence`, `optimization-search-policy` | | Qwen-Applications/Trace2Skill @ 3d0b52a | evolution | Trace2Skill pool consolidation | `skill-evolution-playbook` | | JiusiServe/vllm-evolve @ 43cc82c | search | scheduling-algorithm evolution, search policy | `optimization-search-policy`, `dense-performance-evidence`, `inference-benchmark-design` | -| [OpenBMB/JustGoal repo + paired paper](https://github.com/OpenBMB/JustGoal/tree/f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d) @ `f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d` (repository Apache-2.0 default; NOTICE BSD exceptions; paper license UNKNOWN) | evaluation | Bounded direct inspection, 2026-11-02: complete 21-page paper/appendices and selected DSA wrapper/CUDA workspace/stream contracts; [exact readset and exclusions](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-source.json). Source-only methodology; author-reported comparisons, released failures versus repaired timing and official/stricter oracle differences retained | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-evaluation-contracts.md); candidate/not_run, efficacy not_evaluated; no new implementation/backend coverage or full campaign runner | -| [AutoTuneBench repo + paper v1](https://github.com/li-ch/autotunebench/tree/64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f) @ `64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f`; [arXiv:2609.18123v1](https://arxiv.org/abs/2609.18123v1) (selected repository code/data MIT; paper CC BY 4.0) | evaluation | Bounded source inspection, 2026-11-02: treatment delivery, arm-validity precedence, exported protocol/check/record gaps, weak-denominator and anomaly counterexamples; complete 15-page paper, 6 figures, 7 tables and [exact selected readset/rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json) | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md); source-derived candidate/not_run, efficacy not_evaluated; invalid-arm/source-median conflicts retained, no new controller or measured implementation coverage | -| [Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/6cd3ee98dfe32746b345a6abe521fa4cb9bdc187) @ `6cd3ee98dfe32746b345a6abe521fa4cb9bdc187`; [arXiv:2605.26118v1](https://arxiv.org/html/2605.26118v1) (repository Apache-2.0; paper CC BY 4.0) | evaluation | Bounded source inspection, 2026-11-02: 18-file trial identity/origin, external outcome, selection/finalization/reporting and baseline-oracle readset; complete paper HTML/appendix/tables/algorithm/listing/captions, unused graphic values unverified; [exact hashes, rights and exclusions](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-source.json) | [`inference-benchmark-design` reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-trial-evidence.md); candidate/not_run, efficacy not_evaluated; two measured meanings, named-verdict update limits and manual versus measured-only defaults retained; no runtime integration or whole-model/backend promotion | -| [Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc98043fcf89f3589aa5addb3742777a) @ `1882c6adbc98043fcf89f3589aa5addb3742777a` (selected software/docs Apache-2.0) | evaluation | Bounded source inspection, 2026-11-03: Qwen3.5 GDN/MTP eager record/fold and checkpoint/adoption/retirement contracts, unexecuted fold-test definitions and historical completion denominators; [27-file readset and rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-source.json) | [`inference-benchmark-design` reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md); candidate/not_run, efficacy not_evaluated. Historical author results retain their separate execution revision; no new implementation, backend or measured resource coverage | -| [shenh10/HuggingArch](https://github.com/shenh10/HuggingArch/tree/facbbbc96856e558b9084a94a5070b2e2c86cf0b) @ `facbbbc96856e558b9084a94a5070b2e2c86cf0b` (MIT); companion post dated 2026-10-08, live, not attributable to the pin | evaluation | Bounded source inspection, 2026-11-11: checkpoint join and weight ledger code, validator and parallelism documents, upload speed-of-light check and estimate-time anomaly flag; [12-file readset with digests and scope](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-source-identity.json). Marked `[code]`/`[doc]`/`[tree]`/`[blog]`; post figures absent from the repository recorded as unverified | [`performance-ceiling-analysis` reference](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md): two-gate weight anchor and two-threshold measurement rule; candidate/not_run. Anchors stored weights only; KV, collectives and E2E performance unanchored; no code, specs or measured corpus vendored, and HuggingArch was not run | +| [OpenBMB/JustGoal repo + paired paper](https://github.com/OpenBMB/JustGoal/tree/f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d) @ `f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d` (repository Apache-2.0 default; NOTICE BSD exceptions; paper license UNKNOWN) | evaluation | Bounded direct inspection, 2026-11-02: complete 21-page paper/appendices and selected DSA wrapper/CUDA workspace/stream contracts; [exact readset and exclusions](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/justgoal-source.json). Source-only methodology; author-reported comparisons, released failures versus repaired timing and official/stricter oracle differences retained | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/justgoal-evaluation-contracts.md); candidate/not_run, efficacy not_evaluated; no new implementation/backend coverage or full campaign runner | +| [AutoTuneBench repo + paper v1](https://github.com/li-ch/autotunebench/tree/64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f) @ `64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f`; [arXiv:2609.18123v1](https://arxiv.org/abs/2609.18123v1) (selected repository code/data MIT; paper CC BY 4.0) | evaluation | Bounded source inspection, 2026-11-02: treatment delivery, arm-validity precedence, exported protocol/check/record gaps, weak-denominator and anomaly counterexamples; complete 15-page paper, 6 figures, 7 tables and [exact selected readset/rights](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/autotunebench-source.json) | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/autotunebench-trial-validity.md); source-derived candidate/not_run, efficacy not_evaluated; invalid-arm/source-median conflicts retained, no new controller or measured implementation coverage | +| [Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/6cd3ee98dfe32746b345a6abe521fa4cb9bdc187) @ `6cd3ee98dfe32746b345a6abe521fa4cb9bdc187`; [arXiv:2605.26118v1](https://arxiv.org/html/2605.26118v1) (repository Apache-2.0; paper CC BY 4.0) | evaluation | Bounded source inspection, 2026-11-02: 18-file trial identity/origin, external outcome, selection/finalization/reporting and baseline-oracle readset; complete paper HTML/appendix/tables/algorithm/listing/captions, unused graphic values unverified; [exact hashes, rights and exclusions](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-source.json) | [`inference-benchmark-design` reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-trial-evidence.md); candidate/not_run, efficacy not_evaluated; two measured meanings, named-verdict update limits and manual versus measured-only defaults retained; no runtime integration or whole-model/backend promotion | +| [Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc98043fcf89f3589aa5addb3742777a) @ `1882c6adbc98043fcf89f3589aa5addb3742777a` (selected software/docs Apache-2.0) | evaluation | Bounded source inspection, 2026-11-03: Qwen3.5 GDN/MTP eager record/fold and checkpoint/adoption/retirement contracts, unexecuted fold-test definitions and historical completion denominators; [27-file readset and rights](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-source.json) | [`inference-benchmark-design` reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md); candidate/not_run, efficacy not_evaluated. Historical author results retain their separate execution revision; no new implementation, backend or measured resource coverage | +| [shenh10/HuggingArch](https://github.com/shenh10/HuggingArch/tree/facbbbc96856e558b9084a94a5070b2e2c86cf0b) @ `facbbbc96856e558b9084a94a5070b2e2c86cf0b` (MIT); companion post dated 2026-10-08, live, not attributable to the pin | evaluation | Bounded source inspection, 2026-11-11: checkpoint join and weight ledger code, validator and parallelism documents, upload speed-of-light check and estimate-time anomaly flag; [12-file readset with digests and scope](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-source-identity.json). Marked `[code]`/`[doc]`/`[tree]`/`[blog]`; post figures absent from the repository recorded as unverified | [`performance-ceiling-analysis` reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md): two-gate weight anchor and two-threshold measurement rule; candidate/not_run. Anchors stored weights only; KV, collectives and E2E performance unanchored; no code, specs or measured corpus vendored, and HuggingArch was not run | | Dong1017/OmniRSI @ `codex/skill-eval-bootstrap` (82218b4) | evaluation | skill_distill / skill_eval harness, no_skill/human/distilled contract, Tier A pilot | `npu-skill-factory` (evaluation contract) | | InferMatrixLab/InferRSI PR #9 @ 4ad551a9 | reference | `skill_library/` layout + knowledge records | `npu-skill-factory` (bundle layout) | -| RRSI — arXiv 2609.24972v3 | search; mined via reviewed Pack F | bounded harness proposals; calibrated noise/cost/novelty gates; grouped-ablation limits, replacing the earlier generic constraint summary | `optimization-search-policy`; [exact ledger](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md#f1--rrsi) | +| RRSI — arXiv 2609.24972v3 | search; mined via reviewed Pack F | bounded harness proposals; calibrated noise/cost/novelty gates; grouped-ablation limits, replacing the earlier generic constraint summary | `optimization-search-policy`; [exact ledger](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/references/source-notes.md#f1--rrsi) | | arXiv 2608.01867 (CRISP) | search | critical-step classification | `optimization-search-policy` | | **A1 / Agent Skills** @ `69ef37e9424c0a7ea9dd2293b559e43ec8176379` | reference; Dot A-approved minimal consumption | Specification frontmatter/body/progressive disclosure and client guide catalog/activation/resource sections directly verified 2026-10-15; format guidance, not an execution permission grant | `src/inferrsi/knowledge/skill_catalog.py`, existing CLI/arguments; metadata projection + explicit selection + resource resolution; no installer/router/client configuration | | **CR6 / Stas Bekman ML Engineering Open Book** @ `ee7517e0f562ff0e7d56c3e88c619d0c0c496ba7` (CC BY-SA 4.0 declared by source skill) | expert methodology | `skills/ml-engineering/SKILL.md` scoped principles, `skills/evaluate-cluster/SKILL.md` first 100 lines, targeted performance README MFU/HFU/NUMA/GC sections; scripts identified only, excluded storage/input/recovery lanes not mined | `training-resource-budget` + explicitly attributed/share-alike `references/stas-decision-notes.md`; Accelerator/RAM/Network/OS diagnosis and proposed verification, no local results | @@ -269,17 +269,17 @@ hardware or agent-development activity. | Original source / version | Inspected scope / source notes | Accepted contribution and consumer | Boundary | |---|---|---|---| -| Vidur v2 (same identity as §1) | [A readset](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md) | mined — calibration/queueing uncertainty and constrained shortlist → ceiling/calibration/search | A cross-link, not a second independent source | -| InferenceBench — 2607.20468v1 | [Audit](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md), A readset §§1–6, A–G | mined — bounded search, independent timing integrity, incumbent/final-artifact preservation → search policy and benchmark design | Within-vLLM two-hour main comparison differs from per-engine grid; displayed date/ID anomaly unresolved | -| FlashVector — 2609.17391v1 | [Audit](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md), §§4–5, Algorithm 1, Tables 1–4 | mined — noise-aware cross-layer redirection, host assembly/materialization, full-stack validation → ceiling/host guidance | Component versus service denominators; Model B SLO inconsistency retained; no Network from gRPC alone | -| Agentic Kernel Optimization — 2608.14560v1 | [Audit](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/agentic-kernel.md), main text and A–C | mined — regime dispatch, exactness/timing integrity and stalled-direction correction → search/ceiling cases | PyTorch versus FlashInfer baselines differ; SM utilization versus throughput is unresolved without matched definitions; date anomaly retained | -| Repo-To-Skill — 2609.02749v1 | [Ledger](../src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/repo-to-skill.md), [E readset](../src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/readset.md) §§2–5, A.1–A.2 | mined — case/knowledge/amendment/new-skill admission and independent verification axes → factory | Task-conditioned benchmark pools are not the entire public library; no AREX execution | -| Learning to Commit — 2603.26664v1 | [Ledger](../src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md), §§1–5 | mined — contrastive historical lesson, scope/contradiction/expiration → repository evolution | Accepted implementation is not a timeless invariant; chronology does not prove every contamination path absent | -| Trace2Skill — 2603.25158v5 | [Ledger](../src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/trace2skill.md), E readset §§2–4, A–I | mined — causal error analysis, successful-behavior preservation, consolidation and interaction checks → evolution playbook | Consensus is not acceptance; no universal trace count; code pin not matched to paper | -| SkillOpt — 2605.23904v2 | [Ledger](../src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/skillopt.md), §§3–4, A–C | mined — bounded edits, versioned rejection context and optimizer-memory separation → factory/evolution/evidence | Protocol and cost-denominator discrepancies retained; no automatic promotion | -| Dream-RSI — 2609.14858v2 | [F2 ledger](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md#f2--dream-rsi) | mined — prefix-safe replay and supported actions → search-policy interface knowledge | Replay does not generate unseen branches or prove online improvement; no controller implemented | -| AIDE² — 2609.26457v1 | [F3 ledger](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md#f3--aide²) | mined — inner/private/outer evaluation and solver-versus-improver distinction → evolution/search | Task-solving gains coexist with an inconclusive improver experiment; costs differ by benchmark | -| JitRL — 2601.18510v4 | [F4 ledger](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md#f4--jitrl) | mined — candidate-scoring/reward access and finite-memory evidence → adaptation-interface knowledge | Logit/scoring access is a prerequisite, not a Markdown capability; conditional theorem and matched counter-results retained | +| Vidur v2 (same identity as §1) | [A readset](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/READSET.md) | mined — calibration/queueing uncertainty and constrained shortlist → ceiling/calibration/search | A cross-link, not a second independent source | +| InferenceBench — 2607.20468v1 | [Audit](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md), A readset §§1–6, A–G | mined — bounded search, independent timing integrity, incumbent/final-artifact preservation → search policy and benchmark design | Within-vLLM two-hour main comparison differs from per-engine grid; displayed date/ID anomaly unresolved | +| FlashVector — 2609.17391v1 | [Audit](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md), §§4–5, Algorithm 1, Tables 1–4 | mined — noise-aware cross-layer redirection, host assembly/materialization, full-stack validation → ceiling/host guidance | Component versus service denominators; Model B SLO inconsistency retained; no Network from gRPC alone | +| Agentic Kernel Optimization — 2608.14560v1 | [Audit](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/agentic-kernel.md), main text and A–C | mined — regime dispatch, exactness/timing integrity and stalled-direction correction → search/ceiling cases | PyTorch versus FlashInfer baselines differ; SM utilization versus throughput is unresolved without matched definitions; date anomaly retained | +| Repo-To-Skill — 2609.02749v1 | [Ledger](../src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/repo-to-skill.md), [E readset](../src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/readset.md) §§2–5, A.1–A.2 | mined — case/knowledge/amendment/new-skill admission and independent verification axes → factory | Task-conditioned benchmark pools are not the entire public library; no AREX execution | +| Learning to Commit — 2603.26664v1 | [Ledger](../src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md), §§1–5 | mined — contrastive historical lesson, scope/contradiction/expiration → repository evolution | Accepted implementation is not a timeless invariant; chronology does not prove every contamination path absent | +| Trace2Skill — 2603.25158v5 | [Ledger](../src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/trace2skill.md), E readset §§2–4, A–I | mined — causal error analysis, successful-behavior preservation, consolidation and interaction checks → evolution playbook | Consensus is not acceptance; no universal trace count; code pin not matched to paper | +| SkillOpt — 2605.23904v2 | [Ledger](../src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/skillopt.md), §§3–4, A–C | mined — bounded edits, versioned rejection context and optimizer-memory separation → factory/evolution/evidence | Protocol and cost-denominator discrepancies retained; no automatic promotion | +| Dream-RSI — 2609.14858v2 | [F2 ledger](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/references/source-notes.md#f2--dream-rsi) | mined — prefix-safe replay and supported actions → search-policy interface knowledge | Replay does not generate unseen branches or prove online improvement; no controller implemented | +| AIDE² — 2609.26457v1 | [F3 ledger](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/references/source-notes.md#f3--aide²) | mined — inner/private/outer evaluation and solver-versus-improver distinction → evolution/search | Task-solving gains coexist with an inconclusive improver experiment; costs differ by benchmark | +| JitRL — 2601.18510v4 | [F4 ledger](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/references/source-notes.md#f4--jitrl) | mined — candidate-scoring/reward access and finite-memory evidence → adaptation-interface knowledge | Logit/scoring access is a prerequisite, not a Markdown capability; conditional theorem and matched counter-results retained | Applied owner-review decisions: R1–R6. E-U1 maps to repository evolution; E-U2 to factory admission; E-U3/E-U4 to the evolution playbook; E-U5 to the @@ -294,16 +294,16 @@ Their version-specific APIs are not advertised as this CLI's behavior. No third matrix or automatic discovery/update layer was added. Direct wave-7 source readsets and restrictions: -[inputs B1–B3](../src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/references/sources.md), -[recovery C3–C4](../src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md), -[cross-resource CR1/CR5/CR6](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/sources.md). +[inputs B1–B3](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/training-input-pipeline/references/sources.md), +[recovery C3–C4](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/sources.md), +[cross-resource CR1/CR5/CR6](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/sources.md). CR6's adapted note retains named attribution and CC BY-SA 4.0 share-alike; CR5 retains its 2026-10-15 documentation snapshot; the 2026-10-30 bounded source follow-up resolves the historical full pin/default license and selected allocator/wrapper/affinity paths. Current runtime compatibility remains unqualified. DCP source/version reconciliation, exact links and tutorial snapshot hash: -[`dcp-async-save-performance/references/sources.md`](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/references/sources.md). +[`dcp-async-save-performance/references/sources.md`](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/dcp-async-save-performance/references/sources.md). The live tutorial and pinned implementation differ on caller-side state-dict construction and cleanup details; claims follow the inspected implementation. Approval is save-performance only, not the broader C1 recovery proposal. @@ -329,29 +329,29 @@ English procedures linked below. | Canonical source group | Locally inspected scope / extraction | Canonical material and limits | |---|---|---| -| diffusers / dependency @ `efe46a5b177af67c64b3666da8ddddac7ee12163` | Group/leaf offload and asynchronous prefetch offer different memory-latency tradeoffs; hook compatibility and host allocation must be evaluated. | [diffusers.cuda.diffusion.group_offload_transfer_overlap](../src/inferrsi/knowledge/data/knowledge_packs/dependency/diffusers/cuda/diffusion/group-offload-transfer-overlap.json); imported, locally_verified=false; preserve source readset and counter-signals | -| flashattention / dependency @ `1fc0221183e799363001394a12dd9e78e15f5dda` | Select an attention generation using its actual GPU, dtype and interface requirements; package names and installation paths are not interchangeable. | [flashattention.cuda.attention.generation_specific_dispatch](../src/inferrsi/knowledge/data/knowledge_packs/dependency/flashattention/cuda/attention/generation-specific-dispatch.json); imported, locally_verified=false; preserve source readset and counter-signals | -| flashinfer / dependency @ `3637083a3a3d08538cce60937cd876b17779c6f0` | Paged attention separates planning from execution; auto dispatch and CUDA graph buffer ownership are part of the reproducibility boundary. | [flashinfer.cuda.llm.paged_decode_dispatch](../src/inferrsi/knowledge/data/knowledge_packs/dependency/flashinfer/cuda/llm/paged-decode-dispatch.json); imported, locally_verified=false; preserve source readset and counter-signals | -| pytorch / dependency @ `f15b476814ebc5eade36fb40512e379ebf79dd60` | CUDA graph replay can reduce host launch overhead, but shape specialization, pointer stability and output lifetime remain independent constraints. | [pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes](../src/inferrsi/knowledge/data/knowledge_packs/dependency/pytorch/cuda/compiler/cudagraph-shapes-and-lifetimes.json); imported, locally_verified=false; preserve source readset and counter-signals | -| transformers / dependency @ `2126f65b0ab267e113e78a002447fc7094de83d5` | Dynamic, static, offloaded and quantized KV caches have different contracts; select by workload and API support rather than a universal ranking. | [transformers.common.llm.kv_cache_strategy_contracts](../src/inferrsi/knowledge/data/knowledge_packs/dependency/transformers/common/llm/kv-cache-strategy-contracts.json); imported, locally_verified=false; preserve source readset and counter-signals | -| triton / dependency @ `4eda6cc91e86553a2517d1c764ea2967985553ad` | Fuse a bandwidth-bound reduction when its working set fits on chip; choose launch geometry using the compiled resource footprint. | [triton.common.kernel.bandwidth_fusion_resource_limits](../src/inferrsi/knowledge/data/knowledge_packs/dependency/triton/common/kernel/bandwidth-fusion-resource-limits.json); imported, locally_verified=false; preserve source readset and counter-signals | -| cache_dit / peer @ `a57c826d3527736a0d1f1b0243d47568c350eb5f` | DBCache skips intermediate transformer work using residual similarity, with computed boundary blocks and warmup controlling approximation. | [cache_dit.common.diffusion.residual_cache_calibration](../src/inferrsi/knowledge/data/knowledge_packs/peer/cache_dit/common/diffusion/residual-cache-calibration.json); imported, locally_verified=false; preserve source readset and counter-signals | -| lmdeploy / peer @ `822dddb8b3bc895eaf6cffb45a5514768e7ecb84` | KV compression can increase cache capacity yet reduce throughput; distinguish engine-specific quantizers and validate long-context quality. | [lmdeploy.cuda.llm.kv_quantization_capacity_cost](../src/inferrsi/knowledge/data/knowledge_packs/peer/lmdeploy/cuda/llm/kv-quantization-capacity-cost.json); imported, locally_verified=false; preserve source readset and counter-signals | -| sglang / peer @ `f7ad4345b86d147a1c3095a3cebabca9a33ebaa1` | Move reusable prefix KV across device, host and storage tiers while accounting for sharing scope, layout conversion and prefetch policy. | [sglang.cuda.llm.hierarchical_prefix_cache](../src/inferrsi/knowledge/data/knowledge_packs/peer/sglang/cuda/llm/hierarchical-prefix-cache.json); imported, locally_verified=false; preserve source readset and counter-signals | -| tensorrt_llm / peer @ `c9b18eefa731dc59b0091bdfff6d890aa9d1e98e` | Separate prefill and decode resources when workload interference dominates the added transfer, queueing and layout-conversion costs. | [tensorrt_llm.cuda.llm.prefill_decode_disaggregation](../src/inferrsi/knowledge/data/knowledge_packs/peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json); imported, locally_verified=false; preserve source readset and counter-signals | -| xdit / peer @ `f7c16aa958379a9f0a0302350fe0e1bfcec5f23c` | USP, CFG splitting and PipeFusion have different topology, memory and quality tradeoffs; qualify each before combining them. | [xdit.cuda.diffusion.parallelism_freshness_and_topology](../src/inferrsi/knowledge/data/knowledge_packs/peer/xdit/cuda/diffusion/parallelism-freshness-and-topology.json); imported, locally_verified=false; preserve source readset and counter-signals | -| strata / contribution @ `296b59d2f73c8f04a805df29c2fa97655adace78` | Family geometry, expert residency, CPU/PCIe balance and native engine gates; three existing abstracts inspected | [strata.common.llm.architecture_search_space](../src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/architecture-search-space.json); [strata.common.llm.controlled_execution_tuning](../src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/controlled-execution-tuning.json); [strata.common.llm.engine_validation_contract](../src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/engine-validation-contract.json); English source procedures retained, no device/service validation | -| vllm / contribution @ `2521446ac8c92937b5a32b7dc2a84ef7fa8e99c5` | V1 token/KV ownership, bounded serving measurements and extension gates; three existing abstracts inspected | [vllm.common.llm.bounded_serving_measurement](../src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/bounded-serving-measurement.json); [vllm.common.llm.extension_correctness_gates](../src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/extension-correctness-gates.json); [vllm.common.llm.runtime_and_kv_ownership](../src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/runtime-and-kv-ownership.json); English source procedures retained, no device/service validation | -| sglang / contribution @ `16176278d0180f1a1a578e53e6eef432577876c2` | Radix/HiCache state, TP/DP/EP and controlled tuning; three existing abstracts inspected | [sglang.common.llm.parallelism_and_extension](../src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/parallelism-and-extension.json); [sglang.common.llm.runtime_radix_hicache](../src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/runtime-radix-hicache.json); [sglang.common.llm.tuning_and_validation](../src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/tuning-and-validation.json); English source procedures retained, no device/service validation | -| sglang_omni / contribution @ `2cfb3b67becb437b70751defe37e63d3af4eedb0` | Stage DAG/process transport, preparation/abort lifecycle and audio quality; three existing abstracts inspected | [sglang_omni.common.audio.serving_and_streaming_quality](../src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/audio/serving-and-streaming-quality.json); [sglang_omni.common.omni.extension_lifecycle_validation](../src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/omni/extension-lifecycle-validation.json); [sglang_omni.common.omni.pipeline_state_and_transport](../src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/omni/pipeline-state-and-transport.json); English source procedures retained, no device/service validation | -| vllm_omni / contribution @ `52194f24fa42e14d1c5e096f4bea4dc7f95cffa1` | Stage/replica ownership, diffusion state/output and streaming audio measurement; three existing abstracts inspected | [vllm_omni.common.audio.streaming_serving_measurement](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/audio/streaming-serving-measurement.json); [vllm_omni.common.diffusion.step_state_and_output_validation](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/step-state-and-output-validation.json); [vllm_omni.common.omni.stage_ownership_and_search](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/omni/stage-ownership-and-search.json); English source procedures retained, no device/service validation | -| Internal Drama3 issue #10 | Recorded continuous-admission versus drain occupancy; diagnostic section inspected, original trace not reread | [One reusable case](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.md); INCONCLUSIVE campaign, reported artifact hash, no new gain claim | -| Main audio source guidance | Scheduler-step, graph/state/stream ownership, actual dispatch and complete-service limits at the record's source pin | [Existing audio record](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/audio/concurrency-scaling-state-and-overlap.json); imported guidance stays unverified | +| diffusers / dependency @ `efe46a5b177af67c64b3666da8ddddac7ee12163` | Group/leaf offload and asynchronous prefetch offer different memory-latency tradeoffs; hook compatibility and host allocation must be evaluated. | [diffusers.cuda.diffusion.group_offload_transfer_overlap](../src/inferrsi/knowledge/data/skill_library/records/diffusers.cuda.diffusion.group_offload_transfer_overlap.json); imported, locally_verified=false; preserve source readset and counter-signals | +| flashattention / dependency @ `1fc0221183e799363001394a12dd9e78e15f5dda` | Select an attention generation using its actual GPU, dtype and interface requirements; package names and installation paths are not interchangeable. | [flashattention.cuda.attention.generation_specific_dispatch](../src/inferrsi/knowledge/data/skill_library/records/flashattention.cuda.attention.generation_specific_dispatch.json); imported, locally_verified=false; preserve source readset and counter-signals | +| flashinfer / dependency @ `3637083a3a3d08538cce60937cd876b17779c6f0` | Paged attention separates planning from execution; auto dispatch and CUDA graph buffer ownership are part of the reproducibility boundary. | [flashinfer.cuda.llm.paged_decode_dispatch](../src/inferrsi/knowledge/data/skill_library/records/flashinfer.cuda.llm.paged_decode_dispatch.json); imported, locally_verified=false; preserve source readset and counter-signals | +| pytorch / dependency @ `f15b476814ebc5eade36fb40512e379ebf79dd60` | CUDA graph replay can reduce host launch overhead, but shape specialization, pointer stability and output lifetime remain independent constraints. | [pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes](../src/inferrsi/knowledge/data/skill_library/records/pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes.json); imported, locally_verified=false; preserve source readset and counter-signals | +| transformers / dependency @ `2126f65b0ab267e113e78a002447fc7094de83d5` | Dynamic, static, offloaded and quantized KV caches have different contracts; select by workload and API support rather than a universal ranking. | [transformers.common.llm.kv_cache_strategy_contracts](../src/inferrsi/knowledge/data/skill_library/records/transformers.common.llm.kv_cache_strategy_contracts.json); imported, locally_verified=false; preserve source readset and counter-signals | +| triton / dependency @ `4eda6cc91e86553a2517d1c764ea2967985553ad` | Fuse a bandwidth-bound reduction when its working set fits on chip; choose launch geometry using the compiled resource footprint. | [triton.common.kernel.bandwidth_fusion_resource_limits](../src/inferrsi/knowledge/data/skill_library/records/triton.common.kernel.bandwidth_fusion_resource_limits.json); imported, locally_verified=false; preserve source readset and counter-signals | +| cache_dit / peer @ `a57c826d3527736a0d1f1b0243d47568c350eb5f` | DBCache skips intermediate transformer work using residual similarity, with computed boundary blocks and warmup controlling approximation. | [cache_dit.common.diffusion.residual_cache_calibration](../src/inferrsi/knowledge/data/skill_library/records/cache_dit.common.diffusion.residual_cache_calibration.json); imported, locally_verified=false; preserve source readset and counter-signals | +| lmdeploy / peer @ `822dddb8b3bc895eaf6cffb45a5514768e7ecb84` | KV compression can increase cache capacity yet reduce throughput; distinguish engine-specific quantizers and validate long-context quality. | [lmdeploy.cuda.llm.kv_quantization_capacity_cost](../src/inferrsi/knowledge/data/skill_library/records/lmdeploy.cuda.llm.kv_quantization_capacity_cost.json); imported, locally_verified=false; preserve source readset and counter-signals | +| sglang / peer @ `f7ad4345b86d147a1c3095a3cebabca9a33ebaa1` | Move reusable prefix KV across device, host and storage tiers while accounting for sharing scope, layout conversion and prefetch policy. | [sglang.cuda.llm.hierarchical_prefix_cache](../src/inferrsi/knowledge/data/skill_library/records/sglang.cuda.llm.hierarchical_prefix_cache.json); imported, locally_verified=false; preserve source readset and counter-signals | +| tensorrt_llm / peer @ `c9b18eefa731dc59b0091bdfff6d890aa9d1e98e` | Separate prefill and decode resources when workload interference dominates the added transfer, queueing and layout-conversion costs. | [tensorrt_llm.cuda.llm.prefill_decode_disaggregation](../src/inferrsi/knowledge/data/skill_library/records/tensorrt_llm.cuda.llm.prefill_decode_disaggregation.json); imported, locally_verified=false; preserve source readset and counter-signals | +| xdit / peer @ `f7c16aa958379a9f0a0302350fe0e1bfcec5f23c` | USP, CFG splitting and PipeFusion have different topology, memory and quality tradeoffs; qualify each before combining them. | [xdit.cuda.diffusion.parallelism_freshness_and_topology](../src/inferrsi/knowledge/data/skill_library/records/xdit.cuda.diffusion.parallelism_freshness_and_topology.json); imported, locally_verified=false; preserve source readset and counter-signals | +| strata / contribution @ `296b59d2f73c8f04a805df29c2fa97655adace78` | Family geometry, expert residency, CPU/PCIe balance and native engine gates; three existing abstracts inspected | [strata.common.llm.architecture_search_space](../src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.architecture_search_space.json); [strata.common.llm.controlled_execution_tuning](../src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.controlled_execution_tuning.json); [strata.common.llm.engine_validation_contract](../src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.engine_validation_contract.json); English source procedures retained, no device/service validation | +| vllm / contribution @ `2521446ac8c92937b5a32b7dc2a84ef7fa8e99c5` | V1 token/KV ownership, bounded serving measurements and extension gates; three existing abstracts inspected | [vllm.common.llm.bounded_serving_measurement](../src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.bounded_serving_measurement.json); [vllm.common.llm.extension_correctness_gates](../src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.extension_correctness_gates.json); [vllm.common.llm.runtime_and_kv_ownership](../src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.runtime_and_kv_ownership.json); English source procedures retained, no device/service validation | +| sglang / contribution @ `16176278d0180f1a1a578e53e6eef432577876c2` | Radix/HiCache state, TP/DP/EP and controlled tuning; three existing abstracts inspected | [sglang.common.llm.parallelism_and_extension](../src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.parallelism_and_extension.json); [sglang.common.llm.runtime_radix_hicache](../src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.runtime_radix_hicache.json); [sglang.common.llm.tuning_and_validation](../src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.tuning_and_validation.json); English source procedures retained, no device/service validation | +| sglang_omni / contribution @ `2cfb3b67becb437b70751defe37e63d3af4eedb0` | Stage DAG/process transport, preparation/abort lifecycle and audio quality; three existing abstracts inspected | [sglang_omni.common.audio.serving_and_streaming_quality](../src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.audio.serving_and_streaming_quality.json); [sglang_omni.common.omni.extension_lifecycle_validation](../src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.omni.extension_lifecycle_validation.json); [sglang_omni.common.omni.pipeline_state_and_transport](../src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.omni.pipeline_state_and_transport.json); English source procedures retained, no device/service validation | +| vllm_omni / contribution @ `52194f24fa42e14d1c5e096f4bea4dc7f95cffa1` | Stage/replica ownership, diffusion state/output and streaming audio measurement; three existing abstracts inspected | [vllm_omni.common.audio.streaming_serving_measurement](../src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.audio.streaming_serving_measurement.json); [vllm_omni.common.diffusion.step_state_and_output_validation](../src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.step_state_and_output_validation.json); [vllm_omni.common.omni.stage_ownership_and_search](../src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.omni.stage_ownership_and_search.json); English source procedures retained, no device/service validation | +| Internal Drama3 issue #10 | Recorded continuous-admission versus drain occupancy; diagnostic section inspected, original trace not reread | [One reusable case](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md); INCONCLUSIVE campaign, reported artifact hash, no new gain claim | +| Main audio source guidance | Scheduler-step, graph/state/stream ownership, actual dispatch and complete-service limits at the record's source pin | [Existing audio record](../src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap.json); imported guidance stays unverified | Rejected-direction extraction reuses the same pinned main source summaries: -[Qwen B300](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md), -[Qwen Ascend](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md) -and [Drama3 B300](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md). +[Qwen B300](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md), +[Qwen Ascend](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md) +and [Drama3 B300](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md). These are three additional Case bundles, not new source mechanisms, skills, independent corroboration or hardware validation. Each separates recorded disposition from a proposed diagnostic check and retains the source limits. @@ -373,7 +373,7 @@ map, a state-correctness contract and a measurement/quality contract. | david6666666/drama-3-single-b300-inference | `58a1882a4d941f8d21bfce6be37d3d5e7c9fde80`; RFC-selected commits only (`a6e660cc`, `ec51ba1e`, `a7118fb7`, `6ff8d808`, `33f6d723`) and `benchmark.py`/results; no broad history scan | External GPU reference engine motivating NPU questions; shares the design family of the existing `vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap` import (not independent corroboration) | The supplement links the existing -[Drama3 B300 rejected-directions case](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md) +[Drama3 B300 rejected-directions case](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md) rather than restating it; that case shares this campaign lineage. The two primary matrices are unchanged in authority; this is a reconciliation @@ -383,7 +383,7 @@ note, not a third inventory. All NPU status for Drama3 remains `unverified`. | Source / revision | Inspected scope | Extraction and limits | |---|---|---| -| ml-explore/mlx @ `bff10220c742907f0c68e55ff174aaea56f74e3d` (MIT) | `ops.cpp::add`, Python eval bindings, `transforms.cpp::eval_impl`, array availability/detach, CPU/Metal/CUDA eval retention paths, selected native test source and build conditions | [lazy-evaluation-stream-lifetimes](../src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md), with [pinned readset](../src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md): three framework mechanisms; CODE and unexecuted TEST-SOURCE, empirical validation not_run; no model/service/performance or installed-backend qualification | +| ml-explore/mlx @ `bff10220c742907f0c68e55ff174aaea56f74e3d` (MIT) | `ops.cpp::add`, Python eval bindings, `transforms.cpp::eval_impl`, array availability/detach, CPU/Metal/CUDA eval retention paths, selected native test source and build conditions | [lazy-evaluation-stream-lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md), with [pinned readset](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md): three framework mechanisms; CODE and unexecuted TEST-SOURCE, empirical validation not_run; no model/service/performance or installed-backend qualification | This framework owner does not implement MLX discovery or claim the separate MLX-LM application/cache layer. The generated abstract is a discovery projection, @@ -391,23 +391,23 @@ not an additional source or validated mechanism. ## vLLM-Omni incremental abort/cleanup readset -The [source and responsibility map](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/SKILL.md#source-and-responsibility-map) +The [source and responsibility map](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/SKILL.md#source-and-responsibility-map) links the original engine guide (`52194f24fa42e14d1c5e096f4bea4dc7f95cffa1`, inspected 2026-10-14) and this incremental abort readset (`18c3bd5d2d4323bc7adca1ac3c465c4a53bbfb6e`, inspected 2026-10-22). -Follow the [validation route](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/SKILL.md) +Follow the [validation route](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/SKILL.md) for actual caller, receipt and cleanup questions. Existing readsets and the three catalog records retain their own source attribution and validation limits. | Source / revision | Inspected scope | Reuse, extraction and limits | |---|---|---| -| vllm-project/vllm-omni @ `18c3bd5d2d4323bc7adca1ac3c465c4a53bbfb6e` (Apache-2.0) | Engine-base abort/ack APIs, correlated waiter/router, orchestrator companion and logical cleanup, stage-pool partial outputs and queued release; selected upstream test source and test bootstrap | Extend [existing omni-validation](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/SKILL.md) with [three bounded mechanisms and pinned readset](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md). Reuse the original `52194f24...` engine guide and catalog ownership; no new skill or parallel database. CODE / TEST-SOURCE only; actual device, connector, model, service and performance not_run. Full duplex/session behavior and native abort guarantees deferred. | +| vllm-project/vllm-omni @ `18c3bd5d2d4323bc7adca1ac3c465c4a53bbfb6e` (Apache-2.0) | Engine-base abort/ack APIs, correlated waiter/router, orchestrator companion and logical cleanup, stage-pool partial outputs and queued release; selected upstream test source and test bootstrap | Extend [existing omni-validation](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/SKILL.md) with [three bounded mechanisms and pinned readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md). Reuse the original `52194f24...` engine guide and catalog ownership; no new skill or parallel database. CODE / TEST-SOURCE only; actual device, connector, model, service and performance not_run. Full duplex/session behavior and native abort guarantees deferred. | ## MLX-LM application source extraction | Source / revision | Inspected scope | Extraction and limits | |---|---|---| -| ml-explore/mlx-lm @ `ab7f7048dc341ced2d84b1887eeb17d60b0d1b02` (MIT) | `generate_step`/`stream_generate`, BatchGenerator stage/removal and row filtering, cache factory/KV/BatchKV/LRU prefix paths, selected server model-key/batching/stop branches and native test source | [generation-cache-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md), with [pinned readset](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/source-map.md): three application mechanisms; CODE and unexecuted TEST-SOURCE; empirical validation not_run, no model/service/backend or matched-performance qualification | +| ml-explore/mlx-lm @ `ab7f7048dc341ced2d84b1887eeb17d60b0d1b02` (MIT) | `generate_step`/`stream_generate`, BatchGenerator stage/removal and row filtering, cache factory/KV/BatchKV/LRU prefix paths, selected server model-key/batching/stop branches and native test source | [generation-cache-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md), with [pinned readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/source-map.md): three application mechanisms; CODE and unexecuted TEST-SOURCE; empirical validation not_run, no model/service/backend or matched-performance qualification | MLX-LM owns request/token/cache policy. This source pin does not qualify its compatibility with the separately pinned MLX framework or implement discovery. @@ -417,13 +417,13 @@ The generated abstract is a projection, not independent corroboration. | Source / revision | Inspected scope | Extraction and limits | |---|---|---| -| huggingface/diffusers @ `aea7b70be0f594bcdc15a6f2677ee2ca85e49e8e` (Apache-2.0) | FluxPipeline construction/call, register_modules/components/from_pipe, FlowMatchEulerDiscreteScheduler schedule/index/step and Flux interrupt/callback/normal output tail; selected native fixture and dependency declarations | [diffusers-invocation-state](../src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md), with [pinned source map](../src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/source-map.md): CODE and unexecuted TEST-SOURCE; candidate, empirical validation not_run. The supplied manifest verifies a selected pipeline_utils Git blob, not the full archive tree. No model, concurrent-call, backend or performance qualification. | +| huggingface/diffusers @ `aea7b70be0f594bcdc15a6f2677ee2ca85e49e8e` (Apache-2.0) | FluxPipeline construction/call, register_modules/components/from_pipe, FlowMatchEulerDiscreteScheduler schedule/index/step and Flux interrupt/callback/normal output tail; selected native fixture and dependency declarations | [diffusers-invocation-state](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md), with [pinned source map](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/references/source-map.md): CODE and unexecuted TEST-SOURCE; candidate, empirical validation not_run. The supplied manifest verifies a selected pipeline_utils Git blob, not the full archive tree. No model, concurrent-call, backend or performance qualification. | The existing Diffusers offload card retains its earlier source pin. This bounded invocation readset does not establish caller serialization, general thread safety or exception cleanup. The generated abstract adds discovery, not corroboration. -For entry selection, use the [two-pin responsibility comparison](../src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md#offload-or-invocation-state). +For entry selection, use the [two-pin responsibility comparison](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md#offload-or-invocation-state). The offload dependency record retains its 2026-10-15 documentation access; the invocation source map retains its 2026-10-22 archive read. These dates are not refreshed by navigation checks, and normal-tail hooks do not establish offload @@ -433,7 +433,7 @@ compatibility or exceptional/asynchronous cleanup. | Source / revision | Inspected scope | Reuse, extraction and limits | |---|---|---| -| deepseek-ai/DeepInfer @ `8b16149741a4935e1dec8cea27853f273b374c5b` (Apache-2.0; direct vendors retain separate licenses) | Bounded DFlash producer/verification budget, scheduler confidence/round counters, worker ZOS metadata repair, YoCo boundary/full-layout KV, SWA metadata/ring sizing and released serving hooks; selected test/harness source | Two source-derived candidates: [verification budget](../src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md), [bounded query replay](../src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md); reuse [host scheduling case](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md) and [capability audit](../src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md). Their F labels mean pinned CODE, P means proposed procedure; test/harness reading is not execution. Native/model/device/sampling/quality/performance not_run. | +| deepseek-ai/DeepInfer @ `8b16149741a4935e1dec8cea27853f273b374c5b` (Apache-2.0; direct vendors retain separate licenses) | Bounded DFlash producer/verification budget, scheduler confidence/round counters, worker ZOS metadata repair, YoCo boundary/full-layout KV, SWA metadata/ring sizing and released serving hooks; selected test/harness source | Two source-derived candidates: [verification budget](../src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/SKILL.md), [bounded query replay](../src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/SKILL.md); reuse [host scheduling case](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md) and [capability audit](../src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/references/capability-presence-audit.md). Their F labels mean pinned CODE, P means proposed procedure; test/harness reading is not execution. Native/model/device/sampling/quality/performance not_run. | The source-reading bundle preserves six direct vendor snapshots but omits nine media assets and ten nested gitlinks. Parent-code observations do not certify @@ -446,10 +446,10 @@ the candidate mechanisms. | Source / revision | Inspected scope | Reuse, extraction and limits | | --- | --- | --- | -| [Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/60#issuecomment-6310920198), 2026-10-21; eight Git source inventories in the [owner metadata](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json) | Task/quality/workload separation across text, agentic coding, work agents, TTS, full-duplex interaction and image/video; selected pinned AIPerf/realtime/FDB/video sources rechecked and the current InferRSI metric contract inspected | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [task selection](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md) and [command templates](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md). Candidate guidance; source review and bounded smoke do not qualify all tasks or establish a performance gain. Replay scores, reference parity and task quality retain separate meanings. | -| [NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Speech.cpp/tree/20999cc921d2b14aa58708f61756a8b2e787ba20), inspected 2026-11-01 | Selected cache-aware encoder/config, runner, SDK bounds, endpoint policy, geometry test/CMake and ASR benchmark; complete 14-file readset and Apache-2.0 / parakeet MIT attribution retained in the reference | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [cache-aware ASR stream boundaries](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md): first/steady mel geometry and decoded clock, stream-slot reset/reuse, EOU/EOF/reset and model-test SKIP boundaries. Source-derived candidate / empirical not_run; benchmark figures remain author-reported with precision, FE and trial limits. No model/device/quality/long-audio/load/service qualification. | -| [yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizhang/faster-cosyvoice/tree/f1049c59196c2c1a890079abc56056b43f27cd1d), inspected 2026-11-03; overall source rights UNKNOWN, selected file headers retain Apache-2.0 and unresolved inherited attribution | Bounded PCM client/server-credit, chunk planner, session/worker, Token2Wav and timing/quality-report inspection; [exact readset and per-file rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/faster-cosyvoice-source.json) | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [dependency-aware TTS playout](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/dependency-aware-tts-playout.md): route-origin credit versus client zero-buffer playback, same-session Mel dependencies, cumulative vocoding and partial-cost/token-derived denominators. Original analysis with immutable links only; candidate / empirical not_run / efficacy not_evaluated. Author reports remain separate; no copied implementation/media, new runtime or hardware/quality/service qualification. | -| [Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/156bed151826780e2ce0132f8d0e3540889ae8cd) + [paper v3](https://arxiv.org/abs/2607.02501v3), inspected 2026-11-04 (repository Apache-2.0; paper CC BY 4.0) | Selected fixed CPU-observation-to-complete-CPU-action boundary, client queue/profiler denominators, precision identity, per-task quality and source/artifact lineage; [exact readset and rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json). Later SmolVLA reports remain separate from the August paper. | Extend registered owner `inference-benchmark-design` ([SKILL.md](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md)) with [VLA boundary and quality guidance](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md). Source-derived candidate/not_run; public sample arithmetic is distinct from episode recount and native execution. Generated actions are not control frequency; per-task loss, unequal precision, whole-device memory and unresolved measured-script/dirty-patch identity remain explicit. | +| [Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/60#issuecomment-6310920198), 2026-10-21; eight Git source inventories in the [owner metadata](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) | Task/quality/workload separation across text, agentic coding, work agents, TTS, full-duplex interaction and image/video; selected pinned AIPerf/realtime/FDB/video sources rechecked and the current InferRSI metric contract inspected | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) with [task selection](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/task-benchmark-selection.md) and [command templates](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/benchmark-commands.md). Candidate guidance; source review and bounded smoke do not qualify all tasks or establish a performance gain. Replay scores, reference parity and task quality retain separate meanings. | +| [NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Speech.cpp/tree/20999cc921d2b14aa58708f61756a8b2e787ba20), inspected 2026-11-01 | Selected cache-aware encoder/config, runner, SDK bounds, endpoint policy, geometry test/CMake and ASR benchmark; complete 14-file readset and Apache-2.0 / parakeet MIT attribution retained in the reference | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) with [cache-aware ASR stream boundaries](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md): first/steady mel geometry and decoded clock, stream-slot reset/reuse, EOU/EOF/reset and model-test SKIP boundaries. Source-derived candidate / empirical not_run; benchmark figures remain author-reported with precision, FE and trial limits. No model/device/quality/long-audio/load/service qualification. | +| [yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizhang/faster-cosyvoice/tree/f1049c59196c2c1a890079abc56056b43f27cd1d), inspected 2026-11-03; overall source rights UNKNOWN, selected file headers retain Apache-2.0 and unresolved inherited attribution | Bounded PCM client/server-credit, chunk planner, session/worker, Token2Wav and timing/quality-report inspection; [exact readset and per-file rights](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/faster-cosyvoice-source.json) | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) with [dependency-aware TTS playout](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/dependency-aware-tts-playout.md): route-origin credit versus client zero-buffer playback, same-session Mel dependencies, cumulative vocoding and partial-cost/token-derived denominators. Original analysis with immutable links only; candidate / empirical not_run / efficacy not_evaluated. Author reports remain separate; no copied implementation/media, new runtime or hardware/quality/service qualification. | +| [Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/156bed151826780e2ce0132f8d0e3540889ae8cd) + [paper v3](https://arxiv.org/abs/2607.02501v3), inspected 2026-11-04 (repository Apache-2.0; paper CC BY 4.0) | Selected fixed CPU-observation-to-complete-CPU-action boundary, client queue/profiler denominators, precision identity, per-task quality and source/artifact lineage; [exact readset and rights](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/embodied-cpp-source.json). Later SmolVLA reports remain separate from the August paper. | Extend registered owner `inference-benchmark-design` ([SKILL.md](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md)) with [VLA boundary and quality guidance](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md). Source-derived candidate/not_run; public sample arithmetic is distinct from episode recount and native execution. Generated actions are not control frequency; per-task loss, unequal precision, whole-device memory and unresolved measured-script/dirty-patch identity remain explicit. | ## GPU measurement validity and profiler attribution @@ -457,11 +457,11 @@ General GPU measurement methodology for `inference-benchmark-design`, not an iss | Source / revision | Inspected scope | Reuse, extraction and limits | | --- | --- | --- | -| NVIDIA NCU 2025.3, Nsight Systems 2025.3, CUDA 12.9.1/10.2 archives, hash-pinned SMI document; Triton v3.2.0 @ `9641643da6c52000c807b5eeed05edaec4402a67`; PyTorch 2.7; [robust-kbench paper v1](https://arxiv.org/html/2509.14279v1); [typed source inventory](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json) | Selected clock/cache/replay, timing/graph, metric-name, flush, extension-identity and benchmark-exploit contracts, inspected 2026-11-15 | Extend `inference-benchmark-design` with [GPU measurement validity and profiler attribution](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md). Candidate / empirical not_run / efficacy not_evaluated; normalization/adoption is an unverified author proposal, four task-supplied cases lack raw evidence, and source review does not establish a gain or current-device metric availability. | +| NVIDIA NCU 2025.3, Nsight Systems 2025.3, CUDA 12.9.1/10.2 archives, hash-pinned SMI document; Triton v3.2.0 @ `9641643da6c52000c807b5eeed05edaec4402a67`; PyTorch 2.7; [robust-kbench paper v1](https://arxiv.org/html/2509.14279v1); [typed source inventory](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) | Selected clock/cache/replay, timing/graph, metric-name, flush, extension-identity and benchmark-exploit contracts, inspected 2026-11-15 | Extend `inference-benchmark-design` with [GPU measurement validity and profiler attribution](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/gpu-measurement-validity.md). Candidate / empirical not_run / efficacy not_evaluated; normalization/adoption is an unverified author proposal, four task-supplied cases lack raw evidence, and source review does not establish a gain or current-device metric availability. | ## SGLang versioned entry selection -The [three-pin responsibility comparison](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/SKILL.md#choose-among-source-pins) +The [three-pin responsibility comparison](../src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/SKILL.md#choose-among-source-pins) links the existing LLM engine snapshot, CUDA HiCache peer and bounded diffusion readset. Original contribution access (2026-10-14), HiCache access (2026-10-15) and diffusion archive inspection (2026-10-22) remain separate. This navigation @@ -472,28 +472,28 @@ source maps, notices and imported/candidate states stay with their owners. | Source / revision | Inspected scope | Reuse, extraction and limits | | --- | --- | --- | -| antirez/ds4 @ `ee7e8405232c8971000cc7ba2b945b00bf6d20d3`; AREX-Skill method @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected SSD budgeting, shared prefill workspace, session/token/image identity, disk KV/tool replay, speculation and topology/test contracts | [DwarfStar operating graph](../src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/SKILL.md) with bounded source hashes, bundled request helper and owner-local references. Actual portable create/verify method, not an unexecuted CLI run. Native C adaptation and partial drafting were user-approved. Accelerated/model/physical-distributed/quality/performance verification remains blocked; CPU checks do not establish those claims. Source MIT notice retained; `NO_LICENSE` is the unauthenticated resolver fallback, not a legal conclusion. | +| antirez/ds4 @ `ee7e8405232c8971000cc7ba2b945b00bf6d20d3`; AREX-Skill method @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected SSD budgeting, shared prefill workspace, session/token/image identity, disk KV/tool replay, speculation and topology/test contracts | [DwarfStar operating graph](../src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/SKILL.md) with bounded source hashes, bundled request helper and owner-local references. Actual portable create/verify method, not an unexecuted CLI run. Native C adaptation and partial drafting were user-approved. Accelerated/model/physical-distributed/quality/performance verification remains blocked; CPU checks do not establish those claims. Source MIT notice retained; `NO_LICENSE` is the unauthenticated resolver fallback, not a legal conclusion. | ## Splash portable AREX extraction | Source / revision | Inspected scope | Reuse, extraction and limits | | --- | --- | --- | -| incoai/splash @ `39a97385f8560cf8a93972cec9b0a963c9195250`; AREX method @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected format/memory planning, scheduler/cache/state/persistence, API/security/protocol, DFlash2/ANE/numerical and benchmark contracts | [Splash graph](../src/inferrsi/knowledge/data/skill_library/repositories/splash/SKILL.md) with bounded file hashes, three safe offline helpers and retained notices. Actual compatible-agent portable workflow with user-approved native/source-module adaptation. Python checks are not Metal/ANE/model/disk-state or service qualification; required native evidence remains blocked. Source Apache-2.0 and third-party notices are independent of the `NO_LICENSE` unauthenticated-CLI fallback. | +| incoai/splash @ `39a97385f8560cf8a93972cec9b0a963c9195250`; AREX method @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected format/memory planning, scheduler/cache/state/persistence, API/security/protocol, DFlash2/ANE/numerical and benchmark contracts | [Splash graph](../src/inferrsi/knowledge/data/skill_library/systems/engines/splash/SKILL.md) with bounded file hashes, three safe offline helpers and retained notices. Actual compatible-agent portable workflow with user-approved native/source-module adaptation. Python checks are not Metal/ANE/model/disk-state or service qualification; required native evidence remains blocked. Source Apache-2.0 and third-party notices are independent of the `NO_LICENSE` unauthenticated-CLI fallback. | ## Lily portable AREX extraction | Source / revision | Inspected scope | Reuse, extraction and limits | | --- | --- | --- | -| perplexityai/pplx-garden `lily/` @ `1ed972ed3f0bd5616c997c9507c25616c63394fc`; AREX method @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected exact architecture/affine packing, Metal ownership, strict API/session prefixes, native-test gating and versioned benchmark methods | [Lily graph](../src/inferrsi/knowledge/data/skill_library/repositories/lily/SKILL.md), bounded hashes, safe offline helpers and Apache-2.0/NOTICE retention. Actual portable native-Rust/Metal adaptation, not a DisCo CLI/model run. Cargo/native/model/quality/performance evidence unavailable; later long-context prefill negatives and runner/report version mismatch are retained. `NO_LICENSE` is the unauthenticated resolver fallback, not a legal finding. | +| perplexityai/pplx-garden `lily/` @ `1ed972ed3f0bd5616c997c9507c25616c63394fc`; AREX method @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected exact architecture/affine packing, Metal ownership, strict API/session prefixes, native-test gating and versioned benchmark methods | [Lily graph](../src/inferrsi/knowledge/data/skill_library/systems/engines/lily/SKILL.md), bounded hashes, safe offline helpers and Apache-2.0/NOTICE retention. Actual portable native-Rust/Metal adaptation, not a DisCo CLI/model run. Cargo/native/model/quality/performance evidence unavailable; later long-context prefill negatives and runner/report version mismatch are retained. `NO_LICENSE` is the unauthenticated resolver fallback, not a legal finding. | ## Husky document-only research adaptation | Source / version | Inspected scope | Reuse and limits | | --- | --- | --- | -| Official https://husky.underdog.ai/; displayed2026-09-20, read2026-10-27; unversioned | Public-page report only; no canonical source repository, implementation revision or retained receipts identified | [Document-research candidate](../src/inferrsi/knowledge/data/skill_library/repositories/husky/SKILL.md), bounded attribution and original verification questions. User-approved AREX-informed document adaptation, not complete repo-to-skill/native verification. Source license UNKNOWN; NO_LICENSE resolver unavailable. No new measured coverage. | +| Official https://husky.underdog.ai/; displayed2026-09-20, read2026-10-27; unversioned | Public-page report only; no canonical source repository, implementation revision or retained receipts identified | [Document-research candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/husky/SKILL.md), bounded attribution and original verification questions. User-approved AREX-informed document adaptation, not complete repo-to-skill/native verification. Source license UNKNOWN; NO_LICENSE resolver unavailable. No new measured coverage. | ## LoopX Goal governance source knowledge | Source / revision | Inspected scope | Reuse, extraction and limits | | --- | --- | --- | -| [loopx-project/loopx](https://github.com/loopx-project/loopx/tree/f094b2cfb2acde7a4a0d4d875bba52f64ac31ba9) @ `f094b2cfb2acde7a4a0d4d875bba52f64ac31ba9` (Apache-2.0; original NOTICE and historical MIT text retained) | Direct bounded source inspection, 2026-10-26: TypeScript run/spend owners, Python adapters and health guidance, selected canonical Todo provider/read seams, direct local accounting transaction/admission dependencies, retired Todo-event guard/document, and named tests; exact bytes and inspected sections in [source ledger](../src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/source-ledger.json) | mined — [existing rsi-evidence owner](../src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/README.md) gains authority/projection distinctions, ordered next-action and local receipt-recovery mechanisms with counterexamples; optimization-search-policy links that owner. Source-derived candidate; empirical `not_run`, agent efficacy not evaluated. No LoopX installation/execution, external exactly-once, provider hard cost cap, new runtime or hardware/network coverage is established. | +| [loopx-project/loopx](https://github.com/loopx-project/loopx/tree/f094b2cfb2acde7a4a0d4d875bba52f64ac31ba9) @ `f094b2cfb2acde7a4a0d4d875bba52f64ac31ba9` (Apache-2.0; original NOTICE and historical MIT text retained) | Direct bounded source inspection, 2026-10-26: TypeScript run/spend owners, Python adapters and health guidance, selected canonical Todo provider/read seams, direct local accounting transaction/admission dependencies, retired Todo-event guard/document, and named tests; exact bytes and inspected sections in [source ledger](../src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/source-ledger.json) | mined — [existing rsi-evidence owner](../src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md) gains authority/projection distinctions, ordered next-action and local receipt-recovery mechanisms with counterexamples; optimization-search-policy links that owner. Source-derived candidate; empirical `not_run`, agent efficacy not evaluated. No LoopX installation/execution, external exactly-once, provider hard cost cap, new runtime or hardware/network coverage is established. | The current readset does not replace RFC #137's historical `a6f28057b49fe03bb234fa6ce1bac5d61df0c58f` reference or implement its checkpoint @@ -505,37 +505,37 @@ resources do not reproduce runtime behavior or promote the candidate. | Source / revision | Inspected scope | Reuse and limits | | --- | --- | --- | -| mw00/project-maya @ `249f9a87d1a9d72bec2b684d704b0aa9296b1c9c`, VERSION1.1.34; AREX @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Bounded GLM/DS4/MM3 model/residency/state/control and quality/calibration evidence, plus shallow new Qwen dispatch/format/guard/license map | [Complete candidate](../src/inferrsi/knowledge/data/skill_library/repositories/project-maya/SKILL.md),82-file source inventory, source MIT notices and3 offline helpers. Actual portable native-C++ adaptation with4 original pure host policy tests passed. Required GPU/model/state/vision/quality/service evidence remains blocked; no DisCo CLI/model run or automatic promotion. NO_LICENSE is the actual unavailable gh resolver fallback, not legal absence. | +| mw00/project-maya @ `249f9a87d1a9d72bec2b684d704b0aa9296b1c9c`, VERSION1.1.34; AREX @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Bounded GLM/DS4/MM3 model/residency/state/control and quality/calibration evidence, plus shallow new Qwen dispatch/format/guard/license map | [Complete candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/SKILL.md),82-file source inventory, source MIT notices and3 offline helpers. Actual portable native-C++ adaptation with4 original pure host policy tests passed. Required GPU/model/state/vision/quality/service evidence remains blocked; no DisCo CLI/model run or automatic promotion. NO_LICENSE is the actual unavailable gh resolver fallback, not legal absence. | ## vLLM Metal portable AREX source and CPU extraction | Source / revision | Inspected scope | Reuse and limits | | --- | --- | --- | -| vllm-project/vllm-metal @ `e7df7a6ff215f99c7842552d230fdbc709835934`, package0.32.0; AREX @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected platform/config/ABI/loading, cache/offload owner-consumer and speculative/deferred/GQA measurement contracts; explicit analysis spans in the [readset](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/source-identity.json) | [Candidate graph](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/SKILL.md), upstream Apache-2.0 notice and3 offline helpers. Installed pure Python API/parser checks only; dependency health failed and required Apple MLX/Metal/MPS/model/quality/service evidence remains blocked. Actual portable workflow adaptation, not DisCo CLI/model or managed import. NO_LICENSE is the actual unauthenticated resolver fallback. | +| vllm-project/vllm-metal @ `e7df7a6ff215f99c7842552d230fdbc709835934`, package0.32.0; AREX @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected platform/config/ABI/loading, cache/offload owner-consumer and speculative/deferred/GQA measurement contracts; explicit analysis spans in the [readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/source-identity.json) | [Candidate graph](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/SKILL.md), upstream Apache-2.0 notice and3 offline helpers. Installed pure Python API/parser checks only; dependency health failed and required Apple MLX/Metal/MPS/model/quality/service evidence remains blocked. Actual portable workflow adaptation, not DisCo CLI/model or managed import. NO_LICENSE is the actual unauthenticated resolver fallback. | ## mistral.rs Metal paged-attention extraction | Source / revision | Inspected scope | Reuse and limits | | --- | --- | --- | -| EricLBuehler/mistral.rs @ `9459891e7697238225885518119682ac8209a612`, workspace 0.9.5; AREX @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected Rust owner/native MSL consumer/test assertions for decode, prefill and actual-model preparation; [bounded readset](../src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/source-identity.json). Candle `66a8cf184a5a519671454066b1b9efd446ec9f5c` is declared lineage, not a full dependency audit | [Candidate graph](../src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/SKILL.md), root MIT and component Apache-2.0/NOTICE, three offline helpers. Actual portable Rust/source adaptation; no Rust compilation, native Metal/kernel/model/quality/service or efficacy run. Required backend blocked; no DisCo CLI/model or managed import. NO_LICENSE is actual unauthenticated resolver fallback. | +| EricLBuehler/mistral.rs @ `9459891e7697238225885518119682ac8209a612`, workspace 0.9.5; AREX @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected Rust owner/native MSL consumer/test assertions for decode, prefill and actual-model preparation; [bounded readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/source-identity.json). Candle `66a8cf184a5a519671454066b1b9efd446ec9f5c` is declared lineage, not a full dependency audit | [Candidate graph](../src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/SKILL.md), root MIT and component Apache-2.0/NOTICE, three offline helpers. Actual portable Rust/source adaptation; no Rust compilation, native Metal/kernel/model/quality/service or efficacy run. Required backend blocked; no DisCo CLI/model or managed import. NO_LICENSE is actual unauthenticated resolver fallback. | ## MLX Swift LM lifecycle and recurrent Metal extraction | Source / revision | Inspected scope | Reuse and limits | | --- | --- | --- | -| ml-explore/mlx-swift-lm @ `3d066d3a0ffdc7e70c476a3d31488ea62dfdd0ac`; AREX @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected actual container/mutex/iterator/worker/loop and embedded GatedDelta MSL/host/caller/test contracts; [bounded readset](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/source-identity.json). MLX Swift manifest range is not a resolved dependency pin | [Candidate graph](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/SKILL.md), retained MIT, two offline support helpers, source-only masked-output parity gap. Actual partial Swift/source adaptation; no compile/original host-control/native/Metal/model/quality/service or efficacy run. Required native blocked; no DisCo CLI/model/managed import. NO_LICENSE is actual unauthenticated resolver fallback. | +| ml-explore/mlx-swift-lm @ `3d066d3a0ffdc7e70c476a3d31488ea62dfdd0ac`; AREX @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected actual container/mutex/iterator/worker/loop and embedded GatedDelta MSL/host/caller/test contracts; [bounded readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/source-identity.json). MLX Swift manifest range is not a resolved dependency pin | [Candidate graph](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/SKILL.md), retained MIT, two offline support helpers, source-only masked-output parity gap. Actual partial Swift/source adaptation; no compile/original host-control/native/Metal/model/quality/service or efficacy run. Required native blocked; no DisCo CLI/model/managed import. NO_LICENSE is actual unauthenticated resolver fallback. | ## MLC LLM compiled Metal and runtime contracts | Source / revision | Selected scope | Reuse and boundary | | --- | --- | --- | -| mlc-ai/mlc-llm @ `8978ea9aec626d64096aaa92fbc89bda2e3ea802`; TVM gitlink mlc-ai/relax @ `201fa0001c81431ccce05419b442c47779ea917c` | Canonical artifact/task/parameter metadata and consumer boundary; Apple target/export; generic cache lowering/config; actual pinned TVM Metal codegen/module/storage/stream/test assertions. [Exact two-owner readset](../src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/source-identity.json) | [Candidate graph](../src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/SKILL.md), both Apache LICENSE/NOTICE owners, three declaration helpers. Opt-in WebLLM manifest is not native ABI support; kind=metal fallback may not run. No original compiled CPU/native/compiler/Metal/model/fit/quality/service or efficacy execution; native blocked. | +| mlc-ai/mlc-llm @ `8978ea9aec626d64096aaa92fbc89bda2e3ea802`; TVM gitlink mlc-ai/relax @ `201fa0001c81431ccce05419b442c47779ea917c` | Canonical artifact/task/parameter metadata and consumer boundary; Apple target/export; generic cache lowering/config; actual pinned TVM Metal codegen/module/storage/stream/test assertions. [Exact two-owner readset](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/source-identity.json) | [Candidate graph](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/SKILL.md), both Apache LICENSE/NOTICE owners, three declaration helpers. Opt-in WebLLM manifest is not native ABI support; kind=metal fallback may not run. No original compiled CPU/native/compiler/Metal/model/fit/quality/service or efficacy execution; native blocked. | ## Flare preparation and complete-session source knowledge | Source / revision | Inspected scope | Reuse, extraction and limits | | --- | --- | --- | -| NVIDIA/flare-local-inference → [NVIDIA/local-inference](https://github.com/NVIDIA/local-inference/tree/bce032c014bae34502ba22d3304fa084d5b40afe) @ `bce032c014bae34502ba22d3304fa084d5b40afe` (Apache-2.0) | Bounded source inspection, 2026-10-28: preparation/package documentation, actual ABI/prepared validators and call path, cache/scheduler/worker, offload geometry/index/record/transfer, server test assertions and qualification/provenance; [20-file pin/blob/readset ledger](../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md) | mined — existing [weight-loading preparation owner](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/flare-preparation-boundary.md) plus one [Flare session candidate](../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md): full KV/recurrent/convolution/tail state, cancellation/transfer/fences, five restore-admission outcomes and timing/namespace/cost diagnostics. Research prototype SM120/SM121, GDS deferred; native/model/device/service and efficacy not_run/not evaluated. Root source license does not grant model/dependency rights. | +| NVIDIA/flare-local-inference → [NVIDIA/local-inference](https://github.com/NVIDIA/local-inference/tree/bce032c014bae34502ba22d3304fa084d5b40afe) @ `bce032c014bae34502ba22d3304fa084d5b40afe` (Apache-2.0) | Bounded source inspection, 2026-10-28: preparation/package documentation, actual ABI/prepared validators and call path, cache/scheduler/worker, offload geometry/index/record/transfer, server test assertions and qualification/provenance; [20-file pin/blob/readset ledger](../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md) | mined — existing [weight-loading preparation owner](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/flare-preparation-boundary.md) plus one [Flare session candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md): full KV/recurrent/convolution/tail state, cancellation/transfer/fences, five restore-admission outcomes and timing/namespace/cost diagnostics. Research prototype SM120/SM121, GDS deferred; native/model/device/service and efficacy not_run/not evaluated. Root source license does not grant model/dependency rights. | The redirect preserves the fixed source commit; no ABI/version upgrade is implied. [Construction and evaluation plan](reviews/issue148-flare-source-distillation.md) @@ -547,28 +547,28 @@ create a repo runtime descriptor, backend port or measured resource effect. | Reference (origin @ revision) | Inspected scope | Extraction and remaining limits | | --- | --- | --- | -| xororz/local-dream @ `db162df4d9bb6efc6595250423d8df26241f4c27`; AREX-Skill @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected Android backend/host/generation consumers, native dispatch/shared-lock/cancellation/prompt-cache/selected residency, separate core/DiT packaging/ABI and host-test definitions; [bounded readset and rights](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/repo-provenance.md), [exact identity ledger](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/source-identity.json). Dependency bodies, full app and guide site excluded | [local-dream candidate](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/SKILL.md) plus one declaration-only helper. Actual source CC BY-NC 4.0; API NOASSERTION is not permission or relicensing. Android/MNN/QNN/Hexagon/model/media/fit/thermal/service/performance and skill efficacy not_run. No measured resource cell or managed DisCo import. | +| xororz/local-dream @ `db162df4d9bb6efc6595250423d8df26241f4c27`; AREX-Skill @ `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` | Selected Android backend/host/generation consumers, native dispatch/shared-lock/cancellation/prompt-cache/selected residency, separate core/DiT packaging/ABI and host-test definitions; [bounded readset and rights](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/repo-provenance.md), [exact identity ledger](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/source-identity.json). Dependency bodies, full app and guide site excluded | [local-dream candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/SKILL.md) plus one declaration-only helper. Actual source CC BY-NC 4.0; API NOASSERTION is not permission or relicensing. Android/MNN/QNN/Hexagon/model/media/fit/thermal/service/performance and skill efficacy not_run. No measured resource cell or managed DisCo import. | ## TorchNPU framework operating graph | Source / revision | Inspected scope | Extraction and qualification boundary | | --- | --- | --- | -| Ascend/pytorch @ `046934140b96836c3fcef14d0f39ae54dc0a06bf`; parent-pinned Ascend/op-plugin @ `ad946817034b3270e309c81c54f0ee808d0b62cb` | Selected spans in 48 parent and six op-plugin implementation, documentation and test-definition files: installation/build, PrivateUse1/dispatch, generator/fallback and representative Add/extension adapters, allocator/stream/event/graph, HCCL, AMP and profiling. [Exact readset](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-manifest.json) separates Git blob identities from inspected checkout bytes and normalized spans; [provenance](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/repo-provenance.md) retains every gitlink and unread dependency boundary | [torch-npu candidate graph](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md), eight child procedures and three bounded helpers. Whole-tree inventory is distinct from semantic inspection. Source and documentation notices remain separate from resolver `NOASSERTION`; Linux/NPU/CANN, HCCL and optional compiler execution are `BLOCKED_REQUIRED_BACKEND / not_run`. No source build, native/model/quality/service/performance result, candidate promotion or managed DisCo import. | +| Ascend/pytorch @ `046934140b96836c3fcef14d0f39ae54dc0a06bf`; parent-pinned Ascend/op-plugin @ `ad946817034b3270e309c81c54f0ee808d0b62cb` | Selected spans in 48 parent and six op-plugin implementation, documentation and test-definition files: installation/build, PrivateUse1/dispatch, generator/fallback and representative Add/extension adapters, allocator/stream/event/graph, HCCL, AMP and profiling. [Exact readset](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/source-manifest.json) separates Git blob identities from inspected checkout bytes and normalized spans; [provenance](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/repo-provenance.md) retains every gitlink and unread dependency boundary | [torch-npu candidate graph](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/SKILL.md), eight child procedures and three bounded helpers. Whole-tree inventory is distinct from semantic inspection. Source and documentation notices remain separate from resolver `NOASSERTION`; Linux/NPU/CANN, HCCL and optional compiler execution are `BLOCKED_REQUIRED_BACKEND / not_run`. No source build, native/model/quality/service/performance result, candidate promotion or managed DisCo import. | ## PyTorch framework operating graph | Source / revision | Inspected scope | Extraction and qualification boundary | | --- | --- | --- | -| pytorch/pytorch @ `9a90a4fefcaafa63ac984621963a6fcc66aee68c`, source version `2.16.0a0` | Bounded implementation, documentation and test-definition spans across installation/build, tensor/dispatch/autograd, training/data, compiler/export, memory/streams/graphs, distributed/checkpoint, profiling and custom operators. [Exact readset](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-readset.json) retains Git blob and inspected checkout identities separately; [directory coverage](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-directory-coverage.md) distinguishes tree discovery from substantive inspection and unpopulated dependencies | [pytorch candidate graph](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/SKILL.md), eight child procedures, bounded public-API helpers and exact LICENSE/NOTICE bytes. Resolver `NOASSERTION` is retained separately from source terms. Preparation and CPU observations use the distinct `2.15.0+cpu` wheel, not this source build. Required accelerator, native extension, distributed-device and deployment checks remain blocked/not_run; no quality/service/performance or efficacy claim, candidate promotion, managed DisCo import or implicit TorchNPU compatibility pairing. | +| pytorch/pytorch @ `9a90a4fefcaafa63ac984621963a6fcc66aee68c`, source version `2.16.0a0` | Bounded implementation, documentation and test-definition spans across installation/build, tensor/dispatch/autograd, training/data, compiler/export, memory/streams/graphs, distributed/checkpoint, profiling and custom operators. [Exact readset](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/source-readset.json) retains Git blob and inspected checkout identities separately; [directory coverage](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/source-directory-coverage.md) distinguishes tree discovery from substantive inspection and unpopulated dependencies | [pytorch candidate graph](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/SKILL.md), eight child procedures, bounded public-API helpers and exact LICENSE/NOTICE bytes. Resolver `NOASSERTION` is retained separately from source terms. Preparation and CPU observations use the distinct `2.15.0+cpu` wheel, not this source build. Required accelerator, native extension, distributed-device and deployment checks remain blocked/not_run; no quality/service/performance or efficacy claim, candidate promotion, managed DisCo import or implicit TorchNPU compatibility pairing. | ## vLLM Ascend v0.27.1rc1 serving scope | Source / revision | Inspected scope | Extraction and qualification boundary | | --- | --- | --- | -| vllm-project/vllm-ascend at `3b31886237ed65c435e0965b10e8002ff5769766` | Release table, plugin hooks, DeepSeek-V4/DSpark evolution, Kimi K3/MiniMax-M3/GLM5+ optimization, DSA-CP/FlashComm1/Fused-MC2 and fusion dispatch, reversals and numerical oracles. [History screening and semantic scope](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/history-scope.md). [Source locators](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/sources.md) | [Release-specific diagnosis](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md). Candidate; device execution and downstream efficacy unverified. Older ACLGraph and checkpoint-adaptation owners retain their own source revisions. | +| vllm-project/vllm-ascend at `3b31886237ed65c435e0965b10e8002ff5769766` | Release table, plugin hooks, DeepSeek-V4/DSpark evolution, Kimi K3/MiniMax-M3/GLM5+ optimization, DSA-CP/FlashComm1/Fused-MC2 and fusion dispatch, reversals and numerical oracles. [History screening and semantic scope](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/history-scope.md). [Source locators](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/sources.md) | [Release-specific diagnosis](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/SKILL.md). Candidate; device execution and downstream efficacy unverified. Older ACLGraph and checkpoint-adaptation owners retain their own source revisions. | ## DeepSeek streamed expert-cache oracle contracts | Source | Inspected scope | Canonical extraction and boundary | | --- | --- | --- | -| [ronak-create/deepseek-v4-in-c](https://github.com/ronak-create/deepseek-v4-in-c/tree/d0402056d0da4bf7edfd2d4977de479a0db9d201) @ `d0402056d0da4bf7edfd2d4977de479a0db9d201` (Apache-2.0 root and selected-file declarations; third-party/model rights excluded) | Bounded source inspection, 2026-11-03: byte/fixture geometry, begin/end slot lifetime, actual model/oracle callback wiring and tiny-reference limits; [selected hashes, regions and rights](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-source.json) | [weight-loading-optimization](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md) ([expert-cache oracle reference](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-oracles.md)): independent buffered bytes, usable residents during misses, ordered accumulation and oracle reachability. Source-derived candidate/not_run; historical corruption reports are not reproduced tests, and tiny fixtures do not establish released-model quality or performance. | +| [ronak-create/deepseek-v4-in-c](https://github.com/ronak-create/deepseek-v4-in-c/tree/d0402056d0da4bf7edfd2d4977de479a0db9d201) @ `d0402056d0da4bf7edfd2d4977de479a0db9d201` (Apache-2.0 root and selected-file declarations; third-party/model rights excluded) | Bounded source inspection, 2026-11-03: byte/fixture geometry, begin/end slot lifetime, actual model/oracle callback wiring and tiny-reference limits; [selected hashes, regions and rights](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/deepseek-expert-cache-source.json) | [weight-loading-optimization](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md) ([expert-cache oracle reference](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/deepseek-expert-cache-oracles.md)): independent buffered bytes, usable residents during misses, ordered accumulation and oracle reachability. Source-derived candidate/not_run; historical corruption reports are not reproduced tests, and tiny fixtures do not establish released-model quality or performance. | diff --git a/docs/reviews/issue148-flare-source-distillation.md b/docs/reviews/issue148-flare-source-distillation.md index 03930f60e..decbb8688 100644 --- a/docs/reviews/issue148-flare-source-distillation.md +++ b/docs/reviews/issue148-flare-source-distillation.md @@ -12,10 +12,10 @@ NVIDIA/local-inference at that same commit. The actual commit tree is | Existing owner | Overlap decision | Delivered addition | |---|---|---| -| [weight-loading-optimization](../../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md#10-separate-preparation-from-serving) | Generic loading/startup already has an owner | One preparation-boundary reference and typed Flare source; compilation/export/load/readiness/request phases stay distinct | -| [audit-vllm-kv-lifetimes](../../src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md) | Scheduler/connector lifetime concepts overlap, but concrete vLLM contracts differ | Related route only; no source or runtime replacement | -| [kv-tier-management](../../src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md) | Tier policy overlaps, but LMCache MP configuration is not Flare's protocol | Related route only; no copied knobs or backend claims | -| [audit-flare-session-lifecycle](../../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md) | Complete KV plus recurrent/convolution/tail state and scheduler-owned transfer tickets were not covered by those owners | One CUDA-scoped registered candidate with complete-state/fence/admission diagnostics | +| [weight-loading-optimization](../../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md#10-separate-preparation-from-serving) | Generic loading/startup already has an owner | One preparation-boundary reference and typed Flare source; compilation/export/load/readiness/request phases stay distinct | +| [audit-vllm-kv-lifetimes](../../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md) | Scheduler/connector lifetime concepts overlap, but concrete vLLM contracts differ | Related route only; no source or runtime replacement | +| [kv-tier-management](../../src/inferrsi/knowledge/data/skill_library/systems/engines/lmcache/kv-tier-management/SKILL.md) | Tier policy overlaps, but LMCache MP configuration is not Flare's protocol | Related route only; no copied knobs or backend claims | +| [audit-flare-session-lifecycle](../../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md) | Complete KV plus recurrent/convolution/tail state and scheduler-owned transfer tickets were not covered by those owners | One CUDA-scoped registered candidate with complete-state/fence/admission diagnostics | The construction uses AREX-Skill `ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6` as a **method**, adapting its @@ -29,7 +29,7 @@ AREX native verification or a new construction framework. The anchor is source/task-agnostic; scoped capabilities are preparation compatibility, complete restoration and admission/lifetime diagnosis. Strategy is reuse-existing: the existing generic owner plus one Flare-specific owner. -The [20-file ledger](../../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-ledger.json) +The [20-file ledger](../../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/source-ledger.json) identifies exact bytes and inspected symbols/sections. Kernels, model recipes, exporter implementation and unrelated repository areas are excluded. Upstream scripts are reference-only: preparation and qualification require model/package, diff --git a/docs/reviews/issue91-afd-p2p.md b/docs/reviews/issue91-afd-p2p.md index fcfcea5f4..2d060582c 100644 --- a/docs/reviews/issue91-afd-p2p.md +++ b/docs/reviews/issue91-afd-p2p.md @@ -14,9 +14,9 @@ is part of this delivery. The scoped capabilities are (1) role/world/subgroup rank interpretation with uneven Attention/FFN counts and (2) one per-stage, per-peer payload round trip. The payload scope is CUDA P2pNccl eager dense FFN, Attention-side gating disabled. -The complete [candidate](../../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) -owns both rules, with one [source ledger](../../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md) -and the full package-supported [license/notice resource](../../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/afd-plugin-LICENSE.md). +The complete [candidate](../../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) +owns both rules, with one [source ledger](../../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md) +and the full package-supported [license/notice resource](../../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/references/afd-plugin-LICENSE.md). The [historical #105 intake](issue91-afd-plugin-intake.md) remains unchanged. Its located-not-mined conclusion described that earlier delivery; this bounded diff --git a/docs/reviews/issue91-afd-plugin-intake.md b/docs/reviews/issue91-afd-plugin-intake.md index 407448a37..2f1731bd5 100644 --- a/docs/reviews/issue91-afd-plugin-intake.md +++ b/docs/reviews/issue91-afd-plugin-intake.md @@ -37,7 +37,7 @@ The README recommends Decode, while the connector module describes support for p Reuse the repository identity `afd_plugin`; do not add another registry. The future primary owner, if a procedure is later accepted, should be an AFD repository capability reached through that existing identity. No owner is registered by this intake. -The existing [vLLM KV-lifetime procedure](../../src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md) owns scheduler/connector KV retention and completion obligations at a different pin. It does not supply this hidden-state Attention/FFN rank mapping. [Elastic serving placement](../../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) separates replica migration and sequence-parallel group changes; neither is this AFD partition. [Ascend parallelism guidance](../../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md) is not evidence for CUDA P2pNccl or the excluded NPU CAM branches. Existing owners remain unchanged. +The existing [vLLM KV-lifetime procedure](../../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md) owns scheduler/connector KV retention and completion obligations at a different pin. It does not supply this hidden-state Attention/FFN rank mapping. [Elastic serving placement](../../src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) separates replica migration and sequence-parallel group changes; neither is this AFD partition. [Ascend parallelism guidance](../../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/SKILL.md) is not evidence for CUDA P2pNccl or the excluded NPU CAM branches. Existing owners remain unchanged. A later authorized extraction should: diff --git a/docs/reviews/issue91-akg.md b/docs/reviews/issue91-akg.md index a2ad9eac3..a93f4df64 100644 --- a/docs/reviews/issue91-akg.md +++ b/docs/reviews/issue91-akg.md @@ -17,9 +17,9 @@ The InferRSI library/reader contract remains authoritative. and the selected `inequality-convert-to-if` control flow. - **Ground:** repository identity and discovery prefix resolved to [mindspore-ai/akg @ 5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff](https://github.com/mindspore-ai/akg/tree/5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff). - The [canonical source ledger](../../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/akg-affine-tail-guards.md#source-ledger) + The [canonical source ledger](../../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/akg-affine-tail-guards.md#source-ledger) records exact implementation/fixture/license ranges. -- **Construct:** reuse [kernel-writing-guide](../../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md). +- **Construct:** reuse [kernel-writing-guide](../../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md). Its existing tail-mask and verify-before-benchmark guidance owns the question. The added reference supplies target/option/loop gates, an explicit example, failure observations and a bounded verification procedure. No new skill or diff --git a/docs/reviews/issue91-awesome-ascend.md b/docs/reviews/issue91-awesome-ascend.md index 3a90383ba..ba810c541 100644 --- a/docs/reviews/issue91-awesome-ascend.md +++ b/docs/reviews/issue91-awesome-ascend.md @@ -97,7 +97,7 @@ All paths are relative to the same full source commit above. The eight blobs are | `skills/training/torch-npu-comm-test/references/supported-ops.md` | 49-58,81-89,143-152,195-224: documented shape and P2P intent checked against code | Detailed communication clauses map to immutable lines in the -[owner-local readset](../../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md). +[owner-local readset](../../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md). ## Candidate old/new cases diff --git a/docs/reviews/issue91-cannbot.md b/docs/reviews/issue91-cannbot.md index 235270f0a..8822eb1b6 100644 --- a/docs/reviews/issue91-cannbot.md +++ b/docs/reviews/issue91-cannbot.md @@ -11,7 +11,7 @@ Refs [issue #91](https://github.com/InferMatrixLab/InferRSI/issues/91). - Capability `Q`: one decision rule for accepting a fixed-stage PyTorch/Ascend optimization report: current execution identity, quality result and comparable measurements must agree; callback completion has a narrower meaning. -- Evidence `X`: the bounded [source ledger and trace](../../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md). +- Evidence `X`: the bounded [source ledger and trace](../../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md). Exact Git objects establish identity; source reading establishes instructions and callback predicates. No upstream code, installer, agent host, model, API, device or paid evaluation was executed. diff --git a/docs/reviews/issue91-dlrover.md b/docs/reviews/issue91-dlrover.md index e7500dfd4..41773a947 100644 --- a/docs/reviews/issue91-dlrover.md +++ b/docs/reviews/issue91-dlrover.md @@ -11,7 +11,7 @@ Refs [#91](https://github.com/InferMatrixLab/InferRSI/issues/91) and its publication, tracker update and durable recovery; trace full/sharded replica transport through its consumer; select bounded follow-up checks by failure domain. - Evidence `X`: seven immutable files, read ranges and Apache-2.0 attribution in - the [canonical source inventory](../../src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md). + the [canonical source inventory](../../src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/sources.md). Source text was fetched through the GitHub connector. No source package was installed or imported; documentation intent is kept separate from implementation. - Construction strategy: `reuse-existing`, single `extend-repo-skill` workflow diff --git a/docs/reviews/issue91-fastsafetensors.md b/docs/reviews/issue91-fastsafetensors.md index e5138b17b..6d18d6225 100644 --- a/docs/reviews/issue91-fastsafetensors.md +++ b/docs/reviews/issue91-fastsafetensors.md @@ -22,7 +22,7 @@ No efficacy evaluation has been executed. - **Ground (`X`):** resolve the historical prefix from a clean source checkout; inspect caller, owner-only copier selection, collective adapters, readiness, output lifetime and author tests. The owner-local - [readset](../../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/fastsafetensors-transfer-readset.md) + [readset](../../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/fastsafetensors-transfer-readset.md) maps each rule to immutable paths/lines and Apache-2.0 attribution. Other source claims, including inherited I/O/save performance, remain carried evidence. - **Construct (`G_tilde`):** one existing skill with additive §9 and one source diff --git a/docs/reviews/issue91-sd-cpp-loader-storage.md b/docs/reviews/issue91-sd-cpp-loader-storage.md index 9df1fc349..ca3e13042 100644 --- a/docs/reviews/issue91-sd-cpp-loader-storage.md +++ b/docs/reviews/issue91-sd-cpp-loader-storage.md @@ -1,7 +1,7 @@ # Stable-diffusion.cpp catalog-to-storage construction This bounded Issue #91 slice extends the existing -[`audit-sd-cpp-residency`](../../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) +[`audit-sd-cpp-residency`](../../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) candidate. It preserves its stable ID, eight residency steps, MIT text, original source pin and historical archive receipt. No engine, dependency, evaluator or hardware execution is introduced. Material author: Codex / `gpt-6.1-sol`, @@ -12,8 +12,8 @@ reasoning effort `xhigh`. | AREX phase | Decision and native InferRSI asset | |---|---| | Scope | Fill the catalog/source → actual byte delivery → immediate storage-consumer gap. Select ordinary non-ZIP safetensors and its shard index; leave generic startup optimization with `weight-loading-optimization`. Do not replace request-time residency guidance. | -| Ground | Fixed leejet/stable-diffusion.cpp `b13a48219d5cdf408edcb93da6cd69b612ed03d3`, tree `df847d004c6529eefb02bb629d7ee26e740121a9`. Twelve C++ blobs and retained MIT text have exact byte identity checks; semantic ranges are in the [new ledger](../../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md#fixed-readset-and-attribution). The prior nineteen-blob receipt remains historical. | -| Construct | Reuse one owner and the native metadata/readers. Add [two loader/storage decisions](../../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md), a root route, additive source/validation notes and scoped entries in both matrices. No new ID, backend, counter or parallel registry is needed. | +| Ground | Fixed leejet/stable-diffusion.cpp `b13a48219d5cdf408edcb93da6cd69b612ed03d3`, tree `df847d004c6529eefb02bb629d7ee26e740121a9`. Twelve C++ blobs and retained MIT text have exact byte identity checks; semantic ranges are in the [new ledger](../../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md#fixed-readset-and-attribution). The prior nineteen-blob receipt remains historical. | +| Construct | Reuse one owner and the native metadata/readers. Add [two loader/storage decisions](../../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md), a root route, additive source/validation notes and scoped entries in both matrices. No new ID, backend, counter or parallel registry is needed. | | Verify | Check actual owner/resource discovery, metadata, local links, immutable source ranges and retained license. Preserve the old eight-step body and source identity. Bind generated records to a real source commit only after source freeze. Exact execution and independent-review receipts belong to the PR; package/schema checks do not qualify native loading or agent efficacy. | AREX portable distill/extend guidance is used from fixed diff --git a/docs/reviews/issue91-vllm-ascend-host-offload.md b/docs/reviews/issue91-vllm-ascend-host-offload.md index ee452c414..8e0f1a3b6 100644 --- a/docs/reviews/issue91-vllm-ascend-host-offload.md +++ b/docs/reviews/issue91-vllm-ascend-host-offload.md @@ -2,13 +2,13 @@ ## Construction decision -This bounded Issue #91 contribution extends the existing [host-memory-management](../../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/SKILL.md) owner. The user problem is host descriptor retention after native CPU KV-offload submission or wait. The added rule follows normal event-query-gated pool reuse, with readiness, ownership and physical RAM release kept separate. +This bounded Issue #91 contribution extends the existing [host-memory-management](../../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/SKILL.md) owner. The user problem is host descriptor retention after native CPU KV-offload submission or wait. The added rule follows normal event-query-gated pool reuse, with readiness, ownership and physical RAM release kept separate. The AREX portable distill/extend method was applied as scope → ground → construct → verify. The existing InferRSI ownership/schema/readers remain authoritative; this is not a DisCo managed import or a routing/license migration. One source case owns the new behavior; no new skill ID, backend, count, runtime helper or dependency is introduced. Existing four mechanisms, CUDA/Python/NUMA examples and source history remain. Only the blanket NPU limitation is narrowed to route to the versioned case; historical broad API, threshold and performance statements are not revalidated by this extension. -The [source note](../../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload.md) supplies the bounded procedure and actual caller/consumer/cleanup order. The [seven-file ledger](../../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload-readset.json) pins vllm-project/vllm-ascend, repository ID 924058625, commit 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf and actual tree 5ca8fb3b2c6b9a64d74a3b9a74403ce44014e4a5, with exact Git blobs, SHA256, byte counts, real EOF bounds and selected ranges. Source verification date is 2026-10-23. The acquisition cache used the commit as its request label; reconstructed Git tree bytes bind the actual tree above. +The [source note](../../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-native-offload.md) supplies the bounded procedure and actual caller/consumer/cleanup order. The [seven-file ledger](../../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-native-offload-readset.json) pins vllm-project/vllm-ascend, repository ID 924058625, commit 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf and actual tree 5ca8fb3b2c6b9a64d74a3b9a74403ce44014e4a5, with exact Git blobs, SHA256, byte counts, real EOF bounds and selected ranges. Source verification date is 2026-10-23. The acquisition cache used the commit as its request label; reconstructed Git tree bytes bind the actual tree above. -Full original Apache-2.0 [LICENSE bytes](../../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-LICENSE.md) are retained (11,357 bytes); contributor and Huawei notices remain attributed. No root NOTICE entry exists in the inspected root tree; this is not a third-party-license audit. This source addition does not relicense the owner's older material. +Full original Apache-2.0 [LICENSE bytes](../../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-LICENSE.md) are retained (11,357 bytes); contributor and Huawei notices remain attributed. No root NOTICE entry exists in the inspected root tree; this is not a third-party-license audit. This source addition does not relicense the owner's older material. ```mermaid flowchart LR diff --git a/docs/reviews/issue91-vllm-rlt-finalization.md b/docs/reviews/issue91-vllm-rlt-finalization.md index 7ffd5906f..a454cca55 100644 --- a/docs/reviews/issue91-vllm-rlt-finalization.md +++ b/docs/reviews/issue91-vllm-rlt-finalization.md @@ -8,7 +8,7 @@ The source anchor is `InferMatrixLab/vllm-rlt@4a350344027960a48384626aa646f1765f Portable AREX `distill-ml-knowledge` scope → ground → construct → verify informs the direct, task-agnostic source construction. The existing `extend-repo-skill` ownership check found no RLT operating owner; the separately approved `audit-recurrent-kv-finalization` capability is the only new node. InferRSI's existing typed metadata, canonical library, registry and readers are the packaging contract. No DisCo managed import, extra router, dependency installation or evaluation framework is introduced. -The grounded evidence is the intake's bounded manager/runner/test/license material plus the fixed caller, request, metadata, Ouro consumer and Torch dispatch tuple. The [source notes](../../src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md) explain the call chain and exclusions; the [file ledger](../../src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json) records exact Git blobs, SHA256, ranges and carried versus added acquisition. The original Apache-2.0 LICENSE and model/configuration NOTICE bytes are retained. No implementation or model weights are copied. +The grounded evidence is the intake's bounded manager/runner/test/license material plus the fixed caller, request, metadata, Ouro consumer and Torch dispatch tuple. The [source notes](../../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md) explain the call chain and exclusions; the [file ledger](../../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json) records exact Git blobs, SHA256, ranges and carried versus added acquisition. The original Apache-2.0 LICENSE and model/configuration NOTICE bytes are retained. No implementation or model weights are copied. ```mermaid flowchart LR diff --git a/docs/reviews/issue91-vllm-rlt-intake.md b/docs/reviews/issue91-vllm-rlt-intake.md index a11cd1082..9390a3cf2 100644 --- a/docs/reviews/issue91-vllm-rlt-intake.md +++ b/docs/reviews/issue91-vllm-rlt-intake.md @@ -39,7 +39,7 @@ All source links use the full revision above. Blob identities cover retrieved by ## Ownership, applicability and follow-up -Reuse the existing `vllm_rlt` identity. A future accepted repository capability should own the recurrent-depth KV contract rather than extending the [vLLM connector KV-lifetime procedure](../../src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md): that procedure covers remote-KV completion and retention at a separate vLLM pin. [vLLM admission guidance](../../src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md) addresses resource admission, not early-exit history propagation. Neither existing owner establishes this engine's semantics. No new owner, alias or registry is created here. +Reuse the existing `vllm_rlt` identity. A future accepted repository capability should own the recurrent-depth KV contract rather than extending the [vLLM connector KV-lifetime procedure](../../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md): that procedure covers remote-KV completion and retention at a separate vLLM pin. [vLLM admission guidance](../../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/SKILL.md) addresses resource admission, not early-exit history propagation. Neither existing owner establishes this engine's semantics. No new owner, alias or registry is created here. A later authorized extraction should: diff --git a/docs/searchspace-matrix.md b/docs/searchspace-matrix.md index 916c92920..6281badd6 100644 --- a/docs/searchspace-matrix.md +++ b/docs/searchspace-matrix.md @@ -26,47 +26,47 @@ The primary compute and HBM resource. Most current work lives here. | Lifecycle step | What it covers | Candidate Skills | Cases / Experience | Evidence | Maturity | |---|---|---|---|---|---| -| **Diagnose / Configure / Verify** | CUDA eager dense-FFN Attention/FFN rank namespaces and per-stage/per-peer hidden-state ownership | [audit-afd-p2p-exchanges](../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) | A=3,F=2 control sender differs from data peer; matching total tokens does not preserve peer rows | afd-plugin @ df8d6a0c403c3ca1b4ed9cf1687e7204908bebec; [source ledger](../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md), [proposed cases](reviews/issue91-afd-p2p.md) | 🟢 (source-derived candidate; native/model/efficacy not_run; no Network qualification) | -| **Diagnose / Configure / Optimize / Verify** | Chip-qualified task fusion, normalized-versus-fused controls, ICB hazards/scratch and GPU-resident accepted-prefix state | [lithos-metal](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/SKILL.md), [compile-fusion](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/SKILL.md), [generation-state](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/SKILL.md) | Layer mixer and MLP regions differ; CPU/synthetic ICB checks cannot establish full-model/DSpark numerics or E2E rates | Lithos Metal @ `6653b7df5f2c4392ec7b038960f2796b1499923f`; bounded source readset | Source-derived candidate; efficacy not_run | -| **Observe / Diagnose / Verify** | MiniMax-H3 joint audio/video AdaLN schedule/row-map validity, Metal BF16 rounding and actual int8 dispatch versus approximate controls | [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md) | Fifty layers/reuse 1 does not disable M5 int8 or BF16 head; test-source parity is not full media quality | h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; [precision contracts](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md) | 🟢 (source-derived candidate; native/quality/performance not_run) | -| **Observe / Diagnose / Verify** | FastH3 effective parameters/ambient override, semantic checkpoint transformations, partial/prefix VSA membership and sparse eligibility/fallback | [audit-h3-metal-fasth3-semantics](../src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) | Shape agreement is not conversion; sparse-disabled remains dense fine+coarse; parameter normalization does not erase ambient token enable | h3-metal @ 4ba47a3eabdbebf801ac6664b31f8ec168bc57a8; [semantic counterchecks](../src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/semantic-contracts.md) | 🟢 (source-derived candidate; native/sparse/schedule/media/performance not_run) | -| **Diagnose / Configure / Verify** | Workflow model-weight residency, partial unload versus wrapper removal, patch identity and static/dynamic backend dispatch | [comfy-ui](../src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/SKILL.md) | CPU dynamic construction reroutes to the static patcher; wrapper/free-byte bookkeeping is not physical allocator release | ComfyUI @ b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c; [model ownership](../src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/model-residency.md) | 🟢 (source-derived audit; device/native allocator unverified) | -| **Observe / Diagnose / Verify** | Native diffusion managed budget versus device headroom, graph-segment weight protection and optional prefetch completion | [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | CPU bypasses accelerator capacity and segmentation gates; pin release is not physical free; backend capability does not prove overlap | stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; [phase residency](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md) | 🟢 (source-derived audit; native/device/model unverified) | +| **Diagnose / Configure / Verify** | CUDA eager dense-FFN Attention/FFN rank namespaces and per-stage/per-peer hidden-state ownership | [audit-afd-p2p-exchanges](../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) | A=3,F=2 control sender differs from data peer; matching total tokens does not preserve peer rows | afd-plugin @ df8d6a0c403c3ca1b4ed9cf1687e7204908bebec; [source ledger](../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md), [proposed cases](reviews/issue91-afd-p2p.md) | 🟢 (source-derived candidate; native/model/efficacy not_run; no Network qualification) | +| **Diagnose / Configure / Optimize / Verify** | Chip-qualified task fusion, normalized-versus-fused controls, ICB hazards/scratch and GPU-resident accepted-prefix state | [lithos-metal](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/SKILL.md), [compile-fusion](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/compile-fusion/SKILL.md), [generation-state](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/generation-state/SKILL.md) | Layer mixer and MLP regions differ; CPU/synthetic ICB checks cannot establish full-model/DSpark numerics or E2E rates | Lithos Metal @ `6653b7df5f2c4392ec7b038960f2796b1499923f`; bounded source readset | Source-derived candidate; efficacy not_run | +| **Observe / Diagnose / Verify** | MiniMax-H3 joint audio/video AdaLN schedule/row-map validity, Metal BF16 rounding and actual int8 dispatch versus approximate controls | [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md) | Fifty layers/reuse 1 does not disable M5 int8 or BF16 head; test-source parity is not full media quality | h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; [precision contracts](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md) | 🟢 (source-derived candidate; native/quality/performance not_run) | +| **Observe / Diagnose / Verify** | FastH3 effective parameters/ambient override, semantic checkpoint transformations, partial/prefix VSA membership and sparse eligibility/fallback | [audit-h3-metal-fasth3-semantics](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) | Shape agreement is not conversion; sparse-disabled remains dense fine+coarse; parameter normalization does not erase ambient token enable | h3-metal @ 4ba47a3eabdbebf801ac6664b31f8ec168bc57a8; [semantic counterchecks](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/semantic-contracts.md) | 🟢 (source-derived candidate; native/sparse/schedule/media/performance not_run) | +| **Diagnose / Configure / Verify** | Workflow model-weight residency, partial unload versus wrapper removal, patch identity and static/dynamic backend dispatch | [comfy-ui](../src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/SKILL.md) | CPU dynamic construction reroutes to the static patcher; wrapper/free-byte bookkeeping is not physical allocator release | ComfyUI @ b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c; [model ownership](../src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/model-residency.md) | 🟢 (source-derived audit; device/native allocator unverified) | +| **Observe / Diagnose / Verify** | Native diffusion managed budget versus device headroom, graph-segment weight protection and optional prefetch completion | [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | CPU bypasses accelerator capacity and segmentation gates; pin release is not physical free; backend capability does not prove overlap | stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; [phase residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md) | 🟢 (source-derived audit; native/device/model unverified) | | **Observe / Profile** | System-level GPU/NPU timeline, per-kernel stats, SOL%, AI Core utilization ratios, step-trace four-tuple | `ascend-performance-analysis`, `dense-performance-evidence` (L2) | CANN CSV pipeline, MindIE-SD profiling-analyze lessons | MindIE-SD @ 8481d39b; TRT-LLM nsys/ncu @ f33e0e5f | 🟢 | | **Observe / Profile** | Single-operator benchmark (msprof op, 8 CSV metrics, per-core occupancy) | `ascend-operator-profiling` | ops-profiling loop | awesome-ascend-skills @ 75ec89ce | 🟢 | | **Observe / Profile** | Trace-to-source attribution: representative-step selection, kernel→source-module ownership via code_ref, compact vs full detail views | `cann-attribution-overlap` | Selection reason + shapes + streams + code_ref retained per kernel owner | cann/oam-tools @ 8440935f (pack-mediated, wave 6) | 🟢 | | **Diagnose** | Bottleneck classification: compute-bound vs memory-bound vs AI-CPU fallback vs comm-exposed | `ascend-performance-analysis` (5-layer pipeline), `performance-ceiling-analysis` (L1–L3), `cann-attribution-overlap` (attribution-aware) | SOL% / mte2-ratio / mac-ratio heuristic | MindIE-SD; TRT-LLM roofline @ f33e0e5f; LLM-Viewer @ 1893e4b5 | 🟢 | | **Diagnose** | Roofline model, arithmetic intensity, hardware table entry for new accelerator | `performance-ceiling-analysis` (L1) | FLOPs formulas per layer type | LLM-Viewer `roofline_model.py`, `hardware_params.py` | 🟢 | -| **Diagnose / Configure / Verify** | TorchAir configuration-to-capture path: backend selection, graph keys/recapture, persistent eager fallback, input staging and output lifetime | [ascend-graph-mode](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md) | Separate registered-backend documentation from `CompilerConfig`; same shape is not address stability, output retention or replay coverage; [source map](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-path.md) | Official TorchAir @ e1118e8b0fc1b95190ab53b732c0872989550e20; historical awesome-ascend-skills import retained separately | Candidate source procedure; TorchNPU/CANN native graph, model and service validation not_run | +| **Diagnose / Configure / Verify** | TorchAir configuration-to-capture path: backend selection, graph keys/recapture, persistent eager fallback, input staging and output lifetime | [ascend-graph-mode](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/SKILL.md) | Separate registered-backend documentation from `CompilerConfig`; same shape is not address stability, output retention or replay coverage; [source map](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-graph-path.md) | Official TorchAir @ e1118e8b0fc1b95190ab53b732c0872989550e20; historical awesome-ascend-skills import retained separately | Candidate source procedure; TorchNPU/CANN native graph, model and service validation not_run | | **Configure** | Parallelism strategy (USP/CP/TP/PP), topology-aware card-group selection | `ascend-distributed-parallel` | USP activation requirement, masking upper-bound formula, AlltoAllV workaround | MindIE-SD dit-parallel-opt @ 8481d39b (9 reference files) | 🟢 | | **Configure** | Model adapter: FP8 dequantization, KV-head replication, multimodal feature validation order | `ascend-model-adaptation` | vllm-ascend FP8/multimodal lessons; [separate runtime-wrapper route](../src/inferrsi/knowledge/data/skill_library/README.md#vllm-ascend-version-and-responsibility-routing) | vllm-ascend @ bc8115ed | 🟢 | | **Optimize** | Kernel-level: Triton/CuTe/TileIR writing, kernel tuning, fusion scope analysis | `kernel-writing-guide` | Framework routing (Triton vs CuTe vs AscendC), 6 universal correctness rules, verify-before-benchmark workflow | TRT-LLM kernel-triton-writing + kernel-cute-writing @ f33e0e5f | 🟢 | -| **Diagnose / Configure / Verify** | Existing causal depthwise convolution: API/state ownership, dtype/layout/packed boundaries, gathered rows, circular cursor and build/binding evidence | [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md) | [Five forward/update decision cases](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/causal-conv1d-dispatch-state.md); invalid layout/state and unproved dispatch stop qualification | Dao-AILab/causal-conv1d @ cd81f0413cad2fc1e6f17e785ac39f59aae690cd; [contract/readset](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/causal-conv1d-contract.md); source and test intent only | Candidate; native/build/binary compatibility, downstream readiness, numerics, performance and efficacy not evaluated; no new measured coverage | -| **Diagnose / Configure / Verify** | HF Qwen3.5 conv/GDN binding lifetime, independent fallback, dense history/recurrent state and cache-free classification-prefill limits | [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md) | [Two-release contract](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-binding.md) and [five source cases](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/hf-qwen35-gdn-binding.md); resolve actual method/callable and prior state before reuse | Transformers v5.10.2 @ 0dad7b822255a0ae261ec45ae937371e859ffd1a and v5.17.0 @ 856157a2f3e9594954310df18fdccc31ffddebe9; selected HF source only | Candidate; native/model/numerical/device/performance not_run, efficacy not_evaluated; no new measured coverage | -| **Diagnose / Configure / Verify** | AKG static CUDA Affine loop tails: effective target/option gates, full point-loop bounds and protected input/output accesses | [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md) | [Tail-guard reference](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/akg-affine-tail-guards.md): separation precedence, non-divisible fixture and diagnostic boundary | AKG @ 5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff; source + FileCheck expectations only | Candidate; compiler/IR command, CUDA runtime and agent efficacy not run; no new measured coverage | +| **Diagnose / Configure / Verify** | Existing causal depthwise convolution: API/state ownership, dtype/layout/packed boundaries, gathered rows, circular cursor and build/binding evidence | [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md) | [Five forward/update decision cases](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/causal-conv1d-dispatch-state.md); invalid layout/state and unproved dispatch stop qualification | Dao-AILab/causal-conv1d @ cd81f0413cad2fc1e6f17e785ac39f59aae690cd; [contract/readset](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/causal-conv1d-contract.md); source and test intent only | Candidate; native/build/binary compatibility, downstream readiness, numerics, performance and efficacy not evaluated; no new measured coverage | +| **Diagnose / Configure / Verify** | HF Qwen3.5 conv/GDN binding lifetime, independent fallback, dense history/recurrent state and cache-free classification-prefill limits | [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md) | [Two-release contract](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/hf-qwen35-gdn-binding.md) and [five source cases](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/hf-qwen35-gdn-binding.md); resolve actual method/callable and prior state before reuse | Transformers v5.10.2 @ 0dad7b822255a0ae261ec45ae937371e859ffd1a and v5.17.0 @ 856157a2f3e9594954310df18fdccc31ffddebe9; selected HF source only | Candidate; native/model/numerical/device/performance not_run, efficacy not_evaluated; no new measured coverage | +| **Diagnose / Configure / Verify** | AKG static CUDA Affine loop tails: effective target/option gates, full point-loop bounds and protected input/output accesses | [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md) | [Tail-guard reference](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/akg-affine-tail-guards.md): separation precedence, non-divisible fixture and diagnostic boundary | AKG @ 5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff; source + FileCheck expectations only | Candidate; compiler/IR command, CUDA runtime and agent efficacy not run; no new measured coverage | | **Optimize** | Operator-level: staged accuracy-gated tuning campaign (code-level → scheduling → instruction-level), metric→action routing; AscendC tiling design per operator class (reduction/broadcast/elewise/matmul/conversion) | `ascend-operator-tuning`, `ascendc-tiling-design` | PyPTO frontend/swimlane/incore playbooks; fullload-vs-split, four-element tiling template, chip-generation (DAV_2201/3510) routing | awesome-ascend-skills pypto-op-perf-tune + ascendc-tiling-design @ 75ec89c | 🟢 | | **Optimize** | Casebook of past NPU optimization decisions: AI-CPU fallback, memory-bound, comm-exposed, graph-compilation, scheduling idle gaps | `ascend-optimization-casebook`, `inference-optimization-casebook` | Five bottleneck classes with symptom→diagnosis→action→evidence schema | MindIE-SD profiling-analyze + dit-perf-opt + perf-gate + dit-parallel-opt @ 8481d39b | 🟢 | -| **Optimize** | Bounded experiment selection, distinct mechanism hypotheses, incumbent preservation and final-artifact integrity | `optimization-search-policy` | [Sarathi chunk-budget case](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md): decode-first accounting, prefill overhead and TTFT/TBT tradeoffs; no universal budget. [Infera provenance](../src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md) adds a seed-only comparator and evaluated-trial ledger example; its trace-ranking check is proposed. Methodology provenance stays in Reference Matrix §2; no hardware coverage from agent-paper scores | vllm-evolve @ 43cc82c; scoped A/F paper amendments; separate Sarathi case and bounded Infera Git readset in Reference Matrix | 🟢 (source-derived candidate procedure; empirical not_run) | +| **Optimize** | Bounded experiment selection, distinct mechanism hypotheses, incumbent preservation and final-artifact integrity | `optimization-search-policy` | [Sarathi chunk-budget case](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/cases/sarathi-chunk-budget.md): decode-first accounting, prefill overhead and TTFT/TBT tradeoffs; no universal budget. [Infera provenance](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/README.md) adds a seed-only comparator and evaluated-trial ledger example; its trace-ranking check is proposed. Methodology provenance stays in Reference Matrix §2; no hardware coverage from agent-paper scores | vllm-evolve @ 43cc82c; scoped A/F paper amendments; separate Sarathi case and bounded Infera Git readset in Reference Matrix | 🟢 (source-derived candidate procedure; empirical not_run) | | **Optimize** | Bounded multistream overlap: dependency-DAG constrained (data/state/event/collective_order edges; shared input ≠ dependency), minimal-first, execution-mode sync contract (eager streams vs TorchAir) | `cann-attribution-overlap` | Metric separation (wall / union-of-busy / summed durations / summed duration+wait; repeated-instance product = estimate); device-memory domain qualification; formula conflicts carried unresolved | cann/oam-tools @ 8440935f (pack-mediated, wave 6) | 🟢 | | **Verify** | Evidence taxonomy (L0 correctness → L5 production), claim-evidence matching | `dense-performance-evidence`, `rsi-evidence`, `claim-contract` | InferRSI PASS/FAIL/INCONCLUSIVE protocol | InferRSI @ 4c8e1285 | 🟢 | -| **Diagnose / Verify** | Fixed-stage PyTorch/Ascend model report: bind current stage and actually loaded candidate to quality-validator output and comparable baseline | [ascend-performance-analysis](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md) | [Stage-evidence rule](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md): workflow instructions versus registered marker/keyword checks; completion is not numerical validation | canonical cann/cannbot @ c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110; selected CANN-2.0 scope | Candidate procedure only; host/model/backend not_run, agent efficacy not evaluated; no new measured coverage | +| **Diagnose / Verify** | Fixed-stage PyTorch/Ascend model report: bind current stage and actually loaded candidate to quality-validator output and comparable baseline | [ascend-performance-analysis](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/SKILL.md) | [Stage-evidence rule](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md): workflow instructions versus registered marker/keyword checks; completion is not numerical validation | canonical cann/cannbot @ c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110; selected CANN-2.0 scope | Candidate procedure only; host/model/backend not_run, agent efficacy not evaluated; no new measured coverage | | **Verify** | Same-window A/B protocol, noise floor, acceptance-state discipline; correctness-first overlap acceptance (fallback works, no KV/state reorder, actual overlap, no outside-module regression) | `ascend-optimization-casebook` (evidence-required section per case), `dense-performance-evidence`, `rsi-evidence`, `cann-attribution-overlap` | MindIE-SD perf-gate protocol: exploration vs acceptance state, three-proof requirement | MindIE-SD perf-gate @ 8481d39b; cann/oam-tools @ 8440935f | 🟢 | | **Observe / Configure / Optimize** | DALI CPU/GPU queue overlap and operator buffer sizing under explicit executor-mode gates | `training-input-pipeline` | Separated queues require async+pipelined and non-dynamic mode; output synchronization retained | Reference B1, DALI @ 6fe47c48d3b0 | 🟢 (Python API evidence, not C++/device validation) | | **Diagnose / Configure / Optimize** | Training capacity versus throughput, parameter/optimizer tier budgets, recomputation/offload choices; DCP exposed staging pause | `training-resource-budget`, `dcp-async-save-performance` | DeepSpeed stage gates; expert MFU/HFU denominator discipline; historical MindSpeed selection/allocator/rescue source boundaries | Reference CR1, CR6, CR5, C1 async-only | 🟢 (MindSpeed historical source candidate; current support unqualified) | | **Observe / Diagnose / Configure / Optimize / Verify** | vLLM V1 token/input/active-request budgets, allocation/preemption gates and async-KV admission; cap changes are conditional hypotheses | `diagnose-vllm-admission-pressure` | Queue length and KV usage do not establish allocation pressure; stats are gated and incomplete; verification proposed only | vLLM @ 7f730b4cea9a, Dot bundle; pinned scheduler/connector source ledger | 🟢 (candidate diagnosis; no runtime or gain validation) | -| **Diagnose / Verify** | Context partial-ubatch progress, topology/input graph reuse and backend copy/event readiness | [audit-llama-runtime-state](../src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md); [owner map](../src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) | Retained completed ubatches differ from whole-batch rollback; frontend reuse differs from backend captured graphs | llama.cpp @ `707616feb5f379e99ff7442680a55b4a1704c7c9`; runtime/source notes and native test assertions; static inspection 2026-10-21 carried | 🟢 (source-derived candidate; native/backend execution not run) | +| **Diagnose / Verify** | Context partial-ubatch progress, topology/input graph reuse and backend copy/event readiness | [audit-llama-runtime-state](../src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md); [owner map](../src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) | Retained completed ubatches differ from whole-batch rollback; frontend reuse differs from backend captured graphs | llama.cpp @ `707616feb5f379e99ff7442680a55b4a1704c7c9`; runtime/source notes and native test assertions; static inspection 2026-10-21 carried | 🟢 (source-derived candidate; native/backend execution not run) | | **Observe / Diagnose / Verify** | KV receive/save completion, delayed-free ownership, in-flight-write fences and failure reporting | `audit-vllm-kv-lifetimes` | Event ledger is proposed, not a telemetry exporter; abstract obligations require concrete connector/caller evidence | vLLM @ 7f730b4cea9a, Dot bundle | 🟢 (contract intelligence; concrete backend unverified) | | **Diagnose / Configure / Optimize / Verify** | Runtime ACLGraph dispatch, stable input addresses, capture-resource limits and branch-specific replay/update ordering | `ascend-aclgraph-capture-replay` | Caller owns persistent buffers; DEBUG checks positional tensors only; reduced capture sets trade coverage for resources; [version/responsibility routing](../src/inferrsi/knowledge/data/skill_library/README.md#vllm-ascend-version-and-responsibility-routing) | vllm-ascend @ 8a8e1e05ecb6, Dot bundle; wrapper/guide/tests inspected by supplier | 🟢 (conditional proposals; tests not run) | | **Diagnose / Configure / Optimize / Verify** | Device-HBM stream-use lifetime, fragmentation/counter interpretation and gated allocator controls | `torchnpu-device-memory-lifecycle` | Lifetime recording does not insert data dependencies; expandable segments change counters/control legality; GC synchronization/comment conflict retained | TorchNPU @ 67511c53cd4c, Dot bundle; device allocator ledger | 🟢 (source-supported procedure; NPU execution not run) | -| **Diagnose / Verify** | Serving model: operator/context dependencies, calibration/load domain, fidelity provenance and joint-SLO prediction uncertainty | [calibrate-serving-simulation](../src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), `performance-ceiling-analysis` | Vidur near-capacity CPU-overhead counterexample; simulator output is not a bound. [Infera source reference](../src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md) separates scalar tuner scoring from DES percentiles and retains missing-SLO/stale-document limits; no service guarantee | Vidur v2 and bounded Infera Git readset; Reference Matrix systems sources | 🟢 (prediction guidance; calibration not run) | -| **Diagnose / Verify** | Reusable-prefix compute work versus host/network KV loading cost | [performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md), [Mooncake condition](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md) | Check bytes, FLOPs and effective rate at one GPU-group scope; favorable Equation 2 still requires exposed-cost/SLO measurements | Mooncake FAST '25 §2.2, Table 1, Eqs. 1-2; separate public-paper source, not recovered foundation archive | 🟢 (analytical source condition; current feasibility and E2E benefit unverified; no SSD credit) | -| **Diagnose / Verify** | Offline offload feasibility across GPU, CPU RAM and disk | [performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md), [FlexGen condition](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/flexgen-offload-feasibility.md) | Joint weights/KV/activation placement; RAM resident plus staging/working peaks, SSD state capacity and disk↔CPU cost, effective batch versus GPU batch, exposed CPU/transfer cost and block-latency objective | FlexGen official ICML 2023, §§3–4.3 and selected A.3; separate public source, foundation archive still unavailable | Source-only candidate; empirical not_run, fitted/perfect-overlap model and relaxed OOM limits; no blanket RAM/SSD, Network/OS or current-engine qualification | -| **Diagnose / Configure / Optimize / Verify** | Serving service order versus HBM admission and suspended-KV recovery; replica migration versus elastic sequence groups | [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/SKILL.md), [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | [Hypothetical pressure/migration cases](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md); cohort/fairness and destination-reservation costs | Orca proceedings, FastServe v3, Llumnix v1, LoongServe v2; implementation pins remain separate | 🟢 (conditional paper-derived plans) | -| **Diagnose / Optimize / Verify** | Training activation lifetime, rematerialization and reserved-address packing | [plan-activation-residency](../src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/SKILL.md) | Dependency repair and capacity feasibility checked separately; no recovery-checkpoint coverage | Checkmate v3, OLLA v2; [C readset](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md) | 🟢 (planning, not solver/runtime validation) | -| **Diagnose / Configure / Optimize / Verify** | Training stage/mesh/sharding feasibility and split F/B/W execution under total-HBM and optimizer-semantics constraints | [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md), [assess-split-backward-schedule](../src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/SKILL.md) | Alpa conditional plans; Zero Bubble activation caps versus total memory; rollback versus stale updates | Alpa v3, Zero Bubble v1 | 🟢 (author evidence, no current stack validation) | -| **Diagnose / Optimize / Verify** | Attention IO, operation-class ceilings, output ownership, work supply, async dependencies and FP8 layout/accuracy | [diagnose-attention-bottlenecks](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md), `kernel-writing-guide` | [Author-reported attention cases](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md); compiler/register and precision counter-signals | FlashAttention v2 / FA2 v1 / FA3 v2 | 🟢 (Accelerator only; no Ascend implementation claim) | -| **Diagnose / Verify** | Diffusion stage/component-use readiness, retention and mutable request-scheduler aliases; client timeout/deletion versus worker/device completion | [audit-sglang-diffusion-lifetimes](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | Default ParallelExecutor and concrete rank/strategy must be traced; closing a use does not imply eviction or device cancellation | SGLang @ df13d5dce31a; bounded multimodal_gen source readset, separate from LLM/HiCache | 🟢 (candidate diagnosis; native/quality/performance not run) | -| **Diagnose / Verify** | Synchronous Ouro recurrent KV finalization: copying skipped deeper planes versus marking validity, complete-prefix attention reads, request-event ordering and retirement before page reuse | [audit-recurrent-kv-finalization](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) | Copy invocation and written-position bookkeeping are distinct; consumers and reclaimers follow the recorded-event wait contract | vLLM-RLT @ `4a350344027960a48384626aa646f1765fda6d4f`; [caller/consumer and retirement ledger](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md) | Source-derived candidate; CPU/CUDA/model/quality/performance/efficacy not_run; no host-capacity, Storage, Network or OS qualification | +| **Diagnose / Verify** | Serving model: operator/context dependencies, calibration/load domain, fidelity provenance and joint-SLO prediction uncertainty | [calibrate-serving-simulation](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/SKILL.md), `performance-ceiling-analysis` | Vidur near-capacity CPU-overhead counterexample; simulator output is not a bound. [Infera source reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/README.md) separates scalar tuner scoring from DES percentiles and retains missing-SLO/stale-document limits; no service guarantee | Vidur v2 and bounded Infera Git readset; Reference Matrix systems sources | 🟢 (prediction guidance; calibration not run) | +| **Diagnose / Verify** | Reusable-prefix compute work versus host/network KV loading cost | [performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md), [Mooncake condition](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md) | Check bytes, FLOPs and effective rate at one GPU-group scope; favorable Equation 2 still requires exposed-cost/SLO measurements | Mooncake FAST '25 §2.2, Table 1, Eqs. 1-2; separate public-paper source, not recovered foundation archive | 🟢 (analytical source condition; current feasibility and E2E benefit unverified; no SSD credit) | +| **Diagnose / Verify** | Offline offload feasibility across GPU, CPU RAM and disk | [performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md), [FlexGen condition](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/flexgen-offload-feasibility.md) | Joint weights/KV/activation placement; RAM resident plus staging/working peaks, SSD state capacity and disk↔CPU cost, effective batch versus GPU batch, exposed CPU/transfer cost and block-latency objective | FlexGen official ICML 2023, §§3–4.3 and selected A.3; separate public source, foundation archive still unavailable | Source-only candidate; empirical not_run, fitted/perfect-overlap model and relaxed OOM limits; no blanket RAM/SSD, Network/OS or current-engine qualification | +| **Diagnose / Configure / Optimize / Verify** | Serving service order versus HBM admission and suspended-KV recovery; replica migration versus elastic sequence groups | [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/SKILL.md), [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) | [Hypothetical pressure/migration cases](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md); cohort/fairness and destination-reservation costs | Orca proceedings, FastServe v3, Llumnix v1, LoongServe v2; implementation pins remain separate | 🟢 (conditional paper-derived plans) | +| **Diagnose / Optimize / Verify** | Training activation lifetime, rematerialization and reserved-address packing | [plan-activation-residency](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/SKILL.md) | Dependency repair and capacity feasibility checked separately; no recovery-checkpoint coverage | Checkmate v3, OLLA v2; [C readset](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md) | 🟢 (planning, not solver/runtime validation) | +| **Diagnose / Configure / Optimize / Verify** | Training stage/mesh/sharding feasibility and split F/B/W execution under total-HBM and optimizer-semantics constraints | [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/SKILL.md), [assess-split-backward-schedule](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-split-backward-schedule/SKILL.md) | Alpa conditional plans; Zero Bubble activation caps versus total memory; rollback versus stale updates | Alpa v3, Zero Bubble v1 | 🟢 (author evidence, no current stack validation) | +| **Diagnose / Optimize / Verify** | Attention IO, operation-class ceilings, output ownership, work supply, async dependencies and FP8 layout/accuracy | [diagnose-attention-bottlenecks](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/SKILL.md), `kernel-writing-guide` | [Author-reported attention cases](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md); compiler/register and precision counter-signals | FlashAttention v2 / FA2 v1 / FA3 v2 | 🟢 (Accelerator only; no Ascend implementation claim) | +| **Diagnose / Verify** | Diffusion stage/component-use readiness, retention and mutable request-scheduler aliases; client timeout/deletion versus worker/device completion | [audit-sglang-diffusion-lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | Default ParallelExecutor and concrete rank/strategy must be traced; closing a use does not imply eviction or device cancellation | SGLang @ df13d5dce31a; bounded multimodal_gen source readset, separate from LLM/HiCache | 🟢 (candidate diagnosis; native/quality/performance not run) | +| **Diagnose / Verify** | Synchronous Ouro recurrent KV finalization: copying skipped deeper planes versus marking validity, complete-prefix attention reads, request-event ordering and retirement before page reuse | [audit-recurrent-kv-finalization](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) | Copy invocation and written-position bookkeeping are distinct; consumers and reclaimers follow the recorded-event wait contract | vLLM-RLT @ `4a350344027960a48384626aa646f1765fda6d4f`; [caller/consumer and retirement ledger](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md) | Source-derived candidate; CPU/CUDA/model/quality/performance/efficacy not_run; no host-capacity, Storage, Network or OS qualification | **Second-level targets in this row:** Kernel, Operator, KV Cache, Attention, Diffusion denoising steps, VAE/Codec, Graph compilation, Serving/Runtime admission, device-HBM allocator lifecycle, Training activation residency and pipeline scheduling @@ -75,7 +75,7 @@ The primary compute and HBM resource. Most current work lives here. ### vLLM version and responsibility routing For the vLLM admission/KV rows above, the -[existing responsibility map](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/SKILL.md#choose-the-source-and-responsibility) +[existing responsibility map](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/SKILL.md#choose-the-source-and-responsibility) separates broad engine/configuration/extension guidance at `2521446ac8c92937b5a32b7dc2a84ef7fa8e99c5` from focused scheduler branches and connector obligations at `7f730b4cea9a66520a9200265928ebe016b48a39`. @@ -89,13 +89,13 @@ Host-side memory: CPU DRAM, pinned memory, page tables, PyTorch tensor lifecycle | Lifecycle step | What it covers | Candidate Skills | Cases / Experience | Evidence | Maturity | |---|---|---|---|---|---| -| **Diagnose / Configure / Verify** | Program/pipeline cache, GPU allocation adoption, exact host-prefix snapshots and pending-request idle release | [lithos-metal serving-cache](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/SKILL.md) | Allocation reuse is separate from sequence validity; host snapshots and OS file cache have distinct owners | Lithos Metal pinned serving/Session/PrefixCache/lifecycle paths | Source-derived candidate; full-model residency and continuation unverified | -| **Observe / Diagnose / Verify** | MiniMax-H3 TE/vision/input-output VAE phase lifetime, prepared DiT/decoder retention, shared/mapped weight and host media/output residency | [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md) | Preview/cache changes overlap; output tiling retains full RGB; checkpoint bytes and tracked tensors do not establish total RAM fit | h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; [phase ledger](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/phase-residency.md) | 🟢 (source-derived candidate; physical/compressed/swap peaks unmeasured) | -| **Diagnose / Configure / Verify** | Workflow-output signatures and object identity, ancestor/fingerprint validity versus retention, lazy consumer references and host/pinned state | [comfy-ui](../src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/SKILL.md) | A fixed JSON filename is not an external-state fingerprint; NOT_IDEMPOTENT node identity does not force every-run invalidation | ComfyUI @ b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c; [cache ledger](../src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/graph-cache-invalidation.md) | 🟢 (source-derived procedure; executor cache suite not run) | -| **Observe / Diagnose / Verify** | Native diffusion parameter storage, compute staging and retained runtime buffers across conditioning/denoise/VAE phases; loader read/conversion buffers and shared mapping owners | [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | Device fit does not establish sufficient host RAM; initial host preparation may precede async transfer; releasing one owner is not final unmap; track host/device peaks separately | stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; [validation](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/validation-boundaries.md), [loader/storage](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md) | 🟢 (source-derived ownership audit; host peaks unmeasured) | +| **Diagnose / Configure / Verify** | Program/pipeline cache, GPU allocation adoption, exact host-prefix snapshots and pending-request idle release | [lithos-metal serving-cache](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/SKILL.md) | Allocation reuse is separate from sequence validity; host snapshots and OS file cache have distinct owners | Lithos Metal pinned serving/Session/PrefixCache/lifecycle paths | Source-derived candidate; full-model residency and continuation unverified | +| **Observe / Diagnose / Verify** | MiniMax-H3 TE/vision/input-output VAE phase lifetime, prepared DiT/decoder retention, shared/mapped weight and host media/output residency | [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md) | Preview/cache changes overlap; output tiling retains full RGB; checkpoint bytes and tracked tensors do not establish total RAM fit | h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; [phase ledger](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/phase-residency.md) | 🟢 (source-derived candidate; physical/compressed/swap peaks unmeasured) | +| **Diagnose / Configure / Verify** | Workflow-output signatures and object identity, ancestor/fingerprint validity versus retention, lazy consumer references and host/pinned state | [comfy-ui](../src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/SKILL.md) | A fixed JSON filename is not an external-state fingerprint; NOT_IDEMPOTENT node identity does not force every-run invalidation | ComfyUI @ b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c; [cache ledger](../src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/graph-cache-invalidation.md) | 🟢 (source-derived procedure; executor cache suite not run) | +| **Observe / Diagnose / Verify** | Native diffusion parameter storage, compute staging and retained runtime buffers across conditioning/denoise/VAE phases; loader read/conversion buffers and shared mapping owners | [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | Device fit does not establish sufficient host RAM; initial host preparation may precede async transfer; releasing one owner is not final unmap; track host/device peaks separately | stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; [validation](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/validation-boundaries.md), [loader/storage](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md) | 🟢 (source-derived ownership audit; host peaks unmeasured) | | **Observe / Profile** | Host overhead in nsys: GPU idle ratio (>0.30 aggregate/>0.15 generation), host-prep exposed ratio (M3b>0.05), NVTX breakdown, inter-kernel gaps; cgroup memory pressure (PSI) and event counters | `host-overhead-profiling`, `cgroup-resource-controls` (pressure/events observation) | GPU idle + host-prep verdict metrics; NVTX root-cause breakdown; regression comparison across two traces | TRT-LLM perf-host-analysis @ f33e0e5f; kernel cgroup-v2 docs (pack-mediated) | 🟢 | | **Diagnose** | YES/NO bottleneck verdict; multi-rank straggler detection; NVTX-to-source function mapping | `host-overhead-profiling` | M3b>0.05 AND M3c>0.50 together confirm host-prep bottleneck | TRT-LLM perf-host-analysis @ f33e0e5f | 🟢 | -| **Configure / Verify** | Copy-stream ordering, pinned CPU mutation/completion and host/device allocation lifetimes; version-aware GIL detection; conditional NUMA preflight; versioned Ascend native descriptor-pool ownership | `host-memory-management` | Separate producer/consumer ordering, application reuse and allocator cache reuse; no universal tensor-size or pinned-memory performance threshold. Ascend normal head-event query gates descriptor reuse; waiting alone does not recycle | PyTorch v2.8.0 @ ba56102387ef21a3b04b357e5b183d48f0afefc7 ([readset](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-host-copy-readset.json)); CPython v3.13.0 documentation; legacy NUMA guidance; vllm-ascend @ 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf | Candidate source procedure; CUDA/NPU performance, NUMA and free-threaded stack compatibility not_run | +| **Configure / Verify** | Copy-stream ordering, pinned CPU mutation/completion and host/device allocation lifetimes; version-aware GIL detection; conditional NUMA preflight; versioned Ascend native descriptor-pool ownership | `host-memory-management` | Separate producer/consumer ordering, application reuse and allocator cache reuse; no universal tensor-size or pinned-memory performance threshold. Ascend normal head-event query gates descriptor reuse; waiting alone does not recycle | PyTorch v2.8.0 @ ba56102387ef21a3b04b357e5b183d48f0afefc7 ([readset](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/pytorch-host-copy-readset.json)); CPython v3.13.0 documentation; legacy NUMA guidance; vllm-ascend @ 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf | Candidate source procedure; CUDA/NPU performance, NUMA and free-threaded stack compatibility not_run | | **Configure** | KV host-tier sizing and multi-L1 composition; cgroup memory protection and limits (`memory.high/low/max`) | `kv-tier-management`, `cgroup-resource-controls` | Retain the respective tier-capacity and OS memory-control contracts; these settings do not establish transfer completion | LMCache @ c3b875a; kernel cgroup-v2 docs (pack-mediated) | Existing candidate guidance; no new source read or validation | | **Optimize** | line_profiler loop targeting `_prepare_tp_inputs`; NVTX regression triage to alternative targets (`_fetch_new_requests` +643%, `broadcast_requests`, `_update_requests` +75%); runtime KV tier management (store/retain/prefetch/evict policies, concurrency-vs-RAM admission dial) | `host-overhead-optimization`, `kv-tier-management` | Iterative profile→identify→optimize→verify loop; RAM-release lever ranking | TRT-LLM perf-host-optimization @ f33e0e5f; LMCache MP mode @ c3b875a | 🟢 | | **Verify** | Memory-event verification: matched baseline/outcome windows, PSI + event deltas, configuration provenance, rollback contract | `dense-performance-evidence` (evidence levels apply), `cgroup-resource-controls` (verification contract) | Application outcomes + sibling pressure tracked together; raw counter deltas retained | kernel cgroup-v2 docs (pack-mediated, wave 6) | 🟢 (proposed procedure, unvalidated) | @@ -104,10 +104,10 @@ Host-side memory: CPU DRAM, pinned memory, page tables, PyTorch tensor lifecycle | **Observe / Diagnose / Configure / Optimize** | Input queue/cached-shard working-set budgets; offload CPU/pinned-memory costs and locality | `training-input-pipeline`, `training-resource-budget` | Per-worker samples differ from bytes; host budget includes input/checkpoint/offload together | Reference B1/B2/B3, CR1/CR6/CR5 | 🟢 (candidate decisions, no measurements) | | **Configure / Verify** | Shared-memory snapshot readiness, positive-step agreement, replica/staging lifetime | `checkpoint-recovery-consistency` | Worker survival differs from host survival; state completeness remains separate | Reference C3/C4 | 🟢 (source contracts; recovery untested) | | **Diagnose / Configure / Optimize / Verify** | TorchNPU pinned-host ownership, pending-copy reuse, host-specific byte accounting and default/expandable cache controls | `torchnpu-pinned-host-memory-lifecycle` | External registration ownership and ATen internals unresolved; per-block threshold is not a total-cache cap; version-warning conflict retained | TorchNPU @ 67511c53cd4c, Dot bundle; host allocator/copy/config ledger | 🟢 (conditional procedure; no runtime validation) | -| **Diagnose / Configure / Optimize / Verify** | Suspended KV host residency, predicted recovery and migration staging under actual host capacity/bandwidth | [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/SKILL.md), [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | Running-batch bytes exclude other unfinished state; device space is also required for restoration | FastServe v3, Llumnix v1; hypothetical cases, not observed failures | 🟢 (conditional host mechanism) | -| **Diagnose / Optimize / Verify** | CPU optimizer state/compute ownership and tiled update/return cost | [training-resource-budget](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/SKILL.md) | [ZeRO-Offload delta](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/knowledge/offload-delta.md); staleness requires a separate semantic decision | ZeRO-Offload v1, distinct from the existing DeepSpeed config pin | 🟢 (author evidence, unmeasured locally) | -| **Diagnose / Optimize / Verify** | Host feature materialization, invariant reuse and assembly lifetime | [host-overhead-optimization](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md) | [Cross-layer procedure](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md); local gain requires full-stack SLO verification | FlashVector v1, same source identity as Reference Matrix §2 | 🟢 (scoped host mechanism, not service gain) | -| **Diagnose / Verify** | Host-side request/job records, scheduler-object sharing and output-finalizer task lifetime | [audit-sglang-diffusion-lifetimes](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | Req field is not proof of object isolation; socket close and store deletion do not establish worker termination; device HBM excluded | SGLang @ df13d5dce31a; same bounded source identity | 🟢 (source-derived candidate, no measured memory saving) | +| **Diagnose / Configure / Optimize / Verify** | Suspended KV host residency, predicted recovery and migration staging under actual host capacity/bandwidth | [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/SKILL.md), [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) | Running-batch bytes exclude other unfinished state; device space is also required for restoration | FastServe v3, Llumnix v1; hypothetical cases, not observed failures | 🟢 (conditional host mechanism) | +| **Diagnose / Optimize / Verify** | CPU optimizer state/compute ownership and tiled update/return cost | [training-resource-budget](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/SKILL.md) | [ZeRO-Offload delta](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/knowledge/offload-delta.md); staleness requires a separate semantic decision | ZeRO-Offload v1, distinct from the existing DeepSpeed config pin | 🟢 (author evidence, unmeasured locally) | +| **Diagnose / Optimize / Verify** | Host feature materialization, invariant reuse and assembly lifetime | [host-overhead-optimization](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/SKILL.md) | [Cross-layer procedure](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md); local gain requires full-stack SLO verification | FlashVector v1, same source identity as Reference Matrix §2 | 🟢 (scoped host mechanism, not service gain) | +| **Diagnose / Verify** | Host-side request/job records, scheduler-object sharing and output-finalizer task lifetime | [audit-sglang-diffusion-lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | Req field is not proof of object isolation; socket close and store deletion do not establish worker termination; device HBM excluded | SGLang @ df13d5dce31a; same bounded source identity | 🟢 (source-derived candidate, no measured memory saving) | | **Diagnose / Verify** | Host-owned reader/task/result and slot prompt-token lifecycle under cancellation | `audit-llama-runtime-state` | Waiter removal, queued-work cleanup and slot release are distinct; stateful prompts may remain; no host byte-saving claim | llama.cpp @ 707616feb5f3; server queue/context source and cancellation test assertion | 🟢 (candidate lifecycle diagnosis; native server case not run) | **Second-level targets:** PyTorch tensor lifecycle, request scheduling, KV cache CPU offload, TorchNPU pinned-host allocator lifecycle, CPU optimizer ownership and host feature assembly @@ -120,16 +120,16 @@ Storage I/O: model weight loading, checkpoint save/restore, profiling artifact I | Lifecycle step | What it covers | Candidate Skills | Cases / Experience | Evidence | Maturity | |---|---|---|---|---|---| -| **Diagnose / Verify** | Native diffusion catalog/source identity → non-ZIP safetensors byte reads → host staging or retained mapping → immediate ModelManager completion/cleanup | [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | Stamps/internal revisions are not content hashes; valid metadata or a destination callback is not read completion; failure cleanup covers this load's newly created blocks | stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; [loader/storage boundaries](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md) | 🟢 (bounded source-derived candidate; other formats, native I/O and performance unqualified) | -| **Diagnose / Configure / Verify** | Prepared-pack structural identity and exact-prefix disk blobs, hashes, quota and bounded flush | [lithos-metal serving-cache](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/SKILL.md) | Weight payload size/mtime is not byte authentication; queued writes are not persistence; corrupt/stale entries prefill | Lithos Metal pinned pack/PrefixStore paths | Source-derived candidate; durability/performance unverified | -| **Diagnose / Configure / Verify** | Raw tensor-schema compatibility, file-backed DiT versus two-slot BF16 SSD streaming, Qwen read-worker/ring-depth distinction and submit/join completion | [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md) | SSD mode disables runtime int8; compare matching BF16 paths. F_NOCACHE request and zero join wait do not prove bypass or free I/O | h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; [I/O tradeoffs](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md#three-different-ioresidency-paths) | 🟢 (source-derived candidate; native I/O/overlap/performance not_run) | +| **Diagnose / Verify** | Native diffusion catalog/source identity → non-ZIP safetensors byte reads → host staging or retained mapping → immediate ModelManager completion/cleanup | [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | Stamps/internal revisions are not content hashes; valid metadata or a destination callback is not read completion; failure cleanup covers this load's newly created blocks | stable-diffusion.cpp @ b13a48219d5cdf408edcb93da6cd69b612ed03d3; [loader/storage boundaries](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md) | 🟢 (bounded source-derived candidate; other formats, native I/O and performance unqualified) | +| **Diagnose / Configure / Verify** | Prepared-pack structural identity and exact-prefix disk blobs, hashes, quota and bounded flush | [lithos-metal serving-cache](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/SKILL.md) | Weight payload size/mtime is not byte authentication; queued writes are not persistence; corrupt/stale entries prefill | Lithos Metal pinned pack/PrefixStore paths | Source-derived candidate; durability/performance unverified | +| **Diagnose / Configure / Verify** | Raw tensor-schema compatibility, file-backed DiT versus two-slot BF16 SSD streaming, Qwen read-worker/ring-depth distinction and submit/join completion | [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md) | SSD mode disables runtime int8; compare matching BF16 paths. F_NOCACHE request and zero join wait do not prove bypass or free I/O | h3.c @ 8974cc055ea9c02fcd14cc27dfda3e1027c05153; [I/O tradeoffs](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md#three-different-ioresidency-paths) | 🟢 (source-derived candidate; native I/O/overlap/performance not_run) | | **Observe / Profile** | Workload-matched block-device measurement (fio methodology: signature recording, requested-vs-achieved depth, cache-state ledger, run-shape classification), per-device attribution (iostat), per-process attribution (pidstat) | `storage-io-measurement`, `weight-loading-optimization` (§7 phase timers) | Bounded observation templates; unit/latency-provenance schema; logical-vs-physical I/O distinction | fio @ acb7a302 + sysstat @ efd0c445 (pack-mediated, wave 6); fastsafetensors perf.py @ 219e256; mindie-agent docs @ 6a81b41 | 🟢 (methodology only — running benchmarks or manipulating caches requires separate authorization) | | **Diagnose** | Slow startup attribution: distinguish graph compilation/capture from weight I/O using phase traces | `ascend-graph-mode`, `ascend-model-adaptation` (dummy-to-real sequence) | Graph source identifies capture/compile work but does not rule out concurrent or preceding storage bottlenecks | TorchAir @ e1118e8b0fc1b95190ab53b732c0872989550e20; historical MindIE-SD/vllm-ascend guidance | Incidental diagnosis only; no added storage implementation or measured startup benefit | | **Diagnose** | Device saturation and attribution: workload matching, achieved-depth validation, %util/%iowait saturation guards, per-process I/O vs CPU-scheduling-pressure separation | `storage-io-measurement` | No universal percent threshold; corroboration required before naming a storage bottleneck | fio/sysstat pack (wave 6) | 🟢 | | **Diagnose** | Load-path judgment (source/format/placement/duplication) and startup fixed-overhead accounting (bytes+counts lines, hit-criteria warmup validation) | `weight-loading-optimization` | Four-dimension load-path judgment; fixed-vs-per-request split; 7 accounting errors | MindIE-SD host-opt + host-overhead-account @ 8481d39b | 🟢 | | **Configure** | Checkpoint format for fast loading (FP8 paired tensors, shard loading order) | `ascend-model-adaptation` | FP8 paired-tensor loading pattern | vllm-ascend @ bc8115ed | 🟡 | | **Optimize** | Serving-level load path: device-side weight prefetch (switch-gated, 50%-of-theoretical start), CPU-tier offload ordering by change surface, duplication removal | `weight-loading-optimization` | `npu_prefetch` decision rules; OOM fallback tiers; construction vs inference OOM | cannbot model-infer-prefetch + diffusers memory-optimization @ 75ec89c; MindIE-SD resource-fallback-tiers @ 8481d39b | 🟢 (serving level) | -| **Optimize** | Disk-level loading mechanics: mmap/mlock/dio/lazy loading modes, pinned bounce-buffer parallel reads, GDS direct-to-device, byte-range/shard-parallel loading, incremental mlock growth | [weight-loading-optimization](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md) (§6-7); [llama.cpp owner map](../src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) | Mode trade-off table; NUMA-disables-prefetch coupling; block-size formula; readiness overlap; static fit planner | ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT; static inspection 2026-10-22 carried); foundation-model-stack/fastsafetensors @ 219e256 (Apache-2.0) | 🟢 (wave 5; CUDA/ROCm/POSIX-bound — no Ascend path in these sources) | +| **Optimize** | Disk-level loading mechanics: mmap/mlock/dio/lazy loading modes, pinned bounce-buffer parallel reads, GDS direct-to-device, byte-range/shard-parallel loading, incremental mlock growth | [weight-loading-optimization](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md) (§6-7); [llama.cpp owner map](../src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) | Mode trade-off table; NUMA-disables-prefetch coupling; block-size formula; readiness overlap; static fit planner | ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT; static inspection 2026-10-22 carried); foundation-model-stack/fastsafetensors @ 219e256 (Apache-2.0) | 🟢 (wave 5; CUDA/ROCm/POSIX-bound — no Ascend path in these sources) | | **Optimize** | Runtime KV-to-disk tier: L2 local adapters (fs/nixl/raw_block/dax), persist/recover, per-adapter serde capacity lever | `kv-tier-management` | Adapter selection table; persist/recover semantics | LMCache MP mode @ c3b875a (Apache-2.0) | 🟢 | | **Optimize** | Checkpoint save/restore: synchronous sharded writes, per-file atomic publish, opt-in fsync, reload via existing loader; whole-set atomicity / async / distributed / crash durability explicitly unsupported | `weight-loading-optimization` (§8) | Supported-vs-unknown table; mixed-set rename-failure risk as caller obligation | fastsafetensors saver.py @ 219e256 | 🟡 (fastsafetensors-specific semantics extracted; its save-performance optimization remains unsourced — no perf harness or documented tuning levers) | | **Verify** | Load-path measurement protocol (phase timers, fadvise cold-cache, baseline comparison); storage-side verification: unit/latency-provenance-aware schema, two-layer evidence gate (device metric + application outcome) | `weight-loading-optimization` (§7), `storage-io-measurement` (prime rule, §5) | Author-reported numbers labeled as such; device metrics never merged with application outcomes | fastsafetensors perf.py @ 219e256; fio/sysstat pack (wave 6) | 🟢 (methodology; no empirical validation) | @@ -139,7 +139,7 @@ Storage I/O: model weight loading, checkpoint save/restore, profiling artifact I | **Verify** | Observe final upload errors and step-time impact separately from backend durability and distributed recovery validity | `dcp-async-save-performance` | Async completion is not crash-durability evidence | PyTorch DCP save/stager/thread-writer paths @ 7d239835e773 | 🟢 (verification procedure only) | | **Diagnose / Configure / Optimize** | Dataset shard residency, compressed/raw admission, cache churn, FFCV mmap/process cache and traversal order | `training-input-pipeline` | Mosaic minimum-cache implementation gate; distributed QUASI_RANDOM excluded | Reference B2/B3 | 🟢 (sample correctness and throughput untested) | | **Configure / Optimize** | DeepSpeed NVMe parameter/optimizer buffer pools and tile read/compute/write overlap intent | `training-resource-budget` | Parameter and optimizer budgets distinct; buffer_size unit/dispatch not verified | Reference CR1 | 🟢 (configuration evidence only) | -| **Configure / Verify** | Complete-generation selection; NVIDIA dirty-to-final publication; DLRover queued/shard/done/tracker boundaries | `checkpoint-recovery-consistency` | DLRover common commit counts done files; payload serialization and POSIX synchronization require separate durability evidence. Rank coverage does not prove state completeness | Reference C3/C4; [DLRover persistence](../src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md) | 🟢 (source-derived recovery procedure; recovery/durability untested) | +| **Configure / Verify** | Complete-generation selection; NVIDIA dirty-to-final publication; DLRover queued/shard/done/tracker boundaries | `checkpoint-recovery-consistency` | DLRover common commit counts done files; payload serialization and POSIX synchronization require separate durability evidence. Rank coverage does not prove state completeness | Reference C3/C4; [DLRover persistence](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md) | 🟢 (source-derived recovery procedure; recovery/durability untested) | **Second-level targets:** Model weight sharding, checkpoint I/O, profiling output archival @@ -157,14 +157,14 @@ Inter-device and inter-node communication: HCCL (Ascend), NCCL (CUDA), InfiniBan | **Configure** | Collective-in-graph capture (HCCL graph mode), comm-stream masking; masking formula `1-1/n` when F≥C | `ascend-distributed-parallel` | Masking upper bound derivation; AlltoAllV defect workaround | MindIE-SD @ 8481d39b | 🟢 | | **Optimize** | USP activation requirement (ParallelConfig wrapping), strategy selection, AlltoAllV workaround; evidence-gated compute/communication overlap of independently-ready work | `ascend-distributed-parallel`, `cann-attribution-overlap` | USP must set ParallelConfig; verify allToAll appears not just allGather; overlap preserves ordering and joins, no comm recommendation without comm evidence | MindIE-SD dit-parallel-opt @ 8481d39b (9 ref files); cann/oam-tools @ 8440935f | 🟢 | | **Verify** | Masking ratio measurement, before/after comm-fraction comparison; strategy activation attribution | `ascend-distributed-parallel` (attribution method), `ascend-performance-analysis`, `dense-performance-evidence`, `cann-attribution-overlap` (overlap acceptance) | Step-trace four-tuple as ground truth; scope-effectiveness-check.md | MindIE-SD perf-gate + dit-parallel-opt; cann/oam-tools @ 8440935f | 🟢 | -| **Observe / Diagnose / Verify** | Native HCCL-test versus PyTorch/HCCL wrapper selection; per-operation tensor-byte meaning, completed-call timing, rank-local reports and actual oracle coverage | `ascend-distributed-parallel` ([test acceptance](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md)) | Unchecked operations can print PASS; subgroup tags and P2P peer schedules need separate references; async timing does not prove overlap | awesome-ascend-skills @ `75ec89ce9d401be8ac95980e1c3f42f8d4315057`, wrapper code and source docs | 🟢 (candidate evidence procedure; native collective/transport, backend and agent efficacy not_run; host-KV deferred, no RAM increment) | -| **Diagnose / Configure / Verify** | One native multi-rank OPBASE AICPU mesh AllReduce path: actual algorithm/engine, rank-scaled scratch chunks, peer writes, joins and local reduction | [ascend-distributed-parallel](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md) | [Native source map](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce.md): cost-selector versus size-heuristic routes; final selection after fallback; simulator semantics versus completed native numerical checks | Official cann/hccl @ 96fab0a2acb761e7f8ab0f94aeb5138f200a345b, selected implementation/test definitions | Candidate source procedure only; HCOMM/ACL transport, other devices/engines/collectives, native numerical and performance validation not_run | +| **Observe / Diagnose / Verify** | Native HCCL-test versus PyTorch/HCCL wrapper selection; per-operation tensor-byte meaning, completed-call timing, rank-local reports and actual oracle coverage | `ascend-distributed-parallel` ([test acceptance](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md)) | Unchecked operations can print PASS; subgroup tags and P2P peer schedules need separate references; async timing does not prove overlap | awesome-ascend-skills @ `75ec89ce9d401be8ac95980e1c3f42f8d4315057`, wrapper code and source docs | 🟢 (candidate evidence procedure; native collective/transport, backend and agent efficacy not_run; host-KV deferred, no RAM increment) | +| **Diagnose / Configure / Verify** | One native multi-rank OPBASE AICPU mesh AllReduce path: actual algorithm/engine, rank-scaled scratch chunks, peer writes, joins and local reduction | [ascend-distributed-parallel](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/SKILL.md) | [Native source map](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-allreduce.md): cost-selector versus size-heuristic routes; final selection after fallback; simulator semantics versus completed native numerical checks | Official cann/hccl @ 96fab0a2acb761e7f8ab0f94aeb5138f200a345b, selected implementation/test definitions | Candidate source procedure only; HCOMM/ACL transport, other devices/engines/collectives, native numerical and performance validation not_run | | **Configure / Optimize** | Remote dataset shard download look-ahead versus cache churn | `training-input-pipeline` | Network bytes/readiness are distinct from local-cache footprint | Reference B2 | 🟢 (candidate mechanism; no network measurements) | -| **Configure / Optimize / Verify** | File-owner whole-tensor broadcast versus TP scatter; adjacent whole-tensor broadcast coalescing | `weight-loading-optimization` (§9) | Group-rank order, coalescing eligibility, source readiness, run-storage retention and early close; local clones/H2D excluded | fastsafetensors @ 219e256, [full-pin delivery readset](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/fastsafetensors-transfer-readset.md) | 🟢 (source-derived candidate procedure; no runtime, transport, NPU or gain qualification) | -| **Diagnose / Configure / Verify** | Saver/loader agreement, complete-iteration coverage and DLRover full/sharded Gloo byte/metadata transport | `checkpoint-recovery-consistency` | Same-host shared memory is excluded; replica group/placement and restore consumer need payload checks. CPU/Gloo source is not HCCL/NPU qualification | Reference C3/C4; [DLRover replica boundaries](../src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md#replica-transport-and-its-consumer) | 🟢 (source-derived transport procedure; distributed restore untested) | +| **Configure / Optimize / Verify** | File-owner whole-tensor broadcast versus TP scatter; adjacent whole-tensor broadcast coalescing | `weight-loading-optimization` (§9) | Group-rank order, coalescing eligibility, source readiness, run-storage retention and early close; local clones/H2D excluded | fastsafetensors @ 219e256, [full-pin delivery readset](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/fastsafetensors-transfer-readset.md) | 🟢 (source-derived candidate procedure; no runtime, transport, NPU or gain qualification) | +| **Diagnose / Configure / Verify** | Saver/loader agreement, complete-iteration coverage and DLRover full/sharded Gloo byte/metadata transport | `checkpoint-recovery-consistency` | Same-host shared memory is excluded; replica group/placement and restore consumer need payload checks. CPU/Gloo source is not HCCL/NPU qualification | Reference C3/C4; [DLRover replica boundaries](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md#replica-transport-and-its-consumer) | 🟢 (source-derived transport procedure; distributed restore untested) | | **Observe / Diagnose / Verify** | Collective kind/payload/ranks, algbw versus busbw, rank-side GC stalls versus fabric stalls | `training-resource-budget` (attributed expert note) | No network-script execution or universal bandwidth/gain claim | Reference CR6 | 🟢 (expert methodology) | -| **Diagnose / Configure / Optimize / Verify** | Cross-node request-state pre-copy/catch-up, migration versus ongoing sequence-attention traffic | [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | Total bytes, duration and final downtime differ; charge congestion and protected-class regressions | Llumnix v1, LoongServe v2; actual inter-node placement required | 🟢 (paper decision contract; transport unvalidated) | -| **Diagnose / Configure / Optimize / Verify** | Cross-mesh shard intersections, required slow-link sends and local allgather replacement cost | [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | Separate inter-node traffic from local accelerator collectives; include staging and per-rank memory | Alpa v3 AL2; no current compiler or transport qualification | 🟢 (conditional communication mechanism) | +| **Diagnose / Configure / Optimize / Verify** | Cross-node request-state pre-copy/catch-up, migration versus ongoing sequence-attention traffic | [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) | Total bytes, duration and final downtime differ; charge congestion and protected-class regressions | Llumnix v1, LoongServe v2; actual inter-node placement required | 🟢 (paper decision contract; transport unvalidated) | +| **Diagnose / Configure / Optimize / Verify** | Cross-mesh shard intersections, required slow-link sends and local allgather replacement cost | [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/SKILL.md) | Separate inter-node traffic from local accelerator collectives; include staging and per-rank memory | Alpa v3 AL2; no current compiler or transport qualification | 🟢 (conditional communication mechanism) | Host/device transfers and same-host IPC are excluded from this row. Network coverage above comes only from remote downloads, distributed agreement/replicas @@ -186,11 +186,11 @@ System-level: scheduler, DVFS, process isolation, environment variables, driver | **Diagnose** | SIGKILL residual state: per-step time uniformly elevated by ~1 order of magnitude | `ascend-device-health` | npu-smi Health=OK does not prove card is clean; 4-step smoke required | MindIE-SD dit-parallel-opt @ 8481d39b | 🟢 | | **Configure** | Card-group selection, env variables (`HCCL_OP_EXPANSION_MODE=AIV`), proc-mem occupancy check; cgroup quota/weight/cpuset configuration with hierarchy/delegation/permission preflight | `ascend-device-health`, `cgroup-resource-controls` | Card-group change = contract change; all baselines must be re-measured on new group; identify the authorized owner before proposing changes | MindIE-SD dit-parallel-opt + perf-gate @ 8481d39b; kernel cgroup-v2 docs (pack-mediated) | 🟢 | | **Configure** | Env-preflight: verify device availability, driver health, Python env before measurement | `performance-ceiling-analysis` (env-preflight references TRT-LLM exec-env-check) | TRT-LLM exec-env-check pattern | TRT-LLM @ f33e0e5f | 🟡 | -| **Optimize** | CPU affinity check-then-pin (accelerator-local NUMA node, cross-round consistency), NUMA placement mode semantics (distribute/isolate/numactl) with page-cache-drop and numa_balancing rules, generation-vs-batch thread/affinity splits, thread-count env discipline, device-busy preflight with graceful-to-force PID escalation, structured ERROR/WARNING env checks, container flags and ulimits; cgroup resource reallocation as capacity spend (quota/weight/headroom reviews gated on observed stalls) | [os-process-placement](../src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/SKILL.md), `cgroup-resource-controls`; [llama.cpp owner map](../src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) | OMP_NUM_THREADS/PROC_BIND per scenario; `CPU_AFFINITY_CONF`; npu-smi usages preflight; `ASCEND_HOME_PATH` named-var correctness; rollback manifest before any change | TRT-LLM perf-host-optimization @ f33e0e5f; awesome-ascend-skills @ 75ec89c; ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT; static inspection 2026-10-22 carried); kernel cgroup-v2 docs (pack-mediated); historical vllm-ascend Dockerfile/thread-env shipping defaults @ bc8115ed (Reference Matrix); ACLGraph @ 8a8e1e05ecb6 adds no OS-control evidence | 🟢 (isolcpus and scheduler priorities nice/ionice/SCHED_FIFO/RR remain unsourced) | +| **Optimize** | CPU affinity check-then-pin (accelerator-local NUMA node, cross-round consistency), NUMA placement mode semantics (distribute/isolate/numactl) with page-cache-drop and numa_balancing rules, generation-vs-batch thread/affinity splits, thread-count env discipline, device-busy preflight with graceful-to-force PID escalation, structured ERROR/WARNING env checks, container flags and ulimits; cgroup resource reallocation as capacity spend (quota/weight/headroom reviews gated on observed stalls) | [os-process-placement](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/os-process-placement/SKILL.md), `cgroup-resource-controls`; [llama.cpp owner map](../src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) | OMP_NUM_THREADS/PROC_BIND per scenario; `CPU_AFFINITY_CONF`; npu-smi usages preflight; `ASCEND_HOME_PATH` named-var correctness; rollback manifest before any change | TRT-LLM perf-host-optimization @ f33e0e5f; awesome-ascend-skills @ 75ec89c; ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT; static inspection 2026-10-22 carried); kernel cgroup-v2 docs (pack-mediated); historical vllm-ascend Dockerfile/thread-env shipping defaults @ bc8115ed (Reference Matrix); ACLGraph @ 8a8e1e05ecb6 adds no OS-control evidence | 🟢 (isolcpus and scheduler priorities nice/ionice/SCHED_FIFO/RR remain unsourced) | | **Verify** | Card health check sequence: npu-smi Health, 4-step smoke verify, proc-mem occupancy; cgroup change verification (matched windows, application metrics, counter deltas, provenance, rollback) | `ascend-device-health`, `cgroup-resource-controls` | Health=OK ≠ card usable; SIGKILL residual state only detectable via smoke run | MindIE-SD @ 8481d39b; kernel cgroup-v2 docs (pack-mediated) | 🟢 | | **Diagnose / Configure** | CPU optimizer share, NUMA locality and per-rank garbage-collection stalls; training/helper lifetime in recovery | `training-resource-budget`, `checkpoint-recovery-consistency` | Gate CPUAdam cores to stage-3 super_offload; fallback saver lifetime matters; historical MindSpeed 910A/910B binding requires actual device/NUMA/CPU mapping beyond tool presence | Reference CR1/CR6, C3; CR5 pinned affinity source | 🟢 (source guidance; no scheduling or privileged changes) | | **Configure / Optimize** | FFCV OS-managed versus process-managed dataset caching | `training-input-pipeline` | Cache ownership is independent of nice/ionice/realtime scheduling | Reference B3 | 🟢 (configuration decision, not runtime result) | -| **Diagnose / Configure / Verify** | Scoped RTK learning transition: manager discovery versus consumer PATH resolution; unknown-path and binary-identity limits; platform-specific repair, diagnostic effects and process-restart boundary | [repository-evolution-analysis](../src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/SKILL.md) | Manager discovery is distinct from consumer command resolution; repair and process-environment refresh have separate boundaries | Magpie @ `9848a9062a65c49e074c09770dc7de73dd012bea`, parent `4b5d6d725fae3d8a263f6e43c707400e3765b84f`; [RTK transition](../src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/references/rtk-consumer-path-transition.md) | Source-derived conditional lesson only; upstream tests, consumer execution and platform repair not_run. No scheduler/cgroup/performance or skill-efficacy coverage | +| **Diagnose / Configure / Verify** | Scoped RTK learning transition: manager discovery versus consumer PATH resolution; unknown-path and binary-identity limits; platform-specific repair, diagnostic effects and process-restart boundary | [repository-evolution-analysis](../src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/SKILL.md) | Manager discovery is distinct from consumer command resolution; repair and process-environment refresh have separate boundaries | Magpie @ `9848a9062a65c49e074c09770dc7de73dd012bea`, parent `4b5d6d725fae3d8a263f6e43c707400e3765b84f`; [RTK transition](../src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/references/rtk-consumer-path-transition.md) | Source-derived conditional lesson only; upstream tests, consumer execution and platform repair not_run. No scheduler/cgroup/performance or skill-efficacy coverage | **Second-level targets:** NPU driver, CANN runtime, container/process isolation @@ -201,15 +201,15 @@ System-level: scheduler, DVFS, process isolation, environment variables, driver | Gap | Row | Priority | Next action | |-----|-----|----------|-------------| | Further native hardware-source coverage | Accelerator / Communication | Later source expansion | Native NCCL remains unmined. Existing CUTLASS/DeepGEMM/FlashInfer slices do not cover broad dispatch. Investigate UCX when a multi-node workload requires it, and ROCm when an AMD target exists; none is newly covered by this delivery. | -| Earlier six-paper foundation archive unavailable | Uncredited original-archive mechanisms; original synthesis and standalone paper candidates | Input blocked for original-archive import | Supply the approved archive/readsets before importing its ZeRO-Infinity, FlexGen, Sarathi-Serve, DistServe, PagedAttention or Mooncake candidates; do not substitute config/discovery evidence. Separately, the fresh official-edition [DistServe phase-placement reference](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/README.md) is retained under the existing search-policy owner as candidate / not_run source knowledge; it does not recover that archive or qualify runtime, device or service behavior. The separately sourced Mooncake analytical condition above does not recover the archive; novelty against the missing synthesis remains UNKNOWN. | +| Earlier six-paper foundation archive unavailable | Uncredited original-archive mechanisms; original synthesis and standalone paper candidates | Input blocked for original-archive import | Supply the approved archive/readsets before importing its ZeRO-Infinity, FlexGen, Sarathi-Serve, DistServe, PagedAttention or Mooncake candidates; do not substitute config/discovery evidence. Separately, the fresh official-edition [DistServe phase-placement reference](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/distserve-phase-placement/README.md) is retained under the existing search-policy owner as candidate / not_run source knowledge; it does not recover that archive or qualify runtime, device or service behavior. The separately sourced Mooncake analytical condition above does not recover the archive; novelty against the missing synthesis remains UNKNOWN. | | Reviewed A–F qualification | Scoped Accelerator / Host / Network rows above | Separate future task | Candidates and author evidence do not establish runtime compatibility, measured gains or NPU support; no reproduction authorized | | Save-path performance qualification (checkpoint) | SSD / Optimize + Verify | Low | DCP asynchronous staging, pinned-buffer reuse and completion gates are now sourced in `dcp-async-save-performance`; no local gains, NPU compatibility or backend durability verified. fastsafetensors-specific performance remains unsourced; no repeat mining authorized | | Wave-7 input pipeline qualification | Accelerator / RAM / SSD / Network | Low | Mode-specific DALI behavior, FFCV current toolchain/distributed limits, Mosaic cache-churn and sample correctness remain unmeasured; no hardware run approved | | Distributed recovery qualification | RAM / SSD / Network / Verify | Low | C3/C4 contracts extracted; replica transport, whole-state completeness, world-size changes and actual failure recovery remain unvalidated | | Cross-resource applicability and historical evidence | Accelerator / RAM / OS | Low | DeepSpeed buffer_size units/dispatch outside inspected config; MindSpeed historical pin/default license and selected allocator/wrapper/affinity paths resolved, while full search/cache behavior, current compatibility and measured benefit remain open; CR6 is expert methodology, not measured gains | | OS/System isolcpus and scheduler priorities (nice / ionice / SCHED_FIFO / RR) | OS / Optimize | Low | cgroup v2 controls are now sourced (`cgroup-resource-controls`); the scheduler-priority class remains without a source and deferred | -| vLLM-RLT recurrent-depth KV finalization | Accelerator / KV correctness / Verify | Source-guidance candidate; native/efficacy not_run | [One operating candidate](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) traces synchronous host-KVMetadata exit/copy/validity → later depth-specific attention → normal release/wait/free. LAST_EXITED planes, SHARED single plane, bookkeeping and CUDA completion remain distinct; DeviceKVMetadata/async/speculation/capacity/transport excluded. [Intake](reviews/issue91-vllm-rlt-intake.md) retained | -| AFD P2pNccl completion, graph and transport qualification | Accelerator / Communication / Verify | Candidate audit delivered; native qualification gap remains | [audit-afd-p2p-exchanges](../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) covers rank and peer/stage ownership at its pin. The `afd_plugin` runtime descriptor remains unverified; underlying completion, DBO/graph and physical/multi-host transport are unqualified. Historical [intake](reviews/issue91-afd-plugin-intake.md) retained; no new Network coverage | +| vLLM-RLT recurrent-depth KV finalization | Accelerator / KV correctness / Verify | Source-guidance candidate; native/efficacy not_run | [One operating candidate](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) traces synchronous host-KVMetadata exit/copy/validity → later depth-specific attention → normal release/wait/free. LAST_EXITED planes, SHARED single plane, bookkeeping and CUDA completion remain distinct; DeviceKVMetadata/async/speculation/capacity/transport excluded. [Intake](reviews/issue91-vllm-rlt-intake.md) retained | +| AFD P2pNccl completion, graph and transport qualification | Accelerator / Communication / Verify | Candidate audit delivered; native qualification gap remains | [audit-afd-p2p-exchanges](../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) covers rank and peer/stage ownership at its pin. The `afd_plugin` runtime descriptor remains unverified; underlying completion, DBO/graph and physical/multi-host transport are unqualified. Historical [intake](reviews/issue91-afd-plugin-intake.md) retained; no new Network coverage | **Resolved gaps since initial matrix (all high/medium priority cleared):** - ~~Host overhead profiling + optimization skills~~ → `host-overhead-profiling`, `host-overhead-optimization` ✅ @@ -316,18 +316,18 @@ for translation, filename, package installation or an unexamined trace. | Resource | Stage / named subtargets | Existing canonical support | Boundary | |---|---|---|---| -| Accelerator | Diagnose/Configure/Verify: attention-generation dispatch, paged-decode planning, compiler guards, KV strategies, fusion resource limits | [Dependency records](../src/inferrsi/knowledge/data/knowledge_packs/README.md); FlashAttention, FlashInfer, PyTorch, Transformers, Triton | Source guidance only; selected backend, pointer lifetime, precision and resource envelope require target validation | -| Accelerator | Configure/Optimize/Verify: residual reuse, KV compression, sequence partition and pipeline freshness | [Peer records](../src/inferrsi/knowledge/data/knowledge_packs/README.md); Cache-DiT, LMDeploy, xDiT | Approximation/capacity gains do not establish latency or quality gains; preserved LMDeploy regression | -| RAM / Host | Diagnose/Configure: group offload staging, CPU expert work, host prefix residency | Diffusers dependency card; [Strata tuning](../src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/controlled-execution-tuning.json); SGLang HiCache peer card | Host budget and transfer/recompute cost remain distinct from device HBM | -| SSD / Storage | Configure/Verify: prefix L3 sharing, prefetch/layout/eviction and hit-state comparisons | [SGLang HiCache](../src/inferrsi/knowledge/data/knowledge_packs/peer/sglang/cuda/llm/hierarchical-prefix-cache.json) | Storage backend qualification pending; larger private host cache is not cross-instance sharing | -| Communication / Network | Diagnose/Configure/Verify: disaggregated KV transfer, topology-dependent sequence parallelism | [TRT-LLM disaggregation](../src/inferrsi/knowledge/data/knowledge_packs/peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json); [xDiT](../src/inferrsi/knowledge/data/knowledge_packs/peer/xdit/cuda/diffusion/parallelism-freshness-and-topology.json) | Interconnect/layout/queue cost, backpressure and recovery; H2D/IPC are not counted here | +| Accelerator | Diagnose/Configure/Verify: attention-generation dispatch, paged-decode planning, compiler guards, KV strategies, fusion resource limits | [Dependency records](../src/inferrsi/knowledge/data/skill_library/records/README.md); FlashAttention, FlashInfer, PyTorch, Transformers, Triton | Source guidance only; selected backend, pointer lifetime, precision and resource envelope require target validation | +| Accelerator | Configure/Optimize/Verify: residual reuse, KV compression, sequence partition and pipeline freshness | [Peer records](../src/inferrsi/knowledge/data/skill_library/records/README.md); Cache-DiT, LMDeploy, xDiT | Approximation/capacity gains do not establish latency or quality gains; preserved LMDeploy regression | +| RAM / Host | Diagnose/Configure: group offload staging, CPU expert work, host prefix residency | Diffusers dependency card; [Strata tuning](../src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.controlled_execution_tuning.json); SGLang HiCache peer card | Host budget and transfer/recompute cost remain distinct from device HBM | +| SSD / Storage | Configure/Verify: prefix L3 sharing, prefetch/layout/eviction and hit-state comparisons | [SGLang HiCache](../src/inferrsi/knowledge/data/skill_library/records/sglang.cuda.llm.hierarchical_prefix_cache.json) | Storage backend qualification pending; larger private host cache is not cross-instance sharing | +| Communication / Network | Diagnose/Configure/Verify: disaggregated KV transfer, topology-dependent sequence parallelism | [TRT-LLM disaggregation](../src/inferrsi/knowledge/data/skill_library/records/tensorrt_llm.cuda.llm.prefill_decode_disaggregation.json); [xDiT](../src/inferrsi/knowledge/data/skill_library/records/xdit.cuda.diffusion.parallelism_freshness_and_topology.json) | Interconnect/layout/queue cost, backpressure and recovery; H2D/IPC are not counted here | | OS / System | No new OS-control mechanism credited | Existing OS-specific rows retained | Process/stage ownership and installed access are not scheduler-priority or cgroup evidence | -| Accelerator / Host | Observe/Verify: actual active batch, admission/drain scope and state lifetime | [Shared internal case](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.md); [main audio guidance](../src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/audio/concurrency-scaling-state-and-overlap.json) | Recorded observation versus interpretation; frozen evidence preserved, no new service gain | +| Accelerator / Host | Observe/Verify: actual active batch, admission/drain scope and state lifetime | [Shared internal case](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md); [main audio guidance](../src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap.json) | Recorded observation versus interpretation; frozen evidence preserved, no new service gain | The existing casebook/search-policy path now also exposes rejected directions -for [Qwen B300](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md), -[Qwen Ascend](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md) -and [Drama3](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md). +for [Qwen B300](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md), +[Qwen Ascend](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md) +and [Drama3](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md). They inform Observe/Diagnose/Verify under their recorded Accelerator/Host scope; no new green coverage, Network/OS capability or measured gain is credited. Current applicability and a changed condition must be established before retry. @@ -374,8 +374,8 @@ rows above remain authoritative; the two-matrix contract is unchanged. | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | |---|---|---|---| -| Accelerator | Observe / Diagnose / Verify: submission versus completion, fence/event dependency ordering and selected backend dispatch | [lazy-evaluation-stream-lifetimes](../src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) | Pinned CPU/Metal/CUDA source paths; no native device execution or measured gain | -| RAM / Host | Observe / Diagnose / Verify: graph-reference detachment versus in-flight Data retention and result availability | [execution/lifetime contract](../src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md) | Backend-specific retention mechanisms; allocator eviction and physical-memory release remain outside the readset | +| Accelerator | Observe / Diagnose / Verify: submission versus completion, fence/event dependency ordering and selected backend dispatch | [lazy-evaluation-stream-lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) | Pinned CPU/Metal/CUDA source paths; no native device execution or measured gain | +| RAM / Host | Observe / Diagnose / Verify: graph-reference detachment versus in-flight Data retention and result availability | [execution/lifetime contract](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md) | Backend-specific retention mechanisms; allocator eviction and physical-memory release remain outside the readset | No new measured or green coverage is credited. Stream dependency guidance does not add a Communication/Network capability; no SSD or OS control is introduced. @@ -384,15 +384,15 @@ not add a Communication/Network capability; no SSD or OS control is introduced. ## vLLM-Omni incremental abort guidance Route general stage/model/output decisions through the -[engine source and responsibility map](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/SKILL.md#source-and-responsibility-map). +[engine source and responsibility map](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/SKILL.md#source-and-responsibility-map). For receipt, wait-only timeout or retained transfer resources, follow -[abort validation](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/SKILL.md) -and its [three completion boundaries](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md#keep-three-completion-boundaries-separate). +[abort validation](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/SKILL.md) +and its [three completion boundaries](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md#keep-three-completion-boundaries-separate). The map keeps the engine and incremental abort source snapshots separate. | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | |---|---|---|---| -| Accelerator / RAM / Host | Diagnose / Verify: abort submission, logical acknowledgment, partial AR output and separately queued transfer reclamation | [Abort acknowledgment reference](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md) under the existing validation skill | Source control-flow and queue ownership, not native device synchronization or measured release completion | +| Accelerator / RAM / Host | Diagnose / Verify: abort submission, logical acknowledgment, partial AR output and separately queued transfer reclamation | [Abort acknowledgment reference](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md) under the existing validation skill | Source control-flow and queue ownership, not native device synchronization or measured release completion | | RAM / Host | Observe / Diagnose / Verify: correlated waiter lifetime, blocking submission, wait-only timeout and late result routing | Same canonical reference, source `18c3bd5d2d4323bc7adca1ac3c465c4a53bbfb6e` | Queue timeout does not certify engine cancellation; actual API/backend/next-request validation not_run | These extend source-derived guidance without a new measured/green cell. No new @@ -402,8 +402,8 @@ SSD, Network or OS-control coverage is credited by local RPC or release-task cod | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | |---|---|---|---| -| RAM / Host | Observe / Diagnose / Configure / Verify: request UID/stage/cache-row alignment, prefix identity/copy/trim/eviction and cooperative stop | [generation-cache-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md) | Logical cache state and allocated storage differ; native model/row/stop behavior unrun | -| Accelerator | Observe / Diagnose / Verify: prefill cache evaluation and asynchronous decode submission/read boundary | [generation/batch contract](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/generation-batch.md) | Application calls MLX primitives; CPU checks cannot certify Metal/CUDA dispatch, quality or serving performance | +| RAM / Host | Observe / Diagnose / Configure / Verify: request UID/stage/cache-row alignment, prefix identity/copy/trim/eviction and cooperative stop | [generation-cache-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md) | Logical cache state and allocated storage differ; native model/row/stop behavior unrun | +| Accelerator | Observe / Diagnose / Verify: prefill cache evaluation and asynchronous decode submission/read boundary | [generation/batch contract](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/generation-batch.md) | Application calls MLX primitives; CPU checks cannot certify Metal/CUDA dispatch, quality or serving performance | No measured or green coverage is credited. Application batching is not a new Network/OS capability; no SSD mechanism is added. `common` indexes guidance and @@ -413,12 +413,12 @@ does not certify backend compatibility. | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | |---|---|---|---| -| RAM / Host | Observe / Diagnose / Configure / Verify: wrapper versus reused component identity, per-invocation scheduler schedule/index ownership | [diffusers-invocation-state](../src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md) | Configuration means selecting the existing invocation/scheduler owner; no new manager. Real caller admission and runtime overlap are unverified. | -| Accelerator | Observe / Diagnose / Verify: post-step callback, next-iteration interrupt observation and normal output/cleanup tail | [invocation ownership path](../src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/ownership.md) | Cooperative loop skipping does not revoke submitted kernels; model/device/quality, exception cleanup and performance remain unrun. | +| RAM / Host | Observe / Diagnose / Configure / Verify: wrapper versus reused component identity, per-invocation scheduler schedule/index ownership | [diffusers-invocation-state](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md) | Configuration means selecting the existing invocation/scheduler owner; no new manager. Real caller admission and runtime overlap are unverified. | +| Accelerator | Observe / Diagnose / Verify: post-step callback, next-iteration interrupt observation and normal output/cleanup tail | [invocation ownership path](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/references/ownership.md) | Cooperative loop skipping does not revoke submitted kernels; model/device/quality, exception cleanup and performance remain unrun. | For group/leaf offload staging and host/device transfer tradeoffs, select the -separate CUDA dependency [offload record](../src/inferrsi/knowledge/data/knowledge_packs/dependency/diffusers/cuda/diffusion/group-offload-transfer-overlap.json). -The [two-pin invocation/offload boundary](../src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md#offload-or-invocation-state) +separate CUDA dependency [offload record](../src/inferrsi/knowledge/data/skill_library/records/diffusers.cuda.diffusion.group_offload_transfer_overlap.json). +The [two-pin invocation/offload boundary](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md#offload-or-invocation-state) adds routing only: the common invocation candidate does not qualify that CUDA recipe, an NPU path or a combined stack. Existing coverage/maturity stays unchanged. @@ -430,9 +430,9 @@ framework guidance without certifying backend compatibility. | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | |---|---|---|---| -| Accelerator | Diagnose / Configure / Verify: configured-width DFlash drafts versus budgeted verification rows; lagged CPU totals and GPU ZOS per-request metadata correction | [budget-speculative-verification](../src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md) | Source constraints and cost objective, not sampling-distribution proof, dispatch qualification or measured gain | -| Accelerator / RAM / Host | Diagnose / Configure / Verify: retained query dependency closure, separately preserved full-layout KV, SWA valid-state bounds and nested ring-sizing gates | [bound-windowed-attention-replay](../src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md) | Model-specific boundary/rollback/cache contract; native equivalence and resource measurements unrun | -| RAM / Host | Observe / Diagnose / Verify: host work overlap, one-step retained state, repair and drain obligations | [Existing host-overhead owner case](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md) | Source-level schedule, no reproduced host-gap reduction or complete-service latency result | +| Accelerator | Diagnose / Configure / Verify: configured-width DFlash drafts versus budgeted verification rows; lagged CPU totals and GPU ZOS per-request metadata correction | [budget-speculative-verification](../src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/SKILL.md) | Source constraints and cost objective, not sampling-distribution proof, dispatch qualification or measured gain | +| Accelerator / RAM / Host | Diagnose / Configure / Verify: retained query dependency closure, separately preserved full-layout KV, SWA valid-state bounds and nested ring-sizing gates | [bound-windowed-attention-replay](../src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/SKILL.md) | Model-specific boundary/rollback/cache contract; native equivalence and resource measurements unrun | +| RAM / Host | Observe / Diagnose / Verify: host work overlap, one-step retained state, repair and drain obligations | [Existing host-overhead owner case](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md) | Source-level schedule, no reproduced host-gap reduction or complete-service latency result | These rows add source-derived procedure guidance; native execution and measured gains remain unverified. They do not add measured resource coverage: @@ -444,12 +444,12 @@ separate from the public release's inspected code paths. | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / Host / service boundary | Observe / Verify: select task quality and reference parity; freeze workload, timing, SLOs, units and failure accounting; distinguish agent replay from live task success | [inference-benchmark-design task reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md) and [commands](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md) | Cross-task measurement guidance, not new measured hardware coverage. A text HTTP smoke cannot validate TTS, duplex, image/video quality, repeated A/B or production capacity. No runtime adapter or automatic evaluator is added. | -| Accelerator / Host / decoded-audio boundary | Observe / Verify: derive cache-aware ASR startup/steady mel geometry and decoded frontier; distinguish slot reset/reuse, EOU, EOF and full stream reset; retain optional-model test coverage | [inference-benchmark-design ASR reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md) | NeMo-Speech.cpp `20999cc921d2b14aa58708f61756a8b2e787ba20`; selected source and test definitions, author ASR benchmark only. Candidate / empirical not_run; no executed model, device, long-stream/load, ASR quality or service gain and no new runtime adapter. | +| Accelerator / Host / service boundary | Observe / Verify: select task quality and reference parity; freeze workload, timing, SLOs, units and failure accounting; distinguish agent replay from live task success | [inference-benchmark-design task reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/task-benchmark-selection.md) and [commands](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/benchmark-commands.md) | Cross-task measurement guidance, not new measured hardware coverage. A text HTTP smoke cannot validate TTS, duplex, image/video quality, repeated A/B or production capacity. No runtime adapter or automatic evaluator is added. | +| Accelerator / Host / decoded-audio boundary | Observe / Verify: derive cache-aware ASR startup/steady mel geometry and decoded frontier; distinguish slot reset/reuse, EOU, EOF and full stream reset; retain optional-model test coverage | [inference-benchmark-design ASR reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md) | NeMo-Speech.cpp `20999cc921d2b14aa58708f61756a8b2e787ba20`; selected source and test definitions, author ASR benchmark only. Candidate / empirical not_run; no executed model, device, long-stream/load, ASR quality or service gain and no new runtime adapter. | ## SGLang scope selection -Use the [versioned entry comparison](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/SKILL.md#choose-among-source-pins) +Use the [versioned entry comparison](../src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/SKILL.md#choose-among-source-pins) to distinguish LLM engine/Radix ownership, the separate CUDA HiCache peer's prefix-KV host/storage policy, and multimodal_gen diffusion stage/instance, component-use and timeout/job-finalizer lifetimes. This adds routing only. @@ -460,7 +460,7 @@ Network coverage. Existing resource cells and maturity boundaries remain unchang | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator | Observe / Diagnose / Configure / Verify: TMA full/empty stage ownership and parity, WGMMA group completion and SS/RS layouts, register/SMEM/epilogue budgets, software persistent scheduling and FP8 scale/promotion contracts | [reason-hopper-kernels](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md), with [systematic source selection](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/sources.md) | Candidate mechanisms qualified by selected pinned code and versioned documentation; remaining source bodies unmined, Blackwell-only recipes excluded. No GPU compilation, accuracy, profiling or performance run. | +| Accelerator | Observe / Diagnose / Configure / Verify: TMA full/empty stage ownership and parity, WGMMA group completion and SS/RS layouts, register/SMEM/epilogue budgets, software persistent scheduling and FP8 scale/promotion contracts | [reason-hopper-kernels](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/SKILL.md), with [systematic source selection](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/sources.md) | Candidate mechanisms qualified by selected pinned code and versioned documentation; remaining source bodies unmined, Blackwell-only recipes excluded. No GPU compilation, accuracy, profiling or performance run. | This adds conditional Hopper guidance without new measured or green coverage. Registers, SMEM and HBM remain device resources; local CTA/cluster coordination @@ -470,35 +470,35 @@ and tile scheduling add no RAM/Host, SSD, Network or OS-control coverage. | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| RAM / Accelerator / SSD | Diagnose / Configure: routed-expert cache versus non-routed weights, context, shared prefill workspace and per-session recurrent/KV state | [DwarfStar runtime-memory](../src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/SKILL.md) | Bounded source-derived budgeting guidance; native GPU/model SSD I/O and actual peak memory are not validated. | -| Host / Accelerator / service boundary | Diagnose / Verify: token and image prefix identity, disk payload compatibility, tool replay, speculation rollback and actual batching/fallback path | [DwarfStar serving-state](../src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/SKILL.md) | Model-free CPU bookkeeping and source contracts are distinct from accelerated correctness, physical TCP/RDMA or service performance. This adds no measured Network coverage. | -| RAM / Accelerator / SSD | Diagnose / Configure / Verify: Root routing: bind model/backend/cache-state identity, then select runtime-memory, serving-state or qualification ownership | [dwarfstar](../src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/SKILL.md) | Navigation to the existing focused rows; no additional mechanism or native/model/SSD-I/O/Network/performance qualification. | +| RAM / Accelerator / SSD | Diagnose / Configure: routed-expert cache versus non-routed weights, context, shared prefill workspace and per-session recurrent/KV state | [DwarfStar runtime-memory](../src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/runtime-memory/SKILL.md) | Bounded source-derived budgeting guidance; native GPU/model SSD I/O and actual peak memory are not validated. | +| Host / Accelerator / service boundary | Diagnose / Verify: token and image prefix identity, disk payload compatibility, tool replay, speculation rollback and actual batching/fallback path | [DwarfStar serving-state](../src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/serving-state/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/qualification/SKILL.md) | Model-free CPU bookkeeping and source contracts are distinct from accelerated correctness, physical TCP/RDMA or service performance. This adds no measured Network coverage. | +| RAM / Accelerator / SSD | Diagnose / Configure / Verify: Root routing: bind model/backend/cache-state identity, then select runtime-memory, serving-state or qualification ownership | [dwarfstar](../src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/SKILL.md) | Navigation to the existing focused rows; no additional mechanism or native/model/SSD-I/O/Network/performance qualification. | ## Splash source-derived planning and state | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / RAM / SSD | Diagnose / Configure: mandatory memory floors, expert/PLE stores, dynamic KV/state, disk spill and precision boundaries | [runtime-planning](../src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/SKILL.md) | Source and CPU syntax checks, not measured device/process peak memory or actual spill IO. | -| Host / Accelerator / service boundary | Diagnose / Verify: salted prompt identity versus global image reuse, composite-state commit/reclaim, persistent namespace and process-local Responses history, adaptive DFlash2 row budgeting | [serving-cache](../src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/SKILL.md) | Python codec/LRU and descriptive statistics do not qualify native Metal/ANE/model state, service SLOs or efficacy. No measured Network coverage is added. | -| Accelerator / RAM / SSD | Diagnose / Configure / Verify: Root routing: bind representation, memory/cache-state and actual-path identity before runtime-planning, serving-cache or qualification | [splash](../src/inferrsi/knowledge/data/skill_library/repositories/splash/SKILL.md) | Navigation to focused rows; required Metal/ANE/model/disk-state evidence remains blocked, with no new performance, service or Network qualification. | +| Accelerator / RAM / SSD | Diagnose / Configure: mandatory memory floors, expert/PLE stores, dynamic KV/state, disk spill and precision boundaries | [runtime-planning](../src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/SKILL.md) | Source and CPU syntax checks, not measured device/process peak memory or actual spill IO. | +| Host / Accelerator / service boundary | Diagnose / Verify: salted prompt identity versus global image reuse, composite-state commit/reclaim, persistent namespace and process-local Responses history, adaptive DFlash2 row budgeting | [serving-cache](../src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/serving-cache/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/SKILL.md) | Python codec/LRU and descriptive statistics do not qualify native Metal/ANE/model state, service SLOs or efficacy. No measured Network coverage is added. | +| Accelerator / RAM / SSD | Diagnose / Configure / Verify: Root routing: bind representation, memory/cache-state and actual-path identity before runtime-planning, serving-cache or qualification | [splash](../src/inferrsi/knowledge/data/skill_library/systems/engines/splash/SKILL.md) | Navigation to focused rows; required Metal/ANE/model/disk-state evidence remains blocked, with no new performance, service or Network qualification. | ## Lily exact-model source contracts | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / RAM | Diagnose / Configure / Verify: exact affine-Q4 plus role-Q8, shared buffer views, state/scratch/sequence capacity and completion ownership | [model-runtime](../src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/SKILL.md) | Source/Python selected-rule checks, not Rust loading, actual memory fit, Metal synchronization or model quality. | -| Host / Accelerator / service boundary | Diagnose / Verify: strict API, extra lookahead token, strict token prefix before key preference, actually-fed state retention and version/metric qualification | [serving-cache](../src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/SKILL.md) | Compiler/native/model stages not run; CPU references and ignored/cfg-omitted tests are not GPU evidence. No measured service or Network coverage is added. | -| Accelerator / RAM | Diagnose / Configure / Verify: Root routing: exact model/affine payload, token frontier and extra lookahead-slot identity before model-runtime, serving-cache and qualification checks | [lily](../src/inferrsi/knowledge/data/skill_library/repositories/lily/SKILL.md) | Navigation to existing focused owners; no added native memory-fit, Metal state, model-quality, service or Network qualification. | +| Accelerator / RAM | Diagnose / Configure / Verify: exact affine-Q4 plus role-Q8, shared buffer views, state/scratch/sequence capacity and completion ownership | [model-runtime](../src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/SKILL.md) | Source/Python selected-rule checks, not Rust loading, actual memory fit, Metal synchronization or model quality. | +| Host / Accelerator / service boundary | Diagnose / Verify: strict API, extra lookahead token, strict token prefix before key preference, actually-fed state retention and version/metric qualification | [serving-cache](../src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/SKILL.md) | Compiler/native/model stages not run; CPU references and ignored/cfg-omitted tests are not GPU evidence. No measured service or Network coverage is added. | +| Accelerator / RAM | Diagnose / Configure / Verify: Root routing: exact model/affine payload, token frontier and extra lookahead-slot identity before model-runtime, serving-cache and qualification checks | [lily](../src/inferrsi/knowledge/data/skill_library/systems/engines/lily/SKILL.md) | Navigation to existing focused owners; no added native memory-fit, Metal state, model-quality, service or Network qualification. | ## Husky research questions, not measured coverage | Resource | Stage | Owner | Evidence boundary | | --- | --- | --- | --- | -| Unqualified / conditional host-device questions | Diagnose / propose a falsifiable comparison and required artifacts | [Husky document research](../src/inferrsi/knowledge/data/skill_library/repositories/husky/SKILL.md) | Author-reported unversioned page and original audit questions only. No inspected implementation, backend validation, measured service improvement or expanded resource coverage. | +| Unqualified / conditional host-device questions | Diagnose / propose a falsifiable comparison and required artifacts | [Husky document research](../src/inferrsi/knowledge/data/skill_library/systems/engines/husky/SKILL.md) | Author-reported unversioned page and original audit questions only. No inspected implementation, backend validation, measured service improvement or expanded resource coverage. | ## LoopX Goal governance (cross-cutting workflow) | Scope / stage | Named decision mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Cross-cutting Observe / Diagnose / Verify | Current Todo authority versus Markdown/index/audit projections; missing active Goal state, complete terminal evidence and ordered capability/workspace/inbox/repair decisions | [rsi-evidence Goal governance](../src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/README.md) | Pinned LoopX source/doc/test assertions, `f094b2cfb2acde7a4a0d4d875bba52f64ac31ba9`; source-derived candidate, runtime and agent evaluation `not_run` | -| Cross-cutting Diagnose / Verify, consumed by experiment selection | Local accounting effect/request identity, index CAS, commit repair versus replay lookup, exact Goal instance and external-effect crash-window limits | Same owner; [optimization-search-policy](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md#goal-admission-and-accounting-source-knowledge) links it | Local receipt idempotency does not prove external exactly-once or a supplier-side hard cost cap; a decision/receipt is not a campaign verdict | +| Cross-cutting Observe / Diagnose / Verify | Current Todo authority versus Markdown/index/audit projections; missing active Goal state, complete terminal evidence and ordered capability/workspace/inbox/repair decisions | [rsi-evidence Goal governance](../src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md) | Pinned LoopX source/doc/test assertions, `f094b2cfb2acde7a4a0d4d875bba52f64ac31ba9`; source-derived candidate, runtime and agent evaluation `not_run` | +| Cross-cutting Diagnose / Verify, consumed by experiment selection | Local accounting effect/request identity, index CAS, commit repair versus replay lookup, exact Goal instance and external-effect crash-window limits | Same owner; [optimization-search-policy](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/SKILL.md#goal-admission-and-accounting-source-knowledge) links it | Local receipt idempotency does not prove external exactly-once or a supplier-side hard cost cap; a decision/receipt is not a campaign verdict | This adds workflow knowledge, with no new green or measured resource cell. Agent coordination/accounting is not Accelerator, Communication/Network or @@ -510,45 +510,45 @@ unchanged. | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / RAM / SSD | Diagnose / plan: family/representation guards, physical-pool ownership, expert tiers, private state and scratch overlap | `project-maya`: [model-runtime](../src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/SKILL.md) | Source plus bounded host arithmetic/simulation; not actual accelerator allocation, pressure, tensor payload or residency performance. | -| Host / Accelerator / service boundary | Diagnose / verify: API snapshots versus native slots, parallel state, completion, speculative retained state, calibration controls and quality evidence | `project-maya`: [serving-state](../src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/SKILL.md) | Required CUDA/HIP/model/parallel restore/cancellation evidence remains unavailable. HTTP procedure adds no measured Network/OS or service-gain coverage. Qwen deep state/kernels remain unmined. | +| Accelerator / RAM / SSD | Diagnose / plan: family/representation guards, physical-pool ownership, expert tiers, private state and scratch overlap | `project-maya`: [model-runtime](../src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/SKILL.md) | Source plus bounded host arithmetic/simulation; not actual accelerator allocation, pressure, tensor payload or residency performance. | +| Host / Accelerator / service boundary | Diagnose / verify: API snapshots versus native slots, parallel state, completion, speculative retained state, calibration controls and quality evidence | `project-maya`: [serving-state](../src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/SKILL.md) | Required CUDA/HIP/model/parallel restore/cancellation evidence remains unavailable. HTTP procedure adds no measured Network/OS or service-gain coverage. Qwen deep state/kernels remain unmined. | ## vLLM Metal selected source contracts | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / RAM / SSD | Diagnose / configure: lazy KV capacity versus resident commitment, one physical unified pool, host/disk ownership, transfer/sleep/reuse and identity | `vllm-metal`: [cache-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/SKILL.md) | Selected substantive source/test assertions plus dense-uniform arithmetic only; real transfers/pressure/sleep/model/backend evidence blocked. No measured memory-fit or IO gain. | -| Accelerator / host control boundary | Diagnose / verify: native ABI/backend/runner pairing, GDN state and deferred output ownership, executed dispatch versus selector, serving evidence | `vllm-metal`: [platform-config](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/SKILL.md) | Installed pure API and selected parser/default tests do not qualify Apple/native/model/quality/service. No new Network/OS control or green resource coverage. | +| Accelerator / RAM / SSD | Diagnose / configure: lazy KV capacity versus resident commitment, one physical unified pool, host/disk ownership, transfer/sleep/reuse and identity | `vllm-metal`: [cache-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/SKILL.md) | Selected substantive source/test assertions plus dense-uniform arithmetic only; real transfers/pressure/sleep/model/backend evidence blocked. No measured memory-fit or IO gain. | +| Accelerator / host control boundary | Diagnose / verify: native ABI/backend/runner pairing, GDN state and deferred output ownership, executed dispatch versus selector, serving evidence | `vllm-metal`: [platform-config](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/SKILL.md) and [qualification](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/SKILL.md) | Installed pure API and selected parser/default tests do not qualify Apple/native/model/quality/service. No new Network/OS control or green resource coverage. | ## mistral.rs Metal paged-attention source contracts | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / RAM | Diagnose / configure: HND/packed cache/scale/GQA contracts, instantaneous scratch versus capacity and prefill output/metadata reservation | `mistral-rs`: [decode-contract](../src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/SKILL.md) and [prefill-contract](../src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/SKILL.md) | Source-derived byte/metadata support only; no actual storage/allocator/pressure/parity or physical fit. No measured resource coverage. | -| Accelerator / host control boundary | Diagnose / verify: absolute positions/masks/ranges and actual-model warmup/pipeline cache versus executed native test evidence | `mistral-rs`: [qualification](../src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/SKILL.md) | Compiler and native backend not_run; shape inventory is not GPU simulation or dispatch. No new SSD/Network/OS control, quality/service or performance qualification. | +| Accelerator / RAM | Diagnose / configure: HND/packed cache/scale/GQA contracts, instantaneous scratch versus capacity and prefill output/metadata reservation | `mistral-rs`: [decode-contract](../src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/SKILL.md) and [prefill-contract](../src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/SKILL.md) | Source-derived byte/metadata support only; no actual storage/allocator/pressure/parity or physical fit. No measured resource coverage. | +| Accelerator / host control boundary | Diagnose / verify: absolute positions/masks/ranges and actual-model warmup/pipeline cache versus executed native test evidence | `mistral-rs`: [qualification](../src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/SKILL.md) | Compiler and native backend not_run; shape inventory is not GPU simulation or dispatch. No new SSD/Network/OS control, quality/service or performance qualification. | ## MLX Swift LM ownership and recurrent Metal source contracts | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Host control / accelerator lifetime | Diagnose / verify: exclusive async access, Sendable transfer, lookahead/cancellation/finalization/default-stream sync versus task join | `mlx-swift-lm`: [generation-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/SKILL.md) | Caller evidence declarations only; no actual task/device execution, native cleanup, backgrounding or safe shared-cache proof. No measured resource coverage. | -| Accelerator / RAM | Diagnose / configure: recurrent FP32 state/layout, Dk tails, public kernel/ops gate, source mask discrepancy and dispatch/parity evidence | `mlx-swift-lm`: [gated-delta-contract](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/SKILL.md) | Static source/declared bytes only; no actual provider stride/allocator/dispatch/numerical or physical-fit qualification. No new SSD/Network/OS control, quality/service or performance result. | +| Host control / accelerator lifetime | Diagnose / verify: exclusive async access, Sendable transfer, lookahead/cancellation/finalization/default-stream sync versus task join | `mlx-swift-lm`: [generation-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/SKILL.md) | Caller evidence declarations only; no actual task/device execution, native cleanup, backgrounding or safe shared-cache proof. No measured resource coverage. | +| Accelerator / RAM | Diagnose / configure: recurrent FP32 state/layout, Dk tails, public kernel/ops gate, source mask discrepancy and dispatch/parity evidence | `mlx-swift-lm`: [gated-delta-contract](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/SKILL.md) | Static source/declared bytes only; no actual provider stride/allocator/dispatch/numerical or physical-fit qualification. No new SSD/Network/OS control, quality/service or performance result. | ## MLC LLM artifact, target and runtime source contracts | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / RAM | Diagnose / configure: parameter-only resource declarations versus runtime KV/workspace/limit origins | `mlc-llm`: [artifact-contract](../src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/SKILL.md) and [runtime-kv-limits](../src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/SKILL.md) | Source/hash/declared minima only; no authenticated weights/allocator/free-memory/cache or physical-fit evidence. No measured resource coverage. | -| Accelerator / host completion boundary | Diagnose / verify: SDK/export/prefix, runnable versus serializable module, private/staging buffers, copy/flush/wait/default-stream contracts | `mlc-llm`: [metal-build-runtime](../src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/SKILL.md) | Native compile/load/dispatch/sync/parity not_run. No new SSD/Network/OS control or quality/service/performance qualification. | +| Accelerator / RAM | Diagnose / configure: parameter-only resource declarations versus runtime KV/workspace/limit origins | `mlc-llm`: [artifact-contract](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/SKILL.md) and [runtime-kv-limits](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/SKILL.md) | Source/hash/declared minima only; no authenticated weights/allocator/free-memory/cache or physical-fit evidence. No measured resource coverage. | +| Accelerator / host completion boundary | Diagnose / verify: SDK/export/prefix, runnable versus serializable module, private/staging buffers, copy/flush/wait/default-stream contracts | `mlc-llm`: [metal-build-runtime](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/SKILL.md) | Native compile/load/dispatch/sync/parity not_run. No new SSD/Network/OS control or quality/service/performance qualification. | ## Flare preparation and complete-session source contracts | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator | Diagnose / Verify: complete KV page chain plus continuation row; publish/use/retire fences, shared inbound ticket and five admission outcomes | [audit-flare-session-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md) | Fixed `bce032c014bae34502ba22d3304fa084d5b40afe` CODE/DOC/TEST-SOURCE; NVIDIA SM120/SM121 research prototype; no kernel/model/device/performance qualification | -| RAM / Host | Diagnose / Verify: registered/staging storage lifetime through accounted transfer completion/drain; waiter cancellation does not free ticket destinations | [audit-flare-session-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md), [lifetime detail](../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/lifecycle.md) | Source lifetime contract; no physical-RAM peak, bandwidth or pressure result | -| SSD | Diagnose / Verify: record namespace/layout/row/chain completeness, restart/corruption versus unreadable-file handling; conditional transfer-versus-additional-prefill cost | [audit-flare-session-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md), [diagnostic detail](../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md) | No GDS implementation or measured storage gain; actual device/server tests not_run | -| SSD / RAM / Accelerator | Startup Diagnose / Verify: Python preparation/native export ownership, nonportable live handles, execution ABI 4 versus prepared contract 2, separate phase accounting | Existing [weight-loading owner](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md), [preparation detail](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/flare-preparation-boundary.md) | Source-derived candidate; no preparation/runtime execution or Python-to-Rust performance-causation claim | +| Accelerator | Diagnose / Verify: complete KV page chain plus continuation row; publish/use/retire fences, shared inbound ticket and five admission outcomes | [audit-flare-session-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md) | Fixed `bce032c014bae34502ba22d3304fa084d5b40afe` CODE/DOC/TEST-SOURCE; NVIDIA SM120/SM121 research prototype; no kernel/model/device/performance qualification | +| RAM / Host | Diagnose / Verify: registered/staging storage lifetime through accounted transfer completion/drain; waiter cancellation does not free ticket destinations | [audit-flare-session-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md), [lifetime detail](../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/lifecycle.md) | Source lifetime contract; no physical-RAM peak, bandwidth or pressure result | +| SSD | Diagnose / Verify: record namespace/layout/row/chain completeness, restart/corruption versus unreadable-file handling; conditional transfer-versus-additional-prefill cost | [audit-flare-session-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md), [diagnostic detail](../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md) | No GDS implementation or measured storage gain; actual device/server tests not_run | +| SSD / RAM / Accelerator | Startup Diagnose / Verify: Python preparation/native export ownership, nonportable live handles, execution ABI 4 versus prepared contract 2, separate phase accounting | Existing [weight-loading owner](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md), [preparation detail](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/flare-preparation-boundary.md) | Source-derived candidate; no preparation/runtime execution or Python-to-Rust performance-causation claim | There is no new measured/green resource cell, Network or OS coverage. Multi-rank source branches and tests do not qualify multi-host transport or multi-GPU @@ -560,21 +560,21 @@ behavior. The source-derived candidates remain empirical **not_run**; the | Resource | Stages and mechanism | Owner and evidence | Qualification boundary | |---|---|---|---| -| Accelerator | Diagnose / Configure / Verify: concrete Qwen3-Next GDN caller, requested/effective prefill recurrence, separate in-tree conv fwd/update, mixed/speculative metadata, state-row and progress contract | [kernel-writing-guide reference](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md), [four source cases](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.md); vLLM @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` | Source-derived candidate; native not_run, agent efficacy not_evaluated. HF caller binding, external implementation equivalence, physical allocation/recycling, target hardware and performance remain unqualified. | +| Accelerator | Diagnose / Configure / Verify: concrete Qwen3-Next GDN caller, requested/effective prefill recurrence, separate in-tree conv fwd/update, mixed/speculative metadata, state-row and progress contract | [kernel-writing-guide reference](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md), [four source cases](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-source-contract.md); vLLM @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` | Source-derived candidate; native not_run, agent efficacy not_evaluated. HF caller binding, external implementation equivalence, physical allocation/recycling, target hardware and performance remain unqualified. | ## SGLang GDN operator qualification | Resource | Scoped stages and question | Canonical owner / fixed source | Evidence boundary | |---|---|---|---| -| Accelerator | Diagnose / Configure / Verify: Qwen3.5/Qwen3-Next linear-layer caller, per-runner GDN prefill/decode and architecture predicates, separate conv stride/alias selection, packed metadata, state rows/write-back and conditional classification-prefill transfer | [kernel-writing-guide reference](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md), [four source cases](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.md); SGLang @ `16176278d0180f1a1a578e53e6eef432577876c2` | Source-derived candidate/not_run; agent efficacy not_evaluated. No HF binding, installed/native backend, sm89 compatibility, numerical/model-quality or performance claim. No new RAM, Storage, Network or OS coverage. | +| Accelerator | Diagnose / Configure / Verify: Qwen3.5/Qwen3-Next linear-layer caller, per-runner GDN prefill/decode and architecture predicates, separate conv stride/alias selection, packed metadata, state rows/write-back and conditional classification-prefill transfer | [kernel-writing-guide reference](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md), [four source cases](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/sglang-gdn-source-contract.md); SGLang @ `16176278d0180f1a1a578e53e6eef432577876c2` | Source-derived candidate/not_run; agent efficacy not_evaluated. No HF binding, installed/native backend, sm89 compatibility, numerical/model-quality or performance claim. No new RAM, Storage, Network or OS coverage. | ## Local Dream Android diffusion source contracts | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Host control / accelerator lifetime | Diagnose / verify: desired versus serving full configuration, startup-only preferences, grace/restart, process-spawn versus health and actual shape | [local-dream](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/SKILL.md), [Android lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/android-lifecycle.md) | Static source and exact declaration comparison, not Android lifecycle execution, native readiness, tensor shape or safe reuse proof. | -| Accelerator / RAM / persistent-cache boundary | Diagnose / configure: MNN/QNN/Anima/DiT dispatch, shared generation/upscale exclusion, callback cancellation, mode/layout versus model-revision cache identity | [local-dream native contracts](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/native-contracts.md) | No actual allocations/cache parity/model/image quality/fit/thermal/performance result; not a measured SSD gain or universal cache-safety claim. | -| Native packaging / remote exposure boundary | Diagnose / verify: core executable, CPU DiT library versus DSP skels, ABI and SDK provenance; unauthenticated LAN control versus readiness | [local-dream packaging](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/packaging-and-abi.md), [host limits](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/android-lifecycle.md) | Required native evidence BLOCKED_REQUIRED_BACKEND / not_run; no authorization to expose ports, Network transport optimization, OS control, native compatibility or service gain. | +| Host control / accelerator lifetime | Diagnose / verify: desired versus serving full configuration, startup-only preferences, grace/restart, process-spawn versus health and actual shape | [local-dream](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/SKILL.md), [Android lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/android-lifecycle.md) | Static source and exact declaration comparison, not Android lifecycle execution, native readiness, tensor shape or safe reuse proof. | +| Accelerator / RAM / persistent-cache boundary | Diagnose / configure: MNN/QNN/Anima/DiT dispatch, shared generation/upscale exclusion, callback cancellation, mode/layout versus model-revision cache identity | [local-dream native contracts](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/native-contracts.md) | No actual allocations/cache parity/model/image quality/fit/thermal/performance result; not a measured SSD gain or universal cache-safety claim. | +| Native packaging / remote exposure boundary | Diagnose / verify: core executable, CPU DiT library versus DSP skels, ABI and SDK provenance; unauthenticated LAN control versus readiness | [local-dream packaging](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/packaging-and-abi.md), [host limits](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/android-lifecycle.md) | Required native evidence BLOCKED_REQUIRED_BACKEND / not_run; no authorization to expose ports, Network transport optimization, OS control, native compatibility or service gain. | This source-derived candidate adds no measured/green resource coverage. Matching configuration/ABI/SDK declarations are not native execution; qualification remains @@ -585,67 +585,67 @@ NOASSERTION and InferRSI's repository license. | Resource | Stage / named mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator (guidance only) | Observe / Profile / Diagnose / Optimize / Verify: Ascend specialization of the shared workflow: contiguous packed-layout conversion, TransData attribution and CANN stream/lifetime checks | [ascend-video-diffusion-optimization](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/SKILL.md) | First-hand candidate guidance; field notes are untraceable/non-reproducible. No native, numerical or measured-gain qualification. | -| Accelerator (workflow guidance only) | Observe / Profile / Diagnose / Configure / Optimize / Verify: Video-stage planning and acceptance: residency/quantization trade-offs, dependency-and-lifetime gated overlap, attention eligibility and memory/quality-gated VAE chunking | [video-diffusion-optimization](../src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/SKILL.md) | First-hand candidate workflow, not independently validated; no model/backend/numerical/performance qualification or additional Host/Storage/Network/OS mechanism. | +| Accelerator (guidance only) | Observe / Profile / Diagnose / Optimize / Verify: Ascend specialization of the shared workflow: contiguous packed-layout conversion, TransData attribution and CANN stream/lifetime checks | [ascend-video-diffusion-optimization](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-video-diffusion-optimization/SKILL.md) | First-hand candidate guidance; field notes are untraceable/non-reproducible. No native, numerical or measured-gain qualification. | +| Accelerator (workflow guidance only) | Observe / Profile / Diagnose / Configure / Optimize / Verify: Video-stage planning and acceptance: residency/quantization trade-offs, dependency-and-lifetime gated overlap, attention eligibility and memory/quality-gated VAE chunking | [video-diffusion-optimization](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/SKILL.md) | First-hand candidate workflow, not independently validated; no model/backend/numerical/performance qualification or additional Host/Storage/Network/OS mechanism. | ## JustGoal controlled-evaluation guidance (source-derived) | Resource context | Stage / scoped question | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator (methodology only) | Verify: same-model/budget/scope controls, independent complete-output correctness, released versus repaired counts and workspace/stream/cache timing contracts | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-evaluation-contracts.md) and [source record](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-source.json); JustGoal repo + paired paper | Source-derived candidate support for benchmark verification only. Author-reported results; native/device/performance not_run, efficacy not_evaluated. No new hardware implementation, measured resource coverage, backend compatibility or RAM/Storage/Network/OS coverage; no full campaign runner. | +| Accelerator (methodology only) | Verify: same-model/budget/scope controls, independent complete-output correctness, released versus repaired counts and workspace/stream/cache timing contracts | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/justgoal-evaluation-contracts.md) and [source record](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/justgoal-source.json); JustGoal repo + paired paper | Source-derived candidate support for benchmark verification only. Author-reported results; native/device/performance not_run, efficacy not_evaluated. No new hardware implementation, measured resource coverage, backend compatibility or RAM/Storage/Network/OS coverage; no full campaign runner. | ## AutoTuneBench trial-validity guidance (source-derived) | Resource context | Stage / scoped question | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator (methodology only) | Verify: trace treatment to the eligible consumer, apply frozen arm validity before interpreting a null, and separate declared protocol, reached checks and persisted gate evidence | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md) and [source record](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json); AutoTuneBench repo + paired paper | Source-derived candidate / empirical not_run; paper measurements remain author-reported, efficacy not_evaluated. Contextual source bars are not InferRSI defaults. No new implementation, measured hardware/resource coverage, backend compatibility, or RAM/Storage/Network/OS coverage. | +| Accelerator (methodology only) | Verify: trace treatment to the eligible consumer, apply frozen arm validity before interpreting a null, and separate declared protocol, reached checks and persisted gate evidence | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/autotunebench-trial-validity.md) and [source record](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/autotunebench-source.json); AutoTuneBench repo + paired paper | Source-derived candidate / empirical not_run; paper measurements remain author-reported, efficacy not_evaluated. Contextual source bars are not InferRSI defaults. No new implementation, measured hardware/resource coverage, backend compatibility, or RAM/Storage/Network/OS coverage. | ## Xe-Forge trial-evidence guidance (source-derived) | Resource | Stage and mechanism | Source and owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator (methodology only) | Verify: bind saved candidate and comparator identities, distinguish tool origin from resolved performance, inspect host gate/ratio persistence, and follow manual versus measured-only delivery and baseline-oracle boundaries | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-trial-evidence.md) and [source record](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-source.json); Xe-Forge repo + paired paper | Source-derived candidate / empirical not_run; paper/README results remain author-reported, efficacy not_evaluated. No new implementation, measured hardware/resource coverage, backend compatibility, whole-model/service promotion or RAM/Storage/Network/OS coverage; draft extensions remain proposals. | +| Accelerator (methodology only) | Verify: bind saved candidate and comparator identities, distinguish tool origin from resolved performance, inspect host gate/ratio persistence, and follow manual versus measured-only delivery and baseline-oracle boundaries | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-trial-evidence.md) and [source record](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-source.json); Xe-Forge repo + paired paper | Source-derived candidate / empirical not_run; paper/README results remain author-reported, efficacy not_evaluated. No new implementation, measured hardware/resource coverage, backend compatibility, whole-model/service promotion or RAM/Storage/Network/OS coverage; draft extensions remain proposals. | ## NInfer state and completion validation (source-derived) | Resource context | Stage / scoped question | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / Host / service boundary (methodology only) | Observe / Verify: freeze coupled checkpoint and same-block state oracle; distinguish zero accepted drafts from zero adopted inputs, logical commit from device retirement, and selected steady intervals from complete-cohort/termination accounting | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md) and [source record](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-source.json); NInfer `1882c6adbc98043fcf89f3589aa5addb3742777a` | Source-derived candidate / empirical not_run; unexecuted test definitions and historical author results with a separate execution revision. No new implementation, measured hardware/resource coverage, backend compatibility, native state/model/quality/performance result or service acceptance; efficacy not_evaluated. | +| Accelerator / Host / service boundary (methodology only) | Observe / Verify: freeze coupled checkpoint and same-block state oracle; distinguish zero accepted drafts from zero adopted inputs, logical commit from device retirement, and selected steady intervals from complete-cohort/termination accounting | [inference-benchmark-design reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md) and [source record](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-source.json); NInfer `1882c6adbc98043fcf89f3589aa5addb3742777a` | Source-derived candidate / empirical not_run; unexecuted test definitions and historical author results with a separate execution revision. No new implementation, measured hardware/resource coverage, backend compatibility, native state/model/quality/performance result or service acceptance; efficacy not_evaluated. | ## TorchNPU framework source procedures | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / RAM / Host | Diagnose / configure / verify: PrivateUse1 registration versus native readiness, operator schema/fallback/Meta boundaries, stream ordering versus allocation lifetime, pinned copies, graph storage and AMP state | [torch-npu operating graph](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md), [memory route](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md), [graph route](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md) | Selected source spans and test definitions, not actual allocations, dispatch, numerical parity, compiler conformance, physical fit or measured resource benefit. Earlier allocator skill pins remain independent. | -| Communication / Network / Accelerator | Diagnose / verify: rank/device mapping, matching collective sequence, HCCL work handles, communication-stream ownership and host versus consumer completion | `torch-npu`: [HCCL route](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md), [source and dependency boundaries](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/repo-provenance.md) | HCCL process-group source and test assertions do not qualify transport, multi-rank correctness, overlap or throughput. Required Linux/Ascend/CANN/HCCL execution remains `BLOCKED_REQUIRED_BACKEND / not_run`; no new measured/green resource cell or OS control. | +| Accelerator / RAM / Host | Diagnose / configure / verify: PrivateUse1 registration versus native readiness, operator schema/fallback/Meta boundaries, stream ordering versus allocation lifetime, pinned copies, graph storage and AMP state | [torch-npu operating graph](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/SKILL.md), [memory route](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md), [graph route](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md) | Selected source spans and test definitions, not actual allocations, dispatch, numerical parity, compiler conformance, physical fit or measured resource benefit. Earlier allocator skill pins remain independent. | +| Communication / Network / Accelerator | Diagnose / verify: rank/device mapping, matching collective sequence, HCCL work handles, communication-stream ownership and host versus consumer completion | `torch-npu`: [HCCL route](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md), [source and dependency boundaries](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/repo-provenance.md) | HCCL process-group source and test assertions do not qualify transport, multi-rank correctness, overlap or throughput. Required Linux/Ascend/CANN/HCCL execution remains `BLOCKED_REQUIRED_BACKEND / not_run`; no new measured/green resource cell or OS control. | ## PyTorch framework source procedures | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / RAM / Host | Diagnose / configure / verify: tensor/alias/gradient state, training and data ownership, compiler guards and export contracts, allocator accounting, stream dependencies and graph input/output lifetimes | [pytorch operating graph](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/SKILL.md), [memory/graph route](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md), [compiler/export route](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md) | Source version `2.16.0a0` and observed CPU wheel `2.15.0+cpu` are distinct. CPU observations do not qualify pinned-source execution, CUDA/ROCm/MPS/XPU, native custom operators, memory fit or model quality. | -| SSD / Storage / RAM | Diagnose / verify: serialization trust and state identity, checkpoint ownership, distributed restore and capture/staging/upload completion boundaries | `pytorch`: [distributed/checkpoint route](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md); existing [DCP async-save owner](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md) retains its separate source pin | No source-build checkpoint run, backend durability, distributed recovery or measured storage benefit is established. A state-dict round trip in a separate installed wheel does not qualify those properties. | -| Communication / Network / Accelerator | Diagnose / verify: process/rank/device mapping, backend and collective schedule, DDP/FSDP2/DTensor state and completion | `pytorch`: [distributed/checkpoint route](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md), [source and runtime boundary](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/repo-provenance.md) | Selected source and CPU controls do not establish NCCL or other accelerator transport, multi-device/multi-node correctness, overlap or throughput. Required backend checks remain `BLOCKED_REQUIRED_BACKEND`; no new measured/green resource cell, OS control or compatible latest PyTorch/TorchNPU stack is asserted. | +| Accelerator / RAM / Host | Diagnose / configure / verify: tensor/alias/gradient state, training and data ownership, compiler guards and export contracts, allocator accounting, stream dependencies and graph input/output lifetimes | [pytorch operating graph](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/SKILL.md), [memory/graph route](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md), [compiler/export route](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md) | Source version `2.16.0a0` and observed CPU wheel `2.15.0+cpu` are distinct. CPU observations do not qualify pinned-source execution, CUDA/ROCm/MPS/XPU, native custom operators, memory fit or model quality. | +| SSD / Storage / RAM | Diagnose / verify: serialization trust and state identity, checkpoint ownership, distributed restore and capture/staging/upload completion boundaries | `pytorch`: [distributed/checkpoint route](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md); existing [DCP async-save owner](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/dcp-async-save-performance/SKILL.md) retains its separate source pin | No source-build checkpoint run, backend durability, distributed recovery or measured storage benefit is established. A state-dict round trip in a separate installed wheel does not qualify those properties. | +| Communication / Network / Accelerator | Diagnose / verify: process/rank/device mapping, backend and collective schedule, DDP/FSDP2/DTensor state and completion | `pytorch`: [distributed/checkpoint route](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md), [source and runtime boundary](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/repo-provenance.md) | Selected source and CPU controls do not establish NCCL or other accelerator transport, multi-device/multi-node correctness, overlap or throughput. Required backend checks remain `BLOCKED_REQUIRED_BACKEND`; no new measured/green resource cell, OS control or compatible latest PyTorch/TorchNPU stack is asserted. | ## vLLM Ascend v0.27.1rc1 serving diagnosis | Resource | Stage / mechanism | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| Accelerator / Host | Diagnose engine-process plugin discovery and MiniMax-M3 mixed KV storage from hardware capability through per-layer resolution | [vllm-ascend](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md), [cache path](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/minimax-cache.md) | Entry-point metadata and cache specs can be inspected separately from device dispatch and numerics. | -| Communication / Accelerator | Verify DSpark draft budget, effective P/D ranks and target capture tiers | [vllm-ascend](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md), [DSpark PD](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/dspark-pd.md) | TP4 acceptance-length fixture and two-node 950DT deployment have separate evidence requirements. | -| Accelerator / RAM / Host | Diagnose fused preprocessing, grouped GEMM/activation/quant, residual specialization, KV index reuse and reverted optimizations | [vllm-ascend](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md), [fusion](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/fused-operators.md), [model performance](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/model-performance.md) | Whole-history discovery and selected semantic review remain separate from native performance validation. | +| Accelerator / Host | Diagnose engine-process plugin discovery and MiniMax-M3 mixed KV storage from hardware capability through per-layer resolution | [vllm-ascend](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/SKILL.md), [cache path](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/minimax-cache.md) | Entry-point metadata and cache specs can be inspected separately from device dispatch and numerics. | +| Communication / Accelerator | Verify DSpark draft budget, effective P/D ranks and target capture tiers | [vllm-ascend](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/SKILL.md), [DSpark PD](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/dspark-pd.md) | TP4 acceptance-length fixture and two-node 950DT deployment have separate evidence requirements. | +| Accelerator / RAM / Host | Diagnose fused preprocessing, grouped GEMM/activation/quant, residual specialization, KV index reuse and reverted optimizations | [vllm-ascend](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/SKILL.md), [fusion](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/fused-operators.md), [model performance](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/model-performance.md) | Whole-history discovery and selected semantic review remain separate from native performance validation. | ## DeepSeek streamed expert-cache verification | Resource | Stage / scoped question | Canonical owner | Evidence boundary | | --- | --- | --- | --- | -| SSD / Storage + RAM / Host | Diagnose / Verify: independent tensor-byte identity after aligned/coalesced reads, resident versus pending slot lifetime, ordered consumer accumulation and actual oracle callback coverage | [weight-loading-optimization](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md) ([expert-cache oracle reference](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-oracles.md)) and [source record](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-source.json); DeepSeek source at `d0402056d0da4bf7edfd2d4977de479a0db9d201` | Source-derived candidate / empirical not_run; byte oracle shares tensor metadata, resident snapshot covers w1, and tiny PyTorch/batched checks are source definitions. No measured resource gain, full-model quality, native/device execution or agent efficacy is established. | +| SSD / Storage + RAM / Host | Diagnose / Verify: independent tensor-byte identity after aligned/coalesced reads, resident versus pending slot lifetime, ordered consumer accumulation and actual oracle callback coverage | [weight-loading-optimization](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md) ([expert-cache oracle reference](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/deepseek-expert-cache-oracles.md)) and [source record](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/deepseek-expert-cache-source.json); DeepSeek source at `d0402056d0da4bf7edfd2d4977de479a0db9d201` | Source-derived candidate / empirical not_run; byte oracle shares tensor metadata, resident snapshot covers w1, and tiny PyTorch/batched checks are source definitions. No measured resource gain, full-model quality, native/device execution or agent efficacy is established. | ## Faster CosyVoice evaluation source context -The existing [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) -uses [dependency-aware TTS playout](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/dependency-aware-tts-playout.md) -and its [source record](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/faster-cosyvoice-source.json) +The existing [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) +uses [dependency-aware TTS playout](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/dependency-aware-tts-playout.md) +and its [source record](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/faster-cosyvoice-source.json) as methodology/source context for server PCM credit, client playback observations, chunk dependencies and partial cost denominators. This adds no numbered resource-layer mapping, lifecycle-stage mapping, supplementary @@ -656,7 +656,7 @@ not_evaluated; author reports are not local quality or service results. ## GPU measurement validity and profiler attribution -[GPU measurement validity and profiler attribution](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md) +[GPU measurement validity and profiler attribution](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/gpu-measurement-validity.md) extends `inference-benchmark-design` with clock/cache controls, conditional within-run attribution, work-count signatures and timing-trap checks. This is evaluation guidance, not a hardware-resource optimization mechanism or added @@ -666,12 +666,12 @@ observations are unverified and establish no device or service gain. ## Embodied.cpp benchmark methodology -The [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) +The [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) method uses [Embodied.cpp at 156bed15](https://github.com/SEU-PAISys/Embodied.cpp/tree/156bed151826780e2ce0132f8d0e3540889ae8cd) and [arXiv:2607.02501v3](https://arxiv.org/abs/2607.02501v3) as source context for observation-to-action timing, generated versus replayed actions, precision -identity and per-task closed-loop quality. Its [operating reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md) -and [readset](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json) +identity and per-task closed-loop quality. Its [operating reference](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md) +and [readset](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/embodied-cpp-source.json) retain the later author's measurement and source-version limits. These are comparison methods. Experiment hardware is source context rather diff --git a/docs/skill-coverage.md b/docs/skill-coverage.md index bcd78b5ae..ad1b8b1ca 100644 --- a/docs/skill-coverage.md +++ b/docs/skill-coverage.md @@ -1,10 +1,10 @@ # Skill coverage view -Generated from the registry, the skill sidecars and the two maintained +Generated from the registry, SKILL.md frontmatter and the two maintained matrices. **Do not edit by hand**; regenerate after any registry, skill, matrix or frozen-plan change. -- Input digest: `sha256:057218d1284e26df8db68c1177abd0a2ff73edbb175adc8441fe92a66a79e124` +- Input digest: `sha256:93ad79f2b36af340d3c1c819dcb2f9b651d0f6078913aa726bd9ee1f01e9df54` - Regenerate: `python tools/build_skill_coverage.py` - Verify without writing: `python tools/build_skill_coverage.py --check` @@ -21,10 +21,10 @@ matrix: every field is copied from an owner that already maintains it. | Field | Owner of the fact | |---|---| | Skill id, canonical path, backend | [`skill-index.json`](../src/inferrsi/knowledge/data/skill_library/skill-index.json) | -| Declared task, status, validation state, provenance readsets | the skill's `SKILL.md` frontmatter and `metadata.json` sidecar | -| Declared applicability conditions, workflow input/output/failure/refusal | `applicability` in the same `metadata.json` sidecar; descriptive, not evidential | +| Declared task, status, validation state, provenance readsets | the skill's `SKILL.md` frontmatter | +| Declared applicability conditions, workflow input/output/failure/refusal | `applicability` in the same frontmatter; descriptive, not evidential | | Resource layer, lifecycle stage | [SearchSpace Matrix](searchspace-matrix.md) | -| Reasoned resource-layer exclusion | `searchspace_not_applicable` in the sidecar | +| Reasoned resource-layer exclusion | `searchspace_not_applicable` in the frontmatter | | Source identity and inspected scope | [Reference Matrix](reference-matrix.md) | | Evaluation selection | frozen `evidence//plan/protocol-*.json` | | Delivery PR and issue | not derived here; see [issue #91](https://github.com/InferMatrixLab/InferRSI/issues/91) and `docs/reviews/issue91-*.md` | @@ -33,128 +33,135 @@ matrix: every field is copied from an owner that already maintains it. | Measure | Value | |---|---| -| Registered skill ids | 80 | -| By category | frameworks 5, hardware 22, methods 20, repositories 27, workflows 6 | -| By backend | ascend 16, common 59, cuda 5 | +| Registered skill ids | 88 | +| By category | casebooks 2, playbooks 17, systems 53, tasks 8, workflows 8 | +| By backend | ascend 16, common 67, cuda 5 | | Recorded on a numbered SearchSpace resource layer | 51 | | Recorded only in a supplementary guidance table (†) | 21 | -| Resource layers explicitly not applicable | 7 | +| Resource layers explicitly not applicable | 15 | | Conflicting resource-layer declarations | 0 | | Recorded on two or more resource layers | 24 | | Recorded in the Reference Matrix | 72 | -| With a `metadata.json` sidecar | 80 | | Selected by a frozen evaluation plan | 2 | -| `SKILL.md` documents in the library | 143 | +| `SKILL.md` documents in the library | 151 | | Unregistered documents (excluded from every count above) | 63 | A complete inventory is not a coverage claim. Every registered row below has an entry; that says the listing is exhaustive, not that inference optimization is fully covered or that any procedure has measured benefit. -## Registered: Hardware and platform (22) +## Registered: Casebooks and prior outcomes (2) | Skill | Backend | Declared task | Applicability | Conditions | SearchSpace layer / stage | Reference Matrix | State | Gaps | |---|---|---|---|---|---|---|---|---| -| [ascend-aclgraph-capture-replay](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md) | ascend | Diagnose vLLM-Ascend ACLGraph capture/replay eligibility, stable-input and ordering contracts, and capture-resource ... | Diagnose vLLM-Ascend ACLGraph eligibility, batch descriptor matching, capture/replay ... | input_contract, produced_artifact, refusal_point
5 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (vllm-project/vllm-ascend @ `8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf` ...) | candidate · not_run · 1 readset(s) | no-eval | -| [ascend-device-health](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md) | ascend | Pre-run health checks and recovery procedures for Ascend NPU devices on shared machines: card-group topology ... | Pre-run Ascend device-health, topology, occupancy and timing checks before a trusted ... | failure_modes, input_contract, produced_artifact
3 advice | 4 Network · Diagnose
5 OS · Observe / Profile / Diagnose / Configure / Verify | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39) | candidate · not_run · 0 readset(s) | no-eval | -| [ascend-distributed-parallel](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md) | ascend | Select and validate distributed parallelism strategy on Ascend NPU: topology-aware card-group selection, USP/CP/TP/PP ... | Select and diagnose Ascend parallel strategies and their activation/overlap, with a ... | failure_modes, input_contract, produced_artifact
4 advice | 1 Accelerator · Configure
4 Network · Diagnose / Configure / Optimize / Verify / Observe | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39)
1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c)
1 · Systems sources (cann/hccl @ `96fab0a2acb761e7f8ab0f94aeb5138f200a345b` (CANN Open ...) | candidate · not_run · 3 readset(s) | no-eval | -| [ascend-graph-mode](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md) | ascend | Diagnose an Ascend TorchAir ACL graph path from versioned configuration through FX compilation, capture-key selection ... | Diagnose the TorchAir Python ACL graph path from configuration and FX compilation through ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Diagnose / Configure / Verify
3 SSD · Diagnose | 1 · Systems sources (Ascend/torchair @ `e1118e8b0fc1b95190ab53b732c0872989550e20` ...)
1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c) | candidate · not_run · 3 readset(s) | no-eval | -| [ascend-model-adaptation](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md) | ascend | Patterns for porting LLM/VLM model adapters to Ascend NPU: FP8 checkpoint handling, KV replication with TP, multimodal ... | Checkpoint onboarding for LLM/VLM adapters on Ascend: weight/scale loading, KV-head ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Configure
3 SSD · Diagnose / Configure | 1 · Systems sources (vllm-project/vllm-ascend @ bc8115ed) | candidate · not_run · 1 readset(s) | no-eval | -| [ascend-operator-profiling](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md) | ascend | Benchmark a single Ascend NPU operator using msprof op: build, collect with warmup, read the 8 CSV metrics files ... | Build, measure and diagnose one Ascend operator in isolation using msprof op, before ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Observe / Profile
5 OS · Observe / Profile | 1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c) | candidate · not_run · 0 readset(s) | no-eval | -| [ascend-operator-tuning](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md) | ascend | Run a staged, accuracy-gated tuning campaign on a single Ascend NPU operator: set a numeric target up front, block ... | Run a bounded, accuracy-gated tuning campaign for a single Ascend operator already ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Optimize | 1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c) | candidate · not_run · 0 readset(s) | no-eval | -| [ascend-optimization-casebook](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md) | ascend | Casebook of NPU optimization cases for Ascend inference systems: symptom → diagnosis → action → evidence records for ... | Find source-derived prior cases for an already classified Ascend inference bottleneck ... | input_contract, produced_artifact, refusal_point
2 advice | 1 Accelerator · Optimize / Verify | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39) | candidate · not_run · 0 readset(s) | no-eval | -| [ascend-performance-analysis](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md) | ascend | Diagnose performance bottlenecks on Ascend NPU systems using CANN profiling data. Maps GPU profiling concepts (nsys ... | Interpret Ascend CANN profiling evidence from step breakdown through ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Observe / Profile / Diagnose / Verify
4 Network · Observe / Profile / Diagnose / Verify
5 OS · Diagnose | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39)
1 · Systems sources (cann/cannbot @ `c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110` (selected ...) | candidate · not_run · 1 readset(s) | no-eval | -| [ascend-video-diffusion-optimization](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/SKILL.md) | ascend | Run and verify goal-driven Ascend video diffusion optimization. Specializes the backend-agnostic ... | Ascend NPU video diffusion inference and serving optimization | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: Video diffusion campaign guidance (unvalidated) | — | candidate · not_run · 1 readset(s) | no-Reference-Matrix-row, no-eval | -| [ascendc-tiling-design](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md) | ascend | Design tiling for a custom AscendC operator: normalize the shape first (axis-collapse or flatten), answer the ... | Design or restructure AscendC tiling for the covered reduction, broadcast and elementwise ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Optimize | 1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c) | candidate · not_run · 0 readset(s) | no-eval | -| [cann-attribution-overlap](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md) | ascend | Attribute NPU trace kernels to model source and plan bounded multistream overlap: select a representative step, map ... | Attribute representative CANN trace kernels to source owners and plan bounded multistream ... | failure_modes, input_contract, produced_artifact
4 advice | 1 Accelerator · Observe / Profile / Diagnose / Optimize / Verify
4 Network · Diagnose / Optimize / Verify | 1 · Systems sources (cann/oam-tools @ `8440935f` (!302 import; later master `f3cf4115` ...) | candidate · not_run · 0 readset(s) | no-eval | -| [cgroup-resource-controls](../src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md) | common | Diagnose and configure cgroup v2 resource controls for serving/training workloads: distinguish quota stalls from ... | Audit Linux cgroup v2 CPU quota/shares, effective placement, memory pressure/protection ... | failure_modes, input_contract, produced_artifact
4 advice | 2 RAM/Host · Observe / Profile / Configure / Verify
5 OS · Observe / Profile / Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Linux cgroup v2 docs — kernel.org live 2026-10-15 (revision ...) | candidate · not_run · 0 readset(s) | no-eval | -| [diagnose-attention-bottlenecks](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md) | cuda | Diagnose the next attention-kernel bottleneck after IO reduction using evidence about HBM traffic, work ownership ... | Diagnose attention-kernel bottlenecks after IO reduction, tile/pipeline regressions and ... | input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Optimize / Verify | 1 · Systems sources (FlashAttention — [2205.14135v2](https://arxiv.org/abs/2205.14135v2)) | candidate · not_run · 9 readset(s) | no-eval | -| [host-memory-management](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/SKILL.md) | common | Audit pinned host buffers and asynchronous CUDA transfers: distinguish copy ordering, host mutation/reuse and device ... | Audit host preparation, pinned-host pressure and asynchronous CUDA-copy ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host · Configure / Verify | 1 · Systems sources (python/cpython @ `60403a5409ff2c3f3b07dd2ca91a7a3e096839c7` (v3.13.0 ...)
1 · Systems sources (pytorch/pytorch @ `ba56102387ef21a3b04b357e5b183d48f0afefc7` (v2.8.0 ...)
1 · Systems sources (vllm-project/vllm-ascend @ `8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf` ...) | candidate · not_run · 4 readset(s) | no-eval | -| [host-overhead-optimization](../src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md) | common | Profile and reduce CPU/host overhead in inference serving using line_profiler. Iterative ... | Use line-level profiling to reduce exposed Python request-preparation, scheduling and ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host · Optimize / Diagnose / Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval | -| [host-overhead-profiling](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md) | cuda | Detect and diagnose CPU/host overhead in inference serving from nsys traces. Answers: is host overhead the bottleneck? ... | Detect exposed host overhead and diagnose per-step host regressions in CUDA inference ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host · Observe / Profile / Diagnose | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval | -| [os-process-placement](../src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/SKILL.md) | common | Place and configure inference/training processes at the OS level before measuring or optimizing: check then pin CPU ... | Prepare and record CPU/NUMA process placement, phase-specific thread settings, device ... | failure_modes, input_contract, produced_artifact
3 advice | 5 OS · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c)
1 · Systems sources (ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT ...)
1 · Systems sources (vllm-project/vllm-ascend @ bc8115ed) | candidate · not_run · 1 readset(s) | no-eval | -| [reason-hopper-kernels](../src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md) | cuda | Reason about Hopper SM90 CUDA GEMM kernels using TMA and mbarrier lifetimes, WGMMA warpgroup participation, register ... | Source-derived Hopper SM90/SM90a kernel reasoning for TMA/mbarrier, WGMMA ... | architectures, excluded_mechanisms, failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: Hopper kernel reasoning (source-derived) | 1 · Systems sources (NVlabs/kda @ `acdd79343c5e9d0c7cb96289f26fd3f1051ce785` → ...) | candidate · not_run · 3 readset(s) | no-eval | -| [storage-io-measurement](../src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/SKILL.md) | common | Measure storage I/O for LLM/diffusion serving without conflating device metrics with application outcomes: match the ... | Measure and attribute storage traffic for model loading or serving while separating ... | input_contract, produced_artifact, refusal_point
4 advice | 3 SSD · Observe / Profile / Diagnose / Verify | 1 · Systems sources (axboe/fio @ acb7a302)
1 · Systems sources (sysstat/sysstat @ efd0c445) | candidate · not_run · 0 readset(s) | no-eval | -| [torchnpu-device-memory-lifecycle](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md) | ascend | Diagnose TorchNPU device-HBM reuse, cross-stream lifetimes, cache retention and fragmentation, then propose ... | Diagnose TorchNPU device-HBM reuse, stream lifetimes, fragmentation and ... | input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Ascend/pytorch (TorchNPU) @ `67511c53cd4c88014f6123de7097677e60e85faa` ...) | candidate · not_run · 1 readset(s) | no-eval | -| [torchnpu-pinned-host-memory-lifecycle](../src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md) | ascend | Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy lifetime, reuse and cache controls, then propose ... | Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy ownership/readiness ... | input_contract, produced_artifact, refusal_point
3 advice | 2 RAM/Host · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Ascend/pytorch (TorchNPU) @ `67511c53cd4c88014f6123de7097677e60e85faa` ...) | candidate · not_run · 1 readset(s) | no-eval | - -## Registered: Frameworks (5) +| [ascend-optimization-casebook](../src/inferrsi/knowledge/data/skill_library/casebooks/ascend/ascend-optimization-casebook/SKILL.md) | ascend | Casebook of NPU optimization cases for Ascend inference systems: symptom → diagnosis → action → evidence records for ... | Find source-derived prior cases for an already classified Ascend inference bottleneck ... | input_contract, produced_artifact, refusal_point
2 advice | 1 Accelerator · Optimize / Verify | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39) | candidate · not_run · 0 readset(s) | no-eval | +| [inference-optimization-casebook](../src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | Structuring and querying a casebook of past optimization decisions as decision support ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval | + +## Registered: Engineering playbooks (17) | Skill | Backend | Declared task | Applicability | Conditions | SearchSpace layer / stage | Reference Matrix | State | Gaps | |---|---|---|---|---|---|---|---|---| -| [dcp-async-save-performance](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md) | common | Diagnose training-visible checkpoint pauses using PyTorch Distributed Checkpoint asynchronous saving and staging ... | Source-supported diagnosis of training-visible PyTorch DCP save pauses ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize
2 RAM/Host · Observe / Profile / Configure / Optimize / Verify † guidance: PyTorch framework source procedures
3 SSD · Observe / Profile / Configure / Optimize / Verify † guidance: PyTorch framework source procedures | 1 · Systems sources (C1 / DCP async-save-only approval — pytorch/pytorch @ ...) | candidate · not_run · 1 readset(s) | no-eval | -| [diffusers-invocation-state](../src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md) | common | Trace Diffusers pipeline-wrapper and scheduler invocation state when repeated calls, from_pipe reuse or interruption ... | FluxPipeline and FlowMatchEulerDiscreteScheduler invocation ownership at this source pin ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 2 RAM/Host † guidance: Diffusers invocation guidance (source-derived) | Diffusers invocation source extraction (huggingface/diffusers @ `aea7b70be0f594bcdc15a6f2677ee2ca85e49e8e` ...) | candidate · not_run · 1 readset(s) | no-eval | -| [lazy-evaluation-stream-lifetimes](../src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) | common | Diagnose MLX lazy evaluation, cross-stream ordering and array-buffer lifetimes when timing, memory use or asynchronous ... | MLX evaluation/stream/buffer diagnosis at the pinned implementation; qualify the actual ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: MLX framework guidance (source-derived) | MLX framework source extraction (ml-explore/mlx @ `bff10220c742907f0c68e55ff174aaea56f74e3d` (MIT)) | candidate · not_run · 1 readset(s) | no-eval | -| [pytorch](../src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/SKILL.md) | common | Operate and diagnose PyTorch installation, tensors and autograd, training and data, compilation and export, accelerator ... | Major user workflows and selected deep mechanisms of pinned pytorch/pytorch source; exact ... | failure_modes, input_contract, produced_artifact, refusal_point | 1 Accelerator † guidance: PyTorch framework source procedures
2 RAM/Host † guidance: PyTorch framework source procedures
3 SSD † guidance: PyTorch framework source procedures
4 Network † guidance: PyTorch framework source procedures | PyTorch framework operating graph (pytorch/pytorch @ `9a90a4fefcaafa63ac984621963a6fcc66aee68c`, source ...) | candidate · not_run · 1 readset(s) | no-eval | -| [torch-npu](../src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md) | ascend | Operate and diagnose TorchNPU on Ascend across installation, PrivateUse1 dispatch, operator development, memory and ... | Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack ... | failure_modes, input_contract, produced_artifact, refusal_point | 1 Accelerator † guidance: TorchNPU framework source procedures
2 RAM/Host † guidance: TorchNPU framework source procedures
4 Network † guidance: TorchNPU framework source procedures | TorchNPU framework operating graph (Ascend/pytorch @ `046934140b96836c3fcef14d0f39ae54dc0a06bf` ...) | candidate · not_run · 2 readset(s) | no-eval | - -## Registered: Repositories (27) +| [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/SKILL.md) | common | Assess whether LLM serving queue delays justify token-boundary priority preemption, accounting for suspended KV, host ... | Diagnosing whether LLM serving queue delay justifies token-boundary priority preemption ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify
2 RAM/Host · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Orca — OSDI 2022 proceedings) | candidate · not_run · 1 readset(s) | no-eval | +| [assess-split-backward-schedule](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-split-backward-schedule/SKILL.md) | common | Evaluate memory-capped split-backward and V-shaped pipeline scheduling proposals, preserving synchronous update ... | Assessing memory-capped split-backward and V-shaped pipeline schedule proposals when a ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Zero Bubble — [2401.10241v1](https://arxiv.org/abs/2401.10241v1)) | candidate · not_run · 1 readset(s) | no-eval | +| [audit-fixed-workload-claim](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/audit-fixed-workload-claim/SKILL.md) | common | Audit a performance claim from supplied fixed-workload reports and an explicit measurement contract. Check aggregation ... | Auditing a performance claim against supplied fixed-workload reports and an explicit ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | N/A: SKILL.md §§ Preserve the observation hierarchy, Check what was actually compared, and Return an auditable conclusion audit supplied measurements and verdicts; device identity is comparison metadata, not a hardware-resource mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [calibrate-serving-simulation](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/SKILL.md) | common | Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical ... | Building an evidence-gated serving-configuration shortlist when a profiled simulator or ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Verify | 1 · Systems sources (Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2))
1 · Systems sources ([AMD Infera @ ff1136e4](https://github.com/AMD-AGI/Infera/tree/ff1136e ...) | candidate · not_run · 3 readset(s) | no-eval | +| [checkpoint-recovery-consistency](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/SKILL.md) | common | Assess distributed checkpoint recovery using DLRover memory agreement, persistence completion and replica transport ... | Assessing distributed checkpoint recovery through DLRover memory agreement, persistence ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 2 RAM/Host · Configure / Verify
3 SSD · Configure / Verify
4 Network · Diagnose / Configure / Verify
5 OS · Diagnose / Configure | 1 · Systems sources (C3 / DLRover FlashCheckpoint @ `e35b1ad90e2928ca6bacf5d22117f71356 ...)
1 · Systems sources (C4 / NVIDIA Resiliency Extension @ ...) | candidate · not_run · 1 readset(s) | no-eval | +| [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify
4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval | +| [dense-performance-evidence](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Observe / Profile / Verify
2 RAM/Host · Verify
4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)
2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — | +| [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) | common | Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including ... | Comparing migration of a running LLM request between compatible model replicas with ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify
2 RAM/Host · Diagnose / Configure / Optimize / Verify
4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1)) | candidate · not_run · 1 readset(s) | no-eval | +| [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point
9 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)
2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)
2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)
GPU measurement validity and profiler attribution (NVIDIA NCU 2025.3, Nsight Systems 2025.3, CUDA 12.9.1/10.2 archives ...)
Task-specific benchmark selection (issue #60) ([Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/15 ...)
Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)
Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...)
Task-specific benchmark selection (issue #60) ([yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizha ...) | candidate · not_run · 16 readset(s) | — | +| [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton ... | failure_modes, input_contract, produced_artifact, refusal_point
6 advice | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)
1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)
1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)
1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)
1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-eval | +| [optimization-search-policy](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | Selecting, ordering and stopping optimization experiments: candidate generation ... | failure_modes, input_contract, produced_artifact, refusal_point
5 advice | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)
2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)
2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))
2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-eval | +| [performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md) | common | Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical ... | inference serving systems (LLM, diffusion, audio) | 2 advice | 1 Accelerator · Diagnose / Verify
5 OS · Configure | 1 · Systems sources (FlexGen — [ICML 2023 / PMLR 202:31094–31116](https://proceedings.mlr.p ...)
1 · Systems sources (Mooncake — FAST '25 proceedings, pp. 155-170)
1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
1 · Systems sources (ebugger/Empirical-Roofline-Toolkit @ 0771fa9)
1 · Systems sources (hahnyuan/LLM-Viewer @ 1893e4b)
2 · Methodology and corpus references ([shenh10/HuggingArch](https://github.com/shenh10/HuggingArch/tree/facb ...) | candidate · not_run · 20 readset(s) | no-eval | +| [plan-activation-residency](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/SKILL.md) | common | Propose training activation and gradient memory reductions by distinguishing lifetime reordering, address packing and ... | Proposing training activation and gradient memory reductions when pressure appears to ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Optimize / Verify | 1 · Systems sources (Checkmate — [1910.02653v3](https://arxiv.org/abs/1910.02653v3)) | candidate · not_run · 1 readset(s) | no-eval | +| [training-input-pipeline](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/training-input-pipeline/SKILL.md) | common | Diagnose training input stalls and choose bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic ... | Diagnosing training input stalls and choosing bounded queue, shard-cache and ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Observe / Configure / Optimize
2 RAM/Host · Observe / Diagnose / Configure / Optimize
3 SSD · Diagnose / Configure / Optimize
4 Network · Configure / Optimize
5 OS · Configure / Optimize | 1 · Systems sources (B1 / NVIDIA DALI @ `6fe47c48d3b0ca335f5e1afd3c77c750b2c7e041` ...)
1 · Systems sources (B2 / Mosaic Streaming @ `d99bf9c7cdf2dd4ed62a4960ed35270b93337a5f` ...)
1 · Systems sources (B3 / FFCV @ `3a12966b3afe3a81733a732e633317d747bfaac7` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [training-resource-budget](../src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/SKILL.md) | common | Choose training offload and resource-budget changes using pinned DeepSpeed configuration, attributed expert diagnosis ... | Choosing training offload and resource-budget changes from pinned DeepSpeed ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Diagnose / Configure / Optimize
2 RAM/Host · Observe / Diagnose / Configure / Optimize / Verify
3 SSD · Configure / Optimize
4 Network · Observe / Diagnose / Verify
5 OS · Diagnose / Configure | 1 · Systems sources (CR1 / DeepSpeed ZeRO-Infinity @ `5f266e26d216a705d9c64cac88fb813ce ...)
1 · Systems sources (CR5 / Ascend MindSpeed adaptive memory — historical `core_r0.8.0` ...)
1 · Systems sources (ZeRO-Offload — [2101.06840v1](https://arxiv.org/abs/2101.06840v1))
2 · Methodology and corpus references (CR6 / Stas Bekman ML Engineering Open Book @ ...) | candidate · not_run · 9 readset(s) | no-eval | +| [video-diffusion-optimization](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/SKILL.md) | common | Run and verify goal-driven video diffusion optimization, backend-agnostic. Use when the user asks for latency, RTF ... | Backend-agnostic video diffusion inference and serving optimization | — | 1 Accelerator † guidance: Video diffusion campaign guidance (unvalidated) | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [weight-loading-optimization](../src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md) | common | Optimize the serving-level weight load path from storage to device: judge the load path on four dimensions (source ... | Diagnosing the serving-level weight load path from storage to device and choosing changes ... | failure_modes, input_contract, produced_artifact, refusal_point
6 advice | 1 Accelerator † guidance: Flare preparation and complete-session source contracts
2 RAM/Host † guidance: Flare preparation and complete-session source contracts; DeepSeek streamed expert-cache verification
3 SSD · Observe / Profile / Diagnose / Optimize / Verify † guidance: Flare preparation and complete-session source contracts; DeepSeek streamed expert-cache verification
4 Network · Configure / Optimize / Verify | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39)
1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c)
1 · Systems sources (foundation-model-stack/fastsafetensors @ 219e256)
1 · Systems sources (ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT ...)
1 · Systems sources (mindie-agent/mindie-agent @ 6a81b41)
DeepSeek streamed expert-cache oracle contracts ([ronak-create/deepseek-v4-in-c](https://github.com/ronak-create/deepse ...) | candidate · not_run · 5 readset(s) | no-eval | + +## Registered: Systems and implementations (53) | Skill | Backend | Declared task | Applicability | Conditions | SearchSpace layer / stage | Reference Matrix | State | Gaps | |---|---|---|---|---|---|---|---|---| -| [audit-afd-p2p-exchanges](../src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) | cuda | Audit CUDA P2pNccl Attention/FFN rank namespaces and per-stage, per-peer hidden-state ownership when uneven worker ... | Source-based CUDA P2pNccl rank/payload audit; selected eager dense-FFN path with ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Verify | 1 · Systems sources (vllm-project/afd-plugin @ `df8d6a0c403c3ca1b4ed9cf1687e7204908bebec` ...) | candidate · not_run · 1 readset(s) | no-eval | -| [audit-flare-session-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md) | cuda | Audit Flare complete prefix/session restoration, inbound transfer tickets, publication/use/retirement fences and ... | Pinned Flare complete-session restore and admission audit for its NVIDIA SM120/SM121 ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: Flare preparation and complete-session source contracts
2 RAM/Host † guidance: Flare preparation and complete-session source contracts
3 SSD † guidance: Flare preparation and complete-session source contracts | Flare preparation and complete-session source knowledge (NVIDIA/flare-local-inference → [NVIDIA/local-inference](https://github ...) | candidate · not_run · 2 readset(s) | no-eval | -| [audit-h3-metal-fasth3-semantics](../src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) | common | Audit h3-metal FastH3 effective configuration, checkpoint tensor and schedule semantics, partial VSA tiles, sparse ... | h3-metal FastH3 requested/effective configuration, checkpoint transformation and ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Verify | 1 · Systems sources (zuiho-kai/h3-metal @ `4ba47a3eabdbebf801ac6664b31f8ec168bc57a8` ...) | candidate · not_run · 3 readset(s) | no-eval | -| [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md) | common | Audit MiniMax-H3 generation in h3.c on Apple Silicon by tracing conditioning, AdaLN precompute, DiT residency, Metal ... | MiniMax-H3 joint audio/video C/Metal runtime audit: raw checkpoint compatibility, input ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Verify
2 RAM/Host · Observe / Diagnose / Verify
3 SSD · Diagnose / Configure / Verify | 1 · Systems sources (antirez/h3.c @ `8974cc055ea9c02fcd14cc27dfda3e1027c05153` (MIT ...) | candidate · not_run · 3 readset(s) | no-eval | -| [audit-llama-runtime-state](../src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md) | common | Audit llama.cpp context memory, partial decode progress, graph reuse and server cancellation when a retry, stale ... | Pinned llama.cpp context-memory, graph-reuse and server task-lifecycle diagnosis ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Verify
2 RAM/Host · Diagnose / Verify | 1 · Systems sources (ggml-org/llama.cpp @ `707616feb5f379e99ff7442680a55b4a1704c7c9` (MIT ...) | candidate · not_run · 1 readset(s) | no-eval | -| [audit-recurrent-kv-finalization](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) | common | Audit vLLM-RLT synchronous early-exit KV propagation, later depth-specific attention reads, and request storage ... | Pinned vLLM-RLT native Ouro synchronous scheduling with host KVMetadata: LAST_EXITED ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Verify | 1 · Systems sources (InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ ...) | candidate · not_run · 2 readset(s) | no-eval | -| [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | common | Audit stable-diffusion.cpp image-generation memory pressure by separating catalog freshness and weight-byte delivery ... | Source-grounded audit of native image-generation weight/runtime residency, optional ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Verify
2 RAM/Host · Observe / Diagnose / Verify
3 SSD · Diagnose / Verify | 1 · Systems sources (leejet/stable-diffusion.cpp @ `b13a48219d5cdf408edcb93da6cd69b612ed03d ...) | candidate · not_run · 2 readset(s) | no-eval | -| [audit-sglang-diffusion-lifetimes](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | common | Diagnose SGLang multimodal_gen stage state, component residency and request timeout/deletion when grouped outputs ... | Bounded monolithic multimodal_gen composed-pipeline state and lifecycle diagnosis; actual ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Verify
2 RAM/Host · Diagnose / Verify | 1 · Systems sources (sgl-project/sglang @ `df13d5dce31abfeff0e701a62e8c39a136758e6c` ...) | candidate · not_run · 1 readset(s) | no-eval | -| [audit-vllm-kv-lifetimes](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md) | common | Audit vLLM V1 asynchronous KV load/save completion, block retention, preemption, and receive-failure contracts when ... | Pinned vLLM V1 scheduler/connector ownership audit for asynchronous KV load/save ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Verify | 1 · Systems sources (vllm-project/vllm @ `7f730b4cea9a66520a9200265928ebe016b48a39` ...) | candidate · not_run · 1 readset(s) | no-eval | -| [bound-windowed-attention-replay](../src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md) | common | Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved ... | Decoders with an explicit producer/consumer boundary and a demonstrated dependency ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: DeepInfer bounded source guidance
2 RAM/Host † guidance: DeepInfer bounded source guidance | DeepInfer source extraction (deepseek-ai/DeepInfer @ `8b16149741a4935e1dec8cea27853f273b374c5b` ...) | candidate · not_run · 2 readset(s) | no-eval | -| [budget-speculative-verification](../src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md) | common | Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an ... | Engines with multi-token speculative decoding whose verification forward cost grows with ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: DeepInfer bounded source guidance | DeepInfer source extraction (deepseek-ai/DeepInfer @ `8b16149741a4935e1dec8cea27853f273b374c5b` ...) | candidate · not_run · 2 readset(s) | no-eval | -| [comfy-ui](../src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/SKILL.md) | common | Audit ComfyUI workflow-output cache validity, lazy dependencies and model-weight residency as separate state contracts ... | Source-grounded ComfyUI workflow-output cache and model-weight ownership audit; confirm ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Verify
2 RAM/Host · Diagnose / Configure / Verify | 1 · Systems sources (Comfy-Org/ComfyUI @ `b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c` (GNU ...) | candidate · not_run · 1 readset(s) | no-eval | -| [diagnose-vllm-admission-pressure](../src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md) | common | Diagnose vLLM V1 admission stalls and KV-pressure preemption by separating token, input, active-request ... | Pinned vLLM V1 branch-based diagnosis of queued requests, low scheduled-token counts and ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Configure / Optimize / Verify | 1 · Systems sources (vllm-project/vllm @ `7f730b4cea9a66520a9200265928ebe016b48a39` ...) | candidate · not_run · 14 readset(s) | no-eval | -| [dwarfstar](../src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/SKILL.md) | common | Investigate DwarfStar model fit, expert SSD streaming, session and vision cache correctness, speculative sampling and ... | Pinned DwarfStar source investigation and experiment planning; not arbitrary GGUF support ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: DwarfStar source-derived execution/state graph
2 RAM/Host † guidance: DwarfStar source-derived execution/state graph
3 SSD † guidance: DwarfStar source-derived execution/state graph | DwarfStar portable AREX extraction (antirez/ds4 @ `ee7e8405232c8971000cc7ba2b945b00bf6d20d3`; AREX-Skill ...) | candidate · partial_cpu_only · 2 readset(s) | no-eval | -| [generation-cache-lifecycle](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md) | common | Trace MLX-LM prefill, generation batching, prompt-cache reuse and cooperative stopping when request state, cache rows ... | MLX-LM generation, batch-row and prefix-cache diagnosis; model/backend/runtime ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 2 RAM/Host † guidance: MLX-LM application guidance (source-derived) | MLX-LM application source extraction (ml-explore/mlx-lm @ `ab7f7048dc341ced2d84b1887eeb17d60b0d1b02` (MIT)) | candidate · not_run · 1 readset(s) | no-eval | -| [husky](../src/inferrsi/knowledge/data/skill_library/repositories/husky/SKILL.md) | common | Review Husky's official-document claims as research hypotheses, separating author reports, implementation evidence ... | Review Husky document claims and specify missing evidence; common catalog scope is not ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | **unknown** | Husky document-only research adaptation (Official https://husky.underdog.ai/; displayed2026-09-20 ...) | candidate · not_run · 2 readset(s) | no-SearchSpace-layer, no-eval | -| [kv-tier-management](../src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md) | common | Manage runtime KV cache tiers (GPU → CPU RAM → local disk) for LLM serving: decide when KV is stored, retained ... | LMCache multiprocess runtime KV tier policy across engine GPU cache, host L1 and ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host · Configure / Optimize
3 SSD · Optimize | 1 · Systems sources (LMCache/LMCache @ c3b875a) | candidate · not_run · 0 readset(s) | no-eval | -| [lily](../src/inferrsi/knowledge/data/skill_library/repositories/lily/SKILL.md) | common | Investigate Lily's exact Qwen3.6-35B-A3B MLX-affine-Q4 contract, Metal state and kernels, strict greedy text API ... | Exact Lily source/checkpoint/layout/API/cache/qualification investigation; common catalog ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: Lily exact-model source contracts
2 RAM/Host † guidance: Lily exact-model source contracts | Lily portable AREX extraction (perplexityai/pplx-garden `lily/` @ `1ed972ed3f0bd5616c997c9507c25616c6 ...) | candidate · not_run · 2 readset(s) | no-eval | -| [lithos-metal](../src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/SKILL.md) | common | Inspect Lithos Metal compilation and fusion, GPU-resident generation and DSpark state, or local serving and ... | Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (lithos-ai/lithos-metal @ `6653b7df5f2c4392ec7b038960f2796b1499923f` ...) | candidate · not_run · 1 readset(s) | no-eval | -| [local-dream](../src/inferrsi/knowledge/data/skill_library/repositories/local-dream/SKILL.md) | common | Audit Local Dream Android desired-versus-serving process state, remote exposure, native backend and DiT packaging/ABI ... | Selected Android app/process and native diffusion boundary contracts; common is a ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host † guidance: Local Dream Android diffusion source contracts | Local Dream Android diffusion boundary source (xororz/local-dream @ `db162df4d9bb6efc6595250423d8df26241f4c27` ...) | candidate · not_run · 2 readset(s) | no-eval | -| [mistral-rs](../src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/SKILL.md) | common | Investigate mistral.rs Metal paged-attention decode layouts, prefill coordinates and workspace, actual-model prewarm ... | Selected native Metal paged decode/prefill layout/metadata/workspace and ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: mistral.rs Metal paged-attention source contracts
2 RAM/Host † guidance: mistral.rs Metal paged-attention source contracts | mistral.rs Metal paged-attention extraction (EricLBuehler/mistral.rs @ `9459891e7697238225885518119682ac8209a612` ...) | candidate · not_run · 2 readset(s) | no-eval | -| [mlc-llm](../src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/SKILL.md) | common | Investigate MLC LLM compiled Metal artifact/schema, Apple target and pinned TVM module/storage/stream contracts, and ... | Selected artifact/Apple target/pinned TVM Metal consumer and KV configuration source ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: MLC LLM artifact, target and runtime source contracts
2 RAM/Host † guidance: MLC LLM artifact, target and runtime source contracts | MLC LLM compiled Metal and runtime contracts (mlc-ai/mlc-llm @ `8978ea9aec626d64096aaa92fbc89bda2e3ea802`; TVM ...) | candidate · not_run · 2 readset(s) | no-eval | -| [mlx-swift-lm](../src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/SKILL.md) | common | Investigate MLX Swift LM generation ownership, cancellation and settled task completion, and recurrent GatedDelta Metal ... | Selected Swift ownership/cancellation/settling and actual GatedDelta Metal/ops contracts ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: MLX Swift LM ownership and recurrent Metal source contracts
2 RAM/Host † guidance: MLX Swift LM ownership and recurrent Metal source contracts | MLX Swift LM lifecycle and recurrent Metal extraction (ml-explore/mlx-swift-lm @ `3d066d3a0ffdc7e70c476a3d31488ea62dfdd0ac` ...) | candidate · not_run · 2 readset(s) | no-eval | -| [project-maya](../src/inferrsi/knowledge/data/skill_library/repositories/project-maya/SKILL.md) | common | Investigate Project Maya model-family dispatch, representation and memory tiers, API versus native state, speculative ... | Model-specific dispatch/residency, API/native state and qualification investigation ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: Project Maya model-specific source contracts
2 RAM/Host † guidance: Project Maya model-specific source contracts
3 SSD † guidance: Project Maya model-specific source contracts | Project Maya source and CPU-slice extraction (mw00/project-maya @ `249f9a87d1a9d72bec2b684d704b0aa9296b1c9c` ...) | candidate · partial_cpu_only · 2 readset(s) | no-eval | -| [splash](../src/inferrsi/knowledge/data/skill_library/repositories/splash/SKILL.md) | common | Investigate Splash model-format compatibility, automatic memory planning, Metal and ANE execution, DFlash2 ... | Pinned Splash representation, planning, state/serving and verification investigation ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: Splash source-derived planning and state
2 RAM/Host † guidance: Splash source-derived planning and state
3 SSD † guidance: Splash source-derived planning and state | Splash portable AREX extraction (incoai/splash @ `39a97385f8560cf8a93972cec9b0a963c9195250`; AREX ...) | candidate · partial_cpu_only · 2 readset(s) | no-eval | -| [vllm-ascend](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md) | ascend | Analyze vLLM Ascend v0.27.1rc1 performance and commit evolution for DeepSeek-V4, Kimi K3, MiniMax-M3 and GLM5+, fused ... | Release-pinned performance diagnosis and historical optimization analysis for ... | input_contract, produced_artifact, refusal_point
5 advice | 1 Accelerator † guidance: vLLM Ascend v0.27.1rc1 serving diagnosis
2 RAM/Host † guidance: vLLM Ascend v0.27.1rc1 serving diagnosis | vLLM Ascend v0.27.1rc1 serving scope (vllm-project/vllm-ascend at `3b31886237ed65c435e0965b10e8002ff5769766`) | candidate · not_run · 3 readset(s) | no-eval | -| [vllm-metal](../src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/SKILL.md) | common | Investigate vllm-metal Apple inference through platform and native-ABI selection, paged KV/offload ownership ... | Selected plugin preparation, KV/offload ownership and speculative/deferred/dispatch ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: vLLM Metal selected source contracts
2 RAM/Host † guidance: vLLM Metal selected source contracts
3 SSD † guidance: vLLM Metal selected source contracts | vLLM Metal portable AREX source and CPU extraction (vllm-project/vllm-metal @ `e7df7a6ff215f99c7842552d230fdbc709835934` ...) | candidate · partial_cpu_only · 2 readset(s) | no-eval | - -## Registered: Methods (20) +| [ascend-aclgraph-capture-replay](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-aclgraph-capture-replay/SKILL.md) | ascend | Diagnose vLLM-Ascend ACLGraph capture/replay eligibility, stable-input and ordering contracts, and capture-resource ... | Diagnose vLLM-Ascend ACLGraph eligibility, batch descriptor matching, capture/replay ... | input_contract, produced_artifact, refusal_point
5 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (vllm-project/vllm-ascend @ `8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [ascend-device-health](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-device-health/SKILL.md) | ascend | Pre-run health checks and recovery procedures for Ascend NPU devices on shared machines: card-group topology ... | Pre-run Ascend device-health, topology, occupancy and timing checks before a trusted ... | failure_modes, input_contract, produced_artifact
3 advice | 4 Network · Diagnose
5 OS · Observe / Profile / Diagnose / Configure / Verify | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39) | candidate · not_run · 0 readset(s) | no-eval | +| [ascend-distributed-parallel](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/SKILL.md) | ascend | Select and validate distributed parallelism strategy on Ascend NPU: topology-aware card-group selection, USP/CP/TP/PP ... | Select and diagnose Ascend parallel strategies and their activation/overlap, with a ... | failure_modes, input_contract, produced_artifact
4 advice | 1 Accelerator · Configure
4 Network · Diagnose / Configure / Optimize / Verify / Observe | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39)
1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c)
1 · Systems sources (cann/hccl @ `96fab0a2acb761e7f8ab0f94aeb5138f200a345b` (CANN Open ...) | candidate · not_run · 3 readset(s) | no-eval | +| [ascend-graph-mode](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/SKILL.md) | ascend | Diagnose an Ascend TorchAir ACL graph path from versioned configuration through FX compilation, capture-key selection ... | Diagnose the TorchAir Python ACL graph path from configuration and FX compilation through ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Diagnose / Configure / Verify
3 SSD · Diagnose | 1 · Systems sources (Ascend/torchair @ `e1118e8b0fc1b95190ab53b732c0872989550e20` ...)
1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c) | candidate · not_run · 3 readset(s) | no-eval | +| [ascend-model-adaptation](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-model-adaptation/SKILL.md) | ascend | Patterns for porting LLM/VLM model adapters to Ascend NPU: FP8 checkpoint handling, KV replication with TP, multimodal ... | Checkpoint onboarding for LLM/VLM adapters on Ascend: weight/scale loading, KV-head ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Configure
3 SSD · Diagnose / Configure | 1 · Systems sources (vllm-project/vllm-ascend @ bc8115ed) | candidate · not_run · 1 readset(s) | no-eval | +| [ascend-operator-profiling](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-operator-profiling/SKILL.md) | ascend | Benchmark a single Ascend NPU operator using msprof op: build, collect with warmup, read the 8 CSV metrics files ... | Build, measure and diagnose one Ascend operator in isolation using msprof op, before ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Observe / Profile
5 OS · Observe / Profile | 1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c) | candidate · not_run · 0 readset(s) | no-eval | +| [ascend-operator-tuning](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-operator-tuning/SKILL.md) | ascend | Run a staged, accuracy-gated tuning campaign on a single Ascend NPU operator: set a numeric target up front, block ... | Run a bounded, accuracy-gated tuning campaign for a single Ascend operator already ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Optimize | 1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c) | candidate · not_run · 0 readset(s) | no-eval | +| [ascend-performance-analysis](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/SKILL.md) | ascend | Diagnose performance bottlenecks on Ascend NPU systems using CANN profiling data. Maps GPU profiling concepts (nsys ... | Interpret Ascend CANN profiling evidence from step breakdown through ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Observe / Profile / Diagnose / Verify
4 Network · Observe / Profile / Diagnose / Verify
5 OS · Diagnose | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39)
1 · Systems sources (cann/cannbot @ `c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110` (selected ...) | candidate · not_run · 1 readset(s) | no-eval | +| [ascend-video-diffusion-optimization](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-video-diffusion-optimization/SKILL.md) | ascend | Run and verify goal-driven Ascend video diffusion optimization. Specializes the backend-agnostic ... | Ascend NPU video diffusion inference and serving optimization | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: Video diffusion campaign guidance (unvalidated) | — | candidate · not_run · 1 readset(s) | no-Reference-Matrix-row, no-eval | +| [ascendc-tiling-design](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascendc-tiling-design/SKILL.md) | ascend | Design tiling for a custom AscendC operator: normalize the shape first (axis-collapse or flatten), answer the ... | Design or restructure AscendC tiling for the covered reduction, broadcast and elementwise ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Optimize | 1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c) | candidate · not_run · 0 readset(s) | no-eval | +| [audit-afd-p2p-exchanges](../src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) | cuda | Audit CUDA P2pNccl Attention/FFN rank namespaces and per-stage, per-peer hidden-state ownership when uneven worker ... | Source-based CUDA P2pNccl rank/payload audit; selected eager dense-FFN path with ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Verify | 1 · Systems sources (vllm-project/afd-plugin @ `df8d6a0c403c3ca1b4ed9cf1687e7204908bebec` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [audit-flare-session-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md) | cuda | Audit Flare complete prefix/session restoration, inbound transfer tickets, publication/use/retirement fences and ... | Pinned Flare complete-session restore and admission audit for its NVIDIA SM120/SM121 ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: Flare preparation and complete-session source contracts
2 RAM/Host † guidance: Flare preparation and complete-session source contracts
3 SSD † guidance: Flare preparation and complete-session source contracts | Flare preparation and complete-session source knowledge (NVIDIA/flare-local-inference → [NVIDIA/local-inference](https://github ...) | candidate · not_run · 2 readset(s) | no-eval | +| [audit-h3-metal-fasth3-semantics](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) | common | Audit h3-metal FastH3 effective configuration, checkpoint tensor and schedule semantics, partial VSA tiles, sparse ... | h3-metal FastH3 requested/effective configuration, checkpoint transformation and ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Verify | 1 · Systems sources (zuiho-kai/h3-metal @ `4ba47a3eabdbebf801ac6664b31f8ec168bc57a8` ...) | candidate · not_run · 3 readset(s) | no-eval | +| [audit-h3-runtime-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md) | common | Audit MiniMax-H3 generation in h3.c on Apple Silicon by tracing conditioning, AdaLN precompute, DiT residency, Metal ... | MiniMax-H3 joint audio/video C/Metal runtime audit: raw checkpoint compatibility, input ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Verify
2 RAM/Host · Observe / Diagnose / Verify
3 SSD · Diagnose / Configure / Verify | 1 · Systems sources (antirez/h3.c @ `8974cc055ea9c02fcd14cc27dfda3e1027c05153` (MIT ...) | candidate · not_run · 3 readset(s) | no-eval | +| [audit-llama-runtime-state](../src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md) | common | Audit llama.cpp context memory, partial decode progress, graph reuse and server cancellation when a retry, stale ... | Pinned llama.cpp context-memory, graph-reuse and server task-lifecycle diagnosis ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Verify
2 RAM/Host · Diagnose / Verify | 1 · Systems sources (ggml-org/llama.cpp @ `707616feb5f379e99ff7442680a55b4a1704c7c9` (MIT ...) | candidate · not_run · 1 readset(s) | no-eval | +| [audit-recurrent-kv-finalization](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) | common | Audit vLLM-RLT synchronous early-exit KV propagation, later depth-specific attention reads, and request storage ... | Pinned vLLM-RLT native Ouro synchronous scheduling with host KVMetadata: LAST_EXITED ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Verify | 1 · Systems sources (InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ ...) | candidate · not_run · 2 readset(s) | no-eval | +| [audit-sd-cpp-residency](../src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | common | Audit stable-diffusion.cpp image-generation memory pressure by separating catalog freshness and weight-byte delivery ... | Source-grounded audit of native image-generation weight/runtime residency, optional ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Verify
2 RAM/Host · Observe / Diagnose / Verify
3 SSD · Diagnose / Verify | 1 · Systems sources (leejet/stable-diffusion.cpp @ `b13a48219d5cdf408edcb93da6cd69b612ed03d ...) | candidate · not_run · 2 readset(s) | no-eval | +| [audit-sglang-diffusion-lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | common | Diagnose SGLang multimodal_gen stage state, component residency and request timeout/deletion when grouped outputs ... | Bounded monolithic multimodal_gen composed-pipeline state and lifecycle diagnosis; actual ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Verify
2 RAM/Host · Diagnose / Verify | 1 · Systems sources (sgl-project/sglang @ `df13d5dce31abfeff0e701a62e8c39a136758e6c` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [audit-vllm-kv-lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md) | common | Audit vLLM V1 asynchronous KV load/save completion, block retention, preemption, and receive-failure contracts when ... | Pinned vLLM V1 scheduler/connector ownership audit for asynchronous KV load/save ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Verify | 1 · Systems sources (vllm-project/vllm @ `7f730b4cea9a66520a9200265928ebe016b48a39` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [bound-windowed-attention-replay](../src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/SKILL.md) | common | Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved ... | Decoders with an explicit producer/consumer boundary and a demonstrated dependency ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: DeepInfer bounded source guidance
2 RAM/Host † guidance: DeepInfer bounded source guidance | DeepInfer source extraction (deepseek-ai/DeepInfer @ `8b16149741a4935e1dec8cea27853f273b374c5b` ...) | candidate · not_run · 2 readset(s) | no-eval | +| [budget-speculative-verification](../src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/SKILL.md) | common | Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an ... | Engines with multi-token speculative decoding whose verification forward cost grows with ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: DeepInfer bounded source guidance | DeepInfer source extraction (deepseek-ai/DeepInfer @ `8b16149741a4935e1dec8cea27853f273b374c5b` ...) | candidate · not_run · 2 readset(s) | no-eval | +| [cann-attribution-overlap](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/cann-attribution-overlap/SKILL.md) | ascend | Attribute NPU trace kernels to model source and plan bounded multistream overlap: select a representative step, map ... | Attribute representative CANN trace kernels to source owners and plan bounded multistream ... | failure_modes, input_contract, produced_artifact
4 advice | 1 Accelerator · Observe / Profile / Diagnose / Optimize / Verify
4 Network · Diagnose / Optimize / Verify | 1 · Systems sources (cann/oam-tools @ `8440935f` (!302 import; later master `f3cf4115` ...) | candidate · not_run · 0 readset(s) | no-eval | +| [cgroup-resource-controls](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/cgroup-resource-controls/SKILL.md) | common | Diagnose and configure cgroup v2 resource controls for serving/training workloads: distinguish quota stalls from ... | Audit Linux cgroup v2 CPU quota/shares, effective placement, memory pressure/protection ... | failure_modes, input_contract, produced_artifact
4 advice | 2 RAM/Host · Observe / Profile / Configure / Verify
5 OS · Observe / Profile / Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Linux cgroup v2 docs — kernel.org live 2026-10-15 (revision ...) | candidate · not_run · 0 readset(s) | no-eval | +| [comfy-ui](../src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/SKILL.md) | common | Audit ComfyUI workflow-output cache validity, lazy dependencies and model-weight residency as separate state contracts ... | Source-grounded ComfyUI workflow-output cache and model-weight ownership audit; confirm ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Verify
2 RAM/Host · Diagnose / Configure / Verify | 1 · Systems sources (Comfy-Org/ComfyUI @ `b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c` (GNU ...) | candidate · not_run · 1 readset(s) | no-eval | +| [dcp-async-save-performance](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/dcp-async-save-performance/SKILL.md) | common | Diagnose training-visible checkpoint pauses using PyTorch Distributed Checkpoint asynchronous saving and staging ... | Source-supported diagnosis of training-visible PyTorch DCP save pauses ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize
2 RAM/Host · Observe / Profile / Configure / Optimize / Verify † guidance: PyTorch framework source procedures
3 SSD · Observe / Profile / Configure / Optimize / Verify † guidance: PyTorch framework source procedures | 1 · Systems sources (C1 / DCP async-save-only approval — pytorch/pytorch @ ...) | candidate · not_run · 1 readset(s) | no-eval | +| [diagnose-attention-bottlenecks](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/SKILL.md) | cuda | Diagnose the next attention-kernel bottleneck after IO reduction using evidence about HBM traffic, work ownership ... | Diagnose attention-kernel bottlenecks after IO reduction, tile/pipeline regressions and ... | input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Optimize / Verify | 1 · Systems sources (FlashAttention — [2205.14135v2](https://arxiv.org/abs/2205.14135v2)) | candidate · not_run · 9 readset(s) | no-eval | +| [diagnose-vllm-admission-pressure](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/SKILL.md) | common | Diagnose vLLM V1 admission stalls and KV-pressure preemption by separating token, input, active-request ... | Pinned vLLM V1 branch-based diagnosis of queued requests, low scheduled-token counts and ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Observe / Diagnose / Configure / Optimize / Verify | 1 · Systems sources (vllm-project/vllm @ `7f730b4cea9a66520a9200265928ebe016b48a39` ...) | candidate · not_run · 14 readset(s) | no-eval | +| [diffusers-invocation-state](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md) | common | Trace Diffusers pipeline-wrapper and scheduler invocation state when repeated calls, from_pipe reuse or interruption ... | FluxPipeline and FlowMatchEulerDiscreteScheduler invocation ownership at this source pin ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 2 RAM/Host † guidance: Diffusers invocation guidance (source-derived) | Diffusers invocation source extraction (huggingface/diffusers @ `aea7b70be0f594bcdc15a6f2677ee2ca85e49e8e` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [dwarfstar](../src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/SKILL.md) | common | Investigate DwarfStar model fit, expert SSD streaming, session and vision cache correctness, speculative sampling and ... | Pinned DwarfStar source investigation and experiment planning; not arbitrary GGUF support ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: DwarfStar source-derived execution/state graph
2 RAM/Host † guidance: DwarfStar source-derived execution/state graph
3 SSD † guidance: DwarfStar source-derived execution/state graph | DwarfStar portable AREX extraction (antirez/ds4 @ `ee7e8405232c8971000cc7ba2b945b00bf6d20d3`; AREX-Skill ...) | candidate · partial_cpu_only · 2 readset(s) | no-eval | +| [generation-cache-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md) | common | Trace MLX-LM prefill, generation batching, prompt-cache reuse and cooperative stopping when request state, cache rows ... | MLX-LM generation, batch-row and prefix-cache diagnosis; model/backend/runtime ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 2 RAM/Host † guidance: MLX-LM application guidance (source-derived) | MLX-LM application source extraction (ml-explore/mlx-lm @ `ab7f7048dc341ced2d84b1887eeb17d60b0d1b02` (MIT)) | candidate · not_run · 1 readset(s) | no-eval | +| [host-memory-management](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/SKILL.md) | common | Audit pinned host buffers and asynchronous CUDA transfers: distinguish copy ordering, host mutation/reuse and device ... | Audit host preparation, pinned-host pressure and asynchronous CUDA-copy ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host · Configure / Verify | 1 · Systems sources (python/cpython @ `60403a5409ff2c3f3b07dd2ca91a7a3e096839c7` (v3.13.0 ...)
1 · Systems sources (pytorch/pytorch @ `ba56102387ef21a3b04b357e5b183d48f0afefc7` (v2.8.0 ...)
1 · Systems sources (vllm-project/vllm-ascend @ `8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf` ...) | candidate · not_run · 4 readset(s) | no-eval | +| [host-overhead-optimization](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/SKILL.md) | common | Profile and reduce CPU/host overhead in inference serving using line_profiler. Iterative ... | Use line-level profiling to reduce exposed Python request-preparation, scheduling and ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host · Optimize / Diagnose / Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval | +| [host-overhead-profiling](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/host-overhead-profiling/SKILL.md) | cuda | Detect and diagnose CPU/host overhead in inference serving from nsys traces. Answers: is host overhead the bottleneck? ... | Detect exposed host overhead and diagnose per-step host regressions in CUDA inference ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host · Observe / Profile / Diagnose | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval | +| [husky](../src/inferrsi/knowledge/data/skill_library/systems/engines/husky/SKILL.md) | common | Review Husky's official-document claims as research hypotheses, separating author reports, implementation evidence ... | Review Husky document claims and specify missing evidence; common catalog scope is not ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | **unknown** | Husky document-only research adaptation (Official https://husky.underdog.ai/; displayed2026-09-20 ...) | candidate · not_run · 2 readset(s) | no-SearchSpace-layer, no-eval | +| [kv-tier-management](../src/inferrsi/knowledge/data/skill_library/systems/engines/lmcache/kv-tier-management/SKILL.md) | common | Manage runtime KV cache tiers (GPU → CPU RAM → local disk) for LLM serving: decide when KV is stored, retained ... | LMCache multiprocess runtime KV tier policy across engine GPU cache, host L1 and ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host · Configure / Optimize
3 SSD · Optimize | 1 · Systems sources (LMCache/LMCache @ c3b875a) | candidate · not_run · 0 readset(s) | no-eval | +| [lazy-evaluation-stream-lifetimes](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) | common | Diagnose MLX lazy evaluation, cross-stream ordering and array-buffer lifetimes when timing, memory use or asynchronous ... | MLX evaluation/stream/buffer diagnosis at the pinned implementation; qualify the actual ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: MLX framework guidance (source-derived) | MLX framework source extraction (ml-explore/mlx @ `bff10220c742907f0c68e55ff174aaea56f74e3d` (MIT)) | candidate · not_run · 1 readset(s) | no-eval | +| [lily](../src/inferrsi/knowledge/data/skill_library/systems/engines/lily/SKILL.md) | common | Investigate Lily's exact Qwen3.6-35B-A3B MLX-affine-Q4 contract, Metal state and kernels, strict greedy text API ... | Exact Lily source/checkpoint/layout/API/cache/qualification investigation; common catalog ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: Lily exact-model source contracts
2 RAM/Host † guidance: Lily exact-model source contracts | Lily portable AREX extraction (perplexityai/pplx-garden `lily/` @ `1ed972ed3f0bd5616c997c9507c25616c6 ...) | candidate · not_run · 2 readset(s) | no-eval | +| [lithos-metal](../src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/SKILL.md) | common | Inspect Lithos Metal compilation and fusion, GPU-resident generation and DSpark state, or local serving and ... | Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (lithos-ai/lithos-metal @ `6653b7df5f2c4392ec7b038960f2796b1499923f` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [local-dream](../src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/SKILL.md) | common | Audit Local Dream Android desired-versus-serving process state, remote exposure, native backend and DiT packaging/ABI ... | Selected Android app/process and native diffusion boundary contracts; common is a ... | failure_modes, input_contract, produced_artifact
3 advice | 2 RAM/Host † guidance: Local Dream Android diffusion source contracts | Local Dream Android diffusion boundary source (xororz/local-dream @ `db162df4d9bb6efc6595250423d8df26241f4c27` ...) | candidate · not_run · 2 readset(s) | no-eval | +| [mistral-rs](../src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/SKILL.md) | common | Investigate mistral.rs Metal paged-attention decode layouts, prefill coordinates and workspace, actual-model prewarm ... | Selected native Metal paged decode/prefill layout/metadata/workspace and ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: mistral.rs Metal paged-attention source contracts
2 RAM/Host † guidance: mistral.rs Metal paged-attention source contracts | mistral.rs Metal paged-attention extraction (EricLBuehler/mistral.rs @ `9459891e7697238225885518119682ac8209a612` ...) | candidate · not_run · 2 readset(s) | no-eval | +| [mlc-llm](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/SKILL.md) | common | Investigate MLC LLM compiled Metal artifact/schema, Apple target and pinned TVM module/storage/stream contracts, and ... | Selected artifact/Apple target/pinned TVM Metal consumer and KV configuration source ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: MLC LLM artifact, target and runtime source contracts
2 RAM/Host † guidance: MLC LLM artifact, target and runtime source contracts | MLC LLM compiled Metal and runtime contracts (mlc-ai/mlc-llm @ `8978ea9aec626d64096aaa92fbc89bda2e3ea802`; TVM ...) | candidate · not_run · 2 readset(s) | no-eval | +| [mlx-swift-lm](../src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/SKILL.md) | common | Investigate MLX Swift LM generation ownership, cancellation and settled task completion, and recurrent GatedDelta Metal ... | Selected Swift ownership/cancellation/settling and actual GatedDelta Metal/ops contracts ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: MLX Swift LM ownership and recurrent Metal source contracts
2 RAM/Host † guidance: MLX Swift LM ownership and recurrent Metal source contracts | MLX Swift LM lifecycle and recurrent Metal extraction (ml-explore/mlx-swift-lm @ `3d066d3a0ffdc7e70c476a3d31488ea62dfdd0ac` ...) | candidate · not_run · 2 readset(s) | no-eval | +| [os-process-placement](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/os-process-placement/SKILL.md) | common | Place and configure inference/training processes at the OS level before measuring or optimizing: check then pin CPU ... | Prepare and record CPU/NUMA process placement, phase-specific thread settings, device ... | failure_modes, input_contract, produced_artifact
3 advice | 5 OS · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c)
1 · Systems sources (ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT ...)
1 · Systems sources (vllm-project/vllm-ascend @ bc8115ed) | candidate · not_run · 1 readset(s) | no-eval | +| [project-maya](../src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/SKILL.md) | common | Investigate Project Maya model-family dispatch, representation and memory tiers, API versus native state, speculative ... | Model-specific dispatch/residency, API/native state and qualification investigation ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: Project Maya model-specific source contracts
2 RAM/Host † guidance: Project Maya model-specific source contracts
3 SSD † guidance: Project Maya model-specific source contracts | Project Maya source and CPU-slice extraction (mw00/project-maya @ `249f9a87d1a9d72bec2b684d704b0aa9296b1c9c` ...) | candidate · partial_cpu_only · 2 readset(s) | no-eval | +| [pytorch](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/SKILL.md) | common | Operate and diagnose PyTorch installation, tensors and autograd, training and data, compilation and export, accelerator ... | Major user workflows and selected deep mechanisms of pinned pytorch/pytorch source; exact ... | failure_modes, input_contract, produced_artifact, refusal_point | 1 Accelerator † guidance: PyTorch framework source procedures
2 RAM/Host † guidance: PyTorch framework source procedures
3 SSD † guidance: PyTorch framework source procedures
4 Network † guidance: PyTorch framework source procedures | PyTorch framework operating graph (pytorch/pytorch @ `9a90a4fefcaafa63ac984621963a6fcc66aee68c`, source ...) | candidate · not_run · 1 readset(s) | no-eval | +| [reason-hopper-kernels](../src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/SKILL.md) | cuda | Reason about Hopper SM90 CUDA GEMM kernels using TMA and mbarrier lifetimes, WGMMA warpgroup participation, register ... | Source-derived Hopper SM90/SM90a kernel reasoning for TMA/mbarrier, WGMMA ... | architectures, excluded_mechanisms, failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: Hopper kernel reasoning (source-derived) | 1 · Systems sources (NVlabs/kda @ `acdd79343c5e9d0c7cb96289f26fd3f1051ce785` → ...) | candidate · not_run · 3 readset(s) | no-eval | +| [splash](../src/inferrsi/knowledge/data/skill_library/systems/engines/splash/SKILL.md) | common | Investigate Splash model-format compatibility, automatic memory planning, Metal and ANE execution, DFlash2 ... | Pinned Splash representation, planning, state/serving and verification investigation ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator † guidance: Splash source-derived planning and state
2 RAM/Host † guidance: Splash source-derived planning and state
3 SSD † guidance: Splash source-derived planning and state | Splash portable AREX extraction (incoai/splash @ `39a97385f8560cf8a93972cec9b0a963c9195250`; AREX ...) | candidate · partial_cpu_only · 2 readset(s) | no-eval | +| [storage-io-measurement](../src/inferrsi/knowledge/data/skill_library/systems/platforms/host/storage-io-measurement/SKILL.md) | common | Measure storage I/O for LLM/diffusion serving without conflating device metrics with application outcomes: match the ... | Measure and attribute storage traffic for model loading or serving while separating ... | input_contract, produced_artifact, refusal_point
4 advice | 3 SSD · Observe / Profile / Diagnose / Verify | 1 · Systems sources (axboe/fio @ acb7a302)
1 · Systems sources (sysstat/sysstat @ efd0c445) | candidate · not_run · 0 readset(s) | no-eval | +| [torch-npu](../src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/SKILL.md) | ascend | Operate and diagnose TorchNPU on Ascend across installation, PrivateUse1 dispatch, operator development, memory and ... | Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack ... | failure_modes, input_contract, produced_artifact, refusal_point | 1 Accelerator † guidance: TorchNPU framework source procedures
2 RAM/Host † guidance: TorchNPU framework source procedures
4 Network † guidance: TorchNPU framework source procedures | TorchNPU framework operating graph (Ascend/pytorch @ `046934140b96836c3fcef14d0f39ae54dc0a06bf` ...) | candidate · not_run · 2 readset(s) | no-eval | +| [torchnpu-device-memory-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-device-memory-lifecycle/SKILL.md) | ascend | Diagnose TorchNPU device-HBM reuse, cross-stream lifetimes, cache retention and fragmentation, then propose ... | Diagnose TorchNPU device-HBM reuse, stream lifetimes, fragmentation and ... | input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Ascend/pytorch (TorchNPU) @ `67511c53cd4c88014f6123de7097677e60e85faa` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [torchnpu-pinned-host-memory-lifecycle](../src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md) | ascend | Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy lifetime, reuse and cache controls, then propose ... | Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy ownership/readiness ... | input_contract, produced_artifact, refusal_point
3 advice | 2 RAM/Host · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Ascend/pytorch (TorchNPU) @ `67511c53cd4c88014f6123de7097677e60e85faa` ...) | candidate · not_run · 1 readset(s) | no-eval | +| [vllm-ascend](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/SKILL.md) | ascend | Analyze vLLM Ascend v0.27.1rc1 performance and commit evolution for DeepSeek-V4, Kimi K3, MiniMax-M3 and GLM5+, fused ... | Release-pinned performance diagnosis and historical optimization analysis for ... | input_contract, produced_artifact, refusal_point
5 advice | 1 Accelerator † guidance: vLLM Ascend v0.27.1rc1 serving diagnosis
2 RAM/Host † guidance: vLLM Ascend v0.27.1rc1 serving diagnosis | vLLM Ascend v0.27.1rc1 serving scope (vllm-project/vllm-ascend at `3b31886237ed65c435e0965b10e8002ff5769766`) | candidate · not_run · 3 readset(s) | no-eval | +| [vllm-metal](../src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/SKILL.md) | common | Investigate vllm-metal Apple inference through platform and native-ABI selection, paged KV/offload ownership ... | Selected plugin preparation, KV/offload ownership and speculative/deferred/dispatch ... | failure_modes, input_contract, produced_artifact
3 advice | 1 Accelerator † guidance: vLLM Metal selected source contracts
2 RAM/Host † guidance: vLLM Metal selected source contracts
3 SSD † guidance: vLLM Metal selected source contracts | vLLM Metal portable AREX source and CPU extraction (vllm-project/vllm-metal @ `e7df7a6ff215f99c7842552d230fdbc709835934` ...) | candidate · partial_cpu_only · 2 readset(s) | no-eval | + +## Registered: Task routing (8) | Skill | Backend | Declared task | Applicability | Conditions | SearchSpace layer / stage | Reference Matrix | State | Gaps | |---|---|---|---|---|---|---|---|---| -| [assess-serving-preemption](../src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/SKILL.md) | common | Assess whether LLM serving queue delays justify token-boundary priority preemption, accounting for suspended KV, host ... | Diagnosing whether LLM serving queue delay justifies token-boundary priority preemption ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify
2 RAM/Host · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Orca — OSDI 2022 proceedings) | candidate · not_run · 1 readset(s) | no-eval | -| [assess-split-backward-schedule](../src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/SKILL.md) | common | Evaluate memory-capped split-backward and V-shaped pipeline scheduling proposals, preserving synchronous update ... | Assessing memory-capped split-backward and V-shaped pipeline schedule proposals when a ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Zero Bubble — [2401.10241v1](https://arxiv.org/abs/2401.10241v1)) | candidate · not_run · 1 readset(s) | no-eval | -| [audit-fixed-workload-claim](../src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/SKILL.md) | common | Audit a performance claim from supplied fixed-workload reports and an explicit measurement contract. Check aggregation ... | Auditing a performance claim against supplied fixed-workload reports and an explicit ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | N/A: SKILL.md §§ Preserve the observation hierarchy, Check what was actually compared, and Return an auditable conclusion audit supplied measurements and verdicts; device identity is comparison metadata, not a hardware-resource mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | -| [calibrate-serving-simulation](../src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/SKILL.md) | common | Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical ... | Building an evidence-gated serving-configuration shortlist when a profiled simulator or ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Verify | 1 · Systems sources (Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2))
1 · Systems sources ([AMD Infera @ ff1136e4](https://github.com/AMD-AGI/Infera/tree/ff1136e ...) | candidate · not_run · 3 readset(s) | no-eval | -| [checkpoint-recovery-consistency](../src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md) | common | Assess distributed checkpoint recovery using DLRover memory agreement, persistence completion and replica transport ... | Assessing distributed checkpoint recovery through DLRover memory agreement, persistence ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 2 RAM/Host · Configure / Verify
3 SSD · Configure / Verify
4 Network · Diagnose / Configure / Verify
5 OS · Diagnose / Configure | 1 · Systems sources (C3 / DLRover FlashCheckpoint @ `e35b1ad90e2928ca6bacf5d22117f71356 ...)
1 · Systems sources (C4 / NVIDIA Resiliency Extension @ ...) | candidate · not_run · 1 readset(s) | no-eval | -| [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify
4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval | -| [dense-performance-evidence](../src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Observe / Profile / Verify
2 RAM/Host · Verify
4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)
2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — | -| [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | common | Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including ... | Comparing migration of a running LLM request between compatible model replicas with ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify
2 RAM/Host · Diagnose / Configure / Optimize / Verify
4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1)) | candidate · not_run · 1 readset(s) | no-eval | -| [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point
9 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)
2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)
2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)
GPU measurement validity and profiler attribution (NVIDIA NCU 2025.3, Nsight Systems 2025.3, CUDA 12.9.1/10.2 archives ...)
Task-specific benchmark selection (issue #60) ([Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/15 ...)
Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)
Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...)
Task-specific benchmark selection (issue #60) ([yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizha ...) | candidate · not_run · 16 readset(s) | — | -| [inference-optimization-casebook](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | Structuring and querying a casebook of past optimization decisions as decision support ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval | -| [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton ... | failure_modes, input_contract, produced_artifact, refusal_point
6 advice | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)
1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)
1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)
1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)
1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-eval | -| [optimization-search-policy](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | Selecting, ordering and stopping optimization experiments: candidate generation ... | failure_modes, input_contract, produced_artifact, refusal_point
5 advice | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)
2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)
2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))
2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-eval | -| [performance-ceiling-analysis](../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md) | common | Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical ... | inference serving systems (LLM, diffusion, audio) | 2 advice | 1 Accelerator · Diagnose / Verify
5 OS · Configure | 1 · Systems sources (FlexGen — [ICML 2023 / PMLR 202:31094–31116](https://proceedings.mlr.p ...)
1 · Systems sources (Mooncake — FAST '25 proceedings, pp. 155-170)
1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
1 · Systems sources (ebugger/Empirical-Roofline-Toolkit @ 0771fa9)
1 · Systems sources (hahnyuan/LLM-Viewer @ 1893e4b)
2 · Methodology and corpus references ([shenh10/HuggingArch](https://github.com/shenh10/HuggingArch/tree/facb ...) | candidate · not_run · 20 readset(s) | no-eval | -| [plan-activation-residency](../src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/SKILL.md) | common | Propose training activation and gradient memory reductions by distinguishing lifetime reordering, address packing and ... | Proposing training activation and gradient memory reductions when pressure appears to ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Diagnose / Optimize / Verify | 1 · Systems sources (Checkmate — [1910.02653v3](https://arxiv.org/abs/1910.02653v3)) | candidate · not_run · 1 readset(s) | no-eval | -| [repository-evolution-analysis](../src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/SKILL.md) | common | Analyze a repository's commit history to build a capability-evolution model: what the system can do, how it got there ... | any software repository with a recoverable git history | — | 5 OS · Diagnose / Configure / Verify | 2 · Methodology and corpus references (LearningToCommit/LearningToCommit @ 7e5a1d7)
2 · Methodology and corpus references (yetone/magpie @ 77800032) | candidate · not_run · 0 readset(s) | no-eval | -| [skill-evolution-playbook](../src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/SKILL.md) | common | Practical design choices for evolving agent skills from execution traces: when one trajectory should update a skill ... | Design choices for evolving agent skills from execution traces, drawn from Trace2Skill ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | N/A: SKILL.md §§ When should one trajectory update a skill?, How large should a skill edit be?, and When should skills split or merge? govern skill-text evolution and held-out evaluation, not hardware-resource state or control. | 2 · Methodology and corpus references (Qwen-Applications/Trace2Skill @ 3d0b52a)
2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | no-eval | -| [training-input-pipeline](../src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/SKILL.md) | common | Diagnose training input stalls and choose bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic ... | Diagnosing training input stalls and choosing bounded queue, shard-cache and ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Observe / Configure / Optimize
2 RAM/Host · Observe / Diagnose / Configure / Optimize
3 SSD · Diagnose / Configure / Optimize
4 Network · Configure / Optimize
5 OS · Configure / Optimize | 1 · Systems sources (B1 / NVIDIA DALI @ `6fe47c48d3b0ca335f5e1afd3c77c750b2c7e041` ...)
1 · Systems sources (B2 / Mosaic Streaming @ `d99bf9c7cdf2dd4ed62a4960ed35270b93337a5f` ...)
1 · Systems sources (B3 / FFCV @ `3a12966b3afe3a81733a732e633317d747bfaac7` ...) | candidate · not_run · 1 readset(s) | no-eval | -| [training-resource-budget](../src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/SKILL.md) | common | Choose training offload and resource-budget changes using pinned DeepSpeed configuration, attributed expert diagnosis ... | Choosing training offload and resource-budget changes from pinned DeepSpeed ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | 1 Accelerator · Diagnose / Configure / Optimize
2 RAM/Host · Observe / Diagnose / Configure / Optimize / Verify
3 SSD · Configure / Optimize
4 Network · Observe / Diagnose / Verify
5 OS · Diagnose / Configure | 1 · Systems sources (CR1 / DeepSpeed ZeRO-Infinity @ `5f266e26d216a705d9c64cac88fb813ce ...)
1 · Systems sources (CR5 / Ascend MindSpeed adaptive memory — historical `core_r0.8.0` ...)
1 · Systems sources (ZeRO-Offload — [2101.06840v1](https://arxiv.org/abs/2101.06840v1))
2 · Methodology and corpus references (CR6 / Stas Bekman ML Engineering Open Book @ ...) | candidate · not_run · 9 readset(s) | no-eval | -| [video-diffusion-optimization](../src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/SKILL.md) | common | Run and verify goal-driven video diffusion optimization, backend-agnostic. Use when the user asks for latency, RTF ... | Backend-agnostic video diffusion inference and serving optimization | — | 1 Accelerator † guidance: Video diffusion campaign guidance (unvalidated) | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | -| [weight-loading-optimization](../src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md) | common | Optimize the serving-level weight load path from storage to device: judge the load path on four dimensions (source ... | Diagnosing the serving-level weight load path from storage to device and choosing changes ... | failure_modes, input_contract, produced_artifact, refusal_point
6 advice | 1 Accelerator † guidance: Flare preparation and complete-session source contracts
2 RAM/Host † guidance: Flare preparation and complete-session source contracts; DeepSeek streamed expert-cache verification
3 SSD · Observe / Profile / Diagnose / Optimize / Verify † guidance: Flare preparation and complete-session source contracts; DeepSeek streamed expert-cache verification
4 Network · Configure / Optimize / Verify | 1 · Systems sources (Ascend/MindIE-SD @ 8481d39)
1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)
1 · Systems sources (ascend-ai-coding/awesome-ascend-skills @ 75ec89c)
1 · Systems sources (foundation-model-stack/fastsafetensors @ 219e256)
1 · Systems sources (ggml-org/llama.cpp @ `3e390af18de1131dda27913c00511725b4274c8a` (MIT ...)
1 · Systems sources (mindie-agent/mindie-agent @ 6a81b41)
DeepSeek streamed expert-cache oracle contracts ([ronak-create/deepseek-v4-in-c](https://github.com/ronak-create/deepse ...) | candidate · not_run · 5 readset(s) | no-eval | - -## Registered: Workflows (6) +| [inference-task-router](../src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/SKILL.md) | common | Select the starting route for day-0 model support, inference performance optimization, debugging, accuracy repair ... | Classifying an inference engineering request and identifying reusable procedures. | input_contract, produced_artifact, refusal_point | N/A: This entry selects a task and its evidence requirements before resource-layer analysis. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [task-accuracy](../src/inferrsi/knowledge/data/skill_library/tasks/task-accuracy/SKILL.md) | common | Use when inference completes but tokens, audio, frames, tensors or a quality metric diverge from a reference. Trace the ... | Modality-specific reference parity and accuracy regression repair. | input_contract, produced_artifact, refusal_point | N/A: The procedure selects supporting evidence and tests within the caller's declared search space. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [task-day-0](../src/inferrsi/knowledge/data/skill_library/tasks/task-day-0/SKILL.md) | common | Use for Day-0 onboarding of a new checkpoint or architecture to a named inference engine. Produce a reproducible target ... | New model support in an existing inference engine. | input_contract, produced_artifact, refusal_point | N/A: The procedure selects supporting evidence and tests within the caller's declared search space. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [task-debug](../src/inferrsi/knowledge/data/skill_library/tasks/task-debug/SKILL.md) | common | Use for inference crashes, hangs, out-of-memory failures or execution-path errors. Localize the first failing boundary ... | Run-specific failure reproduction, fault isolation and regression verification. | input_contract, produced_artifact, refusal_point | N/A: The procedure selects supporting evidence and tests within the caller's declared search space. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [task-derived-engine](../src/inferrsi/knowledge/data/skill_library/tasks/task-derived-engine/SKILL.md) | common | Use for a fork or extension that retains an upstream inference execution core, including a vLLM or vLLM-Omni ... | Development and qualification of an engine that retains an upstream execution core. | input_contract, produced_artifact, refusal_point | N/A: The procedure selects supporting evidence and tests within the caller's declared search space. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [task-independent-engine](../src/inferrsi/knowledge/data/skill_library/tasks/task-independent-engine/SKILL.md) | common | Use when the deliverable owns its execution, memory and device integration on a target OS and hardware stack. Prove a ... | Independent runtime bring-up across OS, model format, device and backend boundaries. | input_contract, produced_artifact, refusal_point | N/A: The procedure selects supporting evidence and tests within the caller's declared search space. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [task-performance](../src/inferrsi/knowledge/data/skill_library/tasks/task-performance/SKILL.md) | common | Use when a correct inference baseline misses a latency, throughput, memory or cost target. Compare compatible changes ... | Optimization of a specified metric under fixed workload, quality and resource constraints. | input_contract, produced_artifact, refusal_point | N/A: The procedure selects supporting evidence and tests within the caller's declared search space. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [task-version-alignment](../src/inferrsi/knowledge/data/skill_library/tasks/task-version-alignment/SKILL.md) | common | Use for upgrades spanning inference engines, plugins, drivers or operator libraries. Isolate the incompatible boundary ... | Compatibility of explicit working and target version combinations. | input_contract, produced_artifact, refusal_point | N/A: The procedure selects supporting evidence and tests within the caller's declared search space. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | + +## Registered: Cross-stage workflows (8) | Skill | Backend | Declared task | Applicability | Conditions | SearchSpace layer / stage | Reference Matrix | State | Gaps | |---|---|---|---|---|---|---|---|---| -| [cite-check-downstream-pr](../src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md) | common | Review a downstream PR description for correct acceptance issue links, SHA pins, and artifact citations. | Review of a downstream PR description that cites RSI acceptance work for a performance or ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | N/A: SKILL.md §§ Checklist and Red flags check acceptance links, revision/artifact citations and claim wording; they provide no hardware-resource mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | -| [claim-contract](../src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/SKILL.md) | common | Enforce ThinkFlowLab claim contract wording and verdict rules without embedding campaign-specific revision pins. | Wording and verdict rules for any public performance or quality claim produced from an ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Verify | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | -| [competitor-scan](../src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/SKILL.md) | common | Structure a lightweight competitive scan without inventing benchmark numbers or campaign verdicts. | Qualitative positioning of a new RSI repository against other projects before acceptance ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | N/A: SKILL.md §§ Frame, Rules and Deliverable produce a qualitative competitive comparison with evidence limits, not a hardware-resource decision or implementation mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | -| [draft-acceptance-issue](../src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/SKILL.md) | common | Draft a campaign acceptance issue with checklist, RunSpec, evidence paths, and placeholders for baseline/candidate SHAs. | Opening a new acceptance campaign in an RSI-style repository, before any baseline or ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | N/A: SKILL.md §§ Steps and Output prepare an acceptance issue, RunSpec, frozen gates and evidence paths; device identifiers are campaign metadata rather than a hardware-resource mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | -| [npu-skill-factory](../src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/SKILL.md) | common | Design and operation of the InferRSI NPU skill factory: artifact lifecycle, skill evaluation contract, corpus ... | InferRSI skill lifecycle management | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | N/A: SKILL.md §§ Choose the evidence product, Artifact lifecycle, and What this skill does not do define skill-distillation/evaluation methodology; NPU task tiers are evaluation prerequisites, not a hardware-resource mechanism. | 2 · Methodology and corpus references (Dong1017/OmniRSI @ `codex/skill-eval-bootstrap` (82218b4))
2 · Methodology and corpus references (InferMatrixLab/InferRSI PR #9 @ 4ad551a9)
2 · Methodology and corpus references (VectorSpaceLab/AREX-Skill @ ac3fe1af)
2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 14 readset(s) | no-eval | -| [rsi-evidence](../src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/SKILL.md) | common | Guide agents to establish acceptance criteria, freeze and cite RSI evidence using plan/results/patch/validator paths ... | Preparing acceptance criteria, documenting or reviewing an optimization campaign whose ... | failure_modes, input_contract, produced_artifact, refusal_point
6 advice | 1 Accelerator · Verify | — | candidate · not_run · 1 readset(s) | no-Reference-Matrix-row, no-eval | +| [cite-check-downstream-pr](../src/inferrsi/knowledge/data/skill_library/workflows/governance/cite-check-downstream-pr/SKILL.md) | common | Review a downstream PR description for correct acceptance issue links, SHA pins, and artifact citations. | Review of a downstream PR description that cites RSI acceptance work for a performance or ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | N/A: SKILL.md §§ Checklist and Red flags check acceptance links, revision/artifact citations and claim wording; they provide no hardware-resource mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [claim-contract](../src/inferrsi/knowledge/data/skill_library/workflows/governance/claim-contract/SKILL.md) | common | Enforce ThinkFlowLab claim contract wording and verdict rules without embedding campaign-specific revision pins. | Wording and verdict rules for any public performance or quality claim produced from an ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | 1 Accelerator · Verify | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [competitor-scan](../src/inferrsi/knowledge/data/skill_library/workflows/research/competitor-scan/SKILL.md) | common | Structure a lightweight competitive scan without inventing benchmark numbers or campaign verdicts. | Qualitative positioning of a new RSI repository against other projects before acceptance ... | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | N/A: SKILL.md §§ Frame, Rules and Deliverable produce a qualitative competitive comparison with evidence limits, not a hardware-resource decision or implementation mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [draft-acceptance-issue](../src/inferrsi/knowledge/data/skill_library/workflows/governance/draft-acceptance-issue/SKILL.md) | common | Draft a campaign acceptance issue with checklist, RunSpec, evidence paths, and placeholders for baseline/candidate SHAs. | Opening a new acceptance campaign in an RSI-style repository, before any baseline or ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | N/A: SKILL.md §§ Steps and Output prepare an acceptance issue, RunSpec, frozen gates and evidence paths; device identifiers are campaign metadata rather than a hardware-resource mechanism. | — | candidate · not_run · 0 readset(s) | no-Reference-Matrix-row, no-eval | +| [npu-skill-factory](../src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/SKILL.md) | common | Design and operation of the InferRSI NPU skill factory: artifact lifecycle, skill evaluation contract, corpus ... | InferRSI skill lifecycle management | failure_modes, input_contract, produced_artifact, refusal_point
3 advice | N/A: SKILL.md §§ Choose the evidence product, Artifact lifecycle, and What this skill does not do define skill-distillation/evaluation methodology; NPU task tiers are evaluation prerequisites, not a hardware-resource mechanism. | 2 · Methodology and corpus references (Dong1017/OmniRSI @ `codex/skill-eval-bootstrap` (82218b4))
2 · Methodology and corpus references (InferMatrixLab/InferRSI PR #9 @ 4ad551a9)
2 · Methodology and corpus references (VectorSpaceLab/AREX-Skill @ ac3fe1af)
2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 14 readset(s) | no-eval | +| [repository-evolution-analysis](../src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/SKILL.md) | common | Analyze a repository's commit history to build a capability-evolution model: what the system can do, how it got there ... | any software repository with a recoverable git history | — | 5 OS · Diagnose / Configure / Verify | 2 · Methodology and corpus references (LearningToCommit/LearningToCommit @ 7e5a1d7)
2 · Methodology and corpus references (yetone/magpie @ 77800032) | candidate · not_run · 0 readset(s) | no-eval | +| [rsi-evidence](../src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/SKILL.md) | common | Guide agents to establish acceptance criteria, freeze and cite RSI evidence using plan/results/patch/validator paths ... | Preparing acceptance criteria, documenting or reviewing an optimization campaign whose ... | failure_modes, input_contract, produced_artifact, refusal_point
6 advice | 1 Accelerator · Verify | — | candidate · not_run · 1 readset(s) | no-Reference-Matrix-row, no-eval | +| [skill-evolution-playbook](../src/inferrsi/knowledge/data/skill_library/workflows/maintenance/skill-evolution-playbook/SKILL.md) | common | Practical design choices for evolving agent skills from execution traces: when one trajectory should update a skill ... | Design choices for evolving agent skills from execution traces, drawn from Trace2Skill ... | failure_modes, input_contract, produced_artifact, refusal_point
4 advice | N/A: SKILL.md §§ When should one trajectory update a skill?, How large should a skill edit be?, and When should skills split or merge? govern skill-text evolution and held-out evaluation, not hardware-resource state or control. | 2 · Methodology and corpus references (Qwen-Applications/Trace2Skill @ 3d0b52a)
2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | no-eval | ## Documents outside the registry @@ -165,76 +172,75 @@ an entrypoint routes to procedures below it rather than adding one. | Directory | Kind | |---|---| -| `frameworks/pytorch/pytorch/sub-skills/compiler-and-export` | sub-skill | -| `frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension` | sub-skill | -| `frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint` | sub-skill | -| `frameworks/pytorch/pytorch/sub-skills/installation-and-build` | sub-skill | -| `frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs` | sub-skill | -| `frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging` | sub-skill | -| `frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd` | sub-skill | -| `frameworks/pytorch/pytorch/sub-skills/training-and-data` | sub-skill | -| `frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl` | sub-skill | -| `frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile` | sub-skill | -| `frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility` | sub-skill | -| `frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events` | sub-skill | -| `frameworks/torch-npu/torch-npu/sub-skills/operator-development` | sub-skill | -| `frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch` | sub-skill | -| `frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging` | sub-skill | -| `frameworks/torch-npu/torch-npu/sub-skills/training-and-precision` | sub-skill | -| `repositories/dwarfstar/sub-skills/qualification` | sub-skill | -| `repositories/dwarfstar/sub-skills/runtime-memory` | sub-skill | -| `repositories/dwarfstar/sub-skills/serving-state` | sub-skill | -| `repositories/lily/sub-skills/model-runtime` | sub-skill | -| `repositories/lily/sub-skills/qualification` | sub-skill | -| `repositories/lily/sub-skills/serving-cache` | sub-skill | -| `repositories/lithos-metal/sub-skills/compile-fusion` | sub-skill | -| `repositories/lithos-metal/sub-skills/generation-state` | sub-skill | -| `repositories/lithos-metal/sub-skills/serving-cache` | sub-skill | -| `repositories/mistral-rs/sub-skills/decode-contract` | sub-skill | -| `repositories/mistral-rs/sub-skills/prefill-contract` | sub-skill | -| `repositories/mistral-rs/sub-skills/qualification` | sub-skill | -| `repositories/mlc-llm/sub-skills/artifact-contract` | sub-skill | -| `repositories/mlc-llm/sub-skills/metal-build-runtime` | sub-skill | -| `repositories/mlc-llm/sub-skills/runtime-kv-limits` | sub-skill | -| `repositories/mlx-swift-lm/sub-skills/gated-delta-contract` | sub-skill | -| `repositories/mlx-swift-lm/sub-skills/generation-lifecycle` | sub-skill | -| `repositories/project-maya/sub-skills/model-runtime` | sub-skill | -| `repositories/project-maya/sub-skills/qualification` | sub-skill | -| `repositories/project-maya/sub-skills/serving-state` | sub-skill | -| `repositories/sglang` | engine entrypoint | -| `repositories/sglang-omni` | engine entrypoint | -| `repositories/sglang-omni/sub-skills/sglang-omni-architecture` | sub-skill | -| `repositories/sglang-omni/sub-skills/sglang-omni-extension` | sub-skill | -| `repositories/sglang-omni/sub-skills/sglang-omni-serving` | sub-skill | -| `repositories/sglang/sub-skills/sglang-parallel-extension` | sub-skill | -| `repositories/sglang/sub-skills/sglang-runtime-memory` | sub-skill | -| `repositories/sglang/sub-skills/sglang-tuning-validation` | sub-skill | -| `repositories/splash/sub-skills/qualification` | sub-skill | -| `repositories/splash/sub-skills/runtime-planning` | sub-skill | -| `repositories/splash/sub-skills/serving-cache` | sub-skill | -| `repositories/strata` | engine entrypoint | -| `repositories/strata/sub-skills/design-space` | sub-skill | -| `repositories/strata/sub-skills/tune-execution` | sub-skill | -| `repositories/strata/sub-skills/validate-engine` | sub-skill | -| `repositories/vllm` | engine entrypoint | -| `repositories/vllm-metal/sub-skills/cache-lifecycle` | sub-skill | -| `repositories/vllm-metal/sub-skills/platform-config` | sub-skill | -| `repositories/vllm-metal/sub-skills/qualification` | sub-skill | -| `repositories/vllm-omni` | engine entrypoint | -| `repositories/vllm-omni/sub-skills/omni-architecture` | sub-skill | -| `repositories/vllm-omni/sub-skills/omni-serving` | sub-skill | -| `repositories/vllm-omni/sub-skills/omni-validation` | sub-skill | -| `repositories/vllm/sub-skills/architecture-search-space` | sub-skill | -| `repositories/vllm/sub-skills/extend-validate` | sub-skill | -| `repositories/vllm/sub-skills/operation-performance` | sub-skill | -| `workflows/inference-engine-research` | router | +| `systems/engines/dwarfstar/sub-skills/qualification` | sub-skill | +| `systems/engines/dwarfstar/sub-skills/runtime-memory` | sub-skill | +| `systems/engines/dwarfstar/sub-skills/serving-state` | sub-skill | +| `systems/engines/lily/sub-skills/model-runtime` | sub-skill | +| `systems/engines/lily/sub-skills/qualification` | sub-skill | +| `systems/engines/lily/sub-skills/serving-cache` | sub-skill | +| `systems/engines/lithos-metal/sub-skills/compile-fusion` | sub-skill | +| `systems/engines/lithos-metal/sub-skills/generation-state` | sub-skill | +| `systems/engines/lithos-metal/sub-skills/serving-cache` | sub-skill | +| `systems/engines/mistral-rs/sub-skills/decode-contract` | sub-skill | +| `systems/engines/mistral-rs/sub-skills/prefill-contract` | sub-skill | +| `systems/engines/mistral-rs/sub-skills/qualification` | sub-skill | +| `systems/engines/mlc-llm/sub-skills/artifact-contract` | sub-skill | +| `systems/engines/mlc-llm/sub-skills/metal-build-runtime` | sub-skill | +| `systems/engines/mlc-llm/sub-skills/runtime-kv-limits` | sub-skill | +| `systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract` | sub-skill | +| `systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle` | sub-skill | +| `systems/engines/project-maya/sub-skills/model-runtime` | sub-skill | +| `systems/engines/project-maya/sub-skills/qualification` | sub-skill | +| `systems/engines/project-maya/sub-skills/serving-state` | sub-skill | +| `systems/engines/sglang` | engine entrypoint | +| `systems/engines/sglang-omni` | engine entrypoint | +| `systems/engines/sglang-omni/sub-skills/sglang-omni-architecture` | sub-skill | +| `systems/engines/sglang-omni/sub-skills/sglang-omni-extension` | sub-skill | +| `systems/engines/sglang-omni/sub-skills/sglang-omni-serving` | sub-skill | +| `systems/engines/sglang/sub-skills/sglang-parallel-extension` | sub-skill | +| `systems/engines/sglang/sub-skills/sglang-runtime-memory` | sub-skill | +| `systems/engines/sglang/sub-skills/sglang-tuning-validation` | sub-skill | +| `systems/engines/splash/sub-skills/qualification` | sub-skill | +| `systems/engines/splash/sub-skills/runtime-planning` | sub-skill | +| `systems/engines/splash/sub-skills/serving-cache` | sub-skill | +| `systems/engines/strata` | engine entrypoint | +| `systems/engines/strata/sub-skills/design-space` | sub-skill | +| `systems/engines/strata/sub-skills/tune-execution` | sub-skill | +| `systems/engines/strata/sub-skills/validate-engine` | sub-skill | +| `systems/engines/vllm` | engine entrypoint | +| `systems/engines/vllm-metal/sub-skills/cache-lifecycle` | sub-skill | +| `systems/engines/vllm-metal/sub-skills/platform-config` | sub-skill | +| `systems/engines/vllm-metal/sub-skills/qualification` | sub-skill | +| `systems/engines/vllm-omni` | engine entrypoint | +| `systems/engines/vllm-omni/sub-skills/omni-architecture` | sub-skill | +| `systems/engines/vllm-omni/sub-skills/omni-serving` | sub-skill | +| `systems/engines/vllm-omni/sub-skills/omni-validation` | sub-skill | +| `systems/engines/vllm/sub-skills/architecture-search-space` | sub-skill | +| `systems/engines/vllm/sub-skills/extend-validate` | sub-skill | +| `systems/engines/vllm/sub-skills/operation-performance` | sub-skill | +| `systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export` | sub-skill | +| `systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension` | sub-skill | +| `systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint` | sub-skill | +| `systems/frameworks/pytorch/pytorch/sub-skills/installation-and-build` | sub-skill | +| `systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs` | sub-skill | +| `systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging` | sub-skill | +| `systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd` | sub-skill | +| `systems/frameworks/pytorch/pytorch/sub-skills/training-and-data` | sub-skill | +| `systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl` | sub-skill | +| `systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile` | sub-skill | +| `systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility` | sub-skill | +| `systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events` | sub-skill | +| `systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development` | sub-skill | +| `systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch` | sub-skill | +| `systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging` | sub-skill | +| `systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision` | sub-skill | +| `workflows/research/inference-engine-research` | router | ## Integrity checks -- Registered ids resolve to an existing `SKILL.md`: 80 of 80. +- Registered ids resolve to an existing `SKILL.md`: 88 of 88. - Duplicate ids or paths: none; `skill_catalog` rejects them before this view is built. - Skill links in the two matrices that do not resolve: 0. -- Registered skills without a `metadata.json` sidecar: 0. ## Recorded gaps @@ -242,13 +248,12 @@ an entrypoint routes to procedures below it rather than adding one. |---|---| | `no-SearchSpace-layer` | 1: husky | | `conflicting-SearchSpace-layer` | 0: — | -| `no-Reference-Matrix-row` | 8: ascend-video-diffusion-optimization, audit-fixed-workload-claim, cite-check-downstream-pr, claim-contract, competitor-scan, draft-acceptance-issue, rsi-evidence, video-diffusion-optimization | -| `no-sidecar` | 0: — | +| `no-Reference-Matrix-row` | 16: ascend-video-diffusion-optimization, audit-fixed-workload-claim, cite-check-downstream-pr, claim-contract, competitor-scan, draft-acceptance-issue, inference-task-router, rsi-evidence, task-accuracy, task-day-0, task-debug, task-derived-engine, task-independent-engine, task-performance, task-version-alignment, video-diffusion-optimization | | `no-applicability` | 0: — | -| `no-eval` | 78: as listed in the Gaps column above | +| `no-eval` | 86: as listed in the Gaps column above | These are inventory gaps, not efficacy results. Missing layer mappings stay -unknown unless the sidecar records a reasoned N/A. N/A is not coverage; +unknown unless the owner records a reasoned N/A. N/A is not coverage; a conflicting matrix mapping remains visible and is excluded from coverage counts until reconciled. Supplementary guidance stays distinct from numbered layer coverage. A candidate may legitimately remain untested. @@ -278,7 +283,7 @@ read each campaign's own `README.md` for its verdict and limits. ## Drift and maintenance The generator is the only writer of this file. After changing the registry, -a skill body, a sidecar, either matrix or a frozen plan, regenerate and +a skill document, either matrix or a frozen plan, regenerate and commit both files together. `--check` rebuilds in memory and diffs against the committed file, exiting non-zero on any difference, so CI can detect a view that drifted from its sources. Only the affected rows need review; diff --git a/docs/skill-eval-pr.md b/docs/skill-eval-pr.md index f08a76cc7..fa2ad49f4 100644 --- a/docs/skill-eval-pr.md +++ b/docs/skill-eval-pr.md @@ -305,8 +305,7 @@ content or another boundary. The automated report does not invent that decision or silently count the gap as passed. Content/source review, authorship admission, and merge authority remain independent. -Knowledge-only changes under `src/inferrsi/knowledge/data/repo_packs/**/knowledge/**` and -`src/inferrsi/knowledge/data/knowledge_packs/**` now trigger bounded automatic planning. +Changes under `src/inferrsi/knowledge/data/skill_library/**` trigger bounded automatic planning. The older pack path patterns remain in the workflow for migration diffs. Unmapped paths remain uncovered, select no paid work, and publish insufficient evidence rather than a pass. Other repository-pack paths can use the existing **plan-only dispatch** for an explicit scope report. No unrelated audit grader diff --git a/docs/skill-layout-migration.csv b/docs/skill-layout-migration.csv new file mode 100644 index 000000000..02a755684 --- /dev/null +++ b/docs/skill-layout-migration.csv @@ -0,0 +1,96 @@ +kind,stable_id,old_path,new_path,old_checkout_revision,new_checkout_revision +skill,ascend-aclgraph-capture-replay,hardware/ascend/ascend-aclgraph-capture-replay,systems/platforms/ascend/ascend-aclgraph-capture-replay,sha256:465f798272822a70c55121f99ccf683a2db2330352c6379682f2da759ee18f43,sha256:d381713d28bf452f90503141bcfca2063b5dfac7f4558628e25faf91d9468386 +skill,ascend-device-health,hardware/ascend/ascend-device-health,systems/platforms/ascend/ascend-device-health,sha256:fa189ea81e8f30336c355373c90e6de696aea727a5d912c9bf8483ae45f92017,sha256:03f7d3b229d21e9cbb5e7246b9e6443febe6ddaa9a59a08d4f7f27a96036376c +skill,ascend-distributed-parallel,hardware/ascend/ascend-distributed-parallel,systems/platforms/ascend/ascend-distributed-parallel,sha256:2d71fb10a539a0df31d71c136519cb3326a44e96aab82b12c84fa8ac21b746ff,sha256:dcb23816671423173148ccadaeafdaaa807621549d0ea0a386ef43e5496af4c4 +skill,ascend-graph-mode,hardware/ascend/ascend-graph-mode,systems/platforms/ascend/ascend-graph-mode,sha256:6064546fcdee9e2a4452598d30a527ebb91aaccbff37bfa0a41927bd4973f81b,sha256:080bd5eedb7d100dc4a5821eb85d0ee5685fb97a4c546111416590a4c66e39db +skill,ascend-model-adaptation,hardware/ascend/ascend-model-adaptation,systems/platforms/ascend/ascend-model-adaptation,sha256:d7347190a4aa9f084b9db144ecb2c3bf0260b974fa79d0b172ee0b686d2d76d5,sha256:fe3758c7e2e411ff2963e73016c76912d7ac7f7d552e5c0a61a98ebde8a85ac6 +skill,ascend-operator-profiling,hardware/ascend/ascend-operator-profiling,systems/platforms/ascend/ascend-operator-profiling,sha256:678ff74e8b1eb4386b4cde89c64cf235e83cbbb9034eb83c14889458e40ef64c,sha256:5aab9c463fb01e80cba3a13041acd95c36e1c745f0cb5cae52fd274a9a720849 +skill,ascend-operator-tuning,hardware/ascend/ascend-operator-tuning,systems/platforms/ascend/ascend-operator-tuning,sha256:4e491003f99c52e4b3304d81d033504252d7b0dce813527599e5dcc977e5f17c,sha256:c34564b4e7988870c7c6038a262ebf9cd04f5eea3bf26b1783848a18e86c8c7a +skill,ascend-optimization-casebook,hardware/ascend/ascend-optimization-casebook,casebooks/ascend/ascend-optimization-casebook,sha256:83ca56c085dbc70a9e6bee1c1e7a8ce122d9a5f14a91580a4a162713df3d3582,sha256:17deaadcc3ea337e2a45b9c413d9846e6744caf75699b90f45c4f8f00f171285 +skill,ascend-performance-analysis,hardware/ascend/ascend-performance-analysis,systems/platforms/ascend/ascend-performance-analysis,sha256:b579b2c7bb7c5a0ab2df7f959b67a90cceadc62c284003f7c3d4fbf562279fc5,sha256:8fca01098e84022a37ce257f35309b18a293f6d6c2d863869db1ab728295324d +skill,ascend-video-diffusion-optimization,hardware/ascend/ascend-video-diffusion-optimization,systems/platforms/ascend/ascend-video-diffusion-optimization,sha256:f769d91c74f49de5487d31c1764d8fb43e9be64d05c2d511f794d40821d1941f,sha256:13ab5e3d4d73a1edc91b08c9d534c430586560978eec2492df844ea3b2f6ecad +skill,ascendc-tiling-design,hardware/ascend/ascendc-tiling-design,systems/platforms/ascend/ascendc-tiling-design,sha256:92feb81a367cda8fad23ea331c01bbd53274c3a9da71edc63ea9954607c29b65,sha256:1a039fad08d49d5215f785dbb1cc5a593807a905e1c8abfc05d8d40de3c3d732 +skill,assess-serving-preemption,methods/assess-serving-preemption,playbooks/planning/assess-serving-preemption,sha256:05588c78028d508ff4e085777809e800c0accf256c071a92c22fd649c21bccb0,sha256:57c8da67b15123f52947bb55abec8beae6f25abcefb9881535a72de3556df0d9 +skill,assess-split-backward-schedule,methods/assess-split-backward-schedule,playbooks/planning/assess-split-backward-schedule,sha256:8c36510723c989bf32e587f6f18c2d6297d4e51e43f41b64be545b63833bbaed,sha256:15019511215b7111ba697999ccc14f6b7ce27b0ff21de7ab2e1be7752d4d4078 +skill,audit-afd-p2p-exchanges,repositories/afd-plugin/audit-afd-p2p-exchanges,systems/engines/afd-plugin/audit-afd-p2p-exchanges,sha256:35277be35b02244efe86693b20b2a0d9bf833c165cd05a87046530b668fd77bf,sha256:e2c1f2ff0758fd990fbb20c1b6e1bdcb6ef30d5539ae18896302dc5077f99e7c +skill,audit-fixed-workload-claim,methods/audit-fixed-workload-claim,playbooks/measurement/audit-fixed-workload-claim,sha256:d6baa9a85b5f96b987e818659f08ea5fa366f9d033ddda5d1f9cacdd8dbaabec,sha256:c3de5160f4e66d255c147798e720849bcb69ce528fc4d62cc366cebc6ac43454 +skill,audit-flare-session-lifecycle,repositories/flare-local-inference/audit-flare-session-lifecycle,systems/engines/flare-local-inference/audit-flare-session-lifecycle,sha256:82fa5b82fe785403a97fbde21ef9e76595a07f43d8cd5a15aff95bf7742cc2e4,sha256:42f2b0c9726e9143a586369accef88f767d2b8f7fdf1ef5c53b8aedd407f389a +skill,audit-h3-metal-fasth3-semantics,repositories/h3-metal/audit-h3-metal-fasth3-semantics,systems/engines/h3-metal/audit-h3-metal-fasth3-semantics,sha256:dc7a7263c78052f906015552a8998afd832ee6432c6ac8bd751572a97b370862,sha256:21896e284a5584ba727b8841639c09aa05e985c6cea1c64a475c62869699b5df +skill,audit-h3-runtime-residency,repositories/h3-c/audit-h3-runtime-residency,systems/engines/h3-c/audit-h3-runtime-residency,sha256:11c628f851ac14b68073b4c973f9b84e37d4e04b7ed58d2ba37a9fd358e18bc1,sha256:3e0c1665c7957e8a4fd980146176c746a521ef972b267c420ba4be001d90520d +skill,audit-llama-runtime-state,repositories/llama-cpp/audit-llama-runtime-state,systems/engines/llama-cpp/audit-llama-runtime-state,sha256:ff92584a000d129dc8874774ea64dc2860ba6d135d5fde97a38adfbfaa10bd77,sha256:92d0320136c7acdbfa95db5b87a41d72b2ca1e1a4d4a49ae4b7d5803fe2631e6 +skill,audit-recurrent-kv-finalization,repositories/vllm-rlt/audit-recurrent-kv-finalization,systems/engines/vllm-rlt/audit-recurrent-kv-finalization,sha256:d1e3fa1900c4e395de4d3644a6af976abca666e019e6d6d56d4d7f853723f35d,sha256:a6e56602b29202af5478389dd0514af221e6e28868c439596ced22c82ce85629 +skill,audit-sd-cpp-residency,repositories/stable-diffusion-cpp/audit-sd-cpp-residency,systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency,sha256:8b51e7691301e0649c33e5442dca5beebfc28816784cca94c80cb7512b7469e8,sha256:506b40fcf0a41edb782464f0e8c12a16fa3b083405458fd0f934a26f919ce11a +skill,audit-sglang-diffusion-lifetimes,repositories/sglang/audit-sglang-diffusion-lifetimes,systems/engines/sglang/audit-sglang-diffusion-lifetimes,sha256:04642ba8170571cfa4445c060f7ede58dbda78ed7ca365f35d093e50795363e6,sha256:f3ab9e9b27ea0edc01c7c5ae37bfb504ff1de1d6e8ed48309edbbb6377e87dc3 +skill,audit-vllm-kv-lifetimes,repositories/vllm/audit-vllm-kv-lifetimes,systems/engines/vllm/audit-vllm-kv-lifetimes,sha256:eb9888f6c71bb313ca9516a58caff03c48c9078b6f85ace4f1a5b875ecd004f5,sha256:397043ec8301002750b5602802eaf91552bbc4fa350781ea4f1dafe87e44265e +skill,bound-windowed-attention-replay,repositories/deepinfer/bound-windowed-attention-replay,systems/engines/deepinfer/bound-windowed-attention-replay,sha256:ac6e6a80799c0fe3f348940af84084335d9feafed7bdb9cbb844cc260d2fb73b,sha256:bb51c49df442b3ac3fc1d1d0585ff2f3bffb5bac47997e2ea63524610af4f9a9 +skill,budget-speculative-verification,repositories/deepinfer/budget-speculative-verification,systems/engines/deepinfer/budget-speculative-verification,sha256:5edaeeebcc7c00890162ed2656abbd8b92ac6e349edc3719d11dfe01c1ac065d,sha256:2bc62cda2beff7dd3898e81f9c5eda756e860577d5f63321e548febab965977c +skill,calibrate-serving-simulation,methods/calibrate-serving-simulation,playbooks/measurement/calibrate-serving-simulation,sha256:38e8706e71b1cd17f269d3d1ddc080269a9bbc998f3df04eca1a9c33ad6740dd,sha256:4aef85416ac96b566b929bd0e45d387a173af99e057dfbbe181951cc4cd44684 +skill,cann-attribution-overlap,hardware/ascend/cann-attribution-overlap,systems/platforms/ascend/cann-attribution-overlap,sha256:41fbf471c406bf7ca086c20400d13018d567143d1687e17dcb8bb0c88bdfac71,sha256:359f5b8b0f2abd6bbaa27f5a3d2f96ef8df7de7c468643b859f6e02d7b6488cd +case,case.admission-drain-occupancy,methods/inference-optimization-casebook/cases/admission-drain-occupancy.json,casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.json,sha256:9219b7fec8bd72877925a27956b394e36b85c54775c8b97b35f0bd843830cb03,sha256:9219b7fec8bd72877925a27956b394e36b85c54775c8b97b35f0bd843830cb03 +case,case.drama3-rejected-directions,methods/inference-optimization-casebook/cases/drama3-rejected-directions.json,casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.json,sha256:4cd73101fe76691a6a7360242925fe731cd01db167ea0b485940e835e2ebafe7,sha256:4cd73101fe76691a6a7360242925fe731cd01db167ea0b485940e835e2ebafe7 +case,case.qwen-ascend-rejected-directions,methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json,casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json,sha256:86209f049d3f7bc7f5e09e6262a93ede238e14923cd411c9d2130eb83c0277e1,sha256:86209f049d3f7bc7f5e09e6262a93ede238e14923cd411c9d2130eb83c0277e1 +case,case.qwen-b300-rejected-directions,methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json,casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json,sha256:c6fbc3bc68158a174b687d360d6c74fb19b33204029e3c791690376b193752fe,sha256:c6fbc3bc68158a174b687d360d6c74fb19b33204029e3c791690376b193752fe +case,case.sarathi-chunk-budget,methods/optimization-search-policy/cases/sarathi-chunk-budget.json,playbooks/planning/optimization-search-policy/cases/sarathi-chunk-budget.json,sha256:f453054229ae7978890dbddf90c656f73d6cf9650a3230c5ec01570e92d5c462,sha256:f453054229ae7978890dbddf90c656f73d6cf9650a3230c5ec01570e92d5c462 +case,case.sglang-gdn-source-contract,methods/kernel-writing-guide/cases/sglang-gdn-source-contract.json,playbooks/optimization/kernel-writing-guide/cases/sglang-gdn-source-contract.json,sha256:2bef40d1b32a842c5afa24d4d2a43c972e4288aec0731f17d1c6d4b5dc10556f,sha256:2bef40d1b32a842c5afa24d4d2a43c972e4288aec0731f17d1c6d4b5dc10556f +case,case.vllm-gdn-source-contract,methods/kernel-writing-guide/cases/vllm-gdn-source-contract.json,playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-source-contract.json,sha256:6046e9d41bb2e85fe9cf9f97c9aab1ed9e4dcd22c476eac68fee7099b027c2eb,sha256:6046e9d41bb2e85fe9cf9f97c9aab1ed9e4dcd22c476eac68fee7099b027c2eb +skill,cgroup-resource-controls,hardware/host/cgroup-resource-controls,systems/platforms/host/cgroup-resource-controls,sha256:f12c8db5f86c03bae57e4853c7ace1f31aaf8444da02f9e37726e6ba9cca5c66,sha256:98a69101255260ddf6929698cc68b464aa1d4089483553f867a8e8455e1e93dc +skill,checkpoint-recovery-consistency,methods/checkpoint-recovery-consistency,playbooks/planning/checkpoint-recovery-consistency,sha256:f31c7f2e2254909c72a8fd775165478ca3a127b3033c1e6cfb05ee9076f84272,sha256:d0f9d6bd0594a0c4de8d734f1a88508faefd282511a4bc175e9bfd492a907a96 +skill,choose-parallel-training-plan,methods/choose-parallel-training-plan,playbooks/planning/choose-parallel-training-plan,sha256:c1d5bc51bc3e94006da7261f2f439b742c0fcd32478a50b0ee6e35ffff5de362,sha256:0d80fdea8ad2b6a9880848d0d0bf07a255871fb0d8778dd0e2944bd222495cd8 +skill,cite-check-downstream-pr,workflows/cite-check-downstream-pr,workflows/governance/cite-check-downstream-pr,sha256:b59a81c3420bb154eb1f7821da5bc899de38509dd5ffcdd99ce4db160f22e228,sha256:61b89c41a00b0f44cde7f933774411c3b85ab9871ef7242456f5cfc04a041cfd +skill,claim-contract,workflows/claim-contract,workflows/governance/claim-contract,sha256:117b361544ada6af2a1d9428ec2efcb72598dd590f53f595c8ece3e14f7b1109,sha256:8744eb3d57e51d4f42cc22f63086d27e6dc636676630391ad1a52bb72da516bb +skill,comfy-ui,repositories/comfy-ui,systems/engines/comfy-ui,sha256:0e9bb9faf0d72177ccafeb8439aa999aca737bc4887ef43b749ff5cd77c9fe8f,sha256:ef1faa7d9328b464aad10f409a40f8f2273a6ad4746772553f5df782958ff5c0 +skill,competitor-scan,workflows/competitor-scan,workflows/research/competitor-scan,sha256:5b4c3192a7dbaf135e0d220edb98a72934d11c1b985f8a19c4663ef6006f834f,sha256:ef21805d22a24ea0d4e0e4ff7e8fb63849905a1db14d16872aacc92d3d4ad8fa +skill,dcp-async-save-performance,frameworks/pytorch/dcp-async-save-performance,systems/frameworks/pytorch/dcp-async-save-performance,sha256:53139f3429d72bb796fc4e6a154b1122a503fe0019e9d2fd7b0a7f9229f7b86a,sha256:a56ab8c684ee899f5a3a4242b09be1319adf26ef5aac9ab02fd2f34f3a1c13ef +skill,dense-performance-evidence,methods/dense-performance-evidence,playbooks/measurement/dense-performance-evidence,sha256:5916990bcb5d676583e662824c265fba1d37c00ca9754954dc367e35575f189c,sha256:4a5cea1fc88bfd0a92302db79f668bc0bcb032ebe2e353beee3969715e24e533 +skill,diagnose-attention-bottlenecks,hardware/cuda/diagnose-attention-bottlenecks,systems/platforms/cuda/diagnose-attention-bottlenecks,sha256:9bd54d7131fa7f2403faa3b3c6a845ea2f022df70ae7992e026e3f0c00dad960,sha256:91adbfe01745db9f4251a4a071618d624d92a1fbfafe48f9277e2e0ddcfed819 +skill,diagnose-vllm-admission-pressure,repositories/vllm/diagnose-vllm-admission-pressure,systems/engines/vllm/diagnose-vllm-admission-pressure,sha256:2c30b6989736b1a05d7f42832dfc3d6c57493c9f6891526918d6686ea2919ce4,sha256:40915d4a09a8c0786aeb21fda9fda470ddbee56b7bd7d656c7ba27e9c0ea7de4 +skill,diffusers-invocation-state,frameworks/diffusers/diffusers-invocation-state,systems/frameworks/diffusers/diffusers-invocation-state,sha256:207e2a44aac3be624a0251e3e22c4f969fc6def18279c704221d1803d077d084,sha256:0e8d7420c81cbeaeef7def3a2cb74c916925dfc0b4fb615c76b2ec7c41e0f845 +skill,draft-acceptance-issue,workflows/draft-acceptance-issue,workflows/governance/draft-acceptance-issue,sha256:b9ac44938d14d6dac5ecef63aa9b7597a55a2bbb0bc3eb77d739b81cfd6538cf,sha256:5c15057d1322f9eefeaba649772679e9736191e9984fc77528aeeaf278637ee3 +skill,dwarfstar,repositories/dwarfstar,systems/engines/dwarfstar,sha256:198e61a45415cb667c19f939a073f8b792d02316b67c55817ac07a9a4ffed790,sha256:b3cc9a70b1c0bc5e7b49666a0b2b79fbda7fbc1ebaefc1a8d70bb9e8e1e98ec4 +skill,evaluate-elastic-serving-placement,methods/evaluate-elastic-serving-placement,playbooks/planning/evaluate-elastic-serving-placement,sha256:fd248599edc07d3b72431c087ae3d0d0ab9ac360f8cedfd0ef4b451186fd6cbc,sha256:0e10a38dbb353e98116d3478e59899c4b8fc0aa9fc19a823c2176bf4932c9d73 +skill,generation-cache-lifecycle,repositories/mlx-lm/generation-cache-lifecycle,systems/engines/mlx-lm/generation-cache-lifecycle,sha256:ba72a8d0bbcb8a8cb3df8976ea8190e388be6ab8e2209502423feaeb55cfec25,sha256:34fbae9024630b62fbece6702f9c91faa2618a1c31c1a313a894c114f1aad7bd +skill,host-memory-management,hardware/host/host-memory-management,systems/platforms/host/host-memory-management,sha256:7597c9ef77a3cf17e44dbcb0a4733260b6db9dae8fdc18a3292954213fca50d8,sha256:9df46c2daf5789c227d3135543a940304b78396631493b8d5f84cb63ea6aa0c7 +skill,host-overhead-optimization,hardware/host/host-overhead-optimization,systems/platforms/host/host-overhead-optimization,sha256:c79da9c995d6d72e7405901083cf6e2542d75b95a059110ccd61ba6973bccf63,sha256:121fc4f58c1a4088ef978eb06daec04df66844bb23508bda0bd13d79da3a848c +skill,host-overhead-profiling,hardware/cuda/host-overhead-profiling,systems/platforms/cuda/host-overhead-profiling,sha256:e4d4c6f40121164d44f80becc7d8af5a7f730655de23175316e1458d6af9a095,sha256:1fee30220142be8f46231348bb5b52c7f8602cde027b653a669ed32e08cb6105 +skill,husky,repositories/husky,systems/engines/husky,unavailable_at_cutover,unavailable_at_cutover +skill,inference-benchmark-design,methods/inference-benchmark-design,playbooks/measurement/inference-benchmark-design,sha256:1a5839566b5072cc76bc1eb3f7401b547b4799d9aba603db518f58989ed49e3d,sha256:a623f2e44fa0904d14baa5c2387061014bff20618a7b966c2d8b0676effac1b3 +skill,inference-optimization-casebook,methods/inference-optimization-casebook,casebooks/inference/inference-optimization-casebook,sha256:e04323f3717db845aa5730b7bae8d3caf57f001829ac5020896322e9b6446655,sha256:af6b98cf6a8a140075e2511bf21bfb783954c89111dbc94e2d30bd7b86179abb +skill,inference-task-router,,tasks/inference-task-router,,sha256:e1803be3152e6f90320357364f305deeb8cd4ef52f3a6a5b5db9167226d3f7a1 +skill,kernel-writing-guide,methods/kernel-writing-guide,playbooks/optimization/kernel-writing-guide,sha256:3e0e9e8715314ca82b1c7fd74433b8101f86288e434e9469603e47be6058a580,sha256:3ac2feac032b7fb661559311b1d44ea26fd20383e764c785a6a6ab661697e94c +skill,kv-tier-management,repositories/lmcache/kv-tier-management,systems/engines/lmcache/kv-tier-management,sha256:af8fa5ace8389d5ac78bea20d65fc5c88c933ab5da55cd4a05413f303632c7db,sha256:660a1c66749011a10e5a20c34e0ccf162011544b0fb25c3fe9b5378f5ffa801c +skill,lazy-evaluation-stream-lifetimes,frameworks/mlx/lazy-evaluation-stream-lifetimes,systems/frameworks/mlx/lazy-evaluation-stream-lifetimes,sha256:9f4b9e3fa38fdd563bca9d857641711fc63478faf65986b61dfa0156fae63d89,sha256:e6d96a7427ea32f544ba3b1c14094f38981de62d471545c2be00e9756092edbb +skill,lily,repositories/lily,systems/engines/lily,sha256:c2d4b015afe40620c553c33c9eb778828693d1595963542570c8b65d7475d239,sha256:d0f31b12b6cd5e663a85064f21969acf2583aa285333a41b2919616b098b0c2f +skill,lithos-metal,repositories/lithos-metal,systems/engines/lithos-metal,sha256:1bac5c3211295563646f01ae2af75391720976826d6676ddfccfc7892f6ea3a6,sha256:36e4fae8dcba3f8c8bfbf7fdbd6b24c46a20675ed00f013afe043c4df6fdd1a4 +skill,local-dream,repositories/local-dream,systems/engines/local-dream,sha256:e71deaa063931fd966ca2a8fb80ca4b5737c66bc820c215a1e1eb84adb041bf9,sha256:d264f6049de48eddee088bd9907a883f6e3517d8467677037a41bdf852468f2b +skill,mistral-rs,repositories/mistral-rs,systems/engines/mistral-rs,sha256:757744d70a9a229582b99d5dc2bc13b0d73786e7c439088a52db67c92b852601,sha256:e4e241ff2c31ac524bb5ea42eff039ab7d646d1d58b9540a6210476a5e7eb596 +skill,mlc-llm,repositories/mlc-llm,systems/engines/mlc-llm,sha256:b435c088c64346a929fca010ed4a0245cfb3763808727689a3475299a4dc0201,sha256:82e13b83d52a5270000b7ba7c2dbd12d782a8c0db1920715b19c5193edf0f070 +skill,mlx-swift-lm,repositories/mlx-swift-lm,systems/engines/mlx-swift-lm,sha256:27d02692e2e10b06395f0e21091355884a355b7244c6c2adc9270447d35634ae,sha256:fa3036ed4fc0f4a29673a833f2c7f9548b49016fe2255ad2d6023ba4ba922d2c +skill,npu-skill-factory,workflows/npu-skill-factory,workflows/maintenance/npu-skill-factory,sha256:bcf0628294b7f729e49e0437b74aee585a4f82404942d3e445d3fbf5e9aa4313,sha256:4a32eb345b5c632ddc93c679f67c1bcd7d4071cc86e68060b5a9843ee70d592c +skill,optimization-search-policy,methods/optimization-search-policy,playbooks/planning/optimization-search-policy,sha256:16216b0872b0837438b798ee65d26265451baf0acaa282a9cf16bdef0fb95eee,sha256:a37ff22f61bf8678d5a7002074bb88b24d2517ba3ac296e36975bb54b65d4fd1 +skill,os-process-placement,hardware/host/os-process-placement,systems/platforms/host/os-process-placement,sha256:51dc54e5263e15365035b76b5c966a6f9f53bfc5131818d0552b495d9c9aec24,sha256:f766de055f59cf7e351aabc35d01642c2e971ac8175ef3f0c950d63867b5717a +skill,performance-ceiling-analysis,methods/performance-ceiling-analysis,playbooks/measurement/performance-ceiling-analysis,sha256:2c56e58bfe664a6b61c28d64300252f67a8877caa1de33643321de506e51f962,sha256:ad83aaa41a10fc4915ea83ded9352911d3ae94dd47378f8d2e68db2fcb33ccaa +skill,plan-activation-residency,methods/plan-activation-residency,playbooks/planning/plan-activation-residency,sha256:ce985e5d42daac9201620c8149e4a3a8d8401951125973a89435b57e5dca4749,sha256:a8551582fdfdb107a2d635e35cd043042d6578512cf301a4940f390385002f17 +skill,project-maya,repositories/project-maya,systems/engines/project-maya,sha256:19e448ea2832dd1a0a9a3bd38019d678d32dd284e91da19141b28d7d481caec2,sha256:b6d52d493aa98e0261026475b692ad46fb6ce2c548b06c71ee79c96be90750da +skill,pytorch,frameworks/pytorch/pytorch,systems/frameworks/pytorch/pytorch,sha256:ca42adcf1674428169883557c6962d654b7307026b04f712c4e0a24e688f3397,sha256:e5ed588d3ca75005b0021182fe2a266b7ac3d263c0bbe0efa96e668f5cdd1fc5 +skill,reason-hopper-kernels,hardware/cuda/reason-hopper-kernels,systems/platforms/cuda/reason-hopper-kernels,sha256:89bab26b2a4666d3ee0c1fe5a0b110a61ac910b01bcba3c629c5006cc3e71d97,sha256:9b57f6812a5c46268c4f068a16608c0c7ee015b84079ae3564afc30ad546081e +skill,repository-evolution-analysis,methods/repository-evolution-analysis,workflows/research/repository-evolution-analysis,sha256:455e03625c93176cc9e8aabf5d83f2aeb4cb70d382902d53d595aad335efd11f,sha256:f10b5d57f2ae31ee7bcad9907d8374b925a821090f8a73a4d8c7567e7d23a491 +skill,rsi-evidence,workflows/rsi-evidence,workflows/governance/rsi-evidence,sha256:273da6715afa3ad96ce98567a6f4c68e347b0a93750e15cad8bd6a5ae538228b,sha256:959ade47c93ab2b5fbaa43332c597ae672c4e6428c2b53354b37284db87ac9cc +skill,skill-evolution-playbook,methods/skill-evolution-playbook,workflows/maintenance/skill-evolution-playbook,sha256:02fbd46234549d9a5762613f7f106330812f78680dfb48b2f8aa31119f624980,sha256:61ab81a692582f3f577d2ce04b6211e9587a420de2fc4414b95163fedb28570b +skill,splash,repositories/splash,systems/engines/splash,sha256:a457894180d533552d48fa3bb372c32e7ce20a3cefa3f2f1104aed148da8c520,sha256:b50a0e69182674faf115d40bd95256aff0eefde894794169e12861c2a0382428 +skill,storage-io-measurement,hardware/host/storage-io-measurement,systems/platforms/host/storage-io-measurement,sha256:1e952d85bcf822befe6952fcc9cd6848b6bca4fe7d6d3af05e6d2c6de01a1d39,sha256:f9ca652a5bddc2d9f20184fed81afdf122ec1b2d3f07d7368cc4683ab19fab3a +skill,task-accuracy,,tasks/task-accuracy,,sha256:5267d2773666629bdc6c49d5b79de77da70297722bd53f99ec1268c57a5848bb +skill,task-day-0,,tasks/task-day-0,,sha256:50c42cedf843870e6a8fa17219e7e3ce5049dd2b7cefa8305f8c3683b8833e95 +skill,task-debug,,tasks/task-debug,,sha256:9345d416516f3325ffe415b05d0e9e2e5a7c9144aa10a81ba6407b3ef8213879 +skill,task-derived-engine,,tasks/task-derived-engine,,sha256:ea82623db3db137d9e195f4e5c8cf3be42f0bc6199444f8a9a5530f65bbd0f75 +skill,task-independent-engine,,tasks/task-independent-engine,,sha256:f3e551786fe01450fdc827e1cd4ca22441a7231ea84421c0997dcd5733793800 +skill,task-performance,,tasks/task-performance,,sha256:38f6e79eb4c8cc728ed6a0cd71923c22f81e112b82ce23ddd6f773cb2e799f74 +skill,task-version-alignment,,tasks/task-version-alignment,,sha256:fe85ab688e4ff47dcf5b39304d04a20c5a71891e0e86452edd6132f368324553 +skill,torch-npu,frameworks/torch-npu/torch-npu,systems/frameworks/torch-npu/torch-npu,sha256:dab34d1dcd1fe61b1fd1223d592a3d1ad2d878ae5d3f6d4edf8d1ec29487462f,sha256:613aa917878ee15b82f5553e53a19f857eb765b8de7f8b12e1c9cb97974924a8 +skill,torchnpu-device-memory-lifecycle,hardware/ascend/torchnpu-device-memory-lifecycle,systems/platforms/ascend/torchnpu-device-memory-lifecycle,sha256:3f9eb5aab18fa97d54f233d550b85c9e6f7ccbef971cca70027fa4fa23dcc74c,sha256:f03c1bcd8f076d178c47e8e059f56ecad98e62748d801a2bbeda610c9f7dafef +skill,torchnpu-pinned-host-memory-lifecycle,hardware/ascend/torchnpu-pinned-host-memory-lifecycle,systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle,sha256:474f85c692fd6641c026bac8c41e8d188076ca0fcf7d19e555aa8f0e93d151b1,sha256:a3b5322477a60ba1c5ec77c7db942a167d2b1b8e0fa072d6b2f0a0895f7b17c0 +skill,training-input-pipeline,methods/training-input-pipeline,playbooks/optimization/training-input-pipeline,sha256:ba83e66fb6c16755771185b749315aed8a34a2f657c90c535519a4f33c4845ca,sha256:c4f9ec76f0d0562338b7ac300f88ec44940f9b2166dd605b07940d5bd96ec7b8 +skill,training-resource-budget,methods/training-resource-budget,playbooks/planning/training-resource-budget,sha256:fc62feb1db046829d5fb2fb8e34e41289e82708864e29bcf6bee042712652bc4,sha256:6518ed8ac0acb6a115f268ad92cb00fbd4dd1facbd75f3637017b87d30edf766 +skill,video-diffusion-optimization,methods/video-diffusion-optimization,playbooks/optimization/video-diffusion-optimization,sha256:a96e4f59b6d10eaa6844d00af6a36ad4d2c74292b7ca1430c9bbcf05acdc5e07,sha256:ecab0de13854a7cb554664a4990324da330aa0cd072d38ad86357855d1c571bc +skill,vllm-ascend,repositories/vllm-ascend,systems/engines/vllm-ascend,sha256:ac23b18fa700b198f8fec56b985c431494cb92dc24214604e9d17f506b7d7243,sha256:99727482a25faea9453eaed7bbf9bbe2daa099dcfd59dc260eef97b75f58efc7 +skill,vllm-metal,repositories/vllm-metal,systems/engines/vllm-metal,sha256:7894e1e928641af983fc50f17fc28dce58b155580770d5a5fe320a3c4b136d43,sha256:7b2526daf2f91cef733a46ffe23258b97d7a73839bb07602ad52bd75d2c980c7 +skill,weight-loading-optimization,methods/weight-loading-optimization,playbooks/optimization/weight-loading-optimization,sha256:0312bb29e803700cebbf4bb97cee969fdcbd94b977344f9e4ab4788514966bbe,sha256:6695449f13bbd0e832690f6e390acfb464b711aa53b997100fea05d0878deef9 diff --git a/evidence/_template/plan/campaign-plan.md b/evidence/_template/plan/campaign-plan.md index 0a3476f41..d744b39d0 100644 --- a/evidence/_template/plan/campaign-plan.md +++ b/evidence/_template/plan/campaign-plan.md @@ -29,7 +29,7 @@ versioned search strategy and search state, and is expected to change. For row 3, use user-provided criteria, including a referenced accepted plan. When none are supplied, follow the -[ceiling-analysis acceptance procedure](../../../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/acceptance-criteria.md). +[ceiling-analysis acceptance procedure](../../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md). Keep partial requirements and label derived fields. Record the criteria source (`user`, `ceiling-derived` or `mixed`), request/plan reference, derived fields, analysis evidence (skill revision, baseline, calibration, attainable range, @@ -62,8 +62,8 @@ D-01 | plane= | stage=, of E2E One variable per round and one plane per step. Justify conditional headroom against the campaign target and locally measured noise, following -[performance-ceiling-analysis](../../../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md) -and its [bounds guidance](../../../src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md). +[performance-ceiling-analysis](../../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md) +and its [bounds guidance](../../../src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md). Record conditionally removable exposed critical-path time, assumptions and uncertainty. A non-dominant stage is not by itself a reason for rejection: overlap, synchronization, capacity, queueing, batching and bottleneck migration can diff --git a/plugins/inferrsi/plugin.json b/plugins/inferrsi/plugin.json index f3bdaf053..d95c2731c 100644 --- a/plugins/inferrsi/plugin.json +++ b/plugins/inferrsi/plugin.json @@ -2,7 +2,7 @@ "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", "name": "inferrsi", "version": "0.1.0", - "description": "Research inference optimizations and prepare, run and inspect evidence-based comparisons.", + "description": "Bring up models and engines, align inference stacks, diagnose performance, accuracy and runtime failures, and verify changes against evidence.", "author": {"name": "InferMatrixLab"}, "homepage": "https://github.com/InferMatrixLab/InferRSI", "repository": "https://github.com/InferMatrixLab/InferRSI", diff --git a/plugins/inferrsi/skills/inferrsi/SKILL.md b/plugins/inferrsi/skills/inferrsi/SKILL.md index 5eda5b810..f70dcf6f6 100644 --- a/plugins/inferrsi/skills/inferrsi/SKILL.md +++ b/plugins/inferrsi/skills/inferrsi/SKILL.md @@ -1,6 +1,6 @@ --- name: inferrsi -description: Use InferRSI to research inference-engine bottlenecks, run optimization task sessions, prepare controlled baseline/candidate comparisons, and inspect experiment evidence. Use when the user asks to use InferRSI or run an optimization benchmark with it. +description: Use InferRSI for day-zero model support, performance optimization, debugging, accuracy repair, engine and plugin version alignment, derived-engine development and independent runtime bring-up. Use it to research bottlenecks, run benchmark sessions and inspect controlled comparisons and evidence. --- # InferRSI @@ -42,6 +42,34 @@ Keep the current directory in the target workspace. Pass the intended Use `doctor` only when target inspection is useful; knowledge-only research does not require a device probe. A detected backend is not model qualification. +## Load the primary inference task + +For every inference task, select the row matching the user's requested deliverable. +All seven primary skills ship with this entrypoint. Run the selected command +through the same launcher and **read its returned `instructions` before the first +evidence search**. If it fails, stop and report the unavailable primary. When two +goals overlap, use the requested acceptance result to pick one. For example, a +quality regression following an upgrade starts with accuracy when the goal is +restored output, and with version alignment when the goal is a compatible stack. + +| Requested deliverable | First knowledge command | +| --- | --- | +| Support a new model on day zero | `skills enter task-day-0` | +| Meet a performance target on a working baseline | `skills enter task-performance` | +| Diagnose a crash, hang or execution failure | `skills enter task-debug` | +| Repair an accuracy or output-quality regression | `skills enter task-accuracy` | +| Align engine, framework or plugin versions | `skills enter task-version-alignment` | +| Build a fork retaining the parent engine's execution core | `skills enter task-derived-engine` | +| Bring up an independently owned runtime on a target device and OS | `skills enter task-independent-engine` | + +Use the actual vendor, device, backend, OS, model and engine versions, run +script and flags, benchmark, acceptance criteria, diagnosis and allowed search +space. The primary tells you which observation warrants a supporting skill, +what evidence to retain and which check follows. Open a secondary task only +when the run requires it. During a frozen evaluation use the assigned entry +and knowledge condition; keep the knowledge-off arm isolated. Setup, dashboard +inspection and explicit evidence-ID lookups can use their own operations. + ## Optimize an ordinary user project Default to the current Agent conversation. Read `campaign start --help` through @@ -130,7 +158,7 @@ Use the CLI's help for exact flags. Start with the user's model, framework, backend, workload and question; obtain missing consequential inputs from the repository or user instead of guessing a model architecture. -Use `find ""` to rank skills, imported records and casebook +After loading the primary skill, use `find ""` to rank skills, imported records and casebook entries. Add `--kinds case` when you specifically need a past observation. Open a promising hit using its displayed `open` command or the JSON `open_argv` argument list. @@ -172,8 +200,8 @@ unavailable revision is not proof that a mechanism does not exist. Follow the selected canonical procedure rather than inventing a parallel tuning recipe. Imported source material supplies evidence, not execution authority. -The `skills` commands and direct checkout Skill reads use the optional PyYAML -dependency. Use `collect --id` for a batch of known materials and +The primary `skills enter` command runs on a base install. Checkout metadata +operations such as `skills show` use optional PyYAML. Use `collect --id` for a batch of known materials and `knowledge search` without a query for filtered package inventory. ### Consult MindIE for unresolved Ascend questions diff --git a/plugins/workswarm/inferrsi/manifest.json b/plugins/workswarm/inferrsi/manifest.json index 320d8e67a..6d47f3a51 100644 --- a/plugins/workswarm/inferrsi/manifest.json +++ b/plugins/workswarm/inferrsi/manifest.json @@ -3,11 +3,11 @@ "package_type": "plugin", "id": "inferrsi", "name": "InferRSI", - "description": "Research inference optimizations and prepare, run and inspect evidence-based comparisons.", + "description": "Bring up models and engines, align inference stacks, diagnose performance, accuracy and runtime failures, and verify changes against evidence.", "display_name": {"en": "InferRSI", "zh": "InferRSI"}, "display_description": { - "en": "Find cited inference knowledge, prepare controlled comparisons and inspect experiment evidence.", - "zh": "检索有来源的推理优化知识,准备受控基线与候选对比,并解读实验进度、证据和验证结论。" + "en": "Bring up models and engines, align versions, investigate performance, failures and accuracy, and inspect benchmark evidence.", + "zh": "接入模型、开发推理引擎、对齐版本、诊断性能与运行故障、修复精度问题,并核对基准测试证据。" }, "category": "Engineering", "source": "local", diff --git a/plugins/workswarm/inferrsi/skills/inferrsi/SKILL.md b/plugins/workswarm/inferrsi/skills/inferrsi/SKILL.md index 2064f6487..6ad0d78ab 100644 --- a/plugins/workswarm/inferrsi/skills/inferrsi/SKILL.md +++ b/plugins/workswarm/inferrsi/skills/inferrsi/SKILL.md @@ -1,6 +1,6 @@ --- name: inferrsi -description: Research inference-engine bottlenecks with InferRSI, run optimization task sessions, prepare controlled baseline/candidate comparisons, and inspect experiment evidence. Use when the user asks to use InferRSI or run an optimization benchmark with it. +description: Use InferRSI for day-zero model support, performance optimization, debugging, accuracy repair, engine and plugin version alignment, derived-engine development and independent runtime bring-up. Use it to research bottlenecks, run benchmark sessions and inspect controlled comparisons and evidence. --- # InferRSI in WorkSwarm @@ -34,12 +34,40 @@ operations require no separate pip installation. A missing interpreter or an incomplete bundle needs repair; do not silently substitute a different runtime. Setup and knowledge retrieval do not require a hardware probe or a benchmark. +## Load the primary inference task + +For every inference task, select the row matching the user's requested deliverable. +All seven primary skills ship with this entrypoint. Run the selected command +through the same launcher and **read its returned `instructions` before the first +evidence search**. If it fails, stop and report the unavailable primary. When two +goals overlap, use the requested acceptance result to pick one. For example, a +quality regression following an upgrade starts with accuracy when the goal is +restored output, and with version alignment when the goal is a compatible stack. + +| Requested deliverable | First knowledge command | +| --- | --- | +| Support a new model on day zero | `skills enter task-day-0` | +| Meet a performance target on a working baseline | `skills enter task-performance` | +| Diagnose a crash, hang or execution failure | `skills enter task-debug` | +| Repair an accuracy or output-quality regression | `skills enter task-accuracy` | +| Align engine, framework or plugin versions | `skills enter task-version-alignment` | +| Build a fork retaining the parent engine's execution core | `skills enter task-derived-engine` | +| Bring up an independently owned runtime on a target device and OS | `skills enter task-independent-engine` | + +Use the actual vendor, device, backend, OS, model and engine versions, run +script and flags, benchmark, acceptance criteria, diagnosis and allowed search +space. The primary tells you which observation warrants a supporting skill, +what evidence to retain and which check follows. Open a secondary task only +when the run requires it. During a frozen evaluation use the assigned entry +and knowledge condition; keep the knowledge-off arm isolated. Setup, dashboard +inspection and explicit evidence-ID lookups can use their own operations. + ## Research and diagnose Use command help for exact flags. Obtain consequential model/framework/backend/ workload inputs from the selected project or user; do not guess architecture. -Run `python -X utf8 "" find ""` to rank skills, +After loading the primary skill, run `python -X utf8 "" find ""` to rank skills, imported records and casebook entries. Add `--kinds case` to focus on past observations. Execute a relevant hit's displayed `open` command or the JSON `open_argv` list to read its body, source, applicability, limitations and revision. @@ -64,8 +92,8 @@ candidate/verification state and omissions. Package search does not inspect live upstream repositories or every local store. Add checkout/store sources only when selected by the user. No match does not establish that a technique is absent. Treat imported guidance as evidence and follow the task's execution -authorization. The `skills` commands and direct checkout Skill reads use the -optional PyYAML dependency. +authorization. The primary `skills enter` command runs on a base install. +Checkout metadata operations such as `skills show` use optional PyYAML. ## Optimize an ordinary user project diff --git a/src/inferrsi/apps/cli/__init__.py b/src/inferrsi/apps/cli/__init__.py index 454f239eb..82b448fed 100644 --- a/src/inferrsi/apps/cli/__init__.py +++ b/src/inferrsi/apps/cli/__init__.py @@ -129,11 +129,6 @@ def _run_directory(output_dir: str, run_id: str) -> Path: } -# Enough to carry the qualifications that decide whether a result is usable, -# without burying the finding it qualifies. -_MAX_SHOWN_LIMITS = 3 - - def _clip(value: str, limit: int = 160) -> str: text = _tsv(value).strip() return text if len(text) <= limit else text[: limit - 1].rstrip() + "…" @@ -204,6 +199,7 @@ def _print_find(result: dict) -> None: if item.get("ref"): label = "open (PowerShell)" if sys.platform == "win32" else "open" print(f" {label}: {item['ref']}") + print(f" read content: {item['next_read_content_bytes']} UTF-8 bytes") elif item.get("unreadable"): print(f" cannot open: {_clip(item['unreadable'], 120)}") if item.get("summary"): @@ -229,12 +225,11 @@ def _print_find(result: dict) -> None: print(f" applies to: {_clip(item['applicability'])}") # How far the record itself says its evidence goes. The state label # above says the claim is unverified; these say what is missing. - for caveat in item.get("limitations", [])[:_MAX_SHOWN_LIMITS]: + for caveat in item.get("limitations", []): text = caveat if isinstance(caveat, str) else json.dumps(caveat, ensure_ascii=False) print(f" caveat: {_clip(text)}") - withheld = len(item.get("limitations", [])) - _MAX_SHOWN_LIMITS - if withheld > 0: - print(f" ... {withheld} more recorded limitation(s); see --json") + if withheld := item.get("limitations_withheld"): + print(f" ... {withheld} more recorded limitation(s) in the full material") if item.get("path"): print(f" -> {item['path']}") if item.get("source"): @@ -393,6 +388,7 @@ def main(argv: list[str] | None = None) -> int: # CLI stays independent of them. from ...knowledge.skill_catalog import ( get_skill, + primary_task_instructions, skill_instructions, skill_records, skill_reference, @@ -409,6 +405,9 @@ def main(argv: list[str] | None = None) -> int: else: for record in records: print(f"{record['name']}\t{record['description']}") + elif args.skills_action == "enter": + instructions = primary_task_instructions(root, args.name) + _print({"name": args.name, "instructions": instructions}) elif args.skills_action == "show": record = get_skill(root, args.name) record["instructions"] = skill_instructions(root, args.name) diff --git a/src/inferrsi/apps/cli/arguments.py b/src/inferrsi/apps/cli/arguments.py index 07fec480c..1aadaafec 100644 --- a/src/inferrsi/apps/cli/arguments.py +++ b/src/inferrsi/apps/cli/arguments.py @@ -486,8 +486,12 @@ def parser(*, parser_class=argparse.ArgumentParser) -> argparse.ArgumentParser: help="Skill library or flat skill directory (default: installed library)", ) actions = skills.add_subparsers(dest="skills_action", required=True) - listing = actions.add_parser("list", help="List frontmatter and sidecar metadata only") + listing = actions.add_parser("list", help="List skill metadata without the instruction body") listing.add_argument("--json", action="store_true") + enter = actions.add_parser( + "enter", help="Load the selected primary inference task before evidence search" + ) + enter.add_argument("name", help="Registered primary task skill (for example, task-accuracy)") show = actions.add_parser("show", help="Load one skill's instructions and resource names") show.add_argument("name") resource = actions.add_parser("resource", help="Resolve one resource without executing it") diff --git a/src/inferrsi/apps/workflows/campaign.py b/src/inferrsi/apps/workflows/campaign.py index ccb45d236..92242a9c1 100644 --- a/src/inferrsi/apps/workflows/campaign.py +++ b/src/inferrsi/apps/workflows/campaign.py @@ -98,12 +98,18 @@ def _supplies_no_observation(result: dict) -> bool: key in view for view in result.get("items", []) for key in PROVENANCE_FIELDS) +def _captured_resource(resource: dict) -> dict: + keys = ("path", "role", "sha256", "content") + if resource.get("locator") == "skill-procedure": + keys += ("locator", "source_sha256", "source_byte_ranges") + return {key: resource[key] for key in keys} + + def _contexts(result: dict) -> list[dict]: return [{ "ref": context["ref"], "revision": context["revision"], "content_bytes": context["content_bytes"], - "resources": [{key: resource[key] for key in ("path", "role", "sha256", "content")} - for resource in context["resources"]], + "resources": [_captured_resource(resource) for resource in context["resources"]], "omissions": context["omissions"], "sources": context.get("sources", []), } for context in result.get("contexts", [])] diff --git a/src/inferrsi/knowledge/_skill_selection.py b/src/inferrsi/knowledge/_skill_selection.py index b7a6b74a3..910b09623 100644 --- a/src/inferrsi/knowledge/_skill_selection.py +++ b/src/inferrsi/knowledge/_skill_selection.py @@ -1,11 +1,12 @@ from __future__ import annotations +import hashlib import json import posixpath import re from ._source_snapshot import source_entry -from .catalog import packaged_skill_id +from .catalog import _skill_search_source, packaged_skill_id FIND_RELEVANCE = "find-lexical-v1" TASK_STOP_WORDS = frozenset({"a", "an", "and", "for", "in", "of", "on", "the", "to", "with"}) @@ -130,12 +131,32 @@ def record_context(record: dict, view: dict, content: dict[str, bytes], availabl }, None +def skill_procedure(path: str, raw: bytes) -> tuple[bytes, dict]: + """Select the instruction bytes of a single-file skill for a budgeted context.""" + if posixpath.basename(path) != "SKILL.md": + return raw, {} + try: + text = raw.decode("utf-8") + except UnicodeDecodeError: + return raw, {} # The caller handles non-UTF8 optional resources. + procedure, metadata = _skill_search_source(text) + if metadata is None: + return raw, {} + data = procedure.encode("utf-8") + # The projection removes one contiguous block. These half-open UTF-8 byte + # ranges reconstruct the delivered bytes from the exact pinned file. + start = next((i for i, (a, b) in enumerate(zip(raw, data)) if a != b), len(data)) + end = start + len(raw) - len(data) + return data, {"locator": "skill-procedure", "source_byte_ranges": [[0, start], [end, len(raw)]]} + + def context_view( index: tuple[str, list[str], list[str]], view: dict, content: dict[str, bytes], cue: str | None, available: int, ) -> tuple[dict | None, dict | None]: core, required, optional = index - required_bytes = sum(len(content[path]) for path in required) + selected = {path: skill_procedure(path, content[path]) for path in [*required, *optional]} + required_bytes = sum(len(selected[path][0]) for path in required) if required_bytes > available: return None, { "ref": view["ref"], "reason": "context_limit", @@ -147,12 +168,17 @@ def context_view( def include(path: str) -> None: nonlocal used - resources.append({ + data, locator = selected[path] + resource = { "path": path, "sha256": view["resources"][path], "locator": "whole-resource", "role": "core" if path == core else "reference", - "content": content[path].decode("utf-8"), - }) - used += len(content[path]) + "content": data.decode("utf-8"), + } + if locator: + resource.update(**locator, source_sha256=view["resources"][path], + sha256=hashlib.sha256(data).hexdigest()) + resources.append(resource) + used += len(data) for path in required: include(path) @@ -165,7 +191,7 @@ def include(path: str) -> None: omissions.append({"path": path, "reason": "not_context_resource"}) continue try: - candidates[path] = content[path].decode("utf-8") + candidates[path] = selected[path][0].decode("utf-8") except UnicodeDecodeError: omissions.append({"path": path, "reason": "not_utf8"}) continue @@ -189,7 +215,7 @@ def include(path: str) -> None: for path in candidates if path not in matched ) for path in paths: - if used + len(content[path]) > available: + if used + len(selected[path][0]) > available: omissions.append({"path": path, "reason": "context_limit"}) continue include(path) @@ -198,7 +224,7 @@ def include(path: str) -> None: "content_bytes": used, "omissions": omissions, "sources": view["sources"], "attribution": view["attribution"], "limitations": view["limitations"] + [ - "Selected resources are complete files; this view is not a standalone Bundle.", + "For a skill-procedure, concatenate the reported half-open byte ranges of its source.", "Follow omitted or other linked resources through the exact owner revision.", ], }, None diff --git a/src/inferrsi/knowledge/_source_snapshot.py b/src/inferrsi/knowledge/_source_snapshot.py index 24dff851f..ea02a9d25 100644 --- a/src/inferrsi/knowledge/_source_snapshot.py +++ b/src/inferrsi/knowledge/_source_snapshot.py @@ -3,17 +3,13 @@ import re from typing import Any -# One shared snapshot keeps the repository-wide source revision and registry digest out of -# per-owner records, so a one-owner edit regenerates only that owner's record. -# LIBRARY is the repository path of the corpus at the pinned source_revision. -# After the K02 move and the explicit re-pin, this agrees with the current checkout -# path src/inferrsi/knowledge/data/skill_library. skill_documents.original and -# wheel bundle keys stay at the package-relative prefix data/skill_library/. +# The build pin names the source corpus; installed bundle keys remain package-relative. MANIFEST = "_provenance.json" REPOSITORY = "https://github.com/InferMatrixLab/InferRSI" LIBRARY = "src/inferrsi/knowledge/data/skill_library" BUNDLE_LIBRARY = "data/skill_library" REGISTRY = f"{LIBRARY}/skill-index.json" +# Historical schema for pre-unification snapshots read by existing callers. STORED_SCHEMA = 2 diff --git a/src/inferrsi/knowledge/catalog.py b/src/inferrsi/knowledge/catalog.py index 361a4a6b9..8972a3e11 100644 --- a/src/inferrsi/knowledge/catalog.py +++ b/src/inferrsi/knowledge/catalog.py @@ -6,6 +6,7 @@ import json import re from importlib.resources import files +from pathlib import PurePosixPath from typing import TYPE_CHECKING, Any from ..resources.repository_packs import REPO_TYPES @@ -97,66 +98,226 @@ def _validate_record( raise ValueError(f"Imported knowledge must state locally_verified=false: {record['id']}") -def knowledge_records(*, file_fingerprints: bool = True) -> list[dict[str, Any]]: - data = files("inferrsi").joinpath("knowledge", "data") +REPOSITORY = _source_snapshot.REPOSITORY +COMPILED = "_compiled" +INPUTS = "inputs.json" + + +DEFAULT_RECOMMENDATIONS = [ + "Read the pinned SKILL.md and its prerequisites before choosing this procedure.", + "Read the packaged references and owner-local cases alongside the procedure.", +] + + +def recommendations(metadata: dict) -> list[str]: + """Owner-supplied procedure advice, or generic navigation when absent. + + A declared empty list means the owner states there is no procedure-specific + advice; it is not the same as an absent field and does not fall back. + """ + if "recommendations" in metadata: + return list(metadata["recommendations"]) + return list(DEFAULT_RECOMMENDATIONS) + + +def _split_skill_metadata(header: str) -> tuple[str, dict | None]: + """Separate the Agent Skills metadata.inferrsi block without importing YAML.""" + # Only the canonical literal-block representation is read without PyYAML. + # The source builder checks its interpretation against the full YAML mapping. + section = re.search(r"^metadata:[ \t]*(?:#[^\n]*)?\r?\n", header, re.MULTILINE) + if section is None: + if re.search(r"""^metadata:.*\binferrsi['"]?\s*:""", header, re.MULTILINE): + raise ValueError("Use a metadata.inferrsi literal block of JSON with 2/4-space indents") + return header, None + rest = header[section.end():] + next_field = re.search(r"^[^ \t#\r\n]", rest, re.MULTILINE) + end = section.end() + next_field.start() if next_field else len(header) + children = header[section.end():end] + marker = re.search( + r"^ inferrsi:[ \t]+\|[+-]?[ \t]*(?:#[^\n]*)?\r?\n", children, re.MULTILINE, + ) + if marker is None: + if re.search(r"""^ +['"]?inferrsi['"]?\s*:""", children, re.MULTILINE): + raise ValueError("Use a metadata.inferrsi literal block of JSON with 2/4-space indents") + return header, None + block = re.match(r"(?: [^\n]*(?:\n|$)|\r?\n)+", children[marker.end():]) + if block is None: + raise ValueError("Missing metadata.inferrsi JSON object") + data = json.loads(block.group()) + if not isinstance(data, dict): + raise ValueError("Skill metadata.inferrsi must contain a JSON object") + siblings = children[:marker.start()] + children[marker.end()+block.end():] + if any(line.strip() and not line.lstrip().startswith("#") for line in siblings.splitlines()): + return header[:section.end()] + siblings + header[end:], data + return header[:section.start()] + header[end:], data + + +def _skill_search_source(text: str) -> tuple[str, dict | None]: + """Preserve the original lexical fields. Structured metadata has its own weights.""" + if "inferrsi" not in text: + return text, None + match = re.match(r"\A---[ \t]*\r?\n(.*?)^---[ \t]*\r?$", text, re.MULTILINE | re.DOTALL) + if match is None: + return text, None + header, metadata = _split_skill_metadata(match[1]) + return text[:match.start(1)] + header + text[match.end(1):], metadata + + +def skill_views(root, *, file_fingerprints=True, record_id: str | None = None): + """The only compiled inputs are YAML headers and immutable source hashes.""" + built = root.joinpath(COMPILED) + snapshot = json.loads(built.joinpath(_source_snapshot.MANIFEST).read_bytes()) + revision, registry_hash = _source_snapshot._snapshot(snapshot) + raw = root.joinpath("skill-index.json").read_bytes() + if file_fingerprints and hashlib.sha256(raw).hexdigest() != registry_hash: + raise ValueError("Skill registry disagrees with its pinned snapshot") + registry = json.loads(raw) + if registry.get("schema_version") != 1 or not isinstance(registry.get("skills"), dict): + raise ValueError("Invalid packaged skill registry") + entries = registry["skills"] + inputs = json.loads(built.joinpath(INPUTS).read_bytes()) + if not isinstance(inputs, dict) or (record_id is None and entries.keys() != inputs.keys()): + raise ValueError("Compiled skill IDs disagree with registry") + for name, registration in entries.items(): + if record_id is not None and packaged_skill_id(name, registration["backend"]) != record_id: + continue + relative = registration["path"] + path = PurePosixPath(relative) + if path.is_absolute() or ".." in path.parts or path.name != name or "\\" in relative: + raise ValueError(f"Invalid registered skill path: {relative}") + directory = f"{_source_snapshot.LIBRARY}/{relative}" + owner = root.joinpath(*path.parts) + data = inputs.get(name) + if (not isinstance(data, dict) or set(data) != {"header", "files"} + or not isinstance(data["header"], dict) or not isinstance(data["files"], dict)): + raise ValueError(f"Invalid compiled inputs for {name}") + metadata = dict(data["header"]) + if metadata.get("name") != name or not metadata.get("description"): + raise ValueError(f"Invalid compiled header for {name}") + fingerprints = data["files"] + if "SKILL.md" not in fingerprints: + raise ValueError(f"Missing entry fingerprint for {name}") + for key, digest in fingerprints.items(): + if (not key or key.startswith("/") or "\\" in key + or any(p in {"", ".", ".."} for p in key.split("/")) + or not isinstance(digest, str) or not re.fullmatch(r"[0-9a-f]{64}", digest)): + raise ValueError(f"Invalid source fingerprint for {name}: {key}") + raw = owner.joinpath("SKILL.md").read_bytes() + if file_fingerprints and hashlib.sha256(raw).hexdigest() != fingerprints["SKILL.md"]: + raise ValueError(f"Stale skill metadata for {name}") + _, embedded = _skill_search_source(raw.decode("utf-8")) + if embedded is not None: + if "metadata.json" in fingerprints: + raise ValueError(f"Skill declares metadata in two files: {name}") + metadata.update(embedded) + elif "metadata.json" in fingerprints: + # A historical compiled snapshot can still be opened at its own pin. + legacy = owner.joinpath("metadata.json").read_bytes() + if (file_fingerprints + and hashlib.sha256(legacy).hexdigest() != fingerprints["metadata.json"]): + raise ValueError(f"Stale skill metadata for {name}") + metadata.update(json.loads(legacy)) + backend = registration["backend"] + state = metadata.get("status", "unspecified") + entry = directory + "/SKILL.md" + hashes = {} + if file_fingerprints: + hashes = dict(sorted({ + _source_snapshot.REGISTRY: registry_hash, + **{f"{directory}/{k}": v for k, v in fingerprints.items()}, + }.items())) + paths = list(hashes) + yield { + "schema_version": 1, + "id": packaged_skill_id(name, backend), + "title": name, + "repo": "inferrsi", + "tier": "contribution", + "backend": backend, + "component": "skills", + "scenario": "skills.discovery", + "status": "imported", + "summary": metadata["description"], + "tags": metadata.get("tags", []), + "origin": metadata.get("origin", ["unknown"]), + "sources": ([{ + "type": "git", + "url": f"{REPOSITORY}/blob/{revision}/{entry}", + "commit": revision, + "paths": paths, + "content_sha256": hashes[entry], + }] if file_fingerprints else [{ + "type": "git", "commit": revision, "url": f"{REPOSITORY}/tree/{revision}", + }]), + "index_type": "packaged-skill", + "skill_documents": { + "original": f"data/skill_library/{relative}/SKILL.md", + }, + "checkout_ref": f"checkout:{name}", + "source_files": hashes, + "skill_metadata": metadata, + "applicability": metadata.get("applicability", {}), + "recommendations": recommendations(metadata), + "limitations": [ + "This generated abstract points to the complete packaged skill; " + "external evidence and upstream repositories remain source links.", + "The imported status describes this index, not the original skill; " + f"its declared status is {state}.", + "Reading or indexing does not authorize execution, validate efficacy, " + "promote a candidate or grant a license.", + "The backend categorizes the procedure's hardware dependency; it does not " + "certify compatibility, and skills.discovery is a discovery scenario.", + *metadata.get("limitations", []), + ], + "verification": {"locally_verified": False, "evidence_level": "guidance"}, + } + + +def _authored_record(item) -> dict: + if not item.is_file() or not item.name.endswith(".json"): + raise ValueError(f"Unexpected authored knowledge resource: {item.name}") + parts = item.name[:-5].split(".", 3) + if len(parts) != 4: + raise ValueError(f"Invalid knowledge filename: {item.name}") + repo, backend, component, entry = parts + record = json.loads(item.read_bytes()) + if not isinstance(record, dict) or record.get("index_type") == "packaged-skill": + raise ValueError(f"Invalid authored knowledge record: {item.name}") + tier = record.get("tier", "contribution") + if repo not in KNOWLEDGE_SCOPES.get(tier, ()): + raise ValueError(f"Knowledge tier disagrees with repository scope: {item.name}") + _validate_record(record, repo, (backend, component, entry + ".json"), tier) + if item.name != record["id"] + ".json": + raise ValueError(f"Knowledge ID disagrees with filename: {item.name}") + return record + + +def knowledge_records( + *, file_fingerprints: bool = True, include_skill_views: bool = True, +) -> list[dict[str, Any]]: + root = files("inferrsi").joinpath("knowledge", "data", "skill_library") records = [] - seen = set() - for tier, repos in KNOWLEDGE_SCOPES.items(): - for repo in repos: - knowledge = ( - data.joinpath("repo_packs", repo, "knowledge") - if tier == "contribution" - else data.joinpath("knowledge_packs", tier, repo) - ) - if not knowledge.is_dir(): - continue - snapshot = knowledge.joinpath(_source_snapshot.MANIFEST) - snapshot = ( - json.loads(snapshot.read_text(encoding="utf-8")) if snapshot.is_file() else None - ) - for parts, item in _walk(knowledge): - if not item.name.endswith(".json") or parts == (_source_snapshot.MANIFEST,): - continue - record = json.loads(item.read_text(encoding="utf-8")) - # Generated skill records store schema v2; readers always receive schema v1. - # A pack with a snapshot holds only generated records. - if isinstance(record, dict) and record.get("schema_version") == ( - _source_snapshot.STORED_SCHEMA - ): - if snapshot is None: - raise ValueError(f"Knowledge record lacks its source snapshot: {repo}") - if file_fingerprints: - record = _source_snapshot.resolve(record, snapshot) - else: - revision = snapshot.get("source_revision") - if not isinstance(revision, str) or not re.fullmatch( - r"[0-9a-f]{40}", revision - ): - raise ValueError("Knowledge source commit is unavailable") - record = { - **record, - "schema_version": 1, - "sources": [ - { - "type": "git", - "commit": revision, - "url": _source_snapshot.REPOSITORY + "/tree/" + revision, - } - ], - } - elif snapshot is not None: - raise ValueError( - f"Source-snapshot pack holds a record that is not schema " - f"{_source_snapshot.STORED_SCHEMA}: {repo}/{'/'.join(parts)}" - ) - if not file_fingerprints: - record = without_file_fingerprints(record) - _validate_record(record, repo, parts, tier) - if record["id"] in seen: - raise ValueError(f"Duplicate knowledge ID: {record['id']}") - seen.add(record["id"]) - records.append(record) - return records + authored = root.joinpath("records") + for item in sorted(authored.iterdir(), key=lambda p: p.name): + if item.is_file() and ( + item.name in {"README.md", ".DS_Store"} + or item.name.casefold() in {"desktop.ini", "thumbs.db"} + ): + continue + records.append(_authored_record(item)) + if include_skill_views: + for view in skill_views(root, file_fingerprints=file_fingerprints): + parts = (view["backend"], "skills", view["title"] + ".json") + _validate_record(view, "inferrsi", parts) + records.append(view) + order = {(tier, repo): (t, r) for t, (tier, repos) in enumerate(KNOWLEDGE_SCOPES.items()) + for r, repo in enumerate(repos)} + records.sort(key=lambda row: ( + order[row.get("tier", "contribution"), row["repo"]], row["id"] + )) + if len({row["id"] for row in records}) != len(records): + raise ValueError("Duplicate knowledge ID") + return records if file_fingerprints else without_file_fingerprints(records) def without_file_fingerprints(value): @@ -290,8 +451,24 @@ def search_knowledge( return {"items": items, "retrieval": retrieval} +def lookup_record(record_id: str) -> dict | None: + """Validate a single package owner without parsing unrelated records or skill sidecars.""" + if not re.fullmatch(r"[a-z0-9][a-z0-9._-]*", record_id): + return None + root = files("inferrsi").joinpath("knowledge", "data", "skill_library") + item = root.joinpath("records", record_id + ".json") + record = _authored_record(item) if item.is_file() else None + if re.fullmatch(r"inferrsi\.[a-z0-9]+\.skills\.[a-z0-9_]+", record_id): + for view in skill_views(root, record_id=record_id): + if record is not None: + raise ValueError(f"Duplicate knowledge ID: {record_id}") + _validate_record(view, "inferrsi", (view["backend"], "skills", view["title"] + ".json")) + record = view + return record + + def get_knowledge(record_id: str) -> dict[str, Any]: - for record in knowledge_records(): - if record["id"] == record_id: - return record - raise ValueError(f"Unknown knowledge ID: {record_id}") + record = lookup_record(record_id) + if record is None: + raise ValueError(f"Unknown knowledge ID: {record_id}") + return record diff --git a/src/inferrsi/knowledge/data/knowledge_packs/README.md b/src/inferrsi/knowledge/data/knowledge_packs/README.md deleted file mode 100644 index 947b40009..000000000 --- a/src/inferrsi/knowledge/data/knowledge_packs/README.md +++ /dev/null @@ -1,203 +0,0 @@ -# Three learning tiers - -The tiers describe **how InferRSI learns from a source**, independently of its -backend, component and evidence maturity. A technique can suggest a downstream -experiment; repository conventions stay with their owning repository. - -| Tier / CLI value | Sources | Learning output | -| --- | --- | --- | -| Contribution / `contribution` | vLLM, vLLM-Omni, vllm-ascend; legacy repository packs also retain imported Strata, SGLang and SGLang-Omni procedures | L2C blind attempt → comparison → scoped skill; separate performance evidence/candidates | -| Dependency / `dependency` | FlashInfer, FlashAttention, PyTorch/Inductor, Triton, Diffusers, Transformers, MLX | API/capability constraints and downstream impact; no blind coding session required | -| Peer technique / `peer` | SGLang, TensorRT-LLM, LMDeploy, xDiT, Cache-DiT, Qwen3-TTS engine, DeepInfer | Technique, hardware, reported benefit, costs, quality risks and a reproduction hypothesis | - -This is an acquisition policy, not a ranking or fixed dependency graph. -Cache-DiT, for example, can also be a direct dependency of vLLM-Omni. Its entry -is classified as a peer technique for this initial extraction; reclassify an -entry when the learning use case changes instead of duplicating its claims. - -For `pytorch/pytorch`, the [compiler/checkpoint scope map](../skill_library/README.md#pytorch-compiler-and-checkpoint-scope-routing) -relates the [compiler dependency card](dependency/pytorch/cuda/compiler/cudagraph-shapes-and-lifetimes.json) -to the existing [DCP async-save owner](../skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md), -keeping its fixed implementation and live recipe separate from compiler -documentation, with each owner's completion and version boundaries intact. - -For TensorRT-LLM, the [methodology and disaggregation map](../skill_library/README.md#tensorrt-llm-source-and-responsibility-routing) -routes the existing mixed-source performance methods alongside the -[PD/KV-transfer card](peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json). -It assesses shared measurement controls and separate startup, host and network -costs while retaining the card's source revision and transfer-hypothesis limits. - -## Storage and compatibility - -```text -inferrsi/data/ - repo_packs/REPO/knowledge/BACKEND/COMPONENT/ENTRY.json Contribution knowledge - knowledge_packs/ - README.md Cross-repository navigation - dependency/REPO/BACKEND/COMPONENT/ENTRY.json Dependency impact - peer/REPO/BACKEND/COMPONENT/ENTRY.json Technique candidates - skill_library/ Full categorized skills/resources - hardware/, frameworks/, repositories/ Hardware and implementation owners - methods/, workflows/ General methods and workflows -runs/learning/RUN_ID/ Future learning logs/state -``` - -The existing [vLLM-Omni records](../repo_packs/vllm_omni/knowledge/README.md) -retain their IDs, paths and contents. Schema-v1 records without `tier` belong -to `contribution`. New external records declare `tier`; the loader checks it -against the owning directory as well as ID/repository/backend/component. -`repo` always identifies the source owner, not a validated downstream target. - -The [InferRSI skill catalog](../repo_packs/inferrsi/knowledge/README.md) -provides generated contribution-tier records for registered skills. Their full -procedures, sidecars, references and cases now live in the -[packaged skill library](../skill_library/README.md); `skill_documents.original` -links each record to its procedure. Hardware guidance lives under the matching -`hardware/ascend`, `hardware/cuda` or `hardware/host` directory; other owners are -frameworks, repositories, methods and workflows. Search with `--repo-type inferrsi` -and optionally `--backend ascend` or `--backend cuda`. Original state, provenance -and unknown licensing are preserved separately from the unverified catalog -record. Backend classification and `skills.discovery` imply no qualification. - -Future reviewed L2C skills use these same owner directories; provenance and -learning tier do not introduce another top-level skill category. The learning -log path is a future workflow destination, not a running L2C integration. -There are no learned L2C skills or vLLM/vllm-ascend performance -records in this import. Existing imported vLLM-Omni guidance does not become -L2C training output merely by receiving a tier. - -External knowledge packs do not register executable repository targets. The -five-engine library separately provides repository descriptors for caller-supplied -external commands; it adds no built-in launcher, scheduler, CI lane or GPU dependency. -Learning-source choices and runtime repository descriptors remain separate registries. - -Legacy schema-v1 abstracts in `repo_packs/` retain their contribution-tier paths -and IDs for compatibility, including the five-engine library. These imported -procedures are not reviewed L2C outputs. A source can have records in more than -one tier: SGLang's repository procedures remain here while its separately sourced -HiCache transfer card belongs to `peer`. Tier filtering applies to each record; -unfiltered retrieval returns both without duplicating IDs or inheriting validation. - -## Retrieval - -```bash -inferrsi knowledge search --tier dependency --limit 20 -inferrsi knowledge search --tier peer --backend cuda --scenario llm.serving -inferrsi knowledge search 'offload' --repo-type diffusers --backend cuda -inferrsi context --repo-type xdit --backend cuda --scenario diffusion.t2v -inferrsi knowledge search --tier contribution --repo-type vllm_omni --backend ascend -``` - -Tier, source repository, backend and scenario filters intersect **before** the -result limit. Unfiltered search includes all tiers. A repository filter is -exact: searching vllm_omni does not silently inject FlashInfer or xDiT records. -Select those sources explicitly when developing a transfer hypothesis. -`common` still means backend-independent guidance within its source repository, -not global applicability or proof of support on every backend. Hardware/model/ -software conditions in `applicability` require human or agent inspection. -Human-readable context preserves downstream impact alongside provenance, -limitations and the explicit unverified state. - -## Initial extraction: 2026-10-15 - -This is a **point-in-time bootstrap**, one focused card for each of the eleven -requested external sources. It is not an exhaustive inventory, a daily commit -scan or a claim that a feature was newly added that day. Sources were read at -fixed default-branch revisions; each record stores immutable document/code -URLs, the commit, access date and SHA-256 of the fetched source content. These -source revisions are attribution, not acceptance-campaign pins. - -Claims summarize upstream material. `downstream_impact`, validation steps and -cautions are InferRSI synthesis and transfer hypotheses. They are not assertions -that another repository implements the same API or achieves the same result. -All cards are `status=imported`, `locally_verified=false`; no performance -campaign PASS is claimed. Numeric upstream reports retain their workload and -hardware and remain unverified. Most cards deliberately avoid aggregate README -speedup numbers whose conditions or isolated contribution are unclear. - -| Tier | Source | Card | -| --- | --- | --- | -| dependency | flashinfer | [FlashInfer paged decode: freeze backend selection and graph buffers](dependency/flashinfer/cuda/llm/paged-decode-dispatch.json) | -| dependency | flashattention | [FlashAttention generations have distinct hardware and API contracts](dependency/flashattention/cuda/attention/generation-specific-dispatch.json) | -| dependency | pytorch | [Inductor CUDA graphs: distinguish compile guards, capture shapes and tensor lifetime](dependency/pytorch/cuda/compiler/cudagraph-shapes-and-lifetimes.json) | -| dependency | triton | [Triton fusion trades memory traffic for register and shared-memory pressure](dependency/triton/common/kernel/bandwidth-fusion-resource-limits.json) | -| dependency | diffusers | [Diffusers group offload exchanges VRAM for host memory and transfer scheduling](dependency/diffusers/cuda/diffusion/group-offload-transfer-overlap.json) | -| dependency | transformers | [Transformers cache strategies trade compilation, capacity and latency](dependency/transformers/common/llm/kv-cache-strategy-contracts.json) | -| peer | sglang | [SGLang HiCache: reuse prefixes only when transfer beats recomputation](peer/sglang/cuda/llm/hierarchical-prefix-cache.json) | -| peer | tensorrt_llm | [TensorRT-LLM disaggregation trades phase interference for KV transfer](peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json) | -| peer | lmdeploy | [LMDeploy KV quantization: preserve capacity gains and latency counterexamples](peer/lmdeploy/cuda/llm/kv-quantization-capacity-cost.json) | -| peer | xdit | [xDiT parallelism: separate current-state attention from stale-state pipelines](peer/xdit/cuda/diffusion/parallelism-freshness-and-topology.json) | -| peer | cache_dit | [Cache-DiT residual reuse needs model- and schedule-specific calibration](peer/cache_dit/common/diffusion/residual-cache-calibration.json) | - -## DeepInfer extraction: 2026-10-20 - -Three source-derived candidates at the same immutable DeepInfer revision. -Owner review on 2026-10-22 corrected query/KV ownership, nested feature gates, -verification budgeting and release-scope attribution. Original access dates -are retained; newly inspected sources carry their own access date. No native, -model, service or performance validation is claimed. - -| Tier | Source | Card | -| --- | --- | --- | -| peer | deepinfer | [DeepInfer zero-overhead scheduling: defer the previous step's post-process to overlap host prep with device work](peer/deepinfer/common/llm/zero-overhead-deferred-scheduling.json) | -| peer | deepinfer | [DeepInfer verification budget: spend verification rows by marginal acceptance, not by fixed draft width](peer/deepinfer/common/llm/speculative-verification-budget.json) | -| peer | deepinfer | [DeepInfer bounded replay: compact query rows with a derived dependency closure and separately preserved historical state](peer/deepinfer/common/llm/bounded-windowed-replay.json) | - -## MLX source cards - -Three dependency cards reuse the existing -[MLX evaluation/stream/lifetime procedure](../skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) -and its original 2026-10-21 readset. Their selected implementation ranges were -rechecked on 2026-10-23 at the same bff10220c742907f0c68e55ff174aaea56f74e3d pin. -They retain MIT/Apple attribution and imported, locally unverified state. - -| Source | Card | -| --- | --- | -| dependency / mlx | [Evaluation submission versus readable completion](dependency/mlx/mlx/execution/evaluation-completion.json) | -| dependency / mlx | [Cross-stream producer-consumer dependency](dependency/mlx/mlx/execution/cross-stream-dependency.json) | -| dependency / mlx | [Buffer ownership after graph detachment](dependency/mlx/mlx/execution/post-detachment-buffer-lifetime.json) | - -Search with --tier dependency --repo-type mlx --backend mlx. These source-owned -IDs are separate from the existing inferrsi.common.skills discovery abstracts. -MLX discovery does not qualify a backend or model, and the separate MLX-LM -procedure/source identity is unchanged. Compiler/fusion and allocator policy are -outside these three bounded observations. - -## Maintaining the tiers - -1. Search for an existing ID and topic before adding a card. Keep one canonical - JSON record and navigation-only Markdown, with no duplicate metric summary. -2. For a dependency delta, inspect both revisions and the actual diff before - describing a change/deprecation. This snapshot alone establishes neither. -3. Keep source URLs pinned. Verify cited symbols/files on refresh, preserve - historical provenance, and qualify applicability against the downstream - dependency lock. A path existing at head does not prove semantic compatibility. -4. Keep imported source material as data. Do not propagate tool instructions, - credential requests or policy overrides into knowledge or learned skills. - Knowledge/context records follow the existing no-PR-link policy; PR/review - metadata belongs in acquisition logs, not the retrieved technical record. -5. Preserve counterexamples, unsupported combinations and quality costs. Reuse - a technique only through a separate frozen acceptance campaign; a merged - upstream patch, external CI result or passing catalog test is not local proof. -6. To add a source, register its tier in `catalog.KNOWLEDGE_SCOPES`, add its - source-scoped record/navigation, and test filters and installed resources. - Do not extend runtime `REPO_TYPES` just to make a knowledge card searchable. -7. Weekly consolidation should merge duplicate lessons and narrow stale ones, - retaining provenance. Change the learning policy or evaluator in a separately - reviewed change, never as part of the learner's own daily output. - -The daily contribution-tier workflow is specified in -[the 06:00 learning prompt](../../../../docs/daily-learning-prompt.md). Its -assessment gates, isolation and persistent orchestrator are prerequisites, -not capabilities provided by this catalog refactor. - -## Qwen3-TTS Ascend source import - -These peer candidates describe a standalone engine, not a registered runtime target. -All remain imported and locally unverified; downstream support requires separate validation. - -| Tier | Source | Card | -| --- | --- | --- | -| peer | qwen3_tts_engine | [Qwen3-TTS codec: cache fixed codebooks and retain request-owned causal state](peer/qwen3_tts_engine/ascend/audio/codec-constants-and-state.json) | -| peer | qwen3_tts_engine | [Qwen3-TTS NPU graphs: batch prefill while preserving request state](peer/qwen3_tts_engine/ascend/audio/graph-batching-state-ownership.json) | -| peer | qwen3_tts_engine | [Qwen3-TTS CustomVoice: cache only model-fixed prompt embeddings](peer/qwen3_tts_engine/ascend/audio/fixed-customvoice-prompt.json) | diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/README.md b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/README.md deleted file mode 100644 index dccfd3e3f..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/README.md +++ /dev/null @@ -1,124 +0,0 @@ -# InferRSI skill discovery index - -Generated abstracts for all 80 registered skills in `data/skill_library/`. -Their complete procedures, references and cases ship in the same package. -`skill-index.json` maps stable skill IDs to categorized directories. The -reference and searchspace matrices -remain the maintained knowledge views. Do not edit generated files by hand. - -Index status `imported` and `locally_verified: false` describe these discovery -records. Original status, provenance, license declarations and limitations -are preserved under `skill_metadata`; absent fields remain absent. Unknown -licenses remain unknown. Indexing grants no execution permission or validation. - -These are knowledge abstracts (`--kind knowledge`), with namespaced package -IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA -procedures use their corresponding backend; shared methods use `common`. - -`_provenance.json` holds the shared source snapshot: the full source -revision and the registry hash. Stored records (schema 2) keep only their own -fields and resource hashes; the catalog restores the schema-1 `sources` entry and -registry hash from the snapshot, so editing one skill regenerates only its record -and the snapshot. - -```bash -inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50 -inferrsi collect --repo-type inferrsi --kind knowledge --limit 50 --material -``` - -After committing changes to skill bodies, resources or the registry, regenerate from a -development installation with the skills extra. The source revision must -contain the current input paths and bytes (Git checkout filters are respected). -`source_files` hashes identify the canonical Git bytes, not a validation result. - -```bash -python tools/build_skill_index.py --source-revision -python tools/build_skill_index.py --source-revision --check -``` - -`--check` detects missing, stale and unexpected generated records without writing; -use the `source_revision` recorded in `_provenance.json`. -References and cases ship with their owning skill, not as duplicate knowledge abstracts. - -| Skill | Backend | Original declared status | -| --- | --- | --- | -| [ascend-aclgraph-capture-replay](ascend/skills/ascend-aclgraph-capture-replay.json) | ascend | candidate | -| [ascend-device-health](ascend/skills/ascend-device-health.json) | ascend | candidate | -| [ascend-distributed-parallel](ascend/skills/ascend-distributed-parallel.json) | ascend | candidate | -| [ascend-graph-mode](ascend/skills/ascend-graph-mode.json) | ascend | candidate | -| [ascend-model-adaptation](ascend/skills/ascend-model-adaptation.json) | ascend | candidate | -| [ascend-operator-profiling](ascend/skills/ascend-operator-profiling.json) | ascend | candidate | -| [ascend-operator-tuning](ascend/skills/ascend-operator-tuning.json) | ascend | candidate | -| [ascend-optimization-casebook](ascend/skills/ascend-optimization-casebook.json) | ascend | candidate | -| [ascend-performance-analysis](ascend/skills/ascend-performance-analysis.json) | ascend | candidate | -| [ascend-video-diffusion-optimization](ascend/skills/ascend-video-diffusion-optimization.json) | ascend | candidate | -| [ascendc-tiling-design](ascend/skills/ascendc-tiling-design.json) | ascend | candidate | -| [assess-serving-preemption](common/skills/assess-serving-preemption.json) | common | candidate | -| [assess-split-backward-schedule](common/skills/assess-split-backward-schedule.json) | common | candidate | -| [audit-afd-p2p-exchanges](cuda/skills/audit-afd-p2p-exchanges.json) | cuda | candidate | -| [audit-fixed-workload-claim](common/skills/audit-fixed-workload-claim.json) | common | candidate | -| [audit-flare-session-lifecycle](cuda/skills/audit-flare-session-lifecycle.json) | cuda | candidate | -| [audit-h3-metal-fasth3-semantics](common/skills/audit-h3-metal-fasth3-semantics.json) | common | candidate | -| [audit-h3-runtime-residency](common/skills/audit-h3-runtime-residency.json) | common | candidate | -| [audit-llama-runtime-state](common/skills/audit-llama-runtime-state.json) | common | candidate | -| [audit-recurrent-kv-finalization](common/skills/audit-recurrent-kv-finalization.json) | common | candidate | -| [audit-sd-cpp-residency](common/skills/audit-sd-cpp-residency.json) | common | candidate | -| [audit-sglang-diffusion-lifetimes](common/skills/audit-sglang-diffusion-lifetimes.json) | common | candidate | -| [audit-vllm-kv-lifetimes](common/skills/audit-vllm-kv-lifetimes.json) | common | candidate | -| [bound-windowed-attention-replay](common/skills/bound-windowed-attention-replay.json) | common | candidate | -| [budget-speculative-verification](common/skills/budget-speculative-verification.json) | common | candidate | -| [calibrate-serving-simulation](common/skills/calibrate-serving-simulation.json) | common | candidate | -| [cann-attribution-overlap](ascend/skills/cann-attribution-overlap.json) | ascend | candidate | -| [cgroup-resource-controls](common/skills/cgroup-resource-controls.json) | common | candidate | -| [checkpoint-recovery-consistency](common/skills/checkpoint-recovery-consistency.json) | common | candidate | -| [choose-parallel-training-plan](common/skills/choose-parallel-training-plan.json) | common | candidate | -| [cite-check-downstream-pr](common/skills/cite-check-downstream-pr.json) | common | candidate | -| [claim-contract](common/skills/claim-contract.json) | common | candidate | -| [comfy-ui](common/skills/comfy-ui.json) | common | candidate | -| [competitor-scan](common/skills/competitor-scan.json) | common | candidate | -| [dcp-async-save-performance](common/skills/dcp-async-save-performance.json) | common | candidate | -| [dense-performance-evidence](common/skills/dense-performance-evidence.json) | common | candidate | -| [diagnose-attention-bottlenecks](cuda/skills/diagnose-attention-bottlenecks.json) | cuda | candidate | -| [diagnose-vllm-admission-pressure](common/skills/diagnose-vllm-admission-pressure.json) | common | candidate | -| [diffusers-invocation-state](common/skills/diffusers-invocation-state.json) | common | candidate | -| [draft-acceptance-issue](common/skills/draft-acceptance-issue.json) | common | candidate | -| [dwarfstar](common/skills/dwarfstar.json) | common | candidate | -| [evaluate-elastic-serving-placement](common/skills/evaluate-elastic-serving-placement.json) | common | candidate | -| [generation-cache-lifecycle](common/skills/generation-cache-lifecycle.json) | common | candidate | -| [host-memory-management](common/skills/host-memory-management.json) | common | candidate | -| [host-overhead-optimization](common/skills/host-overhead-optimization.json) | common | candidate | -| [host-overhead-profiling](cuda/skills/host-overhead-profiling.json) | cuda | candidate | -| [husky](common/skills/husky.json) | common | candidate | -| [inference-benchmark-design](common/skills/inference-benchmark-design.json) | common | candidate | -| [inference-optimization-casebook](common/skills/inference-optimization-casebook.json) | common | candidate | -| [kernel-writing-guide](common/skills/kernel-writing-guide.json) | common | candidate | -| [kv-tier-management](common/skills/kv-tier-management.json) | common | candidate | -| [lazy-evaluation-stream-lifetimes](common/skills/lazy-evaluation-stream-lifetimes.json) | common | candidate | -| [lily](common/skills/lily.json) | common | candidate | -| [lithos-metal](common/skills/lithos-metal.json) | common | candidate | -| [local-dream](common/skills/local-dream.json) | common | candidate | -| [mistral-rs](common/skills/mistral-rs.json) | common | candidate | -| [mlc-llm](common/skills/mlc-llm.json) | common | candidate | -| [mlx-swift-lm](common/skills/mlx-swift-lm.json) | common | candidate | -| [npu-skill-factory](common/skills/npu-skill-factory.json) | common | candidate | -| [optimization-search-policy](common/skills/optimization-search-policy.json) | common | candidate | -| [os-process-placement](common/skills/os-process-placement.json) | common | candidate | -| [performance-ceiling-analysis](common/skills/performance-ceiling-analysis.json) | common | candidate | -| [plan-activation-residency](common/skills/plan-activation-residency.json) | common | candidate | -| [project-maya](common/skills/project-maya.json) | common | candidate | -| [pytorch](common/skills/pytorch.json) | common | candidate | -| [reason-hopper-kernels](cuda/skills/reason-hopper-kernels.json) | cuda | candidate | -| [repository-evolution-analysis](common/skills/repository-evolution-analysis.json) | common | candidate | -| [rsi-evidence](common/skills/rsi-evidence.json) | common | candidate | -| [skill-evolution-playbook](common/skills/skill-evolution-playbook.json) | common | candidate | -| [splash](common/skills/splash.json) | common | candidate | -| [storage-io-measurement](common/skills/storage-io-measurement.json) | common | candidate | -| [torch-npu](ascend/skills/torch-npu.json) | ascend | candidate | -| [torchnpu-device-memory-lifecycle](ascend/skills/torchnpu-device-memory-lifecycle.json) | ascend | candidate | -| [torchnpu-pinned-host-memory-lifecycle](ascend/skills/torchnpu-pinned-host-memory-lifecycle.json) | ascend | candidate | -| [training-input-pipeline](common/skills/training-input-pipeline.json) | common | candidate | -| [training-resource-budget](common/skills/training-resource-budget.json) | common | candidate | -| [video-diffusion-optimization](common/skills/video-diffusion-optimization.json) | common | candidate | -| [vllm-ascend](ascend/skills/vllm-ascend.json) | ascend | candidate | -| [vllm-metal](common/skills/vllm-metal.json) | common | candidate | -| [weight-loading-optimization](common/skills/weight-loading-optimization.json) | common | candidate | diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json deleted file mode 100644 index 4dbd175a4..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_aclgraph_capture_replay", - "title": "ascend-aclgraph-capture-replay", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose vLLM-Ascend ACLGraph capture/replay eligibility, stable-input and ordering contracts, and capture-resource failures at a pinned implementation. Use for evidence-gated graph-mode proposals, not allocator, compiler-fusion, or network tuning.", - "tags": [ - "ascend", - "vllm-ascend", - "aclgraph", - "runtime", - "stream-synchronization" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md" - }, - "checkout_ref": "checkout:ascend-aclgraph-capture-replay", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md": "60dcf15b8d5678521d5650160b3404091a03fdc843ffe31a6e13a249b1f6a8be", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json": "255ffa4b9e7469d4b0e5c34cf9effff75b78413e9a1df87232d07d5d28212db6", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/references/source-notes.md": "6ffbfb68cc7e04923b4fbca2a73a7555b7c3b45a6f40819bb5f3213ef05cf88d" - }, - "skill_metadata": { - "name": "ascend-aclgraph-capture-replay", - "description": "Diagnose vLLM-Ascend ACLGraph capture/replay eligibility, stable-input and ordering contracts, and capture-resource failures at a pinned implementation. Use for evidence-gated graph-mode proposals, not allocator, compiler-fusion, or network tuning.", - "tags": [ - "ascend", - "vllm-ascend", - "aclgraph", - "runtime", - "stream-synchronization" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Diagnose vLLM-Ascend ACLGraph eligibility, batch descriptor matching, capture/replay dispatch, caller-buffer ownership, multi-stream ordering, and capture-resource failures.", - "prerequisites": "An existing eager correctness baseline; authorization before any new run or live configuration change.", - "input_contract": "Installed stack and device, requested/effective graph mode, runner/backend settings, capture sizes and batch descriptors, error phase, replay observations and HBM/stream evidence.", - "produced_artifact": "A bounded diagnosis, smallest reversible proposal, expected tradeoff and correctness/resource verification plan with unresolved compatibility or authorization questions.", - "refusal_point": "Stop on output mismatch, unstable replay addresses, capture exhaustion, stream resource errors without capture-size reduction, unknown ordering or version drift; missing evidence permits only a conditional proposal." - }, - "recommendations": [ - "Diagnose the effective runtime mode and batch descriptor before changing capture sizes; intended warmup/profile bypass is not a cache defect.", - "Verify caller BatchDescriptor stability and dispatch mode before investigating graph replay issues.", - "For stream-resource exhaustion, reduce cudagraph_capture_sizes or max_cudagraph_capture_size.", - "Trace stable replay buffers to the owning caller and preserve synchronization/update ordering; the wrapper does not stage new inputs into persistent buffers.", - "Compare eager and graph execution on the same workload, separating initial capture, repeated replay, compiler changes and memory/stream costs; a replay log alone proves neither coverage nor speedup." - ] - }, - "applicability": { - "scope": "Diagnose vLLM-Ascend ACLGraph eligibility, batch descriptor matching, capture/replay dispatch, caller-buffer ownership, multi-stream ordering, and capture-resource failures.", - "prerequisites": "An existing eager correctness baseline; authorization before any new run or live configuration change.", - "input_contract": "Installed stack and device, requested/effective graph mode, runner/backend settings, capture sizes and batch descriptors, error phase, replay observations and HBM/stream evidence.", - "produced_artifact": "A bounded diagnosis, smallest reversible proposal, expected tradeoff and correctness/resource verification plan with unresolved compatibility or authorization questions.", - "refusal_point": "Stop on output mismatch, unstable replay addresses, capture exhaustion, stream resource errors without capture-size reduction, unknown ordering or version drift; missing evidence permits only a conditional proposal." - }, - "recommendations": [ - "Diagnose the effective runtime mode and batch descriptor before changing capture sizes; intended warmup/profile bypass is not a cache defect.", - "Verify caller BatchDescriptor stability and dispatch mode before investigating graph replay issues.", - "For stream-resource exhaustion, reduce cudagraph_capture_sizes or max_cudagraph_capture_size.", - "Trace stable replay buffers to the owning caller and preserve synchronization/update ordering; the wrapper does not stage new inputs into persistent buffers.", - "Compare eager and graph execution on the same workload, separating initial capture, repeated replay, compiler changes and memory/stream costs; a replay log alone proves neither coverage nor speedup." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json deleted file mode 100644 index c8d52899e..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json +++ /dev/null @@ -1,94 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_device_health", - "title": "ascend-device-health", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Pre-run health checks and recovery procedures for Ascend NPU devices on shared machines: card-group topology validation, Health status checks, SIGKILL residual-state detection, 4-step smoke verification, process occupancy check, and DVFS warmup discipline. Use before any experiment that requires a trusted performance baseline.\n", - "tags": [ - "ascend", - "npu", - "device-health", - "topology", - "shared-machine", - "os-system" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-device-health/SKILL.md" - }, - "checkout_ref": "checkout:ascend-device-health", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md": "03829039860ece35519f2e4c339a48b5014ec25e07a2c150a3efb6a3a4e35b4a", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/metadata.json": "f643185563dbf528a5a1f2b0561685ae34302a9efc9900c6753a7168b4e8b329" - }, - "skill_metadata": { - "name": "ascend-device-health", - "description": "Pre-run health checks and recovery procedures for Ascend NPU devices on shared machines: card-group topology validation, Health status checks, SIGKILL residual-state detection, 4-step smoke verification, process occupancy check, and DVFS warmup discipline. Use before any experiment that requires a trusted performance baseline.\n", - "tags": [ - "ascend", - "npu", - "device-health", - "topology", - "shared-machine", - "os-system" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Pre-run Ascend device-health, topology, occupancy and timing checks before a trusted baseline, multi-card run or graph capture, including after a killed job.", - "hardware": "Ascend NPU card groups with topology and health visible through npu-smi.", - "input_contract": "Intended card group, current topology/health and process occupancy, prior termination history and expected workload step timing.", - "produced_artifact": "A pre-run health/occupancy and step-timing record, with recovery observations and a baseline-remeasurement requirement after a card-group change.", - "failure_modes": "Health=OK can miss degraded post-kill timing; foreign processes, unwarmed clocks or comparing different card groups can contaminate the baseline." - }, - "recommendations": [ - "Read topology each session rather than hardcoding card numbers, then check health and foreign-process occupancy before the workload smoke check.", - "Use the short step-timing check to investigate post-kill degradation; record affected cards and timings instead of treating Health=OK as sufficient evidence.", - "Warm up operator measurements and remeasure the baseline after changing card groups; set only environment variables needed by the run." - ] - }, - "applicability": { - "scope": "Pre-run Ascend device-health, topology, occupancy and timing checks before a trusted baseline, multi-card run or graph capture, including after a killed job.", - "hardware": "Ascend NPU card groups with topology and health visible through npu-smi.", - "input_contract": "Intended card group, current topology/health and process occupancy, prior termination history and expected workload step timing.", - "produced_artifact": "A pre-run health/occupancy and step-timing record, with recovery observations and a baseline-remeasurement requirement after a card-group change.", - "failure_modes": "Health=OK can miss degraded post-kill timing; foreign processes, unwarmed clocks or comparing different card groups can contaminate the baseline." - }, - "recommendations": [ - "Read topology each session rather than hardcoding card numbers, then check health and foreign-process occupancy before the workload smoke check.", - "Use the short step-timing check to investigate post-kill degradation; record affected cards and timings instead of treating Health=OK as sufficient evidence.", - "Warm up operator measurements and remeasure the baseline after changing card groups; set only environment variables needed by the run." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json deleted file mode 100644 index 15c626bd2..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json +++ /dev/null @@ -1,165 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_distributed_parallel", - "title": "ascend-distributed-parallel", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Select and validate distributed parallelism strategy on Ascend NPU: topology-aware card-group selection, USP/CP/TP/PP strategy choice, communication masking setup and verification, attribution of parallelism gains, and diagnosis of strategies that appear to activate but have no effect. Covers native HCCL multi-rank AllReduce dispatch, scratch and synchronization boundaries, AlltoAllV defect workaround, and source-scoped communication-test timing and correctness coverage.\n", - "tags": [ - "ascend", - "npu", - "distributed", - "parallelism", - "usp", - "hccl", - "communication", - "allreduce", - "topology-dispatch", - "collective-correctness" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md" - }, - "checkout_ref": "checkout:ascend-distributed-parallel", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md": "0b79d95e8d5abf0a822f8ae6bfa3779cbb40595c5b6f0fb934233275f5899ef8", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json": "096a12d4a5b98bd057e1a87f972038fd43556e82e484b5134943b6e45e880178", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md": "0cb27a3d24409171bb414d7ebf0c49ad62be5f46482f29624695496ba9eb3e63", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-LICENSE.md": "05205a02acebd671c78d3b876c1f6097b84cce89780f5d9bdc204bfbea92d0c1", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce-readset.json": "4a3d403986dfceaa28605f2f30b79545e1efdc27f35c88755402b75b4d13d7e2", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce.md": "4770bba0090ed5ead3d2fd3b2cce2d21ba708acae3e0b4d859cc503499da5b93" - }, - "skill_metadata": { - "name": "ascend-distributed-parallel", - "description": "Select and validate distributed parallelism strategy on Ascend NPU: topology-aware card-group selection, USP/CP/TP/PP strategy choice, communication masking setup and verification, attribution of parallelism gains, and diagnosis of strategies that appear to activate but have no effect. Covers native HCCL multi-rank AllReduce dispatch, scratch and synchronization boundaries, AlltoAllV defect workaround, and source-scoped communication-test timing and correctness coverage.\n", - "tags": [ - "ascend", - "npu", - "distributed", - "parallelism", - "usp", - "hccl", - "communication", - "allreduce", - "topology-dispatch", - "collective-correctness" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ascend-ai-coding/awesome-ascend-skills", - "commit": "75ec89ce9d401be8ac95980e1c3f42f8d4315057", - "paths": [ - "README.md", - "skills/training/hccl-test/scripts/quick-verify.sh", - "skills/training/hccl-test/scripts/parse-hccl-result.py", - "skills/training/torch-npu-comm-test/scripts/comm-bench.py", - "skills/training/torch-npu-comm-test/references/supported-ops.md" - ], - "readset": "references/awesome-ascend-communication-tests.md" - }, - { - "type": "git", - "url": "https://gitcode.com/cann/hccl", - "commit": "96fab0a2acb761e7f8ab0f94aeb5138f200a345b", - "paths": [ - "src/common/hccl_common.h", - "src/ops/all_reduce/all_reduce.cc", - "src/ops/op_common/op_common.cc", - "src/ops/op_common/selector/selector_engine.cc", - "src/ops/op_common/selector/cost_model.cc", - "src/ops/all_reduce/selector/all_reduce_auto_selector.cc", - "src/ops/all_reduce/algorithm/executor/ins_v2_all_reduce_sole_executor.cc", - "src/ops/op_common/algorithm/topo_match/topo_match_one_level.cc", - "src/ops/op_common/algorithm/template/alg_template_base.h", - "src/ops/op_common/algorithm/template/aicpu/kernel_launch.cc", - "src/ops/all_reduce/algorithm/template/aicpu/ins_temp_all_reduce_mesh_1D_one_shot.cc", - "src/ops/op_common/algorithm/template/wrapper/alg_data_trans_wrapper.cc", - "test/st/algorithm/testcase/all_reduce_testcase.cc", - "test/st/algorithm/utils/src/hccl_verifier/hccl_verifier.cc", - "test/st/algorithm/utils/src/hccl_verifier/semantics_check/allreduce_semantics_checker.cc", - "src/common/log.h", - "LICENSE" - ], - "source_channel": "official_repository", - "inspected_scope": "selected_native_implementation_and_simulator_test_definitions", - "license": "CANN Open Software License Agreement Version 2.0", - "readset": "references/hccl-allreduce-readset.json" - } - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Only the declared communication-test and native HCCL source slices are normalized; inherited strategy guidance and its source identities remain unchanged.", - "Wrapper inspection does not qualify native HCCL/TorchNPU implementations, transport algorithms, numerical correctness, backend compatibility or measured gains.", - "Native source coverage is one selected multi-rank AICPU mesh one-shot AllReduce path at the HCCL pin; legacy device routes, other engines/collectives and HCOMM transport/runtime implementations are not qualified.", - "Inspected upstream tests use simulated task-graph semantics; neither those tests nor native numerical, memory, bandwidth, model-quality or service experiments were run." - ], - "provenance_readsets": [ - "references/awesome-ascend-communication-tests.md", - "references/hccl-allreduce.md", - "references/hccl-allreduce-readset.json" - ], - "consumption_mode": "packaged", - "references": [ - "references/awesome-ascend-communication-tests.md", - "references/hccl-allreduce.md", - "references/hccl-allreduce-readset.json", - "references/hccl-LICENSE.md" - ], - "applicability": { - "scope": "Select and diagnose Ascend parallel strategies and their activation/overlap, with a separately bounded native multi-rank HCCL AllReduce dispatch and correctness procedure.", - "input_contract": "Installed collective/runtime builds, rank/device map and topology, workload/strategy settings, count/dtype/reduction, effective dispatch, per-rank outputs and completed-collective/application timing evidence.", - "produced_artifact": "A topology- and workload-qualified strategy/dispatch diagnosis, activation and correctness checks, stage-cost attribution and a target-validation proposal; isolated test completion is not a service result.", - "failure_modes": "Requested parallel settings can silently fall back; stale local-head/chunk/shard parameters can disable a path, and changed topology/card count can confound strategy comparisons." - }, - "recommendations": [ - "Confirm the final collective algorithm/engine after fallback and observe sharding/collective activation before interpreting correct outputs or tuning performance.", - "Preserve peer/thread waits and test distinct rank/position data with explicit completion on every rank; keep enqueue time, collective completion and application-stage timing separate.", - "Recompute topology-derived parameters and remeasure the anchor after changing card groups or parallel degree; inherited diffusion topology restrictions are not universal HCCL gates.", - "Require producer/consumer timeline and exposed-communication evidence for overlap; compare the real pipeline wait cost rather than extrapolating small collectives from bytes." - ] - }, - "applicability": { - "scope": "Select and diagnose Ascend parallel strategies and their activation/overlap, with a separately bounded native multi-rank HCCL AllReduce dispatch and correctness procedure.", - "input_contract": "Installed collective/runtime builds, rank/device map and topology, workload/strategy settings, count/dtype/reduction, effective dispatch, per-rank outputs and completed-collective/application timing evidence.", - "produced_artifact": "A topology- and workload-qualified strategy/dispatch diagnosis, activation and correctness checks, stage-cost attribution and a target-validation proposal; isolated test completion is not a service result.", - "failure_modes": "Requested parallel settings can silently fall back; stale local-head/chunk/shard parameters can disable a path, and changed topology/card count can confound strategy comparisons." - }, - "recommendations": [ - "Confirm the final collective algorithm/engine after fallback and observe sharding/collective activation before interpreting correct outputs or tuning performance.", - "Preserve peer/thread waits and test distinct rank/position data with explicit completion on every rank; keep enqueue time, collective completion and application-stage timing separate.", - "Recompute topology-derived parameters and remeasure the anchor after changing card groups or parallel degree; inherited diffusion topology restrictions are not universal HCCL gates.", - "Require producer/consumer timeline and exposed-communication evidence for overlap; compare the real pipeline wait cost rather than extrapolating small collectives from bytes." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Only the declared communication-test and native HCCL source slices are normalized; inherited strategy guidance and its source identities remain unchanged.", - "Wrapper inspection does not qualify native HCCL/TorchNPU implementations, transport algorithms, numerical correctness, backend compatibility or measured gains.", - "Native source coverage is one selected multi-rank AICPU mesh one-shot AllReduce path at the HCCL pin; legacy device routes, other engines/collectives and HCOMM transport/runtime implementations are not qualified.", - "Inspected upstream tests use simulated task-graph semantics; neither those tests nor native numerical, memory, bandwidth, model-quality or service experiments were run." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json deleted file mode 100644 index ec159f03b..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json +++ /dev/null @@ -1,144 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_graph_mode", - "title": "ascend-graph-mode", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose an Ascend TorchAir ACL graph path from versioned configuration through FX compilation, capture-key selection, eager fallback, input staging, replay and output lifetime. Use when repeated inference has host overhead, capture churn, stale outputs or an uncertain backend; require source-matched configuration and separate device validation.", - "tags": [ - "ascend", - "npu", - "graph-mode", - "aclgraph", - "npugraph-ex", - "torchair", - "capture-key", - "input-staging", - "output-lifetime" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md" - }, - "checkout_ref": "checkout:ascend-graph-mode", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md": "a7341b5bf3b43f32d6114e2109e5043e46f3ddeef9b7bb3b20c1c50236bab638", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json": "017a288c514f6f0aab9d00247af7923258816f0987db15e09a5025a1f3937aa2", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/legacy-source-notes.md": "a51c655aff59741cd7421277bd71dffeb9907560950459ae0a776207e496de4a", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-LICENSE.md": "bc07d81fa332b6543463588a120518c57b5ce8582d3747de75d2f6b09b068c01", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-path.md": "272424571ff439278c8f2675406eae7da9a6482c581973e24319739fa0e58348", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-readset.json": "eb84cbd5c355a88ee013dc7e428b274162e0e13e85845b58526113b0d6a2b46b" - }, - "skill_metadata": { - "name": "ascend-graph-mode", - "description": "Diagnose an Ascend TorchAir ACL graph path from versioned configuration through FX compilation, capture-key selection, eager fallback, input staging, replay and output lifetime. Use when repeated inference has host overhead, capture churn, stale outputs or an uncertain backend; require source-matched configuration and separate device validation.", - "tags": [ - "ascend", - "npu", - "graph-mode", - "aclgraph", - "npugraph-ex", - "torchair", - "capture-key", - "input-staging", - "output-lifetime" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Historical awesome-ascend-skills/cannbot import retains its short pin and UNKNOWN license in references/legacy-source-notes.md; it was not reread for this source slice.", - "The new official TorchAir read follows Python configuration/code generation and capture/replay ownership only; native TorchNPU/CANN, GE execution, transformations, static kernel compilation and native communication remain uninspected.", - "Selected stub and NPU smoke test definitions were read, not executed. Independent source pins do not establish stack compatibility, correctness, performance or service evidence." - ], - "provenance_readsets": [ - "references/torchair-graph-readset.json", - "references/torchair-graph-path.md", - "references/legacy-source-notes.md" - ], - "consumption_mode": "packaged", - "sources": [ - { - "type": "git", - "url": "https://github.com/Ascend/torchair", - "commit": "e1118e8b0fc1b95190ab53b732c0872989550e20", - "paths": [ - "python/torchair/configs/compiler_config.py", - "python/torchair/configs/aclgraph_config.py", - "python/torchair/configs/debug_config.py", - "python/torchair/configs/npugraphex_config.py", - "python/torchair/npu_fx_compiler.py", - "python/torchair/_acl_concrete_graph/fx2acl_converter.py", - "python/torchair/_acl_concrete_graph/acl_graph.py", - "docs/zh/npugraph_ex/quick_start.md", - "docs/zh/npugraph_ex/api/npugraph_ex/compiler_config.md", - "tests/st/torchair_aclgraph_st.py", - "tests/smoke/aclgraph_test.py", - "LICENSE" - ], - "source_channel": "official_repository", - "inspected_scope": "selected_python_implementation_documentation_and_test_definitions", - "license": "BSD-3-Clause", - "readset": "references/torchair-graph-readset.json" - } - ], - "references": [ - "references/torchair-graph-path.md", - "references/torchair-graph-readset.json", - "references/torchair-LICENSE.md", - "references/legacy-source-notes.md", - "../ascend-aclgraph-capture-replay/SKILL.md" - ], - "applicability": { - "scope": "Diagnose the TorchAir Python ACL graph path from configuration and FX compilation through capture-key selection, fallback, input staging, replay and output lifetime.", - "prerequisites": "Eager correctness established first; configuration must match the installed backend, with authorization and workload-specific tolerances for any NPU comparison.", - "input_contract": "Installed hardware/runtime versions, actual compile backend/options, shape/dtype/stride and alias/mutation contract, retained outputs, capture/replay/fallback observations and workload mix.", - "produced_artifact": "A branch-specific diagnosis and bounded configuration or lifetime proposal with correctness, cold/warm cost, resource and concurrency checks still required on the actual stack.", - "failure_modes": "Shape equality or clone_input=False does not prove capture reuse or zero-copy; capture limits can trigger persistent fallback and later replay can overwrite retained outputs." - }, - "recommendations": [ - "Use the installed CompilerConfig/backend surface and a fresh compiled instance for option changes; do not mix registered-backend snippets with callable-backend configuration.", - "Observe capture, recapture, replay and persistent fallback separately, including changed addresses and revisiting old keys after a capture-limit event.", - "Preserve earlier results before another replay and check views/input-output aliases; keep cold capture, input/output copying, optional compiler passes and service timing separate." - ] - }, - "applicability": { - "scope": "Diagnose the TorchAir Python ACL graph path from configuration and FX compilation through capture-key selection, fallback, input staging, replay and output lifetime.", - "prerequisites": "Eager correctness established first; configuration must match the installed backend, with authorization and workload-specific tolerances for any NPU comparison.", - "input_contract": "Installed hardware/runtime versions, actual compile backend/options, shape/dtype/stride and alias/mutation contract, retained outputs, capture/replay/fallback observations and workload mix.", - "produced_artifact": "A branch-specific diagnosis and bounded configuration or lifetime proposal with correctness, cold/warm cost, resource and concurrency checks still required on the actual stack.", - "failure_modes": "Shape equality or clone_input=False does not prove capture reuse or zero-copy; capture limits can trigger persistent fallback and later replay can overwrite retained outputs." - }, - "recommendations": [ - "Use the installed CompilerConfig/backend surface and a fresh compiled instance for option changes; do not mix registered-backend snippets with callable-backend configuration.", - "Observe capture, recapture, replay and persistent fallback separately, including changed addresses and revisiting old keys after a capture-limit event.", - "Preserve earlier results before another replay and check views/input-output aliases; keep cold capture, input/output copying, optional compiler passes and service timing separate." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Historical awesome-ascend-skills/cannbot import retains its short pin and UNKNOWN license in references/legacy-source-notes.md; it was not reread for this source slice.", - "The new official TorchAir read follows Python configuration/code generation and capture/replay ownership only; native TorchNPU/CANN, GE execution, transformations, static kernel compilation and native communication remain uninspected.", - "Selected stub and NPU smoke test definitions were read, not executed. Independent source pins do not establish stack compatibility, correctness, performance or service evidence." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json deleted file mode 100644 index 13bfbb69e..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_model_adaptation", - "title": "ascend-model-adaptation", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Patterns for porting LLM/VLM model adapters to Ascend NPU: FP8 checkpoint handling, KV replication with TP, multimodal feature validation order, graph mode startup, and feature-status reporting. Source-grounded from vllm-ascend production experience. Use when adapting a new model checkpoint to run on Ascend via vllm-ascend or a similar framework.\n", - "tags": [ - "ascend", - "npu", - "model-adaptation", - "fp8", - "multimodal", - "vllm-ascend" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md" - }, - "checkout_ref": "checkout:ascend-model-adaptation", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md": "ff1f0d941ac429041ca295871135e1b14e22c2819c404dd2435576c1d0cbc838", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json": "e5191f402b0ccf069a0aafc9873b990704426f652c0048ad027082f5e47537dc", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/references/source-notes.md": "80df6181be3853190fd8fb35d4f853eb0056f4001071fe380dde0c3ef4ccde1b" - }, - "skill_metadata": { - "name": "ascend-model-adaptation", - "description": "Patterns for porting LLM/VLM model adapters to Ascend NPU: FP8 checkpoint handling, KV replication with TP, multimodal feature validation order, graph mode startup, and feature-status reporting. Source-grounded from vllm-ascend production experience. Use when adapting a new model checkpoint to run on Ascend via vllm-ascend or a similar framework.\n", - "tags": [ - "ascend", - "npu", - "model-adaptation", - "fp8", - "multimodal", - "vllm-ascend" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Checkpoint onboarding for LLM/VLM adapters on Ascend: weight/scale loading, KV-head replication under tensor parallelism and staged multimodal feature checks.", - "prerequisites": "Check the installed CANN and checkpoint format against the versioned guidance; final sign-off requires real weights and at least one full-layer validation.", - "input_contract": "Checkpoint weights/configuration and scale keys, attention-head/TP configuration, model modalities and expected graph/capacity features.", - "produced_artifact": "A checkpoint/feature report separating dummy and real-weight evidence, applicable feature outcomes, and theoretical versus demonstrated serving capacity.", - "failure_modes": "Dummy-only success misses scale/remapping/sharding defects; blindly sharding replicated KV heads or accepting reduced-layer results can conceal loading or full-model failures.", - "refusal_point": "Stop on missing FP8 paired scale keys, unsupported tensor parallel KV-head split without replication, unverified multimodal visual encoder output, or failed base endpoint validation." - }, - "recommendations": [ - "Gate onboarding in order: dummy architecture check, real-weight loading, model endpoint, text request and then multimodal request when the model supports VL; retain the evidence each stage actually supplies.", - "Check weight/scale pairing and each attention tensor loading path, especially when TP exceeds the KV-head count; verify historical FP8 guidance against the actual stack.", - "Expand capacity/concurrency only after basic modality and expected replay checks; mark MoE-only features inapplicable for non-MoE models and require full-layer validation before sign-off." - ] - }, - "applicability": { - "scope": "Checkpoint onboarding for LLM/VLM adapters on Ascend: weight/scale loading, KV-head replication under tensor parallelism and staged multimodal feature checks.", - "prerequisites": "Check the installed CANN and checkpoint format against the versioned guidance; final sign-off requires real weights and at least one full-layer validation.", - "input_contract": "Checkpoint weights/configuration and scale keys, attention-head/TP configuration, model modalities and expected graph/capacity features.", - "produced_artifact": "A checkpoint/feature report separating dummy and real-weight evidence, applicable feature outcomes, and theoretical versus demonstrated serving capacity.", - "failure_modes": "Dummy-only success misses scale/remapping/sharding defects; blindly sharding replicated KV heads or accepting reduced-layer results can conceal loading or full-model failures.", - "refusal_point": "Stop on missing FP8 paired scale keys, unsupported tensor parallel KV-head split without replication, unverified multimodal visual encoder output, or failed base endpoint validation." - }, - "recommendations": [ - "Gate onboarding in order: dummy architecture check, real-weight loading, model endpoint, text request and then multimodal request when the model supports VL; retain the evidence each stage actually supplies.", - "Check weight/scale pairing and each attention tensor loading path, especially when TP exceeds the KV-head count; verify historical FP8 guidance against the actual stack.", - "Expand capacity/concurrency only after basic modality and expected replay checks; mark MoE-only features inapplicable for non-MoE models and require full-layer validation before sign-off." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json deleted file mode 100644 index 2973fe628..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json +++ /dev/null @@ -1,98 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_operator_profiling", - "title": "ascend-operator-profiling", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Benchmark a single Ascend NPU operator using msprof op: build, collect with warmup, read the 8 CSV metrics files, derive a bottleneck verdict, and iterate. Use when validating operator performance in isolation before system-level profiling.\n", - "tags": [ - "ascend", - "npu", - "operator", - "profiling", - "msprof", - "benchmark" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md" - }, - "checkout_ref": "checkout:ascend-operator-profiling", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md": "401981cf7828edcab0c9ea882f7ecd83badfa3c2a95006e912cf99d67b3d6a12", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json": "b2b93637f01546ecb1b7076be6798d234988c594cccdcb764ad222af9ad21af9" - }, - "skill_metadata": { - "name": "ascend-operator-profiling", - "description": "Benchmark a single Ascend NPU operator using msprof op: build, collect with warmup, read the 8 CSV metrics files, derive a bottleneck verdict, and iterate. Use when validating operator performance in isolation before system-level profiling.\n", - "tags": [ - "ascend", - "npu", - "operator", - "profiling", - "msprof", - "benchmark" - ], - "status": "candidate", - "synced-from": "https://gitcode.com/cann/cannbot-skills external/cannbot/ops/ops-profiling", - "synced-date": "2026-05-26", - "synced-commit": "ac5bbd2b4cf427d011874e11f8d1e8b1bef66eda", - "license": "UNKNOWN", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Build, measure and diagnose one Ascend operator in isolation using msprof op, before interpreting system-level or multi-kernel performance.", - "prerequisites": "A built executable or test harness and access to real NPU hardware with msprof op for collection.", - "input_contract": "Operator executable/harness, collection settings and OPPROF output directory, plus the operator directory used to archive each measurement round.", - "produced_artifact": "Per-round raw CSV archives and observational min/average/max summaries, followed by a bottleneck interpretation and comparison of operator duration and core balance.", - "failure_modes": "Unwarmed or reduced-frequency runs distort timing; separately budgeting simultaneous GM reads/writes misses shared bandwidth, and tiny workloads can be dominated by launch overhead." - }, - "recommendations": [ - "Collect warmup and repeated measurements into a named round, preserve the raw CSVs, and distinguish the summary statistics from the subsequent bottleneck judgment.", - "Compare task duration, dominant pipeline ratios and per-core balance together; change one optimization point, rebuild as needed and archive the new round before deciding to keep it.", - "Check current versus rated frequency and shared read/write bandwidth before blaming the kernel; aggregate msprof ratios do not provide an instruction-by-instruction timeline." - ] - }, - "applicability": { - "scope": "Build, measure and diagnose one Ascend operator in isolation using msprof op, before interpreting system-level or multi-kernel performance.", - "prerequisites": "A built executable or test harness and access to real NPU hardware with msprof op for collection.", - "input_contract": "Operator executable/harness, collection settings and OPPROF output directory, plus the operator directory used to archive each measurement round.", - "produced_artifact": "Per-round raw CSV archives and observational min/average/max summaries, followed by a bottleneck interpretation and comparison of operator duration and core balance.", - "failure_modes": "Unwarmed or reduced-frequency runs distort timing; separately budgeting simultaneous GM reads/writes misses shared bandwidth, and tiny workloads can be dominated by launch overhead." - }, - "recommendations": [ - "Collect warmup and repeated measurements into a named round, preserve the raw CSVs, and distinguish the summary statistics from the subsequent bottleneck judgment.", - "Compare task duration, dominant pipeline ratios and per-core balance together; change one optimization point, rebuild as needed and archive the new round before deciding to keep it.", - "Check current versus rated frequency and shared read/write bandwidth before blaming the kernel; aggregate msprof ratios do not provide an instruction-by-instruction timeline." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-tuning.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-tuning.json deleted file mode 100644 index ff8c356f5..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-tuning.json +++ /dev/null @@ -1,98 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_operator_tuning", - "title": "ascend-operator-tuning", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Run a staged, accuracy-gated tuning campaign on a single Ascend NPU operator: set a numeric target up front, block tuning until an on-device accuracy pass is logged, escalate code-level then scheduling then instruction-level tuning with per-stage stall counters and kill-switches, and route each change by the metric contract (core utilization, bubble rate, load balance). Use after ascend-operator-profiling has measured a baseline and the operator is confirmed as the bottleneck.\n", - "tags": [ - "ascend", - "npu", - "operator", - "tuning", - "performance", - "pypto", - "tile" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md" - }, - "checkout_ref": "checkout:ascend-operator-tuning", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md": "38b151bc6b4e9734c6fa41efb7a30343559400c0c75bbdd26bd8a968b8ea4592", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json": "776fe88247ddacd3cf245920c580a9ff8f2a80fbc0ad1fa7d8be6d63b15a66f7" - }, - "skill_metadata": { - "name": "ascend-operator-tuning", - "description": "Run a staged, accuracy-gated tuning campaign on a single Ascend NPU operator: set a numeric target up front, block tuning until an on-device accuracy pass is logged, escalate code-level then scheduling then instruction-level tuning with per-stage stall counters and kill-switches, and route each change by the metric contract (core utilization, bubble rate, load balance). Use after ascend-operator-profiling has measured a baseline and the operator is confirmed as the bottleneck.\n", - "tags": [ - "ascend", - "npu", - "operator", - "tuning", - "performance", - "pypto", - "tile" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Run a bounded, accuracy-gated tuning campaign for a single Ascend operator already identified as the bottleneck, escalating from code-level to scheduling and instruction-level changes.", - "prerequisites": "A measured operator baseline, an explicit numeric target and a logged on-device accuracy pass before tuning; performance comparisons use real hardware.", - "input_contract": "Operator implementation, fixed shape/dtype/card and warmup policy, accuracy result, per-core work/wait measurements and round logs.", - "produced_artifact": "A bounded campaign record of each change, accuracy and measured timing delta, kept/reverted rounds, current stage and documented stop condition.", - "refusal_point": "Do not start without the baseline and accuracy gate; revert accuracy failures, and stop/escalate repeated failures, timeouts, exhausted stages or the campaign budget.", - "failure_modes": "Scheduler-idle and predecessor-wait bottlenecks need different fixes; output inplace-reshape can corrupt slice writes, and a changed analyzer log format can produce a misleading zero-core report." - }, - "recommendations": [ - "Change one parameter and recheck accuracy each round; keep the failure record when reverting and compare the final configuration against the same baseline workload and warmup policy.", - "Route scheduler bubbles separately from predecessor dependencies and enter instruction-level tuning only after isolating a specific long task.", - "Verify the analyzer parsed real cores before trusting its diagnosis, and rederive PyPTO tile/buffer/scheduling settings for the target SoC rather than copying numeric examples." - ] - }, - "applicability": { - "scope": "Run a bounded, accuracy-gated tuning campaign for a single Ascend operator already identified as the bottleneck, escalating from code-level to scheduling and instruction-level changes.", - "prerequisites": "A measured operator baseline, an explicit numeric target and a logged on-device accuracy pass before tuning; performance comparisons use real hardware.", - "input_contract": "Operator implementation, fixed shape/dtype/card and warmup policy, accuracy result, per-core work/wait measurements and round logs.", - "produced_artifact": "A bounded campaign record of each change, accuracy and measured timing delta, kept/reverted rounds, current stage and documented stop condition.", - "refusal_point": "Do not start without the baseline and accuracy gate; revert accuracy failures, and stop/escalate repeated failures, timeouts, exhausted stages or the campaign budget.", - "failure_modes": "Scheduler-idle and predecessor-wait bottlenecks need different fixes; output inplace-reshape can corrupt slice writes, and a changed analyzer log format can produce a misleading zero-core report." - }, - "recommendations": [ - "Change one parameter and recheck accuracy each round; keep the failure record when reverting and compare the final configuration against the same baseline workload and warmup policy.", - "Route scheduler bubbles separately from predecessor dependencies and enter instruction-level tuning only after isolating a specific long task.", - "Verify the analyzer parsed real cores before trusting its diagnosis, and rederive PyPTO tile/buffer/scheduling settings for the target SoC rather than copying numeric examples." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-optimization-casebook.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-optimization-casebook.json deleted file mode 100644 index 93dee7f17..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-optimization-casebook.json +++ /dev/null @@ -1,90 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_optimization_casebook", - "title": "ascend-optimization-casebook", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Casebook of NPU optimization cases for Ascend inference systems: symptom → diagnosis → action → evidence records for memory-bound, compute-bound, communication-exposed, graph-compilation, and AI-CPU-fallback bottleneck classes. Use when a bottleneck has been classified and you need prior art for what to try next. Complement to inference-optimization-casebook (which covers the casebook *pattern*; this carries the actual NPU cases).\n", - "tags": [ - "ascend", - "npu", - "optimization", - "casebook", - "prior-art" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md" - }, - "checkout_ref": "checkout:ascend-optimization-casebook", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md": "647f5e9a288f2c4e1617ba53df59d9d827df749eb4fe86182d21a9c41175caf8", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/metadata.json": "e5e49bf7b24fbc32ddaf1aa8ec14051ef6393533b12bb2c55fff2fb0da927541" - }, - "skill_metadata": { - "name": "ascend-optimization-casebook", - "description": "Casebook of NPU optimization cases for Ascend inference systems: symptom → diagnosis → action → evidence records for memory-bound, compute-bound, communication-exposed, graph-compilation, and AI-CPU-fallback bottleneck classes. Use when a bottleneck has been classified and you need prior art for what to try next. Complement to inference-optimization-casebook (which covers the casebook *pattern*; this carries the actual NPU cases).\n", - "tags": [ - "ascend", - "npu", - "optimization", - "casebook", - "prior-art" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Find source-derived prior cases for an already classified Ascend inference bottleneck, including fallback, memory traffic, exposed communication, graph startup and host scheduling gaps.", - "prerequisites": "A bottleneck classification from profiling and a check of the selected case preconditions/mechanism before applying it.", - "input_contract": "Bottleneck symptoms and matching profiler evidence, model/hardware/CANN/configuration context and, for a new case, measured before/after metrics.", - "produced_artifact": "A matching prior-art case and evidence requirements for a later measured application; a new case record requires a verified win or clear failure with a metric comparison.", - "refusal_point": "Do not add an INCONCLUSIVE case or one without a measured metric comparison; a subjective direction is insufficient evidence." - }, - "recommendations": [ - "Classify first, then check the matching case preconditions and use its owning procedure to apply and measure the candidate; the casebook itself does not perform profiling.", - "Keep profiler mechanism evidence and same-window end-to-end outcomes together, and retain locally_verified=false until the applied case has actually run." - ] - }, - "applicability": { - "scope": "Find source-derived prior cases for an already classified Ascend inference bottleneck, including fallback, memory traffic, exposed communication, graph startup and host scheduling gaps.", - "prerequisites": "A bottleneck classification from profiling and a check of the selected case preconditions/mechanism before applying it.", - "input_contract": "Bottleneck symptoms and matching profiler evidence, model/hardware/CANN/configuration context and, for a new case, measured before/after metrics.", - "produced_artifact": "A matching prior-art case and evidence requirements for a later measured application; a new case record requires a verified win or clear failure with a metric comparison.", - "refusal_point": "Do not add an INCONCLUSIVE case or one without a measured metric comparison; a subjective direction is insufficient evidence." - }, - "recommendations": [ - "Classify first, then check the matching case preconditions and use its owning procedure to apply and measure the candidate; the casebook itself does not perform profiling.", - "Keep profiler mechanism evidence and same-window end-to-end outcomes together, and retain locally_verified=false until the applied case has actually run." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-performance-analysis.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-performance-analysis.json deleted file mode 100644 index 6a0df1922..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-performance-analysis.json +++ /dev/null @@ -1,129 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_performance_analysis", - "title": "ascend-performance-analysis", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose performance bottlenecks on Ascend NPU systems using CANN profiling data. Maps GPU profiling concepts (nsys, ncu, SOL%, roofline) to their NPU equivalents (msprof, torch_npu.profiler, CSV/JSON outputs, AI Core utilization, HCCL communication analysis). Covers measurement discipline on shared NPU machines and evidence acceptance for automated NPU optimization stages. Use when profiling data is available or a stage report needs its execution and validation evidence checked.\n", - "tags": [ - "ascend", - "npu", - "profiling", - "performance", - "cann" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md" - }, - "checkout_ref": "checkout:ascend-performance-analysis", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md": "a057b7f86a24f34a8f84abbbb4910d58d4e50ff47060e1c404600017e3febe89", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/metadata.json": "f3fb35b14b5b1054a9192459862451149395368f4a75f430caef2f41ad050296", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-CANN-2.0-LICENSE.md": "89b1c114df62cfd183004f38d18d07f186436c3b37203dd69c3e84becf9d2282", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md": "f56e4125325c9c1b9235a1d64fd732861c7686444492422daff44ba8dd648be4" - }, - "skill_metadata": { - "name": "ascend-performance-analysis", - "description": "Diagnose performance bottlenecks on Ascend NPU systems using CANN profiling data. Maps GPU profiling concepts (nsys, ncu, SOL%, roofline) to their NPU equivalents (msprof, torch_npu.profiler, CSV/JSON outputs, AI Core utilization, HCCL communication analysis). Covers measurement discipline on shared NPU machines and evidence acceptance for automated NPU optimization stages. Use when profiling data is available or a stage report needs its execution and validation evidence checked.\n", - "tags": [ - "ascend", - "npu", - "profiling", - "performance", - "cann" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://gitcode.com/cann/cannbot", - "commit": "c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110", - "paths": [ - "LICENSE", - "README.md", - ".gitmodules", - "plugins-official/LICENSE", - "plugins-official/README.md", - "plugins-official/model-infer-optimize/.claude-plugin/plugin.json", - "plugins-official/model-infer-optimize/.codex-plugin/plugin.json", - "plugins-official/model-infer-optimize/AGENTS.md", - "plugins-official/model-infer-optimize/plugin-sources.json", - "plugins-official/model-infer-optimize/plugin-install.json", - "plugins-official/model-infer-optimize/workflows/optimize-workflow.md", - "plugins-official/model-infer-optimize/agents/model-infer-analyzer.md", - "plugins-official/model-infer-optimize/agents/model-infer-implementer.md", - "plugins-official/model-infer-optimize/agents/model-infer-reviewer.md", - "plugins-official/model-infer-optimize/hooks/hooks.json", - "plugins-official/model-infer-optimize/hooks/pre_tool_use.py", - "plugins-official/model-infer-optimize/hooks/post_tool_use.py", - "plugins-official/model-infer-optimize/hooks/subagent_stop.py", - "plugins-official/model-infer-optimize/workflows/scripts/archive_progress.py" - ], - "readset": "references/cannbot-stage-evidence.md" - } - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "Only the incremental canonical CannBot fixed-stage evidence boundary is normalized; inherited MindIE-SD guidance remains unchanged.", - "CannBot callback predicates do not establish numerical accuracy or performance; dependent skills, agent-host integration and NPU execution are unverified." - ], - "provenance_readsets": [ - "references/cannbot-stage-evidence.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Interpret Ascend CANN profiling evidence from step breakdown through kernel/utilization/communication analysis, and assess execution/validator evidence in automated optimization-stage reports.", - "prerequisites": "Usable warm-state captures with representative steps, required files and non-null utilization columns; healthy cards and a consistent valid topology for multi-card comparisons.", - "input_contract": "CANN step/kernel/utilization/communication outputs with verified column semantics; for stage acceptance, the loaded candidate, baseline, quality-validator output and comparable measurements.", - "produced_artifact": "An evidence-qualified bottleneck classification and optimization direction; stage conclusions retain missing-execution/validator INCONCLUSIVE or observed gate-failure FAIL.", - "failure_modes": "Summing concurrent communication durations double-counts overlap; misidentified columns can drop communication, and a successful export can still lack usable utilization data." - }, - "recommendations": [ - "Start from the step-level computing/communication/overlap/free breakdown and stop deepening the analysis once the bottleneck is located; do not sum concurrent streams as wall time.", - "Inspect actual column headers and enum values on the installed CANN version, then verify files and non-null metrics before interpreting utilization heuristics.", - "Use same-window interleaved baseline/candidate measurements on shared machines; progress markers, hashes and source-derived thresholds do not substitute for quality and execution evidence." - ] - }, - "applicability": { - "scope": "Interpret Ascend CANN profiling evidence from step breakdown through kernel/utilization/communication analysis, and assess execution/validator evidence in automated optimization-stage reports.", - "prerequisites": "Usable warm-state captures with representative steps, required files and non-null utilization columns; healthy cards and a consistent valid topology for multi-card comparisons.", - "input_contract": "CANN step/kernel/utilization/communication outputs with verified column semantics; for stage acceptance, the loaded candidate, baseline, quality-validator output and comparable measurements.", - "produced_artifact": "An evidence-qualified bottleneck classification and optimization direction; stage conclusions retain missing-execution/validator INCONCLUSIVE or observed gate-failure FAIL.", - "failure_modes": "Summing concurrent communication durations double-counts overlap; misidentified columns can drop communication, and a successful export can still lack usable utilization data." - }, - "recommendations": [ - "Start from the step-level computing/communication/overlap/free breakdown and stop deepening the analysis once the bottleneck is located; do not sum concurrent streams as wall time.", - "Inspect actual column headers and enum values on the installed CANN version, then verify files and non-null metrics before interpreting utilization heuristics.", - "Use same-window interleaved baseline/candidate measurements on shared machines; progress markers, hashes and source-derived thresholds do not substitute for quality and execution evidence." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "Only the incremental canonical CannBot fixed-stage evidence boundary is normalized; inherited MindIE-SD guidance remains unchanged.", - "CannBot callback predicates do not establish numerical accuracy or performance; dependent skills, agent-host integration and NPU execution are unverified." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-video-diffusion-optimization.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-video-diffusion-optimization.json deleted file mode 100644 index cf46fd63c..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-video-diffusion-optimization.json +++ /dev/null @@ -1,121 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascend_video_diffusion_optimization", - "title": "ascend-video-diffusion-optimization", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Run and verify goal-driven Ascend video diffusion optimization. Specializes the backend-agnostic video-diffusion-optimization skeleton with Ascend NPU operator, layout and profiler references. Use when the user asks for latency, RTF, throughput, or HBM targets across VLM, DiT, MoE, attention, or VAE on Ascend NPU. Not for text-only LLMs.", - "tags": [ - "ascend", - "npu", - "video-diffusion", - "dit", - "moe", - "attention", - "vae", - "performance", - "profiling" - ], - "origin": [ - "first-party" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/SKILL.md" - }, - "checkout_ref": "checkout:ascend-video-diffusion-optimization", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/SKILL.md": "53c21e64f0dd47f9510faef1f2c01e1283899787ba7ebfe70cdabd8e57a76f00", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/metadata.json": "35f725c9d9e0a8638b234da798232d2ae49a37b70f0b0360710bce333f9265ca", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/references/field-notes-video-diffusion.md": "dadac7b23462fdc325218c72ddf8a2ac553151761a5d3e45b854b53d2c8c6908" - }, - "skill_metadata": { - "name": "ascend-video-diffusion-optimization", - "description": "Run and verify goal-driven Ascend video diffusion optimization. Specializes the backend-agnostic video-diffusion-optimization skeleton with Ascend NPU operator, layout and profiler references. Use when the user asks for latency, RTF, throughput, or HBM targets across VLM, DiT, MoE, attention, or VAE on Ascend NPU. Not for text-only LLMs.", - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "tags": [ - "ascend", - "npu", - "video-diffusion", - "dit", - "moe", - "attention", - "vae", - "performance", - "profiling" - ], - "empirical_validation": "not_run", - "verification": { - "locally_verified": false, - "evidence_level": "guidance", - "note": "Procedure is first-hand workflow guidance. The linked field notes are not reproducible as published. No acceptance campaign under this repository's evidence contract backs this skill, and no public performance claim is made." - }, - "applicability": { - "scope": "Ascend NPU video diffusion inference and serving optimization", - "hardware": "Ascend NPU; the measure-change-verify workflow skeleton is backend-agnostic and lives in methods/video-diffusion-optimization, while the operator, layout and profiler references here are Ascend-specific", - "prerequisites": "A reproducible benchmark, a frozen quality check, and numeric gates defined in the governing acceptance issue", - "input_contract": "Issue-scoped performance and quality gates, timing boundary, representative and held-out workloads, frozen model/runtime/input settings and an authorized optimization budget.", - "produced_artifact": "An experiment ledger and final report of the frozen goal, baseline/final samples, quality/resource checks, accepted or reverted changes and remaining gap; verdicts require the acceptance issue evidence.", - "failure_modes": "Packed-layout conversion from noncontiguous weights can corrupt outputs; missing side-stream lifetime/order handling or omitted layout-copy costs can invalidate an optimization comparison." - }, - "limitations": [ - "Ascend specialization of the backend-agnostic methods/video-diffusion-optimization skeleton; every operator, layout and profiler reference (TransData, FRACTAL_NZ, CANN) is Ascend-specific and must be re-derived per backend.", - "Candidate workflow guidance; not independently validated beyond the field notes.", - "Field notes are non-reproducible as published and must not be cited as evidence.", - "Model identifiers, campaign revisions, and numeric gates are intentionally excluded and remain issue-scoped.", - "Remaining figures are order-of-magnitude recollections." - ], - "provenance_readsets": [ - "references/field-notes-video-diffusion.md" - ], - "references": [ - "../../../methods/video-diffusion-optimization/SKILL.md", - "../../../methods/video-diffusion-optimization/scripts/evaluate_target.py", - "../../../methods/video-diffusion-optimization/assets/campaign.template.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Follow the common goal-contract and measure-change-verify loop, including final restarted-artifact acceptance; an operator win does not establish end-to-end improvement.", - "Materialize contiguous weights before FRACTAL_NZ conversion when strides differ from physical order, and attribute TransData/copy costs before proposing fusion.", - "Use field notes only to form hypotheses; derive thresholds, chunking, precision and sparsity from the target model measurements, and preserve stream dependencies and tensor lifetimes." - ] - }, - "applicability": { - "scope": "Ascend NPU video diffusion inference and serving optimization", - "hardware": "Ascend NPU; the measure-change-verify workflow skeleton is backend-agnostic and lives in methods/video-diffusion-optimization, while the operator, layout and profiler references here are Ascend-specific", - "prerequisites": "A reproducible benchmark, a frozen quality check, and numeric gates defined in the governing acceptance issue", - "input_contract": "Issue-scoped performance and quality gates, timing boundary, representative and held-out workloads, frozen model/runtime/input settings and an authorized optimization budget.", - "produced_artifact": "An experiment ledger and final report of the frozen goal, baseline/final samples, quality/resource checks, accepted or reverted changes and remaining gap; verdicts require the acceptance issue evidence.", - "failure_modes": "Packed-layout conversion from noncontiguous weights can corrupt outputs; missing side-stream lifetime/order handling or omitted layout-copy costs can invalidate an optimization comparison." - }, - "recommendations": [ - "Follow the common goal-contract and measure-change-verify loop, including final restarted-artifact acceptance; an operator win does not establish end-to-end improvement.", - "Materialize contiguous weights before FRACTAL_NZ conversion when strides differ from physical order, and attribute TransData/copy costs before proposing fusion.", - "Use field notes only to form hypotheses; derive thresholds, chunking, precision and sparsity from the target model measurements, and preserve stream dependencies and tensor lifetimes." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Ascend specialization of the backend-agnostic methods/video-diffusion-optimization skeleton; every operator, layout and profiler reference (TransData, FRACTAL_NZ, CANN) is Ascend-specific and must be re-derived per backend.", - "Candidate workflow guidance; not independently validated beyond the field notes.", - "Field notes are non-reproducible as published and must not be cited as evidence.", - "Model identifiers, campaign revisions, and numeric gates are intentionally excluded and remain issue-scoped.", - "Remaining figures are order-of-magnitude recollections." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascendc-tiling-design.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascendc-tiling-design.json deleted file mode 100644 index ad59f1b04..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/ascendc-tiling-design.json +++ /dev/null @@ -1,94 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.ascendc_tiling_design", - "title": "ascendc-tiling-design", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Design tiling for a custom AscendC operator: normalize the shape first (axis-collapse or flatten), answer the fullload-vs-split question for UB, apply the universal four-element template (multi-core split, UB split, buffer plan, branch matrix), and route per operator class (reduction, broadcast, elementwise, matmul, conversion) using algorithm-selection and chip-generation tables (DAV_2201 vs DAV_3510). Use when writing or restructuring an AscendC operator's tiling logic, before running any performance tuning campaign.\n", - "tags": [ - "ascend", - "npu", - "ascendc", - "tiling", - "operator", - "kernel", - "ub" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md" - }, - "checkout_ref": "checkout:ascendc-tiling-design", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md": "93aa1b775ced304cebfe6c4dec0503ddb2260ecb3cb01edc17ce329446c5ce2b", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/metadata.json": "03cfd09faa3edf42f9d08d0d8878fe5cb7c7a376abacf211dd8f500c698f3a8f" - }, - "skill_metadata": { - "name": "ascendc-tiling-design", - "description": "Design tiling for a custom AscendC operator: normalize the shape first (axis-collapse or flatten), answer the fullload-vs-split question for UB, apply the universal four-element template (multi-core split, UB split, buffer plan, branch matrix), and route per operator class (reduction, broadcast, elementwise, matmul, conversion) using algorithm-selection and chip-generation tables (DAV_2201 vs DAV_3510). Use when writing or restructuring an AscendC operator's tiling logic, before running any performance tuning campaign.\n", - "tags": [ - "ascend", - "npu", - "ascendc", - "tiling", - "operator", - "kernel", - "ub" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Design or restructure AscendC tiling for the covered reduction, broadcast and elementwise classes, plus the bounded mxfp8 fusion and small-channel transpose cases.", - "input_contract": "Operator class and semantics, normalized shapes/axes, dtype, target chip generation, valid/aligned lengths and intermediate-buffer requirements.", - "produced_artifact": "A multi-core split, UB chunk formula, buffer-size/total-usage plan and branch matrix with edge tests; these are design outputs, not benchmark results.", - "failure_modes": "Budgeting only inputs omits intermediate UB storage; mixing valid and aligned lengths or copy units corrupts boundaries, and choosing unsupported generation-specific primitives invalidates the design." - }, - "recommendations": [ - "Normalize axes first, then decide full-load versus chunking from the entire live working set; derive tile size from buffer, axis, API and load-balance constraints.", - "Keep valid compute/copy lengths separate from aligned storage offsets and sizes, and complete dtype, tail/alignment and boundary branches before tuning.", - "Select primitives by the target generation and preserve class-specific coverage limits; re-evaluate sample constants and API semantics instead of treating formulas as tested performance evidence." - ] - }, - "applicability": { - "scope": "Design or restructure AscendC tiling for the covered reduction, broadcast and elementwise classes, plus the bounded mxfp8 fusion and small-channel transpose cases.", - "input_contract": "Operator class and semantics, normalized shapes/axes, dtype, target chip generation, valid/aligned lengths and intermediate-buffer requirements.", - "produced_artifact": "A multi-core split, UB chunk formula, buffer-size/total-usage plan and branch matrix with edge tests; these are design outputs, not benchmark results.", - "failure_modes": "Budgeting only inputs omits intermediate UB storage; mixing valid and aligned lengths or copy units corrupts boundaries, and choosing unsupported generation-specific primitives invalidates the design." - }, - "recommendations": [ - "Normalize axes first, then decide full-load versus chunking from the entire live working set; derive tile size from buffer, axis, API and load-balance constraints.", - "Keep valid compute/copy lengths separate from aligned storage offsets and sizes, and complete dtype, tail/alignment and boundary branches before tuning.", - "Select primitives by the target generation and preserve class-specific coverage limits; re-evaluate sample constants and API semantics instead of treating formulas as tested performance evidence." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/cann-attribution-overlap.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/cann-attribution-overlap.json deleted file mode 100644 index 64ce54902..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/cann-attribution-overlap.json +++ /dev/null @@ -1,98 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.cann_attribution_overlap", - "title": "cann-attribution-overlap", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Attribute NPU trace kernels to model source and plan bounded multistream overlap: select a representative step, map kernels to source owners via code_ref, build the module/operator dependency DAG (data, state, event, collective-order edges), classify serial/candidate/pending groups, choose minimal-first overlap with an explicit execution-mode synchronization contract, and accept only through correctness-first checks. Keeps the !302 import lineage distinct from later revisions, carries known formula/threshold conflicts as unresolved, and qualifies memory evidence by domain (device HBM/UB counters are not host RAM). Planning procedure only — no formula automation for disputed definitions.\n", - "tags": [ - "cann", - "npu", - "attribution", - "dependency-dag", - "multistream", - "overlap", - "trace" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md" - }, - "checkout_ref": "checkout:cann-attribution-overlap", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md": "0e6f094dc554abea12e07def50f20b233b47127779a8aefad595b4ab1efd24c2", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/metadata.json": "7ad137076afb55fbea74c6c5c0c0ca8eb3a8e615a765ec23199e8df75a8abd4f" - }, - "skill_metadata": { - "name": "cann-attribution-overlap", - "description": "Attribute NPU trace kernels to model source and plan bounded multistream overlap: select a representative step, map kernels to source owners via code_ref, build the module/operator dependency DAG (data, state, event, collective-order edges), classify serial/candidate/pending groups, choose minimal-first overlap with an explicit execution-mode synchronization contract, and accept only through correctness-first checks. Keeps the !302 import lineage distinct from later revisions, carries known formula/threshold conflicts as unresolved, and qualifies memory evidence by domain (device HBM/UB counters are not host RAM). Planning procedure only — no formula automation for disputed definitions.\n", - "tags": [ - "cann", - "npu", - "attribution", - "dependency-dag", - "multistream", - "overlap", - "trace" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Attribute representative CANN trace kernels to source owners and plan bounded multistream overlap using data, state, event and collective-order dependencies.", - "prerequisites": "Pin the installed CANN/TorchNPU/TorchAir versions before choosing one execution-mode synchronization API family; communication proposals require communication evidence.", - "input_contract": "Representative-step selection and shapes, kernel/stream inventory with code_ref, adjacent wait windows and dependency edges; communication analysis also needs rank/transport evidence.", - "produced_artifact": "An attribution map, serial/candidate/pending dependency groups and a minimal overlap plan with joins, enable/fallback path, resource tradeoffs and acceptance checks.", - "failure_modes": "Kernel names and mechanical coverage can misidentify ownership; logical independence can hide resource contention, and overlap ratios or summed durations can hide unchanged wall time or new tails." - }, - "recommendations": [ - "Join hotspots to source owners and adjacent wait windows before selecting a target; keep compute cost, waiting and host gaps separate.", - "Build explicit data/state/event/collective-order edges, then verify one correct overlap point before increasing overlap or partitioning resources.", - "Require observed overlap, preserved state/order and fallback behavior plus latency and outside-module checks; report wall time, interval unions and duration sums separately.", - "Carry disputed formulas and source-revision distinctions forward; device free time alone does not identify Python, GIL or OS causes without matching host evidence." - ] - }, - "applicability": { - "scope": "Attribute representative CANN trace kernels to source owners and plan bounded multistream overlap using data, state, event and collective-order dependencies.", - "prerequisites": "Pin the installed CANN/TorchNPU/TorchAir versions before choosing one execution-mode synchronization API family; communication proposals require communication evidence.", - "input_contract": "Representative-step selection and shapes, kernel/stream inventory with code_ref, adjacent wait windows and dependency edges; communication analysis also needs rank/transport evidence.", - "produced_artifact": "An attribution map, serial/candidate/pending dependency groups and a minimal overlap plan with joins, enable/fallback path, resource tradeoffs and acceptance checks.", - "failure_modes": "Kernel names and mechanical coverage can misidentify ownership; logical independence can hide resource contention, and overlap ratios or summed durations can hide unchanged wall time or new tails." - }, - "recommendations": [ - "Join hotspots to source owners and adjacent wait windows before selecting a target; keep compute cost, waiting and host gaps separate.", - "Build explicit data/state/event/collective-order edges, then verify one correct overlap point before increasing overlap or partitioning resources.", - "Require observed overlap, preserved state/order and fallback behavior plus latency and outside-module checks; report wall time, interval unions and duration sums separately.", - "Carry disputed formulas and source-revision distinctions forward; device free time alone does not identify Python, GIL or OS causes without matching host evidence." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torch-npu.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torch-npu.json deleted file mode 100644 index 85782eb4e..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torch-npu.json +++ /dev/null @@ -1,231 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.torch_npu", - "title": "torch-npu", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Operate and diagnose TorchNPU on Ascend across installation, PrivateUse1 dispatch, operator development, memory and streams, graphs and compilation, HCCL, precision and profiling, with pinned source evidence and explicit backend limits.", - "tags": [ - "torch-npu", - "ascend", - "pytorch" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md" - }, - "checkout_ref": "checkout:torch-npu", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md": "7e92d5c8d3d49ebe72613a2eac0855b8d7faa670277f6d1a1f6b84ed0fe7441b", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/metadata.json": "d0fd35c69bea254ddc3c494d3ed61033c8ccdcb4262cbfa4f66a53c2ae70772f", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/NOTICE.md": "ae6554ebec75eadce7d3bcb7490d2be7e49ff923d9c29fdd93549bc6c94ca3af", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/capability-inventory.md": "e6bae7bbeecea8b16c9e911ac40aa385f181da855336e0ac1fe6d9ed1c3a52e9", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-LICENSE.txt": "325488df8d01b0601a46b395904b6ce72f4870cd608f49268904e994dab15f97", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-THIRD-PARTY.txt": "cb91e5fa816c6f6ff7a6a3cf32110c3402b1d4c76778de106130b227782013d0", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/source-identities.json": "f1ee46af6bfcea7e41959ceab981515cf902fe7f1d11998e69145cb86d897c37", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-LICENSE.txt": "99317d26c99c8037890c770fd67fa84cc78d210581ed563d3b1abe42976f47be", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-THIRD-PARTY.txt": "167f93c40acf8a26e4f6cc293df727c04ef869e54a10e7517ce2de60eaf2593f", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-docs-LICENSE.txt": "9e5f1b3c610b9c2da5c313bf81d577a7d1acec686bdb0384edefa6df0f90cd94", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/repo-provenance.md": "18cb87b57a1ffa25e0954f629ddc1fac3137a2ee2d5d362be30083e5bfc12dc9", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/repo-routing-metadata.json": "eeab2fedf7db673c0ce2d8f53641d4bcaa9b0fd84b0aa30cf746f894c548a268", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-manifest.json": "b1d4cbdad4cde3838d3bbc16ccede8c79825dd7280a3c3c081507d7f51e86726", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-map.md": "d64683d861c1cca6f61a5e90500f98c7f5dd9a7256035433c11e39cc44eaad15", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-script-inventory.md": "c43c83daa152856c4fab278f25d05504735e25c427231bcf177b72e01536819d", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/support-workflows.md": "002121bded102c51966754aff40c3f621cc2e69d35ff29fa187582d496bb8ce4", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/inspect_environment.py": "a157f5947217c5baadb2a4957a68c91fa1dbef343c97ac2f7f4d59bf1e4cb3b7", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/operator_contract.py": "45f076bb5353798d4b494f7b7e990796dfb1418d7ef44643521bd669bc348b32", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/runtime_smoke.py": "e5c7d587e54a6e4fd5dd8c45d5ef36990143ee40e24ff1680d49b7637894c248", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md": "7fe7e11dba31a55268fe9649628f84c21adf6aab33e134716abeac0dffbb306e", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/metadata.json": "3d4783529a657997ce9942ad61e4bcdf9e65891fe1325ffdfba53e592165dc8d", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/hccl.md": "bd6b67c243032785f60f100d75c450885bfb1e6585af9100482fca3dfbe86919", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/source-map.md": "06bebd8bcae81b253573a4096a905cd264da2a75a9b13548f646c2ab9b698fd5", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/troubleshooting.md": "946650a1fa1c3f805328e3071ed7a0ed907b71af6a4d746d3fb4bad869d5aaad", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md": "0b3217a1605d7a48c6bd4bd521e2b0258b76955ac80824c0014037020fdcdc48", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/metadata.json": "bbf071c25b3cee9e0ee5c420e6d702b43bb8bc26feeb0e460a12118ca96622dc", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/graphs.md": "ec9e13041eb2d5abf5e2055e667fcef79391dad0b32756b963efb69d2be7354c", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/source-map.md": "89aedad45ab2eb524053909060344e5b11d3b9923b7cff8d1e661a6a097b00f2", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/troubleshooting.md": "35be0f57c998a5a6ef24f46f67f3a37e17b0d519a2604a38ec0498b21f374437", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/SKILL.md": "bb324b9fb8e0056fcd3c63211d7e48f06f4dd4a9cdc8dd050bb5c156f23d6ba7", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/metadata.json": "6046945dcde6d3e5f20748e58e6392cd906242c3130d59111cf3170fbda4fc8a", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/source-map.md": "b5266c944b0abd8bbb35d50aa6eddb9452725ba2308523f0bc8414dbc87f8802", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/stack.md": "4e9a8abbf45eff3b3e79de9b0fa0907479b77f729544cd1c772d2334606fca94", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/troubleshooting.md": "799214c15da2ce43f2f9ab37347e174fff3346c3e27507022efd7a2aa30fcef6", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md": "2f45f2dca635d52859d2a9146aa01edb4381344d9996fa8436373c12ef4cd61d", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/metadata.json": "425049f0312f22da960bcbe2ab40433a13b42f1090c6d4bfda06fc45b7240403", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/lifetimes.md": "bc5551f922534d1db0cb6ade6ea1dbbd4f26419cd9aa0b27f9aec9415d691743", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/source-map.md": "05114bd16d7cf22c5604a3f1885f44b8be225fa2644302a8ef805193a46601c2", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/troubleshooting.md": "b9d5d4d61087f27b2761c9184c99f98da583948c393f6a3c6d189b87ef9f1466", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/SKILL.md": "ce24c7537f0722733a086d046a760578c7cc76f1a3d9030fcceda147513d5b93", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/metadata.json": "079e2a1079753a8177f5ad8812bcca6dc715c70a49132f53ce5b915710f02f26", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/development.md": "70994e5deeb138ca3dce578a743515840a91ff6303f0328eaff6de8fc308313a", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/source-map.md": "29863b23b49040728c4d928a149477aeb966da833cb4ce21a6bad5212ca30c9e", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/troubleshooting.md": "0c9828b96ab7943ab0b7006ac8520e58072fe7f2544a78cb9111c1f224255c6d", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/SKILL.md": "9adda9cba726252229bf322fe7ea021864100a49330a196cb4abb563d86c83e9", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/metadata.json": "cafa3c5d32454ae4fd2bb37d874e83bfcf5479e769c8c915f1db432e046fb8a5", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/dispatch.md": "19d4a9120bd1fcf4aea2dcacbf8e66c0e9f3fc3ca5e56fc6c468a3721daff973", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/source-map.md": "432913d63c0e1a54ad75da6b7e51489dfd4589a208a51b0a0245ba679cd29c4f", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/troubleshooting.md": "193c7499076e77ba82772ac5d939cc377d9301f7e6825353b98df09f737c85cf", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/SKILL.md": "621ddc124b4da8b06da5f0af187cfee8ce330d497495eab0b284c88ee47c403a", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/metadata.json": "008c7a128e6e73d86c4453d3b244afc90139929c24f8420ea5ed7aaaed519066", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/profiling.md": "f61c6f33cd9b63546c44ba14fed1ba48ce0f0318becf7a88fe9a1d658f5e83f9", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/source-map.md": "2550e9a28209a7df0a0d2f571af39985ca629c42a1be6f845e50fd265fc6ccb8", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/troubleshooting.md": "c60466ad74a0e36b6f74001a0ab756fd2dfc674d059e038fbcd63770038396c6", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/SKILL.md": "7d89e992b5cad7383e5513c067f4e590370197225208c8e97eee3be9eaf3332a", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/metadata.json": "73e71cdcb98e187f1470eb71ec9030fa870e46d8cf6eaf396860b712c491aefa", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/source-map.md": "3bfc78d9348f9a1072120f82d7a1a7f042b4fe2ce908a005e22716a797dc0c7b", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/training.md": "0695532020c71d061ddf40ad16536621e004c23dd05d27aaa9f5b46101f6ef62", - "src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/troubleshooting.md": "0f7fb3879d51c793ca0fd6d14fd34352f94297f075295d01488ffab4c5fc0fa4" - }, - "skill_metadata": { - "name": "torch-npu", - "description": "Operate and diagnose TorchNPU on Ascend across installation, PrivateUse1 dispatch, operator development, memory and streams, graphs and compilation, HCCL, precision and profiling, with pinned source evidence and explicit backend limits.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "references": [ - "../../../hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md", - "../../../hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md" - ], - "provenance_readsets": [ - "references/source-map.md", - "references/source-manifest.json" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/Ascend/pytorch", - "commit": "046934140b96836c3fcef14d0f39ae54dc0a06bf", - "paths": [ - "COMPATIBILITY.en.md", - "README.md", - "docs/en/api/environment_variable/op_execution/ASCEND_LAUNCH_BLOCKING.md", - "docs/en/installation_guide/building_from_source.md", - "fxrt/README.md", - "generate_code.sh", - "setup.py", - "test/distributed/test_hccl_group.py", - "test/npu/test_aclgraph_multi_stream.py", - "test/npu/test_amp.py", - "test/npu/test_cpu_fallback_control.py", - "test/npu/test_import_contract.py", - "test/npu/test_stream.py", - "tools/setup_helpers/version.py", - "torch_npu/__init__.py", - "torch_npu/_inductor/__init__.py", - "torch_npu/_init/core/module_loader.py", - "torch_npu/_init/registry/backend.py", - "torch_npu/_init/registry/distributed.py", - "torch_npu/_init/registry/dynamo.py", - "torch_npu/_init/registry/registry_manager.py", - "torch_npu/contrib/transfer_to_npu.py", - "torch_npu/csrc/aten/AutoCastOps.cpp", - "torch_npu/csrc/aten/VariableFallbackKernel.cpp", - "torch_npu/csrc/core/npu/CachingHostAllocator.cpp", - "torch_npu/csrc/core/npu/NPUCachingAllocator.cpp", - "torch_npu/csrc/core/npu/NPUEvent.cpp", - "torch_npu/csrc/core/npu/NPUGraph.cpp", - "torch_npu/csrc/core/npu/NPUHooksInterface.cpp", - "torch_npu/csrc/core/npu/NPUStream.cpp", - "torch_npu/csrc/distributed/ProcessGroupHCCL.cpp", - "torch_npu/csrc/framework/utils/CpuFallbackUtils.h", - "torch_npu/csrc/profiler/profiler_mgr.cpp", - "torch_npu/distributed/tensor/__init__.py", - "torch_npu/dynamo/__init__.py", - "torch_npu/npu/__init__.py", - "torch_npu/npu/_sanitizer.py", - "torch_npu/npu/amp/autocast_mode.py", - "torch_npu/npu/amp/grad_scaler.py", - "torch_npu/npu/graphs.py", - "torch_npu/npu/streams.py", - "torch_npu/onnx/__init__.py", - "torch_npu/optim/__init__.py", - "torch_npu/profiler/profiler.py", - "torch_npu/profiler/scheduler.py", - "torch_npu/utils/serialization.py", - "torchnpugen/gen_backend_stubs.py", - "version.txt" - ], - "readset": "references/source-manifest.json" - }, - { - "type": "git", - "url": "https://github.com/Ascend/op-plugin", - "commit": "ad946817034b3270e309c81c54f0ee808d0b62cb", - "paths": [ - "examples/cpp_extension_full/torch_lib_impl/cpp_extension_full/ops.py", - "examples/cpp_extension_full/torch_lib_impl/csrc/add_custom.cpp", - "op_plugin/config/op_plugin_functions.yaml", - "op_plugin/ops/aclops/AddKernelNpu.cpp", - "op_plugin/ops/opapi/AddKernelNpuOpApi.cpp", - "torchnpugen/gen_op_plugin_functions.py" - ], - "readset": "references/source-manifest.json" - } - ] - }, - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "recommendations": [ - "Read the pinned SKILL.md and its prerequisites before choosing this procedure.", - "Read the packaged references and owner-local cases alongside the procedure." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-device-memory-lifecycle.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-device-memory-lifecycle.json deleted file mode 100644 index 139fcb6f0..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-device-memory-lifecycle.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.torchnpu_device_memory_lifecycle", - "title": "torchnpu-device-memory-lifecycle", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose TorchNPU device-HBM reuse, cross-stream lifetimes, cache retention and fragmentation, then propose version-gated allocator changes. Use for NPU allocator OOM or reuse investigations, not pinned-host RAM or network transport.", - "tags": [ - "ascend", - "torchnpu", - "device-memory", - "allocator", - "lifecycle" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md" - }, - "checkout_ref": "checkout:torchnpu-device-memory-lifecycle", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md": "8428df7157af024fa17cfe0b3818ed54bd9f468306dda46eb8d4f0edf4f3f230", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/metadata.json": "02de593e638e71c6debb763b6b0d5a5596e0b810bbe48450ead28dc43de83f99", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/references/source-notes.md": "80cba4d3a884e7ee2618326cf74b9f9c3a34e4d9cf443707f6bbe673f2d5ba65" - }, - "skill_metadata": { - "name": "torchnpu-device-memory-lifecycle", - "description": "Diagnose TorchNPU device-HBM reuse, cross-stream lifetimes, cache retention and fragmentation, then propose version-gated allocator changes. Use for NPU allocator OOM or reuse investigations, not pinned-host RAM or network transport.", - "tags": [ - "ascend", - "torchnpu", - "device-memory", - "allocator", - "lifecycle" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Diagnose TorchNPU device-HBM reuse, stream lifetimes, fragmentation and graph/private-pool retention before proposing allocator changes.", - "input_contract": "Allocation failure/size sequence, device and allocator path, installed versions/settings, stream and graph-pool timeline, retained tensor references and allocator statistics with synchronization boundaries.", - "produced_artifact": "A suspected allocator state transition with source evidence IDs, one eligible action, resource cost, counter-signals and a correctness-first verification/rollback plan.", - "refusal_point": "Missing evidence supports only a conditional proposal, not a diagnosed leak; stop when external/custom/shared allocation ownership is untraced and keep baseline settings for unsupported versions or live-tensor capacity pressure." - }, - "recommendations": [ - "Separate stream lifetime protection from producer/consumer ordering, and distinguish live tensors, pending events, inactive splits and private-pool retention before tuning.", - "Match split or expandable-segment proposals to the observed allocation pattern and actual feature gates; do not combine incompatible controls or import comment thresholds as defaults.", - "Treat cache release as synchronization/allocation work at a quiescent boundary; compare identical allocation sequences in fresh processes at the same synchronization points." - ] - }, - "applicability": { - "scope": "Diagnose TorchNPU device-HBM reuse, stream lifetimes, fragmentation and graph/private-pool retention before proposing allocator changes.", - "input_contract": "Allocation failure/size sequence, device and allocator path, installed versions/settings, stream and graph-pool timeline, retained tensor references and allocator statistics with synchronization boundaries.", - "produced_artifact": "A suspected allocator state transition with source evidence IDs, one eligible action, resource cost, counter-signals and a correctness-first verification/rollback plan.", - "refusal_point": "Missing evidence supports only a conditional proposal, not a diagnosed leak; stop when external/custom/shared allocation ownership is untraced and keep baseline settings for unsupported versions or live-tensor capacity pressure." - }, - "recommendations": [ - "Separate stream lifetime protection from producer/consumer ordering, and distinguish live tensors, pending events, inactive splits and private-pool retention before tuning.", - "Match split or expandable-segment proposals to the observed allocation pattern and actual feature gates; do not combine incompatible controls or import comment thresholds as defaults.", - "Treat cache release as synchronization/allocation work at a quiescent boundary; compare identical allocation sequences in fresh processes at the same synchronization points." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-pinned-host-memory-lifecycle.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-pinned-host-memory-lifecycle.json deleted file mode 100644 index 42c3ecdea..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-pinned-host-memory-lifecycle.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.torchnpu_pinned_host_memory_lifecycle", - "title": "torchnpu-pinned-host-memory-lifecycle", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy lifetime, reuse and cache controls, then propose version-gated host-allocator changes. Use for pin_memory pressure or host allocation stalls, not device HBM or network transport.", - "tags": [ - "ascend", - "torchnpu", - "host-memory", - "pinned-memory", - "lifecycle" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md" - }, - "checkout_ref": "checkout:torchnpu-pinned-host-memory-lifecycle", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md": "23e93ea7c98f99743509fdc04139d78ed2e7123c63e80b366c6beaba7eb70d0d", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/metadata.json": "0fa51de079663043fd5785beffb153d989b1cac5833dbd2c99535f6b782a5e95", - "src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/references/source-notes.md": "435193590b4a2ba32e2e4c6fa9d7e8f889528d412cbbb48e35ec5433f7dab384" - }, - "skill_metadata": { - "name": "torchnpu-pinned-host-memory-lifecycle", - "description": "Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy lifetime, reuse and cache controls, then propose version-gated host-allocator changes. Use for pin_memory pressure or host allocation stalls, not device HBM or network transport.", - "tags": [ - "ascend", - "torchnpu", - "host-memory", - "pinned-memory", - "lifecycle" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy ownership/readiness, reuse and path-specific cache controls.", - "input_contract": "Installed versions/settings, host pressure and allocation/reuse distribution, stream timeline, owned versus registered/pageable buffers, selected host allocator path and host statistics/allocation timings.", - "produced_artifact": "An ownership classification, source evidence IDs, one eligible allocator proposal, benefit/resource cost and rollback criteria for a later fresh-process comparison.", - "refusal_point": "Keep baseline settings for live-buffer growth, pageable copies, stable reuse, unverified versions or unsupported paths; stop tuning on event/runtime errors and escalate external ownership or gate conflicts." - }, - "recommendations": [ - "Do not mutate an H2D source or read a D2H destination before completion; non_blocking=True alone proves neither overlap nor external-buffer lifetime tracking.", - "Interpret host allocated/active bytes using the selected host path, and test round/cache controls only in supported default-path fresh processes; a per-buffer cache threshold is not a total-cache cap.", - "Keep host and device allocator options separate and use an explicit quiescent boundary for host cache release; cache clearing is not a copy-completion barrier." - ] - }, - "applicability": { - "scope": "Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy ownership/readiness, reuse and path-specific cache controls.", - "input_contract": "Installed versions/settings, host pressure and allocation/reuse distribution, stream timeline, owned versus registered/pageable buffers, selected host allocator path and host statistics/allocation timings.", - "produced_artifact": "An ownership classification, source evidence IDs, one eligible allocator proposal, benefit/resource cost and rollback criteria for a later fresh-process comparison.", - "refusal_point": "Keep baseline settings for live-buffer growth, pageable copies, stable reuse, unverified versions or unsupported paths; stop tuning on event/runtime errors and escalate external ownership or gate conflicts." - }, - "recommendations": [ - "Do not mutate an H2D source or read a D2H destination before completion; non_blocking=True alone proves neither overlap nor external-buffer lifetime tracking.", - "Interpret host allocated/active bytes using the selected host path, and test round/cache controls only in supported default-path fresh processes; a per-buffer cache threshold is not a total-cache cap.", - "Keep host and device allocator options separate and use an explicit quiescent boundary for host cache release; cache clearing is not a copy-completion barrier." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/vllm-ascend.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/vllm-ascend.json deleted file mode 100644 index 5e5d0e944..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/ascend/skills/vllm-ascend.json +++ /dev/null @@ -1,236 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.ascend.skills.vllm_ascend", - "title": "vllm-ascend", - "repo": "inferrsi", - "tier": "contribution", - "backend": "ascend", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Analyze vLLM Ascend v0.27.1rc1 performance and commit evolution for DeepSeek-V4, Kimi K3, MiniMax-M3 and GLM5+, fused operators, DSA-CP, FlashComm1, Fused-MC2 and DSpark, with effective dispatch, rollback and test-oracle guidance.", - "tags": [ - "vllm-ascend", - "v0.27.1rc1", - "MiniMax-M3", - "FP8", - "KV cache", - "DeepSeek-V4-Flash", - "DSpark", - "PD", - "plugin registration", - "DeepSeek-V4", - "Kimi K3", - "MiniMax M3", - "GLM5", - "GLM5.1", - "GLM5.2", - "GLM5.3", - "DSA-CP", - "FlashComm1", - "Fused-MC2", - "fusion", - "performance", - "commit-history" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/vllm-ascend/SKILL.md" - }, - "checkout_ref": "checkout:vllm-ascend", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md": "0e8611f404a947613669ae024ac853eeadc02619e3a218c65b2583a3476a2452", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/metadata.json": "356a876d601238270474816b2a6effe57a537846e5e3d5a576151d433a568fe6", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/communication-config.md": "66daf21685237429e45a64a9d100a10d1f16872a4785037013f0fa798069a319", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/deepseek-v4-evolution.md": "e970d58612aad3cc8244e0b78ae0ce24967f3931659796d2cc519e052f34c113", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/dspark-pd.md": "2f5112acfff4f114998c2737ef3ca4cb81a16c754541421f95fc08e30e13860b", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/fused-operators.md": "97bdae687fb0159612e61744696a256a99bab466726984bec672c1aad257a3ff", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/history-reviewed.json": "c75c3997ba8e5cd569d9252abd58d5f8ecc406696ae9fafc34a4da6a7732251f", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/history-scope.md": "7c57b79dcd2e920e241d50cfa2a40d31fbad62768c7b55254017e51b1d7b7945", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/minimax-cache.md": "6a6196eb2e69bf70796f8436630a5e524b67068c439d482727193b961cb6228e", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/model-performance.md": "6c9bda4499239fe7a206b578b8190ab36c5454d15b57eb4f591eb2ebe1e037e7", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/plugin-registration.md": "ab1b090f22bd0315d2a2e14324064a1c04653fd4eabeffa60de80b9e606de75c", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/sources.md": "9b588e6c4887ff7b5081a72f6fb63b5f53c2f716571b5855acbffc85dbbdc2ab", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/scripts/screen_history.py": "922aa44cc796503628edaff3779d69e5560cf73fa8175859e2fbd701e9ad69e6" - }, - "skill_metadata": { - "name": "vllm-ascend", - "description": "Analyze vLLM Ascend v0.27.1rc1 performance and commit evolution for DeepSeek-V4, Kimi K3, MiniMax-M3 and GLM5+, fused operators, DSA-CP, FlashComm1, Fused-MC2 and DSpark, with effective dispatch, rollback and test-oracle guidance.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "vllm-ascend", - "v0.27.1rc1", - "MiniMax-M3", - "FP8", - "KV cache", - "DeepSeek-V4-Flash", - "DSpark", - "PD", - "plugin registration", - "DeepSeek-V4", - "Kimi K3", - "MiniMax M3", - "GLM5", - "GLM5.1", - "GLM5.2", - "GLM5.3", - "DSA-CP", - "FlashComm1", - "Fused-MC2", - "fusion", - "performance", - "commit-history" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/vllm-ascend", - "commit": "3b31886237ed65c435e0965b10e8002ff5769766", - "license": "Apache-2.0", - "paths": [ - "LICENSE", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/user_guide/release_notes.md", - "setup.py", - "tests/0.27.1rc_vllm-ascend_overall_conclusion_release_notes_EN.md", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_sfa_cp_a2a.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_chunk_kda_aclnn.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_sparse_attention_overlap.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_k3_situ_fusion_cases.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_copy_and_expand_dflash_dspark.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_kimi_k3_fusions.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_tp_rmsnorm_rope.py", - "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", - "tests/e2e/pull_request/eight_card/test_glm5_2.py", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", - "tests/e2e/pull_request/one_card/compile/test_norm_quant_fusion.py", - "tests/e2e/pull_request/one_card/test_minimax_m3_sparse_attn.py", - "tests/e2e/pull_request/sixteen_card/test_kimi_k3.py", - "tests/e2e/pull_request/utils.py", - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/attention/test_dsv4_block_geometry.py", - "tests/ut/kv_transfer/test_layerwise_multi_cache_spec.py", - "tests/ut/models/minimax_m3/test_minimax_m3_eplb.py", - "tests/ut/models/minimax_m3/test_msa_m3.py", - "tests/ut/models/test_kimi_k3_adapter.py", - "tests/ut/ops/test_kimi_kda.py", - "tests/ut/patch/platform/test_patch_parallel_config.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/quantization/methods/test_fp8_block.py", - "tests/ut/spec_decode/test_dspark_speculator.py", - "tests/ut/spec_decode/test_llm_base_proposer.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_ascend_forward_context.py", - "tests/ut/test_platform.py", - "tests/ut/worker/test_device_metadata.py", - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/__init__.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_attn_kv_plan.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/sparse_flash_mla.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/muls_add_pass.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/common/ops/sequence_parallel.py", - "vllm_ascend/models/deepseek_v4/compressor.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/models/deepseek_v4/model.py", - "vllm_ascend/models/minimax_m3/minimax_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/ops/kimi_kda.py", - "vllm_ascend/ops/rope_dsv4.py", - "vllm_ascend/ops/triton/docs/sfa_cp_a2a_fused.md", - "vllm_ascend/ops/triton/dsa_cp.py", - "vllm_ascend/patch/platform/patch_parallel_config.py", - "vllm_ascend/patch/worker/patch_deepseek_v2.py", - "vllm_ascend/patch/worker/patch_minimax_m2.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/configs/modelslim_config.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/device_metadata.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py" - ], - "readset": "references/sources.md" - } - ], - "provenance_readsets": [ - "references/sources.md", - "references/history-scope.md", - "references/history-reviewed.json" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Release-pinned performance diagnosis and historical optimization analysis for DeepSeek-V4, Kimi K3, MiniMax-M3, GLM5+, fusion and communication/speculation paths.", - "prerequisites": "Pinned release evidence or a matching installed stack and role-specific logs.", - "input_contract": "Source and checkpoint revisions, quantization metadata, device and operators, V1/V2 runner, P/D role, TP/DP/PP/PCP/DCP, requested versus derived config, dominant phase and numerical/performance traces.", - "produced_artifact": "First failing boundary with effective values, source locator and smallest discriminating check.", - "refusal_point": "Report an evidence gap when package, role, cache dtype or release identity cannot be established." - }, - "recommendations": [ - "Check the registered general hook in the failing process.", - "Resolve the per-layer cache skip set before interpreting FP8 storage.", - "Compare both DSpark role budgets and connector layouts before changing capture tiers.", - "Check optimization reversals and exact current dispatch before reusing an old performance change.", - "Use commit history for provenance and numerical or serving oracles for validation." - ], - "limitations": [ - "Source-derived guidance for the selected release paths only.", - "Native NPU execution, performance and behavioral skill evaluation not run." - ] - }, - "applicability": { - "scope": "Release-pinned performance diagnosis and historical optimization analysis for DeepSeek-V4, Kimi K3, MiniMax-M3, GLM5+, fusion and communication/speculation paths.", - "prerequisites": "Pinned release evidence or a matching installed stack and role-specific logs.", - "input_contract": "Source and checkpoint revisions, quantization metadata, device and operators, V1/V2 runner, P/D role, TP/DP/PP/PCP/DCP, requested versus derived config, dominant phase and numerical/performance traces.", - "produced_artifact": "First failing boundary with effective values, source locator and smallest discriminating check.", - "refusal_point": "Report an evidence gap when package, role, cache dtype or release identity cannot be established." - }, - "recommendations": [ - "Check the registered general hook in the failing process.", - "Resolve the per-layer cache skip set before interpreting FP8 storage.", - "Compare both DSpark role budgets and connector layouts before changing capture tiers.", - "Check optimization reversals and exact current dispatch before reusing an old performance change.", - "Use commit history for provenance and numerical or serving oracles for validation." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived guidance for the selected release paths only.", - "Native NPU execution, performance and behavioral skill evaluation not run." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/assess-serving-preemption.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/assess-serving-preemption.json deleted file mode 100644 index fc49e11b6..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/assess-serving-preemption.json +++ /dev/null @@ -1,101 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.assess_serving_preemption", - "title": "assess-serving-preemption", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Assess whether LLM serving queue delays justify token-boundary priority preemption, accounting for suspended KV, host swapping or recomputation, and long-request service guarantees. Use for admission-versus-service-order diagnosis, not current scheduler implementation or tenant-fairness certification.", - "tags": [], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/assess-serving-preemption/SKILL.md" - }, - "checkout_ref": "checkout:assess-serving-preemption", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/SKILL.md": "0a5f48d9352c5bf460775ef47f9136829978811f4a16e84983e299782cae305d", - "src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/metadata.json": "3e2cea37f5669e7c60f7c01c390a48ea960b15cbd6de8f5340cf8fe2afb9e1a1", - "src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/references/source-notes.md": "f29fcecfba0490008a354c11b42403983fe165e56ff907ad61cc6040d2b6407f" - }, - "skill_metadata": { - "name": "assess-serving-preemption", - "description": "Assess whether LLM serving queue delays justify token-boundary priority preemption, accounting for suspended KV, host swapping or recomputation, and long-request service guarantees. Use for admission-versus-service-order diagnosis, not current scheduler implementation or tenant-fairness certification.", - "status": "candidate", - "empirical_validation": "not_run", - "source_pins": "orca-osdi22-and-fastserve-2305.05920v3", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Diagnosing whether LLM serving queue delay justifies token-boundary priority preemption, including suspended KV, host swap or recomputation and long-request service guarantees; admission-versus-service-order diagnosis, not current scheduler implementation or tenant-fairness certification", - "prerequisites": "The packaged source notes are read before quoting a metric or comparing a baseline; installing, benchmarking, changing scheduling or admission, migrating state, editing a repository or running a reproduction needs a separately authorized task", - "input_contract": "The late boundary (first token, token gap, completion or a stated cohort's service) with its exact percentile or attainment target and denominator; model and KV format, engine revision, safe interruption boundary, request batch limit, input/output/context distributions, arrival bursts, device live/reserved/allocated bytes, active and suspended requests, host budget, transfer path, recovery time and queue/phase/service timings; per-tenant or priority-class obligations kept apart from aggregate latency", - "produced_artifact": "A read-only decision naming the observed failure boundary, the smallest candidate that addresses it, unchanged assumptions, missing evidence, resource cost, affected cohorts, counter-signal and next authorized gate, plus a proposed verification plan; not a validated configuration", - "failure_modes": [ - "An ordinary iteration boundary is taken to permit interrupting a prefill halfway through", - "Unknown output length is treated as known from the prompt", - "KV is counted for the running batch only, omitting started requests that are suspended", - "Reservations are removed because some reserved capacity is not live", - "Recompute-based eviction is proposed without a progress invariant, so two jobs can repeatedly displace each other", - "Host-device copies or same-host IPC are charged to Network instead of their local resource" - ], - "refusal_point": "Withholds a preemption recommendation when the apparent gain can be explained by different kernels, unmatched parallelism or an incomparable latency metric; unknown safe boundaries, unsupported state recovery, uncalibrated timing or unclear service obligations are blockers" - }, - "recommendations": [ - "Name the late boundary and its exact target and denominator before choosing among batching, admission, preemption and state recovery.", - "Send prefill interference to choose-prefill-decode-schedule and physical KV slack to diagnose-kv-block-efficiency when those candidates are available; otherwise report the diagnosed boundary and the missing evidence.", - "In a later authorized test, isolate service ordering first and state recovery second, and evaluate their interaction only after both are understood.", - "Weigh FastServe's same-engine FCFS comparison above its historical-vLLM headline, and do not read its P95 aggregate gain as a starvation or tenant-fairness guarantee." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Diagnosing whether LLM serving queue delay justifies token-boundary priority preemption, including suspended KV, host swap or recomputation and long-request service guarantees; admission-versus-service-order diagnosis, not current scheduler implementation or tenant-fairness certification", - "prerequisites": "The packaged source notes are read before quoting a metric or comparing a baseline; installing, benchmarking, changing scheduling or admission, migrating state, editing a repository or running a reproduction needs a separately authorized task", - "input_contract": "The late boundary (first token, token gap, completion or a stated cohort's service) with its exact percentile or attainment target and denominator; model and KV format, engine revision, safe interruption boundary, request batch limit, input/output/context distributions, arrival bursts, device live/reserved/allocated bytes, active and suspended requests, host budget, transfer path, recovery time and queue/phase/service timings; per-tenant or priority-class obligations kept apart from aggregate latency", - "produced_artifact": "A read-only decision naming the observed failure boundary, the smallest candidate that addresses it, unchanged assumptions, missing evidence, resource cost, affected cohorts, counter-signal and next authorized gate, plus a proposed verification plan; not a validated configuration", - "failure_modes": [ - "An ordinary iteration boundary is taken to permit interrupting a prefill halfway through", - "Unknown output length is treated as known from the prompt", - "KV is counted for the running batch only, omitting started requests that are suspended", - "Reservations are removed because some reserved capacity is not live", - "Recompute-based eviction is proposed without a progress invariant, so two jobs can repeatedly displace each other", - "Host-device copies or same-host IPC are charged to Network instead of their local resource" - ], - "refusal_point": "Withholds a preemption recommendation when the apparent gain can be explained by different kernels, unmatched parallelism or an incomparable latency metric; unknown safe boundaries, unsupported state recovery, uncalibrated timing or unclear service obligations are blockers" - }, - "recommendations": [ - "Name the late boundary and its exact target and denominator before choosing among batching, admission, preemption and state recovery.", - "Send prefill interference to choose-prefill-decode-schedule and physical KV slack to diagnose-kv-block-efficiency when those candidates are available; otherwise report the diagnosed boundary and the missing evidence.", - "In a later authorized test, isolate service ordering first and state recovery second, and evaluate their interaction only after both are understood.", - "Weigh FastServe's same-engine FCFS comparison above its historical-vLLM headline, and do not read its P95 aggregate gain as a starvation or tenant-fairness guarantee." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/assess-split-backward-schedule.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/assess-split-backward-schedule.json deleted file mode 100644 index 7a1572ae0..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/assess-split-backward-schedule.json +++ /dev/null @@ -1,98 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.assess_split_backward_schedule", - "title": "assess-split-backward-schedule", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Evaluate memory-capped split-backward and V-shaped pipeline scheduling proposals, preserving synchronous update, global-batch and optimizer-validation semantics.", - "tags": [], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/assess-split-backward-schedule/SKILL.md" - }, - "checkout_ref": "checkout:assess-split-backward-schedule", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/SKILL.md": "1e64f28ea71672fb53c56f6108b87864738c6d03d6dcd32eedaef9e0625e96e3", - "src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/metadata.json": "a089dd2c2e5a4b41e970e426c63b5b8f65acc77d2e5a40a44b38543b5d730a00", - "src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/references/source-notes.md": "61b1609f71c37b61d136b8c4d39f793de87cffb1cf31c8ec737a1c3fa6fb1b4e" - }, - "skill_metadata": { - "name": "assess-split-backward-schedule", - "description": "Evaluate memory-capped split-backward and V-shaped pipeline scheduling proposals, preserving synchronous update, global-batch and optimizer-validation semantics.", - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Assessing memory-capped split-backward and V-shaped pipeline schedule proposals when a pipeline trace suggests input-gradient propagation waits behind deferrable weight-gradient work or an optimizer barrier, while preserving synchronous update, global-batch and optimizer-validation semantics", - "prerequisites": "The packaged evidence notes are read before citing the paper; installing or running frameworks, benchmarking schedules, changing training semantics, configuring hardware or modifying repositories needs separate authorization", - "input_contract": "Stage and microbatch counts, microbatch and global batch, precision, optimizer and clipping/NaN behavior; F, input-gradient B, weight-gradient W and transfer timings; per-pass retained and released bytes; total device peak and margin; collective ordering; kernel-splitting support; calculated bubble rate kept apart from measured idle time and useful samples/s", - "produced_artifact": "A candidate schedule and fallback with per-pass order rationale, peak-memory and transfer costs, unchanged semantic requirements, counter-signals and a future verification proposal; a reviewable plan, not an enabled schedule", - "failure_modes": [ - "Completion of B is equated with releasing all activations, so W inputs retained by deferral go uncounted", - "The equal-time zero-bubble construction replaces observed F/B/W and transfer times, or kernel-splitting and synchronization overhead is left out", - "The schedule is chosen by minimum modeled bubble rate instead of whole-step throughput under the memory and quality constraint", - "pMB activation capacity is read as total memory, hiding HBM, workspace, buffers or per-rank imbalance", - "Removing the optimizer barrier drops global norm, clipping or NaN decisions, rollback and re-update, or the correctness of work that depends on a tentative update", - "Optimizer post-validation is conflated with accepting stale-gradient delayed updates" - ], - "refusal_point": "Withholds an enablement recommendation under frequent clipping/NaNs, small available memory, strongly unequal passes, unsupported split kernels, hidden collectives or dynamic shapes, and retains the synchronized fallback while optimizer reversibility and rollback coverage are unverified" - }, - "recommendations": [ - "Compare ordinary 1F1B with a memory-capped split schedule first; add V-shaped two-chunk placement only when memory permits, and at matched global batch also try smaller microbatches with more microbatches.", - "Decide optimizer post-validation separately from the schedule change.", - "In a later authorized comparison, hold model, global batch and precision fixed and compare real with calculated idle intervals." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Assessing memory-capped split-backward and V-shaped pipeline schedule proposals when a pipeline trace suggests input-gradient propagation waits behind deferrable weight-gradient work or an optimizer barrier, while preserving synchronous update, global-batch and optimizer-validation semantics", - "prerequisites": "The packaged evidence notes are read before citing the paper; installing or running frameworks, benchmarking schedules, changing training semantics, configuring hardware or modifying repositories needs separate authorization", - "input_contract": "Stage and microbatch counts, microbatch and global batch, precision, optimizer and clipping/NaN behavior; F, input-gradient B, weight-gradient W and transfer timings; per-pass retained and released bytes; total device peak and margin; collective ordering; kernel-splitting support; calculated bubble rate kept apart from measured idle time and useful samples/s", - "produced_artifact": "A candidate schedule and fallback with per-pass order rationale, peak-memory and transfer costs, unchanged semantic requirements, counter-signals and a future verification proposal; a reviewable plan, not an enabled schedule", - "failure_modes": [ - "Completion of B is equated with releasing all activations, so W inputs retained by deferral go uncounted", - "The equal-time zero-bubble construction replaces observed F/B/W and transfer times, or kernel-splitting and synchronization overhead is left out", - "The schedule is chosen by minimum modeled bubble rate instead of whole-step throughput under the memory and quality constraint", - "pMB activation capacity is read as total memory, hiding HBM, workspace, buffers or per-rank imbalance", - "Removing the optimizer barrier drops global norm, clipping or NaN decisions, rollback and re-update, or the correctness of work that depends on a tentative update", - "Optimizer post-validation is conflated with accepting stale-gradient delayed updates" - ], - "refusal_point": "Withholds an enablement recommendation under frequent clipping/NaNs, small available memory, strongly unequal passes, unsupported split kernels, hidden collectives or dynamic shapes, and retains the synchronized fallback while optimizer reversibility and rollback coverage are unverified" - }, - "recommendations": [ - "Compare ordinary 1F1B with a memory-capped split schedule first; add V-shaped two-chunk placement only when memory permits, and at matched global batch also try smaller microbatches with more microbatches.", - "Decide optimizer post-validation separately from the schedule change.", - "In a later authorized comparison, hold model, global batch and precision fixed and compare real with calculated idle intervals." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-fixed-workload-claim.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-fixed-workload-claim.json deleted file mode 100644 index 840835fda..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-fixed-workload-claim.json +++ /dev/null @@ -1,96 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.audit_fixed_workload_claim", - "title": "audit-fixed-workload-claim", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit a performance claim from supplied fixed-workload reports and an explicit measurement contract. Check aggregation, retained observations, comparability and quality evidence without launching new benchmarks.", - "tags": [ - "performance", - "audit", - "measurement" - ], - "origin": [ - "internal" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/audit-fixed-workload-claim/SKILL.md" - }, - "checkout_ref": "checkout:audit-fixed-workload-claim", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/SKILL.md": "2be89b1340a3deb978c8e1792d18aeecbe8f5de81d4daf542389f53826b8ebee", - "src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/metadata.json": "c3c6ee33ce318c6ec052b848f9be9797b2b1febd8ea563face20db018613a110" - }, - "skill_metadata": { - "name": "audit-fixed-workload-claim", - "description": "Audit a performance claim from supplied fixed-workload reports and an explicit measurement contract. Check aggregation, retained observations, comparability and quality evidence without launching new benchmarks.", - "status": "candidate", - "origin": [ - "internal" - ], - "tags": [ - "performance", - "audit", - "measurement" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Auditing a performance claim against supplied fixed-workload reports and an explicit measurement contract: aggregation, retained observations, comparability and quality evidence", - "input_contract": "Baseline and candidate fixed-workload reports with request observations grouped by trial and workload; the measurement contract stating metric, unit, timing boundary, improvement direction, aggregation rule and order, fixed exclusions, threshold and comparison operator; the declared quality and completion evidence; the requested output schema", - "produced_artifact": "An audit conclusion showing baseline and candidate aggregates, the calculation, the threshold comparison and the evidence or missing information that controls the verdict; it interprets supplied evidence and does not itself establish a measured improvement", - "failure_modes": [ - "A pooled request median silently replaces a median of per-trial means, or the reverse, reweighting trials whose request counts differ", - "Late requests, failures or outliers are dropped, or requests move between trials, outside the exclusions the contract fixes", - "Values are rounded before the threshold comparison", - "A missing rule that permits different verdicts is resolved toward the favorable one", - "A speed improvement is read as task success, or a quality pass as equivalent measured work", - "Different workloads or incomplete observations are repaired by a favorable aggregate" - ], - "refusal_point": "Does not replace missing evidence with a pass, does not support a percentage claim on an undefined denominator, and does not launch a new benchmark or alter the contract to resolve an unfavorable result; a failed threshold is reported apart from an unverified comparison or missing quality evidence" - }, - "recommendations": [ - "Recover the contract's metric, direction, aggregation order and comparison operator before reading the report's own verdict.", - "Treat a named minimum bound as >= and a maximum bound as <= unless the contract requires a strict comparison.", - "Keep separate regimes and metrics separate unless the contract defines their aggregation." - ], - "limitations": [ - "Guidance for supplied reports; no local efficacy validation or measured performance gain is established." - ], - "searchspace_not_applicable": "SKILL.md §§ Preserve the observation hierarchy, Check what was actually compared, and Return an auditable conclusion audit supplied measurements and verdicts; device identity is comparison metadata, not a hardware-resource mechanism." - }, - "applicability": { - "scope": "Auditing a performance claim against supplied fixed-workload reports and an explicit measurement contract: aggregation, retained observations, comparability and quality evidence", - "input_contract": "Baseline and candidate fixed-workload reports with request observations grouped by trial and workload; the measurement contract stating metric, unit, timing boundary, improvement direction, aggregation rule and order, fixed exclusions, threshold and comparison operator; the declared quality and completion evidence; the requested output schema", - "produced_artifact": "An audit conclusion showing baseline and candidate aggregates, the calculation, the threshold comparison and the evidence or missing information that controls the verdict; it interprets supplied evidence and does not itself establish a measured improvement", - "failure_modes": [ - "A pooled request median silently replaces a median of per-trial means, or the reverse, reweighting trials whose request counts differ", - "Late requests, failures or outliers are dropped, or requests move between trials, outside the exclusions the contract fixes", - "Values are rounded before the threshold comparison", - "A missing rule that permits different verdicts is resolved toward the favorable one", - "A speed improvement is read as task success, or a quality pass as equivalent measured work", - "Different workloads or incomplete observations are repaired by a favorable aggregate" - ], - "refusal_point": "Does not replace missing evidence with a pass, does not support a percentage claim on an undefined denominator, and does not launch a new benchmark or alter the contract to resolve an unfavorable result; a failed threshold is reported apart from an unverified comparison or missing quality evidence" - }, - "recommendations": [ - "Recover the contract's metric, direction, aggregation order and comparison operator before reading the report's own verdict.", - "Treat a named minimum bound as >= and a maximum bound as <= unless the contract requires a strict comparison.", - "Keep separate regimes and metrics separate unless the contract defines their aggregation." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Guidance for supplied reports; no local efficacy validation or measured performance gain is established." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-h3-metal-fasth3-semantics.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-h3-metal-fasth3-semantics.json deleted file mode 100644 index 6ad3cf672..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-h3-metal-fasth3-semantics.json +++ /dev/null @@ -1,184 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.audit_h3_metal_fasth3_semantics", - "title": "audit-h3-metal-fasth3-semantics", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit h3-metal FastH3 effective configuration, checkpoint tensor and schedule semantics, partial VSA tiles, sparse eligibility and dense-plus-coarse fallback without treating source or header checks as native parity.", - "tags": [ - "h3-metal", - "FastH3", - "MiniMax-H3", - "apple-silicon", - "metal", - "qkv-layout", - "swiglu", - "bf16", - "vsa", - "sparse-attention", - "fallback", - "effective-configuration" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md" - }, - "checkout_ref": "checkout:audit-h3-metal-fasth3-semantics", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/LICENSE": "663e51edcd9a3a8f858018ce6924ad2895a24c269c26c39ad75325bcd0f7e5bc", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md": "e5205de6eba52b3d1693666af755640697717e52ee7834e1ae30e4071b208d54", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/metadata.json": "f1681981db4f9077ee27ff71571dceecd69daf25136bd374b7d9ad3b59762a6a", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/THIRD_PARTY_NOTICES.md": "a0d77db7d2aca0eda811ab3c386cc32b3143f2c2609a042cf4642954f0f6fb09", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/preflight.md": "5cf745005914c74b18a09f127d1d02ce0cb64f6cd7899501ce3cb40938c42a4c", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/repo-provenance.md": "6a4c14207223dcc2584f7133bd14ff4590c2258fc2bd44f3721b6bf63e3e35cf", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/semantic-contracts.md": "3f518160e8a227272e698486e6c67acd364af0d74755103ffc868b2d66052c20", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/source-identity.json": "b0e63197725c6a1baa4888ca3c48ccd4926b6e6f0f8bf781ae76ac97ffd1ceb1", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/source-notes.md": "f9430103644eacd9c55ace6f84d8587fc0bc063bfcd9cb60e6045b139a9fee05", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/troubleshooting.md": "ed552d87f9fb9fb2c56de64df61f3fc6e2333cb049116cd0736f784c8a327612", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/scripts/audit_fasth3.py": "05bfb9348cf88a76f4fba8bc8a600fe54246d83a98c1685bc4847ace28e2fbac" - }, - "skill_metadata": { - "name": "audit-h3-metal-fasth3-semantics", - "description": "Audit h3-metal FastH3 effective configuration, checkpoint tensor and schedule semantics, partial VSA tiles, sparse eligibility and dense-plus-coarse fallback without treating source or header checks as native parity.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NO_LICENSE", - "tags": [ - "h3-metal", - "FastH3", - "MiniMax-H3", - "apple-silicon", - "metal", - "qkv-layout", - "swiglu", - "bf16", - "vsa", - "sparse-attention", - "fallback", - "effective-configuration" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/zuiho-kai/h3-metal", - "commit": "4ba47a3eabdbebf801ac6664b31f8ec168bc57a8", - "paths": [ - "LICENSE", - "THIRD_PARTY_NOTICES.md", - "Makefile", - "README.md", - "docs/USAGE.md", - "h3.h", - "h3.c", - "main.c", - "h3_host.h", - "h3_host.c", - "h3_dit.c", - "h3_dit_schedule.c", - "h3_gpu.m", - "h3_shaders.metal", - "tests/test_h3.c", - "tests/test_vsa_coarse.c", - "tests/test_real_dit_schedule.c", - "tests/bench_dit.c", - "tools/fasth3_keymap.py", - "tools/fasth3_keymap.md", - "bench/run_fox2.sh" - ], - "readset": "references/source-notes.md", - "identity": "references/source-identity.json", - "license": "MIT; selected shader-derived components separately BSD-3-Clause", - "third_party_notices": "references/THIRD_PARTY_NOTICES.md" - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill", - "cli/packages/coding-agent/src/disco/skills/extend-repo-skill" - ], - "scope": "Portable source/CPU-only preparation, extraction, safe-script disposition, difficult cases and verification adaptation to InferRSI ownership; no DisCo CLI/model run or managed import." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "capability-specific", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Bounded fork-specific source audit, bundled selected-declaration helper and assertion-backed cases. Native Apple/Metal/model evidence remains blocked; no numerical/performance/skill-efficacy claim." - }, - "applicability": { - "scope": "h3-metal FastH3 requested/effective configuration, checkpoint transformation and schedule/RoPE semantics, VSA prefix/partial tiles and dispatch/fallback audit. Common catalog placement is discovery, not CPU inference or Apple qualification.", - "prerequisites": "Access to the fork-specific source and semantic contracts; native checks additionally need their matching Metal/model resources and separate execution authority.", - "input_contract": "Source revision/patches, requested parameters and explicitly captured environment controls, checkpoint path/key-family/config identity, requested/aligned frames, latent dimensions, packed prefix segments and sparse-pipeline availability or unknown.", - "produced_artifact": "A ledger of requested and normalized parameters, ambient overrides, source invariants, counterexamples, observed/unknown facts and the smallest discriminating next check.", - "failure_modes": "Ignoring ambient overrides, mixing checkpoint identity switches, using global rather than per-head Q/K/V concatenation, pooling padded boundary rows, or double-adding coarse attention can invalidate the semantic audit.", - "refusal_point": "Stop on conflicting configuration, mixed checkpoint identity, missing resources or the authorized boundary; name the missing native test/fixture and leave it blocked/not_run." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: native Apple/Metal, actual FastH3 checkpoint/schedule/sparse parity, audio/video quality, target memory fit, performance and service evidence not_run.", - "Offline helper audits caller declarations and selected arithmetic only, not engine parameter validity, payload/dtype/conversion equivalence, attention numerics, actual environment or dispatch.", - "Bounded file/range readset only; full kernels, model/config/weights, encoders/VAEs, device/runtime behavior and external FastVideo conformance are not established.", - "Native coarse VSA and original-H3 schedule test definitions do not cover sparse fine or FastH3 BF16 schedule execution; no native tests were run.", - "Key-map and benchmark source scripts are excluded from execution; checked-in reports, source comments and existing measurement documents are not new local evidence.", - "NO_LICENSE is the actual unauthenticated machine-resolver fallback, not absence of licensing. Inspected MIT source and BSD-3-Clause notices are retained; model rights remain separate/unverified." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/source-identity.json", - "references/repo-provenance.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Record requested parameters, normalized values and later environment overrides separately, including requested versus aligned frame counts.", - "Audit checkpoint transformations and schedule consumers beyond tensor names/shapes; tiny permutation fixtures can falsify mappings but do not establish model parity.", - "Resolve sparse eligibility and the dense-fine-plus-gated-coarse fallback explicitly; preserve unknown pipeline availability." - ] - }, - "applicability": { - "scope": "h3-metal FastH3 requested/effective configuration, checkpoint transformation and schedule/RoPE semantics, VSA prefix/partial tiles and dispatch/fallback audit. Common catalog placement is discovery, not CPU inference or Apple qualification.", - "prerequisites": "Access to the fork-specific source and semantic contracts; native checks additionally need their matching Metal/model resources and separate execution authority.", - "input_contract": "Source revision/patches, requested parameters and explicitly captured environment controls, checkpoint path/key-family/config identity, requested/aligned frames, latent dimensions, packed prefix segments and sparse-pipeline availability or unknown.", - "produced_artifact": "A ledger of requested and normalized parameters, ambient overrides, source invariants, counterexamples, observed/unknown facts and the smallest discriminating next check.", - "failure_modes": "Ignoring ambient overrides, mixing checkpoint identity switches, using global rather than per-head Q/K/V concatenation, pooling padded boundary rows, or double-adding coarse attention can invalidate the semantic audit.", - "refusal_point": "Stop on conflicting configuration, mixed checkpoint identity, missing resources or the authorized boundary; name the missing native test/fixture and leave it blocked/not_run." - }, - "recommendations": [ - "Record requested parameters, normalized values and later environment overrides separately, including requested versus aligned frame counts.", - "Audit checkpoint transformations and schedule consumers beyond tensor names/shapes; tiny permutation fixtures can falsify mappings but do not establish model parity.", - "Resolve sparse eligibility and the dense-fine-plus-gated-coarse fallback explicitly; preserve unknown pipeline availability." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "BLOCKED_REQUIRED_BACKEND: native Apple/Metal, actual FastH3 checkpoint/schedule/sparse parity, audio/video quality, target memory fit, performance and service evidence not_run.", - "Offline helper audits caller declarations and selected arithmetic only, not engine parameter validity, payload/dtype/conversion equivalence, attention numerics, actual environment or dispatch.", - "Bounded file/range readset only; full kernels, model/config/weights, encoders/VAEs, device/runtime behavior and external FastVideo conformance are not established.", - "Native coarse VSA and original-H3 schedule test definitions do not cover sparse fine or FastH3 BF16 schedule execution; no native tests were run.", - "Key-map and benchmark source scripts are excluded from execution; checked-in reports, source comments and existing measurement documents are not new local evidence.", - "NO_LICENSE is the actual unauthenticated machine-resolver fallback, not absence of licensing. Inspected MIT source and BSD-3-Clause notices are retained; model rights remain separate/unverified." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-h3-runtime-residency.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-h3-runtime-residency.json deleted file mode 100644 index cfb24be71..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-h3-runtime-residency.json +++ /dev/null @@ -1,211 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.audit_h3_runtime_residency", - "title": "audit-h3-runtime-residency", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit MiniMax-H3 generation in h3.c on Apple Silicon by tracing conditioning, AdaLN precompute, DiT residency, Metal precision paths, SSD streaming and audio/video decoding.", - "tags": [ - "h3.c", - "MiniMax-H3", - "apple-silicon", - "metal", - "diffusion", - "video", - "audio", - "multimodal-conditioning", - "adaln", - "model-residency", - "ssd-streaming", - "int8", - "bf16" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md" - }, - "checkout_ref": "checkout:audit-h3-runtime-residency", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/LICENSE": "663e51edcd9a3a8f858018ce6924ad2895a24c269c26c39ad75325bcd0f7e5bc", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md": "fdb4d9483c6efa836f9e59617da605fa9c2186e11c2041b05d5c81850ba0475d", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/metadata.json": "952bd2947b68e56d249f10798e028d191a8ffbdb2f123775be66946d3e2ef77e", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/THIRD_PARTY_NOTICES.md": "a0d77db7d2aca0eda811ab3c386cc32b3143f2c2609a042cf4642954f0f6fb09", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md": "57ce01754ba96c378f13a91305ef3d418b926f99b2753f0af99315572bce063a", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-weight-inventory.json": "fbaacaeec340d3ce893556ee864405a0746af4d13530a38cb3d5aa9d60929271", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/phase-residency.md": "2836a550eb2be7f5a862f51ec7b135ebd9715bf3c4faa84a7ecbc9d9c11f5d8d", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md": "7e166c89193f4c7aed852ec95346a2de2b5008dac002e2e1862be1b3e57f6093", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/source-identity.json": "730f8dd23648a3f8f47b9726cc613d893a79d1817d515eec609a9b06d8a9ff90", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/source-notes.md": "7304bf9369efc157dc7aa40686dd828cc14db45c49442e908b3d3cc4b329eb86", - "src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/validation.md": "0754873a758e048480938314894e25b504ae0c269549312e7f8e6020d83fe294" - }, - "skill_metadata": { - "name": "audit-h3-runtime-residency", - "description": "Audit MiniMax-H3 generation in h3.c on Apple Silicon by tracing conditioning, AdaLN precompute, DiT residency, Metal precision paths, SSD streaming and audio/video decoding.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "MIT", - "tags": [ - "h3.c", - "MiniMax-H3", - "apple-silicon", - "metal", - "diffusion", - "video", - "audio", - "multimodal-conditioning", - "adaln", - "model-residency", - "ssd-streaming", - "int8", - "bf16" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/antirez/h3.c", - "commit": "8974cc055ea9c02fcd14cc27dfda3e1027c05153", - "paths": [ - "h3.h", - "h3_internal.h", - "h3.c", - "h3_weights.c", - "h3_weights.h", - "h3_safetensors.c", - "h3_host.c", - "h3_text_encoder.c", - "h3_vision_encoder.c", - "h3_multimodal.c", - "h3_video_encoder.c", - "h3_audio_vae.c", - "h3_video_vae.c", - "h3_dit_schedule.c", - "h3_dit_schedule.h", - "h3_dit.c", - "h3_gpu.m", - "h3_shaders.metal", - "main.c", - "h3_cli.c", - "Makefile", - "README.md", - "LICENSE", - "THIRD_PARTY_NOTICES.md", - "tests/test_h3.c", - "tests/test_bf16.c", - "tests/test_real_dit_schedule.c", - "tests/test_real_prompt.c", - "tests/test_real_multimodal_text.c", - "tests/test_semantic_dit.c", - "tests/test_semantic_vae.c", - "tests/test_real_audio_vae.c", - "tests/test_real_video_vae.c", - "tests/bench_dit.c" - ], - "readset": "references/source-notes.md", - "identity": "references/source-identity.json", - "license": "MIT", - "third_party_notices": "references/THIRD_PARTY_NOTICES.md" - }, - { - "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI", - "commit": "f804ec1503e977787c572d5d27d1f3258017b01f", - "paths": [ - "src/inferrsi/data/skill_library/workflows/inference-engine-research/references/minimax-h3-apple.md", - "src/inferrsi/data/skill_library/workflows/inference-engine-research/references/minimax-h3-weight-inventory.json" - ], - "scope": "Prior model/engine comparison and safetensors header observations, reused without reissuing range requests; inventory bytes preserved." - }, - { - "type": "document", - "url": "https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/42ed227ee7df40d41602854ae760620d6eb651fe/FL2VA", - "version": "42ed227ee7df40d41602854ae760620d6eb651fe", - "scope": "29 JSON-header digests and component aggregates retained from prior inspection; payload and complete model license not newly verified.", - "license": "UNKNOWN", - "readset": "references/minimax-h3-weight-inventory.json" - }, - { - "type": "document", - "url": "https://huggingface.co/appautomaton/minimax-h3-base-8bit-mlx/tree/9a08116a349ab469b097775b5a483affcb5ccd34", - "version": "9a08116a349ab469b097775b5a483affcb5ccd34", - "scope": "Four selected export JSON-header digests/packing aggregates retained from prior inspection; payload and model/export license not newly verified.", - "license": "UNKNOWN", - "readset": "references/minimax-h3-weight-inventory.json" - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "capability-specific", - "strategy": "direct", - "status": "portable-candidate", - "native_import": "not-run", - "method": "Manual pinned-source distillation informed by the existing Repo-To-Skill ledger; no AREX/DisCo execution or behavioral efficacy claim." - }, - "applicability": { - "scope": "MiniMax-H3 joint audio/video C/Metal runtime audit: raw checkpoint compatibility, input encoders, schedule precompute, phased/cached residency, numerical controls and SSD streaming on Apple Silicon. Common catalog placement is a discovery classification, not cross-backend qualification.", - "prerequisites": "Access to the pinned source architecture and selected component/format inventory; native checks need matching model, fixture and backend resources within an authorized experiment budget.", - "input_contract": "Runtime revision/patches, macOS/SDK/GPU and memory budget, FL2VA/Ref2VA selection, component formats, prompt and ordered media, requested/aligned frames, canvases, steps/seed, layer/reuse/token/precision policies and preview/interactive-cache state.", - "produced_artifact": "A phase/component/owner ledger with resident weights, runtime/host buffers, overlap, release/reuse boundaries and observations/unknowns, followed by a discriminating check, actual result and remaining gap.", - "failure_modes": "Estimating fit from model-file bytes, confusing runtime int8 with incompatible MLX bundles, ignoring preview/cache overlap, or reducing DiT residency to address a VAE/host-output peak can produce a wrong capacity diagnosis.", - "refusal_point": "Stop at unavailable model/fixture/backend resources or the accepted experiment budget; missing target evidence remains unverified/INCONCLUSIVE." - }, - "limitations": [ - "Native h3.c/Metal execution, target peak memory, audio/video quality, performance, service behavior and skill efficacy were not run.", - "Only the bounded readset was inspected; complete kernel, driver/MPSGraph, tokenizer, FFmpeg, concurrent-caller and failure-path coverage is not established.", - "The retained inventory authenticates only prior JSON header observations, not weight payloads; full official versus selected truncated/packed export inventories are distinct.", - "Upstream 128 GB M5 Max results and tracked DiT bytes do not qualify a 64 GiB target, a different workload or total physical memory.", - "MIT covers the h3.c source; retained BSD-3-Clause notices and independent model/export/tool licenses remain separate. UNKNOWN source-license fields are unresolved." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/source-identity.json", - "references/minimax-h3-weight-inventory.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve the actual conditioning and output components before counting residency; do not add alternative checkpoints or count shared files twice.", - "Trace one-shot, preview and interactive ownership separately, including AdaLN schedule dependencies and prepared-cache reset boundaries.", - "Classify approximation and precision controls before comparison, then measure the relevant phase with tensor, physical/compressed/swap and storage observations." - ] - }, - "applicability": { - "scope": "MiniMax-H3 joint audio/video C/Metal runtime audit: raw checkpoint compatibility, input encoders, schedule precompute, phased/cached residency, numerical controls and SSD streaming on Apple Silicon. Common catalog placement is a discovery classification, not cross-backend qualification.", - "prerequisites": "Access to the pinned source architecture and selected component/format inventory; native checks need matching model, fixture and backend resources within an authorized experiment budget.", - "input_contract": "Runtime revision/patches, macOS/SDK/GPU and memory budget, FL2VA/Ref2VA selection, component formats, prompt and ordered media, requested/aligned frames, canvases, steps/seed, layer/reuse/token/precision policies and preview/interactive-cache state.", - "produced_artifact": "A phase/component/owner ledger with resident weights, runtime/host buffers, overlap, release/reuse boundaries and observations/unknowns, followed by a discriminating check, actual result and remaining gap.", - "failure_modes": "Estimating fit from model-file bytes, confusing runtime int8 with incompatible MLX bundles, ignoring preview/cache overlap, or reducing DiT residency to address a VAE/host-output peak can produce a wrong capacity diagnosis.", - "refusal_point": "Stop at unavailable model/fixture/backend resources or the accepted experiment budget; missing target evidence remains unverified/INCONCLUSIVE." - }, - "recommendations": [ - "Resolve the actual conditioning and output components before counting residency; do not add alternative checkpoints or count shared files twice.", - "Trace one-shot, preview and interactive ownership separately, including AdaLN schedule dependencies and prepared-cache reset boundaries.", - "Classify approximation and precision controls before comparison, then measure the relevant phase with tensor, physical/compressed/swap and storage observations." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Native h3.c/Metal execution, target peak memory, audio/video quality, performance, service behavior and skill efficacy were not run.", - "Only the bounded readset was inspected; complete kernel, driver/MPSGraph, tokenizer, FFmpeg, concurrent-caller and failure-path coverage is not established.", - "The retained inventory authenticates only prior JSON header observations, not weight payloads; full official versus selected truncated/packed export inventories are distinct.", - "Upstream 128 GB M5 Max results and tracked DiT bytes do not qualify a 64 GiB target, a different workload or total physical memory.", - "MIT covers the h3.c source; retained BSD-3-Clause notices and independent model/export/tool licenses remain separate. UNKNOWN source-license fields are unresolved." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-llama-runtime-state.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-llama-runtime-state.json deleted file mode 100644 index 10a469a12..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-llama-runtime-state.json +++ /dev/null @@ -1,130 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.audit_llama_runtime_state", - "title": "audit-llama-runtime-state", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit llama.cpp context memory, partial decode progress, graph reuse and server cancellation when a retry, stale prefix, unexpected rebuild or unreleased slot needs an ownership explanation; requires pinned source or a concrete runtime trace.", - "tags": [ - "llama.cpp", - "context", - "state", - "graph-reuse", - "cancellation", - "correctness" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md" - }, - "checkout_ref": "checkout:audit-llama-runtime-state", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md": "95e81b6bc096a954fa73139ea5bc001d1efa6c99747161da62f012b494e5c013", - "src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/metadata.json": "22f1d01b48ea6e2b2ef2deeb20c1404ce8b586eb18442b5adb1390310b4b7503", - "src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/references/source-notes.md": "3d12a401e9decb55a2f4f3b2bfa0d360ef5110f63a0469dfe285654e78054190" - }, - "skill_metadata": { - "name": "audit-llama-runtime-state", - "description": "Audit llama.cpp context memory, partial decode progress, graph reuse and server cancellation when a retry, stale prefix, unexpected rebuild or unreleased slot needs an ownership explanation; requires pinned source or a concrete runtime trace.", - "tags": [ - "llama.cpp", - "context", - "state", - "graph-reuse", - "cancellation", - "correctness" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ggml-org/llama.cpp", - "commit": "707616feb5f379e99ff7442680a55b4a1704c7c9", - "paths": [ - "LICENSE", - "include/llama.h", - "src/llama-context.cpp", - "src/llama-context.h", - "src/llama-memory.h", - "src/llama-memory-recurrent.cpp", - "src/llama-graph.cpp", - "src/llama-graph.h", - "ggml/src/ggml-backend.cpp", - "tools/server/server-context.cpp", - "tools/server/server-queue.cpp", - "tools/server/server-queue.h", - "tests/llama-context/cases-kv.cpp", - "tests/llama-context/cases-rs.cpp", - "tests/CMakeLists.txt", - "tests/test-llama-archs.cpp", - "tests/test-backend-ops.cpp", - ".github/workflows/build-cann.yml", - "tools/server/tests/unit/test_completion.py" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Pinned llama.cpp context-memory, graph-reuse and server task-lifecycle diagnosis; concrete model/backend behavior requires native validation.", - "prerequisites": "Pinned llama.cpp source or a concrete runtime trace, access to the relevant memory/graph/server ownership evidence and an identified caller recovery policy that remains in force.", - "input_contract": "Source/build and model revisions, memory type, actual backend/device and execution mode, context/batch/ubatch parameters, sequence IDs/positions/output flags and return status; graph cases need dispatch evidence, server cases need correlated HTTP/task/slot/cache logs.", - "produced_artifact": "An owner/event ledger of ubatch progress, retained memory, graph-gate decisions, input readiness and slot release/retention, with the first supported broken obligation or smallest missing observation.", - "failure_modes": "Assuming whole-batch rollback after partial decode failure, treating token count as a complete graph-reuse key, or equating posted cancellation with released slots can hide retained state or misdiagnose correct deferred work.", - "refusal_point": "Stop before reset or repair if revision, memory type, actual backend, sequence ownership or partial progress is unknown; escalate unsupported rollback or unresolved in-flight work." - }, - "limitations": [ - "Source-derived candidate; native runtime and efficacy validation not run.", - "No cross-backend preemption, model-quality or performance claim.", - "KV, recurrent and hybrid state contracts must be checked independently." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Reconcile per-sequence retained positions with completed ubatches before retrying a failed logical batch.", - "Check both graph allow_reuse and input can_reuse gates, then establish copy readiness independently of graph identity.", - "Trace cancellation posting, normal task processing, slot release and prompt retention as separate events." - ] - }, - "applicability": { - "scope": "Pinned llama.cpp context-memory, graph-reuse and server task-lifecycle diagnosis; concrete model/backend behavior requires native validation.", - "prerequisites": "Pinned llama.cpp source or a concrete runtime trace, access to the relevant memory/graph/server ownership evidence and an identified caller recovery policy that remains in force.", - "input_contract": "Source/build and model revisions, memory type, actual backend/device and execution mode, context/batch/ubatch parameters, sequence IDs/positions/output flags and return status; graph cases need dispatch evidence, server cases need correlated HTTP/task/slot/cache logs.", - "produced_artifact": "An owner/event ledger of ubatch progress, retained memory, graph-gate decisions, input readiness and slot release/retention, with the first supported broken obligation or smallest missing observation.", - "failure_modes": "Assuming whole-batch rollback after partial decode failure, treating token count as a complete graph-reuse key, or equating posted cancellation with released slots can hide retained state or misdiagnose correct deferred work.", - "refusal_point": "Stop before reset or repair if revision, memory type, actual backend, sequence ownership or partial progress is unknown; escalate unsupported rollback or unresolved in-flight work." - }, - "recommendations": [ - "Reconcile per-sequence retained positions with completed ubatches before retrying a failed logical batch.", - "Check both graph allow_reuse and input can_reuse gates, then establish copy readiness independently of graph identity.", - "Trace cancellation posting, normal task processing, slot release and prompt retention as separate events." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; native runtime and efficacy validation not run.", - "No cross-backend preemption, model-quality or performance claim.", - "KV, recurrent and hybrid state contracts must be checked independently." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-recurrent-kv-finalization.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-recurrent-kv-finalization.json deleted file mode 100644 index 1aea709a6..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-recurrent-kv-finalization.json +++ /dev/null @@ -1,146 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.audit_recurrent_kv_finalization", - "title": "audit-recurrent-kv-finalization", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit vLLM-RLT synchronous early-exit KV propagation, later depth-specific attention reads, and request storage retirement. Use when shallow token exits leave deeper history or release order in question.", - "tags": [ - "vllm-rlt", - "recurrent", - "kv-cache", - "early-exit", - "last-exited", - "shared", - "validity", - "completion", - "lifetime" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md" - }, - "checkout_ref": "checkout:audit-recurrent-kv-finalization", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md": "a468d75f25e77b96e6fb5376692f7a89c0464cc00d4df035f1dc6cc923f713cc", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/metadata.json": "dfd605f8f31740565063fad0785dea3e7751b8deea4d43e1a66ca30d92ae901f", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/NOTICE.md": "cca794656897ecdc37f983f7f23c4cd12306dbe3c9cd2f54fece6340753ae4cf", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json": "5e52db0a0bda4673b9aac46a3fc035b824780def72921fee6796edb2ed9fca65", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md": "0956f59cd9a1fac27da53228dd60c51a07b87639a43c09bfab296c4aeeceaab9" - }, - "skill_metadata": { - "name": "audit-recurrent-kv-finalization", - "description": "Audit vLLM-RLT synchronous early-exit KV propagation, later depth-specific attention reads, and request storage retirement. Use when shallow token exits leave deeper history or release order in question.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "Apache-2.0", - "tags": [ - "vllm-rlt", - "recurrent", - "kv-cache", - "early-exit", - "last-exited", - "shared", - "validity", - "completion", - "lifetime" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/InferMatrixLab/vllm-rlt", - "commit": "4a350344027960a48384626aa646f1765fda6d4f", - "paths": [ - "README.md", - "LICENSE", - "NOTICE", - "vllm_rlt/core/kv_cache_manager.py", - "vllm_rlt/worker/model_runner.py", - "tests/test_kv_cache.py", - "pyproject.toml", - "vllm_rlt/config.py", - "vllm_rlt/engine/stage_engine.py", - "vllm_rlt/core/scheduler.py", - "vllm_rlt/worker/executor.py", - "vllm_rlt/attention/__init__.py", - "vllm_rlt/attention/backend.py", - "vllm_rlt/models/ouro.py", - "vllm_rlt/layers/__init__.py", - "vllm_rlt/__init__.py", - "tests/conftest.py", - "vllm_rlt/request.py", - "vllm_rlt/worker/metadata.py", - "vllm_rlt/kernels/paged_attention.py", - "vllm_rlt/worker/buffers.py" - ], - "readset": "references/source-notes.md", - "identity": "references/source-identity.json", - "license": "Apache-2.0", - "third_party_notices": "references/NOTICE.md" - } - ], - "applicability": { - "scope": "Pinned vLLM-RLT native Ouro synchronous scheduling with host KVMetadata: LAST_EXITED propagation versus SHARED, later depth-specific Torch-reference attention and normal release/wait/free order. No speculation, connector, preemption, prefix sharing, split/packed prefill or graph/static-buffer path qualification.", - "prerequisites": "The selected native Ouro synchronous path uses host KVMetadata, no speculation/connector/preemption, atomic prefill and disabled graph/static-buffer/packed-prefill paths; changed execution conditions need a new bounded check.", - "input_contract": "Repository revision, model/configuration, recurrent-layer set, loop count, cache layout and block size, dtype/device, attention backend and effective flags, plus the traced token/depth/layer and its next consumer.", - "produced_artifact": "A bounded token/finalization/consumer/retirement trace separating source call order, observed data and written validity, completion evidence and remaining native model/backend/performance gaps.", - "failure_modes": "Substituting mark_finalized for K/V propagation, accepting allocated pages as a complete written prefix, applying LAST_EXITED copy expectations to SHARED, or treating event recording as completion can invalidate the ownership diagnosis.", - "refusal_point": "On a failed prerequisite or complete-prefix check, retain the first failing token/depth/layer and trace the missing operation. Stop the conclusion when flags or metadata ownership leave the selected path." - }, - "limitations": [ - "Source-derived candidate; native CPU/CUDA, model correctness/quality, performance and agent efficacy are not validated.", - "Allocated capacity, host written-position validity, copied K/V bytes, event recording and completed device work are distinct; host-row checks do not qualify DeviceKVMetadata.", - "Repository Apache-2.0 and retained model/configuration NOTICE attribution do not constitute a whole-tree or model-weight license audit." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Distinguish copied K/V values, logical written positions and allocated capacity; follow the later depth-specific attention consumer.", - "Preserve executed shallower values during LAST_EXITED propagation and use SHARED as a separate one-plane control.", - "Follow the latest request event through runner release before scheduler cache release; page reuse is not evidence of erased old bytes." - ] - }, - "applicability": { - "scope": "Pinned vLLM-RLT native Ouro synchronous scheduling with host KVMetadata: LAST_EXITED propagation versus SHARED, later depth-specific Torch-reference attention and normal release/wait/free order. No speculation, connector, preemption, prefix sharing, split/packed prefill or graph/static-buffer path qualification.", - "prerequisites": "The selected native Ouro synchronous path uses host KVMetadata, no speculation/connector/preemption, atomic prefill and disabled graph/static-buffer/packed-prefill paths; changed execution conditions need a new bounded check.", - "input_contract": "Repository revision, model/configuration, recurrent-layer set, loop count, cache layout and block size, dtype/device, attention backend and effective flags, plus the traced token/depth/layer and its next consumer.", - "produced_artifact": "A bounded token/finalization/consumer/retirement trace separating source call order, observed data and written validity, completion evidence and remaining native model/backend/performance gaps.", - "failure_modes": "Substituting mark_finalized for K/V propagation, accepting allocated pages as a complete written prefix, applying LAST_EXITED copy expectations to SHARED, or treating event recording as completion can invalidate the ownership diagnosis.", - "refusal_point": "On a failed prerequisite or complete-prefix check, retain the first failing token/depth/layer and trace the missing operation. Stop the conclusion when flags or metadata ownership leave the selected path." - }, - "recommendations": [ - "Distinguish copied K/V values, logical written positions and allocated capacity; follow the later depth-specific attention consumer.", - "Preserve executed shallower values during LAST_EXITED propagation and use SHARED as a separate one-plane control.", - "Follow the latest request event through runner release before scheduler cache release; page reuse is not evidence of erased old bytes." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; native CPU/CUDA, model correctness/quality, performance and agent efficacy are not validated.", - "Allocated capacity, host written-position validity, copied K/V bytes, event recording and completed device work are distinct; host-row checks do not qualify DeviceKVMetadata.", - "Repository Apache-2.0 and retained model/configuration NOTICE attribution do not constitute a whole-tree or model-weight license audit." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json deleted file mode 100644 index 835abd4c6..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json +++ /dev/null @@ -1,157 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.audit_sd_cpp_residency", - "title": "audit-sd-cpp-residency", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit stable-diffusion.cpp image-generation memory pressure by separating catalog freshness and weight-byte delivery from model parameter storage, compute weight residency, retained runtime buffers and segment prefetch under the actual backend.", - "tags": [ - "stable-diffusion-cpp", - "diffusion", - "model-residency", - "workspace", - "prefetch", - "state-ownership", - "weight-loading", - "storage-io" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md" - }, - "checkout_ref": "checkout:audit-sd-cpp-residency", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/LICENSE": "b53fa08f515cb5a6fff7b9fd8fcd0961a4b80df29d74372d25e1f9171aa042ee", - "src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md": "4c04d39812fb566e3ecb5807c170d5c32848eb2b93a1f0fedcf66ba9a9c92860", - "src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/metadata.json": "412c649b16526038ee0299f8b2a610bb7f120eaab25cd83575809fe03b604b2e", - "src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md": "c1ec047aa37fca6b374f9bfd750e8f30d45af1d48fe728db23873c0b84ad94e8", - "src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md": "7b537cb972347dd68e303960f142892346d8b19b884e4d08016ea36a23bdaa7a", - "src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-identity.json": "af4830370b6eddc2d176a9ae14da4ab56a90b0fe048a01a89ee051df4365d8e9", - "src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md": "c493af8f6fd11ab7c9a37c9eec78cac31d57c04403a790035efbb4fff99f4669", - "src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/validation-boundaries.md": "3aaf79b36192bf8ff3e8f95ac90f2e769aa065dfac1fe70368824f3591849b8d" - }, - "skill_metadata": { - "name": "audit-sd-cpp-residency", - "description": "Audit stable-diffusion.cpp image-generation memory pressure by separating catalog freshness and weight-byte delivery from model parameter storage, compute weight residency, retained runtime buffers and segment prefetch under the actual backend.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "MIT", - "tags": [ - "stable-diffusion-cpp", - "diffusion", - "model-residency", - "workspace", - "prefetch", - "state-ownership", - "weight-loading", - "storage-io" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/leejet/stable-diffusion.cpp", - "commit": "b13a48219d5cdf408edcb93da6cd69b612ed03d3", - "paths": [ - "src/stable-diffusion.cpp", - "src/pipeline/image.cpp", - "src/pipeline/diffusion_engine.h", - "src/pipeline/diffusion_engine.cpp", - "src/core/ggml_runner.cpp", - "src/core/segment_weight_pipeline.cpp", - "src/core/compute_workspace.cpp", - "src/model_manager.cpp", - "src/model_manager_prefetch.cpp", - "src/device_residency_manager.h", - ".github/workflows/build.yml", - ".gitmodules", - "cmake/ggml.cmake", - "CMakeLists.txt", - "examples/cli/main.cpp", - "include/stable-diffusion.h", - "LICENSE", - "src/model_manager.h", - "src/model_loader.h", - "src/model_loader_files.cpp", - "src/model_loader.cpp", - "src/model_manager_files.cpp", - "src/model.h", - "src/core/util.h", - "src/core/util.cpp", - "src/model_io/tensor_storage.h", - "src/model_io/safetensors_io.cpp" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Source-grounded audit of native image-generation weight/runtime residency, optional segment prefetch and bounded non-ZIP safetensors catalog/read/staging ownership; confirm the actual source, GGML, backend, model and executing path independently.", - "prerequisites": "Access to the pinned source/storage boundaries and evidence of the actual image runner/backend dispatch; native capacity or prefetch checks require matching GGML, model and backend resources.", - "input_contract": "Runtime/GGML revisions and selection, compiled/loaded backends and devices, model/components and numerical image settings, effective memory/prefetch/placement settings and actual dispatch; retain available phase allocations, cache/workspace, transfer events and completed-image counts.", - "produced_artifact": "A phase-by-runner/backend/device ownership table for stored and resident weights, retained runtime, pending allocations, pin/prefetch ownership and release/eviction, with one discriminating check and its actual status.", - "failure_modes": "Treating max_vram as proof of segmentation, unpin as physical freeing, or configured prefetch as completed useful overlap can misidentify the capacity cause; missing observations cannot be read as zero.", - "refusal_point": "Stop when the cause is resolved or matching GGML/model/backend resources are unavailable; invoking this audit does not authorize dependency installation, model downloads, policy changes or device provisioning." - }, - "limitations": [ - "Source-derived candidate; native inference, allocator behavior, image quality and performance were not run.", - "Eighteen bounded readset paths plus one separately identity-checked README match the pinned Git commit/tree blob IDs and sizes (19 blobs total); header coverage is limited to selected owner/state declarations, and this does not authenticate every archive member or submodule contents.", - "The loader/storage extension separately authenticates twelve bounded C++ source blobs and the retained MIT text; its line-scoped readset does not replace the historical archive receipt. Catalog stamps are not content hashes; read/mapping/backend calls do not establish storage throughput or device completion.", - "GGML and other submodules are not expanded; matching private GGML headers are required even for a system library.", - "CPU bypasses the inspected accelerator capacity gate and segmentation eligibility, so CPU checks cannot qualify accelerator memory budgets or prefetch.", - "Video/audio pipelines, detailed model kernels, approximate denoising caches and full multi-device execution remain outside this readset.", - "Loader rules select ordinary non-ZIP safetensors; full GGUF/Torch/ZIP parsers, special quantization/conversion correctness and native failure injection remain unqualified." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/loader-storage-boundaries.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Separate parameter storage, compute-resident weights, retained workspace/output caches and pending allocations for each runner and phase.", - "Verify the actual monolithic, segmented or multi-device path before interpreting a memory budget or prefetch setting.", - "For a policy comparison, hold model, inputs, numerical settings and device fixed; check completed images and quality before latency, and separate startup, first and warm requests." - ] - }, - "applicability": { - "scope": "Source-grounded audit of native image-generation weight/runtime residency, optional segment prefetch and bounded non-ZIP safetensors catalog/read/staging ownership; confirm the actual source, GGML, backend, model and executing path independently.", - "prerequisites": "Access to the pinned source/storage boundaries and evidence of the actual image runner/backend dispatch; native capacity or prefetch checks require matching GGML, model and backend resources.", - "input_contract": "Runtime/GGML revisions and selection, compiled/loaded backends and devices, model/components and numerical image settings, effective memory/prefetch/placement settings and actual dispatch; retain available phase allocations, cache/workspace, transfer events and completed-image counts.", - "produced_artifact": "A phase-by-runner/backend/device ownership table for stored and resident weights, retained runtime, pending allocations, pin/prefetch ownership and release/eviction, with one discriminating check and its actual status.", - "failure_modes": "Treating max_vram as proof of segmentation, unpin as physical freeing, or configured prefetch as completed useful overlap can misidentify the capacity cause; missing observations cannot be read as zero.", - "refusal_point": "Stop when the cause is resolved or matching GGML/model/backend resources are unavailable; invoking this audit does not authorize dependency installation, model downloads, policy changes or device provisioning." - }, - "recommendations": [ - "Separate parameter storage, compute-resident weights, retained workspace/output caches and pending allocations for each runner and phase.", - "Verify the actual monolithic, segmented or multi-device path before interpreting a memory budget or prefetch setting.", - "For a policy comparison, hold model, inputs, numerical settings and device fixed; check completed images and quality before latency, and separate startup, first and warm requests." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; native inference, allocator behavior, image quality and performance were not run.", - "Eighteen bounded readset paths plus one separately identity-checked README match the pinned Git commit/tree blob IDs and sizes (19 blobs total); header coverage is limited to selected owner/state declarations, and this does not authenticate every archive member or submodule contents.", - "The loader/storage extension separately authenticates twelve bounded C++ source blobs and the retained MIT text; its line-scoped readset does not replace the historical archive receipt. Catalog stamps are not content hashes; read/mapping/backend calls do not establish storage throughput or device completion.", - "GGML and other submodules are not expanded; matching private GGML headers are required even for a system library.", - "CPU bypasses the inspected accelerator capacity gate and segmentation eligibility, so CPU checks cannot qualify accelerator memory budgets or prefetch.", - "Video/audio pipelines, detailed model kernels, approximate denoising caches and full multi-device execution remain outside this readset.", - "Loader rules select ordinary non-ZIP safetensors; full GGUF/Torch/ZIP parsers, special quantization/conversion correctness and native failure injection remain unqualified." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-sglang-diffusion-lifetimes.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-sglang-diffusion-lifetimes.json deleted file mode 100644 index cbd424883..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-sglang-diffusion-lifetimes.json +++ /dev/null @@ -1,136 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.audit_sglang_diffusion_lifetimes", - "title": "audit-sglang-diffusion-lifetimes", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose SGLang multimodal_gen stage state, component residency and request timeout/deletion when grouped outputs interfere, a module remains resident, or stopped waiting is mistaken for stopped execution; requires pinned source or a concrete trace.", - "tags": [ - "sglang", - "diffusion", - "stage", - "state", - "residency", - "cancellation" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md" - }, - "checkout_ref": "checkout:audit-sglang-diffusion-lifetimes", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/LICENSE": "1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747", - "src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md": "b0a4ba4751087126143ef86983798deb0ed00e24b6ff06bf6ef76a9df6e274f2", - "src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/metadata.json": "e2cb137978960d29b3644967a29dab930c768d3453bce8aff2aad96fd90a56d9", - "src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md": "d0fcfd1177ae2f351041c5b2abba7f5121cec133006d9e4dca8f74ddb567332d" - }, - "skill_metadata": { - "name": "audit-sglang-diffusion-lifetimes", - "description": "Diagnose SGLang multimodal_gen stage state, component residency and request timeout/deletion when grouped outputs interfere, a module remains resident, or stopped waiting is mistaken for stopped execution; requires pinned source or a concrete trace.", - "tags": [ - "sglang", - "diffusion", - "stage", - "state", - "residency", - "cancellation" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "license": "Apache-2.0", - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/sgl-project/sglang", - "commit": "df13d5dce31abfeff0e701a62e8c39a136758e6c", - "paths": [ - "python/sglang/multimodal_gen/runtime/entrypoints/control_requests.py", - "python/sglang/multimodal_gen/runtime/entrypoints/openai/utils.py", - "python/sglang/multimodal_gen/runtime/entrypoints/openai/video_api.py", - "python/sglang/multimodal_gen/runtime/managers/gpu_worker.py", - "python/sglang/multimodal_gen/runtime/managers/memory_managers/component_manager.py", - "python/sglang/multimodal_gen/runtime/managers/scheduler.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/composed_pipeline_base.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/diffusion_scheduler_utils.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/executors/parallel_executor.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/executors/pipeline_executor.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/schedule_batch.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/stages/dedup.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/stages/denoising.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/stages/timestep_preparation.py", - "python/sglang/multimodal_gen/runtime/scheduler_client.py", - ".github/workflows/diffusion-ci-gt-gen-npu.yml", - ".github/workflows/diffusion-ci-gt-gen.yml", - "LICENSE", - "python/pyproject.toml", - "python/sglang/multimodal_gen/test/unit/test_component_residency.py", - "python/sglang/multimodal_gen/test/unit/test_pipeline_executor.py", - "python/sglang/multimodal_gen/test/unit/test_pipeline_residency_manager_install.py", - "python/sglang/multimodal_gen/test/unit/test_scheduler_client.py" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Bounded monolithic multimodal_gen composed-pipeline state and lifecycle diagnosis; actual backend/model behavior requires native validation.", - "prerequisites": "Pinned source or a concrete trace of the selected composed-pipeline executor and installed residency manager, with the applicable monolithic route identified.", - "input_contract": "Source/build, backend and executor/pipeline class, model revision, modality/shape, steps/scheduler/seed/output count, grouped/serial route, ranks, placement, warmup/save settings and symptom; timeout cases need correlated request/job and client/scheduler/worker events.", - "produced_artifact": "A version-bound owner/event ledger, first unresolved boundary, causal hypothesis, smallest real discriminating check, exact results and missing dependencies.", - "failure_modes": "Assuming all executors share the SyncExecutor contract, treating closed component use as eviction, assuming grouped schedulers are isolated, or reading socket close/job deletion as stopped device compute can misdiagnose ownership.", - "refusal_point": "Stop performance claims when the executing path, cleanup or output quality is unresolved; retain the failure and a known baseline rather than inferring a blanket leak or safety conclusion." - }, - "limitations": [ - "Source-derived candidate; upstream runtime and efficacy validation not run.", - "Supplier verified one selected source blob, not the whole archive tree.", - "No automatic grouped isolation, device cancellation, cross-rank recovery or performance claim." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Trace the actual executor and separate request, stage, component-use, module, scheduler, finalizer and allocator-cache lifetimes.", - "Check scheduler aliases and declared output-copy policies before introducing overlap; compare real serial/grouped outputs with fixed timesteps and seeds.", - "Correlate worker, finalizer and device completion after client timeout or deletion; preserve an unknown execution outcome until that evidence exists." - ] - }, - "applicability": { - "scope": "Bounded monolithic multimodal_gen composed-pipeline state and lifecycle diagnosis; actual backend/model behavior requires native validation.", - "prerequisites": "Pinned source or a concrete trace of the selected composed-pipeline executor and installed residency manager, with the applicable monolithic route identified.", - "input_contract": "Source/build, backend and executor/pipeline class, model revision, modality/shape, steps/scheduler/seed/output count, grouped/serial route, ranks, placement, warmup/save settings and symptom; timeout cases need correlated request/job and client/scheduler/worker events.", - "produced_artifact": "A version-bound owner/event ledger, first unresolved boundary, causal hypothesis, smallest real discriminating check, exact results and missing dependencies.", - "failure_modes": "Assuming all executors share the SyncExecutor contract, treating closed component use as eviction, assuming grouped schedulers are isolated, or reading socket close/job deletion as stopped device compute can misdiagnose ownership.", - "refusal_point": "Stop performance claims when the executing path, cleanup or output quality is unresolved; retain the failure and a known baseline rather than inferring a blanket leak or safety conclusion." - }, - "recommendations": [ - "Trace the actual executor and separate request, stage, component-use, module, scheduler, finalizer and allocator-cache lifetimes.", - "Check scheduler aliases and declared output-copy policies before introducing overlap; compare real serial/grouped outputs with fixed timesteps and seeds.", - "Correlate worker, finalizer and device completion after client timeout or deletion; preserve an unknown execution outcome until that evidence exists." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; upstream runtime and efficacy validation not run.", - "Supplier verified one selected source blob, not the whole archive tree.", - "No automatic grouped isolation, device cancellation, cross-rank recovery or performance claim." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-vllm-kv-lifetimes.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-vllm-kv-lifetimes.json deleted file mode 100644 index 7ed6ae591..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/audit-vllm-kv-lifetimes.json +++ /dev/null @@ -1,105 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.audit_vllm_kv_lifetimes", - "title": "audit-vllm-kv-lifetimes", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit vLLM V1 asynchronous KV load/save completion, block retention, preemption, and receive-failure contracts when requests stall or blocks may be reused too early; requires a concrete connector trace.", - "tags": [ - "vllm", - "kv-cache", - "lifecycle", - "correctness" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md" - }, - "checkout_ref": "checkout:audit-vllm-kv-lifetimes", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md": "4c3f27459df239f9f4fc971c559d4c9ddfbfebd70507a8a93be8eae59a30b69d", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/metadata.json": "8ded6e60c562131616dc8cdcf0d7efc04aef1884885da0b30fb16f42d7433b7c", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/references/source-notes.md": "3665b034594fc8cc767c408002f4e16f8803369242405027c7f37ce6c9ecebe9" - }, - "skill_metadata": { - "name": "audit-vllm-kv-lifetimes", - "description": "Audit vLLM V1 asynchronous KV load/save completion, block retention, preemption, and receive-failure contracts when requests stall or blocks may be reused too early; requires a concrete connector trace.", - "tags": [ - "vllm", - "kv-cache", - "lifecycle", - "correctness" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/vllm", - "commit": "7f730b4cea9a66520a9200265928ebe016b48a39", - "paths": [ - "vllm/distributed/kv_transfer/kv_connector/v1/base.py", - "vllm/v1/core/sched/scheduler.py" - ], - "readset": "references/source-notes.md" - } - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Pinned vLLM V1 scheduler/connector ownership audit for asynchronous KV load/save completion, retained blocks, preemption and failed receives; concrete connector conformance is not established by the interface.", - "prerequisites": "A concrete connector and worker/model-runner call trace at identified revisions, with request/block events sufficient to compare actual callers against the pinned scheduler/interface contracts.", - "input_contract": "Engine/connector revision and class, producer/consumer roles, KV-group count/block namespace, failure policy, concurrent-batch and graph modes; correlate bounded request IDs with allocation, matched tokens, load/save events, receive failures, preemption, finish and pool return.", - "produced_artifact": "An ownership/event ledger and bounded contract-conformance report naming the earliest evidenced broken obligation, owner, affected requests/blocks and smallest missing observation, with a proposed targeted regression check.", - "failure_modes": "Equating reserved tokens with usable KV, finished responses with reusable blocks, or send completion with scheduler write-fence completion can hide unsafe reuse; missing failed-receive completion can leave waits unreleased.", - "refusal_point": "Escalate absent concrete caller evidence, ambiguous block namespaces, unsupported layouts or revision drift; stop before inferring a backend fix, freeing retained blocks or claiming a reproduced bug." - }, - "recommendations": [ - "Trace lookup, allocation, worker execution and completion propagation before changing request state; repeated or undecided lookup and ordinary admission waits can be valid.", - "Check delayed-free ownership and scheduler in-flight-write fences separately, including empty-step save/metadata ordering in the real worker.", - "Keep receive failure and finished_recving correlated, and use request-level failure evidence for multi-group layouts while preserving the existing recovery policy." - ] - }, - "applicability": { - "scope": "Pinned vLLM V1 scheduler/connector ownership audit for asynchronous KV load/save completion, retained blocks, preemption and failed receives; concrete connector conformance is not established by the interface.", - "prerequisites": "A concrete connector and worker/model-runner call trace at identified revisions, with request/block events sufficient to compare actual callers against the pinned scheduler/interface contracts.", - "input_contract": "Engine/connector revision and class, producer/consumer roles, KV-group count/block namespace, failure policy, concurrent-batch and graph modes; correlate bounded request IDs with allocation, matched tokens, load/save events, receive failures, preemption, finish and pool return.", - "produced_artifact": "An ownership/event ledger and bounded contract-conformance report naming the earliest evidenced broken obligation, owner, affected requests/blocks and smallest missing observation, with a proposed targeted regression check.", - "failure_modes": "Equating reserved tokens with usable KV, finished responses with reusable blocks, or send completion with scheduler write-fence completion can hide unsafe reuse; missing failed-receive completion can leave waits unreleased.", - "refusal_point": "Escalate absent concrete caller evidence, ambiguous block namespaces, unsupported layouts or revision drift; stop before inferring a backend fix, freeing retained blocks or claiming a reproduced bug." - }, - "recommendations": [ - "Trace lookup, allocation, worker execution and completion propagation before changing request state; repeated or undecided lookup and ordinary admission waits can be valid.", - "Check delayed-free ownership and scheduler in-flight-write fences separately, including empty-step save/metadata ordering in the real worker.", - "Keep receive failure and finished_recving correlated, and use request-level failure evidence for multi-group layouts while preserving the existing recovery policy." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/bound-windowed-attention-replay.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/bound-windowed-attention-replay.json deleted file mode 100644 index a4fd7d10f..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/bound-windowed-attention-replay.json +++ /dev/null @@ -1,132 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.bound_windowed_attention_replay", - "title": "bound-windowed-attention-replay", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved historical state satisfy the full dependency closure. Use when a model has shared hidden state or preserved-compressed KV and the retained region must be derived rather than assumed. Derives the retain rule from window plus rollback depth, separates query-row compaction from dropping historical keys, and names what becomes unrecoverable.", - "tags": [ - "deepinfer", - "sliding-window-attention", - "prefix-replay", - "kv-cache", - "metadata", - "correctness" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md" - }, - "checkout_ref": "checkout:bound-windowed-attention-replay", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md": "4703beb6e5194ef2ff2c0b5ec69f64e35417a4bc0dfb50a10c80a920b77ef3b5", - "src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json": "c68254b1a96a8834ae46c7a1d9035856fa4783315b219aa95e31461bd8ac58c9", - "src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/source-notes.md": "fd917bd5ac80bee9c9f4525f8e0300f6d67dc144e2a16632af42a67f3d53d098", - "src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/validation.md": "60e71e70c5519bcdf6c171eec2be2f342e8801bedd746d6005f11adc646dfaff" - }, - "skill_metadata": { - "name": "bound-windowed-attention-replay", - "description": "Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved historical state satisfy the full dependency closure. Use when a model has shared hidden state or preserved-compressed KV and the retained region must be derived rather than assumed. Derives the retain rule from window plus rollback depth, separates query-row compaction from dropping historical keys, and names what becomes unrecoverable.", - "tags": [ - "deepinfer", - "sliding-window-attention", - "prefix-replay", - "kv-cache", - "metadata", - "correctness" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/deepseek-ai/DeepInfer", - "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", - "paths": [ - "csrc/engine/input_metadata/prepare_tensor_input_v4.cpp", - "deepinfer/engine/llm_engine.py", - "deepinfer/model_executor/input_metadata/yoco.py", - "deepinfer/model_executor/model_metadata_utils.py", - "deepinfer/model_executor/models/decoder_layer.py", - "deepinfer/model_executor/models/v4_modules/attn.py", - "deepinfer/model_executor/models/v4_modules/yoco.py", - "deepinfer/worker/cache_engine.py", - "deepinfer/worker/model_runner.py", - "deepinfer/environ.py", - "LICENSE" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Decoders with an explicit producer/consumer boundary and a demonstrated dependency closure for compact query rows, including separately preserved historical state", - "prerequisites": "A derived dependency closure; a full-length KV write at the boundary; known rollback depth; control over metadata construction", - "hardware": "Backend-independent retain arithmetic; window sizes, boundary index and ring lengths come from the model config and do not transfer", - "input_contract": "The actual producer/consumer split and transitive dependency closure, boundary index, window and rollback depth, consumer KV sharing/compression layout and separately preserved historical keys.", - "produced_artifact": "A dependency listing, enforced retain length, compact query/position/store-slot/visibility metadata, a separate historical-key retention statement and a diagnostic equivalence-reference mode.", - "failure_modes": "Applying window-plus-rollback without dependency closure, discarding full-layout state, clipping away the sampling row, or comparing across preemption can invalidate replay; compact queries do not imply historical keys were dropped.", - "refusal_point": "Stop if required consumer state is unavailable without a preserved full-length copy, the boundary representation cannot be captured, or comparison preemption moves the anchor. Do not use for clipped-region outputs or required bit-identical preemption replay." - }, - "limitations": [ - "Source-derived candidate; no equivalence run was performed.", - "Applies to architectures with a demonstrated boundary dependency, not to arbitrary sliding-window stacks.", - "Window sizes, boundary-layer index and ring lengths are model-config specific and are not transferable.", - "Query-row compaction does not imply dropping historical keys; the two decisions are separate.", - "Prompt logprobs are unrecoverable for the clipped region; the reference forces them off with a warning.", - "The truncation path is off by default in the reference source." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/validation.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Derive every consumer dependency before computing keep; preserve the required full-layout representation separately from the compact query rows.", - "Use the visibility-only diagnostic mode to isolate equivalence from row gathering, and retain one sampling row for a fully clipped chunk.", - "Check metadata at chunk/store boundaries, actual prompt-logprob handling and zero preemption during equivalence comparison." - ] - }, - "applicability": { - "scope": "Decoders with an explicit producer/consumer boundary and a demonstrated dependency closure for compact query rows, including separately preserved historical state", - "prerequisites": "A derived dependency closure; a full-length KV write at the boundary; known rollback depth; control over metadata construction", - "hardware": "Backend-independent retain arithmetic; window sizes, boundary index and ring lengths come from the model config and do not transfer", - "input_contract": "The actual producer/consumer split and transitive dependency closure, boundary index, window and rollback depth, consumer KV sharing/compression layout and separately preserved historical keys.", - "produced_artifact": "A dependency listing, enforced retain length, compact query/position/store-slot/visibility metadata, a separate historical-key retention statement and a diagnostic equivalence-reference mode.", - "failure_modes": "Applying window-plus-rollback without dependency closure, discarding full-layout state, clipping away the sampling row, or comparing across preemption can invalidate replay; compact queries do not imply historical keys were dropped.", - "refusal_point": "Stop if required consumer state is unavailable without a preserved full-length copy, the boundary representation cannot be captured, or comparison preemption moves the anchor. Do not use for clipped-region outputs or required bit-identical preemption replay." - }, - "recommendations": [ - "Derive every consumer dependency before computing keep; preserve the required full-layout representation separately from the compact query rows.", - "Use the visibility-only diagnostic mode to isolate equivalence from row gathering, and retain one sampling row for a fully clipped chunk.", - "Check metadata at chunk/store boundaries, actual prompt-logprob handling and zero preemption during equivalence comparison." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no equivalence run was performed.", - "Applies to architectures with a demonstrated boundary dependency, not to arbitrary sliding-window stacks.", - "Window sizes, boundary-layer index and ring lengths are model-config specific and are not transferable.", - "Query-row compaction does not imply dropping historical keys; the two decisions are separate.", - "Prompt logprobs are unrecoverable for the clipped region; the reference forces them off with a warning.", - "The truncation path is off by default in the reference source." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/budget-speculative-verification.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/budget-speculative-verification.json deleted file mode 100644 index 6e7d3b214..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/budget-speculative-verification.json +++ /dev/null @@ -1,132 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.budget_speculative_verification", - "title": "budget-speculative-verification", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an engine already runs a draft head with multi-token speculation and verification cost grows with query rows. Separates draft cost from verify cost, builds a cumulative-acceptance rate vector, fits a cost model, and allocates a per-step row budget.", - "tags": [ - "deepinfer", - "speculative-decoding", - "mtp", - "verification", - "cost-model", - "scheduling" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md" - }, - "checkout_ref": "checkout:budget-speculative-verification", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md": "f157562d1edc905cb4fc07246567e3c5b844efb9fbf5fe39f0d40553a1b1a307", - "src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json": "1aa80a3109072c1ec211f38dbeb9e9cb9edb56aeac187946b43e65dd7502e46f", - "src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/source-notes.md": "6fa955022f16cd598f44fd6521901d1cb868b06cec545cabe2b82769a641b5d6", - "src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/validation.md": "673d93f6700052974b340fab25b3438e0e6344960569ac9abec7d7720f662288" - }, - "skill_metadata": { - "name": "budget-speculative-verification", - "description": "Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an engine already runs a draft head with multi-token speculation and verification cost grows with query rows. Separates draft cost from verify cost, builds a cumulative-acceptance rate vector, fits a cost model, and allocates a per-step row budget.", - "tags": [ - "deepinfer", - "speculative-decoding", - "mtp", - "verification", - "cost-model", - "scheduling" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/deepseek-ai/DeepInfer", - "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", - "paths": [ - "csrc/engine/mtp_varlen.hpp", - "csrc/engine/scheduler.hpp", - "deepinfer/worker/worker.py", - "deepinfer/environ.py", - "deepinfer/sampling_params.py", - "deepinfer/model_executor/models/mtp/dflash.py", - "deepinfer/model_executor/layers/sampler.py", - "deepinfer/model_executor/layers/tilelang_ops/fused_sample.py", - "LICENSE" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Engines with multi-token speculative decoding whose verification forward cost grows with query rows", - "prerequisites": "A calibrated per-position acceptance signal; a measured (rows, step_time) curve; a handle on metadata construction", - "hardware": "Backend-independent procedure; the reference implementation targets NVIDIA SM100 and Ascend 950 only", - "input_contract": "Draft width and row representation, the defined individual/conditional versus cumulative acceptance semantics, verify rows/time curve, sampler cost and acceptance round actually available at schedule time.", - "produced_artifact": "A per-step verification row budget, per-request query lengths from one through draft_width plus one, cumulative acceptance vectors and predicted-versus-realized acceptance error logs.", - "failure_modes": "Confusing verification savings with reduced drafting, misreading confidence semantics or available rounds, allowing later-position confidence to select an earlier token, or generating non-prefix grants can invalidate the budget or sampling claim.", - "refusal_point": "Stop if unthrottled drafting or sampler cost dominates, the signal cannot be made cumulative, or no round supplies acceptance estimates; the procedure also does not apply without a per-step cost model." - }, - "limitations": [ - "Source-derived candidate; no local efficacy validation and no reproduced gain.", - "The reference budgets verification rows only; draft compute is unchanged.", - "The budgeted path is off by default in the reference source.", - "Budgeting can lower accepted length; equal accepted length is not an acceptance criterion.", - "Per-position independence of the confidence used for verification is a correctness requirement, not only a scheduling convenience.", - "Cross-backend rejection-comparison parity is unproven; the reference has no equality assertion.", - "Numeric acceptance rates and warmup curves are deployment-specific and are not carried here." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/validation.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Measure draft, verify and sampler costs separately and fit the verify curve for graph sizes that will actually replay.", - "Preserve the one-row floor and prefix grants, and audit CPU/GPU allocation timing against the per-position decision-independence contract.", - "Report verification rows, accepted tokens, total step time and throughput separately; use an independently specified sampling oracle rather than requiring unchanged accepted length." - ] - }, - "applicability": { - "scope": "Engines with multi-token speculative decoding whose verification forward cost grows with query rows", - "prerequisites": "A calibrated per-position acceptance signal; a measured (rows, step_time) curve; a handle on metadata construction", - "hardware": "Backend-independent procedure; the reference implementation targets NVIDIA SM100 and Ascend 950 only", - "input_contract": "Draft width and row representation, the defined individual/conditional versus cumulative acceptance semantics, verify rows/time curve, sampler cost and acceptance round actually available at schedule time.", - "produced_artifact": "A per-step verification row budget, per-request query lengths from one through draft_width plus one, cumulative acceptance vectors and predicted-versus-realized acceptance error logs.", - "failure_modes": "Confusing verification savings with reduced drafting, misreading confidence semantics or available rounds, allowing later-position confidence to select an earlier token, or generating non-prefix grants can invalidate the budget or sampling claim.", - "refusal_point": "Stop if unthrottled drafting or sampler cost dominates, the signal cannot be made cumulative, or no round supplies acceptance estimates; the procedure also does not apply without a per-step cost model." - }, - "recommendations": [ - "Measure draft, verify and sampler costs separately and fit the verify curve for graph sizes that will actually replay.", - "Preserve the one-row floor and prefix grants, and audit CPU/GPU allocation timing against the per-position decision-independence contract.", - "Report verification rows, accepted tokens, total step time and throughput separately; use an independently specified sampling oracle rather than requiring unchanged accepted length." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation and no reproduced gain.", - "The reference budgets verification rows only; draft compute is unchanged.", - "The budgeted path is off by default in the reference source.", - "Budgeting can lower accepted length; equal accepted length is not an acceptance criterion.", - "Per-position independence of the confidence used for verification is a correctness requirement, not only a scheduling convenience.", - "Cross-backend rejection-comparison parity is unproven; the reference has no equality assertion.", - "Numeric acceptance rates and warmup curves are deployment-specific and are not carried here." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/calibrate-serving-simulation.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/calibrate-serving-simulation.json deleted file mode 100644 index 26ad55a7a..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/calibrate-serving-simulation.json +++ /dev/null @@ -1,138 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.calibrate_serving_simulation", - "title": "calibrate-serving-simulation", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical predictions from physical bounds and near-capacity queueing risk.", - "tags": [], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/calibrate-serving-simulation/SKILL.md" - }, - "checkout_ref": "checkout:calibrate-serving-simulation", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/SKILL.md": "500231dd32f470dc90831ff1cdf5a8f0c791660958db92fcfbaa333162c28e5a", - "src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/metadata.json": "69fbc43e3b354670600dc8e8423957f28c90e30b5270f2659020cc4473f25509", - "src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md": "fe614b1c62f1d4a9db797a85d9b11931ae32169c40109c7b69ecd648fb6e9d1e", - "src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/source-record.json": "61fafa5e92ae6312f8792e4630a35c78da4c1810fac4bf9cde385824c3894f36", - "src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/source-notes.md": "9696d3c0ef96b6611787446bd6f96d4d6255ca8d107105de8c1d9af83edb9fa5" - }, - "skill_metadata": { - "name": "calibrate-serving-simulation", - "description": "Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical predictions from physical bounds and near-capacity queueing risk.", - "status": "candidate", - "empirical_validation": "not_run", - "source_pack": "A", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "paper", - "id": "arxiv:2405.05465", - "version": "v2", - "url": "https://arxiv.org/abs/2405.05465v2", - "sections": [ - "3-7", - "A.1-A.2" - ], - "readset": "references/source-notes.md" - }, - { - "type": "git", - "url": "https://github.com/AMD-AGI/Infera", - "commit": "ff1136e4a51998eba2933ed63cfb9c10120f86ea", - "paths": [ - "LICENSE", - "manual/simulation/tuning_agent.md", - "manual/simulation/boundaries.md", - "manual/simulation/anchors.md", - "manual/simulation/workloads.md", - "infera/projection/agents/tuning_agent/history.py", - "infera/projection/agents/tuning_agent/cli.py", - "infera/projection/agents/tuning_agent/evaluator.py", - "infera/projection/agents/tuning_agent/inference_tuning.py", - "infera/projection/agents/tuning_agent/inference_agent.py", - "infera/projection/core/projection/inference_projection/launcher.py", - "infera/projection/core/projection/inference_projection/des.py", - "tests/unit/projection/test_tuning_slo.py", - "tests/unit/projection/test_tuning_objectives.py", - "tests/unit/projection/test_tuning_variable_lengths.py", - "tests/unit/projection/test_des_disaggregated.py" - ], - "readset": "references/infera-serving-search/source-record.json" - } - ], - "applicability": { - "scope": "Building an evidence-gated serving-configuration shortlist when a profiled simulator or cost model is proposed to reduce deployment experiments; extends performance-ceiling-analysis, optimization-search-policy and inference-benchmark-design without replacing their ceilings or measurement contract", - "prerequisites": "The packaged Vidur evidence and evidence-type notes are read; reading the skill does not authorize execution, and no validation check runs under it without separate execution authorization", - "input_contract": "Model, shape and dtype; actual kernel/runtime revision; graph mode; GPU, host and topology; request length and arrival distribution; total resources and cost basis; memory limits; latency definitions and percentile SLOs", - "produced_artifact": "A bounded decision memo with input identities, model assumptions, evidence type, exact metrics and denominators, unsupported dimensions, nondominated candidates with predicted ranges, joint constraint checks, the next discriminating trial and rollback/finalization requirements, keeping prediction and measurement columns apart; a shortlist for future measurement, not a measured result", - "failure_modes": [ - "Learned runtime output is labeled a physical bound", - "A missing identity is filled with a paper default, or topology is substituted, instead of being marked unknown", - "A low-load check stands in for an SLO claim at the intended load regime and capacity boundary", - "A predictor that relies on summed KV volume hides batches with equal volume but unequal context distributions", - "Capacity binary search is applied without verifying its monotonic/stationary premise or treating overloaded samples explicitly", - "One predicted winner is reported when uncertainty exceeds rank separation" - ], - "refusal_point": "Stops predictive certification when profiles are missing, topology is substituted, error grows near capacity or host overhead is omitted; labels an ordering unresolved when uncertainty exceeds rank separation; asserts neither universal error below 9% nor current vLLM, asynchronous PP or NPU support from the historical simulator paper" - }, - "recommendations": [ - "Classify every estimate as a physical bound, measured envelope, Amdahl-style conditional impact or empirical prediction before comparing configurations.", - "Keep prefill and decode separate in the operator signatures, and record how sharding and memory planning enter the model.", - "Recalibrate after workload drift, kernel/graph/runtime changes, new hardware or topology, or mismatched residuals." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/infera-serving-search/README.md", - "references/infera-serving-search/source-record.json" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Building an evidence-gated serving-configuration shortlist when a profiled simulator or cost model is proposed to reduce deployment experiments; extends performance-ceiling-analysis, optimization-search-policy and inference-benchmark-design without replacing their ceilings or measurement contract", - "prerequisites": "The packaged Vidur evidence and evidence-type notes are read; reading the skill does not authorize execution, and no validation check runs under it without separate execution authorization", - "input_contract": "Model, shape and dtype; actual kernel/runtime revision; graph mode; GPU, host and topology; request length and arrival distribution; total resources and cost basis; memory limits; latency definitions and percentile SLOs", - "produced_artifact": "A bounded decision memo with input identities, model assumptions, evidence type, exact metrics and denominators, unsupported dimensions, nondominated candidates with predicted ranges, joint constraint checks, the next discriminating trial and rollback/finalization requirements, keeping prediction and measurement columns apart; a shortlist for future measurement, not a measured result", - "failure_modes": [ - "Learned runtime output is labeled a physical bound", - "A missing identity is filled with a paper default, or topology is substituted, instead of being marked unknown", - "A low-load check stands in for an SLO claim at the intended load regime and capacity boundary", - "A predictor that relies on summed KV volume hides batches with equal volume but unequal context distributions", - "Capacity binary search is applied without verifying its monotonic/stationary premise or treating overloaded samples explicitly", - "One predicted winner is reported when uncertainty exceeds rank separation" - ], - "refusal_point": "Stops predictive certification when profiles are missing, topology is substituted, error grows near capacity or host overhead is omitted; labels an ordering unresolved when uncertainty exceeds rank separation; asserts neither universal error below 9% nor current vLLM, asynchronous PP or NPU support from the historical simulator paper" - }, - "recommendations": [ - "Classify every estimate as a physical bound, measured envelope, Amdahl-style conditional impact or empirical prediction before comparing configurations.", - "Keep prefill and decode separate in the operator signatures, and record how sharding and memory planning enter the model.", - "Recalibrate after workload drift, kernel/graph/runtime changes, new hardware or topology, or mismatched residuals." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/cgroup-resource-controls.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/cgroup-resource-controls.json deleted file mode 100644 index 1ff1aefcf..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/cgroup-resource-controls.json +++ /dev/null @@ -1,100 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.cgroup_resource_controls", - "title": "cgroup-resource-controls", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose and configure cgroup v2 resource controls for serving/training workloads: distinguish quota stalls from sibling contention (cpu.max vs cpu.weight, cpu.stat vs cpu.stat.local), audit requested-versus-effective cpuset with migration-cost budgeting, triage memory pressure with PSI plus event deltas (high/max/low semantics), and gate io.max caps on confirmed block-backed traffic. Includes hierarchy/delegation/permission preflight and a rollback contract. Extra resource allocation is treated as a capacity spend, never as an efficiency improvement. Control-plane procedures only — applying settings requires separate authorization.\n", - "tags": [ - "cgroup", - "cgroupv2", - "cpu", - "cpuset", - "memory", - "io", - "resource-controls", - "os" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md" - }, - "checkout_ref": "checkout:cgroup-resource-controls", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md": "fd1f5a7e6a55714730044155f5b38c1d700383d4e16c3e91963bbfbe1c773af5", - "src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/metadata.json": "155aa391e573fa5bd90a089f4e3d06f357795f5d89c8cf9260e3e8ec3636b243" - }, - "skill_metadata": { - "name": "cgroup-resource-controls", - "description": "Diagnose and configure cgroup v2 resource controls for serving/training workloads: distinguish quota stalls from sibling contention (cpu.max vs cpu.weight, cpu.stat vs cpu.stat.local), audit requested-versus-effective cpuset with migration-cost budgeting, triage memory pressure with PSI plus event deltas (high/max/low semantics), and gate io.max caps on confirmed block-backed traffic. Includes hierarchy/delegation/permission preflight and a rollback contract. Extra resource allocation is treated as a capacity spend, never as an efficiency improvement. Control-plane procedures only — applying settings requires separate authorization.\n", - "tags": [ - "cgroup", - "cgroupv2", - "cpu", - "cpuset", - "memory", - "io", - "resource-controls", - "os" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Audit Linux cgroup v2 CPU quota/shares, effective placement, memory pressure/protection and conditional block-device IO controls for serving or training workloads.", - "prerequisites": "Identify workload cgroup membership, kernel/controller support, ancestor limits and the authorized file owner; prepare previous values and stop conditions before a separately authorized trial.", - "input_contract": "Target PID and hierarchy/namespace view, requested/effective cpusets, time-aligned CPU/memory/IO counters and application/sibling observations; IO proposals require resolved block-backed traffic.", - "produced_artifact": "An evidence-qualified control review/proposal with capacity tradeoff, configuration provenance, rollback manifest and matched-window application/sibling checks.", - "failure_modes": "Ancestor constraints can invalidate requested placement; pressure movement alone is not application causality, and unsupported IO accounting or shared-inode attribution can misassign traffic." - }, - "recommendations": [ - "Separate own-quota throttling, ancestor limits and sibling contention before choosing a cpu.max or cpu.weight review.", - "Compare requested with effective CPU/node sets and budget cpuset.mems migration before proposing placement; align memory PSI with local versus descendant event deltas.", - "Treat increased headroom as a shared-capacity spend and verify application outcomes and sibling impact, not just counter improvements.", - "Resolve actual block-backed traffic and supported IO controls before proposing caps; preserve inaccessible-ancestor uncertainty and do not infer container write authority." - ] - }, - "applicability": { - "scope": "Audit Linux cgroup v2 CPU quota/shares, effective placement, memory pressure/protection and conditional block-device IO controls for serving or training workloads.", - "prerequisites": "Identify workload cgroup membership, kernel/controller support, ancestor limits and the authorized file owner; prepare previous values and stop conditions before a separately authorized trial.", - "input_contract": "Target PID and hierarchy/namespace view, requested/effective cpusets, time-aligned CPU/memory/IO counters and application/sibling observations; IO proposals require resolved block-backed traffic.", - "produced_artifact": "An evidence-qualified control review/proposal with capacity tradeoff, configuration provenance, rollback manifest and matched-window application/sibling checks.", - "failure_modes": "Ancestor constraints can invalidate requested placement; pressure movement alone is not application causality, and unsupported IO accounting or shared-inode attribution can misassign traffic." - }, - "recommendations": [ - "Separate own-quota throttling, ancestor limits and sibling contention before choosing a cpu.max or cpu.weight review.", - "Compare requested with effective CPU/node sets and budget cpuset.mems migration before proposing placement; align memory PSI with local versus descendant event deltas.", - "Treat increased headroom as a shared-capacity spend and verify application outcomes and sibling impact, not just counter improvements.", - "Resolve actual block-backed traffic and supported IO controls before proposing caps; preserve inaccessible-ancestor uncertainty and do not infer container write authority." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/checkpoint-recovery-consistency.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/checkpoint-recovery-consistency.json deleted file mode 100644 index b7a6c9107..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/checkpoint-recovery-consistency.json +++ /dev/null @@ -1,116 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.checkpoint_recovery_consistency", - "title": "checkpoint-recovery-consistency", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Assess distributed checkpoint recovery using DLRover memory agreement, persistence completion and replica transport, and NVIDIA Resiliency Extension complete-iteration selection. Distinguish worker, host and replica loss; separate queued saves, publication, application-state validity and durable recovery.\n", - "tags": [ - "checkpoint", - "recovery", - "distributed", - "shared-memory", - "replication" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md" - }, - "checkout_ref": "checkout:checkpoint-recovery-consistency", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md": "6b7db1eb2780dec27858cb7aa0a3c093c5b5dfa4c30261aebfeaa255213ff688", - "src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/metadata.json": "4185ff9f8b4c06d2a984753127780f29ceebf50d87b225a7697bfecccc2a4a60", - "src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md": "055f4357f79026fcbcc766bbe77db610376ec353623f8898f5d14a30ee69e7bc", - "src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md": "9425b1aab963a2c8d35f3db897bd15ee69699bf5b0d0c8425c3c8bba3e9a0233" - }, - "skill_metadata": { - "name": "checkpoint-recovery-consistency", - "description": "Assess distributed checkpoint recovery using DLRover memory agreement, persistence completion and replica transport, and NVIDIA Resiliency Extension complete-iteration selection. Distinguish worker, host and replica loss; separate queued saves, publication, application-state validity and durable recovery.\n", - "tags": [ - "checkpoint", - "recovery", - "distributed", - "shared-memory", - "replication" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Assessing distributed checkpoint recovery through DLRover memory agreement, persistence completion and replica transport, and NVIDIA Resiliency Extension complete-iteration selection; distinguishes worker, host and replica loss and separates queued saves, publication, application-state validity and durable recovery. Save performance belongs to dcp-async-save-performance", - "prerequisites": "The pinned sources and limits are read; reading authorizes no launchers, failure injection or checkpoint loading, and restore evidence needs separate authorization", - "input_contract": "The declared failure domain; actual memory, storage and replica placement and helper lifetime; step or iteration identity, the expected shard/rank set and finalization result; the application's model, optimizer and caller-required RNG/scheduler/data-position state contract; the target environment's format and world-size/replication assumptions", - "produced_artifact": "An answer to each of the four acceptance questions (snapshot survival, cross-participant completeness, application-state completeness, restorability) at its own evidence level, a proposed per-generation recovery evidence record including errors and skipped saves, and a proposed verification split by worker, host, replica and persistent-storage loss; no recovery success, durability, NPU support or measured gain", - "failure_modes": [ - "Equal steps or complete rank IDs are taken to verify tensor contents or whole application semantics", - "A skipped save, where not every participant acquired shared-memory access, is reported as a new recovery point", - "A returned step paired with an empty state dictionary is read as a successful restore", - "A local maximum filename or one clean shard substitutes for global coverage", - "The .dirty-then-rename publication is read as power-loss durability", - "An asynchronous save request is treated as a completed checkpoint, or queue concurrency is assumed unlimited", - "One failure domain is tested and recovery from all of them is claimed" - ], - "refusal_point": "Claims no recovery success, durability, NPU support or measured gain, does not upgrade allocation, metadata or a completed collective into proven recovery, and neither injects failures nor authorizes them" - }, - "recommendations": [ - "Answer the four acceptance questions separately; record each at its own evidence level.", - "Count replication as added availability only when copies occupy independent failure domains, and budget clean, dirty and replicated shards on each host.", - "Keep the one-outstanding asynchronous save restriction until the target queue version is checked.", - "Send save-throughput questions to dcp-async-save-performance; a faster return or higher write throughput does not establish recoverability." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/sources.md" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Assessing distributed checkpoint recovery through DLRover memory agreement, persistence completion and replica transport, and NVIDIA Resiliency Extension complete-iteration selection; distinguishes worker, host and replica loss and separates queued saves, publication, application-state validity and durable recovery. Save performance belongs to dcp-async-save-performance", - "prerequisites": "The pinned sources and limits are read; reading authorizes no launchers, failure injection or checkpoint loading, and restore evidence needs separate authorization", - "input_contract": "The declared failure domain; actual memory, storage and replica placement and helper lifetime; step or iteration identity, the expected shard/rank set and finalization result; the application's model, optimizer and caller-required RNG/scheduler/data-position state contract; the target environment's format and world-size/replication assumptions", - "produced_artifact": "An answer to each of the four acceptance questions (snapshot survival, cross-participant completeness, application-state completeness, restorability) at its own evidence level, a proposed per-generation recovery evidence record including errors and skipped saves, and a proposed verification split by worker, host, replica and persistent-storage loss; no recovery success, durability, NPU support or measured gain", - "failure_modes": [ - "Equal steps or complete rank IDs are taken to verify tensor contents or whole application semantics", - "A skipped save, where not every participant acquired shared-memory access, is reported as a new recovery point", - "A returned step paired with an empty state dictionary is read as a successful restore", - "A local maximum filename or one clean shard substitutes for global coverage", - "The .dirty-then-rename publication is read as power-loss durability", - "An asynchronous save request is treated as a completed checkpoint, or queue concurrency is assumed unlimited", - "One failure domain is tested and recovery from all of them is claimed" - ], - "refusal_point": "Claims no recovery success, durability, NPU support or measured gain, does not upgrade allocation, metadata or a completed collective into proven recovery, and neither injects failures nor authorizes them" - }, - "recommendations": [ - "Answer the four acceptance questions separately; record each at its own evidence level.", - "Count replication as added availability only when copies occupy independent failure domains, and budget clean, dirty and replicated shards on each host.", - "Keep the one-outstanding asynchronous save restriction until the target queue version is checked.", - "Send save-throughput questions to dcp-async-save-performance; a faster return or higher write throughput does not establish recoverability." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/choose-parallel-training-plan.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/choose-parallel-training-plan.json deleted file mode 100644 index e8e8a84f0..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/choose-parallel-training-plan.json +++ /dev/null @@ -1,98 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.choose_parallel_training_plan", - "title": "choose-parallel-training-plan", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for memory, topology, resharding and planning cost.", - "tags": [], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/choose-parallel-training-plan/SKILL.md" - }, - "checkout_ref": "checkout:choose-parallel-training-plan", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md": "25d04aed9782b2a8ff1186828becdc7fce93c4418039ceaef9b8dcc5ec9bcd24", - "src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/metadata.json": "4990d0a2f1a269111a680f45a5d91221af37e3f8b3ce514fea84e97878e767cd", - "src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/references/source-notes.md": "9f9184d46b19312138a6e805e439e43dc0594a0cc6b46675255a582dc946ca5b" - }, - "skill_metadata": { - "name": "choose-parallel-training-plan", - "description": "Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for memory, topology, resharding and planning cost.", - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Comparing stage, device-mesh and operator-sharding candidates for static distributed training when heterogeneous stage memory or work, or communication cost, makes a uniform data/tensor/pipeline plan suspect; derived from historical Alpa, not a current compiler configuration guide", - "prerequisites": "The packaged source notes are read; the skill authorizes no compiler or solver install or run, parallelism reconfiguration, repository edit, hardware task or benchmark", - "input_contract": "Model operator graph and shapes; global batch, microbatch and accumulation; precision and optimizer; physical links and contention; candidate shard layouts; stage latency, workspace and in-flight activation memory; the objective and expected run length. A device count without topology is insufficient", - "produced_artifact": "The best supported candidate and a simpler fallback with a memory and critical-path ledger, planning cost and amortization, semantic invariants, counter-signals, missing evidence and a proposed matched comparison; a plan, not a reconfiguration", - "failure_modes": [ - "Equal arithmetic work is taken to mean equal kernel time, carrying over Alpa's zero compute-cost simplification", - "Mean memory replaces the largest per-rank peak with multiple in-flight activation sets", - "Inter-stage transfer and microbatch count stay outside the objective, as in historical Alpa, without the gap being reported", - "Repeated slow-link sends to replicated destination tiles are not compared with one cross-mesh transfer plus local allgather", - "Inter-node Network traffic is merged with intra-node accelerator links", - "Static cost estimates are applied to dynamic shapes, asymmetric devices or changing communication contention" - ], - "refusal_point": "Rejects infeasible stage/mesh pairs, claims no global optimality outside the declared search space, and stops at the plan in paper-only mode" - }, - "recommendations": [ - "Keep a simple reference plan with the same training semantics and global batch as the comparison point.", - "Separate fixed-model scaling from model-size weak scaling before quoting a speedup; Alpa's 9.7x MoE ratio belongs to a specific weak-scaling workload, baseline and 25-Gbps cluster.", - "Weigh planning cost against the expected run length; a shorter job may not amortize it." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Comparing stage, device-mesh and operator-sharding candidates for static distributed training when heterogeneous stage memory or work, or communication cost, makes a uniform data/tensor/pipeline plan suspect; derived from historical Alpa, not a current compiler configuration guide", - "prerequisites": "The packaged source notes are read; the skill authorizes no compiler or solver install or run, parallelism reconfiguration, repository edit, hardware task or benchmark", - "input_contract": "Model operator graph and shapes; global batch, microbatch and accumulation; precision and optimizer; physical links and contention; candidate shard layouts; stage latency, workspace and in-flight activation memory; the objective and expected run length. A device count without topology is insufficient", - "produced_artifact": "The best supported candidate and a simpler fallback with a memory and critical-path ledger, planning cost and amortization, semantic invariants, counter-signals, missing evidence and a proposed matched comparison; a plan, not a reconfiguration", - "failure_modes": [ - "Equal arithmetic work is taken to mean equal kernel time, carrying over Alpa's zero compute-cost simplification", - "Mean memory replaces the largest per-rank peak with multiple in-flight activation sets", - "Inter-stage transfer and microbatch count stay outside the objective, as in historical Alpa, without the gap being reported", - "Repeated slow-link sends to replicated destination tiles are not compared with one cross-mesh transfer plus local allgather", - "Inter-node Network traffic is merged with intra-node accelerator links", - "Static cost estimates are applied to dynamic shapes, asymmetric devices or changing communication contention" - ], - "refusal_point": "Rejects infeasible stage/mesh pairs, claims no global optimality outside the declared search space, and stops at the plan in paper-only mode" - }, - "recommendations": [ - "Keep a simple reference plan with the same training semantics and global batch as the comparison point.", - "Separate fixed-model scaling from model-size weak scaling before quoting a speedup; Alpa's 9.7x MoE ratio belongs to a specific weak-scaling workload, baseline and 25-Gbps cluster.", - "Weigh planning cost against the expected run length; a shorter job may not amortize it." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/cite-check-downstream-pr.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/cite-check-downstream-pr.json deleted file mode 100644 index a7ff52622..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/cite-check-downstream-pr.json +++ /dev/null @@ -1,107 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.cite_check_downstream_pr", - "title": "cite-check-downstream-pr", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Review a downstream PR description for correct acceptance issue links, SHA pins, and artifact citations.", - "tags": [ - "review", - "downstream", - "citation", - "evidence", - "verdict" - ], - "origin": [ - "first-party" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/workflows/cite-check-downstream-pr/SKILL.md" - }, - "checkout_ref": "checkout:cite-check-downstream-pr", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md": "485932cdf3b1fba5dc69be73e75d38f86a7b79442d26e82a296c963a7598d226", - "src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/metadata.json": "ade23e3def5c7b875fba4084d32fa63b1af90bebeade72506265c974405eb55d" - }, - "skill_metadata": { - "name": "cite-check-downstream-pr", - "description": "Review a downstream PR description for correct acceptance issue links, SHA pins, and artifact citations.", - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "review", - "downstream", - "citation", - "evidence", - "verdict" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Review of a downstream PR description that cites RSI acceptance work for a performance or quality improvement", - "prerequisites": "The PR description, the acceptance issue it cites, and the evidence paths it names; reviewing is read-only and needs no local run", - "input_contract": "One PR description plus the cited acceptance issue and its evidence tree; the reviewer reads claims and citations, not the candidate code", - "produced_artifact": "A per-checklist review result that either accepts the citation or marks the summary unverified, using only PASS/FAIL/INCONCLUSIVE vocabulary", - "failure_modes": [ - "No acceptance issue URL, only a chat thread", - "Branch names instead of full baseline and candidate SHAs", - "Microbenchmark evidence offered for an end-to-end latency claim", - "A threshold restated or invented relative to the acceptance issue", - "Evidence path absent, stale or on a different SHA than the cited candidate" - ], - "refusal_point": "Refuses to restate a performance or quality claim whose artifacts are missing, stale or off-SHA; reports the summary as unverified instead" - }, - "recommendations": [ - "Read the cited acceptance issue before the PR description so an invented threshold is visible immediately.", - "Resolve every listed evidence path and compare its SHA with the candidate SHA the PR names.", - "Report an unverifiable claim as unverified rather than repeating its number." - ], - "limitations": [ - "Checklist-only procedure; it validates citation discipline, not the measurement behind an artifact.", - "A passing review does not establish that the campaign evidence itself is correct." - ], - "provenance_readsets": [], - "searchspace_not_applicable": "SKILL.md §§ Checklist and Red flags check acceptance links, revision/artifact citations and claim wording; they provide no hardware-resource mechanism." - }, - "applicability": { - "scope": "Review of a downstream PR description that cites RSI acceptance work for a performance or quality improvement", - "prerequisites": "The PR description, the acceptance issue it cites, and the evidence paths it names; reviewing is read-only and needs no local run", - "input_contract": "One PR description plus the cited acceptance issue and its evidence tree; the reviewer reads claims and citations, not the candidate code", - "produced_artifact": "A per-checklist review result that either accepts the citation or marks the summary unverified, using only PASS/FAIL/INCONCLUSIVE vocabulary", - "failure_modes": [ - "No acceptance issue URL, only a chat thread", - "Branch names instead of full baseline and candidate SHAs", - "Microbenchmark evidence offered for an end-to-end latency claim", - "A threshold restated or invented relative to the acceptance issue", - "Evidence path absent, stale or on a different SHA than the cited candidate" - ], - "refusal_point": "Refuses to restate a performance or quality claim whose artifacts are missing, stale or off-SHA; reports the summary as unverified instead" - }, - "recommendations": [ - "Read the cited acceptance issue before the PR description so an invented threshold is visible immediately.", - "Resolve every listed evidence path and compare its SHA with the candidate SHA the PR names.", - "Report an unverifiable claim as unverified rather than repeating its number." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Checklist-only procedure; it validates citation discipline, not the measurement behind an artifact.", - "A passing review does not establish that the campaign evidence itself is correct." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/claim-contract.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/claim-contract.json deleted file mode 100644 index 59af96d85..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/claim-contract.json +++ /dev/null @@ -1,104 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.claim_contract", - "title": "claim-contract", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Enforce ThinkFlowLab claim contract wording and verdict rules without embedding campaign-specific revision pins.", - "tags": [ - "claim", - "wording", - "verdict", - "evidence", - "portable" - ], - "origin": [ - "first-party" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/workflows/claim-contract/SKILL.md" - }, - "checkout_ref": "checkout:claim-contract", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/SKILL.md": "3af1dbbe81d91cb05fc2ef15a2ae25d7592a08558e1a5535f49c52f33a488c55", - "src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/metadata.json": "25c0137cf347f75d7ff8f8305d9ff5f055c6881ddf4b708f923c3176de16c153" - }, - "skill_metadata": { - "name": "claim-contract", - "description": "Enforce ThinkFlowLab claim contract wording and verdict rules without embedding campaign-specific revision pins.", - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "claim", - "wording", - "verdict", - "evidence", - "portable" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Wording and verdict rules for any public performance or quality claim produced from an RSI campaign", - "prerequisites": "A completed campaign with pinned baseline and candidate SHAs and an evidence tree; the rules themselves carry no numeric threshold", - "input_contract": "A claim draft, the campaign RunSpec and its evidence tree under evidence//", - "produced_artifact": "A claim sentence in the pinned form, or a refusal when the campaign verdict is not PASS", - "failure_modes": [ - "Claim asserted before the campaign reaches PASS with artifacts", - "Baseline or candidate SHA written into a shared template, skill or default CI environment", - "Microbenchmark result presented as end-to-end completion", - "A lesson transferred across an unverified repo x revision x backend x device x model x scenario tuple" - ], - "refusal_point": "Refuses any public claim lacking a PASS verdict and artifacts under evidence/, and refuses numeric thresholds because thresholds belong to the campaign" - }, - "recommendations": [ - "Keep campaign SHAs in the acceptance issue; a shared template that carries them will be wrong for the next campaign.", - "Write the verdict from the evidence tree before writing the claim sentence.", - "Treat each repo x revision x backend x device x model x scenario tuple as an independent claim." - ], - "limitations": [ - "States wording and verdict rules only; it selects no numeric gate and validates no measurement.", - "SIAS plane boundaries constrain where a change belongs, not whether it is correct." - ], - "provenance_readsets": [] - }, - "applicability": { - "scope": "Wording and verdict rules for any public performance or quality claim produced from an RSI campaign", - "prerequisites": "A completed campaign with pinned baseline and candidate SHAs and an evidence tree; the rules themselves carry no numeric threshold", - "input_contract": "A claim draft, the campaign RunSpec and its evidence tree under evidence//", - "produced_artifact": "A claim sentence in the pinned form, or a refusal when the campaign verdict is not PASS", - "failure_modes": [ - "Claim asserted before the campaign reaches PASS with artifacts", - "Baseline or candidate SHA written into a shared template, skill or default CI environment", - "Microbenchmark result presented as end-to-end completion", - "A lesson transferred across an unverified repo x revision x backend x device x model x scenario tuple" - ], - "refusal_point": "Refuses any public claim lacking a PASS verdict and artifacts under evidence/, and refuses numeric thresholds because thresholds belong to the campaign" - }, - "recommendations": [ - "Keep campaign SHAs in the acceptance issue; a shared template that carries them will be wrong for the next campaign.", - "Write the verdict from the evidence tree before writing the claim sentence.", - "Treat each repo x revision x backend x device x model x scenario tuple as an independent claim." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "States wording and verdict rules only; it selects no numeric gate and validates no measurement.", - "SIAS plane boundaries constrain where a change belongs, not whether it is correct." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json deleted file mode 100644 index b6f244a39..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json +++ /dev/null @@ -1,136 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.comfy_ui", - "title": "comfy-ui", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit ComfyUI workflow-output cache validity, lazy dependencies and model-weight residency as separate state contracts using a pinned source revision and explicit runtime evidence.", - "tags": [ - "comfyui", - "workflow", - "cache-invalidation", - "model-residency", - "state-ownership" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/comfy-ui/SKILL.md" - }, - "checkout_ref": "checkout:comfy-ui", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/LICENSE": "3972dc9744f6499f0f9b2dbf76696f2ae7ad8af9b23dde66d6af86c9dfb36986", - "src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/SKILL.md": "c993bd6ff8e9b5f82f39f175ec8242341136e4ee0808af414e1d94ad3e196c5e", - "src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/metadata.json": "81ad1c73b12426e4bf7961889f4b9a1c92739c6be18380402f7866e021cac67b", - "src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/graph-cache-invalidation.md": "614c9a4e6943d24d7d7be7d632682ce8b1dcc239124a37d9dc1d487113772eac", - "src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/model-residency.md": "014c68386e7cdfdb7b4f566836e8b2ff61bd5a5a506e7b4723f83fe80c5d4de7", - "src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/repo-provenance.md": "ca996ef515af42a3ce304747fa3f12a31c00533deaace5ba2ce21aefa0a62e2e", - "src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/validation-boundaries.md": "1913dedbeb558d872d6a0e015cc11d822910c3cb259a236db29fb60ad8a64263" - }, - "skill_metadata": { - "name": "comfy-ui", - "description": "Audit ComfyUI workflow-output cache validity, lazy dependencies and model-weight residency as separate state contracts using a pinned source revision and explicit runtime evidence.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "GPL-3.0", - "tags": [ - "comfyui", - "workflow", - "cache-invalidation", - "model-residency", - "state-ownership" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/Comfy-Org/ComfyUI", - "commit": "b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c", - "paths": [ - "execution.py", - "comfy_execution/caching.py", - "comfy_execution/graph.py", - "comfy/model_management.py", - "comfy/model_patcher.py", - "comfy/sampler_helpers.py", - "server.py", - "main.py", - "comfy_execution/graph_utils.py", - "comfy_execution/cache_provider.py", - "nodes.py", - "comfy/sample.py", - "comfy/samplers.py", - "tests/execution/test_execution.py", - "tests-unit/execution_test/test_execution_list.py", - "tests-unit/comfy_test/model_management_test.py", - "tests/inference/test_inference.py", - ".github/workflows/test-execution.yml", - "LICENSE", - "pyproject.toml", - "tests-unit/execution_test/test_cache_provider.py", - "AGENTS.md" - ], - "readset": "references/repo-provenance.md" - } - ], - "applicability": { - "scope": "Source-grounded ComfyUI workflow-output cache and model-weight ownership audit; confirm the actual revision, node registry, cache caller and static/dynamic patcher before transfer.", - "prerequisites": "Access to the pinned provenance and the actual node registry/cache caller; residency questions additionally require the concrete patcher and effective dynamic/static mode. Native memory conclusions need the actual backend.", - "input_contract": "Exact revision, API-format prompt/output targets, relevant node classes/constants/links/hidden inputs and external state, selected cache caller/mode and patcher settings, plus available execution/load/memory/timing observations.", - "produced_artifact": "A state/owner/key/lifetime/consumer ledger with invalidation or release events, evidence and unresolved boundaries, followed by a discriminating check and its actual result.", - "failure_modes": "Treating identical JSON as proof of unchanged external inputs, confusing eviction with key validity, or equating workflow cache hits with device-resident model weights can explain the wrong state contract.", - "refusal_point": "Stop when the cause/action is supported or required environment, backend or model resources are unavailable; do not manufacture a hit, hardware result or performance threshold." - }, - "limitations": [ - "Source-derived candidate; no local executor-cache, device-residency or performance qualification.", - "Seven self-authored cases exercised only the unchanged pure upstream graph_utils module; they do not verify PromptExecutor, cache signatures or model behavior.", - "The dynamic native allocator and third-party custom-node behavior remain outside the verified scope." - ], - "provenance_readsets": [ - "references/repo-provenance.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Draw the affected dependency cone with declared, lazy and dynamically expanded links, then mark unfingerprinted external state unresolved.", - "Separate output/object cache validity and retention from patch identity, partial unload, full detach and allocator-visible release.", - "Change one test variable while fixing prompt, output target, model, backend, precision, seed and cache mode; check values and executed nodes before timing." - ] - }, - "applicability": { - "scope": "Source-grounded ComfyUI workflow-output cache and model-weight ownership audit; confirm the actual revision, node registry, cache caller and static/dynamic patcher before transfer.", - "prerequisites": "Access to the pinned provenance and the actual node registry/cache caller; residency questions additionally require the concrete patcher and effective dynamic/static mode. Native memory conclusions need the actual backend.", - "input_contract": "Exact revision, API-format prompt/output targets, relevant node classes/constants/links/hidden inputs and external state, selected cache caller/mode and patcher settings, plus available execution/load/memory/timing observations.", - "produced_artifact": "A state/owner/key/lifetime/consumer ledger with invalidation or release events, evidence and unresolved boundaries, followed by a discriminating check and its actual result.", - "failure_modes": "Treating identical JSON as proof of unchanged external inputs, confusing eviction with key validity, or equating workflow cache hits with device-resident model weights can explain the wrong state contract.", - "refusal_point": "Stop when the cause/action is supported or required environment, backend or model resources are unavailable; do not manufacture a hit, hardware result or performance threshold." - }, - "recommendations": [ - "Draw the affected dependency cone with declared, lazy and dynamically expanded links, then mark unfingerprinted external state unresolved.", - "Separate output/object cache validity and retention from patch identity, partial unload, full detach and allocator-visible release.", - "Change one test variable while fixing prompt, output target, model, backend, precision, seed and cache mode; check values and executed nodes before timing." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local executor-cache, device-residency or performance qualification.", - "Seven self-authored cases exercised only the unchanged pure upstream graph_utils module; they do not verify PromptExecutor, cache signatures or model behavior.", - "The dynamic native allocator and third-party custom-node behavior remain outside the verified scope." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/competitor-scan.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/competitor-scan.json deleted file mode 100644 index ce143e91c..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/competitor-scan.json +++ /dev/null @@ -1,101 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.competitor_scan", - "title": "competitor-scan", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Structure a lightweight competitive scan without inventing benchmark numbers or campaign verdicts.", - "tags": [ - "competitive-analysis", - "positioning", - "claim", - "unverified" - ], - "origin": [ - "first-party" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/workflows/competitor-scan/SKILL.md" - }, - "checkout_ref": "checkout:competitor-scan", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/SKILL.md": "0ede58ace65a9324a538b4816add1684f230ba536d4364fe5a5dd2c0bbed3776", - "src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/metadata.json": "e1a279c0b730c80fa34ca985c004d4084dae5ee77685f7f2b642cf469bfffffc" - }, - "skill_metadata": { - "name": "competitor-scan", - "description": "Structure a lightweight competitive scan without inventing benchmark numbers or campaign verdicts.", - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "competitive-analysis", - "positioning", - "claim", - "unverified" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Qualitative positioning of a new RSI repository against other projects before acceptance evidence exists", - "prerequisites": "A selected repository and readable public material from the alternatives; no benchmark run is required", - "input_contract": "A target repository plus whatever the alternatives publish about what they optimize and how they prove it", - "produced_artifact": "A short Markdown note covering problem, alternatives, differentiation, limits and open questions", - "failure_modes": [ - "Numeric comparison written without a PASS campaign in our own evidence tree", - "Third-party figure repeated as if reproduced under our RunSpec", - "Scan presented as a substitute for acceptance evidence" - ], - "refusal_point": "Refuses any numeric comparison not sourced from our own evidence//results/ with a PASS verdict" - }, - "recommendations": [ - "Answer the four framing questions qualitatively first; the absence of a published claim contract is itself a finding.", - "Label every third-party number unverified unless it was reproduced under our RunSpec.", - "Close the note with open questions rather than a conclusion the evidence cannot support." - ], - "limitations": [ - "Produces positioning material only; it establishes no performance or quality result.", - "Third-party claim contracts are read from public material and may be incomplete." - ], - "provenance_readsets": [], - "searchspace_not_applicable": "SKILL.md §§ Frame, Rules and Deliverable produce a qualitative competitive comparison with evidence limits, not a hardware-resource decision or implementation mechanism." - }, - "applicability": { - "scope": "Qualitative positioning of a new RSI repository against other projects before acceptance evidence exists", - "prerequisites": "A selected repository and readable public material from the alternatives; no benchmark run is required", - "input_contract": "A target repository plus whatever the alternatives publish about what they optimize and how they prove it", - "produced_artifact": "A short Markdown note covering problem, alternatives, differentiation, limits and open questions", - "failure_modes": [ - "Numeric comparison written without a PASS campaign in our own evidence tree", - "Third-party figure repeated as if reproduced under our RunSpec", - "Scan presented as a substitute for acceptance evidence" - ], - "refusal_point": "Refuses any numeric comparison not sourced from our own evidence//results/ with a PASS verdict" - }, - "recommendations": [ - "Answer the four framing questions qualitatively first; the absence of a published claim contract is itself a finding.", - "Label every third-party number unverified unless it was reproduced under our RunSpec.", - "Close the note with open questions rather than a conclusion the evidence cannot support." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Produces positioning material only; it establishes no performance or quality result.", - "Third-party claim contracts are read from public material and may be incomplete." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dcp-async-save-performance.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dcp-async-save-performance.json deleted file mode 100644 index 5bf38ed9f..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dcp-async-save-performance.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.dcp_async_save_performance", - "title": "dcp-async-save-performance", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose training-visible checkpoint pauses using PyTorch Distributed Checkpoint asynchronous saving and staging. Separate snapshot capture, staging completion, upload completion and backend durability; bound in-flight requests and cached CPU buffers before proposing overlap. Use for DCP save performance, not distributed recovery validation.\n", - "tags": [ - "checkpoint", - "training", - "staging", - "host-memory", - "storage", - "pytorch" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md" - }, - "checkout_ref": "checkout:dcp-async-save-performance", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md": "645139ee0c88ec865cad952e24cda35f4648d4b777b732c9b7d274b6c7830845", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/metadata.json": "49764fa454403fb38891d82ee63de1e07a90d3ff6a3cd28550aee7ea34cbdcb0", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/references/sources.md": "22afe4f4d77f441096cc657d55d763473bfcbea8fb531c8e493fe49ab84955f7" - }, - "skill_metadata": { - "name": "dcp-async-save-performance", - "description": "Diagnose training-visible checkpoint pauses using PyTorch Distributed Checkpoint asynchronous saving and staging. Separate snapshot capture, staging completion, upload completion and backend durability; bound in-flight requests and cached CPU buffers before proposing overlap. Use for DCP save performance, not distributed recovery validation.\n", - "tags": [ - "checkpoint", - "training", - "staging", - "host-memory", - "storage", - "pytorch" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/sources.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Source-supported diagnosis of training-visible PyTorch DCP save pauses, snapshot/staging/upload lifetimes and bounded CPU staging-buffer reuse; distributed recovery validity and backend durability need separate evidence.", - "prerequisites": "Known correspondence between the installed PyTorch API and source/version notes, identified checkpointed-state mutation boundaries and an initial one-outstanding-save policy before overlap is considered.", - "input_contract": "Installed PyTorch revision, stager, storage writer and thread/process executor, checkpoint contents, ranks per host and state-mutation boundaries; performance assessment needs caller/staging/upload waits and host/resource observations.", - "produced_artifact": "A completion/lifetime diagnosis and smallest justified staging/cache/backpressure lever, with separate caller-pause, staging, upload and step-time measurements proposed for authorized validation.", - "failure_modes": "Mutating source state before staging completes, overwriting cached snapshots before prior upload completes, ignoring caller-side state_dict work or treating async return as durability can corrupt the snapshot or misstate the pause.", - "refusal_point": "Reject staging overlap that races checkpointed-state mutation; do not infer backend durability from asynchronous return or writer completion." - }, - "recommendations": [ - "Place the staging wait before the earliest mutation of any checkpointed state, including forward-mutated buffers rather than only optimizer.step.", - "Wait for the previous upload before reusing a stager's CPU storage; observe the final upload before closing caller-owned staging resources.", - "Measure checkpoint and intervening steps with the same state/cadence, retaining first-use and cached-buffer costs plus per-host memory and I/O pressure." - ] - }, - "applicability": { - "scope": "Source-supported diagnosis of training-visible PyTorch DCP save pauses, snapshot/staging/upload lifetimes and bounded CPU staging-buffer reuse; distributed recovery validity and backend durability need separate evidence.", - "prerequisites": "Known correspondence between the installed PyTorch API and source/version notes, identified checkpointed-state mutation boundaries and an initial one-outstanding-save policy before overlap is considered.", - "input_contract": "Installed PyTorch revision, stager, storage writer and thread/process executor, checkpoint contents, ranks per host and state-mutation boundaries; performance assessment needs caller/staging/upload waits and host/resource observations.", - "produced_artifact": "A completion/lifetime diagnosis and smallest justified staging/cache/backpressure lever, with separate caller-pause, staging, upload and step-time measurements proposed for authorized validation.", - "failure_modes": "Mutating source state before staging completes, overwriting cached snapshots before prior upload completes, ignoring caller-side state_dict work or treating async return as durability can corrupt the snapshot or misstate the pause.", - "refusal_point": "Reject staging overlap that races checkpointed-state mutation; do not infer backend durability from asynchronous return or writer completion." - }, - "recommendations": [ - "Place the staging wait before the earliest mutation of any checkpointed state, including forward-mutated buffers rather than only optimizer.step.", - "Wait for the previous upload before reusing a stager's CPU storage; observe the final upload before closing caller-owned staging resources.", - "Measure checkpoint and intervening steps with the same state/cadence, retaining first-use and cached-buffer costs plus per-host memory and I/O pressure." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dense-performance-evidence.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dense-performance-evidence.json deleted file mode 100644 index e4b68fc93..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dense-performance-evidence.json +++ /dev/null @@ -1,110 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.dense_performance_evidence", - "title": "dense-performance-evidence", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps evidence types to the questions they answer and the claims they can support. Use to decide which evidence to collect next, avoid over-claiming from cheap evidence, and structure a measurement campaign.\n", - "tags": [ - "performance", - "evidence", - "profiling", - "measurement" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/dense-performance-evidence/SKILL.md" - }, - "checkout_ref": "checkout:dense-performance-evidence", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md": "53cc1867a13f54d1006d50517dea8848869ca50df3f0cd610b2424cc8d18c933", - "src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/metadata.json": "041780985dcaf76809d29dc0d4e242359b17b6bcfaadab2d05674fa4699a5427" - }, - "skill_metadata": { - "name": "dense-performance-evidence", - "description": "Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps evidence types to the questions they answer and the claims they can support. Use to decide which evidence to collect next, avoid over-claiming from cheap evidence, and structure a measurement campaign.\n", - "tags": [ - "performance", - "evidence", - "profiling", - "measurement" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Deciding which performance evidence to collect next and which claim each evidence level supports, from correctness through microbenchmark, component profile, isolated scenario benchmark and multi-regime serving benchmark to a production A/B experiment; also structuring a measurement campaign", - "prerequisites": "Level 0 correctness evidence is collected before any performance measurement; a Level 5 experiment needs production infrastructure, traffic splitting and statistical power", - "input_contract": "The claim to be made or the question to answer, and the evidence already collected with its level, workload, hardware/software context and controls", - "produced_artifact": "The minimum evidence level each claim requires, the next evidence to collect under the escalation rule, and negative-result records with hypothesis, evidence level, the metric that did not improve, context and the reason if known; a classification of evidence, not a measurement", - "failure_modes": [ - "An md5 match or a test pass is read as semantically correct output or as support for a performance claim", - "A microbenchmark improvement is reported as an inference throughput gain", - "Absolute kernel durations or single collective latencies are compared across devices or sessions", - "A win on one fixed workload or concurrency is generalized to production traffic", - "Worse metrics, per-task regressions or negative results are discarded, or a shared task budget is taken as equal total cost" - ], - "refusal_point": "Withholds any claim above the level its evidence reaches, leaves an unresolved source protocol or denominator mismatch unresolved, and lets neither upstream scores nor local packaging checks promote an InferRSI candidate" - }, - "recommendations": [ - "Collect Level 0 first and profile at Level 2 before implementing; escalate to Level 3 only after a promising Level 2 result, and to Level 4 only after a Level 3 win worth validating across regimes.", - "Collect a Level 1 microbenchmark only for an isolated-kernel question after Level 2 has confirmed that kernel is on the critical path.", - "For a Level 1 claim, pin input shape and dtype, exclude warmup, synchronize before timing, run at least 10 repetitions and report median and IQR; for a Level 3 claim, match hardware, software stack, model revision and inputs, run at least 3 repetitions and alternate A/B/A order.", - "Read the admission/drain occupancy case when interpreting finite service cohorts; phase-selected occupancy is not end-to-end gain." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "references": [ - "../inference-optimization-casebook/cases/admission-drain-occupancy.md", - "../inference-optimization-casebook/cases/admission-drain-occupancy.json" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Deciding which performance evidence to collect next and which claim each evidence level supports, from correctness through microbenchmark, component profile, isolated scenario benchmark and multi-regime serving benchmark to a production A/B experiment; also structuring a measurement campaign", - "prerequisites": "Level 0 correctness evidence is collected before any performance measurement; a Level 5 experiment needs production infrastructure, traffic splitting and statistical power", - "input_contract": "The claim to be made or the question to answer, and the evidence already collected with its level, workload, hardware/software context and controls", - "produced_artifact": "The minimum evidence level each claim requires, the next evidence to collect under the escalation rule, and negative-result records with hypothesis, evidence level, the metric that did not improve, context and the reason if known; a classification of evidence, not a measurement", - "failure_modes": [ - "An md5 match or a test pass is read as semantically correct output or as support for a performance claim", - "A microbenchmark improvement is reported as an inference throughput gain", - "Absolute kernel durations or single collective latencies are compared across devices or sessions", - "A win on one fixed workload or concurrency is generalized to production traffic", - "Worse metrics, per-task regressions or negative results are discarded, or a shared task budget is taken as equal total cost" - ], - "refusal_point": "Withholds any claim above the level its evidence reaches, leaves an unresolved source protocol or denominator mismatch unresolved, and lets neither upstream scores nor local packaging checks promote an InferRSI candidate" - }, - "recommendations": [ - "Collect Level 0 first and profile at Level 2 before implementing; escalate to Level 3 only after a promising Level 2 result, and to Level 4 only after a Level 3 win worth validating across regimes.", - "Collect a Level 1 microbenchmark only for an isolated-kernel question after Level 2 has confirmed that kernel is on the critical path.", - "For a Level 1 claim, pin input shape and dtype, exclude warmup, synchronize before timing, run at least 10 repetitions and report median and IQR; for a Level 3 claim, match hardware, software stack, model revision and inputs, run at least 3 repetitions and alternate A/B/A order.", - "Read the admission/drain occupancy case when interpreting finite service cohorts; phase-selected occupancy is not end-to-end gain." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/diagnose-vllm-admission-pressure.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/diagnose-vllm-admission-pressure.json deleted file mode 100644 index 16f37f06a..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/diagnose-vllm-admission-pressure.json +++ /dev/null @@ -1,121 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.diagnose_vllm_admission_pressure", - "title": "diagnose-vllm-admission-pressure", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose vLLM V1 admission stalls and KV-pressure preemption by separating token, input, active-request, cache-allocation, and connector-wait constraints at a pinned scheduler revision.", - "tags": [ - "vllm", - "scheduler", - "admission", - "kv-cache" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md" - }, - "checkout_ref": "checkout:diagnose-vllm-admission-pressure", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md": "d99fc28d48127d3bc627f2a0cab6f3f16158a77da49428838a08865adaea9905", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/metadata.json": "a4a4c4b7645852bce4abafde8e4128baacbdf1bc2a69362f7c3120db597fbe40", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md": "82b24be82451ee121892f61575d0cf4ecb9920813a25ccb26cce2b45dda6d8b3", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/migration-is-not-free-memory.md": "2ac168f2725cba7fc92dfcc67c01bf2258dff6a2dbe1ddb063bc60c175d82d0a", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/parallelism-change-with-state-cost.md": "5accc81086e4500d66f894ef6af9951e1cbb76941f6c631aab5c5561a786d8d8", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/preemption-under-kv-pressure.md": "462200a6d0b193ef01f32374a463e0f2838ca8edf33d9ab898def65ffd0b490a", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md": "d4ef29869959b835a5dcb06ddb2fd8e6cff49bb176994dc1d46eb9a852f6228d", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/fastserve.md": "9b4e275843000ef453d508d9c3c9c3f8ea1de5545fbd01d7abe74baa8a4931a7", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/llumnix.md": "f252ce33ac2db017701ca1d49813c5ec1996c4c92900cf35bb6bda7fc59c0f18", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/loongserve.md": "7bc6d1e256d461b04acc8ed856cf99108faceda9cde729ab95bcca596fe019ca", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/orca.md": "f5a3b5870d64226408a8fcda5d69483769d823f8a5efd7f11698c8da483ab195", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md": "9866bd0c78b2b638998b3b632f6bae26aef032462c81abee336d309af5b07831", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/llumnix-readset.md": "3a2819d65ad24087c08530492ef39d8ea046ed1f780d53512f1d23e2b3578487", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md": "f1a383e7bdbbd5a5a472d7f6e1292c8a17d4910ff736ca5a6aa422e1bee74f8c", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md": "a497ef23b7e60c0fbc15d95b7e07cd20547cfe56f37c24dfe6ed70d79ef7a3cf", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/source-notes.md": "84033fd514773f10c0bddd9914fabb980497ba2b308f226a922c3e1888413827" - }, - "skill_metadata": { - "name": "diagnose-vllm-admission-pressure", - "description": "Diagnose vLLM V1 admission stalls and KV-pressure preemption by separating token, input, active-request, cache-allocation, and connector-wait constraints at a pinned scheduler revision.", - "tags": [ - "vllm", - "scheduler", - "admission", - "kv-cache" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/papers-20261015/cases/migration-is-not-free-memory.md", - "references/papers-20261015/cases/parallelism-change-with-state-cost.md", - "references/papers-20261015/cases/preemption-under-kv-pressure.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/scheduling-state-boundaries.md", - "references/papers-20261015/notes/fastserve.md", - "references/papers-20261015/notes/llumnix.md", - "references/papers-20261015/notes/loongserve.md", - "references/papers-20261015/notes/orca.md", - "references/papers-20261015/references/fastserve-readset.md", - "references/papers-20261015/references/llumnix-readset.md", - "references/papers-20261015/references/loongserve-readset.md", - "references/papers-20261015/references/orca-readset.md", - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Pinned vLLM V1 branch-based diagnosis of queued requests, low scheduled-token counts and KV-pressure preemption, separating admission/token/input/allocation/connector-wait constraints.", - "prerequisites": "The deployed scheduler must be mapped to the pinned branches, with effective configuration and a bounded request/step trace; another revision requires re-establishing that correspondence.", - "input_contract": "Deployed revision, scheduler class/configuration, connector role, KV layout, policy and speculative/encoder/concurrent-batch features; per-step queues, tokens, both budgets, allocation results, victims, transfer/free state and timing.", - "produced_artifact": "A per-step accounting ledger, observed limiting branch and competing explanation, missing instrumentation and one conditional configuration/progress hypothesis with predicted effects and trade-offs.", - "failure_modes": "Inferring memory exhaustion from a queue or high KV usage, conflating token/input budgets, ignoring pending frees or reading async token accounting as load completion can attribute admission stalls to the wrong branch.", - "refusal_point": "When required state is missing, leave the diagnosis unresolved and request the smallest trace; escalate allocation or connector behavior outside the readset and stop before live mutation or performance claims." - }, - "recommendations": [ - "Trace the first actual blocking or skip branch and record running versus admission caps and token versus input budgets separately.", - "Inspect deferred frees and request-free eligibility before blaming preemption victim selection, then reconcile restored budgets for an already-scheduled victim.", - "Reconstruct waiting admission independently, including queue precedence and async prefix states; change no setting without a branch-specific hypothesis." - ] - }, - "applicability": { - "scope": "Pinned vLLM V1 branch-based diagnosis of queued requests, low scheduled-token counts and KV-pressure preemption, separating admission/token/input/allocation/connector-wait constraints.", - "prerequisites": "The deployed scheduler must be mapped to the pinned branches, with effective configuration and a bounded request/step trace; another revision requires re-establishing that correspondence.", - "input_contract": "Deployed revision, scheduler class/configuration, connector role, KV layout, policy and speculative/encoder/concurrent-batch features; per-step queues, tokens, both budgets, allocation results, victims, transfer/free state and timing.", - "produced_artifact": "A per-step accounting ledger, observed limiting branch and competing explanation, missing instrumentation and one conditional configuration/progress hypothesis with predicted effects and trade-offs.", - "failure_modes": "Inferring memory exhaustion from a queue or high KV usage, conflating token/input budgets, ignoring pending frees or reading async token accounting as load completion can attribute admission stalls to the wrong branch.", - "refusal_point": "When required state is missing, leave the diagnosis unresolved and request the smallest trace; escalate allocation or connector behavior outside the readset and stop before live mutation or performance claims." - }, - "recommendations": [ - "Trace the first actual blocking or skip branch and record running versus admission caps and token versus input budgets separately.", - "Inspect deferred frees and request-free eligibility before blaming preemption victim selection, then reconcile restored budgets for an already-scheduled victim.", - "Reconstruct waiting admission independently, including queue precedence and async prefix states; change no setting without a branch-specific hypothesis." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/diffusers-invocation-state.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/diffusers-invocation-state.json deleted file mode 100644 index 9f2c08b76..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/diffusers-invocation-state.json +++ /dev/null @@ -1,118 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.diffusers_invocation_state", - "title": "diffusers-invocation-state", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Trace Diffusers pipeline-wrapper and scheduler invocation state when repeated calls, from_pipe reuse or interruption produce unexpected schedules or retained request state; scoped to FluxPipeline and FlowMatchEulerDiscreteScheduler source evidence.", - "tags": [ - "diffusers", - "flux", - "scheduler", - "state-ownership", - "interruption" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md" - }, - "checkout_ref": "checkout:diffusers-invocation-state", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md": "7581b56336d07213a87d01477793ae1a8acf0b7cf876fce36c9a761ee32c1ff4", - "src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/metadata.json": "66f8d20c1644aba964adfff3b439bfd3ecd3875885191e6ce10721bc50877b1a", - "src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/ownership.md": "40688cc44183027efbf2fca8d3548c299cdf2e33fef54f76ab3549353eb9bd1b", - "src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/source-map.md": "10562a75683bd9bd07ec2e37a1cbd0f93f2534d6f85b55a566adb2d97485215f", - "src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/validation.md": "5a0b854b87d091d7d505b0bdd5c03330052507a851ab6edf35b93a6675f2d4f7" - }, - "skill_metadata": { - "name": "diffusers-invocation-state", - "description": "Trace Diffusers pipeline-wrapper and scheduler invocation state when repeated calls, from_pipe reuse or interruption produce unexpected schedules or retained request state; scoped to FluxPipeline and FlowMatchEulerDiscreteScheduler source evidence.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "Apache-2.0", - "tags": [ - "diffusers", - "flux", - "scheduler", - "state-ownership", - "interruption" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/huggingface/diffusers", - "commit": "aea7b70be0f594bcdc15a6f2677ee2ca85e49e8e", - "paths": [ - "src/diffusers/pipelines/pipeline_utils.py", - "src/diffusers/pipelines/flux/pipeline_flux.py", - "src/diffusers/schedulers/scheduling_flow_match_euler_discrete.py", - "tests/pipelines/flux/test_pipeline_flux.py", - "setup.py", - "LICENSE" - ], - "readset": "references/source-map.md" - } - ], - "applicability": { - "scope": "FluxPipeline and FlowMatchEulerDiscreteScheduler invocation ownership at this source pin; inspect the actual caller contract.", - "prerequisites": "Access to the pinned ownership path and an identified caller admission/serialization contract; other pipeline or scheduler classes need their own contracts.", - "input_contract": "Diffusers/PyTorch revisions, pipeline/scheduler classes and object identities, construction/from_pipe path, actual components, scheduler config and resolved timesteps/sigmas, shape/dtype, callback/interrupt timing, output/cleanup path and overlapping intervals.", - "produced_artifact": "An object-identity/ownership map and call-to-schedule-reset/step/interrupt/output/cleanup trace, the caller admission contract, one next check and actual evidence status.", - "failure_modes": "Assuming from_pipe deep-copies scheduler state, passing an enumeration index instead of a scheduler timestep, or treating cooperative iteration skipping as kernel cancellation/exception cleanup can misidentify invocation ownership.", - "refusal_point": "Stop when the ownership question is resolved or a required fixture/backend is absent; do not infer service-wide race or serialization behavior from this readset." - }, - "limitations": [ - "Source-derived candidate; no installed API, native scheduler, model, concurrency or service validation.", - "Supplied manifest verifies a selected pipeline_utils Git blob, not the whole source tree.", - "Other pipelines and schedulers require separate contracts; common indexing is not backend qualification." - ], - "provenance_readsets": [ - "references/source-map.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Check wrapper and scheduler identities explicitly and follow the effective schedule reset and step-index progression.", - "Trace interruption through its loop-boundary consumer and normal decode/cleanup tail while preserving the caller's existing shared-state serialization.", - "Use a real scheduler CPU check only for its scheduler cycle; keep pipeline reuse/interruption and serving claims tied to matching fixtures and execution evidence." - ] - }, - "applicability": { - "scope": "FluxPipeline and FlowMatchEulerDiscreteScheduler invocation ownership at this source pin; inspect the actual caller contract.", - "prerequisites": "Access to the pinned ownership path and an identified caller admission/serialization contract; other pipeline or scheduler classes need their own contracts.", - "input_contract": "Diffusers/PyTorch revisions, pipeline/scheduler classes and object identities, construction/from_pipe path, actual components, scheduler config and resolved timesteps/sigmas, shape/dtype, callback/interrupt timing, output/cleanup path and overlapping intervals.", - "produced_artifact": "An object-identity/ownership map and call-to-schedule-reset/step/interrupt/output/cleanup trace, the caller admission contract, one next check and actual evidence status.", - "failure_modes": "Assuming from_pipe deep-copies scheduler state, passing an enumeration index instead of a scheduler timestep, or treating cooperative iteration skipping as kernel cancellation/exception cleanup can misidentify invocation ownership.", - "refusal_point": "Stop when the ownership question is resolved or a required fixture/backend is absent; do not infer service-wide race or serialization behavior from this readset." - }, - "recommendations": [ - "Check wrapper and scheduler identities explicitly and follow the effective schedule reset and step-index progression.", - "Trace interruption through its loop-boundary consumer and normal decode/cleanup tail while preserving the caller's existing shared-state serialization.", - "Use a real scheduler CPU check only for its scheduler cycle; keep pipeline reuse/interruption and serving claims tied to matching fixtures and execution evidence." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no installed API, native scheduler, model, concurrency or service validation.", - "Supplied manifest verifies a selected pipeline_utils Git blob, not the whole source tree.", - "Other pipelines and schedulers require separate contracts; common indexing is not backend qualification." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/draft-acceptance-issue.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/draft-acceptance-issue.json deleted file mode 100644 index 24bf2a933..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/draft-acceptance-issue.json +++ /dev/null @@ -1,105 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.draft_acceptance_issue", - "title": "draft-acceptance-issue", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Draft a campaign acceptance issue with checklist, RunSpec, evidence paths, and placeholders for baseline/candidate SHAs.", - "tags": [ - "acceptance", - "issue", - "runspec", - "template", - "campaign" - ], - "origin": [ - "first-party" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/workflows/draft-acceptance-issue/SKILL.md" - }, - "checkout_ref": "checkout:draft-acceptance-issue", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/SKILL.md": "e22c427a27ab7f41e8627fccba1c65717915af6d7f7c04d91463eb1ba1b4c53f", - "src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/metadata.json": "57473329e01f341881cc298c916b70ea8b718bf7b516d3428227c2682f172fe0" - }, - "skill_metadata": { - "name": "draft-acceptance-issue", - "description": "Draft a campaign acceptance issue with checklist, RunSpec, evidence paths, and placeholders for baseline/candidate SHAs.", - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "acceptance", - "issue", - "runspec", - "template", - "campaign" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Opening a new acceptance campaign in an RSI-style repository, before any baseline or candidate run", - "prerequisites": "A chosen campaign subject and an empty evidence// tree copied from evidence/_template/; SHAs may still be placeholders", - "input_contract": "A campaign id of the form {model-or-scenario}/{device}x{count}, any user-supplied criteria or accepted plan, and available workload/baseline/calibration evidence for deriving missing targets", - "produced_artifact": "An acceptance issue body with campaign id, pinned baseline and candidate SHAs, RunSpec, criteria source and derivation including gaps and freeze authority, gates, evidence paths, initial verdict and a downstream citation blurb", - "failure_modes": [ - "Gates written after the search loop has already run", - "Campaign SHAs committed into recipes/, benchmarks/, skills or workflow defaults", - "PASS claimed without a validator/ freeze and a results/ rationale" - ], - "refusal_point": "Refuses to state a verdict other than INCONCLUSIVE while the validator freeze or results rationale is absent" - }, - "recommendations": [ - "Use user-supplied criteria; when absent, follow the performance-ceiling-analysis acceptance procedure to formulate targets, preserving partial requirements and identifying derived fields and evidence gaps.", - "Write the gates in plan/ before any search loop; a gate chosen afterwards cannot reject the candidate honestly.", - "Use the and placeholders until the campaign fixes them, and keep the real values in the issue.", - "Set the initial verdict to INCONCLUSIVE so the absence of evidence is recorded rather than assumed away." - ], - "limitations": [ - "Drafts an issue only; it freezes no gate and runs no measurement.", - "The placeholder SHA convention depends on the campaign owner filling them in later." - ], - "provenance_readsets": [], - "searchspace_not_applicable": "SKILL.md §§ Steps and Output prepare an acceptance issue, RunSpec, frozen gates and evidence paths; device identifiers are campaign metadata rather than a hardware-resource mechanism." - }, - "applicability": { - "scope": "Opening a new acceptance campaign in an RSI-style repository, before any baseline or candidate run", - "prerequisites": "A chosen campaign subject and an empty evidence// tree copied from evidence/_template/; SHAs may still be placeholders", - "input_contract": "A campaign id of the form {model-or-scenario}/{device}x{count}, any user-supplied criteria or accepted plan, and available workload/baseline/calibration evidence for deriving missing targets", - "produced_artifact": "An acceptance issue body with campaign id, pinned baseline and candidate SHAs, RunSpec, criteria source and derivation including gaps and freeze authority, gates, evidence paths, initial verdict and a downstream citation blurb", - "failure_modes": [ - "Gates written after the search loop has already run", - "Campaign SHAs committed into recipes/, benchmarks/, skills or workflow defaults", - "PASS claimed without a validator/ freeze and a results/ rationale" - ], - "refusal_point": "Refuses to state a verdict other than INCONCLUSIVE while the validator freeze or results rationale is absent" - }, - "recommendations": [ - "Use user-supplied criteria; when absent, follow the performance-ceiling-analysis acceptance procedure to formulate targets, preserving partial requirements and identifying derived fields and evidence gaps.", - "Write the gates in plan/ before any search loop; a gate chosen afterwards cannot reject the candidate honestly.", - "Use the and placeholders until the campaign fixes them, and keep the real values in the issue.", - "Set the initial verdict to INCONCLUSIVE so the absence of evidence is recorded rather than assumed away." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Drafts an issue only; it freezes no gate and runs no measurement.", - "The placeholder SHA convention depends on the campaign owner filling them in later." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dwarfstar.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dwarfstar.json deleted file mode 100644 index 886f71cb4..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/dwarfstar.json +++ /dev/null @@ -1,180 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.dwarfstar", - "title": "dwarfstar", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Investigate DwarfStar model fit, expert SSD streaming, session and vision cache correctness, speculative sampling and distributed execution using pinned source contracts and explicit native verification boundaries.", - "tags": [ - "dwarfstar", - "ds4", - "moe", - "ssd-streaming", - "session-state", - "kv-cache", - "speculation", - "tensor-parallelism" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/dwarfstar/SKILL.md" - }, - "checkout_ref": "checkout:dwarfstar", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/LICENSE": "4a8a1358f5b048627d71e4988de0bafd386f4988991929629e6cb9e29b7b9b2f", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/SKILL.md": "a04a252ecdcd4710a3273595f4bd1f3eb8e24d5bb1bfe005f058a78388254dc9", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/metadata.json": "c883f6bc05e585f072cf03f734b7a296a65842c7921d3afcfe89e1977405fbd2", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/repo-provenance.md": "7d72f9c85ee60fe003f5f5a585b52e9989ebcedca625b584fa39a976d4433998", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/repo-routing-metadata.json": "8e195b420b67151932af5f435c284f49677cc7863e650c83cfa9e762fd00db0b", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/source-identity.json": "a38fe07d003accd83b7626d7e189f7b49b4b73e7a223e4d441b0e33add5d9c91", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/scripts/request_template.py": "62e4cde002f2aebeaddd63e6e1f04fc727c271d2cbdaa00856158baadee5340b", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/SKILL.md": "3b8eb25f6b8e4729b48cce5619daa24422bee251f119ea7f9cc1faf0097ec6bb", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/references/troubleshooting.md": "cd832ab2a420490524251a042c0c83c2f75bbfdc12e914b91da8f91edf6860e9", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/references/verification-contract.md": "2801f6683c2974d1cfa5b86c6268fd33f175f1cbcc9ed85c932e9b3504fac3e1", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/SKILL.md": "ad1d34d429baaa41fb72c94ff4e554b26b2f5b701cc4deed03f1d066c7d3fc52", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/references/memory-ledger.md": "765915ecd3a318cc7908658d5a79bbdff615fc24409501febcb7ff8af65a1050", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/references/troubleshooting.md": "42e4785b7d3c1375efa324583897327f28ef76c0b7788092da92095bddeb1440", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/SKILL.md": "71c27212847146376d72e6cc6d22abe2f7eed56402c5917cd7dfcda0b2aa5982", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/references/state-contracts.md": "8f74a07c947021e3069468268f6b5aa2548a17db518c5c0505b84dde73a1f31c", - "src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/references/troubleshooting.md": "c9eebe4109d418c4dfb9bb2425c145ebc1a52da9b665e0556149bc31a442e2fc" - }, - "skill_metadata": { - "name": "dwarfstar", - "description": "Investigate DwarfStar model fit, expert SSD streaming, session and vision cache correctness, speculative sampling and distributed execution using pinned source contracts and explicit native verification boundaries.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "partial_cpu_only", - "tags": [ - "dwarfstar", - "ds4", - "moe", - "ssd-streaming", - "session-state", - "kv-cache", - "speculation", - "tensor-parallelism" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/antirez/ds4", - "commit": "ee7e8405232c8971000cc7ba2b945b00bf6d20d3", - "paths": [ - "README.md", - "LICENSE", - "Makefile", - "ds4.h", - "ds4.c", - "ds4_ssd.c", - "ds4_ssd.h", - "ds4_kvstore.c", - "ds4_kvstore.h", - "ds4_server.c", - "ds4_tp.c", - "ds4_tp.h", - "docs/SSD_STREAMING.md", - "docs/SERVER.md", - "docs/SPECULATIVE_DECODING.md", - "docs/DISTRIBUTED.md", - "docs/CUDA_MULTI_GPU.md", - "docs/TESTING.md", - "tests/test_session_state.c", - "tests/test_sampling.c", - "tests/test_tp_commands.c", - "tests/test_engine_mgpu_placement.c", - "ds4_cuda.cu", - "QA_BEFORE_RELEASES.md" - ], - "readset": "references/source-identity.json", - "license": "MIT" - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Portable instructions executed by compatible agent with native C adaptation; no DisCo CLI/model run or managed import." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "User-approved native C adaptation and partial drafting; manual stable-ID subskill orchestration and integrated create/verify stages. Required-backend blocks disable automatic admission." - }, - "applicability": { - "scope": "Pinned DwarfStar source investigation and experiment planning; not arbitrary GGUF support or cross-backend qualification.", - "prerequisites": "Access to the pinned native-source contract and relevant memory/state/qualification route; native verification needs a compatible build, matching hardware/weights and task-specific execution authority.", - "input_contract": "Engine revision/patches, actual model artifact/representation, selected backend/build and topology, memory/storage budget, context/chunk/session count, image inputs, sampling/speculation policy, cache state and quality/service requirements.", - "produced_artifact": "A bounded executing-tuple report with observed signal, source-backed mechanism, alternate explanation, proposed falsifier, actual check outcome and unresolved backend/model/service gap.", - "failure_modes": "Inferring model support from flags, aliases, file mapping or quantization names can select the wrong graph; treating CPU bookkeeping or parser checks as native backend/service qualification can overstate the result.", - "refusal_point": "Stop at unavailable hardware/weights, unsafe exposure or the authorized budget; retain memory-guard refusals rather than disabling the guard to claim fit." - }, - "limitations": [ - "Metal/CUDA/ROCm inference, actual model output, SSD runtime I/O, GPU recurrent rollback, physical distributed behavior, service quality/performance and skill efficacy remain unverified.", - "CPU build/help and selected bookkeeping tests do not clear BLOCKED_REQUIRED_BACKEND.", - "NO_LICENSE is the actual unauthenticated GitHub CLI resolver fallback, not a legal conclusion; pinned source MIT notice is retained.", - "Only selected source bodies were inspected; full kernels, conversion/imatrix tools and every workflow are not covered." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Choose the memory, serving-state or qualification route by the observed symptom and identify the owner, validity key and release/rollback boundary.", - "Freeze comparison controls, test the smallest invariant first and retain fallback messages, cancellations, refusals and unavailable fixtures.", - "Base model support on the actual loaded artifact and graph; use native backend evidence for model, tensor-parallel or service conclusions." - ] - }, - "applicability": { - "scope": "Pinned DwarfStar source investigation and experiment planning; not arbitrary GGUF support or cross-backend qualification.", - "prerequisites": "Access to the pinned native-source contract and relevant memory/state/qualification route; native verification needs a compatible build, matching hardware/weights and task-specific execution authority.", - "input_contract": "Engine revision/patches, actual model artifact/representation, selected backend/build and topology, memory/storage budget, context/chunk/session count, image inputs, sampling/speculation policy, cache state and quality/service requirements.", - "produced_artifact": "A bounded executing-tuple report with observed signal, source-backed mechanism, alternate explanation, proposed falsifier, actual check outcome and unresolved backend/model/service gap.", - "failure_modes": "Inferring model support from flags, aliases, file mapping or quantization names can select the wrong graph; treating CPU bookkeeping or parser checks as native backend/service qualification can overstate the result.", - "refusal_point": "Stop at unavailable hardware/weights, unsafe exposure or the authorized budget; retain memory-guard refusals rather than disabling the guard to claim fit." - }, - "recommendations": [ - "Choose the memory, serving-state or qualification route by the observed symptom and identify the owner, validity key and release/rollback boundary.", - "Freeze comparison controls, test the smallest invariant first and retain fallback messages, cancellations, refusals and unavailable fixtures.", - "Base model support on the actual loaded artifact and graph; use native backend evidence for model, tensor-parallel or service conclusions." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Metal/CUDA/ROCm inference, actual model output, SSD runtime I/O, GPU recurrent rollback, physical distributed behavior, service quality/performance and skill efficacy remain unverified.", - "CPU build/help and selected bookkeeping tests do not clear BLOCKED_REQUIRED_BACKEND.", - "NO_LICENSE is the actual unauthenticated GitHub CLI resolver fallback, not a legal conclusion; pinned source MIT notice is retained.", - "Only selected source bodies were inspected; full kernels, conversion/imatrix tools and every workflow are not covered." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/evaluate-elastic-serving-placement.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/evaluate-elastic-serving-placement.json deleted file mode 100644 index 4d2ac6984..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/evaluate-elastic-serving-placement.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.evaluate_elastic_serving_placement", - "title": "evaluate-elastic-serving-placement", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including state-copy, reservation, recomputation, interruption, and priority costs. Use for cross-instance memory imbalance or highly variable long-context serving, not connector wiring or live scaling.", - "tags": [], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md" - }, - "checkout_ref": "checkout:evaluate-elastic-serving-placement", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md": "82d0099d3c33706f46c657d4fa7f76d3d486e019b36760d1de001921328a60cc", - "src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/metadata.json": "091a3b95d1a8996b5ad917e7948efb4ed598e31c7163709377f8e9199cebfe26", - "src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/references/source-notes.md": "ea416831fd7d0874e06d0a76fb0c7196df53072b717673a0cdf635db6bfd917b" - }, - "skill_metadata": { - "name": "evaluate-elastic-serving-placement", - "description": "Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including state-copy, reservation, recomputation, interruption, and priority costs. Use for cross-instance memory imbalance or highly variable long-context serving, not connector wiring or live scaling.", - "status": "candidate", - "empirical_validation": "not_run", - "source_pins": "llumnix-2406.03243v1-and-loongserve-2404.09526v2", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Comparing migration of a running LLM request between compatible model replicas with changing its sequence-parallel execution group, including state-copy, reservation, recomputation, interruption and priority costs, for cross-instance memory imbalance or highly variable long-context serving; not connector wiring or live scaling", - "prerequisites": "The relevant Llumnix or LoongServe branch and its readset in the packaged source notes are read; moving state, altering admission or priority, scaling replicas, changing parallelism, running experiments or editing frameworks needs separate authorization", - "input_contract": "Exact model and cache identity and format; weight residency; request phase, context and generated length; source and destination KV bytes; physical free/reserved memory and virtual accounting; batch state; current parallel group; queue and service times; actual topology and transport for each path; the target SLO statistic and affected priority or tenant cohorts", - "produced_artifact": "The smallest justified branch with missing capability or evidence, expected saved work or wait, added state, traffic and replica cost, affected SLO and cohorts, counter-signals, the proposed next gate and a future verification plan; an evidence-based recommendation, not a validated deployment rule", - "failure_modes": [ - "Model-weight repartitioning, pipeline configuration, instance migration and sequence sharding are treated as interchangeable uses of parallelism", - "Aggregate free bytes are taken to mean a single blocked request fits, conflating internal KV-block slack with cross-instance fragmentation", - "Llumnix's 20-30 ms measured downtime is substituted for total transfer time", - "Virtual headroom is counted as extra memory or as a tenant-fairness guarantee", - "A zero-overhead transition label is read as every group change being free, or a token sum hides length-sensitive attention work", - "Ongoing distributed-attention communication goes uncharged because existing KV does not migrate" - ], - "refusal_point": "Withholds the candidate when every destination is full, pre-copy catch-up is unproven, model or cache compatibility is unknown, or the measured work saved does not exceed exposed costs; current-code review and target-system tests remain a later authorization gate" - }, - "recommendations": [ - "Decide first whether the question is replica relocation, a sequence-parallel group change, or prefix reuse and phase pools, which evaluate-kv-cache-transfer-gates and choose-prefill-decode-schedule cover when available.", - "Name each transfer path separately; only inter-node transport is Network.", - "Keep a feasible static, no-migration baseline in any later comparison." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Comparing migration of a running LLM request between compatible model replicas with changing its sequence-parallel execution group, including state-copy, reservation, recomputation, interruption and priority costs, for cross-instance memory imbalance or highly variable long-context serving; not connector wiring or live scaling", - "prerequisites": "The relevant Llumnix or LoongServe branch and its readset in the packaged source notes are read; moving state, altering admission or priority, scaling replicas, changing parallelism, running experiments or editing frameworks needs separate authorization", - "input_contract": "Exact model and cache identity and format; weight residency; request phase, context and generated length; source and destination KV bytes; physical free/reserved memory and virtual accounting; batch state; current parallel group; queue and service times; actual topology and transport for each path; the target SLO statistic and affected priority or tenant cohorts", - "produced_artifact": "The smallest justified branch with missing capability or evidence, expected saved work or wait, added state, traffic and replica cost, affected SLO and cohorts, counter-signals, the proposed next gate and a future verification plan; an evidence-based recommendation, not a validated deployment rule", - "failure_modes": [ - "Model-weight repartitioning, pipeline configuration, instance migration and sequence sharding are treated as interchangeable uses of parallelism", - "Aggregate free bytes are taken to mean a single blocked request fits, conflating internal KV-block slack with cross-instance fragmentation", - "Llumnix's 20-30 ms measured downtime is substituted for total transfer time", - "Virtual headroom is counted as extra memory or as a tenant-fairness guarantee", - "A zero-overhead transition label is read as every group change being free, or a token sum hides length-sensitive attention work", - "Ongoing distributed-attention communication goes uncharged because existing KV does not migrate" - ], - "refusal_point": "Withholds the candidate when every destination is full, pre-copy catch-up is unproven, model or cache compatibility is unknown, or the measured work saved does not exceed exposed costs; current-code review and target-system tests remain a later authorization gate" - }, - "recommendations": [ - "Decide first whether the question is replica relocation, a sequence-parallel group change, or prefix reuse and phase pools, which evaluate-kv-cache-transfer-gates and choose-prefill-decode-schedule cover when available.", - "Name each transfer path separately; only inter-node transport is Network.", - "Keep a feasible static, no-migration baseline in any later comparison." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/generation-cache-lifecycle.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/generation-cache-lifecycle.json deleted file mode 100644 index 017b43134..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/generation-cache-lifecycle.json +++ /dev/null @@ -1,120 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.generation_cache_lifecycle", - "title": "generation-cache-lifecycle", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Trace MLX-LM prefill, generation batching, prompt-cache reuse and cooperative stopping when request state, cache rows or retained prefixes become inconsistent.", - "tags": [ - "mlx-lm", - "generation", - "batching", - "kv-cache", - "cancellation" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md" - }, - "checkout_ref": "checkout:generation-cache-lifecycle", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md": "bf1177cb9985c12a9bf11edb64771018e1e314c4713920de8fd4a989ff70115f", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/metadata.json": "c2e30a2a3365dd27cac7916dc26aa356f1277d07ebee891c173ee37b77111fa1", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/cache-stop.md": "bba6b5c73261065309cbcf11b1e591eef48916f0c79ceaaf74d98a9fa96b0992", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/generation-batch.md": "f16d3c57d45e63bcaffd3871aa41c76972ee11f7dbf1e12b45d9b170303cad9b", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/source-map.md": "91d26181f246307f2113f94b12ac9587d689b654dcd216e9f14b11cf5ebb650c", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/validation.md": "73b5dd347017baba519ecc85415aead9993c1751d116b9ba4f07689f58c5f783" - }, - "skill_metadata": { - "name": "generation-cache-lifecycle", - "description": "Trace MLX-LM prefill, generation batching, prompt-cache reuse and cooperative stopping when request state, cache rows or retained prefixes become inconsistent.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "MIT", - "tags": [ - "mlx-lm", - "generation", - "batching", - "kv-cache", - "cancellation" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ml-explore/mlx-lm", - "commit": "ab7f7048dc341ced2d84b1887eeb17d60b0d1b02", - "paths": [ - "mlx_lm/generate.py", - "mlx_lm/models/cache.py", - "mlx_lm/server.py", - "tests/test_generate.py", - "tests/test_prompt_cache.py", - "LICENSE", - "pyproject.toml", - "mlx_lm/cache_prompt.py", - ".github/workflows/pull_request.yml" - ], - "readset": "references/source-map.md" - } - ], - "applicability": { - "scope": "MLX-LM generation, batch-row and prefix-cache diagnosis; model/backend/runtime compatibility requires separate qualification.", - "prerequisites": "Access to the independently pinned MLX-LM generation/cache contracts and the selected entry path; model, backend and runtime compatibility is separately established for any native claim.", - "input_contract": "MLX-LM/MLX revisions and builds, backend/device, model/tokenizer/adapter/draft identities, entry point, tokenized prompts, cache classes, batching/prefill limits, per-request sampling/processors/stops, seed and KV quantization; failure traces include UIDs, stages, lengths, terminal reason and next-request result.", - "produced_artifact": "An entry-path decision, UID/stage/cache-row table, prefix identity and cache-class assumptions, terminal/cleanup trace and actual evidence status with a targeted verification plan.", - "failure_modes": "Ignoring server batching gates, filtering only some per-request row state, reusing a text-only prefix key or treating a stop flag as immediate MLX work cancellation can corrupt lifecycle reasoning.", - "refusal_point": "Stop at the first unsupported contract or missing required fixture/backend; retain a targeted plan without inferring paged allocation, radix sharing or instantaneous prefill interruption." - }, - "limitations": [ - "Source-derived candidate; no local native-backend, model, service or efficacy validation.", - "Static content checks do not promote this candidate or establish installed API conformance." - ], - "provenance_readsets": [ - "references/source-map.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve the library versus server entry and its batching gates before transferring BatchGenerator behavior.", - "Keep UID, caches, tokens, samplers, processors, lengths, stop matchers and pending outputs aligned under the same removal selection, including prompt-stage requests.", - "Check exact token/model/adapter/draft identity and cache-type copy/trim semantics, then trace cooperative removal through survivor and next-request behavior." - ] - }, - "applicability": { - "scope": "MLX-LM generation, batch-row and prefix-cache diagnosis; model/backend/runtime compatibility requires separate qualification.", - "prerequisites": "Access to the independently pinned MLX-LM generation/cache contracts and the selected entry path; model, backend and runtime compatibility is separately established for any native claim.", - "input_contract": "MLX-LM/MLX revisions and builds, backend/device, model/tokenizer/adapter/draft identities, entry point, tokenized prompts, cache classes, batching/prefill limits, per-request sampling/processors/stops, seed and KV quantization; failure traces include UIDs, stages, lengths, terminal reason and next-request result.", - "produced_artifact": "An entry-path decision, UID/stage/cache-row table, prefix identity and cache-class assumptions, terminal/cleanup trace and actual evidence status with a targeted verification plan.", - "failure_modes": "Ignoring server batching gates, filtering only some per-request row state, reusing a text-only prefix key or treating a stop flag as immediate MLX work cancellation can corrupt lifecycle reasoning.", - "refusal_point": "Stop at the first unsupported contract or missing required fixture/backend; retain a targeted plan without inferring paged allocation, radix sharing or instantaneous prefill interruption." - }, - "recommendations": [ - "Resolve the library versus server entry and its batching gates before transferring BatchGenerator behavior.", - "Keep UID, caches, tokens, samplers, processors, lengths, stop matchers and pending outputs aligned under the same removal selection, including prompt-stage requests.", - "Check exact token/model/adapter/draft identity and cache-type copy/trim semantics, then trace cooperative removal through survivor and next-request behavior." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local native-backend, model, service or efficacy validation.", - "Static content checks do not promote this candidate or establish installed API conformance." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/host-memory-management.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/host-memory-management.json deleted file mode 100644 index e0d3dea55..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/host-memory-management.json +++ /dev/null @@ -1,187 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.host_memory_management", - "title": "host-memory-management", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit pinned host buffers and asynchronous CUDA transfers: distinguish copy ordering, host mutation/reuse and device allocator lifetime; use bounded PyTorch source evidence, a reusable staging-slot example and a hardware validation recipe. Also route Ascend native descriptor retention, Python preprocessing and NUMA placement to their scoped sources.", - "tags": [ - "host", - "cpu", - "memory", - "pinned-memory", - "h2d", - "gil", - "numa", - "inference", - "ascend", - "descriptor-pool", - "cuda-streams", - "allocator-lifetime", - "buffer-reuse" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/host/host-memory-management/SKILL.md" - }, - "checkout_ref": "checkout:host-memory-management", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/SKILL.md": "4c8db9f603cd581d002e8f41da65af9cbd03699be05c4965a55de47e044e4dd8", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/metadata.json": "9fe5988b136da9aeff9d00b0c6c607cac00b4071efaf637d287a350e5d4fcafb", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/cpython-PSF-LICENSE.md": "33f1a470702f459c160751bf3188b3fa7462fc761a006e03e80ef3f0a7a01ed6", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/cpython-free-threading-readset.json": "46336623b3c573fad97d4c686d9836baab807adf10f4bea05352d7a46236e004", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/legacy-source-notes.md": "bb1c0fb7a4d1a0301ca9e699e733224f3dcf0ded3cfe13e612e5c04d48a8cab5", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-LICENSE.md": "a8a2c6b67cf8a597ffaa5c82380a6bd1b82629c0417e00673d8013d662eb500a", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-host-copy-readset.json": "881258c97edfee0a89f2392c2ff61cd7fe37ff8344f18620f281e841fb92cbc8", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-host-copy.md": "fa9da5c39d37ebe44e5a336a7b88d7aece6a0f118ec47cb38271e29925e11117", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-LICENSE.md": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload-readset.json": "cc6c2011be746002ba3ff96ccefab1809863748ddd17eb102b75e7b0e134fe55", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload.md": "6c2a72b5ba3708a01d229e0576570837a617e6a42dbb0b5312a26a65a6c31542" - }, - "skill_metadata": { - "name": "host-memory-management", - "description": "Audit pinned host buffers and asynchronous CUDA transfers: distinguish copy ordering, host mutation/reuse and device allocator lifetime; use bounded PyTorch source evidence, a reusable staging-slot example and a hardware validation recipe. Also route Ascend native descriptor retention, Python preprocessing and NUMA placement to their scoped sources.", - "tags": [ - "host", - "cpu", - "memory", - "pinned-memory", - "h2d", - "gil", - "numa", - "inference", - "ascend", - "descriptor-pool", - "cuda-streams", - "allocator-lifetime", - "buffer-reuse" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/vllm-ascend", - "commit": "8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf", - "paths": [ - "LICENSE", - "requirements.txt", - "vllm_ascend/distributed/kv_transfer/__init__.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/npu.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/cpu_npu.py", - "csrc/torch_binding.cpp", - "tests/ut/kv_offload/test_native_cpu_offload.py" - ], - "readset": "references/vllm-ascend-native-offload-readset.json" - }, - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "ba56102387ef21a3b04b357e5b183d48f0afefc7", - "paths": [ - "aten/src/ATen/core/CachingHostAllocator.h", - "aten/src/ATen/cuda/CachingHostAllocator.cpp", - "aten/src/ATen/native/cuda/Copy.cu", - "aten/src/ATen/native/Memory.cpp", - "c10/cuda/CUDAFunctions.h", - "docs/source/notes/cuda.rst", - "torch/cuda/streams.py", - "torch/_tensor_docs.py", - "LICENSE" - ], - "source_channel": "official_repository", - "inspected_scope": "selected_implementation_and_official_documentation", - "license": "BSD-3-Clause", - "readset": "references/pytorch-host-copy-readset.json" - }, - { - "type": "git", - "url": "https://github.com/python/cpython", - "commit": "60403a5409ff2c3f3b07dd2ca91a7a3e096839c7", - "paths": [ - "Doc/howto/free-threading-python.rst", - "LICENSE" - ], - "source_channel": "official_repository", - "inspected_scope": "official_documentation_only", - "license": "PSF-2.0", - "readset": "references/cpython-free-threading-readset.json" - } - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Legacy live source links remain unnormalized in references/legacy-source-notes.md; only the declared immutable source slices are normalized.", - "Added Ascend case is limited to normal native single-tier CPU descriptor-pool ownership; vLLM compatibility and CANN retention/teardown contracts remain unqualified.", - "PyTorch source-derived CUDA transfer/host allocator candidate; hardware, graph implementation, custom allocators, NUMA and service efficacy are not qualified.", - "CPython v3.13.0 documentation corrects GIL detection only; no free-threaded interpreter or inference-stack compatibility test was run." - ], - "provenance_readsets": [ - "references/vllm-ascend-native-offload.md", - "references/vllm-ascend-native-offload-readset.json", - "references/pytorch-host-copy-readset.json", - "references/cpython-free-threading-readset.json" - ], - "references": [ - "references/vllm-ascend-native-offload.md", - "references/vllm-ascend-native-offload-readset.json", - "references/vllm-ascend-LICENSE.md", - "references/pytorch-host-copy.md", - "references/pytorch-host-copy-readset.json", - "references/pytorch-LICENSE.md", - "references/cpython-free-threading-readset.json", - "references/cpython-PSF-LICENSE.md", - "references/legacy-source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Audit host preparation, pinned-host pressure and asynchronous CUDA-copy ownership/order/lifetime, with separately scoped routing for Ascend descriptor retention, Python preprocessing and NUMA placement.", - "input_contract": "Host buffer ownership and alias/mutation behavior, copy direction/dtype/layout, producer/consumer streams, live/in-flight/cache budget and the actual interpreter/placement context.", - "produced_artifact": "An ownership and dependency audit with bounded transfer candidates and a correctness-first validation plan separating enqueue, pinning, transfer, model and end-to-end cost.", - "failure_modes": "A stream wait does not prevent CPU mutation and record_stream does not create a data dependency; cached host storage may outlive Python scope, and the eager reusable-slot example is not capture-safe." - }, - "recommendations": [ - "Track copy-before-consumer order, CPU payload reuse and device storage lifetime as three independent obligations; wait for D2H completion before CPU reads.", - "Compare pageable, per-request pinning and bounded reuse on the real workload while budgeting all live and in-flight slots, cache and preprocessing memory.", - "Keep the eager staging-slot preconditions explicit and validate ordering/lifetime under stress before performance; a small passing sample does not establish a hazardous pattern is safe." - ] - }, - "applicability": { - "scope": "Audit host preparation, pinned-host pressure and asynchronous CUDA-copy ownership/order/lifetime, with separately scoped routing for Ascend descriptor retention, Python preprocessing and NUMA placement.", - "input_contract": "Host buffer ownership and alias/mutation behavior, copy direction/dtype/layout, producer/consumer streams, live/in-flight/cache budget and the actual interpreter/placement context.", - "produced_artifact": "An ownership and dependency audit with bounded transfer candidates and a correctness-first validation plan separating enqueue, pinning, transfer, model and end-to-end cost.", - "failure_modes": "A stream wait does not prevent CPU mutation and record_stream does not create a data dependency; cached host storage may outlive Python scope, and the eager reusable-slot example is not capture-safe." - }, - "recommendations": [ - "Track copy-before-consumer order, CPU payload reuse and device storage lifetime as three independent obligations; wait for D2H completion before CPU reads.", - "Compare pageable, per-request pinning and bounded reuse on the real workload while budgeting all live and in-flight slots, cache and preprocessing memory.", - "Keep the eager staging-slot preconditions explicit and validate ordering/lifetime under stress before performance; a small passing sample does not establish a hazardous pattern is safe." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Legacy live source links remain unnormalized in references/legacy-source-notes.md; only the declared immutable source slices are normalized.", - "Added Ascend case is limited to normal native single-tier CPU descriptor-pool ownership; vLLM compatibility and CANN retention/teardown contracts remain unqualified.", - "PyTorch source-derived CUDA transfer/host allocator candidate; hardware, graph implementation, custom allocators, NUMA and service efficacy are not qualified.", - "CPython v3.13.0 documentation corrects GIL detection only; no free-threaded interpreter or inference-stack compatibility test was run." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/host-overhead-optimization.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/host-overhead-optimization.json deleted file mode 100644 index cd4258e0a..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/host-overhead-optimization.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.host_overhead_optimization", - "title": "host-overhead-optimization", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Profile and reduce CPU/host overhead in inference serving using line_profiler. Iterative profile-analyze-optimize-validate loop targeting Python execution in request preparation, scheduling, and model execution coordination.\n", - "tags": [ - "host", - "cpu", - "optimization", - "line-profiler", - "inference" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/host/host-overhead-optimization/SKILL.md" - }, - "checkout_ref": "checkout:host-overhead-optimization", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md": "36ef88826a32ac3147b869001004ded17a8d5a491e25bfe105005ba39ea9d1eb", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/metadata.json": "db3c63ca77a46e647a806e094ae662504768a958d537630a4d7bcabde35d6da3", - "src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md": "f452bf40132c9195d739f1c86b3aa18857138c7ed0f1861af5853d93b9955924" - }, - "skill_metadata": { - "name": "host-overhead-optimization", - "description": "Profile and reduce CPU/host overhead in inference serving using line_profiler. Iterative profile-analyze-optimize-validate loop targeting Python execution in request preparation, scheduling, and model execution coordination.\n", - "tags": [ - "host", - "cpu", - "optimization", - "line-profiler", - "inference" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Use line-level profiling to reduce exposed Python request-preparation, scheduling and execution-coordination overhead in a bounded serving optimization loop.", - "prerequisites": "Confirm host overhead is the limiting factor with the detection procedure before tuning; choose a representative workload and consistent CPU affinity for comparisons.", - "input_contract": "Detection/NVTX triage, qualified target function paths and line-profiler output, workload/affinity settings, related correctness tests and serving benchmark metrics.", - "produced_artifact": "Per-round hotspot/change/rollback and correctness records plus a final before/after benchmark report, failed attempts, remaining hotspots and scoped follow-up recommendations.", - "failure_modes": "Optimizing the default preparation function can miss a different regressing owner; caching/hoisting/preallocation can violate lifetime or invalidation, and a line-level speedup can leave serving throughput unchanged." - }, - "recommendations": [ - "Profile the actual top regressing function by absolute exposed time, drilling into dominant unprofiled subcalls instead of assuming _prepare_tp_inputs is the owner.", - "Prefer one small evidence-supported change with explicit ownership/lifetime/invalidation checks, run relevant correctness tests and revert failed or regressing candidates.", - "Re-profile identical workloads after each accepted change and compare hotspot time, total function time and serving metrics; reassess the next bottleneck and stay within the authorized budget." - ] - }, - "applicability": { - "scope": "Use line-level profiling to reduce exposed Python request-preparation, scheduling and execution-coordination overhead in a bounded serving optimization loop.", - "prerequisites": "Confirm host overhead is the limiting factor with the detection procedure before tuning; choose a representative workload and consistent CPU affinity for comparisons.", - "input_contract": "Detection/NVTX triage, qualified target function paths and line-profiler output, workload/affinity settings, related correctness tests and serving benchmark metrics.", - "produced_artifact": "Per-round hotspot/change/rollback and correctness records plus a final before/after benchmark report, failed attempts, remaining hotspots and scoped follow-up recommendations.", - "failure_modes": "Optimizing the default preparation function can miss a different regressing owner; caching/hoisting/preallocation can violate lifetime or invalidation, and a line-level speedup can leave serving throughput unchanged." - }, - "recommendations": [ - "Profile the actual top regressing function by absolute exposed time, drilling into dominant unprofiled subcalls instead of assuming _prepare_tp_inputs is the owner.", - "Prefer one small evidence-supported change with explicit ownership/lifetime/invalidation checks, run relevant correctness tests and revert failed or regressing candidates.", - "Re-profile identical workloads after each accepted change and compare hotspot time, total function time and serving metrics; reassess the next bottleneck and stay within the authorized budget." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/husky.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/husky.json deleted file mode 100644 index f15954a81..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/husky.json +++ /dev/null @@ -1,143 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.husky", - "title": "husky", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Review Husky's official-document claims as research hypotheses, separating author reports, implementation evidence, negative results, future directions and the missing artifacts needed for independent qualification.", - "tags": [ - "husky", - "document-research", - "evidence-qualification", - "negative-results" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/husky/SKILL.md" - }, - "checkout_ref": "checkout:husky", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/husky/SKILL.md": "ac86afdabf9adffb684cb635971934438724bdfde8628ec549727ea23c40cf3b", - "src/inferrsi/knowledge/data/skill_library/repositories/husky/metadata.json": "b1cb0a5be02d1170125bfc3e16c044b9c1463a611a14d36fac40638524fc2e52", - "src/inferrsi/knowledge/data/skill_library/repositories/husky/references/author-report.md": "4c1351ca42b2535a7c03c4da87706040650ef10d86eadcdb496e881a147fc0f7", - "src/inferrsi/knowledge/data/skill_library/repositories/husky/references/repo-provenance.md": "37de912e2e2a36b44233560e6a8368b994428bff587ed4f04b1cb5fbaba0a9e7", - "src/inferrsi/knowledge/data/skill_library/repositories/husky/references/source-availability.json": "d3523e53c89d4aee5bfaa17db84501aa4c4eddd59bcee486df5d48d3aa48173d", - "src/inferrsi/knowledge/data/skill_library/repositories/husky/references/troubleshooting.md": "3948711437d25822d389dd857c72dc7fcc15727c9048d2e3eae3317d97aca6dd", - "src/inferrsi/knowledge/data/skill_library/repositories/husky/references/verification-protocol.md": "83c5a966ed3f721b412b8d8417af8336c5a8c1e46044a888720936bf2f1deeab", - "src/inferrsi/knowledge/data/skill_library/repositories/husky/scripts/audit_evidence_card.py": "291e62701ef7a37a5bfab963ea4e20abb94245bc015584223a101f18b1490895" - }, - "skill_metadata": { - "name": "husky", - "description": "Review Husky's official-document claims as research hypotheses, separating author reports, implementation evidence, negative results, future directions and the missing artifacts needed for independent qualification.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "husky", - "document-research", - "evidence-qualification", - "negative-results" - ], - "sources": [ - { - "type": "document", - "url": "https://husky.underdog.ai/", - "displayed_publication_date": "2026-09-20", - "accessed": "2026-10-27", - "license": "UNKNOWN", - "scope": "Public unversioned official-page view; bounded author-report summary, not source code or retained receipts." - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "AREX-informed document-only adaptation explicitly accepted by the user; not complete repo-to-skill or native verification." - } - ], - "construction": { - "anchor_kind": "document", - "distillation_form": "document-research-procedure", - "strategy": "arex-informed-document-adaptation", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Single coherent research workflow, bounded source summary, original verification questions and offline schema checks. Repository/environment/native stages remain unavailable." - }, - "applicability": { - "scope": "Review Husky document claims and specify missing evidence; common catalog scope is not inference backend compatibility.", - "prerequisites": "Access to the bounded official-document inventory and verification protocol; implementation and native reproduction remain unavailable until separately supplied and authenticated.", - "input_contract": "The exact claim, source/view date, workload and comparison intent, supporting evidence classes and available implementation/model/output/measurement artifacts; unknown identity, licensing and availability remain explicit.", - "produced_artifact": "A reviewable claim/evidence request recording source availability, mechanism hypothesis, counter-signal, missing identities/artifacts, proposed falsifying check and actual result or bounded INCONCLUSIVE conclusion.", - "failure_modes": "Treating a public-page reading as code inspection, proposed extensions as shipped behavior, summed independent ratios as one gain, or a structurally valid self-reported card as authentic evidence can overstate qualification.", - "refusal_point": "Stop at missing source/receipts, unsafe or private data, unsupported resources or the authorized budget; unavailable artifacts yield an unverified/INCONCLUSIVE research conclusion." - }, - "limitations": [ - "No canonical public repository, implementation revision, retained receipts or source-code license was identified; unknowns remain unknown.", - "The unversioned official page is author-reported and unreproduced; no immutable source HTML hash is claimed.", - "Native reproduction is unavailable; schema checks do not authenticate supplied identities, artifacts, output equality or performance.", - "Research protocols and ablation questions are original proposals, not claims about uninspected implementation behavior.", - "NO_LICENSE is the unavailable repository-license resolver result, not a legal absence finding or grant to copy publisher material.", - "Final informed candidate admission is separate; no automatic import, native qualification or skill efficacy is claimed." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-availability.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Separate author reports, inspected implementation, executed experiments and new interpretations, including negative findings and future work.", - "Request immutable identities and artifacts, then state one mechanism, competing explanation and smallest ablation with frozen semantic requirements.", - "Use the offline card auditor only to identify structural gaps; it authenticates neither artifacts nor execution and leaves qualification unadjudicated." - ] - }, - "applicability": { - "scope": "Review Husky document claims and specify missing evidence; common catalog scope is not inference backend compatibility.", - "prerequisites": "Access to the bounded official-document inventory and verification protocol; implementation and native reproduction remain unavailable until separately supplied and authenticated.", - "input_contract": "The exact claim, source/view date, workload and comparison intent, supporting evidence classes and available implementation/model/output/measurement artifacts; unknown identity, licensing and availability remain explicit.", - "produced_artifact": "A reviewable claim/evidence request recording source availability, mechanism hypothesis, counter-signal, missing identities/artifacts, proposed falsifying check and actual result or bounded INCONCLUSIVE conclusion.", - "failure_modes": "Treating a public-page reading as code inspection, proposed extensions as shipped behavior, summed independent ratios as one gain, or a structurally valid self-reported card as authentic evidence can overstate qualification.", - "refusal_point": "Stop at missing source/receipts, unsafe or private data, unsupported resources or the authorized budget; unavailable artifacts yield an unverified/INCONCLUSIVE research conclusion." - }, - "recommendations": [ - "Separate author reports, inspected implementation, executed experiments and new interpretations, including negative findings and future work.", - "Request immutable identities and artifacts, then state one mechanism, competing explanation and smallest ablation with frozen semantic requirements.", - "Use the offline card auditor only to identify structural gaps; it authenticates neither artifacts nor execution and leaves qualification unadjudicated." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "No canonical public repository, implementation revision, retained receipts or source-code license was identified; unknowns remain unknown.", - "The unversioned official page is author-reported and unreproduced; no immutable source HTML hash is claimed.", - "Native reproduction is unavailable; schema checks do not authenticate supplied identities, artifacts, output equality or performance.", - "Research protocols and ablation questions are original proposals, not claims about uninspected implementation behavior.", - "NO_LICENSE is the unavailable repository-license resolver result, not a legal absence finding or grant to copy publisher material.", - "Final informed candidate admission is separate; no automatic import, native qualification or skill efficacy is claimed." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json deleted file mode 100644 index 9c4e06251..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json +++ /dev/null @@ -1,734 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.inference_benchmark_design", - "title": "inference-benchmark-design", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image, video and VLA inference. Covers AIPerf AgentX/Exgentic replay, vLLM/SGLang benchmark boundaries, VLA action-chunk timing and quality, SLO-aware metrics and InferRSI integration.", - "tags": [ - "benchmark", - "evaluation", - "serving", - "measurement", - "task-quality", - "reference-parity", - "agentic-coding", - "work-agents", - "aiperf", - "tts", - "full-duplex", - "image-generation", - "video-generation", - "asr", - "stream-boundaries", - "treatment-delivery", - "trial-validity", - "protocol-audit", - "trial-evidence", - "result-origin", - "oracle-boundary", - "candidate-selection", - "accepted-prefix", - "state-restoration", - "completion-accounting", - "vla", - "embodied", - "action-chunks", - "closed-loop", - "gpu-timing", - "profiler-attribution", - "measurement-validity" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/inference-benchmark-design/SKILL.md" - }, - "checkout_ref": "checkout:inference-benchmark-design", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md": "7782936c8b43dc6d27f416da6aa7162a397e3321adc044e8e7515a14791b720b", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json": "bffa79dd93915986252728939f6cba16ee721164910c58ab1f78a2e69299db1d", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json": "bc343ece2bf40c4e8aa4bd8d84f32465d00633255557feda5fe7799ce24e8688", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md": "cb3a7a5d94d056f868af07f92a16a6fef31eafd3199ed07ae79e93798e320b1d", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md": "0898424e0bff3a6246e1eeeda6abd6c5bd974c047fe32f3bbd5ab9d31ce17cd3", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md": "4e0ea9c1bf6b096a49a91ad5266e90b8a456f4f5aca15eba8ac270a53c2309b5", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/dependency-aware-tts-playout.md": "e94866e718adc3bd7e272a4856e9a90b6f27cbc0ff3a44101e68e6995ed79e2c", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md": "0bc3ca87159cce960c7b32f62b82c275f850b98077ae07da0481bd554a3cd71d", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json": "056985be689554570a9f12b8e228ce71bdca5be940f9a2d7f6e1963e15a48130", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/faster-cosyvoice-source.json": "9efe8918208adddfb9a31d2abed080923a5ae0dfb6332438179af0de66f54e13", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md": "96e273d8a193b62540962084cdca7cb4d93ef43473a38b1dcb7228088de793aa", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-evaluation-contracts.md": "d25df795fd301e8a1bb436e5f8122d0bac2441c43104e59a3b310fd5a64f2f3e", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-source.json": "60f2784384ea1a3b08a2ecafb60207d8e3707247dc7bc177b95b463d81f1cc16", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-source.json": "ccda0fc6781e3c4dceb41521ab952445840b4afcc9f5a7d968857f76e8db6272", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md": "2b3de210d1790eec92319addadf0e38fcd1529fa2f23b5cddf96a789b783f4ac", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md": "0d4bb080a754e26846e81680023db0dff5e6a4c96be4327c411c3b72ac4ef8e2", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-source.json": "6c21791b9c8ecbc9645c6deb659bd897e17f70525536ea25e884a62fc5fb6b25", - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-trial-evidence.md": "802c4dfe83983cbd6740231d5d0e2b459792b685de8dfbefb6dc51a912e23604" - }, - "skill_metadata": { - "name": "inference-benchmark-design", - "description": "Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image, video and VLA inference. Covers AIPerf AgentX/Exgentic replay, vLLM/SGLang benchmark boundaries, VLA action-chunk timing and quality, SLO-aware metrics and InferRSI integration.", - "tags": [ - "benchmark", - "evaluation", - "serving", - "measurement", - "task-quality", - "reference-parity", - "agentic-coding", - "work-agents", - "aiperf", - "tts", - "full-duplex", - "image-generation", - "video-generation", - "asr", - "stream-boundaries", - "treatment-delivery", - "trial-validity", - "protocol-audit", - "trial-evidence", - "result-origin", - "oracle-boundary", - "candidate-selection", - "accepted-prefix", - "state-restoration", - "completion-accounting", - "vla", - "embodied", - "action-chunks", - "closed-loop", - "gpu-timing", - "profiler-attribution", - "measurement-validity" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "document", - "url": "https://github.com/InferMatrixLab/InferRSI/issues/60#issuecomment-6310920198", - "version": "2026-10-21 report", - "sha256": "c56a1b169d70d9cd2bfcf073b13e001354945d71c54ebde5740f3036e4ee4496", - "readset": "references/task-benchmark-selection.md", - "scope": "Distilled task matrix, metric definitions, integration boundaries and command templates; original report is source/documentation review, not a runtime benchmark." - }, - { - "type": "git", - "url": "https://github.com/vllm-project/vllm-omni", - "commit": "b20e5bac96a029172a59559438ad0590460aeb3c", - "paths": [ - "benchmarks/accuracy/README.md", - "benchmarks/accuracy/image_to_image/README.md", - "benchmarks/accuracy/text_to_image/README.md", - "benchmarks/diffusion/backends.py", - "benchmarks/diffusion/diffusion_benchmark_serving.py", - "benchmarks/tts/README.md", - "benchmarks/tts/bench_tts.py", - "docs/cli/bench/serve.md", - "tests/e2e/accuracy/wan22_i2v/test_wan22_i2v_video_similarity.py", - "vllm_omni/benchmarks/data_modules/seed_tts_eval.py", - "vllm_omni/benchmarks/duplex_session_metrics.py", - "vllm_omni/benchmarks/metrics/metrics.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/sgl-project/sglang", - "commit": "844d2b5decf6257aa192744cbdaaffb8ad797263", - "paths": [ - "docs/cookbook/base/benchmarks/diffusion_model_benchmark.mdx", - "docs/docs/developer_guide/bench_serving.mdx", - "docs/docs/developer_guide/evaluating_new_models.mdx", - "python/sglang/benchmark/serving.py", - "python/sglang/multimodal_gen/benchmarks/bench_offline_throughput.py", - "python/sglang/multimodal_gen/benchmarks/bench_serving.py", - "python/sglang/multimodal_gen/test/test_utils.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/vllm-project/vllm", - "commit": "a9bf45ed8a856a0b1839197152d0d26fabb6599c", - "paths": [ - "docs/benchmarking/cli.md", - "vllm/benchmarks/serve.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/ai-dynamo/aiperf", - "commit": "30420ce581d4fc9eeb34b4d47defe367ab8c3a11", - "paths": [ - "docs/accuracy/accuracy-benchmarking.md", - "docs/benchmark-datasets.md", - "docs/metrics-reference.md", - "docs/tutorials/agentic-code-generator.md", - "docs/tutorials/agentx-mvp.md", - "docs/tutorials/h-cua-perf.md", - "docs/tutorials/tracelab-trace.md", - "docs/tutorials/weka-trace.md", - "docs/tutorials/working-with-profile-exports.md", - "src/aiperf/dataset/loader/exgentic_v2.py", - "src/aiperf/metrics/types/inter_token_latency_metric.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/sgl-project/sglang-omni", - "commit": "f31cf9e9af749040ffa6dbdf064882c443a2fd37", - "paths": [ - "benchmarks/README.md", - "benchmarks/duplex/serving.py", - "benchmarks/metrics/performance.py", - "benchmarks/metrics/speaker_similarity.py", - "benchmarks/metrics/wer.py", - "docs/developer_reference/full_duplex_bench.md", - "docs/developer_reference/full_duplex_metrics.md" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/Lucky-Lance/OmniInteract", - "commit": "de304cef35fd9a50a5caadb5090c34cfbf0dd868", - "paths": [ - "eval/README.md", - "eval/evaluation/compute_unified_eval.py", - "eval/evaluation/summarize_unified_eval.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/OpenBMB/Omni-DuplexEval", - "commit": "962cec448ac37a377ffb963b476778777d11d346", - "paths": [ - "README.md" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI", - "commit": "b3ef95a6dd3abd85c19b5527fe46e80ccb3521d4", - "paths": [ - "docs/quickstart.md", - "src/inferrsi/harness/evaluation.py", - "src/inferrsi/model/protocols/metrics.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/NVIDIA/NeMo-Speech.cpp", - "commit": "20999cc921d2b14aa58708f61756a8b2e787ba20", - "paths": [ - "BENCHMARK.md", - "LICENSE", - "NOTICE", - "THIRD_PARTY_NOTICES.md", - "src/asr/encoder/cache_aware_encoder.cpp", - "src/asr/encoder/cache_aware_encoder.h", - "src/asr/encoder/fastconformer.h", - "src/asr/recognizer.cpp", - "src/asr/runner.cpp", - "src/asr/runner.h", - "src/asr/vad/vad_endpointer.cpp", - "src/asr/vad/vad_endpointer.h", - "tests/cpp/asr/CMakeLists.txt", - "tests/cpp/asr/test_cache_stream_geometry.cpp" - ], - "readset": "references/cache-aware-asr-stream-boundaries.md" - }, - { - "type": "git", - "url": "https://github.com/OpenBMB/JustGoal", - "commit": "f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d", - "paths": [ - "LICENSE", - "NOTICE", - "README.md", - "goal_full_shape/fable-5.1/dsa_decode_fwd/dsa_sparse_attention_h16_ckv512_kpe64_topk2048_ps64/dsa_decode.cu", - "goal_full_shape/fable-5.1/dsa_decode_fwd/dsa_sparse_attention_h16_ckv512_kpe64_topk2048_ps64/main.py", - "goal_single_shape/fable-5.1/dsa_decode_fwd/t8_topk2048/dsa_kernel.cu", - "goal_single_shape/fable-5.1/dsa_decode_fwd/t8_topk2048/dsa_kernel.py", - "kda05_reproduction_full_shape/opus-4.8/fa4_bwd/attention_bwd_causal_h16_d128/cute_fabwd_bshd_v5c_fusedzero/helpers/NOT_INCLUDED.md" - ], - "readset": "references/justgoal-source.json", - "license": "Apache-2.0 repository default; NOTICE exceptions and excluded proprietary helper retained", - "scope": "Selected kernel entry/workspace/stream contracts and distribution boundary; source-only, no execution" - }, - { - "type": "document", - "url": "https://github.com/OpenBMB/JustGoal/blob/f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d/paper.pdf", - "version": "Repository-paired PDF at f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d", - "sha256": "6bfa1a2728546def4c48e15cf55f31d92a543264787318f1522236c80a0ae8c6", - "readset": "references/justgoal-source.json", - "license": "UNKNOWN", - "scope": "Complete 21-page paper and appendices: controlled comparison, independent correctness, repair/coverage and timing evidence; author-reported, no PDF/figure redistribution" - }, - { - "type": "git", - "url": "https://github.com/li-ch/autotunebench", - "commit": "64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f", - "paths": [ - "ARTIFACT.md", - "README.md", - "CITATION.cff", - "LICENSE", - "config/protocol.yaml", - "pdb/ingest_validate.py", - "pdb/store.py", - "agents/loop.py", - "agents/adapter.py", - "orchestrator/cli.py", - "gates/g2_runtime.py", - "gates/g3_correctness.py", - "gates/g5_graduation.py", - "gates/schema.py", - "harness/kernel_bench.py", - "figs/plot_cand_ab_saturation.py", - "scripts/kernelbench_tabulate.py", - "experiments/kernelbench_l1/results/README.md", - "experiments/kernelbench_l1/results/TABLE.md", - "experiments/astra_parity/README.md", - "observability/iterations.jsonl", - "experiments/kernelbench_l1/kernelbench_tasks/UPSTREAM_LICENSE", - "experiments/kernelbench_l1/kernelbench_tasks/README.md" - ], - "readset": "references/autotunebench-trial-validity.md" - }, - { - "type": "paper", - "id": "arxiv:2609.18123", - "version": "v1", - "url": "https://arxiv.org/abs/2609.18123v1", - "sections": [ - "1-8", - "References", - "Figures 1-6", - "Tables 1-7" - ], - "readset": "references/autotunebench-trial-validity.md" - }, - { - "type": "git", - "url": "https://github.com/IntelLabs/Xe-Forge", - "commit": "6cd3ee98dfe32746b345a6abe521fa4cb9bdc187", - "paths": [ - "CONTRIBUTING.md", - "LICENSE", - "README.md", - "docs/CORRECTNESS.md", - "docs/KERNEL_REPO.md", - "pyproject.toml", - "src/xe_forge/claude/generator.py", - "src/xe_forge/claude/templates/CLAUDE.md.j2", - "src/xe_forge/cli.py", - "src/xe_forge/core/trial_manager.py", - "src/xe_forge/engines/claude_engine.py", - "src/xe_forge/external.py", - "src/xe_forge/skills/__init__.py", - "src/xe_forge/skills/benchmark.py", - "src/xe_forge/skills/trial.py", - "src/xe_forge/skills/upstream.py", - "tests/test_benchmark_gate.py", - "tests/test_trial_manager_ranking.py" - ], - "readset": "references/xeforge-source.json", - "license": "Apache-2.0 repository default; DCO and curated-example exclusions retained", - "scope": "Selected source-bound trial origin/comparison identity, host outcomes, measured-only selection and baseline-oracle contracts; source-only, no execution" - }, - { - "type": "paper", - "id": "arxiv:2605.26118", - "version": "v1", - "url": "https://arxiv.org/html/2605.26118v1", - "sections": [ - "Abstract", - "I-VIII", - "References", - "Appendix A", - "Tables 1-3", - "Algorithm 1", - "Listing 1", - "Figure captions 1-15" - ], - "readset": "references/xeforge-source.json", - "license": "CC-BY-4.0", - "scope": "Complete HTML text/appendix and captions; historical author-reported context, unused graphic values remain unverified" - }, - { - "type": "git", - "url": "https://github.com/Neroued/ninfer", - "commit": "1882c6adbc98043fcf89f3589aa5addb3742777a", - "paths": [ - "LICENSE", - "README.md", - "docs/maintainer/engine-architecture.md", - "docs/maintainer/speculative-decoding.md", - "docs/maintainer/context-management.md", - "docs/performance/methodology.md", - "docs/performance/qwen3.8-27b.md", - "include/ninfer/ops/gdn_replay.h", - "src/core/gdn_replay_records.h", - "src/core/gdn_replay_records.cpp", - "src/core/linear_attention_state.h", - "src/core/linear_attention_state.cpp", - "src/models/qwen3_5/state/state_image.h", - "src/models/qwen3_5/state/state_image.cpp", - "src/models/qwen3_5/execution/gdn.cpp", - "src/models/qwen3_5/execution/text.cpp", - "src/models/qwen3_5/program/decode.cpp", - "src/models/qwen3_5/program/commit.cpp", - "src/models/qwen3_5/program/mtp.cpp", - "src/models/qwen3_5/session/commit.cpp", - "src/models/qwen3_5/session/binding.cpp", - "src/models/qwen3_5/session/checkpoints.cpp", - "src/runtime/engine/engine_execution.inl", - "src/runtime/executor/contract.h", - "src/runtime/executor/local_executor.h", - "src/ops/linear_attention/gated_delta_net/replay/fold.cpp", - "tests/ops/linear_attention/gated_delta_net/test_fold.cpp" - ], - "readset": "references/ninfer-source.json", - "license": "Apache-2.0", - "scope": "Selected Qwen3.5 GDN/MTP state-restoration, accepted-prefix and completion evidence contracts; historical results retain their separate reported revision. Source-only, no native execution." - }, - { - "type": "git", - "url": "https://github.com/yuekaizhang/faster-cosyvoice", - "commit": "f1049c59196c2c1a890079abc56056b43f27cd1d", - "paths": [ - "README.md", - "README_zh.md", - "docs/benchmark.md", - "docs/benchmark_zh.md", - "docs/usage.md", - "docs/usage_zh.md", - "examples/offline_inference.py", - "examples/stream_client.py", - "faster_cosyvoice/config/settings.py", - "faster_cosyvoice/llm/engine.py", - "faster_cosyvoice/server/app.py", - "faster_cosyvoice/server/arguments.py", - "faster_cosyvoice/server/audio_encode.py", - "faster_cosyvoice/server/leading_silence.py", - "faster_cosyvoice/server/openai_speech.py", - "faster_cosyvoice/server/protocol.py", - "faster_cosyvoice/streaming/chunker.py", - "faster_cosyvoice/streaming/session.py", - "faster_cosyvoice/streaming/token2wav_batcher.py", - "faster_cosyvoice/streaming/token2wav_scheduler.py", - "faster_cosyvoice/token2wav/builders.py", - "faster_cosyvoice/token2wav/cosyvoice/flow/DiT/modules.py", - "faster_cosyvoice/token2wav/cosyvoice/flow/flow.py", - "faster_cosyvoice/token2wav/cosyvoice/hifigan/generator.py", - "faster_cosyvoice/token2wav/flashinfer_dit.py", - "faster_cosyvoice/token2wav/flashinfer_flow.py", - "faster_cosyvoice/token2wav/frontend.py", - "faster_cosyvoice/token2wav/matcha_audio.py", - "faster_cosyvoice/token2wav/token2wav.py", - "pyproject.toml", - "tests/test_session.py", - "tests/test_stream_client.py", - "tests/test_token2wav_batcher.py" - ], - "readset": "references/faster-cosyvoice-source.json", - "license": "UNKNOWN overall; selected file-level Apache-2.0 headers and unresolved inherited attribution retained", - "scope": "Selected PCM/playout, routed-credit, chunk-dependency and timing-denominator source audit; original analysis and immutable links only, no code/media copying or native/model/quality/service execution" - }, - { - "type": "git", - "url": "https://github.com/SEU-PAISys/Embodied.cpp", - "commit": "156bed151826780e2ce0132f8d0e3540889ae8cd", - "paths": [ - "adapter/typed_io.py", - "eval/client/libero_profile.py", - "eval/client/vla_cpp_client.py", - "eval/conf/libero_smolvla_object_q8_eval.yaml", - "eval/README.md", - "eval/smolvla_q8_fixed_boundary_20261002.json", - "eval/smolvla_q8_object_followup_20260930.md", - "eval/SMOLVLA_VALIDATION.md", - "eval/THREE_MODEL_VALIDATION.md", - "eval/VLA_BENCHMARK_STANDARD.md", - "LICENSE", - "README.md", - "scripts/aggregate_smolvla_object_eval.py", - "scripts/bench_vla_boundary.py", - "scripts/profile_smolvla_python_baseline.py", - "serving/vla-server.cpp", - "serving/vla.proto" - ], - "readset": "references/embodied-cpp-source.json", - "license": "Apache-2.0", - "scope": "Selected CPU observation-to-action benchmark, client/profiler/aggregation and author artifact contracts; source-derived methodology, not native execution or quality/performance acceptance" - }, - { - "type": "paper", - "id": "arxiv:2607.02501", - "version": "v3", - "url": "https://arxiv.org/abs/2607.02501v3", - "sections": [ - "4 (VLA discussion and Table 3)" - ], - "readset": "references/embodied-cpp-source.json", - "license": "CC-BY-4.0", - "scope": "Independently normalized VLA metric semantics; predates the later SmolVLA experiment" - }, - { - "type": "document", - "url": "https://docs.nvidia.com/nsight-compute/2025.3/ProfilingGuide/index.html", - "version": "Nsight Compute 2025.3", - "sha256": "c9f3b812ce50a46a751f2c92eff628177554f4f6678bb3381468b1dac91b607e", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Replay, Overhead, Hardware Model, Metrics Structure, Range and Precision, Reproducibility, Clock Control and Cache Control; source contracts only, no profiler execution." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/nsight-compute/2025.3/NsightComputeCli/index.html", - "version": "Nsight Compute 2025.3", - "sha256": "c5129a8afe2439c04906f19312393972ae123062d2ee0894b3ba2b11af43471e", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Metric Collection and Profiling option defaults for clock/cache control and metric queries; actual target support must be checked separately." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/nsight-systems/2025.3/UserGuide/index.html", - "version": "Nsight Systems 2025.3", - "sha256": "1d42e35005fa9a4cb21b2c65d0018e4b065c3756dbba308dc0c41f79746e751f", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected CLI Profile options, CUDA graph/node tracing and CUDA API/kernel/launch/MemOps statistics reports; absence of NCU replay controls is an author inference, not a launcher guarantee." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/12.9.1/cuda-c-best-practices-guide/index.html", - "version": "CUDA Toolkit 12.9.1 archive", - "sha256": "0fcaf0c361d00de56a9870ba96db871a0b880904121e1dfb215ab9882470120b", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Assess/Optimize, Application Profiling, Amdahl's Law, Timing and Instruction Throughput; normalization/adoption procedure is an unverified author synthesis." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/12.9.1/cuda-c-programming-guide/index.html", - "version": "CUDA Toolkit 12.9.1 archive", - "sha256": "06499a0c6bfc6925cf54c76f6ba86a4986a081c44c602e10d4ef3b4f33a33865", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Concurrent Kernel Execution, CUDA Graphs (structure, definition/instantiation/execution) and Events; no constant launch/replay overhead or device test claimed." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/10.2/nsight-compute/docs/NsightComputeCli/index.html", - "version": "CUDA Toolkit 10.2 archive, Nsight Compute CLI", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected nvprof Metric Comparison for dram__bytes_read.sum, dram__bytes_write.sum and smsp__inst_executed.sum. Legacy names do not guarantee current chip/tool availability; no raw-byte digest retained for this browser read." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/deploy/nvidia-smi/index.html", - "version": "Unversioned NVIDIA System Management Interface page, retrieved 2026-11-15; content pinned by SHA-256", - "sha256": "94d1737cdff11b5ef89b121e97365706feeb8ccc18978fecec571bdbbe6ec9ed", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Performance State, Clocks Event Reasons, clocks, temperature and power reporting. Public page may change; digest identifies inspected HTML, which is not redistributed." - }, - { - "type": "git", - "url": "https://github.com/triton-lang/triton", - "commit": "9641643da6c52000c807b5eeed05edaec4402a67", - "paths": [ - "python/triton/testing.py" - ], - "readset": "references/gpu-measurement-validity.md", - "license": "MIT", - "scope": "v3.2.0 do_bench: write-based cache.zero_ flush, preliminary estimate and per-sample event boundaries; tag resolved with git ls-remote, source inspected in public browser. No flush comparison run." - }, - { - "type": "document", - "url": "https://docs.pytorch.org/docs/2.7/cpp_extension.html", - "version": "PyTorch 2.7", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected load/load_inline and the extension name, build-directory and pybind11 module contract. Distinct names for separately loaded variants are author advice, not a documented universal duplicate-name failure; no raw-byte digest retained for this browser read." - }, - { - "type": "paper", - "url": "https://arxiv.org/html/2509.14279v1", - "id": "arxiv:2509.14279", - "version": "v1", - "sections": [ - "3", - "Appendix A", - "Appendix C.3" - ], - "readset": "references/gpu-measurement-validity.md", - "license": "CC-BY-4.0", - "scope": "Lange et al., Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization. Inspected benchmark loopholes, compromised-task filtering/examples and eager/compiled baseline comparisons; reports remain author-reported, not locally reproduced." - } - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Selecting task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image and video inference, and designing goodput-based multi-regime comparisons that fit InferRSI's evidence trail; interpreting treatment delivery, arm validity and stated-versus-observed protocol enforcement in supplied tuning reports; interpreting source-bound kernel trial origin, host evaluator outcomes, unresolved performance and measured-only versus manual candidate selection; designing and interpreting the selected NInfer hybrid speculative state-restoration and completion validation contract; designing or auditing VLA deployment comparisons with explicit observation-to-action boundaries, generated/replayed horizons, per-task quality and source/precision identity; assessing GPU timing validity and profiler-based attribution of kernel/inference gains", - "prerequisites": "The primary objective, timing boundary, quality gates, sample/failure accounting, required regimes, repetitions, seeds and decision rule are frozen before optimization; baseline and candidate share hardware, software stack, input distribution, benchmark configuration, warmup and measurement protocol, and on a shared machine also one time window and dedicated devices; cache-aware ASR also requires the selected model/config and first/steady geometry, decoded clock and reset/EOU/EOF boundaries to be established; VLA comparisons also freeze generated/replayed horizons, initial-state/noise protocol and intended precision or quantization treatment", - "input_contract": "The task family and deployment use case; candidate quality evaluators and reference parity; the declared SLO and the caller's policy on which regimes must improve or must not regress; for an audit, a supplied fixed-workload report with request-to-trial grouping, exclusion and failure counts and the declared aggregation rule; for a treatment audit, the frozen delivery/arm-validity rules, run/candidate identities, feedback and consumer-input evidence, and per-arm raw records; for a kernel trial audit, complete saved candidate bytes and IDs, baseline/spec/reference/variant identities, result-origin fields, raw host outcome and return code, named gate and persisted ratio, actual oracle and the selecting/delivering caller; for the NInfer slice, a complete checkpoint/frontier identity, physical verify-block/state oracle, final adopted extents and correlated execution/termination records; for VLA audits, the declared CPU input/output boundary, source/fixture/checkpoint/processor identities, per-task trial scope and available raw records or explicit artifact gaps", - "produced_artifact": "A frozen benchmark design with evaluators, reference parity, primary objective, timing boundary, quality gates, required regimes, repetition and seed rules and an explicit decision config, plus the evidence-tree layout a PASS claim must produce; for a supplied report, the computed threshold result and missing evidence. Candidate knowledge, not qualification of any model or backend; for a treatment audit, a contract-bound eligibility and interpretation report separating treatment exposure, observed checks, comparator identity, anomaly disposition and source conflicts; for a kernel trial audit, a source-bound interpretation separating artifact/cohort identity, origin, oracle, performance resolution and selected-versus-delivered eligibility, without promoting kernel parity into a gain or whole-model acceptance; for the NInfer slice, a source-bound validation plan or report interpretation that separates state-oracle evidence, final adoption, retirement and workload denominators; for VLA reports, a timing/denominator and precision reconciliation with per-task quality/completion accounting and source-artifact limits; for GPU attribution, a source-grounded measurement plan or interpretation binding environment, paired timing, work counts, predicted signature and E2E decomposition with explicit assumptions and unresolved evidence", - "failure_modes": [ - "Raw throughput is optimized in place of goodput, so requests served outside the SLO count toward a win", - "One concurrency level or one undisclosed aggregate score stands in for all required regimes", - "Recorded agent task scores or replay completion are read as current task success, or a VBench prompt workload as computed VBench quality", - "An upstream better/worse/inconclusive label is used as InferRSI's PASS/FAIL/INCONCLUSIVE verdict", - "Runs on the same hardware class, in different time windows or on shared devices are compared as if pinned", - "A quality pass is taken to certify that the timed work actually occurred", - "Best-seen results are reported as delivered without restarting and validating the final code/config artifact", - "A cache-aware ASR EOU is treated as EOF or full reset, nominal chunk duration as the decoded mel frontier, or shape-only geometry PASS as a native model-state test", - "An enabled treatment or stored feedback summary is taken as proof of delivery to the next eligible consumer", - "A failed arm-validity condition is overridden by a favorable best timing or a saturation/null interpretation", - "Complete gate keys or zero recorded failures are read as proof that every declared protocol check ran", - "A tool-origin measured marker or completed status is treated as a resolved performance gain", - "A displayed/manual best, baseline copy or edited candidate is reported as the eligible measured delivery", - "Ratio omission alone is assumed to clear an earlier ratio, or baseline agreement is treated as independent semantic correctness", - "Zero accepted drafts is treated as zero adopted input, or a matching token prefix is used to splice unrelated State and KV checkpoints", - "ResultReady or a commit receipt is treated as DeviceRetired, or selected full-batch intervals are compared with complete-cohort rates under one unlabeled denominator", - "Generated-action amortization is read as action-response latency or actual control frequency", - "Matching structural pair fields are taken to certify matched effective compute precision or quality equivalence", - "An aggregate VLA success rate conceals task-specific losses or missing/skipped episodes", - "A public current script, recorded experiment revision or binary hash is treated as the exact measured dirty source patch", - "Whole-device fallback memory is relabelled as process-attributed memory or compared with simulator-inclusive/allocator quantities", - "NCU replay durations are substituted for normal E2E evidence, or an untouched region is treated as a universal clock without checking common drift and critical-path assumptions", - "Unexplained gains, changed baseline modes or same-named JIT loads are adopted without binding actual work, module identity and predicted profiler signatures" - ], - "refusal_point": "Records missing coverage and high-variance results as INCONCLUSIVE rather than a win; a run without a frozen RunSpec, or whose validator changed after the plan was frozen, is INCONCLUSIVE regardless of metric values; sets no universal numeric gate or noise threshold; withholds causal treatment benefit/null conclusions when required delivery or arm eligibility is unverified, and preserves conflicts between stated protocol, exported code and source reports; withholds a kernel gain for unresolved performance or an unbound comparison and identifies missing independent semantic evidence when only baseline agreement is available; withholds native state-correctness and cancellation/reuse conclusions when the required same-block oracle, checkpoint identity or matching receipt is missing; withholds VLA quality-preservation, runtime-only attribution, control-rate or current-build conclusions when their required precision, per-task, timestamp or measured-source evidence is missing" - }, - "recommendations": [ - "Freeze one primary objective, timing boundary, quality gates and sample/failure accounting before optimization.", - "Choose the required regimes from the deployment use case, report each separately, and record which must improve or must not regress in a versioned decision config.", - "Use the replay trace that matches the actual deployment pattern.", - "State the concurrency, trace, model, SKU and microbenchmark scope boundaries in any report that cites benchmark results.", - "Trace treatment to its eligible consumer and apply the frozen arm-validity precedence before attributing benefit, saturation or a null.", - "Bind the complete saved candidate and comparator cohort; keep result origin, actual oracle, named gate/ratio and delivered eligibility separate.", - "For hybrid speculative comparisons, freeze the coupled restore point, same-physical-block oracle, zero-draft/zero-adoption controls and complete-cohort versus steady-phase denominators before interpreting a result.", - "Separate complete VLA request cost, generated/replayed action denominators and actual control cadence; retain per-task quality and measured-source/precision limits.", - "For GPU comparisons, retain timestamped clocks/temperature/power, same-run region shares and work counts; test the predicted mechanism signature and paired E2E change before adopting a gain." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "Task benchmark knowledge is distilled from issue #60 with selected pinned-source rechecks; no all-task runtime qualification or optimization benefit is established.", - "Benchmark feasibility and package-consumption checks do not promote this candidate or replace task quality, repeated A/B and service qualification.", - "The pinned NeMo-Speech.cpp ASR reference covers selected source/test definitions and author-reported benchmark boundaries; native model/device, decoder internals, quality, long-audio/load and service behavior remain not_run.", - "JustGoal contributes source-only controlled-evaluation and kernel-acceptance guidance; its repaired/retested timings, released-pass counts and protocol differences are author-reported, not locally reproduced or a complete campaign runner.", - "The AutoTuneBench reference is source-only: original operational checkpoint/prompt receipts remain unavailable, source contradictions remain explicit, and no tuning, device/performance or agent-efficacy validation was run.", - "The Xe-Forge reference is source-only and limited to inspected host/manual/Claude trial paths; named-verdict update predicates and caller defaults remain explicit. Native tests, backend/model/quality/performance/service behavior and agent efficacy were not evaluated, and source/package checks do not promote whole-model support.", - "The NInfer reference inspects selected Qwen3.5 GDN/MTP source contracts and unexecuted test definitions; its historical author measurements use a different engine revision. Native state/model/device/quality/performance and efficacy remain unvalidated.", - "The Faster CosyVoice reference is source-only: server route credit and client simulated playback remain distinct, reported costs/quality retain their boundaries, overall source rights are UNKNOWN, and native/model/device/audio quality/service/performance are not_run with efficacy not_evaluated.", - "The Embodied.cpp reference is source-derived: its public latency arrays allow arithmetic audit, but raw episode recount, exact measured script/dirty patch, effective dispatch and current model/device/service or efficacy qualification remain unresolved.", - "The GPU measurement-validity reference is source-only. The within-run normalization and explanation-first adoption procedure is an unverified author proposal; four task-supplied numerical observations have no supplied raw evidence. GPU/profiler, model/quality, L1–L4 and efficacy validation were not run; legacy metric mappings require target-specific queries." - ], - "provenance_readsets": [ - "references/task-benchmark-selection.md", - "references/benchmark-commands.md", - "references/cache-aware-asr-stream-boundaries.md", - "references/justgoal-evaluation-contracts.md", - "references/justgoal-source.json", - "references/autotunebench-trial-validity.md", - "references/autotunebench-source.json", - "references/xeforge-trial-evidence.md", - "references/xeforge-source.json", - "references/ninfer-state-and-completion-contracts.md", - "references/ninfer-source.json", - "references/dependency-aware-tts-playout.md", - "references/faster-cosyvoice-source.json", - "references/embodied-cpp-boundary-and-quality.md", - "references/embodied-cpp-source.json", - "references/gpu-measurement-validity.md" - ], - "references": [ - "../inference-optimization-casebook/cases/admission-drain-occupancy.md", - "../inference-optimization-casebook/cases/admission-drain-occupancy.json", - "../../repositories/deepinfer/budget-speculative-verification/SKILL.md", - "../../repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md" - ], - "consumption_mode": "packaged", - "searchspace_not_applicable": "SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism." - }, - "applicability": { - "scope": "Selecting task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image and video inference, and designing goodput-based multi-regime comparisons that fit InferRSI's evidence trail; interpreting treatment delivery, arm validity and stated-versus-observed protocol enforcement in supplied tuning reports; interpreting source-bound kernel trial origin, host evaluator outcomes, unresolved performance and measured-only versus manual candidate selection; designing and interpreting the selected NInfer hybrid speculative state-restoration and completion validation contract; designing or auditing VLA deployment comparisons with explicit observation-to-action boundaries, generated/replayed horizons, per-task quality and source/precision identity; assessing GPU timing validity and profiler-based attribution of kernel/inference gains", - "prerequisites": "The primary objective, timing boundary, quality gates, sample/failure accounting, required regimes, repetitions, seeds and decision rule are frozen before optimization; baseline and candidate share hardware, software stack, input distribution, benchmark configuration, warmup and measurement protocol, and on a shared machine also one time window and dedicated devices; cache-aware ASR also requires the selected model/config and first/steady geometry, decoded clock and reset/EOU/EOF boundaries to be established; VLA comparisons also freeze generated/replayed horizons, initial-state/noise protocol and intended precision or quantization treatment", - "input_contract": "The task family and deployment use case; candidate quality evaluators and reference parity; the declared SLO and the caller's policy on which regimes must improve or must not regress; for an audit, a supplied fixed-workload report with request-to-trial grouping, exclusion and failure counts and the declared aggregation rule; for a treatment audit, the frozen delivery/arm-validity rules, run/candidate identities, feedback and consumer-input evidence, and per-arm raw records; for a kernel trial audit, complete saved candidate bytes and IDs, baseline/spec/reference/variant identities, result-origin fields, raw host outcome and return code, named gate and persisted ratio, actual oracle and the selecting/delivering caller; for the NInfer slice, a complete checkpoint/frontier identity, physical verify-block/state oracle, final adopted extents and correlated execution/termination records; for VLA audits, the declared CPU input/output boundary, source/fixture/checkpoint/processor identities, per-task trial scope and available raw records or explicit artifact gaps", - "produced_artifact": "A frozen benchmark design with evaluators, reference parity, primary objective, timing boundary, quality gates, required regimes, repetition and seed rules and an explicit decision config, plus the evidence-tree layout a PASS claim must produce; for a supplied report, the computed threshold result and missing evidence. Candidate knowledge, not qualification of any model or backend; for a treatment audit, a contract-bound eligibility and interpretation report separating treatment exposure, observed checks, comparator identity, anomaly disposition and source conflicts; for a kernel trial audit, a source-bound interpretation separating artifact/cohort identity, origin, oracle, performance resolution and selected-versus-delivered eligibility, without promoting kernel parity into a gain or whole-model acceptance; for the NInfer slice, a source-bound validation plan or report interpretation that separates state-oracle evidence, final adoption, retirement and workload denominators; for VLA reports, a timing/denominator and precision reconciliation with per-task quality/completion accounting and source-artifact limits; for GPU attribution, a source-grounded measurement plan or interpretation binding environment, paired timing, work counts, predicted signature and E2E decomposition with explicit assumptions and unresolved evidence", - "failure_modes": [ - "Raw throughput is optimized in place of goodput, so requests served outside the SLO count toward a win", - "One concurrency level or one undisclosed aggregate score stands in for all required regimes", - "Recorded agent task scores or replay completion are read as current task success, or a VBench prompt workload as computed VBench quality", - "An upstream better/worse/inconclusive label is used as InferRSI's PASS/FAIL/INCONCLUSIVE verdict", - "Runs on the same hardware class, in different time windows or on shared devices are compared as if pinned", - "A quality pass is taken to certify that the timed work actually occurred", - "Best-seen results are reported as delivered without restarting and validating the final code/config artifact", - "A cache-aware ASR EOU is treated as EOF or full reset, nominal chunk duration as the decoded mel frontier, or shape-only geometry PASS as a native model-state test", - "An enabled treatment or stored feedback summary is taken as proof of delivery to the next eligible consumer", - "A failed arm-validity condition is overridden by a favorable best timing or a saturation/null interpretation", - "Complete gate keys or zero recorded failures are read as proof that every declared protocol check ran", - "A tool-origin measured marker or completed status is treated as a resolved performance gain", - "A displayed/manual best, baseline copy or edited candidate is reported as the eligible measured delivery", - "Ratio omission alone is assumed to clear an earlier ratio, or baseline agreement is treated as independent semantic correctness", - "Zero accepted drafts is treated as zero adopted input, or a matching token prefix is used to splice unrelated State and KV checkpoints", - "ResultReady or a commit receipt is treated as DeviceRetired, or selected full-batch intervals are compared with complete-cohort rates under one unlabeled denominator", - "Generated-action amortization is read as action-response latency or actual control frequency", - "Matching structural pair fields are taken to certify matched effective compute precision or quality equivalence", - "An aggregate VLA success rate conceals task-specific losses or missing/skipped episodes", - "A public current script, recorded experiment revision or binary hash is treated as the exact measured dirty source patch", - "Whole-device fallback memory is relabelled as process-attributed memory or compared with simulator-inclusive/allocator quantities", - "NCU replay durations are substituted for normal E2E evidence, or an untouched region is treated as a universal clock without checking common drift and critical-path assumptions", - "Unexplained gains, changed baseline modes or same-named JIT loads are adopted without binding actual work, module identity and predicted profiler signatures" - ], - "refusal_point": "Records missing coverage and high-variance results as INCONCLUSIVE rather than a win; a run without a frozen RunSpec, or whose validator changed after the plan was frozen, is INCONCLUSIVE regardless of metric values; sets no universal numeric gate or noise threshold; withholds causal treatment benefit/null conclusions when required delivery or arm eligibility is unverified, and preserves conflicts between stated protocol, exported code and source reports; withholds a kernel gain for unresolved performance or an unbound comparison and identifies missing independent semantic evidence when only baseline agreement is available; withholds native state-correctness and cancellation/reuse conclusions when the required same-block oracle, checkpoint identity or matching receipt is missing; withholds VLA quality-preservation, runtime-only attribution, control-rate or current-build conclusions when their required precision, per-task, timestamp or measured-source evidence is missing" - }, - "recommendations": [ - "Freeze one primary objective, timing boundary, quality gates and sample/failure accounting before optimization.", - "Choose the required regimes from the deployment use case, report each separately, and record which must improve or must not regress in a versioned decision config.", - "Use the replay trace that matches the actual deployment pattern.", - "State the concurrency, trace, model, SKU and microbenchmark scope boundaries in any report that cites benchmark results.", - "Trace treatment to its eligible consumer and apply the frozen arm-validity precedence before attributing benefit, saturation or a null.", - "Bind the complete saved candidate and comparator cohort; keep result origin, actual oracle, named gate/ratio and delivered eligibility separate.", - "For hybrid speculative comparisons, freeze the coupled restore point, same-physical-block oracle, zero-draft/zero-adoption controls and complete-cohort versus steady-phase denominators before interpreting a result.", - "Separate complete VLA request cost, generated/replayed action denominators and actual control cadence; retain per-task quality and measured-source/precision limits.", - "For GPU comparisons, retain timestamped clocks/temperature/power, same-run region shares and work counts; test the predicted mechanism signature and paired E2E change before adopting a gain." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "Task benchmark knowledge is distilled from issue #60 with selected pinned-source rechecks; no all-task runtime qualification or optimization benefit is established.", - "Benchmark feasibility and package-consumption checks do not promote this candidate or replace task quality, repeated A/B and service qualification.", - "The pinned NeMo-Speech.cpp ASR reference covers selected source/test definitions and author-reported benchmark boundaries; native model/device, decoder internals, quality, long-audio/load and service behavior remain not_run.", - "JustGoal contributes source-only controlled-evaluation and kernel-acceptance guidance; its repaired/retested timings, released-pass counts and protocol differences are author-reported, not locally reproduced or a complete campaign runner.", - "The AutoTuneBench reference is source-only: original operational checkpoint/prompt receipts remain unavailable, source contradictions remain explicit, and no tuning, device/performance or agent-efficacy validation was run.", - "The Xe-Forge reference is source-only and limited to inspected host/manual/Claude trial paths; named-verdict update predicates and caller defaults remain explicit. Native tests, backend/model/quality/performance/service behavior and agent efficacy were not evaluated, and source/package checks do not promote whole-model support.", - "The NInfer reference inspects selected Qwen3.5 GDN/MTP source contracts and unexecuted test definitions; its historical author measurements use a different engine revision. Native state/model/device/quality/performance and efficacy remain unvalidated.", - "The Faster CosyVoice reference is source-only: server route credit and client simulated playback remain distinct, reported costs/quality retain their boundaries, overall source rights are UNKNOWN, and native/model/device/audio quality/service/performance are not_run with efficacy not_evaluated.", - "The Embodied.cpp reference is source-derived: its public latency arrays allow arithmetic audit, but raw episode recount, exact measured script/dirty patch, effective dispatch and current model/device/service or efficacy qualification remain unresolved.", - "The GPU measurement-validity reference is source-only. The within-run normalization and explanation-first adoption procedure is an unverified author proposal; four task-supplied numerical observations have no supplied raw evidence. GPU/profiler, model/quality, L1–L4 and efficacy validation were not run; legacy metric mappings require target-specific queries." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json deleted file mode 100644 index 20c8cc2c1..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json +++ /dev/null @@ -1,116 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.inference_optimization_casebook", - "title": "inference-optimization-casebook", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook pattern from TensorRT-LLM's perf-optimization-casebook skill. Use when building a record of optimization experiments or when deciding which optimization to try next based on prior art.\n", - "tags": [ - "optimization", - "casebook", - "prior-art", - "decision-support" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/inference-optimization-casebook/SKILL.md" - }, - "checkout_ref": "checkout:inference-optimization-casebook", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/SKILL.md": "89a6a82c692f7f5192f4d7f4c3a5ff08a4df3b1c4bdaa0aafb7e04292d8aa5b4", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.json": "4f8544292b2cc3391f4d4d75994cb81fdd295eff99e55b7e60374c6c122de09e", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.md": "e88bbac5d253a8e7e3a3d5e21047f23f2970c11ac23887d4303ab94fca52dd27", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.json": "a2dc5db8aba62081c5743b12f854e74ae500c53bd57e2486260d0b5ae478638a", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md": "6f143bc363a3290fc008c08e57cb075f1776d1c8cca8a6a59297d5b1fbbf61bf", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json": "b2c1808919ae525efb82148a24b465459799422fefc6d420891d7dbeb36cd31a", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md": "be396983812262515899a3e2ba4f01b23ff92e1b836881882c7580372b8ae552", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json": "90e3cdca085e2d122e648aae527bd6a61de6d4fddb66c393ca25ac4541cd9122", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md": "cce2930f544a0f78f21797fba4d8ded298e4a81f76c34d7977b0e31d361bc738", - "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/metadata.json": "963488658f327d2379183d70357eabf832d70a6a70db2aa67f548493f4088d97" - }, - "skill_metadata": { - "name": "inference-optimization-casebook", - "description": "How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook pattern from TensorRT-LLM's perf-optimization-casebook skill. Use when building a record of optimization experiments or when deciding which optimization to try next based on prior art.\n", - "tags": [ - "optimization", - "casebook", - "prior-art", - "decision-support" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Structuring and querying a casebook of past optimization decisions as decision support, when recording optimization experiments or choosing the next optimization from prior art; the casebook is reference material and neither runs profiling nor makes decisions", - "prerequisites": "Whole-body text access to the case files for free-text search; applying a case first requires reading the nearest checked-in config and the model's deployment guide", - "input_contract": "For a query: natural search terms such as model name, knob, mechanism word or deployment feature, plus the current model and hardware. For a new case: mechanism, applies-when and does-not-apply conditions, procedure, expected effect, accuracy risk, verification, rollback and a real measurement trail with run ID, verdict, exact hardware SKU, model revision, workload and before/after result", - "produced_artifact": "Ranked matching cases read as precedents and diagnostic options, or a new or deprecated case record in the template; a precedent's procedure and expected direction of effect, not a guaranteed configuration or a measured result", - "failure_modes": [ - "The query is pre-translated into canonical vocabulary or bounded by structured fields, so synonyms in the body text are missed", - "A case's specific values, such as batch size, tensor-parallel degree or kernel parameters, are copied instead of re-derived for the current model and hardware", - "The precedent's expected direction of effect stands in for measurement after the change is applied", - "A rejected direction is read as a permanent blacklist, or one discarded implementation as ruling out its mechanism", - "A confounded run, a missing experiment and a failed gate are recorded as the same evidence state", - "Cross-system cases are bootstrapped without re-verifying that the mechanism holds on the target hardware" - ], - "refusal_point": "Adds no case for an INCONCLUSIVE optimization-effect claim, a single-configuration result without plausible generalization, or a result without a real measurement trail, except a scoped diagnostic observation that keeps its verdict and configuration boundary; manufactures no PASS, and reading the records does not reopen a paused campaign" - }, - "recommendations": [ - "Search the full case bodies with free text first, then rank the hits by bottleneck class and signals.", - "Before retrying a rejected direction, state which condition changed or which uncertainty a bounded check resolves, run an unchanged control and choose the least expensive discriminating check.", - "Mark bootstrapped cases source_grounded or author_reported until a local measurement confirms or refutes them.", - "When a case's conditions no longer hold, mark it deprecated with the reason instead of deleting it." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Structuring and querying a casebook of past optimization decisions as decision support, when recording optimization experiments or choosing the next optimization from prior art; the casebook is reference material and neither runs profiling nor makes decisions", - "prerequisites": "Whole-body text access to the case files for free-text search; applying a case first requires reading the nearest checked-in config and the model's deployment guide", - "input_contract": "For a query: natural search terms such as model name, knob, mechanism word or deployment feature, plus the current model and hardware. For a new case: mechanism, applies-when and does-not-apply conditions, procedure, expected effect, accuracy risk, verification, rollback and a real measurement trail with run ID, verdict, exact hardware SKU, model revision, workload and before/after result", - "produced_artifact": "Ranked matching cases read as precedents and diagnostic options, or a new or deprecated case record in the template; a precedent's procedure and expected direction of effect, not a guaranteed configuration or a measured result", - "failure_modes": [ - "The query is pre-translated into canonical vocabulary or bounded by structured fields, so synonyms in the body text are missed", - "A case's specific values, such as batch size, tensor-parallel degree or kernel parameters, are copied instead of re-derived for the current model and hardware", - "The precedent's expected direction of effect stands in for measurement after the change is applied", - "A rejected direction is read as a permanent blacklist, or one discarded implementation as ruling out its mechanism", - "A confounded run, a missing experiment and a failed gate are recorded as the same evidence state", - "Cross-system cases are bootstrapped without re-verifying that the mechanism holds on the target hardware" - ], - "refusal_point": "Adds no case for an INCONCLUSIVE optimization-effect claim, a single-configuration result without plausible generalization, or a result without a real measurement trail, except a scoped diagnostic observation that keeps its verdict and configuration boundary; manufactures no PASS, and reading the records does not reopen a paused campaign" - }, - "recommendations": [ - "Search the full case bodies with free text first, then rank the hits by bottleneck class and signals.", - "Before retrying a rejected direction, state which condition changed or which uncertainty a bounded check resolves, run an unchanged control and choose the least expensive discriminating check.", - "Mark bootstrapped cases source_grounded or author_reported until a local measurement confirms or refutes them.", - "When a case's conditions no longer hold, mark it deprecated with the reason instead of deleting it." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json deleted file mode 100644 index 4c2419600..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json +++ /dev/null @@ -1,321 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.kernel_writing_guide", - "title": "kernel-writing-guide", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL, CUDA C++, AscendC/TileIR), universal correctness rules that apply regardless of framework, and the verify-before-benchmark workflow. Use before starting any kernel implementation to avoid common correctness and performance pitfalls.\n", - "tags": [ - "kernel", - "triton", - "cute", - "cuda", - "ascendc", - "performance", - "accelerator", - "akg", - "affine", - "tail-guards", - "causal-conv1d", - "state-update", - "vllm", - "gdn", - "causal-convolution", - "transformers", - "qwen3.5", - "binding", - "classification-prefill", - "sglang", - "prefill", - "state-ownership" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/kernel-writing-guide/SKILL.md" - }, - "checkout_ref": "checkout:kernel-writing-guide", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md": "360361f592ec3a24834198372cbf92aaedc1720404d182df94315eb0e01d58dd", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/causal-conv1d-dispatch-state.md": "2eb67dd522021bb2234616d7f660e6ba99acb43676f48cb23141bc69bf3e0029", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/hf-qwen35-gdn-binding.md": "b3a55af3f45f01c48e1cb18c8b029e5949a494c3e51dc444a5cccbad9acbba0a", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.json": "5237ac437eb76e910bb1d08bfcc364b87dea1b947de5e67eebce511ea1ef1d23", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.md": "8b093faf0ab775d5605ecdd9653fb3689c2b839cba2641c5d5cda17c721bb2ac", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-causal-conv-contract.md": "2f3dcee66e2dfe7a98bee21a36b892538998de5de3caf1bc63257137f9984236", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.json": "31cfac1e780113c8233b451af7a7562af983aa2f4209938017cc67347ee5dd66", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.md": "7f9ebe8de7e55d3db2a119b6a56e15c37448fd7d4e022a47122a143206d31687", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/metadata.json": "766aa688d573c790789e2836f952315e91c4e06806fca3d04523a8a748bf9c32", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/akg-affine-tail-guards.md": "113deb2db10b171db3425bb3f1792bee763706b794c8cc2e222b4cc0d0114118", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/causal-conv1d-contract.md": "b3a6245cf5e9deeba0dffd4a15ddcf97da6a4315f4d3131921779438832663ad", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-binding.md": "848d3c99b47ef711e8820bbc4f6bcbd1db7211197811c9d855201025448d806b", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-source.json": "df3f4650b5cf92d4fd3d5a4cc8aefb91dd4b0c2affaa0c873138d5651c50e6bd", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/licenses/sglang-LICENSE.md": "1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/pplx-gdn-caller-context.json": "0219fc9ad785da4e638c1680bc9eaf28672aa32313824f52bdcaed61d29bca87", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md": "c9f27b0231ddec3d14d6fa0831d1494ffd8e11bd8ad704e04be3bc4a4ffc0e58", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-source-files.json": "4c7ec2cfd2c29297e185ce09b4d9b2040fbe3635943c112f9ebb71dca8ab381b", - "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md": "eec09a6e95983c46b67d5f066d34419c17819899f5d8e3c65f2f163f2ada0fde" - }, - "skill_metadata": { - "name": "kernel-writing-guide", - "description": "Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL, CUDA C++, AscendC/TileIR), universal correctness rules that apply regardless of framework, and the verify-before-benchmark workflow. Use before starting any kernel implementation to avoid common correctness and performance pitfalls.\n", - "tags": [ - "kernel", - "triton", - "cute", - "cuda", - "ascendc", - "performance", - "accelerator", - "akg", - "affine", - "tail-guards", - "causal-conv1d", - "state-update", - "vllm", - "gdn", - "causal-convolution", - "transformers", - "qwen3.5", - "binding", - "classification-prefill", - "sglang", - "prefill", - "state-ownership" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/Dao-AILab/causal-conv1d", - "commit": "cd81f0413cad2fc1e6f17e785ac39f59aae690cd", - "paths": [ - "LICENSE", - "README.md", - "causal_conv1d/__init__.py", - "causal_conv1d/causal_conv1d_interface.py", - "causal_conv1d/cpp_functions.py", - "csrc/causal_conv1d.cpp", - "csrc/causal_conv1d_fwd.cu", - "csrc/causal_conv1d_update.cu", - "setup.py", - "tests/test_causal_conv1d.py" - ], - "readset": "references/causal-conv1d-contract.md", - "license": "BSD-3-Clause", - "scope": "Forward/update caller contracts, selected packed/state kernel branches and build/binding evidence; no native execution." - }, - { - "type": "git", - "url": "https://github.com/vllm-project/vllm", - "commit": "537a79f0fc5ba564d2bef43d54915b2f80f9f77d", - "paths": [ - "LICENSE", - "vllm/model_executor/models/qwen3_next.py", - "vllm/model_executor/layers/mamba/gdn/qwen_gdn_linear_attn.py", - "vllm/model_executor/layers/mamba/gdn/base.py", - "vllm/model_executor/layers/mamba/mamba_utils.py", - "vllm/model_executor/layers/mamba/ops/causal_conv1d.py", - "vllm/v1/attention/backends/gdn_attn.py", - "vllm/v1/attention/backends/utils.py", - "vllm/envs.py", - "vllm/third_party/flash_linear_attention/LICENSE", - "tests/v1/attention/test_gdn_metadata_builder.py", - "tests/kernels/mamba/test_causal_conv1d.py" - ], - "readset": "references/vllm-gdn-causal-conv-contract.md" - }, - { - "type": "git", - "url": "https://github.com/huggingface/transformers", - "commit": "0dad7b822255a0ae261ec45ae937371e859ffd1a", - "paths": [ - "LICENSE", - "src/transformers/models/qwen3_5/modeling_qwen3_5.py", - "src/transformers/models/qwen3_5/configuration_qwen3_5.py", - "src/transformers/cache_utils.py", - "src/transformers/utils/import_utils.py" - ], - "readset": "references/hf-qwen35-gdn-source.json", - "license": "Apache-2.0", - "scope": "v5.10.2 Qwen3.5 conv/GDN binding, cache and local numerical contract; candidate, source-only" - }, - { - "type": "git", - "url": "https://github.com/huggingface/transformers", - "commit": "856157a2f3e9594954310df18fdccc31ffddebe9", - "paths": [ - "LICENSE", - "src/transformers/models/qwen3_5/modeling_qwen3_5.py", - "src/transformers/models/qwen3_5/configuration_qwen3_5.py", - "src/transformers/cache_utils.py", - "src/transformers/utils/import_utils.py", - "src/transformers/integrations/hub_kernels.py", - "src/transformers/integrations/__init__.py", - "src/transformers/masking_utils.py" - ], - "readset": "references/hf-qwen35-gdn-source.json", - "license": "Apache-2.0", - "scope": "v5.17.0 Qwen3.5 conv/GDN binding, cache and local numerical contract; candidate, source-only" - }, - { - "type": "git", - "url": "https://github.com/sgl-project/sglang", - "commit": "16176278d0180f1a1a578e53e6eef432577876c2", - "paths": [ - "LICENSE", - "python/sglang/kernels/ops/mamba/causal_conv1d.py", - "python/sglang/kernels/ops/mamba/causal_conv1d_triton.py", - "python/sglang/srt/arg_groups/fields/exec_.py", - "python/sglang/srt/configs/hybrid_arch.py", - "python/sglang/srt/configs/linear_attn_model_registry.py", - "python/sglang/srt/configs/mamba_utils.py", - "python/sglang/srt/configs/qwen3_5.py", - "python/sglang/srt/configs/qwen3_next.py", - "python/sglang/srt/layers/attention/attention_registry.py", - "python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py", - "python/sglang/srt/layers/attention/linear/gdn_backend.py", - "python/sglang/srt/layers/attention/linear/kernels/gdn_cutedsl.py", - "python/sglang/srt/layers/attention/linear/kernels/gdn_flashinfer.py", - "python/sglang/srt/layers/attention/linear/kernels/gdn_triton.py", - "python/sglang/srt/layers/attention/linear/utils.py", - "python/sglang/srt/layers/attention/mamba/causal_conv1d.py", - "python/sglang/srt/layers/radix_linear_attention.py", - "python/sglang/srt/mem_cache/memory_pool.py", - "python/sglang/srt/models/qwen3_5.py", - "python/sglang/srt/models/qwen3_5_text.py", - "python/sglang/srt/models/qwen3_next.py", - "test/registered/layers/mamba/test_causal_conv1d.py", - "test/registered/unit/configs/test_linear_attn_model_registry.py", - "test/registered/unit/layers/attention/test_gdn_prefill_backend_policy.py", - "test/registered/unit/layers/attention/test_linear_attn_config.py", - "test/registered/unit/layers/test_gdn_mis_metadata.py" - ], - "readset": "references/sglang-gdn-causal-conv-contract.md", - "license": "Apache-2.0", - "scope": "Bounded Qwen3.5/Qwen3-Next caller, GDN phase/architecture policy, separate conv wrappers and state metadata; no native execution or external kernel equivalence." - }, - { - "type": "git", - "url": "https://github.com/InferMatrixLab/system1-omni", - "commit": "509bdd2161ca5bef8f5e158d466514862fac7c9e", - "paths": [ - "src/models/pplx_decider/conv.py", - "src/models/pplx_decider/engine.py", - "src/models/pplx_decider/gdn.py" - ], - "readset": "references/pplx-gdn-caller-context.json", - "license": "UNKNOWN", - "scope": "Bounded secondary PPLX caller context for the HF/vLLM/SGLang cross-map: source-level default/override, conv and FI adapter/state guards only; no implementation vendored or execution qualification." - } - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton, CuTe DSL, CUDA C++, AscendC or TileIR, applying framework-independent correctness rules, and verifying the executing kernel against a reference before benchmarking; covers fused elementwise chains, reductions, tiled GEMM, attention variants, custom collectives, causal convolution and stateful serving-operator qualification", - "prerequisites": "The applicable operator, caller/state and layout contracts are read before substitution, including the causal-conv1d contract and HF Qwen3.5 GDN binding; the CuTe DSL environment uses Linux x86_64, Python 3.10–3.13 and Ampere+ (SM80+) with nvidia-cutlass-dsl installed and no NPU target; the CANN programming guide for the specific SoC is consulted for AscendC", - "input_contract": "The candidate operation or operation chain, its shapes, dtypes, quantization and layout requirements, and the target execution phase; the target accelerator and framework constraints; for serving operators, the concrete caller, effective phase and backend, state-row mapping, cache or buffer lifetime and in-place-write behavior; a reference implementation with the same signature, a callable kernel wrapper and fresh CUDA or NPU test tensors; verification shapes and dtypes plus profiler output for the target metric", - "produced_artifact": "A write-or-decline decision for the candidate operation with the profiled bottleneck, fusable pair or required dtype, shape or quantization variant unsupported by the standard implementation that justifies it, a framework choice routed by pattern and accelerator, a correctness checklist covering fp32 transcendental round-trips, sync-free wrappers, C division semantics, pre-allocated outputs, fp32 matmul accumulators and masked boundary loads and stores, a starting block or tile size with its autotune and warmup cost noted, a kernel shipping kernel_fn, reference_fn and get_inputs, a reference-parity result across the verification shapes and dtypes, and a post-verification profiler reading on the target metric; for an existing causal-convolution or stateful serving operator, a qualification record naming the caller, phase, backend, state rows and in-place writes, with source selection kept apart from executed dispatch; a routing and correctness decision, not a complete implementation or a measured speedup", - "failure_modes": [ - "A kernel is written for a standalone elementwise op, a standalone matmul or standard attention where the vendor library already wins", - "Host-side scheduling cost is treated as a kernel problem instead of being routed to host-overhead-profiling", - "Fusion is justified by removed Python call overhead rather than by the DRAM traffic the shared data no longer makes", - "A transcendental runs on fp16/bf16 inputs without the fp32 cast round-trip, so the error stays silent through benchmarking", - "tl.dot accumulates in the input dtype because Triton is assumed to promote fp16/bf16 to fp32 automatically", - "Negative offsets are computed assuming Python floor division while Triton or AscendC rounds toward zero", - "An unmasked load or store is accepted because the benchmark shapes happen to divide the block size", - "A kernel is benchmarked before it matches its reference, or a shape-boundary, accumulator-dtype or precision failure is read as a tuning problem", - "A verification pass is reported as a win although the profiler metric did not move, so the kernel was never on the critical path", - "Autotune compile cost is measured as steady-state serving latency instead of being pre-warmed or cached", - "A CuTe DSL pattern is routed to an NPU target, or a CUDA tail-guard, attention primitive or SM-matched example is assumed to carry to Ascend", - "A causal-conv1d or GDN operator is substituted on build success, import success or matching names without resolving dtype, strides, packed boundaries, state rows and cursors", - "An in-tree vLLM wrapper, a GDN prefill flag or a cache-free classification adapter is read as establishing the executed convolution path or generation-state support" - ], - "refusal_point": "Declines to start a kernel until a profiler puts the target on the critical path near its hardware ceiling, a concrete fusable pair is identified, or the required dtype, shape or quantization variant is unsupported by the standard implementation, and sends host-bound cases elsewhere; refuses to benchmark an unverified kernel and withholds a gain whose profiler metric did not move; keeps operator source selection separate from executed dispatch, numerical parity and hardware qualification, so build or import success qualifies nothing; treats AscendC and TileIR content as source-inferred pending CANN and SoC verification with no local NPU run, treats the AKG Affine FileCheck expectations as IR structure rather than executed CUDA correctness, and reads author-reported overlap and FP8 cases as counterexamples to unconditional tuning rules; claims no Ascend support from NVIDIA primitives", - "hardware": "CUDA Ampere+ and Ascend NPU have separate framework routes for each kernel pattern; CuTe DSL requires Linux x86_64, Python 3.10-3.13 and SM80+ with nvidia-cutlass-dsl and is unavailable on NPU; profiling uses ncu on CUDA and msprof on Ascend; the AscendC and TileIR guidance is source-inferred with no local NPU verification and requires validation against the specific CANN version and SoC" - }, - "recommendations": [ - "Confirm with nsys/ncu or msprof that the target kernel sits on the critical path near its hardware ceiling, name the adjacent operations whose shared data would otherwise round-trip through DRAM, or identify a required dtype, shape or quantization variant unsupported by the standard implementation, before writing any kernel.", - "Ship every kernel with kernel_fn, reference_fn and get_inputs, pass reference parity across the verification shapes and dtypes first, and profile afterwards to confirm the target metric actually moved.", - "Cast fp16/bf16 to fp32 and back for any math beyond +, -, * and /, declare tl.dot accumulators as tl.float32, and mask every load and store whose shape does not divide the block size.", - "Start Triton at BLOCK_SIZE 1024 on Hopper and 512 on Ampere using powers of two, take GEMM tiles from the source skeleton configs (128/256/64 for Hopper, 64/128/32 for smaller cards) and autotune from there.", - "On Ascend, reach for TileIR before raw AscendC, size tiles to fit the AI Core Unified Buffer and check cube-unit alignment against the CANN guide for the specific SoC.", - "Match a CuTe DSL example to the GPU architecture (SM80 Ampere, SM90 Hopper, SM100 Blackwell) and study its structure instead of copying it, and pre-warm or cache autotuned kernels at startup so first-run compile cost stays out of serving latency." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "The typed vLLM entry covers only the new GDN reference; older source inventories remain incomplete.", - "The vLLM and HF references identify separate source call contracts; no binary equivalence, executed backend, hardware qualification, numerical/model parity, performance or agent-efficacy evaluation.", - "SGLang source contract only; native/backend/sm89 qualification, HF classification binding, numerical/model quality, performance and agent efficacy are not evaluated.", - "The secondary PPLX caller readset has UNKNOWN repository licensing and source-only default/adapter evidence; no worker/kernel execution, model/quality/performance or stateful reuse is qualified." - ], - "provenance_readsets": [ - "references/akg-affine-tail-guards.md", - "references/causal-conv1d-contract.md", - "references/vllm-gdn-causal-conv-contract.md", - "references/hf-qwen35-gdn-binding.md", - "references/hf-qwen35-gdn-source.json", - "references/sglang-gdn-causal-conv-contract.md", - "references/pplx-gdn-caller-context.json" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton, CuTe DSL, CUDA C++, AscendC or TileIR, applying framework-independent correctness rules, and verifying the executing kernel against a reference before benchmarking; covers fused elementwise chains, reductions, tiled GEMM, attention variants, custom collectives, causal convolution and stateful serving-operator qualification", - "prerequisites": "The applicable operator, caller/state and layout contracts are read before substitution, including the causal-conv1d contract and HF Qwen3.5 GDN binding; the CuTe DSL environment uses Linux x86_64, Python 3.10–3.13 and Ampere+ (SM80+) with nvidia-cutlass-dsl installed and no NPU target; the CANN programming guide for the specific SoC is consulted for AscendC", - "input_contract": "The candidate operation or operation chain, its shapes, dtypes, quantization and layout requirements, and the target execution phase; the target accelerator and framework constraints; for serving operators, the concrete caller, effective phase and backend, state-row mapping, cache or buffer lifetime and in-place-write behavior; a reference implementation with the same signature, a callable kernel wrapper and fresh CUDA or NPU test tensors; verification shapes and dtypes plus profiler output for the target metric", - "produced_artifact": "A write-or-decline decision for the candidate operation with the profiled bottleneck, fusable pair or required dtype, shape or quantization variant unsupported by the standard implementation that justifies it, a framework choice routed by pattern and accelerator, a correctness checklist covering fp32 transcendental round-trips, sync-free wrappers, C division semantics, pre-allocated outputs, fp32 matmul accumulators and masked boundary loads and stores, a starting block or tile size with its autotune and warmup cost noted, a kernel shipping kernel_fn, reference_fn and get_inputs, a reference-parity result across the verification shapes and dtypes, and a post-verification profiler reading on the target metric; for an existing causal-convolution or stateful serving operator, a qualification record naming the caller, phase, backend, state rows and in-place writes, with source selection kept apart from executed dispatch; a routing and correctness decision, not a complete implementation or a measured speedup", - "failure_modes": [ - "A kernel is written for a standalone elementwise op, a standalone matmul or standard attention where the vendor library already wins", - "Host-side scheduling cost is treated as a kernel problem instead of being routed to host-overhead-profiling", - "Fusion is justified by removed Python call overhead rather than by the DRAM traffic the shared data no longer makes", - "A transcendental runs on fp16/bf16 inputs without the fp32 cast round-trip, so the error stays silent through benchmarking", - "tl.dot accumulates in the input dtype because Triton is assumed to promote fp16/bf16 to fp32 automatically", - "Negative offsets are computed assuming Python floor division while Triton or AscendC rounds toward zero", - "An unmasked load or store is accepted because the benchmark shapes happen to divide the block size", - "A kernel is benchmarked before it matches its reference, or a shape-boundary, accumulator-dtype or precision failure is read as a tuning problem", - "A verification pass is reported as a win although the profiler metric did not move, so the kernel was never on the critical path", - "Autotune compile cost is measured as steady-state serving latency instead of being pre-warmed or cached", - "A CuTe DSL pattern is routed to an NPU target, or a CUDA tail-guard, attention primitive or SM-matched example is assumed to carry to Ascend", - "A causal-conv1d or GDN operator is substituted on build success, import success or matching names without resolving dtype, strides, packed boundaries, state rows and cursors", - "An in-tree vLLM wrapper, a GDN prefill flag or a cache-free classification adapter is read as establishing the executed convolution path or generation-state support" - ], - "refusal_point": "Declines to start a kernel until a profiler puts the target on the critical path near its hardware ceiling, a concrete fusable pair is identified, or the required dtype, shape or quantization variant is unsupported by the standard implementation, and sends host-bound cases elsewhere; refuses to benchmark an unverified kernel and withholds a gain whose profiler metric did not move; keeps operator source selection separate from executed dispatch, numerical parity and hardware qualification, so build or import success qualifies nothing; treats AscendC and TileIR content as source-inferred pending CANN and SoC verification with no local NPU run, treats the AKG Affine FileCheck expectations as IR structure rather than executed CUDA correctness, and reads author-reported overlap and FP8 cases as counterexamples to unconditional tuning rules; claims no Ascend support from NVIDIA primitives", - "hardware": "CUDA Ampere+ and Ascend NPU have separate framework routes for each kernel pattern; CuTe DSL requires Linux x86_64, Python 3.10-3.13 and SM80+ with nvidia-cutlass-dsl and is unavailable on NPU; profiling uses ncu on CUDA and msprof on Ascend; the AscendC and TileIR guidance is source-inferred with no local NPU verification and requires validation against the specific CANN version and SoC" - }, - "recommendations": [ - "Confirm with nsys/ncu or msprof that the target kernel sits on the critical path near its hardware ceiling, name the adjacent operations whose shared data would otherwise round-trip through DRAM, or identify a required dtype, shape or quantization variant unsupported by the standard implementation, before writing any kernel.", - "Ship every kernel with kernel_fn, reference_fn and get_inputs, pass reference parity across the verification shapes and dtypes first, and profile afterwards to confirm the target metric actually moved.", - "Cast fp16/bf16 to fp32 and back for any math beyond +, -, * and /, declare tl.dot accumulators as tl.float32, and mask every load and store whose shape does not divide the block size.", - "Start Triton at BLOCK_SIZE 1024 on Hopper and 512 on Ampere using powers of two, take GEMM tiles from the source skeleton configs (128/256/64 for Hopper, 64/128/32 for smaller cards) and autotune from there.", - "On Ascend, reach for TileIR before raw AscendC, size tiles to fit the AI Core Unified Buffer and check cube-unit alignment against the CANN guide for the specific SoC.", - "Match a CuTe DSL example to the GPU architecture (SM80 Ampere, SM90 Hopper, SM100 Blackwell) and study its structure instead of copying it, and pre-warm or cache autotuned kernels at startup so first-run compile cost stays out of serving latency." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "The typed vLLM entry covers only the new GDN reference; older source inventories remain incomplete.", - "The vLLM and HF references identify separate source call contracts; no binary equivalence, executed backend, hardware qualification, numerical/model parity, performance or agent-efficacy evaluation.", - "SGLang source contract only; native/backend/sm89 qualification, HF classification binding, numerical/model quality, performance and agent efficacy are not evaluated.", - "The secondary PPLX caller readset has UNKNOWN repository licensing and source-only default/adapter evidence; no worker/kernel execution, model/quality/performance or stateful reuse is qualified." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/kv-tier-management.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/kv-tier-management.json deleted file mode 100644 index ee8731dc0..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/kv-tier-management.json +++ /dev/null @@ -1,96 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.kv_tier_management", - "title": "kv-tier-management", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Manage runtime KV cache tiers (GPU → CPU RAM → local disk) for LLM serving: decide when KV is stored, retained, prefetched, or evicted; size the concurrency-versus-RAM-pressure trade-off with an explicit in-flight admission dial; order RAM-release levers; and measure with token-level L1/L2 hit rates and prefetch-pressure counters. Use when configuring KV offload for a serving stack. Covers runtime KV offload only — model startup/loading, checkpoint save/restore, and block-device profiling are separate concerns (see weight-loading-optimization).\n", - "tags": [ - "kv-cache", - "offload", - "l1-l2-tiering", - "eviction", - "prefetch", - "serving", - "lmcache" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md" - }, - "checkout_ref": "checkout:kv-tier-management", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md": "e4b78f368d70c8abb74a935acb910414015f1aeec1b50f7a295e836822ec4b27", - "src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/metadata.json": "85f7242b83334af11f57799875738b498bfe4680a51f36727792dfbe91806704" - }, - "skill_metadata": { - "name": "kv-tier-management", - "description": "Manage runtime KV cache tiers (GPU → CPU RAM → local disk) for LLM serving: decide when KV is stored, retained, prefetched, or evicted; size the concurrency-versus-RAM-pressure trade-off with an explicit in-flight admission dial; order RAM-release levers; and measure with token-level L1/L2 hit rates and prefetch-pressure counters. Use when configuring KV offload for a serving stack. Covers runtime KV offload only — model startup/loading, checkpoint save/restore, and block-device profiling are separate concerns (see weight-loading-optimization).\n", - "tags": [ - "kv-cache", - "offload", - "l1-l2-tiering", - "eviction", - "prefetch", - "serving", - "lmcache" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "LMCache multiprocess runtime KV tier policy across engine GPU cache, host L1 and configured L2 adapters, covering store/retention, prefetch admission, eviction and RAM pressure; startup loading and checkpoint recovery have separate owners.", - "prerequisites": "The deployment uses the inspected MP-mode semantics rather than single-process knobs; selected GDS/DAX/raw-block paths need their own platform and adapter prerequisites. Lazy offload additionally requires vLLM prefix caching.", - "input_contract": "MP versus single-process mode, store/retention policy and rationale, L1 manager capacities and eviction settings, L2 adapter capacities/eviction, chunk-size distribution and prefetch in-flight setting, together with token-level hit and pressure counters.", - "produced_artifact": "A mode- and adapter-specific tier-policy choice with retention rationale, an ordered RAM-release plan and a verification record separating L1/L2 token hits, prefetch pressure, store lag and per-manager usage.", - "failure_modes": "Mixing MP and in-process controls, treating key-count eviction as a byte fraction for unequal chunks, omitting fs capacity management, or increasing count-based prefetch admission without tracking L1 pressure can make the policy or memory claim unusable." - }, - "recommendations": [ - "Choose store retention and prefetched-key retention separately; use skip_l1 with noop only for buffer-only operation and account for failed stores remaining in L1.", - "Tune the count-based prefetch in-flight limit against L1 allocation failures and in-flight bytes; eager or warm prefetch can raise pressure before useful reuse.", - "Record per-adapter capacity/eviction and actual chunk-size uniformity; report token-level L1/L2 hits and sum per-l1_tag bytes rather than ratios." - ] - }, - "applicability": { - "scope": "LMCache multiprocess runtime KV tier policy across engine GPU cache, host L1 and configured L2 adapters, covering store/retention, prefetch admission, eviction and RAM pressure; startup loading and checkpoint recovery have separate owners.", - "prerequisites": "The deployment uses the inspected MP-mode semantics rather than single-process knobs; selected GDS/DAX/raw-block paths need their own platform and adapter prerequisites. Lazy offload additionally requires vLLM prefix caching.", - "input_contract": "MP versus single-process mode, store/retention policy and rationale, L1 manager capacities and eviction settings, L2 adapter capacities/eviction, chunk-size distribution and prefetch in-flight setting, together with token-level hit and pressure counters.", - "produced_artifact": "A mode- and adapter-specific tier-policy choice with retention rationale, an ordered RAM-release plan and a verification record separating L1/L2 token hits, prefetch pressure, store lag and per-manager usage.", - "failure_modes": "Mixing MP and in-process controls, treating key-count eviction as a byte fraction for unequal chunks, omitting fs capacity management, or increasing count-based prefetch admission without tracking L1 pressure can make the policy or memory claim unusable." - }, - "recommendations": [ - "Choose store retention and prefetched-key retention separately; use skip_l1 with noop only for buffer-only operation and account for failed stores remaining in L1.", - "Tune the count-based prefetch in-flight limit against L1 allocation failures and in-flight bytes; eager or warm prefetch can raise pressure before useful reuse.", - "Record per-adapter capacity/eviction and actual chunk-size uniformity; report token-level L1/L2 hits and sum per-l1_tag bytes rather than ratios." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lazy-evaluation-stream-lifetimes.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lazy-evaluation-stream-lifetimes.json deleted file mode 100644 index 789d06801..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lazy-evaluation-stream-lifetimes.json +++ /dev/null @@ -1,126 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.lazy_evaluation_stream_lifetimes", - "title": "lazy-evaluation-stream-lifetimes", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose MLX lazy evaluation, cross-stream ordering and array-buffer lifetimes when timing, memory use or asynchronous results disagree with expectations.", - "tags": [ - "mlx", - "lazy-evaluation", - "stream", - "buffer-lifetime", - "synchronization" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" - }, - "checkout_ref": "checkout:lazy-evaluation-stream-lifetimes", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md": "a95569bcc0da6e715fd74783a6a1e0489e92b680c582f15dafbf98df589d505a", - "src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/metadata.json": "cfa57ac908025135113d04c3d64b9967f8d9a3af4a00707673d94b8227c85808", - "src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md": "c4bc41e08aca68f661325f61e6f0db1760a079dc362f295c847bdc374fee3bf6", - "src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md": "798b9cb89ed6b885eb47464ca9f137d2802a54571b9327e631d7742d3e5abd5d", - "src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/validation.md": "ebfc00bf7d6ab0a1dc415953ae84d56ccdc713d5614aff85e433c4a843793bff" - }, - "skill_metadata": { - "name": "lazy-evaluation-stream-lifetimes", - "description": "Diagnose MLX lazy evaluation, cross-stream ordering and array-buffer lifetimes when timing, memory use or asynchronous results disagree with expectations.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "MIT", - "tags": [ - "mlx", - "lazy-evaluation", - "stream", - "buffer-lifetime", - "synchronization" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ml-explore/mlx", - "commit": "bff10220c742907f0c68e55ff174aaea56f74e3d", - "paths": [ - "mlx/ops.cpp", - "mlx/transforms.cpp", - "mlx/array.cpp", - "mlx/backend/cpu/eval.cpp", - "mlx/backend/metal/eval.cpp", - "mlx/backend/cuda/eval.cpp", - "python/src/transforms.cpp", - "python/tests/test_eval.py", - "LICENSE", - "mlx/scheduler.cpp", - "python/tests/test_memory.py", - "CMakeLists.txt", - "setup.py", - "python/tests/test_device.py", - "python/tests/mlx_tests.py", - ".github/workflows/build_and_test.yml" - ], - "readset": "references/source-map.md" - } - ], - "applicability": { - "scope": "MLX evaluation/stream/buffer diagnosis at the pinned implementation; qualify the actual backend separately.", - "prerequisites": "Access to the pinned MLX execution/lifetime contract and a bounded reproduction with an identified evaluation/read boundary; device-dependent conclusions need the actual selected backend.", - "input_contract": "MLX revision/build, backend/device, Python/C++ entry, array shapes/dtypes, primitive streams, evaluation/read boundary and transformation context; timing needs warmup/work boundaries, while memory needs live references, in-flight buffers, allocator cache and synchronized observations.", - "produced_artifact": "A producer/evaluation/dependency-wait/backend/availability-cleanup trace, ownership table, observed symptom and one discriminating check with build-bound actual evidence status.", - "failure_modes": "Timing graph construction or submission as execution, assuming evaluated means completed, synchronizing only the default stream, or treating graph detachment/Python deletion as reusable storage can misdiagnose time or memory.", - "refusal_point": "Stop when the trace answers the question; if the required backend is absent, retain the source-supported diagnosis and exact runtime gap rather than substituting CPU import success." - }, - "limitations": [ - "Source-derived candidate; no local native-backend, model, service or efficacy validation.", - "Static content checks do not promote this candidate or establish installed API conformance." - ], - "provenance_readsets": [ - "references/source-map.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Separate graph construction, async_eval submission, eval and host reads, and record which stream the measurement actually synchronizes.", - "Preserve the dependency-wait path and selected backend buffer-retention/donation contract before changing streams or assuming storage can be reused.", - "Distinguish live, in-flight and allocator-cached memory; inspect the real error/cleanup path before proposing a leak fix or hiding the failure." - ] - }, - "applicability": { - "scope": "MLX evaluation/stream/buffer diagnosis at the pinned implementation; qualify the actual backend separately.", - "prerequisites": "Access to the pinned MLX execution/lifetime contract and a bounded reproduction with an identified evaluation/read boundary; device-dependent conclusions need the actual selected backend.", - "input_contract": "MLX revision/build, backend/device, Python/C++ entry, array shapes/dtypes, primitive streams, evaluation/read boundary and transformation context; timing needs warmup/work boundaries, while memory needs live references, in-flight buffers, allocator cache and synchronized observations.", - "produced_artifact": "A producer/evaluation/dependency-wait/backend/availability-cleanup trace, ownership table, observed symptom and one discriminating check with build-bound actual evidence status.", - "failure_modes": "Timing graph construction or submission as execution, assuming evaluated means completed, synchronizing only the default stream, or treating graph detachment/Python deletion as reusable storage can misdiagnose time or memory.", - "refusal_point": "Stop when the trace answers the question; if the required backend is absent, retain the source-supported diagnosis and exact runtime gap rather than substituting CPU import success." - }, - "recommendations": [ - "Separate graph construction, async_eval submission, eval and host reads, and record which stream the measurement actually synchronizes.", - "Preserve the dependency-wait path and selected backend buffer-retention/donation contract before changing streams or assuming storage can be reused.", - "Distinguish live, in-flight and allocator-cached memory; inspect the real error/cleanup path before proposing a leak fix or hiding the failure." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local native-backend, model, service or efficacy validation.", - "Static content checks do not promote this candidate or establish installed API conformance." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lily.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lily.json deleted file mode 100644 index 3be7a0279..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lily.json +++ /dev/null @@ -1,202 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.lily", - "title": "lily", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Investigate Lily's exact Qwen3.6-35B-A3B MLX-affine-Q4 contract, Metal state and kernels, strict greedy text API, token-prefix session cache and source-qualified validation or benchmark plans.", - "tags": [ - "lily", - "rust", - "metal", - "apple-silicon", - "mlx-affine-q4", - "session-prefix", - "numerical-validation", - "benchmark-provenance" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/lily/SKILL.md" - }, - "checkout_ref": "checkout:lily", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/lily/LICENSE": "b9e2b50a48a09fb39dafa71ab1a11a6e87389addda17faa99d55f5f74a0001ac", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/NOTICE.md": "7a01e666044fb8aca1fd88c636bca43bcc95b8b1fecbe101dce4c7c80fbb5759", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/SKILL.md": "db815674d5d36964902041d310d3df844d00ab2e6bef0ce4b72ed2696ab2b9aa", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/metadata.json": "52f01d58613e1a93e9cfd958ceca9ab5061baa07c5f94572393520434892b420", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/references/repo-provenance.md": "96182f6cb3666f89adf1aed6e216c0a84ce3a04a0685a4257e286100ba8ee367", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/references/repo-routing-metadata.json": "f2f3becb51056e03081c7c4140e19572a6318c3d63015884df56eb49d38a00fc", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/references/source-identity.json": "4595559c11611c855be150919fca3eeb89e7f96c7a8dd133b9f79a5cd8e05e62", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/SKILL.md": "dc800f07fa0e82213483c7b848dea95d609386296006bfec7a34cbb194a83de6", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/references/model-and-memory.md": "d5a862303d646ebbaf745818caed8f96c7323cd442cb6d6e2aae00b0361080a1", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/references/troubleshooting.md": "b32fab02b561c3ab4e683abdfa80cd4a8f929c514147709febaa6b6d1df65909", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/scripts/screen_config.py": "5961bb6f31ceca43473683127b37faee268234040a013ae49bbb4898eacb6f04", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/SKILL.md": "0231b50b9bebb68d0fc5f2ce19aa7c6e5ffd498d8eef4cc00d39e82908cbbe91", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/references/troubleshooting.md": "01607534ce1f09fb046255c78f8c1c436aac61874b93b65a9d62f3eb480496c2", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/references/verification-contract.md": "6934779f67b878bf7cdb5f2cbe461f6716c243c662a29272c6db37f64cb317c7", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/scripts/audit_benchmark_record.py": "59534f58d46ad438c7bf88886a9b05a8716c922d817f2d9c58ff1b71ae50ee85", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/SKILL.md": "7e062ff9af76125ef08c98b4465e0439c66e9e237f0c39ea0285005cc5496cfb", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/references/api-and-session.md": "7262bb077c24f2f7549d450473ec7944829940d09cca89db9c12f37ac90d5db0", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/references/troubleshooting.md": "6e2f36d2f7ab9eae665d87a676e34c0ebd2f46f3df6fa5f2e589a306749a87b5", - "src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/scripts/request_template.py": "d6b00c9f0b38c7fb0a0e52e396993a832542ea0aa6e2456026fabc0c39a4d33a" - }, - "skill_metadata": { - "name": "lily", - "description": "Investigate Lily's exact Qwen3.6-35B-A3B MLX-affine-Q4 contract, Metal state and kernels, strict greedy text API, token-prefix session cache and source-qualified validation or benchmark plans.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "lily", - "rust", - "metal", - "apple-silicon", - "mlx-affine-q4", - "session-prefix", - "numerical-validation", - "benchmark-provenance" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/perplexityai/pplx-garden", - "commit": "1ed972ed3f0bd5616c997c9507c25616c63394fc", - "paths": [ - "lily/README.md", - "lily/Cargo.toml", - "lily/Cargo.lock", - "lily/rust-toolchain.toml", - "lily/CONTRIBUTING.md", - "lily/LICENSE", - "lily/NOTICE", - "lily/src/lib.rs", - "lily/src/config.rs", - "lily/src/weights.rs", - "lily/src/safetensors.rs", - "lily/src/tensor.rs", - "lily/src/metal.rs", - "lily/src/model.rs", - "lily/src/serve.rs", - "lily/src/serve/session.rs", - "lily/src/chat.rs", - "lily/src/generate.rs", - "lily/src/tokenizer.rs", - "lily/src/kernels/attention.rs", - "lily/src/kernels/gdn.rs", - "lily/src/kernels/gemm.rs", - "lily/src/kernels/skinny.rs", - "lily/src/bin/lily-bench.rs", - "lily/tests/support/cpu_ref.rs", - "lily/tests/test_shader_compile.rs", - "lily/tests/test_e2e_35b.rs", - "lily/tests/test_tokenizer.rs", - "lily/tests/unit/weights.rs", - "lily/tests/unit/serve.rs", - "lily/tests/unit/serve/session.rs", - "lily/tests/unit/generate.rs", - "lily/tests/unit/kernels/gemm.rs", - "lily/tests/unit/kernels/gdn.rs", - "lily/tests/unit/kernels/skinny.rs", - "lily/docs/2026-09-01-performance-mlx-0.31.2.md", - "lily/docs/2026-09-02-performance-mlx-0.32.2.md", - "lily/benchmarks/run_matrix.py", - "lily/benchmarks/bench_mlx.py", - "lily/benchmarks/requirements.txt" - ], - "license": "Apache-2.0", - "scope": "Lily subtree only; independent lily/LICENSE and NOTICE retained.", - "readset": "references/source-identity.json" - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Portable native Rust/Metal adaptation executed by a compatible agent; not a DisCo CLI/model run." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "User-approved native Rust/Metal and partial drafting adaptation; stable-ID lanes, integrated source graph and assertion/helper verification. Missing compiler/backend/checkpoint remains explicit." - }, - "applicability": { - "scope": "Exact Lily source/checkpoint/layout/API/cache/qualification investigation; common catalog classification is not CPU inference or cross-device support.", - "prerequisites": "Access to the exact Lily source/model-layout/API contracts; native checks require the matching Rust/Metal/checkpoint environment and frozen comparison requirements.", - "input_contract": "Source revision, Cargo lock/toolchain, macOS/chip/GPU family, checkpoint and tensor payload identities, affine configuration/layout, tokenizer/template/stops, sequence/output budgets, greedy policy, input history, prefix-cache state and quality/measurement requirements.", - "produced_artifact": "A source/model/device-bound representation, serving-cache or qualification diagnosis with the observation, invariant, alternate cause, smallest falsifier, actual result and remaining native gap.", - "failure_modes": "Equal bit width, file size, API names or prompt_cache_key do not establish tensor/token/state equivalence; ignoring the extra lookahead slot can understate request capacity, and green CPU references do not replace matched-model Metal goldens.", - "refusal_point": "Stop at unsupported formats/APIs, missing compiler/backend/checkpoint, unfrozen gates or the authorized budget; do not weaken ignored/golden tests to obtain a green result." - }, - "limitations": [ - "Cargo/Rust1.92, supported M5/macOS26/Metal and model resources are unavailable; native evidence remains BLOCKED_REQUIRED_BACKEND.", - "Offline Python helper checks are not Rust serde/loader/tokenizer, native kernel/state/cache, model quality, service or performance validation.", - "Current benchmark source and later report have distinct package/harness identities; negative long-context prefill observations remain author-reported.", - "Only selected source bodies are mined; complete kernel/format/platform/failure coverage and behavioral skill efficacy remain unverified.", - "NO_LICENSE is the actual unauthenticated GitHub CLI fallback, not a legal conclusion; lily Apache-2.0 LICENSE and NOTICE are independently retained." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve exact affine representation and strict API semantics, then trace token frontier, cache key and restore/release ownership.", - "Retain the source's extra request lookahead slot in capacity arithmetic rather than counting only prompt and visible output.", - "Separate reported benchmark version from the checked-out runner and distinguish offline screening, native compilation, Metal operators and matched-model goldens." - ] - }, - "applicability": { - "scope": "Exact Lily source/checkpoint/layout/API/cache/qualification investigation; common catalog classification is not CPU inference or cross-device support.", - "prerequisites": "Access to the exact Lily source/model-layout/API contracts; native checks require the matching Rust/Metal/checkpoint environment and frozen comparison requirements.", - "input_contract": "Source revision, Cargo lock/toolchain, macOS/chip/GPU family, checkpoint and tensor payload identities, affine configuration/layout, tokenizer/template/stops, sequence/output budgets, greedy policy, input history, prefix-cache state and quality/measurement requirements.", - "produced_artifact": "A source/model/device-bound representation, serving-cache or qualification diagnosis with the observation, invariant, alternate cause, smallest falsifier, actual result and remaining native gap.", - "failure_modes": "Equal bit width, file size, API names or prompt_cache_key do not establish tensor/token/state equivalence; ignoring the extra lookahead slot can understate request capacity, and green CPU references do not replace matched-model Metal goldens.", - "refusal_point": "Stop at unsupported formats/APIs, missing compiler/backend/checkpoint, unfrozen gates or the authorized budget; do not weaken ignored/golden tests to obtain a green result." - }, - "recommendations": [ - "Resolve exact affine representation and strict API semantics, then trace token frontier, cache key and restore/release ownership.", - "Retain the source's extra request lookahead slot in capacity arithmetic rather than counting only prompt and visible output.", - "Separate reported benchmark version from the checked-out runner and distinguish offline screening, native compilation, Metal operators and matched-model goldens." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Cargo/Rust1.92, supported M5/macOS26/Metal and model resources are unavailable; native evidence remains BLOCKED_REQUIRED_BACKEND.", - "Offline Python helper checks are not Rust serde/loader/tokenizer, native kernel/state/cache, model quality, service or performance validation.", - "Current benchmark source and later report have distinct package/harness identities; negative long-context prefill observations remain author-reported.", - "Only selected source bodies are mined; complete kernel/format/platform/failure coverage and behavioral skill efficacy remain unverified.", - "NO_LICENSE is the actual unauthenticated GitHub CLI fallback, not a legal conclusion; lily Apache-2.0 LICENSE and NOTICE are independently retained." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lithos-metal.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lithos-metal.json deleted file mode 100644 index ae15e433c..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/lithos-metal.json +++ /dev/null @@ -1,151 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.lithos_metal", - "title": "lithos-metal", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Inspect Lithos Metal compilation and fusion, GPU-resident generation and DSpark state, or local serving and exact-prefix caches on Apple silicon; use when changing recipes, diagnosing stale state, or qualifying an inference claim.", - "tags": [ - "lithos-metal", - "metal", - "apple-silicon", - "repository-guide" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/lithos-metal/SKILL.md" - }, - "checkout_ref": "checkout:lithos-metal", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/SKILL.md": "4c71ba2f26c275829466bba2412738aa27799842e642c75cdfd36d9ed8713f7f", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/metadata.json": "c5fa717d6f79d54e6aa2b0c5b672e80ac7412fbe5901ae24fda78a24c584ba67", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/mlx-license.md": "ccfab7ccb2ea306f71531c8ca77bb55507606cd90768b1e32b8b52ab5b48cf01", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/notices.md": "b2c9d2481ef2c9d16b3240c951ae65db417e5d309cd06b7917db9d84fe175886", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/package-api.md": "4f3ecfabafb2a15eddf21ccf964b8c37635bbb7862418a2c404e7049c0fa2898", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/repo-provenance.md": "c0c7c3dd45eab0d55bcf154760156db58e8e365f9cfb2c6bd246dd6a06f3ef3c", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/source-notes.md": "110dbc995c7fc727632017d4c26c71df7ecf12d4df5d89df0cd4afd75c5892f2", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/troubleshooting.md": "1894bbe1a18018f81318b11e2150201ee97f3819f84a7a81f3cf5fb01549c165", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/upstream-notice.md": "cd80bddd6b6204dbc61c6ac762ed206c73d9264755d9884383d0860de29d185d", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/scripts/inspect_contract.py": "f8365f87af2135fcf0a2c8baaefac563c30989282fe080a02d958a37a215515c", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/SKILL.md": "e0d927763f3774440abddefc45913528946cd29f7f089766fab47b212f9b6693", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/metadata.json": "b3ff3702531e023b49beded950151ec1256aab4fbf4c02e9212ef0202d3ad72e", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/references/mechanisms.md": "6173a8b73ec5138cb12791f108f3d5d83278a117d443d48187c842e16bdc6eff", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/references/troubleshooting.md": "f18ca9324d9a80f2ba327a1ad4ccf84a01049080366fe12410f2800d5f178232", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/SKILL.md": "bd8cd27ddb18a04c9e2b31ce9f16e676864133e160267473979de828b1915f63", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/metadata.json": "dacba1953fad82b32b98f76bd48009b04109f1ae446cc68be522b02a64306157", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/references/state-machine.md": "d812ad31d4e343539119cf82f8ce241fe14c691dd73398048893cc5de3ab8e22", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/references/troubleshooting.md": "7ca629e66ef3b0a67077a7d2536befef683315ad01939728a9e6e7523cc8414c", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/SKILL.md": "01c38c2678d6e8292a3908bf5619b332432c5f1289fd5d5a67eb44a2141adb90", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/metadata.json": "2fca222efac7752e29f236f40a0bc38275045826b55955687e39c64ffb69f58f", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/references/troubleshooting.md": "52c6e27a76547475b5c66121f873f4d2c1662c811a69da64ce85614f1bd44749", - "src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/references/workflows.md": "298d27d3c748fe22774b872ed0465b707e53812379a7a538c95f25d1727fc796" - }, - "skill_metadata": { - "name": "lithos-metal", - "description": "Inspect Lithos Metal compilation and fusion, GPU-resident generation and DSpark state, or local serving and exact-prefix caches on Apple silicon; use when changing recipes, diagnosing stale state, or qualifying an inference claim.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "lithos-metal", - "metal", - "apple-silicon", - "repository-guide" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/lithos-ai/lithos-metal", - "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", - "paths": [ - "kernels/common/spec_ops.metal", - "monolith/backends/metal/base.py", - "monolith/backends/metal/registry.py", - "monolith/compiler/arena.py", - "monolith/compiler/barriers.py", - "monolith/compiler/coverage.py", - "monolith/compiler/decoder_fusion.py", - "monolith/compiler/emit.py", - "monolith/compiler/region_fusion.py", - "monolith/compiler/static_fusion.py", - "monolith/core/step_state.py", - "monolith/generate.py", - "monolith/runtime/engine.py", - "monolith/runtime/prefix_cache.py", - "monolith/runtime/prefix_store.py", - "monolith/serve.py", - "monolith/serving/cache.py", - "monolith/serving/lifecycle.py", - "monolith/serving/setup.py", - "monolith/spec/dspark/model.py", - "runtime/src/metal_core.mm" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately.", - "prerequisites": "Access to the pinned package/API and selected compiler, generation or serving contracts; revision/API drift needs a fresh bounded inspection, and efficacy claims need campaign-owned gates and matching native/model evidence.", - "input_contract": "Actual source/package version, backend/configuration, chip/core count and OS, target/draft checkpoint identities, context and prompt-token IDs, sampling/numerical options and the affected recipe/runtime/session/cache boundary.", - "produced_artifact": "A routed region/dependency, committed-state or cache-identity/lifecycle ledger, followed by the smallest supported finding, missing checks and next bounded action.", - "failure_modes": "Treating session adoption/cache hits as generation correctness, fewer dispatches as faster rounds, another chip recipe as the default, CPU cache checks as GPU rollback or health liveness as loaded/warm model state can yield unsupported conclusions." - }, - "limitations": [ - "Source-derived candidate; empirical efficacy validation not_run.", - "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", - "No model downloads, live DisCo deployment or inference backend integration is provided." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "license": "Apache-2.0", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "recommendations": [ - "Follow session replacement across recipe geometry, committed generation state and serving adoption/cache identity before interpreting a cache hit.", - "Keep an unfused/native or target-only control with matching decoded weights, numerical mode, row bounds and sampling policy; verify dependencies and persistent state before timing.", - "Distinguish package/CPU checks, synthetic Metal execution, model continuation and HTTP workload evidence; chip recipes and target-compute projections do not establish another device or E2E rate." - ] - }, - "applicability": { - "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately.", - "prerequisites": "Access to the pinned package/API and selected compiler, generation or serving contracts; revision/API drift needs a fresh bounded inspection, and efficacy claims need campaign-owned gates and matching native/model evidence.", - "input_contract": "Actual source/package version, backend/configuration, chip/core count and OS, target/draft checkpoint identities, context and prompt-token IDs, sampling/numerical options and the affected recipe/runtime/session/cache boundary.", - "produced_artifact": "A routed region/dependency, committed-state or cache-identity/lifecycle ledger, followed by the smallest supported finding, missing checks and next bounded action.", - "failure_modes": "Treating session adoption/cache hits as generation correctness, fewer dispatches as faster rounds, another chip recipe as the default, CPU cache checks as GPU rollback or health liveness as loaded/warm model state can yield unsupported conclusions." - }, - "recommendations": [ - "Follow session replacement across recipe geometry, committed generation state and serving adoption/cache identity before interpreting a cache hit.", - "Keep an unfused/native or target-only control with matching decoded weights, numerical mode, row bounds and sampling policy; verify dependencies and persistent state before timing.", - "Distinguish package/CPU checks, synthetic Metal execution, model continuation and HTTP workload evidence; chip recipes and target-compute projections do not establish another device or E2E rate." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; empirical efficacy validation not_run.", - "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", - "No model downloads, live DisCo deployment or inference backend integration is provided." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/local-dream.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/local-dream.json deleted file mode 100644 index b2668764c..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/local-dream.json +++ /dev/null @@ -1,191 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.local_dream", - "title": "local-dream", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit Local Dream Android desired-versus-serving process state, remote exposure, native backend and DiT packaging/ABI selection, shared generation cancellation and prompt-cache identity without mistaking declarations or source tests for on-device diffusion qualification.", - "tags": [ - "local-dream", - "android", - "snapdragon", - "qnn", - "hexagon", - "mnn", - "diffusion", - "lifecycle", - "cancellation", - "prompt-cache", - "abi", - "lan-exposure" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/local-dream/SKILL.md" - }, - "checkout_ref": "checkout:local-dream", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/SKILL.md": "e51fd3c71d693070abeba57ca67dcca82c7347cb94eb98f43e7c455bcacbaa46", - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/metadata.json": "63b71cc6a237d8c4e00db875d15594df09d6e3cf87a1cb25183a311b4d480804", - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/android-lifecycle.md": "f85e5fce39f5de5f88548fd8c21978240c810f899e03ff05ed72eb3508ee5e36", - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/native-contracts.md": "40206892f5e421f9d8edf303870f5c4aefa3e87123537178d45f438f3d04b7ce", - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/packaging-and-abi.md": "9a19bca59e19c3bfbfc1a15f5fc756c13d954ace367812aeb000551f4484ad06", - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/repo-provenance.md": "0b848082c0ce893d8ae26d779c5b663d914d1a208cb2293c4a84668cdde86293", - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/source-identity.json": "b85f028bb96a37852e9d9780b5d5052fdd9c8f3a704ad391b6381adb58291347", - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/upstream-license.md": "41003d4a74749c0220e33dd415042164b5a1093ed401f36277234f772d22d3d0", - "src/inferrsi/knowledge/data/skill_library/repositories/local-dream/scripts/screen_declarations.py": "ccd7d0d55be9a47484ec3a74498d8edef8aba55287f92287775c8d82050c8932" - }, - "skill_metadata": { - "name": "local-dream", - "description": "Audit Local Dream Android desired-versus-serving process state, remote exposure, native backend and DiT packaging/ABI selection, shared generation cancellation and prompt-cache identity without mistaking declarations or source tests for on-device diffusion qualification.", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "local-dream", - "android", - "snapdragon", - "qnn", - "hexagon", - "mnn", - "diffusion", - "lifecycle", - "cancellation", - "prompt-cache", - "abi", - "lan-exposure" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/xororz/local-dream", - "commit": "db162df4d9bb6efc6595250423d8df26241f4c27", - "license": "CC-BY-NC-4.0; repository API SPDX NOASSERTION retained separately", - "readset": "references/source-identity.json", - "paths": [ - ".gitmodules", - "LICENSE", - "README.md", - "app/build.gradle.kts", - "app/src/main/cpp/CMakeLists.txt", - "app/src/main/cpp/CMakePresets.json", - "app/src/main/cpp/build.sh", - "app/src/main/cpp/dit/CMakeLists.txt", - "app/src/main/cpp/dit/DitEngine.cpp", - "app/src/main/cpp/dit/build.sh", - "app/src/main/cpp/include/DitEngine.h", - "app/src/main/cpp/src/Config.hpp", - "app/src/main/cpp/src/Pipeline.hpp", - "app/src/main/cpp/src/PipelineAnima.hpp", - "app/src/main/cpp/src/PipelineDit.hpp", - "app/src/main/cpp/src/PipelineSd15Cpu.hpp", - "app/src/main/cpp/src/PipelineSd15Npu.hpp", - "app/src/main/cpp/src/PipelineSdxl.hpp", - "app/src/main/cpp/src/PipelineSdxlMnn.hpp", - "app/src/main/cpp/src/PromptCache.hpp", - "app/src/main/cpp/src/QnnRuntime.hpp", - "app/src/main/cpp/src/main.cpp", - "app/src/main/java/io/github/xororz/localdream/data/DitEngine.kt", - "app/src/main/java/io/github/xororz/localdream/data/DitResolution.kt", - "app/src/main/java/io/github/xororz/localdream/data/Model.kt", - "app/src/main/java/io/github/xororz/localdream/remote/RemoteApiClient.kt", - "app/src/main/java/io/github/xororz/localdream/remote/RemoteHostServer.kt", - "app/src/main/java/io/github/xororz/localdream/remote/RemoteProtocol.kt", - "app/src/main/java/io/github/xororz/localdream/service/BackendService.kt", - "app/src/main/java/io/github/xororz/localdream/service/BackgroundGenerationService.kt", - "app/src/main/java/io/github/xororz/localdream/service/RemoteHostService.kt", - "app/src/main/java/io/github/xororz/localdream/ui/screens/ModelRunScreen.kt", - "app/src/main/java/io/github/xororz/localdream/ui/screens/ModelRunSupport.kt", - "app/src/test/java/io/github/xororz/localdream/ExampleUnitTest.kt", - "app/src/test/java/io/github/xororz/localdream/ModelStorageTest.kt" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "readset": "references/source-identity.json", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill/SKILL.md", - "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/repository-evidence.md", - "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/input-output-and-structure.md", - "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/planning-and-writing.md", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env/SKILL.md", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill/SKILL.md", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill/references/usability-test-cases.md" - ] - } - ], - "applicability": { - "scope": "Selected Android app/process and native diffusion boundary contracts; common is a discovery category, not CPU/device compatibility or native qualification.", - "prerequisites": "Access to the pinned Android lifecycle/native/packaging contracts and identified installed artifacts; stronger conclusions require an authorized, frozen workload on the actual selected Android/ARM64 backend.", - "input_contract": "Source revision, installed APK/version/flavor/ABI, native executable and loaded DiT library, QNN/Hexagon/MNN builds, model/export/weights/tokenizer identities, requested and serving configuration, and lifecycle/readiness/generation/cache observations.", - "produced_artifact": "A bounded failed-boundary report with counter-signals, identities observed versus declared, the smallest missing discriminator and retained INCONCLUSIVE native scope.", - "failure_modes": "Reading UI flags or equal declared configurations as actual dispatch/readiness, treating cancellation/SSE progress as native completion, or interpreting model-directory caches as authenticated model-revision namespaces can misidentify stale state." - }, - "limitations": [ - "Source-derived candidate; required Android ARM64/MNN/QNN/Hexagon/model execution BLOCKED_REQUIRED_BACKEND / not_run; qualification INCONCLUSIVE.", - "Portable AREX-informed source analysis and InferRSI candidate integration, not DisCo CLI execution, managed import or full AREX verification.", - "Original upstream Gradle/JVM/native tests, APK build/sign/install, loader/device/model/media-quality/memory-fit/thermal/service/performance and agent-efficacy checks not_run.", - "Offline helper compares supplied exact configuration and ABI/SDK labels only; it does not authenticate process state, readiness, artifact bytes, cache correctness or runtime behavior.", - "LAN control is unauthenticated in the inspected implementation despite stale comments; audit guidance does not authorize enabling host mode or exposing ports.", - "NOASSERTION is machine metadata, not license absence or commercial permission. Pinned LICENSE is CC BY-NC 4.0; retain restriction and attribution; SDK/model/dependency rights separate.", - "Full UI/navigation/import/download/storage/history/scheduler/kernel/security review and external dependency bodies excluded." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Compare requested and serving model/backend/dimensions/bind address through stop/reopen and health observations; never enable host mode as a diagnostic default.", - "Trace the actual MNN/QNN/Anima/DiT path and separate native library/ABI identity from UI runOnCpu flags and declared SDK labels.", - "Keep progress, cancellation, native completion, release and saved final output separate; check prompt-cache mode/version/dimensions and textual-inversion caller guards before proposing cache changes." - ] - }, - "applicability": { - "scope": "Selected Android app/process and native diffusion boundary contracts; common is a discovery category, not CPU/device compatibility or native qualification.", - "prerequisites": "Access to the pinned Android lifecycle/native/packaging contracts and identified installed artifacts; stronger conclusions require an authorized, frozen workload on the actual selected Android/ARM64 backend.", - "input_contract": "Source revision, installed APK/version/flavor/ABI, native executable and loaded DiT library, QNN/Hexagon/MNN builds, model/export/weights/tokenizer identities, requested and serving configuration, and lifecycle/readiness/generation/cache observations.", - "produced_artifact": "A bounded failed-boundary report with counter-signals, identities observed versus declared, the smallest missing discriminator and retained INCONCLUSIVE native scope.", - "failure_modes": "Reading UI flags or equal declared configurations as actual dispatch/readiness, treating cancellation/SSE progress as native completion, or interpreting model-directory caches as authenticated model-revision namespaces can misidentify stale state." - }, - "recommendations": [ - "Compare requested and serving model/backend/dimensions/bind address through stop/reopen and health observations; never enable host mode as a diagnostic default.", - "Trace the actual MNN/QNN/Anima/DiT path and separate native library/ABI identity from UI runOnCpu flags and declared SDK labels.", - "Keep progress, cancellation, native completion, release and saved final output separate; check prompt-cache mode/version/dimensions and textual-inversion caller guards before proposing cache changes." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; required Android ARM64/MNN/QNN/Hexagon/model execution BLOCKED_REQUIRED_BACKEND / not_run; qualification INCONCLUSIVE.", - "Portable AREX-informed source analysis and InferRSI candidate integration, not DisCo CLI execution, managed import or full AREX verification.", - "Original upstream Gradle/JVM/native tests, APK build/sign/install, loader/device/model/media-quality/memory-fit/thermal/service/performance and agent-efficacy checks not_run.", - "Offline helper compares supplied exact configuration and ABI/SDK labels only; it does not authenticate process state, readiness, artifact bytes, cache correctness or runtime behavior.", - "LAN control is unauthenticated in the inspected implementation despite stale comments; audit guidance does not authorize enabling host mode or exposing ports.", - "NOASSERTION is machine metadata, not license absence or commercial permission. Pinned LICENSE is CC BY-NC 4.0; retain restriction and attribution; SDK/model/dependency rights separate.", - "Full UI/navigation/import/download/storage/history/scheduler/kernel/security review and external dependency bodies excluded." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mistral-rs.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mistral-rs.json deleted file mode 100644 index f443caf6c..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mistral-rs.json +++ /dev/null @@ -1,192 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.mistral_rs", - "title": "mistral-rs", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Investigate mistral.rs Metal paged-attention decode layouts, prefill coordinates and workspace, actual-model prewarm and native test evidence without treating static metadata or prepared pipelines as executed GPU correctness or performance.", - "tags": [ - "mistral-rs", - "metal", - "candle", - "paged-attention", - "cache-layout", - "prefill-metadata", - "workspace", - "prewarm", - "native-evidence" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/mistral-rs/SKILL.md" - }, - "checkout_ref": "checkout:mistral-rs", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/LICENSE": "b40bb09c7b3b6f697a8920cf9d2cb3ff8cfcf77a0fcb6d264a92842b43c4dcae", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/SKILL.md": "8917b30fb86437dea7508275ccce1992194d6e0c4858d537a410bf1f900953d0", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/metadata.json": "3ca0ca5f3a3c9e4c3e2c7d4587940b83123cf05f4e3213b305ccdd1ae89cd679", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/apache-component-license.md": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/paged-component-notice.md": "a1718d8c5a25f15e61bea98a158880e96faa5b7174e82c4e1470fbacb23a3559", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/repo-provenance.md": "fab87dacbe0a7298e42137cab7409f443ece0dfefba60961040031e36d96adc8", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/repo-routing-metadata.json": "0debda77a8f6b1b1c06176e3c2447767c1953b9fe7e4c84ef01b3ac90735df70", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/scope-and-notices.md": "938203995f5e58cd6cefa278225b2d5a3ea7dc7f07cae86201f9fc71553ac020", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/source-identity.json": "6a8c0b0834228b3a298e7e548c9d8cb774c00f661135f51c12ee613efe1decf3", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/SKILL.md": "73620deb4efad4681da10a07bb982e180423bd69fec3572d16ab556c530b89ec", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/helper-contract.md": "802d8af65343eeb29b0950d3c305d0f24cb7a4298628ade5f7cdcf4358c9e228", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/layout-and-features.md": "6804a771ed0dd2d8d60613d52ef87d4297af2a6445b4fe61d08adeab4be331aa", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/troubleshooting.md": "a6de920f174d6918a72093921d344c700108b0c01e1f0d09f2b9051800b1fa7d", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/workspace-and-evidence.md": "7ac5028be6fd48f9c6cf067bf832efc8edd681195f2fece67ea119211bdac106", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/scripts/audit_decode_metadata.py": "de28f586f27278fee25f6c96f1e00ee63528cdf746f57c83bdf367c31c0f6e6b", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/SKILL.md": "21015790789933e106f6ea929fd1778d862315520b55425213c679e9b7a44e95", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/helper-contract.md": "607449105bf3bcbcfe785dc9448a7a587c7daeb528ca63e0cd959b074a9bc409", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/prefill-and-metadata.md": "e8a738a668a23068b8ff1907d521a3a26f7090e7a791f3a715efdb6001338dc0", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/troubleshooting.md": "f584a02528e2249f5d12f223d8933e9d40212e5aba8dbe545261164b00ed6f8b", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/scripts/screen_prefill_metadata.py": "769366c19d621ec1b42c722ed224fa154a63c3410a0cffc9072a13abd9c71bbd", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/SKILL.md": "745d88dbf6c764f64f8b31763e57bef720788830ddf17b518456323000325fbb", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/prewarm-and-evidence.md": "6b72ae4b4599018ce2da5c54d0c28fadcafb7ce75b81ce4c18c06a440aab6056", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/troubleshooting.md": "d933bea6e9e493faa8cd0f4103ea33197ace7f7f901359f8cf0d6bab2edcdb4d", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/warmup-inventory.md": "1e7af7d63c6853ede4232b5a12cf68aba6597c4a305015366f1c2c917e3caf01", - "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/scripts/warmup_shape_inventory.py": "6aa920ca1cb534dec41f5dd2d833c3e760b234ab2c6439b6c4250b8a30b74055" - }, - "skill_metadata": { - "name": "mistral-rs", - "description": "Investigate mistral.rs Metal paged-attention decode layouts, prefill coordinates and workspace, actual-model prewarm and native test evidence without treating static metadata or prepared pipelines as executed GPU correctness or performance.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "mistral-rs", - "metal", - "candle", - "paged-attention", - "cache-layout", - "prefill-metadata", - "workspace", - "prewarm", - "native-evidence" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/EricLBuehler/mistral.rs", - "commit": "9459891e7697238225885518119682ac8209a612", - "version": "0.9.5", - "license": "MIT root; identified paged attention portions Apache-2.0 with NOTICE", - "readset": "references/source-identity.json", - "paths": [ - ".github/workflows/ci.yml", - "AGENTS.md", - "Cargo.lock", - "Cargo.toml", - "LICENSE", - "README.md", - "docs/src/content/docs/guides/perf/paged-attention.mdx", - "examples/python/paged_attention.py", - "mistralrs-core/src/paged_attention/cache_engine.rs", - "mistralrs-core/src/paged_attention/metal_hnd_tests.rs", - "mistralrs-core/src/paged_attention/mod.rs", - "mistralrs-core/src/pipeline/metal_warmup.rs", - "mistralrs-paged-attn/Cargo.toml", - "mistralrs-paged-attn/LICENSE-APACHE", - "mistralrs-paged-attn/NOTICE", - "mistralrs-paged-attn/build.rs", - "mistralrs-paged-attn/src/metal/backend/mod.rs", - "mistralrs-paged-attn/src/metal/backend/paged_attention.rs", - "mistralrs-paged-attn/src/metal/backend/paged_prefill.rs", - "mistralrs-paged-attn/src/metal/kernels/mod.rs", - "mistralrs-paged-attn/src/metal/kernels/paged_decode_hnd.metal", - "mistralrs-paged-attn/src/metal/kernels/paged_prefill.metal", - "mistralrs-paged-attn/src/metal/mod.rs", - "mistralrs-paged-attn/tests/metal_prefill.rs" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Portable compatible-agent native Rust/source adaptation; no DisCo CLI/model execution or managed import." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Stable-ID bounded decode/prefill/qualification author lanes, whole-graph reconciliation, hard cases and offline helper/static/scaffold/wheel checks; compiler and actual Apple evidence not_run." - }, - "applicability": { - "scope": "Selected native Metal paged decode/prefill layout/metadata/workspace and preparation/evidence investigation. Common catalog categorization is not CPU inference or hardware certification.", - "prerequisites": "Access to the pinned Rust/Candle/MSL decode, prefill and qualification contracts; native comparisons need the matched Rust/Apple/Candle/shader stack and separately authorized frozen references.", - "input_contract": "mistral.rs/Candle/native artifact revisions, Rust toolchain, Apple SDK/OS/device, actual model/cache dtype and representation, shape/strides/offsets, block table, context and execution limits.", - "produced_artifact": "A source/build-bound route and invariant report with actual versus declared facts, safe support result, counter-signal and smallest next native check; unavailable Metal/kernel/model evidence remains blocked.", - "failure_modes": "Equating element counts with physical alias/layout identity, transferring decode geometry to prefill, or treating prewarm/pipeline preparation as actual dispatch can misclassify eligibility and native correctness." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: actual Rust/Candle/Apple Metal compilation, native tensor/cache/dispatch/parity/model/quality/service/performance evidence not_run.", - "Cargo/rustc and compatible Apple SDK/device unavailable; no Rust package/toolchain or native dependency installation, live Rust API inspection or source test execution claimed.", - "Offline helpers screen caller declarations, source byte formulas or boundary enumeration only; no tensor authentication, GPU simulation, native eligibility, scratch-capacity or physical-fit proof.", - "Generic model/loading/quantization/server/agentic/distributed/CUDA paths and unmined specialized kernels remain explicit long-tail gaps; broad skill efficacy unmeasured.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback, not legal absence; root MIT and identified component Apache-2.0/NOTICE retained independently." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Select decode, prefill or qualification by the failed contract, then follow cache/metadata bytes from producer through view to native consumer.", - "Keep static predicates, pipeline preparation, kernel dispatch, numerical correctness and model/service outcomes as separate evidence steps; retain skipped or gated tests.", - "Use the matched Candle revision and native shader/build inputs; broad workspace tests can require model fixtures, tokens and downloads." - ] - }, - "applicability": { - "scope": "Selected native Metal paged decode/prefill layout/metadata/workspace and preparation/evidence investigation. Common catalog categorization is not CPU inference or hardware certification.", - "prerequisites": "Access to the pinned Rust/Candle/MSL decode, prefill and qualification contracts; native comparisons need the matched Rust/Apple/Candle/shader stack and separately authorized frozen references.", - "input_contract": "mistral.rs/Candle/native artifact revisions, Rust toolchain, Apple SDK/OS/device, actual model/cache dtype and representation, shape/strides/offsets, block table, context and execution limits.", - "produced_artifact": "A source/build-bound route and invariant report with actual versus declared facts, safe support result, counter-signal and smallest next native check; unavailable Metal/kernel/model evidence remains blocked.", - "failure_modes": "Equating element counts with physical alias/layout identity, transferring decode geometry to prefill, or treating prewarm/pipeline preparation as actual dispatch can misclassify eligibility and native correctness." - }, - "recommendations": [ - "Select decode, prefill or qualification by the failed contract, then follow cache/metadata bytes from producer through view to native consumer.", - "Keep static predicates, pipeline preparation, kernel dispatch, numerical correctness and model/service outcomes as separate evidence steps; retain skipped or gated tests.", - "Use the matched Candle revision and native shader/build inputs; broad workspace tests can require model fixtures, tokens and downloads." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "BLOCKED_REQUIRED_BACKEND: actual Rust/Candle/Apple Metal compilation, native tensor/cache/dispatch/parity/model/quality/service/performance evidence not_run.", - "Cargo/rustc and compatible Apple SDK/device unavailable; no Rust package/toolchain or native dependency installation, live Rust API inspection or source test execution claimed.", - "Offline helpers screen caller declarations, source byte formulas or boundary enumeration only; no tensor authentication, GPU simulation, native eligibility, scratch-capacity or physical-fit proof.", - "Generic model/loading/quantization/server/agentic/distributed/CUDA paths and unmined specialized kernels remain explicit long-tail gaps; broad skill efficacy unmeasured.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback, not legal absence; root MIT and identified component Apache-2.0/NOTICE retained independently." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mlc-llm.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mlc-llm.json deleted file mode 100644 index e7b320479..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mlc-llm.json +++ /dev/null @@ -1,202 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.mlc_llm", - "title": "mlc-llm", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Investigate MLC LLM compiled Metal artifact/schema, Apple target and pinned TVM module/storage/stream contracts, and compile/runtime KV limits without confusing canonical metadata, codegen or automatic estimates with runnable native model qualification.", - "tags": [ - "mlc-llm", - "tvm", - "metal", - "artifact-contract", - "target", - "sdk", - "module", - "stream", - "kv-cache", - "compiled-limits" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/mlc-llm/SKILL.md" - }, - "checkout_ref": "checkout:mlc-llm", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/SKILL.md": "e0fd1accb4216191c56e7e4a8074633f36d9063727c2546e0d2eed14220d8cf4", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/metadata.json": "f20ca326ca0966e8d56b5fa698f971001a4af77370a145631e98476601610fa9", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/mlc-notice.md": "3a7dfb37bb96ce6f5a1cc0d6d4c8da3b8a35e6ec4692647ba0a37f6155c7e694", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/repo-provenance.md": "423f21e1de6d7ab2e11c0bdf540357efde025ae4535bdf76ed9770b1a59ba35a", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/repo-routing-metadata.json": "5309c27e7b60c4fe7aaef28bf6911e7e3c936b58b1a907aa8486bf0e40b4c0d0", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/scope-and-notices.md": "8c573c234f44a0808a731774921fb5228fc0844cd238496a2c4d61d0120bdc79", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/source-identity.json": "75e91b0f7fdac12466e62eedfb49baa6f1b3016111d7479aa2b04e5c799454b2", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/tvm-license.md": "e62264b483aca7a9f8c1ac69388b476971b43975ced90ac11dbd9e9c530edb46", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/tvm-notice.md": "967cd2fb97712c43271e2b1c4f636c4d8823898869f0f76f65bf8a3232584f0f", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/SKILL.md": "933fdd05ca853f47e3ace024cf1e08df7511834c31676851c28803b6dcb1c681", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/contract-mechanisms.md": "82187f4a1478352c68a5690101e9798973ef63f9b6d717e06188b03a1832bb8c", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/helper-contract.md": "48492a36435ff1340a4db4e0959f052cd72683ced26bb8ab35baa84bd79ccd7b", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/troubleshooting.md": "41b002a31b031a78ca372504de4c9cccc9353d6d80d989b8d62e56260c291e2e", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/scripts/audit_artifact_declarations.py": "874a16cb16d75a14cb51b4fbe442f4fac46feb21ac242063f5f845a0fcbabaa4", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/SKILL.md": "25dbd48f4fe84b7d759005d04a2aa9db33da987cafac7975a148e62cc9efae34", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/helper-contract.md": "5428efd6dca0654a58e9e226805e15c9a9f1b3f1ef60db234139985d107acbfa", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/modules-and-streams.md": "56faaf149f95ee821c242ccca8115a6f3b166d34392499798ae8532fcd10372d", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/native-evidence.md": "133322e879fd9010a7513afd278d937838aa2ec97d593ed11d324cc59793374e", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/targets-and-export.md": "547eb8e56af4f7b58415eb58c338ea1a33ff8edf8b58fa523700947dc7e04821", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/troubleshooting.md": "e392b8329ed183bba5fd79fbd260c3fbfb389016ee71880a957741e71a0d59c0", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/scripts/audit_apple_target_declarations.py": "d5bf16f7def4cfe28b143a700ca45a9636ddaeac12ead04cbaecdcc8b4b71aa5", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/SKILL.md": "6b4914cb60831c07d4c748162251c49fc16b229e0d62b4c5a5663719c45b16de", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/cache-lowering-and-evidence.md": "a118f5bc584ea885e8c573978eaf1e32d448910860b8135196c5c1acd42aee05", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/helper-contract.md": "da33306472cbda5a33605b80fe52d6cb1fa886b3b7eea13adbdf0ce8a6024c66", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/limits-and-workspace.md": "77105db95de2b8ea27ed6953b5441c8af061fb745ddb13d3904f8773fc21e879", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/troubleshooting.md": "b89255fc04507185bc8bcf74de779730b009b12d13bb38ad0193205dfed8e921", - "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/scripts/audit_kv_limit_declarations.py": "d82a6dda5f5931eb369003a5e1f419bfbdd82304766a18be8ea1e1849e26b77d" - }, - "skill_metadata": { - "name": "mlc-llm", - "description": "Investigate MLC LLM compiled Metal artifact/schema, Apple target and pinned TVM module/storage/stream contracts, and compile/runtime KV limits without confusing canonical metadata, codegen or automatic estimates with runnable native model qualification.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "mlc-llm", - "tvm", - "metal", - "artifact-contract", - "target", - "sdk", - "module", - "stream", - "kv-cache", - "compiled-limits" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/mlc-ai/mlc-llm", - "commit": "8978ea9aec626d64096aaa92fbc89bda2e3ea802", - "license": "Apache-2.0 with retained NOTICE and source-specific component notices", - "readset": "references/source-identity.json", - "paths": [ - ".github/workflows/lint.yml", - ".gitmodules", - "CONTRIBUTORS.md", - "LICENSE", - "NOTICE", - "README.md", - "cpp/serve/config.cc", - "cpp/serve/config.h", - "docs/community/guideline.rst", - "docs/compilation/model_artifact_manifest.rst", - "pyproject.toml", - "python/mlc_llm/__init__.py", - "python/mlc_llm/base.py", - "python/mlc_llm/compiler_pass/dispatch_kv_cache_creation.py", - "python/mlc_llm/interface/compile.py", - "python/mlc_llm/interface/convert_weight.py", - "python/mlc_llm/interface/gen_config.py", - "python/mlc_llm/model/gemma4/gemma4_model.py", - "python/mlc_llm/model/model.py", - "python/mlc_llm/protocol/__init__.py", - "python/mlc_llm/protocol/artifact_manifest.py", - "python/mlc_llm/serve/__init__.py", - "python/mlc_llm/serve/engine.py", - "python/mlc_llm/support/auto_target.py", - "tests/python/conftest.py", - "tests/python/integration/test_model_compile.py", - "tests/python/model/test_kv_cache.py", - "tests/python/protocol/test_artifact_manifest.py", - "tests/python/support/test_auto_target.py" - ] - }, - { - "type": "git", - "url": "https://github.com/mlc-ai/relax", - "commit": "201fa0001c81431ccce05419b442c47779ea917c", - "license": "Apache-2.0 with retained NOTICE and source-specific component notices", - "readset": "references/source-identity.json", - "paths": [ - "AGENTS.md", - "LICENSE", - "NOTICE", - "src/backend/metal/codegen/codegen_metal.cc", - "src/backend/metal/codegen/metal_fallback_module.cc", - "src/backend/metal/codegen/metal_fallback_module.h", - "src/backend/metal/runtime/metal_common.h", - "src/backend/metal/runtime/metal_device_api.mm", - "src/backend/metal/runtime/metal_module.mm", - "tests/python/codegen/test_target_codegen_metal.py" - ] - } - ], - "applicability": { - "scope": "Selected artifact/Apple target/pinned TVM Metal consumer and KV configuration source contracts. Common catalog categorization is not CPU inference or Apple qualification.", - "prerequisites": "Access to the pinned MLC and exact TVM fork/gitlink contracts; native checks require separately prepared matching MLC/TVM/FFI and the applicable Apple SDK/device stack.", - "input_contract": "MLC revision, TVM gitlink/build, model/tokenizer/quantized-parameter identity, target/host/SDK, compiled artifact/format, runtime device and the actual compiled/runtime configuration sources.", - "produced_artifact": "An identity and actual-versus-declared boundary report for artifact, Metal runtime or KV-limit ownership, with a negative signal and smallest missing native discriminator.", - "failure_modes": "Treating a schema hash as payload authentication, an opt-in WebLLM artifact contract as a native-engine ABI, serializable Metal fallback as runnable, or automatic capacity estimates as free memory can create unsupported qualification claims." - }, - "limitations": [ - "Source-derived candidate; original compiled CPU/native tests, compiler/live API/CLI/model/Metal/fit/quality/service/performance execution not_run; required native BLOCKED_REQUIRED_BACKEND.", - "Opt-in artifact manifest first targets WebLLM; native engine ignoring that ABI is not qualified by canonical metadata agreement.", - "Serializable kind=metal fallback can be non-runnable; codegen/export/pipeline/submission and actual load/dispatch/completion differ.", - "Helpers screen narrow explicit canonical metadata, literal target strings or limit clamping only; no general Pydantic/TVM emulation, actual tensor/weight/export authentication or GPU/memory simulation.", - "Full model/tokenizer/quantization/VLM/RNN/speculation/disaggregation/CUDA/scheduler/dependency implementation and broad skill efficacy remain outside selected contracts.", - "NO_LICENSE is unavailable machine SPDX metadata, not legal absence; both source owners Apache LICENSE/NOTICE bytes are independently retained." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Identify the interface's actual consumer before using a valid artifact manifest as evidence of native model/task support.", - "Separate codegen/export from runtime factory/load, pipeline creation, dispatch and stream completion; a serializable kind=metal fallback can still be non-runnable.", - "Trace runtime preference, compile-time bounds, nested workspace inputs and explicit overrides separately before interpreting a KV or working-set estimate." - ] - }, - "applicability": { - "scope": "Selected artifact/Apple target/pinned TVM Metal consumer and KV configuration source contracts. Common catalog categorization is not CPU inference or Apple qualification.", - "prerequisites": "Access to the pinned MLC and exact TVM fork/gitlink contracts; native checks require separately prepared matching MLC/TVM/FFI and the applicable Apple SDK/device stack.", - "input_contract": "MLC revision, TVM gitlink/build, model/tokenizer/quantized-parameter identity, target/host/SDK, compiled artifact/format, runtime device and the actual compiled/runtime configuration sources.", - "produced_artifact": "An identity and actual-versus-declared boundary report for artifact, Metal runtime or KV-limit ownership, with a negative signal and smallest missing native discriminator.", - "failure_modes": "Treating a schema hash as payload authentication, an opt-in WebLLM artifact contract as a native-engine ABI, serializable Metal fallback as runnable, or automatic capacity estimates as free memory can create unsupported qualification claims." - }, - "recommendations": [ - "Identify the interface's actual consumer before using a valid artifact manifest as evidence of native model/task support.", - "Separate codegen/export from runtime factory/load, pipeline creation, dispatch and stream completion; a serializable kind=metal fallback can still be non-runnable.", - "Trace runtime preference, compile-time bounds, nested workspace inputs and explicit overrides separately before interpreting a KV or working-set estimate." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; original compiled CPU/native tests, compiler/live API/CLI/model/Metal/fit/quality/service/performance execution not_run; required native BLOCKED_REQUIRED_BACKEND.", - "Opt-in artifact manifest first targets WebLLM; native engine ignoring that ABI is not qualified by canonical metadata agreement.", - "Serializable kind=metal fallback can be non-runnable; codegen/export/pipeline/submission and actual load/dispatch/completion differ.", - "Helpers screen narrow explicit canonical metadata, literal target strings or limit clamping only; no general Pydantic/TVM emulation, actual tensor/weight/export authentication or GPU/memory simulation.", - "Full model/tokenizer/quantization/VLM/RNN/speculation/disaggregation/CUDA/scheduler/dependency implementation and broad skill efficacy remain outside selected contracts.", - "NO_LICENSE is unavailable machine SPDX metadata, not legal absence; both source owners Apache LICENSE/NOTICE bytes are independently retained." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mlx-swift-lm.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mlx-swift-lm.json deleted file mode 100644 index 9bb1bdf28..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/mlx-swift-lm.json +++ /dev/null @@ -1,200 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.mlx_swift_lm", - "title": "mlx-swift-lm", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Investigate MLX Swift LM generation ownership, cancellation and settled task completion, and recurrent GatedDelta Metal versus ops contracts without confusing source predicates, final info or offline metadata with executed native correctness.", - "tags": [ - "mlx-swift-lm", - "swift", - "metal", - "generation", - "ownership", - "cancellation", - "task-join", - "recurrent-state", - "gated-delta", - "native-evidence" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/mlx-swift-lm/SKILL.md" - }, - "checkout_ref": "checkout:mlx-swift-lm", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/LICENSE": "7b1d86acac816ce8192209d7e01141c129b4d48d474c15cc5fb4af62f497c323", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/SKILL.md": "501f73e7b327b04b6b80501b42b20dab1ac15c875817e6f4c4b66fe3e69b4fc4", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/metadata.json": "6d5bd7d5f3566b75c8c0b53be95d524db039a517566826c211c978987d78b253", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/repo-provenance.md": "f1b15edf89c597db0c3fc0bff7db6b2451924ce9273c39623bc0cbdccb568d31", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/repo-routing-metadata.json": "8fc804347771cd768a7ed027f2041d96c9832fe5f8c5869e6593a3021b1bef8b", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/scope-and-notices.md": "18e56805a8f275d35af3d8e42f3aa1525871b43400921a5d2dca4897c2047214", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/source-identity.json": "518da7bb64f22ff8ee6beb0e0f0579e7cd0da2fd70fa69fd8288294b0c08ba24", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/SKILL.md": "04b97959b4e207380a06d932e9a557954a9fb9aa880c243439da6d4a981854ed", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/helper-contract.md": "1358531257e00c0f9fb2eb9fb7a40f9d749f5435c90168c8923b6646353dce6f", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/recurrence-and-state.md": "ff0ebbeacb8219d93a890418d18bdf2856e6326d67f974a7050de3a0e5a3c5a3", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/troubleshooting.md": "3bbecbf91b241e7b37b3e8c753bf3fbcbdbba0b0f4d0ddf2f65eb653007f6cff", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/verification-boundaries.md": "21b6fceb692f29b410dc967f82e31d0d082904cea827443ffc05f6f940191dfe", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/scripts/audit_gated_delta_declarations.py": "3a256c37c867866e49dfdc2893cbb3f32d061e389fe4395555eff899bc0415ff", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/SKILL.md": "7effaa20ddaf32e3b4008c95888ccd09bc2bbc3bb5ff17c710f989b36be49535", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/evidence-schema.md": "d40e44910d8fc43ee3c731735902b1c1ba506c2ae972bb53dbacfaf37e38509e", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/native-evidence.md": "3e9a630a30932989faea5f52b347318cb94c73c7d4d6c81c0465ccaee2818671", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/ownership-and-termination.md": "56fdec7ebc89b5c217c7e5c203886bef1d3c2eb5794852897bdb5941ab1db2a0", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/troubleshooting.md": "de28b8ca9cf95be2a4e5762f938acfadb6d4b53459dbf86df4b2b2ec64ec347f", - "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/scripts/audit_generation_evidence.py": "2aba6caf42ca1b3fc55299fd96419af8681c37750c21a7ba75205a43baa9c7b1" - }, - "skill_metadata": { - "name": "mlx-swift-lm", - "description": "Investigate MLX Swift LM generation ownership, cancellation and settled task completion, and recurrent GatedDelta Metal versus ops contracts without confusing source predicates, final info or offline metadata with executed native correctness.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "mlx-swift-lm", - "swift", - "metal", - "generation", - "ownership", - "cancellation", - "task-join", - "recurrent-state", - "gated-delta", - "native-evidence" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ml-explore/mlx-swift-lm", - "commit": "3d066d3a0ffdc7e70c476a3d31488ea62dfdd0ac", - "version": "main 3.x API context; Swift tools 6.2", - "license": "MIT; unselected vendored xgrammar separately Apache-2.0", - "readset": "references/source-identity.json", - "paths": [ - ".github/workflows/integration_tests.yml", - ".github/workflows/pull_request.yml", - "AGENTS.md", - "CONTRIBUTING.md", - "LICENSE", - "Libraries/MLXLLM/Models/Qwen35.swift", - "Libraries/MLXLLM/Models/Qwen3Next.swift", - "Libraries/MLXLMCommon/Documentation.docc/upgrade.md", - "Libraries/MLXLMCommon/Documentation.docc/using.md", - "Libraries/MLXLMCommon/Evaluate.swift", - "Libraries/MLXLMCommon/GatedDelta.swift", - "Libraries/MLXLMCommon/KVCache.swift", - "Libraries/MLXLMCommon/ModelContainer.swift", - "Libraries/MLXLMCommon/ModelFactory.swift", - "Libraries/MLXLMCommon/TokenLoopHandler.swift", - "Libraries/MLXLMCommon/Tokenizer.swift", - "Libraries/MLXLMCommon/UserInput.swift", - "Libraries/MLXLMCommon/Utilities/GenerationWorker.swift", - "Libraries/MLXLMCommon/Utilities/SerialAccessContainer.swift", - "Libraries/MLXVLM/Models/Qwen35.swift", - "Package.swift", - "README.md", - "Tests/MLXLMTests/CancellationTests.swift", - "Tests/MLXLMTests/GatedDeltaQKNormTests.swift", - "Tests/MLXLMTests/GatedDeltaTests.swift", - "Tests/MLXLMTests/GenerationExecutionTests.swift", - "Tests/MLXLMTests/Qwen35GDNDecodeBitwiseTests.swift", - "Tests/MLXLMTests/Qwen35MTPTests.swift", - "skills/mlx-swift-lm/SKILL.md", - "skills/mlx-swift-lm/references/concurrency.md", - "skills/mlx-swift-lm/references/generation.md", - "skills/mlx-swift-lm/references/kv-cache.md", - "skills/mlx-swift-lm/references/model-container.md", - "skills/mlx-swift-lm/references/model-porting.md" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Actual portable compatible-agent source/native-Swift adaptation; no DisCo CLI/model run or managed installation." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Stable-ID lifecycle and recurrent-kernel author lanes, integrated root/local references, hard cases and offline support/static/scaffold/package checks; native Swift/Xcode/MLX compiler/tests not_run." - }, - "applicability": { - "scope": "Selected Swift ownership/cancellation/settling and actual GatedDelta Metal/ops contracts. Common categorization is not CPU inference or Apple qualification.", - "prerequisites": "Access to the Swift lifecycle and selected GatedDelta contracts, with resolved MLX/native identities for a comparison; native checks require a prepared matching Swift/Xcode/Apple stack and separate authorization.", - "input_contract": "Exact source revision, resolved Swift/MLX dependencies, Xcode/SDK/OS/device, API family and native artifact identity; the selected question supplies lifecycle ownership/completion observations or recurrent producer shape/dtype/layout and actual consumer/fallback.", - "produced_artifact": "An identity and actual-versus-declared report for generation ownership or recurrent kernel invariants, with safe support result, counter-signal and smallest separately authorized native check.", - "failure_modes": "Conflating lexical isolation with exclusive access across await or completed task join, treating useKernel as observed dispatch, calling an ops fallback CPU execution, or assuming a manifest range is a resolved dependency can invalidate the conclusion." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: native Swift/MLX compiler, host-control tests and required Apple Metal cancellation/backgrounding/kernel/cache/model/fit/quality/service/performance evidence not_run.", - "Swift/Xcode/Apple SDK/MLX native stack unavailable; no live Swift API, package resolution, original source/native tests or CLI execution claimed.", - "Helpers audit caller-declared evidence or selected metadata/byte/source-predicate contracts only; no tasks/recurrence/GPU simulation, authenticated observation, actual dispatch or numerical validation.", - "Inspected masked kernel and ops output handling differ at source level; no masked-output native parity case executed, confirmed numeric bug or fix claimed.", - "Full ChatSession/cache reconciliation, models/loading/quantization/VLM/tools/grammar/LoRA/training/FoundationModels and external MLX manager/provider internals are bounded gaps; broad skill efficacy unmeasured.", - "NO_LICENSE is actual unauthenticated gh resolver fallback, not legal absence; root MIT retained independently." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve mutable-context/cache ownership and task settlement separately from lexical isolation and queued GPU work before cache reuse or producer shutdown.", - "Follow recurrent shape/dtype/layout to the actual Metal or ops consumer; predicate eligibility and useKernel alone do not prove dispatch.", - "Record the actual dependency resolution and use the documented Xcode macOS test route; the source states swift test does not work for this package." - ] - }, - "applicability": { - "scope": "Selected Swift ownership/cancellation/settling and actual GatedDelta Metal/ops contracts. Common categorization is not CPU inference or Apple qualification.", - "prerequisites": "Access to the Swift lifecycle and selected GatedDelta contracts, with resolved MLX/native identities for a comparison; native checks require a prepared matching Swift/Xcode/Apple stack and separate authorization.", - "input_contract": "Exact source revision, resolved Swift/MLX dependencies, Xcode/SDK/OS/device, API family and native artifact identity; the selected question supplies lifecycle ownership/completion observations or recurrent producer shape/dtype/layout and actual consumer/fallback.", - "produced_artifact": "An identity and actual-versus-declared report for generation ownership or recurrent kernel invariants, with safe support result, counter-signal and smallest separately authorized native check.", - "failure_modes": "Conflating lexical isolation with exclusive access across await or completed task join, treating useKernel as observed dispatch, calling an ops fallback CPU execution, or assuming a manifest range is a resolved dependency can invalidate the conclusion." - }, - "recommendations": [ - "Resolve mutable-context/cache ownership and task settlement separately from lexical isolation and queued GPU work before cache reuse or producer shutdown.", - "Follow recurrent shape/dtype/layout to the actual Metal or ops consumer; predicate eligibility and useKernel alone do not prove dispatch.", - "Record the actual dependency resolution and use the documented Xcode macOS test route; the source states swift test does not work for this package." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "BLOCKED_REQUIRED_BACKEND: native Swift/MLX compiler, host-control tests and required Apple Metal cancellation/backgrounding/kernel/cache/model/fit/quality/service/performance evidence not_run.", - "Swift/Xcode/Apple SDK/MLX native stack unavailable; no live Swift API, package resolution, original source/native tests or CLI execution claimed.", - "Helpers audit caller-declared evidence or selected metadata/byte/source-predicate contracts only; no tasks/recurrence/GPU simulation, authenticated observation, actual dispatch or numerical validation.", - "Inspected masked kernel and ops output handling differ at source level; no masked-output native parity case executed, confirmed numeric bug or fix claimed.", - "Full ChatSession/cache reconciliation, models/loading/quantization/VLM/tools/grammar/LoRA/training/FoundationModels and external MLX manager/provider internals are bounded gaps; broad skill efficacy unmeasured.", - "NO_LICENSE is actual unauthenticated gh resolver fallback, not legal absence; root MIT retained independently." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/npu-skill-factory.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/npu-skill-factory.json deleted file mode 100644 index 04c2cf1c4..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/npu-skill-factory.json +++ /dev/null @@ -1,181 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.npu_skill_factory", - "title": "npu-skill-factory", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Design and operation of the InferRSI NPU skill factory: artifact lifecycle, skill evaluation contract, corpus selection, hide-and-re-distill procedure, and skill evolution cycle. Use when deciding where new skill artifacts belong, how to distill a candidate skill, or how to run the No-Skill vs Human-Skill vs Distilled-Skill comparison.\n", - "tags": [ - "skill-factory", - "distillation", - "evaluation", - "npu", - "ascend" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/workflows/npu-skill-factory/SKILL.md" - }, - "checkout_ref": "checkout:npu-skill-factory", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/SKILL.md": "94ad2b670863f45f26b85ce99a96a64c3d2050a74363931c59847bc2fceefb43", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/metadata.json": "c862565b814e98e0ebc4923e32eef2aa67a8df2357c255903d744f40aaea32c8", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/COMPARISON.md": "38587678a96581f5c2dc697d982127343565504bffa191352811dffcc165aa23", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/INTEGRATION_NOTES.md": "fbc4a96b6f6c477706eed83eaa4fcc29a14f881637629d6c004601ed049f2523", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/QA.md": "3f3785ec39a6938a5d1f2e874e99153fe92c0e666dd3e78a499c8b47d891346a", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/README.md": "221785e91c45588f82b3caab6d52f1ea1d102b16139c20a0802921e2789e6d28", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/UNREAD_LEADS.md": "6c1d87ebe491d27666703149eb05ad582c7e5aff77f0ac5c11a7b89c946f96ee", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md": "f9fc21753ea41eef77ca020d78d031e945e5dd835b5d09c5ff0e53749ceb2ca4", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/knowledge/evidence-routing.md": "406d5d74c3fef4c1a23d495492e21a084c3c46495a1a84b385703ed104358874", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md": "9c2d1d738e896d05c00f53e6adf14c9df2501ecc3f0020e71b5a1c2192547086", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/protocol-details.md": "41298c6bde3a28f0712fa06a18bbc4ac59aea6d17cc9eb1318744f4121e3bab6", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/readset.md": "6f953d20d8958c68339c9ceeae348604d0fea3e9c6ccb9b64a63d3b210426a22", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/repo-to-skill.md": "5ebdac2a2bbbe6614f9d4096bc9895bd685acdf76414e2c3c1fb7b50d57c738d", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/skillopt.md": "148075daa67e19f4b4351710498438d17c279a9bce56b2d01cab26ff3be33e99", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/source-manifest.json": "716cd49ed01824ee9f6b6fc55ebf2b24fdd5d5a3e677f6e28e34a472616bbe25", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/trace2skill.md": "230d13b77ba819b5e3edee147523e8cad6bfce95afccc140fb4946d6c8747a2e", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/updates/bounded-updates.md": "d02c5bd0109846d1e1987de074810321cb4af277cd3f66ab2ac268aadd5ee142", - "src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/skill-factory-design-20261014.md": "fb8ce1ad83b7a6ddc657e89ee2eebb710873b213cfd85bdd6d823c8691b3cb9f" - }, - "skill_metadata": { - "name": "npu-skill-factory", - "description": "Design and operation of the InferRSI NPU skill factory: artifact lifecycle, skill evaluation contract, corpus selection, hide-and-re-distill procedure, and skill evolution cycle. Use when deciding where new skill artifacts belong, how to distill a candidate skill, or how to run the No-Skill vs Human-Skill vs Distilled-Skill comparison.\n", - "schema_version": 1, - "status": "candidate", - "sources": [ - { - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1af", - "paths": [ - "docs/disco-meta-skills.md", - "docs/disco-workflows.md", - "src/disco/skills/create-paper-module-skill/SKILL.md", - "src/disco/skills/create-paper-module-skill/scripts/create_skill_skeleton.py", - "src/disco/skills/create-paper-module-skill/scripts/validate_skill_tree.py", - "src/disco/modes/skill-policy.ts", - "src/core/repo-skills-library-manager.ts" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/SkillOpt/SkillOpt", - "commit": "343db229", - "paths": [ - "skillopt/evaluation/gate.py", - "skillopt/optimizer/skill.py", - "skillopt/optimizer/clip.py", - "skillopt_sleep/cycle.py", - "skillopt_sleep/harvest_sources.py" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/WillowMindIE/MindIE-SD", - "commit": "8481d39b", - "paths": [ - ".agents/skills/profiling-analyze/SKILL.md", - ".agents/skills/dit-perf-opt/SKILL.md", - ".agents/skills/perf-gate/SKILL.md", - ".agents/skills/dit-parallel-opt/SKILL.md" - ], - "type": "git", - "revision_precision": "prefix" - } - ], - "verification": { - "locally_verified": false, - "evidence_level": "source_grounded", - "note": "Historical repository extraction and sibling schema at 82218b4, not a current local factory implementation. Reviewed paper additions and exact versions live in references/papers-20261015/references/readset.md and docs/reference-matrix.md; no efficacy or paper-to-code equivalence established." - }, - "applicability": { - "scope": "InferRSI skill lifecycle management", - "prerequisites": "Current candidate inventory and source/evaluation evidence; historical sibling execution interfaces are not required for read-only artifact admission", - "input_contract": "A candidate inventory plus the source and evaluation evidence for each candidate skill; read-only artifact admission needs no historical sibling execution interface", - "produced_artifact": "An admitted or refused skill candidate with its source, evidence and limitation record, and its stable id and owning path", - "failure_modes": [ - "A candidate admitted on an inventory line without source or evaluation evidence", - "A stable id or path assigned twice across owners", - "Indexing or admission reported as validation or promotion", - "Paper-to-code equivalence asserted without a pinned readset" - ], - "refusal_point": "Refuses admission when a candidate has neither source evidence nor a declared evaluation state, and records the refusal instead of leaving the field blank" - }, - "recommendations": [ - "Read the methodology counterexamples in references/papers-20261015/cases/ before admitting a candidate derived from a paper.", - "Keep the source snapshot revision as research evidence; it is not a campaign default.", - "Record an unvalidated candidate as candidate with not_run rather than omitting the state." - ], - "tags": [ - "skill-factory", - "distillation", - "evaluation", - "npu", - "ascend" - ], - "profile": "local-skill-v1", - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation." - ], - "provenance_readsets": [ - "references/papers-20261015/cases/methodology-counterexamples.md", - "references/papers-20261015/COMPARISON.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/evidence-routing.md", - "references/papers-20261015/QA.md", - "references/papers-20261015/README.md", - "references/papers-20261015/references/learning-to-commit.md", - "references/papers-20261015/references/protocol-details.md", - "references/papers-20261015/references/readset.md", - "references/papers-20261015/references/repo-to-skill.md", - "references/papers-20261015/references/skillopt.md", - "references/papers-20261015/references/trace2skill.md", - "references/papers-20261015/UNREAD_LEADS.md", - "references/papers-20261015/updates/bounded-updates.md" - ], - "consumption_mode": "packaged", - "searchspace_not_applicable": "SKILL.md §§ Choose the evidence product, Artifact lifecycle, and What this skill does not do define skill-distillation/evaluation methodology; NPU task tiers are evaluation prerequisites, not a hardware-resource mechanism." - }, - "applicability": { - "scope": "InferRSI skill lifecycle management", - "prerequisites": "Current candidate inventory and source/evaluation evidence; historical sibling execution interfaces are not required for read-only artifact admission", - "input_contract": "A candidate inventory plus the source and evaluation evidence for each candidate skill; read-only artifact admission needs no historical sibling execution interface", - "produced_artifact": "An admitted or refused skill candidate with its source, evidence and limitation record, and its stable id and owning path", - "failure_modes": [ - "A candidate admitted on an inventory line without source or evaluation evidence", - "A stable id or path assigned twice across owners", - "Indexing or admission reported as validation or promotion", - "Paper-to-code equivalence asserted without a pinned readset" - ], - "refusal_point": "Refuses admission when a candidate has neither source evidence nor a declared evaluation state, and records the refusal instead of leaving the field blank" - }, - "recommendations": [ - "Read the methodology counterexamples in references/papers-20261015/cases/ before admitting a candidate derived from a paper.", - "Keep the source snapshot revision as research evidence; it is not a campaign default.", - "Record an unvalidated candidate as candidate with not_run rather than omitting the state." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json deleted file mode 100644 index 91e68b62b..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json +++ /dev/null @@ -1,170 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.optimization_search_policy", - "title": "optimization-search-policy", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop.\n", - "tags": [ - "optimization", - "search", - "policy", - "experiment-design" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/optimization-search-policy/SKILL.md" - }, - "checkout_ref": "checkout:optimization-search-policy", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md": "493001e1266b91434d873d688c978486bbac4a4b86fb657492746877b652e4fb", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json": "681871983ec4033ef9861ad270bc0d312f9134a76933b8f7ce740659ef1e4dd5", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md": "2c2a8147001d24d1abc8672f15cf0ca759ae461b734413c8dcb7497634bb6821", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/metadata.json": "43e6378cdc801a71fe510e6adc17079016220c94bd6debfa8366b3cc7117d493", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/README.md": "56e16efce447dfd4990a024ff1d6f7cb428f8e3e1f7b32a7e4d05f57139b836a", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/source-record.json": "21cc9562f1c19d6b57957c01b53e0adf8700d05ffaaa7764afe07c4da1284de1", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/INTEGRATION_NOTES.md": "157e74ba180426b279b548a729769f152bf18f9a78ffca4ebfcaf26b7d0c5292", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/MECHANISMS.md": "18bb2f5d7e52773c516420a142edfb531d0b9c822a2934d60f3234ac1cb9d40e", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/NEXT_GATE_LEADS.md": "16a264c7159f9c2c700ce8ba46e63f3e636b277c987da4970f6d2fa085fcb053", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/knowledge/adaptation-interfaces.md": "5e761957cb5ecd93926e6ffc13deee7c830c6e8f0ee3c3846e9dd3b50dc74446", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md": "54d83c3713045d59a48821bcf93598374ba4a630e3891ad39498f919317c4fb8", - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/updates/optimization-search-policy.md": "d3db44fa9a6831b86a02c638b323ed943e69fe256866213f10d68c72d762879e" - }, - "skill_metadata": { - "name": "optimization-search-policy", - "description": "Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop.\n", - "tags": [ - "optimization", - "search", - "policy", - "experiment-design" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Selecting, ordering and stopping optimization experiments: candidate generation, exploration versus exploitation, mechanism diversity, negative experiment memory, cost-aware escalation, budget allocation and stopping criteria, drawn from vllm-evolve, the TRT-LLM casebook and SkillOpt", - "prerequisites": "The useful work, quality contract, allowed approximations, timing boundary and resource budget are frozen before search, and a complete restartable incumbent is preserved; reading the procedures authorizes no execution, framework change, new resource spending or publication", - "input_contract": "Profiling data, existing config, prior experiment and negative records and the casebook for the current bottleneck class, model and hardware; each target stage's fraction of E2E time; the target, locally measured uncertainty and the authorized budget", - "produced_artifact": "An ordered experiment plan: one candidate change per round with an explicit hypothesis and its motivating evidence, a bounded shortlist of distinct mechanisms with assumptions, uncertainty, expected impact and evaluation cost, negative records turned into search constraints, and a stop-or-switch decision; when authorized rounds run, a recorded verdict per round with best-seen and delivered outcomes kept apart", - "failure_modes": [ - "The evaluation protocol, measurement setup or baseline changes in the same round as the candidate", - "Benchmark budget is spent on a candidate that fails correctness", - "A matching keyword in a rejection record is used as a causal diagnosis or automatic veto, or one failed implementation excludes a whole solution family", - "Expensive work starts on a stage without computing its Amdahl contribution, so a large local speedup yields little E2E gain", - "One mechanism family keeps being sampled only because budget remains, without re-profiling after the limiting stage moves", - "Roofline proximity or a utilization threshold is used as a universal stopping rule", - "Screening is allowed to skip the frozen acceptance regimes or required service-level evidence" - ], - "refusal_point": "Discards a correctness-breaking, worse or inconclusive change instead of integrating it, preserves no-change when candidates cannot be separated, does not count a shortcut outside the authorized semantics as an optimization, and does not let a successful probe qualify a broken final state" - }, - "recommendations": [ - "Resolve acceptance criteria before search: retain supplied criteria, otherwise use performance-ceiling-analysis to derive targets, record provenance and freeze supported gates.", - "Before generating a candidate, consult the casebook, the negative record and the target stage's Amdahl contribution.", - "Record every worse or inconclusive result with hypothesis, metric, context, evidence level and failure type, and derive a search constraint from it.", - "When about three experiments on one hypothesis return inconclusive from measurement variance, fix the protocol or move to a different hypothesis.", - "Before reporting, restore and verify the best fully valid incumbent through its canonical launcher." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/adaptation-interfaces.md", - "references/papers-20261015/MECHANISMS.md", - "references/papers-20261015/NEXT_GATE_LEADS.md", - "references/papers-20261015/references/source-notes.md", - "references/papers-20261015/updates/optimization-search-policy.md", - "cases/sarathi-chunk-budget.json", - "references/distserve-phase-placement/README.md", - "references/distserve-phase-placement/source-record.json" - ], - "references": [ - "../inference-optimization-casebook/cases/qwen-b300-rejected-directions.md", - "../inference-optimization-casebook/cases/qwen-b300-rejected-directions.json", - "../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md", - "../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json", - "../inference-optimization-casebook/cases/drama3-rejected-directions.md", - "../inference-optimization-casebook/cases/drama3-rejected-directions.json", - "../../workflows/rsi-evidence/references/loopx-goal-governance/README.md", - "cases/sarathi-chunk-budget.md", - "cases/sarathi-chunk-budget.json", - "../../repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md", - "../assess-serving-preemption/SKILL.md", - "../calibrate-serving-simulation/SKILL.md", - "../performance-ceiling-analysis/SKILL.md", - "../../repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md", - "../calibrate-serving-simulation/references/infera-serving-search/README.md", - "../calibrate-serving-simulation/references/infera-serving-search/source-record.json" - ], - "consumption_mode": "packaged", - "sources": [ - { - "type": "paper", - "id": "usenix:osdi24:zhong-yinmin", - "version": "OSDI 24 proceedings, July 2024, pages 193-210", - "url": "https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin", - "sections": [ - "3.3", - "4.1-4.2", - "6.1-6.5", - "7", - "Appendices A-C" - ], - "readset": "references/distserve-phase-placement/source-record.json" - }, - { - "type": "unknown", - "scope": "Pre-existing sources remain in their original procedure/readsets; this amendment does not normalize that inventory" - } - ] - }, - "applicability": { - "scope": "Selecting, ordering and stopping optimization experiments: candidate generation, exploration versus exploitation, mechanism diversity, negative experiment memory, cost-aware escalation, budget allocation and stopping criteria, drawn from vllm-evolve, the TRT-LLM casebook and SkillOpt", - "prerequisites": "The useful work, quality contract, allowed approximations, timing boundary and resource budget are frozen before search, and a complete restartable incumbent is preserved; reading the procedures authorizes no execution, framework change, new resource spending or publication", - "input_contract": "Profiling data, existing config, prior experiment and negative records and the casebook for the current bottleneck class, model and hardware; each target stage's fraction of E2E time; the target, locally measured uncertainty and the authorized budget", - "produced_artifact": "An ordered experiment plan: one candidate change per round with an explicit hypothesis and its motivating evidence, a bounded shortlist of distinct mechanisms with assumptions, uncertainty, expected impact and evaluation cost, negative records turned into search constraints, and a stop-or-switch decision; when authorized rounds run, a recorded verdict per round with best-seen and delivered outcomes kept apart", - "failure_modes": [ - "The evaluation protocol, measurement setup or baseline changes in the same round as the candidate", - "Benchmark budget is spent on a candidate that fails correctness", - "A matching keyword in a rejection record is used as a causal diagnosis or automatic veto, or one failed implementation excludes a whole solution family", - "Expensive work starts on a stage without computing its Amdahl contribution, so a large local speedup yields little E2E gain", - "One mechanism family keeps being sampled only because budget remains, without re-profiling after the limiting stage moves", - "Roofline proximity or a utilization threshold is used as a universal stopping rule", - "Screening is allowed to skip the frozen acceptance regimes or required service-level evidence" - ], - "refusal_point": "Discards a correctness-breaking, worse or inconclusive change instead of integrating it, preserves no-change when candidates cannot be separated, does not count a shortcut outside the authorized semantics as an optimization, and does not let a successful probe qualify a broken final state" - }, - "recommendations": [ - "Resolve acceptance criteria before search: retain supplied criteria, otherwise use performance-ceiling-analysis to derive targets, record provenance and freeze supported gates.", - "Before generating a candidate, consult the casebook, the negative record and the target stage's Amdahl contribution.", - "Record every worse or inconclusive result with hypothesis, metric, context, evidence level and failure type, and derive a search constraint from it.", - "When about three experiments on one hypothesis return inconclusive from measurement variance, fix the protocol or move to a different hypothesis.", - "Before reporting, restore and verify the best fully valid incumbent through its canonical launcher." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/os-process-placement.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/os-process-placement.json deleted file mode 100644 index 4c80c9f7b..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/os-process-placement.json +++ /dev/null @@ -1,119 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.os_process_placement", - "title": "os-process-placement", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Place and configure inference/training processes at the OS level before measuring or optimizing: check then pin CPU affinity to the accelerator-local NUMA node (taskset/numactl), keep affinity constant across measurement rounds, set thread-count environment variables (OMP_NUM_THREADS, OMP_PROC_BIND, CPU_AFFINITY_CONF) per scenario, run a device-busy preflight (npu-smi/nvidia-smi usages, graceful-then-force PID cleanup), and structure environment checks as ERROR vs WARNING with per-failure remediation. Use before any performance measurement on shared multi-socket or multi-tenant hosts. Does not cover cgroups, cpuset, isolcpus, or scheduler priorities — no source evidence for those.\n", - "tags": [ - "os", - "numa", - "cpu-affinity", - "process-placement", - "threads", - "environment", - "preflight" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/host/os-process-placement/SKILL.md" - }, - "checkout_ref": "checkout:os-process-placement", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/SKILL.md": "2003492e4d89fde0443b180d74933e764bd1979435e109c8d26e20eb8f6e5053", - "src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/metadata.json": "d53e53f724c7b6f261cf6c9b579afd20b2279cb1ffd0af89f8ce9f475046fa76", - "src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/references/llama-numa-thread-readset.md": "748b254b3b043a5ddb0f480eb4a566854d27eedd7b111499863e9bf0a5a6951e" - }, - "skill_metadata": { - "name": "os-process-placement", - "description": "Place and configure inference/training processes at the OS level before measuring or optimizing: check then pin CPU affinity to the accelerator-local NUMA node (taskset/numactl), keep affinity constant across measurement rounds, set thread-count environment variables (OMP_NUM_THREADS, OMP_PROC_BIND, CPU_AFFINITY_CONF) per scenario, run a device-busy preflight (npu-smi/nvidia-smi usages, graceful-then-force PID cleanup), and structure environment checks as ERROR vs WARNING with per-failure remediation. Use before any performance measurement on shared multi-socket or multi-tenant hosts. Does not cover cgroups, cpuset, isolcpus, or scheduler priorities — no source evidence for those.\n", - "tags": [ - "os", - "numa", - "cpu-affinity", - "process-placement", - "threads", - "environment", - "preflight" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ggml-org/llama.cpp", - "commit": "3e390af18de1131dda27913c00511725b4274c8a", - "paths": [ - "common/arg.cpp", - "common/common.cpp", - "src/llama.cpp", - "src/llama-model-loader.cpp", - "src/llama-mmap.cpp", - "ggml/src/ggml-cpu/ggml-cpu.c", - "ggml/src/ggml-cpu/ggml-cpu.cpp", - "LICENSE" - ], - "readset": "references/llama-numa-thread-readset.md" - } - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Only the existing llama.cpp NUMA/thread slice is normalized at 3e390af18de1131dda27913c00511725b4274c8a; other original sources remain unnormalized, with their identities and limits retained in the procedure/readsets.", - "Linux NUMA source paths and separate configuration controls do not establish universally optimal thread counts, affinity combinations or measured placement gains." - ], - "provenance_readsets": [ - "references/llama-numa-thread-readset.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Prepare and record CPU/NUMA process placement, phase-specific thread settings, device contention and environment prerequisites before shared-host performance measurements.", - "prerequisites": "No outstanding hard environment errors: required devices/driver, planned device count and any container prerequisites must be present before running.", - "input_contract": "Current affinity/NUMA policy, accelerator locality, workload phase/thread controls, device occupancy, runtime environment and relevant container/resource limits.", - "produced_artifact": "A recorded placement/thread/environment configuration and preflight outcome, with errors requiring remediation and warnings retained for the measurement campaign.", - "failure_modes": "Changing affinity or thread settings between rounds confounds timing; combining independent affinity mechanisms is order-sensitive, and small driver memory usage can be mistaken for device occupation." - }, - "recommendations": [ - "Inspect affinity and NUMA policy before placement and hold them constant across comparisons; document and requalify a round when placement changes.", - "Choose one affinity-mask authority and distinguish generation, batch and frontend thread controls; source template thread counts are starting points rather than tuned constants.", - "Record device occupancy and structured ERROR/WARNING results before measuring, using the official environment setup for derived CANN paths." - ] - }, - "applicability": { - "scope": "Prepare and record CPU/NUMA process placement, phase-specific thread settings, device contention and environment prerequisites before shared-host performance measurements.", - "prerequisites": "No outstanding hard environment errors: required devices/driver, planned device count and any container prerequisites must be present before running.", - "input_contract": "Current affinity/NUMA policy, accelerator locality, workload phase/thread controls, device occupancy, runtime environment and relevant container/resource limits.", - "produced_artifact": "A recorded placement/thread/environment configuration and preflight outcome, with errors requiring remediation and warnings retained for the measurement campaign.", - "failure_modes": "Changing affinity or thread settings between rounds confounds timing; combining independent affinity mechanisms is order-sensitive, and small driver memory usage can be mistaken for device occupation." - }, - "recommendations": [ - "Inspect affinity and NUMA policy before placement and hold them constant across comparisons; document and requalify a round when placement changes.", - "Choose one affinity-mask authority and distinguish generation, batch and frontend thread controls; source template thread counts are starting points rather than tuned constants.", - "Record device occupancy and structured ERROR/WARNING results before measuring, using the official environment setup for derived CANN paths." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Only the existing llama.cpp NUMA/thread slice is normalized at 3e390af18de1131dda27913c00511725b4274c8a; other original sources remain unnormalized, with their identities and limits retained in the procedure/readsets.", - "Linux NUMA source paths and separate configuration controls do not establish universally optimal thread counts, affinity combinations or measured placement gains." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/performance-ceiling-analysis.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/performance-ceiling-analysis.json deleted file mode 100644 index 8e5e02401..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/performance-ceiling-analysis.json +++ /dev/null @@ -1,243 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.performance_ceiling_analysis", - "title": "performance-ceiling-analysis", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical, empirical component, end-to-end) and a prioritized measurement plan. Use to formulate acceptance targets when the user has not supplied them, assess a requested speedup, or identify the binding bottleneck.\n", - "tags": [ - "performance", - "roofline", - "profiling", - "bottleneck", - "inference" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/performance-ceiling-analysis/SKILL.md" - }, - "checkout_ref": "checkout:performance-ceiling-analysis", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md": "39cbe5db2bdcb11ed86d08cbe1e308fd231231d5bcdb76061f7c6b59dbf316e4", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/metadata.json": "38b3c171e24d80bb9ff71d4decc7dfc9e7b0b5dca2bbd265698fd27b6980ef79", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/acceptance-criteria.md": "e96efc42005af591a87b57dd6f1e9bffa8e351bcd023bd093ac37239a0486638", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/flexgen-offload-feasibility.md": "984cc41c4412eafa6d20e109dea8927ad321a668776bc86bbffda0a94dfa9cc7", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md": "2f1a22b29391928544884754f5177694cf0a9a71aa502ed3785bc17363ee3b9d", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-source-identity.json": "c4c000cf25e990d5fedbfaa206c9ee1e48429da50251c0553975ac2ff3efe0b5", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md": "e64b62fded4490708414a898f540b0319c54d2b205fd5d060acaab419734d100", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/INTEGRATION_NOTES.md": "083e462e0a204dce599362d46712fcfb392d053c5cda56216e22e28d4b8fa0fa", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/MECHANISMS.md": "742c249082ceca230326f4407a40130fe624553b796ae5a157a8a4c15a7e1398", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/README.md": "17b2d1c7c14124d59ac8a0d5ad4200ad2803d991e784a1a98cd97188cc364999", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md": "66459ce393858b2b2160f1e2973373bfc4b9902a12644631934b4ebe3140296a", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/STATIC_CHECKS.md": "cb0aa7d148352ebb7b1678fc42a557a099575054716a341dfa25b6a998029064", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/cases/decision-cases.md": "781a675a6533e0a18c044aaaee0d010da7013b6504654b8b8292a35176db801e", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md": "ce036a4d8a0908da204515c47e2ff9623548d09de264fa65bd58fedce52a78a0", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/references/source-notes.md": "f9a8254e45473ddb2b73e576957b6d3db83c62b99dff1922f4d1e606e569559e", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md": "5780d6eb5257612ec45deb94f85f334a6ecabf7e466d1678706041f6549bda79", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md": "6593a63fce9c474b515663a560c94b7df048dc5f6e77a95b8e54e4a64cd10a09", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md": "6ff3a7bc31f1172db92aedaf7df1f1aac688bf769227b583a4cd82bfad52da67", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/agentic-kernel.md": "7190573b85f94a4391bb0d887ccbd37a31b3f08bb0531114ad7afd72fe1567a7", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md": "3926369acd88ff3c370bee5718e74dd15f3219c3e4f6311095e30e917bffa4cf", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md": "21f47111197a0b7f8d6384ea918d139dc8318a62ca3313ce060048271f50b639", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/prior-coverage.md": "35d47ac078591221fcb666219bcd824f9400fc17aafecdc0499d8336f7178238", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/vidur.md": "d38602af5219f5feeb49e6cd6bf53d66098c77fad141dae676f1d8a6743cf25d", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/roofline-contract.md": "419ec4bcaec45ac579a133bce083d4f48779f26e664d7a86f720fa65f343b1d1", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/roofline-example.json": "597cd3205df043cff3ae9d15924c77934e8d8eb300c00c4840f6bb2921f90faf", - "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/scripts/roofline.py": "524de48a6764401695fe1444f1fa3d38f363069b0cd2fba4715a86ca4b28a5d4" - }, - "skill_metadata": { - "name": "performance-ceiling-analysis", - "description": "Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical, empirical component, end-to-end) and a prioritized measurement plan. Use to formulate acceptance targets when the user has not supplied them, assess a requested speedup, or identify the binding bottleneck.\n", - "schema_version": 1, - "status": "candidate", - "sources": [ - { - "url": "https://github.com/NVIDIA/TensorRT-LLM", - "commit": "f33e0e5fd55b", - "paths": [ - ".claude/skills/perf-analysis/SKILL.md", - ".claude/skills/perf-nsight-compute-analysis/references/roofline-analysis.md", - ".claude/skills/perf-workload-profiling/SKILL.md", - ".claude/skills/perf-optimization/SKILL.md", - ".claude/skills/perf-optimization-casebook/SKILL.md", - ".claude/skills/exec-env-check/SKILL.md", - ".claude/skills/exec-remote-slurm/SKILL.md", - ".claude/agents/perf-profiling-specialist.md" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/hahnyuan/LLM-Viewer", - "commit": "1893e4b5705d", - "paths": [ - "roofline_model.py", - "model_analyzer.py", - "hardwares/hardware_params.py", - "configs/DiT.py" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/ebugger/Empirical-Roofline-Toolkit", - "commit": "0771fa9621d0", - "paths": [ - "Empirical_Roofline_Tool-1.1.0/README.md", - "Empirical_Roofline_Tool-1.1.0/Drivers/driver1.c", - "Empirical_Roofline_Tool-1.1.0/Kernels/kernel1.c", - "Empirical_Roofline_Tool-1.1.0/Scripts/roofline.py" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "type": "paper", - "id": "usenix:fast25-qin", - "version": "FAST '25 proceedings, February 25-27, 2025, pp. 155-170", - "url": "https://www.usenix.org/system/files/fast25-qin.pdf", - "content_sha256": "543e56c541523c5e807d2231370f96749b2294bc901266397a7265b18ae2c130", - "sections": [ - "2.2, Table 1, Equations 1-2", - "3.1 and 5.3 (storage-scope limits only)" - ], - "readset": "references/mooncake-prefix-reuse-feasibility.md", - "license": "UNKNOWN" - }, - { - "type": "paper", - "id": "pmlr-v202-sheng23a", - "version": "ICML 2023 official ten-author edition, PMLR 202:31094-31116", - "url": "https://proceedings.mlr.press/v202/sheng23a/sheng23a.pdf", - "content_sha256": "7cd9673c37640011d38ef4b8595f5e1e09a85b1f803a3734f19c97ca66c75b9e", - "sections": [ - "1 and 3; 4.1-4.3, Figures 1-3, Algorithm 1, Equation 1", - "4.4 (scope boundary only); 6 and 6.1, Tables 2-3 (denominators and ablations)", - "Appendix A.3 selected cost/peak-memory formulas, PDF pages 15-17; A.2 ending only (theorem boundary)" - ], - "readset": "references/flexgen-offload-feasibility.md", - "license": "CC-BY-4.0" - }, - { - "type": "git", - "url": "https://github.com/shenh10/HuggingArch", - "commit": "facbbbc96856e558b9084a94a5070b2e2c86cf0b", - "paths": [ - "docs/validation.md", - "backend/arch/validate/_matching.py", - "backend/arch/validate/_weight_ledger.py", - "backend/arch/validate/_ablation.py", - "docs/parallelism.md", - "docs/evaluation_model.md", - "docs/measured_calibration.md", - "docs/known_limitations.md", - "kernel_bench/emit.py", - "backend/inference/op_calibration.py", - "backend/measure/scope.py", - "LICENSE" - ], - "readset": "references/huggingarch-checkpoint-anchor.md", - "identity": "references/huggingarch-source-identity.json", - "license": "MIT" - }, - { - "type": "document", - "url": "https://shenhan.cc/blog/huggingarch", - "displayed_publication_date": "2026-10-08", - "accessed": "2026-11-10", - "sha256": "31d0e0a9bbf9e32723fb6ab09179fbefb2fbba1043863f1475ecc7ea7ce24ed4", - "readset": "references/huggingarch-checkpoint-anchor.md", - "scope": "Live, unversioned companion post; sections 2.5, 2.6, 3.6 and 3.7 read from one retrieval, whose HTML bytes the sha256 identifies. Published before the pinned commit, so not attributable to it; figures absent from the repository are recorded as unverified.", - "license": "UNKNOWN" - } - ], - "verification": { - "locally_verified": false, - "evidence_level": "source_grounded", - "note": "Historical repository methods plus reviewed paper extensions: references/papers-20261015/READSET.md records exact paper versions and inspection scope. Physical/conditional bounds, observed envelopes, simulator predictions and application measurements remain distinct. The bundled pure helper computes supplied L1 counts only; no automatic workload estimation, local accelerator measurements or E2E qualification. references/huggingarch-checkpoint-anchor.md records the checkpoint gates and the two-threshold rule from HuggingArch at a full commit, with per-file digests and inspection scope in references/huggingarch-source-identity.json; per-rank weight bytes are anchored only after the second gate passes." - }, - "applicability": { - "scope": "inference serving systems (LLM, diffusion, audio)", - "hardware": "any; CUDA examples from sources, NPU adaptation required", - "prerequisites": "profiler access (nsys/ncu or NPU equivalent), or willingness to work at L1 analytical only" - }, - "tags": [ - "performance", - "roofline", - "profiling", - "bottleneck", - "inference" - ], - "profile": "local-skill-v1", - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Acceptance-target preparation is agent guidance: the CLI and pure L1 helper do not infer gates, and unsupported estimates cannot replace user criteria or qualify campaign outcomes.", - "Mooncake's prefix-reuse condition is an analytical approximation; current cache validity, placement, effective rates, overlap and SLO outcomes remain unverified.", - "FlexGen's cross-tier feasibility comparison assumes fitted rates and perfect overlap; relaxed placements may OOM, the printed bls wording remains unresolved, and current compatibility, latency and efficacy are unverified.", - "The HuggingArch checkpoint anchor covers stored weights only. KV size and growth, activation traffic, collectives and end-to-end serving performance stay unanchored by it. Per-rank weight bytes stay unanchored until the second gate passes, and that gate checks the declared layout and format, not the placement a framework uses at run time." - ], - "provenance_readsets": [ - "references/flexgen-offload-feasibility.md", - "references/huggingarch-checkpoint-anchor.md", - "references/huggingarch-source-identity.json", - "references/mooncake-prefix-reuse-feasibility.md", - "references/papers-20261015/cases/decision-cases.md", - "references/papers-20261015/govern-optimization-experiments/procedure.md", - "references/papers-20261015/govern-optimization-experiments/references/source-notes.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/bounds-predictions-and-decisions.md", - "references/papers-20261015/MECHANISMS.md", - "references/papers-20261015/README.md", - "references/papers-20261015/READSET.md", - "references/papers-20261015/redirect-cross-layer-optimization/procedure.md", - "references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md", - "references/papers-20261015/references/agentic-kernel.md", - "references/papers-20261015/references/flashvector.md", - "references/papers-20261015/references/inferencebench.md", - "references/papers-20261015/references/prior-coverage.md", - "references/papers-20261015/references/vidur.md", - "references/papers-20261015/STATIC_CHECKS.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Use supplied acceptance criteria unchanged. When absent, follow references/acceptance-criteria.md to derive targets from matching baseline, calibrated attainable performance and uncertainty; retain partial requirements and label derived fields.", - "Read the full procedure and linked resources; preserve the distinction between analytical bounds, observed attainable envelopes, predictions and measured application outcomes." - ] - }, - "applicability": { - "scope": "inference serving systems (LLM, diffusion, audio)", - "hardware": "any; CUDA examples from sources, NPU adaptation required", - "prerequisites": "profiler access (nsys/ncu or NPU equivalent), or willingness to work at L1 analytical only" - }, - "recommendations": [ - "Use supplied acceptance criteria unchanged. When absent, follow references/acceptance-criteria.md to derive targets from matching baseline, calibrated attainable performance and uncertainty; retain partial requirements and label derived fields.", - "Read the full procedure and linked resources; preserve the distinction between analytical bounds, observed attainable envelopes, predictions and measured application outcomes." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Acceptance-target preparation is agent guidance: the CLI and pure L1 helper do not infer gates, and unsupported estimates cannot replace user criteria or qualify campaign outcomes.", - "Mooncake's prefix-reuse condition is an analytical approximation; current cache validity, placement, effective rates, overlap and SLO outcomes remain unverified.", - "FlexGen's cross-tier feasibility comparison assumes fitted rates and perfect overlap; relaxed placements may OOM, the printed bls wording remains unresolved, and current compatibility, latency and efficacy are unverified.", - "The HuggingArch checkpoint anchor covers stored weights only. KV size and growth, activation traffic, collectives and end-to-end serving performance stay unanchored by it. Per-rank weight bytes stay unanchored until the second gate passes, and that gate checks the declared layout and format, not the placement a framework uses at run time." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/plan-activation-residency.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/plan-activation-residency.json deleted file mode 100644 index f2a0f611c..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/plan-activation-residency.json +++ /dev/null @@ -1,100 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.plan_activation_residency", - "title": "plan-activation-residency", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Propose training activation and gradient memory reductions by distinguishing lifetime reordering, address packing and selective rematerialization from supplied evidence.", - "tags": [], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/plan-activation-residency/SKILL.md" - }, - "checkout_ref": "checkout:plan-activation-residency", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/SKILL.md": "542d8a1e6963f5019986e1fbd79d3a247a045a51487c09313026087ac457c15e", - "src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/metadata.json": "0289c62f93d087cd6a4ba4140487925c4b3258a9db1f7004ff1dfb22fd6d4582", - "src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/references/source-notes.md": "5ff52655cfd06fcf970ed2a7d4d4fe1c31115b102af2d41497de0c74848ed2b3" - }, - "skill_metadata": { - "name": "plan-activation-residency", - "description": "Propose training activation and gradient memory reductions by distinguishing lifetime reordering, address packing and selective rematerialization from supplied evidence.", - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Proposing training activation and gradient memory reductions when pressure appears to come from live tensors or allocator-reserved memory, by distinguishing lifetime reordering, address packing and selective rematerialization; covers activation checkpointing for recomputation, not persistent recovery checkpoints", - "prerequisites": "The packaged evidence and limits are read before citing paper results; installing or running a solver or framework, modifying allocation or checkpoint policies, reproducing workloads or provisioning hardware needs separate authorization", - "input_contract": "The actual forward/backward/update DAG, shapes and precision, tensor sizes and consumer lifetimes; resident versus reserved peak bytes; operator, workspace and temporary-output peaks; batch and accumulation policy; optimizer and collective barriers; representative compute and transfer costs; whether the goal is fit, step time or time to a fixed training objective", - "produced_artifact": "A preferred candidate, simpler fallback, resource ledger, critical-path prediction, assumptions, counter-signals and unresolved-evidence list, with source claims kept apart from proposed decisions, plus a proposed matched comparison; a reviewable proposal, not an implementation", - "failure_modes": [ - "Fragmentation and live-byte excess are not distinguished at the same phase", - "OLLA address packing is read as tensor placement across CPU/SSD tiers", - "Reordering moves a release or update past a dependency such as shared weights, accumulation, gradient clipping, optimizer state or collectives", - "A static packing plan ignores alignment, aliasing, workspace, asynchronous overlap or shape variability", - "Added recomputation consumes the compute window that previously hid transfers", - "Checkmate's rounding is taken to guarantee the memory cap, or its 10% allowance is used as a portable safety margin", - "A solver timeout is read as proof of infeasibility" - ], - "refusal_point": "Returns the missing classification instead of a preferred checkpoint interval when only total OOM memory is known, and escalates when a legal dependency graph, memory margin or correctness model is unavailable" - }, - "recommendations": [ - "Where ordering is legal, try earlier last-use and gradient release before adding computation or transfer.", - "Check dependency validity and memory feasibility separately for an approximate schedule.", - "Account for planning cost over the expected number of repeated steps." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Proposing training activation and gradient memory reductions when pressure appears to come from live tensors or allocator-reserved memory, by distinguishing lifetime reordering, address packing and selective rematerialization; covers activation checkpointing for recomputation, not persistent recovery checkpoints", - "prerequisites": "The packaged evidence and limits are read before citing paper results; installing or running a solver or framework, modifying allocation or checkpoint policies, reproducing workloads or provisioning hardware needs separate authorization", - "input_contract": "The actual forward/backward/update DAG, shapes and precision, tensor sizes and consumer lifetimes; resident versus reserved peak bytes; operator, workspace and temporary-output peaks; batch and accumulation policy; optimizer and collective barriers; representative compute and transfer costs; whether the goal is fit, step time or time to a fixed training objective", - "produced_artifact": "A preferred candidate, simpler fallback, resource ledger, critical-path prediction, assumptions, counter-signals and unresolved-evidence list, with source claims kept apart from proposed decisions, plus a proposed matched comparison; a reviewable proposal, not an implementation", - "failure_modes": [ - "Fragmentation and live-byte excess are not distinguished at the same phase", - "OLLA address packing is read as tensor placement across CPU/SSD tiers", - "Reordering moves a release or update past a dependency such as shared weights, accumulation, gradient clipping, optimizer state or collectives", - "A static packing plan ignores alignment, aliasing, workspace, asynchronous overlap or shape variability", - "Added recomputation consumes the compute window that previously hid transfers", - "Checkmate's rounding is taken to guarantee the memory cap, or its 10% allowance is used as a portable safety margin", - "A solver timeout is read as proof of infeasibility" - ], - "refusal_point": "Returns the missing classification instead of a preferred checkpoint interval when only total OOM memory is known, and escalates when a legal dependency graph, memory margin or correctness model is unavailable" - }, - "recommendations": [ - "Where ordering is legal, try earlier last-use and gradient release before adding computation or transfer.", - "Check dependency validity and memory feasibility separately for an approximate schedule.", - "Account for planning cost over the expected number of repeated steps." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/project-maya.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/project-maya.json deleted file mode 100644 index 6eeeb9586..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/project-maya.json +++ /dev/null @@ -1,247 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.project_maya", - "title": "project-maya", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Investigate Project Maya model-family dispatch, representation and memory tiers, API versus native state, speculative decoding and source-qualified verification without treating safe CPU checks as CUDA/HIP or model evidence.", - "tags": [ - "project-maya", - "model-specific-inference", - "cuda", - "rocm", - "expert-tiering", - "native-state", - "speculation", - "evidence-boundaries" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/project-maya/SKILL.md" - }, - "checkout_ref": "checkout:project-maya", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/LICENSE": "574cffb7c57769a9271043ac0c2b07c5e7e7621cfb81081b901d8981e791d121", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/SKILL.md": "c619a3472be02147a847a24ca97c56afd8b56b6ffd22ce66fcae453155e34bab", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/metadata.json": "c48af69c430f23a8b39a2dbc09495bef54b48147c64068b39cfb3551f4e94c63", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/license-boundaries.md": "090839df90d6e96f55794c2ad821d7154dc6a6aa4211cdbb08a1e5f6f2e52a92", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/repo-provenance.md": "7bb6b6c14d1a9b77e906804505c723569f952e5d47559e636d2e6621994216bf", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/repo-routing-metadata.json": "0acbb111e92316e74d537200cd033f3d7d0f69972bf0953c8cea9dfb2ecb4a34", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/source-identity.json": "fc2287d5bde587cf040bbf515161d25e84a7bb42989cf01343582af67e4b8eeb", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/troubleshooting.md": "735d4bf3a3b243c29e6371af37dc06c1fa43b39a85811ba9714b911bf520ff67", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/SKILL.md": "1b68800cc2dd9b7b03cee3343fb8619135dc3563550cc20386606c295358df6f", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/model-and-memory.md": "ea7b905fa72cbfda5513f4c6ae77c552044af71e9313e396bcd3628884183bfb", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/notices.md": "6f6d9a96e1e67da6fe25f54482da0b69bd256fb688cc0956668f7319308f310e", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/troubleshooting.md": "97c00435d57a5b9067fe8e5885219d5818208a25f225a14c240d7160dcd697f7", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/scripts/audit_runtime_plan.py": "b096ec10c4f0e4b057997fe7b946007ab4c06dd263ab09d803e29e498601da00", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/SKILL.md": "5e264000198a9bbdded9c35db2df682e8eee4408ab741f18bc002e9be103b401", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/references/troubleshooting.md": "0c42256af4869e5cd1eb88fdaacfe65adf51dd180bdc03a2b9fccf220a19ccd8", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/references/verification-contract.md": "eaeb9319df27b66840cdfcba5c4788a287775d1493308517d9124fd0608eefec", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/scripts/summarize_decode_records.py": "cc0ac3c403aba79a131f631290d68e1640283e9f0d564aeef431b577824e3036", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/SKILL.md": "830d8bccf6b36e1ad6cef899d5147afb74c0514516c68d02060bdfb550b56920", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/api-and-history.md": "a51fa298017f266decd801441eb411fd68f7906a90a4b0d18083d1af31fc7d52", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/native-state.md": "e5a7ee5e777998db0fb5422cf5b1ab18dbd67b4cbbfcfd6628ed6e40d8709c61", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/troubleshooting.md": "97dea7b2d26fe165d2f64a1a63c91d099dddb2d3b95c16b667f5173b3ffa9d2d", - "src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/scripts/request_template.py": "d91b03fbbd7003012a92bd7259d39604b9a91ee5707da2d3366e7c0fb47edd6a" - }, - "skill_metadata": { - "name": "project-maya", - "description": "Investigate Project Maya model-family dispatch, representation and memory tiers, API versus native state, speculative decoding and source-qualified verification without treating safe CPU checks as CUDA/HIP or model evidence.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "partial_cpu_only", - "tags": [ - "project-maya", - "model-specific-inference", - "cuda", - "rocm", - "expert-tiering", - "native-state", - "speculation", - "evidence-boundaries" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/mw00/project-maya", - "commit": "249f9a87d1a9d72bec2b684d704b0aa9296b1c9c", - "version": "VERSION1.1.34; CMake0.1.27 differs", - "license": "MIT (root source); model/optional assets separate", - "readset": "references/source-identity.json", - "paths": [ - "CMakeLists.txt", - "LICENSE", - "LICENSE.deepseek-v41", - "LICENSE.minimax-m31", - "LICENSE.qwen-community-1.0", - "README.md", - "VERSION", - "bench/results/2026-10-06-glm-spec/README.md", - "bench/results/MAYA-S.md", - "docs/MODELS.md", - "docs/RESPONSES.md", - "docs/SETTINGS.md", - "include/strata/core/glm_model.hpp", - "include/strata/core/mtp_rows.hpp", - "include/strata/program/conv_cache.hpp", - "include/strata/spec/controller.hpp", - "include/strata/spec/draft_policy.hpp", - "include/strata/spec/suffix_drafter.hpp", - "maya.py", - "maya.sh", - "serve/response_config.py", - "serve/response_jobs.py", - "serve/response_schema.py", - "serve/response_store.py", - "serve/responses.py", - "serve/server.py", - "serve/test_response_jobs.py", - "serve/test_response_store.py", - "serve/test_responses.py", - "setup.py", - "setup.sh", - "src/core/glm_fast_path.cu", - "src/core/glm_memory.hpp", - "src/core/glm_model.cu", - "src/core/glm_mtp.cu", - "src/core/glm_mtp_rows.cu", - "src/core/glm_multi.cu", - "src/core/glm_window.cu", - "src/ds4/ds4_main.cpp", - "src/ds4/ds4_model.cu", - "src/ds4/ds4_model.hpp", - "src/mm3/mm3_geo.hpp", - "src/mm3/mm3_lane_plan.hpp", - "src/mm3/mm3_main.cpp", - "src/mm3/mm3_model.cu", - "src/mm3/mm3_model.hpp", - "src/mm3/mm3_pack.hpp", - "src/mm3/mm3_sample.hpp", - "src/program/conv_cache_test.cpp", - "src/program/generate.cpp", - "src/spec/controller.cpp", - "src/spec/controller_test.cpp", - "src/spec/draft_policy.cpp", - "src/spec/draft_policy_test.cpp", - "src/spec/suffix_drafter.cpp", - "src/spec/suffix_drafter_test.cpp", - "tests/core/mtp_rows_test.cpp", - "tests/glm_memory_test.cpp", - "tests/glm_pack_graph_test.cpp", - "tests/glm_rows_attn_test.cu", - "tests/glm_rows_kernels_test.cu", - "tests/mm3_lane_plan_test.cpp", - "third_party/beellama.cpp/LICENSE", - "third_party/exllamav3/LICENSE", - "third_party/ggml/LICENSE", - "tools/calibrate_draft.py", - "tools/calibrate_ds4.py", - "tools/calibrate_ds4_tp.py", - "tools/calibrate_glm.py", - "tools/calibrate_mm3_prefill.py", - "tools/ds4/pack.py", - "tools/ds4/pack_gguf.py", - "tools/exl3_pack.py", - "tools/iq_pack.py", - "tools/maya_names.py", - "tools/maya_quant/kl_eval.py", - "tools/maya_quants.py", - "tools/mm3/calibrate.py", - "tools/mm3/pack.py", - "tools/mm3/pack_gguf.py", - "tools/test_bench.py", - "tools/test_calibrate_glm.py" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Compatible-agent portable native C++ adaptation, not DisCo CLI/model execution." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Stable-ID isolated author lanes, integrated runtime graph, assertion/helper/static and selected direct-compiler CPU verification; required accelerator/model blocks retained." - }, - "applicability": { - "scope": "Model-specific dispatch/residency, API/native state and qualification investigation. Common catalog placement is not CPU inference or cross-device certification.", - "prerequisites": "Access to the selected model-family runtime/state/qualification contract; native changes need frozen correctness/comparison requirements and the matching authorized CUDA/HIP/model resources.", - "input_contract": "Full source revision, actual executable/model family, model and asset identities, representation, compiler/runtime/device, rendered workload, sampling policy, single/parallel route, exact tokens/images and relevant completion observations.", - "produced_artifact": "A route/identity/evidence-class report with invariant, proposed check, actual result, counter-signal, missing resource and next authorized action; missing native requirements remain INCONCLUSIVE.", - "failure_modes": "Transferring coverage through a common facade or quantization name, confusing API history with native KV/private state, double-counting unified RAM/VRAM, or repairing a slot header to appear compatible can invalidate diagnosis.", - "refusal_point": "When required CUDA/HIP/model evidence is unavailable, retain BLOCKED_REQUIRED_BACKEND and an unverified/INCONCLUSIVE conclusion; exit-zero skips or CPU simulations cannot clear it." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: no actual CUDA/HIP/model/device, quantized payload, vision, parallel/state/restore or cancellation-cleanup evidence.", - "Safe CPU slices and offline helpers cannot establish complete engine builds, memory fit, numerical/model quality, service/performance or broad skill efficacy.", - "Qwen coverage is shallow routing/format/guard evidence; its deep runtime/state/kernels remain unmined/unverified. Other family coverage is bounded rather than exhaustive.", - "Source reports and source-specific calibration margins are not local measurements or portable acceptance gates.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback; root MIT is retained independently and optional/model asset licenses remain separate." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Select the family-specific route and preserve the explicitly shallow Qwen coverage rather than transferring deeper GLM/DeepSeek/MiniMax evidence.", - "Separate shared weights, private conversation state, API history and native KV; count unified physical memory once.", - "Preserve exact token/image identities and completion boundaries before reuse, then test one falsifiable hypothesis without modifying slot headers to bypass compatibility." - ] - }, - "applicability": { - "scope": "Model-specific dispatch/residency, API/native state and qualification investigation. Common catalog placement is not CPU inference or cross-device certification.", - "prerequisites": "Access to the selected model-family runtime/state/qualification contract; native changes need frozen correctness/comparison requirements and the matching authorized CUDA/HIP/model resources.", - "input_contract": "Full source revision, actual executable/model family, model and asset identities, representation, compiler/runtime/device, rendered workload, sampling policy, single/parallel route, exact tokens/images and relevant completion observations.", - "produced_artifact": "A route/identity/evidence-class report with invariant, proposed check, actual result, counter-signal, missing resource and next authorized action; missing native requirements remain INCONCLUSIVE.", - "failure_modes": "Transferring coverage through a common facade or quantization name, confusing API history with native KV/private state, double-counting unified RAM/VRAM, or repairing a slot header to appear compatible can invalidate diagnosis.", - "refusal_point": "When required CUDA/HIP/model evidence is unavailable, retain BLOCKED_REQUIRED_BACKEND and an unverified/INCONCLUSIVE conclusion; exit-zero skips or CPU simulations cannot clear it." - }, - "recommendations": [ - "Select the family-specific route and preserve the explicitly shallow Qwen coverage rather than transferring deeper GLM/DeepSeek/MiniMax evidence.", - "Separate shared weights, private conversation state, API history and native KV; count unified physical memory once.", - "Preserve exact token/image identities and completion boundaries before reuse, then test one falsifiable hypothesis without modifying slot headers to bypass compatibility." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "BLOCKED_REQUIRED_BACKEND: no actual CUDA/HIP/model/device, quantized payload, vision, parallel/state/restore or cancellation-cleanup evidence.", - "Safe CPU slices and offline helpers cannot establish complete engine builds, memory fit, numerical/model quality, service/performance or broad skill efficacy.", - "Qwen coverage is shallow routing/format/guard evidence; its deep runtime/state/kernels remain unmined/unverified. Other family coverage is bounded rather than exhaustive.", - "Source reports and source-specific calibration margins are not local measurements or portable acceptance gates.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback; root MIT is retained independently and optional/model asset licenses remain separate." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/pytorch.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/pytorch.json deleted file mode 100644 index fe9f76c58..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/pytorch.json +++ /dev/null @@ -1,249 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.pytorch", - "title": "pytorch", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Operate and diagnose PyTorch installation, tensors and autograd, training and data, compilation and export, accelerator lifetimes, distributed execution, checkpointing, profiling and custom operators from pinned repository evidence.", - "tags": [ - "pytorch", - "tensor", - "autograd", - "training", - "compiler", - "distributed", - "repository-guide" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/frameworks/pytorch/pytorch/SKILL.md" - }, - "checkout_ref": "checkout:pytorch", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/LICENSE": "bd018feef8825e88181c84eb7e3aa4eafb8f08a20d9fd6ef948569610c4a3e43", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/SKILL.md": "c4bd561c73955f0c6e386b6cf3414ec772902711b2a5de6bd0613f49cbf71cb0", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/metadata.json": "b7563cebe8cb7e9d9f97a16aa6d3c3f07ed0bbced0fe20cbc570a49e3e2c5ca0", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/notices.md": "1928d322834e8fdccf061f830ba56fedc2c287ed1f25785763f213465c6e0baa", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/repo-provenance.md": "f86fba682b47f4030fe86700f4d21608d83ce5808abfb2c947dff867e589ec99", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/repo-routing-metadata.json": "908bed12129dabd490b58d958a208e720f93379ada7e70a52b78a1393457a310", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-directory-coverage.md": "ecae69ea63b2b528dc2c2aa98df502be21c126b972e73ce6e7864b240ebc95b9", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-readset.json": "4fe665a587dfd9bb02db6dc8f701d7417946c46986c1f5b960701c67304d0529", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-script-map.md": "1d669129ae4e9e2d030763fcfa66aee683c5f457537ab76db9b711100d506dfa", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/troubleshooting.md": "55df7c1517e1cf9127548d789197659612858d39d7020341960128bcbc10cba8", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/upstream-NOTICE": "c2cc7bf0caec7652c2b460a8a470bea1677f241e4ab8e431df34cf17f5a9fec0", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/scripts/inspect_environment.py": "bfb0086d93b975086c87a9b84adc629c8f468c5c4b7ceeb686206415e5a3880f", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md": "a870cb530de2d2043ac70676222a37ada6be5e8d201a2003ea55dfc4bc45c11d", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/metadata.json": "0f6e522a6cdbf4f3699ac1f13ec7c1f5b03d1913659799e8936e09647f172987", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/troubleshooting.md": "ff1b48d2acbc13f80a530a4bb3b94fc7fb7b2e553f040f778d3edfe0aee67989", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/workflows.md": "3a68c6a183426df13ccaac999abd3e94e05b0784851c8238f435c737578f0b96", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/scripts/export_contract.py": "24457ab73ca4cf7fb33de6e2a735d1de3ed2fec36f76cb57f1f1e05136c0c509", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/SKILL.md": "d8e057495573baf418459993c7f4ac5763b60ab881b4997764293d6b50f8280d", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/metadata.json": "7b863d222739d5764a814eb65373c3c9ca90aec89e02373f9988d15002587a4c", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/troubleshooting.md": "8383d490b1ffaeba3b77f07cc8eec76bf65aeee145177d7e17560d17754e747b", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/workflows.md": "74d77673576f7951b1872615cddaac4ff6cf2a14d9f35e61ad801747cc9f0b1b", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/scripts/custom_operator.py": "959ccc9dd18003bda5013c7726ba33e9c391b45c96598e76a087b956a36d6919", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md": "5325f039662563a6eaefccf36d2cf532f4424dd70666fde8f40d4d27a6f6db5e", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/metadata.json": "a3db54cd8b15cc9dc7e9cb6c04f2bc24557cdb3b6d63b3ff7fe2dc97ef3d7a2b", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/troubleshooting.md": "8afe03529f4586d131f003a20d95c610ac9d21abe625332e7ba0018b52e42b89", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/workflows.md": "d55c5e1b6d479f8947c5a8be42629370c4b8789a0af017f184d20f0006d29f39", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/rank_contract.py": "19494458f3866c11c9db4d97a90aeba338d3d5d7cf89179c1e4cb6ffd6fdc5cc", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/state_roundtrip.py": "f99df15d1b0a28ba99ce2fc9878374653d22c2e5fbfb449e4db89ce886b548e6", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/SKILL.md": "671bbebffd47b81966940bfec0765b981aa3e57593773ad089c90d16417acfe6", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/metadata.json": "25324903936db0142cf7da0d5caf172d2e4a3e58e12f827446cb7302cf08da8e", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/troubleshooting.md": "96b1358821914d81c9eb3ec0d72cacd1e1ad0868d2e1790e68ea46281026f623", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/workflows.md": "2ed61b4922760b5f7a366ec5a51a58cf33c37d3715cd5e094a769065fa57addf", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md": "09a8e64a961ec27b72d3d9b8bfbd7545549f8bd17d3d6caef81b1cf72b2076f5", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/metadata.json": "5247f1482678c98cb4e52ea15decba5fe9d2dd29435ea101e49ec719d2561664", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/troubleshooting.md": "3398de27efe1d01457d204df8b5edba3cf7faf4a2fdf6052c74ea0af28fb1cd8", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/workflows.md": "f385877df2d8c6a97c1482aa3cbfba710342649d92c6856f44c34cbeb64af325", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/scripts/cuda_lifetimes.py": "1d9d8bb5d038e35ebcd46e97062e33cbdc9f57dd04366b91edd95edee0fedd3e", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/SKILL.md": "7791883ade0d48b06ca0781e3da7499c1909edf31050aa48c4f78cd9554b64d5", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/metadata.json": "0870cd3c14b16da9c5d461621e2301c2cc9236dc68dbe08312e09ab36c2a58ba", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/troubleshooting.md": "984c5116cd2ed0ecc93d0f6e6d324c8b6c34093c9c4dae33981e31ecc9c4ed9e", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/workflows.md": "79313a2fc9037132db87214be4df91e37ba2a9c90c861c18e7a0d6c72ee19585", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/scripts/profile_step.py": "4a475f0bb2d8a06fcdc87d8661b893338279ddf4a4a83609c64c1f9a73028d7c", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/SKILL.md": "5515beb0b1e803ddad5e1c9fda40694b99cd00e98289de05ec1af3e4569d3196", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/metadata.json": "8f8859e9fe1401326db1f84e8a6fa89447910337cafae8abcd1dce3a9debdb17", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/troubleshooting.md": "4cf95feeab8a5cb87ea943158fd1a90c441979e834615050c8d88be686a845f1", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/workflows.md": "93b699c5623710f411bb371084924ab3a45057ea4d59fcad36ee52ab0f26aa26", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/scripts/tensor_contracts.py": "e5d893e42e2b91e9dd376b4b870e603193c91b096957972e2a93fa49f484b0c1", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/SKILL.md": "32751dcf5afe68283ec4ce4a51a24331a30a3cde3a4960edb86107f6910dd106", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/metadata.json": "8e929683dbd9cb8541588653fc3d34d2ef175f08cf8d78b0f50906bc1e7dd0bf", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/troubleshooting.md": "4e5c687a20e8066a9673625fc780667f490f9558a343d4d7d47d7ad278180824", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/workflows.md": "791c86dc3c13c3c87da3c4144cafb082e84f9ba2978bb71dea99f0cf649fe727", - "src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/scripts/tiny_training.py": "46af58f6ba0c7d69373efc416fcc2ae8dcada717ed29336ac84f84857b1240bb" - }, - "skill_metadata": { - "name": "pytorch", - "description": "Operate and diagnose PyTorch installation, tensors and autograd, training and data, compilation and export, accelerator lifetimes, distributed execution, checkpointing, profiling and custom operators from pinned repository evidence.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "tensor", - "autograd", - "training", - "compiler", - "distributed", - "repository-guide" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "AGENTS.md", - "LICENSE", - "NOTICE", - "README.md", - "SECURITY.md", - "aten/src/ATen/core/dispatch/DispatchKeyExtractor.h", - "aten/src/ATen/core/dispatch/Dispatcher.h", - "aten/src/ATen/native/transformers/cuda/sdp_utils.cpp", - "c10/cuda/CUDACachingAllocator.cpp", - "cmake/EnvVarForwarding.cmake", - "docs/source/distributed/checkpoint.md", - "docs/source/distributed/fsdp.fully_shard.md", - "docs/source/distributed/tensor.md", - "docs/source/func.md", - "docs/source/nested.md", - "docs/source/notes/amp_examples.md", - "docs/source/notes/autograd.md", - "docs/source/notes/cuda.md", - "docs/source/notes/ddp.md", - "docs/source/notes/extending.md", - "docs/source/notes/modules.md", - "docs/source/notes/numerical_accuracy.md", - "docs/source/notes/randomness.md", - "docs/source/notes/serialization.md", - "docs/source/onnx.md", - "docs/source/profiler.md", - "docs/source/quantization.md", - "docs/source/sparse.md", - "docs/source/user_guide/torch_compiler/torch.compiler_troubleshooting.md", - "docs/source/utils/mobile_optimizer.md", - "pyproject.toml", - "setup.py", - "test/export/test_export.py", - "test/profiler/test_profiler.py", - "test/test_autograd.py", - "test/test_cuda.py", - "test/test_custom_ops.py", - "test/test_dataloader.py", - "test/test_serialization.py", - "torch/__init__.py", - "torch/_dynamo/convert_frame.py", - "torch/_dynamo/guards.py", - "torch/_inductor/compile_fx.py", - "torch/_inductor/graph.py", - "torch/_library/custom_ops.py", - "torch/csrc/distributed/c10d/ProcessGroup.hpp", - "torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp", - "torch/distributed/checkpoint/format_utils.py", - "torch/distributed/checkpoint/state_dict_saver.py", - "torch/distributed/distributed_c10d.py", - "torch/distributed/elastic/rendezvous/c10d_rendezvous_backend.py", - "torch/distributed/rendezvous.py", - "torch/distributed/run.py", - "torch/export/__init__.py", - "torch/fx/__init__.py", - "torch/jit/__init__.py", - "torch/library.py", - "torch/nn/functional.py", - "torch/onnx/__init__.py", - "torch/optim/optimizer.py", - "torch/profiler/profiler.py", - "torch/serialization.py", - "torch/utils/benchmark/utils/timer.py", - "torch/utils/checkpoint.py", - "torch/utils/collect_env.py", - "torch/utils/cpp_extension.py", - "torch/utils/data/dataloader.py", - "version.txt" - ], - "readset": "references/source-readset.json" - } - ], - "provenance_readsets": [ - "references/source-readset.json" - ], - "references": [ - "../../../hardware/host/host-memory-management/SKILL.md", - "../dcp-async-save-performance/SKILL.md" - ], - "applicability": { - "scope": "Major user workflows and selected deep mechanisms of pinned pytorch/pytorch source; exact device/operator qualification remains task-specific.", - "prerequisites": [ - "An explicitly selected installed torch environment for runtime helpers.", - "Actual required devices, supported build and trusted ranks/storage for backend/distributed operations." - ], - "input_contract": "A concrete model/tensor/data/environment reproduction and desired operation or expected behavior.", - "produced_artifact": "A route-specific environment, tensor/gradient, graph, memory/stream, rank/state or measurement contract with checked and missing evidence.", - "failure_modes": [ - "Transferring pinned-source behavior to another release/backend without checking compatibility.", - "Confusing source inventory, CPU diagnostics and target backend execution." - ], - "refusal_point": "Withhold backend success and performance conclusions when required runtime or workload evidence is missing." - }, - "limitations": [ - "Source-derived candidate; empirical efficacy not_run.", - "Required CUDA/ROCm/MPS/XPU/external accelerator and NCCL/graph checks unavailable locally retain BLOCKED_REQUIRED_BACKEND.", - "Installed CPU wheel 2.15.0+cpu is separate from pinned source 2.16.0a0; CPU diagnostics are not same-source native verification.", - "No automatic compatibility pairing with independently pinned TorchNPU.", - "Directory inventory does not imply every file/operator/device was semantically read or run." - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Major user workflows and selected deep mechanisms of pinned pytorch/pytorch source; exact device/operator qualification remains task-specific.", - "prerequisites": [ - "An explicitly selected installed torch environment for runtime helpers.", - "Actual required devices, supported build and trusted ranks/storage for backend/distributed operations." - ], - "input_contract": "A concrete model/tensor/data/environment reproduction and desired operation or expected behavior.", - "produced_artifact": "A route-specific environment, tensor/gradient, graph, memory/stream, rank/state or measurement contract with checked and missing evidence.", - "failure_modes": [ - "Transferring pinned-source behavior to another release/backend without checking compatibility.", - "Confusing source inventory, CPU diagnostics and target backend execution." - ], - "refusal_point": "Withhold backend success and performance conclusions when required runtime or workload evidence is missing." - }, - "recommendations": [ - "Read the pinned SKILL.md and its prerequisites before choosing this procedure.", - "Read the packaged references and owner-local cases alongside the procedure." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; empirical efficacy not_run.", - "Required CUDA/ROCm/MPS/XPU/external accelerator and NCCL/graph checks unavailable locally retain BLOCKED_REQUIRED_BACKEND.", - "Installed CPU wheel 2.15.0+cpu is separate from pinned source 2.16.0a0; CPU diagnostics are not same-source native verification.", - "No automatic compatibility pairing with independently pinned TorchNPU.", - "Directory inventory does not imply every file/operator/device was semantically read or run." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/repository-evolution-analysis.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/repository-evolution-analysis.json deleted file mode 100644 index 1d711a421..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/repository-evolution-analysis.json +++ /dev/null @@ -1,159 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.repository_evolution_analysis", - "title": "repository-evolution-analysis", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Analyze a repository's commit history to build a capability-evolution model: what the system can do, how it got there, and what patterns allow it to iterate quickly. Use to produce history-derived knowledge for agent skill conditioning, and to design temporal-split benchmarks for skill evaluation.\n", - "tags": [ - "repository-intelligence", - "history", - "skill-distillation", - "evaluation" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/repository-evolution-analysis/SKILL.md" - }, - "checkout_ref": "checkout:repository-evolution-analysis", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/SKILL.md": "61bc684d67d6c0876c4956057f11c88802cfc4b771553d00411634d8ea226b3d", - "src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/metadata.json": "98ef753a0e74c46f59a69ae267bdc0a5d293761e6f27edc1822fa8bd4548c4d6", - "src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md": "c21eb4a6aabe811a49a6540a34c54dbcfdcf7efae3052a79b54034c7838026a5", - "src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/references/rtk-consumer-path-transition.md": "a70b1d128fb5e0ac12292a290d624f218a0831d7581fe8980b62661473e0f2b7" - }, - "skill_metadata": { - "name": "repository-evolution-analysis", - "description": "Analyze a repository's commit history to build a capability-evolution model: what the system can do, how it got there, and what patterns allow it to iterate quickly. Use to produce history-derived knowledge for agent skill conditioning, and to design temporal-split benchmarks for skill evaluation.\n", - "schema_version": 1, - "status": "candidate", - "sources": [ - { - "url": "https://github.com/LearningToCommit/LearningToCommit", - "commit": "7e5a1d77", - "paths": [ - "data_construction/s1_scan_commits.py", - "data_construction/s3_tag.py", - "data_construction/s4_sample.py", - "data_construction/s5_build.py", - "modules/agent.py", - "modules/prompt_builder.py", - "modules/trajectory_to_md.py", - "system_prompts/solve_with_skill.txt", - "system_prompts/learn_reflect.txt", - "evaluation/pairwise_judge.py", - "evaluation/programmatic_metrics.py" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/yetone/magpie", - "commit": "77800032", - "paths": [ - "internal/", - "docs/" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/yetone/magpie", - "commit": "4b5d6d725fae3d8a263f6e43c707400e3765b84f", - "paths": [ - "internal/library/rtk.go", - "internal/library/rtk_upgrade.go", - "internal/proc/path_other.go", - "internal/proc/path_windows.go", - "internal/provider/install_windows.go" - ], - "type": "git", - "revision_precision": "full" - }, - { - "url": "https://github.com/yetone/magpie", - "commit": "9848a9062a65c49e074c09770dc7de73dd012bea", - "paths": [ - "internal/library/rtk.go", - "internal/library/rtk_upgrade.go", - "internal/proc/path_other.go", - "internal/proc/path_windows.go", - "internal/provider/install_windows.go", - "internal/library/rtk_path.go", - "internal/library/rtk_path_other.go", - "internal/library/rtk_path_windows.go", - "internal/library/rtk_path_test.go", - "internal/library/rtk_test.go", - "internal/library/rtk_upgrade_test.go", - "internal/library/library_test.go", - "internal/gui/library.go", - "internal/gui/assets/library.js", - "internal/gui/tests/library-rtk-path.test.cjs", - "library_cli.go", - "LICENSE" - ], - "type": "git", - "revision_precision": "full" - } - ], - "verification": { - "locally_verified": false, - "evidence_level": "source_grounded", - "note": "Carried LearningToCommit pipeline and Magpie history analysis, not causal evidence of development speed. The reviewed paper amendment is linked from SKILL.md; paper-to-code equivalence is not established. First-hand Magpie coordination research remains pending Dot delivery and owner review. The RTK worked transition is pinned static source/test inspection only; no upstream execution or heldout efficacy evaluation." - }, - "applicability": { - "scope": "any software repository with a recoverable git history", - "prerequisites": "git history access, optionally LearningToCommit-style benchmark for evaluation" - }, - "tags": [ - "repository-intelligence", - "history", - "skill-distillation", - "evaluation" - ], - "profile": "local-skill-v1", - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "The RTK transition is a learning example, not an independent heldout task; upstream tests were inspected, not executed.", - "Existing audit and synthetic KV comparison adapters are not a generic historical coding-task evaluator." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "references": [ - "references/rtk-consumer-path-transition.md" - ] - }, - "applicability": { - "scope": "any software repository with a recoverable git history", - "prerequisites": "git history access, optionally LearningToCommit-style benchmark for evaluation" - }, - "recommendations": [ - "Read the pinned SKILL.md and its prerequisites before choosing this procedure.", - "Read the packaged references and owner-local cases alongside the procedure." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "The RTK transition is a learning example, not an independent heldout task; upstream tests were inspected, not executed.", - "Existing audit and synthetic KV comparison adapters are not a generic historical coding-task evaluator." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/rsi-evidence.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/rsi-evidence.json deleted file mode 100644 index 34bb1969f..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/rsi-evidence.json +++ /dev/null @@ -1,124 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.rsi_evidence", - "title": "rsi-evidence", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Guide agents to establish acceptance criteria, freeze and cite RSI evidence using plan/results/patch/validator paths and PASS/FAIL/INCONCLUSIVE verdicts.", - "tags": [ - "evidence", - "verdict", - "runspec", - "provenance", - "review" - ], - "origin": [ - "first-party" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/workflows/rsi-evidence/SKILL.md" - }, - "checkout_ref": "checkout:rsi-evidence", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/SKILL.md": "065b18fda03c714596a175495f4594ac06a9f304182190cc8fab1272a0f5cea7", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/metadata.json": "5d78e62f31978b1c9314766eaf402c83b7be1669266aa0374dfe4186f3d381ee", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/LICENSE-MIT.md": "1df1cf519b9e5d8acbbc72820299d7aa95afa1acbfc1f43a15109ad1d59975f3", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/NOTICE.md": "3c3c36a40e1d65f3a6b20ec3a97319cb753e857f1d120f302cd828e107120f6d", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/README.md": "c5802bb4d71fbf95e40a6a5282c0afcba594ab5b34d40bec97874928d8376a00", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/decision-cases.md": "c2c1d3d75c4fcb515dd36ff84d8dc940c25d54c5757a29a80d1af88eb8b66115", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/mechanisms.md": "4fa09099d362a8088fdb0e6a767f72aae6218b92fe5fd75b6fb78c2bb2a318fb", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/source-ledger.json": "60294951d702db924119300d89f4ed01d970a7036e214aa0d04c78f6842f92d9", - "src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/omni-experiments.md": "8fafc1e05ef21ab60158fc626c73ca1b8c615ada59f90fd23199496daa46ac48" - }, - "skill_metadata": { - "name": "rsi-evidence", - "description": "Guide agents to establish acceptance criteria, freeze and cite RSI evidence using plan/results/patch/validator paths and PASS/FAIL/INCONCLUSIVE verdicts.", - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "evidence", - "verdict", - "runspec", - "provenance", - "review" - ], - "empirical_validation": "not_run", - "provenance_readsets": [ - "references/loopx-goal-governance/README.md" - ], - "applicability": { - "scope": "Preparing acceptance criteria, documenting or reviewing an optimization campaign whose verdict will be cited downstream", - "prerequisites": "For preparation, the user requirements and available model/hardware/workload evidence; for verdicts, pinned baseline and candidate SHAs and an evidence tree under evidence//; conclusions about LoopX goal authority additionally need its governance reference", - "input_contract": "Campaign artifacts in plan/, results/, patch/ and validator/, plus the RunSpec that fixes repo x revision x backend x device x model x scenario", - "produced_artifact": "An evidence record that links paths instead of pasting metrics, and states exactly one of PASS, FAIL or INCONCLUSIVE with its rationale", - "failure_modes": [ - "Metrics pasted into an issue comment with no link to an evidence path", - "A verdict vocabulary outside PASS/FAIL/INCONCLUSIVE", - "PASS recorded on a cleanup or integrity-probe failure", - "Output reused across trials, aliased into a reference, or written outside its own trial directory", - "Source fingerprint taken over a directory overlap that excludes changed tracked files", - "Org-wide default SHAs assumed in a skill or CI instead of read from the acceptance issue" - ], - "refusal_point": "Refuses PASS when the validator freeze, the results rationale or the pinned SHAs are missing, and reports INCONCLUSIVE for that campaign instead" - }, - "recommendations": [ - "Use user-supplied acceptance criteria; otherwise follow performance-ceiling-analysis to formulate targets. Preserve partial requirements and record each derived field's evidence before freezing gates.", - "Link evidence paths in the report; a metric quoted only in a comment cannot be re-checked later.", - "Fingerprint tracked source even when the output directory overlaps it, and exclude only the current run's generated files.", - "Treat a missing upstream-cache release proof as INCONCLUSIVE rather than a PASS with a caveat.", - "Read baseline and candidate SHAs from the acceptance issue for that campaign, never from a skill default.", - "Freeze optional required_trial_checks for complete requests, effective graph controls and clean worker exits; require literal true per trial and retain supporting receipts." - ], - "limitations": [ - "Documents evidence discipline; it performs no measurement and certifies no campaign.", - "The LoopX goal-governance reference is a source-derived candidate and adds no execution permission or empirical validation." - ] - }, - "applicability": { - "scope": "Preparing acceptance criteria, documenting or reviewing an optimization campaign whose verdict will be cited downstream", - "prerequisites": "For preparation, the user requirements and available model/hardware/workload evidence; for verdicts, pinned baseline and candidate SHAs and an evidence tree under evidence//; conclusions about LoopX goal authority additionally need its governance reference", - "input_contract": "Campaign artifacts in plan/, results/, patch/ and validator/, plus the RunSpec that fixes repo x revision x backend x device x model x scenario", - "produced_artifact": "An evidence record that links paths instead of pasting metrics, and states exactly one of PASS, FAIL or INCONCLUSIVE with its rationale", - "failure_modes": [ - "Metrics pasted into an issue comment with no link to an evidence path", - "A verdict vocabulary outside PASS/FAIL/INCONCLUSIVE", - "PASS recorded on a cleanup or integrity-probe failure", - "Output reused across trials, aliased into a reference, or written outside its own trial directory", - "Source fingerprint taken over a directory overlap that excludes changed tracked files", - "Org-wide default SHAs assumed in a skill or CI instead of read from the acceptance issue" - ], - "refusal_point": "Refuses PASS when the validator freeze, the results rationale or the pinned SHAs are missing, and reports INCONCLUSIVE for that campaign instead" - }, - "recommendations": [ - "Use user-supplied acceptance criteria; otherwise follow performance-ceiling-analysis to formulate targets. Preserve partial requirements and record each derived field's evidence before freezing gates.", - "Link evidence paths in the report; a metric quoted only in a comment cannot be re-checked later.", - "Fingerprint tracked source even when the output directory overlaps it, and exclude only the current run's generated files.", - "Treat a missing upstream-cache release proof as INCONCLUSIVE rather than a PASS with a caveat.", - "Read baseline and candidate SHAs from the acceptance issue for that campaign, never from a skill default.", - "Freeze optional required_trial_checks for complete requests, effective graph controls and clean worker exits; require literal true per trial and retain supporting receipts." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Documents evidence discipline; it performs no measurement and certifies no campaign.", - "The LoopX goal-governance reference is a source-derived candidate and adds no execution permission or empirical validation." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/skill-evolution-playbook.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/skill-evolution-playbook.json deleted file mode 100644 index 1083d3487..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/skill-evolution-playbook.json +++ /dev/null @@ -1,111 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.skill_evolution_playbook", - "title": "skill-evolution-playbook", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Practical design choices for evolving agent skills from execution traces: when one trajectory should update a skill, how conflicting lessons are handled, how negative evidence is preserved, how large an edit should be, and when skills should split or merge. Drawn from Trace2Skill and SkillOpt. Use when designing or operating a skill evolution pipeline.\n", - "tags": [ - "skill-evolution", - "trace2skill", - "skillopt", - "rsi" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/skill-evolution-playbook/SKILL.md" - }, - "checkout_ref": "checkout:skill-evolution-playbook", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/SKILL.md": "9cd37f21fe0d862069e4798edca8a04d59f8b840e55d521901594a63dddd9ba3", - "src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/metadata.json": "a5fe391d145eb5e7039f142fea7211135abbd8f3d8835bfbf83cc2b05d645da5" - }, - "skill_metadata": { - "name": "skill-evolution-playbook", - "description": "Practical design choices for evolving agent skills from execution traces: when one trajectory should update a skill, how conflicting lessons are handled, how negative evidence is preserved, how large an edit should be, and when skills should split or merge. Drawn from Trace2Skill and SkillOpt. Use when designing or operating a skill evolution pipeline.\n", - "tags": [ - "skill-evolution", - "trace2skill", - "skillopt", - "rsi" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Design choices for evolving agent skills from execution traces, drawn from Trace2Skill and SkillOpt: when a trajectory should update a skill, how conflicting lessons are handled, how negative evidence is preserved, how large an edit should be and when skills split or merge; for designing or operating a skill evolution pipeline", - "prerequisites": "A final-test partition kept outside edit selection and the incumbent skill retained for rollback; changing a skill's scope first needs explicit review of its identity and consumers", - "input_contract": "Execution trajectories or trace IDs with the diagnosed causal failure and the successful behavior an amendment must preserve; the incumbent skill and its revision lineage; held-out evaluation results for candidate edits", - "produced_artifact": "A scoped case or amendment proposal, or a consolidated candidate edit, with rejection records and the versioned revision lineage; a candidate for independently evaluated acceptance, not a replacement decision or an efficacy claim", - "failure_modes": [ - "An anecdote, or agreement among analysts, is taken as efficacy", - "Consolidation output is treated as a replacement decision instead of a candidate awaiting independent acceptance", - "A tie or uncertain result is reported as improvement", - "A conflict is resolved by voting for whichever outcome appeared more often in the pool", - "A rejection is applied to every future use of an idea instead of the comparison it constrains", - "Paper-specific section counts or length budgets are used as universal defaults, or a skill is split on length alone", - "A stronger task solver is assumed to be a stronger improver, or replay rankings and private inner scores are read as outer generalization" - ], - "refusal_point": "Records a conflict that conditioning cannot resolve, where the same context produces different outcomes across runs, as INCONCLUSIVE and flags it for human review instead of choosing a lesson; claims no universal minimum trace count, paper-to-code equivalence or local efficacy" - }, - "recommendations": [ - "Condition a conflicting lesson on its context (when X, do A; when Y, avoid A) before choosing one side.", - "Store each rejection record with its evaluation evidence, outside the deployed task procedure, and give it a retirement condition.", - "Bound the target passage and intended behavior before editing, inspect the actual diff and preserve rollback state.", - "Prefer an amendment when the existing skill owns the workflow; create a new candidate only for a genuinely distinct capability." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "searchspace_not_applicable": "SKILL.md §§ When should one trajectory update a skill?, How large should a skill edit be?, and When should skills split or merge? govern skill-text evolution and held-out evaluation, not hardware-resource state or control." - }, - "applicability": { - "scope": "Design choices for evolving agent skills from execution traces, drawn from Trace2Skill and SkillOpt: when a trajectory should update a skill, how conflicting lessons are handled, how negative evidence is preserved, how large an edit should be and when skills split or merge; for designing or operating a skill evolution pipeline", - "prerequisites": "A final-test partition kept outside edit selection and the incumbent skill retained for rollback; changing a skill's scope first needs explicit review of its identity and consumers", - "input_contract": "Execution trajectories or trace IDs with the diagnosed causal failure and the successful behavior an amendment must preserve; the incumbent skill and its revision lineage; held-out evaluation results for candidate edits", - "produced_artifact": "A scoped case or amendment proposal, or a consolidated candidate edit, with rejection records and the versioned revision lineage; a candidate for independently evaluated acceptance, not a replacement decision or an efficacy claim", - "failure_modes": [ - "An anecdote, or agreement among analysts, is taken as efficacy", - "Consolidation output is treated as a replacement decision instead of a candidate awaiting independent acceptance", - "A tie or uncertain result is reported as improvement", - "A conflict is resolved by voting for whichever outcome appeared more often in the pool", - "A rejection is applied to every future use of an idea instead of the comparison it constrains", - "Paper-specific section counts or length budgets are used as universal defaults, or a skill is split on length alone", - "A stronger task solver is assumed to be a stronger improver, or replay rankings and private inner scores are read as outer generalization" - ], - "refusal_point": "Records a conflict that conditioning cannot resolve, where the same context produces different outcomes across runs, as INCONCLUSIVE and flags it for human review instead of choosing a lesson; claims no universal minimum trace count, paper-to-code equivalence or local efficacy" - }, - "recommendations": [ - "Condition a conflicting lesson on its context (when X, do A; when Y, avoid A) before choosing one side.", - "Store each rejection record with its evaluation evidence, outside the deployed task procedure, and give it a retirement condition.", - "Bound the target passage and intended behavior before editing, inspect the actual diff and preserve rollback state.", - "Prefer an amendment when the existing skill owns the workflow; create a new candidate only for a genuinely distinct capability." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/splash.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/splash.json deleted file mode 100644 index 394c77c37..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/splash.json +++ /dev/null @@ -1,212 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.splash", - "title": "splash", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Investigate Splash model-format compatibility, automatic memory planning, Metal and ANE execution, DFlash2 verification, cache and request identity, and correctness-gated serving measurements.", - "tags": [ - "splash", - "apple-silicon", - "metal", - "ane", - "dflash2", - "memory-plan", - "prefix-cache", - "persistent-cache" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/splash/SKILL.md" - }, - "checkout_ref": "checkout:splash", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/splash/LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/SKILL.md": "619b87c282dcec4befa319085764fb53fa506a05c6a7112500bcca662f059ece", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/THIRD_PARTY_NOTICES": "2fedf1277bf732f8435c3e3be2798f52f25f7e2bd4155865cb366da55b88e50b", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/metadata.json": "9d0613387fdaee61de7e79146e2a896f19ac7f5c1d355f74afe767074f64dd2a", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/references/repo-provenance.md": "20e3462f1b6a8752eb0fa6f36e76867f142a49c1953f498f895e29c4fd4f0204", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/references/repo-routing-metadata.json": "287ae591f42949b8d33312659ec8303c47d5c73093c64c125edb1cf6ea8de8b9", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/references/source-identity.json": "2f778882ec440538a46f48332650412b446722246c8dc5690083f19da7dce61d", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/scripts/request_template.py": "086d773d42edf67c43cedae16974061b09ac66458ee47fe961fc01733f3b2a0e", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/SKILL.md": "4fddc39811c12ec4b3279ad68a8864b7f1ef80e838ea03788b54cb025a03ae16", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/references/troubleshooting.md": "dbd2179a48eb3fc0ecfd54c5df8a690d0cb2e1814acc19ac3b0aabf184092c92", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/references/verification-contract.md": "9efffa0ecfd3b6771dfce631137eccc6d939c3239f60b1fadcd0c6c1ed7700be", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/scripts/summarize_rounds.py": "107a844954e3909500c60eb1ca365ac9b5e7dc3d91122dbae46f402a5cef4e3b", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/SKILL.md": "f0fe3d3f4053937767ae586559cc7fba89dacb6c896186a029fff220414e8d22", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/references/memory-and-formats.md": "04edf10f4a965c512f6a262b1f8295f3c033d615849e49d21d3d94c78b549f2b", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/references/troubleshooting.md": "5a1896f9fb01f6c8c9ef78d94f3c965d6d64e6b54223c6d7901eaed0548c5575", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/scripts/parse_budget.py": "b3a3b0847280333c2bfbe5d03350dc866307d5290160ec64c7e3d462a34f6301", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/SKILL.md": "c16f911498e116ff9b31b5a6f0ddd3ae6517b51c0e9d079c337f36a5bcac1df8", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/references/state-and-api.md": "592603177473ac8e024b7dc19a3d3faedd7c1e0312137eab8bf9a62d2f567be6", - "src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/references/troubleshooting.md": "4fda461042a3de43c109d7a3cec67542c7d61fe68412bf91606220f2e13e576e" - }, - "skill_metadata": { - "name": "splash", - "description": "Investigate Splash model-format compatibility, automatic memory planning, Metal and ANE execution, DFlash2 verification, cache and request identity, and correctness-gated serving measurements.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "partial_cpu_only", - "tags": [ - "splash", - "apple-silicon", - "metal", - "ane", - "dflash2", - "memory-plan", - "prefix-cache", - "persistent-cache" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/incoai/splash", - "commit": "39a97385f8560cf8a93972cec9b0a963c9195250", - "paths": [ - "README.md", - "DEVELOPMENT.md", - "docs/performance.md", - "LICENSE", - "THIRD_PARTY_NOTICES", - "Makefile", - "server/serve_options.py", - "server/frontend.py", - "server/lru.py", - "server/protocol.py", - "server/http_security.py", - "server/origins.py", - "server/runtime.py", - "install/gguf.py", - "runtime/engine/MemoryPlan.hpp", - "runtime/engine/MemoryPlan.cpp", - "runtime/engine/MemoryGovernor.cpp", - "runtime/engine/MemoryControl.cpp", - "runtime/engine/MemoryAudit.hpp", - "runtime/engine/MemoryAudit.cpp", - "runtime/engine/KvCache.hpp", - "runtime/engine/KvCache.cpp", - "runtime/engine/Cache.hpp", - "runtime/engine/Cache.cpp", - "runtime/engine/StateCache.hpp", - "runtime/engine/StateCache.cpp", - "runtime/engine/Scheduler.cpp", - "runtime/engine/WriteBehind.hpp", - "runtime/engine/WriteBehind.cpp", - "runtime/engine/RuntimeResources.mm", - "runtime/engine/Protocol.hpp", - "runtime/engine/Protocol.cpp", - "runtime/ops/DraftSelector.hpp", - "runtime/ops/DraftSelector.cpp", - "runtime/ops/Sampling.hpp", - "runtime/ops/Sampling.cpp", - "runtime/metal/kernels/decode/sampling.metal", - "runtime/model/Runtime.hpp", - "runtime/model/Runtime.cpp", - "runtime/model/QwenState.cpp", - "runtime/model/ModelDescriptor.mm", - "runtime/model/DFlashDraft.hpp", - "runtime/model/VerifyBudget.hpp", - "dev/benchmarks/abba.py", - "dev/benchmarks/backend_regression.py", - "dev/benchmarks/logit_compare.py", - "dev/benchmarks/http_regression.py", - "dev/tests/server/test_serve_options.py", - "dev/tests/server/test_lru_cache.py", - "dev/tests/server/test_protocol_python.py", - "dev/tests/tools/test_abba.py", - "dev/tests/engine/engine_memory_plan_test.cpp", - "dev/tests/engine/memory_audit_test.cpp" - ], - "license": "Apache-2.0", - "readset": "references/source-identity.json" - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Actual portable compatible-agent workflow with native/source-module adaptation, not DisCo CLI/model execution." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "User-approved native Metal/C++ and isolated source-Python adaptation; stable-ID lanes, integrated graph, assertion cases and create/verify stages. Required-backend block disables automatic admission." - }, - "applicability": { - "scope": "Pinned Splash representation, planning, state/serving and verification investigation; common discovery does not qualify a different Mac/model/backend.", - "prerequisites": "Access to the pinned representation/planning, serving-cache or qualification route; native checks need matching Apple/backend/model resources, an authorized workload and frozen semantic/measurement controls.", - "input_contract": "Source/build, macOS/device, target and compatible draft artifacts/formats, tokenizer/template/vision selection, context/cache/memory controls, KV/recurrent precision, ANE and verifier-row policy, concurrency/priority/sampling, quality controls and cache state.", - "produced_artifact": "An execution-identity report with observation, source-backed invariant, alternate explanation, smallest discriminating check, actual result and unresolved Metal/ANE/model/disk-state gap.", - "failure_modes": "Treating an API alias as new weights, equal bit width as loader/kernel compatibility, or a fixed seed as fixed cache/batch/adaptive-verification scheduling can invalidate the comparison; CPU cache/codec checks do not prove GPU state.", - "refusal_point": "Stop at unsupported formats, unavailable backend/model resources, unsafe service exposure or the authorized budget." - }, - "limitations": [ - "Required Metal/ANE/model/actual device and disk-state execution remains BLOCKED_REQUIRED_BACKEND; Python policy/codec checks do not clear it.", - "Service performance, quality, native recurrent-state recovery and skill efficacy remain unverified.", - "Only selected bodies were mined; not all formats, kernels, supported devices or release/client workflows.", - "NO_LICENSE is the actual unauthenticated GitHub CLI resolver fallback, not a legal conclusion; Apache-2.0 and third-party notices are retained." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve representation, memory/cache owner, validity key and reclaim/rollback boundary before tuning a numeric control.", - "Freeze semantic and workload controls beyond the seed, and retain actual-path fallbacks, admission waits, evictions, cancellations and errors.", - "Inspect options without starting the normal serve path, which can download target/draft weights; installation and serving require separate task authority." - ] - }, - "applicability": { - "scope": "Pinned Splash representation, planning, state/serving and verification investigation; common discovery does not qualify a different Mac/model/backend.", - "prerequisites": "Access to the pinned representation/planning, serving-cache or qualification route; native checks need matching Apple/backend/model resources, an authorized workload and frozen semantic/measurement controls.", - "input_contract": "Source/build, macOS/device, target and compatible draft artifacts/formats, tokenizer/template/vision selection, context/cache/memory controls, KV/recurrent precision, ANE and verifier-row policy, concurrency/priority/sampling, quality controls and cache state.", - "produced_artifact": "An execution-identity report with observation, source-backed invariant, alternate explanation, smallest discriminating check, actual result and unresolved Metal/ANE/model/disk-state gap.", - "failure_modes": "Treating an API alias as new weights, equal bit width as loader/kernel compatibility, or a fixed seed as fixed cache/batch/adaptive-verification scheduling can invalidate the comparison; CPU cache/codec checks do not prove GPU state.", - "refusal_point": "Stop at unsupported formats, unavailable backend/model resources, unsafe service exposure or the authorized budget." - }, - "recommendations": [ - "Resolve representation, memory/cache owner, validity key and reclaim/rollback boundary before tuning a numeric control.", - "Freeze semantic and workload controls beyond the seed, and retain actual-path fallbacks, admission waits, evictions, cancellations and errors.", - "Inspect options without starting the normal serve path, which can download target/draft weights; installation and serving require separate task authority." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Required Metal/ANE/model/actual device and disk-state execution remains BLOCKED_REQUIRED_BACKEND; Python policy/codec checks do not clear it.", - "Service performance, quality, native recurrent-state recovery and skill efficacy remain unverified.", - "Only selected bodies were mined; not all formats, kernels, supported devices or release/client workflows.", - "NO_LICENSE is the actual unauthenticated GitHub CLI resolver fallback, not a legal conclusion; Apache-2.0 and third-party notices are retained." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/storage-io-measurement.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/storage-io-measurement.json deleted file mode 100644 index 11a25df7f..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/storage-io-measurement.json +++ /dev/null @@ -1,98 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.storage_io_measurement", - "title": "storage-io-measurement", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Measure storage I/O for LLM/diffusion serving without conflating device metrics with application outcomes: match the fio workload to the observed loader signature, validate requested-versus-achieved I/O depth, keep a cache-state ledger (warm/cold/unknown with method), classify startup vs steady-state runs, read outputs with unit and latency-provenance awareness, attribute traffic per device (iostat) and per process (pidstat), and gate any \"storage bottleneck\" claim through a saturation-evidence check plus a separate application-outcome layer. Measurement methodology only — running benchmarks or manipulating caches requires separate authorization.\n", - "tags": [ - "storage", - "fio", - "iostat", - "pidstat", - "measurement", - "io", - "evidence" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/host/storage-io-measurement/SKILL.md" - }, - "checkout_ref": "checkout:storage-io-measurement", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/SKILL.md": "63b883c10a8d3e7d918abfc7175b72e2b401ff1e02697f94b8382ec1c8f369b7", - "src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/metadata.json": "5017aa6c04b8cc45d8549b71bc283bfced8e7e1bdb29cf0b305a0447a3c12a9d" - }, - "skill_metadata": { - "name": "storage-io-measurement", - "description": "Measure storage I/O for LLM/diffusion serving without conflating device metrics with application outcomes: match the fio workload to the observed loader signature, validate requested-versus-achieved I/O depth, keep a cache-state ledger (warm/cold/unknown with method), classify startup vs steady-state runs, read outputs with unit and latency-provenance awareness, attribute traffic per device (iostat) and per process (pidstat), and gate any \"storage bottleneck\" claim through a saturation-evidence check plus a separate application-outcome layer. Measurement methodology only — running benchmarks or manipulating caches requires separate authorization.\n", - "tags": [ - "storage", - "fio", - "iostat", - "pidstat", - "measurement", - "io", - "evidence" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Measure and attribute storage traffic for model loading or serving while separating device/process observations from application outcomes and cache/startup effects.", - "prerequisites": "Resolve backing devices and participating worker PIDs first; active fio benchmarks or cache manipulation require separate authorization, with bounded observation during already-authorized runs.", - "input_contract": "Observed loader access pattern, tool/OS/filesystem versions, file-to-device mapping, worker PIDs, phase timestamps, cache-state method and raw IO/job/application measurements.", - "produced_artifact": "A workload- and cache-qualified IO evidence record with units, achieved depth, errors/bytes/completion and device/process attribution, plus a separately measured application outcome where benefit is claimed.", - "refusal_point": "Reject unspecified or copied-example benchmark workloads; do not name a storage bottleneck from util/iowait alone or infer application gain from synthetic bandwidth." - }, - "recommendations": [ - "Match the fio read pattern to the observed loader and verify achieved rather than requested depth; more queued work can increase latency.", - "Record warm/cold/unknown cache state and its method, separate startup from repeated/steady reads, and check completed bytes for truncated runs.", - "Align per-device and per-process observations with the application phase, retain units and latency provenance, and corroborate saturation using traffic, queues and process activity.", - "Only run a separately authorized bounded read-only job against an explicit existing regular file, then remeasure the real application before accepting a benefit." - ] - }, - "applicability": { - "scope": "Measure and attribute storage traffic for model loading or serving while separating device/process observations from application outcomes and cache/startup effects.", - "prerequisites": "Resolve backing devices and participating worker PIDs first; active fio benchmarks or cache manipulation require separate authorization, with bounded observation during already-authorized runs.", - "input_contract": "Observed loader access pattern, tool/OS/filesystem versions, file-to-device mapping, worker PIDs, phase timestamps, cache-state method and raw IO/job/application measurements.", - "produced_artifact": "A workload- and cache-qualified IO evidence record with units, achieved depth, errors/bytes/completion and device/process attribution, plus a separately measured application outcome where benefit is claimed.", - "refusal_point": "Reject unspecified or copied-example benchmark workloads; do not name a storage bottleneck from util/iowait alone or infer application gain from synthetic bandwidth." - }, - "recommendations": [ - "Match the fio read pattern to the observed loader and verify achieved rather than requested depth; more queued work can increase latency.", - "Record warm/cold/unknown cache state and its method, separate startup from repeated/steady reads, and check completed bytes for truncated runs.", - "Align per-device and per-process observations with the application phase, retain units and latency provenance, and corroborate saturation using traffic, queues and process activity.", - "Only run a separately authorized bounded read-only job against an explicit existing regular file, then remeasure the real application before accepting a benefit." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/training-input-pipeline.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/training-input-pipeline.json deleted file mode 100644 index 6658d52ac..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/training-input-pipeline.json +++ /dev/null @@ -1,117 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.training_input_pipeline", - "title": "training-input-pipeline", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose training input stalls and choose bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic Streaming and FFCV. Preserve execution-mode gates, sample ordering and per-host memory budgets; separate remote downloads, local storage reads and host/device transfers.\n", - "tags": [ - "training", - "input-pipeline", - "prefetch", - "dataset", - "memory", - "storage" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/training-input-pipeline/SKILL.md" - }, - "checkout_ref": "checkout:training-input-pipeline", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/SKILL.md": "64bd2c6b795d966045d1d3cca55976d3d49b08b06df7c7e06a70c2824cf55b7e", - "src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/metadata.json": "debad47896f600c7a1e19e477e34026c975dfde320a786dca1ddd02030a5feb1", - "src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/references/sources.md": "60e0f3e88655d8e1d5eb4797f39ca4370deda01614f265b8d2ea4559d8ac020f" - }, - "skill_metadata": { - "name": "training-input-pipeline", - "description": "Diagnose training input stalls and choose bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic Streaming and FFCV. Preserve execution-mode gates, sample ordering and per-host memory budgets; separate remote downloads, local storage reads and host/device transfers.\n", - "tags": [ - "training", - "input-pipeline", - "prefetch", - "dataset", - "memory", - "storage" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Diagnosing training input stalls and choosing bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic Streaming and FFCV, while preserving execution-mode gates, sample ordering and per-host memory budgets and separating remote downloads, local storage reads and host/device transfers", - "prerequisites": "The pinned readset and its license constraints are read; reading authorizes no commands, cache manipulation or experiment", - "input_contract": "Dataset format, sample-size distribution, workers and ranks per host, ordering/shuffle, local-cache state and executor mode; a per-host memory budget that includes checkpoint staging and runtime offload; timings that separate remote fetch, decompression/decoding, transformation, batch readiness and device transfer", - "produced_artifact": "One proposed queue, shard-cache or cache-ownership change with its source-backed rule and boundary, the evidence proposed to confirm it, and a verification contract for a separately authorized comparison; no input-throughput, model-quality or hardware result", - "failure_modes": [ - "Low accelerator utilization is taken to identify which producer stage is late", - "Missing legacy DALI statistics are read as zero use, or the execution mode is switched silently to obtain them, which changes the comparison", - "DALI run() returning is read as output-ready timing instead of the outputs() synchronization boundary", - "Mosaic predownload is treated as a byte cap or request count, or per-worker sample targets are multiplied into an exact network-byte prediction", - "Look-ahead is increased when the local cache cannot retain the working set, so evicted shards are downloaded again", - "cache_limit is read as a bound on total process RAM, or the constructor minimum as an optimum or a guarantee against thrashing", - "An FFCV memory map is taken as proof that its pages are resident" - ], - "refusal_point": "Rejects changes that violate sample coverage or the stated reproducibility contract even if batches arrive faster, does not report queue depth alone as a performance result, and draws no NPU acceleration claim from the CUDA-oriented DALI source" - }, - "recommendations": [ - "Record dataset format, sample sizes, workers per host, ordering, cache state and executor mode before proposing one change.", - "Increase Mosaic look-ahead only when readiness starvation is observed and cache headroom exists; otherwise address shard and working-set residency first.", - "Check the exact DALI configuration surface before use, because the constructor and deserialization paths use different queue keys.", - "In a later comparison, separate warm-cache throughput from cold retrieval and initialization, and report step time with input wait, host/HBM peaks and actual I/O." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/sources.md" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Diagnosing training input stalls and choosing bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic Streaming and FFCV, while preserving execution-mode gates, sample ordering and per-host memory budgets and separating remote downloads, local storage reads and host/device transfers", - "prerequisites": "The pinned readset and its license constraints are read; reading authorizes no commands, cache manipulation or experiment", - "input_contract": "Dataset format, sample-size distribution, workers and ranks per host, ordering/shuffle, local-cache state and executor mode; a per-host memory budget that includes checkpoint staging and runtime offload; timings that separate remote fetch, decompression/decoding, transformation, batch readiness and device transfer", - "produced_artifact": "One proposed queue, shard-cache or cache-ownership change with its source-backed rule and boundary, the evidence proposed to confirm it, and a verification contract for a separately authorized comparison; no input-throughput, model-quality or hardware result", - "failure_modes": [ - "Low accelerator utilization is taken to identify which producer stage is late", - "Missing legacy DALI statistics are read as zero use, or the execution mode is switched silently to obtain them, which changes the comparison", - "DALI run() returning is read as output-ready timing instead of the outputs() synchronization boundary", - "Mosaic predownload is treated as a byte cap or request count, or per-worker sample targets are multiplied into an exact network-byte prediction", - "Look-ahead is increased when the local cache cannot retain the working set, so evicted shards are downloaded again", - "cache_limit is read as a bound on total process RAM, or the constructor minimum as an optimum or a guarantee against thrashing", - "An FFCV memory map is taken as proof that its pages are resident" - ], - "refusal_point": "Rejects changes that violate sample coverage or the stated reproducibility contract even if batches arrive faster, does not report queue depth alone as a performance result, and draws no NPU acceleration claim from the CUDA-oriented DALI source" - }, - "recommendations": [ - "Record dataset format, sample sizes, workers per host, ordering, cache state and executor mode before proposing one change.", - "Increase Mosaic look-ahead only when readiness starvation is observed and cache headroom exists; otherwise address shard and working-set residency first.", - "Check the exact DALI configuration surface before use, because the constructor and deserialization paths use different queue keys.", - "In a later comparison, separate warm-cache throughput from cold retrieval and initialization, and report step time with input wait, host/HBM peaks and actual I/O." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/training-resource-budget.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/training-resource-budget.json deleted file mode 100644 index 449bfa25f..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/training-resource-budget.json +++ /dev/null @@ -1,133 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.training_resource_budget", - "title": "training-resource-budget", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Choose training offload and resource-budget changes using pinned DeepSpeed configuration, attributed expert diagnosis and a historical MindSpeed case. Balance accelerator capacity, CPU work, pinned RAM and local storage; keep host/device transfer exposure distinct from collective network bottlenecks.\n", - "tags": [ - "training", - "offload", - "resource-budget", - "memory", - "cpu", - "numa" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/training-resource-budget/SKILL.md" - }, - "checkout_ref": "checkout:training-resource-budget", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/SKILL.md": "d3805c5f605faae663ad5a674a56c1e1e5fdd211cd58833c008e557cfd7e4c37", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/metadata.json": "9ae014b0ca5ca5d32e31e05b52e7436ad1c197fdce8d731fd4d9c79965fd4f2d", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/mindspeed-historical.md": "ad65c27f278c3d0ca3599a7630c02d3e529477a05b71b7703bd2d75f0d8fdd2c", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/mindspeed-source-readset.md": "4462d69ed6a941a03639c756968be3ce54f947e51f6ca618f4fb1cd43b643b35", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/INTEGRATION_NOTES.md": "c55c6ad2131ef788866411dcc7056f221420ebff30c95c6bc919b09a4818cb73", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/cases/paper-grounded-cases.md": "19bd25b1a944adb2351ebc6b1d795150ea033041a44f6cc261e881677812a101", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/knowledge/offload-delta.md": "f8740fc5c8bbbe5ac13ff1cfec7ca499569d0b46e8cd56cf1f5a44c3a5e72242", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/notes/training-memory-scheduling.md": "9aa6d3e01452dff95c8a8940ff62ead8c146ce215c9175b78c9f2ef508a7b2de", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md": "2a5c166767f56fa47e7eba7386f1f36da8cb73702e5b5bbe14d6a456ca9ebfae", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/sources.md": "53d70671a183de4aaa3369f9caf2383a23b6f8a0a7b04464a8d8a639150afc6a", - "src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/stas-decision-notes.md": "f1a993e6c4ca9a035a2effcaf5500081f227b0cc56cc885a6b22fda0041fc744" - }, - "skill_metadata": { - "name": "training-resource-budget", - "description": "Choose training offload and resource-budget changes using pinned DeepSpeed configuration, attributed expert diagnosis and a historical MindSpeed case. Balance accelerator capacity, CPU work, pinned RAM and local storage; keep host/device transfer exposure distinct from collective network bottlenecks.\n", - "tags": [ - "training", - "offload", - "resource-budget", - "memory", - "cpu", - "numa" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Choosing training offload and resource-budget changes from pinned DeepSpeed configuration, attributed expert diagnosis and a historical MindSpeed case, balancing accelerator capacity, CPU work, pinned RAM and local storage while keeping host/device transfer exposure distinct from collective network bottlenecks", - "prerequisites": "The source boundaries are read; garbage-collection or affinity changes, privileged memlock changes and delayed parameter updates each need separate approval, and the skill authorizes no newest-wheel installation, scheduler-priority change, benchmark or failure injection; historical MindSpeed reuse requires compatible allocator symbols, tracked swappable storage, sufficient host capacity and verified device/NUMA/CPU mapping, not merely the presence of npu-smi and lspci", - "input_contract": "State attributed to parameters, optimizer state, gradients or activations with its current memory tier, lifetime and next consumer; whether the failure is allocation capacity or throughput; a per-host budget for input buffers, checkpoint snapshots, offloaded state, OS cache and CPU workers with the actual ranks sharing the host; the host/device link recorded apart from collective traffic; the exact source version and enabled execution mode", - "produced_artifact": "A candidate decision record per proposal: the observed limiting interval and owning state or resource, source version and execution mode, capacity released and resources consumed with units and uncertainty, overlap readiness and mutation boundaries with expected contention, and the fixed-workload step and correctness evidence required for later validation", - "failure_modes": [ - "A change that lets the model fit is presented as a speedup, although it can slow each step", - "A tensor copied to CPU is taken as an InfiniBand/RoCE bottleneck, or CPU/NVMe offload is counted as network evidence", - "Parameter max_in_cpu is read as a byte or total-host-memory cap instead of an element count", - "NVMe optimizer placement is assumed to remove CPU optimizer demand", - "The parameter buffer-pool count is transplanted to the optimizer, or all optimizers are assumed to hold identical states", - "Configuration presence or schema documentation is taken as evidence that a knob or overlap actually ran", - "Ordinary overlap is taken as permission for delayed, stale parameter updates" - ], - "refusal_point": "Claims no hardware validation, measured gain or current NPU compatibility, keeps source support and extracted coverage separate from measured improvement, and does not reuse the MindSpeed headline gain; historical source observations do not establish current-stack compatibility, whole-model correctness or measured benefit" - }, - "recommendations": [ - "Attribute the state and separate a capacity failure from a throughput bottleneck before choosing an offload surface.", - "Verify the consumer of buffer_size before calculating bytes or tuning it, and measure the executing path before claiming a knob ran.", - "Compare the whole step's critical path, not host capacity alone.", - "Use plan-activation-residency, choose-parallel-training-plan and assess-split-backward-schedule for live-range, parallel-plan and backward-ordering questions." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/mindspeed-historical.md", - "references/mindspeed-source-readset.md", - "references/papers-20261015/cases/paper-grounded-cases.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/offload-delta.md", - "references/papers-20261015/notes/training-memory-scheduling.md", - "references/papers-20261015/references/source-readset.md", - "references/sources.md", - "references/stas-decision-notes.md" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Choosing training offload and resource-budget changes from pinned DeepSpeed configuration, attributed expert diagnosis and a historical MindSpeed case, balancing accelerator capacity, CPU work, pinned RAM and local storage while keeping host/device transfer exposure distinct from collective network bottlenecks", - "prerequisites": "The source boundaries are read; garbage-collection or affinity changes, privileged memlock changes and delayed parameter updates each need separate approval, and the skill authorizes no newest-wheel installation, scheduler-priority change, benchmark or failure injection; historical MindSpeed reuse requires compatible allocator symbols, tracked swappable storage, sufficient host capacity and verified device/NUMA/CPU mapping, not merely the presence of npu-smi and lspci", - "input_contract": "State attributed to parameters, optimizer state, gradients or activations with its current memory tier, lifetime and next consumer; whether the failure is allocation capacity or throughput; a per-host budget for input buffers, checkpoint snapshots, offloaded state, OS cache and CPU workers with the actual ranks sharing the host; the host/device link recorded apart from collective traffic; the exact source version and enabled execution mode", - "produced_artifact": "A candidate decision record per proposal: the observed limiting interval and owning state or resource, source version and execution mode, capacity released and resources consumed with units and uncertainty, overlap readiness and mutation boundaries with expected contention, and the fixed-workload step and correctness evidence required for later validation", - "failure_modes": [ - "A change that lets the model fit is presented as a speedup, although it can slow each step", - "A tensor copied to CPU is taken as an InfiniBand/RoCE bottleneck, or CPU/NVMe offload is counted as network evidence", - "Parameter max_in_cpu is read as a byte or total-host-memory cap instead of an element count", - "NVMe optimizer placement is assumed to remove CPU optimizer demand", - "The parameter buffer-pool count is transplanted to the optimizer, or all optimizers are assumed to hold identical states", - "Configuration presence or schema documentation is taken as evidence that a knob or overlap actually ran", - "Ordinary overlap is taken as permission for delayed, stale parameter updates" - ], - "refusal_point": "Claims no hardware validation, measured gain or current NPU compatibility, keeps source support and extracted coverage separate from measured improvement, and does not reuse the MindSpeed headline gain; historical source observations do not establish current-stack compatibility, whole-model correctness or measured benefit" - }, - "recommendations": [ - "Attribute the state and separate a capacity failure from a throughput bottleneck before choosing an offload surface.", - "Verify the consumer of buffer_size before calculating bytes or tuning it, and measure the executing path before claiming a knob ran.", - "Compare the whole step's critical path, not host capacity alone.", - "Use plan-activation-residency, choose-parallel-training-plan and assess-split-backward-schedule for live-range, parallel-plan and backward-ordering questions." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/video-diffusion-optimization.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/video-diffusion-optimization.json deleted file mode 100644 index 1b68386b6..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/video-diffusion-optimization.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.video_diffusion_optimization", - "title": "video-diffusion-optimization", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Run and verify goal-driven video diffusion optimization, backend-agnostic. Use when the user asks for latency, RTF, throughput, or memory targets across VLM, DiT, MoE, attention, or VAE, on any accelerator. Not for text-only LLMs.", - "tags": [ - "video-diffusion", - "dit", - "moe", - "attention", - "vae", - "performance", - "profiling", - "serving", - "optimization" - ], - "origin": [ - "first-party" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/video-diffusion-optimization/SKILL.md" - }, - "checkout_ref": "checkout:video-diffusion-optimization", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/SKILL.md": "fc0e7baf9913ce7448247772a1e224a9cbd827492cf9459b1f17f492b9f67b70", - "src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/assets/campaign.template.json": "1a37f1d136612143ddab8f0cbbf424c80433b4da5312edb18d07c0b9d87ef657", - "src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/metadata.json": "b27d69142dfea5dd2820170475c2bf6cced2d5eefd1c43b3413c41849af0fd02", - "src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/scripts/evaluate_target.py": "9e84df051e03d567746fd73ae7243efb2f3c5ec806fef824d3c4d42a8eb9704c" - }, - "skill_metadata": { - "name": "video-diffusion-optimization", - "description": "Run and verify goal-driven video diffusion optimization, backend-agnostic. Use when the user asks for latency, RTF, throughput, or memory targets across VLM, DiT, MoE, attention, or VAE, on any accelerator. Not for text-only LLMs.", - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "tags": [ - "video-diffusion", - "dit", - "moe", - "attention", - "vae", - "performance", - "profiling", - "serving", - "optimization" - ], - "empirical_validation": "not_run", - "verification": { - "locally_verified": false, - "evidence_level": "guidance", - "note": "Procedure is first-hand workflow guidance. The linked field notes are not reproducible as published. No acceptance campaign under this repository's evidence contract backs this skill, and no public performance claim is made." - }, - "applicability": { - "scope": "Backend-agnostic video diffusion inference and serving optimization", - "hardware": "Any accelerator (Ascend NPU, CUDA GPU, and others); operator names, memory layouts, and profiler references are re-derived per backend", - "prerequisites": "A reproducible benchmark, a frozen quality check, and numeric gates defined in the governing acceptance issue" - }, - "limitations": [ - "Backend-agnostic skeleton only. The first-hand field notes are Ascend-specific and live in the ascend-video-diffusion-optimization skill; every operator, layout and profiler reference (TransData, FRACTAL_NZ, CANN) must be re-derived per backend.", - "Candidate workflow guidance; not independently validated.", - "Model identifiers, campaign revisions, and numeric gates are intentionally excluded and remain issue-scoped." - ], - "provenance_readsets": [], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Backend-agnostic video diffusion inference and serving optimization", - "hardware": "Any accelerator (Ascend NPU, CUDA GPU, and others); operator names, memory layouts, and profiler references are re-derived per backend", - "prerequisites": "A reproducible benchmark, a frozen quality check, and numeric gates defined in the governing acceptance issue" - }, - "recommendations": [ - "Read the pinned SKILL.md and its prerequisites before choosing this procedure.", - "Read the packaged references and owner-local cases alongside the procedure." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Backend-agnostic skeleton only. The first-hand field notes are Ascend-specific and live in the ascend-video-diffusion-optimization skill; every operator, layout and profiler reference (TransData, FRACTAL_NZ, CANN) must be re-derived per backend.", - "Candidate workflow guidance; not independently validated.", - "Model identifiers, campaign revisions, and numeric gates are intentionally excluded and remain issue-scoped." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/vllm-metal.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/vllm-metal.json deleted file mode 100644 index ad78d146f..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/vllm-metal.json +++ /dev/null @@ -1,204 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.vllm_metal", - "title": "vllm-metal", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Investigate vllm-metal Apple inference through platform and native-ABI selection, paged KV/offload ownership, speculative/deferred state and actual dispatch/serving evidence without upgrading source or CPU checks to backend qualification.", - "tags": [ - "vllm-metal", - "apple-silicon", - "mlx", - "mps", - "paged-kv", - "kv-offload", - "speculation", - "deferred-execution", - "evidence-boundaries" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/vllm-metal/SKILL.md" - }, - "checkout_ref": "checkout:vllm-metal", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/SKILL.md": "5f83b1b989e8dc239a6f34a6b3bd63872533ecc1404483a69dabfb1865648439", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/metadata.json": "55652ec6de744d92044951e25ea89aa18c92b14846dd4b597444d773bb8c31e3", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/repo-provenance.md": "970bda4d1081704aa82c94d6544a98bbf479430c4fc2c82a1a82f206043aa72a", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/repo-routing-metadata.json": "230a2b383bcf2855af4de50374c41f11b0b0841268ae45e424c1c97f679b477a", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/scope-and-notices.md": "aaef98eaea4cff56f39852e1d131163ebc01334e79034b1890f67a2660727d3e", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/source-identity.json": "9da529574e81f9f91d86c01153eb09a8b1e728dc5327c40364fff74085598c1f", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/SKILL.md": "dbfb06896615dc994f4a0a1e3789904acdf512ec316cbab6347165b764bc592f", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/cache-and-offload.md": "9829c14cf20f22575c44c1a0fcf18f63578443710e1a592c9fea709a43a3b888", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/helper-contract.md": "56a7494b58c47e07d1c85471ddc6075b15e34dc7a852b937002416dbd55b65ce", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/troubleshooting.md": "a4600c6d17a980b6442fc4c36f1c9ae148a2710c644126a9a898675b01f95366", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/scripts/screen_pool_budget.py": "dd615f7d3239f48510ce83416fb14fba8d5da2b9017627bdd312567be1bd7956", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/SKILL.md": "e2f5c5207b5c7665d3388d94fdace35e25f073a7833cbddaf14514a05f5ca5b9", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/configuration.md": "69949b93f4f319145e37d1b28487e37bd4ef21622f3169583d9cff46234a9c76", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/platform-and-loading.md": "7f11dc9e4bb9ae7bac25fcd6a2909cbea0eb2ae329e28ec9a7a2f4085c56e5cc", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/troubleshooting.md": "837cef82321fa050af3f01804fe337b0a0462255261fea5c21158ecfa3d0c325", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/scripts/check_environment_plan.py": "2603200fef87e3afad7820b84994f7e43e6041b45c17d0ed613d0d6229535bf2", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/SKILL.md": "37d9ae55ab4d0afd4ff8053c1955445e5a090188841ecc7941a5e22c5dd5eb39", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/dispatch-and-measurement.md": "360719d4253ad7e5a36823171d432aa8ca0f74d1ae565c7803af1990d99f2af9", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/record-schema.md": "f3cefb8b131613535dcacf30d130ffc2dea991acfb495adb4e1c7050046903d7", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/speculation-and-pipeline.md": "64faced7f31d489e9b0c4b9523a803ff1a76dca0f9315bdc6d0b83935bd338a4", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/troubleshooting.md": "a2e85696120ffc2af4249ac9c1e1b96320e02287e5099577c0474c025ab4f5f6", - "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/scripts/summarize_serving_records.py": "78e660a13a6a6941d006cb11fc2f401a34f489e54273860ae6a208dca6a6a3c2" - }, - "skill_metadata": { - "name": "vllm-metal", - "description": "Investigate vllm-metal Apple inference through platform and native-ABI selection, paged KV/offload ownership, speculative/deferred state and actual dispatch/serving evidence without upgrading source or CPU checks to backend qualification.", - "license": "NO_LICENSE", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "partial_cpu_only", - "tags": [ - "vllm-metal", - "apple-silicon", - "mlx", - "mps", - "paged-kv", - "kv-offload", - "speculation", - "deferred-execution", - "evidence-boundaries" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/vllm-metal", - "commit": "e7df7a6ff215f99c7842552d230fdbc709835934", - "version": "0.32.0", - "license": "Apache-2.0 (retained source); dependency/model assets separate", - "readset": "references/source-identity.json", - "paths": [ - "LICENSE", - "README.md", - "docs/CONTRIBUTING.md", - "docs/benchmarking-macos.md", - "docs/configuration.md", - "docs/gqa-decode.md", - "docs/installation.md", - "docs/kv_offloading.md", - "docs/pytorch-mps.md", - "docs/speculative_decoding.md", - "pyproject.toml", - "scripts/lint.sh", - "scripts/test.sh", - "setup.py", - "tests/conftest.py", - "tests/test_config.py", - "tests/test_envs.py", - "tests/test_kv_offload_budget.py", - "tests/test_kv_offload_tiering.py", - "tests/test_kv_offload_worker.py", - "tests/test_macos_defaults.py", - "tools/avg_gen_length.py", - "tools/native_sampling_probe.py", - "vllm_metal/__init__.py", - "vllm_metal/config.py", - "vllm_metal/envs.py", - "vllm_metal/metal/build.py", - "vllm_metal/metal/paged_attention.cpp", - "vllm_metal/platform.py", - "vllm_metal/v1/cache_policy.py", - "vllm_metal/v1/decode_pipeline.py", - "vllm_metal/v1/kv_offload/config.py", - "vllm_metal/v1/kv_offload/connector.py", - "vllm_metal/v1/kv_offload/fs_tier.py", - "vllm_metal/v1/kv_offload/shared_region.py", - "vllm_metal/v1/kv_offload/spec.py", - "vllm_metal/v1/kv_offload/worker.py", - "vllm_metal/v1/model_lifecycle.py" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Compatible-agent portable source/Python partial adaptation, not DisCo CLI/model execution or managed import." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Stable-ID manual author sequencing plus bounded cache author lane; integrated graph followed by assertion/helper/static and selected pure Python source tests; Apple native blocks retained." - }, - "applicability": { - "scope": "Selected plugin preparation, KV/offload ownership and speculative/deferred/dispatch evidence investigation. Common catalog scope does not certify CPU inference or cross-backend qualification.", - "prerequisites": "Access to the plugin-specific platform/ABI, cache-lifecycle and qualification contracts; actual engine changes require separate authority, resource ownership, frozen gates and cleanup, with a supported artifact pairing.", - "input_contract": "Source/package/runtime/native artifact identities, Python/OS/chip/device, backend/executor, checkpoint/tokenizer/draft identities, format/dtype, effective feature flags and workload.", - "produced_artifact": "An identity/route/invariant report with actual safe-check result, counter-signal and next authorized native check; partial CPU evidence stays separate from unverified transfers, sleep/wake, model and HTTP outcomes.", - "failure_modes": "Using lazy import success as native registration or ABI evidence, treating a feature switch as dispatch, or conflating cache capacity with physical residency and kernel time with HTTP time can overstate plugin behavior." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: actual Apple MLX/Metal and MPS kernels/model loading, cache transfers/sleep/wake and real worker dispatch are not verified.", - "Installed pure API inspection and selected parser/default tests do not establish a healthy inference dependency set: pip check failed missing requirements, a FastAPI mismatch and unsupported host platform.", - "No model payload authentication, numerical/model quality, physical memory fit, HTTP service/performance or broad skill efficacy validation.", - "Detailed GGUF/AWQ/quantization/MoE, DFlash/DSpark, multimodal, diffusion, STT/pooling/LoRA and distributed paths remain bounded route evidence or explicit long-tail gaps.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback, not legal absence; upstream Apache-2.0 LICENSE retained independently." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve platform registration, backend/runner and native ABI pairing before capacity tuning; a package visibility check establishes only its reported software facts.", - "Keep MLX and MPS, full-attention and recurrent state, logical cache capacity and physical residency as separate contracts.", - "Preserve failed paths and distinguish parser/API checks from real transfers, sleep/wake, model output and HTTP performance evidence." - ] - }, - "applicability": { - "scope": "Selected plugin preparation, KV/offload ownership and speculative/deferred/dispatch evidence investigation. Common catalog scope does not certify CPU inference or cross-backend qualification.", - "prerequisites": "Access to the plugin-specific platform/ABI, cache-lifecycle and qualification contracts; actual engine changes require separate authority, resource ownership, frozen gates and cleanup, with a supported artifact pairing.", - "input_contract": "Source/package/runtime/native artifact identities, Python/OS/chip/device, backend/executor, checkpoint/tokenizer/draft identities, format/dtype, effective feature flags and workload.", - "produced_artifact": "An identity/route/invariant report with actual safe-check result, counter-signal and next authorized native check; partial CPU evidence stays separate from unverified transfers, sleep/wake, model and HTTP outcomes.", - "failure_modes": "Using lazy import success as native registration or ABI evidence, treating a feature switch as dispatch, or conflating cache capacity with physical residency and kernel time with HTTP time can overstate plugin behavior." - }, - "recommendations": [ - "Resolve platform registration, backend/runner and native ABI pairing before capacity tuning; a package visibility check establishes only its reported software facts.", - "Keep MLX and MPS, full-attention and recurrent state, logical cache capacity and physical residency as separate contracts.", - "Preserve failed paths and distinguish parser/API checks from real transfers, sleep/wake, model output and HTTP performance evidence." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "BLOCKED_REQUIRED_BACKEND: actual Apple MLX/Metal and MPS kernels/model loading, cache transfers/sleep/wake and real worker dispatch are not verified.", - "Installed pure API inspection and selected parser/default tests do not establish a healthy inference dependency set: pip check failed missing requirements, a FastAPI mismatch and unsupported host platform.", - "No model payload authentication, numerical/model quality, physical memory fit, HTTP service/performance or broad skill efficacy validation.", - "Detailed GGUF/AWQ/quantization/MoE, DFlash/DSpark, multimodal, diffusion, STT/pooling/LoRA and distributed paths remain bounded route evidence or explicit long-tail gaps.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback, not legal absence; upstream Apache-2.0 LICENSE retained independently." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/weight-loading-optimization.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/weight-loading-optimization.json deleted file mode 100644 index 4121bb9f8..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/weight-loading-optimization.json +++ /dev/null @@ -1,243 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.common.skills.weight_loading_optimization", - "title": "weight-loading-optimization", - "repo": "inferrsi", - "tier": "contribution", - "backend": "common", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Optimize the serving-level weight load path from storage to device: judge the load path on four dimensions (source, format, placement, duplication), account startup fixed overhead separately from per-request cost with bytes-and-counts lines and hit-criteria warmup validation, verify sharded weight downloads against the safetensors index, apply device-side weight prefetch only on profiled memory-bound hot spots, and order CPU-tier offload candidates by change surface. Includes source-scoped disk-loading modes, pinned staging, synchronous checkpoint save semantics and fastsafetensors broadcast/scatter/coalesced delivery; also audit streamed expert-cache byte correctness and begin/end overlap contracts. Use when startup/loading latency dominates or a streamed expert-cache change needs correctness verification; route DCP asynchronous save performance and distributed recovery validity to their separate skills.\n", - "tags": [ - "storage", - "weight-loading", - "startup", - "prefetch", - "offload", - "serving", - "npu", - "collectives", - "tensor-parallel", - "Flare", - "model-preparation", - "prepared-runtime-package", - "expert-cache", - "loader-correctness" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/methods/weight-loading-optimization/SKILL.md" - }, - "checkout_ref": "checkout:weight-loading-optimization", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md": "b970ef27da2de14c20d3fe734cc5ee0cf8445c396f53dab588bd180943921833", - "src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/metadata.json": "007142544806e5b065b8c48b6e467f49c65b4a55465d09d7a88e0997287bb212", - "src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-oracles.md": "5e42c4a01f9ac810663979dee0658c4cbe87060594c16622ae6ec0b854e44b29", - "src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-source.json": "65bf0623616b4f633d8f51b30134798cbf71176716c8a8613209e2fc48ff84bd", - "src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/fastsafetensors-transfer-readset.md": "5a39f14b102b0cee304541e011aa31e44e6b13527d1bee9987c3f3b66c064d3d", - "src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/flare-preparation-boundary.md": "1d7922785a196bdbafdc9791fe79428465a5acc0292ca8f99446f8c764b35454", - "src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/llama-loading-readset.md": "6c640f28ee3f0253a17b81461c865cb9f947367002cb75bf72550c4572d02e97" - }, - "skill_metadata": { - "name": "weight-loading-optimization", - "description": "Optimize the serving-level weight load path from storage to device: judge the load path on four dimensions (source, format, placement, duplication), account startup fixed overhead separately from per-request cost with bytes-and-counts lines and hit-criteria warmup validation, verify sharded weight downloads against the safetensors index, apply device-side weight prefetch only on profiled memory-bound hot spots, and order CPU-tier offload candidates by change surface. Includes source-scoped disk-loading modes, pinned staging, synchronous checkpoint save semantics and fastsafetensors broadcast/scatter/coalesced delivery; also audit streamed expert-cache byte correctness and begin/end overlap contracts. Use when startup/loading latency dominates or a streamed expert-cache change needs correctness verification; route DCP asynchronous save performance and distributed recovery validity to their separate skills.\n", - "tags": [ - "storage", - "weight-loading", - "startup", - "prefetch", - "offload", - "serving", - "npu", - "collectives", - "tensor-parallel", - "Flare", - "model-preparation", - "prepared-runtime-package", - "expert-cache", - "loader-correctness" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ggml-org/llama.cpp", - "commit": "3e390af18de1131dda27913c00511725b4274c8a", - "paths": [ - "common/arg.cpp", - "common/common.cpp", - "common/fit.cpp", - "src/llama-model.cpp", - "src/llama-model-loader.cpp", - "src/llama-model-loader.h", - "src/llama-mmap.cpp", - "src/llama-mmap.h", - "LICENSE" - ], - "readset": "references/llama-loading-readset.md" - }, - { - "type": "git", - "url": "https://github.com/foundation-model-stack/fastsafetensors", - "commit": "219e2561a41ede2d5d8eba611c248bb0effbf580", - "paths": [ - "fastsafetensors/loader.py", - "fastsafetensors/file_buffer.py", - "fastsafetensors/tensor_factory.py", - "fastsafetensors/parallel_loader.py", - "fastsafetensors/frameworks/_torch.py", - "fastsafetensors/frameworks/_paddle.py", - "docs/overview.md", - "examples/run_parallel.py", - "tests/unit/test_broadcast_runs.py", - "tests/unit/test_subgroup.py", - "LICENSE" - ], - "readset": "references/fastsafetensors-transfer-readset.md" - }, - { - "type": "git", - "url": "https://github.com/NVIDIA/flare-local-inference", - "resolved_url": "https://github.com/NVIDIA/local-inference", - "commit": "bce032c014bae34502ba22d3304fa084d5b40afe", - "paths": [ - "README.md", - "docs/runtime-package-design.md", - "helios/src/runtime_package/format.rs", - "helios/src/runtime_package/model.rs", - "helios/src/execution/manifest.rs", - "helios/src/model/contract.rs", - "helios/src/model/catalog.rs", - "LICENSE" - ], - "readset": "references/flare-preparation-boundary.md", - "license": "Apache-2.0" - }, - { - "type": "git", - "url": "https://github.com/ronak-create/deepseek-v4-in-c", - "commit": "d0402056d0da4bf7edfd2d4977de479a0db9d201", - "paths": [ - "LICENSE", - "README.md", - "Makefile", - "src/cache/dsv4_cache.c", - "src/cache/dsv4_cache.h", - "src/io/dsv4_st.c", - "src/io/dsv4_st.h", - "src/model/dsv4_layer.h", - "src/model/dsv4_layer.c", - "src/cli/dsv4_run.c", - "tests/unit/test_cache.c", - "tests/unit/test_model_oracle.c", - "tools/make_st_fixture.py", - "tools/make_tiny_checkpoint.py", - "tools/dsv4_ref.py" - ], - "readset": "references/deepseek-expert-cache-oracles.md", - "license": "Apache-2.0 (repository declaration and selected-file notices; third-party/model rights excluded)" - } - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Diagnosing the serving-level weight load path from storage to device and choosing changes to it when startup or loading latency dominates: locating the cost by source, format, placement and duplication, accounting once-per-process loading and warmup apart from per-request cost, verifying sharded downloads, restricting device-side prefetch to profiled memory-bound hot spots and ordering CPU-tier offload; also covers source-scoped disk-loading modes, pinned staging, synchronous checkpoint save semantics, fastsafetensors broadcast/scatter/coalesced delivery and the separation of model preparation from serving. Request-time host overhead belongs to host-overhead-profiling, DCP asynchronous save performance to dcp-async-save-performance and distributed recovery validity to checkpoint-recovery-consistency; for disk-streamed routed experts, audit independent byte identity, resident/pending slot ownership, ordered accumulation and actual oracle callback coverage; the DeepSeek source slice describes POSIX storage, C/OpenMP and pthread ownership; target validation must identify its actual direct/buffered and reader-pool paths", - "prerequisites": "Flag names and API entry paths are re-verified against the pinned source revisions, and torch_npu.npu_prefetch details against the target CANN version, before they are relied on; card state is checked before launch and process placement fixed before measuring, through ascend-device-health and os-process-placement; the Diffusers enable_model_cpu_offload path requires accelerate", - "input_contract": "The current load path's byte source, file format, host-first or direct device placement and duplicated copies across processes or instances; startup timings for loading and warmup with byte and access counts per segment, and the expected warmup hit criteria; the safetensors index weight_map and the downloaded shard set; a profile showing memory-bound hot spots and the adjacent compute or communication segments; free device memory and whether an OOM occurs during construction or inference; for cross-rank delivery, the file-to-loader-group-rank map, tensor names, shapes, dtypes and call order, the required replicated or sliced output, the model's expected shard layout and the collective backend; for streamed expert-cache changes, the selected direct/buffered I/O and reader-pool path, tensor offsets and cache-slot budget, requested keys, source callback table, independent byte/numerical reference and checked fixture coverage", - "produced_artifact": "The located load-path cost on the four dimensions with its matching lever, a startup ledger that keeps fixed overhead apart from per-request cost, a structural shard-completeness result naming the missing shards, a prefetch proposal behind an enable_prefetch switch defaulting to False or a recorded decision not to prefetch, an offload candidate order, the recorded loading mode, a distributed delivery choice with its proposed verification, and a preparation/serving record of serialized and rebuilt state, live resource owners and ABI/prepared-contract versions; proposed changes and verification targets, not a measured startup or delivery gain; a source-bound expert-cache byte/state audit and oracle-path coverage record, with synthetic-fixture, fallback and SKIP limits", - "failure_modes": [ - "Loading and warmup are amortized into per-request latency, or a host segment without timing points is treated as costing nothing", - "Warmup is accepted on total time while mismatched warmup shapes have warmed a different set of cache entries", - "In-flight .incomplete shards are re-downloaded as corrupt, or gap-free shard numbering is accepted without matching the index weight_map", - "Prefetch is forced without a profiled memory-bound hot spot or a non-memory-bound dependency window", - "Resolution or CFG tier-downs are applied to a construction-phase OOM, or offload is adopted when compute already hides the transfer", - "A free-memory probe runs on an mmap-backed load, or a run whose first mlock failure latched with only a warning is treated as fully resident", - "Disk counters that include cache and pipes are read as physical writes, or a run without a page-cache drop is reported as a cold start", - "The ceil-block scatter partition is used without checking it against the model's expected shard layout, or ranks run different orders or plans so their collectives mismatch", - "A faster H2D copy, local clone or rank-local filtered read is counted as a cross-rank communication gain", - "Serial and concurrent cache paths agree on identically corrupted placement bytes, or a contiguous fixture misses a separated-run alignment failure", - "Resident bytes or ordered per-k accumulation change while misses are pending, or an exit-zero SKIP/non-overlapped oracle is counted as overlap validation" - ], - "refusal_point": "Withholds a load-path win until the verification checklist holds and keeps the distributed-delivery checks as verification targets rather than results; does not force prefetch without a safe dependency window and disables it when neither a smaller size nor a moved position removes a regression; records unsupported save behaviors as unknown; rejects mismatched execution ABI or prepared-contract versions instead of editing their tags; and implies no Ascend support from the llama.cpp or fastsafetensors mechanisms; withholds expert-cache correctness/overlap conclusions when byte coverage, actual I/O/pool dispatch or model-oracle callback reachability is missing", - "hardware": "NPU for the device-side prefetch API (torch_npu.npu_prefetch, checked against the target CANN version) and the Diffusers offload playbook; the prefetch and fastsafetensors methodology transfers while the fastsafetensors and llama.cpp APIs do not, and cross-rank completion follows each collective backend's own wait contract" - }, - "recommendations": [ - "Report preparation/startup/readiness, warmups and measured work as separate numbers, with load latency split into init, header, io and get phases, the cold-cache protocol stated and the effective loading mode identical across compared runs.", - "Place each prefetch behind an enable_prefetch switch defaulting to False, start max_size at 50% of the theoretical size, verify a single operator first and step up to 0.7 and then 1.0 only after verified wins.", - "Try offload candidates in order of change surface: batch size, resolution, CFG/feature flags, CPU or layered offload, tensor-parallel degree, then activation checkpointing; re-estimate the un-overlapped transfer share after adopting each tier.", - "Record whole-set atomic publication, asynchronous checkpointing, multi-rank save coordination and crash durability as unknown for the fastsafetensors saver instead of assuming them.", - "Before adopting coalesced delivery, compare it disabled and enabled with identical checkpoint, rank mapping, backend, topology, dtype, cache state, chunk/queue budgets and consumer retention, then measure complete startup under the same accounting protocol.", - "For streamed expert-cache changes, compare against independent buffered tensor bytes, exercise separated-run geometry, check resident bytes during pending reads, preserve ordered per-k accumulation and verify that the numerical oracle selects the actual callbacks." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Normalized provenance covers the existing llama.cpp loading slice at 3e390af18de1131dda27913c00511725b4274c8a and the fastsafetensors distributed-delivery slice at 219e2561a41ede2d5d8eba611c248bb0effbf580; other inherited claims remain unnormalized with original identities and limits retained.", - "The readsets record static mechanisms and source recommendations, not validated startup, memory, placement or backend performance.", - "The fastsafetensors delivery readset does not revalidate inherited I/O/save claims or qualify uneven scatter, accelerator/NPU compatibility, inter-node transport or agent efficacy.", - "Flare adds source-only preparation/serving ownership and ABI/contract checks; live exporter handles, native backend execution, startup gains and model/dependency rights are not qualified.", - "The DeepSeek increment records source/test definitions only: its buffered byte oracle shares tensor metadata, its resident snapshot covers w1 bytes, and its tiny PyTorch/batched checks do not qualify the released model, backend performance or agent efficacy." - ], - "provenance_readsets": [ - "references/llama-loading-readset.md", - "references/fastsafetensors-transfer-readset.md", - "references/flare-preparation-boundary.md", - "references/deepseek-expert-cache-oracles.md", - "references/deepseek-expert-cache-source.json" - ], - "consumption_mode": "packaged" - }, - "applicability": { - "scope": "Diagnosing the serving-level weight load path from storage to device and choosing changes to it when startup or loading latency dominates: locating the cost by source, format, placement and duplication, accounting once-per-process loading and warmup apart from per-request cost, verifying sharded downloads, restricting device-side prefetch to profiled memory-bound hot spots and ordering CPU-tier offload; also covers source-scoped disk-loading modes, pinned staging, synchronous checkpoint save semantics, fastsafetensors broadcast/scatter/coalesced delivery and the separation of model preparation from serving. Request-time host overhead belongs to host-overhead-profiling, DCP asynchronous save performance to dcp-async-save-performance and distributed recovery validity to checkpoint-recovery-consistency; for disk-streamed routed experts, audit independent byte identity, resident/pending slot ownership, ordered accumulation and actual oracle callback coverage; the DeepSeek source slice describes POSIX storage, C/OpenMP and pthread ownership; target validation must identify its actual direct/buffered and reader-pool paths", - "prerequisites": "Flag names and API entry paths are re-verified against the pinned source revisions, and torch_npu.npu_prefetch details against the target CANN version, before they are relied on; card state is checked before launch and process placement fixed before measuring, through ascend-device-health and os-process-placement; the Diffusers enable_model_cpu_offload path requires accelerate", - "input_contract": "The current load path's byte source, file format, host-first or direct device placement and duplicated copies across processes or instances; startup timings for loading and warmup with byte and access counts per segment, and the expected warmup hit criteria; the safetensors index weight_map and the downloaded shard set; a profile showing memory-bound hot spots and the adjacent compute or communication segments; free device memory and whether an OOM occurs during construction or inference; for cross-rank delivery, the file-to-loader-group-rank map, tensor names, shapes, dtypes and call order, the required replicated or sliced output, the model's expected shard layout and the collective backend; for streamed expert-cache changes, the selected direct/buffered I/O and reader-pool path, tensor offsets and cache-slot budget, requested keys, source callback table, independent byte/numerical reference and checked fixture coverage", - "produced_artifact": "The located load-path cost on the four dimensions with its matching lever, a startup ledger that keeps fixed overhead apart from per-request cost, a structural shard-completeness result naming the missing shards, a prefetch proposal behind an enable_prefetch switch defaulting to False or a recorded decision not to prefetch, an offload candidate order, the recorded loading mode, a distributed delivery choice with its proposed verification, and a preparation/serving record of serialized and rebuilt state, live resource owners and ABI/prepared-contract versions; proposed changes and verification targets, not a measured startup or delivery gain; a source-bound expert-cache byte/state audit and oracle-path coverage record, with synthetic-fixture, fallback and SKIP limits", - "failure_modes": [ - "Loading and warmup are amortized into per-request latency, or a host segment without timing points is treated as costing nothing", - "Warmup is accepted on total time while mismatched warmup shapes have warmed a different set of cache entries", - "In-flight .incomplete shards are re-downloaded as corrupt, or gap-free shard numbering is accepted without matching the index weight_map", - "Prefetch is forced without a profiled memory-bound hot spot or a non-memory-bound dependency window", - "Resolution or CFG tier-downs are applied to a construction-phase OOM, or offload is adopted when compute already hides the transfer", - "A free-memory probe runs on an mmap-backed load, or a run whose first mlock failure latched with only a warning is treated as fully resident", - "Disk counters that include cache and pipes are read as physical writes, or a run without a page-cache drop is reported as a cold start", - "The ceil-block scatter partition is used without checking it against the model's expected shard layout, or ranks run different orders or plans so their collectives mismatch", - "A faster H2D copy, local clone or rank-local filtered read is counted as a cross-rank communication gain", - "Serial and concurrent cache paths agree on identically corrupted placement bytes, or a contiguous fixture misses a separated-run alignment failure", - "Resident bytes or ordered per-k accumulation change while misses are pending, or an exit-zero SKIP/non-overlapped oracle is counted as overlap validation" - ], - "refusal_point": "Withholds a load-path win until the verification checklist holds and keeps the distributed-delivery checks as verification targets rather than results; does not force prefetch without a safe dependency window and disables it when neither a smaller size nor a moved position removes a regression; records unsupported save behaviors as unknown; rejects mismatched execution ABI or prepared-contract versions instead of editing their tags; and implies no Ascend support from the llama.cpp or fastsafetensors mechanisms; withholds expert-cache correctness/overlap conclusions when byte coverage, actual I/O/pool dispatch or model-oracle callback reachability is missing", - "hardware": "NPU for the device-side prefetch API (torch_npu.npu_prefetch, checked against the target CANN version) and the Diffusers offload playbook; the prefetch and fastsafetensors methodology transfers while the fastsafetensors and llama.cpp APIs do not, and cross-rank completion follows each collective backend's own wait contract" - }, - "recommendations": [ - "Report preparation/startup/readiness, warmups and measured work as separate numbers, with load latency split into init, header, io and get phases, the cold-cache protocol stated and the effective loading mode identical across compared runs.", - "Place each prefetch behind an enable_prefetch switch defaulting to False, start max_size at 50% of the theoretical size, verify a single operator first and step up to 0.7 and then 1.0 only after verified wins.", - "Try offload candidates in order of change surface: batch size, resolution, CFG/feature flags, CPU or layered offload, tensor-parallel degree, then activation checkpointing; re-estimate the un-overlapped transfer share after adopting each tier.", - "Record whole-set atomic publication, asynchronous checkpointing, multi-rank save coordination and crash durability as unknown for the fastsafetensors saver instead of assuming them.", - "Before adopting coalesced delivery, compare it disabled and enabled with identical checkpoint, rank mapping, backend, topology, dtype, cache state, chunk/queue budgets and consumer retention, then measure complete startup under the same accounting protocol.", - "For streamed expert-cache changes, compare against independent buffered tensor bytes, exercise separated-run geometry, check resident bytes during pending reads, preserve ordered per-k accumulation and verify that the numerical oracle selects the actual callbacks." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Normalized provenance covers the existing llama.cpp loading slice at 3e390af18de1131dda27913c00511725b4274c8a and the fastsafetensors distributed-delivery slice at 219e2561a41ede2d5d8eba611c248bb0effbf580; other inherited claims remain unnormalized with original identities and limits retained.", - "The readsets record static mechanisms and source recommendations, not validated startup, memory, placement or backend performance.", - "The fastsafetensors delivery readset does not revalidate inherited I/O/save claims or qualify uneven scatter, accelerator/NPU compatibility, inter-node transport or agent efficacy.", - "Flare adds source-only preparation/serving ownership and ABI/contract checks; live exporter handles, native backend execution, startup gains and model/dependency rights are not qualified.", - "The DeepSeek increment records source/test definitions only: its buffered byte oracle shares tensor metadata, its resident snapshot covers w1 bytes, and its tiny PyTorch/batched checks do not qualify the released model, backend performance or agent efficacy." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/audit-afd-p2p-exchanges.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/audit-afd-p2p-exchanges.json deleted file mode 100644 index 1d95a6d38..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/audit-afd-p2p-exchanges.json +++ /dev/null @@ -1,132 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.cuda.skills.audit_afd_p2p_exchanges", - "title": "audit-afd-p2p-exchanges", - "repo": "inferrsi", - "tier": "contribution", - "backend": "cuda", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit CUDA P2pNccl Attention/FFN rank namespaces and per-stage, per-peer hidden-state ownership when uneven worker counts or a mismatched return slice make an AFD exchange ambiguous.", - "tags": [ - "afd-plugin", - "cuda", - "p2p", - "rank-mapping", - "hidden-states", - "ownership" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md" - }, - "checkout_ref": "checkout:audit-afd-p2p-exchanges", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md": "a46110891eca860915e314d6921a22856d631453b22b17837fe30262d6780535", - "src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/metadata.json": "ae8c7582265dd6902cc4172293fc709244cc915df9390672fde25687b9e79b4b", - "src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/afd-plugin-LICENSE.md": "0e8566ea362276878e1f38f97da49964dbea301c15d4aa9d727f97a4c39fd943", - "src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md": "5073d87450cb60d1b245308dff8807e73749c07de976d50f2fa26694fad81a72" - }, - "skill_metadata": { - "name": "audit-afd-p2p-exchanges", - "description": "Audit CUDA P2pNccl Attention/FFN rank namespaces and per-stage, per-peer hidden-state ownership when uneven worker counts or a mismatched return slice make an AFD exchange ambiguous.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "Apache-2.0", - "tags": [ - "afd-plugin", - "cuda", - "p2p", - "rank-mapping", - "hidden-states", - "ownership" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/afd-plugin", - "commit": "df8d6a0c403c3ca1b4ed9cf1687e7204908bebec", - "paths": [ - "README.md", - "LICENSE", - "afd_plugin/config.py", - "afd_plugin/distributed/topology.py", - "afd_plugin/connectors/factory.py", - "afd_plugin/connectors/metadata.py", - "afd_plugin/connectors/gpu/p2p.py", - "afd_plugin/v1/worker/attention_metadata.py", - "afd_plugin/v1/worker/attention_model_runner_v2.py", - "afd_plugin/v1/worker/ffn_worker.py", - "afd_plugin/v1/worker/ffn_model_runner.py", - "afd_plugin/model_executor/models/deepseek_v2.py", - "afd_plugin/model_executor/models/forward_context.py", - "tests/unit/distributed/test_topology_partition.py" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Source-based CUDA P2pNccl rank/payload audit; selected eager dense-FFN path with compute_gate_on_attention=false. Repository identity afd_plugin; runtime/backend compatibility unverified.", - "prerequisites": "Access to the pinned source ledger and caller-level evidence for the selected eager dense FFN path with Attention-side gating disabled; any proposed cross-stream reuse needs an established communicator completion contract.", - "input_contract": "Plugin and dependency revisions, connector and role, A/F worker counts, DP/PCP/TP coordinates, worker/device ranks, subgroup roster, stage, per-peer token counts, payload shape/dtype/device, transfer context and returned slices.", - "produced_artifact": "A bounded rank-namespace and per-peer payload ownership ledger, the earliest evidenced mismatch or missing edge, and a proposed verification plan.", - "failure_modes": "Inferring data peers from control senders, accepting equal total row counts despite a peer permutation, or treating an even-split fallback or Python return as proof of ownership/completion can produce a wrong exchange diagnosis.", - "refusal_point": "Stop on A < F, invalid role ranks, duplicate/missing assignments or an unresolved rank namespace; return the conflicting callers and ledger entries without inventing ranks or attributing the fault to networking." - }, - "limitations": [ - "Source-derived candidate; native exchange, model quality, backend compatibility, performance and agent efficacy not_run.", - "Selected Python readset is Apache-2.0; the retained MIT notice concerns excluded NPU async CAM implementation and adapted bindings.", - "No qualification of graph/DBO, remote-expert MoE, multi-host transport, complete token-count projection or underlying communicator completion semantics." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "references": [ - "../../vllm/audit-vllm-kv-lifetimes/SKILL.md", - "../../../methods/inference-benchmark-design/SKILL.md", - "../../../methods/dense-performance-evidence/SKILL.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Record role, world and subgroup-local ranks separately; reconstruct the actual subgroup roster before interpreting send/receive peers.", - "Track distinct row identities and stage-specific peer lengths through concatenation and splitting; use same-total swapped lengths as a countercheck.", - "Establish the last communicator/stream completion obligation before reusing or releasing a receive buffer." - ] - }, - "applicability": { - "scope": "Source-based CUDA P2pNccl rank/payload audit; selected eager dense-FFN path with compute_gate_on_attention=false. Repository identity afd_plugin; runtime/backend compatibility unverified.", - "prerequisites": "Access to the pinned source ledger and caller-level evidence for the selected eager dense FFN path with Attention-side gating disabled; any proposed cross-stream reuse needs an established communicator completion contract.", - "input_contract": "Plugin and dependency revisions, connector and role, A/F worker counts, DP/PCP/TP coordinates, worker/device ranks, subgroup roster, stage, per-peer token counts, payload shape/dtype/device, transfer context and returned slices.", - "produced_artifact": "A bounded rank-namespace and per-peer payload ownership ledger, the earliest evidenced mismatch or missing edge, and a proposed verification plan.", - "failure_modes": "Inferring data peers from control senders, accepting equal total row counts despite a peer permutation, or treating an even-split fallback or Python return as proof of ownership/completion can produce a wrong exchange diagnosis.", - "refusal_point": "Stop on A < F, invalid role ranks, duplicate/missing assignments or an unresolved rank namespace; return the conflicting callers and ledger entries without inventing ranks or attributing the fault to networking." - }, - "recommendations": [ - "Record role, world and subgroup-local ranks separately; reconstruct the actual subgroup roster before interpreting send/receive peers.", - "Track distinct row identities and stage-specific peer lengths through concatenation and splitting; use same-total swapped lengths as a countercheck.", - "Establish the last communicator/stream completion obligation before reusing or releasing a receive buffer." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; native exchange, model quality, backend compatibility, performance and agent efficacy not_run.", - "Selected Python readset is Apache-2.0; the retained MIT notice concerns excluded NPU async CAM implementation and adapted bindings.", - "No qualification of graph/DBO, remote-expert MoE, multi-host transport, complete token-count projection or underlying communicator completion semantics." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/audit-flare-session-lifecycle.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/audit-flare-session-lifecycle.json deleted file mode 100644 index 39b0e9717..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/audit-flare-session-lifecycle.json +++ /dev/null @@ -1,163 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.cuda.skills.audit_flare_session_lifecycle", - "title": "audit-flare-session-lifecycle", - "repo": "inferrsi", - "tier": "contribution", - "backend": "cuda", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Audit Flare complete prefix/session restoration, inbound transfer tickets, publication/use/retirement fences and host-restore admission outcomes. Use when a lookup hit does not become reused tokens, a cancelled waiter leaves transfers in flight, or a restarted session must recover KV pages together with recurrent, convolution and partial-tail state.", - "tags": [ - "Flare", - "flare-local-inference", - "local-inference", - "cuda", - "session-state", - "kv-cache", - "recurrent-state", - "convolution", - "restore-admission", - "transfer-ticket", - "fences", - "offload" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md" - }, - "checkout_ref": "checkout:audit-flare-session-lifecycle", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", - "src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md": "02a5f7c18e44201215ecec17c6ac96b27be9148bcc189df287f39d1b372b7721", - "src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/metadata.json": "9c667c7735ebe4fb07247e098218f6b03a1c1fafa776f1372f8d948ea8ea756b", - "src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md": "0908a0eae53a2e10775a8b6ee3e3944b2fa410174402b5a1a8c98eb674dd4b4d", - "src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/lifecycle.md": "107ebe3c22e5dba8007c3b8ea61149f10bc55c51b557a4230f3e5de859bb0f7c", - "src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-ledger.json": "61e51a97ffd7c1fa7866bdd440b024d0c5e0065635ac8d0ba53d2c69c7f9e470", - "src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md": "fc7ceb819b91ef1731d8b7783f543c92ef6bb5c97ef5f897c055824f8a560168" - }, - "skill_metadata": { - "name": "audit-flare-session-lifecycle", - "description": "Audit Flare complete prefix/session restoration, inbound transfer tickets, publication/use/retirement fences and host-restore admission outcomes. Use when a lookup hit does not become reused tokens, a cancelled waiter leaves transfers in flight, or a restarted session must recover KV pages together with recurrent, convolution and partial-tail state.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "Apache-2.0", - "tags": [ - "Flare", - "flare-local-inference", - "local-inference", - "cuda", - "session-state", - "kv-cache", - "recurrent-state", - "convolution", - "restore-admission", - "transfer-ticket", - "fences", - "offload" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/NVIDIA/flare-local-inference", - "resolved_url": "https://github.com/NVIDIA/local-inference", - "commit": "bce032c014bae34502ba22d3304fa084d5b40afe", - "paths": [ - "README.md", - "LICENSE", - "docs/runtime-package-design.md", - "docs/prefix-cache-offload.md", - "helios/src/runtime_package/format.rs", - "helios/src/runtime_package/model.rs", - "helios/src/execution/manifest.rs", - "helios/src/model/contract.rs", - "helios/src/model/catalog.rs", - "helios/src/cache.rs", - "helios/src/offload/mod.rs", - "helios/src/offload/geometry.rs", - "helios/src/offload/index.rs", - "helios/src/offload/record.rs", - "helios/src/offload/transfer.rs", - "helios/src/scheduler.rs", - "helios/src/server/runtime.rs", - "tests/server/test_prefix_cache_offload.py", - "qualify/performance.py", - "qualify/provenance.py" - ], - "readset": "references/source-notes.md", - "identity": "references/source-ledger.json", - "license": "Apache-2.0" - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "task-agnostic", - "strategy": "reuse-existing", - "status": "source-derived-candidate", - "method": "AREX scope/ground/construct/verify and repository evidence/usability methods adapted to InferRSI native owners and packaging; no DisCo runtime or managed import." - }, - "applicability": { - "scope": "Pinned Flare complete-session restore and admission audit for its NVIDIA SM120/SM121 research prototype. CUDA catalog scope is content classification, not device or backend qualification.", - "prerequisites": "Access to package/source identity and bounded restore traces, with preparation compatibility established before cache-state diagnosis. Running, freeing resources or changing configuration requires separate authority.", - "input_contract": "Execution ABI and prepared-contract version, ordinary/speculative configuration, model/layout identity, request IDs and consumed-token endpoints, GPU/host prefixes, continuation row, ticket/waiter identities, transfer events, fences and resource counts.", - "produced_artifact": "A checkpoint/ticket ownership table, five-outcome admission ledger and diagnosis identifying the earliest unsupported transition and smallest missing observation.", - "failure_modes": "A KV-only lookup hit can omit continuation state; admission can be mistaken for actual reuse; client cancellation can be mistaken for transfer completion; mixing absolute timestamps from client and native clocks can corrupt the diagnosis.", - "refusal_point": "Stop with an unresolved edge when package identity, layout, transfer completion or clock provenance is unknown." - }, - "limitations": [ - "Source-only candidate; native Flare/model/GPU/service and skill efficacy were not evaluated.", - "GDS/Direct transport is deferred and explicitly rejected by the pinned implementation.", - "Multi-rank source branches do not establish multi-GPU, multi-host or hardware failure qualification.", - "Root Apache-2.0 attribution does not grant model, asset or separately licensed dependency rights.", - "Only the selected symbols/sections were inspected; kernels, exporter implementation and whole-repository behavior are outside coverage." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/source-ledger.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Pair the immutable KV parent chain with the complete continuation row at the same consumed-token endpoint before calling a checkpoint restorable.", - "Distinguish lookup outcomes, destination reservation, ticket creation, transfer completion, publication and actual reused-token evidence.", - "Follow scheduler-owned tickets through waiter cancellation and check write/use fences before allowing storage reuse." - ] - }, - "applicability": { - "scope": "Pinned Flare complete-session restore and admission audit for its NVIDIA SM120/SM121 research prototype. CUDA catalog scope is content classification, not device or backend qualification.", - "prerequisites": "Access to package/source identity and bounded restore traces, with preparation compatibility established before cache-state diagnosis. Running, freeing resources or changing configuration requires separate authority.", - "input_contract": "Execution ABI and prepared-contract version, ordinary/speculative configuration, model/layout identity, request IDs and consumed-token endpoints, GPU/host prefixes, continuation row, ticket/waiter identities, transfer events, fences and resource counts.", - "produced_artifact": "A checkpoint/ticket ownership table, five-outcome admission ledger and diagnosis identifying the earliest unsupported transition and smallest missing observation.", - "failure_modes": "A KV-only lookup hit can omit continuation state; admission can be mistaken for actual reuse; client cancellation can be mistaken for transfer completion; mixing absolute timestamps from client and native clocks can corrupt the diagnosis.", - "refusal_point": "Stop with an unresolved edge when package identity, layout, transfer completion or clock provenance is unknown." - }, - "recommendations": [ - "Pair the immutable KV parent chain with the complete continuation row at the same consumed-token endpoint before calling a checkpoint restorable.", - "Distinguish lookup outcomes, destination reservation, ticket creation, transfer completion, publication and actual reused-token evidence.", - "Follow scheduler-owned tickets through waiter cancellation and check write/use fences before allowing storage reuse." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-only candidate; native Flare/model/GPU/service and skill efficacy were not evaluated.", - "GDS/Direct transport is deferred and explicitly rejected by the pinned implementation.", - "Multi-rank source branches do not establish multi-GPU, multi-host or hardware failure qualification.", - "Root Apache-2.0 attribution does not grant model, asset or separately licensed dependency rights.", - "Only the selected symbols/sections were inspected; kernels, exporter implementation and whole-repository behavior are outside coverage." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/diagnose-attention-bottlenecks.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/diagnose-attention-bottlenecks.json deleted file mode 100644 index 209a59552..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/diagnose-attention-bottlenecks.json +++ /dev/null @@ -1,97 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.cuda.skills.diagnose_attention_bottlenecks", - "title": "diagnose-attention-bottlenecks", - "repo": "inferrsi", - "tier": "contribution", - "backend": "cuda", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Diagnose the next attention-kernel bottleneck after IO reduction using evidence about HBM traffic, work ownership, parallelism, scalar work, asynchronous scheduling and precision. Produce a bounded optimization hypothesis and correctness-first verification plan; do not infer current library support or NPU portability from GPU papers.", - "tags": [], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md" - }, - "checkout_ref": "checkout:diagnose-attention-bottlenecks", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md": "a25738dbb3b9daa15c5827cb536a20fe13f7c8b9dd14aa3fb7cfa7e132879431", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/metadata.json": "6239c885aa83936a7bd9c49643b1c79b3734a17234af7c03d95839efc622c79e", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/mechanisms.md": "fe08b2292fa587be3874774ddf41063b84c579a0085df422ec6710d8aa90b8b7", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/INTEGRATION_NOTES.md": "371b01ee201ef51fecbfd6da892d517aaba295930b9395025f03f34e21c030a1", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/NEXT_GATE_LEADS.md": "35b8b58ae5db02b2df51ccbe5cdf870a7383d17f345f9ee5eb7b0a4146dfc74b", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/README.md": "e0b1ec1b39c10067f5ddce58320851ad089568c4b2296a24777389097cb5b388", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/io-reduction-can-plateau.md": "5cc35f514224ac6e6efd2da6db221bba468b2f4db66807295e67b240bcdfdba5", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md": "80c8a60a6df55f3d84fab23e03567d8b38e23bcdbd63cc09881e46db1e2888ab", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/partitioning-after-fusion.md": "4d4cabe0991eab554dcfbd4fa9c8199e61fa4767709b0c4cd646e2588bb398a2", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/knowledge/attention-cost-model-boundaries.md": "5ed8f888443e38ccb72545b6025447aab7b92936f78c151fed680e582126239b", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/sources.md": "53436e0de4e6dda2d29426591bf61f8526378cdc6f9ec7967b88b1bcd4c61e6f" - }, - "skill_metadata": { - "name": "diagnose-attention-bottlenecks", - "description": "Diagnose the next attention-kernel bottleneck after IO reduction using evidence about HBM traffic, work ownership, parallelism, scalar work, asynchronous scheduling and precision. Produce a bounded optimization hypothesis and correctness-first verification plan; do not infer current library support or NPU portability from GPU papers.", - "status": "candidate", - "empirical_validation": "not_run", - "source_pins": "2205.14135v2;2307.08691v1;2407.08608v2", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/mechanisms.md", - "references/papers-20261015/cases/io-reduction-can-plateau.md", - "references/papers-20261015/cases/overlap-and-fp8-counter-signals.md", - "references/papers-20261015/cases/partitioning-after-fusion.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/attention-cost-model-boundaries.md", - "references/papers-20261015/NEXT_GATE_LEADS.md", - "references/papers-20261015/README.md", - "references/sources.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Diagnose attention-kernel bottlenecks after IO reduction, tile/pipeline regressions and precision or hardware-port assumptions.", - "input_contract": "Actual kernel/device revisions, attention shapes and semantics, operand/accumulator types, correctness/quality criteria, and existing traffic, resource, parallelism and stall observations.", - "produced_artifact": "A bottleneck diagnosis with confidence, one bounded candidate, resource exchange, counter-signal, missing evidence, source ID and next authorization gate.", - "refusal_point": "Return conditional hypotheses when observations are missing; reject or defer changes with wrong outputs/gradients, unsafe lifetimes/order, unsupported primitives, unacceptable accuracy or gains erased by omitted work." - }, - "recommendations": [ - "Preserve masks, dropout replay, online-softmax normalization and gradient obligations before comparing costs; exact algebra does not imply bitwise equality.", - "After fusion, reclassify the exposed limit using scalar work, independent output tiles and resource ceilings; larger tiles and deeper pipelines can reduce residency or spill.", - "Include precision preprocessing/conversion in the measurement boundary and check generated asynchronous scheduling; validate accelerator-specific primitives separately before a port." - ] - }, - "applicability": { - "scope": "Diagnose attention-kernel bottlenecks after IO reduction, tile/pipeline regressions and precision or hardware-port assumptions.", - "input_contract": "Actual kernel/device revisions, attention shapes and semantics, operand/accumulator types, correctness/quality criteria, and existing traffic, resource, parallelism and stall observations.", - "produced_artifact": "A bottleneck diagnosis with confidence, one bounded candidate, resource exchange, counter-signal, missing evidence, source ID and next authorization gate.", - "refusal_point": "Return conditional hypotheses when observations are missing; reject or defer changes with wrong outputs/gradients, unsafe lifetimes/order, unsupported primitives, unacceptable accuracy or gains erased by omitted work." - }, - "recommendations": [ - "Preserve masks, dropout replay, online-softmax normalization and gradient obligations before comparing costs; exact algebra does not imply bitwise equality.", - "After fusion, reclassify the exposed limit using scalar work, independent output tiles and resource ceilings; larger tiles and deeper pipelines can reduce residency or spill.", - "Include precision preprocessing/conversion in the measurement boundary and check generated asynchronous scheduling; validate accelerator-specific primitives separately before a port." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/host-overhead-profiling.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/host-overhead-profiling.json deleted file mode 100644 index 015a1669b..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/host-overhead-profiling.json +++ /dev/null @@ -1,92 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.cuda.skills.host_overhead_profiling", - "title": "host-overhead-profiling", - "repo": "inferrsi", - "tier": "contribution", - "backend": "cuda", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Detect and diagnose CPU/host overhead in inference serving from nsys traces. Answers: is host overhead the bottleneck? What specific work is causing it? Covers GPU idle ratio, host-prep exposed ratio, NVTX breakdown, inter-kernel gap analysis, and multi-rank straggler detection.\n", - "tags": [ - "host", - "cpu", - "profiling", - "nsys", - "inference" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md" - }, - "checkout_ref": "checkout:host-overhead-profiling", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md": "326e566fbe30d3ba04e4b50c3e87aad9df037f0ac4ad6fe129165ae97dc5f91d", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/metadata.json": "bc4236c65e37378f4a74e0b88c47776055c186aca44c000b1f68e8c63cc4b277" - }, - "skill_metadata": { - "name": "host-overhead-profiling", - "description": "Detect and diagnose CPU/host overhead in inference serving from nsys traces. Answers: is host overhead the bottleneck? What specific work is causing it? Covers GPU idle ratio, host-prep exposed ratio, NVTX breakdown, inter-kernel gap analysis, and multi-rank straggler detection.\n", - "tags": [ - "host", - "cpu", - "profiling", - "nsys", - "inference" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Detect exposed host overhead and diagnose per-step host regressions in CUDA inference serving from existing nsys timelines.", - "prerequisites": "An existing serving nsys trace and Python with sqlite3; comparative root-cause analysis needs baseline and target traces, with nsys export available for report-format input.", - "input_contract": "One trace for detection/breakdown or two comparable traces for regression analysis, including the workload/phase context needed to isolate steady-state forward steps.", - "produced_artifact": "A detection verdict with metric/phase evidence, per-step NVTX/kernel/API breakdown and ranked host-operation deltas for a targeted optimization handoff.", - "failure_modes": "Warmup/loading or mismatched phase mixtures distort per-step comparisons; duplicated rank ranges, unresolved NVTX text IDs or overlapping rank end times can miscompute host gaps." - }, - "recommendations": [ - "Isolate comparable steady-state forward iterations and inspect context/generation separately; aggregate utilization can hide a phase-specific host bottleneck.", - "Confirm exposed preparation using both performance impact and idle attribution, retain communication as a confounder, and triage absolute NVTX time deltas before line-level optimization.", - "Resolve NVTX text IDs, account for TP rank duplication and use the maximum overlapping end time for gaps; do not compare full-trace totals as per-step overhead." - ] - }, - "applicability": { - "scope": "Detect exposed host overhead and diagnose per-step host regressions in CUDA inference serving from existing nsys timelines.", - "prerequisites": "An existing serving nsys trace and Python with sqlite3; comparative root-cause analysis needs baseline and target traces, with nsys export available for report-format input.", - "input_contract": "One trace for detection/breakdown or two comparable traces for regression analysis, including the workload/phase context needed to isolate steady-state forward steps.", - "produced_artifact": "A detection verdict with metric/phase evidence, per-step NVTX/kernel/API breakdown and ranked host-operation deltas for a targeted optimization handoff.", - "failure_modes": "Warmup/loading or mismatched phase mixtures distort per-step comparisons; duplicated rank ranges, unresolved NVTX text IDs or overlapping rank end times can miscompute host gaps." - }, - "recommendations": [ - "Isolate comparable steady-state forward iterations and inspect context/generation separately; aggregate utilization can hide a phase-specific host bottleneck.", - "Confirm exposed preparation using both performance impact and idle attribution, retain communication as a confounder, and triage absolute NVTX time deltas before line-level optimization.", - "Resolve NVTX text IDs, account for TP rank duplication and use the maximum overlapping end time for gaps; do not compare full-trace totals as per-step overhead." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/reason-hopper-kernels.json b/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/reason-hopper-kernels.json deleted file mode 100644 index d21b61156..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/cuda/skills/reason-hopper-kernels.json +++ /dev/null @@ -1,283 +0,0 @@ -{ - "schema_version": 2, - "id": "inferrsi.cuda.skills.reason_hopper_kernels", - "title": "reason-hopper-kernels", - "repo": "inferrsi", - "tier": "contribution", - "backend": "cuda", - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": "Reason about Hopper SM90 CUDA GEMM kernels using TMA and mbarrier lifetimes, WGMMA warpgroup participation, register and pipeline budgets, layouts, software scheduling and FP8 scaling. Use for an H100/H200-family kernel diagnosis or a Blackwell-to-Hopper applicability check; source-derived candidate guidance with no local GPU validation.", - "tags": [ - "cuda", - "Hopper", - "SM90", - "SM90a", - "H100", - "H200", - "TMA", - "mbarrier", - "WGMMA", - "warp-specialization", - "registers", - "pipeline", - "swizzle", - "persistent-kernels", - "FP8", - "DeepGEMM", - "CUTLASS", - "KernelWiki", - "KDA" - ], - "origin": [ - "external" - ], - "index_type": "packaged-skill", - "skill_documents": { - "original": "data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md" - }, - "checkout_ref": "checkout:reason-hopper-kernels", - "source_files": { - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md": "cec4e3aa1eb64c31c54b5f5ca40af2b61b7919d94be4174968581d5f752c0538", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/metadata.json": "79ca64dc97ad34331036dc071e86954c98ac568e651adfae74cce74f1ca61c7b", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/fp8.md": "31d9c211c11c2224007abbdcd4059913c4a54833fe2d2f926411c5238fcd3501", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/cutlass-bsd.md": "42fec630f410aa308f70a51a89fadcd19586fa620f9831a32bee528a9a10000e", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/deepgemm-mit.md": "6c73a30c14a2938abaa6afe98ebb19e6d81dac5bd4ace0cb05c61c20afa37b85", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/kernelwiki-kda-mit.md": "84aab57158a914f448b38d8be29dd842b634a08c192d844f416a256beed4ae94", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/notices.md": "876ec016644bf5331b3f257985e126c0e61980e3bfd82569b7b0c3f6b2c6c64c", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/resources-and-scheduling.md": "1ce25b426736072435a1ead9a2a8e1f56df5d0abb8198e8c4fdcb34321db4964", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/source-inventory.json": "1dc09c895c970a0c9708158bcc6dd65cdb736274fd2c35d44542c70407d3849d", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/sources.md": "6b74b9e774f2a80db0860510d9e6940d36284430f29c1e124f115a9eaf25833d", - "src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/synchronization.md": "4322f6b3c0e4368bf8c192b4ec752529767f50d19c0c38dad8fdf9cd781e41d9" - }, - "skill_metadata": { - "name": "reason-hopper-kernels", - "description": "Reason about Hopper SM90 CUDA GEMM kernels using TMA and mbarrier lifetimes, WGMMA warpgroup participation, register and pipeline budgets, layouts, software scheduling and FP8 scaling. Use for an H100/H200-family kernel diagnosis or a Blackwell-to-Hopper applicability check; source-derived candidate guidance with no local GPU validation.", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "UNKNOWN", - "tags": [ - "cuda", - "Hopper", - "SM90", - "SM90a", - "H100", - "H200", - "TMA", - "mbarrier", - "WGMMA", - "warp-specialization", - "registers", - "pipeline", - "swizzle", - "persistent-kernels", - "FP8", - "DeepGEMM", - "CUTLASS", - "KernelWiki", - "KDA" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/NVlabs/kda", - "commit": "acdd79343c5e9d0c7cb96289f26fd3f1051ce785", - "paths": [ - ".gitmodules", - "THIRD_PARTY_NOTICES.md", - "LICENSE", - "third_party_licenses/MIT-hanlab.txt", - "skills/KernelWiki" - ], - "readset": "references/sources.md", - "identity": "references/source-inventory.json", - "scope": "KDA-to-KernelWiki gitlink and licensing provenance only; first-party runtime code not mined." - }, - { - "type": "git", - "url": "https://github.com/mit-han-lab/KernelWiki", - "commit": "76d27b56f804e7e7295d4c570e1e5d7eef4b0a75", - "paths": [ - "README.md", - "SKILL.md", - "artifacts/kernels/deepgemm/full/PROVENANCE.yaml", - "artifacts/prs/cutlass/PR-2139/PROVENANCE.yaml", - "artifacts/prs/cutlass/PR-2750/key-files/examples/python/CuTeDSL/blackwell/tutorial_gemm/fp16_gemm_1.py", - "artifacts/prs/cutlass/PR-3091/PROVENANCE.yaml", - "artifacts/prs/cutlass/PR-3091/key-files/examples/python/CuTeDSL/hopper/grouped_gemm.py", - "artifacts/prs/cutlass/PR-3091/key-files/python/CuTeDSL/cutlass/utils/tensormap_manager.py", - "data/refresh-cutoff.yaml", - "data/tool-versions.yaml", - "sources/blogs/deepgemm.md", - "sources/prs/cutlass/PR-2139.md", - "sources/prs/cutlass/PR-2965.md", - "sources/prs/sglang/PR-16014.md", - "sources/prs/sglang/PR-19794.md", - "sources/prs/sglang/PR-9559.md", - "sources/prs/vllm/PR-20396.md", - "sources/prs/vllm/PR-23666.md", - "sources/prs/vllm/PR-33285.md", - "wiki/hardware/mbarrier.md", - "wiki/hardware/tma.md", - "wiki/kernels/deepgemm.md", - "wiki/kernels/fp8-block-scale-gemm.md", - "wiki/kernels/grouped-gemm.md", - "wiki/languages/cuda-cpp.md", - "wiki/migration/register-to-tmem.md", - "wiki/migration/wgmma-to-tcgen05.md", - "wiki/patterns/compute-bound.md", - "wiki/patterns/low-sm-utilization.md", - "wiki/patterns/memory-bound.md", - "wiki/patterns/pipeline-stalls.md", - "wiki/patterns/register-pressure.md", - "wiki/patterns/tail-effect.md", - "wiki/techniques/cache-policy.md", - "wiki/techniques/double-buffering.md", - "wiki/techniques/fine-grained-quantization.md", - "wiki/techniques/persistent-kernels.md", - "wiki/techniques/ping-pong-scheduling.md", - "wiki/techniques/pipeline-stages.md", - "wiki/techniques/register-budgeting.md", - "wiki/techniques/swizzling.md", - "wiki/techniques/tile-scheduling.md", - "wiki/techniques/vectorized-loads.md", - "wiki/techniques/warp-specialization.md" - ], - "readset": "references/sources.md", - "identity": "references/source-inventory.json", - "license": "UNKNOWN", - "scope": "All synthesis and source frontmatters screened; selected Hopper bodies mined with explicit qualifications. MIT assertion inherited from KDA; no root LICENSE at this pin." - }, - { - "type": "git", - "url": "https://github.com/deepseek-ai/DeepGEMM", - "commit": "891d57b4db1071624b5c8fa0d1e51cb317fa709f", - "paths": [ - "LICENSE", - "deep_gemm/include/deep_gemm/impls/sm90_fp8_gemm_1d1d.cuh", - "deep_gemm/include/deep_gemm/mma/sm90.cuh", - "deep_gemm/include/deep_gemm/ptx/wgmma.cuh", - "deep_gemm/include/deep_gemm/scheduler/gemm.cuh" - ], - "readset": "references/sources.md", - "identity": "references/source-inventory.json", - "license": "MIT", - "scope": "Selected code sections directly inspected; original bytes compared with KernelWiki where recorded; no native build or execution." - }, - { - "type": "git", - "url": "https://github.com/NVIDIA/cutlass", - "commit": "ca4fdbea708ad940c905359788372b8add9f85e0", - "paths": [ - "LICENSE.txt", - "examples/67_hopper_fp8_warp_specialized_gemm_with_blockwise_scaling/67_hopper_fp8_warp_specialized_gemm_with_blockwise_scaling.cu", - "include/cutlass/gemm/collective/sm90_mma_tma_gmma_ss_warpspecialized_fp8_blockwise_scaling.hpp", - "include/cutlass/gemm/kernel/sm90_gemm_array_tma_warpspecialized_pingpong.hpp" - ], - "readset": "references/sources.md", - "identity": "references/source-inventory.json", - "license": "BSD-3-Clause", - "scope": "Selected code sections directly inspected; original bytes compared with KernelWiki where recorded; no native build or execution." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/12.8.0/hopper-tuning-guide/index.html", - "version": "CUDA 12.8.0", - "content_sha256": "0fb0a5dd532f43e04c41112aa07bb3fddfab03fc8e68392509f0bec2c03ca418", - "readset": "references/source-inventory.json", - "scope": "Selected architecture/ISA sections directly inspected; source HTML not packaged." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/12.8.0/parallel-thread-execution/index.html", - "version": "CUDA 12.8.0 / PTX ISA 8.7", - "content_sha256": "48849aa7851b9ff8893efc750f1a0db46e6023852cb76d001d68d57ffa49fa06", - "readset": "references/source-inventory.json", - "scope": "Selected architecture/ISA sections directly inspected; source HTML not packaged." - } - ], - "applicability": { - "scope": "Source-derived Hopper SM90/SM90a kernel reasoning for TMA/mbarrier, WGMMA, resource/layout/pipeline/software scheduling and selected FP8 GEMM. CUDA categorization does not certify other GPU architectures, library dispatch, or a combined tested stack.", - "architectures": [ - "sm90", - "sm90a" - ], - "excluded_mechanisms": [ - "tcgen05", - "TMEM", - "CLC", - "two-SM MMA", - "native NVFP4/block-scaled tcgen05 MMA" - ], - "input_contract": "Device/compute capability, compiler target and selected implementation, kernel/library revisions, shapes/strides/alignment, numeric and grouping/epilogue semantics, correctness criterion and measurement boundary.", - "produced_artifact": "A source-qualified, reviewable hypothesis with one bounded change, resource/accuracy exchange, counter-signal, stop condition and next verification step.", - "failure_modes": "SM90 tags or import/dispatch guards can hide an unbuilt or unselected SM90a instruction path; extra stages or register caps can increase spills or lower residency." - }, - "limitations": [ - "Native compilation, GPU correctness, numerical accuracy, profiling, performance, model/service behavior and skill efficacy were not run.", - "KernelWiki is Blackwell-first; frontmatter tags, verified labels and snippet claims are not Hopper compatibility proofs.", - "Only selected code sections and nine source summary bodies were mined; remaining source headers and synthesis dispositions are explicit in the inventory.", - "Source numerical claims, fixed tuning defaults and unsafe teaching snippets are not imported as local evidence or recipes.", - "Pinned KernelWiki has no root LICENSE; KDA MIT assertion and its sanitized-distribution discrepancy are retained, with third-party terms kept separate." - ], - "provenance_readsets": [ - "references/sources.md", - "references/source-inventory.json", - "references/notices.md" - ], - "references": [ - "../diagnose-attention-bottlenecks/SKILL.md", - "../../../methods/kernel-writing-guide/SKILL.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "For WGMMA advice, confirm the compiled and selected SM90a path; do not transplant Blackwell-only instruction or memory mechanisms.", - "Resolve producer/consumer ownership and completion before tuning stage depth; budget live accumulators, padded shared memory, scales and epilogue storage together.", - "For later validation, check tails, short K, stage wraparound and grouped offsets against the frozen reference, inspect actual PTX/SASS resources, and include setup/conversion/reduction costs." - ] - }, - "applicability": { - "scope": "Source-derived Hopper SM90/SM90a kernel reasoning for TMA/mbarrier, WGMMA, resource/layout/pipeline/software scheduling and selected FP8 GEMM. CUDA categorization does not certify other GPU architectures, library dispatch, or a combined tested stack.", - "architectures": [ - "sm90", - "sm90a" - ], - "excluded_mechanisms": [ - "tcgen05", - "TMEM", - "CLC", - "two-SM MMA", - "native NVFP4/block-scaled tcgen05 MMA" - ], - "input_contract": "Device/compute capability, compiler target and selected implementation, kernel/library revisions, shapes/strides/alignment, numeric and grouping/epilogue semantics, correctness criterion and measurement boundary.", - "produced_artifact": "A source-qualified, reviewable hypothesis with one bounded change, resource/accuracy exchange, counter-signal, stop condition and next verification step.", - "failure_modes": "SM90 tags or import/dispatch guards can hide an unbuilt or unselected SM90a instruction path; extra stages or register caps can increase spills or lower residency." - }, - "recommendations": [ - "For WGMMA advice, confirm the compiled and selected SM90a path; do not transplant Blackwell-only instruction or memory mechanisms.", - "Resolve producer/consumer ownership and completion before tuning stage depth; budget live accumulators, padded shared memory, scales and epilogue storage together.", - "For later validation, check tails, short K, stage wraparound and grouped offsets against the frozen reference, inspect actual PTX/SASS resources, and include setup/conversion/reduction costs." - ], - "limitations": [ - "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; its declared status is candidate.", - "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", - "Native compilation, GPU correctness, numerical accuracy, profiling, performance, model/service behavior and skill efficacy were not run.", - "KernelWiki is Blackwell-first; frontmatter tags, verified labels and snippet claims are not Hopper compatibility proofs.", - "Only selected code sections and nine source summary bodies were mined; remaining source headers and synthesis dispositions are explicit in the inventory.", - "Source numerical claims, fixed tuning defaults and unsafe teaching snippets are not imported as local evidence or recipes.", - "Pinned KernelWiki has no root LICENSE; KDA MIT assertion and its sanitized-distribution discrepancy are retained, with third-party terms kept separate." - ], - "verification": { - "locally_verified": false, - "evidence_level": "guidance" - } -} diff --git a/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/README.md b/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/README.md deleted file mode 100644 index fd8be6bce..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# SGLang knowledge - -Named JSON records are retrieval abstracts with source attribution, conditions, -validation plans and explicit verification state. The complete English skills -provide the detailed procedures and evidence maps; they are not per-entry -Markdown copies of the catalog records. - -Full library: [English skill](../../../skill_library/repositories/sglang/SKILL.md). - -All entries are imported guidance with `locally_verified=false`. `common` means -backend-independent guidance within this repository, not verified backend -support. The pack uses caller-supplied external commands and supplies no model -launcher. GPU/NPU execution, model quality and performance need separate -target-environment evidence. - -## Entries - -| Component | Record | -| --- | --- | -| llm | [SGLang scheduler, RadixCache and HiCache state ownership](common/llm/runtime-radix-hicache.json) | -| llm | [SGLang TP/DP/EP, PD transfer and extension ownership](common/llm/parallelism-and-extension.json) | -| llm | [SGLang controlled tuning and model-quality gates](common/llm/tuning-and-validation.json) | - -## Maintenance - -Keep IDs stable and add a navigation row for every record. Retain pinned source -attribution and limitations; omit PR links. Keep detailed procedures in the -linked full skills, and update abstracts when their source contracts change. -Catalog or CPU tests do not promote imported guidance to validated runtime -evidence. Follow the shared [knowledge maintenance contract](../../vllm_omni/knowledge/README.md#maintaining-the-pack). diff --git a/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/README.md b/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/README.md deleted file mode 100644 index 728b35c29..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# SGLang-Omni knowledge - -Named JSON records are retrieval abstracts with source attribution, conditions, -validation plans and explicit verification state. The complete English skills -provide the detailed procedures and evidence maps; they are not per-entry -Markdown copies of the catalog records. - -Full library: [English skill](../../../skill_library/repositories/sglang-omni/SKILL.md). - -All entries are imported guidance with `locally_verified=false`. `common` means -backend-independent guidance within this repository, not verified backend -support. The pack uses caller-supplied external commands and supplies no model -launcher. GPU/NPU execution, model quality and performance need separate -target-environment evidence. - -## Entries - -| Component | Record | -| --- | --- | -| omni | [SGLang-Omni stage DAG, process replicas and transport lifetime](common/omni/pipeline-state-and-transport.json) | -| audio | [SGLang-Omni serving recipes and streaming audio measurements](common/audio/serving-and-streaming-quality.json) | -| omni | [SGLang-Omni extension ownership and lifecycle validation](common/omni/extension-lifecycle-validation.json) | - -## Maintenance - -Keep IDs stable and add a navigation row for every record. Retain pinned source -attribution and limitations; omit PR links. Keep detailed procedures in the -linked full skills, and update abstracts when their source contracts change. -Catalog or CPU tests do not promote imported guidance to validated runtime -evidence. Follow the shared [knowledge maintenance contract](../../vllm_omni/knowledge/README.md#maintaining-the-pack). diff --git a/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/README.md b/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/README.md deleted file mode 100644 index 3dba46a64..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# Strata knowledge - -Named JSON records are retrieval abstracts with source attribution, conditions, -validation plans and explicit verification state. The complete English skills -provide the detailed procedures and evidence maps; they are not per-entry -Markdown copies of the catalog records. - -Full library: [English skill](../../../skill_library/repositories/strata/SKILL.md). - -All entries are imported guidance with `locally_verified=false`. `common` means -backend-independent guidance within this repository, not verified backend -support. The pack uses caller-supplied external commands and supplies no model -launcher. GPU/NPU execution, model quality and performance need separate -target-environment evidence. - -## Entries - -| Component | Record | -| --- | --- | -| llm | [Strata family contracts and constrained inference search](common/llm/architecture-search-space.json) | -| llm | [Strata expert residency, CPU/PCIe balance and bounded tuning](common/llm/controlled-execution-tuning.json) | -| llm | [Strata engine validation from native builds to request lifecycle](common/llm/engine-validation-contract.json) | - -## Maintenance - -Keep IDs stable and add a navigation row for every record. Retain pinned source -attribution and limitations; omit PR links. Keep detailed procedures in the -linked full skills, and update abstracts when their source contracts change. -Catalog or CPU tests do not promote imported guidance to validated runtime -evidence. Follow the shared [knowledge maintenance contract](../../vllm_omni/knowledge/README.md#maintaining-the-pack). diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/README.md b/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/README.md deleted file mode 100644 index f9d72561c..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# vLLM knowledge - -Named JSON records are retrieval abstracts with source attribution, conditions, -validation plans and explicit verification state. The complete English skills -provide the detailed procedures and evidence maps; they are not per-entry -Markdown copies of the catalog records. - -Full library: [English skill](../../../skill_library/repositories/vllm/SKILL.md). - -All entries are imported guidance with `locally_verified=false`. `common` means -backend-independent guidance within this repository, not verified backend -support. The pack uses caller-supplied external commands and supplies no model -launcher. GPU/NPU execution, model quality and performance need separate -target-environment evidence. - -## Entries - -| Component | Record | -| --- | --- | -| llm | [vLLM V1 scheduling, token budgets and KV ownership](common/llm/runtime-and-kv-ownership.json) | -| llm | [vLLM bounded serving experiments and workload controls](common/llm/bounded-serving-measurement.json) | -| llm | [vLLM extension boundaries and correctness gates](common/llm/extension-correctness-gates.json) | - -## Maintenance - -Keep IDs stable and add a navigation row for every record. Retain pinned source -attribution and limitations; omit PR links. Keep detailed procedures in the -linked full skills, and update abstracts when their source contracts change. -Catalog or CPU tests do not promote imported guidance to validated runtime -evidence. Follow the shared [knowledge maintenance contract](../../vllm_omni/knowledge/README.md#maintaining-the-pack). diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/README.md b/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/README.md deleted file mode 100644 index df67f13eb..000000000 --- a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/README.md +++ /dev/null @@ -1,139 +0,0 @@ -# vLLM-Omni knowledge - -This is the contribution tier of the [three-tier knowledge catalog](../../../knowledge_packs/README.md). -The external dependency and peer tiers have separate source owners; existing -records here keep their IDs, paths and verification state. - -Each named JSON file is the canonical catalog record for one reusable experience -or source-guided topic. This README provides organization and navigation rather -than a second per-entry summary. Existing experience records retain their IDs -and source history. New topic abstracts link to complete English procedures -through `skill_documents`. - -Full library: [English skill](../../../skill_library/repositories/vllm-omni/SKILL.md). - -## Ownership and hierarchy - -```text -data/ - repo_packs// - pack.json Repository capabilities - knowledge/ - README.md This guide and entry links - common//.json Backend-independent methods - cuda//.json CUDA-scoped experience - ascend//.json Ascend-scoped experience -evidence// Frozen plans, artifacts and verdicts -``` - -Only directories with records are created. This pack covers diffusion, -multi-stage orchestration (`omni`) and streaming audio. `common` means common -**within that repository**, not globally validated across repositories. ROCm -currently retrieves common methods only. There are no ROCm-specific -optimization results in this pack. - -| Axis | Stored in | Meaning | -| --- | --- | --- | -| Repository ownership | repo directory and repo field | Prevents mixing repositories | -| Backend scope | backend directory and backend field | common or one concrete backend | -| Component | component directory and component field | Diffusion, omni orchestration or audio | -| Workload | scenario | Exact image/video scenario or a component wildcard | -| Model and prerequisites | applicability and tags | Model, SKU, software, shapes, dtype and workload conditions | -| Origin and maturity | status, sources, verification | Imported guidance/report versus locally supported evidence | - -A model is a field rather than another directory level: a shared method can -apply to several models without copying records. Keep an entry ID stable. -Add a separate record when another backend needs different prerequisites, -failure modes or measurements; do not overwrite the original qualification. - -## Record contract and retrieval - -Every record carries its ID/title/summary, repository/backend/component/scenario, -schema_version, import status, source attribution and explicit verification. -Applicability, claims, recommendations, validation_plan and limitations hold -the technical content. The loader checks path/ID/scope consistency and pinned -source attribution before making a record available. - -The CLI reads JSON directly; it never parses this README or executes imported -instructions. Repository, backend and scenario filters run before keyword -matching and the result limit. A backend query includes that backend plus -common methods. Scenario aliases diffusion.t2i and diffusion.t2v still work. -Hardware/model/software conditions require inspection of applicability; they -are not an automatic eligibility check. - -```bash -inferrsi knowledge search --repo-type vllm_omni --backend ascend --scenario diffusion.t2i -inferrsi knowledge show vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i -inferrsi context --repo-type vllm_omni --backend cuda --scenario diffusion.image_generation -``` - -Human-readable context is rendered from the same JSON used by search/show. -When present, `skill_documents` identifies package-relative paths to the full -English skills, which contain detailed procedures and source maps. -Those skills are the source material for the concise indexed abstract, not a -second per-entry catalog summary. This entry list contains navigation only; -tests check that every packaged record is listed. - -## Entries - -| Backend | Record | -| --- | --- | -| ascend | [MiniMax-H3 reference KV reuse must balance ownership after Ulysses](ascend/diffusion/minimax-h3-reference-kv-reuse.json) | -| ascend | [MiniMax-H3 SP head buckets reduce exposed All-to-All with quality still pending](ascend/diffusion/minimax-h3-sp-head-buckets.json) | -| ascend | [Qwen-Image-2.1 eager text-to-image on one 910B1: reuse plus interleaved RoPE, 30% not met](ascend/diffusion/qwen-image-21-eager-t2i.json) | -| common | [vLLM-Omni streaming audio, deployment and measurement boundaries](common/audio/streaming-serving-measurement.json) | -| common | [Diffusion measurement and profiling discipline](common/diffusion/measurement-and-profiling.json) | -| common | [Production diffusion integration and compatibility gates](common/diffusion/production-integration-gates.json) | -| common | [vLLM-Omni diffusion step state and output/transport validation](common/diffusion/step-state-and-output-validation.json) | -| common | [Upstream review gates for optional acceleration and evidence ownership](common/diffusion/upstream-review-gates.json) | -| common | [vLLM-Omni stage ownership and constrained execution search](common/omni/stage-ownership-and-search.json) | -| cuda | [Audio concurrency scaling: mixed prefill, state ownership and codec overlap](cuda/audio/concurrency-scaling-state-and-overlap.json) | -| cuda | [MiniMax-H3 exact AdaLN schedule cache: capacity benefit can exceed latency benefit](cuda/diffusion/minimax-h3-adaln-schedule-cache.json) | -| cuda | [MiniMax-H3 online CUDA MXFP8 is an opt-in approximate deployment lane](cuda/diffusion/minimax-h3-cuda-mxfp8.json) | -| cuda | [MiniMax-H3 modulation fusion changes rounding and requires platform guards](cuda/diffusion/minimax-h3-fp32-modulation.json) | -| cuda | [MiniMax-H3 guarded Q/K RMSNorm and packed RoPE fusion](cuda/diffusion/minimax-h3-qk-norm-rope.json) | -| cuda | [MiniMax-H3 Ref2VA TeaCache needs task-specific calibration and step defaults](cuda/diffusion/minimax-h3-ref2va-teacache.json) | -| cuda | [MiniMax-H3 stacked VAE tiling trades activation memory for fewer decode calls](cuda/diffusion/minimax-h3-vae-stacked-tiling.json) | - -## Attribution and evidence - -Knowledge records and exported context contain no PR or review-thread links. -Keep repository/revision/date attribution, known authors and immutable -technical-document URLs. A repository URL plus a commit identifies the source -context; it is **not** a link to raw measurement evidence. Review guidance is -attributed to its reviewed source revision and remains guidance. - -All current entries remain status=imported and -verification.locally_verified=false. Evidence levels describe content: -guidance or upstream_reported. Source merge/open status, where retained, is a -historical observation at accessed_at; it is not a confidence level. -Removing links and reorganizing files does not verify any performance claim. - -Reported numbers and counterexamples retain their hardware, workload, quality -and sampling limits. Missing raw outputs or frozen validators remain explicit. -Campaign artifacts belong in evidence/ and run directories; knowledge stores -the reusable lesson and its boundaries. - -## Maintaining the pack - -1. Find an existing entry by topic and scope before adding a new file. Use a - concise slug and match its repo.backend.component.slug_with_underscores ID. -2. Write conditions, the actionable lesson, validation steps, limitations and - negative results in JSON. Attribute known authors; omit PR links. -3. Keep imported claims unverified. Local qualification requires a separate - frozen campaign with exact source/environment/workload and complete quality - and performance artifacts. CPU tests, a merged change or a source report - alone do not promote an entry. -4. When new evidence narrows or contradicts a lesson, update its conditions and - limitations without erasing the negative result or reassigning its ID. - Preserve the historical campaign artifacts. -5. Add a navigation row here, then run the catalog/CLI tests and the - noneditable-package resource check. No manual index of metrics is needed. -6. For a topic backed by the full skill library, retain valid `skill_documents` - paths to English procedures and reconcile the concise abstract when the - detailed source contracts change. Do not replace or silently repin historical records - while adding a newer source snapshot. - -Repository-wide and backend-wide methods stay in their owning packs. External -dependency and peer techniques have explicit source/tier filters in the -three-tier catalog; common is never an implicit global inheritance mechanism. diff --git a/src/inferrsi/knowledge/data/skill_library/README.md b/src/inferrsi/knowledge/data/skill_library/README.md index d39b95311..be0c2b2a0 100644 --- a/src/inferrsi/knowledge/data/skill_library/README.md +++ b/src/inferrsi/knowledge/data/skill_library/README.md @@ -1,26 +1,33 @@ # InferRSI skill library -The English-only library contains **142 skill documents**: 79 registered roots, -21 separately indexed engine/router documents and 42 AREX graph sub-skills. -Complete procedures, metadata, references, cases and source notices ship with the package. - -## Browse by owner - -| Directory | Content | -| --- | --- | -| [Hardware](hardware/README.md) | Ascend (14 registered skills), CUDA (3) and host resource guidance (5), with retained source-depth navigation | -| [Frameworks](frameworks/) | PyTorch and TorchNPU operating graphs, PyTorch distributed checkpointing and MLX evaluation/lifetimes | -| [Repositories](repositories/) | Engine entry skills, focused procedures, references and cases | -| [Methods](methods/) | Optimization, benchmarking, analysis, search and casebooks | -| [Workflows](workflows/) | Research, acceptance, evidence, source checking and skill lifecycle | - -Only this README and [`skill-index.json`](skill-index.json) live at the library -root. Supporting material belongs to its owner's `references/`, `cases/` or +The main InferRSI agent selects one of the [seven primary inference skills](tasks/) +by the requested outcome, runs `inferrsi skills enter ` and reads +the returned procedure before the first supporting evidence search. Use the actual model, device, engine version, script, gates +and diagnosis to decide which supporting skill to load next. The +[compatibility router](tasks/inference-task-router/SKILL.md) retains older route +resource paths. The library has 88 registered bundles and 151 English +`SKILL.md` documents, including nested and unregistered entries. + +| Question | Directory | Choose a second-level category | +| --- | --- | --- | +| Which procedure owns the inference task? | [Tasks](tasks/) | Day-0, performance, debug, accuracy, version alignment, derived engine or independent engine | +| How does this implementation work? | [Systems](systems/) | [Platforms](systems/platforms/README.md), [frameworks](systems/frameworks/) or [engines](systems/engines/) | +| How should I perform this engineering task? | [Playbooks](playbooks/) | [Measurement](playbooks/measurement/), [planning](playbooks/planning/) or [optimization](playbooks/optimization/) | +| How do several steps fit together? | [Workflows](workflows/) | [Research](workflows/research/), [governance](workflows/governance/) or [knowledge maintenance](workflows/maintenance/) | +| What happened the last time this was tried? | [Casebooks](casebooks/) | [Inference experiments](casebooks/inference/) or [Ascend-specific cases](casebooks/ascend/) | + +For example, an Ascend capture/replay procedure belongs in `systems/platforms/ascend/`. +Prior NPU optimization outcomes belong in `casebooks/ascend/`. Cross-platform +benchmark design belongs in `playbooks/measurement/`. Put a bundle in one +canonical location; link to it from other categories by its registered ID. + +The library also owns authored findings under `records/`, search configuration +under `config/` and generated build inputs under `_compiled/`. Supporting material belongs to its owner's `references/`, `cases/` or existing resource directory. ## Read registered skills -[`skill-index.json`](skill-index.json) maps 79 stable skill IDs to their canonical +[`skill-index.json`](skill-index.json) maps 88 stable skill IDs to their canonical paths and backend scopes. Install the optional `skills` extra for these commands: ```bash @@ -29,7 +36,7 @@ inferrsi skills show ascend-graph-mode inferrsi skills resource inference-optimization-casebook cases/qwen-ascend-rejected-directions.md ``` -The [generated catalog](../repo_packs/inferrsi/knowledge/README.md) links each +The runtime catalog links each registered procedure through `skill_documents.original`. Core catalog reads do not require the optional YAML parser: @@ -43,9 +50,9 @@ categorized library with `skill-index.json`. Backend scope does not certify device compatibility; readers do not execute helpers or promote candidates. For Goal-state authority, ordered next-action decisions and local quota receipt -recovery, start with the existing [RSI evidence owner](workflows/rsi-evidence/SKILL.md#goal-authority-and-accounting-decisions) -and its [LoopX source reference](workflows/rsi-evidence/references/loopx-goal-governance/README.md). -The [search-policy route](methods/optimization-search-policy/SKILL.md#goal-admission-and-accounting-source-knowledge) +recovery, start with the existing [RSI evidence owner](workflows/governance/rsi-evidence/SKILL.md#goal-authority-and-accounting-decisions) +and its [LoopX source reference](workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md). +The [search-policy route](playbooks/planning/optimization-search-policy/SKILL.md#goal-admission-and-accounting-source-knowledge) reuses that owner. These are source-derived candidate additions, with runtime and agent efficacy unmeasured; no new skill or registry ID is added. @@ -55,36 +62,35 @@ Choose by the question and the retained source version. These existing candidate | Existing owner | Retained source version | Use for | Existing evidence inventory | Original semantic inspection date | | --- | --- | --- | --- | --- | -| [ascend-model-adaptation](hardware/ascend/ascend-model-adaptation/SKILL.md) | Historical `bc8115ed` documentation pin | Checkpoint onboarding, FP8 loading, KV-head replication and multimodal validation order | The entry cites `.agents/skills/vllm-ascend-model-adapter/references/` lessons, including `fp8-on-npu-lessons.md` and `multimodal-ep-aclgraph-lessons.md`; no bundled owner readset | Not individually recorded in the retained entry; the commit date is not an inspection date | -| [ascend-aclgraph-capture-replay](hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md) | `8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf` runtime pin | Wrapper dispatch, caller-owned stable inputs, replay/update ordering and capture-resource diagnosis | Existing [source notes](hardware/ascend/ascend-aclgraph-capture-replay/references/source-notes.md): wrapper, graph-mode guide and selected test cases | 2026-10-15 supplier inspection, carried unchanged; local consolidation does not repeat it | +| [ascend-model-adaptation](systems/platforms/ascend/ascend-model-adaptation/SKILL.md) | Historical `bc8115ed` documentation pin | Checkpoint onboarding, FP8 loading, KV-head replication and multimodal validation order | The entry cites `.agents/skills/vllm-ascend-model-adapter/references/` lessons, including `fp8-on-npu-lessons.md` and `multimodal-ep-aclgraph-lessons.md`; no bundled owner readset | Not individually recorded in the retained entry; the commit date is not an inspection date | +| [ascend-aclgraph-capture-replay](systems/platforms/ascend/ascend-aclgraph-capture-replay/SKILL.md) | `8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf` runtime pin | Wrapper dispatch, caller-owned stable inputs, replay/update ordering and capture-resource diagnosis | Existing [source notes](systems/platforms/ascend/ascend-aclgraph-capture-replay/references/source-notes.md): wrapper, graph-mode guide and selected test cases | 2026-10-15 supplier inspection, carried unchanged; local consolidation does not repeat it | The historical short pin resolves to [`bc8115ed7fdc7e500320b5de70742ef049b4c1a2`](https://github.com/vllm-project/vllm-ascend/commit/bc8115ed7fdc7e500320b5de70742ef049b4c1a2). That resolves Git identity only; it does not add a semantic reread or an inspection date. Both entries keep their candidate / empirical validation not_run state and separate compatibility limits. -The [Reference Matrix](https://github.com/InferMatrixLab/InferRSI/blob/main/docs/reference-matrix.md) retains the older `bc8115ed` Dockerfile/thread-env shipping-default scope under [os-process-placement](hardware/host/os-process-placement/SKILL.md). ACLGraph capture/replay supplies no new OS-control evidence; see the [OS/System row](https://github.com/InferMatrixLab/InferRSI/blob/main/docs/searchspace-matrix.md#5--os--system). +The [Reference Matrix](https://github.com/InferMatrixLab/InferRSI/blob/main/docs/reference-matrix.md) retains the older `bc8115ed` Dockerfile/thread-env shipping-default scope under [os-process-placement](systems/platforms/host/os-process-placement/SKILL.md). ACLGraph capture/replay supplies no new OS-control evidence; see the [OS/System row](https://github.com/InferMatrixLab/InferRSI/blob/main/docs/searchspace-matrix.md#5--os--system). ## Focused llama.cpp runtime audit -The registered [audit-llama-runtime-state](repositories/llama-cpp/audit-llama-runtime-state/SKILL.md) +The registered [audit-llama-runtime-state](systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md) candidate separates partial decode progress, frontend graph reuse, backend input readiness and server task/slot cancellation. Its pinned source contracts are untested natively; -[loading](methods/weight-loading-optimization/SKILL.md) and -[host placement](hardware/host/os-process-placement/SKILL.md) remain with their -existing owners. The [scope and source map](repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) +[loading](playbooks/optimization/weight-loading-optimization/SKILL.md) and +[host placement](systems/platforms/host/os-process-placement/SKILL.md) remain with their +existing owners. The [scope and source map](systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) links all three responsibilities and their separate source snapshots. ## PyTorch compiler and checkpoint scope routing -The [PyTorch operating graph](frameworks/pytorch/pytorch/SKILL.md) routes eight +The [PyTorch operating graph](systems/frameworks/pytorch/pytorch/SKILL.md) routes eight workflows across installation/build, tensor/autograd, training/data, compiler/export, memory/streams/graphs, distributed/checkpoint, profiling and -custom operators. The [TorchNPU operating graph](frameworks/torch-npu/torch-npu/SKILL.md) +custom operators. The [TorchNPU operating graph](systems/frameworks/torch-npu/torch-npu/SKILL.md) routes eight Ascend adapter workflows, including PrivateUse1 dispatch, operator development, HCCL, precision and explicit compiler selection. Both are source-derived candidates; installed CPU observations do not qualify the pinned source build or the required accelerator and distributed backends. -Their registered IDs are `pytorch` and `torch-npu`. Nested procedures and -sidecars are packaged resources of those roots, rather than separate IDs: +Their registered IDs are `pytorch` and `torch-npu`. Nested SKILL.md files carry their own procedure and metadata and are packaged resources of those roots: ```bash inferrsi skills resource pytorch sub-skills/compiler-and-export/SKILL.md @@ -99,19 +105,19 @@ completion contracts; their shared repository name does not make a tested stack. | Need | Existing entry | Retained evidence | Completion / applicability boundary | | --- | --- | --- | --- | -| Inductor guards, capture shapes, replay addresses or live outputs | [Compiler dependency card](../knowledge_packs/dependency/pytorch/cuda/compiler/cudagraph-shapes-and-lifetimes.json) | Technical documents at `f15b476814ebc5eade36fb40512e379ebf79dd60`, accessed 2026-10-15; upstream-reported, imported/unverified | Compile/recompile guards, graph recording/replay and output lifetime are independent constraints; this is CUDA/compiler guidance | -| Training-visible save pauses, snapshot buffers and cleanup | [DCP async-save procedure](frameworks/pytorch/dcp-async-save-performance/SKILL.md) and [source ledger](frameworks/pytorch/dcp-async-save-performance/references/sources.md) | Implementation at `7d239835e773c91689eb298692097f4a3340491f`, inspected 2026-10-15; candidate / empirical validation `not_run` | Preserve caller capture, staging, upload, cached-buffer reuse and caller-owned cleanup boundaries; writer completion is separate from backend durability and distributed recovery | -| Tutorial examples and API/version differences | [DCP documentation/implementation reconciliation](frameworks/pytorch/dcp-async-save-performance/references/sources.md#documentation-matched-against-implementation) | Live async recipe retrieved 2026-10-15, with its own retained HTML hash and page metadata | The recipe's 2.4+ / 2.9+ labels do not identify the inspected main commit or establish a matching wheel; source notices and recipe attribution remain in the ledger | +| Inductor guards, capture shapes, replay addresses or live outputs | [Compiler dependency card](records/pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes.json) | Technical documents at `f15b476814ebc5eade36fb40512e379ebf79dd60`, accessed 2026-10-15; upstream-reported, imported/unverified | Compile/recompile guards, graph recording/replay and output lifetime are independent constraints; this is CUDA/compiler guidance | +| Training-visible save pauses, snapshot buffers and cleanup | [DCP async-save procedure](systems/frameworks/pytorch/dcp-async-save-performance/SKILL.md) and [source ledger](systems/frameworks/pytorch/dcp-async-save-performance/references/sources.md) | Implementation at `7d239835e773c91689eb298692097f4a3340491f`, inspected 2026-10-15; candidate / empirical validation `not_run` | Preserve caller capture, staging, upload, cached-buffer reuse and caller-owned cleanup boundaries; writer completion is separate from backend durability and distributed recovery | +| Tutorial examples and API/version differences | [DCP documentation/implementation reconciliation](systems/frameworks/pytorch/dcp-async-save-performance/references/sources.md#documentation-matched-against-implementation) | Live async recipe retrieved 2026-10-15, with its own retained HTML hash and page metadata | The recipe's 2.4+ / 2.9+ labels do not identify the inspected main commit or establish a matching wheel; source notices and recipe attribution remain in the ledger | Known mutation and storage lifetimes are a shared diagnostic prerequisite, with different checks: the compiler card calls for retaining outputs across later calls and exercising input mutation; DCP locates the next mutation of checkpointed state -and follows its [completion/lifetime contract](frameworks/pytorch/dcp-async-save-performance/SKILL.md#completion-and-lifetime-contract). +and follows its [completion/lifetime contract](systems/frameworks/pytorch/dcp-async-save-performance/SKILL.md#completion-and-lifetime-contract). Neither owner's completion event establishes the other's safe reuse or eligibility. Keep timing boundaries separate too: compiler checks distinguish cold cost, recompilation, capture and steady-state replay; DCP measures -[caller, staging, upload and step time](frameworks/pytorch/dcp-async-save-performance/SKILL.md#evidence-to-request-in-a-separately-authorized-measurement), +[caller, staging, upload and step time](systems/frameworks/pytorch/dcp-async-save-performance/SKILL.md#evidence-to-request-in-a-separately-authorized-measurement), including first-use versus reused buffers. These are owner-specific checks, not a combined compiler/checkpoint calling protocol or compatibility result. @@ -119,12 +125,12 @@ not a combined compiler/checkpoint calling protocol or compatibility result. TensorRT-LLM's retained methodology row covers **skill documents**, not an engine-runtime audit. It records prefix `f33e0e5fd`; procedures keep -`f33e0e5f`, while the [ceiling sidecar](methods/performance-ceiling-analysis/metadata.json) +`f33e0e5f`, while the [ceiling skill metadata](playbooks/measurement/performance-ceiling-analysis/SKILL.md) retains the longer **prefix** `f33e0e5fd55b`. An original inspection date for that methodology extraction is not individually recorded. These are partial source identities, not a verified common runtime version. -The separate [PD/KV-transfer peer card](../knowledge_packs/peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json) +The separate [PD/KV-transfer peer card](records/tensorrt_llm.cuda.llm.prefill_decode_disaggregation.json) records `c9b18eefa731dc59b0091bdfff6d890aa9d1e98e`, accessed 2026-10-15. Its upstream-reported technique remains imported/unverified. The methods below remain candidate / empirical validation `not_run`, with their other sources @@ -133,20 +139,20 @@ guidance; it establishes no cross-pin implementation compatibility. | Decision / resource | Existing entry | Shared prerequisite or accounting judgment | Keep separate | | --- | --- | --- | --- | -| Evidence and measurement | [Evidence levels](methods/dense-performance-evidence/SKILL.md#claim-evidence-matching), [headroom inputs](methods/performance-ceiling-analysis/SKILL.md#evidence-to-inspect-first) and [case reuse](methods/inference-optimization-casebook/SKILL.md#adapt-do-not-copy) | Apply the peer card's fixed input/output-length and arrival distributions, same total device budget and SLO workload; report TTFT, TPOT, tail latency, goodput and total GPU/network resources | Analytical/component evidence and a precedent do not establish service gain; re-derive model/hardware-specific values | -| Accelerator | [Kernel verification](methods/kernel-writing-guide/SKILL.md#workflow-verify-before-benchmark) | A kernel change still needs reference parity over target shapes/dtypes and critical-path profiling | Kernel correctness does not establish reduced prefill/decode interference or end-to-end PD benefit | -| RAM / Host | [Serving-trace prerequisites](hardware/cuda/host-overhead-profiling/SKILL.md#prerequisites) and [confirmed host bottleneck](hardware/host/host-overhead-optimization/SKILL.md#confirming-the-bottleneck) | Establish exposed host work from serving traces before attributing a PD regression to CPU preparation | A line profiler locates CPU time; it does not establish that CPU is limiting or measure the KV interconnect | -| SSD / Storage | [Startup accounting](methods/weight-loading-optimization/SKILL.md#2-account-startup-fixed-overhead-separately) | Loading/warmup is once per process; request KV delivery needs its own per-request bytes and timing | Do not combine those denominators; disk-loading mechanics retain their separately attributed sources | -| Communication / Network | [Disaggregation peer card](../knowledge_packs/peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json) | Consider PD with measured prefill/decode interference, compatible KV layouts and qualified NVIDIA GPU transport; include transfer, layout-conversion and queue costs, and verify backpressure, cancellation and failed-worker recovery | Host/device loading overlap and local H2D do not establish this interconnect path; diffusion-stage splitting does not inherit the autoregressive PD cost model | -| OS / System | [Placement discipline](hardware/host/os-process-placement/SKILL.md#1-cpu-affinity-check-then-pin-then-keep-constant) | Record and control affinity, environment and device contention across matched pool comparisons | Placement methodology controls an experiment; it does not implement PD routing or OS-priority scheduling | +| Evidence and measurement | [Evidence levels](playbooks/measurement/dense-performance-evidence/SKILL.md#claim-evidence-matching), [headroom inputs](playbooks/measurement/performance-ceiling-analysis/SKILL.md#evidence-to-inspect-first) and [case reuse](casebooks/inference/inference-optimization-casebook/SKILL.md#adapt-do-not-copy) | Apply the peer card's fixed input/output-length and arrival distributions, same total device budget and SLO workload; report TTFT, TPOT, tail latency, goodput and total GPU/network resources | Analytical/component evidence and a precedent do not establish service gain; re-derive model/hardware-specific values | +| Accelerator | [Kernel verification](playbooks/optimization/kernel-writing-guide/SKILL.md#workflow-verify-before-benchmark) | A kernel change still needs reference parity over target shapes/dtypes and critical-path profiling | Kernel correctness does not establish reduced prefill/decode interference or end-to-end PD benefit | +| RAM / Host | [Serving-trace prerequisites](systems/platforms/cuda/host-overhead-profiling/SKILL.md#prerequisites) and [confirmed host bottleneck](systems/platforms/host/host-overhead-optimization/SKILL.md#confirming-the-bottleneck) | Establish exposed host work from serving traces before attributing a PD regression to CPU preparation | A line profiler locates CPU time; it does not establish that CPU is limiting or measure the KV interconnect | +| SSD / Storage | [Startup accounting](playbooks/optimization/weight-loading-optimization/SKILL.md#2-account-startup-fixed-overhead-separately) | Loading/warmup is once per process; request KV delivery needs its own per-request bytes and timing | Do not combine those denominators; disk-loading mechanics retain their separately attributed sources | +| Communication / Network | [Disaggregation peer card](records/tensorrt_llm.cuda.llm.prefill_decode_disaggregation.json) | Consider PD with measured prefill/decode interference, compatible KV layouts and qualified NVIDIA GPU transport; include transfer, layout-conversion and queue costs, and verify backpressure, cancellation and failed-worker recovery | Host/device loading overlap and local H2D do not establish this interconnect path; diffusion-stage splitting does not inherit the autoregressive PD cost model | +| OS / System | [Placement discipline](systems/platforms/host/os-process-placement/SKILL.md#1-cpu-affinity-check-then-pin-then-keep-constant) | Record and control affinity, environment and device contention across matched pool comparisons | Placement methodology controls an experiment; it does not implement PD routing or OS-priority scheduling | ## vLLM version and responsibility routing -The existing [vLLM responsibility map](repositories/vllm/SKILL.md#choose-the-source-and-responsibility) +The existing [vLLM responsibility map](systems/engines/vllm/SKILL.md#choose-the-source-and-responsibility) separates the broad engine guide at `2521446ac8c92937b5a32b7dc2a84ef7fa8e99c5` (inspection 2026-10-14) from the focused -[admission](repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md) and -[KV lifetime](repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md) procedures at +[admission](systems/engines/vllm/diagnose-vllm-admission-pressure/SKILL.md) and +[KV lifetime](systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md) procedures at `7f730b4cea9a66520a9200265928ebe016b48a39` (supplier inspection 2026-10-15). Their original readsets, source notices and candidate/unverified states remain; this routing adds no semantic reread or compatible-stack result. Paper-era @@ -156,35 +162,35 @@ scheduling/placement proposals remain separate from both implementation pins. Five repository entry skills, 15 focused sub-skills and one research router provide source-grounded procedures. These 21 documents are available through -the links and catalog records below, separately from the 79-ID registry. +the links and catalog records below, separately from the 88-ID registry. The registered ComfyUI and stable-diffusion.cpp audits are listed alongside these imports and use generated abstracts rather than runtime repository packs. | Repository | Focus | Entry | Catalog | | --- | --- | --- | --- | -| Lithos Metal source guide | Task compiler/fusion, GPU-resident generation/DSpark state and exact serving/cache ownership; candidate | [Skill](repositories/lithos-metal/SKILL.md) | [Record](../repo_packs/inferrsi/knowledge/common/skills/lithos-metal.json) | -| ComfyUI source audit | Workflow-output cache validity and model-weight residency; source-derived candidate | [Skill](repositories/comfy-ui/SKILL.md) | [Record](../repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json) | -| stable-diffusion.cpp source audit | Native image weight/runtime capacity and protected segment prefetch; source-derived candidate | [Skill](repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | [Record](../repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json) | -| h3.c / MiniMax-H3 source audit | Conditioning, AdaLN precompute, phased/cached residency, precision and BF16 SSD streaming; source-derived candidate | [Skill](repositories/h3-c/audit-h3-runtime-residency/SKILL.md) | [Record](../repo_packs/inferrsi/knowledge/common/skills/audit-h3-runtime-residency.json) | -| h3-metal / FastH3 semantic audit | Effective parameters and ambient overrides, tensor/schedule transformations, VSA prefix/partial tiles and fallback; source-derived candidate | [Skill](repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) | [Record](../repo_packs/inferrsi/knowledge/common/skills/audit-h3-metal-fasth3-semantics.json) | -| AFD plugin P2pNccl audit | CUDA eager dense-FFN rank namespaces and per-stage/per-peer payload ownership; source-derived candidate | [Skill](repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) | `inferrsi.cuda.skills.audit_afd_p2p_exchanges` | -| vLLM-RLT source audit | Synchronous early-exit KV propagation, later deeper attention and normal storage retirement; source-derived candidate | [Skill](repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) | [Record](../repo_packs/inferrsi/knowledge/common/skills/audit-recurrent-kv-finalization.json) | -| Strata | Heterogeneous MoE, placement, memory and speculative execution | [Skill](repositories/strata/SKILL.md) | [Records](../repo_packs/strata/knowledge/README.md) | -| vLLM | Token scheduling, KV ownership and parallel execution | [Skill](repositories/vllm/SKILL.md) | [Records](../repo_packs/vllm/knowledge/README.md) | -| SGLang | Radix/HiCache, prefill/decode, extensions and a separately pinned [diffusion lifetime candidate](repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | [Skill](repositories/sglang/SKILL.md) | [Records](../repo_packs/sglang/knowledge/README.md) | -| SGLang-Omni | Stage pipelines, streaming and transport | [Skill](repositories/sglang-omni/SKILL.md) | [Records](../repo_packs/sglang_omni/knowledge/README.md) | -| vLLM-Omni | Model/deploy separation, AR/diffusion and output contracts | [Skill](repositories/vllm-omni/SKILL.md) | [Records](../repo_packs/vllm_omni/knowledge/README.md) | -| Research router | Constrained engine design and validation | [Skill](workflows/inference-engine-research/SKILL.md) | — | +| Lithos Metal source guide | Task compiler/fusion, GPU-resident generation/DSpark state and exact serving/cache ownership; candidate | [Skill](systems/engines/lithos-metal/SKILL.md) | `inferrsi.common.skills.lithos_metal` | +| ComfyUI source audit | Workflow-output cache validity and model-weight residency; source-derived candidate | [Skill](systems/engines/comfy-ui/SKILL.md) | `inferrsi.common.skills.comfy_ui` | +| stable-diffusion.cpp source audit | Native image weight/runtime capacity and protected segment prefetch; source-derived candidate | [Skill](systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | `inferrsi.common.skills.audit_sd_cpp_residency` | +| h3.c / MiniMax-H3 source audit | Conditioning, AdaLN precompute, phased/cached residency, precision and BF16 SSD streaming; source-derived candidate | [Skill](systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md) | `inferrsi.common.skills.audit_h3_runtime_residency` | +| h3-metal / FastH3 semantic audit | Effective parameters and ambient overrides, tensor/schedule transformations, VSA prefix/partial tiles and fallback; source-derived candidate | [Skill](systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) | `inferrsi.common.skills.audit_h3_metal_fasth3_semantics` | +| AFD plugin P2pNccl audit | CUDA eager dense-FFN rank namespaces and per-stage/per-peer payload ownership; source-derived candidate | [Skill](systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) | `inferrsi.cuda.skills.audit_afd_p2p_exchanges` | +| vLLM-RLT source audit | Synchronous early-exit KV propagation, later deeper attention and normal storage retirement; source-derived candidate | [Skill](systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) | `inferrsi.common.skills.audit_recurrent_kv_finalization` | +| Strata | Heterogeneous MoE, placement, memory and speculative execution | [Skill](systems/engines/strata/SKILL.md) | [Records](records/README.md) | +| vLLM | Token scheduling, KV ownership and parallel execution | [Skill](systems/engines/vllm/SKILL.md) | [Records](records/README.md) | +| SGLang | Radix/HiCache, prefill/decode, extensions and a separately pinned [diffusion lifetime candidate](systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | [Skill](systems/engines/sglang/SKILL.md) | [Records](records/README.md) | +| SGLang-Omni | Stage pipelines, streaming and transport | [Skill](systems/engines/sglang-omni/SKILL.md) | [Records](records/README.md) | +| vLLM-Omni | Model/deploy separation, AR/diffusion and output contracts | [Skill](systems/engines/vllm-omni/SKILL.md) | [Records](records/README.md) | +| Research router | Constrained engine design and validation | [Skill](workflows/research/inference-engine-research/SKILL.md) | — | For vLLM-Omni, the -[source and responsibility map](repositories/vllm-omni/SKILL.md#source-and-responsibility-map) +[source and responsibility map](systems/engines/vllm-omni/SKILL.md#source-and-responsibility-map) routes broad stage/model/output decisions and the separate -[abort validation](repositories/vllm-omni/sub-skills/omni-validation/SKILL.md) +[abort validation](systems/engines/vllm-omni/sub-skills/omni-validation/SKILL.md) with their original inspection dates and source pins. -See the [collection overview](workflows/inference-engine-research/references/overview.md), -[import boundaries](workflows/inference-engine-research/references/import.md) and -[Strata design-space analysis](repositories/strata/references/design-space-analysis.md). +See the [collection overview](workflows/research/inference-engine-research/references/overview.md), +[import boundaries](workflows/research/inference-engine-research/references/import.md) and +[Strata design-space analysis](systems/engines/strata/references/design-space-analysis.md). Catalog records link detailed procedures through `skill_documents.en`; `knowledge show` and `context` retain these package-relative pointers. @@ -215,14 +221,14 @@ and regenerate abstracts with `tools/build_skill_index.py`. For engine knowledge, update the procedure and catalog abstract together and retain source revisions and licenses in the -[source manifest](workflows/inference-engine-research/references/source-manifest.json). +[source manifest](workflows/research/inference-engine-research/references/source-manifest.json). Source snapshots are research provenance, not campaign defaults. Check local links, dependencies, catalog reads and noneditable installed-package access; retain `locally_verified=false` until a separate qualified campaign exists. ## MLX framework guidance -[Diagnose MLX evaluation and lifetimes](frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) +[Diagnose MLX evaluation and lifetimes](systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) separates graph construction, cross-stream ordering, backend buffer retention and result availability. It is source-derived candidate guidance; CPU, Metal and CUDA runtime behavior and performance remain unverified. MLX-LM request/cache policy and @@ -230,7 +236,7 @@ MLX platform discovery belong to their separate owners. ## MLX-LM application guidance -[Trace MLX-LM generation and cache lifecycle](repositories/mlx-lm/generation-cache-lifecycle/SKILL.md) +[Trace MLX-LM generation and cache lifecycle](systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md) links prefill/decode boundaries, three-stage batch-row ownership, prefix-cache identity and cooperative stopping. It is a source-derived candidate with no installed API, model, native-backend or service validation. MLX framework execution @@ -238,7 +244,7 @@ and MLX platform discovery remain separate responsibilities. ## Diffusers invocation guidance -[Trace Diffusers invocation state](frameworks/diffusers/diffusers-invocation-state/SKILL.md) +[Trace Diffusers invocation state](systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md) separates pipeline-wrapper fields, reused component identities, scheduler progress and cooperative interruption. The candidate is scoped to FluxPipeline and FlowMatchEulerDiscreteScheduler at its source pin. Runtime/model/backend behavior @@ -246,13 +252,13 @@ and performance remain unverified; the existing offload card keeps its own pin. For host-memory/transfer questions, read the separate CUDA dependency record `diffusers.cuda.diffusion.group_offload_transfer_overlap`. The -[version and responsibility comparison](frameworks/diffusers/diffusers-invocation-state/SKILL.md#offload-or-invocation-state) +[version and responsibility comparison](systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md#offload-or-invocation-state) keeps its earlier documentation pin/access date separate from the invocation candidate; neither supplies a combined-stack or backend qualification. ## SGLang versioned guidance -The [SGLang entry comparison](repositories/sglang/SKILL.md#choose-among-source-pins) +The [SGLang entry comparison](systems/engines/sglang/SKILL.md#choose-among-source-pins) separates the LLM engine graph, CUDA HiCache peer card and multimodal_gen diffusion candidate. Read external HiCache material by its stable catalog ID; the versioned routes retain their original dates and validation limits. Shared navigation does @@ -260,18 +266,18 @@ not establish cross-pin/backend compatibility or new Network coverage. ## Hopper kernel reasoning -The CUDA-owned [reason-hopper-kernels](hardware/cuda/reason-hopper-kernels/SKILL.md) +The CUDA-owned [reason-hopper-kernels](systems/platforms/cuda/reason-hopper-kernels/SKILL.md) candidate distills the pinned KDA → KernelWiki lineage into TMA/mbarrier/WGMMA lifetime reasoning, register/pipeline/layout/software-scheduling tradeoffs and -selected FP8 GEMM contracts. Its [coverage inventory](hardware/cuda/reason-hopper-kernels/references/source-inventory.json) +selected FP8 GEMM contracts. Its [coverage inventory](systems/platforms/cuda/reason-hopper-kernels/references/source-inventory.json) separates mined mechanisms, unmined source headers and Blackwell-only recipes; -[source/license qualifications](hardware/cuda/reason-hopper-kernels/references/sources.md) +[source/license qualifications](systems/platforms/cuda/reason-hopper-kernels/references/sources.md) retain contradictions and distribution discrepancies. GPU execution and efficacy remain not_run. ## DwarfStar portable AREX candidate -[DwarfStar](repositories/dwarfstar/SKILL.md) routes memory/SSD, serving-state and +[DwarfStar](systems/engines/dwarfstar/SKILL.md) routes memory/SSD, serving-state and qualification questions to one source-pinned operating graph. The portable AREX create/verify workflow was adapted to native C with the user-approved partial backend boundary. CPU checks do not qualify accelerated/model/service execution; @@ -279,20 +285,20 @@ required backend gaps remain explicit. Its generated catalog ID is `inferrsi.common.skills.dwarfstar`. ## Splash portable AREX candidate -[Splash](repositories/splash/SKILL.md) links source-pinned format/memory planning, +[Splash](systems/engines/splash/SKILL.md) links source-pinned format/memory planning, serving/cache identity and DFlash2/Metal/ANE qualification guidance. The portable AREX workflow uses the user-approved native/source-module adaptation. CPU logic checks do not clear required native backend blocks. Catalog ID: `inferrsi.common.skills.splash`. ## Lily portable AREX candidate -[Lily](repositories/lily/SKILL.md) owns exact architecture/MLX-affine-Q4 runtime, +[Lily](systems/engines/lily/SKILL.md) owns exact architecture/MLX-affine-Q4 runtime, strict text API/token-prefix state and native/benchmark qualification guidance. Its compiler/Metal/model gaps remain blocked; offline Python helpers are not Rust execution. Catalog ID: `inferrsi.common.skills.lily`. ## Husky document-only research candidate -[Husky](repositories/husky/SKILL.md) separates an attributed official-page report +[Husky](systems/engines/husky/SKILL.md) separates an attributed official-page report from implementation, reproduction and original research proposals. Repository, revision, source license and raw receipts remain unknown; its offline helper screens supplied metadata only. This is not complete repo-to-skill or native @@ -300,7 +306,7 @@ verification. Catalog ID: `inferrsi.common.skills.husky`. ## Project Maya portable AREX candidate -[Project Maya](repositories/project-maya/SKILL.md) owns model-family dispatch and +[Project Maya](systems/engines/project-maya/SKILL.md) owns model-family dispatch and physical residency planning, API versus native state, and speculation/calibration qualification. Selected direct-compiler CPU policies and safe offline helpers do not clear CUDA/HIP/model/state/quality/service gaps. New Qwen coverage is shallow @@ -309,23 +315,23 @@ routing/format/guard evidence, not deep kernel qualification. Catalog ID: ## vLLM Metal source and CPU candidate -The [vllm-metal graph](repositories/vllm-metal/SKILL.md) routes platform/native ABI, KV/offload lifetime and speculative/deferred/dispatch evidence. Three bundled offline helpers support bounded syntax, dense-pool arithmetic and accounting. Required Apple Metal/MPS/model/service/quality verification remains blocked; CPU source tests do not promote it. +The [vllm-metal graph](systems/engines/vllm-metal/SKILL.md) routes platform/native ABI, KV/offload lifetime and speculative/deferred/dispatch evidence. Three bundled offline helpers support bounded syntax, dense-pool arithmetic and accounting. Required Apple Metal/MPS/model/service/quality verification remains blocked; CPU source tests do not promote it. ## mistral.rs native paged-attention source candidate -The [mistral-rs graph](repositories/mistral-rs/SKILL.md) owns selected Metal decode layout/scales/workspace, prefill coordinates/masks/workspace and actual-model prewarm/evidence. Three offline metadata/formula/inventory helpers add no GPU simulation or execution proof. Rust compilation and Apple native/model/quality/service evidence remain not_run / blocked. +The [mistral-rs graph](systems/engines/mistral-rs/SKILL.md) owns selected Metal decode layout/scales/workspace, prefill coordinates/masks/workspace and actual-model prewarm/evidence. Three offline metadata/formula/inventory helpers add no GPU simulation or execution proof. Rust compilation and Apple native/model/quality/service evidence remain not_run / blocked. ## MLX Swift LM ownership and recurrent Metal source candidate -The [mlx-swift-lm graph](repositories/mlx-swift-lm/SKILL.md) adds selected Swift exclusive-access/cancellation/task-settlement and actual GatedDelta native/ops contracts. Two bounded evidence/metadata helpers authenticate neither observations nor dispatch. Swift compilation and required Apple native/model/quality/service evidence remain not_run / blocked; source-only masked-output parity gap is retained. +The [mlx-swift-lm graph](systems/engines/mlx-swift-lm/SKILL.md) adds selected Swift exclusive-access/cancellation/task-settlement and actual GatedDelta native/ops contracts. Two bounded evidence/metadata helpers authenticate neither observations nor dispatch. Swift compilation and required Apple native/model/quality/service evidence remain not_run / blocked; source-only masked-output parity gap is retained. ## MLC LLM compiled Metal source contracts -The [mlc-llm candidate](repositories/mlc-llm/SKILL.md) covers opt-in artifact/schema/resource declarations, Apple target and pinned TVM module/storage/stream consumers, and compiled/runtime KV limits. Helpers authenticate no binary/weights/dispatch or memory fit; required native/model/quality/service evidence remains not_run. +The [mlc-llm candidate](systems/engines/mlc-llm/SKILL.md) covers opt-in artifact/schema/resource declarations, Apple target and pinned TVM module/storage/stream consumers, and compiled/runtime KV limits. Helpers authenticate no binary/weights/dispatch or memory fit; required native/model/quality/service evidence remains not_run. ## Local Dream Android diffusion source contracts -The [local-dream candidate](repositories/local-dream/SKILL.md) separates desired +The [local-dream candidate](systems/engines/local-dream/SKILL.md) separates desired and serving process state, grace/restart/readiness and unauthenticated LAN exposure, MNN/QNN/Anima/DiT dispatch, shared cancellation/cache lifetimes and CPU-library versus DSP-skels/ABI packaging. One offline declaration screen is diff --git a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/_provenance.json b/src/inferrsi/knowledge/data/skill_library/_compiled/_provenance.json similarity index 55% rename from src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/_provenance.json rename to src/inferrsi/knowledge/data/skill_library/_compiled/_provenance.json index 881fff2ad..e9c6505a5 100644 --- a/src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/_provenance.json +++ b/src/inferrsi/knowledge/data/skill_library/_compiled/_provenance.json @@ -1,9 +1,9 @@ { "schema_version": 1, "repository": "https://github.com/InferMatrixLab/InferRSI", - "source_revision": "14e6f17bd254aaa05da0d5a2e3aea35ed337b139", + "source_revision": "217d0b386d9c8d53bc06fc0d485f2e99722d5847", "registry": { "path": "src/inferrsi/knowledge/data/skill_library/skill-index.json", - "sha256": "7904275c3383380db179287171e1430617aaa9ac4f439291eff8a71426987fbb" + "sha256": "c6c235483034dfce3c0a4ec3bf7d68046ea2cf8c933f069833ecaef65cdf2d70" } } diff --git a/src/inferrsi/knowledge/data/skill_library/_compiled/inputs.json b/src/inferrsi/knowledge/data/skill_library/_compiled/inputs.json new file mode 100644 index 000000000..8fbac40d7 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/_compiled/inputs.json @@ -0,0 +1,1383 @@ +{ + "ascend-aclgraph-capture-replay": { + "header": { + "name": "ascend-aclgraph-capture-replay", + "description": "Diagnose vLLM-Ascend ACLGraph capture/replay eligibility, stable-input and ordering contracts, and capture-resource failures at a pinned implementation. Use for evidence-gated graph-mode proposals, not allocator, compiler-fusion, or network tuning." + }, + "files": { + "SKILL.md": "ca2a29e7577f6a3bdadea1741158dc60bd22fd2d0e8475638f54eb5307734543", + "references/source-notes.md": "6ffbfb68cc7e04923b4fbca2a73a7555b7c3b45a6f40819bb5f3213ef05cf88d" + } + }, + "ascend-device-health": { + "header": { + "name": "ascend-device-health", + "description": "Pre-run health checks and recovery procedures for Ascend NPU devices on shared machines: card-group topology validation, Health status checks, SIGKILL residual-state detection, 4-step smoke verification, process occupancy check, and DVFS warmup discipline. Use before any experiment that requires a trusted performance baseline.\n" + }, + "files": { + "SKILL.md": "526bc4c3a277e3d1773a2bed81db60ad99f077aa1569302351ff66a4a4c27f7d" + } + }, + "ascend-distributed-parallel": { + "header": { + "name": "ascend-distributed-parallel", + "description": "Select and validate distributed parallelism strategy on Ascend NPU: topology-aware card-group selection, USP/CP/TP/PP strategy choice, communication masking setup and verification, attribution of parallelism gains, and diagnosis of strategies that appear to activate but have no effect. Covers native HCCL multi-rank AllReduce dispatch, scratch and synchronization boundaries, AlltoAllV defect workaround, and source-scoped communication-test timing and correctness coverage.\n" + }, + "files": { + "SKILL.md": "cd9a681bd117a38004a6374e9aacba773cbc6cdfea6f7a3eb465bda564175ddf", + "references/awesome-ascend-communication-tests.md": "0cb27a3d24409171bb414d7ebf0c49ad62be5f46482f29624695496ba9eb3e63", + "references/hccl-LICENSE.md": "05205a02acebd671c78d3b876c1f6097b84cce89780f5d9bdc204bfbea92d0c1", + "references/hccl-allreduce-readset.json": "4a3d403986dfceaa28605f2f30b79545e1efdc27f35c88755402b75b4d13d7e2", + "references/hccl-allreduce.md": "4770bba0090ed5ead3d2fd3b2cce2d21ba708acae3e0b4d859cc503499da5b93" + } + }, + "ascend-graph-mode": { + "header": { + "name": "ascend-graph-mode", + "description": "Diagnose an Ascend TorchAir ACL graph path from versioned configuration through FX compilation, capture-key selection, eager fallback, input staging, replay and output lifetime. Use when repeated inference has host overhead, capture churn, stale outputs or an uncertain backend; require source-matched configuration and separate device validation." + }, + "files": { + "SKILL.md": "f22e7b06c16b5a5427c3bab6b34b4ef6cd27e10af1d1705a1276c0f7440eeb41", + "references/legacy-source-notes.md": "a51c655aff59741cd7421277bd71dffeb9907560950459ae0a776207e496de4a", + "references/torchair-LICENSE.md": "bc07d81fa332b6543463588a120518c57b5ce8582d3747de75d2f6b09b068c01", + "references/torchair-graph-path.md": "272424571ff439278c8f2675406eae7da9a6482c581973e24319739fa0e58348", + "references/torchair-graph-readset.json": "eb84cbd5c355a88ee013dc7e428b274162e0e13e85845b58526113b0d6a2b46b" + } + }, + "ascend-model-adaptation": { + "header": { + "name": "ascend-model-adaptation", + "description": "Patterns for porting LLM/VLM model adapters to Ascend NPU: FP8 checkpoint handling, KV replication with TP, multimodal feature validation order, graph mode startup, and feature-status reporting. Source-grounded from vllm-ascend production experience. Use when adapting a new model checkpoint to run on Ascend via vllm-ascend or a similar framework.\n" + }, + "files": { + "SKILL.md": "bf1630e5f7f716dac4c8d59235dd2b088e126c80a544f01ff6daabb6ac0abdf0", + "references/source-notes.md": "500966fe5ddd3b52bb8405b406ba9c707b5e68c0ec8317f8b1ac8c109d38936b" + } + }, + "ascend-operator-profiling": { + "header": { + "name": "ascend-operator-profiling", + "description": "Benchmark a single Ascend NPU operator using msprof op: build, collect with warmup, read the 8 CSV metrics files, derive a bottleneck verdict, and iterate. Use when validating operator performance in isolation before system-level profiling.\n" + }, + "files": { + "SKILL.md": "927b19960629173945604ffae9dfe5982429c86c2b33c3d8b8ad7552a3e2f388" + } + }, + "ascend-operator-tuning": { + "header": { + "name": "ascend-operator-tuning", + "description": "Run a staged, accuracy-gated tuning campaign on a single Ascend NPU operator: set a numeric target up front, block tuning until an on-device accuracy pass is logged, escalate code-level then scheduling then instruction-level tuning with per-stage stall counters and kill-switches, and route each change by the metric contract (core utilization, bubble rate, load balance). Use after ascend-operator-profiling has measured a baseline and the operator is confirmed as the bottleneck.\n" + }, + "files": { + "SKILL.md": "b226f5597b964bf2616d7faa68f7fa047789695904ba6e44e9c8ca2b7741d2ef" + } + }, + "ascend-optimization-casebook": { + "header": { + "name": "ascend-optimization-casebook", + "description": "Casebook of NPU optimization cases for Ascend inference systems: symptom → diagnosis → action → evidence records for memory-bound, compute-bound, communication-exposed, graph-compilation, and AI-CPU-fallback bottleneck classes. Use when a bottleneck has been classified and you need prior art for what to try next. Complement to inference-optimization-casebook (which covers the casebook *pattern*; this carries the actual NPU cases).\n" + }, + "files": { + "SKILL.md": "b1ffddd0784743840606c151e7cf23f6eb5466e90f93f6d91214fded73c0be4a" + } + }, + "ascend-performance-analysis": { + "header": { + "name": "ascend-performance-analysis", + "description": "Diagnose performance bottlenecks on Ascend NPU systems using CANN profiling data. Maps GPU profiling concepts (nsys, ncu, SOL%, roofline) to their NPU equivalents (msprof, torch_npu.profiler, CSV/JSON outputs, AI Core utilization, HCCL communication analysis). Covers measurement discipline on shared NPU machines and evidence acceptance for automated NPU optimization stages. Use when profiling data is available or a stage report needs its execution and validation evidence checked.\n" + }, + "files": { + "SKILL.md": "8fdbb60d366a2e03f08d17a4604210833a54104fc0a22cd2be68c2cc2052deba", + "references/cannbot-CANN-2.0-LICENSE.md": "89b1c114df62cfd183004f38d18d07f186436c3b37203dd69c3e84becf9d2282", + "references/cannbot-stage-evidence.md": "f56e4125325c9c1b9235a1d64fd732861c7686444492422daff44ba8dd648be4" + } + }, + "ascend-video-diffusion-optimization": { + "header": { + "name": "ascend-video-diffusion-optimization", + "description": "Run and verify goal-driven Ascend video diffusion optimization. Specializes the backend-agnostic video-diffusion-optimization skeleton with Ascend NPU operator, layout and profiler references. Use when the user asks for latency, RTF, throughput, or HBM targets across VLM, DiT, MoE, attention, or VAE on Ascend NPU. Not for text-only LLMs." + }, + "files": { + "SKILL.md": "955e5b65c86cf6b39e9dd042c4f0d5d3e4869d708daba2a77bc4cac7b8dbff33", + "references/field-notes-video-diffusion.md": "dadac7b23462fdc325218c72ddf8a2ac553151761a5d3e45b854b53d2c8c6908" + } + }, + "ascendc-tiling-design": { + "header": { + "name": "ascendc-tiling-design", + "description": "Design tiling for a custom AscendC operator: normalize the shape first (axis-collapse or flatten), answer the fullload-vs-split question for UB, apply the universal four-element template (multi-core split, UB split, buffer plan, branch matrix), and route per operator class (reduction, broadcast, elementwise, matmul, conversion) using algorithm-selection and chip-generation tables (DAV_2201 vs DAV_3510). Use when writing or restructuring an AscendC operator's tiling logic, before running any performance tuning campaign.\n" + }, + "files": { + "SKILL.md": "5d7afc6e33b5224c53dfca2f97a2e56c44d51216ab59c3ec6bf984511983ce8a" + } + }, + "assess-serving-preemption": { + "header": { + "name": "assess-serving-preemption", + "description": "Assess whether LLM serving queue delays justify token-boundary priority preemption, accounting for suspended KV, host swapping or recomputation, and long-request service guarantees. Use for admission-versus-service-order diagnosis, not current scheduler implementation or tenant-fairness certification." + }, + "files": { + "SKILL.md": "7167ef93338f5acdf2eab867372748259bfd7cfc8480ea7ddefe166882e04570", + "references/source-notes.md": "f29fcecfba0490008a354c11b42403983fe165e56ff907ad61cc6040d2b6407f" + } + }, + "assess-split-backward-schedule": { + "header": { + "name": "assess-split-backward-schedule", + "description": "Evaluate memory-capped split-backward and V-shaped pipeline scheduling proposals, preserving synchronous update, global-batch and optimizer-validation semantics." + }, + "files": { + "SKILL.md": "f777607e0158371f4025efff3fa63e57e1622e2a86ec087cc580f076681e9ac5", + "references/source-notes.md": "61b1609f71c37b61d136b8c4d39f793de87cffb1cf31c8ec737a1c3fa6fb1b4e" + } + }, + "audit-afd-p2p-exchanges": { + "header": { + "name": "audit-afd-p2p-exchanges", + "description": "Audit CUDA P2pNccl Attention/FFN rank namespaces and per-stage, per-peer hidden-state ownership when uneven worker counts or a mismatched return slice make an AFD exchange ambiguous." + }, + "files": { + "SKILL.md": "2325744f75fcb60608da0715aee5a5fc6160cf8a672de18a379fa46ec394e022", + "references/afd-plugin-LICENSE.md": "0e8566ea362276878e1f38f97da49964dbea301c15d4aa9d727f97a4c39fd943", + "references/source-notes.md": "5073d87450cb60d1b245308dff8807e73749c07de976d50f2fa26694fad81a72" + } + }, + "audit-fixed-workload-claim": { + "header": { + "name": "audit-fixed-workload-claim", + "description": "Audit a performance claim from supplied fixed-workload reports and an explicit measurement contract. Check aggregation, retained observations, comparability and quality evidence without launching new benchmarks." + }, + "files": { + "SKILL.md": "30d5303418fd5748649fa38ffd525a885749ee8d93a02b41b7efc27ae74fb471" + } + }, + "audit-flare-session-lifecycle": { + "header": { + "name": "audit-flare-session-lifecycle", + "description": "Audit Flare complete prefix/session restoration, inbound transfer tickets, publication/use/retirement fences and host-restore admission outcomes. Use when a lookup hit does not become reused tokens, a cancelled waiter leaves transfers in flight, or a restarted session must recover KV pages together with recurrent, convolution and partial-tail state." + }, + "files": { + "LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", + "SKILL.md": "bf226236185f3e1009f062e73b81cc517d64d5483734f4ee07fa1fd2ac4e9f6e", + "references/diagnostics.md": "86d10d9aa848c9539664b2220ae712f23273e8af4dcae5d1b6c8ed3befdfbfa7", + "references/lifecycle.md": "107ebe3c22e5dba8007c3b8ea61149f10bc55c51b557a4230f3e5de859bb0f7c", + "references/source-ledger.json": "61e51a97ffd7c1fa7866bdd440b024d0c5e0065635ac8d0ba53d2c69c7f9e470", + "references/source-notes.md": "f174a0c38303cc50b9b27a0ef57de91147d6c6a24d5205b9b0bd2ecfda339806" + } + }, + "audit-h3-metal-fasth3-semantics": { + "header": { + "name": "audit-h3-metal-fasth3-semantics", + "description": "Audit h3-metal FastH3 effective configuration, checkpoint tensor and schedule semantics, partial VSA tiles, sparse eligibility and dense-plus-coarse fallback without treating source or header checks as native parity." + }, + "files": { + "LICENSE": "663e51edcd9a3a8f858018ce6924ad2895a24c269c26c39ad75325bcd0f7e5bc", + "SKILL.md": "988eed832af47f41e574995d78bdb55117ed8ee14b6127f9b18209d0ec5d843e", + "references/THIRD_PARTY_NOTICES.md": "a0d77db7d2aca0eda811ab3c386cc32b3143f2c2609a042cf4642954f0f6fb09", + "references/preflight.md": "5cf745005914c74b18a09f127d1d02ce0cb64f6cd7899501ce3cb40938c42a4c", + "references/repo-provenance.md": "9c6010d8b0e8db1c4e611ff39a2ee052169db97f240f00d7412b7b8095dd087e", + "references/semantic-contracts.md": "3f518160e8a227272e698486e6c67acd364af0d74755103ffc868b2d66052c20", + "references/source-identity.json": "b0e63197725c6a1baa4888ca3c48ccd4926b6e6f0f8bf781ae76ac97ffd1ceb1", + "references/source-notes.md": "f9430103644eacd9c55ace6f84d8587fc0bc063bfcd9cb60e6045b139a9fee05", + "references/troubleshooting.md": "ed552d87f9fb9fb2c56de64df61f3fc6e2333cb049116cd0736f784c8a327612", + "scripts/audit_fasth3.py": "05bfb9348cf88a76f4fba8bc8a600fe54246d83a98c1685bc4847ace28e2fbac" + } + }, + "audit-h3-runtime-residency": { + "header": { + "name": "audit-h3-runtime-residency", + "description": "Audit MiniMax-H3 generation in h3.c on Apple Silicon by tracing conditioning, AdaLN precompute, DiT residency, Metal precision paths, SSD streaming and audio/video decoding." + }, + "files": { + "LICENSE": "663e51edcd9a3a8f858018ce6924ad2895a24c269c26c39ad75325bcd0f7e5bc", + "SKILL.md": "dd9539c5189582df0ffac7cf24461e377298507566cec44ca3bb9dec1def4b8d", + "references/THIRD_PARTY_NOTICES.md": "a0d77db7d2aca0eda811ab3c386cc32b3143f2c2609a042cf4642954f0f6fb09", + "references/minimax-h3-apple.md": "9e11b867fd7e3dede66e73de7410558bcb6b5015a0a218bed376b4b24bebad76", + "references/minimax-h3-weight-inventory.json": "fbaacaeec340d3ce893556ee864405a0746af4d13530a38cb3d5aa9d60929271", + "references/phase-residency.md": "2836a550eb2be7f5a862f51ec7b135ebd9715bf3c4faa84a7ecbc9d9c11f5d8d", + "references/precision-and-streaming.md": "7e166c89193f4c7aed852ec95346a2de2b5008dac002e2e1862be1b3e57f6093", + "references/source-identity.json": "730f8dd23648a3f8f47b9726cc613d893a79d1817d515eec609a9b06d8a9ff90", + "references/source-notes.md": "7304bf9369efc157dc7aa40686dd828cc14db45c49442e908b3d3cc4b329eb86", + "references/validation.md": "0754873a758e048480938314894e25b504ae0c269549312e7f8e6020d83fe294" + } + }, + "audit-llama-runtime-state": { + "header": { + "name": "audit-llama-runtime-state", + "description": "Audit llama.cpp context memory, partial decode progress, graph reuse and server cancellation when a retry, stale prefix, unexpected rebuild or unreleased slot needs an ownership explanation; requires pinned source or a concrete runtime trace." + }, + "files": { + "SKILL.md": "fb53c38204f3eb5fcadaa4870cc8b111bf2f3b92696a0a2dd2d9b41d4a905e88", + "references/source-notes.md": "3d12a401e9decb55a2f4f3b2bfa0d360ef5110f63a0469dfe285654e78054190" + } + }, + "audit-recurrent-kv-finalization": { + "header": { + "name": "audit-recurrent-kv-finalization", + "description": "Audit vLLM-RLT synchronous early-exit KV propagation, later depth-specific attention reads, and request storage retirement. Use when shallow token exits leave deeper history or release order in question." + }, + "files": { + "LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", + "SKILL.md": "fb59f58e1c1ee9490f9c933b9c5ac7788bff367511e6f947c96f53939fc8f521", + "references/NOTICE.md": "cca794656897ecdc37f983f7f23c4cd12306dbe3c9cd2f54fece6340753ae4cf", + "references/source-identity.json": "5e52db0a0bda4673b9aac46a3fc035b824780def72921fee6796edb2ed9fca65", + "references/source-notes.md": "0956f59cd9a1fac27da53228dd60c51a07b87639a43c09bfab296c4aeeceaab9" + } + }, + "audit-sd-cpp-residency": { + "header": { + "name": "audit-sd-cpp-residency", + "description": "Audit stable-diffusion.cpp image-generation memory pressure by separating catalog freshness and weight-byte delivery from model parameter storage, compute weight residency, retained runtime buffers and segment prefetch under the actual backend." + }, + "files": { + "LICENSE": "b53fa08f515cb5a6fff7b9fd8fcd0961a4b80df29d74372d25e1f9171aa042ee", + "SKILL.md": "0a2ea4dcff90a283cbf1e835f6e8b93a160ffc9f0e6400aa00aae3ad0306b184", + "references/loader-storage-boundaries.md": "c1ec047aa37fca6b374f9bfd750e8f30d45af1d48fe728db23873c0b84ad94e8", + "references/phase-residency.md": "7b537cb972347dd68e303960f142892346d8b19b884e4d08016ea36a23bdaa7a", + "references/source-identity.json": "af4830370b6eddc2d176a9ae14da4ab56a90b0fe048a01a89ee051df4365d8e9", + "references/source-notes.md": "c493af8f6fd11ab7c9a37c9eec78cac31d57c04403a790035efbb4fff99f4669", + "references/validation-boundaries.md": "3aaf79b36192bf8ff3e8f95ac90f2e769aa065dfac1fe70368824f3591849b8d" + } + }, + "audit-sglang-diffusion-lifetimes": { + "header": { + "name": "audit-sglang-diffusion-lifetimes", + "description": "Diagnose SGLang multimodal_gen stage state, component residency and request timeout/deletion when grouped outputs interfere, a module remains resident, or stopped waiting is mistaken for stopped execution; requires pinned source or a concrete trace." + }, + "files": { + "LICENSE": "1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747", + "SKILL.md": "c2f28dae816151a9fc03ee69dea415f2d1415612f33c1fefe960a0f55b30c9b2", + "references/source-notes.md": "d0fcfd1177ae2f351041c5b2abba7f5121cec133006d9e4dca8f74ddb567332d" + } + }, + "audit-vllm-kv-lifetimes": { + "header": { + "name": "audit-vllm-kv-lifetimes", + "description": "Audit vLLM V1 asynchronous KV load/save completion, block retention, preemption, and receive-failure contracts when requests stall or blocks may be reused too early; requires a concrete connector trace." + }, + "files": { + "SKILL.md": "10f7ad0b6c04089fa32e9d08b2e0b45b159659ac0e42de8fb91eb19bfe07bb06", + "references/source-notes.md": "3665b034594fc8cc767c408002f4e16f8803369242405027c7f37ce6c9ecebe9" + } + }, + "bound-windowed-attention-replay": { + "header": { + "name": "bound-windowed-attention-replay", + "description": "Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved historical state satisfy the full dependency closure. Use when a model has shared hidden state or preserved-compressed KV and the retained region must be derived rather than assumed. Derives the retain rule from window plus rollback depth, separates query-row compaction from dropping historical keys, and names what becomes unrecoverable." + }, + "files": { + "SKILL.md": "06e007f578f19bda7e047bf860fbbe0c899791553fab99e92ef20e929caded54", + "references/source-notes.md": "fd917bd5ac80bee9c9f4525f8e0300f6d67dc144e2a16632af42a67f3d53d098", + "references/validation.md": "60e71e70c5519bcdf6c171eec2be2f342e8801bedd746d6005f11adc646dfaff" + } + }, + "budget-speculative-verification": { + "header": { + "name": "budget-speculative-verification", + "description": "Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an engine already runs a draft head with multi-token speculation and verification cost grows with query rows. Separates draft cost from verify cost, builds a cumulative-acceptance rate vector, fits a cost model, and allocates a per-step row budget." + }, + "files": { + "SKILL.md": "6e0dfacc7aa59b383761798656fd52e99b66fc7794c65dc3fc4f1c180b73072c", + "references/source-notes.md": "6fa955022f16cd598f44fd6521901d1cb868b06cec545cabe2b82769a641b5d6", + "references/validation.md": "673d93f6700052974b340fab25b3438e0e6344960569ac9abec7d7720f662288" + } + }, + "calibrate-serving-simulation": { + "header": { + "name": "calibrate-serving-simulation", + "description": "Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical predictions from physical bounds and near-capacity queueing risk." + }, + "files": { + "SKILL.md": "36c1a9f9723347e77e6076b4efb1e35c5967b21c0fd44eaf003ab176c5786bbb", + "references/infera-serving-search/README.md": "fe614b1c62f1d4a9db797a85d9b11931ae32169c40109c7b69ecd648fb6e9d1e", + "references/infera-serving-search/source-record.json": "61fafa5e92ae6312f8792e4630a35c78da4c1810fac4bf9cde385824c3894f36", + "references/source-notes.md": "9696d3c0ef96b6611787446bd6f96d4d6255ca8d107105de8c1d9af83edb9fa5" + } + }, + "cann-attribution-overlap": { + "header": { + "name": "cann-attribution-overlap", + "description": "Attribute NPU trace kernels to model source and plan bounded multistream overlap: select a representative step, map kernels to source owners via code_ref, build the module/operator dependency DAG (data, state, event, collective-order edges), classify serial/candidate/pending groups, choose minimal-first overlap with an explicit execution-mode synchronization contract, and accept only through correctness-first checks. Keeps the !302 import lineage distinct from later revisions, carries known formula/threshold conflicts as unresolved, and qualifies memory evidence by domain (device HBM/UB counters are not host RAM). Planning procedure only — no formula automation for disputed definitions.\n" + }, + "files": { + "SKILL.md": "3c0f31b2c7d6b4c121139d8aaa95f40aa2ae3f9eb23ce07dfea8b865e89731e1" + } + }, + "cgroup-resource-controls": { + "header": { + "name": "cgroup-resource-controls", + "description": "Diagnose and configure cgroup v2 resource controls for serving/training workloads: distinguish quota stalls from sibling contention (cpu.max vs cpu.weight, cpu.stat vs cpu.stat.local), audit requested-versus-effective cpuset with migration-cost budgeting, triage memory pressure with PSI plus event deltas (high/max/low semantics), and gate io.max caps on confirmed block-backed traffic. Includes hierarchy/delegation/permission preflight and a rollback contract. Extra resource allocation is treated as a capacity spend, never as an efficiency improvement. Control-plane procedures only — applying settings requires separate authorization.\n" + }, + "files": { + "SKILL.md": "3603cbef2516b00b6079f8b773b4a8aca28659475982c695d3eb36230ae6daef" + } + }, + "checkpoint-recovery-consistency": { + "header": { + "name": "checkpoint-recovery-consistency", + "description": "Assess distributed checkpoint recovery using DLRover memory agreement, persistence completion and replica transport, and NVIDIA Resiliency Extension complete-iteration selection. Distinguish worker, host and replica loss; separate queued saves, publication, application-state validity and durable recovery.\n" + }, + "files": { + "SKILL.md": "cc53f838da409daff0f85c3d4d3867307c00b4f7822cdc042274bd1aea8a95c1", + "references/dlrover-persistence-replicas.md": "055f4357f79026fcbcc766bbe77db610376ec353623f8898f5d14a30ee69e7bc", + "references/sources.md": "9425b1aab963a2c8d35f3db897bd15ee69699bf5b0d0c8425c3c8bba3e9a0233" + } + }, + "choose-parallel-training-plan": { + "header": { + "name": "choose-parallel-training-plan", + "description": "Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for memory, topology, resharding and planning cost." + }, + "files": { + "SKILL.md": "0e56b44e712af224252fdd8061cc9c0dfc2b22c54f010664644ca61c93f4b27f", + "references/source-notes.md": "9f9184d46b19312138a6e805e439e43dc0594a0cc6b46675255a582dc946ca5b" + } + }, + "cite-check-downstream-pr": { + "header": { + "name": "cite-check-downstream-pr", + "description": "Review a downstream PR description for correct acceptance issue links, SHA pins, and artifact citations." + }, + "files": { + "SKILL.md": "68ff74b48de1607297aa35e2e8b2eaea12674d938f8c8167b93e55b1044ba858" + } + }, + "claim-contract": { + "header": { + "name": "claim-contract", + "description": "Enforce ThinkFlowLab claim contract wording and verdict rules without embedding campaign-specific revision pins." + }, + "files": { + "SKILL.md": "49cf787427e7842fc0b8cbc1e58091fc4b77a5a48c0cb8fb7aaecd91f8715670" + } + }, + "comfy-ui": { + "header": { + "name": "comfy-ui", + "description": "Audit ComfyUI workflow-output cache validity, lazy dependencies and model-weight residency as separate state contracts using a pinned source revision and explicit runtime evidence." + }, + "files": { + "LICENSE": "3972dc9744f6499f0f9b2dbf76696f2ae7ad8af9b23dde66d6af86c9dfb36986", + "SKILL.md": "904de714cd5fad63552b3c62dfdbbaa31b4a02de13e20eba752427b0540f1778", + "references/graph-cache-invalidation.md": "614c9a4e6943d24d7d7be7d632682ce8b1dcc239124a37d9dc1d487113772eac", + "references/model-residency.md": "014c68386e7cdfdb7b4f566836e8b2ff61bd5a5a506e7b4723f83fe80c5d4de7", + "references/repo-provenance.md": "ca996ef515af42a3ce304747fa3f12a31c00533deaace5ba2ce21aefa0a62e2e", + "references/validation-boundaries.md": "1913dedbeb558d872d6a0e015cc11d822910c3cb259a236db29fb60ad8a64263" + } + }, + "competitor-scan": { + "header": { + "name": "competitor-scan", + "description": "Structure a lightweight competitive scan without inventing benchmark numbers or campaign verdicts." + }, + "files": { + "SKILL.md": "7f2613a1fc8e9fc58c3058c4946b85e8d7d49359143d650f711b88b448307ab6" + } + }, + "dcp-async-save-performance": { + "header": { + "name": "dcp-async-save-performance", + "description": "Diagnose training-visible checkpoint pauses using PyTorch Distributed Checkpoint asynchronous saving and staging. Separate snapshot capture, staging completion, upload completion and backend durability; bound in-flight requests and cached CPU buffers before proposing overlap. Use for DCP save performance, not distributed recovery validation.\n" + }, + "files": { + "SKILL.md": "ed761a1e7efcb7350e9436eab86e708053ba4ba28dd038c135d63f7540e46101", + "references/sources.md": "22afe4f4d77f441096cc657d55d763473bfcbea8fb531c8e493fe49ab84955f7" + } + }, + "dense-performance-evidence": { + "header": { + "name": "dense-performance-evidence", + "description": "Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps evidence types to the questions they answer and the claims they can support. Use to decide which evidence to collect next, avoid over-claiming from cheap evidence, and structure a measurement campaign.\n" + }, + "files": { + "SKILL.md": "8fc261fce4be42ef66b3f98cf35bf3b5f8ee6f622494083ddb343f2645d6bf15" + } + }, + "diagnose-attention-bottlenecks": { + "header": { + "name": "diagnose-attention-bottlenecks", + "description": "Diagnose the next attention-kernel bottleneck after IO reduction using evidence about HBM traffic, work ownership, parallelism, scalar work, asynchronous scheduling and precision. Produce a bounded optimization hypothesis and correctness-first verification plan; do not infer current library support or NPU portability from GPU papers." + }, + "files": { + "SKILL.md": "dabb1e8f06875a72dfa78c4d684fb4bf380667d34151dbc1a6355b355a432df9", + "references/mechanisms.md": "fe08b2292fa587be3874774ddf41063b84c579a0085df422ec6710d8aa90b8b7", + "references/papers-20261015/INTEGRATION_NOTES.md": "371b01ee201ef51fecbfd6da892d517aaba295930b9395025f03f34e21c030a1", + "references/papers-20261015/NEXT_GATE_LEADS.md": "35b8b58ae5db02b2df51ccbe5cdf870a7383d17f345f9ee5eb7b0a4146dfc74b", + "references/papers-20261015/README.md": "e0b1ec1b39c10067f5ddce58320851ad089568c4b2296a24777389097cb5b388", + "references/papers-20261015/cases/io-reduction-can-plateau.md": "5cc35f514224ac6e6efd2da6db221bba468b2f4db66807295e67b240bcdfdba5", + "references/papers-20261015/cases/overlap-and-fp8-counter-signals.md": "80c8a60a6df55f3d84fab23e03567d8b38e23bcdbd63cc09881e46db1e2888ab", + "references/papers-20261015/cases/partitioning-after-fusion.md": "4d4cabe0991eab554dcfbd4fa9c8199e61fa4767709b0c4cd646e2588bb398a2", + "references/papers-20261015/knowledge/attention-cost-model-boundaries.md": "5ed8f888443e38ccb72545b6025447aab7b92936f78c151fed680e582126239b", + "references/sources.md": "53436e0de4e6dda2d29426591bf61f8526378cdc6f9ec7967b88b1bcd4c61e6f" + } + }, + "diagnose-vllm-admission-pressure": { + "header": { + "name": "diagnose-vllm-admission-pressure", + "description": "Diagnose vLLM V1 admission stalls and KV-pressure preemption by separating token, input, active-request, cache-allocation, and connector-wait constraints at a pinned scheduler revision." + }, + "files": { + "SKILL.md": "e0dc4374adaf70324bd21f7a5bde5d7cadd7fd84167bcec5ca5004191b1a5cc3", + "references/papers-20261015/INTEGRATION_NOTES.md": "de414b2394f59f896df5b32aeb02aad43021b5c5c984bec8c31ac11f13f903a3", + "references/papers-20261015/cases/migration-is-not-free-memory.md": "2ac168f2725cba7fc92dfcc67c01bf2258dff6a2dbe1ddb063bc60c175d82d0a", + "references/papers-20261015/cases/parallelism-change-with-state-cost.md": "5accc81086e4500d66f894ef6af9951e1cbb76941f6c631aab5c5561a786d8d8", + "references/papers-20261015/cases/preemption-under-kv-pressure.md": "462200a6d0b193ef01f32374a463e0f2838ca8edf33d9ab898def65ffd0b490a", + "references/papers-20261015/knowledge/scheduling-state-boundaries.md": "d4ef29869959b835a5dcb06ddb2fd8e6cff49bb176994dc1d46eb9a852f6228d", + "references/papers-20261015/notes/fastserve.md": "9b4e275843000ef453d508d9c3c9c3f8ea1de5545fbd01d7abe74baa8a4931a7", + "references/papers-20261015/notes/llumnix.md": "f252ce33ac2db017701ca1d49813c5ec1996c4c92900cf35bb6bda7fc59c0f18", + "references/papers-20261015/notes/loongserve.md": "7bc6d1e256d461b04acc8ed856cf99108faceda9cde729ab95bcca596fe019ca", + "references/papers-20261015/notes/orca.md": "f5a3b5870d64226408a8fcda5d69483769d823f8a5efd7f11698c8da483ab195", + "references/papers-20261015/references/fastserve-readset.md": "9866bd0c78b2b638998b3b632f6bae26aef032462c81abee336d309af5b07831", + "references/papers-20261015/references/llumnix-readset.md": "3a2819d65ad24087c08530492ef39d8ea046ed1f780d53512f1d23e2b3578487", + "references/papers-20261015/references/loongserve-readset.md": "f1a383e7bdbbd5a5a472d7f6e1292c8a17d4910ff736ca5a6aa422e1bee74f8c", + "references/papers-20261015/references/orca-readset.md": "a497ef23b7e60c0fbc15d95b7e07cd20547cfe56f37c24dfe6ed70d79ef7a3cf", + "references/source-notes.md": "84033fd514773f10c0bddd9914fabb980497ba2b308f226a922c3e1888413827" + } + }, + "diffusers-invocation-state": { + "header": { + "name": "diffusers-invocation-state", + "description": "Trace Diffusers pipeline-wrapper and scheduler invocation state when repeated calls, from_pipe reuse or interruption produce unexpected schedules or retained request state; scoped to FluxPipeline and FlowMatchEulerDiscreteScheduler source evidence." + }, + "files": { + "SKILL.md": "c7743a838e3f50627a998716b5caee99951ac15c14a4b2e848daf7d0bd240a3e", + "references/ownership.md": "40688cc44183027efbf2fca8d3548c299cdf2e33fef54f76ab3549353eb9bd1b", + "references/source-map.md": "10562a75683bd9bd07ec2e37a1cbd0f93f2534d6f85b55a566adb2d97485215f", + "references/validation.md": "5a0b854b87d091d7d505b0bdd5c03330052507a851ab6edf35b93a6675f2d4f7" + } + }, + "draft-acceptance-issue": { + "header": { + "name": "draft-acceptance-issue", + "description": "Draft a campaign acceptance issue with checklist, RunSpec, evidence paths, and placeholders for baseline/candidate SHAs." + }, + "files": { + "SKILL.md": "2c14533c067c51b9763f80a9df0bf5eb5acd398f4265790c57039fed96d4c497" + } + }, + "dwarfstar": { + "header": { + "name": "dwarfstar", + "description": "Investigate DwarfStar model fit, expert SSD streaming, session and vision cache correctness, speculative sampling and distributed execution using pinned source contracts and explicit native verification boundaries.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "LICENSE": "4a8a1358f5b048627d71e4988de0bafd386f4988991929629e6cb9e29b7b9b2f", + "SKILL.md": "1f0aa283e08ded546a7fa7505f8772c44be48715e61e9e6edb841af49c6d8da0", + "references/repo-provenance.md": "7d72f9c85ee60fe003f5f5a585b52e9989ebcedca625b584fa39a976d4433998", + "references/repo-routing-metadata.json": "8e195b420b67151932af5f435c284f49677cc7863e650c83cfa9e762fd00db0b", + "references/source-identity.json": "a38fe07d003accd83b7626d7e189f7b49b4b73e7a223e4d441b0e33add5d9c91", + "scripts/request_template.py": "62e4cde002f2aebeaddd63e6e1f04fc727c271d2cbdaa00856158baadee5340b", + "sub-skills/qualification/SKILL.md": "3b8eb25f6b8e4729b48cce5619daa24422bee251f119ea7f9cc1faf0097ec6bb", + "sub-skills/qualification/references/troubleshooting.md": "cd832ab2a420490524251a042c0c83c2f75bbfdc12e914b91da8f91edf6860e9", + "sub-skills/qualification/references/verification-contract.md": "2801f6683c2974d1cfa5b86c6268fd33f175f1cbcc9ed85c932e9b3504fac3e1", + "sub-skills/runtime-memory/SKILL.md": "ad1d34d429baaa41fb72c94ff4e554b26b2f5b701cc4deed03f1d066c7d3fc52", + "sub-skills/runtime-memory/references/memory-ledger.md": "765915ecd3a318cc7908658d5a79bbdff615fc24409501febcb7ff8af65a1050", + "sub-skills/runtime-memory/references/troubleshooting.md": "42e4785b7d3c1375efa324583897327f28ef76c0b7788092da92095bddeb1440", + "sub-skills/serving-state/SKILL.md": "71c27212847146376d72e6cc6d22abe2f7eed56402c5917cd7dfcda0b2aa5982", + "sub-skills/serving-state/references/state-contracts.md": "8f74a07c947021e3069468268f6b5aa2548a17db518c5c0505b84dde73a1f31c", + "sub-skills/serving-state/references/troubleshooting.md": "c9eebe4109d418c4dfb9bb2425c145ebc1a52da9b665e0556149bc31a442e2fc" + } + }, + "evaluate-elastic-serving-placement": { + "header": { + "name": "evaluate-elastic-serving-placement", + "description": "Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including state-copy, reservation, recomputation, interruption, and priority costs. Use for cross-instance memory imbalance or highly variable long-context serving, not connector wiring or live scaling." + }, + "files": { + "SKILL.md": "8668e20cea75717fe536cd310b40b5cc1b94e0a25ad097675d3bdaae7c7d8886", + "references/source-notes.md": "ea416831fd7d0874e06d0a76fb0c7196df53072b717673a0cdf635db6bfd917b" + } + }, + "generation-cache-lifecycle": { + "header": { + "name": "generation-cache-lifecycle", + "description": "Trace MLX-LM prefill, generation batching, prompt-cache reuse and cooperative stopping when request state, cache rows or retained prefixes become inconsistent." + }, + "files": { + "SKILL.md": "d1a1c3c88e1a84d612aa04138de136e3495e62e91245d962bf0c9b6789359858", + "references/cache-stop.md": "bba6b5c73261065309cbcf11b1e591eef48916f0c79ceaaf74d98a9fa96b0992", + "references/generation-batch.md": "f16d3c57d45e63bcaffd3871aa41c76972ee11f7dbf1e12b45d9b170303cad9b", + "references/source-map.md": "91d26181f246307f2113f94b12ac9587d689b654dcd216e9f14b11cf5ebb650c", + "references/validation.md": "73b5dd347017baba519ecc85415aead9993c1751d116b9ba4f07689f58c5f783" + } + }, + "host-memory-management": { + "header": { + "name": "host-memory-management", + "description": "Audit pinned host buffers and asynchronous CUDA transfers: distinguish copy ordering, host mutation/reuse and device allocator lifetime; use bounded PyTorch source evidence, a reusable staging-slot example and a hardware validation recipe. Also route Ascend native descriptor retention, Python preprocessing and NUMA placement to their scoped sources." + }, + "files": { + "SKILL.md": "1406a94aaaed8a882a4fca2f78d3dc60e335bdb2e4ae8ce994ea952cf29a92e3", + "references/cpython-PSF-LICENSE.md": "33f1a470702f459c160751bf3188b3fa7462fc761a006e03e80ef3f0a7a01ed6", + "references/cpython-free-threading-readset.json": "46336623b3c573fad97d4c686d9836baab807adf10f4bea05352d7a46236e004", + "references/legacy-source-notes.md": "bb1c0fb7a4d1a0301ca9e699e733224f3dcf0ded3cfe13e612e5c04d48a8cab5", + "references/pytorch-LICENSE.md": "a8a2c6b67cf8a597ffaa5c82380a6bd1b82629c0417e00673d8013d662eb500a", + "references/pytorch-host-copy-readset.json": "881258c97edfee0a89f2392c2ff61cd7fe37ff8344f18620f281e841fb92cbc8", + "references/pytorch-host-copy.md": "fa9da5c39d37ebe44e5a336a7b88d7aece6a0f118ec47cb38271e29925e11117", + "references/vllm-ascend-LICENSE.md": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", + "references/vllm-ascend-native-offload-readset.json": "cc6c2011be746002ba3ff96ccefab1809863748ddd17eb102b75e7b0e134fe55", + "references/vllm-ascend-native-offload.md": "6c2a72b5ba3708a01d229e0576570837a617e6a42dbb0b5312a26a65a6c31542" + } + }, + "host-overhead-optimization": { + "header": { + "name": "host-overhead-optimization", + "description": "Profile and reduce CPU/host overhead in inference serving using line_profiler. Iterative profile-analyze-optimize-validate loop targeting Python execution in request preparation, scheduling, and model execution coordination.\n" + }, + "files": { + "SKILL.md": "8a004388fb04385f6da6f14b2b63b3925a2ae58657d03f41325319fc6db3d1da", + "references/deepinfer-zero-overhead-scheduling.md": "f452bf40132c9195d739f1c86b3aa18857138c7ed0f1861af5853d93b9955924" + } + }, + "host-overhead-profiling": { + "header": { + "name": "host-overhead-profiling", + "description": "Detect and diagnose CPU/host overhead in inference serving from nsys traces. Answers: is host overhead the bottleneck? What specific work is causing it? Covers GPU idle ratio, host-prep exposed ratio, NVTX breakdown, inter-kernel gap analysis, and multi-rank straggler detection.\n" + }, + "files": { + "SKILL.md": "c734c6be297e3069b20dfd8f10eea529be0ac77b750914411c1351b9137d24fc" + } + }, + "husky": { + "header": { + "name": "husky", + "description": "Review Husky's official-document claims as research hypotheses, separating author reports, implementation evidence, negative results, future directions and the missing artifacts needed for independent qualification.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "SKILL.md": "dcc95d498466d2591513cbe2845374fd415ea83286b59f2eb99e9d749e0113cd", + "references/author-report.md": "4c1351ca42b2535a7c03c4da87706040650ef10d86eadcdb496e881a147fc0f7", + "references/repo-provenance.md": "37de912e2e2a36b44233560e6a8368b994428bff587ed4f04b1cb5fbaba0a9e7", + "references/source-availability.json": "d3523e53c89d4aee5bfaa17db84501aa4c4eddd59bcee486df5d48d3aa48173d", + "references/troubleshooting.md": "3948711437d25822d389dd857c72dc7fcc15727c9048d2e3eae3317d97aca6dd", + "references/verification-protocol.md": "83c5a966ed3f721b412b8d8417af8336c5a8c1e46044a888720936bf2f1deeab", + "scripts/audit_evidence_card.py": "291e62701ef7a37a5bfab963ea4e20abb94245bc015584223a101f18b1490895" + } + }, + "inference-benchmark-design": { + "header": { + "name": "inference-benchmark-design", + "description": "Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image, video and VLA inference. Covers AIPerf AgentX/Exgentic replay, vLLM/SGLang benchmark boundaries, VLA action-chunk timing and quality, SLO-aware metrics and InferRSI integration." + }, + "files": { + "SKILL.md": "d2a78d46556eb49eb711e0ae842ddf248dd00966a3837489d7762f6a2abff641", + "references/autotunebench-source.json": "bc343ece2bf40c4e8aa4bd8d84f32465d00633255557feda5fe7799ce24e8688", + "references/autotunebench-trial-validity.md": "cb3a7a5d94d056f868af07f92a16a6fef31eafd3199ed07ae79e93798e320b1d", + "references/benchmark-commands.md": "0898424e0bff3a6246e1eeeda6abd6c5bd974c047fe32f3bbd5ab9d31ce17cd3", + "references/cache-aware-asr-stream-boundaries.md": "4e0ea9c1bf6b096a49a91ad5266e90b8a456f4f5aca15eba8ac270a53c2309b5", + "references/dependency-aware-tts-playout.md": "e94866e718adc3bd7e272a4856e9a90b6f27cbc0ff3a44101e68e6995ed79e2c", + "references/embodied-cpp-boundary-and-quality.md": "0bc3ca87159cce960c7b32f62b82c275f850b98077ae07da0481bd554a3cd71d", + "references/embodied-cpp-source.json": "056985be689554570a9f12b8e228ce71bdca5be940f9a2d7f6e1963e15a48130", + "references/faster-cosyvoice-source.json": "9efe8918208adddfb9a31d2abed080923a5ae0dfb6332438179af0de66f54e13", + "references/gpu-measurement-validity.md": "af8808e4ffab71eb285750630f76a54f910b98ae62367c03808d9f9716c071a1", + "references/justgoal-evaluation-contracts.md": "d25df795fd301e8a1bb436e5f8122d0bac2441c43104e59a3b310fd5a64f2f3e", + "references/justgoal-source.json": "60f2784384ea1a3b08a2ecafb60207d8e3707247dc7bc177b95b463d81f1cc16", + "references/ninfer-source.json": "ccda0fc6781e3c4dceb41521ab952445840b4afcc9f5a7d968857f76e8db6272", + "references/ninfer-state-and-completion-contracts.md": "3be946daf69d0b56f4fa56ccbc301dacb076473169f4921376a100dc64b57caf", + "references/task-benchmark-selection.md": "9866457c245ca9f351e8c0f3688dda688ebb1a08c25a1e9665ff1a3bbde12007", + "references/xeforge-source.json": "6c21791b9c8ecbc9645c6deb659bd897e17f70525536ea25e884a62fc5fb6b25", + "references/xeforge-trial-evidence.md": "802c4dfe83983cbd6740231d5d0e2b459792b685de8dfbefb6dc51a912e23604" + } + }, + "inference-optimization-casebook": { + "header": { + "name": "inference-optimization-casebook", + "description": "How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook pattern from TensorRT-LLM's perf-optimization-casebook skill. Use when building a record of optimization experiments or when deciding which optimization to try next based on prior art.\n" + }, + "files": { + "SKILL.md": "cec75f3a721dd13e6913b597d5373833a14c39d9048951a30c4864dd2c49aac8", + "cases/admission-drain-occupancy.json": "4f8544292b2cc3391f4d4d75994cb81fdd295eff99e55b7e60374c6c122de09e", + "cases/admission-drain-occupancy.md": "e88bbac5d253a8e7e3a3d5e21047f23f2970c11ac23887d4303ab94fca52dd27", + "cases/drama3-rejected-directions.json": "a2dc5db8aba62081c5743b12f854e74ae500c53bd57e2486260d0b5ae478638a", + "cases/drama3-rejected-directions.md": "6f143bc363a3290fc008c08e57cb075f1776d1c8cca8a6a59297d5b1fbbf61bf", + "cases/qwen-ascend-rejected-directions.json": "b2c1808919ae525efb82148a24b465459799422fefc6d420891d7dbeb36cd31a", + "cases/qwen-ascend-rejected-directions.md": "be396983812262515899a3e2ba4f01b23ff92e1b836881882c7580372b8ae552", + "cases/qwen-b300-rejected-directions.json": "90e3cdca085e2d122e648aae527bd6a61de6d4fddb66c393ca25ac4541cd9122", + "cases/qwen-b300-rejected-directions.md": "cce2930f544a0f78f21797fba4d8ded298e4a81f76c34d7977b0e31d361bc738" + } + }, + "inference-task-router": { + "header": { + "name": "inference-task-router", + "description": "Select the starting route for day-0 model support, inference performance optimization, debugging, accuracy repair, version alignment, derived-engine development or independent runtime bring-up." + }, + "files": { + "SKILL.md": "3dae791f61b26810332e05fbabe6fd7113abf9ed8609aef663535a267525849c", + "references/routes/accuracy.json": "5a672842ec26262d8827f7ce8d92de9bd956859ff5244189c0519b8f0dd188e3", + "references/routes/day0-model.json": "77b26d1300e5535d8b595cef95c6c803ea1322e92f38b68ae2905f69355a814f", + "references/routes/debug.json": "75241e43d12fb961d3f48d1b683e4df321c0f18e683959aedca9722e78a8b78f", + "references/routes/derived-engine.json": "3ab030bcd04db76781d20dccb27b62963b4fd996e05edfb0c38e2680018d8222", + "references/routes/independent-engine.json": "f2208c40871306fe4be21f84a099c551013eda26e55bff02d95ba60739b8fade", + "references/routes/performance.json": "4904e6b403f8390ddc6d11c962491ec42683ec7e29eb4a2be4075bb052bd31fa", + "references/routes/version-align.json": "4243e305578e50ddc88bf4465447e716bbe9d22e4a0122189ad58a50e2267bf1" + } + }, + "kernel-writing-guide": { + "header": { + "name": "kernel-writing-guide", + "description": "Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL, CUDA C++, AscendC/TileIR), universal correctness rules that apply regardless of framework, and the verify-before-benchmark workflow. Use before starting any kernel implementation to avoid common correctness and performance pitfalls.\n" + }, + "files": { + "SKILL.md": "c6a4ef16ca5b0daf3b69221788eceafac33709a063a0885cd2e3dff315c81643", + "cases/causal-conv1d-dispatch-state.md": "2eb67dd522021bb2234616d7f660e6ba99acb43676f48cb23141bc69bf3e0029", + "cases/hf-qwen35-gdn-binding.md": "b3a55af3f45f01c48e1cb18c8b029e5949a494c3e51dc444a5cccbad9acbba0a", + "cases/sglang-gdn-source-contract.json": "5237ac437eb76e910bb1d08bfcc364b87dea1b947de5e67eebce511ea1ef1d23", + "cases/sglang-gdn-source-contract.md": "8b093faf0ab775d5605ecdd9653fb3689c2b839cba2641c5d5cda17c721bb2ac", + "cases/vllm-gdn-causal-conv-contract.md": "2f3dcee66e2dfe7a98bee21a36b892538998de5de3caf1bc63257137f9984236", + "cases/vllm-gdn-source-contract.json": "31cfac1e780113c8233b451af7a7562af983aa2f4209938017cc67347ee5dd66", + "cases/vllm-gdn-source-contract.md": "7f9ebe8de7e55d3db2a119b6a56e15c37448fd7d4e022a47122a143206d31687", + "references/akg-affine-tail-guards.md": "113deb2db10b171db3425bb3f1792bee763706b794c8cc2e222b4cc0d0114118", + "references/causal-conv1d-contract.md": "b3a6245cf5e9deeba0dffd4a15ddcf97da6a4315f4d3131921779438832663ad", + "references/hf-qwen35-gdn-binding.md": "848d3c99b47ef711e8820bbc4f6bcbd1db7211197811c9d855201025448d806b", + "references/hf-qwen35-gdn-source.json": "df3f4650b5cf92d4fd3d5a4cc8aefb91dd4b0c2affaa0c873138d5651c50e6bd", + "references/licenses/sglang-LICENSE.md": "1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747", + "references/pplx-gdn-caller-context.json": "0219fc9ad785da4e638c1680bc9eaf28672aa32313824f52bdcaed61d29bca87", + "references/sglang-gdn-causal-conv-contract.md": "c9f27b0231ddec3d14d6fa0831d1494ffd8e11bd8ad704e04be3bc4a4ffc0e58", + "references/sglang-gdn-source-files.json": "4c7ec2cfd2c29297e185ce09b4d9b2040fbe3635943c112f9ebb71dca8ab381b", + "references/vllm-gdn-causal-conv-contract.md": "eec09a6e95983c46b67d5f066d34419c17819899f5d8e3c65f2f163f2ada0fde" + } + }, + "kv-tier-management": { + "header": { + "name": "kv-tier-management", + "description": "Manage runtime KV cache tiers (GPU → CPU RAM → local disk) for LLM serving: decide when KV is stored, retained, prefetched, or evicted; size the concurrency-versus-RAM-pressure trade-off with an explicit in-flight admission dial; order RAM-release levers; and measure with token-level L1/L2 hit rates and prefetch-pressure counters. Use when configuring KV offload for a serving stack. Covers runtime KV offload only — model startup/loading, checkpoint save/restore, and block-device profiling are separate concerns (see weight-loading-optimization).\n" + }, + "files": { + "SKILL.md": "e6938f20791e374c016553e2ede729e65f56f8fd28b32894df6cb8248d708217" + } + }, + "lazy-evaluation-stream-lifetimes": { + "header": { + "name": "lazy-evaluation-stream-lifetimes", + "description": "Diagnose MLX lazy evaluation, cross-stream ordering and array-buffer lifetimes when timing, memory use or asynchronous results disagree with expectations." + }, + "files": { + "SKILL.md": "2545b8694a7b1a2ad30f1c95529b177b4117695e0b55fd748c3599975561f19a", + "references/execution-lifetime.md": "c4bc41e08aca68f661325f61e6f0db1760a079dc362f295c847bdc374fee3bf6", + "references/source-map.md": "798b9cb89ed6b885eb47464ca9f137d2802a54571b9327e631d7742d3e5abd5d", + "references/validation.md": "ebfc00bf7d6ab0a1dc415953ae84d56ccdc713d5614aff85e433c4a843793bff" + } + }, + "lily": { + "header": { + "name": "lily", + "description": "Investigate Lily's exact Qwen3.6-35B-A3B MLX-affine-Q4 contract, Metal state and kernels, strict greedy text API, token-prefix session cache and source-qualified validation or benchmark plans.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "LICENSE": "b9e2b50a48a09fb39dafa71ab1a11a6e87389addda17faa99d55f5f74a0001ac", + "NOTICE.md": "7a01e666044fb8aca1fd88c636bca43bcc95b8b1fecbe101dce4c7c80fbb5759", + "SKILL.md": "d24b7a1367949b8724179587b8cb94ba7d40511e998de14e4f8307f77dd2b2da", + "references/repo-provenance.md": "96182f6cb3666f89adf1aed6e216c0a84ce3a04a0685a4257e286100ba8ee367", + "references/repo-routing-metadata.json": "f2f3becb51056e03081c7c4140e19572a6318c3d63015884df56eb49d38a00fc", + "references/source-identity.json": "4595559c11611c855be150919fca3eeb89e7f96c7a8dd133b9f79a5cd8e05e62", + "sub-skills/model-runtime/SKILL.md": "dc800f07fa0e82213483c7b848dea95d609386296006bfec7a34cbb194a83de6", + "sub-skills/model-runtime/references/model-and-memory.md": "d5a862303d646ebbaf745818caed8f96c7323cd442cb6d6e2aae00b0361080a1", + "sub-skills/model-runtime/references/troubleshooting.md": "b32fab02b561c3ab4e683abdfa80cd4a8f929c514147709febaa6b6d1df65909", + "sub-skills/model-runtime/scripts/screen_config.py": "5961bb6f31ceca43473683127b37faee268234040a013ae49bbb4898eacb6f04", + "sub-skills/qualification/SKILL.md": "0231b50b9bebb68d0fc5f2ce19aa7c6e5ffd498d8eef4cc00d39e82908cbbe91", + "sub-skills/qualification/references/troubleshooting.md": "01607534ce1f09fb046255c78f8c1c436aac61874b93b65a9d62f3eb480496c2", + "sub-skills/qualification/references/verification-contract.md": "6934779f67b878bf7cdb5f2cbe461f6716c243c662a29272c6db37f64cb317c7", + "sub-skills/qualification/scripts/audit_benchmark_record.py": "59534f58d46ad438c7bf88886a9b05a8716c922d817f2d9c58ff1b71ae50ee85", + "sub-skills/serving-cache/SKILL.md": "7e062ff9af76125ef08c98b4465e0439c66e9e237f0c39ea0285005cc5496cfb", + "sub-skills/serving-cache/references/api-and-session.md": "7262bb077c24f2f7549d450473ec7944829940d09cca89db9c12f37ac90d5db0", + "sub-skills/serving-cache/references/troubleshooting.md": "6e2f36d2f7ab9eae665d87a676e34c0ebd2f46f3df6fa5f2e589a306749a87b5", + "sub-skills/serving-cache/scripts/request_template.py": "d6b00c9f0b38c7fb0a0e52e396993a832542ea0aa6e2456026fabc0c39a4d33a" + } + }, + "lithos-metal": { + "header": { + "name": "lithos-metal", + "description": "Inspect Lithos Metal compilation and fusion, GPU-resident generation and DSpark state, or local serving and exact-prefix caches on Apple silicon; use when changing recipes, diagnosing stale state, or qualifying an inference claim." + }, + "files": { + "LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", + "SKILL.md": "ec4d44acecb314008872ba587d0c05ec98edaac11b75805f33b2ef3126c0b66f", + "references/mlx-license.md": "ccfab7ccb2ea306f71531c8ca77bb55507606cd90768b1e32b8b52ab5b48cf01", + "references/notices.md": "b2c9d2481ef2c9d16b3240c951ae65db417e5d309cd06b7917db9d84fe175886", + "references/package-api.md": "4f3ecfabafb2a15eddf21ccf964b8c37635bbb7862418a2c404e7049c0fa2898", + "references/repo-provenance.md": "c0c7c3dd45eab0d55bcf154760156db58e8e365f9cfb2c6bd246dd6a06f3ef3c", + "references/source-notes.md": "110dbc995c7fc727632017d4c26c71df7ecf12d4df5d89df0cd4afd75c5892f2", + "references/troubleshooting.md": "1894bbe1a18018f81318b11e2150201ee97f3819f84a7a81f3cf5fb01549c165", + "references/upstream-notice.md": "cd80bddd6b6204dbc61c6ac762ed206c73d9264755d9884383d0860de29d185d", + "scripts/inspect_contract.py": "f8365f87af2135fcf0a2c8baaefac563c30989282fe080a02d958a37a215515c", + "sub-skills/compile-fusion/SKILL.md": "ad07dc883a708bc3df29158c8a408ab60f2b89d783325cd67604f9342e40e975", + "sub-skills/compile-fusion/references/mechanisms.md": "6173a8b73ec5138cb12791f108f3d5d83278a117d443d48187c842e16bdc6eff", + "sub-skills/compile-fusion/references/troubleshooting.md": "f18ca9324d9a80f2ba327a1ad4ccf84a01049080366fe12410f2800d5f178232", + "sub-skills/generation-state/SKILL.md": "c244ebcc61f821c38ed3099bd49442705dccae0512be9471eefe1df5f1c2c2fb", + "sub-skills/generation-state/references/state-machine.md": "d812ad31d4e343539119cf82f8ce241fe14c691dd73398048893cc5de3ab8e22", + "sub-skills/generation-state/references/troubleshooting.md": "7ca629e66ef3b0a67077a7d2536befef683315ad01939728a9e6e7523cc8414c", + "sub-skills/serving-cache/SKILL.md": "26b78f2d30ee8393d025ceba39d39f9ffe9cf113711f23d52de83f3543a12174", + "sub-skills/serving-cache/references/troubleshooting.md": "52c6e27a76547475b5c66121f873f4d2c1662c811a69da64ce85614f1bd44749", + "sub-skills/serving-cache/references/workflows.md": "298d27d3c748fe22774b872ed0465b707e53812379a7a538c95f25d1727fc796" + } + }, + "local-dream": { + "header": { + "name": "local-dream", + "description": "Audit Local Dream Android desired-versus-serving process state, remote exposure, native backend and DiT packaging/ABI selection, shared generation cancellation and prompt-cache identity without mistaking declarations or source tests for on-device diffusion qualification.", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "SKILL.md": "0c8bfb62bd75363ef8e49391168657622bcda8fc3ac757c6b85b7ab26b36b720", + "references/android-lifecycle.md": "f85e5fce39f5de5f88548fd8c21978240c810f899e03ff05ed72eb3508ee5e36", + "references/native-contracts.md": "40206892f5e421f9d8edf303870f5c4aefa3e87123537178d45f438f3d04b7ce", + "references/packaging-and-abi.md": "9a19bca59e19c3bfbfc1a15f5fc756c13d954ace367812aeb000551f4484ad06", + "references/repo-provenance.md": "0b848082c0ce893d8ae26d779c5b663d914d1a208cb2293c4a84668cdde86293", + "references/source-identity.json": "b85f028bb96a37852e9d9780b5d5052fdd9c8f3a704ad391b6381adb58291347", + "references/upstream-license.md": "41003d4a74749c0220e33dd415042164b5a1093ed401f36277234f772d22d3d0", + "scripts/screen_declarations.py": "ccd7d0d55be9a47484ec3a74498d8edef8aba55287f92287775c8d82050c8932" + } + }, + "mistral-rs": { + "header": { + "name": "mistral-rs", + "description": "Investigate mistral.rs Metal paged-attention decode layouts, prefill coordinates and workspace, actual-model prewarm and native test evidence without treating static metadata or prepared pipelines as executed GPU correctness or performance.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "LICENSE": "b40bb09c7b3b6f697a8920cf9d2cb3ff8cfcf77a0fcb6d264a92842b43c4dcae", + "SKILL.md": "cab0196b8447db6ca5704cfeaa4563d1a7a30d46ca2d931f8e1903e5bff96bbd", + "references/apache-component-license.md": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", + "references/paged-component-notice.md": "a1718d8c5a25f15e61bea98a158880e96faa5b7174e82c4e1470fbacb23a3559", + "references/repo-provenance.md": "fab87dacbe0a7298e42137cab7409f443ece0dfefba60961040031e36d96adc8", + "references/repo-routing-metadata.json": "0debda77a8f6b1b1c06176e3c2447767c1953b9fe7e4c84ef01b3ac90735df70", + "references/scope-and-notices.md": "938203995f5e58cd6cefa278225b2d5a3ea7dc7f07cae86201f9fc71553ac020", + "references/source-identity.json": "6a8c0b0834228b3a298e7e548c9d8cb774c00f661135f51c12ee613efe1decf3", + "sub-skills/decode-contract/SKILL.md": "73620deb4efad4681da10a07bb982e180423bd69fec3572d16ab556c530b89ec", + "sub-skills/decode-contract/references/helper-contract.md": "802d8af65343eeb29b0950d3c305d0f24cb7a4298628ade5f7cdcf4358c9e228", + "sub-skills/decode-contract/references/layout-and-features.md": "6804a771ed0dd2d8d60613d52ef87d4297af2a6445b4fe61d08adeab4be331aa", + "sub-skills/decode-contract/references/troubleshooting.md": "a6de920f174d6918a72093921d344c700108b0c01e1f0d09f2b9051800b1fa7d", + "sub-skills/decode-contract/references/workspace-and-evidence.md": "7ac5028be6fd48f9c6cf067bf832efc8edd681195f2fece67ea119211bdac106", + "sub-skills/decode-contract/scripts/audit_decode_metadata.py": "de28f586f27278fee25f6c96f1e00ee63528cdf746f57c83bdf367c31c0f6e6b", + "sub-skills/prefill-contract/SKILL.md": "21015790789933e106f6ea929fd1778d862315520b55425213c679e9b7a44e95", + "sub-skills/prefill-contract/references/helper-contract.md": "607449105bf3bcbcfe785dc9448a7a587c7daeb528ca63e0cd959b074a9bc409", + "sub-skills/prefill-contract/references/prefill-and-metadata.md": "e8a738a668a23068b8ff1907d521a3a26f7090e7a791f3a715efdb6001338dc0", + "sub-skills/prefill-contract/references/troubleshooting.md": "f584a02528e2249f5d12f223d8933e9d40212e5aba8dbe545261164b00ed6f8b", + "sub-skills/prefill-contract/scripts/screen_prefill_metadata.py": "769366c19d621ec1b42c722ed224fa154a63c3410a0cffc9072a13abd9c71bbd", + "sub-skills/qualification/SKILL.md": "745d88dbf6c764f64f8b31763e57bef720788830ddf17b518456323000325fbb", + "sub-skills/qualification/references/prewarm-and-evidence.md": "6b72ae4b4599018ce2da5c54d0c28fadcafb7ce75b81ce4c18c06a440aab6056", + "sub-skills/qualification/references/troubleshooting.md": "d933bea6e9e493faa8cd0f4103ea33197ace7f7f901359f8cf0d6bab2edcdb4d", + "sub-skills/qualification/references/warmup-inventory.md": "1e7af7d63c6853ede4232b5a12cf68aba6597c4a305015366f1c2c917e3caf01", + "sub-skills/qualification/scripts/warmup_shape_inventory.py": "6aa920ca1cb534dec41f5dd2d833c3e760b234ab2c6439b6c4250b8a30b74055" + } + }, + "mlc-llm": { + "header": { + "name": "mlc-llm", + "description": "Investigate MLC LLM compiled Metal artifact/schema, Apple target and pinned TVM module/storage/stream contracts, and compile/runtime KV limits without confusing canonical metadata, codegen or automatic estimates with runnable native model qualification.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", + "SKILL.md": "c0a3f0a56d02615ac19627aa3e78c9587ca9194e38f4c16303bd9c54af4ee051", + "references/mlc-notice.md": "3a7dfb37bb96ce6f5a1cc0d6d4c8da3b8a35e6ec4692647ba0a37f6155c7e694", + "references/repo-provenance.md": "423f21e1de6d7ab2e11c0bdf540357efde025ae4535bdf76ed9770b1a59ba35a", + "references/repo-routing-metadata.json": "5309c27e7b60c4fe7aaef28bf6911e7e3c936b58b1a907aa8486bf0e40b4c0d0", + "references/scope-and-notices.md": "8c573c234f44a0808a731774921fb5228fc0844cd238496a2c4d61d0120bdc79", + "references/source-identity.json": "75e91b0f7fdac12466e62eedfb49baa6f1b3016111d7479aa2b04e5c799454b2", + "references/tvm-license.md": "e62264b483aca7a9f8c1ac69388b476971b43975ced90ac11dbd9e9c530edb46", + "references/tvm-notice.md": "967cd2fb97712c43271e2b1c4f636c4d8823898869f0f76f65bf8a3232584f0f", + "sub-skills/artifact-contract/SKILL.md": "933fdd05ca853f47e3ace024cf1e08df7511834c31676851c28803b6dcb1c681", + "sub-skills/artifact-contract/references/contract-mechanisms.md": "82187f4a1478352c68a5690101e9798973ef63f9b6d717e06188b03a1832bb8c", + "sub-skills/artifact-contract/references/helper-contract.md": "48492a36435ff1340a4db4e0959f052cd72683ced26bb8ab35baa84bd79ccd7b", + "sub-skills/artifact-contract/references/troubleshooting.md": "41b002a31b031a78ca372504de4c9cccc9353d6d80d989b8d62e56260c291e2e", + "sub-skills/artifact-contract/scripts/audit_artifact_declarations.py": "874a16cb16d75a14cb51b4fbe442f4fac46feb21ac242063f5f845a0fcbabaa4", + "sub-skills/metal-build-runtime/SKILL.md": "25dbd48f4fe84b7d759005d04a2aa9db33da987cafac7975a148e62cc9efae34", + "sub-skills/metal-build-runtime/references/helper-contract.md": "5428efd6dca0654a58e9e226805e15c9a9f1b3f1ef60db234139985d107acbfa", + "sub-skills/metal-build-runtime/references/modules-and-streams.md": "56faaf149f95ee821c242ccca8115a6f3b166d34392499798ae8532fcd10372d", + "sub-skills/metal-build-runtime/references/native-evidence.md": "133322e879fd9010a7513afd278d937838aa2ec97d593ed11d324cc59793374e", + "sub-skills/metal-build-runtime/references/targets-and-export.md": "547eb8e56af4f7b58415eb58c338ea1a33ff8edf8b58fa523700947dc7e04821", + "sub-skills/metal-build-runtime/references/troubleshooting.md": "e392b8329ed183bba5fd79fbd260c3fbfb389016ee71880a957741e71a0d59c0", + "sub-skills/metal-build-runtime/scripts/audit_apple_target_declarations.py": "d5bf16f7def4cfe28b143a700ca45a9636ddaeac12ead04cbaecdcc8b4b71aa5", + "sub-skills/runtime-kv-limits/SKILL.md": "6b4914cb60831c07d4c748162251c49fc16b229e0d62b4c5a5663719c45b16de", + "sub-skills/runtime-kv-limits/references/cache-lowering-and-evidence.md": "a118f5bc584ea885e8c573978eaf1e32d448910860b8135196c5c1acd42aee05", + "sub-skills/runtime-kv-limits/references/helper-contract.md": "da33306472cbda5a33605b80fe52d6cb1fa886b3b7eea13adbdf0ce8a6024c66", + "sub-skills/runtime-kv-limits/references/limits-and-workspace.md": "77105db95de2b8ea27ed6953b5441c8af061fb745ddb13d3904f8773fc21e879", + "sub-skills/runtime-kv-limits/references/troubleshooting.md": "b89255fc04507185bc8bcf74de779730b009b12d13bb38ad0193205dfed8e921", + "sub-skills/runtime-kv-limits/scripts/audit_kv_limit_declarations.py": "d82a6dda5f5931eb369003a5e1f419bfbdd82304766a18be8ea1e1849e26b77d" + } + }, + "mlx-swift-lm": { + "header": { + "name": "mlx-swift-lm", + "description": "Investigate MLX Swift LM generation ownership, cancellation and settled task completion, and recurrent GatedDelta Metal versus ops contracts without confusing source predicates, final info or offline metadata with executed native correctness.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "LICENSE": "7b1d86acac816ce8192209d7e01141c129b4d48d474c15cc5fb4af62f497c323", + "SKILL.md": "3c35893e685d020085fba738a43e3dc958fec36ed440db2f85355ce165364228", + "references/repo-provenance.md": "f1b15edf89c597db0c3fc0bff7db6b2451924ce9273c39623bc0cbdccb568d31", + "references/repo-routing-metadata.json": "8fc804347771cd768a7ed027f2041d96c9832fe5f8c5869e6593a3021b1bef8b", + "references/scope-and-notices.md": "18e56805a8f275d35af3d8e42f3aa1525871b43400921a5d2dca4897c2047214", + "references/source-identity.json": "518da7bb64f22ff8ee6beb0e0f0579e7cd0da2fd70fa69fd8288294b0c08ba24", + "sub-skills/gated-delta-contract/SKILL.md": "04b97959b4e207380a06d932e9a557954a9fb9aa880c243439da6d4a981854ed", + "sub-skills/gated-delta-contract/references/helper-contract.md": "1358531257e00c0f9fb2eb9fb7a40f9d749f5435c90168c8923b6646353dce6f", + "sub-skills/gated-delta-contract/references/recurrence-and-state.md": "ff0ebbeacb8219d93a890418d18bdf2856e6326d67f974a7050de3a0e5a3c5a3", + "sub-skills/gated-delta-contract/references/troubleshooting.md": "3bbecbf91b241e7b37b3e8c753bf3fbcbdbba0b0f4d0ddf2f65eb653007f6cff", + "sub-skills/gated-delta-contract/references/verification-boundaries.md": "21b6fceb692f29b410dc967f82e31d0d082904cea827443ffc05f6f940191dfe", + "sub-skills/gated-delta-contract/scripts/audit_gated_delta_declarations.py": "3a256c37c867866e49dfdc2893cbb3f32d061e389fe4395555eff899bc0415ff", + "sub-skills/generation-lifecycle/SKILL.md": "7effaa20ddaf32e3b4008c95888ccd09bc2bbc3bb5ff17c710f989b36be49535", + "sub-skills/generation-lifecycle/references/evidence-schema.md": "d40e44910d8fc43ee3c731735902b1c1ba506c2ae972bb53dbacfaf37e38509e", + "sub-skills/generation-lifecycle/references/native-evidence.md": "3e9a630a30932989faea5f52b347318cb94c73c7d4d6c81c0465ccaee2818671", + "sub-skills/generation-lifecycle/references/ownership-and-termination.md": "56fdec7ebc89b5c217c7e5c203886bef1d3c2eb5794852897bdb5941ab1db2a0", + "sub-skills/generation-lifecycle/references/troubleshooting.md": "de28b8ca9cf95be2a4e5762f938acfadb6d4b53459dbf86df4b2b2ec64ec347f", + "sub-skills/generation-lifecycle/scripts/audit_generation_evidence.py": "2aba6caf42ca1b3fc55299fd96419af8681c37750c21a7ba75205a43baa9c7b1" + } + }, + "npu-skill-factory": { + "header": { + "name": "npu-skill-factory", + "description": "Design and operation of the InferRSI NPU skill factory: artifact lifecycle, skill evaluation contract, corpus selection, hide-and-re-distill procedure, and skill evolution cycle. Use when deciding where new skill artifacts belong, how to distill a candidate skill, or how to run the No-Skill vs Human-Skill vs Distilled-Skill comparison.\n" + }, + "files": { + "SKILL.md": "0de28180da76b6cb9d2c9456733011d00d9801f7662afd32959c4c5d2c05a183", + "references/papers-20261015/COMPARISON.md": "38587678a96581f5c2dc697d982127343565504bffa191352811dffcc165aa23", + "references/papers-20261015/INTEGRATION_NOTES.md": "fbc4a96b6f6c477706eed83eaa4fcc29a14f881637629d6c004601ed049f2523", + "references/papers-20261015/QA.md": "3f3785ec39a6938a5d1f2e874e99153fe92c0e666dd3e78a499c8b47d891346a", + "references/papers-20261015/README.md": "221785e91c45588f82b3caab6d52f1ea1d102b16139c20a0802921e2789e6d28", + "references/papers-20261015/UNREAD_LEADS.md": "6c1d87ebe491d27666703149eb05ad582c7e5aff77f0ac5c11a7b89c946f96ee", + "references/papers-20261015/cases/methodology-counterexamples.md": "f9fc21753ea41eef77ca020d78d031e945e5dd835b5d09c5ff0e53749ceb2ca4", + "references/papers-20261015/knowledge/evidence-routing.md": "406d5d74c3fef4c1a23d495492e21a084c3c46495a1a84b385703ed104358874", + "references/papers-20261015/references/learning-to-commit.md": "9c2d1d738e896d05c00f53e6adf14c9df2501ecc3f0020e71b5a1c2192547086", + "references/papers-20261015/references/protocol-details.md": "41298c6bde3a28f0712fa06a18bbc4ac59aea6d17cc9eb1318744f4121e3bab6", + "references/papers-20261015/references/readset.md": "6f953d20d8958c68339c9ceeae348604d0fea3e9c6ccb9b64a63d3b210426a22", + "references/papers-20261015/references/repo-to-skill.md": "5ebdac2a2bbbe6614f9d4096bc9895bd685acdf76414e2c3c1fb7b50d57c738d", + "references/papers-20261015/references/skillopt.md": "148075daa67e19f4b4351710498438d17c279a9bce56b2d01cab26ff3be33e99", + "references/papers-20261015/references/source-manifest.json": "716cd49ed01824ee9f6b6fc55ebf2b24fdd5d5a3e677f6e28e34a472616bbe25", + "references/papers-20261015/references/trace2skill.md": "230d13b77ba819b5e3edee147523e8cad6bfce95afccc140fb4946d6c8747a2e", + "references/papers-20261015/updates/bounded-updates.md": "529c363e9c678f33f7c0be0420d26c64d056321f041726362dba69e53e1e375e", + "references/skill-factory-design-20261014.md": "0b2e05bb6a4a3b3f6df0b04abc40ec125be60714b87f609a390246ee2a292d2d" + } + }, + "optimization-search-policy": { + "header": { + "name": "optimization-search-policy", + "description": "Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop.\n" + }, + "files": { + "SKILL.md": "1ad13f1f5dbef1a25810d1f77a3c79c6dbce85683af6796cc9ddd708bfb6c470", + "cases/sarathi-chunk-budget.json": "681871983ec4033ef9861ad270bc0d312f9134a76933b8f7ce740659ef1e4dd5", + "cases/sarathi-chunk-budget.md": "2c2a8147001d24d1abc8672f15cf0ca759ae461b734413c8dcb7497634bb6821", + "references/distserve-phase-placement/README.md": "56e16efce447dfd4990a024ff1d6f7cb428f8e3e1f7b32a7e4d05f57139b836a", + "references/distserve-phase-placement/source-record.json": "21cc9562f1c19d6b57957c01b53e0adf8700d05ffaaa7764afe07c4da1284de1", + "references/papers-20261015/INTEGRATION_NOTES.md": "157e74ba180426b279b548a729769f152bf18f9a78ffca4ebfcaf26b7d0c5292", + "references/papers-20261015/MECHANISMS.md": "18bb2f5d7e52773c516420a142edfb531d0b9c822a2934d60f3234ac1cb9d40e", + "references/papers-20261015/NEXT_GATE_LEADS.md": "16a264c7159f9c2c700ce8ba46e63f3e636b277c987da4970f6d2fa085fcb053", + "references/papers-20261015/knowledge/adaptation-interfaces.md": "5e761957cb5ecd93926e6ffc13deee7c830c6e8f0ee3c3846e9dd3b50dc74446", + "references/papers-20261015/references/source-notes.md": "54d83c3713045d59a48821bcf93598374ba4a630e3891ad39498f919317c4fb8", + "references/papers-20261015/updates/optimization-search-policy.md": "d3db44fa9a6831b86a02c638b323ed943e69fe256866213f10d68c72d762879e" + } + }, + "os-process-placement": { + "header": { + "name": "os-process-placement", + "description": "Place and configure inference/training processes at the OS level before measuring or optimizing: check then pin CPU affinity to the accelerator-local NUMA node (taskset/numactl), keep affinity constant across measurement rounds, set thread-count environment variables (OMP_NUM_THREADS, OMP_PROC_BIND, CPU_AFFINITY_CONF) per scenario, run a device-busy preflight (npu-smi/nvidia-smi usages, graceful-then-force PID cleanup), and structure environment checks as ERROR vs WARNING with per-failure remediation. Use before any performance measurement on shared multi-socket or multi-tenant hosts. Does not cover cgroups, cpuset, isolcpus, or scheduler priorities — no source evidence for those.\n" + }, + "files": { + "SKILL.md": "1978f4eb9117b8c97225eeec1809e3b693eec9c0186847929bc1a71f86b023b8", + "references/llama-numa-thread-readset.md": "748b254b3b043a5ddb0f480eb4a566854d27eedd7b111499863e9bf0a5a6951e" + } + }, + "performance-ceiling-analysis": { + "header": { + "name": "performance-ceiling-analysis", + "description": "Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical, empirical component, end-to-end) and a prioritized measurement plan. Use to formulate acceptance targets when the user has not supplied them, assess a requested speedup, or identify the binding bottleneck.\n" + }, + "files": { + "SKILL.md": "23ba1b652c26812001f9b9a72af63fa0f03588d983b53445a4bd5ff9e5f816c8", + "references/acceptance-criteria.md": "e96efc42005af591a87b57dd6f1e9bffa8e351bcd023bd093ac37239a0486638", + "references/flexgen-offload-feasibility.md": "5c72ae959c6cc72aa9e96a17a2934d6bfcfb055b25fda3221c99382dc09910d9", + "references/huggingarch-checkpoint-anchor.md": "ca03747be29a93b579849ee361b67632e541218a68310e0f3369868c6a45da7e", + "references/huggingarch-source-identity.json": "c4c000cf25e990d5fedbfaa206c9ee1e48429da50251c0553975ac2ff3efe0b5", + "references/mooncake-prefix-reuse-feasibility.md": "6df0843a5fbe7cc6b05e58835b65976db5ce9fff526e9c7ed9a9d70c021c71bc", + "references/papers-20261015/INTEGRATION_NOTES.md": "083e462e0a204dce599362d46712fcfb392d053c5cda56216e22e28d4b8fa0fa", + "references/papers-20261015/MECHANISMS.md": "742c249082ceca230326f4407a40130fe624553b796ae5a157a8a4c15a7e1398", + "references/papers-20261015/README.md": "17b2d1c7c14124d59ac8a0d5ad4200ad2803d991e784a1a98cd97188cc364999", + "references/papers-20261015/READSET.md": "66459ce393858b2b2160f1e2973373bfc4b9902a12644631934b4ebe3140296a", + "references/papers-20261015/STATIC_CHECKS.md": "cb0aa7d148352ebb7b1678fc42a557a099575054716a341dfa25b6a998029064", + "references/papers-20261015/cases/decision-cases.md": "781a675a6533e0a18c044aaaee0d010da7013b6504654b8b8292a35176db801e", + "references/papers-20261015/govern-optimization-experiments/procedure.md": "ce036a4d8a0908da204515c47e2ff9623548d09de264fa65bd58fedce52a78a0", + "references/papers-20261015/govern-optimization-experiments/references/source-notes.md": "f9a8254e45473ddb2b73e576957b6d3db83c62b99dff1922f4d1e606e569559e", + "references/papers-20261015/knowledge/bounds-predictions-and-decisions.md": "5780d6eb5257612ec45deb94f85f334a6ecabf7e466d1678706041f6549bda79", + "references/papers-20261015/redirect-cross-layer-optimization/procedure.md": "6593a63fce9c474b515663a560c94b7df048dc5f6e77a95b8e54e4a64cd10a09", + "references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md": "6ff3a7bc31f1172db92aedaf7df1f1aac688bf769227b583a4cd82bfad52da67", + "references/papers-20261015/references/agentic-kernel.md": "7190573b85f94a4391bb0d887ccbd37a31b3f08bb0531114ad7afd72fe1567a7", + "references/papers-20261015/references/flashvector.md": "3926369acd88ff3c370bee5718e74dd15f3219c3e4f6311095e30e917bffa4cf", + "references/papers-20261015/references/inferencebench.md": "21f47111197a0b7f8d6384ea918d139dc8318a62ca3313ce060048271f50b639", + "references/papers-20261015/references/prior-coverage.md": "35d47ac078591221fcb666219bcd824f9400fc17aafecdc0499d8336f7178238", + "references/papers-20261015/references/vidur.md": "d38602af5219f5feeb49e6cd6bf53d66098c77fad141dae676f1d8a6743cf25d", + "references/roofline-contract.md": "419ec4bcaec45ac579a133bce083d4f48779f26e664d7a86f720fa65f343b1d1", + "references/roofline-example.json": "597cd3205df043cff3ae9d15924c77934e8d8eb300c00c4840f6bb2921f90faf", + "scripts/roofline.py": "524de48a6764401695fe1444f1fa3d38f363069b0cd2fba4715a86ca4b28a5d4" + } + }, + "plan-activation-residency": { + "header": { + "name": "plan-activation-residency", + "description": "Propose training activation and gradient memory reductions by distinguishing lifetime reordering, address packing and selective rematerialization from supplied evidence." + }, + "files": { + "SKILL.md": "c1918b60480a34e7cf1a1a431b089f42288f860a7b50f7df1ba0bce2658d3fe0", + "references/source-notes.md": "5ff52655cfd06fcf970ed2a7d4d4fe1c31115b102af2d41497de0c74848ed2b3" + } + }, + "project-maya": { + "header": { + "name": "project-maya", + "description": "Investigate Project Maya model-family dispatch, representation and memory tiers, API versus native state, speculative decoding and source-qualified verification without treating safe CPU checks as CUDA/HIP or model evidence.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "LICENSE": "574cffb7c57769a9271043ac0c2b07c5e7e7621cfb81081b901d8981e791d121", + "SKILL.md": "0e043e437d1ccc917ad5c20f79bf07c55b16764a15c88537725b68cac2ed1200", + "references/license-boundaries.md": "090839df90d6e96f55794c2ad821d7154dc6a6aa4211cdbb08a1e5f6f2e52a92", + "references/repo-provenance.md": "7bb6b6c14d1a9b77e906804505c723569f952e5d47559e636d2e6621994216bf", + "references/repo-routing-metadata.json": "0acbb111e92316e74d537200cd033f3d7d0f69972bf0953c8cea9dfb2ecb4a34", + "references/source-identity.json": "fc2287d5bde587cf040bbf515161d25e84a7bb42989cf01343582af67e4b8eeb", + "references/troubleshooting.md": "735d4bf3a3b243c29e6371af37dc06c1fa43b39a85811ba9714b911bf520ff67", + "sub-skills/model-runtime/SKILL.md": "1b68800cc2dd9b7b03cee3343fb8619135dc3563550cc20386606c295358df6f", + "sub-skills/model-runtime/references/model-and-memory.md": "ea7b905fa72cbfda5513f4c6ae77c552044af71e9313e396bcd3628884183bfb", + "sub-skills/model-runtime/references/notices.md": "6f6d9a96e1e67da6fe25f54482da0b69bd256fb688cc0956668f7319308f310e", + "sub-skills/model-runtime/references/troubleshooting.md": "97c00435d57a5b9067fe8e5885219d5818208a25f225a14c240d7160dcd697f7", + "sub-skills/model-runtime/scripts/audit_runtime_plan.py": "b096ec10c4f0e4b057997fe7b946007ab4c06dd263ab09d803e29e498601da00", + "sub-skills/qualification/SKILL.md": "5e264000198a9bbdded9c35db2df682e8eee4408ab741f18bc002e9be103b401", + "sub-skills/qualification/references/troubleshooting.md": "0c42256af4869e5cd1eb88fdaacfe65adf51dd180bdc03a2b9fccf220a19ccd8", + "sub-skills/qualification/references/verification-contract.md": "eaeb9319df27b66840cdfcba5c4788a287775d1493308517d9124fd0608eefec", + "sub-skills/qualification/scripts/summarize_decode_records.py": "cc0ac3c403aba79a131f631290d68e1640283e9f0d564aeef431b577824e3036", + "sub-skills/serving-state/SKILL.md": "830d8bccf6b36e1ad6cef899d5147afb74c0514516c68d02060bdfb550b56920", + "sub-skills/serving-state/references/api-and-history.md": "a51fa298017f266decd801441eb411fd68f7906a90a4b0d18083d1af31fc7d52", + "sub-skills/serving-state/references/native-state.md": "e5a7ee5e777998db0fb5422cf5b1ab18dbd67b4cbbfcfd6628ed6e40d8709c61", + "sub-skills/serving-state/references/troubleshooting.md": "97dea7b2d26fe165d2f64a1a63c91d099dddb2d3b95c16b667f5173b3ffa9d2d", + "sub-skills/serving-state/scripts/request_template.py": "d91b03fbbd7003012a92bd7259d39604b9a91ee5707da2d3366e7c0fb47edd6a" + } + }, + "pytorch": { + "header": { + "name": "pytorch", + "description": "Operate and diagnose PyTorch installation, tensors and autograd, training and data, compilation and export, accelerator lifetimes, distributed execution, checkpointing, profiling and custom operators from pinned repository evidence." + }, + "files": { + "LICENSE": "bd018feef8825e88181c84eb7e3aa4eafb8f08a20d9fd6ef948569610c4a3e43", + "SKILL.md": "cd04f507c7e3ea6212d47c1a68cc517fcd98205de355f412bb0b64acba39084d", + "references/notices.md": "c7b57d1226e749a396db4598510bf60afe1fefa1a3517fc4d90a0143ac783f20", + "references/repo-provenance.md": "e443eb08718412b59f60b6c9702b6408826bc2cf946e71aa8d2da474916a62d4", + "references/repo-routing-metadata.json": "908bed12129dabd490b58d958a208e720f93379ada7e70a52b78a1393457a310", + "references/source-directory-coverage.md": "ecae69ea63b2b528dc2c2aa98df502be21c126b972e73ce6e7864b240ebc95b9", + "references/source-readset.json": "4fe665a587dfd9bb02db6dc8f701d7417946c46986c1f5b960701c67304d0529", + "references/source-script-map.md": "1d669129ae4e9e2d030763fcfa66aee683c5f457537ab76db9b711100d506dfa", + "references/troubleshooting.md": "55df7c1517e1cf9127548d789197659612858d39d7020341960128bcbc10cba8", + "references/upstream-NOTICE": "c2cc7bf0caec7652c2b460a8a470bea1677f241e4ab8e431df34cf17f5a9fec0", + "scripts/inspect_environment.py": "bfb0086d93b975086c87a9b84adc629c8f468c5c4b7ceeb686206415e5a3880f", + "sub-skills/compiler-and-export/SKILL.md": "ac156fb42254308c25dd96efeffd15ccd84a86912733c7b57f214c6a7ddaf40b", + "sub-skills/compiler-and-export/references/troubleshooting.md": "ff1b48d2acbc13f80a530a4bb3b94fc7fb7b2e553f040f778d3edfe0aee67989", + "sub-skills/compiler-and-export/references/workflows.md": "3a68c6a183426df13ccaac999abd3e94e05b0784851c8238f435c737578f0b96", + "sub-skills/compiler-and-export/scripts/export_contract.py": "24457ab73ca4cf7fb33de6e2a735d1de3ed2fec36f76cb57f1f1e05136c0c509", + "sub-skills/custom-operators-and-extension/SKILL.md": "51f524127d1a798f6c79c28dde71c7d02984fab607655ff2fbe388d80ab7e1d2", + "sub-skills/custom-operators-and-extension/references/troubleshooting.md": "8383d490b1ffaeba3b77f07cc8eec76bf65aeee145177d7e17560d17754e747b", + "sub-skills/custom-operators-and-extension/references/workflows.md": "74d77673576f7951b1872615cddaac4ff6cf2a14d9f35e61ad801747cc9f0b1b", + "sub-skills/custom-operators-and-extension/scripts/custom_operator.py": "959ccc9dd18003bda5013c7726ba33e9c391b45c96598e76a087b956a36d6919", + "sub-skills/distributed-and-checkpoint/SKILL.md": "116ec05dad073f37f5e223459f2fa0aebc88302ba3af5f9408b9156d41af4a47", + "sub-skills/distributed-and-checkpoint/references/troubleshooting.md": "8afe03529f4586d131f003a20d95c610ac9d21abe625332e7ba0018b52e42b89", + "sub-skills/distributed-and-checkpoint/references/workflows.md": "d55c5e1b6d479f8947c5a8be42629370c4b8789a0af017f184d20f0006d29f39", + "sub-skills/distributed-and-checkpoint/scripts/rank_contract.py": "19494458f3866c11c9db4d97a90aeba338d3d5d7cf89179c1e4cb6ffd6fdc5cc", + "sub-skills/distributed-and-checkpoint/scripts/state_roundtrip.py": "f99df15d1b0a28ba99ce2fc9878374653d22c2e5fbfb449e4db89ce886b548e6", + "sub-skills/installation-and-build/SKILL.md": "cf423ea34d30efe5062336cc804aef92068f0dba5bfa1a6a0c8cdc9b5af8c9b2", + "sub-skills/installation-and-build/references/troubleshooting.md": "96b1358821914d81c9eb3ec0d72cacd1e1ad0868d2e1790e68ea46281026f623", + "sub-skills/installation-and-build/references/workflows.md": "2ed61b4922760b5f7a366ec5a51a58cf33c37d3715cd5e094a769065fa57addf", + "sub-skills/memory-streams-and-graphs/SKILL.md": "845f019569cb5f425837df3c79d111b708aee0c638b600dbfda318a71021c4c5", + "sub-skills/memory-streams-and-graphs/references/troubleshooting.md": "3398de27efe1d01457d204df8b5edba3cf7faf4a2fdf6052c74ea0af28fb1cd8", + "sub-skills/memory-streams-and-graphs/references/workflows.md": "f385877df2d8c6a97c1482aa3cbfba710342649d92c6856f44c34cbeb64af325", + "sub-skills/memory-streams-and-graphs/scripts/cuda_lifetimes.py": "1d9d8bb5d038e35ebcd46e97062e33cbdc9f57dd04366b91edd95edee0fedd3e", + "sub-skills/profiling-and-debugging/SKILL.md": "54dce03592d7b84314ca5ca42cdd2d1dc6847b7602be25e4d233c1688cadd2b7", + "sub-skills/profiling-and-debugging/references/troubleshooting.md": "984c5116cd2ed0ecc93d0f6e6d324c8b6c34093c9c4dae33981e31ecc9c4ed9e", + "sub-skills/profiling-and-debugging/references/workflows.md": "79313a2fc9037132db87214be4df91e37ba2a9c90c861c18e7a0d6c72ee19585", + "sub-skills/profiling-and-debugging/scripts/profile_step.py": "4a475f0bb2d8a06fcdc87d8661b893338279ddf4a4a83609c64c1f9a73028d7c", + "sub-skills/tensor-dispatch-autograd/SKILL.md": "4890ededcfba72266878b58fb50b98c227d852f4caa4eb4eb3907b3a0b0fcb9a", + "sub-skills/tensor-dispatch-autograd/references/troubleshooting.md": "4cf95feeab8a5cb87ea943158fd1a90c441979e834615050c8d88be686a845f1", + "sub-skills/tensor-dispatch-autograd/references/workflows.md": "93b699c5623710f411bb371084924ab3a45057ea4d59fcad36ee52ab0f26aa26", + "sub-skills/tensor-dispatch-autograd/scripts/tensor_contracts.py": "e5d893e42e2b91e9dd376b4b870e603193c91b096957972e2a93fa49f484b0c1", + "sub-skills/training-and-data/SKILL.md": "8b760d2220c4ceae20d61c115e09c6906390b2f355692e18cf7feb6fde610b6a", + "sub-skills/training-and-data/references/troubleshooting.md": "4e5c687a20e8066a9673625fc780667f490f9558a343d4d7d47d7ad278180824", + "sub-skills/training-and-data/references/workflows.md": "791c86dc3c13c3c87da3c4144cafb082e84f9ba2978bb71dea99f0cf649fe727", + "sub-skills/training-and-data/scripts/tiny_training.py": "46af58f6ba0c7d69373efc416fcc2ae8dcada717ed29336ac84f84857b1240bb" + } + }, + "reason-hopper-kernels": { + "header": { + "name": "reason-hopper-kernels", + "description": "Reason about Hopper SM90 CUDA GEMM kernels using TMA and mbarrier lifetimes, WGMMA warpgroup participation, register and pipeline budgets, layouts, software scheduling and FP8 scaling. Use for an H100/H200-family kernel diagnosis or a Blackwell-to-Hopper applicability check; source-derived candidate guidance with no local GPU validation." + }, + "files": { + "SKILL.md": "1bbc5031877458f771482d5faf5e2bfa040a2c7f52c63ed43aa847325b7405eb", + "references/fp8.md": "31d9c211c11c2224007abbdcd4059913c4a54833fe2d2f926411c5238fcd3501", + "references/licenses/cutlass-bsd.md": "42fec630f410aa308f70a51a89fadcd19586fa620f9831a32bee528a9a10000e", + "references/licenses/deepgemm-mit.md": "6c73a30c14a2938abaa6afe98ebb19e6d81dac5bd4ace0cb05c61c20afa37b85", + "references/licenses/kernelwiki-kda-mit.md": "84aab57158a914f448b38d8be29dd842b634a08c192d844f416a256beed4ae94", + "references/notices.md": "876ec016644bf5331b3f257985e126c0e61980e3bfd82569b7b0c3f6b2c6c64c", + "references/resources-and-scheduling.md": "1ce25b426736072435a1ead9a2a8e1f56df5d0abb8198e8c4fdcb34321db4964", + "references/source-inventory.json": "1dc09c895c970a0c9708158bcc6dd65cdb736274fd2c35d44542c70407d3849d", + "references/sources.md": "6b74b9e774f2a80db0860510d9e6940d36284430f29c1e124f115a9eaf25833d", + "references/synchronization.md": "4322f6b3c0e4368bf8c192b4ec752529767f50d19c0c38dad8fdf9cd781e41d9" + } + }, + "repository-evolution-analysis": { + "header": { + "name": "repository-evolution-analysis", + "description": "Analyze a repository's commit history to build a capability-evolution model: what the system can do, how it got there, and what patterns allow it to iterate quickly. Use to produce history-derived knowledge for agent skill conditioning, and to design temporal-split benchmarks for skill evaluation.\n" + }, + "files": { + "SKILL.md": "0f48cc6a77b3ac861af698048be7c250c3ef5c225dc238b9a0e6d71208cbd93e", + "references/capability-presence-audit.md": "c21eb4a6aabe811a49a6540a34c54dbcfdcf7efae3052a79b54034c7838026a5", + "references/rtk-consumer-path-transition.md": "a70b1d128fb5e0ac12292a290d624f218a0831d7581fe8980b62661473e0f2b7" + } + }, + "rsi-evidence": { + "header": { + "name": "rsi-evidence", + "description": "Guide agents to establish acceptance criteria, freeze and cite RSI evidence using plan/results/patch/validator paths and PASS/FAIL/INCONCLUSIVE verdicts." + }, + "files": { + "SKILL.md": "5edd2bd544da050d2688f37c85ae6e7927eaee24af764e3c96b6cec04982990a", + "references/loopx-goal-governance/LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", + "references/loopx-goal-governance/LICENSE-MIT.md": "1df1cf519b9e5d8acbbc72820299d7aa95afa1acbfc1f43a15109ad1d59975f3", + "references/loopx-goal-governance/NOTICE.md": "3c3c36a40e1d65f3a6b20ec3a97319cb753e857f1d120f302cd828e107120f6d", + "references/loopx-goal-governance/README.md": "c86eb179cea1b42f346f5c1dc076292ba086be907f20b0beee45c43cf0f8d6fe", + "references/loopx-goal-governance/decision-cases.md": "c2c1d3d75c4fcb515dd36ff84d8dc940c25d54c5757a29a80d1af88eb8b66115", + "references/loopx-goal-governance/mechanisms.md": "4fa09099d362a8088fdb0e6a767f72aae6218b92fe5fd75b6fb78c2bb2a318fb", + "references/loopx-goal-governance/source-ledger.json": "4a0f1f02134af5b7d54a1f13e104b8066372d3d4a1a4a9e717514632fff05893", + "references/omni-experiments.md": "8fafc1e05ef21ab60158fc626c73ca1b8c615ada59f90fd23199496daa46ac48" + } + }, + "skill-evolution-playbook": { + "header": { + "name": "skill-evolution-playbook", + "description": "Practical design choices for evolving agent skills from execution traces: when one trajectory should update a skill, how conflicting lessons are handled, how negative evidence is preserved, how large an edit should be, and when skills should split or merge. Drawn from Trace2Skill and SkillOpt. Use when designing or operating a skill evolution pipeline.\n" + }, + "files": { + "SKILL.md": "f988a0569be93fec550da52484d22d970b0a6c31394ea463ad6ee40e0877ede6" + } + }, + "splash": { + "header": { + "name": "splash", + "description": "Investigate Splash model-format compatibility, automatic memory planning, Metal and ANE execution, DFlash2 verification, cache and request identity, and correctness-gated serving measurements.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "LICENSE": "cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30", + "SKILL.md": "17b7f9d978bb02ec9cc246824ad247ecf55930f11d015b1606dce73060329651", + "THIRD_PARTY_NOTICES": "2fedf1277bf732f8435c3e3be2798f52f25f7e2bd4155865cb366da55b88e50b", + "references/repo-provenance.md": "20e3462f1b6a8752eb0fa6f36e76867f142a49c1953f498f895e29c4fd4f0204", + "references/repo-routing-metadata.json": "287ae591f42949b8d33312659ec8303c47d5c73093c64c125edb1cf6ea8de8b9", + "references/source-identity.json": "2f778882ec440538a46f48332650412b446722246c8dc5690083f19da7dce61d", + "scripts/request_template.py": "086d773d42edf67c43cedae16974061b09ac66458ee47fe961fc01733f3b2a0e", + "sub-skills/qualification/SKILL.md": "4fddc39811c12ec4b3279ad68a8864b7f1ef80e838ea03788b54cb025a03ae16", + "sub-skills/qualification/references/troubleshooting.md": "dbd2179a48eb3fc0ecfd54c5df8a690d0cb2e1814acc19ac3b0aabf184092c92", + "sub-skills/qualification/references/verification-contract.md": "9efffa0ecfd3b6771dfce631137eccc6d939c3239f60b1fadcd0c6c1ed7700be", + "sub-skills/qualification/scripts/summarize_rounds.py": "107a844954e3909500c60eb1ca365ac9b5e7dc3d91122dbae46f402a5cef4e3b", + "sub-skills/runtime-planning/SKILL.md": "f0fe3d3f4053937767ae586559cc7fba89dacb6c896186a029fff220414e8d22", + "sub-skills/runtime-planning/references/memory-and-formats.md": "04edf10f4a965c512f6a262b1f8295f3c033d615849e49d21d3d94c78b549f2b", + "sub-skills/runtime-planning/references/troubleshooting.md": "5a1896f9fb01f6c8c9ef78d94f3c965d6d64e6b54223c6d7901eaed0548c5575", + "sub-skills/runtime-planning/scripts/parse_budget.py": "b3a3b0847280333c2bfbe5d03350dc866307d5290160ec64c7e3d462a34f6301", + "sub-skills/serving-cache/SKILL.md": "c16f911498e116ff9b31b5a6f0ddd3ae6517b51c0e9d079c337f36a5bcac1df8", + "sub-skills/serving-cache/references/state-and-api.md": "592603177473ac8e024b7dc19a3d3faedd7c1e0312137eab8bf9a62d2f567be6", + "sub-skills/serving-cache/references/troubleshooting.md": "4fda461042a3de43c109d7a3cec67542c7d61fe68412bf91606220f2e13e576e" + } + }, + "storage-io-measurement": { + "header": { + "name": "storage-io-measurement", + "description": "Measure storage I/O for LLM/diffusion serving without conflating device metrics with application outcomes: match the fio workload to the observed loader signature, validate requested-versus-achieved I/O depth, keep a cache-state ledger (warm/cold/unknown with method), classify startup vs steady-state runs, read outputs with unit and latency-provenance awareness, attribute traffic per device (iostat) and per process (pidstat), and gate any \"storage bottleneck\" claim through a saturation-evidence check plus a separate application-outcome layer. Measurement methodology only — running benchmarks or manipulating caches requires separate authorization.\n" + }, + "files": { + "SKILL.md": "7faf89f304d6fdad2d27978f9c4bb828400e57e7fbb294dbd43e045bcf94bcc3" + } + }, + "task-accuracy": { + "header": { + "name": "task-accuracy", + "description": "Use when inference completes but tokens, audio, frames, tensors or a quality metric diverge from a reference. Trace the first divergence and verify end-to-end tolerance." + }, + "files": { + "SKILL.md": "a441eb5565cd39674307f2cc870cb0d8d4c90314bb7df80245162df1dc3d1ae3" + } + }, + "task-day-0": { + "header": { + "name": "task-day-0", + "description": "Use for Day-0 onboarding of a new checkpoint or architecture to a named inference engine. Produce a reproducible target run, stage parity evidence and a regression fixture." + }, + "files": { + "SKILL.md": "ca187de2205b3aa86b4ca5123fb0107a8235aa84f9d942d3771a8a665df1a0a3" + } + }, + "task-debug": { + "header": { + "name": "task-debug", + "description": "Use for inference crashes, hangs, out-of-memory failures or execution-path errors. Localize the first failing boundary and verify recovery with the original reproducer." + }, + "files": { + "SKILL.md": "2e1e91cf429384c7cb3de68cb9a896a645853f819b23bbe2c10792b2813ae5f0" + } + }, + "task-derived-engine": { + "header": { + "name": "task-derived-engine", + "description": "Use for a fork or extension that retains an upstream inference execution core, including a vLLM or vLLM-Omni derivative. Verify inherited behavior and the intended delta separately." + }, + "files": { + "SKILL.md": "976a01cf23e626fb2112fa637bb76da3f59532d27ee93f22365ea2c752dd0942" + } + }, + "task-independent-engine": { + "header": { + "name": "task-independent-engine", + "description": "Use when the deliverable owns its execution, memory and device integration on a target OS and hardware stack. Prove a native end-to-end path and measure its resource use." + }, + "files": { + "SKILL.md": "37e122098f2df35edc466d5b70598039ae640c8f4bb683b694d9195b0a83089e" + } + }, + "task-performance": { + "header": { + "name": "task-performance", + "description": "Use when a correct inference baseline misses a latency, throughput, memory or cost target. Compare compatible changes under the user's workload and quality gate." + }, + "files": { + "SKILL.md": "bf5a137504566b6ee4eeaee18400d52aae99a1db3c8247dea7247d653103df0b" + } + }, + "task-version-alignment": { + "header": { + "name": "task-version-alignment", + "description": "Use for upgrades spanning inference engines, plugins, drivers or operator libraries. Isolate the incompatible boundary and qualify the requested version combination on the same workload." + }, + "files": { + "SKILL.md": "114a2f41ce5b45f61b7cbb55d1cf9fa99a7c9aaff5e5597e563d9a00a7727c8c" + } + }, + "torch-npu": { + "header": { + "name": "torch-npu", + "description": "Operate and diagnose TorchNPU on Ascend across installation, PrivateUse1 dispatch, operator development, memory and streams, graphs and compilation, HCCL, precision and profiling, with pinned source evidence and explicit backend limits." + }, + "files": { + "SKILL.md": "94b08de8f9666cf19c4f479da32cf6d40bd122a8689afcdac8cea55768e98a30", + "references/NOTICE.md": "a81fb8c00576c86fe6c839d9bf2d4acd32d854ae9fa168ec6c6d9cf6ea2170dd", + "references/capability-inventory.md": "e6bae7bbeecea8b16c9e911ac40aa385f181da855336e0ac1fe6d9ed1c3a52e9", + "references/licenses/op-plugin-LICENSE.txt": "325488df8d01b0601a46b395904b6ce72f4870cd608f49268904e994dab15f97", + "references/licenses/op-plugin-THIRD-PARTY.txt": "cb91e5fa816c6f6ff7a6a3cf32110c3402b1d4c76778de106130b227782013d0", + "references/licenses/source-identities.json": "f1ee46af6bfcea7e41959ceab981515cf902fe7f1d11998e69145cb86d897c37", + "references/licenses/torch-npu-LICENSE.txt": "99317d26c99c8037890c770fd67fa84cc78d210581ed563d3b1abe42976f47be", + "references/licenses/torch-npu-THIRD-PARTY.txt": "167f93c40acf8a26e4f6cc293df727c04ef869e54a10e7517ce2de60eaf2593f", + "references/licenses/torch-npu-docs-LICENSE.txt": "9e5f1b3c610b9c2da5c313bf81d577a7d1acec686bdb0384edefa6df0f90cd94", + "references/repo-provenance.md": "18cb87b57a1ffa25e0954f629ddc1fac3137a2ee2d5d362be30083e5bfc12dc9", + "references/repo-routing-metadata.json": "eeab2fedf7db673c0ce2d8f53641d4bcaa9b0fd84b0aa30cf746f894c548a268", + "references/source-manifest.json": "b1d4cbdad4cde3838d3bbc16ccede8c79825dd7280a3c3c081507d7f51e86726", + "references/source-map.md": "d64683d861c1cca6f61a5e90500f98c7f5dd9a7256035433c11e39cc44eaad15", + "references/source-script-inventory.md": "c43c83daa152856c4fab278f25d05504735e25c427231bcf177b72e01536819d", + "references/support-workflows.md": "002121bded102c51966754aff40c3f621cc2e69d35ff29fa187582d496bb8ce4", + "scripts/inspect_environment.py": "a157f5947217c5baadb2a4957a68c91fa1dbef343c97ac2f7f4d59bf1e4cb3b7", + "scripts/operator_contract.py": "45f076bb5353798d4b494f7b7e990796dfb1418d7ef44643521bd669bc348b32", + "scripts/runtime_smoke.py": "e5c7d587e54a6e4fd5dd8c45d5ef36990143ee40e24ff1680d49b7637894c248", + "sub-skills/distributed-and-hccl/SKILL.md": "0f7a272f9461456ea23872f0636d668654faa3448297778f2ef96cda956ea314", + "sub-skills/distributed-and-hccl/references/hccl.md": "bd6b67c243032785f60f100d75c450885bfb1e6585af9100482fca3dfbe86919", + "sub-skills/distributed-and-hccl/references/source-map.md": "06bebd8bcae81b253573a4096a905cd264da2a75a9b13548f646c2ab9b698fd5", + "sub-skills/distributed-and-hccl/references/troubleshooting.md": "946650a1fa1c3f805328e3071ed7a0ed907b71af6a4d746d3fb4bad869d5aaad", + "sub-skills/graph-and-compile/SKILL.md": "b15b5bafee882a00f57d8a4d3458e4c411b549a97a68be953d237e441b5bfe31", + "sub-skills/graph-and-compile/references/graphs.md": "ec9e13041eb2d5abf5e2055e667fcef79391dad0b32756b963efb69d2be7354c", + "sub-skills/graph-and-compile/references/source-map.md": "89aedad45ab2eb524053909060344e5b11d3b9923b7cff8d1e661a6a097b00f2", + "sub-skills/graph-and-compile/references/troubleshooting.md": "35be0f57c998a5a6ef24f46f67f3a37e17b0d519a2604a38ec0498b21f374437", + "sub-skills/installation-and-compatibility/SKILL.md": "af5f44615fe45cb245b3d6e3881ed4626546614c01782c36bfdb5a46d0843b71", + "sub-skills/installation-and-compatibility/references/source-map.md": "b5266c944b0abd8bbb35d50aa6eddb9452725ba2308523f0bc8414dbc87f8802", + "sub-skills/installation-and-compatibility/references/stack.md": "4e9a8abbf45eff3b3e79de9b0fa0907479b77f729544cd1c772d2334606fca94", + "sub-skills/installation-and-compatibility/references/troubleshooting.md": "799214c15da2ce43f2f9ab37347e174fff3346c3e27507022efd7a2aa30fcef6", + "sub-skills/memory-streams-and-events/SKILL.md": "13fb13dbfb1e1a684ad2c905cd50307b604abb025b93c8f317af79954a82cc46", + "sub-skills/memory-streams-and-events/references/lifetimes.md": "bc5551f922534d1db0cb6ade6ea1dbbd4f26419cd9aa0b27f9aec9415d691743", + "sub-skills/memory-streams-and-events/references/source-map.md": "05114bd16d7cf22c5604a3f1885f44b8be225fa2644302a8ef805193a46601c2", + "sub-skills/memory-streams-and-events/references/troubleshooting.md": "b9d5d4d61087f27b2761c9184c99f98da583948c393f6a3c6d189b87ef9f1466", + "sub-skills/operator-development/SKILL.md": "3f71ee3a85d5d8b79c51b5b01fbd3193df4b8817405a4c2a982823614f7332b0", + "sub-skills/operator-development/references/development.md": "70994e5deeb138ca3dce578a743515840a91ff6303f0328eaff6de8fc308313a", + "sub-skills/operator-development/references/source-map.md": "29863b23b49040728c4d928a149477aeb966da833cb4ce21a6bad5212ca30c9e", + "sub-skills/operator-development/references/troubleshooting.md": "0c9828b96ab7943ab0b7006ac8520e58072fe7f2544a78cb9111c1f224255c6d", + "sub-skills/privateuse1-and-dispatch/SKILL.md": "b983b6ff5f40217a8240956f4bc6d3d89427d7af72756632f1731bc803159a87", + "sub-skills/privateuse1-and-dispatch/references/dispatch.md": "19d4a9120bd1fcf4aea2dcacbf8e66c0e9f3fc3ca5e56fc6c468a3721daff973", + "sub-skills/privateuse1-and-dispatch/references/source-map.md": "432913d63c0e1a54ad75da6b7e51489dfd4589a208a51b0a0245ba679cd29c4f", + "sub-skills/privateuse1-and-dispatch/references/troubleshooting.md": "193c7499076e77ba82772ac5d939cc377d9301f7e6825353b98df09f737c85cf", + "sub-skills/profiling-and-debugging/SKILL.md": "1181c4eba1274a9eedf4dd0a430f7b13a0599d3b7b89dda772fec11411a60cf4", + "sub-skills/profiling-and-debugging/references/profiling.md": "f61c6f33cd9b63546c44ba14fed1ba48ce0f0318becf7a88fe9a1d658f5e83f9", + "sub-skills/profiling-and-debugging/references/source-map.md": "2550e9a28209a7df0a0d2f571af39985ca629c42a1be6f845e50fd265fc6ccb8", + "sub-skills/profiling-and-debugging/references/troubleshooting.md": "c60466ad74a0e36b6f74001a0ab756fd2dfc674d059e038fbcd63770038396c6", + "sub-skills/training-and-precision/SKILL.md": "e665e57af9569c9ae7ff063d8c57049013ed7babb56de82c4d93af108be2f725", + "sub-skills/training-and-precision/references/source-map.md": "3bfc78d9348f9a1072120f82d7a1a7f042b4fe2ce908a005e22716a797dc0c7b", + "sub-skills/training-and-precision/references/training.md": "0695532020c71d061ddf40ad16536621e004c23dd05d27aaa9f5b46101f6ef62", + "sub-skills/training-and-precision/references/troubleshooting.md": "0f7fb3879d51c793ca0fd6d14fd34352f94297f075295d01488ffab4c5fc0fa4" + } + }, + "torchnpu-device-memory-lifecycle": { + "header": { + "name": "torchnpu-device-memory-lifecycle", + "description": "Diagnose TorchNPU device-HBM reuse, cross-stream lifetimes, cache retention and fragmentation, then propose version-gated allocator changes. Use for NPU allocator OOM or reuse investigations, not pinned-host RAM or network transport." + }, + "files": { + "SKILL.md": "11cf02d6ef936b7ea8b1b1f941137f676f8b3a122db4f6d9e48b1be249c304f0", + "references/source-notes.md": "80cba4d3a884e7ee2618326cf74b9f9c3a34e4d9cf443707f6bbe673f2d5ba65" + } + }, + "torchnpu-pinned-host-memory-lifecycle": { + "header": { + "name": "torchnpu-pinned-host-memory-lifecycle", + "description": "Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy lifetime, reuse and cache controls, then propose version-gated host-allocator changes. Use for pin_memory pressure or host allocation stalls, not device HBM or network transport." + }, + "files": { + "SKILL.md": "5cf992ee8b6bf4a7a07d83fa2568e4238feb5b187e61d1bff82a054aa4c58863", + "references/source-notes.md": "435193590b4a2ba32e2e4c6fa9d7e8f889528d412cbbb48e35ec5433f7dab384" + } + }, + "training-input-pipeline": { + "header": { + "name": "training-input-pipeline", + "description": "Diagnose training input stalls and choose bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic Streaming and FFCV. Preserve execution-mode gates, sample ordering and per-host memory budgets; separate remote downloads, local storage reads and host/device transfers.\n" + }, + "files": { + "SKILL.md": "779478e854aa1ffcc453609b1c8c67477154f478dc360f6541c182940f67fd0e", + "references/sources.md": "60e0f3e88655d8e1d5eb4797f39ca4370deda01614f265b8d2ea4559d8ac020f" + } + }, + "training-resource-budget": { + "header": { + "name": "training-resource-budget", + "description": "Choose training offload and resource-budget changes using pinned DeepSpeed configuration, attributed expert diagnosis and a historical MindSpeed case. Balance accelerator capacity, CPU work, pinned RAM and local storage; keep host/device transfer exposure distinct from collective network bottlenecks.\n" + }, + "files": { + "SKILL.md": "e8adf9d115975844d6c7a1b1809c1cbc2baeabc1318198a777050f159a0fe8e2", + "references/mindspeed-historical.md": "ad65c27f278c3d0ca3599a7630c02d3e529477a05b71b7703bd2d75f0d8fdd2c", + "references/mindspeed-source-readset.md": "4462d69ed6a941a03639c756968be3ce54f947e51f6ca618f4fb1cd43b643b35", + "references/papers-20261015/INTEGRATION_NOTES.md": "c55c6ad2131ef788866411dcc7056f221420ebff30c95c6bc919b09a4818cb73", + "references/papers-20261015/cases/paper-grounded-cases.md": "19bd25b1a944adb2351ebc6b1d795150ea033041a44f6cc261e881677812a101", + "references/papers-20261015/knowledge/offload-delta.md": "f8740fc5c8bbbe5ac13ff1cfec7ca499569d0b46e8cd56cf1f5a44c3a5e72242", + "references/papers-20261015/notes/training-memory-scheduling.md": "9aa6d3e01452dff95c8a8940ff62ead8c146ce215c9175b78c9f2ef508a7b2de", + "references/papers-20261015/references/source-readset.md": "2a5c166767f56fa47e7eba7386f1f36da8cb73702e5b5bbe14d6a456ca9ebfae", + "references/sources.md": "53d70671a183de4aaa3369f9caf2383a23b6f8a0a7b04464a8d8a639150afc6a", + "references/stas-decision-notes.md": "f1a993e6c4ca9a035a2effcaf5500081f227b0cc56cc885a6b22fda0041fc744" + } + }, + "video-diffusion-optimization": { + "header": { + "name": "video-diffusion-optimization", + "description": "Run and verify goal-driven video diffusion optimization, backend-agnostic. Use when the user asks for latency, RTF, throughput, or memory targets across VLM, DiT, MoE, attention, or VAE, on any accelerator. Not for text-only LLMs." + }, + "files": { + "SKILL.md": "6466f46b63f6149e9692fdc344fca5f3af9e1961b59fd48f25b778256089c203", + "assets/campaign.template.json": "1a37f1d136612143ddab8f0cbbf424c80433b4da5312edb18d07c0b9d87ef657", + "scripts/evaluate_target.py": "9e84df051e03d567746fd73ae7243efb2f3c5ec806fef824d3c4d42a8eb9704c" + } + }, + "vllm-ascend": { + "header": { + "name": "vllm-ascend", + "description": "Analyze vLLM Ascend v0.27.1rc1 performance and commit evolution for DeepSeek-V4, Kimi K3, MiniMax-M3 and GLM5+, fused operators, DSA-CP, FlashComm1, Fused-MC2 and DSpark, with effective dispatch, rollback and test-oracle guidance." + }, + "files": { + "SKILL.md": "c8d829d6c4536f269ce2c4c816fb475faba0bb6c33c2782ce4d90ae6f7156e74", + "references/communication-config.md": "66daf21685237429e45a64a9d100a10d1f16872a4785037013f0fa798069a319", + "references/deepseek-v4-evolution.md": "e970d58612aad3cc8244e0b78ae0ce24967f3931659796d2cc519e052f34c113", + "references/dspark-pd.md": "2f5112acfff4f114998c2737ef3ca4cb81a16c754541421f95fc08e30e13860b", + "references/fused-operators.md": "97bdae687fb0159612e61744696a256a99bab466726984bec672c1aad257a3ff", + "references/history-reviewed.json": "c75c3997ba8e5cd569d9252abd58d5f8ecc406696ae9fafc34a4da6a7732251f", + "references/history-scope.md": "7c57b79dcd2e920e241d50cfa2a40d31fbad62768c7b55254017e51b1d7b7945", + "references/minimax-cache.md": "6a6196eb2e69bf70796f8436630a5e524b67068c439d482727193b961cb6228e", + "references/model-performance.md": "6c9bda4499239fe7a206b578b8190ab36c5454d15b57eb4f591eb2ebe1e037e7", + "references/plugin-registration.md": "ab1b090f22bd0315d2a2e14324064a1c04653fd4eabeffa60de80b9e606de75c", + "references/sources.md": "9b588e6c4887ff7b5081a72f6fb63b5f53c2f716571b5855acbffc85dbbdc2ab", + "scripts/screen_history.py": "922aa44cc796503628edaff3779d69e5560cf73fa8175859e2fbd701e9ad69e6" + } + }, + "vllm-metal": { + "header": { + "name": "vllm-metal", + "description": "Investigate vllm-metal Apple inference through platform and native-ABI selection, paged KV/offload ownership, speculative/deferred state and actual dispatch/serving evidence without upgrading source or CPU checks to backend qualification.", + "license": "NO_LICENSE", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + }, + "files": { + "LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", + "SKILL.md": "4829402b68f6707d74c8d3316575f78b5375a23428400e6a247c97bce8793f05", + "references/repo-provenance.md": "970bda4d1081704aa82c94d6544a98bbf479430c4fc2c82a1a82f206043aa72a", + "references/repo-routing-metadata.json": "230a2b383bcf2855af4de50374c41f11b0b0841268ae45e424c1c97f679b477a", + "references/scope-and-notices.md": "aaef98eaea4cff56f39852e1d131163ebc01334e79034b1890f67a2660727d3e", + "references/source-identity.json": "9da529574e81f9f91d86c01153eb09a8b1e728dc5327c40364fff74085598c1f", + "sub-skills/cache-lifecycle/SKILL.md": "dbfb06896615dc994f4a0a1e3789904acdf512ec316cbab6347165b764bc592f", + "sub-skills/cache-lifecycle/references/cache-and-offload.md": "9829c14cf20f22575c44c1a0fcf18f63578443710e1a592c9fea709a43a3b888", + "sub-skills/cache-lifecycle/references/helper-contract.md": "56a7494b58c47e07d1c85471ddc6075b15e34dc7a852b937002416dbd55b65ce", + "sub-skills/cache-lifecycle/references/troubleshooting.md": "a4600c6d17a980b6442fc4c36f1c9ae148a2710c644126a9a898675b01f95366", + "sub-skills/cache-lifecycle/scripts/screen_pool_budget.py": "dd615f7d3239f48510ce83416fb14fba8d5da2b9017627bdd312567be1bd7956", + "sub-skills/platform-config/SKILL.md": "e2f5c5207b5c7665d3388d94fdace35e25f073a7833cbddaf14514a05f5ca5b9", + "sub-skills/platform-config/references/configuration.md": "69949b93f4f319145e37d1b28487e37bd4ef21622f3169583d9cff46234a9c76", + "sub-skills/platform-config/references/platform-and-loading.md": "7f11dc9e4bb9ae7bac25fcd6a2909cbea0eb2ae329e28ec9a7a2f4085c56e5cc", + "sub-skills/platform-config/references/troubleshooting.md": "837cef82321fa050af3f01804fe337b0a0462255261fea5c21158ecfa3d0c325", + "sub-skills/platform-config/scripts/check_environment_plan.py": "2603200fef87e3afad7820b84994f7e43e6041b45c17d0ed613d0d6229535bf2", + "sub-skills/qualification/SKILL.md": "37d9ae55ab4d0afd4ff8053c1955445e5a090188841ecc7941a5e22c5dd5eb39", + "sub-skills/qualification/references/dispatch-and-measurement.md": "360719d4253ad7e5a36823171d432aa8ca0f74d1ae565c7803af1990d99f2af9", + "sub-skills/qualification/references/record-schema.md": "f3cefb8b131613535dcacf30d130ffc2dea991acfb495adb4e1c7050046903d7", + "sub-skills/qualification/references/speculation-and-pipeline.md": "64faced7f31d489e9b0c4b9523a803ff1a76dca0f9315bdc6d0b83935bd338a4", + "sub-skills/qualification/references/troubleshooting.md": "a2e85696120ffc2af4249ac9c1e1b96320e02287e5099577c0474c025ab4f5f6", + "sub-skills/qualification/scripts/summarize_serving_records.py": "78e660a13a6a6941d006cb11fc2f401a34f489e54273860ae6a208dca6a6a3c2" + } + }, + "weight-loading-optimization": { + "header": { + "name": "weight-loading-optimization", + "description": "Optimize the serving-level weight load path from storage to device: judge the load path on four dimensions (source, format, placement, duplication), account startup fixed overhead separately from per-request cost with bytes-and-counts lines and hit-criteria warmup validation, verify sharded weight downloads against the safetensors index, apply device-side weight prefetch only on profiled memory-bound hot spots, and order CPU-tier offload candidates by change surface. Includes source-scoped disk-loading modes, pinned staging, synchronous checkpoint save semantics and fastsafetensors broadcast/scatter/coalesced delivery; also audit streamed expert-cache byte correctness and begin/end overlap contracts. Use when startup/loading latency dominates or a streamed expert-cache change needs correctness verification; route DCP asynchronous save performance and distributed recovery validity to their separate skills.\n" + }, + "files": { + "SKILL.md": "53d9c9c594663d271365de4cec6f22e37077fcf178625aa5ca087fbffb5b86a1", + "references/deepseek-expert-cache-oracles.md": "5e42c4a01f9ac810663979dee0658c4cbe87060594c16622ae6ec0b854e44b29", + "references/deepseek-expert-cache-source.json": "65bf0623616b4f633d8f51b30134798cbf71176716c8a8613209e2fc48ff84bd", + "references/fastsafetensors-transfer-readset.md": "5a39f14b102b0cee304541e011aa31e44e6b13527d1bee9987c3f3b66c064d3d", + "references/flare-preparation-boundary.md": "c0d787522451f5b69a6d20b188d573183fa6d67de30f695ea668ca1ab7108929", + "references/llama-loading-readset.md": "6c640f28ee3f0253a17b81461c865cb9f947367002cb75bf72550c4572d02e97" + } + } +} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md b/src/inferrsi/knowledge/data/skill_library/casebooks/ascend/ascend-optimization-casebook/SKILL.md similarity index 82% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/casebooks/ascend/ascend-optimization-casebook/SKILL.md index b92747eb4..6d701cbbc 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/casebooks/ascend/ascend-optimization-casebook/SKILL.md @@ -3,6 +3,42 @@ name: ascend-optimization-casebook description: 'Casebook of NPU optimization cases for Ascend inference systems: symptom → diagnosis → action → evidence records for memory-bound, compute-bound, communication-exposed, graph-compilation, and AI-CPU-fallback bottleneck classes. Use when a bottleneck has been classified and you need prior art for what to try next. Complement to inference-optimization-casebook (which covers the casebook *pattern*; this carries the actual NPU cases). ' +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "optimization", + "casebook", + "prior-art" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Find source-derived prior cases for an already classified Ascend inference bottleneck, including fallback, memory traffic, exposed communication, graph startup and host scheduling gaps.", + "prerequisites": "A bottleneck classification from profiling and a check of the selected case preconditions/mechanism before applying it.", + "input_contract": "Bottleneck symptoms and matching profiler evidence, model/hardware/CANN/configuration context and, for a new case, measured before/after metrics.", + "produced_artifact": "A matching prior-art case and evidence requirements for a later measured application; a new case record requires a verified win or clear failure with a metric comparison.", + "refusal_point": "Do not add an INCONCLUSIVE case or one without a measured metric comparison; a subjective direction is insufficient evidence." + }, + "recommendations": [ + "Classify first, then check the matching case preconditions and use its owning procedure to apply and measure the candidate; the casebook itself does not perform profiling.", + "Keep profiler mechanism evidence and same-window end-to-end outcomes together, and retain locally_verified=false until the applied case has actually run." + ] + } --- # Ascend NPU Optimization Casebook diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/SKILL.md b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/SKILL.md similarity index 64% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/SKILL.md index abf4d37d1..d67d1a861 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/SKILL.md @@ -3,6 +3,51 @@ name: inference-optimization-casebook description: 'How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook pattern from TensorRT-LLM''s perf-optimization-casebook skill. Use when building a record of optimization experiments or when deciding which optimization to try next based on prior art. ' +metadata: + inferrsi: | + { + "tags": [ + "optimization", + "casebook", + "prior-art", + "decision-support" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Structuring and querying a casebook of past optimization decisions as decision support, when recording optimization experiments or choosing the next optimization from prior art; the casebook is reference material and neither runs profiling nor makes decisions", + "prerequisites": "Whole-body text access to the case files for free-text search; applying a case first requires reading the nearest checked-in config and the model's deployment guide", + "input_contract": "For a query: natural search terms such as model name, knob, mechanism word or deployment feature, plus the current model and hardware. For a new case: mechanism, applies-when and does-not-apply conditions, procedure, expected effect, accuracy risk, verification, rollback and a real measurement trail with run ID, verdict, exact hardware SKU, model revision, workload and before/after result", + "produced_artifact": "Ranked matching cases read as precedents and diagnostic options, or a new or deprecated case record in the template; a precedent's procedure and expected direction of effect, not a guaranteed configuration or a measured result", + "failure_modes": [ + "The query is pre-translated into canonical vocabulary or bounded by structured fields, so synonyms in the body text are missed", + "A case's specific values, such as batch size, tensor-parallel degree or kernel parameters, are copied instead of re-derived for the current model and hardware", + "The precedent's expected direction of effect stands in for measurement after the change is applied", + "A rejected direction is read as a permanent blacklist, or one discarded implementation as ruling out its mechanism", + "A confounded run, a missing experiment and a failed gate are recorded as the same evidence state", + "Cross-system cases are bootstrapped without re-verifying that the mechanism holds on the target hardware" + ], + "refusal_point": "Adds no case for an INCONCLUSIVE optimization-effect claim, a single-configuration result without plausible generalization, or a result without a real measurement trail, except a scoped diagnostic observation that keeps its verdict and configuration boundary; manufactures no PASS, and reading the records does not reopen a paused campaign" + }, + "recommendations": [ + "Search the full case bodies with free text first, then rank the hits by bottleneck class and signals.", + "Before retrying a rejected direction, state which condition changed or which uncertainty a bounded check resolves, run an unchanged control and choose the least expensive discriminating check.", + "Mark bootstrapped cases source_grounded or author_reported until a local measurement confirms or refutes them.", + "When a case's conditions no longer hold, mark it deprecated with the reason instead of deleting it." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged" + } --- # Inference Optimization Casebook Pattern diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.json b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.json rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.md b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.md rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.json b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.json rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.md rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md b/src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md rename to src/inferrsi/knowledge/data/skill_library/casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md diff --git a/src/inferrsi/knowledge/data/query_aliases.json b/src/inferrsi/knowledge/data/skill_library/config/query_aliases.json similarity index 100% rename from src/inferrsi/knowledge/data/query_aliases.json rename to src/inferrsi/knowledge/data/skill_library/config/query_aliases.json diff --git a/src/inferrsi/knowledge/data/research_profiles/minimax-h3.json b/src/inferrsi/knowledge/data/skill_library/config/research_profiles/minimax-h3.json similarity index 82% rename from src/inferrsi/knowledge/data/research_profiles/minimax-h3.json rename to src/inferrsi/knowledge/data/skill_library/config/research_profiles/minimax-h3.json index 84d8703e7..1d2edb94f 100644 --- a/src/inferrsi/knowledge/data/research_profiles/minimax-h3.json +++ b/src/inferrsi/knowledge/data/skill_library/config/research_profiles/minimax-h3.json @@ -28,7 +28,7 @@ ], "skill_id": "audit-h3-runtime-residency", "knowledge_ids": ["inferrsi.common.skills.audit_h3_runtime_residency"], - "skill_resource": "data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md", - "worked_example_resource": "data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md", - "inventory_resource": "data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-weight-inventory.json" + "skill_resource": "data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md", + "worked_example_resource": "data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md", + "inventory_resource": "data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/minimax-h3-weight-inventory.json" } diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/metadata.json deleted file mode 100644 index 4d74a60c1..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/metadata.json +++ /dev/null @@ -1,56 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "Apache-2.0", - "tags": [ - "diffusers", - "flux", - "scheduler", - "state-ownership", - "interruption" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/huggingface/diffusers", - "commit": "aea7b70be0f594bcdc15a6f2677ee2ca85e49e8e", - "paths": [ - "src/diffusers/pipelines/pipeline_utils.py", - "src/diffusers/pipelines/flux/pipeline_flux.py", - "src/diffusers/schedulers/scheduling_flow_match_euler_discrete.py", - "tests/pipelines/flux/test_pipeline_flux.py", - "setup.py", - "LICENSE" - ], - "readset": "references/source-map.md" - } - ], - "applicability": { - "scope": "FluxPipeline and FlowMatchEulerDiscreteScheduler invocation ownership at this source pin; inspect the actual caller contract.", - "prerequisites": "Access to the pinned ownership path and an identified caller admission/serialization contract; other pipeline or scheduler classes need their own contracts.", - "input_contract": "Diffusers/PyTorch revisions, pipeline/scheduler classes and object identities, construction/from_pipe path, actual components, scheduler config and resolved timesteps/sigmas, shape/dtype, callback/interrupt timing, output/cleanup path and overlapping intervals.", - "produced_artifact": "An object-identity/ownership map and call-to-schedule-reset/step/interrupt/output/cleanup trace, the caller admission contract, one next check and actual evidence status.", - "failure_modes": "Assuming from_pipe deep-copies scheduler state, passing an enumeration index instead of a scheduler timestep, or treating cooperative iteration skipping as kernel cancellation/exception cleanup can misidentify invocation ownership.", - "refusal_point": "Stop when the ownership question is resolved or a required fixture/backend is absent; do not infer service-wide race or serialization behavior from this readset." - }, - "limitations": [ - "Source-derived candidate; no installed API, native scheduler, model, concurrency or service validation.", - "Supplied manifest verifies a selected pipeline_utils Git blob, not the whole source tree.", - "Other pipelines and schedulers require separate contracts; common indexing is not backend qualification." - ], - "provenance_readsets": [ - "references/source-map.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Check wrapper and scheduler identities explicitly and follow the effective schedule reset and step-index progression.", - "Trace interruption through its loop-boundary consumer and normal decode/cleanup tail while preserving the caller's existing shared-state serialization.", - "Use a real scheduler CPU check only for its scheduler cycle; keep pipeline reuse/interruption and serving claims tied to matching fixtures and execution evidence." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/metadata.json deleted file mode 100644 index 3b4b26ef7..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/metadata.json +++ /dev/null @@ -1,65 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "MIT", - "tags": [ - "mlx", - "lazy-evaluation", - "stream", - "buffer-lifetime", - "synchronization" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ml-explore/mlx", - "commit": "bff10220c742907f0c68e55ff174aaea56f74e3d", - "paths": [ - "mlx/ops.cpp", - "mlx/transforms.cpp", - "mlx/array.cpp", - "mlx/backend/cpu/eval.cpp", - "mlx/backend/metal/eval.cpp", - "mlx/backend/cuda/eval.cpp", - "python/src/transforms.cpp", - "python/tests/test_eval.py", - "LICENSE", - "mlx/scheduler.cpp", - "python/tests/test_memory.py", - "CMakeLists.txt", - "setup.py", - "python/tests/test_device.py", - "python/tests/mlx_tests.py", - ".github/workflows/build_and_test.yml" - ], - "readset": "references/source-map.md" - } - ], - "applicability": { - "scope": "MLX evaluation/stream/buffer diagnosis at the pinned implementation; qualify the actual backend separately.", - "prerequisites": "Access to the pinned MLX execution/lifetime contract and a bounded reproduction with an identified evaluation/read boundary; device-dependent conclusions need the actual selected backend.", - "input_contract": "MLX revision/build, backend/device, Python/C++ entry, array shapes/dtypes, primitive streams, evaluation/read boundary and transformation context; timing needs warmup/work boundaries, while memory needs live references, in-flight buffers, allocator cache and synchronized observations.", - "produced_artifact": "A producer/evaluation/dependency-wait/backend/availability-cleanup trace, ownership table, observed symptom and one discriminating check with build-bound actual evidence status.", - "failure_modes": "Timing graph construction or submission as execution, assuming evaluated means completed, synchronizing only the default stream, or treating graph detachment/Python deletion as reusable storage can misdiagnose time or memory.", - "refusal_point": "Stop when the trace answers the question; if the required backend is absent, retain the source-supported diagnosis and exact runtime gap rather than substituting CPU import success." - }, - "limitations": [ - "Source-derived candidate; no local native-backend, model, service or efficacy validation.", - "Static content checks do not promote this candidate or establish installed API conformance." - ], - "provenance_readsets": [ - "references/source-map.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Separate graph construction, async_eval submission, eval and host reads, and record which stream the measurement actually synchronizes.", - "Preserve the dependency-wait path and selected backend buffer-retention/donation contract before changing streams or assuming storage can be reused.", - "Distinguish live, in-flight and allocator-cached memory; inspect the real error/cleanup path before proposing a leak fix or hiding the failure." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/metadata.json deleted file mode 100644 index ca690ae38..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/metadata.json +++ /dev/null @@ -1,39 +0,0 @@ -{ - "tags": [ - "checkpoint", - "training", - "staging", - "host-memory", - "storage", - "pytorch" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/sources.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Source-supported diagnosis of training-visible PyTorch DCP save pauses, snapshot/staging/upload lifetimes and bounded CPU staging-buffer reuse; distributed recovery validity and backend durability need separate evidence.", - "prerequisites": "Known correspondence between the installed PyTorch API and source/version notes, identified checkpointed-state mutation boundaries and an initial one-outstanding-save policy before overlap is considered.", - "input_contract": "Installed PyTorch revision, stager, storage writer and thread/process executor, checkpoint contents, ranks per host and state-mutation boundaries; performance assessment needs caller/staging/upload waits and host/resource observations.", - "produced_artifact": "A completion/lifetime diagnosis and smallest justified staging/cache/backpressure lever, with separate caller-pause, staging, upload and step-time measurements proposed for authorized validation.", - "failure_modes": "Mutating source state before staging completes, overwriting cached snapshots before prior upload completes, ignoring caller-side state_dict work or treating async return as durability can corrupt the snapshot or misstate the pause.", - "refusal_point": "Reject staging overlap that races checkpointed-state mutation; do not infer backend durability from asynchronous return or writer completion." - }, - "recommendations": [ - "Place the staging wait before the earliest mutation of any checkpointed state, including forward-mutated buffers rather than only optimizer.step.", - "Wait for the previous upload before reusing a stager's CPU storage; observe the final upload before closing caller-owned staging resources.", - "Measure checkpoint and intervening steps with the same state/cadence, retaining first-use and cached-buffer costs plus per-host memory and I/O pressure." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/SKILL.md deleted file mode 100644 index 1d029b97f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/SKILL.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -name: pytorch -description: "Operate and diagnose PyTorch installation, tensors and autograd, training and data, compilation and export, accelerator lifetimes, distributed execution, checkpointing, profiling and custom operators from pinned repository evidence." ---- - -# PyTorch operating graph - -Use this graph to plan a PyTorch workflow or isolate a source, API, numerical, -compiler, device, communication or serialization failure. It covers the major -repository workflows through eight routes. **Candidate** describes knowledge -state; source breadth does not mean every file or backend was executed. - -Read [provenance](references/repo-provenance.md) before transferring a recipe to -another revision. The pinned source is `pytorch/pytorch` at -`9a90a4fefcaafa63ac984621963a6fcc66aee68c` (`2.16.0a0` source version). -Installed-wheel observations are a distinct evidence class. Accelerator and -distributed backend qualification must use the actual target stack. - -| User task | Route | Expected artifact | -| --- | --- | --- | -| Choose a wheel/backend; build Python or LibTorch; repair import/ABI | [installation and build](sub-skills/installation-and-build/SKILL.md) | Environment/build identity and smoke result | -| Tensor layout/device/dtype, missing gradient, sparse/nested, torch.func | [tensor, dispatch and autograd](sub-skills/tensor-dispatch-autograd/SKILL.md) | Tensor/alias/gradient contract | -| nn/optim, training/eval, DataLoader, AMP, SDPA, activation recompute | [training and data](sub-skills/training-and-data/SKILL.md) | Bounded step with state and data checks | -| Dynamo guards, Inductor, FX, export, ONNX, TorchScript/mobile migration | [compiler and export](sub-skills/compiler-and-export/SKILL.md) | Graph/shape/deployment contract | -| OOM, allocated versus reserved bytes, streams, graph replay | [memory, streams and graphs](sub-skills/memory-streams-and-graphs/SKILL.md) | Storage/stream lifetime ledger | -| torchrun, c10d, DDP/FSDP2/DTensor, serialization and DCP | [distributed and checkpoint](sub-skills/distributed-and-checkpoint/SKILL.md) | Rank/collective schedule and restore plan | -| Trace/time/memory, nonfinite results, backend/compiler isolation | [profiling and debugging](sub-skills/profiling-and-debugging/SKILL.md) | Reproduction and bounded measurement | -| torch.library, FakeTensor, gradients, C++/CUDA/SYCL extension | [custom operators and extensions](sub-skills/custom-operators-and-extension/SKILL.md) | Schema, numerical and registration checks | - -Start by identifying source commit or distribution version, wheel backend tag, -Python/OS, device and driver/runtime, workload shapes/dtypes/layouts, grad mode, -state/seed and the smallest failing example. Run the bundled -[environment inspector](scripts/inspect_environment.py) in the chosen environment: - -```bash -python /scripts/inspect_environment.py -python /scripts/inspect_environment.py --require-backend cuda -``` - -`` is this skill's installed directory; helpers run from any current -directory. Inspection reports bounded software facts, never automatic package -repair, model downloads or a performance verdict. Use the installation route -for the source-build recipe and current backend prerequisites. - -For cross-route tasks, preserve the earliest incorrect boundary. A custom op -needs tensor/schema correctness before compiler eligibility; graph replay needs -stream/address lifetimes before timing; distributed resume needs allocation and -state ownership before throughput tuning. Use eager CPU as a control where it -represents the same semantics; preserve a missing device check separately. - -Cross-cutting failures are in [troubleshooting](references/troubleshooting.md). -[Repository coverage](references/source-directory-coverage.md) explicitly maps -all top-level source directories and public module families to their owners or -exclusions. [Readset](references/source-readset.json) distinguishes exact inspected -spans from tree inventory. [Script disposition](references/source-script-map.md) -explains bundled helpers and excluded source tooling. - -Existing owners remain authoritative for their focused procedures: -[host memory management](../../../hardware/host/host-memory-management/SKILL.md) -and [DCP async save performance](../dcp-async-save-performance/SKILL.md). -Their source revisions and evidence stay independent; following a link does not -update their pins or qualify the current PyTorch snapshot. - -Read [notices](references/notices.md) before redistribution. This InferRSI -package preserves the AREX operating-role, visibility and repository-license -fields in local-skill-v1 sidecars. It does not imply deployment into a managed -DisCo installation. No TorchNPU/PyTorch compatibility pairing is inferred. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/metadata.json deleted file mode 100644 index a688df924..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/metadata.json +++ /dev/null @@ -1,131 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "tensor", - "autograd", - "training", - "compiler", - "distributed", - "repository-guide" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "AGENTS.md", - "LICENSE", - "NOTICE", - "README.md", - "SECURITY.md", - "aten/src/ATen/core/dispatch/DispatchKeyExtractor.h", - "aten/src/ATen/core/dispatch/Dispatcher.h", - "aten/src/ATen/native/transformers/cuda/sdp_utils.cpp", - "c10/cuda/CUDACachingAllocator.cpp", - "cmake/EnvVarForwarding.cmake", - "docs/source/distributed/checkpoint.md", - "docs/source/distributed/fsdp.fully_shard.md", - "docs/source/distributed/tensor.md", - "docs/source/func.md", - "docs/source/nested.md", - "docs/source/notes/amp_examples.md", - "docs/source/notes/autograd.md", - "docs/source/notes/cuda.md", - "docs/source/notes/ddp.md", - "docs/source/notes/extending.md", - "docs/source/notes/modules.md", - "docs/source/notes/numerical_accuracy.md", - "docs/source/notes/randomness.md", - "docs/source/notes/serialization.md", - "docs/source/onnx.md", - "docs/source/profiler.md", - "docs/source/quantization.md", - "docs/source/sparse.md", - "docs/source/user_guide/torch_compiler/torch.compiler_troubleshooting.md", - "docs/source/utils/mobile_optimizer.md", - "pyproject.toml", - "setup.py", - "test/export/test_export.py", - "test/profiler/test_profiler.py", - "test/test_autograd.py", - "test/test_cuda.py", - "test/test_custom_ops.py", - "test/test_dataloader.py", - "test/test_serialization.py", - "torch/__init__.py", - "torch/_dynamo/convert_frame.py", - "torch/_dynamo/guards.py", - "torch/_inductor/compile_fx.py", - "torch/_inductor/graph.py", - "torch/_library/custom_ops.py", - "torch/csrc/distributed/c10d/ProcessGroup.hpp", - "torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp", - "torch/distributed/checkpoint/format_utils.py", - "torch/distributed/checkpoint/state_dict_saver.py", - "torch/distributed/distributed_c10d.py", - "torch/distributed/elastic/rendezvous/c10d_rendezvous_backend.py", - "torch/distributed/rendezvous.py", - "torch/distributed/run.py", - "torch/export/__init__.py", - "torch/fx/__init__.py", - "torch/jit/__init__.py", - "torch/library.py", - "torch/nn/functional.py", - "torch/onnx/__init__.py", - "torch/optim/optimizer.py", - "torch/profiler/profiler.py", - "torch/serialization.py", - "torch/utils/benchmark/utils/timer.py", - "torch/utils/checkpoint.py", - "torch/utils/collect_env.py", - "torch/utils/cpp_extension.py", - "torch/utils/data/dataloader.py", - "version.txt" - ], - "readset": "references/source-readset.json" - } - ], - "provenance_readsets": [ - "references/source-readset.json" - ], - "references": [ - "../../../hardware/host/host-memory-management/SKILL.md", - "../dcp-async-save-performance/SKILL.md" - ], - "applicability": { - "scope": "Major user workflows and selected deep mechanisms of pinned pytorch/pytorch source; exact device/operator qualification remains task-specific.", - "prerequisites": [ - "An explicitly selected installed torch environment for runtime helpers.", - "Actual required devices, supported build and trusted ranks/storage for backend/distributed operations." - ], - "input_contract": "A concrete model/tensor/data/environment reproduction and desired operation or expected behavior.", - "produced_artifact": "A route-specific environment, tensor/gradient, graph, memory/stream, rank/state or measurement contract with checked and missing evidence.", - "failure_modes": [ - "Transferring pinned-source behavior to another release/backend without checking compatibility.", - "Confusing source inventory, CPU diagnostics and target backend execution." - ], - "refusal_point": "Withhold backend success and performance conclusions when required runtime or workload evidence is missing." - }, - "limitations": [ - "Source-derived candidate; empirical efficacy not_run.", - "Required CUDA/ROCm/MPS/XPU/external accelerator and NCCL/graph checks unavailable locally retain BLOCKED_REQUIRED_BACKEND.", - "Installed CPU wheel 2.15.0+cpu is separate from pinned source 2.16.0a0; CPU diagnostics are not same-source native verification.", - "No automatic compatibility pairing with independently pinned TorchNPU.", - "Directory inventory does not imply every file/operator/device was semantically read or run." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md deleted file mode 100644 index b23eb562b..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: compiler-and-export -description: "Diagnose PyTorch Dynamo guards, graph breaks and Inductor lowering, or export models through FX, torch.export, ONNX and legacy deployment interfaces." ---- - -# Compiler and export - -Use this route for the task named in the description. Produce a graph and shape contract, a compiler-stage reproduction and a deployment compatibility check. -This is a source-derived **candidate**, with native backend checks separate from -source support. Read the root [provenance](../../references/repo-provenance.md) -when package, API or platform identity differs. - -## Operating sequence - -1. Record the concrete input contract, current behavior, desired result, package - version and backend. Preserve a tiny eager or single-process control. -2. Read [workflows and mechanisms](references/workflows.md) and choose its - matching branch. Check prerequisites before launching a build or device job. -3. Apply the smallest change at the first disagreeing boundary; preserve tensor - shape, dtype, layout, device, mutation and gradient assumptions. -4. Check the predicted output and failure recovery using the linked helper or - recipe. A software check cannot establish an unavailable device capability. -5. Return the artifact named above, checks actually run, unresolved conditions - and the next discriminating check. Missing required backend execution stays - **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. - -Read [troubleshooting](references/troubleshooting.md) when a prerequisite or -check fails. The references contain complete procedures and exact source -permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/metadata.json deleted file mode 100644 index a6fed4c35..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/metadata.json +++ /dev/null @@ -1,54 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "compiler-and-export" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "torch/__init__.py", - "torch/_dynamo/convert_frame.py", - "torch/_dynamo/guards.py", - "torch/_inductor/compile_fx.py", - "torch/_inductor/graph.py", - "torch/export/__init__.py", - "torch/onnx/__init__.py", - "torch/fx/__init__.py" - ], - "readset": "../../references/source-readset.json" - } - ], - "applicability": { - "scope": "Diagnose PyTorch Dynamo guards, graph breaks and Inductor lowering, or export models through FX, torch.export, ONNX and legacy deployment interfaces.", - "prerequisites": [ - "An explicitly selected installed torch environment; actual devices for backend-specific checks." - ], - "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", - "produced_artifact": "Produce a graph and shape contract, a compiler-stage reproduction and a deployment compatibility check.", - "failure_modes": [ - "Transferring source facts to another wheel or backend without checking drift." - ], - "refusal_point": "Withhold backend success when the required device check cannot execute." - }, - "limitations": [ - "Source-supported procedures are not full-repository execution coverage.", - "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/SKILL.md deleted file mode 100644 index 5113312e8..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: custom-operators-and-extension -description: "Implement or qualify PyTorch Python custom operators, FakeTensor and autograd registrations, C++ extensions, and backend dispatch integrations." ---- - -# Custom operators and extension - -Use this route for the task named in the description. Produce a schema, metadata and gradient contract with separate registration, numerical and backend checks. -This is a source-derived **candidate**, with native backend checks separate from -source support. Read the root [provenance](../../references/repo-provenance.md) -when package, API or platform identity differs. - -## Operating sequence - -1. Record the concrete input contract, current behavior, desired result, package - version and backend. Preserve a tiny eager or single-process control. -2. Read [workflows and mechanisms](references/workflows.md) and choose its - matching branch. Check prerequisites before launching a build or device job. -3. Apply the smallest change at the first disagreeing boundary; preserve tensor - shape, dtype, layout, device, mutation and gradient assumptions. -4. Check the predicted output and failure recovery using the linked helper or - recipe. A software check cannot establish an unavailable device capability. -5. Return the artifact named above, checks actually run, unresolved conditions - and the next discriminating check. Missing required backend execution stays - **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. - -Read [troubleshooting](references/troubleshooting.md) when a prerequisite or -check fails. The references contain complete procedures and exact source -permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/metadata.json deleted file mode 100644 index 703293349..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/metadata.json +++ /dev/null @@ -1,51 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "custom-operators-and-extension" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "torch/_library/custom_ops.py", - "torch/library.py", - "torch/utils/cpp_extension.py", - "docs/source/notes/extending.md", - "test/test_custom_ops.py" - ], - "readset": "../../references/source-readset.json" - } - ], - "applicability": { - "scope": "Implement or qualify PyTorch Python custom operators, FakeTensor and autograd registrations, C++ extensions, and backend dispatch integrations.", - "prerequisites": [ - "An explicitly selected installed torch environment; actual devices for backend-specific checks." - ], - "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", - "produced_artifact": "Produce a schema, metadata and gradient contract with separate registration, numerical and backend checks.", - "failure_modes": [ - "Transferring source facts to another wheel or backend without checking drift." - ], - "refusal_point": "Withhold backend success when the required device check cannot execute." - }, - "limitations": [ - "Source-supported procedures are not full-repository execution coverage.", - "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md deleted file mode 100644 index 9a02d9364..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: distributed-and-checkpoint -description: "Set up or debug PyTorch c10d, DDP, FSDP2, DeviceMesh and DTensor, distributed checkpointing and trusted state serialization." ---- - -# Distributed and checkpoint - -Use this route for the task named in the description. Produce a rank and collective schedule and a state-restore contract with explicit trust and backend boundaries. -This is a source-derived **candidate**, with native backend checks separate from -source support. Read the root [provenance](../../references/repo-provenance.md) -when package, API or platform identity differs. - -## Operating sequence - -1. Record the concrete input contract, current behavior, desired result, package - version and backend. Preserve a tiny eager or single-process control. -2. Read [workflows and mechanisms](references/workflows.md) and choose its - matching branch. Check prerequisites before launching a build or device job. -3. Apply the smallest change at the first disagreeing boundary; preserve tensor - shape, dtype, layout, device, mutation and gradient assumptions. -4. Check the predicted output and failure recovery using the linked helper or - recipe. A software check cannot establish an unavailable device capability. -5. Return the artifact named above, checks actually run, unresolved conditions - and the next discriminating check. Missing required backend execution stays - **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. - -Read [troubleshooting](references/troubleshooting.md) when a prerequisite or -check fails. The references contain complete procedures and exact source -permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/metadata.json deleted file mode 100644 index 84b7fc69c..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/metadata.json +++ /dev/null @@ -1,55 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "distributed-and-checkpoint" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "torch/distributed/distributed_c10d.py", - "torch/csrc/distributed/c10d/ProcessGroup.hpp", - "torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp", - "docs/source/notes/ddp.md", - "docs/source/distributed/fsdp.fully_shard.md", - "docs/source/distributed/tensor.md", - "torch/serialization.py", - "torch/distributed/checkpoint/format_utils.py", - "SECURITY.md" - ], - "readset": "../../references/source-readset.json" - } - ], - "applicability": { - "scope": "Set up or debug PyTorch c10d, DDP, FSDP2, DeviceMesh and DTensor, distributed checkpointing and trusted state serialization.", - "prerequisites": [ - "An explicitly selected installed torch environment; actual devices for backend-specific checks." - ], - "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", - "produced_artifact": "Produce a rank and collective schedule and a state-restore contract with explicit trust and backend boundaries.", - "failure_modes": [ - "Transferring source facts to another wheel or backend without checking drift." - ], - "refusal_point": "Withhold backend success when the required device check cannot execute." - }, - "limitations": [ - "Source-supported procedures are not full-repository execution coverage.", - "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/SKILL.md deleted file mode 100644 index 7963fed19..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: installation-and-build -description: "Install, inspect, or build PyTorch for a selected OS, Python ABI and device backend; diagnose import and binary-extension mismatches." ---- - -# Installation and build - -Use this route for the task named in the description. Build a versioned environment plan and distinguish source, wheel, toolkit and driver identities. -This is a source-derived **candidate**, with native backend checks separate from -source support. Read the root [provenance](../../references/repo-provenance.md) -when package, API or platform identity differs. - -## Operating sequence - -1. Record the concrete input contract, current behavior, desired result, package - version and backend. Preserve a tiny eager or single-process control. -2. Read [workflows and mechanisms](references/workflows.md) and choose its - matching branch. Check prerequisites before launching a build or device job. -3. Apply the smallest change at the first disagreeing boundary; preserve tensor - shape, dtype, layout, device, mutation and gradient assumptions. -4. Check the predicted output and failure recovery using the linked helper or - recipe. A software check cannot establish an unavailable device capability. -5. Return the artifact named above, checks actually run, unresolved conditions - and the next discriminating check. Missing required backend execution stays - **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. - -Read [troubleshooting](references/troubleshooting.md) when a prerequisite or -check fails. The references contain complete procedures and exact source -permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/metadata.json deleted file mode 100644 index 15dd74687..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/metadata.json +++ /dev/null @@ -1,50 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "installation-and-build" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "README.md", - "pyproject.toml", - "setup.py", - "cmake/EnvVarForwarding.cmake" - ], - "readset": "../../references/source-readset.json" - } - ], - "applicability": { - "scope": "Install, inspect, or build PyTorch for a selected OS, Python ABI and device backend; diagnose import and binary-extension mismatches.", - "prerequisites": [ - "An explicitly selected installed torch environment; actual devices for backend-specific checks." - ], - "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", - "produced_artifact": "Build a versioned environment plan and distinguish source, wheel, toolkit and driver identities.", - "failure_modes": [ - "Transferring source facts to another wheel or backend without checking drift." - ], - "refusal_point": "Withhold backend success when the required device check cannot execute." - }, - "limitations": [ - "Source-supported procedures are not full-repository execution coverage.", - "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md deleted file mode 100644 index 445c12d2e..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: memory-streams-and-graphs -description: "Diagnose PyTorch accelerator memory, CUDA caching allocation, asynchronous stream dependencies and CUDA graph replay lifetimes." ---- - -# Memory streams and graphs - -Use this route for the task named in the description. Produce a storage and stream lifetime ledger plus a bounded allocator or graph reproduction. -This is a source-derived **candidate**, with native backend checks separate from -source support. Read the root [provenance](../../references/repo-provenance.md) -when package, API or platform identity differs. - -## Operating sequence - -1. Record the concrete input contract, current behavior, desired result, package - version and backend. Preserve a tiny eager or single-process control. -2. Read [workflows and mechanisms](references/workflows.md) and choose its - matching branch. Check prerequisites before launching a build or device job. -3. Apply the smallest change at the first disagreeing boundary; preserve tensor - shape, dtype, layout, device, mutation and gradient assumptions. -4. Check the predicted output and failure recovery using the linked helper or - recipe. A software check cannot establish an unavailable device capability. -5. Return the artifact named above, checks actually run, unresolved conditions - and the next discriminating check. Missing required backend execution stays - **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. - -Read [troubleshooting](references/troubleshooting.md) when a prerequisite or -check fails. The references contain complete procedures and exact source -permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/metadata.json deleted file mode 100644 index 8438743db..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/metadata.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "memory-streams-and-graphs" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "docs/source/notes/cuda.md", - "c10/cuda/CUDACachingAllocator.cpp", - "test/test_cuda.py" - ], - "readset": "../../references/source-readset.json" - } - ], - "applicability": { - "scope": "Diagnose PyTorch accelerator memory, CUDA caching allocation, asynchronous stream dependencies and CUDA graph replay lifetimes.", - "prerequisites": [ - "An explicitly selected installed torch environment; actual devices for backend-specific checks." - ], - "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", - "produced_artifact": "Produce a storage and stream lifetime ledger plus a bounded allocator or graph reproduction.", - "failure_modes": [ - "Transferring source facts to another wheel or backend without checking drift." - ], - "refusal_point": "Withhold backend success when the required device check cannot execute." - }, - "limitations": [ - "Source-supported procedures are not full-repository execution coverage.", - "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/SKILL.md deleted file mode 100644 index c0120c05e..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: profiling-and-debugging -description: "Profile PyTorch operators and memory, benchmark controlled workloads, diagnose compiler stalls, nonfinite results and asynchronous device failures." ---- - -# Profiling and debugging - -Use this route for the task named in the description. Produce a reproducible trace or timing design and a claim bounded by the measured workload. -This is a source-derived **candidate**, with native backend checks separate from -source support. Read the root [provenance](../../references/repo-provenance.md) -when package, API or platform identity differs. - -## Operating sequence - -1. Record the concrete input contract, current behavior, desired result, package - version and backend. Preserve a tiny eager or single-process control. -2. Read [workflows and mechanisms](references/workflows.md) and choose its - matching branch. Check prerequisites before launching a build or device job. -3. Apply the smallest change at the first disagreeing boundary; preserve tensor - shape, dtype, layout, device, mutation and gradient assumptions. -4. Check the predicted output and failure recovery using the linked helper or - recipe. A software check cannot establish an unavailable device capability. -5. Return the artifact named above, checks actually run, unresolved conditions - and the next discriminating check. Missing required backend execution stays - **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. - -Read [troubleshooting](references/troubleshooting.md) when a prerequisite or -check fails. The references contain complete procedures and exact source -permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/metadata.json deleted file mode 100644 index b114715f1..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/metadata.json +++ /dev/null @@ -1,51 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "profiling-and-debugging" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "torch/profiler/profiler.py", - "torch/utils/benchmark/utils/timer.py", - "torch/utils/collect_env.py", - "docs/source/notes/numerical_accuracy.md", - "docs/source/notes/randomness.md" - ], - "readset": "../../references/source-readset.json" - } - ], - "applicability": { - "scope": "Profile PyTorch operators and memory, benchmark controlled workloads, diagnose compiler stalls, nonfinite results and asynchronous device failures.", - "prerequisites": [ - "An explicitly selected installed torch environment; actual devices for backend-specific checks." - ], - "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", - "produced_artifact": "Produce a reproducible trace or timing design and a claim bounded by the measured workload.", - "failure_modes": [ - "Transferring source facts to another wheel or backend without checking drift." - ], - "refusal_point": "Withhold backend success when the required device check cannot execute." - }, - "limitations": [ - "Source-supported procedures are not full-repository execution coverage.", - "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/SKILL.md deleted file mode 100644 index 4cb2718bd..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: tensor-dispatch-autograd -description: "Diagnose PyTorch tensor layouts, dtype and device dispatch, autograd history, function transforms, sparse and nested tensor contracts." ---- - -# Tensor dispatch autograd - -Use this route for the task named in the description. Produce a tensor and gradient contract, then isolate the first wrong operator or dispatch boundary. -This is a source-derived **candidate**, with native backend checks separate from -source support. Read the root [provenance](../../references/repo-provenance.md) -when package, API or platform identity differs. - -## Operating sequence - -1. Record the concrete input contract, current behavior, desired result, package - version and backend. Preserve a tiny eager or single-process control. -2. Read [workflows and mechanisms](references/workflows.md) and choose its - matching branch. Check prerequisites before launching a build or device job. -3. Apply the smallest change at the first disagreeing boundary; preserve tensor - shape, dtype, layout, device, mutation and gradient assumptions. -4. Check the predicted output and failure recovery using the linked helper or - recipe. A software check cannot establish an unavailable device capability. -5. Return the artifact named above, checks actually run, unresolved conditions - and the next discriminating check. Missing required backend execution stays - **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. - -Read [troubleshooting](references/troubleshooting.md) when a prerequisite or -check fails. The references contain complete procedures and exact source -permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/metadata.json deleted file mode 100644 index 3b2f7d5c9..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/metadata.json +++ /dev/null @@ -1,52 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "tensor-dispatch-autograd" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "docs/source/notes/autograd.md", - "aten/src/ATen/core/dispatch/Dispatcher.h", - "aten/src/ATen/core/dispatch/DispatchKeyExtractor.h", - "docs/source/sparse.md", - "docs/source/nested.md", - "docs/source/func.md" - ], - "readset": "../../references/source-readset.json" - } - ], - "applicability": { - "scope": "Diagnose PyTorch tensor layouts, dtype and device dispatch, autograd history, function transforms, sparse and nested tensor contracts.", - "prerequisites": [ - "An explicitly selected installed torch environment; actual devices for backend-specific checks." - ], - "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", - "produced_artifact": "Produce a tensor and gradient contract, then isolate the first wrong operator or dispatch boundary.", - "failure_modes": [ - "Transferring source facts to another wheel or backend without checking drift." - ], - "refusal_point": "Withhold backend success when the required device check cannot execute." - }, - "limitations": [ - "Source-supported procedures are not full-repository execution coverage.", - "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/SKILL.md deleted file mode 100644 index eaf49c972..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: training-and-data -description: "Build or debug PyTorch modules, optimizers, data loaders, mixed precision, activation checkpointing and attention training or evaluation." ---- - -# Training and data - -Use this route for the task named in the description. Produce a bounded training or evaluation recipe with explicit sample, state, gradient and numerical checks. -This is a source-derived **candidate**, with native backend checks separate from -source support. Read the root [provenance](../../references/repo-provenance.md) -when package, API or platform identity differs. - -## Operating sequence - -1. Record the concrete input contract, current behavior, desired result, package - version and backend. Preserve a tiny eager or single-process control. -2. Read [workflows and mechanisms](references/workflows.md) and choose its - matching branch. Check prerequisites before launching a build or device job. -3. Apply the smallest change at the first disagreeing boundary; preserve tensor - shape, dtype, layout, device, mutation and gradient assumptions. -4. Check the predicted output and failure recovery using the linked helper or - recipe. A software check cannot establish an unavailable device capability. -5. Return the artifact named above, checks actually run, unresolved conditions - and the next discriminating check. Missing required backend execution stays - **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. - -Read [troubleshooting](references/troubleshooting.md) when a prerequisite or -check fails. The references contain complete procedures and exact source -permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/metadata.json deleted file mode 100644 index 3561a02af..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/metadata.json +++ /dev/null @@ -1,52 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "pytorch", - "training-and-data" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", - "paths": [ - "torch/utils/data/dataloader.py", - "docs/source/notes/modules.md", - "docs/source/notes/amp_examples.md", - "torch/utils/checkpoint.py", - "torch/nn/functional.py", - "aten/src/ATen/native/transformers/cuda/sdp_utils.cpp" - ], - "readset": "../../references/source-readset.json" - } - ], - "applicability": { - "scope": "Build or debug PyTorch modules, optimizers, data loaders, mixed precision, activation checkpointing and attention training or evaluation.", - "prerequisites": [ - "An explicitly selected installed torch environment; actual devices for backend-specific checks." - ], - "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", - "produced_artifact": "Produce a bounded training or evaluation recipe with explicit sample, state, gradient and numerical checks.", - "failure_modes": [ - "Transferring source facts to another wheel or backend without checking drift." - ], - "refusal_point": "Withhold backend success when the required device check cannot execute." - }, - "limitations": [ - "Source-supported procedures are not full-repository execution coverage.", - "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md deleted file mode 100644 index 1745a0ca6..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/SKILL.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -name: torch-npu -description: "Operate and diagnose TorchNPU on Ascend across installation, PrivateUse1 dispatch, operator development, memory and streams, graphs and compilation, HCCL, precision and profiling, with pinned source evidence and explicit backend limits." ---- - -# TorchNPU operating graph - -Use for Ascend/pytorch's TorchNPU adapter. Collect exact torch/torch-npu builds, -CANN/driver/hardware, the selected eager/compile/distributed mode, tensor -shape/dtype/layout and a bounded reproduction. This is a source-derived -candidate, not a validated installed stack. NPU, CANN, HCCL and optional compiler -checks remain `BLOCKED_REQUIRED_BACKEND` until exercised on the actual backend. - -Choose the narrowest route, then compose routes when the failure crosses an -ownership boundary: - -| User goal or symptom | Owning procedure | -|---|---| -| Install, select versions, missing shared library, build/package mismatch | [Installation and compatibility](sub-skills/installation-and-compatibility/SKILL.md) | -| Missing `torch.npu`, import ordering, unsupported op or hidden CPU fallback | [PrivateUse1 and dispatch](sub-skills/privateuse1-and-dispatch/SKILL.md) | -| Add an overload/custom op; fix schema, backward or Meta support | [Operator development](sub-skills/operator-development/SKILL.md) | -| Async corruption, stream waits, allocation reuse, pinned copies, memory retention | [Memory, streams and events](sub-skills/memory-streams-and-events/SKILL.md) | -| NPUGraph replay, `torch.compile`, graph breaks, compiler choice or eager fallback | [Graph and compile](sub-skills/graph-and-compile/SKILL.md) | -| Multi-rank launch, HCCL/P2P, hangs, work completion or sharding boundary | [Distributed and HCCL](sub-skills/distributed-and-hccl/SKILL.md) | -| Training migration, autocast/scaler, overflow, optimizer or checkpoint state | [Training and precision](sub-skills/training-and-precision/SKILL.md) | -| Trace collection, memory timeline, async exception or sanitizer | [Profiling and debugging](sub-skills/profiling-and-debugging/SKILL.md) | - -For migration shims, serialization/export, fused modules, RPC, DTensor/FSDP, -FXRT deployment and fault tooling, read [support workflows](references/support-workflows.md). -The [capability inventory](references/capability-inventory.md) distinguishes -substantive inspection from directory discovery and names remaining boundaries. - -## Operating contract - -1. Qualify the environment before executing a device workflow. Run - `python /scripts/inspect_environment.py` for package metadata only; - use `--device` explicitly for a tiny allocation/matmul/completion check. -2. State the relevant source mechanism and the actual last inspected boundary. - Use [the evidence ledger](references/source-map.md) and - [immutable provenance](references/repo-provenance.md) for revision alignment. -3. Establish correctness before changing precision, compilation, allocator or - overlap policy. Keep producer ordering, storage lifetime and completion - observations distinct. -4. Use [runtime smoke examples](scripts/runtime_smoke.py) only after selecting - their exact case and `--run`; the default is non-executing. The - [operator contract checker](scripts/operator_contract.py) requires only - standard-library Python and checks declarations, not a device kernel. -5. Return exact stack identities, chosen path, observations and pass/fail/not-run - scope. Restore a known-correct configuration if a bounded experiment fails. - -Expected output is an executable proposal or diagnostic trace with explicit -verification state. Reading this graph does not authorize installations, -multi-rank launches, device reset, stress tests or arbitrary upstream scripts. -Use only the operations needed for the caller's authorized task. - -## Boundaries and related owners - -Do not infer device readiness from registration, NPU coverage from CPU fallback, -memory safety from ordering alone, completed HCCL transfer from a host future, -compilation from an eager fallback, or service gains from submission timing. -CPU/helper/static checks cannot substitute for required accelerator execution. -No convergence, benchmark gain, L4 service result or candidate promotion is -established by this graph. - -Existing skill IDs `torchnpu-device-memory-lifecycle` and -`torchnpu-pinned-host-memory-lifecycle` own focused allocator tuning at their -earlier exact source pin. Metadata declares those relationships; their evidence -is not refreshed here. This graph provides newer end-to-end operating routes -and preserves both revision boundaries. - -Full-repository inventory does not mean every operator, CANN kernel, compiler -pass or device family was inspected. The pinned op-plugin readset covers real -adapter/generator/registration mechanisms; other gitlinks remain explicit -external dependencies. See [source-script decisions](references/source-script-inventory.md) -and [attribution and retained licenses](references/NOTICE.md). diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/metadata.json deleted file mode 100644 index 02a7d2481..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/metadata.json +++ /dev/null @@ -1,116 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "references": [ - "../../../hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md", - "../../../hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md" - ], - "provenance_readsets": [ - "references/source-map.md", - "references/source-manifest.json" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/Ascend/pytorch", - "commit": "046934140b96836c3fcef14d0f39ae54dc0a06bf", - "paths": [ - "COMPATIBILITY.en.md", - "README.md", - "docs/en/api/environment_variable/op_execution/ASCEND_LAUNCH_BLOCKING.md", - "docs/en/installation_guide/building_from_source.md", - "fxrt/README.md", - "generate_code.sh", - "setup.py", - "test/distributed/test_hccl_group.py", - "test/npu/test_aclgraph_multi_stream.py", - "test/npu/test_amp.py", - "test/npu/test_cpu_fallback_control.py", - "test/npu/test_import_contract.py", - "test/npu/test_stream.py", - "tools/setup_helpers/version.py", - "torch_npu/__init__.py", - "torch_npu/_inductor/__init__.py", - "torch_npu/_init/core/module_loader.py", - "torch_npu/_init/registry/backend.py", - "torch_npu/_init/registry/distributed.py", - "torch_npu/_init/registry/dynamo.py", - "torch_npu/_init/registry/registry_manager.py", - "torch_npu/contrib/transfer_to_npu.py", - "torch_npu/csrc/aten/AutoCastOps.cpp", - "torch_npu/csrc/aten/VariableFallbackKernel.cpp", - "torch_npu/csrc/core/npu/CachingHostAllocator.cpp", - "torch_npu/csrc/core/npu/NPUCachingAllocator.cpp", - "torch_npu/csrc/core/npu/NPUEvent.cpp", - "torch_npu/csrc/core/npu/NPUGraph.cpp", - "torch_npu/csrc/core/npu/NPUHooksInterface.cpp", - "torch_npu/csrc/core/npu/NPUStream.cpp", - "torch_npu/csrc/distributed/ProcessGroupHCCL.cpp", - "torch_npu/csrc/framework/utils/CpuFallbackUtils.h", - "torch_npu/csrc/profiler/profiler_mgr.cpp", - "torch_npu/distributed/tensor/__init__.py", - "torch_npu/dynamo/__init__.py", - "torch_npu/npu/__init__.py", - "torch_npu/npu/_sanitizer.py", - "torch_npu/npu/amp/autocast_mode.py", - "torch_npu/npu/amp/grad_scaler.py", - "torch_npu/npu/graphs.py", - "torch_npu/npu/streams.py", - "torch_npu/onnx/__init__.py", - "torch_npu/optim/__init__.py", - "torch_npu/profiler/profiler.py", - "torch_npu/profiler/scheduler.py", - "torch_npu/utils/serialization.py", - "torchnpugen/gen_backend_stubs.py", - "version.txt" - ], - "readset": "references/source-manifest.json" - }, - { - "type": "git", - "url": "https://github.com/Ascend/op-plugin", - "commit": "ad946817034b3270e309c81c54f0ee808d0b62cb", - "paths": [ - "examples/cpp_extension_full/torch_lib_impl/cpp_extension_full/ops.py", - "examples/cpp_extension_full/torch_lib_impl/csrc/add_custom.cpp", - "op_plugin/config/op_plugin_functions.yaml", - "op_plugin/ops/aclops/AddKernelNpu.cpp", - "op_plugin/ops/opapi/AddKernelNpuOpApi.cpp", - "torchnpugen/gen_op_plugin_functions.py" - ], - "readset": "references/source-manifest.json" - } - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md deleted file mode 100644 index 78ce3108d..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -name: distributed-and-hccl -description: "Launch and diagnose TorchNPU HCCL communication, process groups, asynchronous work, stream ownership and higher-level distributed integration boundaries." ---- - -# Run and diagnose HCCL - -Use for NPU distributed launch, collectives/P2P, hangs, work completion, -communication-stream memory or FSDP/DTensor integration. Read [the HCCL procedure](references/hccl.md) -and [failure recovery](references/troubleshooting.md). - -Inputs: ranks/world/local ranks, one selected device per process, rendezvous, -stack/topology, collective order/tensor metadata and timeouts. Output: rank -mapping, ordered collective trace, correctness observations and exact blocked -runtime/dependency gates. - -1. Qualify one-device execution, then establish rank/device mapping. -2. Initialize HCCL with a finite timeout and identical collective sequence. -3. Distinguish producer dependency, work handle, consumer-stream dependency and - host completion. -4. Use [the bundled case](../../scripts/runtime_smoke.py) under `torchrun` - with `--case hccl --run`; it never launches additional workers itself. -5. Escalate to the named higher-level integration only after collective - correctness. Tie failures to the first differing rank/sequence. - -Single-device checks, Gloo CPU collectives and source inspection cannot clear -HCCL `BLOCKED_REQUIRED_BACKEND`. Read [memory](../memory-streams-and-events/SKILL.md) -before changing communication-memory reuse policy. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/metadata.json deleted file mode 100644 index 10f94d6b9..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch", - "distributed-and-hccl" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "provenance_readsets": [ - "references/source-map.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md deleted file mode 100644 index 1f0a9b0c8..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -name: graph-and-compile -description: "Choose and debug TorchNPU manual NPUGraph, npugraphs, npugraph_ex, TorchAir, Inductor and FXRT compilation routes with explicit storage and backend gates." ---- - -# Choose capture and compilation paths - -Use for graph replay, `torch.compile`, graph breaks, stale results, compiler -dependency failures or unexpected eager fallback. Read [the graph procedure](references/graphs.md) -and [failure recovery](references/troubleshooting.md). - -Inputs: exact stack/revisions, backend name, eager function, input shapes/dtypes/ -strides, mutation and aliasing, state/RNG, capture stream and output lifetime. -Output: selected backend contract, eager-versus-compiled/replayed comparison, -capture assumptions and unresolved compiler/device checks. - -1. Establish a correct eager NPU result before capture. -2. Select manual NPUGraph, built-in graph backend, TorchAir, Inductor or FXRT - explicitly; record optional compiler availability. -3. Preserve stable storage and legal stream/capture operations. -4. Compare updated-input replays, then benchmark after warmup and synchronization. -5. Use [the bundled graph case](../../scripts/runtime_smoke.py) with - `--case graph --run` for a tiny manual replay check, not compiler conformance. - -Use [memory](../memory-streams-and-events/SKILL.md) for pool/lifetime questions -and [operator development](../operator-development/SKILL.md) for missing Meta -or custom-op registrations. TorchAir and CANN internals remain external evidence -boundaries; no backend speedup is established by this source study. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/metadata.json deleted file mode 100644 index ff5635a4a..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch", - "graph-and-compile" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "provenance_readsets": [ - "references/source-map.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/SKILL.md deleted file mode 100644 index afde54499..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/SKILL.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -name: installation-and-compatibility -description: "Select a compatible TorchNPU, PyTorch, Python, CANN and driver stack; distinguish released wheels from source builds and diagnose import failures." ---- - -# Install and qualify TorchNPU - -Use for a new Ascend environment, version mismatch, wheel/source build decision, -or missing CANN library. Start with [the stack procedure](references/stack.md). -Use [failure recovery](references/troubleshooting.md) before changing packages. - -Inputs: hardware family, Linux architecture, driver/firmware/CANN versions, -Python ABI, exact torch/torch-npu builds, binary versus source intent, and access -to the device. Output: one explicit compatible stack proposal, installation -commands and separate import/device observations. - -1. Separate the released package matrix from the source branch's build lines. -2. Establish driver, CANN and environment setup before package import. -3. Install matching PyTorch and TorchNPU package lines in a private environment. -4. Run [the bundled probe](../../scripts/inspect_environment.py) without device - execution, then use `--device` only on an authorized NPU host. -5. Record actual versions and all blocked checks; route missing backend - registration to [dispatch](../privateuse1-and-dispatch/SKILL.md). - -Source-derived candidate: no wheel install, source build or NPU operation was -validated by this contribution. A clean pip dependency report is insufficient -to qualify the CANN/driver/device combination. Preserve `BLOCKED_REQUIRED_BACKEND` -when the actual Ascend environment is absent. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/metadata.json deleted file mode 100644 index a26a99537..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch", - "installation-and-compatibility" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "provenance_readsets": [ - "references/source-map.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md deleted file mode 100644 index 43363af9a..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -name: memory-streams-and-events -description: "Reason about TorchNPU device and pinned-host ownership, stream dependencies, events, asynchronous copies, allocator caches and graph interaction." ---- - -# Trace memory and stream lifetimes - -Use for asynchronous corruption, unexpected memory retention, stream races, -pinned copy lifetime or misleading timings. Read [the ownership procedure](references/lifetimes.md) -and [failure recovery](references/troubleshooting.md). - -Inputs: allocation owner, tensor/storage aliases, allocating/using streams, -event edges, host-copy boundaries, allocator configuration and exact build. -Output: ordering diagram, ownership table, synchronized measurements and one -bounded discriminating check. - -1. Identify producer, consumer and completion boundaries. -2. Establish stream ordering and allocation lifetime separately. -3. Classify live, active, reserved and host-pinned memory independently. -4. Run [the bundled stream case](../../scripts/runtime_smoke.py) with - `--case streams --run` on the qualified NPU host. -5. Route capture to [graphs](../graph-and-compile/SKILL.md) and communication - ownership to [HCCL](../distributed-and-hccl/SKILL.md). - -The existing library owners `torchnpu-device-memory-lifecycle` and -`torchnpu-pinned-host-memory-lifecycle` retain their earlier immutable source -pins. Load them by declared metadata reference for allocator-tuning depth; do -not treat this newer operating graph as an update to their evidence. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/metadata.json deleted file mode 100644 index 12d74a5c9..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch", - "memory-streams-and-events" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "provenance_readsets": [ - "references/source-map.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/SKILL.md deleted file mode 100644 index 7df14ba81..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/SKILL.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -name: operator-development -description: "Develop or diagnose TorchNPU operators across schema/code generation, PrivateUse1, autograd, Meta, ACLNN and legacy ACL adapter paths." ---- - -# Develop and validate an NPU operator - -Use for a missing overload, custom operator, wrong dtype/shape/aliasing behavior, -or compile/backward support gap. Read [the implementation procedure](references/development.md) -and [failure recovery](references/troubleshooting.md). - -Inputs: exact schema, intended dtype/layout/device/shape range, mutability, -gradient requirements, PyTorch/TorchNPU/op-plugin/CANN revisions, target hardware -and reference outputs. Output: a schema/registration/kernel-boundary plan, -implementation checklist and correctness matrix with unverified entries. - -1. Validate the contract with [the bundled checker](../../scripts/operator_contract.py). -2. Choose built-in operator adaptation versus a namespaced extension. -3. Implement device, autograd and Meta behavior as separate responsibilities. -4. Exercise scalar, broadcast, out/in-place, dtype and layout boundaries before - compilation or graph capture. -5. Report L0 results from the actual accelerator and retain external-kernel - boundaries. Source inspection does not establish custom kernel correctness. - -Use [dispatch](../privateuse1-and-dispatch/SKILL.md) to identify the actual -entry path and [graphs](../graph-and-compile/SKILL.md) for capture constraints. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/metadata.json deleted file mode 100644 index fb22deeaf..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch", - "operator-development" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "provenance_readsets": [ - "references/source-map.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/SKILL.md deleted file mode 100644 index 6dc570935..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: privateuse1-and-dispatch -description: "Trace TorchNPU initialization, PrivateUse1 registration, lazy device setup, operator dispatch and explicit CPU fallback boundaries." ---- - -# Trace registration and dispatch - -Use when `torch.npu` is missing, a tensor unexpectedly executes on CPU, -PrivateUse1 registration conflicts, an autograd path fails, or import loads -unexpected compiler state. Read [registration and dispatch](references/dispatch.md) -and [failure recovery](references/troubleshooting.md). - -Inputs: plugin/torch revision, fresh-process import order, autoload settings, -operator schema/overload, tensor devices/dtypes and gradient context. Output: -an initialization or dispatch trace with the selected kernel/fallback and the -first failing boundary. - -1. Locate failure in core extension, backend registration, patches, lifecycle - barrier or optional feature setup. -2. Separate renamed PrivateUse1/device methods from actual runtime initialization. -3. For one operator, identify schema, dispatch key, implementation and fallback. -4. Check forward, autograd and Meta paths independently; route missing kernels - to [operator development](../operator-development/SKILL.md). -5. Validate with a fresh interpreter and the actual NPU stack. Record CPU fallback - rather than describing a CPU result as NPU coverage. - -For broad CUDA-to-NPU migration, [the support reference](../../references/support-workflows.md) -describes the opt-in transfer shim and its process-wide effects. Importing this -skill never enables that shim. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/metadata.json deleted file mode 100644 index 907ebcc70..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch", - "privateuse1-and-dispatch" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "provenance_readsets": [ - "references/source-map.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/SKILL.md deleted file mode 100644 index 95867cf7a..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/SKILL.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -name: profiling-and-debugging -description: "Capture bounded TorchNPU CPU/NPU traces, memory timelines and synchronization-aware diagnostics; distinguish collection, parsing, completion and performance claims." ---- - -# Profile and debug TorchNPU - -Use for kernel/communication/host gaps, memory traces, async exceptions or stream -sanitizer diagnosis. Read [the profiling procedure](references/profiling.md) -and [failure recovery](references/troubleshooting.md). - -Inputs: tiny correct reproduction, stack/backend, selected activities, bounded -schedule, trace output location and question to discriminate. Output: trace -collection/analysis status, relevant events/counters, completion boundary and -one source-grounded hypothesis. - -1. Reproduce correctness without profiling. -2. Select CPU/NPU activities and minimal trace options. -3. Advance the schedule with `prof.step()` and verify actual artifacts. -4. For memory timelines include the required shape/memory/stack-or-module options. -5. Use [the bundled profile case](../../scripts/runtime_smoke.py) with - `--case profile --run --output ` on a qualified host. - -For async error location use a fresh bounded run with the documented launch -blocking setting, then restore the normal timing mode. Profiling overhead and -microbenchmarks do not establish L4 service performance. Device trace collection -remains `BLOCKED_REQUIRED_BACKEND` without NPU/CANN profiling support. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/metadata.json deleted file mode 100644 index 83fd984e9..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch", - "profiling-and-debugging" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "provenance_readsets": [ - "references/source-map.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/SKILL.md b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/SKILL.md deleted file mode 100644 index de50b4bbf..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/SKILL.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -name: training-and-precision -description: "Migrate a bounded TorchNPU training step, select autocast and gradient-scaling behavior, compare optimizer/checkpoint state and diagnose precision failures." ---- - -# Validate training and precision - -Use for a first NPU training step, autocast/GradScaler, gradient clipping, -overflow, fused optimizer migration or numerical mismatch. Read [the precision procedure](references/training.md) -and [failure recovery](references/troubleshooting.md). - -Inputs: model/loss/optimizer, input dtype/layout, gradient accumulation, scaler, -distributed mode, expected tolerances and exact stack. Output: one-step -correctness/overflow/state comparison with a named validation scope. - -1. Establish FP32 eager forward/backward and optimizer behavior. -2. Introduce autocast and scaling deliberately; check finite gradients and - skipped steps separately from loss values. -3. Unscale before clipping, and update the scale only at the optimizer boundary. -4. Run [the bundled AMP case](../../scripts/runtime_smoke.py) with - `--case amp --run` on an NPU; preserve disabled-scaler and overflow observations. -5. Test optimizer/checkpoint changes separately, then add distributed or graph - execution through their owning routes. - -A finite one-step loss is not convergence, numerical-quality, throughput or -service evidence. Custom-op gradients belong to [operator development](../operator-development/SKILL.md). diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/metadata.json b/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/metadata.json deleted file mode 100644 index fbe1067bc..000000000 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "tags": [ - "torch-npu", - "ascend", - "pytorch", - "training-and-precision" - ], - "applicability": { - "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", - "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", - "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", - "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", - "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", - "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", - "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." - }, - "limitations": [ - "Source-derived candidate; installed API and NPU/CANN execution were not validated.", - "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", - "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", - "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." - ], - "consumption_mode": "packaged", - "provenance_readsets": [ - "references/source-map.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json deleted file mode 100644 index bd58f81b4..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json +++ /dev/null @@ -1,39 +0,0 @@ -{ - "tags": [ - "ascend", - "vllm-ascend", - "aclgraph", - "runtime", - "stream-synchronization" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Diagnose vLLM-Ascend ACLGraph eligibility, batch descriptor matching, capture/replay dispatch, caller-buffer ownership, multi-stream ordering, and capture-resource failures.", - "prerequisites": "An existing eager correctness baseline; authorization before any new run or live configuration change.", - "input_contract": "Installed stack and device, requested/effective graph mode, runner/backend settings, capture sizes and batch descriptors, error phase, replay observations and HBM/stream evidence.", - "produced_artifact": "A bounded diagnosis, smallest reversible proposal, expected tradeoff and correctness/resource verification plan with unresolved compatibility or authorization questions.", - "refusal_point": "Stop on output mismatch, unstable replay addresses, capture exhaustion, stream resource errors without capture-size reduction, unknown ordering or version drift; missing evidence permits only a conditional proposal." - }, - "recommendations": [ - "Diagnose the effective runtime mode and batch descriptor before changing capture sizes; intended warmup/profile bypass is not a cache defect.", - "Verify caller BatchDescriptor stability and dispatch mode before investigating graph replay issues.", - "For stream-resource exhaustion, reduce cudagraph_capture_sizes or max_cudagraph_capture_size.", - "Trace stable replay buffers to the owning caller and preserve synchronization/update ordering; the wrapper does not stage new inputs into persistent buffers.", - "Compare eager and graph execution on the same workload, separating initial capture, repeated replay, compiler changes and memory/stream costs; a replay log alone proves neither coverage nor speedup." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/metadata.json deleted file mode 100644 index ffc1a92b3..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/metadata.json +++ /dev/null @@ -1,36 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "device-health", - "topology", - "shared-machine", - "os-system" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Pre-run Ascend device-health, topology, occupancy and timing checks before a trusted baseline, multi-card run or graph capture, including after a killed job.", - "hardware": "Ascend NPU card groups with topology and health visible through npu-smi.", - "input_contract": "Intended card group, current topology/health and process occupancy, prior termination history and expected workload step timing.", - "produced_artifact": "A pre-run health/occupancy and step-timing record, with recovery observations and a baseline-remeasurement requirement after a card-group change.", - "failure_modes": "Health=OK can miss degraded post-kill timing; foreign processes, unwarmed clocks or comparing different card groups can contaminate the baseline." - }, - "recommendations": [ - "Read topology each session rather than hardcoding card numbers, then check health and foreign-process occupancy before the workload smoke check.", - "Use the short step-timing check to investigate post-kill degradation; record affected cards and timings instead of treating Health=OK as sufficient evidence.", - "Warm up operator measurements and remeasure the baseline after changing card groups; set only environment variables needed by the run." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json deleted file mode 100644 index d698e50b1..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json +++ /dev/null @@ -1,96 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "distributed", - "parallelism", - "usp", - "hccl", - "communication", - "allreduce", - "topology-dispatch", - "collective-correctness" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ascend-ai-coding/awesome-ascend-skills", - "commit": "75ec89ce9d401be8ac95980e1c3f42f8d4315057", - "paths": [ - "README.md", - "skills/training/hccl-test/scripts/quick-verify.sh", - "skills/training/hccl-test/scripts/parse-hccl-result.py", - "skills/training/torch-npu-comm-test/scripts/comm-bench.py", - "skills/training/torch-npu-comm-test/references/supported-ops.md" - ], - "readset": "references/awesome-ascend-communication-tests.md" - }, - { - "type": "git", - "url": "https://gitcode.com/cann/hccl", - "commit": "96fab0a2acb761e7f8ab0f94aeb5138f200a345b", - "paths": [ - "src/common/hccl_common.h", - "src/ops/all_reduce/all_reduce.cc", - "src/ops/op_common/op_common.cc", - "src/ops/op_common/selector/selector_engine.cc", - "src/ops/op_common/selector/cost_model.cc", - "src/ops/all_reduce/selector/all_reduce_auto_selector.cc", - "src/ops/all_reduce/algorithm/executor/ins_v2_all_reduce_sole_executor.cc", - "src/ops/op_common/algorithm/topo_match/topo_match_one_level.cc", - "src/ops/op_common/algorithm/template/alg_template_base.h", - "src/ops/op_common/algorithm/template/aicpu/kernel_launch.cc", - "src/ops/all_reduce/algorithm/template/aicpu/ins_temp_all_reduce_mesh_1D_one_shot.cc", - "src/ops/op_common/algorithm/template/wrapper/alg_data_trans_wrapper.cc", - "test/st/algorithm/testcase/all_reduce_testcase.cc", - "test/st/algorithm/utils/src/hccl_verifier/hccl_verifier.cc", - "test/st/algorithm/utils/src/hccl_verifier/semantics_check/allreduce_semantics_checker.cc", - "src/common/log.h", - "LICENSE" - ], - "source_channel": "official_repository", - "inspected_scope": "selected_native_implementation_and_simulator_test_definitions", - "license": "CANN Open Software License Agreement Version 2.0", - "readset": "references/hccl-allreduce-readset.json" - } - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Only the declared communication-test and native HCCL source slices are normalized; inherited strategy guidance and its source identities remain unchanged.", - "Wrapper inspection does not qualify native HCCL/TorchNPU implementations, transport algorithms, numerical correctness, backend compatibility or measured gains.", - "Native source coverage is one selected multi-rank AICPU mesh one-shot AllReduce path at the HCCL pin; legacy device routes, other engines/collectives and HCOMM transport/runtime implementations are not qualified.", - "Inspected upstream tests use simulated task-graph semantics; neither those tests nor native numerical, memory, bandwidth, model-quality or service experiments were run." - ], - "provenance_readsets": [ - "references/awesome-ascend-communication-tests.md", - "references/hccl-allreduce.md", - "references/hccl-allreduce-readset.json" - ], - "consumption_mode": "packaged", - "references": [ - "references/awesome-ascend-communication-tests.md", - "references/hccl-allreduce.md", - "references/hccl-allreduce-readset.json", - "references/hccl-LICENSE.md" - ], - "applicability": { - "scope": "Select and diagnose Ascend parallel strategies and their activation/overlap, with a separately bounded native multi-rank HCCL AllReduce dispatch and correctness procedure.", - "input_contract": "Installed collective/runtime builds, rank/device map and topology, workload/strategy settings, count/dtype/reduction, effective dispatch, per-rank outputs and completed-collective/application timing evidence.", - "produced_artifact": "A topology- and workload-qualified strategy/dispatch diagnosis, activation and correctness checks, stage-cost attribution and a target-validation proposal; isolated test completion is not a service result.", - "failure_modes": "Requested parallel settings can silently fall back; stale local-head/chunk/shard parameters can disable a path, and changed topology/card count can confound strategy comparisons." - }, - "recommendations": [ - "Confirm the final collective algorithm/engine after fallback and observe sharding/collective activation before interpreting correct outputs or tuning performance.", - "Preserve peer/thread waits and test distinct rank/position data with explicit completion on every rank; keep enqueue time, collective completion and application-stage timing separate.", - "Recompute topology-derived parameters and remeasure the anchor after changing card groups or parallel degree; inherited diffusion topology restrictions are not universal HCCL gates.", - "Require producer/consumer timeline and exposed-communication evidence for overlap; compare the real pipeline wait cost rather than extrapolating small collectives from bytes." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json deleted file mode 100644 index 144b6045d..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json +++ /dev/null @@ -1,77 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "graph-mode", - "aclgraph", - "npugraph-ex", - "torchair", - "capture-key", - "input-staging", - "output-lifetime" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Historical awesome-ascend-skills/cannbot import retains its short pin and UNKNOWN license in references/legacy-source-notes.md; it was not reread for this source slice.", - "The new official TorchAir read follows Python configuration/code generation and capture/replay ownership only; native TorchNPU/CANN, GE execution, transformations, static kernel compilation and native communication remain uninspected.", - "Selected stub and NPU smoke test definitions were read, not executed. Independent source pins do not establish stack compatibility, correctness, performance or service evidence." - ], - "provenance_readsets": [ - "references/torchair-graph-readset.json", - "references/torchair-graph-path.md", - "references/legacy-source-notes.md" - ], - "consumption_mode": "packaged", - "sources": [ - { - "type": "git", - "url": "https://github.com/Ascend/torchair", - "commit": "e1118e8b0fc1b95190ab53b732c0872989550e20", - "paths": [ - "python/torchair/configs/compiler_config.py", - "python/torchair/configs/aclgraph_config.py", - "python/torchair/configs/debug_config.py", - "python/torchair/configs/npugraphex_config.py", - "python/torchair/npu_fx_compiler.py", - "python/torchair/_acl_concrete_graph/fx2acl_converter.py", - "python/torchair/_acl_concrete_graph/acl_graph.py", - "docs/zh/npugraph_ex/quick_start.md", - "docs/zh/npugraph_ex/api/npugraph_ex/compiler_config.md", - "tests/st/torchair_aclgraph_st.py", - "tests/smoke/aclgraph_test.py", - "LICENSE" - ], - "source_channel": "official_repository", - "inspected_scope": "selected_python_implementation_documentation_and_test_definitions", - "license": "BSD-3-Clause", - "readset": "references/torchair-graph-readset.json" - } - ], - "references": [ - "references/torchair-graph-path.md", - "references/torchair-graph-readset.json", - "references/torchair-LICENSE.md", - "references/legacy-source-notes.md", - "../ascend-aclgraph-capture-replay/SKILL.md" - ], - "applicability": { - "scope": "Diagnose the TorchAir Python ACL graph path from configuration and FX compilation through capture-key selection, fallback, input staging, replay and output lifetime.", - "prerequisites": "Eager correctness established first; configuration must match the installed backend, with authorization and workload-specific tolerances for any NPU comparison.", - "input_contract": "Installed hardware/runtime versions, actual compile backend/options, shape/dtype/stride and alias/mutation contract, retained outputs, capture/replay/fallback observations and workload mix.", - "produced_artifact": "A branch-specific diagnosis and bounded configuration or lifetime proposal with correctness, cold/warm cost, resource and concurrency checks still required on the actual stack.", - "failure_modes": "Shape equality or clone_input=False does not prove capture reuse or zero-copy; capture limits can trigger persistent fallback and later replay can overwrite retained outputs." - }, - "recommendations": [ - "Use the installed CompilerConfig/backend surface and a fresh compiled instance for option changes; do not mix registered-backend snippets with callable-backend configuration.", - "Observe capture, recapture, replay and persistent fallback separately, including changed addresses and revisiting old keys after a capture-limit event.", - "Preserve earlier results before another replay and check views/input-output aliases; keep cold capture, input/output copying, optional compiler passes and service timing separate." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json deleted file mode 100644 index c8fcb9eb0..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json +++ /dev/null @@ -1,39 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "model-adaptation", - "fp8", - "multimodal", - "vllm-ascend" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Checkpoint onboarding for LLM/VLM adapters on Ascend: weight/scale loading, KV-head replication under tensor parallelism and staged multimodal feature checks.", - "prerequisites": "Check the installed CANN and checkpoint format against the versioned guidance; final sign-off requires real weights and at least one full-layer validation.", - "input_contract": "Checkpoint weights/configuration and scale keys, attention-head/TP configuration, model modalities and expected graph/capacity features.", - "produced_artifact": "A checkpoint/feature report separating dummy and real-weight evidence, applicable feature outcomes, and theoretical versus demonstrated serving capacity.", - "failure_modes": "Dummy-only success misses scale/remapping/sharding defects; blindly sharding replicated KV heads or accepting reduced-layer results can conceal loading or full-model failures.", - "refusal_point": "Stop on missing FP8 paired scale keys, unsupported tensor parallel KV-head split without replication, unverified multimodal visual encoder output, or failed base endpoint validation." - }, - "recommendations": [ - "Gate onboarding in order: dummy architecture check, real-weight loading, model endpoint, text request and then multimodal request when the model supports VL; retain the evidence each stage actually supplies.", - "Check weight/scale pairing and each attention tensor loading path, especially when TP exceeds the KV-head count; verify historical FP8 guidance against the actual stack.", - "Expand capacity/concurrency only after basic modality and expected replay checks; mark MoE-only features inapplicable for non-MoE models and require full-layer validation before sign-off." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json deleted file mode 100644 index 58068bb3f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "operator", - "profiling", - "msprof", - "benchmark" - ], - "status": "candidate", - "synced-from": "https://gitcode.com/cann/cannbot-skills external/cannbot/ops/ops-profiling", - "synced-date": "2026-05-26", - "synced-commit": "ac5bbd2b4cf427d011874e11f8d1e8b1bef66eda", - "license": "UNKNOWN", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Build, measure and diagnose one Ascend operator in isolation using msprof op, before interpreting system-level or multi-kernel performance.", - "prerequisites": "A built executable or test harness and access to real NPU hardware with msprof op for collection.", - "input_contract": "Operator executable/harness, collection settings and OPPROF output directory, plus the operator directory used to archive each measurement round.", - "produced_artifact": "Per-round raw CSV archives and observational min/average/max summaries, followed by a bottleneck interpretation and comparison of operator duration and core balance.", - "failure_modes": "Unwarmed or reduced-frequency runs distort timing; separately budgeting simultaneous GM reads/writes misses shared bandwidth, and tiny workloads can be dominated by launch overhead." - }, - "recommendations": [ - "Collect warmup and repeated measurements into a named round, preserve the raw CSVs, and distinguish the summary statistics from the subsequent bottleneck judgment.", - "Compare task duration, dominant pipeline ratios and per-core balance together; change one optimization point, rebuild as needed and archive the new round before deciding to keep it.", - "Check current versus rated frequency and shared read/write bandwidth before blaming the kernel; aggregate msprof ratios do not provide an instruction-by-instruction timeline." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json deleted file mode 100644 index e8c6fb63f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "operator", - "tuning", - "performance", - "pypto", - "tile" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Run a bounded, accuracy-gated tuning campaign for a single Ascend operator already identified as the bottleneck, escalating from code-level to scheduling and instruction-level changes.", - "prerequisites": "A measured operator baseline, an explicit numeric target and a logged on-device accuracy pass before tuning; performance comparisons use real hardware.", - "input_contract": "Operator implementation, fixed shape/dtype/card and warmup policy, accuracy result, per-core work/wait measurements and round logs.", - "produced_artifact": "A bounded campaign record of each change, accuracy and measured timing delta, kept/reverted rounds, current stage and documented stop condition.", - "refusal_point": "Do not start without the baseline and accuracy gate; revert accuracy failures, and stop/escalate repeated failures, timeouts, exhausted stages or the campaign budget.", - "failure_modes": "Scheduler-idle and predecessor-wait bottlenecks need different fixes; output inplace-reshape can corrupt slice writes, and a changed analyzer log format can produce a misleading zero-core report." - }, - "recommendations": [ - "Change one parameter and recheck accuracy each round; keep the failure record when reverting and compare the final configuration against the same baseline workload and warmup policy.", - "Route scheduler bubbles separately from predecessor dependencies and enter instruction-level tuning only after isolating a specific long task.", - "Verify the analyzer parsed real cores before trusting its diagnosis, and rederive PyPTO tile/buffer/scheduling settings for the target SoC rather than copying numeric examples." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/metadata.json deleted file mode 100644 index 75e659d0c..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-optimization-casebook/metadata.json +++ /dev/null @@ -1,34 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "optimization", - "casebook", - "prior-art" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Find source-derived prior cases for an already classified Ascend inference bottleneck, including fallback, memory traffic, exposed communication, graph startup and host scheduling gaps.", - "prerequisites": "A bottleneck classification from profiling and a check of the selected case preconditions/mechanism before applying it.", - "input_contract": "Bottleneck symptoms and matching profiler evidence, model/hardware/CANN/configuration context and, for a new case, measured before/after metrics.", - "produced_artifact": "A matching prior-art case and evidence requirements for a later measured application; a new case record requires a verified win or clear failure with a metric comparison.", - "refusal_point": "Do not add an INCONCLUSIVE case or one without a measured metric comparison; a subjective direction is insufficient evidence." - }, - "recommendations": [ - "Classify first, then check the matching case preconditions and use its owning procedure to apply and measure the candidate; the casebook itself does not perform profiling.", - "Keep profiler mechanism evidence and same-window end-to-end outcomes together, and retain locally_verified=false until the applied case has actually run." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/metadata.json deleted file mode 100644 index 13ba90189..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/metadata.json +++ /dev/null @@ -1,68 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "profiling", - "performance", - "cann" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://gitcode.com/cann/cannbot", - "commit": "c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110", - "paths": [ - "LICENSE", - "README.md", - ".gitmodules", - "plugins-official/LICENSE", - "plugins-official/README.md", - "plugins-official/model-infer-optimize/.claude-plugin/plugin.json", - "plugins-official/model-infer-optimize/.codex-plugin/plugin.json", - "plugins-official/model-infer-optimize/AGENTS.md", - "plugins-official/model-infer-optimize/plugin-sources.json", - "plugins-official/model-infer-optimize/plugin-install.json", - "plugins-official/model-infer-optimize/workflows/optimize-workflow.md", - "plugins-official/model-infer-optimize/agents/model-infer-analyzer.md", - "plugins-official/model-infer-optimize/agents/model-infer-implementer.md", - "plugins-official/model-infer-optimize/agents/model-infer-reviewer.md", - "plugins-official/model-infer-optimize/hooks/hooks.json", - "plugins-official/model-infer-optimize/hooks/pre_tool_use.py", - "plugins-official/model-infer-optimize/hooks/post_tool_use.py", - "plugins-official/model-infer-optimize/hooks/subagent_stop.py", - "plugins-official/model-infer-optimize/workflows/scripts/archive_progress.py" - ], - "readset": "references/cannbot-stage-evidence.md" - } - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "Only the incremental canonical CannBot fixed-stage evidence boundary is normalized; inherited MindIE-SD guidance remains unchanged.", - "CannBot callback predicates do not establish numerical accuracy or performance; dependent skills, agent-host integration and NPU execution are unverified." - ], - "provenance_readsets": [ - "references/cannbot-stage-evidence.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Interpret Ascend CANN profiling evidence from step breakdown through kernel/utilization/communication analysis, and assess execution/validator evidence in automated optimization-stage reports.", - "prerequisites": "Usable warm-state captures with representative steps, required files and non-null utilization columns; healthy cards and a consistent valid topology for multi-card comparisons.", - "input_contract": "CANN step/kernel/utilization/communication outputs with verified column semantics; for stage acceptance, the loaded candidate, baseline, quality-validator output and comparable measurements.", - "produced_artifact": "An evidence-qualified bottleneck classification and optimization direction; stage conclusions retain missing-execution/validator INCONCLUSIVE or observed gate-failure FAIL.", - "failure_modes": "Summing concurrent communication durations double-counts overlap; misidentified columns can drop communication, and a successful export can still lack usable utilization data." - }, - "recommendations": [ - "Start from the step-level computing/communication/overlap/free breakdown and stop deepening the analysis once the bottleneck is located; do not sum concurrent streams as wall time.", - "Inspect actual column headers and enum values on the installed CANN version, then verify files and non-null metrics before interpreting utilization heuristics.", - "Use same-window interleaved baseline/candidate measurements on shared machines; progress markers, hashes and source-derived thresholds do not substitute for quality and execution evidence." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/SKILL.md b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/SKILL.md deleted file mode 100644 index 66eab0871..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/SKILL.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -name: ascend-video-diffusion-optimization -description: 'Run and verify goal-driven Ascend video diffusion optimization. Specializes the backend-agnostic video-diffusion-optimization skeleton with Ascend NPU operator, layout and profiler references. Use when the user asks for latency, RTF, throughput, or HBM targets across VLM, DiT, MoE, attention, or VAE on Ascend NPU. Not for text-only LLMs.' ---- - -# Ascend Video Diffusion Optimization - -Status: candidate workflow guidance, Ascend-specific. The procedure is first-hand and has not been independently validated beyond the [field notes](references/field-notes-video-diffusion.md). Keep model identifiers, campaign revisions, and numeric gates out of this file; they belong to an acceptance issue and its frozen evidence. - -This skill is the Ascend specialization of the backend-agnostic [video diffusion optimization skeleton](../../../methods/video-diffusion-optimization/SKILL.md). Follow that skeleton for the goal contract, baseline discipline, optimization loop, candidate-selection strategy, correctness protection, final acceptance, and final report. This file adds the Ascend-specific operator, layout, and profiler references; it does not repeat the full procedure. - -Own the performance objective through implementation and final verification. Do not stop after profiling, proposing changes, or showing a faster microbenchmark. Continue the measure-change-verify loop until the target is met, the authorized budget is exhausted, or an external blocker prevents execution. - -Read the field notes only for prior failures and starting hypotheses. Treat every observation there as untraceable and non-reproducible. Derive thresholds, chunk sizes, precision policy, and sparsity from the target model's own measurements. - -Verify every stage against the packaged [benchmark design](../../../methods/inference-benchmark-design/SKILL.md) and [evidence discipline](../../../workflows/rsi-evidence/SKILL.md). A microbenchmark or operator parity check is not end-to-end evidence. - -## Ascend-Specific Layout And Profiler References - -Where the common skeleton says "operator names, memory layouts, and profiler references are backend-specific", apply these Ascend facts: - -- `FRACTAL_NZ` is the Ascend packed weight layout. Materialize contiguous weights before converting to `FRACTAL_NZ` when source strides can differ from physical order; a physical-layout mismatch corrupts output. -- `TransData` performs layout conversion between formats. Attribute `TransData` and layout-copy cost before designing fusion or format-preserving wrappers. -- Profiling and stream instrumentation follow `CANN` conventions. Establish producer-consumer events, record tensor lifetimes on side streams, and join streams before their first common consumer. -- Sparse-attention and quantization candidate selection remain backend-agnostic decisions; only the kernel names, packed-layout materialization, and profiler attributions are Ascend-specific. - -For the candidate-selection guidance (condition/scheduling, DiT/MoE/quantization, attention, VAE/long-video), follow the common skeleton and read the [field notes](references/field-notes-video-diffusion.md) for the Ascend-specific failures to avoid. - -The common skeleton's `scripts/evaluate_target.py` and `assets/campaign.template.json` are shared and backend-agnostic; use them as documented there. diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/metadata.json deleted file mode 100644 index fa959015c..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/metadata.json +++ /dev/null @@ -1,55 +0,0 @@ -{ - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "tags": [ - "ascend", - "npu", - "video-diffusion", - "dit", - "moe", - "attention", - "vae", - "performance", - "profiling" - ], - "empirical_validation": "not_run", - "verification": { - "locally_verified": false, - "evidence_level": "guidance", - "note": "Procedure is first-hand workflow guidance. The linked field notes are not reproducible as published. No acceptance campaign under this repository's evidence contract backs this skill, and no public performance claim is made." - }, - "applicability": { - "scope": "Ascend NPU video diffusion inference and serving optimization", - "hardware": "Ascend NPU; the measure-change-verify workflow skeleton is backend-agnostic and lives in methods/video-diffusion-optimization, while the operator, layout and profiler references here are Ascend-specific", - "prerequisites": "A reproducible benchmark, a frozen quality check, and numeric gates defined in the governing acceptance issue", - "input_contract": "Issue-scoped performance and quality gates, timing boundary, representative and held-out workloads, frozen model/runtime/input settings and an authorized optimization budget.", - "produced_artifact": "An experiment ledger and final report of the frozen goal, baseline/final samples, quality/resource checks, accepted or reverted changes and remaining gap; verdicts require the acceptance issue evidence.", - "failure_modes": "Packed-layout conversion from noncontiguous weights can corrupt outputs; missing side-stream lifetime/order handling or omitted layout-copy costs can invalidate an optimization comparison." - }, - "limitations": [ - "Ascend specialization of the backend-agnostic methods/video-diffusion-optimization skeleton; every operator, layout and profiler reference (TransData, FRACTAL_NZ, CANN) is Ascend-specific and must be re-derived per backend.", - "Candidate workflow guidance; not independently validated beyond the field notes.", - "Field notes are non-reproducible as published and must not be cited as evidence.", - "Model identifiers, campaign revisions, and numeric gates are intentionally excluded and remain issue-scoped.", - "Remaining figures are order-of-magnitude recollections." - ], - "provenance_readsets": [ - "references/field-notes-video-diffusion.md" - ], - "references": [ - "../../../methods/video-diffusion-optimization/SKILL.md", - "../../../methods/video-diffusion-optimization/scripts/evaluate_target.py", - "../../../methods/video-diffusion-optimization/assets/campaign.template.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Follow the common goal-contract and measure-change-verify loop, including final restarted-artifact acceptance; an operator win does not establish end-to-end improvement.", - "Materialize contiguous weights before FRACTAL_NZ conversion when strides differ from physical order, and attribute TransData/copy costs before proposing fusion.", - "Use field notes only to form hypotheses; derive thresholds, chunking, precision and sparsity from the target model measurements, and preserve stream dependencies and tensor lifetimes." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/metadata.json deleted file mode 100644 index 988348711..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/metadata.json +++ /dev/null @@ -1,36 +0,0 @@ -{ - "tags": [ - "ascend", - "npu", - "ascendc", - "tiling", - "operator", - "kernel", - "ub" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Design or restructure AscendC tiling for the covered reduction, broadcast and elementwise classes, plus the bounded mxfp8 fusion and small-channel transpose cases.", - "input_contract": "Operator class and semantics, normalized shapes/axes, dtype, target chip generation, valid/aligned lengths and intermediate-buffer requirements.", - "produced_artifact": "A multi-core split, UB chunk formula, buffer-size/total-usage plan and branch matrix with edge tests; these are design outputs, not benchmark results.", - "failure_modes": "Budgeting only inputs omits intermediate UB storage; mixing valid and aligned lengths or copy units corrupts boundaries, and choosing unsupported generation-specific primitives invalidates the design." - }, - "recommendations": [ - "Normalize axes first, then decide full-load versus chunking from the entire live working set; derive tile size from buffer, axis, API and load-balance constraints.", - "Keep valid compute/copy lengths separate from aligned storage offsets and sizes, and complete dtype, tail/alignment and boundary branches before tuning.", - "Select primitives by the target generation and preserve class-specific coverage limits; re-evaluate sample constants and API semantics instead of treating formulas as tested performance evidence." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/metadata.json deleted file mode 100644 index 8e36a12dc..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/metadata.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "tags": [ - "cann", - "npu", - "attribution", - "dependency-dag", - "multistream", - "overlap", - "trace" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Attribute representative CANN trace kernels to source owners and plan bounded multistream overlap using data, state, event and collective-order dependencies.", - "prerequisites": "Pin the installed CANN/TorchNPU/TorchAir versions before choosing one execution-mode synchronization API family; communication proposals require communication evidence.", - "input_contract": "Representative-step selection and shapes, kernel/stream inventory with code_ref, adjacent wait windows and dependency edges; communication analysis also needs rank/transport evidence.", - "produced_artifact": "An attribution map, serial/candidate/pending dependency groups and a minimal overlap plan with joins, enable/fallback path, resource tradeoffs and acceptance checks.", - "failure_modes": "Kernel names and mechanical coverage can misidentify ownership; logical independence can hide resource contention, and overlap ratios or summed durations can hide unchanged wall time or new tails." - }, - "recommendations": [ - "Join hotspots to source owners and adjacent wait windows before selecting a target; keep compute cost, waiting and host gaps separate.", - "Build explicit data/state/event/collective-order edges, then verify one correct overlap point before increasing overlap or partitioning resources.", - "Require observed overlap, preserved state/order and fallback behavior plus latency and outside-module checks; report wall time, interval unions and duration sums separately.", - "Carry disputed formulas and source-revision distinctions forward; device free time alone does not identify Python, GIL or OS causes without matching host evidence." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/metadata.json deleted file mode 100644 index d1f4c947d..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/metadata.json +++ /dev/null @@ -1,36 +0,0 @@ -{ - "tags": [ - "ascend", - "torchnpu", - "device-memory", - "allocator", - "lifecycle" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Diagnose TorchNPU device-HBM reuse, stream lifetimes, fragmentation and graph/private-pool retention before proposing allocator changes.", - "input_contract": "Allocation failure/size sequence, device and allocator path, installed versions/settings, stream and graph-pool timeline, retained tensor references and allocator statistics with synchronization boundaries.", - "produced_artifact": "A suspected allocator state transition with source evidence IDs, one eligible action, resource cost, counter-signals and a correctness-first verification/rollback plan.", - "refusal_point": "Missing evidence supports only a conditional proposal, not a diagnosed leak; stop when external/custom/shared allocation ownership is untraced and keep baseline settings for unsupported versions or live-tensor capacity pressure." - }, - "recommendations": [ - "Separate stream lifetime protection from producer/consumer ordering, and distinguish live tensors, pending events, inactive splits and private-pool retention before tuning.", - "Match split or expandable-segment proposals to the observed allocation pattern and actual feature gates; do not combine incompatible controls or import comment thresholds as defaults.", - "Treat cache release as synchronization/allocation work at a quiescent boundary; compare identical allocation sequences in fresh processes at the same synchronization points." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/metadata.json deleted file mode 100644 index 7450407c6..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/metadata.json +++ /dev/null @@ -1,36 +0,0 @@ -{ - "tags": [ - "ascend", - "torchnpu", - "host-memory", - "pinned-memory", - "lifecycle" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy ownership/readiness, reuse and path-specific cache controls.", - "input_contract": "Installed versions/settings, host pressure and allocation/reuse distribution, stream timeline, owned versus registered/pageable buffers, selected host allocator path and host statistics/allocation timings.", - "produced_artifact": "An ownership classification, source evidence IDs, one eligible allocator proposal, benefit/resource cost and rollback criteria for a later fresh-process comparison.", - "refusal_point": "Keep baseline settings for live-buffer growth, pageable copies, stable reuse, unverified versions or unsupported paths; stop tuning on event/runtime errors and escalate external ownership or gate conflicts." - }, - "recommendations": [ - "Do not mutate an H2D source or read a D2H destination before completion; non_blocking=True alone proves neither overlap nor external-buffer lifetime tracking.", - "Interpret host allocated/active bytes using the selected host path, and test round/cache controls only in supported default-path fresh processes; a per-buffer cache threshold is not a total-cache cap.", - "Keep host and device allocator options separate and use an explicit quiescent boundary for host cache release; cache clearing is not a copy-completion barrier." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/metadata.json deleted file mode 100644 index a372310a2..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/metadata.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "status": "candidate", - "empirical_validation": "not_run", - "source_pins": "2205.14135v2;2307.08691v1;2407.08608v2", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/mechanisms.md", - "references/papers-20261015/cases/io-reduction-can-plateau.md", - "references/papers-20261015/cases/overlap-and-fp8-counter-signals.md", - "references/papers-20261015/cases/partitioning-after-fusion.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/attention-cost-model-boundaries.md", - "references/papers-20261015/NEXT_GATE_LEADS.md", - "references/papers-20261015/README.md", - "references/sources.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Diagnose attention-kernel bottlenecks after IO reduction, tile/pipeline regressions and precision or hardware-port assumptions.", - "input_contract": "Actual kernel/device revisions, attention shapes and semantics, operand/accumulator types, correctness/quality criteria, and existing traffic, resource, parallelism and stall observations.", - "produced_artifact": "A bottleneck diagnosis with confidence, one bounded candidate, resource exchange, counter-signal, missing evidence, source ID and next authorization gate.", - "refusal_point": "Return conditional hypotheses when observations are missing; reject or defer changes with wrong outputs/gradients, unsafe lifetimes/order, unsupported primitives, unacceptable accuracy or gains erased by omitted work." - }, - "recommendations": [ - "Preserve masks, dropout replay, online-softmax normalization and gradient obligations before comparing costs; exact algebra does not imply bitwise equality.", - "After fusion, reclassify the exposed limit using scalar work, independent output tiles and resource ceilings; larger tiles and deeper pipelines can reduce residency or spill.", - "Include precision preprocessing/conversion in the measurement boundary and check generated asynchronous scheduling; validate accelerator-specific primitives separately before a port." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/metadata.json deleted file mode 100644 index f2a7d642d..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/metadata.json +++ /dev/null @@ -1,35 +0,0 @@ -{ - "tags": [ - "host", - "cpu", - "profiling", - "nsys", - "inference" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Detect exposed host overhead and diagnose per-step host regressions in CUDA inference serving from existing nsys timelines.", - "prerequisites": "An existing serving nsys trace and Python with sqlite3; comparative root-cause analysis needs baseline and target traces, with nsys export available for report-format input.", - "input_contract": "One trace for detection/breakdown or two comparable traces for regression analysis, including the workload/phase context needed to isolate steady-state forward steps.", - "produced_artifact": "A detection verdict with metric/phase evidence, per-step NVTX/kernel/API breakdown and ranked host-operation deltas for a targeted optimization handoff.", - "failure_modes": "Warmup/loading or mismatched phase mixtures distort per-step comparisons; duplicated rank ranges, unresolved NVTX text IDs or overlapping rank end times can miscompute host gaps." - }, - "recommendations": [ - "Isolate comparable steady-state forward iterations and inspect context/generation separately; aggregate utilization can hide a phase-specific host bottleneck.", - "Confirm exposed preparation using both performance impact and idle attribution, retain communication as a confounder, and triage absolute NVTX time deltas before line-level optimization.", - "Resolve NVTX text IDs, account for TP rank duplication and use the maximum overlapping end time for gaps; do not compare full-trace totals as per-step overhead." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md b/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md deleted file mode 100644 index bae7a1dbc..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/SKILL.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -name: reason-hopper-kernels -description: Reason about Hopper SM90 CUDA GEMM kernels using TMA and mbarrier lifetimes, WGMMA warpgroup participation, register and pipeline budgets, layouts, software scheduling and FP8 scaling. Use for an H100/H200-family kernel diagnosis or a Blackwell-to-Hopper applicability check; source-derived candidate guidance with no local GPU validation. ---- - -# Reason about Hopper kernels - -Use this procedure to turn an observed Hopper kernel bottleneck into one bounded -change and a correctness-first verification proposal. It does not implement a -kernel or establish installed library support. All operational steps are -**proposed**; the references distinguish inspected source facts, upstream reports -and unresolved claims. Empirical validation is **not_run**. - -Read [source qualification and coverage](references/sources.md) first when using -KernelWiki material: its SM90 tags do not establish Hopper compatibility. The -[inventory](references/source-inventory.json) accounts for every synthesis page -and every source-page header advertising SM90 at the pinned snapshot. Read the -[retained notices](references/notices.md) before reusing source material. - -## Establish the actual contract - -Record device and compute capability, compiler/toolkit and target flags, kernel -and library revisions, selected implementation, shape/strides/alignment, operand, -scale, accumulation and output types, grouping/masking, epilogue and correctness -criterion. Distinguish kernel steady state from quantization, layout conversion, -JIT/descriptor setup, launch overhead and whole-request time. - -Confirm the actual instruction path. Hopper WGMMA needs an architecture-specific -SM90a compilation path; a device-family label, a package import or a generic -SM90 dispatch guard does not prove that path was built or selected. Do not map -Blackwell tcgen05, TMEM, hardware block-scaled MMA, CLC or two-SM MMA recipes to -Hopper. Hopper thread-block clusters and TMA multicast are separate capabilities. - -## Choose the next discriminating check - -| Signal | Inspect and reason about | Small candidate and counter-signal | -| --- | --- | --- | -| TMA/barrier stalls, stale data, hang or size-sensitive failure | [Synchronization](references/synchronization.md): full/empty stages, transaction bytes, phase on reuse, WGMMA completion and epilogue-store lifetime | Repair the ownership protocol before tuning depth. A wait can be a correct dependency, not evidence of insufficient stages. | -| High registers, spills or low resident work | [Resource/layout/scheduling tradeoffs](references/resources-and-scheduling.md): all live accumulators, role budgets, padded SMEM and scale/epilogue storage | Reduce a live range, tile or unroll factor; compare against extra IO or lost instruction parallelism. A register cap can add spills. | -| Load/compute gaps with a long independent K-loop | Same reference: producer/consumer specialization and stage cost | Consider one extra input stage or a role change only after proving independence. Short K or lower residency can make it slower. | -| Low device utilization, thin/grouped GEMM or a long last wave | Same reference: useful tile count, actual residency, software schedule, locality and reduction cost | Compare tile/raster/grouping choices. Persistence cannot create missing work; split-K adds reduction and numerical obligations. | -| FP8 error or scaling/preprocessing cost | [FP8](references/fp8.md): scale meaning/layout, reduction-chunk boundary, partial-sum lifetime and output contract | Change one scale/layout/promotion choice; recheck error and the full conversion-plus-GEMM boundary. A software E8M0 representation is not native block-scaled WGMMA. | - -For attention, start with -[diagnose-attention-bottlenecks](../diagnose-attention-bottlenecks/SKILL.md) for -online-softmax, mask, forward/backward and quality obligations; use this owner's -references only for the selected Hopper mechanism. General kernel lifecycle and -measurement remain with -[kernel-writing-guide](../../../methods/kernel-writing-guide/SKILL.md). - -## Return a reviewable hypothesis - -State the source path/revision and evidence class; observed signal and missing -observations; semantic and architecture prerequisites; one proposed change; the -resource or accuracy exchange; a counter-signal and stop condition; and the next -check. Retain a rejected direction with its changed-condition requirement. - -For later authorized validation, first check tails, zero/short K, repeated stage -wraparound, multiple work tiles, grouped offsets and output/scale layout against -the frozen reference. Inspect generated PTX/SASS and compiler resources on the -actual stack. Then compare the same workload and boundary, accounting for setup, -conversion, reduction, synchronization and epilogue costs. A source report or CPU -packaging check establishes no GPU correctness, performance or service gain. diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/metadata.json deleted file mode 100644 index 024ad9144..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/metadata.json +++ /dev/null @@ -1,190 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "UNKNOWN", - "tags": [ - "cuda", - "Hopper", - "SM90", - "SM90a", - "H100", - "H200", - "TMA", - "mbarrier", - "WGMMA", - "warp-specialization", - "registers", - "pipeline", - "swizzle", - "persistent-kernels", - "FP8", - "DeepGEMM", - "CUTLASS", - "KernelWiki", - "KDA" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/NVlabs/kda", - "commit": "acdd79343c5e9d0c7cb96289f26fd3f1051ce785", - "paths": [ - ".gitmodules", - "THIRD_PARTY_NOTICES.md", - "LICENSE", - "third_party_licenses/MIT-hanlab.txt", - "skills/KernelWiki" - ], - "readset": "references/sources.md", - "identity": "references/source-inventory.json", - "scope": "KDA-to-KernelWiki gitlink and licensing provenance only; first-party runtime code not mined." - }, - { - "type": "git", - "url": "https://github.com/mit-han-lab/KernelWiki", - "commit": "76d27b56f804e7e7295d4c570e1e5d7eef4b0a75", - "paths": [ - "README.md", - "SKILL.md", - "artifacts/kernels/deepgemm/full/PROVENANCE.yaml", - "artifacts/prs/cutlass/PR-2139/PROVENANCE.yaml", - "artifacts/prs/cutlass/PR-2750/key-files/examples/python/CuTeDSL/blackwell/tutorial_gemm/fp16_gemm_1.py", - "artifacts/prs/cutlass/PR-3091/PROVENANCE.yaml", - "artifacts/prs/cutlass/PR-3091/key-files/examples/python/CuTeDSL/hopper/grouped_gemm.py", - "artifacts/prs/cutlass/PR-3091/key-files/python/CuTeDSL/cutlass/utils/tensormap_manager.py", - "data/refresh-cutoff.yaml", - "data/tool-versions.yaml", - "sources/blogs/deepgemm.md", - "sources/prs/cutlass/PR-2139.md", - "sources/prs/cutlass/PR-2965.md", - "sources/prs/sglang/PR-16014.md", - "sources/prs/sglang/PR-19794.md", - "sources/prs/sglang/PR-9559.md", - "sources/prs/vllm/PR-20396.md", - "sources/prs/vllm/PR-23666.md", - "sources/prs/vllm/PR-33285.md", - "wiki/hardware/mbarrier.md", - "wiki/hardware/tma.md", - "wiki/kernels/deepgemm.md", - "wiki/kernels/fp8-block-scale-gemm.md", - "wiki/kernels/grouped-gemm.md", - "wiki/languages/cuda-cpp.md", - "wiki/migration/register-to-tmem.md", - "wiki/migration/wgmma-to-tcgen05.md", - "wiki/patterns/compute-bound.md", - "wiki/patterns/low-sm-utilization.md", - "wiki/patterns/memory-bound.md", - "wiki/patterns/pipeline-stalls.md", - "wiki/patterns/register-pressure.md", - "wiki/patterns/tail-effect.md", - "wiki/techniques/cache-policy.md", - "wiki/techniques/double-buffering.md", - "wiki/techniques/fine-grained-quantization.md", - "wiki/techniques/persistent-kernels.md", - "wiki/techniques/ping-pong-scheduling.md", - "wiki/techniques/pipeline-stages.md", - "wiki/techniques/register-budgeting.md", - "wiki/techniques/swizzling.md", - "wiki/techniques/tile-scheduling.md", - "wiki/techniques/vectorized-loads.md", - "wiki/techniques/warp-specialization.md" - ], - "readset": "references/sources.md", - "identity": "references/source-inventory.json", - "license": "UNKNOWN", - "scope": "All synthesis and source frontmatters screened; selected Hopper bodies mined with explicit qualifications. MIT assertion inherited from KDA; no root LICENSE at this pin." - }, - { - "type": "git", - "url": "https://github.com/deepseek-ai/DeepGEMM", - "commit": "891d57b4db1071624b5c8fa0d1e51cb317fa709f", - "paths": [ - "LICENSE", - "deep_gemm/include/deep_gemm/impls/sm90_fp8_gemm_1d1d.cuh", - "deep_gemm/include/deep_gemm/mma/sm90.cuh", - "deep_gemm/include/deep_gemm/ptx/wgmma.cuh", - "deep_gemm/include/deep_gemm/scheduler/gemm.cuh" - ], - "readset": "references/sources.md", - "identity": "references/source-inventory.json", - "license": "MIT", - "scope": "Selected code sections directly inspected; original bytes compared with KernelWiki where recorded; no native build or execution." - }, - { - "type": "git", - "url": "https://github.com/NVIDIA/cutlass", - "commit": "ca4fdbea708ad940c905359788372b8add9f85e0", - "paths": [ - "LICENSE.txt", - "examples/67_hopper_fp8_warp_specialized_gemm_with_blockwise_scaling/67_hopper_fp8_warp_specialized_gemm_with_blockwise_scaling.cu", - "include/cutlass/gemm/collective/sm90_mma_tma_gmma_ss_warpspecialized_fp8_blockwise_scaling.hpp", - "include/cutlass/gemm/kernel/sm90_gemm_array_tma_warpspecialized_pingpong.hpp" - ], - "readset": "references/sources.md", - "identity": "references/source-inventory.json", - "license": "BSD-3-Clause", - "scope": "Selected code sections directly inspected; original bytes compared with KernelWiki where recorded; no native build or execution." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/12.8.0/hopper-tuning-guide/index.html", - "version": "CUDA 12.8.0", - "content_sha256": "0fb0a5dd532f43e04c41112aa07bb3fddfab03fc8e68392509f0bec2c03ca418", - "readset": "references/source-inventory.json", - "scope": "Selected architecture/ISA sections directly inspected; source HTML not packaged." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/12.8.0/parallel-thread-execution/index.html", - "version": "CUDA 12.8.0 / PTX ISA 8.7", - "content_sha256": "48849aa7851b9ff8893efc750f1a0db46e6023852cb76d001d68d57ffa49fa06", - "readset": "references/source-inventory.json", - "scope": "Selected architecture/ISA sections directly inspected; source HTML not packaged." - } - ], - "applicability": { - "scope": "Source-derived Hopper SM90/SM90a kernel reasoning for TMA/mbarrier, WGMMA, resource/layout/pipeline/software scheduling and selected FP8 GEMM. CUDA categorization does not certify other GPU architectures, library dispatch, or a combined tested stack.", - "architectures": [ - "sm90", - "sm90a" - ], - "excluded_mechanisms": [ - "tcgen05", - "TMEM", - "CLC", - "two-SM MMA", - "native NVFP4/block-scaled tcgen05 MMA" - ], - "input_contract": "Device/compute capability, compiler target and selected implementation, kernel/library revisions, shapes/strides/alignment, numeric and grouping/epilogue semantics, correctness criterion and measurement boundary.", - "produced_artifact": "A source-qualified, reviewable hypothesis with one bounded change, resource/accuracy exchange, counter-signal, stop condition and next verification step.", - "failure_modes": "SM90 tags or import/dispatch guards can hide an unbuilt or unselected SM90a instruction path; extra stages or register caps can increase spills or lower residency." - }, - "limitations": [ - "Native compilation, GPU correctness, numerical accuracy, profiling, performance, model/service behavior and skill efficacy were not run.", - "KernelWiki is Blackwell-first; frontmatter tags, verified labels and snippet claims are not Hopper compatibility proofs.", - "Only selected code sections and nine source summary bodies were mined; remaining source headers and synthesis dispositions are explicit in the inventory.", - "Source numerical claims, fixed tuning defaults and unsafe teaching snippets are not imported as local evidence or recipes.", - "Pinned KernelWiki has no root LICENSE; KDA MIT assertion and its sanitized-distribution discrepancy are retained, with third-party terms kept separate." - ], - "provenance_readsets": [ - "references/sources.md", - "references/source-inventory.json", - "references/notices.md" - ], - "references": [ - "../diagnose-attention-bottlenecks/SKILL.md", - "../../../methods/kernel-writing-guide/SKILL.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "For WGMMA advice, confirm the compiled and selected SM90a path; do not transplant Blackwell-only instruction or memory mechanisms.", - "Resolve producer/consumer ownership and completion before tuning stage depth; budget live accumulators, padded shared memory, scales and epilogue storage together.", - "For later validation, check tails, short K, stage wraparound and grouped offsets against the frozen reference, inspect actual PTX/SASS resources, and include setup/conversion/reduction costs." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/metadata.json deleted file mode 100644 index 792eb68c7..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/metadata.json +++ /dev/null @@ -1,39 +0,0 @@ -{ - "tags": [ - "cgroup", - "cgroupv2", - "cpu", - "cpuset", - "memory", - "io", - "resource-controls", - "os" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Audit Linux cgroup v2 CPU quota/shares, effective placement, memory pressure/protection and conditional block-device IO controls for serving or training workloads.", - "prerequisites": "Identify workload cgroup membership, kernel/controller support, ancestor limits and the authorized file owner; prepare previous values and stop conditions before a separately authorized trial.", - "input_contract": "Target PID and hierarchy/namespace view, requested/effective cpusets, time-aligned CPU/memory/IO counters and application/sibling observations; IO proposals require resolved block-backed traffic.", - "produced_artifact": "An evidence-qualified control review/proposal with capacity tradeoff, configuration provenance, rollback manifest and matched-window application/sibling checks.", - "failure_modes": "Ancestor constraints can invalidate requested placement; pressure movement alone is not application causality, and unsupported IO accounting or shared-inode attribution can misassign traffic." - }, - "recommendations": [ - "Separate own-quota throttling, ancestor limits and sibling contention before choosing a cpu.max or cpu.weight review.", - "Compare requested with effective CPU/node sets and budget cpuset.mems migration before proposing placement; align memory PSI with local versus descendant event deltas.", - "Treat increased headroom as a shared-capacity spend and verify application outcomes and sibling impact, not just counter improvements.", - "Resolve actual block-backed traffic and supported IO controls before proposing caps; preserve inaccessible-ancestor uncertainty and do not infer container write authority." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/metadata.json deleted file mode 100644 index 299878606..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/metadata.json +++ /dev/null @@ -1,111 +0,0 @@ -{ - "tags": [ - "host", - "cpu", - "memory", - "pinned-memory", - "h2d", - "gil", - "numa", - "inference", - "ascend", - "descriptor-pool", - "cuda-streams", - "allocator-lifetime", - "buffer-reuse" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/vllm-ascend", - "commit": "8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf", - "paths": [ - "LICENSE", - "requirements.txt", - "vllm_ascend/distributed/kv_transfer/__init__.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/npu.py", - "vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/cpu_npu.py", - "csrc/torch_binding.cpp", - "tests/ut/kv_offload/test_native_cpu_offload.py" - ], - "readset": "references/vllm-ascend-native-offload-readset.json" - }, - { - "type": "git", - "url": "https://github.com/pytorch/pytorch", - "commit": "ba56102387ef21a3b04b357e5b183d48f0afefc7", - "paths": [ - "aten/src/ATen/core/CachingHostAllocator.h", - "aten/src/ATen/cuda/CachingHostAllocator.cpp", - "aten/src/ATen/native/cuda/Copy.cu", - "aten/src/ATen/native/Memory.cpp", - "c10/cuda/CUDAFunctions.h", - "docs/source/notes/cuda.rst", - "torch/cuda/streams.py", - "torch/_tensor_docs.py", - "LICENSE" - ], - "source_channel": "official_repository", - "inspected_scope": "selected_implementation_and_official_documentation", - "license": "BSD-3-Clause", - "readset": "references/pytorch-host-copy-readset.json" - }, - { - "type": "git", - "url": "https://github.com/python/cpython", - "commit": "60403a5409ff2c3f3b07dd2ca91a7a3e096839c7", - "paths": [ - "Doc/howto/free-threading-python.rst", - "LICENSE" - ], - "source_channel": "official_repository", - "inspected_scope": "official_documentation_only", - "license": "PSF-2.0", - "readset": "references/cpython-free-threading-readset.json" - } - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Legacy live source links remain unnormalized in references/legacy-source-notes.md; only the declared immutable source slices are normalized.", - "Added Ascend case is limited to normal native single-tier CPU descriptor-pool ownership; vLLM compatibility and CANN retention/teardown contracts remain unqualified.", - "PyTorch source-derived CUDA transfer/host allocator candidate; hardware, graph implementation, custom allocators, NUMA and service efficacy are not qualified.", - "CPython v3.13.0 documentation corrects GIL detection only; no free-threaded interpreter or inference-stack compatibility test was run." - ], - "provenance_readsets": [ - "references/vllm-ascend-native-offload.md", - "references/vllm-ascend-native-offload-readset.json", - "references/pytorch-host-copy-readset.json", - "references/cpython-free-threading-readset.json" - ], - "references": [ - "references/vllm-ascend-native-offload.md", - "references/vllm-ascend-native-offload-readset.json", - "references/vllm-ascend-LICENSE.md", - "references/pytorch-host-copy.md", - "references/pytorch-host-copy-readset.json", - "references/pytorch-LICENSE.md", - "references/cpython-free-threading-readset.json", - "references/cpython-PSF-LICENSE.md", - "references/legacy-source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Audit host preparation, pinned-host pressure and asynchronous CUDA-copy ownership/order/lifetime, with separately scoped routing for Ascend descriptor retention, Python preprocessing and NUMA placement.", - "input_contract": "Host buffer ownership and alias/mutation behavior, copy direction/dtype/layout, producer/consumer streams, live/in-flight/cache budget and the actual interpreter/placement context.", - "produced_artifact": "An ownership and dependency audit with bounded transfer candidates and a correctness-first validation plan separating enqueue, pinning, transfer, model and end-to-end cost.", - "failure_modes": "A stream wait does not prevent CPU mutation and record_stream does not create a data dependency; cached host storage may outlive Python scope, and the eager reusable-slot example is not capture-safe." - }, - "recommendations": [ - "Track copy-before-consumer order, CPU payload reuse and device storage lifetime as three independent obligations; wait for D2H completion before CPU reads.", - "Compare pageable, per-request pinning and bounded reuse on the real workload while budgeting all live and in-flight slots, cache and preprocessing memory.", - "Keep the eager staging-slot preconditions explicit and validate ordering/lifetime under stress before performance; a small passing sample does not establish a hazardous pattern is safe." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/metadata.json deleted file mode 100644 index d21287f3a..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/metadata.json +++ /dev/null @@ -1,35 +0,0 @@ -{ - "tags": [ - "host", - "cpu", - "optimization", - "line-profiler", - "inference" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Use line-level profiling to reduce exposed Python request-preparation, scheduling and execution-coordination overhead in a bounded serving optimization loop.", - "prerequisites": "Confirm host overhead is the limiting factor with the detection procedure before tuning; choose a representative workload and consistent CPU affinity for comparisons.", - "input_contract": "Detection/NVTX triage, qualified target function paths and line-profiler output, workload/affinity settings, related correctness tests and serving benchmark metrics.", - "produced_artifact": "Per-round hotspot/change/rollback and correctness records plus a final before/after benchmark report, failed attempts, remaining hotspots and scoped follow-up recommendations.", - "failure_modes": "Optimizing the default preparation function can miss a different regressing owner; caching/hoisting/preallocation can violate lifetime or invalidation, and a line-level speedup can leave serving throughput unchanged." - }, - "recommendations": [ - "Profile the actual top regressing function by absolute exposed time, drilling into dominant unprofiled subcalls instead of assuming _prepare_tp_inputs is the owner.", - "Prefer one small evidence-supported change with explicit ownership/lifetime/invalidation checks, run relevant correctness tests and revert failed or regressing candidates.", - "Re-profile identical workloads after each accepted change and compare hotspot time, total function time and serving metrics; reassess the next bottleneck and stay within the authorized budget." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/metadata.json deleted file mode 100644 index 1cd78a334..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/metadata.json +++ /dev/null @@ -1,58 +0,0 @@ -{ - "tags": [ - "os", - "numa", - "cpu-affinity", - "process-placement", - "threads", - "environment", - "preflight" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ggml-org/llama.cpp", - "commit": "3e390af18de1131dda27913c00511725b4274c8a", - "paths": [ - "common/arg.cpp", - "common/common.cpp", - "src/llama.cpp", - "src/llama-model-loader.cpp", - "src/llama-mmap.cpp", - "ggml/src/ggml-cpu/ggml-cpu.c", - "ggml/src/ggml-cpu/ggml-cpu.cpp", - "LICENSE" - ], - "readset": "references/llama-numa-thread-readset.md" - } - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Only the existing llama.cpp NUMA/thread slice is normalized at 3e390af18de1131dda27913c00511725b4274c8a; other original sources remain unnormalized, with their identities and limits retained in the procedure/readsets.", - "Linux NUMA source paths and separate configuration controls do not establish universally optimal thread counts, affinity combinations or measured placement gains." - ], - "provenance_readsets": [ - "references/llama-numa-thread-readset.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Prepare and record CPU/NUMA process placement, phase-specific thread settings, device contention and environment prerequisites before shared-host performance measurements.", - "prerequisites": "No outstanding hard environment errors: required devices/driver, planned device count and any container prerequisites must be present before running.", - "input_contract": "Current affinity/NUMA policy, accelerator locality, workload phase/thread controls, device occupancy, runtime environment and relevant container/resource limits.", - "produced_artifact": "A recorded placement/thread/environment configuration and preflight outcome, with errors requiring remediation and warnings retained for the measurement campaign.", - "failure_modes": "Changing affinity or thread settings between rounds confounds timing; combining independent affinity mechanisms is order-sensitive, and small driver memory usage can be mistaken for device occupation." - }, - "recommendations": [ - "Inspect affinity and NUMA policy before placement and hold them constant across comparisons; document and requalify a round when placement changes.", - "Choose one affinity-mask authority and distinguish generation, batch and frontend thread controls; source template thread counts are starting points rather than tuned constants.", - "Record device occupancy and structured ERROR/WARNING results before measuring, using the official environment setup for derived CANN paths." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/metadata.json b/src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/metadata.json deleted file mode 100644 index 899caaed8..000000000 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/metadata.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "tags": [ - "storage", - "fio", - "iostat", - "pidstat", - "measurement", - "io", - "evidence" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "Measure and attribute storage traffic for model loading or serving while separating device/process observations from application outcomes and cache/startup effects.", - "prerequisites": "Resolve backing devices and participating worker PIDs first; active fio benchmarks or cache manipulation require separate authorization, with bounded observation during already-authorized runs.", - "input_contract": "Observed loader access pattern, tool/OS/filesystem versions, file-to-device mapping, worker PIDs, phase timestamps, cache-state method and raw IO/job/application measurements.", - "produced_artifact": "A workload- and cache-qualified IO evidence record with units, achieved depth, errors/bytes/completion and device/process attribution, plus a separately measured application outcome where benefit is claimed.", - "refusal_point": "Reject unspecified or copied-example benchmark workloads; do not name a storage bottleneck from util/iowait alone or infer application gain from synthetic bandwidth." - }, - "recommendations": [ - "Match the fio read pattern to the observed loader and verify achieved rather than requested depth; more queued work can increase latency.", - "Record warm/cold/unknown cache state and its method, separate startup from repeated/steady reads, and check completed bytes for truncated runs.", - "Align per-device and per-process observations with the application phase, retain units and latency provenance, and corroborate saturation using traffic, queues and process activity.", - "Only run a separately authorized bounded read-only job against an explicit existing regular file, then remeasure the real application before accepting a benefit." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/metadata.json deleted file mode 100644 index d03ef788f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "status": "candidate", - "empirical_validation": "not_run", - "source_pins": "orca-osdi22-and-fastserve-2305.05920v3", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Diagnosing whether LLM serving queue delay justifies token-boundary priority preemption, including suspended KV, host swap or recomputation and long-request service guarantees; admission-versus-service-order diagnosis, not current scheduler implementation or tenant-fairness certification", - "prerequisites": "The packaged source notes are read before quoting a metric or comparing a baseline; installing, benchmarking, changing scheduling or admission, migrating state, editing a repository or running a reproduction needs a separately authorized task", - "input_contract": "The late boundary (first token, token gap, completion or a stated cohort's service) with its exact percentile or attainment target and denominator; model and KV format, engine revision, safe interruption boundary, request batch limit, input/output/context distributions, arrival bursts, device live/reserved/allocated bytes, active and suspended requests, host budget, transfer path, recovery time and queue/phase/service timings; per-tenant or priority-class obligations kept apart from aggregate latency", - "produced_artifact": "A read-only decision naming the observed failure boundary, the smallest candidate that addresses it, unchanged assumptions, missing evidence, resource cost, affected cohorts, counter-signal and next authorized gate, plus a proposed verification plan; not a validated configuration", - "failure_modes": [ - "An ordinary iteration boundary is taken to permit interrupting a prefill halfway through", - "Unknown output length is treated as known from the prompt", - "KV is counted for the running batch only, omitting started requests that are suspended", - "Reservations are removed because some reserved capacity is not live", - "Recompute-based eviction is proposed without a progress invariant, so two jobs can repeatedly displace each other", - "Host-device copies or same-host IPC are charged to Network instead of their local resource" - ], - "refusal_point": "Withholds a preemption recommendation when the apparent gain can be explained by different kernels, unmatched parallelism or an incomparable latency metric; unknown safe boundaries, unsupported state recovery, uncalibrated timing or unclear service obligations are blockers" - }, - "recommendations": [ - "Name the late boundary and its exact target and denominator before choosing among batching, admission, preemption and state recovery.", - "Send prefill interference to choose-prefill-decode-schedule and physical KV slack to diagnose-kv-block-efficiency when those candidates are available; otherwise report the diagnosed boundary and the missing evidence.", - "In a later authorized test, isolate service ordering first and state recovery second, and evaluate their interaction only after both are understood.", - "Weigh FastServe's same-engine FCFS comparison above its historical-vLLM headline, and do not read its P95 aggregate gain as a starvation or tenant-fairness guarantee." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/SKILL.md b/src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/SKILL.md deleted file mode 100644 index 3ffe30eaf..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/SKILL.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -name: assess-split-backward-schedule -description: Evaluate memory-capped split-backward and V-shaped pipeline scheduling proposals, preserving synchronous update, global-batch and optimizer-validation semantics. ---- - -# Assess a split-backward schedule - -Use when a pipeline trace suggests input-gradient propagation waits behind deferrable parameter-gradient work or an optimizer barrier. This is a candidate paper-derived assessment, not permission to change a training runtime. Read [evidence](references/source-notes.md). - -## Required evidence - -Obtain stage/microbatch counts, microbatch/global batch, precision, optimizer and clipping/NaN behavior; F, input-gradient B, weight-gradient W and transfer timings; per-pass retained/released bytes; total device peak/margin; collective ordering; and kernel-splitting support. Separate calculated bubble rate from measured idle time and useful samples/s. - -## Proposal - -- Identify the critical B dependencies and work W that can legally move after its corresponding B. Account for W inputs retained by deferral; do not equate completion of B with releasing all activations. -- Compare ordinary 1F1B with a memory-capped split schedule. Use observed F/B/W times and transfer costs rather than the equal-time zero-bubble construction. Include kernel-splitting and synchronization overhead. -- When memory permits, compare a larger activation cap with V-shaped two-chunk placement. At matched global batch also consider smaller microbatches with more microbatches. Keep total HBM, workspace, buffers and per-rank imbalance explicit; pMB denotes activation capacity, not total memory. -- Choose whole-step throughput under the memory/quality constraint, not minimum modeled bubble rate. Larger microbatches may beat lower bubbles because kernels use hardware better; more chunks can add communication. -- Keep optimizer post-validation separate. Removing a barrier must preserve global norm/clipping/NaN decisions, rollback/re-update, and correctness of work dependent on a tentative update. If actual optimizer reversibility and rollback coverage are unverified, retain the synchronized fallback. This mechanism is distinct from accepting stale-gradient delayed updates. - -## Output, checks and stopping point - -Return a candidate/fallback with per-pass order rationale, peak-memory and transfer costs, unchanged semantic requirements, counter-signals and a future verification proposal. A later authorized comparison should hold model, global batch and precision fixed and inspect numerical equivalence, full-step time, total peak bytes, exposed communication, rollback frequency/cost and real versus calculated idle intervals. - -Frequent clipping/NaNs, small available memory, strongly unequal passes, unsupported split kernels, hidden collectives or dynamic shapes are reasons to withhold an enablement recommendation. A paper's fixed-seed loss match is not universal convergence or rollback-stress validation. Stop at the reviewable plan; do not install/run frameworks, benchmark schedules, change training semantics, configure hardware or modify repositories without separate authorization. No current/NPU compatibility is established. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/metadata.json deleted file mode 100644 index 18173e4cf..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/metadata.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Assessing memory-capped split-backward and V-shaped pipeline schedule proposals when a pipeline trace suggests input-gradient propagation waits behind deferrable weight-gradient work or an optimizer barrier, while preserving synchronous update, global-batch and optimizer-validation semantics", - "prerequisites": "The packaged evidence notes are read before citing the paper; installing or running frameworks, benchmarking schedules, changing training semantics, configuring hardware or modifying repositories needs separate authorization", - "input_contract": "Stage and microbatch counts, microbatch and global batch, precision, optimizer and clipping/NaN behavior; F, input-gradient B, weight-gradient W and transfer timings; per-pass retained and released bytes; total device peak and margin; collective ordering; kernel-splitting support; calculated bubble rate kept apart from measured idle time and useful samples/s", - "produced_artifact": "A candidate schedule and fallback with per-pass order rationale, peak-memory and transfer costs, unchanged semantic requirements, counter-signals and a future verification proposal; a reviewable plan, not an enabled schedule", - "failure_modes": [ - "Completion of B is equated with releasing all activations, so W inputs retained by deferral go uncounted", - "The equal-time zero-bubble construction replaces observed F/B/W and transfer times, or kernel-splitting and synchronization overhead is left out", - "The schedule is chosen by minimum modeled bubble rate instead of whole-step throughput under the memory and quality constraint", - "pMB activation capacity is read as total memory, hiding HBM, workspace, buffers or per-rank imbalance", - "Removing the optimizer barrier drops global norm, clipping or NaN decisions, rollback and re-update, or the correctness of work that depends on a tentative update", - "Optimizer post-validation is conflated with accepting stale-gradient delayed updates" - ], - "refusal_point": "Withholds an enablement recommendation under frequent clipping/NaNs, small available memory, strongly unequal passes, unsupported split kernels, hidden collectives or dynamic shapes, and retains the synchronized fallback while optimizer reversibility and rollback coverage are unverified" - }, - "recommendations": [ - "Compare ordinary 1F1B with a memory-capped split schedule first; add V-shaped two-chunk placement only when memory permits, and at matched global batch also try smaller microbatches with more microbatches.", - "Decide optimizer post-validation separately from the schedule change.", - "In a later authorized comparison, hold model, global batch and precision fixed and compare real with calculated idle intervals." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/metadata.json deleted file mode 100644 index c066a7d7c..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/metadata.json +++ /dev/null @@ -1,35 +0,0 @@ -{ - "status": "candidate", - "origin": [ - "internal" - ], - "tags": [ - "performance", - "audit", - "measurement" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Auditing a performance claim against supplied fixed-workload reports and an explicit measurement contract: aggregation, retained observations, comparability and quality evidence", - "input_contract": "Baseline and candidate fixed-workload reports with request observations grouped by trial and workload; the measurement contract stating metric, unit, timing boundary, improvement direction, aggregation rule and order, fixed exclusions, threshold and comparison operator; the declared quality and completion evidence; the requested output schema", - "produced_artifact": "An audit conclusion showing baseline and candidate aggregates, the calculation, the threshold comparison and the evidence or missing information that controls the verdict; it interprets supplied evidence and does not itself establish a measured improvement", - "failure_modes": [ - "A pooled request median silently replaces a median of per-trial means, or the reverse, reweighting trials whose request counts differ", - "Late requests, failures or outliers are dropped, or requests move between trials, outside the exclusions the contract fixes", - "Values are rounded before the threshold comparison", - "A missing rule that permits different verdicts is resolved toward the favorable one", - "A speed improvement is read as task success, or a quality pass as equivalent measured work", - "Different workloads or incomplete observations are repaired by a favorable aggregate" - ], - "refusal_point": "Does not replace missing evidence with a pass, does not support a percentage claim on an undefined denominator, and does not launch a new benchmark or alter the contract to resolve an unfavorable result; a failed threshold is reported apart from an unverified comparison or missing quality evidence" - }, - "recommendations": [ - "Recover the contract's metric, direction, aggregation order and comparison operator before reading the report's own verdict.", - "Treat a named minimum bound as >= and a maximum bound as <= unless the contract requires a strict comparison.", - "Keep separate regimes and metrics separate unless the contract defines their aggregation." - ], - "limitations": [ - "Guidance for supplied reports; no local efficacy validation or measured performance gain is established." - ], - "searchspace_not_applicable": "SKILL.md §§ Preserve the observation hierarchy, Check what was actually compared, and Return an auditable conclusion audit supplied measurements and verdicts; device identity is comparison metadata, not a hardware-resource mechanism." -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/SKILL.md b/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/SKILL.md deleted file mode 100644 index 7e2519ec4..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/SKILL.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -name: calibrate-serving-simulation -description: Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical predictions from physical bounds and near-capacity queueing risk. ---- - -# Calibrate a serving-simulation shortlist - -Use when simulation or a cost model is proposed to reduce deployment experiments. This candidate extends existing `performance-ceiling-analysis`, `optimization-search-policy` and `inference-benchmark-design`; it does not replace their ceilings or measurement contract. Reading this skill does not authorize execution; follow the current task's scope. - -## Inputs and prerequisites - -Require model/shape/dtype, actual kernel/runtime revision, graph mode, GPU/host/topology, request length and arrival distribution, total resources/cost basis, memory limits, latency definitions and percentile SLOs. Read [Vidur evidence](references/source-notes.md) and [evidence types](references/source-notes.md). If any essential identity is missing, mark that dimension unknown rather than borrowing a paper default. - -For Infera serving-tuner output, read the [provenance reference](references/infera-serving-search/README.md) before using a shortlist. It distinguishes actual fidelity, the seed-only comparator, evaluated-trial retention and scalar versus DES metrics; independent trace-ranking calibration remains proposed and `not_run`. - -## Procedure (P) - -1. Classify each estimate: physical feasibility bound, measured attainable envelope, Amdahl-style conditional impact, or empirical prediction. Never label learned runtime output a bound. -2. Build operator dependency signatures: current tokens; phase and context-length distribution; KV bytes; communication payload plus topology. Keep prefill and decode separate. Record how sharding and memory planning enter the model. -3. Record calibration domain and held-out coverage. Preserve skewed attention cases, small-model host overhead and tile/shape transitions. If the predictor relies on summed KV volume, challenge batches with equal volume but unequal context distributions. -4. Validate plans must distinguish operator/iteration errors from static normalized execution time and dynamic E2E/queue delay. Include the intended load regime and capacity boundary; do not substitute a low-load check for an SLO claim. No checks are run under this skill without separate execution authorization. -5. Enumerate feasible configuration families under total resource cost and memory budgets. If using capacity binary search, verify the empirical monotonic/stationary premise and define its scheduling-delay threshold. Treat overloaded samples explicitly. -6. Filter on all latency/quality constraints together. Retain nondominated alternatives and uncertainty bands, including differing mechanisms, rather than one predicted winner. If uncertainty exceeds rank separation, label the ordering unresolved. -7. Produce a small future-measurement shortlist whose trials discriminate among competing explanations. Recalibration triggers include workload drift, kernel/graph/runtime changes, new hardware/topology and mismatched residuals. - -## Output - -A bounded decision memo: input identities; model assumptions; evidence type; exact metrics/denominators; unsupported dimensions; candidates and predicted ranges; joint constraint checks; next discriminating trial; rollback/finalization requirements. Prediction and measurement columns must not be conflated. - -## Failure / stop conditions - -Stop predictive certification if profiles are missing, topology is substituted, error grows near capacity, or host overhead is omitted. Do not assert universal error below9%: Vidur AppendixA has12.65% maximum error in a specified near-capacity case. Do not claim current vLLM, asynchronous PP or NPU support from this historical simulator paper. - -## Resource contract - -The local `references/source-notes.md` retains the Vidur evidence. The [Infera reference](references/infera-serving-search/README.md) and its [source record](references/infera-serving-search/source-record.json) retain the separate implementation example and its limits. These owner-local resources support selective consumption. Optional comparison notes and cases are linked through the owning ceiling skill. Existing InferRSI skill names above are reconciliation targets, not runtime dependencies. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/metadata.json deleted file mode 100644 index 5e20e4ad5..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/metadata.json +++ /dev/null @@ -1,77 +0,0 @@ -{ - "status": "candidate", - "empirical_validation": "not_run", - "source_pack": "A", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "paper", - "id": "arxiv:2405.05465", - "version": "v2", - "url": "https://arxiv.org/abs/2405.05465v2", - "sections": [ - "3-7", - "A.1-A.2" - ], - "readset": "references/source-notes.md" - }, - { - "type": "git", - "url": "https://github.com/AMD-AGI/Infera", - "commit": "ff1136e4a51998eba2933ed63cfb9c10120f86ea", - "paths": [ - "LICENSE", - "manual/simulation/tuning_agent.md", - "manual/simulation/boundaries.md", - "manual/simulation/anchors.md", - "manual/simulation/workloads.md", - "infera/projection/agents/tuning_agent/history.py", - "infera/projection/agents/tuning_agent/cli.py", - "infera/projection/agents/tuning_agent/evaluator.py", - "infera/projection/agents/tuning_agent/inference_tuning.py", - "infera/projection/agents/tuning_agent/inference_agent.py", - "infera/projection/core/projection/inference_projection/launcher.py", - "infera/projection/core/projection/inference_projection/des.py", - "tests/unit/projection/test_tuning_slo.py", - "tests/unit/projection/test_tuning_objectives.py", - "tests/unit/projection/test_tuning_variable_lengths.py", - "tests/unit/projection/test_des_disaggregated.py" - ], - "readset": "references/infera-serving-search/source-record.json" - } - ], - "applicability": { - "scope": "Building an evidence-gated serving-configuration shortlist when a profiled simulator or cost model is proposed to reduce deployment experiments; extends performance-ceiling-analysis, optimization-search-policy and inference-benchmark-design without replacing their ceilings or measurement contract", - "prerequisites": "The packaged Vidur evidence and evidence-type notes are read; reading the skill does not authorize execution, and no validation check runs under it without separate execution authorization", - "input_contract": "Model, shape and dtype; actual kernel/runtime revision; graph mode; GPU, host and topology; request length and arrival distribution; total resources and cost basis; memory limits; latency definitions and percentile SLOs", - "produced_artifact": "A bounded decision memo with input identities, model assumptions, evidence type, exact metrics and denominators, unsupported dimensions, nondominated candidates with predicted ranges, joint constraint checks, the next discriminating trial and rollback/finalization requirements, keeping prediction and measurement columns apart; a shortlist for future measurement, not a measured result", - "failure_modes": [ - "Learned runtime output is labeled a physical bound", - "A missing identity is filled with a paper default, or topology is substituted, instead of being marked unknown", - "A low-load check stands in for an SLO claim at the intended load regime and capacity boundary", - "A predictor that relies on summed KV volume hides batches with equal volume but unequal context distributions", - "Capacity binary search is applied without verifying its monotonic/stationary premise or treating overloaded samples explicitly", - "One predicted winner is reported when uncertainty exceeds rank separation" - ], - "refusal_point": "Stops predictive certification when profiles are missing, topology is substituted, error grows near capacity or host overhead is omitted; labels an ordering unresolved when uncertainty exceeds rank separation; asserts neither universal error below 9% nor current vLLM, asynchronous PP or NPU support from the historical simulator paper" - }, - "recommendations": [ - "Classify every estimate as a physical bound, measured envelope, Amdahl-style conditional impact or empirical prediction before comparing configurations.", - "Keep prefill and decode separate in the operator signatures, and record how sharding and memory planning enter the model.", - "Recalibrate after workload drift, kernel/graph/runtime changes, new hardware or topology, or mismatched residuals." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/infera-serving-search/README.md", - "references/infera-serving-search/source-record.json" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/metadata.json deleted file mode 100644 index 8ca288e82..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/metadata.json +++ /dev/null @@ -1,47 +0,0 @@ -{ - "tags": [ - "checkpoint", - "recovery", - "distributed", - "shared-memory", - "replication" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Assessing distributed checkpoint recovery through DLRover memory agreement, persistence completion and replica transport, and NVIDIA Resiliency Extension complete-iteration selection; distinguishes worker, host and replica loss and separates queued saves, publication, application-state validity and durable recovery. Save performance belongs to dcp-async-save-performance", - "prerequisites": "The pinned sources and limits are read; reading authorizes no launchers, failure injection or checkpoint loading, and restore evidence needs separate authorization", - "input_contract": "The declared failure domain; actual memory, storage and replica placement and helper lifetime; step or iteration identity, the expected shard/rank set and finalization result; the application's model, optimizer and caller-required RNG/scheduler/data-position state contract; the target environment's format and world-size/replication assumptions", - "produced_artifact": "An answer to each of the four acceptance questions (snapshot survival, cross-participant completeness, application-state completeness, restorability) at its own evidence level, a proposed per-generation recovery evidence record including errors and skipped saves, and a proposed verification split by worker, host, replica and persistent-storage loss; no recovery success, durability, NPU support or measured gain", - "failure_modes": [ - "Equal steps or complete rank IDs are taken to verify tensor contents or whole application semantics", - "A skipped save, where not every participant acquired shared-memory access, is reported as a new recovery point", - "A returned step paired with an empty state dictionary is read as a successful restore", - "A local maximum filename or one clean shard substitutes for global coverage", - "The .dirty-then-rename publication is read as power-loss durability", - "An asynchronous save request is treated as a completed checkpoint, or queue concurrency is assumed unlimited", - "One failure domain is tested and recovery from all of them is claimed" - ], - "refusal_point": "Claims no recovery success, durability, NPU support or measured gain, does not upgrade allocation, metadata or a completed collective into proven recovery, and neither injects failures nor authorizes them" - }, - "recommendations": [ - "Answer the four acceptance questions separately; record each at its own evidence level.", - "Count replication as added availability only when copies occupy independent failure domains, and budget clean, dirty and replicated shards on each host.", - "Keep the one-outstanding asynchronous save restriction until the target queue version is checked.", - "Send save-throughput questions to dcp-async-save-performance; a faster return or higher write throughput does not establish recoverability." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/sources.md" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/metadata.json deleted file mode 100644 index afc792f30..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/metadata.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Comparing stage, device-mesh and operator-sharding candidates for static distributed training when heterogeneous stage memory or work, or communication cost, makes a uniform data/tensor/pipeline plan suspect; derived from historical Alpa, not a current compiler configuration guide", - "prerequisites": "The packaged source notes are read; the skill authorizes no compiler or solver install or run, parallelism reconfiguration, repository edit, hardware task or benchmark", - "input_contract": "Model operator graph and shapes; global batch, microbatch and accumulation; precision and optimizer; physical links and contention; candidate shard layouts; stage latency, workspace and in-flight activation memory; the objective and expected run length. A device count without topology is insufficient", - "produced_artifact": "The best supported candidate and a simpler fallback with a memory and critical-path ledger, planning cost and amortization, semantic invariants, counter-signals, missing evidence and a proposed matched comparison; a plan, not a reconfiguration", - "failure_modes": [ - "Equal arithmetic work is taken to mean equal kernel time, carrying over Alpa's zero compute-cost simplification", - "Mean memory replaces the largest per-rank peak with multiple in-flight activation sets", - "Inter-stage transfer and microbatch count stay outside the objective, as in historical Alpa, without the gap being reported", - "Repeated slow-link sends to replicated destination tiles are not compared with one cross-mesh transfer plus local allgather", - "Inter-node Network traffic is merged with intra-node accelerator links", - "Static cost estimates are applied to dynamic shapes, asymmetric devices or changing communication contention" - ], - "refusal_point": "Rejects infeasible stage/mesh pairs, claims no global optimality outside the declared search space, and stops at the plan in paper-only mode" - }, - "recommendations": [ - "Keep a simple reference plan with the same training semantics and global batch as the comparison point.", - "Separate fixed-model scaling from model-size weak scaling before quoting a speedup; Alpa's 9.7x MoE ratio belongs to a specific weak-scaling workload, baseline and 25-Gbps cluster.", - "Weigh planning cost against the expected run length; a shorter job may not amortize it." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/metadata.json deleted file mode 100644 index 14777a2c0..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/metadata.json +++ /dev/null @@ -1,46 +0,0 @@ -{ - "tags": [ - "performance", - "evidence", - "profiling", - "measurement" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Deciding which performance evidence to collect next and which claim each evidence level supports, from correctness through microbenchmark, component profile, isolated scenario benchmark and multi-regime serving benchmark to a production A/B experiment; also structuring a measurement campaign", - "prerequisites": "Level 0 correctness evidence is collected before any performance measurement; a Level 5 experiment needs production infrastructure, traffic splitting and statistical power", - "input_contract": "The claim to be made or the question to answer, and the evidence already collected with its level, workload, hardware/software context and controls", - "produced_artifact": "The minimum evidence level each claim requires, the next evidence to collect under the escalation rule, and negative-result records with hypothesis, evidence level, the metric that did not improve, context and the reason if known; a classification of evidence, not a measurement", - "failure_modes": [ - "An md5 match or a test pass is read as semantically correct output or as support for a performance claim", - "A microbenchmark improvement is reported as an inference throughput gain", - "Absolute kernel durations or single collective latencies are compared across devices or sessions", - "A win on one fixed workload or concurrency is generalized to production traffic", - "Worse metrics, per-task regressions or negative results are discarded, or a shared task budget is taken as equal total cost" - ], - "refusal_point": "Withholds any claim above the level its evidence reaches, leaves an unresolved source protocol or denominator mismatch unresolved, and lets neither upstream scores nor local packaging checks promote an InferRSI candidate" - }, - "recommendations": [ - "Collect Level 0 first and profile at Level 2 before implementing; escalate to Level 3 only after a promising Level 2 result, and to Level 4 only after a Level 3 win worth validating across regimes.", - "Collect a Level 1 microbenchmark only for an isolated-kernel question after Level 2 has confirmed that kernel is on the critical path.", - "For a Level 1 claim, pin input shape and dtype, exclude warmup, synchronize before timing, run at least 10 repetitions and report median and IQR; for a Level 3 claim, match hardware, software stack, model revision and inputs, run at least 3 repetitions and alternate A/B/A order.", - "Read the admission/drain occupancy case when interpreting finite service cohorts; phase-selected occupancy is not end-to-end gain." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "references": [ - "../inference-optimization-casebook/cases/admission-drain-occupancy.md", - "../inference-optimization-casebook/cases/admission-drain-occupancy.json" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/metadata.json deleted file mode 100644 index 312f66977..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/metadata.json +++ /dev/null @@ -1,39 +0,0 @@ -{ - "status": "candidate", - "empirical_validation": "not_run", - "source_pins": "llumnix-2406.03243v1-and-loongserve-2404.09526v2", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Comparing migration of a running LLM request between compatible model replicas with changing its sequence-parallel execution group, including state-copy, reservation, recomputation, interruption and priority costs, for cross-instance memory imbalance or highly variable long-context serving; not connector wiring or live scaling", - "prerequisites": "The relevant Llumnix or LoongServe branch and its readset in the packaged source notes are read; moving state, altering admission or priority, scaling replicas, changing parallelism, running experiments or editing frameworks needs separate authorization", - "input_contract": "Exact model and cache identity and format; weight residency; request phase, context and generated length; source and destination KV bytes; physical free/reserved memory and virtual accounting; batch state; current parallel group; queue and service times; actual topology and transport for each path; the target SLO statistic and affected priority or tenant cohorts", - "produced_artifact": "The smallest justified branch with missing capability or evidence, expected saved work or wait, added state, traffic and replica cost, affected SLO and cohorts, counter-signals, the proposed next gate and a future verification plan; an evidence-based recommendation, not a validated deployment rule", - "failure_modes": [ - "Model-weight repartitioning, pipeline configuration, instance migration and sequence sharding are treated as interchangeable uses of parallelism", - "Aggregate free bytes are taken to mean a single blocked request fits, conflating internal KV-block slack with cross-instance fragmentation", - "Llumnix's 20-30 ms measured downtime is substituted for total transfer time", - "Virtual headroom is counted as extra memory or as a tenant-fairness guarantee", - "A zero-overhead transition label is read as every group change being free, or a token sum hides length-sensitive attention work", - "Ongoing distributed-attention communication goes uncharged because existing KV does not migrate" - ], - "refusal_point": "Withholds the candidate when every destination is full, pre-copy catch-up is unproven, model or cache compatibility is unknown, or the measured work saved does not exceed exposed costs; current-code review and target-system tests remain a later authorization gate" - }, - "recommendations": [ - "Decide first whether the question is replica relocation, a sequence-parallel group change, or prefix reuse and phase pools, which evaluate-kv-cache-transfer-gates and choose-prefill-decode-schedule cover when available.", - "Name each transfer path separately; only inter-node transport is Network.", - "Keep a feasible static, no-migration baseline in any later comparison." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md b/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md deleted file mode 100644 index a331d6800..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md +++ /dev/null @@ -1,247 +0,0 @@ ---- -name: inference-benchmark-design -description: 'Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image, video and VLA inference. Covers AIPerf AgentX/Exgentic replay, vLLM/SGLang benchmark boundaries, VLA action-chunk timing and quality, SLO-aware metrics and InferRSI integration.' ---- - -# Inference Benchmark Design - -Status: candidate. Source-grounded from vllm-evolve @ 43cc82c and -InferRSI evidence framework @ 4c8e1285. - -## Select the task and its evaluation contract - -Read [task-specific benchmark selection](references/task-benchmark-selection.md) -when choosing quality evaluators, reference parity and a performance workload. -It covers text, agent replay, TTS, three full-duplex task families, images and -videos, with pinned provenance from InferRSI issue #60. Use the -[command templates](references/benchmark-commands.md) for entry points and a -bounded HTTP smoke. These references are candidate knowledge, not qualification -of every model or backend. - -For cache-aware speech recognition, read the -[ASR stream-boundary procedure](references/cache-aware-asr-stream-boundaries.md) -before selecting tests or comparing chunk latency. It separates startup/steady geometry, decoded -progress, stream-slot reuse, reset, EOU and EOF; its source tests and -author benchmark reports are not local model or serving qualification. - -For streaming TTS measurements tied to playback deadlines or dependent chunks, -read [dependency-aware TTS playout](references/dependency-aware-tts-playout.md). -It traces server PCM credit, client playback events, independent batching work -and partial pipeline costs at one pinned source; it adds no hardware coverage, -runtime or locally reproduced quality/performance result. - -For VLA and robot deployment comparisons, read -[observation-to-action timing and closed-loop quality](references/embodied-cpp-boundary-and-quality.md). -Define the complete CPU-action boundary, distinguish generated and replayed -horizons from the control clock, and reconcile per-task quality, precision and -the measured source revision. Freeze latency/jitter and quality objectives for -the actual control use case; per-generated-action cost is amortization. - -For treatment-enabled tuning comparisons or claimed protocol integrity, read -[treatment delivery and trial validity](references/autotunebench-trial-validity.md). -Trace feedback to the eligible consumer input and apply the frozen arm-validity -precedence before interpreting benefit, saturation or a null. - -For GPU timing comparisons, read -[measurement validity and profiler attribution](references/gpu-measurement-validity.md). -Separate normal E2E timing from profiler replay, test within-run attribution -assumptions and retain work counts plus the predicted mechanism signature. -This is candidate guidance; its task-supplied observations remain unverified. - -For hybrid speculative state-restoration or completion evidence, read the -[NInfer validation contract](references/ninfer-state-and-completion-contracts.md). -Freeze a coupled checkpoint and same-physical-block state oracle; distinguish -zero accepted drafts from zero adopted inputs, logical commit from device -retirement, and selected steady intervals from complete-cohort denominators. -Its source contracts and historical author reports are not native parity results. - -Freeze one primary objective, timing boundary, quality gates and sample/failure -accounting before optimization. Recorded agent task scores and replay completion -do not establish current task success; a VBench prompt workload does not compute -VBench quality. Missing coverage stays INCONCLUSIVE. Numeric gates and required -workload regimes are campaign choices, not repository-wide defaults. - -## Core principle: goodput over throughput - -Raw throughput (tokens/second, frames/second) is misleading because a system -can maximize throughput by accepting requests it then fails to serve within -the user's latency expectation. The meaningful metric is goodput: the rate of -requests that complete AND meet the declared SLO. - -``` -goodput = (requests completed within SLO) / time -``` - -A candidate that improves raw throughput by trading latency away cannot "win" -under a goodput metric if it causes SLO violations. -Source: vllm-evolve README @ 43cc82c. - -## Workload regimes to select for the campaign - -A single benchmark run at one concurrency level does not characterize serving -performance. These source-derived regimes answer different questions; choose -the required set from the deployment use case and freeze it before comparison: - -| Regime | Load level | What it answers | -|--------|-----------|----------------| -| Throughput | Saturating load, with observed bottleneck and queue state | Max sustainable goodput at peak load | -| Latency | Low load | Responsiveness for latency-sensitive workloads | -| Replay | Real or representative trace | Behavior under actual production burstiness | - -An optimization may improve one regime while degrading another. Report all -required regimes separately; do not aggregate into a single score without disclosing -how the aggregation was done. - -For replay, standard trace sources (from vllm-evolve): BurstGPT, Azure, -ShareGPT, Mooncake. Use the trace that matches your actual deployment pattern. - -## Verdict semantics - -The upstream vllm-evolve comparison labels are: - -| Verdict | Condition | -|---------|----------| -| better | Improvement in all required regimes, no regression in any | -| worse | Regression in any required regime | -| inconclusive | Too noisy to distinguish (see below) | - -These labels are not InferRSI's verdict enum. InferRSI uses PASS / FAIL / -INCONCLUSIVE and the campaign's frozen gates. `high_variance_inconclusive` -is a useful diagnostic reason: it means the -measurement signal is not strong enough to support a claim in either direction. -Do not report an inconclusive result as a win. Record it as INCONCLUSIVE and -note the variance level. -Source: vllm-evolve README @ 43cc82c. - -## Statistical requirements - -Freeze repetition count, workload seeds and the decision rule in the campaign. -The upstream guidance recommends multiple seeds and median goodput with IQR or -range. State whether a repetition changes the workload seed or repeats identical -inputs. If using `max - min > threshold × median` as a noise gate, choose the -threshold in the acceptance plan; this skill defines no universal numeric bar. -InferRSI's built-in median/range summaries are descriptive, not a significance -test or automatic enforcement of that noise gate. - -The decision of which regimes must improve and which must not regress is the -caller's policy, not the benchmark's. Encode it in a decision config file -(`decision.yaml` in vllm-evolve) so it is explicit, versioned, and auditable. - -## Audit a supplied fixed-workload report - -Preserve the request-to-trial grouping and every observation covered by the -contract. Late requests and outliers remain included unless the frozen protocol -explicitly excludes them; retain exclusion and failure counts. Apply aggregation -in the declared order: a median of per-trial request means uses every included -request to calculate each trial mean, then takes the median across trials. A -pooled request median is a different statistic, especially with unequal trial -sizes. If the report omits an aggregation rule that changes the verdict, expose -that ambiguity rather than choosing one after seeing the result. - -For a lower-is-better metric with a positive baseline, calculate percentage -reduction as `100 * (baseline - candidate) / baseline` and compare it with the -declared threshold before rounding. Check workload equivalence, output quality -and completion evidence separately from this arithmetic. Report the computed -threshold result and any missing evidence under the frozen contract's verdict -semantics; an attractive aggregate alone does not establish an accepted gain. - -## What must be pinned before comparing two runs - -Every valid A/B comparison requires these to be identical: -- Hardware (exact device SKU, driver version, firmware). -- Software stack (framework version, model weights revision). -- Input distribution (same trace or same generation config and seed). -- Benchmark configuration (concurrency levels, SLO thresholds, timeout). -- Warmup (same warmup protocol, excluded from measurement). -- Measurement protocol (same window duration, same number of seeds). - -If any of these differ between the baseline and candidate run, the comparison -is invalid. "Same hardware class" is not the same as "same device." - -## Shared-machine measurement discipline - -On shared infrastructure (multiple users, shared GPU/NPU allocation): - -- Cross-window comparison is invalid. Run baseline and candidate within the - same time window (interleaved or back-to-back in the same allocation). -- Record neighbor load state before each run. If it changes between runs, - report the comparison as potentially confounded. -- Dedicate devices: do not share GPU devices between the baseline server and - the candidate server in the same comparison run. - -Source: MindIE-SD perf-gate skill @ 8481d39b (window discipline for NPU). - -## What a benchmark cannot prove - -- A benchmark win at one concurrency level does not imply a win at other - concurrency levels. -- A benchmark win with synthetic traces does not imply a win under real - production traffic. -- A benchmark win on one model does not transfer to other models without - re-running. -- A benchmark win on one hardware SKU does not transfer to other SKUs. -- Microbenchmark improvement does not imply E2E benchmark improvement. - -State these scope boundaries explicitly in any report that cites benchmark -results. - -## Search integrity and final-state verification - -Freeze workload hashes, output/work semantics, permitted approximations, quality -checks, timing events and developer-versus-held-out splits. Verify that measured -work actually occurs; a quality pass does not certify a truthful timing path. -Keep fast probes separate from full evaluation, and preserve invalid, failed and -inconclusive trials. A simulator shortlist requires calibrated load/capacity -coverage and does not replace measurements. - -Compare search methods at matched total budgets. Separate search/API cost from -steady-state serving cost; independent per-engine searches are not one equal-budget -engine-selection run. Keep different SLOs, baselines, resources and aggregation -rules explicit. Before reporting, restart the final code/config artifact and -validate that exact state; report best-seen results separately from delivered ones. - -[Reviewed experiment discipline](../performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md) -and [source audit](../performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md) -retain author-reported evidence and counterexamples. No local experiment is -established by integrating these notes. - -For controlled kernel-agent comparisons, use the -[JustGoal source reference](references/justgoal-evaluation-contracts.md). -Keep model/budget/task scope, independent acceptance, released versus repaired -artifacts and workspace/stream/cache contracts explicit. Its results remain -author-reported; it adds no runner or device/performance qualification. - -For source-bound kernel trial records, read -[trial identity, evaluator evidence and candidate selection](references/xeforge-trial-evidence.md). -Keep tool origin, actual oracle, unresolved performance, selection and delivered -eligibility distinct; a retained parity candidate is not a measured gain. - -## Evidence trail requirements (InferRSI compatibility) - -A benchmark campaign that claims PASS under InferRSI's evidence framework -must produce: -``` -evidence// - plan/ - hypothesis.md # what is being tested and why - runspec.json # frozen RunSpec (repo × revision × backend × device × model × scenario) - slo.yaml # declared SLO and threshold - decision.yaml # which regimes must improve / must not regress - results/ - regime-throughput/ # raw metrics, logs, per-seed records - regime-latency/ - regime-replay/ - verdict.json # PASS / FAIL / INCONCLUSIVE with reasons - patch/ - candidate.patch # the exact change being evaluated - validator/ - validate.py # frozen validator script -``` - -A run without a frozen RunSpec is INCONCLUSIVE regardless of the metric values. -A run whose validator was modified after the plan was frozen is INCONCLUSIVE. -Source: InferRSI AGENTS.md and rsi-evidence skill @ 4c8e1285. - -## Scoped internal experience - -Read the single authoritative [admission/drain occupancy case](../inference-optimization-casebook/cases/admission-drain-occupancy.md) when interpreting finite service cohorts. It separates recorded observation, action and interpretation; phase-selected occupancy is not end-to-end gain. The original frozen evidence remains linked, not copied. This shared reference is included in the packaged skill library; standalone folder export is not supported. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json deleted file mode 100644 index 0be7cad20..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json +++ /dev/null @@ -1,593 +0,0 @@ -{ - "tags": [ - "benchmark", - "evaluation", - "serving", - "measurement", - "task-quality", - "reference-parity", - "agentic-coding", - "work-agents", - "aiperf", - "tts", - "full-duplex", - "image-generation", - "video-generation", - "asr", - "stream-boundaries", - "treatment-delivery", - "trial-validity", - "protocol-audit", - "trial-evidence", - "result-origin", - "oracle-boundary", - "candidate-selection", - "accepted-prefix", - "state-restoration", - "completion-accounting", - "vla", - "embodied", - "action-chunks", - "closed-loop", - "gpu-timing", - "profiler-attribution", - "measurement-validity" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "document", - "url": "https://github.com/InferMatrixLab/InferRSI/issues/60#issuecomment-6310920198", - "version": "2026-10-21 report", - "sha256": "c56a1b169d70d9cd2bfcf073b13e001354945d71c54ebde5740f3036e4ee4496", - "readset": "references/task-benchmark-selection.md", - "scope": "Distilled task matrix, metric definitions, integration boundaries and command templates; original report is source/documentation review, not a runtime benchmark." - }, - { - "type": "git", - "url": "https://github.com/vllm-project/vllm-omni", - "commit": "b20e5bac96a029172a59559438ad0590460aeb3c", - "paths": [ - "benchmarks/accuracy/README.md", - "benchmarks/accuracy/image_to_image/README.md", - "benchmarks/accuracy/text_to_image/README.md", - "benchmarks/diffusion/backends.py", - "benchmarks/diffusion/diffusion_benchmark_serving.py", - "benchmarks/tts/README.md", - "benchmarks/tts/bench_tts.py", - "docs/cli/bench/serve.md", - "tests/e2e/accuracy/wan22_i2v/test_wan22_i2v_video_similarity.py", - "vllm_omni/benchmarks/data_modules/seed_tts_eval.py", - "vllm_omni/benchmarks/duplex_session_metrics.py", - "vllm_omni/benchmarks/metrics/metrics.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/sgl-project/sglang", - "commit": "844d2b5decf6257aa192744cbdaaffb8ad797263", - "paths": [ - "docs/cookbook/base/benchmarks/diffusion_model_benchmark.mdx", - "docs/docs/developer_guide/bench_serving.mdx", - "docs/docs/developer_guide/evaluating_new_models.mdx", - "python/sglang/benchmark/serving.py", - "python/sglang/multimodal_gen/benchmarks/bench_offline_throughput.py", - "python/sglang/multimodal_gen/benchmarks/bench_serving.py", - "python/sglang/multimodal_gen/test/test_utils.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/vllm-project/vllm", - "commit": "a9bf45ed8a856a0b1839197152d0d26fabb6599c", - "paths": [ - "docs/benchmarking/cli.md", - "vllm/benchmarks/serve.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/ai-dynamo/aiperf", - "commit": "30420ce581d4fc9eeb34b4d47defe367ab8c3a11", - "paths": [ - "docs/accuracy/accuracy-benchmarking.md", - "docs/benchmark-datasets.md", - "docs/metrics-reference.md", - "docs/tutorials/agentic-code-generator.md", - "docs/tutorials/agentx-mvp.md", - "docs/tutorials/h-cua-perf.md", - "docs/tutorials/tracelab-trace.md", - "docs/tutorials/weka-trace.md", - "docs/tutorials/working-with-profile-exports.md", - "src/aiperf/dataset/loader/exgentic_v2.py", - "src/aiperf/metrics/types/inter_token_latency_metric.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/sgl-project/sglang-omni", - "commit": "f31cf9e9af749040ffa6dbdf064882c443a2fd37", - "paths": [ - "benchmarks/README.md", - "benchmarks/duplex/serving.py", - "benchmarks/metrics/performance.py", - "benchmarks/metrics/speaker_similarity.py", - "benchmarks/metrics/wer.py", - "docs/developer_reference/full_duplex_bench.md", - "docs/developer_reference/full_duplex_metrics.md" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/Lucky-Lance/OmniInteract", - "commit": "de304cef35fd9a50a5caadb5090c34cfbf0dd868", - "paths": [ - "eval/README.md", - "eval/evaluation/compute_unified_eval.py", - "eval/evaluation/summarize_unified_eval.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/OpenBMB/Omni-DuplexEval", - "commit": "962cec448ac37a377ffb963b476778777d11d346", - "paths": [ - "README.md" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI", - "commit": "b3ef95a6dd3abd85c19b5527fe46e80ccb3521d4", - "paths": [ - "docs/quickstart.md", - "src/inferrsi/harness/evaluation.py", - "src/inferrsi/model/protocols/metrics.py" - ], - "readset": "references/task-benchmark-selection.md" - }, - { - "type": "git", - "url": "https://github.com/NVIDIA/NeMo-Speech.cpp", - "commit": "20999cc921d2b14aa58708f61756a8b2e787ba20", - "paths": [ - "BENCHMARK.md", - "LICENSE", - "NOTICE", - "THIRD_PARTY_NOTICES.md", - "src/asr/encoder/cache_aware_encoder.cpp", - "src/asr/encoder/cache_aware_encoder.h", - "src/asr/encoder/fastconformer.h", - "src/asr/recognizer.cpp", - "src/asr/runner.cpp", - "src/asr/runner.h", - "src/asr/vad/vad_endpointer.cpp", - "src/asr/vad/vad_endpointer.h", - "tests/cpp/asr/CMakeLists.txt", - "tests/cpp/asr/test_cache_stream_geometry.cpp" - ], - "readset": "references/cache-aware-asr-stream-boundaries.md" - }, - { - "type": "git", - "url": "https://github.com/OpenBMB/JustGoal", - "commit": "f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d", - "paths": [ - "LICENSE", - "NOTICE", - "README.md", - "goal_full_shape/fable-5.1/dsa_decode_fwd/dsa_sparse_attention_h16_ckv512_kpe64_topk2048_ps64/dsa_decode.cu", - "goal_full_shape/fable-5.1/dsa_decode_fwd/dsa_sparse_attention_h16_ckv512_kpe64_topk2048_ps64/main.py", - "goal_single_shape/fable-5.1/dsa_decode_fwd/t8_topk2048/dsa_kernel.cu", - "goal_single_shape/fable-5.1/dsa_decode_fwd/t8_topk2048/dsa_kernel.py", - "kda05_reproduction_full_shape/opus-4.8/fa4_bwd/attention_bwd_causal_h16_d128/cute_fabwd_bshd_v5c_fusedzero/helpers/NOT_INCLUDED.md" - ], - "readset": "references/justgoal-source.json", - "license": "Apache-2.0 repository default; NOTICE exceptions and excluded proprietary helper retained", - "scope": "Selected kernel entry/workspace/stream contracts and distribution boundary; source-only, no execution" - }, - { - "type": "document", - "url": "https://github.com/OpenBMB/JustGoal/blob/f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d/paper.pdf", - "version": "Repository-paired PDF at f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d", - "sha256": "6bfa1a2728546def4c48e15cf55f31d92a543264787318f1522236c80a0ae8c6", - "readset": "references/justgoal-source.json", - "license": "UNKNOWN", - "scope": "Complete 21-page paper and appendices: controlled comparison, independent correctness, repair/coverage and timing evidence; author-reported, no PDF/figure redistribution" - }, - { - "type": "git", - "url": "https://github.com/li-ch/autotunebench", - "commit": "64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f", - "paths": [ - "ARTIFACT.md", - "README.md", - "CITATION.cff", - "LICENSE", - "config/protocol.yaml", - "pdb/ingest_validate.py", - "pdb/store.py", - "agents/loop.py", - "agents/adapter.py", - "orchestrator/cli.py", - "gates/g2_runtime.py", - "gates/g3_correctness.py", - "gates/g5_graduation.py", - "gates/schema.py", - "harness/kernel_bench.py", - "figs/plot_cand_ab_saturation.py", - "scripts/kernelbench_tabulate.py", - "experiments/kernelbench_l1/results/README.md", - "experiments/kernelbench_l1/results/TABLE.md", - "experiments/astra_parity/README.md", - "observability/iterations.jsonl", - "experiments/kernelbench_l1/kernelbench_tasks/UPSTREAM_LICENSE", - "experiments/kernelbench_l1/kernelbench_tasks/README.md" - ], - "readset": "references/autotunebench-trial-validity.md" - }, - { - "type": "paper", - "id": "arxiv:2609.18123", - "version": "v1", - "url": "https://arxiv.org/abs/2609.18123v1", - "sections": [ - "1-8", - "References", - "Figures 1-6", - "Tables 1-7" - ], - "readset": "references/autotunebench-trial-validity.md" - }, - { - "type": "git", - "url": "https://github.com/IntelLabs/Xe-Forge", - "commit": "6cd3ee98dfe32746b345a6abe521fa4cb9bdc187", - "paths": [ - "CONTRIBUTING.md", - "LICENSE", - "README.md", - "docs/CORRECTNESS.md", - "docs/KERNEL_REPO.md", - "pyproject.toml", - "src/xe_forge/claude/generator.py", - "src/xe_forge/claude/templates/CLAUDE.md.j2", - "src/xe_forge/cli.py", - "src/xe_forge/core/trial_manager.py", - "src/xe_forge/engines/claude_engine.py", - "src/xe_forge/external.py", - "src/xe_forge/skills/__init__.py", - "src/xe_forge/skills/benchmark.py", - "src/xe_forge/skills/trial.py", - "src/xe_forge/skills/upstream.py", - "tests/test_benchmark_gate.py", - "tests/test_trial_manager_ranking.py" - ], - "readset": "references/xeforge-source.json", - "license": "Apache-2.0 repository default; DCO and curated-example exclusions retained", - "scope": "Selected source-bound trial origin/comparison identity, host outcomes, measured-only selection and baseline-oracle contracts; source-only, no execution" - }, - { - "type": "paper", - "id": "arxiv:2605.26118", - "version": "v1", - "url": "https://arxiv.org/html/2605.26118v1", - "sections": [ - "Abstract", - "I-VIII", - "References", - "Appendix A", - "Tables 1-3", - "Algorithm 1", - "Listing 1", - "Figure captions 1-15" - ], - "readset": "references/xeforge-source.json", - "license": "CC-BY-4.0", - "scope": "Complete HTML text/appendix and captions; historical author-reported context, unused graphic values remain unverified" - }, - { - "type": "git", - "url": "https://github.com/Neroued/ninfer", - "commit": "1882c6adbc98043fcf89f3589aa5addb3742777a", - "paths": [ - "LICENSE", - "README.md", - "docs/maintainer/engine-architecture.md", - "docs/maintainer/speculative-decoding.md", - "docs/maintainer/context-management.md", - "docs/performance/methodology.md", - "docs/performance/qwen3.8-27b.md", - "include/ninfer/ops/gdn_replay.h", - "src/core/gdn_replay_records.h", - "src/core/gdn_replay_records.cpp", - "src/core/linear_attention_state.h", - "src/core/linear_attention_state.cpp", - "src/models/qwen3_5/state/state_image.h", - "src/models/qwen3_5/state/state_image.cpp", - "src/models/qwen3_5/execution/gdn.cpp", - "src/models/qwen3_5/execution/text.cpp", - "src/models/qwen3_5/program/decode.cpp", - "src/models/qwen3_5/program/commit.cpp", - "src/models/qwen3_5/program/mtp.cpp", - "src/models/qwen3_5/session/commit.cpp", - "src/models/qwen3_5/session/binding.cpp", - "src/models/qwen3_5/session/checkpoints.cpp", - "src/runtime/engine/engine_execution.inl", - "src/runtime/executor/contract.h", - "src/runtime/executor/local_executor.h", - "src/ops/linear_attention/gated_delta_net/replay/fold.cpp", - "tests/ops/linear_attention/gated_delta_net/test_fold.cpp" - ], - "readset": "references/ninfer-source.json", - "license": "Apache-2.0", - "scope": "Selected Qwen3.5 GDN/MTP state-restoration, accepted-prefix and completion evidence contracts; historical results retain their separate reported revision. Source-only, no native execution." - }, - { - "type": "git", - "url": "https://github.com/yuekaizhang/faster-cosyvoice", - "commit": "f1049c59196c2c1a890079abc56056b43f27cd1d", - "paths": [ - "README.md", - "README_zh.md", - "docs/benchmark.md", - "docs/benchmark_zh.md", - "docs/usage.md", - "docs/usage_zh.md", - "examples/offline_inference.py", - "examples/stream_client.py", - "faster_cosyvoice/config/settings.py", - "faster_cosyvoice/llm/engine.py", - "faster_cosyvoice/server/app.py", - "faster_cosyvoice/server/arguments.py", - "faster_cosyvoice/server/audio_encode.py", - "faster_cosyvoice/server/leading_silence.py", - "faster_cosyvoice/server/openai_speech.py", - "faster_cosyvoice/server/protocol.py", - "faster_cosyvoice/streaming/chunker.py", - "faster_cosyvoice/streaming/session.py", - "faster_cosyvoice/streaming/token2wav_batcher.py", - "faster_cosyvoice/streaming/token2wav_scheduler.py", - "faster_cosyvoice/token2wav/builders.py", - "faster_cosyvoice/token2wav/cosyvoice/flow/DiT/modules.py", - "faster_cosyvoice/token2wav/cosyvoice/flow/flow.py", - "faster_cosyvoice/token2wav/cosyvoice/hifigan/generator.py", - "faster_cosyvoice/token2wav/flashinfer_dit.py", - "faster_cosyvoice/token2wav/flashinfer_flow.py", - "faster_cosyvoice/token2wav/frontend.py", - "faster_cosyvoice/token2wav/matcha_audio.py", - "faster_cosyvoice/token2wav/token2wav.py", - "pyproject.toml", - "tests/test_session.py", - "tests/test_stream_client.py", - "tests/test_token2wav_batcher.py" - ], - "readset": "references/faster-cosyvoice-source.json", - "license": "UNKNOWN overall; selected file-level Apache-2.0 headers and unresolved inherited attribution retained", - "scope": "Selected PCM/playout, routed-credit, chunk-dependency and timing-denominator source audit; original analysis and immutable links only, no code/media copying or native/model/quality/service execution" - }, - { - "type": "git", - "url": "https://github.com/SEU-PAISys/Embodied.cpp", - "commit": "156bed151826780e2ce0132f8d0e3540889ae8cd", - "paths": [ - "adapter/typed_io.py", - "eval/client/libero_profile.py", - "eval/client/vla_cpp_client.py", - "eval/conf/libero_smolvla_object_q8_eval.yaml", - "eval/README.md", - "eval/smolvla_q8_fixed_boundary_20261002.json", - "eval/smolvla_q8_object_followup_20260930.md", - "eval/SMOLVLA_VALIDATION.md", - "eval/THREE_MODEL_VALIDATION.md", - "eval/VLA_BENCHMARK_STANDARD.md", - "LICENSE", - "README.md", - "scripts/aggregate_smolvla_object_eval.py", - "scripts/bench_vla_boundary.py", - "scripts/profile_smolvla_python_baseline.py", - "serving/vla-server.cpp", - "serving/vla.proto" - ], - "readset": "references/embodied-cpp-source.json", - "license": "Apache-2.0", - "scope": "Selected CPU observation-to-action benchmark, client/profiler/aggregation and author artifact contracts; source-derived methodology, not native execution or quality/performance acceptance" - }, - { - "type": "paper", - "id": "arxiv:2607.02501", - "version": "v3", - "url": "https://arxiv.org/abs/2607.02501v3", - "sections": [ - "4 (VLA discussion and Table 3)" - ], - "readset": "references/embodied-cpp-source.json", - "license": "CC-BY-4.0", - "scope": "Independently normalized VLA metric semantics; predates the later SmolVLA experiment" - }, - { - "type": "document", - "url": "https://docs.nvidia.com/nsight-compute/2025.3/ProfilingGuide/index.html", - "version": "Nsight Compute 2025.3", - "sha256": "c9f3b812ce50a46a751f2c92eff628177554f4f6678bb3381468b1dac91b607e", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Replay, Overhead, Hardware Model, Metrics Structure, Range and Precision, Reproducibility, Clock Control and Cache Control; source contracts only, no profiler execution." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/nsight-compute/2025.3/NsightComputeCli/index.html", - "version": "Nsight Compute 2025.3", - "sha256": "c5129a8afe2439c04906f19312393972ae123062d2ee0894b3ba2b11af43471e", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Metric Collection and Profiling option defaults for clock/cache control and metric queries; actual target support must be checked separately." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/nsight-systems/2025.3/UserGuide/index.html", - "version": "Nsight Systems 2025.3", - "sha256": "1d42e35005fa9a4cb21b2c65d0018e4b065c3756dbba308dc0c41f79746e751f", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected CLI Profile options, CUDA graph/node tracing and CUDA API/kernel/launch/MemOps statistics reports; absence of NCU replay controls is an author inference, not a launcher guarantee." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/12.9.1/cuda-c-best-practices-guide/index.html", - "version": "CUDA Toolkit 12.9.1 archive", - "sha256": "0fcaf0c361d00de56a9870ba96db871a0b880904121e1dfb215ab9882470120b", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Assess/Optimize, Application Profiling, Amdahl's Law, Timing and Instruction Throughput; normalization/adoption procedure is an unverified author synthesis." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/12.9.1/cuda-c-programming-guide/index.html", - "version": "CUDA Toolkit 12.9.1 archive", - "sha256": "06499a0c6bfc6925cf54c76f6ba86a4986a081c44c602e10d4ef3b4f33a33865", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Concurrent Kernel Execution, CUDA Graphs (structure, definition/instantiation/execution) and Events; no constant launch/replay overhead or device test claimed." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/cuda/archive/10.2/nsight-compute/docs/NsightComputeCli/index.html", - "version": "CUDA Toolkit 10.2 archive, Nsight Compute CLI", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected nvprof Metric Comparison for dram__bytes_read.sum, dram__bytes_write.sum and smsp__inst_executed.sum. Legacy names do not guarantee current chip/tool availability; no raw-byte digest retained for this browser read." - }, - { - "type": "document", - "url": "https://docs.nvidia.com/deploy/nvidia-smi/index.html", - "version": "Unversioned NVIDIA System Management Interface page, retrieved 2026-11-15; content pinned by SHA-256", - "sha256": "94d1737cdff11b5ef89b121e97365706feeb8ccc18978fecec571bdbbe6ec9ed", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected Performance State, Clocks Event Reasons, clocks, temperature and power reporting. Public page may change; digest identifies inspected HTML, which is not redistributed." - }, - { - "type": "git", - "url": "https://github.com/triton-lang/triton", - "commit": "9641643da6c52000c807b5eeed05edaec4402a67", - "paths": [ - "python/triton/testing.py" - ], - "readset": "references/gpu-measurement-validity.md", - "license": "MIT", - "scope": "v3.2.0 do_bench: write-based cache.zero_ flush, preliminary estimate and per-sample event boundaries; tag resolved with git ls-remote, source inspected in public browser. No flush comparison run." - }, - { - "type": "document", - "url": "https://docs.pytorch.org/docs/2.7/cpp_extension.html", - "version": "PyTorch 2.7", - "readset": "references/gpu-measurement-validity.md", - "scope": "Inspected load/load_inline and the extension name, build-directory and pybind11 module contract. Distinct names for separately loaded variants are author advice, not a documented universal duplicate-name failure; no raw-byte digest retained for this browser read." - }, - { - "type": "paper", - "url": "https://arxiv.org/html/2509.14279v1", - "id": "arxiv:2509.14279", - "version": "v1", - "sections": [ - "3", - "Appendix A", - "Appendix C.3" - ], - "readset": "references/gpu-measurement-validity.md", - "license": "CC-BY-4.0", - "scope": "Lange et al., Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization. Inspected benchmark loopholes, compromised-task filtering/examples and eager/compiled baseline comparisons; reports remain author-reported, not locally reproduced." - } - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Selecting task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image and video inference, and designing goodput-based multi-regime comparisons that fit InferRSI's evidence trail; interpreting treatment delivery, arm validity and stated-versus-observed protocol enforcement in supplied tuning reports; interpreting source-bound kernel trial origin, host evaluator outcomes, unresolved performance and measured-only versus manual candidate selection; designing and interpreting the selected NInfer hybrid speculative state-restoration and completion validation contract; designing or auditing VLA deployment comparisons with explicit observation-to-action boundaries, generated/replayed horizons, per-task quality and source/precision identity; assessing GPU timing validity and profiler-based attribution of kernel/inference gains", - "prerequisites": "The primary objective, timing boundary, quality gates, sample/failure accounting, required regimes, repetitions, seeds and decision rule are frozen before optimization; baseline and candidate share hardware, software stack, input distribution, benchmark configuration, warmup and measurement protocol, and on a shared machine also one time window and dedicated devices; cache-aware ASR also requires the selected model/config and first/steady geometry, decoded clock and reset/EOU/EOF boundaries to be established; VLA comparisons also freeze generated/replayed horizons, initial-state/noise protocol and intended precision or quantization treatment", - "input_contract": "The task family and deployment use case; candidate quality evaluators and reference parity; the declared SLO and the caller's policy on which regimes must improve or must not regress; for an audit, a supplied fixed-workload report with request-to-trial grouping, exclusion and failure counts and the declared aggregation rule; for a treatment audit, the frozen delivery/arm-validity rules, run/candidate identities, feedback and consumer-input evidence, and per-arm raw records; for a kernel trial audit, complete saved candidate bytes and IDs, baseline/spec/reference/variant identities, result-origin fields, raw host outcome and return code, named gate and persisted ratio, actual oracle and the selecting/delivering caller; for the NInfer slice, a complete checkpoint/frontier identity, physical verify-block/state oracle, final adopted extents and correlated execution/termination records; for VLA audits, the declared CPU input/output boundary, source/fixture/checkpoint/processor identities, per-task trial scope and available raw records or explicit artifact gaps", - "produced_artifact": "A frozen benchmark design with evaluators, reference parity, primary objective, timing boundary, quality gates, required regimes, repetition and seed rules and an explicit decision config, plus the evidence-tree layout a PASS claim must produce; for a supplied report, the computed threshold result and missing evidence. Candidate knowledge, not qualification of any model or backend; for a treatment audit, a contract-bound eligibility and interpretation report separating treatment exposure, observed checks, comparator identity, anomaly disposition and source conflicts; for a kernel trial audit, a source-bound interpretation separating artifact/cohort identity, origin, oracle, performance resolution and selected-versus-delivered eligibility, without promoting kernel parity into a gain or whole-model acceptance; for the NInfer slice, a source-bound validation plan or report interpretation that separates state-oracle evidence, final adoption, retirement and workload denominators; for VLA reports, a timing/denominator and precision reconciliation with per-task quality/completion accounting and source-artifact limits; for GPU attribution, a source-grounded measurement plan or interpretation binding environment, paired timing, work counts, predicted signature and E2E decomposition with explicit assumptions and unresolved evidence", - "failure_modes": [ - "Raw throughput is optimized in place of goodput, so requests served outside the SLO count toward a win", - "One concurrency level or one undisclosed aggregate score stands in for all required regimes", - "Recorded agent task scores or replay completion are read as current task success, or a VBench prompt workload as computed VBench quality", - "An upstream better/worse/inconclusive label is used as InferRSI's PASS/FAIL/INCONCLUSIVE verdict", - "Runs on the same hardware class, in different time windows or on shared devices are compared as if pinned", - "A quality pass is taken to certify that the timed work actually occurred", - "Best-seen results are reported as delivered without restarting and validating the final code/config artifact", - "A cache-aware ASR EOU is treated as EOF or full reset, nominal chunk duration as the decoded mel frontier, or shape-only geometry PASS as a native model-state test", - "An enabled treatment or stored feedback summary is taken as proof of delivery to the next eligible consumer", - "A failed arm-validity condition is overridden by a favorable best timing or a saturation/null interpretation", - "Complete gate keys or zero recorded failures are read as proof that every declared protocol check ran", - "A tool-origin measured marker or completed status is treated as a resolved performance gain", - "A displayed/manual best, baseline copy or edited candidate is reported as the eligible measured delivery", - "Ratio omission alone is assumed to clear an earlier ratio, or baseline agreement is treated as independent semantic correctness", - "Zero accepted drafts is treated as zero adopted input, or a matching token prefix is used to splice unrelated State and KV checkpoints", - "ResultReady or a commit receipt is treated as DeviceRetired, or selected full-batch intervals are compared with complete-cohort rates under one unlabeled denominator", - "Generated-action amortization is read as action-response latency or actual control frequency", - "Matching structural pair fields are taken to certify matched effective compute precision or quality equivalence", - "An aggregate VLA success rate conceals task-specific losses or missing/skipped episodes", - "A public current script, recorded experiment revision or binary hash is treated as the exact measured dirty source patch", - "Whole-device fallback memory is relabelled as process-attributed memory or compared with simulator-inclusive/allocator quantities", - "NCU replay durations are substituted for normal E2E evidence, or an untouched region is treated as a universal clock without checking common drift and critical-path assumptions", - "Unexplained gains, changed baseline modes or same-named JIT loads are adopted without binding actual work, module identity and predicted profiler signatures" - ], - "refusal_point": "Records missing coverage and high-variance results as INCONCLUSIVE rather than a win; a run without a frozen RunSpec, or whose validator changed after the plan was frozen, is INCONCLUSIVE regardless of metric values; sets no universal numeric gate or noise threshold; withholds causal treatment benefit/null conclusions when required delivery or arm eligibility is unverified, and preserves conflicts between stated protocol, exported code and source reports; withholds a kernel gain for unresolved performance or an unbound comparison and identifies missing independent semantic evidence when only baseline agreement is available; withholds native state-correctness and cancellation/reuse conclusions when the required same-block oracle, checkpoint identity or matching receipt is missing; withholds VLA quality-preservation, runtime-only attribution, control-rate or current-build conclusions when their required precision, per-task, timestamp or measured-source evidence is missing" - }, - "recommendations": [ - "Freeze one primary objective, timing boundary, quality gates and sample/failure accounting before optimization.", - "Choose the required regimes from the deployment use case, report each separately, and record which must improve or must not regress in a versioned decision config.", - "Use the replay trace that matches the actual deployment pattern.", - "State the concurrency, trace, model, SKU and microbenchmark scope boundaries in any report that cites benchmark results.", - "Trace treatment to its eligible consumer and apply the frozen arm-validity precedence before attributing benefit, saturation or a null.", - "Bind the complete saved candidate and comparator cohort; keep result origin, actual oracle, named gate/ratio and delivered eligibility separate.", - "For hybrid speculative comparisons, freeze the coupled restore point, same-physical-block oracle, zero-draft/zero-adoption controls and complete-cohort versus steady-phase denominators before interpreting a result.", - "Separate complete VLA request cost, generated/replayed action denominators and actual control cadence; retain per-task quality and measured-source/precision limits.", - "For GPU comparisons, retain timestamped clocks/temperature/power, same-run region shares and work counts; test the predicted mechanism signature and paired E2E change before adopting a gain." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "Task benchmark knowledge is distilled from issue #60 with selected pinned-source rechecks; no all-task runtime qualification or optimization benefit is established.", - "Benchmark feasibility and package-consumption checks do not promote this candidate or replace task quality, repeated A/B and service qualification.", - "The pinned NeMo-Speech.cpp ASR reference covers selected source/test definitions and author-reported benchmark boundaries; native model/device, decoder internals, quality, long-audio/load and service behavior remain not_run.", - "JustGoal contributes source-only controlled-evaluation and kernel-acceptance guidance; its repaired/retested timings, released-pass counts and protocol differences are author-reported, not locally reproduced or a complete campaign runner.", - "The AutoTuneBench reference is source-only: original operational checkpoint/prompt receipts remain unavailable, source contradictions remain explicit, and no tuning, device/performance or agent-efficacy validation was run.", - "The Xe-Forge reference is source-only and limited to inspected host/manual/Claude trial paths; named-verdict update predicates and caller defaults remain explicit. Native tests, backend/model/quality/performance/service behavior and agent efficacy were not evaluated, and source/package checks do not promote whole-model support.", - "The NInfer reference inspects selected Qwen3.5 GDN/MTP source contracts and unexecuted test definitions; its historical author measurements use a different engine revision. Native state/model/device/quality/performance and efficacy remain unvalidated.", - "The Faster CosyVoice reference is source-only: server route credit and client simulated playback remain distinct, reported costs/quality retain their boundaries, overall source rights are UNKNOWN, and native/model/device/audio quality/service/performance are not_run with efficacy not_evaluated.", - "The Embodied.cpp reference is source-derived: its public latency arrays allow arithmetic audit, but raw episode recount, exact measured script/dirty patch, effective dispatch and current model/device/service or efficacy qualification remain unresolved.", - "The GPU measurement-validity reference is source-only. The within-run normalization and explanation-first adoption procedure is an unverified author proposal; four task-supplied numerical observations have no supplied raw evidence. GPU/profiler, model/quality, L1–L4 and efficacy validation were not run; legacy metric mappings require target-specific queries." - ], - "provenance_readsets": [ - "references/task-benchmark-selection.md", - "references/benchmark-commands.md", - "references/cache-aware-asr-stream-boundaries.md", - "references/justgoal-evaluation-contracts.md", - "references/justgoal-source.json", - "references/autotunebench-trial-validity.md", - "references/autotunebench-source.json", - "references/xeforge-trial-evidence.md", - "references/xeforge-source.json", - "references/ninfer-state-and-completion-contracts.md", - "references/ninfer-source.json", - "references/dependency-aware-tts-playout.md", - "references/faster-cosyvoice-source.json", - "references/embodied-cpp-boundary-and-quality.md", - "references/embodied-cpp-source.json", - "references/gpu-measurement-validity.md" - ], - "references": [ - "../inference-optimization-casebook/cases/admission-drain-occupancy.md", - "../inference-optimization-casebook/cases/admission-drain-occupancy.json", - "../../repositories/deepinfer/budget-speculative-verification/SKILL.md", - "../../repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md" - ], - "consumption_mode": "packaged", - "searchspace_not_applicable": "SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism." -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/metadata.json deleted file mode 100644 index 0d321e5cb..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/metadata.json +++ /dev/null @@ -1,43 +0,0 @@ -{ - "tags": [ - "optimization", - "casebook", - "prior-art", - "decision-support" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Structuring and querying a casebook of past optimization decisions as decision support, when recording optimization experiments or choosing the next optimization from prior art; the casebook is reference material and neither runs profiling nor makes decisions", - "prerequisites": "Whole-body text access to the case files for free-text search; applying a case first requires reading the nearest checked-in config and the model's deployment guide", - "input_contract": "For a query: natural search terms such as model name, knob, mechanism word or deployment feature, plus the current model and hardware. For a new case: mechanism, applies-when and does-not-apply conditions, procedure, expected effect, accuracy risk, verification, rollback and a real measurement trail with run ID, verdict, exact hardware SKU, model revision, workload and before/after result", - "produced_artifact": "Ranked matching cases read as precedents and diagnostic options, or a new or deprecated case record in the template; a precedent's procedure and expected direction of effect, not a guaranteed configuration or a measured result", - "failure_modes": [ - "The query is pre-translated into canonical vocabulary or bounded by structured fields, so synonyms in the body text are missed", - "A case's specific values, such as batch size, tensor-parallel degree or kernel parameters, are copied instead of re-derived for the current model and hardware", - "The precedent's expected direction of effect stands in for measurement after the change is applied", - "A rejected direction is read as a permanent blacklist, or one discarded implementation as ruling out its mechanism", - "A confounded run, a missing experiment and a failed gate are recorded as the same evidence state", - "Cross-system cases are bootstrapped without re-verifying that the mechanism holds on the target hardware" - ], - "refusal_point": "Adds no case for an INCONCLUSIVE optimization-effect claim, a single-configuration result without plausible generalization, or a result without a real measurement trail, except a scoped diagnostic observation that keeps its verdict and configuration boundary; manufactures no PASS, and reading the records does not reopen a paused campaign" - }, - "recommendations": [ - "Search the full case bodies with free text first, then rank the hits by bottleneck class and signals.", - "Before retrying a rejected direction, state which condition changed or which uncertainty a bounded check resolves, run an unchanged control and choose the least expensive discriminating check.", - "Mark bootstrapped cases source_grounded or author_reported until a local measurement confirms or refutes them.", - "When a case's conditions no longer hold, mark it deprecated with the reason instead of deleting it." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md b/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md deleted file mode 100644 index b5fc258f0..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md +++ /dev/null @@ -1,276 +0,0 @@ ---- -name: kernel-writing-guide -description: 'Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL, CUDA C++, AscendC/TileIR), universal correctness rules that apply regardless of framework, and the verify-before-benchmark workflow. Use before starting any kernel implementation to avoid common correctness and performance pitfalls. - - ' ---- - -# Kernel Writing Guide - -Status: candidate. Source-grounded from TensorRT-LLM `kernel-triton-writing` -and `kernel-cute-writing` skills @ f33e0e5f (NVIDIA/Apache-2.0). NPU-specific -content inferred from architecture parallels; verify against CANN/TileIR docs. - -## SearchSpace mapping - -Accelerator row → Optimize cell - -Second-level targets: Kernel fusion, custom GEMM, attention variants, elementwise -chains, reduction kernels. - -## When to write a custom kernel - -Writing a kernel is expensive. Verify the bottleneck before starting. - -**Write a custom kernel when:** -- A profiler (nsys/ncu or CANN msprof) confirms a specific kernel is on the - critical path and near its hardware ceiling. -- Two or more adjacent operations can fuse: they share data that would otherwise - go to/from DRAM between calls. The gain is the reduction in memory traffic, - not the removal of Python overhead. -- A standard implementation exists but the specific dtype, shape, or quantization - variant you need is not covered. - -**Do NOT write a custom kernel when:** -- A single elementwise operation (relu, sigmoid on its own): the vendor library - is already optimal. -- A standalone matmul: cuBLAS/CANN is better than a handwritten GEMM in almost - all cases. -- Standard attention: use FlashAttention or the vendor's equivalent. -- The bottleneck is host-side scheduling: that is a Python/GIL problem, not a - kernel problem. See `host-overhead-profiling`. - -## Qualify an existing causal convolution operator - -Before replacing or binding causal depthwise convolution, use the -[causal-conv1d contract](references/causal-conv1d-contract.md) to choose a -full-sequence forward call or an in-place state update. Check dtype, strides, -packed boundaries, state rows and circular-buffer cursors before substitution. -The [five decision cases](cases/causal-conv1d-dispatch-state.md) cover invalid -layouts, continuation state and unproved dispatch. Build targets and import -success do not establish the executing operator, numerical parity or a gain. - -For a concrete Transformers Qwen3.5 caller, first use the -[two-version HF binding contract](references/hf-qwen35-gdn-binding.md) and -[source cases](cases/hf-qwen35-gdn-binding.md). Resolve module versus instance -binding, active forward overrides and cache state before substituting conv or -GDN. A cache-free classification adapter does not establish generation-state -support; package presence and matching names do not prove executed dispatch. - -## Framework routing - -| Pattern | CUDA / Ampere+ | Ascend NPU | -|---------|---------------|------------| -| Fused elementwise chain (2+ ops sharing data) | Triton (`@triton.jit`) | TileIR / AscendC | -| Reduction (softmax, LayerNorm, RMSNorm) | Triton | TileIR / AscendC | -| Tiled GEMM with autotune | Triton | AscendC | -| GEMM with explicit shared memory / tensor core MMA | CuTe DSL (CUTLASS 4.x) | AscendC cube unit | -| Flash Attention variant | CuTe DSL or Triton | AscendC (custom) | -| Custom collective (AllReduce variant) | NCCL/custom CUDA | HCCL plugin | - -CuTe DSL requires Linux x86_64, Python 3.10–3.13, Ampere+ (SM80+), and -`pip install nvidia-cutlass-dsl`. It is not available on NPU. - -Source: TRT-LLM kernel-triton-writing + kernel-cute-writing @ f33e0e5f. - -## Universal correctness rules (apply to all frameworks) - -These rules cause silent wrong results if violated. Check them before -benchmarking. - -### 1. Transcendental functions require FP32 inputs - -`exp`, `log`, `erf`, `tanh`, and similar transcendentals cause errors or -undefined behavior with FP16/BF16 inputs in Triton. Pattern: - -```python -# Wrong — silent error or wrong result with fp16/bf16: -result = tl.exp(x) - -# Correct — cast to fp32, compute, cast back: -x_fp32 = x.to(tl.float32) -result = tl.exp(x_fp32).to(x.dtype) -``` - -Rule: any math function beyond `+`, `-`, `*`, `/` needs an fp32 cast round-trip. - -Source: TRT-LLM kernel-triton-writing @ f33e0e5f, §FP16/BF16 Precision Rules. - -### 2. Never call `.item()` inside a kernel wrapper - -`.item()` forces a CPU-GPU synchronization (~50–100 µs per call). This turns -into a pipeline stall if called inside the hot path. - -```python -# Wrong — forces sync: -seed = some_tensor.item() - -# Correct — use data pointer arithmetic, no sync: -seed = (some_tensor.data_ptr() % (2**31)) ^ n_elements -``` - -Source: TRT-LLM kernel-triton-writing @ f33e0e5f, §CPU-GPU Sync Avoidance. - -### 3. C integer division semantics (Triton / AscendC) - -Triton and most NPU DSLs use C semantics for `//` and `%`: they round toward -zero, not toward negative infinity like Python. This only matters when operands -can be negative (which they can in offset calculations). - -``` -Python: -7 // 2 = -4, -7 % 2 = 1 -C/Triton: -7 // 2 = -3, -7 % 2 = -1 -``` - -Safe pattern: ensure all index and offset values are non-negative. If negative -values are possible: `(idx % BLOCK + BLOCK) % BLOCK`. - -Source: TRT-LLM kernel-triton-writing @ f33e0e5f, §C Integer Division Semantics. - -### 4. Pre-allocate output tensors - -Allocating output inside the kernel wrapper on every call adds Python object -creation cost on the hot path. Accept a pre-allocated output buffer as a -parameter, or cache allocation outside the benchmark loop. - -### 5. Use FP32 accumulators for matmul (Triton) - -Triton does NOT auto-promote fp16/bf16 inputs to fp32 for accumulation in -`tl.dot`. Always declare the accumulator as `tl.float32`: - -```python -acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32) -``` - -### 6. Always mask loads and stores for non-divisible shapes - -If `n_elements % BLOCK_SIZE != 0`, an unmasked load reads past the tensor -boundary. Always include a mask: - -```python -mask = offsets < n_elements -x = tl.load(x_ptr + offsets, mask=mask) -tl.store(out_ptr + offsets, result, mask=mask) -``` - -### AKG Affine loop tails (source-derived) - -For AKG-MLIR static CUDA elementwise loops, use the -[Affine tail-guard reference](references/akg-affine-tail-guards.md) to trace the -effective target, option precedence and loop bounds before interpreting a -non-divisible tile. The compiler can expand point-loop bounds and move the body -under an `affine.if`; both input reads and output writes need the valid-index -contract. The pinned fixture's FileCheck expectations describe IR structure, -not executed CUDA correctness or a tuning gain. This addition preserves the -handwritten load/store mask rule above and requires separate validation of -later lowering, synchronization and numerics. CPU/NPU tail strategies are not -qualified by the CUDA fixture. - -## Workflow: verify before benchmark - -### Qualify a stateful serving operator - -For vLLM Qwen3-Next GDN convolution/recurrence reuse, read the -[pinned phase/state contract](references/vllm-gdn-causal-conv-contract.md) and -[representative source cases](cases/vllm-gdn-source-contract.md). Trace the -concrete caller, effective phase/backend, state-row mapping and in-place writes -before choosing an operator. The in-tree vLLM wrappers do not establish a -standalone-package or HF-worker binding. Source selection remains separate -from executed correctness, hardware qualification and performance. - -For SGLang Qwen3.5 or Qwen3-Next, use the separate -[caller/backend contract](references/sglang-gdn-causal-conv-contract.md) and -[four source cases](cases/sglang-gdn-source-contract.md). Resolve the model's -linear layers, per-runner prefill/decode choices, architecture predicates, -conv aliases and state-row metadata before proposing reuse. A GDN prefill flag -does not select the convolution implementation. Keep one-shot classification -prefill distinct from decode, cached continuation and delimiter-based scoring; -source-derived transfer conditions are not an HF binding or execution result. -The [worker/engine cross-map](references/sglang-gdn-causal-conv-contract.md#map-the-inspected-hf-workers-to-vllm-and-sglang) -connects the inspected Clef/PPLX bindings to both engine operator contracts; -its PPLX context follows the default GVA forward and the state-free adapter. - -**Phase 1: Write the kernel with a reference function** - -Every kernel must ship with three exports (used by companion verification and -benchmark scripts): - -```python -kernel_fn = my_kernel_wrapper # the implementation -reference_fn(*args) -> Tensor # PyTorch reference with identical signature -get_inputs() -> list # returns fresh CUDA/NPU tensors for testing -``` - -**Phase 2: Verify against reference** - -Run `scripts/verify_kernel.py` (or equivalent) to check that the kernel output -matches the reference across a range of shapes and dtypes before any performance -measurement. Do not benchmark an unverified kernel. - -Common verification failures: -- Shape boundary errors (off-by-one in mask) -- Accumulator dtype (fp16 accumulation in matmul) -- Transcendental precision (forgot fp32 cast) - -**Phase 3: Profile to confirm bottleneck** - -After verification, profile with ncu (CUDA) or msprof (Ascend) to confirm the -kernel has improved the target metric. An optimization that passes verification -but does not change the profiler output was not on the critical path. - -## Block size selection - -For Triton on CUDA: -- Start with `BLOCK_SIZE = 1024` for H100/Hopper, `512` for A100/Ampere. -- Block sizes must be powers of 2. -- For GEMM tiles (`BLOCK_M`, `BLOCK_N`, `BLOCK_K`): start from the tiled matmul - skeleton configs in the source (128/256/64 for Hopper, 64/128/32 for smaller - cards) and autotune from there. - -For AscendC on Ascend NPU: -- The AI Core's local buffer (Unified Buffer / UB) is the equivalent of shared - memory. Tile sizes must fit in UB. -- Cube unit (matrix unit) alignment requirements differ by SoC; check the CANN - AscendC programming guide for the specific SoC target. -- TileIR (the higher-level DSL above raw AscendC) auto-tiles for standard - patterns; use it before dropping to raw AscendC. - -## Starting from examples - -For CuTe DSL on CUDA: find the closest example in the CUTLASS repo at -`https://github.com/NVIDIA/cutlass/tree/main/examples/python/CuTeDSL`. -Match the GPU architecture first (SM80 for Ampere, SM90 for Hopper, SM100 for -Blackwell). Study the example's structure; do not copy it verbatim since shapes, -dtypes, and problem dimensions differ. - -For AscendC: consult the CANN AscendC operator development guide for your -specific SoC (Ascend 910B, 910C, etc.). Templates and intrinsics differ between -generations. - -## Attention mechanisms and scoped acceptance - -Use [attention diagnosis](../../hardware/cuda/diagnose-attention-bottlenecks/SKILL.md) when IO -reduction exposes scalar work, insufficient independent tiles, reduction costs, -compiler ordering or precision/layout overhead. Its -[mechanism notes](../../hardware/cuda/diagnose-attention-bottlenecks/references/mechanisms.md) -retain FlashAttention 1/2/3 assumptions, operation-class costs and counter-signals. -Tiling, recomputation, ownership and overlap change the resource balance; no -single utilization threshold or paper tile size selects a winner. - -Preserve dense/sparse/quantized semantics, masks, dropout, output/gradient -accuracy and real buffer lifetimes. Verify the executing instruction schedule -before claiming overlap. Include conversion/preprocessing and exactness costs -in the measured boundary. NVIDIA primitives establish no Ascend support. -[Author-reported cases](../../hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md) -are counterexamples to unconditional tuning rules, not local successes. - -## Limitations - -- This skill is a routing and correctness guide, not a complete implementation - reference. The source TRT-LLM skills contain full implementations for common - patterns (fused GELU+dropout, softmax, LayerNorm, tiled GEMM, Flash Attention). -- CuTe DSL patterns are NVIDIA CUDA-only (Linux x86_64, SM80+). -- AscendC/TileIR patterns are source-inferred for this skill — verify against - your specific CANN version and SoC before use. No local NPU verification done. -- Autotune (Triton `@triton.autotune`) adds substantial first-run compile cost. - For serving, pre-warm the kernel at startup or cache the compiled binary. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/metadata.json deleted file mode 100644 index cc50284f2..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/metadata.json +++ /dev/null @@ -1,208 +0,0 @@ -{ - "tags": [ - "kernel", - "triton", - "cute", - "cuda", - "ascendc", - "performance", - "accelerator", - "akg", - "affine", - "tail-guards", - "causal-conv1d", - "state-update", - "vllm", - "gdn", - "causal-convolution", - "transformers", - "qwen3.5", - "binding", - "classification-prefill", - "sglang", - "prefill", - "state-ownership" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/Dao-AILab/causal-conv1d", - "commit": "cd81f0413cad2fc1e6f17e785ac39f59aae690cd", - "paths": [ - "LICENSE", - "README.md", - "causal_conv1d/__init__.py", - "causal_conv1d/causal_conv1d_interface.py", - "causal_conv1d/cpp_functions.py", - "csrc/causal_conv1d.cpp", - "csrc/causal_conv1d_fwd.cu", - "csrc/causal_conv1d_update.cu", - "setup.py", - "tests/test_causal_conv1d.py" - ], - "readset": "references/causal-conv1d-contract.md", - "license": "BSD-3-Clause", - "scope": "Forward/update caller contracts, selected packed/state kernel branches and build/binding evidence; no native execution." - }, - { - "type": "git", - "url": "https://github.com/vllm-project/vllm", - "commit": "537a79f0fc5ba564d2bef43d54915b2f80f9f77d", - "paths": [ - "LICENSE", - "vllm/model_executor/models/qwen3_next.py", - "vllm/model_executor/layers/mamba/gdn/qwen_gdn_linear_attn.py", - "vllm/model_executor/layers/mamba/gdn/base.py", - "vllm/model_executor/layers/mamba/mamba_utils.py", - "vllm/model_executor/layers/mamba/ops/causal_conv1d.py", - "vllm/v1/attention/backends/gdn_attn.py", - "vllm/v1/attention/backends/utils.py", - "vllm/envs.py", - "vllm/third_party/flash_linear_attention/LICENSE", - "tests/v1/attention/test_gdn_metadata_builder.py", - "tests/kernels/mamba/test_causal_conv1d.py" - ], - "readset": "references/vllm-gdn-causal-conv-contract.md" - }, - { - "type": "git", - "url": "https://github.com/huggingface/transformers", - "commit": "0dad7b822255a0ae261ec45ae937371e859ffd1a", - "paths": [ - "LICENSE", - "src/transformers/models/qwen3_5/modeling_qwen3_5.py", - "src/transformers/models/qwen3_5/configuration_qwen3_5.py", - "src/transformers/cache_utils.py", - "src/transformers/utils/import_utils.py" - ], - "readset": "references/hf-qwen35-gdn-source.json", - "license": "Apache-2.0", - "scope": "v5.10.2 Qwen3.5 conv/GDN binding, cache and local numerical contract; candidate, source-only" - }, - { - "type": "git", - "url": "https://github.com/huggingface/transformers", - "commit": "856157a2f3e9594954310df18fdccc31ffddebe9", - "paths": [ - "LICENSE", - "src/transformers/models/qwen3_5/modeling_qwen3_5.py", - "src/transformers/models/qwen3_5/configuration_qwen3_5.py", - "src/transformers/cache_utils.py", - "src/transformers/utils/import_utils.py", - "src/transformers/integrations/hub_kernels.py", - "src/transformers/integrations/__init__.py", - "src/transformers/masking_utils.py" - ], - "readset": "references/hf-qwen35-gdn-source.json", - "license": "Apache-2.0", - "scope": "v5.17.0 Qwen3.5 conv/GDN binding, cache and local numerical contract; candidate, source-only" - }, - { - "type": "git", - "url": "https://github.com/sgl-project/sglang", - "commit": "16176278d0180f1a1a578e53e6eef432577876c2", - "paths": [ - "LICENSE", - "python/sglang/kernels/ops/mamba/causal_conv1d.py", - "python/sglang/kernels/ops/mamba/causal_conv1d_triton.py", - "python/sglang/srt/arg_groups/fields/exec_.py", - "python/sglang/srt/configs/hybrid_arch.py", - "python/sglang/srt/configs/linear_attn_model_registry.py", - "python/sglang/srt/configs/mamba_utils.py", - "python/sglang/srt/configs/qwen3_5.py", - "python/sglang/srt/configs/qwen3_next.py", - "python/sglang/srt/layers/attention/attention_registry.py", - "python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py", - "python/sglang/srt/layers/attention/linear/gdn_backend.py", - "python/sglang/srt/layers/attention/linear/kernels/gdn_cutedsl.py", - "python/sglang/srt/layers/attention/linear/kernels/gdn_flashinfer.py", - "python/sglang/srt/layers/attention/linear/kernels/gdn_triton.py", - "python/sglang/srt/layers/attention/linear/utils.py", - "python/sglang/srt/layers/attention/mamba/causal_conv1d.py", - "python/sglang/srt/layers/radix_linear_attention.py", - "python/sglang/srt/mem_cache/memory_pool.py", - "python/sglang/srt/models/qwen3_5.py", - "python/sglang/srt/models/qwen3_5_text.py", - "python/sglang/srt/models/qwen3_next.py", - "test/registered/layers/mamba/test_causal_conv1d.py", - "test/registered/unit/configs/test_linear_attn_model_registry.py", - "test/registered/unit/layers/attention/test_gdn_prefill_backend_policy.py", - "test/registered/unit/layers/attention/test_linear_attn_config.py", - "test/registered/unit/layers/test_gdn_mis_metadata.py" - ], - "readset": "references/sglang-gdn-causal-conv-contract.md", - "license": "Apache-2.0", - "scope": "Bounded Qwen3.5/Qwen3-Next caller, GDN phase/architecture policy, separate conv wrappers and state metadata; no native execution or external kernel equivalence." - }, - { - "type": "git", - "url": "https://github.com/InferMatrixLab/system1-omni", - "commit": "509bdd2161ca5bef8f5e158d466514862fac7c9e", - "paths": [ - "src/models/pplx_decider/conv.py", - "src/models/pplx_decider/engine.py", - "src/models/pplx_decider/gdn.py" - ], - "readset": "references/pplx-gdn-caller-context.json", - "license": "UNKNOWN", - "scope": "Bounded secondary PPLX caller context for the HF/vLLM/SGLang cross-map: source-level default/override, conv and FI adapter/state guards only; no implementation vendored or execution qualification." - } - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton, CuTe DSL, CUDA C++, AscendC or TileIR, applying framework-independent correctness rules, and verifying the executing kernel against a reference before benchmarking; covers fused elementwise chains, reductions, tiled GEMM, attention variants, custom collectives, causal convolution and stateful serving-operator qualification", - "prerequisites": "The applicable operator, caller/state and layout contracts are read before substitution, including the causal-conv1d contract and HF Qwen3.5 GDN binding; the CuTe DSL environment uses Linux x86_64, Python 3.10–3.13 and Ampere+ (SM80+) with nvidia-cutlass-dsl installed and no NPU target; the CANN programming guide for the specific SoC is consulted for AscendC", - "input_contract": "The candidate operation or operation chain, its shapes, dtypes, quantization and layout requirements, and the target execution phase; the target accelerator and framework constraints; for serving operators, the concrete caller, effective phase and backend, state-row mapping, cache or buffer lifetime and in-place-write behavior; a reference implementation with the same signature, a callable kernel wrapper and fresh CUDA or NPU test tensors; verification shapes and dtypes plus profiler output for the target metric", - "produced_artifact": "A write-or-decline decision for the candidate operation with the profiled bottleneck, fusable pair or required dtype, shape or quantization variant unsupported by the standard implementation that justifies it, a framework choice routed by pattern and accelerator, a correctness checklist covering fp32 transcendental round-trips, sync-free wrappers, C division semantics, pre-allocated outputs, fp32 matmul accumulators and masked boundary loads and stores, a starting block or tile size with its autotune and warmup cost noted, a kernel shipping kernel_fn, reference_fn and get_inputs, a reference-parity result across the verification shapes and dtypes, and a post-verification profiler reading on the target metric; for an existing causal-convolution or stateful serving operator, a qualification record naming the caller, phase, backend, state rows and in-place writes, with source selection kept apart from executed dispatch; a routing and correctness decision, not a complete implementation or a measured speedup", - "failure_modes": [ - "A kernel is written for a standalone elementwise op, a standalone matmul or standard attention where the vendor library already wins", - "Host-side scheduling cost is treated as a kernel problem instead of being routed to host-overhead-profiling", - "Fusion is justified by removed Python call overhead rather than by the DRAM traffic the shared data no longer makes", - "A transcendental runs on fp16/bf16 inputs without the fp32 cast round-trip, so the error stays silent through benchmarking", - "tl.dot accumulates in the input dtype because Triton is assumed to promote fp16/bf16 to fp32 automatically", - "Negative offsets are computed assuming Python floor division while Triton or AscendC rounds toward zero", - "An unmasked load or store is accepted because the benchmark shapes happen to divide the block size", - "A kernel is benchmarked before it matches its reference, or a shape-boundary, accumulator-dtype or precision failure is read as a tuning problem", - "A verification pass is reported as a win although the profiler metric did not move, so the kernel was never on the critical path", - "Autotune compile cost is measured as steady-state serving latency instead of being pre-warmed or cached", - "A CuTe DSL pattern is routed to an NPU target, or a CUDA tail-guard, attention primitive or SM-matched example is assumed to carry to Ascend", - "A causal-conv1d or GDN operator is substituted on build success, import success or matching names without resolving dtype, strides, packed boundaries, state rows and cursors", - "An in-tree vLLM wrapper, a GDN prefill flag or a cache-free classification adapter is read as establishing the executed convolution path or generation-state support" - ], - "refusal_point": "Declines to start a kernel until a profiler puts the target on the critical path near its hardware ceiling, a concrete fusable pair is identified, or the required dtype, shape or quantization variant is unsupported by the standard implementation, and sends host-bound cases elsewhere; refuses to benchmark an unverified kernel and withholds a gain whose profiler metric did not move; keeps operator source selection separate from executed dispatch, numerical parity and hardware qualification, so build or import success qualifies nothing; treats AscendC and TileIR content as source-inferred pending CANN and SoC verification with no local NPU run, treats the AKG Affine FileCheck expectations as IR structure rather than executed CUDA correctness, and reads author-reported overlap and FP8 cases as counterexamples to unconditional tuning rules; claims no Ascend support from NVIDIA primitives", - "hardware": "CUDA Ampere+ and Ascend NPU have separate framework routes for each kernel pattern; CuTe DSL requires Linux x86_64, Python 3.10-3.13 and SM80+ with nvidia-cutlass-dsl and is unavailable on NPU; profiling uses ncu on CUDA and msprof on Ascend; the AscendC and TileIR guidance is source-inferred with no local NPU verification and requires validation against the specific CANN version and SoC" - }, - "recommendations": [ - "Confirm with nsys/ncu or msprof that the target kernel sits on the critical path near its hardware ceiling, name the adjacent operations whose shared data would otherwise round-trip through DRAM, or identify a required dtype, shape or quantization variant unsupported by the standard implementation, before writing any kernel.", - "Ship every kernel with kernel_fn, reference_fn and get_inputs, pass reference parity across the verification shapes and dtypes first, and profile afterwards to confirm the target metric actually moved.", - "Cast fp16/bf16 to fp32 and back for any math beyond +, -, * and /, declare tl.dot accumulators as tl.float32, and mask every load and store whose shape does not divide the block size.", - "Start Triton at BLOCK_SIZE 1024 on Hopper and 512 on Ampere using powers of two, take GEMM tiles from the source skeleton configs (128/256/64 for Hopper, 64/128/32 for smaller cards) and autotune from there.", - "On Ascend, reach for TileIR before raw AscendC, size tiles to fit the AI Core Unified Buffer and check cube-unit alignment against the CANN guide for the specific SoC.", - "Match a CuTe DSL example to the GPU architecture (SM80 Ampere, SM90 Hopper, SM100 Blackwell) and study its structure instead of copying it, and pre-warm or cache autotuned kernels at startup so first-run compile cost stays out of serving latency." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", - "The typed vLLM entry covers only the new GDN reference; older source inventories remain incomplete.", - "The vLLM and HF references identify separate source call contracts; no binary equivalence, executed backend, hardware qualification, numerical/model parity, performance or agent-efficacy evaluation.", - "SGLang source contract only; native/backend/sm89 qualification, HF classification binding, numerical/model quality, performance and agent efficacy are not evaluated.", - "The secondary PPLX caller readset has UNKNOWN repository licensing and source-only default/adapter evidence; no worker/kernel execution, model/quality/performance or stateful reuse is qualified." - ], - "provenance_readsets": [ - "references/akg-affine-tail-guards.md", - "references/causal-conv1d-contract.md", - "references/vllm-gdn-causal-conv-contract.md", - "references/hf-qwen35-gdn-binding.md", - "references/hf-qwen35-gdn-source.json", - "references/sglang-gdn-causal-conv-contract.md", - "references/pplx-gdn-caller-context.json" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/metadata.json deleted file mode 100644 index a6531022e..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/metadata.json +++ /dev/null @@ -1,93 +0,0 @@ -{ - "tags": [ - "optimization", - "search", - "policy", - "experiment-design" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Selecting, ordering and stopping optimization experiments: candidate generation, exploration versus exploitation, mechanism diversity, negative experiment memory, cost-aware escalation, budget allocation and stopping criteria, drawn from vllm-evolve, the TRT-LLM casebook and SkillOpt", - "prerequisites": "The useful work, quality contract, allowed approximations, timing boundary and resource budget are frozen before search, and a complete restartable incumbent is preserved; reading the procedures authorizes no execution, framework change, new resource spending or publication", - "input_contract": "Profiling data, existing config, prior experiment and negative records and the casebook for the current bottleneck class, model and hardware; each target stage's fraction of E2E time; the target, locally measured uncertainty and the authorized budget", - "produced_artifact": "An ordered experiment plan: one candidate change per round with an explicit hypothesis and its motivating evidence, a bounded shortlist of distinct mechanisms with assumptions, uncertainty, expected impact and evaluation cost, negative records turned into search constraints, and a stop-or-switch decision; when authorized rounds run, a recorded verdict per round with best-seen and delivered outcomes kept apart", - "failure_modes": [ - "The evaluation protocol, measurement setup or baseline changes in the same round as the candidate", - "Benchmark budget is spent on a candidate that fails correctness", - "A matching keyword in a rejection record is used as a causal diagnosis or automatic veto, or one failed implementation excludes a whole solution family", - "Expensive work starts on a stage without computing its Amdahl contribution, so a large local speedup yields little E2E gain", - "One mechanism family keeps being sampled only because budget remains, without re-profiling after the limiting stage moves", - "Roofline proximity or a utilization threshold is used as a universal stopping rule", - "Screening is allowed to skip the frozen acceptance regimes or required service-level evidence" - ], - "refusal_point": "Discards a correctness-breaking, worse or inconclusive change instead of integrating it, preserves no-change when candidates cannot be separated, does not count a shortcut outside the authorized semantics as an optimization, and does not let a successful probe qualify a broken final state" - }, - "recommendations": [ - "Resolve acceptance criteria before search: retain supplied criteria, otherwise use performance-ceiling-analysis to derive targets, record provenance and freeze supported gates.", - "Before generating a candidate, consult the casebook, the negative record and the target stage's Amdahl contribution.", - "Record every worse or inconclusive result with hypothesis, metric, context, evidence level and failure type, and derive a search constraint from it.", - "When about three experiments on one hypothesis return inconclusive from measurement variance, fix the protocol or move to a different hypothesis.", - "Before reporting, restore and verify the best fully valid incumbent through its canonical launcher." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/adaptation-interfaces.md", - "references/papers-20261015/MECHANISMS.md", - "references/papers-20261015/NEXT_GATE_LEADS.md", - "references/papers-20261015/references/source-notes.md", - "references/papers-20261015/updates/optimization-search-policy.md", - "cases/sarathi-chunk-budget.json", - "references/distserve-phase-placement/README.md", - "references/distserve-phase-placement/source-record.json" - ], - "references": [ - "../inference-optimization-casebook/cases/qwen-b300-rejected-directions.md", - "../inference-optimization-casebook/cases/qwen-b300-rejected-directions.json", - "../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md", - "../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json", - "../inference-optimization-casebook/cases/drama3-rejected-directions.md", - "../inference-optimization-casebook/cases/drama3-rejected-directions.json", - "../../workflows/rsi-evidence/references/loopx-goal-governance/README.md", - "cases/sarathi-chunk-budget.md", - "cases/sarathi-chunk-budget.json", - "../../repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md", - "../assess-serving-preemption/SKILL.md", - "../calibrate-serving-simulation/SKILL.md", - "../performance-ceiling-analysis/SKILL.md", - "../../repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md", - "../calibrate-serving-simulation/references/infera-serving-search/README.md", - "../calibrate-serving-simulation/references/infera-serving-search/source-record.json" - ], - "consumption_mode": "packaged", - "sources": [ - { - "type": "paper", - "id": "usenix:osdi24:zhong-yinmin", - "version": "OSDI 24 proceedings, July 2024, pages 193-210", - "url": "https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin", - "sections": [ - "3.3", - "4.1-4.2", - "6.1-6.5", - "7", - "Appendices A-C" - ], - "readset": "references/distserve-phase-placement/source-record.json" - }, - { - "type": "unknown", - "scope": "Pre-existing sources remain in their original procedure/readsets; this amendment does not normalize that inventory" - } - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/metadata.json deleted file mode 100644 index 66af8111e..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/metadata.json +++ /dev/null @@ -1,162 +0,0 @@ -{ - "schema_version": 1, - "status": "candidate", - "sources": [ - { - "url": "https://github.com/NVIDIA/TensorRT-LLM", - "commit": "f33e0e5fd55b", - "paths": [ - ".claude/skills/perf-analysis/SKILL.md", - ".claude/skills/perf-nsight-compute-analysis/references/roofline-analysis.md", - ".claude/skills/perf-workload-profiling/SKILL.md", - ".claude/skills/perf-optimization/SKILL.md", - ".claude/skills/perf-optimization-casebook/SKILL.md", - ".claude/skills/exec-env-check/SKILL.md", - ".claude/skills/exec-remote-slurm/SKILL.md", - ".claude/agents/perf-profiling-specialist.md" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/hahnyuan/LLM-Viewer", - "commit": "1893e4b5705d", - "paths": [ - "roofline_model.py", - "model_analyzer.py", - "hardwares/hardware_params.py", - "configs/DiT.py" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/ebugger/Empirical-Roofline-Toolkit", - "commit": "0771fa9621d0", - "paths": [ - "Empirical_Roofline_Tool-1.1.0/README.md", - "Empirical_Roofline_Tool-1.1.0/Drivers/driver1.c", - "Empirical_Roofline_Tool-1.1.0/Kernels/kernel1.c", - "Empirical_Roofline_Tool-1.1.0/Scripts/roofline.py" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "type": "paper", - "id": "usenix:fast25-qin", - "version": "FAST '25 proceedings, February 25-27, 2025, pp. 155-170", - "url": "https://www.usenix.org/system/files/fast25-qin.pdf", - "content_sha256": "543e56c541523c5e807d2231370f96749b2294bc901266397a7265b18ae2c130", - "sections": [ - "2.2, Table 1, Equations 1-2", - "3.1 and 5.3 (storage-scope limits only)" - ], - "readset": "references/mooncake-prefix-reuse-feasibility.md", - "license": "UNKNOWN" - }, - { - "type": "paper", - "id": "pmlr-v202-sheng23a", - "version": "ICML 2023 official ten-author edition, PMLR 202:31094-31116", - "url": "https://proceedings.mlr.press/v202/sheng23a/sheng23a.pdf", - "content_sha256": "7cd9673c37640011d38ef4b8595f5e1e09a85b1f803a3734f19c97ca66c75b9e", - "sections": [ - "1 and 3; 4.1-4.3, Figures 1-3, Algorithm 1, Equation 1", - "4.4 (scope boundary only); 6 and 6.1, Tables 2-3 (denominators and ablations)", - "Appendix A.3 selected cost/peak-memory formulas, PDF pages 15-17; A.2 ending only (theorem boundary)" - ], - "readset": "references/flexgen-offload-feasibility.md", - "license": "CC-BY-4.0" - }, - { - "type": "git", - "url": "https://github.com/shenh10/HuggingArch", - "commit": "facbbbc96856e558b9084a94a5070b2e2c86cf0b", - "paths": [ - "docs/validation.md", - "backend/arch/validate/_matching.py", - "backend/arch/validate/_weight_ledger.py", - "backend/arch/validate/_ablation.py", - "docs/parallelism.md", - "docs/evaluation_model.md", - "docs/measured_calibration.md", - "docs/known_limitations.md", - "kernel_bench/emit.py", - "backend/inference/op_calibration.py", - "backend/measure/scope.py", - "LICENSE" - ], - "readset": "references/huggingarch-checkpoint-anchor.md", - "identity": "references/huggingarch-source-identity.json", - "license": "MIT" - }, - { - "type": "document", - "url": "https://shenhan.cc/blog/huggingarch", - "displayed_publication_date": "2026-10-08", - "accessed": "2026-11-10", - "sha256": "31d0e0a9bbf9e32723fb6ab09179fbefb2fbba1043863f1475ecc7ea7ce24ed4", - "readset": "references/huggingarch-checkpoint-anchor.md", - "scope": "Live, unversioned companion post; sections 2.5, 2.6, 3.6 and 3.7 read from one retrieval, whose HTML bytes the sha256 identifies. Published before the pinned commit, so not attributable to it; figures absent from the repository are recorded as unverified.", - "license": "UNKNOWN" - } - ], - "verification": { - "locally_verified": false, - "evidence_level": "source_grounded", - "note": "Historical repository methods plus reviewed paper extensions: references/papers-20261015/READSET.md records exact paper versions and inspection scope. Physical/conditional bounds, observed envelopes, simulator predictions and application measurements remain distinct. The bundled pure helper computes supplied L1 counts only; no automatic workload estimation, local accelerator measurements or E2E qualification. references/huggingarch-checkpoint-anchor.md records the checkpoint gates and the two-threshold rule from HuggingArch at a full commit, with per-file digests and inspection scope in references/huggingarch-source-identity.json; per-rank weight bytes are anchored only after the second gate passes." - }, - "applicability": { - "scope": "inference serving systems (LLM, diffusion, audio)", - "hardware": "any; CUDA examples from sources, NPU adaptation required", - "prerequisites": "profiler access (nsys/ncu or NPU equivalent), or willingness to work at L1 analytical only" - }, - "tags": [ - "performance", - "roofline", - "profiling", - "bottleneck", - "inference" - ], - "profile": "local-skill-v1", - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Acceptance-target preparation is agent guidance: the CLI and pure L1 helper do not infer gates, and unsupported estimates cannot replace user criteria or qualify campaign outcomes.", - "Mooncake's prefix-reuse condition is an analytical approximation; current cache validity, placement, effective rates, overlap and SLO outcomes remain unverified.", - "FlexGen's cross-tier feasibility comparison assumes fitted rates and perfect overlap; relaxed placements may OOM, the printed bls wording remains unresolved, and current compatibility, latency and efficacy are unverified.", - "The HuggingArch checkpoint anchor covers stored weights only. KV size and growth, activation traffic, collectives and end-to-end serving performance stay unanchored by it. Per-rank weight bytes stay unanchored until the second gate passes, and that gate checks the declared layout and format, not the placement a framework uses at run time." - ], - "provenance_readsets": [ - "references/flexgen-offload-feasibility.md", - "references/huggingarch-checkpoint-anchor.md", - "references/huggingarch-source-identity.json", - "references/mooncake-prefix-reuse-feasibility.md", - "references/papers-20261015/cases/decision-cases.md", - "references/papers-20261015/govern-optimization-experiments/procedure.md", - "references/papers-20261015/govern-optimization-experiments/references/source-notes.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/bounds-predictions-and-decisions.md", - "references/papers-20261015/MECHANISMS.md", - "references/papers-20261015/README.md", - "references/papers-20261015/READSET.md", - "references/papers-20261015/redirect-cross-layer-optimization/procedure.md", - "references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md", - "references/papers-20261015/references/agentic-kernel.md", - "references/papers-20261015/references/flashvector.md", - "references/papers-20261015/references/inferencebench.md", - "references/papers-20261015/references/prior-coverage.md", - "references/papers-20261015/references/vidur.md", - "references/papers-20261015/STATIC_CHECKS.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Use supplied acceptance criteria unchanged. When absent, follow references/acceptance-criteria.md to derive targets from matching baseline, calibrated attainable performance and uncertainty; retain partial requirements and label derived fields.", - "Read the full procedure and linked resources; preserve the distinction between analytical bounds, observed attainable envelopes, predictions and measured application outcomes." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/metadata.json deleted file mode 100644 index eafcb232c..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/metadata.json +++ /dev/null @@ -1,39 +0,0 @@ -{ - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "applicability": { - "scope": "Proposing training activation and gradient memory reductions when pressure appears to come from live tensors or allocator-reserved memory, by distinguishing lifetime reordering, address packing and selective rematerialization; covers activation checkpointing for recomputation, not persistent recovery checkpoints", - "prerequisites": "The packaged evidence and limits are read before citing paper results; installing or running a solver or framework, modifying allocation or checkpoint policies, reproducing workloads or provisioning hardware needs separate authorization", - "input_contract": "The actual forward/backward/update DAG, shapes and precision, tensor sizes and consumer lifetimes; resident versus reserved peak bytes; operator, workspace and temporary-output peaks; batch and accumulation policy; optimizer and collective barriers; representative compute and transfer costs; whether the goal is fit, step time or time to a fixed training objective", - "produced_artifact": "A preferred candidate, simpler fallback, resource ledger, critical-path prediction, assumptions, counter-signals and unresolved-evidence list, with source claims kept apart from proposed decisions, plus a proposed matched comparison; a reviewable proposal, not an implementation", - "failure_modes": [ - "Fragmentation and live-byte excess are not distinguished at the same phase", - "OLLA address packing is read as tensor placement across CPU/SSD tiers", - "Reordering moves a release or update past a dependency such as shared weights, accumulation, gradient clipping, optimizer state or collectives", - "A static packing plan ignores alignment, aliasing, workspace, asynchronous overlap or shape variability", - "Added recomputation consumes the compute window that previously hid transfers", - "Checkmate's rounding is taken to guarantee the memory cap, or its 10% allowance is used as a portable safety margin", - "A solver timeout is read as proof of infeasibility" - ], - "refusal_point": "Returns the missing classification instead of a preferred checkpoint interval when only total OOM memory is known, and escalates when a legal dependency graph, memory margin or correctness model is unavailable" - }, - "recommendations": [ - "Where ordering is legal, try earlier last-use and gradient release before adding computation or transfer.", - "Check dependency validity and memory feasibility separately for an approximate schedule.", - "Account for planning cost over the expected number of repeated steps." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/metadata.json deleted file mode 100644 index 75bd18adf..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/metadata.json +++ /dev/null @@ -1,104 +0,0 @@ -{ - "schema_version": 1, - "status": "candidate", - "sources": [ - { - "url": "https://github.com/LearningToCommit/LearningToCommit", - "commit": "7e5a1d77", - "paths": [ - "data_construction/s1_scan_commits.py", - "data_construction/s3_tag.py", - "data_construction/s4_sample.py", - "data_construction/s5_build.py", - "modules/agent.py", - "modules/prompt_builder.py", - "modules/trajectory_to_md.py", - "system_prompts/solve_with_skill.txt", - "system_prompts/learn_reflect.txt", - "evaluation/pairwise_judge.py", - "evaluation/programmatic_metrics.py" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/yetone/magpie", - "commit": "77800032", - "paths": [ - "internal/", - "docs/" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/yetone/magpie", - "commit": "4b5d6d725fae3d8a263f6e43c707400e3765b84f", - "paths": [ - "internal/library/rtk.go", - "internal/library/rtk_upgrade.go", - "internal/proc/path_other.go", - "internal/proc/path_windows.go", - "internal/provider/install_windows.go" - ], - "type": "git", - "revision_precision": "full" - }, - { - "url": "https://github.com/yetone/magpie", - "commit": "9848a9062a65c49e074c09770dc7de73dd012bea", - "paths": [ - "internal/library/rtk.go", - "internal/library/rtk_upgrade.go", - "internal/proc/path_other.go", - "internal/proc/path_windows.go", - "internal/provider/install_windows.go", - "internal/library/rtk_path.go", - "internal/library/rtk_path_other.go", - "internal/library/rtk_path_windows.go", - "internal/library/rtk_path_test.go", - "internal/library/rtk_test.go", - "internal/library/rtk_upgrade_test.go", - "internal/library/library_test.go", - "internal/gui/library.go", - "internal/gui/assets/library.js", - "internal/gui/tests/library-rtk-path.test.cjs", - "library_cli.go", - "LICENSE" - ], - "type": "git", - "revision_precision": "full" - } - ], - "verification": { - "locally_verified": false, - "evidence_level": "source_grounded", - "note": "Carried LearningToCommit pipeline and Magpie history analysis, not causal evidence of development speed. The reviewed paper amendment is linked from SKILL.md; paper-to-code equivalence is not established. First-hand Magpie coordination research remains pending Dot delivery and owner review. The RTK worked transition is pinned static source/test inspection only; no upstream execution or heldout efficacy evaluation." - }, - "applicability": { - "scope": "any software repository with a recoverable git history", - "prerequisites": "git history access, optionally LearningToCommit-style benchmark for evaluation" - }, - "tags": [ - "repository-intelligence", - "history", - "skill-distillation", - "evaluation" - ], - "profile": "local-skill-v1", - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "The RTK transition is a learning example, not an independent heldout task; upstream tests were inspected, not executed.", - "Existing audit and synthetic KV comparison adapters are not a generic historical coding-task evaluator." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "references": [ - "references/rtk-consumer-path-transition.md" - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/metadata.json deleted file mode 100644 index 07f7a0b92..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/metadata.json +++ /dev/null @@ -1,45 +0,0 @@ -{ - "tags": [ - "skill-evolution", - "trace2skill", - "skillopt", - "rsi" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Design choices for evolving agent skills from execution traces, drawn from Trace2Skill and SkillOpt: when a trajectory should update a skill, how conflicting lessons are handled, how negative evidence is preserved, how large an edit should be and when skills split or merge; for designing or operating a skill evolution pipeline", - "prerequisites": "A final-test partition kept outside edit selection and the incumbent skill retained for rollback; changing a skill's scope first needs explicit review of its identity and consumers", - "input_contract": "Execution trajectories or trace IDs with the diagnosed causal failure and the successful behavior an amendment must preserve; the incumbent skill and its revision lineage; held-out evaluation results for candidate edits", - "produced_artifact": "A scoped case or amendment proposal, or a consolidated candidate edit, with rejection records and the versioned revision lineage; a candidate for independently evaluated acceptance, not a replacement decision or an efficacy claim", - "failure_modes": [ - "An anecdote, or agreement among analysts, is taken as efficacy", - "Consolidation output is treated as a replacement decision instead of a candidate awaiting independent acceptance", - "A tie or uncertain result is reported as improvement", - "A conflict is resolved by voting for whichever outcome appeared more often in the pool", - "A rejection is applied to every future use of an idea instead of the comparison it constrains", - "Paper-specific section counts or length budgets are used as universal defaults, or a skill is split on length alone", - "A stronger task solver is assumed to be a stronger improver, or replay rankings and private inner scores are read as outer generalization" - ], - "refusal_point": "Records a conflict that conditioning cannot resolve, where the same context produces different outcomes across runs, as INCONCLUSIVE and flags it for human review instead of choosing a lesson; claims no universal minimum trace count, paper-to-code equivalence or local efficacy" - }, - "recommendations": [ - "Condition a conflicting lesson on its context (when X, do A; when Y, avoid A) before choosing one side.", - "Store each rejection record with its evaluation evidence, outside the deployed task procedure, and give it a retirement condition.", - "Bound the target passage and intended behavior before editing, inspect the actual diff and preserve rollback state.", - "Prefer an amendment when the existing skill owns the workflow; create a new candidate only for a genuinely distinct capability." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "searchspace_not_applicable": "SKILL.md §§ When should one trajectory update a skill?, How large should a skill edit be?, and When should skills split or merge? govern skill-text evolution and held-out evaluation, not hardware-resource state or control." -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/metadata.json deleted file mode 100644 index 3a92cc24c..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/metadata.json +++ /dev/null @@ -1,48 +0,0 @@ -{ - "tags": [ - "training", - "input-pipeline", - "prefetch", - "dataset", - "memory", - "storage" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Diagnosing training input stalls and choosing bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic Streaming and FFCV, while preserving execution-mode gates, sample ordering and per-host memory budgets and separating remote downloads, local storage reads and host/device transfers", - "prerequisites": "The pinned readset and its license constraints are read; reading authorizes no commands, cache manipulation or experiment", - "input_contract": "Dataset format, sample-size distribution, workers and ranks per host, ordering/shuffle, local-cache state and executor mode; a per-host memory budget that includes checkpoint staging and runtime offload; timings that separate remote fetch, decompression/decoding, transformation, batch readiness and device transfer", - "produced_artifact": "One proposed queue, shard-cache or cache-ownership change with its source-backed rule and boundary, the evidence proposed to confirm it, and a verification contract for a separately authorized comparison; no input-throughput, model-quality or hardware result", - "failure_modes": [ - "Low accelerator utilization is taken to identify which producer stage is late", - "Missing legacy DALI statistics are read as zero use, or the execution mode is switched silently to obtain them, which changes the comparison", - "DALI run() returning is read as output-ready timing instead of the outputs() synchronization boundary", - "Mosaic predownload is treated as a byte cap or request count, or per-worker sample targets are multiplied into an exact network-byte prediction", - "Look-ahead is increased when the local cache cannot retain the working set, so evicted shards are downloaded again", - "cache_limit is read as a bound on total process RAM, or the constructor minimum as an optimum or a guarantee against thrashing", - "An FFCV memory map is taken as proof that its pages are resident" - ], - "refusal_point": "Rejects changes that violate sample coverage or the stated reproducibility contract even if batches arrive faster, does not report queue depth alone as a performance result, and draws no NPU acceleration claim from the CUDA-oriented DALI source" - }, - "recommendations": [ - "Record dataset format, sample sizes, workers per host, ordering, cache state and executor mode before proposing one change.", - "Increase Mosaic look-ahead only when readiness starvation is observed and cache headroom exists; otherwise address shard and working-set residency first.", - "Check the exact DALI configuration surface before use, because the constructor and deserialization paths use different queue keys.", - "In a later comparison, separate warm-cache throughput from cold retrieval and initialization, and report step time with input wait, host/HBM peaks and actual I/O." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/sources.md" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/metadata.json deleted file mode 100644 index 089b91e4f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/metadata.json +++ /dev/null @@ -1,56 +0,0 @@ -{ - "tags": [ - "training", - "offload", - "resource-budget", - "memory", - "cpu", - "numa" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Choosing training offload and resource-budget changes from pinned DeepSpeed configuration, attributed expert diagnosis and a historical MindSpeed case, balancing accelerator capacity, CPU work, pinned RAM and local storage while keeping host/device transfer exposure distinct from collective network bottlenecks", - "prerequisites": "The source boundaries are read; garbage-collection or affinity changes, privileged memlock changes and delayed parameter updates each need separate approval, and the skill authorizes no newest-wheel installation, scheduler-priority change, benchmark or failure injection; historical MindSpeed reuse requires compatible allocator symbols, tracked swappable storage, sufficient host capacity and verified device/NUMA/CPU mapping, not merely the presence of npu-smi and lspci", - "input_contract": "State attributed to parameters, optimizer state, gradients or activations with its current memory tier, lifetime and next consumer; whether the failure is allocation capacity or throughput; a per-host budget for input buffers, checkpoint snapshots, offloaded state, OS cache and CPU workers with the actual ranks sharing the host; the host/device link recorded apart from collective traffic; the exact source version and enabled execution mode", - "produced_artifact": "A candidate decision record per proposal: the observed limiting interval and owning state or resource, source version and execution mode, capacity released and resources consumed with units and uncertainty, overlap readiness and mutation boundaries with expected contention, and the fixed-workload step and correctness evidence required for later validation", - "failure_modes": [ - "A change that lets the model fit is presented as a speedup, although it can slow each step", - "A tensor copied to CPU is taken as an InfiniBand/RoCE bottleneck, or CPU/NVMe offload is counted as network evidence", - "Parameter max_in_cpu is read as a byte or total-host-memory cap instead of an element count", - "NVMe optimizer placement is assumed to remove CPU optimizer demand", - "The parameter buffer-pool count is transplanted to the optimizer, or all optimizers are assumed to hold identical states", - "Configuration presence or schema documentation is taken as evidence that a knob or overlap actually ran", - "Ordinary overlap is taken as permission for delayed, stale parameter updates" - ], - "refusal_point": "Claims no hardware validation, measured gain or current NPU compatibility, keeps source support and extracted coverage separate from measured improvement, and does not reuse the MindSpeed headline gain; historical source observations do not establish current-stack compatibility, whole-model correctness or measured benefit" - }, - "recommendations": [ - "Attribute the state and separate a capacity failure from a throughput bottleneck before choosing an offload surface.", - "Verify the consumer of buffer_size before calculating bytes or tuning it, and measure the executing path before claiming a knob ran.", - "Compare the whole step's critical path, not host capacity alone.", - "Use plan-activation-residency, choose-parallel-training-plan and assess-split-backward-schedule for live-range, parallel-plan and backward-ordering questions." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/mindspeed-historical.md", - "references/mindspeed-source-readset.md", - "references/papers-20261015/cases/paper-grounded-cases.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/offload-delta.md", - "references/papers-20261015/notes/training-memory-scheduling.md", - "references/papers-20261015/references/source-readset.md", - "references/sources.md", - "references/stas-decision-notes.md" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/metadata.json deleted file mode 100644 index a3fa54151..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/metadata.json +++ /dev/null @@ -1,36 +0,0 @@ -{ - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": ["first-party"], - "tags": [ - "video-diffusion", - "dit", - "moe", - "attention", - "vae", - "performance", - "profiling", - "serving", - "optimization" - ], - "empirical_validation": "not_run", - "verification": { - "locally_verified": false, - "evidence_level": "guidance", - "note": "Procedure is first-hand workflow guidance. The linked field notes are not reproducible as published. No acceptance campaign under this repository's evidence contract backs this skill, and no public performance claim is made." - }, - "applicability": { - "scope": "Backend-agnostic video diffusion inference and serving optimization", - "hardware": "Any accelerator (Ascend NPU, CUDA GPU, and others); operator names, memory layouts, and profiler references are re-derived per backend", - "prerequisites": "A reproducible benchmark, a frozen quality check, and numeric gates defined in the governing acceptance issue" - }, - "limitations": [ - "Backend-agnostic skeleton only. The first-hand field notes are Ascend-specific and live in the ascend-video-diffusion-optimization skill; every operator, layout and profiler reference (TransData, FRACTAL_NZ, CANN) must be re-derived per backend.", - "Candidate workflow guidance; not independently validated.", - "Model identifiers, campaign revisions, and numeric gates are intentionally excluded and remain issue-scoped." - ], - "provenance_readsets": [], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/metadata.json b/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/metadata.json deleted file mode 100644 index fa7edd96e..000000000 --- a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/metadata.json +++ /dev/null @@ -1,151 +0,0 @@ -{ - "tags": [ - "storage", - "weight-loading", - "startup", - "prefetch", - "offload", - "serving", - "npu", - "collectives", - "tensor-parallel", - "Flare", - "model-preparation", - "prepared-runtime-package", - "expert-cache", - "loader-correctness" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ggml-org/llama.cpp", - "commit": "3e390af18de1131dda27913c00511725b4274c8a", - "paths": [ - "common/arg.cpp", - "common/common.cpp", - "common/fit.cpp", - "src/llama-model.cpp", - "src/llama-model-loader.cpp", - "src/llama-model-loader.h", - "src/llama-mmap.cpp", - "src/llama-mmap.h", - "LICENSE" - ], - "readset": "references/llama-loading-readset.md" - }, - { - "type": "git", - "url": "https://github.com/foundation-model-stack/fastsafetensors", - "commit": "219e2561a41ede2d5d8eba611c248bb0effbf580", - "paths": [ - "fastsafetensors/loader.py", - "fastsafetensors/file_buffer.py", - "fastsafetensors/tensor_factory.py", - "fastsafetensors/parallel_loader.py", - "fastsafetensors/frameworks/_torch.py", - "fastsafetensors/frameworks/_paddle.py", - "docs/overview.md", - "examples/run_parallel.py", - "tests/unit/test_broadcast_runs.py", - "tests/unit/test_subgroup.py", - "LICENSE" - ], - "readset": "references/fastsafetensors-transfer-readset.md" - }, - { - "type": "git", - "url": "https://github.com/NVIDIA/flare-local-inference", - "resolved_url": "https://github.com/NVIDIA/local-inference", - "commit": "bce032c014bae34502ba22d3304fa084d5b40afe", - "paths": [ - "README.md", - "docs/runtime-package-design.md", - "helios/src/runtime_package/format.rs", - "helios/src/runtime_package/model.rs", - "helios/src/execution/manifest.rs", - "helios/src/model/contract.rs", - "helios/src/model/catalog.rs", - "LICENSE" - ], - "readset": "references/flare-preparation-boundary.md", - "license": "Apache-2.0" - }, - { - "type": "git", - "url": "https://github.com/ronak-create/deepseek-v4-in-c", - "commit": "d0402056d0da4bf7edfd2d4977de479a0db9d201", - "paths": [ - "LICENSE", - "README.md", - "Makefile", - "src/cache/dsv4_cache.c", - "src/cache/dsv4_cache.h", - "src/io/dsv4_st.c", - "src/io/dsv4_st.h", - "src/model/dsv4_layer.h", - "src/model/dsv4_layer.c", - "src/cli/dsv4_run.c", - "tests/unit/test_cache.c", - "tests/unit/test_model_oracle.c", - "tools/make_st_fixture.py", - "tools/make_tiny_checkpoint.py", - "tools/dsv4_ref.py" - ], - "readset": "references/deepseek-expert-cache-oracles.md", - "license": "Apache-2.0 (repository declaration and selected-file notices; third-party/model rights excluded)" - } - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Diagnosing the serving-level weight load path from storage to device and choosing changes to it when startup or loading latency dominates: locating the cost by source, format, placement and duplication, accounting once-per-process loading and warmup apart from per-request cost, verifying sharded downloads, restricting device-side prefetch to profiled memory-bound hot spots and ordering CPU-tier offload; also covers source-scoped disk-loading modes, pinned staging, synchronous checkpoint save semantics, fastsafetensors broadcast/scatter/coalesced delivery and the separation of model preparation from serving. Request-time host overhead belongs to host-overhead-profiling, DCP asynchronous save performance to dcp-async-save-performance and distributed recovery validity to checkpoint-recovery-consistency; for disk-streamed routed experts, audit independent byte identity, resident/pending slot ownership, ordered accumulation and actual oracle callback coverage; the DeepSeek source slice describes POSIX storage, C/OpenMP and pthread ownership; target validation must identify its actual direct/buffered and reader-pool paths", - "prerequisites": "Flag names and API entry paths are re-verified against the pinned source revisions, and torch_npu.npu_prefetch details against the target CANN version, before they are relied on; card state is checked before launch and process placement fixed before measuring, through ascend-device-health and os-process-placement; the Diffusers enable_model_cpu_offload path requires accelerate", - "input_contract": "The current load path's byte source, file format, host-first or direct device placement and duplicated copies across processes or instances; startup timings for loading and warmup with byte and access counts per segment, and the expected warmup hit criteria; the safetensors index weight_map and the downloaded shard set; a profile showing memory-bound hot spots and the adjacent compute or communication segments; free device memory and whether an OOM occurs during construction or inference; for cross-rank delivery, the file-to-loader-group-rank map, tensor names, shapes, dtypes and call order, the required replicated or sliced output, the model's expected shard layout and the collective backend; for streamed expert-cache changes, the selected direct/buffered I/O and reader-pool path, tensor offsets and cache-slot budget, requested keys, source callback table, independent byte/numerical reference and checked fixture coverage", - "produced_artifact": "The located load-path cost on the four dimensions with its matching lever, a startup ledger that keeps fixed overhead apart from per-request cost, a structural shard-completeness result naming the missing shards, a prefetch proposal behind an enable_prefetch switch defaulting to False or a recorded decision not to prefetch, an offload candidate order, the recorded loading mode, a distributed delivery choice with its proposed verification, and a preparation/serving record of serialized and rebuilt state, live resource owners and ABI/prepared-contract versions; proposed changes and verification targets, not a measured startup or delivery gain; a source-bound expert-cache byte/state audit and oracle-path coverage record, with synthetic-fixture, fallback and SKIP limits", - "failure_modes": [ - "Loading and warmup are amortized into per-request latency, or a host segment without timing points is treated as costing nothing", - "Warmup is accepted on total time while mismatched warmup shapes have warmed a different set of cache entries", - "In-flight .incomplete shards are re-downloaded as corrupt, or gap-free shard numbering is accepted without matching the index weight_map", - "Prefetch is forced without a profiled memory-bound hot spot or a non-memory-bound dependency window", - "Resolution or CFG tier-downs are applied to a construction-phase OOM, or offload is adopted when compute already hides the transfer", - "A free-memory probe runs on an mmap-backed load, or a run whose first mlock failure latched with only a warning is treated as fully resident", - "Disk counters that include cache and pipes are read as physical writes, or a run without a page-cache drop is reported as a cold start", - "The ceil-block scatter partition is used without checking it against the model's expected shard layout, or ranks run different orders or plans so their collectives mismatch", - "A faster H2D copy, local clone or rank-local filtered read is counted as a cross-rank communication gain", - "Serial and concurrent cache paths agree on identically corrupted placement bytes, or a contiguous fixture misses a separated-run alignment failure", - "Resident bytes or ordered per-k accumulation change while misses are pending, or an exit-zero SKIP/non-overlapped oracle is counted as overlap validation" - ], - "refusal_point": "Withholds a load-path win until the verification checklist holds and keeps the distributed-delivery checks as verification targets rather than results; does not force prefetch without a safe dependency window and disables it when neither a smaller size nor a moved position removes a regression; records unsupported save behaviors as unknown; rejects mismatched execution ABI or prepared-contract versions instead of editing their tags; and implies no Ascend support from the llama.cpp or fastsafetensors mechanisms; withholds expert-cache correctness/overlap conclusions when byte coverage, actual I/O/pool dispatch or model-oracle callback reachability is missing", - "hardware": "NPU for the device-side prefetch API (torch_npu.npu_prefetch, checked against the target CANN version) and the Diffusers offload playbook; the prefetch and fastsafetensors methodology transfers while the fastsafetensors and llama.cpp APIs do not, and cross-rank completion follows each collective backend's own wait contract" - }, - "recommendations": [ - "Report preparation/startup/readiness, warmups and measured work as separate numbers, with load latency split into init, header, io and get phases, the cold-cache protocol stated and the effective loading mode identical across compared runs.", - "Place each prefetch behind an enable_prefetch switch defaulting to False, start max_size at 50% of the theoretical size, verify a single operator first and step up to 0.7 and then 1.0 only after verified wins.", - "Try offload candidates in order of change surface: batch size, resolution, CFG/feature flags, CPU or layered offload, tensor-parallel degree, then activation checkpointing; re-estimate the un-overlapped transfer share after adopting each tier.", - "Record whole-set atomic publication, asynchronous checkpointing, multi-rank save coordination and crash durability as unknown for the fastsafetensors saver instead of assuming them.", - "Before adopting coalesced delivery, compare it disabled and enabled with identical checkpoint, rank mapping, backend, topology, dtype, cache state, chunk/queue budgets and consumer retention, then measure complete startup under the same accounting protocol.", - "For streamed expert-cache changes, compare against independent buffered tensor bytes, exercise separated-run geometry, check resident bytes during pending reads, preserve ordered per-k accumulation and verify that the numerical oracle selects the actual callbacks." - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Normalized provenance covers the existing llama.cpp loading slice at 3e390af18de1131dda27913c00511725b4274c8a and the fastsafetensors distributed-delivery slice at 219e2561a41ede2d5d8eba611c248bb0effbf580; other inherited claims remain unnormalized with original identities and limits retained.", - "The readsets record static mechanisms and source recommendations, not validated startup, memory, placement or backend performance.", - "The fastsafetensors delivery readset does not revalidate inherited I/O/save claims or qualify uneven scatter, accelerator/NPU compatibility, inter-node transport or agent efficacy.", - "Flare adds source-only preparation/serving ownership and ABI/contract checks; live exporter handles, native backend execution, startup gains and model/dependency rights are not qualified.", - "The DeepSeek increment records source/test definitions only: its buffered byte oracle shares tensor metadata, its resident snapshot covers w1 bytes, and its tiny PyTorch/batched checks do not qualify the released model, backend performance or agent efficacy." - ], - "provenance_readsets": [ - "references/llama-loading-readset.md", - "references/fastsafetensors-transfer-readset.md", - "references/flare-preparation-boundary.md", - "references/deepseek-expert-cache-oracles.md", - "references/deepseek-expert-cache-source.json" - ], - "consumption_mode": "packaged" -} diff --git a/src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/audit-fixed-workload-claim/SKILL.md similarity index 52% rename from src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/audit-fixed-workload-claim/SKILL.md index 42dd11952..95b3c670b 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/audit-fixed-workload-claim/SKILL.md @@ -1,6 +1,43 @@ --- name: audit-fixed-workload-claim description: Audit a performance claim from supplied fixed-workload reports and an explicit measurement contract. Check aggregation, retained observations, comparability and quality evidence without launching new benchmarks. +metadata: + inferrsi: | + { + "status": "candidate", + "origin": [ + "internal" + ], + "tags": [ + "performance", + "audit", + "measurement" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Auditing a performance claim against supplied fixed-workload reports and an explicit measurement contract: aggregation, retained observations, comparability and quality evidence", + "input_contract": "Baseline and candidate fixed-workload reports with request observations grouped by trial and workload; the measurement contract stating metric, unit, timing boundary, improvement direction, aggregation rule and order, fixed exclusions, threshold and comparison operator; the declared quality and completion evidence; the requested output schema", + "produced_artifact": "An audit conclusion showing baseline and candidate aggregates, the calculation, the threshold comparison and the evidence or missing information that controls the verdict; it interprets supplied evidence and does not itself establish a measured improvement", + "failure_modes": [ + "A pooled request median silently replaces a median of per-trial means, or the reverse, reweighting trials whose request counts differ", + "Late requests, failures or outliers are dropped, or requests move between trials, outside the exclusions the contract fixes", + "Values are rounded before the threshold comparison", + "A missing rule that permits different verdicts is resolved toward the favorable one", + "A speed improvement is read as task success, or a quality pass as equivalent measured work", + "Different workloads or incomplete observations are repaired by a favorable aggregate" + ], + "refusal_point": "Does not replace missing evidence with a pass, does not support a percentage claim on an undefined denominator, and does not launch a new benchmark or alter the contract to resolve an unfavorable result; a failed threshold is reported apart from an unverified comparison or missing quality evidence" + }, + "recommendations": [ + "Recover the contract's metric, direction, aggregation order and comparison operator before reading the report's own verdict.", + "Treat a named minimum bound as >= and a maximum bound as <= unless the contract requires a strict comparison.", + "Keep separate regimes and metrics separate unless the contract defines their aggregation." + ], + "limitations": [ + "Guidance for supplied reports; no local efficacy validation or measured performance gain is established." + ], + "searchspace_not_applicable": "SKILL.md §§ Preserve the observation hierarchy, Check what was actually compared, and Return an auditable conclusion audit supplied measurements and verdicts; device identity is comparison metadata, not a hardware-resource mechanism." + } --- # Audit a Fixed-Workload Claim diff --git a/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/SKILL.md new file mode 100644 index 000000000..42e5afd68 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/SKILL.md @@ -0,0 +1,115 @@ +--- +name: calibrate-serving-simulation +description: Build an evidence-gated serving-configuration shortlist from a profiled simulator while separating empirical predictions from physical bounds and near-capacity queueing risk. +metadata: + inferrsi: | + { + "status": "candidate", + "empirical_validation": "not_run", + "source_pack": "A", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "paper", + "id": "arxiv:2405.05465", + "version": "v2", + "url": "https://arxiv.org/abs/2405.05465v2", + "sections": [ + "3-7", + "A.1-A.2" + ], + "readset": "references/source-notes.md" + }, + { + "type": "git", + "url": "https://github.com/AMD-AGI/Infera", + "commit": "ff1136e4a51998eba2933ed63cfb9c10120f86ea", + "paths": [ + "LICENSE", + "manual/simulation/tuning_agent.md", + "manual/simulation/boundaries.md", + "manual/simulation/anchors.md", + "manual/simulation/workloads.md", + "infera/projection/agents/tuning_agent/history.py", + "infera/projection/agents/tuning_agent/cli.py", + "infera/projection/agents/tuning_agent/evaluator.py", + "infera/projection/agents/tuning_agent/inference_tuning.py", + "infera/projection/agents/tuning_agent/inference_agent.py", + "infera/projection/core/projection/inference_projection/launcher.py", + "infera/projection/core/projection/inference_projection/des.py", + "tests/unit/projection/test_tuning_slo.py", + "tests/unit/projection/test_tuning_objectives.py", + "tests/unit/projection/test_tuning_variable_lengths.py", + "tests/unit/projection/test_des_disaggregated.py" + ], + "readset": "references/infera-serving-search/source-record.json" + } + ], + "applicability": { + "scope": "Building an evidence-gated serving-configuration shortlist when a profiled simulator or cost model is proposed to reduce deployment experiments; extends performance-ceiling-analysis, optimization-search-policy and inference-benchmark-design without replacing their ceilings or measurement contract", + "prerequisites": "The packaged Vidur evidence and evidence-type notes are read; reading the skill does not authorize execution, and no validation check runs under it without separate execution authorization", + "input_contract": "Model, shape and dtype; actual kernel/runtime revision; graph mode; GPU, host and topology; request length and arrival distribution; total resources and cost basis; memory limits; latency definitions and percentile SLOs", + "produced_artifact": "A bounded decision memo with input identities, model assumptions, evidence type, exact metrics and denominators, unsupported dimensions, nondominated candidates with predicted ranges, joint constraint checks, the next discriminating trial and rollback/finalization requirements, keeping prediction and measurement columns apart; a shortlist for future measurement, not a measured result", + "failure_modes": [ + "Learned runtime output is labeled a physical bound", + "A missing identity is filled with a paper default, or topology is substituted, instead of being marked unknown", + "A low-load check stands in for an SLO claim at the intended load regime and capacity boundary", + "A predictor that relies on summed KV volume hides batches with equal volume but unequal context distributions", + "Capacity binary search is applied without verifying its monotonic/stationary premise or treating overloaded samples explicitly", + "One predicted winner is reported when uncertainty exceeds rank separation" + ], + "refusal_point": "Stops predictive certification when profiles are missing, topology is substituted, error grows near capacity or host overhead is omitted; labels an ordering unresolved when uncertainty exceeds rank separation; asserts neither universal error below 9% nor current vLLM, asynchronous PP or NPU support from the historical simulator paper" + }, + "recommendations": [ + "Classify every estimate as a physical bound, measured envelope, Amdahl-style conditional impact or empirical prediction before comparing configurations.", + "Keep prefill and decode separate in the operator signatures, and record how sharding and memory planning enter the model.", + "Recalibrate after workload drift, kernel/graph/runtime changes, new hardware or topology, or mismatched residuals." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/infera-serving-search/README.md", + "references/infera-serving-search/source-record.json" + ], + "consumption_mode": "packaged" + } +--- + +# Calibrate a serving-simulation shortlist + +Use when simulation or a cost model is proposed to reduce deployment experiments. This candidate extends existing `performance-ceiling-analysis`, `optimization-search-policy` and `inference-benchmark-design`; it does not replace their ceilings or measurement contract. Reading this skill does not authorize execution; follow the current task's scope. + +## Inputs and prerequisites + +Require model/shape/dtype, actual kernel/runtime revision, graph mode, GPU/host/topology, request length and arrival distribution, total resources/cost basis, memory limits, latency definitions and percentile SLOs. Read [Vidur evidence](references/source-notes.md) and [evidence types](references/source-notes.md). If any essential identity is missing, mark that dimension unknown rather than borrowing a paper default. + +For Infera serving-tuner output, read the [provenance reference](references/infera-serving-search/README.md) before using a shortlist. It distinguishes actual fidelity, the seed-only comparator, evaluated-trial retention and scalar versus DES metrics; independent trace-ranking calibration remains proposed and `not_run`. + +## Procedure (P) + +1. Classify each estimate: physical feasibility bound, measured attainable envelope, Amdahl-style conditional impact, or empirical prediction. Never label learned runtime output a bound. +2. Build operator dependency signatures: current tokens; phase and context-length distribution; KV bytes; communication payload plus topology. Keep prefill and decode separate. Record how sharding and memory planning enter the model. +3. Record calibration domain and held-out coverage. Preserve skewed attention cases, small-model host overhead and tile/shape transitions. If the predictor relies on summed KV volume, challenge batches with equal volume but unequal context distributions. +4. Validate plans must distinguish operator/iteration errors from static normalized execution time and dynamic E2E/queue delay. Include the intended load regime and capacity boundary; do not substitute a low-load check for an SLO claim. No checks are run under this skill without separate execution authorization. +5. Enumerate feasible configuration families under total resource cost and memory budgets. If using capacity binary search, verify the empirical monotonic/stationary premise and define its scheduling-delay threshold. Treat overloaded samples explicitly. +6. Filter on all latency/quality constraints together. Retain nondominated alternatives and uncertainty bands, including differing mechanisms, rather than one predicted winner. If uncertainty exceeds rank separation, label the ordering unresolved. +7. Produce a small future-measurement shortlist whose trials discriminate among competing explanations. Recalibration triggers include workload drift, kernel/graph/runtime changes, new hardware/topology and mismatched residuals. + +## Output + +A bounded decision memo: input identities; model assumptions; evidence type; exact metrics/denominators; unsupported dimensions; candidates and predicted ranges; joint constraint checks; next discriminating trial; rollback/finalization requirements. Prediction and measurement columns must not be conflated. + +## Failure / stop conditions + +Stop predictive certification if profiles are missing, topology is substituted, error grows near capacity, or host overhead is omitted. Do not assert universal error below9%: Vidur AppendixA has12.65% maximum error in a specified near-capacity case. Do not claim current vLLM, asynchronous PP or NPU support from this historical simulator paper. + +## Resource contract + +The local `references/source-notes.md` retains the Vidur evidence. The [Infera reference](references/infera-serving-search/README.md) and its [source record](references/infera-serving-search/source-record.json) retain the separate implementation example and its limits. These owner-local resources support selective consumption. Optional comparison notes and cases are linked through the owning ceiling skill. Existing InferRSI skill names above are reconciliation targets, not runtime dependencies. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/README.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/README.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/README.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/source-record.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/source-record.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/infera-serving-search/source-record.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/infera-serving-search/source-record.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/calibrate-serving-simulation/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/calibrate-serving-simulation/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/dense-performance-evidence/SKILL.md similarity index 64% rename from src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/dense-performance-evidence/SKILL.md index bfcb04763..c206cfb71 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/dense-performance-evidence/SKILL.md @@ -3,6 +3,54 @@ name: dense-performance-evidence description: 'Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps evidence types to the questions they answer and the claims they can support. Use to decide which evidence to collect next, avoid over-claiming from cheap evidence, and structure a measurement campaign. ' +metadata: + inferrsi: | + { + "tags": [ + "performance", + "evidence", + "profiling", + "measurement" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Deciding which performance evidence to collect next and which claim each evidence level supports, from correctness through microbenchmark, component profile, isolated scenario benchmark and multi-regime serving benchmark to a production A/B experiment; also structuring a measurement campaign", + "prerequisites": "Level 0 correctness evidence is collected before any performance measurement; a Level 5 experiment needs production infrastructure, traffic splitting and statistical power", + "input_contract": "The claim to be made or the question to answer, and the evidence already collected with its level, workload, hardware/software context and controls", + "produced_artifact": "The minimum evidence level each claim requires, the next evidence to collect under the escalation rule, and negative-result records with hypothesis, evidence level, the metric that did not improve, context and the reason if known; a classification of evidence, not a measurement", + "failure_modes": [ + "An md5 match or a test pass is read as semantically correct output or as support for a performance claim", + "A microbenchmark improvement is reported as an inference throughput gain", + "Absolute kernel durations or single collective latencies are compared across devices or sessions", + "A win on one fixed workload or concurrency is generalized to production traffic", + "Worse metrics, per-task regressions or negative results are discarded, or a shared task budget is taken as equal total cost" + ], + "refusal_point": "Withholds any claim above the level its evidence reaches, leaves an unresolved source protocol or denominator mismatch unresolved, and lets neither upstream scores nor local packaging checks promote an InferRSI candidate" + }, + "recommendations": [ + "Collect Level 0 first and profile at Level 2 before implementing; escalate to Level 3 only after a promising Level 2 result, and to Level 4 only after a Level 3 win worth validating across regimes.", + "Collect a Level 1 microbenchmark only for an isolated-kernel question after Level 2 has confirmed that kernel is on the critical path.", + "For a Level 1 claim, pin input shape and dtype, exclude warmup, synchronize before timing, run at least 10 repetitions and report median and IQR; for a Level 3 claim, match hardware, software stack, model revision and inputs, run at least 3 repetitions and alternate A/B/A order.", + "Read the admission/drain occupancy case when interpreting finite service cohorts; phase-selected occupancy is not end-to-end gain." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "references": [ + "../../../casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md", + "../../../casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.json" + ], + "consumption_mode": "packaged" + } --- # Dense Performance Evidence Taxonomy @@ -143,8 +191,8 @@ with conditional denominators. Preserve worse metrics and per-task regressions; a shared task budget alone does not prove equal total cost. An unresolved source protocol/denominator mismatch stays unresolved. -[Paper-method comparison](../../workflows/npu-skill-factory/references/papers-20261015/COMPARISON.md) -and [protocol details](../../workflows/npu-skill-factory/references/papers-20261015/references/protocol-details.md) +[Paper-method comparison](../../../workflows/maintenance/npu-skill-factory/references/papers-20261015/COMPARISON.md) +and [protocol details](../../../workflows/maintenance/npu-skill-factory/references/papers-20261015/references/protocol-details.md) provide the reviewed evidence; neither upstream scores nor local packaging checks promote an InferRSI candidate. For performance predictions, distinguish physical bounds, observed envelopes, simulator predictions and measured application results @@ -176,4 +224,4 @@ evidence preservation in rejected edit records). ## Scoped internal experience -Read the single authoritative [admission/drain occupancy case](../inference-optimization-casebook/cases/admission-drain-occupancy.md) when interpreting finite service cohorts. It separates recorded observation, action and interpretation; phase-selected occupancy is not end-to-end gain. The original frozen evidence remains linked, not copied. This shared reference is included in the packaged skill library; standalone folder export is not supported. +Read the single authoritative [admission/drain occupancy case](../../../casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md) when interpreting finite service cohorts. It separates recorded observation, action and interpretation; phase-selected occupancy is not end-to-end gain. The original frozen evidence remains linked, not copied. This shared reference is included in the packaged skill library; standalone folder export is not supported. diff --git a/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md new file mode 100644 index 000000000..8dc890c71 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md @@ -0,0 +1,842 @@ +--- +name: inference-benchmark-design +description: 'Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image, video and VLA inference. Covers AIPerf AgentX/Exgentic replay, vLLM/SGLang benchmark boundaries, VLA action-chunk timing and quality, SLO-aware metrics and InferRSI integration.' +metadata: + inferrsi: | + { + "tags": [ + "benchmark", + "evaluation", + "serving", + "measurement", + "task-quality", + "reference-parity", + "agentic-coding", + "work-agents", + "aiperf", + "tts", + "full-duplex", + "image-generation", + "video-generation", + "asr", + "stream-boundaries", + "treatment-delivery", + "trial-validity", + "protocol-audit", + "trial-evidence", + "result-origin", + "oracle-boundary", + "candidate-selection", + "accepted-prefix", + "state-restoration", + "completion-accounting", + "vla", + "embodied", + "action-chunks", + "closed-loop", + "gpu-timing", + "profiler-attribution", + "measurement-validity" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "document", + "url": "https://github.com/InferMatrixLab/InferRSI/issues/60#issuecomment-6310920198", + "version": "2026-10-21 report", + "sha256": "c56a1b169d70d9cd2bfcf073b13e001354945d71c54ebde5740f3036e4ee4496", + "readset": "references/task-benchmark-selection.md", + "scope": "Distilled task matrix, metric definitions, integration boundaries and command templates; original report is source/documentation review, not a runtime benchmark." + }, + { + "type": "git", + "url": "https://github.com/vllm-project/vllm-omni", + "commit": "b20e5bac96a029172a59559438ad0590460aeb3c", + "paths": [ + "benchmarks/accuracy/README.md", + "benchmarks/accuracy/image_to_image/README.md", + "benchmarks/accuracy/text_to_image/README.md", + "benchmarks/diffusion/backends.py", + "benchmarks/diffusion/diffusion_benchmark_serving.py", + "benchmarks/tts/README.md", + "benchmarks/tts/bench_tts.py", + "docs/cli/bench/serve.md", + "tests/e2e/accuracy/wan22_i2v/test_wan22_i2v_video_similarity.py", + "vllm_omni/benchmarks/data_modules/seed_tts_eval.py", + "vllm_omni/benchmarks/duplex_session_metrics.py", + "vllm_omni/benchmarks/metrics/metrics.py" + ], + "readset": "references/task-benchmark-selection.md" + }, + { + "type": "git", + "url": "https://github.com/sgl-project/sglang", + "commit": "844d2b5decf6257aa192744cbdaaffb8ad797263", + "paths": [ + "docs/cookbook/base/benchmarks/diffusion_model_benchmark.mdx", + "docs/docs/developer_guide/bench_serving.mdx", + "docs/docs/developer_guide/evaluating_new_models.mdx", + "python/sglang/benchmark/serving.py", + "python/sglang/multimodal_gen/benchmarks/bench_offline_throughput.py", + "python/sglang/multimodal_gen/benchmarks/bench_serving.py", + "python/sglang/multimodal_gen/test/test_utils.py" + ], + "readset": "references/task-benchmark-selection.md" + }, + { + "type": "git", + "url": "https://github.com/vllm-project/vllm", + "commit": "a9bf45ed8a856a0b1839197152d0d26fabb6599c", + "paths": [ + "docs/benchmarking/cli.md", + "vllm/benchmarks/serve.py" + ], + "readset": "references/task-benchmark-selection.md" + }, + { + "type": "git", + "url": "https://github.com/ai-dynamo/aiperf", + "commit": "30420ce581d4fc9eeb34b4d47defe367ab8c3a11", + "paths": [ + "docs/accuracy/accuracy-benchmarking.md", + "docs/benchmark-datasets.md", + "docs/metrics-reference.md", + "docs/tutorials/agentic-code-generator.md", + "docs/tutorials/agentx-mvp.md", + "docs/tutorials/h-cua-perf.md", + "docs/tutorials/tracelab-trace.md", + "docs/tutorials/weka-trace.md", + "docs/tutorials/working-with-profile-exports.md", + "src/aiperf/dataset/loader/exgentic_v2.py", + "src/aiperf/metrics/types/inter_token_latency_metric.py" + ], + "readset": "references/task-benchmark-selection.md" + }, + { + "type": "git", + "url": "https://github.com/sgl-project/sglang-omni", + "commit": "f31cf9e9af749040ffa6dbdf064882c443a2fd37", + "paths": [ + "benchmarks/README.md", + "benchmarks/duplex/serving.py", + "benchmarks/metrics/performance.py", + "benchmarks/metrics/speaker_similarity.py", + "benchmarks/metrics/wer.py", + "docs/developer_reference/full_duplex_bench.md", + "docs/developer_reference/full_duplex_metrics.md" + ], + "readset": "references/task-benchmark-selection.md" + }, + { + "type": "git", + "url": "https://github.com/Lucky-Lance/OmniInteract", + "commit": "de304cef35fd9a50a5caadb5090c34cfbf0dd868", + "paths": [ + "eval/README.md", + "eval/evaluation/compute_unified_eval.py", + "eval/evaluation/summarize_unified_eval.py" + ], + "readset": "references/task-benchmark-selection.md" + }, + { + "type": "git", + "url": "https://github.com/OpenBMB/Omni-DuplexEval", + "commit": "962cec448ac37a377ffb963b476778777d11d346", + "paths": [ + "README.md" + ], + "readset": "references/task-benchmark-selection.md" + }, + { + "type": "git", + "url": "https://github.com/InferMatrixLab/InferRSI", + "commit": "b3ef95a6dd3abd85c19b5527fe46e80ccb3521d4", + "paths": [ + "docs/quickstart.md", + "src/inferrsi/harness/evaluation.py", + "src/inferrsi/model/protocols/metrics.py" + ], + "readset": "references/task-benchmark-selection.md" + }, + { + "type": "git", + "url": "https://github.com/NVIDIA/NeMo-Speech.cpp", + "commit": "20999cc921d2b14aa58708f61756a8b2e787ba20", + "paths": [ + "BENCHMARK.md", + "LICENSE", + "NOTICE", + "THIRD_PARTY_NOTICES.md", + "src/asr/encoder/cache_aware_encoder.cpp", + "src/asr/encoder/cache_aware_encoder.h", + "src/asr/encoder/fastconformer.h", + "src/asr/recognizer.cpp", + "src/asr/runner.cpp", + "src/asr/runner.h", + "src/asr/vad/vad_endpointer.cpp", + "src/asr/vad/vad_endpointer.h", + "tests/cpp/asr/CMakeLists.txt", + "tests/cpp/asr/test_cache_stream_geometry.cpp" + ], + "readset": "references/cache-aware-asr-stream-boundaries.md" + }, + { + "type": "git", + "url": "https://github.com/OpenBMB/JustGoal", + "commit": "f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d", + "paths": [ + "LICENSE", + "NOTICE", + "README.md", + "goal_full_shape/fable-5.1/dsa_decode_fwd/dsa_sparse_attention_h16_ckv512_kpe64_topk2048_ps64/dsa_decode.cu", + "goal_full_shape/fable-5.1/dsa_decode_fwd/dsa_sparse_attention_h16_ckv512_kpe64_topk2048_ps64/main.py", + "goal_single_shape/fable-5.1/dsa_decode_fwd/t8_topk2048/dsa_kernel.cu", + "goal_single_shape/fable-5.1/dsa_decode_fwd/t8_topk2048/dsa_kernel.py", + "kda05_reproduction_full_shape/opus-4.8/fa4_bwd/attention_bwd_causal_h16_d128/cute_fabwd_bshd_v5c_fusedzero/helpers/NOT_INCLUDED.md" + ], + "readset": "references/justgoal-source.json", + "license": "Apache-2.0 repository default; NOTICE exceptions and excluded proprietary helper retained", + "scope": "Selected kernel entry/workspace/stream contracts and distribution boundary; source-only, no execution" + }, + { + "type": "document", + "url": "https://github.com/OpenBMB/JustGoal/blob/f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d/paper.pdf", + "version": "Repository-paired PDF at f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d", + "sha256": "6bfa1a2728546def4c48e15cf55f31d92a543264787318f1522236c80a0ae8c6", + "readset": "references/justgoal-source.json", + "license": "UNKNOWN", + "scope": "Complete 21-page paper and appendices: controlled comparison, independent correctness, repair/coverage and timing evidence; author-reported, no PDF/figure redistribution" + }, + { + "type": "git", + "url": "https://github.com/li-ch/autotunebench", + "commit": "64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f", + "paths": [ + "ARTIFACT.md", + "README.md", + "CITATION.cff", + "LICENSE", + "config/protocol.yaml", + "pdb/ingest_validate.py", + "pdb/store.py", + "agents/loop.py", + "agents/adapter.py", + "orchestrator/cli.py", + "gates/g2_runtime.py", + "gates/g3_correctness.py", + "gates/g5_graduation.py", + "gates/schema.py", + "harness/kernel_bench.py", + "figs/plot_cand_ab_saturation.py", + "scripts/kernelbench_tabulate.py", + "experiments/kernelbench_l1/results/README.md", + "experiments/kernelbench_l1/results/TABLE.md", + "experiments/astra_parity/README.md", + "observability/iterations.jsonl", + "experiments/kernelbench_l1/kernelbench_tasks/UPSTREAM_LICENSE", + "experiments/kernelbench_l1/kernelbench_tasks/README.md" + ], + "readset": "references/autotunebench-trial-validity.md" + }, + { + "type": "paper", + "id": "arxiv:2609.18123", + "version": "v1", + "url": "https://arxiv.org/abs/2609.18123v1", + "sections": [ + "1-8", + "References", + "Figures 1-6", + "Tables 1-7" + ], + "readset": "references/autotunebench-trial-validity.md" + }, + { + "type": "git", + "url": "https://github.com/IntelLabs/Xe-Forge", + "commit": "6cd3ee98dfe32746b345a6abe521fa4cb9bdc187", + "paths": [ + "CONTRIBUTING.md", + "LICENSE", + "README.md", + "docs/CORRECTNESS.md", + "docs/KERNEL_REPO.md", + "pyproject.toml", + "src/xe_forge/claude/generator.py", + "src/xe_forge/claude/templates/CLAUDE.md.j2", + "src/xe_forge/cli.py", + "src/xe_forge/core/trial_manager.py", + "src/xe_forge/engines/claude_engine.py", + "src/xe_forge/external.py", + "src/xe_forge/skills/__init__.py", + "src/xe_forge/skills/benchmark.py", + "src/xe_forge/skills/trial.py", + "src/xe_forge/skills/upstream.py", + "tests/test_benchmark_gate.py", + "tests/test_trial_manager_ranking.py" + ], + "readset": "references/xeforge-source.json", + "license": "Apache-2.0 repository default; DCO and curated-example exclusions retained", + "scope": "Selected source-bound trial origin/comparison identity, host outcomes, measured-only selection and baseline-oracle contracts; source-only, no execution" + }, + { + "type": "paper", + "id": "arxiv:2605.26118", + "version": "v1", + "url": "https://arxiv.org/html/2605.26118v1", + "sections": [ + "Abstract", + "I-VIII", + "References", + "Appendix A", + "Tables 1-3", + "Algorithm 1", + "Listing 1", + "Figure captions 1-15" + ], + "readset": "references/xeforge-source.json", + "license": "CC-BY-4.0", + "scope": "Complete HTML text/appendix and captions; historical author-reported context, unused graphic values remain unverified" + }, + { + "type": "git", + "url": "https://github.com/Neroued/ninfer", + "commit": "1882c6adbc98043fcf89f3589aa5addb3742777a", + "paths": [ + "LICENSE", + "README.md", + "docs/maintainer/engine-architecture.md", + "docs/maintainer/speculative-decoding.md", + "docs/maintainer/context-management.md", + "docs/performance/methodology.md", + "docs/performance/qwen3.8-27b.md", + "include/ninfer/ops/gdn_replay.h", + "src/core/gdn_replay_records.h", + "src/core/gdn_replay_records.cpp", + "src/core/linear_attention_state.h", + "src/core/linear_attention_state.cpp", + "src/models/qwen3_5/state/state_image.h", + "src/models/qwen3_5/state/state_image.cpp", + "src/models/qwen3_5/execution/gdn.cpp", + "src/models/qwen3_5/execution/text.cpp", + "src/models/qwen3_5/program/decode.cpp", + "src/models/qwen3_5/program/commit.cpp", + "src/models/qwen3_5/program/mtp.cpp", + "src/models/qwen3_5/session/commit.cpp", + "src/models/qwen3_5/session/binding.cpp", + "src/models/qwen3_5/session/checkpoints.cpp", + "src/runtime/engine/engine_execution.inl", + "src/runtime/executor/contract.h", + "src/runtime/executor/local_executor.h", + "src/ops/linear_attention/gated_delta_net/replay/fold.cpp", + "tests/ops/linear_attention/gated_delta_net/test_fold.cpp" + ], + "readset": "references/ninfer-source.json", + "license": "Apache-2.0", + "scope": "Selected Qwen3.5 GDN/MTP state-restoration, accepted-prefix and completion evidence contracts; historical results retain their separate reported revision. Source-only, no native execution." + }, + { + "type": "git", + "url": "https://github.com/yuekaizhang/faster-cosyvoice", + "commit": "f1049c59196c2c1a890079abc56056b43f27cd1d", + "paths": [ + "README.md", + "README_zh.md", + "docs/benchmark.md", + "docs/benchmark_zh.md", + "docs/usage.md", + "docs/usage_zh.md", + "examples/offline_inference.py", + "examples/stream_client.py", + "faster_cosyvoice/config/settings.py", + "faster_cosyvoice/llm/engine.py", + "faster_cosyvoice/server/app.py", + "faster_cosyvoice/server/arguments.py", + "faster_cosyvoice/server/audio_encode.py", + "faster_cosyvoice/server/leading_silence.py", + "faster_cosyvoice/server/openai_speech.py", + "faster_cosyvoice/server/protocol.py", + "faster_cosyvoice/streaming/chunker.py", + "faster_cosyvoice/streaming/session.py", + "faster_cosyvoice/streaming/token2wav_batcher.py", + "faster_cosyvoice/streaming/token2wav_scheduler.py", + "faster_cosyvoice/token2wav/builders.py", + "faster_cosyvoice/token2wav/cosyvoice/flow/DiT/modules.py", + "faster_cosyvoice/token2wav/cosyvoice/flow/flow.py", + "faster_cosyvoice/token2wav/cosyvoice/hifigan/generator.py", + "faster_cosyvoice/token2wav/flashinfer_dit.py", + "faster_cosyvoice/token2wav/flashinfer_flow.py", + "faster_cosyvoice/token2wav/frontend.py", + "faster_cosyvoice/token2wav/matcha_audio.py", + "faster_cosyvoice/token2wav/token2wav.py", + "pyproject.toml", + "tests/test_session.py", + "tests/test_stream_client.py", + "tests/test_token2wav_batcher.py" + ], + "readset": "references/faster-cosyvoice-source.json", + "license": "UNKNOWN overall; selected file-level Apache-2.0 headers and unresolved inherited attribution retained", + "scope": "Selected PCM/playout, routed-credit, chunk-dependency and timing-denominator source audit; original analysis and immutable links only, no code/media copying or native/model/quality/service execution" + }, + { + "type": "git", + "url": "https://github.com/SEU-PAISys/Embodied.cpp", + "commit": "156bed151826780e2ce0132f8d0e3540889ae8cd", + "paths": [ + "adapter/typed_io.py", + "eval/client/libero_profile.py", + "eval/client/vla_cpp_client.py", + "eval/conf/libero_smolvla_object_q8_eval.yaml", + "eval/README.md", + "eval/smolvla_q8_fixed_boundary_20261002.json", + "eval/smolvla_q8_object_followup_20260930.md", + "eval/SMOLVLA_VALIDATION.md", + "eval/THREE_MODEL_VALIDATION.md", + "eval/VLA_BENCHMARK_STANDARD.md", + "LICENSE", + "README.md", + "scripts/aggregate_smolvla_object_eval.py", + "scripts/bench_vla_boundary.py", + "scripts/profile_smolvla_python_baseline.py", + "serving/vla-server.cpp", + "serving/vla.proto" + ], + "readset": "references/embodied-cpp-source.json", + "license": "Apache-2.0", + "scope": "Selected CPU observation-to-action benchmark, client/profiler/aggregation and author artifact contracts; source-derived methodology, not native execution or quality/performance acceptance" + }, + { + "type": "paper", + "id": "arxiv:2607.02501", + "version": "v3", + "url": "https://arxiv.org/abs/2607.02501v3", + "sections": [ + "4 (VLA discussion and Table 3)" + ], + "readset": "references/embodied-cpp-source.json", + "license": "CC-BY-4.0", + "scope": "Independently normalized VLA metric semantics; predates the later SmolVLA experiment" + }, + { + "type": "document", + "url": "https://docs.nvidia.com/nsight-compute/2025.3/ProfilingGuide/index.html", + "version": "Nsight Compute 2025.3", + "sha256": "c9f3b812ce50a46a751f2c92eff628177554f4f6678bb3381468b1dac91b607e", + "readset": "references/gpu-measurement-validity.md", + "scope": "Inspected Replay, Overhead, Hardware Model, Metrics Structure, Range and Precision, Reproducibility, Clock Control and Cache Control; source contracts only, no profiler execution." + }, + { + "type": "document", + "url": "https://docs.nvidia.com/nsight-compute/2025.3/NsightComputeCli/index.html", + "version": "Nsight Compute 2025.3", + "sha256": "c5129a8afe2439c04906f19312393972ae123062d2ee0894b3ba2b11af43471e", + "readset": "references/gpu-measurement-validity.md", + "scope": "Inspected Metric Collection and Profiling option defaults for clock/cache control and metric queries; actual target support must be checked separately." + }, + { + "type": "document", + "url": "https://docs.nvidia.com/nsight-systems/2025.3/UserGuide/index.html", + "version": "Nsight Systems 2025.3", + "sha256": "1d42e35005fa9a4cb21b2c65d0018e4b065c3756dbba308dc0c41f79746e751f", + "readset": "references/gpu-measurement-validity.md", + "scope": "Inspected CLI Profile options, CUDA graph/node tracing and CUDA API/kernel/launch/MemOps statistics reports; absence of NCU replay controls is an author inference, not a launcher guarantee." + }, + { + "type": "document", + "url": "https://docs.nvidia.com/cuda/archive/12.9.1/cuda-c-best-practices-guide/index.html", + "version": "CUDA Toolkit 12.9.1 archive", + "sha256": "0fcaf0c361d00de56a9870ba96db871a0b880904121e1dfb215ab9882470120b", + "readset": "references/gpu-measurement-validity.md", + "scope": "Inspected Assess/Optimize, Application Profiling, Amdahl's Law, Timing and Instruction Throughput; normalization/adoption procedure is an unverified author synthesis." + }, + { + "type": "document", + "url": "https://docs.nvidia.com/cuda/archive/12.9.1/cuda-c-programming-guide/index.html", + "version": "CUDA Toolkit 12.9.1 archive", + "sha256": "06499a0c6bfc6925cf54c76f6ba86a4986a081c44c602e10d4ef3b4f33a33865", + "readset": "references/gpu-measurement-validity.md", + "scope": "Inspected Concurrent Kernel Execution, CUDA Graphs (structure, definition/instantiation/execution) and Events; no constant launch/replay overhead or device test claimed." + }, + { + "type": "document", + "url": "https://docs.nvidia.com/cuda/archive/10.2/nsight-compute/docs/NsightComputeCli/index.html", + "version": "CUDA Toolkit 10.2 archive, Nsight Compute CLI", + "readset": "references/gpu-measurement-validity.md", + "scope": "Inspected nvprof Metric Comparison for dram__bytes_read.sum, dram__bytes_write.sum and smsp__inst_executed.sum. Legacy names do not guarantee current chip/tool availability; no raw-byte digest retained for this browser read." + }, + { + "type": "document", + "url": "https://docs.nvidia.com/deploy/nvidia-smi/index.html", + "version": "Unversioned NVIDIA System Management Interface page, retrieved 2026-11-15; content pinned by SHA-256", + "sha256": "94d1737cdff11b5ef89b121e97365706feeb8ccc18978fecec571bdbbe6ec9ed", + "readset": "references/gpu-measurement-validity.md", + "scope": "Inspected Performance State, Clocks Event Reasons, clocks, temperature and power reporting. Public page may change; digest identifies inspected HTML, which is not redistributed." + }, + { + "type": "git", + "url": "https://github.com/triton-lang/triton", + "commit": "9641643da6c52000c807b5eeed05edaec4402a67", + "paths": [ + "python/triton/testing.py" + ], + "readset": "references/gpu-measurement-validity.md", + "license": "MIT", + "scope": "v3.2.0 do_bench: write-based cache.zero_ flush, preliminary estimate and per-sample event boundaries; tag resolved with git ls-remote, source inspected in public browser. No flush comparison run." + }, + { + "type": "document", + "url": "https://docs.pytorch.org/docs/2.7/cpp_extension.html", + "version": "PyTorch 2.7", + "readset": "references/gpu-measurement-validity.md", + "scope": "Inspected load/load_inline and the extension name, build-directory and pybind11 module contract. Distinct names for separately loaded variants are author advice, not a documented universal duplicate-name failure; no raw-byte digest retained for this browser read." + }, + { + "type": "paper", + "url": "https://arxiv.org/html/2509.14279v1", + "id": "arxiv:2509.14279", + "version": "v1", + "sections": [ + "3", + "Appendix A", + "Appendix C.3" + ], + "readset": "references/gpu-measurement-validity.md", + "license": "CC-BY-4.0", + "scope": "Lange et al., Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization. Inspected benchmark loopholes, compromised-task filtering/examples and eager/compiled baseline comparisons; reports remain author-reported, not locally reproduced." + } + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Selecting task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents, cache-aware ASR, TTS, full-duplex audio/video, image and video inference, and designing goodput-based multi-regime comparisons that fit InferRSI's evidence trail; interpreting treatment delivery, arm validity and stated-versus-observed protocol enforcement in supplied tuning reports; interpreting source-bound kernel trial origin, host evaluator outcomes, unresolved performance and measured-only versus manual candidate selection; designing and interpreting the selected NInfer hybrid speculative state-restoration and completion validation contract; designing or auditing VLA deployment comparisons with explicit observation-to-action boundaries, generated/replayed horizons, per-task quality and source/precision identity; assessing GPU timing validity and profiler-based attribution of kernel/inference gains", + "prerequisites": "The primary objective, timing boundary, quality gates, sample/failure accounting, required regimes, repetitions, seeds and decision rule are frozen before optimization; baseline and candidate share hardware, software stack, input distribution, benchmark configuration, warmup and measurement protocol, and on a shared machine also one time window and dedicated devices; cache-aware ASR also requires the selected model/config and first/steady geometry, decoded clock and reset/EOU/EOF boundaries to be established; VLA comparisons also freeze generated/replayed horizons, initial-state/noise protocol and intended precision or quantization treatment", + "input_contract": "The task family and deployment use case; candidate quality evaluators and reference parity; the declared SLO and the caller's policy on which regimes must improve or must not regress; for an audit, a supplied fixed-workload report with request-to-trial grouping, exclusion and failure counts and the declared aggregation rule; for a treatment audit, the frozen delivery/arm-validity rules, run/candidate identities, feedback and consumer-input evidence, and per-arm raw records; for a kernel trial audit, complete saved candidate bytes and IDs, baseline/spec/reference/variant identities, result-origin fields, raw host outcome and return code, named gate and persisted ratio, actual oracle and the selecting/delivering caller; for the NInfer slice, a complete checkpoint/frontier identity, physical verify-block/state oracle, final adopted extents and correlated execution/termination records; for VLA audits, the declared CPU input/output boundary, source/fixture/checkpoint/processor identities, per-task trial scope and available raw records or explicit artifact gaps", + "produced_artifact": "A frozen benchmark design with evaluators, reference parity, primary objective, timing boundary, quality gates, required regimes, repetition and seed rules and an explicit decision config, plus the evidence-tree layout a PASS claim must produce; for a supplied report, the computed threshold result and missing evidence. Candidate knowledge, not qualification of any model or backend; for a treatment audit, a contract-bound eligibility and interpretation report separating treatment exposure, observed checks, comparator identity, anomaly disposition and source conflicts; for a kernel trial audit, a source-bound interpretation separating artifact/cohort identity, origin, oracle, performance resolution and selected-versus-delivered eligibility, without promoting kernel parity into a gain or whole-model acceptance; for the NInfer slice, a source-bound validation plan or report interpretation that separates state-oracle evidence, final adoption, retirement and workload denominators; for VLA reports, a timing/denominator and precision reconciliation with per-task quality/completion accounting and source-artifact limits; for GPU attribution, a source-grounded measurement plan or interpretation binding environment, paired timing, work counts, predicted signature and E2E decomposition with explicit assumptions and unresolved evidence", + "failure_modes": [ + "Raw throughput is optimized in place of goodput, so requests served outside the SLO count toward a win", + "One concurrency level or one undisclosed aggregate score stands in for all required regimes", + "Recorded agent task scores or replay completion are read as current task success, or a VBench prompt workload as computed VBench quality", + "An upstream better/worse/inconclusive label is used as InferRSI's PASS/FAIL/INCONCLUSIVE verdict", + "Runs on the same hardware class, in different time windows or on shared devices are compared as if pinned", + "A quality pass is taken to certify that the timed work actually occurred", + "Best-seen results are reported as delivered without restarting and validating the final code/config artifact", + "A cache-aware ASR EOU is treated as EOF or full reset, nominal chunk duration as the decoded mel frontier, or shape-only geometry PASS as a native model-state test", + "An enabled treatment or stored feedback summary is taken as proof of delivery to the next eligible consumer", + "A failed arm-validity condition is overridden by a favorable best timing or a saturation/null interpretation", + "Complete gate keys or zero recorded failures are read as proof that every declared protocol check ran", + "A tool-origin measured marker or completed status is treated as a resolved performance gain", + "A displayed/manual best, baseline copy or edited candidate is reported as the eligible measured delivery", + "Ratio omission alone is assumed to clear an earlier ratio, or baseline agreement is treated as independent semantic correctness", + "Zero accepted drafts is treated as zero adopted input, or a matching token prefix is used to splice unrelated State and KV checkpoints", + "ResultReady or a commit receipt is treated as DeviceRetired, or selected full-batch intervals are compared with complete-cohort rates under one unlabeled denominator", + "Generated-action amortization is read as action-response latency or actual control frequency", + "Matching structural pair fields are taken to certify matched effective compute precision or quality equivalence", + "An aggregate VLA success rate conceals task-specific losses or missing/skipped episodes", + "A public current script, recorded experiment revision or binary hash is treated as the exact measured dirty source patch", + "Whole-device fallback memory is relabelled as process-attributed memory or compared with simulator-inclusive/allocator quantities", + "NCU replay durations are substituted for normal E2E evidence, or an untouched region is treated as a universal clock without checking common drift and critical-path assumptions", + "Unexplained gains, changed baseline modes or same-named JIT loads are adopted without binding actual work, module identity and predicted profiler signatures" + ], + "refusal_point": "Records missing coverage and high-variance results as INCONCLUSIVE rather than a win; a run without a frozen RunSpec, or whose validator changed after the plan was frozen, is INCONCLUSIVE regardless of metric values; sets no universal numeric gate or noise threshold; withholds causal treatment benefit/null conclusions when required delivery or arm eligibility is unverified, and preserves conflicts between stated protocol, exported code and source reports; withholds a kernel gain for unresolved performance or an unbound comparison and identifies missing independent semantic evidence when only baseline agreement is available; withholds native state-correctness and cancellation/reuse conclusions when the required same-block oracle, checkpoint identity or matching receipt is missing; withholds VLA quality-preservation, runtime-only attribution, control-rate or current-build conclusions when their required precision, per-task, timestamp or measured-source evidence is missing" + }, + "recommendations": [ + "Freeze one primary objective, timing boundary, quality gates and sample/failure accounting before optimization.", + "Choose the required regimes from the deployment use case, report each separately, and record which must improve or must not regress in a versioned decision config.", + "Use the replay trace that matches the actual deployment pattern.", + "State the concurrency, trace, model, SKU and microbenchmark scope boundaries in any report that cites benchmark results.", + "Trace treatment to its eligible consumer and apply the frozen arm-validity precedence before attributing benefit, saturation or a null.", + "Bind the complete saved candidate and comparator cohort; keep result origin, actual oracle, named gate/ratio and delivered eligibility separate.", + "For hybrid speculative comparisons, freeze the coupled restore point, same-physical-block oracle, zero-draft/zero-adoption controls and complete-cohort versus steady-phase denominators before interpreting a result.", + "Separate complete VLA request cost, generated/replayed action denominators and actual control cadence; retain per-task quality and measured-source/precision limits.", + "For GPU comparisons, retain timestamped clocks/temperature/power, same-run region shares and work counts; test the predicted mechanism signature and paired E2E change before adopting a gain." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", + "Task benchmark knowledge is distilled from issue #60 with selected pinned-source rechecks; no all-task runtime qualification or optimization benefit is established.", + "Benchmark feasibility and package-consumption checks do not promote this candidate or replace task quality, repeated A/B and service qualification.", + "The pinned NeMo-Speech.cpp ASR reference covers selected source/test definitions and author-reported benchmark boundaries; native model/device, decoder internals, quality, long-audio/load and service behavior remain not_run.", + "JustGoal contributes source-only controlled-evaluation and kernel-acceptance guidance; its repaired/retested timings, released-pass counts and protocol differences are author-reported, not locally reproduced or a complete campaign runner.", + "The AutoTuneBench reference is source-only: original operational checkpoint/prompt receipts remain unavailable, source contradictions remain explicit, and no tuning, device/performance or agent-efficacy validation was run.", + "The Xe-Forge reference is source-only and limited to inspected host/manual/Claude trial paths; named-verdict update predicates and caller defaults remain explicit. Native tests, backend/model/quality/performance/service behavior and agent efficacy were not evaluated, and source/package checks do not promote whole-model support.", + "The NInfer reference inspects selected Qwen3.5 GDN/MTP source contracts and unexecuted test definitions; its historical author measurements use a different engine revision. Native state/model/device/quality/performance and efficacy remain unvalidated.", + "The Faster CosyVoice reference is source-only: server route credit and client simulated playback remain distinct, reported costs/quality retain their boundaries, overall source rights are UNKNOWN, and native/model/device/audio quality/service/performance are not_run with efficacy not_evaluated.", + "The Embodied.cpp reference is source-derived: its public latency arrays allow arithmetic audit, but raw episode recount, exact measured script/dirty patch, effective dispatch and current model/device/service or efficacy qualification remain unresolved.", + "The GPU measurement-validity reference is source-only. The within-run normalization and explanation-first adoption procedure is an unverified author proposal; four task-supplied numerical observations have no supplied raw evidence. GPU/profiler, model/quality, L1–L4 and efficacy validation were not run; legacy metric mappings require target-specific queries." + ], + "provenance_readsets": [ + "references/task-benchmark-selection.md", + "references/benchmark-commands.md", + "references/cache-aware-asr-stream-boundaries.md", + "references/justgoal-evaluation-contracts.md", + "references/justgoal-source.json", + "references/autotunebench-trial-validity.md", + "references/autotunebench-source.json", + "references/xeforge-trial-evidence.md", + "references/xeforge-source.json", + "references/ninfer-state-and-completion-contracts.md", + "references/ninfer-source.json", + "references/dependency-aware-tts-playout.md", + "references/faster-cosyvoice-source.json", + "references/embodied-cpp-boundary-and-quality.md", + "references/embodied-cpp-source.json", + "references/gpu-measurement-validity.md" + ], + "references": [ + "../../../casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md", + "../../../casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.json", + "../../../systems/engines/deepinfer/budget-speculative-verification/SKILL.md", + "../../../systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md" + ], + "consumption_mode": "packaged", + "searchspace_not_applicable": "SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism." + } +--- + +# Inference Benchmark Design + +Status: candidate. Source-grounded from vllm-evolve @ 43cc82c and +InferRSI evidence framework @ 4c8e1285. + +## Select the task and its evaluation contract + +Read [task-specific benchmark selection](references/task-benchmark-selection.md) +when choosing quality evaluators, reference parity and a performance workload. +It covers text, agent replay, TTS, three full-duplex task families, images and +videos, with pinned provenance from InferRSI issue #60. Use the +[command templates](references/benchmark-commands.md) for entry points and a +bounded HTTP smoke. These references are candidate knowledge, not qualification +of every model or backend. + +For cache-aware speech recognition, read the +[ASR stream-boundary procedure](references/cache-aware-asr-stream-boundaries.md) +before selecting tests or comparing chunk latency. It separates startup/steady geometry, decoded +progress, stream-slot reuse, reset, EOU and EOF; its source tests and +author benchmark reports are not local model or serving qualification. + +For streaming TTS measurements tied to playback deadlines or dependent chunks, +read [dependency-aware TTS playout](references/dependency-aware-tts-playout.md). +It traces server PCM credit, client playback events, independent batching work +and partial pipeline costs at one pinned source; it adds no hardware coverage, +runtime or locally reproduced quality/performance result. + +For VLA and robot deployment comparisons, read +[observation-to-action timing and closed-loop quality](references/embodied-cpp-boundary-and-quality.md). +Define the complete CPU-action boundary, distinguish generated and replayed +horizons from the control clock, and reconcile per-task quality, precision and +the measured source revision. Freeze latency/jitter and quality objectives for +the actual control use case; per-generated-action cost is amortization. + +For treatment-enabled tuning comparisons or claimed protocol integrity, read +[treatment delivery and trial validity](references/autotunebench-trial-validity.md). +Trace feedback to the eligible consumer input and apply the frozen arm-validity +precedence before interpreting benefit, saturation or a null. + +For GPU timing comparisons, read +[measurement validity and profiler attribution](references/gpu-measurement-validity.md). +Separate normal E2E timing from profiler replay, test within-run attribution +assumptions and retain work counts plus the predicted mechanism signature. +This is candidate guidance; its task-supplied observations remain unverified. + +For hybrid speculative state-restoration or completion evidence, read the +[NInfer validation contract](references/ninfer-state-and-completion-contracts.md). +Freeze a coupled checkpoint and same-physical-block state oracle; distinguish +zero accepted drafts from zero adopted inputs, logical commit from device +retirement, and selected steady intervals from complete-cohort denominators. +Its source contracts and historical author reports are not native parity results. + +Freeze one primary objective, timing boundary, quality gates and sample/failure +accounting before optimization. Recorded agent task scores and replay completion +do not establish current task success; a VBench prompt workload does not compute +VBench quality. Missing coverage stays INCONCLUSIVE. Numeric gates and required +workload regimes are campaign choices, not repository-wide defaults. + +## Core principle: goodput over throughput + +Raw throughput (tokens/second, frames/second) is misleading because a system +can maximize throughput by accepting requests it then fails to serve within +the user's latency expectation. The meaningful metric is goodput: the rate of +requests that complete AND meet the declared SLO. + +``` +goodput = (requests completed within SLO) / time +``` + +A candidate that improves raw throughput by trading latency away cannot "win" +under a goodput metric if it causes SLO violations. +Source: vllm-evolve README @ 43cc82c. + +## Workload regimes to select for the campaign + +A single benchmark run at one concurrency level does not characterize serving +performance. These source-derived regimes answer different questions; choose +the required set from the deployment use case and freeze it before comparison: + +| Regime | Load level | What it answers | +|--------|-----------|----------------| +| Throughput | Saturating load, with observed bottleneck and queue state | Max sustainable goodput at peak load | +| Latency | Low load | Responsiveness for latency-sensitive workloads | +| Replay | Real or representative trace | Behavior under actual production burstiness | + +An optimization may improve one regime while degrading another. Report all +required regimes separately; do not aggregate into a single score without disclosing +how the aggregation was done. + +For replay, standard trace sources (from vllm-evolve): BurstGPT, Azure, +ShareGPT, Mooncake. Use the trace that matches your actual deployment pattern. + +## Verdict semantics + +The upstream vllm-evolve comparison labels are: + +| Verdict | Condition | +|---------|----------| +| better | Improvement in all required regimes, no regression in any | +| worse | Regression in any required regime | +| inconclusive | Too noisy to distinguish (see below) | + +These labels are not InferRSI's verdict enum. InferRSI uses PASS / FAIL / +INCONCLUSIVE and the campaign's frozen gates. `high_variance_inconclusive` +is a useful diagnostic reason: it means the +measurement signal is not strong enough to support a claim in either direction. +Do not report an inconclusive result as a win. Record it as INCONCLUSIVE and +note the variance level. +Source: vllm-evolve README @ 43cc82c. + +## Statistical requirements + +Freeze repetition count, workload seeds and the decision rule in the campaign. +The upstream guidance recommends multiple seeds and median goodput with IQR or +range. State whether a repetition changes the workload seed or repeats identical +inputs. If using `max - min > threshold × median` as a noise gate, choose the +threshold in the acceptance plan; this skill defines no universal numeric bar. +InferRSI's built-in median/range summaries are descriptive, not a significance +test or automatic enforcement of that noise gate. + +The decision of which regimes must improve and which must not regress is the +caller's policy, not the benchmark's. Encode it in a decision config file +(`decision.yaml` in vllm-evolve) so it is explicit, versioned, and auditable. + +## Audit a supplied fixed-workload report + +Preserve the request-to-trial grouping and every observation covered by the +contract. Late requests and outliers remain included unless the frozen protocol +explicitly excludes them; retain exclusion and failure counts. Apply aggregation +in the declared order: a median of per-trial request means uses every included +request to calculate each trial mean, then takes the median across trials. A +pooled request median is a different statistic, especially with unequal trial +sizes. If the report omits an aggregation rule that changes the verdict, expose +that ambiguity rather than choosing one after seeing the result. + +For a lower-is-better metric with a positive baseline, calculate percentage +reduction as `100 * (baseline - candidate) / baseline` and compare it with the +declared threshold before rounding. Check workload equivalence, output quality +and completion evidence separately from this arithmetic. Report the computed +threshold result and any missing evidence under the frozen contract's verdict +semantics; an attractive aggregate alone does not establish an accepted gain. + +## What must be pinned before comparing two runs + +Every valid A/B comparison requires these to be identical: +- Hardware (exact device SKU, driver version, firmware). +- Software stack (framework version, model weights revision). +- Input distribution (same trace or same generation config and seed). +- Benchmark configuration (concurrency levels, SLO thresholds, timeout). +- Warmup (same warmup protocol, excluded from measurement). +- Measurement protocol (same window duration, same number of seeds). + +If any of these differ between the baseline and candidate run, the comparison +is invalid. "Same hardware class" is not the same as "same device." + +## Shared-machine measurement discipline + +On shared infrastructure (multiple users, shared GPU/NPU allocation): + +- Cross-window comparison is invalid. Run baseline and candidate within the + same time window (interleaved or back-to-back in the same allocation). +- Record neighbor load state before each run. If it changes between runs, + report the comparison as potentially confounded. +- Dedicate devices: do not share GPU devices between the baseline server and + the candidate server in the same comparison run. + +Source: MindIE-SD perf-gate skill @ 8481d39b (window discipline for NPU). + +## What a benchmark cannot prove + +- A benchmark win at one concurrency level does not imply a win at other + concurrency levels. +- A benchmark win with synthetic traces does not imply a win under real + production traffic. +- A benchmark win on one model does not transfer to other models without + re-running. +- A benchmark win on one hardware SKU does not transfer to other SKUs. +- Microbenchmark improvement does not imply E2E benchmark improvement. + +State these scope boundaries explicitly in any report that cites benchmark +results. + +## Search integrity and final-state verification + +Freeze workload hashes, output/work semantics, permitted approximations, quality +checks, timing events and developer-versus-held-out splits. Verify that measured +work actually occurs; a quality pass does not certify a truthful timing path. +Keep fast probes separate from full evaluation, and preserve invalid, failed and +inconclusive trials. A simulator shortlist requires calibrated load/capacity +coverage and does not replace measurements. + +Compare search methods at matched total budgets. Separate search/API cost from +steady-state serving cost; independent per-engine searches are not one equal-budget +engine-selection run. Keep different SLOs, baselines, resources and aggregation +rules explicit. Before reporting, restart the final code/config artifact and +validate that exact state; report best-seen results separately from delivered ones. + +[Reviewed experiment discipline](../performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md) +and [source audit](../performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md) +retain author-reported evidence and counterexamples. No local experiment is +established by integrating these notes. + +For controlled kernel-agent comparisons, use the +[JustGoal source reference](references/justgoal-evaluation-contracts.md). +Keep model/budget/task scope, independent acceptance, released versus repaired +artifacts and workspace/stream/cache contracts explicit. Its results remain +author-reported; it adds no runner or device/performance qualification. + +For source-bound kernel trial records, read +[trial identity, evaluator evidence and candidate selection](references/xeforge-trial-evidence.md). +Keep tool origin, actual oracle, unresolved performance, selection and delivered +eligibility distinct; a retained parity candidate is not a measured gain. + +## Evidence trail requirements (InferRSI compatibility) + +A benchmark campaign that claims PASS under InferRSI's evidence framework +must produce: +``` +evidence// + plan/ + hypothesis.md # what is being tested and why + runspec.json # frozen RunSpec (repo × revision × backend × device × model × scenario) + slo.yaml # declared SLO and threshold + decision.yaml # which regimes must improve / must not regress + results/ + regime-throughput/ # raw metrics, logs, per-seed records + regime-latency/ + regime-replay/ + verdict.json # PASS / FAIL / INCONCLUSIVE with reasons + patch/ + candidate.patch # the exact change being evaluated + validator/ + validate.py # frozen validator script +``` + +A run without a frozen RunSpec is INCONCLUSIVE regardless of the metric values. +A run whose validator was modified after the plan was frozen is INCONCLUSIVE. +Source: InferRSI AGENTS.md and rsi-evidence skill @ 4c8e1285. + +## Scoped internal experience + +Read the single authoritative [admission/drain occupancy case](../../../casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md) when interpreting finite service cohorts. It separates recorded observation, action and interpretation; phase-selected occupancy is not end-to-end gain. The original frozen evidence remains linked, not copied. This shared reference is included in the packaged skill library; standalone folder export is not supported. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/autotunebench-source.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/autotunebench-source.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/autotunebench-trial-validity.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/autotunebench-trial-validity.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/benchmark-commands.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/benchmark-commands.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/dependency-aware-tts-playout.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/dependency-aware-tts-playout.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/dependency-aware-tts-playout.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/dependency-aware-tts-playout.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/embodied-cpp-source.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/embodied-cpp-source.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/faster-cosyvoice-source.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/faster-cosyvoice-source.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/faster-cosyvoice-source.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/faster-cosyvoice-source.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/gpu-measurement-validity.md similarity index 99% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/gpu-measurement-validity.md index 8c681316c..02a08fbb1 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/gpu-measurement-validity.md @@ -6,7 +6,7 @@ adoption procedure is an author proposal derived from those sources, unverified as a complete method. The final section contains task-supplied author reports with unavailable evidence, not locally reproduced cases or efficacy evidence. Source versions, inspected sections and available document hashes are retained -in the owning `metadata.json`; the unversioned SMI page is hash-pinned there. +in the owning [SKILL.md frontmatter](../SKILL.md); the unversioned SMI page is hash-pinned there. ## Freeze the measurement and environment diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-evaluation-contracts.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/justgoal-evaluation-contracts.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-evaluation-contracts.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/justgoal-evaluation-contracts.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-source.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/justgoal-source.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/justgoal-source.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/justgoal-source.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-source.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-source.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-source.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-source.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md similarity index 97% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md index c575f3440..ddcabc4d5 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/ninfer-state-and-completion-contracts.md @@ -81,8 +81,8 @@ All 820 formal requests were reported free of request/CUDA/allocation errors, bu ## Route to the existing responsibility -- For verification-row allocation from acceptance and a cost model, use [budget-speculative-verification](../../../repositories/deepinfer/budget-speculative-verification/SKILL.md). It budgets verification; it does not supply this GDN restore oracle. -- For Ouro LAST_EXITED depth-plane KV propagation and contiguous written-history checks, use [audit-recurrent-kv-finalization](../../../repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md). Its scoped synchronous path excludes speculation/preemption; it is not a replacement for this source's StateImage/frontier contract. +- For verification-row allocation from acceptance and a cost model, use [budget-speculative-verification](../../../../systems/engines/deepinfer/budget-speculative-verification/SKILL.md). It budgets verification; it does not supply this GDN restore oracle. +- For Ouro LAST_EXITED depth-plane KV propagation and contiguous written-history checks, use [audit-recurrent-kv-finalization](../../../../systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md). Its scoped synchronous path excludes speculation/preemption; it is not a replacement for this source's StateImage/frontier contract. - Use this reference to produce a frozen validation plan or source-bound report interpretation. Model/device parity, quality, repeated performance and service acceptance require separately authorized native evidence and frozen campaign gates. ## Rights and evidence boundary diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/task-benchmark-selection.md similarity index 99% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/task-benchmark-selection.md index 05f764ce7..0125ae405 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/task-benchmark-selection.md @@ -229,7 +229,7 @@ SGLang-Omni FDB guide and SGLang video backend. Other upstream details retain the report's attribution rather than claiming a new full audit. The current InferRSI contract was also checked against its source. -The typed source inventory in `metadata.json` identifies the retained readset. +The typed source inventory in the owner's [SKILL.md frontmatter](../SKILL.md) identifies the retained readset. The external VBench link is a locator with an unpinned revision; pin it before an experiment. This reference adds no hardware qualification or efficacy claim. Any accompanying local smoke validates only its recorded path, not all nine diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-source.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-source.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-source.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-source.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-trial-evidence.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-trial-evidence.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/xeforge-trial-evidence.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design/references/xeforge-trial-evidence.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md similarity index 68% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md index 29b5fa549..7f4eb614b 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/SKILL.md @@ -3,6 +3,170 @@ name: performance-ceiling-analysis description: 'Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical, empirical component, end-to-end) and a prioritized measurement plan. Use to formulate acceptance targets when the user has not supplied them, assess a requested speedup, or identify the binding bottleneck. ' +metadata: + inferrsi: | + { + "schema_version": 1, + "status": "candidate", + "sources": [ + { + "url": "https://github.com/NVIDIA/TensorRT-LLM", + "commit": "f33e0e5fd55b", + "paths": [ + ".claude/skills/perf-analysis/SKILL.md", + ".claude/skills/perf-nsight-compute-analysis/references/roofline-analysis.md", + ".claude/skills/perf-workload-profiling/SKILL.md", + ".claude/skills/perf-optimization/SKILL.md", + ".claude/skills/perf-optimization-casebook/SKILL.md", + ".claude/skills/exec-env-check/SKILL.md", + ".claude/skills/exec-remote-slurm/SKILL.md", + ".claude/agents/perf-profiling-specialist.md" + ], + "type": "git", + "revision_precision": "prefix" + }, + { + "url": "https://github.com/hahnyuan/LLM-Viewer", + "commit": "1893e4b5705d", + "paths": [ + "roofline_model.py", + "model_analyzer.py", + "hardwares/hardware_params.py", + "configs/DiT.py" + ], + "type": "git", + "revision_precision": "prefix" + }, + { + "url": "https://github.com/ebugger/Empirical-Roofline-Toolkit", + "commit": "0771fa9621d0", + "paths": [ + "Empirical_Roofline_Tool-1.1.0/README.md", + "Empirical_Roofline_Tool-1.1.0/Drivers/driver1.c", + "Empirical_Roofline_Tool-1.1.0/Kernels/kernel1.c", + "Empirical_Roofline_Tool-1.1.0/Scripts/roofline.py" + ], + "type": "git", + "revision_precision": "prefix" + }, + { + "type": "paper", + "id": "usenix:fast25-qin", + "version": "FAST '25 proceedings, February 25-27, 2025, pp. 155-170", + "url": "https://www.usenix.org/system/files/fast25-qin.pdf", + "content_sha256": "543e56c541523c5e807d2231370f96749b2294bc901266397a7265b18ae2c130", + "sections": [ + "2.2, Table 1, Equations 1-2", + "3.1 and 5.3 (storage-scope limits only)" + ], + "readset": "references/mooncake-prefix-reuse-feasibility.md", + "license": "UNKNOWN" + }, + { + "type": "paper", + "id": "pmlr-v202-sheng23a", + "version": "ICML 2023 official ten-author edition, PMLR 202:31094-31116", + "url": "https://proceedings.mlr.press/v202/sheng23a/sheng23a.pdf", + "content_sha256": "7cd9673c37640011d38ef4b8595f5e1e09a85b1f803a3734f19c97ca66c75b9e", + "sections": [ + "1 and 3; 4.1-4.3, Figures 1-3, Algorithm 1, Equation 1", + "4.4 (scope boundary only); 6 and 6.1, Tables 2-3 (denominators and ablations)", + "Appendix A.3 selected cost/peak-memory formulas, PDF pages 15-17; A.2 ending only (theorem boundary)" + ], + "readset": "references/flexgen-offload-feasibility.md", + "license": "CC-BY-4.0" + }, + { + "type": "git", + "url": "https://github.com/shenh10/HuggingArch", + "commit": "facbbbc96856e558b9084a94a5070b2e2c86cf0b", + "paths": [ + "docs/validation.md", + "backend/arch/validate/_matching.py", + "backend/arch/validate/_weight_ledger.py", + "backend/arch/validate/_ablation.py", + "docs/parallelism.md", + "docs/evaluation_model.md", + "docs/measured_calibration.md", + "docs/known_limitations.md", + "kernel_bench/emit.py", + "backend/inference/op_calibration.py", + "backend/measure/scope.py", + "LICENSE" + ], + "readset": "references/huggingarch-checkpoint-anchor.md", + "identity": "references/huggingarch-source-identity.json", + "license": "MIT" + }, + { + "type": "document", + "url": "https://shenhan.cc/blog/huggingarch", + "displayed_publication_date": "2026-10-08", + "accessed": "2026-11-10", + "sha256": "31d0e0a9bbf9e32723fb6ab09179fbefb2fbba1043863f1475ecc7ea7ce24ed4", + "readset": "references/huggingarch-checkpoint-anchor.md", + "scope": "Live, unversioned companion post; sections 2.5, 2.6, 3.6 and 3.7 read from one retrieval, whose HTML bytes the sha256 identifies. Published before the pinned commit, so not attributable to it; figures absent from the repository are recorded as unverified.", + "license": "UNKNOWN" + } + ], + "verification": { + "locally_verified": false, + "evidence_level": "source_grounded", + "note": "Historical repository methods plus reviewed paper extensions: references/papers-20261015/READSET.md records exact paper versions and inspection scope. Physical/conditional bounds, observed envelopes, simulator predictions and application measurements remain distinct. The bundled pure helper computes supplied L1 counts only; no automatic workload estimation, local accelerator measurements or E2E qualification. references/huggingarch-checkpoint-anchor.md records the checkpoint gates and the two-threshold rule from HuggingArch at a full commit, with per-file digests and inspection scope in references/huggingarch-source-identity.json; per-rank weight bytes are anchored only after the second gate passes." + }, + "applicability": { + "scope": "inference serving systems (LLM, diffusion, audio)", + "hardware": "any; CUDA examples from sources, NPU adaptation required", + "prerequisites": "profiler access (nsys/ncu or NPU equivalent), or willingness to work at L1 analytical only" + }, + "tags": [ + "performance", + "roofline", + "profiling", + "bottleneck", + "inference" + ], + "profile": "local-skill-v1", + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Acceptance-target preparation is agent guidance: the CLI and pure L1 helper do not infer gates, and unsupported estimates cannot replace user criteria or qualify campaign outcomes.", + "Mooncake's prefix-reuse condition is an analytical approximation; current cache validity, placement, effective rates, overlap and SLO outcomes remain unverified.", + "FlexGen's cross-tier feasibility comparison assumes fitted rates and perfect overlap; relaxed placements may OOM, the printed bls wording remains unresolved, and current compatibility, latency and efficacy are unverified.", + "The HuggingArch checkpoint anchor covers stored weights only. KV size and growth, activation traffic, collectives and end-to-end serving performance stay unanchored by it. Per-rank weight bytes stay unanchored until the second gate passes, and that gate checks the declared layout and format, not the placement a framework uses at run time." + ], + "provenance_readsets": [ + "references/flexgen-offload-feasibility.md", + "references/huggingarch-checkpoint-anchor.md", + "references/huggingarch-source-identity.json", + "references/mooncake-prefix-reuse-feasibility.md", + "references/papers-20261015/cases/decision-cases.md", + "references/papers-20261015/govern-optimization-experiments/procedure.md", + "references/papers-20261015/govern-optimization-experiments/references/source-notes.md", + "references/papers-20261015/INTEGRATION_NOTES.md", + "references/papers-20261015/knowledge/bounds-predictions-and-decisions.md", + "references/papers-20261015/MECHANISMS.md", + "references/papers-20261015/README.md", + "references/papers-20261015/READSET.md", + "references/papers-20261015/redirect-cross-layer-optimization/procedure.md", + "references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md", + "references/papers-20261015/references/agentic-kernel.md", + "references/papers-20261015/references/flashvector.md", + "references/papers-20261015/references/inferencebench.md", + "references/papers-20261015/references/prior-coverage.md", + "references/papers-20261015/references/vidur.md", + "references/papers-20261015/STATIC_CHECKS.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Use supplied acceptance criteria unchanged. When absent, follow references/acceptance-criteria.md to derive targets from matching baseline, calibrated attainable performance and uncertainty; retain partial requirements and label derived fields.", + "Read the full procedure and linked resources; preserve the distinction between analytical bounds, observed attainable envelopes, predictions and measured application outcomes." + ] + } --- # Performance Ceiling Analysis @@ -394,7 +558,7 @@ The [modeling comparison](references/papers-20261015/MECHANISMS.md), and [decision cases](references/papers-20261015/cases/decision-cases.md) retain paper-specific costs, counter-signals and unresolved metric discrepancies. Use [simulator calibration](../calibrate-serving-simulation/SKILL.md) for a -prediction-based shortlist and [attention diagnosis](../../hardware/cuda/diagnose-attention-bottlenecks/SKILL.md) +prediction-based shortlist and [attention diagnosis](../../../systems/platforms/cuda/diagnose-attention-bottlenecks/SKILL.md) for algorithm/operation-class changes. These remain candidate guidance, not local performance measurements. The bundled helper only computes explicitly supplied L1 inputs under the contract above. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/acceptance-criteria.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/acceptance-criteria.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/flexgen-offload-feasibility.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/flexgen-offload-feasibility.md similarity index 99% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/flexgen-offload-feasibility.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/flexgen-offload-feasibility.md index aefc1ba43..937228e46 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/flexgen-offload-feasibility.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/flexgen-offload-feasibility.md @@ -13,7 +13,7 @@ Sheng et al., *FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU*, ICML 2023, PMLR 202:31094–31116 ([proceedings](https://proceedings.mlr.press/v202/sheng23a.html), [paper](https://proceedings.mlr.press/v202/sheng23a/sheng23a.pdf)). The -[owner's paper source](../metadata.json) records the fixed edition and digest: +[owner's paper source](../SKILL.md) records the fixed edition and digest: the official ten-author edition, 23 PDF pages, SHA-256 `7cd9673c37640011d38ef4b8595f5e1e09a85b1f803a3734f19c97ca66c75b9e`. The extended-author arXiv v2 edition is not the extraction source. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md similarity index 99% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md index b4df46557..dab1c7b26 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-checkpoint-anchor.md @@ -97,7 +97,7 @@ Every marker carries a locator. Line numbers refer to the files at the pin. 1. Read every shard's safetensors header: tensor names, dtypes, shapes and data offsets. One header read per shard is enough. Loading mechanics stay with - [weight-loading-optimization](../../weight-loading-optimization/SKILL.md). + [weight-loading-optimization](../../../optimization/weight-loading-optimization/SKILL.md). 2. List every tensor the L1 count includes: weights, biases, quantization scale siblings, separately stored tied copies and split parts. Give each a name pattern (`X` at indexed positions), stored dtype, stored shape and number of diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-source-identity.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/huggingarch-source-identity.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/huggingarch-source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md similarity index 96% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md index 5536f5f94..5542f4d26 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/mooncake-prefix-reuse-feasibility.md @@ -4,7 +4,7 @@ Qin et al., *Mooncake: Trading More Storage for Less Computation — A KVCache-centric Architecture for Serving LLM Chatbot*, FAST '25 proceedings, pp. 155-170 ([publisher](https://www.usenix.org/conference/fast25/presentation/qin), [paper](https://www.usenix.org/system/files/fast25-qin.pdf)). The fixed edition -and PDF digest are recorded in [the owner's paper source](../metadata.json). +and PDF digest are recorded in [the owner's paper source](../SKILL.md). This reference uses Section 2.2, Table 1 and Equations 1-2 (proceedings pp. 157-158; PDF pages 4-5). It does not substitute the later arXiv v5 edition. @@ -104,7 +104,7 @@ implementation or execution authorization. says nothing about SSD support in current Mooncake implementations. - Earlier extraction history positively lists `evaluate-kv-cache-transfer-gates` for remote prefix reuse, phase-pool transfer and projected decode-load - admission; see the [retained B integration notes](../../../repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md). + admission; see the [retained B integration notes](../../../../systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md). Its original synthesis/archive remains unavailable. This separate public-paper reference does not recover it; novelty against its unseen text is `UNKNOWN`. - The paper's historical capacity, transfer and GPU-time results are author diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/INTEGRATION_NOTES.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/INTEGRATION_NOTES.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/INTEGRATION_NOTES.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/INTEGRATION_NOTES.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/MECHANISMS.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/MECHANISMS.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/MECHANISMS.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/MECHANISMS.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/README.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/README.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/README.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/README.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/READSET.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/READSET.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/STATIC_CHECKS.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/STATIC_CHECKS.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/STATIC_CHECKS.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/STATIC_CHECKS.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/cases/decision-cases.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/cases/decision-cases.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/cases/decision-cases.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/cases/decision-cases.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/knowledge/bounds-predictions-and-decisions.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/agentic-kernel.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/agentic-kernel.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/agentic-kernel.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/agentic-kernel.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/prior-coverage.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/prior-coverage.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/prior-coverage.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/prior-coverage.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/vidur.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/vidur.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/vidur.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/vidur.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/roofline-contract.md b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/roofline-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/roofline-contract.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/roofline-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/roofline-example.json b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/roofline-example.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/references/roofline-example.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/references/roofline-example.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/scripts/roofline.py b/src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/scripts/roofline.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis/scripts/roofline.py rename to src/inferrsi/knowledge/data/skill_library/playbooks/measurement/performance-ceiling-analysis/scripts/roofline.py diff --git a/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md new file mode 100644 index 000000000..d05d2fb46 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/SKILL.md @@ -0,0 +1,486 @@ +--- +name: kernel-writing-guide +description: 'Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL, CUDA C++, AscendC/TileIR), universal correctness rules that apply regardless of framework, and the verify-before-benchmark workflow. Use before starting any kernel implementation to avoid common correctness and performance pitfalls. + + ' +metadata: + inferrsi: | + { + "tags": [ + "kernel", + "triton", + "cute", + "cuda", + "ascendc", + "performance", + "accelerator", + "akg", + "affine", + "tail-guards", + "causal-conv1d", + "state-update", + "vllm", + "gdn", + "causal-convolution", + "transformers", + "qwen3.5", + "binding", + "classification-prefill", + "sglang", + "prefill", + "state-ownership" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/Dao-AILab/causal-conv1d", + "commit": "cd81f0413cad2fc1e6f17e785ac39f59aae690cd", + "paths": [ + "LICENSE", + "README.md", + "causal_conv1d/__init__.py", + "causal_conv1d/causal_conv1d_interface.py", + "causal_conv1d/cpp_functions.py", + "csrc/causal_conv1d.cpp", + "csrc/causal_conv1d_fwd.cu", + "csrc/causal_conv1d_update.cu", + "setup.py", + "tests/test_causal_conv1d.py" + ], + "readset": "references/causal-conv1d-contract.md", + "license": "BSD-3-Clause", + "scope": "Forward/update caller contracts, selected packed/state kernel branches and build/binding evidence; no native execution." + }, + { + "type": "git", + "url": "https://github.com/vllm-project/vllm", + "commit": "537a79f0fc5ba564d2bef43d54915b2f80f9f77d", + "paths": [ + "LICENSE", + "vllm/model_executor/models/qwen3_next.py", + "vllm/model_executor/layers/mamba/gdn/qwen_gdn_linear_attn.py", + "vllm/model_executor/layers/mamba/gdn/base.py", + "vllm/model_executor/layers/mamba/mamba_utils.py", + "vllm/model_executor/layers/mamba/ops/causal_conv1d.py", + "vllm/v1/attention/backends/gdn_attn.py", + "vllm/v1/attention/backends/utils.py", + "vllm/envs.py", + "vllm/third_party/flash_linear_attention/LICENSE", + "tests/v1/attention/test_gdn_metadata_builder.py", + "tests/kernels/mamba/test_causal_conv1d.py" + ], + "readset": "references/vllm-gdn-causal-conv-contract.md" + }, + { + "type": "git", + "url": "https://github.com/huggingface/transformers", + "commit": "0dad7b822255a0ae261ec45ae937371e859ffd1a", + "paths": [ + "LICENSE", + "src/transformers/models/qwen3_5/modeling_qwen3_5.py", + "src/transformers/models/qwen3_5/configuration_qwen3_5.py", + "src/transformers/cache_utils.py", + "src/transformers/utils/import_utils.py" + ], + "readset": "references/hf-qwen35-gdn-source.json", + "license": "Apache-2.0", + "scope": "v5.10.2 Qwen3.5 conv/GDN binding, cache and local numerical contract; candidate, source-only" + }, + { + "type": "git", + "url": "https://github.com/huggingface/transformers", + "commit": "856157a2f3e9594954310df18fdccc31ffddebe9", + "paths": [ + "LICENSE", + "src/transformers/models/qwen3_5/modeling_qwen3_5.py", + "src/transformers/models/qwen3_5/configuration_qwen3_5.py", + "src/transformers/cache_utils.py", + "src/transformers/utils/import_utils.py", + "src/transformers/integrations/hub_kernels.py", + "src/transformers/integrations/__init__.py", + "src/transformers/masking_utils.py" + ], + "readset": "references/hf-qwen35-gdn-source.json", + "license": "Apache-2.0", + "scope": "v5.17.0 Qwen3.5 conv/GDN binding, cache and local numerical contract; candidate, source-only" + }, + { + "type": "git", + "url": "https://github.com/sgl-project/sglang", + "commit": "16176278d0180f1a1a578e53e6eef432577876c2", + "paths": [ + "LICENSE", + "python/sglang/kernels/ops/mamba/causal_conv1d.py", + "python/sglang/kernels/ops/mamba/causal_conv1d_triton.py", + "python/sglang/srt/arg_groups/fields/exec_.py", + "python/sglang/srt/configs/hybrid_arch.py", + "python/sglang/srt/configs/linear_attn_model_registry.py", + "python/sglang/srt/configs/mamba_utils.py", + "python/sglang/srt/configs/qwen3_5.py", + "python/sglang/srt/configs/qwen3_next.py", + "python/sglang/srt/layers/attention/attention_registry.py", + "python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py", + "python/sglang/srt/layers/attention/linear/gdn_backend.py", + "python/sglang/srt/layers/attention/linear/kernels/gdn_cutedsl.py", + "python/sglang/srt/layers/attention/linear/kernels/gdn_flashinfer.py", + "python/sglang/srt/layers/attention/linear/kernels/gdn_triton.py", + "python/sglang/srt/layers/attention/linear/utils.py", + "python/sglang/srt/layers/attention/mamba/causal_conv1d.py", + "python/sglang/srt/layers/radix_linear_attention.py", + "python/sglang/srt/mem_cache/memory_pool.py", + "python/sglang/srt/models/qwen3_5.py", + "python/sglang/srt/models/qwen3_5_text.py", + "python/sglang/srt/models/qwen3_next.py", + "test/registered/layers/mamba/test_causal_conv1d.py", + "test/registered/unit/configs/test_linear_attn_model_registry.py", + "test/registered/unit/layers/attention/test_gdn_prefill_backend_policy.py", + "test/registered/unit/layers/attention/test_linear_attn_config.py", + "test/registered/unit/layers/test_gdn_mis_metadata.py" + ], + "readset": "references/sglang-gdn-causal-conv-contract.md", + "license": "Apache-2.0", + "scope": "Bounded Qwen3.5/Qwen3-Next caller, GDN phase/architecture policy, separate conv wrappers and state metadata; no native execution or external kernel equivalence." + }, + { + "type": "git", + "url": "https://github.com/InferMatrixLab/system1-omni", + "commit": "509bdd2161ca5bef8f5e158d466514862fac7c9e", + "paths": [ + "src/models/pplx_decider/conv.py", + "src/models/pplx_decider/engine.py", + "src/models/pplx_decider/gdn.py" + ], + "readset": "references/pplx-gdn-caller-context.json", + "license": "UNKNOWN", + "scope": "Bounded secondary PPLX caller context for the HF/vLLM/SGLang cross-map: source-level default/override, conv and FI adapter/state guards only; no implementation vendored or execution qualification." + } + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton, CuTe DSL, CUDA C++, AscendC or TileIR, applying framework-independent correctness rules, and verifying the executing kernel against a reference before benchmarking; covers fused elementwise chains, reductions, tiled GEMM, attention variants, custom collectives, causal convolution and stateful serving-operator qualification", + "prerequisites": "The applicable operator, caller/state and layout contracts are read before substitution, including the causal-conv1d contract and HF Qwen3.5 GDN binding; the CuTe DSL environment uses Linux x86_64, Python 3.10–3.13 and Ampere+ (SM80+) with nvidia-cutlass-dsl installed and no NPU target; the CANN programming guide for the specific SoC is consulted for AscendC", + "input_contract": "The candidate operation or operation chain, its shapes, dtypes, quantization and layout requirements, and the target execution phase; the target accelerator and framework constraints; for serving operators, the concrete caller, effective phase and backend, state-row mapping, cache or buffer lifetime and in-place-write behavior; a reference implementation with the same signature, a callable kernel wrapper and fresh CUDA or NPU test tensors; verification shapes and dtypes plus profiler output for the target metric", + "produced_artifact": "A write-or-decline decision for the candidate operation with the profiled bottleneck, fusable pair or required dtype, shape or quantization variant unsupported by the standard implementation that justifies it, a framework choice routed by pattern and accelerator, a correctness checklist covering fp32 transcendental round-trips, sync-free wrappers, C division semantics, pre-allocated outputs, fp32 matmul accumulators and masked boundary loads and stores, a starting block or tile size with its autotune and warmup cost noted, a kernel shipping kernel_fn, reference_fn and get_inputs, a reference-parity result across the verification shapes and dtypes, and a post-verification profiler reading on the target metric; for an existing causal-convolution or stateful serving operator, a qualification record naming the caller, phase, backend, state rows and in-place writes, with source selection kept apart from executed dispatch; a routing and correctness decision, not a complete implementation or a measured speedup", + "failure_modes": [ + "A kernel is written for a standalone elementwise op, a standalone matmul or standard attention where the vendor library already wins", + "Host-side scheduling cost is treated as a kernel problem instead of being routed to host-overhead-profiling", + "Fusion is justified by removed Python call overhead rather than by the DRAM traffic the shared data no longer makes", + "A transcendental runs on fp16/bf16 inputs without the fp32 cast round-trip, so the error stays silent through benchmarking", + "tl.dot accumulates in the input dtype because Triton is assumed to promote fp16/bf16 to fp32 automatically", + "Negative offsets are computed assuming Python floor division while Triton or AscendC rounds toward zero", + "An unmasked load or store is accepted because the benchmark shapes happen to divide the block size", + "A kernel is benchmarked before it matches its reference, or a shape-boundary, accumulator-dtype or precision failure is read as a tuning problem", + "A verification pass is reported as a win although the profiler metric did not move, so the kernel was never on the critical path", + "Autotune compile cost is measured as steady-state serving latency instead of being pre-warmed or cached", + "A CuTe DSL pattern is routed to an NPU target, or a CUDA tail-guard, attention primitive or SM-matched example is assumed to carry to Ascend", + "A causal-conv1d or GDN operator is substituted on build success, import success or matching names without resolving dtype, strides, packed boundaries, state rows and cursors", + "An in-tree vLLM wrapper, a GDN prefill flag or a cache-free classification adapter is read as establishing the executed convolution path or generation-state support" + ], + "refusal_point": "Declines to start a kernel until a profiler puts the target on the critical path near its hardware ceiling, a concrete fusable pair is identified, or the required dtype, shape or quantization variant is unsupported by the standard implementation, and sends host-bound cases elsewhere; refuses to benchmark an unverified kernel and withholds a gain whose profiler metric did not move; keeps operator source selection separate from executed dispatch, numerical parity and hardware qualification, so build or import success qualifies nothing; treats AscendC and TileIR content as source-inferred pending CANN and SoC verification with no local NPU run, treats the AKG Affine FileCheck expectations as IR structure rather than executed CUDA correctness, and reads author-reported overlap and FP8 cases as counterexamples to unconditional tuning rules; claims no Ascend support from NVIDIA primitives", + "hardware": "CUDA Ampere+ and Ascend NPU have separate framework routes for each kernel pattern; CuTe DSL requires Linux x86_64, Python 3.10-3.13 and SM80+ with nvidia-cutlass-dsl and is unavailable on NPU; profiling uses ncu on CUDA and msprof on Ascend; the AscendC and TileIR guidance is source-inferred with no local NPU verification and requires validation against the specific CANN version and SoC" + }, + "recommendations": [ + "Confirm with nsys/ncu or msprof that the target kernel sits on the critical path near its hardware ceiling, name the adjacent operations whose shared data would otherwise round-trip through DRAM, or identify a required dtype, shape or quantization variant unsupported by the standard implementation, before writing any kernel.", + "Ship every kernel with kernel_fn, reference_fn and get_inputs, pass reference parity across the verification shapes and dtypes first, and profile afterwards to confirm the target metric actually moved.", + "Cast fp16/bf16 to fp32 and back for any math beyond +, -, * and /, declare tl.dot accumulators as tl.float32, and mask every load and store whose shape does not divide the block size.", + "Start Triton at BLOCK_SIZE 1024 on Hopper and 512 on Ampere using powers of two, take GEMM tiles from the source skeleton configs (128/256/64 for Hopper, 64/128/32 for smaller cards) and autotune from there.", + "On Ascend, reach for TileIR before raw AscendC, size tiles to fit the AI Core Unified Buffer and check cube-unit alignment against the CANN guide for the specific SoC.", + "Match a CuTe DSL example to the GPU architecture (SM80 Ampere, SM90 Hopper, SM100 Blackwell) and study its structure instead of copying it, and pre-warm or cache autotuned kernels at startup so first-run compile cost stays out of serving latency." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", + "The typed vLLM entry covers only the new GDN reference; older source inventories remain incomplete.", + "The vLLM and HF references identify separate source call contracts; no binary equivalence, executed backend, hardware qualification, numerical/model parity, performance or agent-efficacy evaluation.", + "SGLang source contract only; native/backend/sm89 qualification, HF classification binding, numerical/model quality, performance and agent efficacy are not evaluated.", + "The secondary PPLX caller readset has UNKNOWN repository licensing and source-only default/adapter evidence; no worker/kernel execution, model/quality/performance or stateful reuse is qualified." + ], + "provenance_readsets": [ + "references/akg-affine-tail-guards.md", + "references/causal-conv1d-contract.md", + "references/vllm-gdn-causal-conv-contract.md", + "references/hf-qwen35-gdn-binding.md", + "references/hf-qwen35-gdn-source.json", + "references/sglang-gdn-causal-conv-contract.md", + "references/pplx-gdn-caller-context.json" + ], + "consumption_mode": "packaged" + } +--- + +# Kernel Writing Guide + +Status: candidate. Source-grounded from TensorRT-LLM `kernel-triton-writing` +and `kernel-cute-writing` skills @ f33e0e5f (NVIDIA/Apache-2.0). NPU-specific +content inferred from architecture parallels; verify against CANN/TileIR docs. + +## SearchSpace mapping + +Accelerator row → Optimize cell + +Second-level targets: Kernel fusion, custom GEMM, attention variants, elementwise +chains, reduction kernels. + +## When to write a custom kernel + +Writing a kernel is expensive. Verify the bottleneck before starting. + +**Write a custom kernel when:** +- A profiler (nsys/ncu or CANN msprof) confirms a specific kernel is on the + critical path and near its hardware ceiling. +- Two or more adjacent operations can fuse: they share data that would otherwise + go to/from DRAM between calls. The gain is the reduction in memory traffic, + not the removal of Python overhead. +- A standard implementation exists but the specific dtype, shape, or quantization + variant you need is not covered. + +**Do NOT write a custom kernel when:** +- A single elementwise operation (relu, sigmoid on its own): the vendor library + is already optimal. +- A standalone matmul: cuBLAS/CANN is better than a handwritten GEMM in almost + all cases. +- Standard attention: use FlashAttention or the vendor's equivalent. +- The bottleneck is host-side scheduling: that is a Python/GIL problem, not a + kernel problem. See `host-overhead-profiling`. + +## Qualify an existing causal convolution operator + +Before replacing or binding causal depthwise convolution, use the +[causal-conv1d contract](references/causal-conv1d-contract.md) to choose a +full-sequence forward call or an in-place state update. Check dtype, strides, +packed boundaries, state rows and circular-buffer cursors before substitution. +The [five decision cases](cases/causal-conv1d-dispatch-state.md) cover invalid +layouts, continuation state and unproved dispatch. Build targets and import +success do not establish the executing operator, numerical parity or a gain. + +For a concrete Transformers Qwen3.5 caller, first use the +[two-version HF binding contract](references/hf-qwen35-gdn-binding.md) and +[source cases](cases/hf-qwen35-gdn-binding.md). Resolve module versus instance +binding, active forward overrides and cache state before substituting conv or +GDN. A cache-free classification adapter does not establish generation-state +support; package presence and matching names do not prove executed dispatch. + +## Framework routing + +| Pattern | CUDA / Ampere+ | Ascend NPU | +|---------|---------------|------------| +| Fused elementwise chain (2+ ops sharing data) | Triton (`@triton.jit`) | TileIR / AscendC | +| Reduction (softmax, LayerNorm, RMSNorm) | Triton | TileIR / AscendC | +| Tiled GEMM with autotune | Triton | AscendC | +| GEMM with explicit shared memory / tensor core MMA | CuTe DSL (CUTLASS 4.x) | AscendC cube unit | +| Flash Attention variant | CuTe DSL or Triton | AscendC (custom) | +| Custom collective (AllReduce variant) | NCCL/custom CUDA | HCCL plugin | + +CuTe DSL requires Linux x86_64, Python 3.10–3.13, Ampere+ (SM80+), and +`pip install nvidia-cutlass-dsl`. It is not available on NPU. + +Source: TRT-LLM kernel-triton-writing + kernel-cute-writing @ f33e0e5f. + +## Universal correctness rules (apply to all frameworks) + +These rules cause silent wrong results if violated. Check them before +benchmarking. + +### 1. Transcendental functions require FP32 inputs + +`exp`, `log`, `erf`, `tanh`, and similar transcendentals cause errors or +undefined behavior with FP16/BF16 inputs in Triton. Pattern: + +```python +# Wrong — silent error or wrong result with fp16/bf16: +result = tl.exp(x) + +# Correct — cast to fp32, compute, cast back: +x_fp32 = x.to(tl.float32) +result = tl.exp(x_fp32).to(x.dtype) +``` + +Rule: any math function beyond `+`, `-`, `*`, `/` needs an fp32 cast round-trip. + +Source: TRT-LLM kernel-triton-writing @ f33e0e5f, §FP16/BF16 Precision Rules. + +### 2. Never call `.item()` inside a kernel wrapper + +`.item()` forces a CPU-GPU synchronization (~50–100 µs per call). This turns +into a pipeline stall if called inside the hot path. + +```python +# Wrong — forces sync: +seed = some_tensor.item() + +# Correct — use data pointer arithmetic, no sync: +seed = (some_tensor.data_ptr() % (2**31)) ^ n_elements +``` + +Source: TRT-LLM kernel-triton-writing @ f33e0e5f, §CPU-GPU Sync Avoidance. + +### 3. C integer division semantics (Triton / AscendC) + +Triton and most NPU DSLs use C semantics for `//` and `%`: they round toward +zero, not toward negative infinity like Python. This only matters when operands +can be negative (which they can in offset calculations). + +``` +Python: -7 // 2 = -4, -7 % 2 = 1 +C/Triton: -7 // 2 = -3, -7 % 2 = -1 +``` + +Safe pattern: ensure all index and offset values are non-negative. If negative +values are possible: `(idx % BLOCK + BLOCK) % BLOCK`. + +Source: TRT-LLM kernel-triton-writing @ f33e0e5f, §C Integer Division Semantics. + +### 4. Pre-allocate output tensors + +Allocating output inside the kernel wrapper on every call adds Python object +creation cost on the hot path. Accept a pre-allocated output buffer as a +parameter, or cache allocation outside the benchmark loop. + +### 5. Use FP32 accumulators for matmul (Triton) + +Triton does NOT auto-promote fp16/bf16 inputs to fp32 for accumulation in +`tl.dot`. Always declare the accumulator as `tl.float32`: + +```python +acc = tl.zeros((BLOCK_M, BLOCK_N), dtype=tl.float32) +``` + +### 6. Always mask loads and stores for non-divisible shapes + +If `n_elements % BLOCK_SIZE != 0`, an unmasked load reads past the tensor +boundary. Always include a mask: + +```python +mask = offsets < n_elements +x = tl.load(x_ptr + offsets, mask=mask) +tl.store(out_ptr + offsets, result, mask=mask) +``` + +### AKG Affine loop tails (source-derived) + +For AKG-MLIR static CUDA elementwise loops, use the +[Affine tail-guard reference](references/akg-affine-tail-guards.md) to trace the +effective target, option precedence and loop bounds before interpreting a +non-divisible tile. The compiler can expand point-loop bounds and move the body +under an `affine.if`; both input reads and output writes need the valid-index +contract. The pinned fixture's FileCheck expectations describe IR structure, +not executed CUDA correctness or a tuning gain. This addition preserves the +handwritten load/store mask rule above and requires separate validation of +later lowering, synchronization and numerics. CPU/NPU tail strategies are not +qualified by the CUDA fixture. + +## Workflow: verify before benchmark + +### Qualify a stateful serving operator + +For vLLM Qwen3-Next GDN convolution/recurrence reuse, read the +[pinned phase/state contract](references/vllm-gdn-causal-conv-contract.md) and +[representative source cases](cases/vllm-gdn-source-contract.md). Trace the +concrete caller, effective phase/backend, state-row mapping and in-place writes +before choosing an operator. The in-tree vLLM wrappers do not establish a +standalone-package or HF-worker binding. Source selection remains separate +from executed correctness, hardware qualification and performance. + +For SGLang Qwen3.5 or Qwen3-Next, use the separate +[caller/backend contract](references/sglang-gdn-causal-conv-contract.md) and +[four source cases](cases/sglang-gdn-source-contract.md). Resolve the model's +linear layers, per-runner prefill/decode choices, architecture predicates, +conv aliases and state-row metadata before proposing reuse. A GDN prefill flag +does not select the convolution implementation. Keep one-shot classification +prefill distinct from decode, cached continuation and delimiter-based scoring; +source-derived transfer conditions are not an HF binding or execution result. +The [worker/engine cross-map](references/sglang-gdn-causal-conv-contract.md#map-the-inspected-hf-workers-to-vllm-and-sglang) +connects the inspected Clef/PPLX bindings to both engine operator contracts; +its PPLX context follows the default GVA forward and the state-free adapter. + +**Phase 1: Write the kernel with a reference function** + +Every kernel must ship with three exports (used by companion verification and +benchmark scripts): + +```python +kernel_fn = my_kernel_wrapper # the implementation +reference_fn(*args) -> Tensor # PyTorch reference with identical signature +get_inputs() -> list # returns fresh CUDA/NPU tensors for testing +``` + +**Phase 2: Verify against reference** + +Run `scripts/verify_kernel.py` (or equivalent) to check that the kernel output +matches the reference across a range of shapes and dtypes before any performance +measurement. Do not benchmark an unverified kernel. + +Common verification failures: +- Shape boundary errors (off-by-one in mask) +- Accumulator dtype (fp16 accumulation in matmul) +- Transcendental precision (forgot fp32 cast) + +**Phase 3: Profile to confirm bottleneck** + +After verification, profile with ncu (CUDA) or msprof (Ascend) to confirm the +kernel has improved the target metric. An optimization that passes verification +but does not change the profiler output was not on the critical path. + +## Block size selection + +For Triton on CUDA: +- Start with `BLOCK_SIZE = 1024` for H100/Hopper, `512` for A100/Ampere. +- Block sizes must be powers of 2. +- For GEMM tiles (`BLOCK_M`, `BLOCK_N`, `BLOCK_K`): start from the tiled matmul + skeleton configs in the source (128/256/64 for Hopper, 64/128/32 for smaller + cards) and autotune from there. + +For AscendC on Ascend NPU: +- The AI Core's local buffer (Unified Buffer / UB) is the equivalent of shared + memory. Tile sizes must fit in UB. +- Cube unit (matrix unit) alignment requirements differ by SoC; check the CANN + AscendC programming guide for the specific SoC target. +- TileIR (the higher-level DSL above raw AscendC) auto-tiles for standard + patterns; use it before dropping to raw AscendC. + +## Starting from examples + +For CuTe DSL on CUDA: find the closest example in the CUTLASS repo at +`https://github.com/NVIDIA/cutlass/tree/main/examples/python/CuTeDSL`. +Match the GPU architecture first (SM80 for Ampere, SM90 for Hopper, SM100 for +Blackwell). Study the example's structure; do not copy it verbatim since shapes, +dtypes, and problem dimensions differ. + +For AscendC: consult the CANN AscendC operator development guide for your +specific SoC (Ascend 910B, 910C, etc.). Templates and intrinsics differ between +generations. + +## Attention mechanisms and scoped acceptance + +Use [attention diagnosis](../../../systems/platforms/cuda/diagnose-attention-bottlenecks/SKILL.md) when IO +reduction exposes scalar work, insufficient independent tiles, reduction costs, +compiler ordering or precision/layout overhead. Its +[mechanism notes](../../../systems/platforms/cuda/diagnose-attention-bottlenecks/references/mechanisms.md) +retain FlashAttention 1/2/3 assumptions, operation-class costs and counter-signals. +Tiling, recomputation, ownership and overlap change the resource balance; no +single utilization threshold or paper tile size selects a winner. + +Preserve dense/sparse/quantized semantics, masks, dropout, output/gradient +accuracy and real buffer lifetimes. Verify the executing instruction schedule +before claiming overlap. Include conversion/preprocessing and exactness costs +in the measured boundary. NVIDIA primitives establish no Ascend support. +[Author-reported cases](../../../systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md) +are counterexamples to unconditional tuning rules, not local successes. + +## Limitations + +- This skill is a routing and correctness guide, not a complete implementation + reference. The source TRT-LLM skills contain full implementations for common + patterns (fused GELU+dropout, softmax, LayerNorm, tiled GEMM, Flash Attention). +- CuTe DSL patterns are NVIDIA CUDA-only (Linux x86_64, SM80+). +- AscendC/TileIR patterns are source-inferred for this skill — verify against + your specific CANN version and SoC before use. No local NPU verification done. +- Autotune (Triton `@triton.autotune`) adds substantial first-run compile cost. + For serving, pre-warm the kernel at startup or cache the compiled binary. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/causal-conv1d-dispatch-state.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/causal-conv1d-dispatch-state.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/causal-conv1d-dispatch-state.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/causal-conv1d-dispatch-state.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/hf-qwen35-gdn-binding.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/hf-qwen35-gdn-binding.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/hf-qwen35-gdn-binding.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/hf-qwen35-gdn-binding.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.json b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/sglang-gdn-source-contract.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/sglang-gdn-source-contract.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/sglang-gdn-source-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/sglang-gdn-source-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-causal-conv-contract.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-causal-conv-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-causal-conv-contract.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-causal-conv-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.json b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-source-contract.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-source-contract.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-source-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/cases/vllm-gdn-source-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/akg-affine-tail-guards.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/akg-affine-tail-guards.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/akg-affine-tail-guards.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/akg-affine-tail-guards.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/causal-conv1d-contract.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/causal-conv1d-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/causal-conv1d-contract.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/causal-conv1d-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-binding.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/hf-qwen35-gdn-binding.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-binding.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/hf-qwen35-gdn-binding.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-source.json b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/hf-qwen35-gdn-source.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/hf-qwen35-gdn-source.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/hf-qwen35-gdn-source.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/licenses/sglang-LICENSE.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/licenses/sglang-LICENSE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/licenses/sglang-LICENSE.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/licenses/sglang-LICENSE.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/pplx-gdn-caller-context.json b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/pplx-gdn-caller-context.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/pplx-gdn-caller-context.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/pplx-gdn-caller-context.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-source-files.json b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/sglang-gdn-source-files.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/sglang-gdn-source-files.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/sglang-gdn-source-files.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/kernel-writing-guide/references/vllm-gdn-causal-conv-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/training-input-pipeline/SKILL.md similarity index 65% rename from src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/training-input-pipeline/SKILL.md index d27999a20..f4ac16a3c 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/training-input-pipeline/SKILL.md @@ -3,6 +3,56 @@ name: training-input-pipeline description: 'Diagnose training input stalls and choose bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic Streaming and FFCV. Preserve execution-mode gates, sample ordering and per-host memory budgets; separate remote downloads, local storage reads and host/device transfers. ' +metadata: + inferrsi: | + { + "tags": [ + "training", + "input-pipeline", + "prefetch", + "dataset", + "memory", + "storage" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Diagnosing training input stalls and choosing bounded queue, shard-cache and cache-ownership changes from DALI, Mosaic Streaming and FFCV, while preserving execution-mode gates, sample ordering and per-host memory budgets and separating remote downloads, local storage reads and host/device transfers", + "prerequisites": "The pinned readset and its license constraints are read; reading authorizes no commands, cache manipulation or experiment", + "input_contract": "Dataset format, sample-size distribution, workers and ranks per host, ordering/shuffle, local-cache state and executor mode; a per-host memory budget that includes checkpoint staging and runtime offload; timings that separate remote fetch, decompression/decoding, transformation, batch readiness and device transfer", + "produced_artifact": "One proposed queue, shard-cache or cache-ownership change with its source-backed rule and boundary, the evidence proposed to confirm it, and a verification contract for a separately authorized comparison; no input-throughput, model-quality or hardware result", + "failure_modes": [ + "Low accelerator utilization is taken to identify which producer stage is late", + "Missing legacy DALI statistics are read as zero use, or the execution mode is switched silently to obtain them, which changes the comparison", + "DALI run() returning is read as output-ready timing instead of the outputs() synchronization boundary", + "Mosaic predownload is treated as a byte cap or request count, or per-worker sample targets are multiplied into an exact network-byte prediction", + "Look-ahead is increased when the local cache cannot retain the working set, so evicted shards are downloaded again", + "cache_limit is read as a bound on total process RAM, or the constructor minimum as an optimum or a guarantee against thrashing", + "An FFCV memory map is taken as proof that its pages are resident" + ], + "refusal_point": "Rejects changes that violate sample coverage or the stated reproducibility contract even if batches arrive faster, does not report queue depth alone as a performance result, and draws no NPU acceleration claim from the CUDA-oriented DALI source" + }, + "recommendations": [ + "Record dataset format, sample sizes, workers per host, ordering, cache state and executor mode before proposing one change.", + "Increase Mosaic look-ahead only when readiness starvation is observed and cache headroom exists; otherwise address shard and working-set residency first.", + "Check the exact DALI configuration surface before use, because the constructor and deserialization paths use different queue keys.", + "In a later comparison, separate warm-cache throughput from cold retrieval and initialization, and report step time with input wait, host/HBM peaks and actual I/O." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/sources.md" + ], + "consumption_mode": "packaged" + } --- # Training Input Pipeline Decisions diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/references/sources.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/training-input-pipeline/references/sources.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-input-pipeline/references/sources.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/training-input-pipeline/references/sources.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/SKILL.md similarity index 83% rename from src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/SKILL.md index edc36a2e4..508826837 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/SKILL.md @@ -1,6 +1,46 @@ --- name: video-diffusion-optimization description: 'Run and verify goal-driven video diffusion optimization, backend-agnostic. Use when the user asks for latency, RTF, throughput, or memory targets across VLM, DiT, MoE, attention, or VAE, on any accelerator. Not for text-only LLMs.' +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "status": "candidate", + "origin": [ + "first-party" + ], + "tags": [ + "video-diffusion", + "dit", + "moe", + "attention", + "vae", + "performance", + "profiling", + "serving", + "optimization" + ], + "empirical_validation": "not_run", + "verification": { + "locally_verified": false, + "evidence_level": "guidance", + "note": "Procedure is first-hand workflow guidance. The linked field notes are not reproducible as published. No acceptance campaign under this repository's evidence contract backs this skill, and no public performance claim is made." + }, + "applicability": { + "scope": "Backend-agnostic video diffusion inference and serving optimization", + "hardware": "Any accelerator (Ascend NPU, CUDA GPU, and others); operator names, memory layouts, and profiler references are re-derived per backend", + "prerequisites": "A reproducible benchmark, a frozen quality check, and numeric gates defined in the governing acceptance issue" + }, + "limitations": [ + "Backend-agnostic skeleton only. The first-hand field notes are Ascend-specific and live in the ascend-video-diffusion-optimization skill; every operator, layout and profiler reference (TransData, FRACTAL_NZ, CANN) must be re-derived per backend.", + "Candidate workflow guidance; not independently validated.", + "Model identifiers, campaign revisions, and numeric gates are intentionally excluded and remain issue-scoped." + ], + "provenance_readsets": [], + "consumption_mode": "packaged" + } --- # Video Diffusion Optimization @@ -9,9 +49,9 @@ Status: candidate workflow guidance, backend-agnostic. The procedure is first-ha Own the performance objective through implementation and final verification. Do not stop after profiling, proposing changes, or showing a faster microbenchmark. Continue the measure-change-verify loop until the target is met, the authorized budget is exhausted, or an external blocker prevents execution. -This skeleton is backend-agnostic. Operator names, memory layouts, and profiler references are backend-specific. The `ascend-video-diffusion-optimization` skill under `hardware/ascend/` specializes this skeleton with Ascend annotations (`TransData`, `FRACTAL_NZ`, `CANN`) and first-hand field notes; re-derive those per backend before following the candidate-selection sections below. +This skeleton is backend-agnostic. Operator names, memory layouts, and profiler references are backend-specific. The `ascend-video-diffusion-optimization` skill under `systems/platforms/ascend/` specializes this skeleton with Ascend annotations (`TransData`, `FRACTAL_NZ`, `CANN`) and first-hand field notes; re-derive those per backend before following the candidate-selection sections below. -Verify every stage against the packaged [benchmark design](../inference-benchmark-design/SKILL.md) and [evidence discipline](../../workflows/rsi-evidence/SKILL.md). A microbenchmark or operator parity check is not end-to-end evidence. +Verify every stage against the packaged [benchmark design](../../measurement/inference-benchmark-design/SKILL.md) and [evidence discipline](../../../workflows/governance/rsi-evidence/SKILL.md). A microbenchmark or operator parity check is not end-to-end evidence. ## 1. Freeze The Goal Contract diff --git a/src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/assets/campaign.template.json b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/assets/campaign.template.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/assets/campaign.template.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/assets/campaign.template.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/scripts/evaluate_target.py b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/scripts/evaluate_target.py old mode 100755 new mode 100644 similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/video-diffusion-optimization/scripts/evaluate_target.py rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/video-diffusion-optimization/scripts/evaluate_target.py diff --git a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md similarity index 68% rename from src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md index 73ef82bae..00dd494fd 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/SKILL.md @@ -3,6 +3,159 @@ name: weight-loading-optimization description: 'Optimize the serving-level weight load path from storage to device: judge the load path on four dimensions (source, format, placement, duplication), account startup fixed overhead separately from per-request cost with bytes-and-counts lines and hit-criteria warmup validation, verify sharded weight downloads against the safetensors index, apply device-side weight prefetch only on profiled memory-bound hot spots, and order CPU-tier offload candidates by change surface. Includes source-scoped disk-loading modes, pinned staging, synchronous checkpoint save semantics and fastsafetensors broadcast/scatter/coalesced delivery; also audit streamed expert-cache byte correctness and begin/end overlap contracts. Use when startup/loading latency dominates or a streamed expert-cache change needs correctness verification; route DCP asynchronous save performance and distributed recovery validity to their separate skills. ' +metadata: + inferrsi: | + { + "tags": [ + "storage", + "weight-loading", + "startup", + "prefetch", + "offload", + "serving", + "npu", + "collectives", + "tensor-parallel", + "Flare", + "model-preparation", + "prepared-runtime-package", + "expert-cache", + "loader-correctness" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/ggml-org/llama.cpp", + "commit": "3e390af18de1131dda27913c00511725b4274c8a", + "paths": [ + "common/arg.cpp", + "common/common.cpp", + "common/fit.cpp", + "src/llama-model.cpp", + "src/llama-model-loader.cpp", + "src/llama-model-loader.h", + "src/llama-mmap.cpp", + "src/llama-mmap.h", + "LICENSE" + ], + "readset": "references/llama-loading-readset.md" + }, + { + "type": "git", + "url": "https://github.com/foundation-model-stack/fastsafetensors", + "commit": "219e2561a41ede2d5d8eba611c248bb0effbf580", + "paths": [ + "fastsafetensors/loader.py", + "fastsafetensors/file_buffer.py", + "fastsafetensors/tensor_factory.py", + "fastsafetensors/parallel_loader.py", + "fastsafetensors/frameworks/_torch.py", + "fastsafetensors/frameworks/_paddle.py", + "docs/overview.md", + "examples/run_parallel.py", + "tests/unit/test_broadcast_runs.py", + "tests/unit/test_subgroup.py", + "LICENSE" + ], + "readset": "references/fastsafetensors-transfer-readset.md" + }, + { + "type": "git", + "url": "https://github.com/NVIDIA/flare-local-inference", + "resolved_url": "https://github.com/NVIDIA/local-inference", + "commit": "bce032c014bae34502ba22d3304fa084d5b40afe", + "paths": [ + "README.md", + "docs/runtime-package-design.md", + "helios/src/runtime_package/format.rs", + "helios/src/runtime_package/model.rs", + "helios/src/execution/manifest.rs", + "helios/src/model/contract.rs", + "helios/src/model/catalog.rs", + "LICENSE" + ], + "readset": "references/flare-preparation-boundary.md", + "license": "Apache-2.0" + }, + { + "type": "git", + "url": "https://github.com/ronak-create/deepseek-v4-in-c", + "commit": "d0402056d0da4bf7edfd2d4977de479a0db9d201", + "paths": [ + "LICENSE", + "README.md", + "Makefile", + "src/cache/dsv4_cache.c", + "src/cache/dsv4_cache.h", + "src/io/dsv4_st.c", + "src/io/dsv4_st.h", + "src/model/dsv4_layer.h", + "src/model/dsv4_layer.c", + "src/cli/dsv4_run.c", + "tests/unit/test_cache.c", + "tests/unit/test_model_oracle.c", + "tools/make_st_fixture.py", + "tools/make_tiny_checkpoint.py", + "tools/dsv4_ref.py" + ], + "readset": "references/deepseek-expert-cache-oracles.md", + "license": "Apache-2.0 (repository declaration and selected-file notices; third-party/model rights excluded)" + } + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Diagnosing the serving-level weight load path from storage to device and choosing changes to it when startup or loading latency dominates: locating the cost by source, format, placement and duplication, accounting once-per-process loading and warmup apart from per-request cost, verifying sharded downloads, restricting device-side prefetch to profiled memory-bound hot spots and ordering CPU-tier offload; also covers source-scoped disk-loading modes, pinned staging, synchronous checkpoint save semantics, fastsafetensors broadcast/scatter/coalesced delivery and the separation of model preparation from serving. Request-time host overhead belongs to host-overhead-profiling, DCP asynchronous save performance to dcp-async-save-performance and distributed recovery validity to checkpoint-recovery-consistency; for disk-streamed routed experts, audit independent byte identity, resident/pending slot ownership, ordered accumulation and actual oracle callback coverage; the DeepSeek source slice describes POSIX storage, C/OpenMP and pthread ownership; target validation must identify its actual direct/buffered and reader-pool paths", + "prerequisites": "Flag names and API entry paths are re-verified against the pinned source revisions, and torch_npu.npu_prefetch details against the target CANN version, before they are relied on; card state is checked before launch and process placement fixed before measuring, through ascend-device-health and os-process-placement; the Diffusers enable_model_cpu_offload path requires accelerate", + "input_contract": "The current load path's byte source, file format, host-first or direct device placement and duplicated copies across processes or instances; startup timings for loading and warmup with byte and access counts per segment, and the expected warmup hit criteria; the safetensors index weight_map and the downloaded shard set; a profile showing memory-bound hot spots and the adjacent compute or communication segments; free device memory and whether an OOM occurs during construction or inference; for cross-rank delivery, the file-to-loader-group-rank map, tensor names, shapes, dtypes and call order, the required replicated or sliced output, the model's expected shard layout and the collective backend; for streamed expert-cache changes, the selected direct/buffered I/O and reader-pool path, tensor offsets and cache-slot budget, requested keys, source callback table, independent byte/numerical reference and checked fixture coverage", + "produced_artifact": "The located load-path cost on the four dimensions with its matching lever, a startup ledger that keeps fixed overhead apart from per-request cost, a structural shard-completeness result naming the missing shards, a prefetch proposal behind an enable_prefetch switch defaulting to False or a recorded decision not to prefetch, an offload candidate order, the recorded loading mode, a distributed delivery choice with its proposed verification, and a preparation/serving record of serialized and rebuilt state, live resource owners and ABI/prepared-contract versions; proposed changes and verification targets, not a measured startup or delivery gain; a source-bound expert-cache byte/state audit and oracle-path coverage record, with synthetic-fixture, fallback and SKIP limits", + "failure_modes": [ + "Loading and warmup are amortized into per-request latency, or a host segment without timing points is treated as costing nothing", + "Warmup is accepted on total time while mismatched warmup shapes have warmed a different set of cache entries", + "In-flight .incomplete shards are re-downloaded as corrupt, or gap-free shard numbering is accepted without matching the index weight_map", + "Prefetch is forced without a profiled memory-bound hot spot or a non-memory-bound dependency window", + "Resolution or CFG tier-downs are applied to a construction-phase OOM, or offload is adopted when compute already hides the transfer", + "A free-memory probe runs on an mmap-backed load, or a run whose first mlock failure latched with only a warning is treated as fully resident", + "Disk counters that include cache and pipes are read as physical writes, or a run without a page-cache drop is reported as a cold start", + "The ceil-block scatter partition is used without checking it against the model's expected shard layout, or ranks run different orders or plans so their collectives mismatch", + "A faster H2D copy, local clone or rank-local filtered read is counted as a cross-rank communication gain", + "Serial and concurrent cache paths agree on identically corrupted placement bytes, or a contiguous fixture misses a separated-run alignment failure", + "Resident bytes or ordered per-k accumulation change while misses are pending, or an exit-zero SKIP/non-overlapped oracle is counted as overlap validation" + ], + "refusal_point": "Withholds a load-path win until the verification checklist holds and keeps the distributed-delivery checks as verification targets rather than results; does not force prefetch without a safe dependency window and disables it when neither a smaller size nor a moved position removes a regression; records unsupported save behaviors as unknown; rejects mismatched execution ABI or prepared-contract versions instead of editing their tags; and implies no Ascend support from the llama.cpp or fastsafetensors mechanisms; withholds expert-cache correctness/overlap conclusions when byte coverage, actual I/O/pool dispatch or model-oracle callback reachability is missing", + "hardware": "NPU for the device-side prefetch API (torch_npu.npu_prefetch, checked against the target CANN version) and the Diffusers offload playbook; the prefetch and fastsafetensors methodology transfers while the fastsafetensors and llama.cpp APIs do not, and cross-rank completion follows each collective backend's own wait contract" + }, + "recommendations": [ + "Report preparation/startup/readiness, warmups and measured work as separate numbers, with load latency split into init, header, io and get phases, the cold-cache protocol stated and the effective loading mode identical across compared runs.", + "Place each prefetch behind an enable_prefetch switch defaulting to False, start max_size at 50% of the theoretical size, verify a single operator first and step up to 0.7 and then 1.0 only after verified wins.", + "Try offload candidates in order of change surface: batch size, resolution, CFG/feature flags, CPU or layered offload, tensor-parallel degree, then activation checkpointing; re-estimate the un-overlapped transfer share after adopting each tier.", + "Record whole-set atomic publication, asynchronous checkpointing, multi-rank save coordination and crash durability as unknown for the fastsafetensors saver instead of assuming them.", + "Before adopting coalesced delivery, compare it disabled and enabled with identical checkpoint, rank mapping, backend, topology, dtype, cache state, chunk/queue budgets and consumer retention, then measure complete startup under the same accounting protocol.", + "For streamed expert-cache changes, compare against independent buffered tensor bytes, exercise separated-run geometry, check resident bytes during pending reads, preserve ordered per-k accumulation and verify that the numerical oracle selects the actual callbacks." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Normalized provenance covers the existing llama.cpp loading slice at 3e390af18de1131dda27913c00511725b4274c8a and the fastsafetensors distributed-delivery slice at 219e2561a41ede2d5d8eba611c248bb0effbf580; other inherited claims remain unnormalized with original identities and limits retained.", + "The readsets record static mechanisms and source recommendations, not validated startup, memory, placement or backend performance.", + "The fastsafetensors delivery readset does not revalidate inherited I/O/save claims or qualify uneven scatter, accelerator/NPU compatibility, inter-node transport or agent efficacy.", + "Flare adds source-only preparation/serving ownership and ABI/contract checks; live exporter handles, native backend execution, startup gains and model/dependency rights are not qualified.", + "The DeepSeek increment records source/test definitions only: its buffered byte oracle shares tensor metadata, its resident snapshot covers w1 bytes, and its tiny PyTorch/batched checks do not qualify the released model, backend performance or agent efficacy." + ], + "provenance_readsets": [ + "references/llama-loading-readset.md", + "references/fastsafetensors-transfer-readset.md", + "references/flare-preparation-boundary.md", + "references/deepseek-expert-cache-oracles.md", + "references/deepseek-expert-cache-source.json" + ], + "consumption_mode": "packaged" + } --- # Weight Loading and Startup Fixed-Overhead Optimization @@ -141,7 +294,7 @@ When weights/activations do not fit device memory: ## 6. Loading modes: mmap / mlock / direct I/O / lazy (llama.cpp pattern) For the division between loading, NUMA/thread placement and runtime-state diagnosis, -use the [llama.cpp owner map](../../repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope). +use the [llama.cpp owner map](../../../systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope). Source pins and static-inspection dates stay with their respective owners. At the pinned revision (3e390af) loading is one unified flag, diff --git a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-oracles.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/deepseek-expert-cache-oracles.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-oracles.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/deepseek-expert-cache-oracles.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-source.json b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/deepseek-expert-cache-source.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/deepseek-expert-cache-source.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/deepseek-expert-cache-source.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/fastsafetensors-transfer-readset.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/fastsafetensors-transfer-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/fastsafetensors-transfer-readset.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/fastsafetensors-transfer-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/flare-preparation-boundary.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/flare-preparation-boundary.md similarity index 92% rename from src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/flare-preparation-boundary.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/flare-preparation-boundary.md index 99058f3d6..f337ac210 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/flare-preparation-boundary.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/flare-preparation-boundary.md @@ -2,7 +2,7 @@ Source-derived candidate; empirical validation **not_run**. This adds one preparation-contract example to the existing loading owner, not another generic -loading skill. The [pinned source map](../../../repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md#preparation-readset) +loading skill. The [pinned source map](../../../../systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md#preparation-readset) defines the immutable files and symbols. No Flare installation or preparation was performed. @@ -57,7 +57,7 @@ conditions before attributing a startup change. "No Python in serving" describes the process boundary; it alone supplies no performance-causation evidence. For request-time prefix reuse, continue with the -[Flare session owner](../../../repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md). +[Flare session owner](../../../../systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md). Its KV chain must accompany recurrent/convolution/partial-tail continuation state. A prepared package is different from a per-session persisted checkpoint. @@ -69,7 +69,7 @@ qualification follows from these contracts. Its GDS transport remains deferred; preparation serialization is unrelated to a claim of direct-storage execution. The selected repository files are under the root Apache-2.0 -[license retained by the Flare owner](../../../repositories/flare-local-inference/audit-flare-session-lifecycle/LICENSE). +[license retained by the Flare owner](../../../../systems/engines/flare-local-inference/audit-flare-session-lifecycle/LICENSE). This notice covers source attribution, not a grant over model weights, tokenizer/assets or separate dependencies. Their own licenses and access conditions remain separate; none was imported or independently cleared here. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/llama-loading-readset.md b/src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/llama-loading-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/weight-loading-optimization/references/llama-loading-readset.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/optimization/weight-loading-optimization/references/llama-loading-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/SKILL.md similarity index 62% rename from src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/SKILL.md index 1260f6a36..b91d9bc4f 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/SKILL.md @@ -1,6 +1,48 @@ --- name: assess-serving-preemption description: Assess whether LLM serving queue delays justify token-boundary priority preemption, accounting for suspended KV, host swapping or recomputation, and long-request service guarantees. Use for admission-versus-service-order diagnosis, not current scheduler implementation or tenant-fairness certification. +metadata: + inferrsi: | + { + "status": "candidate", + "empirical_validation": "not_run", + "source_pins": "orca-osdi22-and-fastserve-2305.05920v3", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "applicability": { + "scope": "Diagnosing whether LLM serving queue delay justifies token-boundary priority preemption, including suspended KV, host swap or recomputation and long-request service guarantees; admission-versus-service-order diagnosis, not current scheduler implementation or tenant-fairness certification", + "prerequisites": "The packaged source notes are read before quoting a metric or comparing a baseline; installing, benchmarking, changing scheduling or admission, migrating state, editing a repository or running a reproduction needs a separately authorized task", + "input_contract": "The late boundary (first token, token gap, completion or a stated cohort's service) with its exact percentile or attainment target and denominator; model and KV format, engine revision, safe interruption boundary, request batch limit, input/output/context distributions, arrival bursts, device live/reserved/allocated bytes, active and suspended requests, host budget, transfer path, recovery time and queue/phase/service timings; per-tenant or priority-class obligations kept apart from aggregate latency", + "produced_artifact": "A read-only decision naming the observed failure boundary, the smallest candidate that addresses it, unchanged assumptions, missing evidence, resource cost, affected cohorts, counter-signal and next authorized gate, plus a proposed verification plan; not a validated configuration", + "failure_modes": [ + "An ordinary iteration boundary is taken to permit interrupting a prefill halfway through", + "Unknown output length is treated as known from the prompt", + "KV is counted for the running batch only, omitting started requests that are suspended", + "Reservations are removed because some reserved capacity is not live", + "Recompute-based eviction is proposed without a progress invariant, so two jobs can repeatedly displace each other", + "Host-device copies or same-host IPC are charged to Network instead of their local resource" + ], + "refusal_point": "Withholds a preemption recommendation when the apparent gain can be explained by different kernels, unmatched parallelism or an incomparable latency metric; unknown safe boundaries, unsupported state recovery, uncalibrated timing or unclear service obligations are blockers" + }, + "recommendations": [ + "Name the late boundary and its exact target and denominator before choosing among batching, admission, preemption and state recovery.", + "Send prefill interference to choose-prefill-decode-schedule and physical KV slack to diagnose-kv-block-efficiency when those candidates are available; otherwise report the diagnosed boundary and the missing evidence.", + "In a later authorized test, isolate service ordering first and state recovery second, and evaluate their interaction only after both are understood.", + "Weigh FastServe's same-engine FCFS comparison above its historical-vLLM headline, and do not read its P95 aggregate gain as a starvation or tenant-fairness guarantee." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged" + } --- # Assess serving preemption diff --git a/src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/assess-serving-preemption/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-serving-preemption/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-split-backward-schedule/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-split-backward-schedule/SKILL.md new file mode 100644 index 000000000..b8719d5b7 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-split-backward-schedule/SKILL.md @@ -0,0 +1,66 @@ +--- +name: assess-split-backward-schedule +description: Evaluate memory-capped split-backward and V-shaped pipeline scheduling proposals, preserving synchronous update, global-batch and optimizer-validation semantics. +metadata: + inferrsi: | + { + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "applicability": { + "scope": "Assessing memory-capped split-backward and V-shaped pipeline schedule proposals when a pipeline trace suggests input-gradient propagation waits behind deferrable weight-gradient work or an optimizer barrier, while preserving synchronous update, global-batch and optimizer-validation semantics", + "prerequisites": "The packaged evidence notes are read before citing the paper; installing or running frameworks, benchmarking schedules, changing training semantics, configuring hardware or modifying repositories needs separate authorization", + "input_contract": "Stage and microbatch counts, microbatch and global batch, precision, optimizer and clipping/NaN behavior; F, input-gradient B, weight-gradient W and transfer timings; per-pass retained and released bytes; total device peak and margin; collective ordering; kernel-splitting support; calculated bubble rate kept apart from measured idle time and useful samples/s", + "produced_artifact": "A candidate schedule and fallback with per-pass order rationale, peak-memory and transfer costs, unchanged semantic requirements, counter-signals and a future verification proposal; a reviewable plan, not an enabled schedule", + "failure_modes": [ + "Completion of B is equated with releasing all activations, so W inputs retained by deferral go uncounted", + "The equal-time zero-bubble construction replaces observed F/B/W and transfer times, or kernel-splitting and synchronization overhead is left out", + "The schedule is chosen by minimum modeled bubble rate instead of whole-step throughput under the memory and quality constraint", + "pMB activation capacity is read as total memory, hiding HBM, workspace, buffers or per-rank imbalance", + "Removing the optimizer barrier drops global norm, clipping or NaN decisions, rollback and re-update, or the correctness of work that depends on a tentative update", + "Optimizer post-validation is conflated with accepting stale-gradient delayed updates" + ], + "refusal_point": "Withholds an enablement recommendation under frequent clipping/NaNs, small available memory, strongly unequal passes, unsupported split kernels, hidden collectives or dynamic shapes, and retains the synchronized fallback while optimizer reversibility and rollback coverage are unverified" + }, + "recommendations": [ + "Compare ordinary 1F1B with a memory-capped split schedule first; add V-shaped two-chunk placement only when memory permits, and at matched global batch also try smaller microbatches with more microbatches.", + "Decide optimizer post-validation separately from the schedule change.", + "In a later authorized comparison, hold model, global batch and precision fixed and compare real with calculated idle intervals." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged" + } +--- + +# Assess a split-backward schedule + +Use when a pipeline trace suggests input-gradient propagation waits behind deferrable parameter-gradient work or an optimizer barrier. This is a candidate paper-derived assessment, not permission to change a training runtime. Read [evidence](references/source-notes.md). + +## Required evidence + +Obtain stage/microbatch counts, microbatch/global batch, precision, optimizer and clipping/NaN behavior; F, input-gradient B, weight-gradient W and transfer timings; per-pass retained/released bytes; total device peak/margin; collective ordering; and kernel-splitting support. Separate calculated bubble rate from measured idle time and useful samples/s. + +## Proposal + +- Identify the critical B dependencies and work W that can legally move after its corresponding B. Account for W inputs retained by deferral; do not equate completion of B with releasing all activations. +- Compare ordinary 1F1B with a memory-capped split schedule. Use observed F/B/W times and transfer costs rather than the equal-time zero-bubble construction. Include kernel-splitting and synchronization overhead. +- When memory permits, compare a larger activation cap with V-shaped two-chunk placement. At matched global batch also consider smaller microbatches with more microbatches. Keep total HBM, workspace, buffers and per-rank imbalance explicit; pMB denotes activation capacity, not total memory. +- Choose whole-step throughput under the memory/quality constraint, not minimum modeled bubble rate. Larger microbatches may beat lower bubbles because kernels use hardware better; more chunks can add communication. +- Keep optimizer post-validation separate. Removing a barrier must preserve global norm/clipping/NaN decisions, rollback/re-update, and correctness of work dependent on a tentative update. If actual optimizer reversibility and rollback coverage are unverified, retain the synchronized fallback. This mechanism is distinct from accepting stale-gradient delayed updates. + +## Output, checks and stopping point + +Return a candidate/fallback with per-pass order rationale, peak-memory and transfer costs, unchanged semantic requirements, counter-signals and a future verification proposal. A later authorized comparison should hold model, global batch and precision fixed and inspect numerical equivalence, full-step time, total peak bytes, exposed communication, rollback frequency/cost and real versus calculated idle intervals. + +Frequent clipping/NaNs, small available memory, strongly unequal passes, unsupported split kernels, hidden collectives or dynamic shapes are reasons to withhold an enablement recommendation. A paper's fixed-seed loss match is not universal convergence or rollback-stress validation. Stop at the reviewable plan; do not install/run frameworks, benchmark schedules, change training semantics, configure hardware or modify repositories without separate authorization. No current/NPU compatibility is established. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-split-backward-schedule/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/assess-split-backward-schedule/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/assess-split-backward-schedule/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/SKILL.md similarity index 63% rename from src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/SKILL.md index 2217cc391..a4760157f 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/SKILL.md @@ -3,6 +3,55 @@ name: checkpoint-recovery-consistency description: 'Assess distributed checkpoint recovery using DLRover memory agreement, persistence completion and replica transport, and NVIDIA Resiliency Extension complete-iteration selection. Distinguish worker, host and replica loss; separate queued saves, publication, application-state validity and durable recovery. ' +metadata: + inferrsi: | + { + "tags": [ + "checkpoint", + "recovery", + "distributed", + "shared-memory", + "replication" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Assessing distributed checkpoint recovery through DLRover memory agreement, persistence completion and replica transport, and NVIDIA Resiliency Extension complete-iteration selection; distinguishes worker, host and replica loss and separates queued saves, publication, application-state validity and durable recovery. Save performance belongs to dcp-async-save-performance", + "prerequisites": "The pinned sources and limits are read; reading authorizes no launchers, failure injection or checkpoint loading, and restore evidence needs separate authorization", + "input_contract": "The declared failure domain; actual memory, storage and replica placement and helper lifetime; step or iteration identity, the expected shard/rank set and finalization result; the application's model, optimizer and caller-required RNG/scheduler/data-position state contract; the target environment's format and world-size/replication assumptions", + "produced_artifact": "An answer to each of the four acceptance questions (snapshot survival, cross-participant completeness, application-state completeness, restorability) at its own evidence level, a proposed per-generation recovery evidence record including errors and skipped saves, and a proposed verification split by worker, host, replica and persistent-storage loss; no recovery success, durability, NPU support or measured gain", + "failure_modes": [ + "Equal steps or complete rank IDs are taken to verify tensor contents or whole application semantics", + "A skipped save, where not every participant acquired shared-memory access, is reported as a new recovery point", + "A returned step paired with an empty state dictionary is read as a successful restore", + "A local maximum filename or one clean shard substitutes for global coverage", + "The .dirty-then-rename publication is read as power-loss durability", + "An asynchronous save request is treated as a completed checkpoint, or queue concurrency is assumed unlimited", + "One failure domain is tested and recovery from all of them is claimed" + ], + "refusal_point": "Claims no recovery success, durability, NPU support or measured gain, does not upgrade allocation, metadata or a completed collective into proven recovery, and neither injects failures nor authorizes them" + }, + "recommendations": [ + "Answer the four acceptance questions separately; record each at its own evidence level.", + "Count replication as added availability only when copies occupy independent failure domains, and budget clean, dirty and replicated shards on each host.", + "Keep the one-outstanding asynchronous save restriction until the target queue version is checked.", + "Send save-throughput questions to dcp-async-save-performance; a faster return or higher write throughput does not establish recoverability." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/sources.md" + ], + "consumption_mode": "packaged" + } --- # Checkpoint Recovery Consistency diff --git a/src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/dlrover-persistence-replicas.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/sources.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/checkpoint-recovery-consistency/references/sources.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/SKILL.md similarity index 50% rename from src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/SKILL.md index ecba7dd33..ab08d9e9b 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/SKILL.md @@ -1,6 +1,46 @@ --- name: choose-parallel-training-plan description: Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for memory, topology, resharding and planning cost. +metadata: + inferrsi: | + { + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "applicability": { + "scope": "Comparing stage, device-mesh and operator-sharding candidates for static distributed training when heterogeneous stage memory or work, or communication cost, makes a uniform data/tensor/pipeline plan suspect; derived from historical Alpa, not a current compiler configuration guide", + "prerequisites": "The packaged source notes are read; the skill authorizes no compiler or solver install or run, parallelism reconfiguration, repository edit, hardware task or benchmark", + "input_contract": "Model operator graph and shapes; global batch, microbatch and accumulation; precision and optimizer; physical links and contention; candidate shard layouts; stage latency, workspace and in-flight activation memory; the objective and expected run length. A device count without topology is insufficient", + "produced_artifact": "The best supported candidate and a simpler fallback with a memory and critical-path ledger, planning cost and amortization, semantic invariants, counter-signals, missing evidence and a proposed matched comparison; a plan, not a reconfiguration", + "failure_modes": [ + "Equal arithmetic work is taken to mean equal kernel time, carrying over Alpa's zero compute-cost simplification", + "Mean memory replaces the largest per-rank peak with multiple in-flight activation sets", + "Inter-stage transfer and microbatch count stay outside the objective, as in historical Alpa, without the gap being reported", + "Repeated slow-link sends to replicated destination tiles are not compared with one cross-mesh transfer plus local allgather", + "Inter-node Network traffic is merged with intra-node accelerator links", + "Static cost estimates are applied to dynamic shapes, asymmetric devices or changing communication contention" + ], + "refusal_point": "Rejects infeasible stage/mesh pairs, claims no global optimality outside the declared search space, and stops at the plan in paper-only mode" + }, + "recommendations": [ + "Keep a simple reference plan with the same training semantics and global batch as the comparison point.", + "Separate fixed-model scaling from model-size weak scaling before quoting a speedup; Alpa's 9.7x MoE ratio belongs to a specific weak-scaling workload, baseline and 25-Gbps cluster.", + "Weigh planning cost against the expected run length; a shorter job may not amortize it." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged" + } --- # Choose a parallel training plan diff --git a/src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/choose-parallel-training-plan/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md similarity index 65% rename from src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md index ae440778b..01190818d 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/SKILL.md @@ -1,6 +1,47 @@ --- name: evaluate-elastic-serving-placement description: Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including state-copy, reservation, recomputation, interruption, and priority costs. Use for cross-instance memory imbalance or highly variable long-context serving, not connector wiring or live scaling. +metadata: + inferrsi: | + { + "status": "candidate", + "empirical_validation": "not_run", + "source_pins": "llumnix-2406.03243v1-and-loongserve-2404.09526v2", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "applicability": { + "scope": "Comparing migration of a running LLM request between compatible model replicas with changing its sequence-parallel execution group, including state-copy, reservation, recomputation, interruption and priority costs, for cross-instance memory imbalance or highly variable long-context serving; not connector wiring or live scaling", + "prerequisites": "The relevant Llumnix or LoongServe branch and its readset in the packaged source notes are read; moving state, altering admission or priority, scaling replicas, changing parallelism, running experiments or editing frameworks needs separate authorization", + "input_contract": "Exact model and cache identity and format; weight residency; request phase, context and generated length; source and destination KV bytes; physical free/reserved memory and virtual accounting; batch state; current parallel group; queue and service times; actual topology and transport for each path; the target SLO statistic and affected priority or tenant cohorts", + "produced_artifact": "The smallest justified branch with missing capability or evidence, expected saved work or wait, added state, traffic and replica cost, affected SLO and cohorts, counter-signals, the proposed next gate and a future verification plan; an evidence-based recommendation, not a validated deployment rule", + "failure_modes": [ + "Model-weight repartitioning, pipeline configuration, instance migration and sequence sharding are treated as interchangeable uses of parallelism", + "Aggregate free bytes are taken to mean a single blocked request fits, conflating internal KV-block slack with cross-instance fragmentation", + "Llumnix's 20-30 ms measured downtime is substituted for total transfer time", + "Virtual headroom is counted as extra memory or as a tenant-fairness guarantee", + "A zero-overhead transition label is read as every group change being free, or a token sum hides length-sensitive attention work", + "Ongoing distributed-attention communication goes uncharged because existing KV does not migrate" + ], + "refusal_point": "Withholds the candidate when every destination is full, pre-copy catch-up is unproven, model or cache compatibility is unknown, or the measured work saved does not exceed exposed costs; current-code review and target-system tests remain a later authorization gate" + }, + "recommendations": [ + "Decide first whether the question is replica relocation, a sequence-parallel group change, or prefix reuse and phase pools, which evaluate-kv-cache-transfer-gates and choose-prefill-decode-schedule cover when available.", + "Name each transfer path separately; only inter-node transport is Network.", + "Keep a feasible static, no-migration baseline in any later comparison." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged" + } --- # Evaluate elastic serving placement diff --git a/src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/evaluate-elastic-serving-placement/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/SKILL.md similarity index 64% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/SKILL.md index b68bbccdd..4e2801984 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/SKILL.md @@ -3,6 +3,101 @@ name: optimization-search-policy description: 'Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate generation, exploration vs exploitation, diversity, negative experiment memory, budget allocation, and stopping criteria. Drawn from vllm-evolve (phase-locked optimization round), TRT-LLM casebook (prior-art matching), and SkillOpt (bounded edit + gate). Use when deciding what to try next and when to stop. ' +metadata: + inferrsi: | + { + "tags": [ + "optimization", + "search", + "policy", + "experiment-design" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Selecting, ordering and stopping optimization experiments: candidate generation, exploration versus exploitation, mechanism diversity, negative experiment memory, cost-aware escalation, budget allocation and stopping criteria, drawn from vllm-evolve, the TRT-LLM casebook and SkillOpt", + "prerequisites": "The useful work, quality contract, allowed approximations, timing boundary and resource budget are frozen before search, and a complete restartable incumbent is preserved; reading the procedures authorizes no execution, framework change, new resource spending or publication", + "input_contract": "Profiling data, existing config, prior experiment and negative records and the casebook for the current bottleneck class, model and hardware; each target stage's fraction of E2E time; the target, locally measured uncertainty and the authorized budget", + "produced_artifact": "An ordered experiment plan: one candidate change per round with an explicit hypothesis and its motivating evidence, a bounded shortlist of distinct mechanisms with assumptions, uncertainty, expected impact and evaluation cost, negative records turned into search constraints, and a stop-or-switch decision; when authorized rounds run, a recorded verdict per round with best-seen and delivered outcomes kept apart", + "failure_modes": [ + "The evaluation protocol, measurement setup or baseline changes in the same round as the candidate", + "Benchmark budget is spent on a candidate that fails correctness", + "A matching keyword in a rejection record is used as a causal diagnosis or automatic veto, or one failed implementation excludes a whole solution family", + "Expensive work starts on a stage without computing its Amdahl contribution, so a large local speedup yields little E2E gain", + "One mechanism family keeps being sampled only because budget remains, without re-profiling after the limiting stage moves", + "Roofline proximity or a utilization threshold is used as a universal stopping rule", + "Screening is allowed to skip the frozen acceptance regimes or required service-level evidence" + ], + "refusal_point": "Discards a correctness-breaking, worse or inconclusive change instead of integrating it, preserves no-change when candidates cannot be separated, does not count a shortcut outside the authorized semantics as an optimization, and does not let a successful probe qualify a broken final state" + }, + "recommendations": [ + "Resolve acceptance criteria before search: retain supplied criteria, otherwise use performance-ceiling-analysis to derive targets, record provenance and freeze supported gates.", + "Before generating a candidate, consult the casebook, the negative record and the target stage's Amdahl contribution.", + "Record every worse or inconclusive result with hypothesis, metric, context, evidence level and failure type, and derive a search constraint from it.", + "When about three experiments on one hypothesis return inconclusive from measurement variance, fix the protocol or move to a different hypothesis.", + "Before reporting, restore and verify the best fully valid incumbent through its canonical launcher." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/papers-20261015/INTEGRATION_NOTES.md", + "references/papers-20261015/knowledge/adaptation-interfaces.md", + "references/papers-20261015/MECHANISMS.md", + "references/papers-20261015/NEXT_GATE_LEADS.md", + "references/papers-20261015/references/source-notes.md", + "references/papers-20261015/updates/optimization-search-policy.md", + "cases/sarathi-chunk-budget.json", + "references/distserve-phase-placement/README.md", + "references/distserve-phase-placement/source-record.json" + ], + "references": [ + "../../../casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md", + "../../../casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json", + "../../../casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md", + "../../../casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json", + "../../../casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md", + "../../../casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.json", + "../../../workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md", + "cases/sarathi-chunk-budget.md", + "cases/sarathi-chunk-budget.json", + "../../../systems/engines/vllm/diagnose-vllm-admission-pressure/SKILL.md", + "../assess-serving-preemption/SKILL.md", + "../../measurement/calibrate-serving-simulation/SKILL.md", + "../../measurement/performance-ceiling-analysis/SKILL.md", + "../../../systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md", + "../../measurement/calibrate-serving-simulation/references/infera-serving-search/README.md", + "../../measurement/calibrate-serving-simulation/references/infera-serving-search/source-record.json" + ], + "consumption_mode": "packaged", + "sources": [ + { + "type": "paper", + "id": "usenix:osdi24:zhong-yinmin", + "version": "OSDI 24 proceedings, July 2024, pages 193-210", + "url": "https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin", + "sections": [ + "3.3", + "4.1-4.2", + "6.1-6.5", + "7", + "Appendices A-C" + ], + "readset": "references/distserve-phase-placement/source-record.json" + }, + { + "type": "unknown", + "scope": "Pre-existing sources remain in their original procedure/readsets; this amendment does not normalize that inventory" + } + ] + } --- # Optimization Search Policy @@ -65,7 +160,7 @@ After Phase 7, return to Phase 1 with the updated context. ## Before candidate search Resolve the acceptance contract using the -[acceptance-criteria procedure](../performance-ceiling-analysis/references/acceptance-criteria.md): +[acceptance-criteria procedure](../../measurement/performance-ceiling-analysis/references/acceptance-criteria.md): use user-supplied criteria; otherwise derive targets with `performance-ceiling-analysis`. Preserve partial requirements, record provenance and freeze the resolved gates before search. Candidate outcomes cannot set or @@ -85,9 +180,9 @@ Before generating a new candidate, consult: in this configuration? If yes, the burden of justification is higher — something about the conditions must have changed to try again. - InferRSI's source-linked options are the [Qwen B300 rejection case](../inference-optimization-casebook/cases/qwen-b300-rejected-directions.md), - [Ascend reported rejection case](../inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md) - and [Drama3 diagnostic case](../inference-optimization-casebook/cases/drama3-rejected-directions.md). + InferRSI's source-linked options are the [Qwen B300 rejection case](../../../casebooks/inference/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md), + [Ascend reported rejection case](../../../casebooks/inference/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.md) + and [Drama3 diagnostic case](../../../casebooks/inference/inference-optimization-casebook/cases/drama3-rejected-directions.md). First distinguish quality rejection, no demonstrated gain, resource failure, confounded control, unintegrated lead and analytical rejection. Match the source revision, workload, device and evidence level before drawing a lesson. @@ -138,7 +233,7 @@ constraint from it. Example: "KV cache compression at this sequence length did not reduce memory traffic (measured: mte2_ratio unchanged at concurrency=4)." That constraint narrows the remaining search space. -For a serving-tuner shortlist, follow the calibration owner's [Infera provenance reference](../calibrate-serving-simulation/references/infera-serving-search/README.md). Keep the deterministic seed-only comparator, complete evaluated-trial ledger and actual fidelity visible. That reference qualifies the scalar/DES metric boundary and the proposed, unrun trace-ranking check. +For a serving-tuner shortlist, follow the calibration owner's [Infera provenance reference](../../measurement/calibrate-serving-simulation/references/infera-serving-search/README.md). Keep the deterministic seed-only comparator, complete evaluated-trial ledger and actual fidelity visible. That reference qualifies the scalar/DES metric boundary and the proposed, unrun trace-ranking check. ## Cost-aware candidate selection @@ -148,9 +243,9 @@ For observed prefill interference with ongoing decodes, consult the Account for decode tokens before partial and new prefills; preserve TTFT, token-gap latency, queueing and prefill-efficiency tradeoffs. This source-derived candidate supplies neither a universal budget nor current-engine validation. -Use [admission diagnosis](../../repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md), +Use [admission diagnosis](../../../systems/engines/vllm/diagnose-vllm-admission-pressure/SKILL.md), [preemption assessment](../assess-serving-preemption/SKILL.md) and -[serving-simulation calibration](../calibrate-serving-simulation/SKILL.md) +[serving-simulation calibration](../../measurement/calibrate-serving-simulation/SKILL.md) for their existing responsibilities. Not all experiments cost the same. Order candidates by cost-adjusted expected @@ -210,8 +305,8 @@ cost, not a paper's fixed percentage. Restore and verify the best fully valid incumbent through its canonical launcher. Report best-seen and delivered outcomes separately; a successful probe does not qualify a broken final state. -The [experiment-governance procedure](../performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md) -and [decision cases](../performance-ceiling-analysis/references/papers-20261015/cases/decision-cases.md) +The [experiment-governance procedure](../../measurement/performance-ceiling-analysis/references/papers-20261015/govern-optimization-experiments/procedure.md) +and [decision cases](../../measurement/performance-ceiling-analysis/references/papers-20261015/cases/decision-cases.md) retain the reviewed A-pack mechanism, costs and limits. InferenceBench's within-vLLM matched two-hour comparison is separate from its grid of independent per-engine two-hour searches. None is an InferRSI measurement. @@ -275,7 +370,7 @@ new resource spending or publication. When budget/stop reasoning depends on a Goal's current authority or a repeated settlement, follow the existing -[RSI evidence owner's LoopX reference](../../workflows/rsi-evidence/references/loopx-goal-governance/README.md). +[RSI evidence owner's LoopX reference](../../../workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md). Use its source/decision/receipt distinctions before selecting another experiment. That owner retains the mechanism and counterexamples; this search policy does not duplicate its accounting rules or turn a local quota into a provider cost diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/cases/sarathi-chunk-budget.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/cases/sarathi-chunk-budget.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/cases/sarathi-chunk-budget.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/cases/sarathi-chunk-budget.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/README.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/distserve-phase-placement/README.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/README.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/distserve-phase-placement/README.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/source-record.json b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/distserve-phase-placement/source-record.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/distserve-phase-placement/source-record.json rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/distserve-phase-placement/source-record.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/INTEGRATION_NOTES.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/INTEGRATION_NOTES.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/INTEGRATION_NOTES.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/INTEGRATION_NOTES.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/MECHANISMS.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/MECHANISMS.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/MECHANISMS.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/MECHANISMS.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/NEXT_GATE_LEADS.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/NEXT_GATE_LEADS.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/NEXT_GATE_LEADS.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/NEXT_GATE_LEADS.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/knowledge/adaptation-interfaces.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/knowledge/adaptation-interfaces.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/knowledge/adaptation-interfaces.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/knowledge/adaptation-interfaces.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/updates/optimization-search-policy.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/updates/optimization-search-policy.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/references/papers-20261015/updates/optimization-search-policy.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy/references/papers-20261015/updates/optimization-search-policy.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/SKILL.md similarity index 51% rename from src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/SKILL.md index 582cfc32e..b37ef27b2 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/SKILL.md @@ -1,6 +1,47 @@ --- name: plan-activation-residency description: Propose training activation and gradient memory reductions by distinguishing lifetime reordering, address packing and selective rematerialization from supplied evidence. +metadata: + inferrsi: | + { + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "applicability": { + "scope": "Proposing training activation and gradient memory reductions when pressure appears to come from live tensors or allocator-reserved memory, by distinguishing lifetime reordering, address packing and selective rematerialization; covers activation checkpointing for recomputation, not persistent recovery checkpoints", + "prerequisites": "The packaged evidence and limits are read before citing paper results; installing or running a solver or framework, modifying allocation or checkpoint policies, reproducing workloads or provisioning hardware needs separate authorization", + "input_contract": "The actual forward/backward/update DAG, shapes and precision, tensor sizes and consumer lifetimes; resident versus reserved peak bytes; operator, workspace and temporary-output peaks; batch and accumulation policy; optimizer and collective barriers; representative compute and transfer costs; whether the goal is fit, step time or time to a fixed training objective", + "produced_artifact": "A preferred candidate, simpler fallback, resource ledger, critical-path prediction, assumptions, counter-signals and unresolved-evidence list, with source claims kept apart from proposed decisions, plus a proposed matched comparison; a reviewable proposal, not an implementation", + "failure_modes": [ + "Fragmentation and live-byte excess are not distinguished at the same phase", + "OLLA address packing is read as tensor placement across CPU/SSD tiers", + "Reordering moves a release or update past a dependency such as shared weights, accumulation, gradient clipping, optimizer state or collectives", + "A static packing plan ignores alignment, aliasing, workspace, asynchronous overlap or shape variability", + "Added recomputation consumes the compute window that previously hid transfers", + "Checkmate's rounding is taken to guarantee the memory cap, or its 10% allowance is used as a portable safety margin", + "A solver timeout is read as proof of infeasibility" + ], + "refusal_point": "Returns the missing classification instead of a preferred checkpoint interval when only total OOM memory is known, and escalates when a legal dependency graph, memory margin or correctness model is unavailable" + }, + "recommendations": [ + "Where ordering is legal, try earlier last-use and gradient release before adding computation or transfer.", + "Check dependency validity and memory feasibility separately for an approximate schedule.", + "Account for planning cost over the expected number of repeated steps." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged" + } --- # Plan activation residency diff --git a/src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/plan-activation-residency/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/plan-activation-residency/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/SKILL.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/SKILL.md similarity index 62% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/SKILL.md index 60640857a..b31c7ee1c 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/SKILL.md @@ -3,6 +3,64 @@ name: training-resource-budget description: 'Choose training offload and resource-budget changes using pinned DeepSpeed configuration, attributed expert diagnosis and a historical MindSpeed case. Balance accelerator capacity, CPU work, pinned RAM and local storage; keep host/device transfer exposure distinct from collective network bottlenecks. ' +metadata: + inferrsi: | + { + "tags": [ + "training", + "offload", + "resource-budget", + "memory", + "cpu", + "numa" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Choosing training offload and resource-budget changes from pinned DeepSpeed configuration, attributed expert diagnosis and a historical MindSpeed case, balancing accelerator capacity, CPU work, pinned RAM and local storage while keeping host/device transfer exposure distinct from collective network bottlenecks", + "prerequisites": "The source boundaries are read; garbage-collection or affinity changes, privileged memlock changes and delayed parameter updates each need separate approval, and the skill authorizes no newest-wheel installation, scheduler-priority change, benchmark or failure injection; historical MindSpeed reuse requires compatible allocator symbols, tracked swappable storage, sufficient host capacity and verified device/NUMA/CPU mapping, not merely the presence of npu-smi and lspci", + "input_contract": "State attributed to parameters, optimizer state, gradients or activations with its current memory tier, lifetime and next consumer; whether the failure is allocation capacity or throughput; a per-host budget for input buffers, checkpoint snapshots, offloaded state, OS cache and CPU workers with the actual ranks sharing the host; the host/device link recorded apart from collective traffic; the exact source version and enabled execution mode", + "produced_artifact": "A candidate decision record per proposal: the observed limiting interval and owning state or resource, source version and execution mode, capacity released and resources consumed with units and uncertainty, overlap readiness and mutation boundaries with expected contention, and the fixed-workload step and correctness evidence required for later validation", + "failure_modes": [ + "A change that lets the model fit is presented as a speedup, although it can slow each step", + "A tensor copied to CPU is taken as an InfiniBand/RoCE bottleneck, or CPU/NVMe offload is counted as network evidence", + "Parameter max_in_cpu is read as a byte or total-host-memory cap instead of an element count", + "NVMe optimizer placement is assumed to remove CPU optimizer demand", + "The parameter buffer-pool count is transplanted to the optimizer, or all optimizers are assumed to hold identical states", + "Configuration presence or schema documentation is taken as evidence that a knob or overlap actually ran", + "Ordinary overlap is taken as permission for delayed, stale parameter updates" + ], + "refusal_point": "Claims no hardware validation, measured gain or current NPU compatibility, keeps source support and extracted coverage separate from measured improvement, and does not reuse the MindSpeed headline gain; historical source observations do not establish current-stack compatibility, whole-model correctness or measured benefit" + }, + "recommendations": [ + "Attribute the state and separate a capacity failure from a throughput bottleneck before choosing an offload surface.", + "Verify the consumer of buffer_size before calculating bytes or tuning it, and measure the executing path before claiming a knob ran.", + "Compare the whole step's critical path, not host capacity alone.", + "Use plan-activation-residency, choose-parallel-training-plan and assess-split-backward-schedule for live-range, parallel-plan and backward-ordering questions." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/mindspeed-historical.md", + "references/mindspeed-source-readset.md", + "references/papers-20261015/cases/paper-grounded-cases.md", + "references/papers-20261015/INTEGRATION_NOTES.md", + "references/papers-20261015/knowledge/offload-delta.md", + "references/papers-20261015/notes/training-memory-scheduling.md", + "references/papers-20261015/references/source-readset.md", + "references/sources.md", + "references/stas-decision-notes.md" + ], + "consumption_mode": "packaged" + } --- # Training Cross-Resource Budget Decisions diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/mindspeed-historical.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/mindspeed-historical.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/mindspeed-historical.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/mindspeed-historical.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/mindspeed-source-readset.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/mindspeed-source-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/mindspeed-source-readset.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/mindspeed-source-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/INTEGRATION_NOTES.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/INTEGRATION_NOTES.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/INTEGRATION_NOTES.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/INTEGRATION_NOTES.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/cases/paper-grounded-cases.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/cases/paper-grounded-cases.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/cases/paper-grounded-cases.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/cases/paper-grounded-cases.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/knowledge/offload-delta.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/knowledge/offload-delta.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/knowledge/offload-delta.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/knowledge/offload-delta.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/notes/training-memory-scheduling.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/notes/training-memory-scheduling.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/notes/training-memory-scheduling.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/notes/training-memory-scheduling.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/papers-20261015/references/source-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/sources.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/sources.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/sources.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/sources.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/stas-decision-notes.md b/src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/stas-decision-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/training-resource-budget/references/stas-decision-notes.md rename to src/inferrsi/knowledge/data/skill_library/playbooks/planning/training-resource-budget/references/stas-decision-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/records/README.md b/src/inferrsi/knowledge/data/skill_library/records/README.md new file mode 100644 index 000000000..0d2315255 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/records/README.md @@ -0,0 +1,9 @@ +# Engineering records + +An engineering finding belongs in one `.json` file here. The ID encodes repository, backend, component and entry. Keep the source revision, applicability, verification state, limitations and negative results with the claim. The tier field expresses the source relationship (contribution, dependency or peer). Existing schema-1 contribution records may omit the tier. + +Use `inferrsi knowledge search --tier peer --json` to browse a tier and `inferrsi knowledge show ` to open the complete record. A procedure belongs with its own SKILL.md, metadata and resources. The catalog builds packaged-skill views from those owners. No per-record navigation list needs editing. + +## Adding or updating a record + +Choose a stable ID, pin the source and describe the observed scope. Separate an upstream report from a locally verified result. Include the environment and comparison needed to transfer the finding. Verify the record with `knowledge show` and search using its repository, backend and scenario. Frozen run output belongs under the repository's `evidence/` directory. diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/cache_dit/common/diffusion/residual-cache-calibration.json b/src/inferrsi/knowledge/data/skill_library/records/cache_dit.common.diffusion.residual_cache_calibration.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/cache_dit/common/diffusion/residual-cache-calibration.json rename to src/inferrsi/knowledge/data/skill_library/records/cache_dit.common.diffusion.residual_cache_calibration.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/deepinfer/common/llm/bounded-windowed-replay.json b/src/inferrsi/knowledge/data/skill_library/records/deepinfer.common.llm.bounded_windowed_replay.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/deepinfer/common/llm/bounded-windowed-replay.json rename to src/inferrsi/knowledge/data/skill_library/records/deepinfer.common.llm.bounded_windowed_replay.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/deepinfer/common/llm/speculative-verification-budget.json b/src/inferrsi/knowledge/data/skill_library/records/deepinfer.common.llm.speculative_verification_budget.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/deepinfer/common/llm/speculative-verification-budget.json rename to src/inferrsi/knowledge/data/skill_library/records/deepinfer.common.llm.speculative_verification_budget.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/deepinfer/common/llm/zero-overhead-deferred-scheduling.json b/src/inferrsi/knowledge/data/skill_library/records/deepinfer.common.llm.zero_overhead_deferred_scheduling.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/deepinfer/common/llm/zero-overhead-deferred-scheduling.json rename to src/inferrsi/knowledge/data/skill_library/records/deepinfer.common.llm.zero_overhead_deferred_scheduling.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/diffusers/cuda/diffusion/group-offload-transfer-overlap.json b/src/inferrsi/knowledge/data/skill_library/records/diffusers.cuda.diffusion.group_offload_transfer_overlap.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/diffusers/cuda/diffusion/group-offload-transfer-overlap.json rename to src/inferrsi/knowledge/data/skill_library/records/diffusers.cuda.diffusion.group_offload_transfer_overlap.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/flashattention/cuda/attention/generation-specific-dispatch.json b/src/inferrsi/knowledge/data/skill_library/records/flashattention.cuda.attention.generation_specific_dispatch.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/flashattention/cuda/attention/generation-specific-dispatch.json rename to src/inferrsi/knowledge/data/skill_library/records/flashattention.cuda.attention.generation_specific_dispatch.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/flashinfer/cuda/llm/paged-decode-dispatch.json b/src/inferrsi/knowledge/data/skill_library/records/flashinfer.cuda.llm.paged_decode_dispatch.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/flashinfer/cuda/llm/paged-decode-dispatch.json rename to src/inferrsi/knowledge/data/skill_library/records/flashinfer.cuda.llm.paged_decode_dispatch.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/lmdeploy/cuda/llm/kv-quantization-capacity-cost.json b/src/inferrsi/knowledge/data/skill_library/records/lmdeploy.cuda.llm.kv_quantization_capacity_cost.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/lmdeploy/cuda/llm/kv-quantization-capacity-cost.json rename to src/inferrsi/knowledge/data/skill_library/records/lmdeploy.cuda.llm.kv_quantization_capacity_cost.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/cross-stream-dependency.json b/src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.cross_stream_dependency.json similarity index 97% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/cross-stream-dependency.json rename to src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.cross_stream_dependency.json index a361cbb48..7e7ccddbf 100644 --- a/src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/cross-stream-dependency.json +++ b/src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.cross_stream_dependency.json @@ -8,7 +8,7 @@ "license": "MIT", "attribution": "Copyright © 2023 Apple Inc. Source observations are paraphrased; no implementation code is copied.", "skill_documents": { - "original": "data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" + "original": "data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" }, "verification": { "locally_verified": false, diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/evaluation-completion.json b/src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.evaluation_completion.json similarity index 97% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/evaluation-completion.json rename to src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.evaluation_completion.json index 3ee155d5b..464cc6fd2 100644 --- a/src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/evaluation-completion.json +++ b/src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.evaluation_completion.json @@ -8,7 +8,7 @@ "license": "MIT", "attribution": "Copyright © 2023 Apple Inc. Source observations are paraphrased; no implementation code is copied.", "skill_documents": { - "original": "data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" + "original": "data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" }, "verification": { "locally_verified": false, diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/post-detachment-buffer-lifetime.json b/src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.post_detachment_buffer_lifetime.json similarity index 97% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/post-detachment-buffer-lifetime.json rename to src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.post_detachment_buffer_lifetime.json index 4277d7852..21e9f6ae5 100644 --- a/src/inferrsi/knowledge/data/knowledge_packs/dependency/mlx/mlx/execution/post-detachment-buffer-lifetime.json +++ b/src/inferrsi/knowledge/data/skill_library/records/mlx.mlx.execution.post_detachment_buffer_lifetime.json @@ -8,7 +8,7 @@ "license": "MIT", "attribution": "Copyright © 2023 Apple Inc. Source observations are paraphrased; no implementation code is copied.", "skill_documents": { - "original": "data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" + "original": "data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" }, "verification": { "locally_verified": false, diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/pytorch/cuda/compiler/cudagraph-shapes-and-lifetimes.json b/src/inferrsi/knowledge/data/skill_library/records/pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/pytorch/cuda/compiler/cudagraph-shapes-and-lifetimes.json rename to src/inferrsi/knowledge/data/skill_library/records/pytorch.cuda.compiler.cudagraph_shapes_and_lifetimes.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/qwen3_tts_engine/ascend/audio/codec-constants-and-state.json b/src/inferrsi/knowledge/data/skill_library/records/qwen3_tts_engine.ascend.audio.codec_constants_and_state.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/qwen3_tts_engine/ascend/audio/codec-constants-and-state.json rename to src/inferrsi/knowledge/data/skill_library/records/qwen3_tts_engine.ascend.audio.codec_constants_and_state.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/qwen3_tts_engine/ascend/audio/fixed-customvoice-prompt.json b/src/inferrsi/knowledge/data/skill_library/records/qwen3_tts_engine.ascend.audio.fixed_customvoice_prompt.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/qwen3_tts_engine/ascend/audio/fixed-customvoice-prompt.json rename to src/inferrsi/knowledge/data/skill_library/records/qwen3_tts_engine.ascend.audio.fixed_customvoice_prompt.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/qwen3_tts_engine/ascend/audio/graph-batching-state-ownership.json b/src/inferrsi/knowledge/data/skill_library/records/qwen3_tts_engine.ascend.audio.graph_batching_state_ownership.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/qwen3_tts_engine/ascend/audio/graph-batching-state-ownership.json rename to src/inferrsi/knowledge/data/skill_library/records/qwen3_tts_engine.ascend.audio.graph_batching_state_ownership.json diff --git a/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/parallelism-and-extension.json b/src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.parallelism_and_extension.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/parallelism-and-extension.json rename to src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.parallelism_and_extension.json index 0c10d027f..355d2fee2 100644 --- a/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/parallelism-and-extension.json +++ b/src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.parallelism_and_extension.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/sglang/sub-skills/sglang-parallel-extension/SKILL.md" + "en": "data/skill_library/systems/engines/sglang/sub-skills/sglang-parallel-extension/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/runtime-radix-hicache.json b/src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.runtime_radix_hicache.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/runtime-radix-hicache.json rename to src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.runtime_radix_hicache.json index eb5b802ca..ca486baed 100644 --- a/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/runtime-radix-hicache.json +++ b/src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.runtime_radix_hicache.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/sglang/sub-skills/sglang-runtime-memory/SKILL.md" + "en": "data/skill_library/systems/engines/sglang/sub-skills/sglang-runtime-memory/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/tuning-and-validation.json b/src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.tuning_and_validation.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/tuning-and-validation.json rename to src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.tuning_and_validation.json index a7f093fc3..e9909406a 100644 --- a/src/inferrsi/knowledge/data/repo_packs/sglang/knowledge/common/llm/tuning-and-validation.json +++ b/src/inferrsi/knowledge/data/skill_library/records/sglang.common.llm.tuning_and_validation.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/sglang/sub-skills/sglang-tuning-validation/SKILL.md" + "en": "data/skill_library/systems/engines/sglang/sub-skills/sglang-tuning-validation/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/sglang/cuda/llm/hierarchical-prefix-cache.json b/src/inferrsi/knowledge/data/skill_library/records/sglang.cuda.llm.hierarchical_prefix_cache.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/sglang/cuda/llm/hierarchical-prefix-cache.json rename to src/inferrsi/knowledge/data/skill_library/records/sglang.cuda.llm.hierarchical_prefix_cache.json diff --git a/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/audio/serving-and-streaming-quality.json b/src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.audio.serving_and_streaming_quality.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/audio/serving-and-streaming-quality.json rename to src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.audio.serving_and_streaming_quality.json index a00aa9560..6af12af12 100644 --- a/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/audio/serving-and-streaming-quality.json +++ b/src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.audio.serving_and_streaming_quality.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-serving/SKILL.md" + "en": "data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-serving/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/omni/extension-lifecycle-validation.json b/src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.omni.extension_lifecycle_validation.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/omni/extension-lifecycle-validation.json rename to src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.omni.extension_lifecycle_validation.json index 643a001a8..00f6fec1a 100644 --- a/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/omni/extension-lifecycle-validation.json +++ b/src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.omni.extension_lifecycle_validation.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-extension/SKILL.md" + "en": "data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-extension/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/omni/pipeline-state-and-transport.json b/src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.omni.pipeline_state_and_transport.json similarity index 96% rename from src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/omni/pipeline-state-and-transport.json rename to src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.omni.pipeline_state_and_transport.json index 629a8f36f..bdf91b58b 100644 --- a/src/inferrsi/knowledge/data/repo_packs/sglang_omni/knowledge/common/omni/pipeline-state-and-transport.json +++ b/src/inferrsi/knowledge/data/skill_library/records/sglang_omni.common.omni.pipeline_state_and_transport.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-architecture/SKILL.md" + "en": "data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-architecture/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/architecture-search-space.json b/src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.architecture_search_space.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/architecture-search-space.json rename to src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.architecture_search_space.json index dbfa13167..b92ce6c88 100644 --- a/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/architecture-search-space.json +++ b/src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.architecture_search_space.json @@ -42,7 +42,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/strata/sub-skills/design-space/SKILL.md" + "en": "data/skill_library/systems/engines/strata/sub-skills/design-space/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/controlled-execution-tuning.json b/src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.controlled_execution_tuning.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/controlled-execution-tuning.json rename to src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.controlled_execution_tuning.json index ad5b58d24..d1a1839ff 100644 --- a/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/controlled-execution-tuning.json +++ b/src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.controlled_execution_tuning.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/strata/sub-skills/tune-execution/SKILL.md" + "en": "data/skill_library/systems/engines/strata/sub-skills/tune-execution/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/engine-validation-contract.json b/src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.engine_validation_contract.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/engine-validation-contract.json rename to src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.engine_validation_contract.json index a5a2b37b5..843a30f80 100644 --- a/src/inferrsi/knowledge/data/repo_packs/strata/knowledge/common/llm/engine-validation-contract.json +++ b/src/inferrsi/knowledge/data/skill_library/records/strata.common.llm.engine_validation_contract.json @@ -42,7 +42,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/strata/sub-skills/validate-engine/SKILL.md" + "en": "data/skill_library/systems/engines/strata/sub-skills/validate-engine/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json b/src/inferrsi/knowledge/data/skill_library/records/tensorrt_llm.cuda.llm.prefill_decode_disaggregation.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json rename to src/inferrsi/knowledge/data/skill_library/records/tensorrt_llm.cuda.llm.prefill_decode_disaggregation.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/transformers/common/llm/kv-cache-strategy-contracts.json b/src/inferrsi/knowledge/data/skill_library/records/transformers.common.llm.kv_cache_strategy_contracts.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/transformers/common/llm/kv-cache-strategy-contracts.json rename to src/inferrsi/knowledge/data/skill_library/records/transformers.common.llm.kv_cache_strategy_contracts.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/dependency/triton/common/kernel/bandwidth-fusion-resource-limits.json b/src/inferrsi/knowledge/data/skill_library/records/triton.common.kernel.bandwidth_fusion_resource_limits.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/dependency/triton/common/kernel/bandwidth-fusion-resource-limits.json rename to src/inferrsi/knowledge/data/skill_library/records/triton.common.kernel.bandwidth_fusion_resource_limits.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/bounded-serving-measurement.json b/src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.bounded_serving_measurement.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/bounded-serving-measurement.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.bounded_serving_measurement.json index 6405f25bc..3ceda6d11 100644 --- a/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/bounded-serving-measurement.json +++ b/src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.bounded_serving_measurement.json @@ -44,7 +44,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/vllm/sub-skills/operation-performance/SKILL.md" + "en": "data/skill_library/systems/engines/vllm/sub-skills/operation-performance/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/extension-correctness-gates.json b/src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.extension_correctness_gates.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/extension-correctness-gates.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.extension_correctness_gates.json index 731e26826..92f69fd25 100644 --- a/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/extension-correctness-gates.json +++ b/src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.extension_correctness_gates.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/vllm/sub-skills/extend-validate/SKILL.md" + "en": "data/skill_library/systems/engines/vllm/sub-skills/extend-validate/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/runtime-and-kv-ownership.json b/src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.runtime_and_kv_ownership.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/runtime-and-kv-ownership.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.runtime_and_kv_ownership.json index faf2a87ca..c07b22137 100644 --- a/src/inferrsi/knowledge/data/repo_packs/vllm/knowledge/common/llm/runtime-and-kv-ownership.json +++ b/src/inferrsi/knowledge/data/skill_library/records/vllm.common.llm.runtime_and_kv_ownership.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/vllm/sub-skills/architecture-search-space/SKILL.md" + "en": "data/skill_library/systems/engines/vllm/sub-skills/architecture-search-space/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/ascend/diffusion/minimax-h3-reference-kv-reuse.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/ascend/diffusion/minimax-h3-reference-kv-reuse.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuse.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/ascend/diffusion/minimax-h3-sp-head-buckets.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.ascend.diffusion.minimax_h3_sp_head_buckets.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/ascend/diffusion/minimax-h3-sp-head-buckets.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.ascend.diffusion.minimax_h3_sp_head_buckets.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/ascend/diffusion/qwen-image-21-eager-t2i.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/ascend/diffusion/qwen-image-21-eager-t2i.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/audio/streaming-serving-measurement.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.audio.streaming_serving_measurement.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/audio/streaming-serving-measurement.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.audio.streaming_serving_measurement.json index c28ce422f..d5f2133da 100644 --- a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/audio/streaming-serving-measurement.json +++ b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.audio.streaming_serving_measurement.json @@ -43,7 +43,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/vllm-omni/sub-skills/omni-serving/SKILL.md" + "en": "data/skill_library/systems/engines/vllm-omni/sub-skills/omni-serving/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/measurement-and-profiling.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.measurement_and_profiling.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/measurement-and-profiling.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.measurement_and_profiling.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/production-integration-gates.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.production_integration_gates.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/production-integration-gates.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.production_integration_gates.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/step-state-and-output-validation.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.step_state_and_output_validation.json similarity index 98% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/step-state-and-output-validation.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.step_state_and_output_validation.json index 821b08f5a..adb6e7d85 100644 --- a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/step-state-and-output-validation.json +++ b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.step_state_and_output_validation.json @@ -63,7 +63,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/SKILL.md" + "en": "data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/upstream-review-gates.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.upstream_review_gates.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/diffusion/upstream-review-gates.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.diffusion.upstream_review_gates.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/omni/stage-ownership-and-search.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.omni.stage_ownership_and_search.json similarity index 97% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/omni/stage-ownership-and-search.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.omni.stage_ownership_and_search.json index 6b88222ac..e8c4cc2c5 100644 --- a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/common/omni/stage-ownership-and-search.json +++ b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.common.omni.stage_ownership_and_search.json @@ -42,7 +42,7 @@ } ], "skill_documents": { - "en": "data/skill_library/repositories/vllm-omni/sub-skills/omni-architecture/SKILL.md" + "en": "data/skill_library/systems/engines/vllm-omni/sub-skills/omni-architecture/SKILL.md" }, "applicability": { "hardware": [], diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/audio/concurrency-scaling-state-and-overlap.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/audio/concurrency-scaling-state-and-overlap.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.audio.concurrency_scaling_state_and_overlap.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-adaln-schedule-cache.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-adaln-schedule-cache.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_adaln_schedule_cache.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-cuda-mxfp8.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_cuda_mxfp8.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-cuda-mxfp8.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_cuda_mxfp8.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-fp32-modulation.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-fp32-modulation.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_fp32_modulation.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-qk-norm-rope.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-qk-norm-rope.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_qk_norm_rope.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-ref2va-teacache.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-ref2va-teacache.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_ref2va_teacache.json diff --git a/src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-vae-stacked-tiling.json b/src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling.json similarity index 100% rename from src/inferrsi/knowledge/data/repo_packs/vllm_omni/knowledge/cuda/diffusion/minimax-h3-vae-stacked-tiling.json rename to src/inferrsi/knowledge/data/skill_library/records/vllm_omni.cuda.diffusion.minimax_h3_vae_stacked_tiling.json diff --git a/src/inferrsi/knowledge/data/knowledge_packs/peer/xdit/cuda/diffusion/parallelism-freshness-and-topology.json b/src/inferrsi/knowledge/data/skill_library/records/xdit.cuda.diffusion.parallelism_freshness_and_topology.json similarity index 100% rename from src/inferrsi/knowledge/data/knowledge_packs/peer/xdit/cuda/diffusion/parallelism-freshness-and-topology.json rename to src/inferrsi/knowledge/data/skill_library/records/xdit.cuda.diffusion.parallelism_freshness_and_topology.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/metadata.json deleted file mode 100644 index 7a0490096..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/metadata.json +++ /dev/null @@ -1,70 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "Apache-2.0", - "tags": [ - "afd-plugin", - "cuda", - "p2p", - "rank-mapping", - "hidden-states", - "ownership" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/afd-plugin", - "commit": "df8d6a0c403c3ca1b4ed9cf1687e7204908bebec", - "paths": [ - "README.md", - "LICENSE", - "afd_plugin/config.py", - "afd_plugin/distributed/topology.py", - "afd_plugin/connectors/factory.py", - "afd_plugin/connectors/metadata.py", - "afd_plugin/connectors/gpu/p2p.py", - "afd_plugin/v1/worker/attention_metadata.py", - "afd_plugin/v1/worker/attention_model_runner_v2.py", - "afd_plugin/v1/worker/ffn_worker.py", - "afd_plugin/v1/worker/ffn_model_runner.py", - "afd_plugin/model_executor/models/deepseek_v2.py", - "afd_plugin/model_executor/models/forward_context.py", - "tests/unit/distributed/test_topology_partition.py" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Source-based CUDA P2pNccl rank/payload audit; selected eager dense-FFN path with compute_gate_on_attention=false. Repository identity afd_plugin; runtime/backend compatibility unverified.", - "prerequisites": "Access to the pinned source ledger and caller-level evidence for the selected eager dense FFN path with Attention-side gating disabled; any proposed cross-stream reuse needs an established communicator completion contract.", - "input_contract": "Plugin and dependency revisions, connector and role, A/F worker counts, DP/PCP/TP coordinates, worker/device ranks, subgroup roster, stage, per-peer token counts, payload shape/dtype/device, transfer context and returned slices.", - "produced_artifact": "A bounded rank-namespace and per-peer payload ownership ledger, the earliest evidenced mismatch or missing edge, and a proposed verification plan.", - "failure_modes": "Inferring data peers from control senders, accepting equal total row counts despite a peer permutation, or treating an even-split fallback or Python return as proof of ownership/completion can produce a wrong exchange diagnosis.", - "refusal_point": "Stop on A < F, invalid role ranks, duplicate/missing assignments or an unresolved rank namespace; return the conflicting callers and ledger entries without inventing ranks or attributing the fault to networking." - }, - "limitations": [ - "Source-derived candidate; native exchange, model quality, backend compatibility, performance and agent efficacy not_run.", - "Selected Python readset is Apache-2.0; the retained MIT notice concerns excluded NPU async CAM implementation and adapted bindings.", - "No qualification of graph/DBO, remote-expert MoE, multi-host transport, complete token-count projection or underlying communicator completion semantics." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "references": [ - "../../vllm/audit-vllm-kv-lifetimes/SKILL.md", - "../../../methods/inference-benchmark-design/SKILL.md", - "../../../methods/dense-performance-evidence/SKILL.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Record role, world and subgroup-local ranks separately; reconstruct the actual subgroup roster before interpreting send/receive peers.", - "Track distinct row identities and stage-specific peer lengths through concatenation and splitting; use same-total swapped lengths as a countercheck.", - "Establish the last communicator/stream completion obligation before reusing or releasing a receive buffer." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/metadata.json deleted file mode 100644 index b37aeb6dc..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/metadata.json +++ /dev/null @@ -1,72 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "GPL-3.0", - "tags": [ - "comfyui", - "workflow", - "cache-invalidation", - "model-residency", - "state-ownership" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/Comfy-Org/ComfyUI", - "commit": "b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c", - "paths": [ - "execution.py", - "comfy_execution/caching.py", - "comfy_execution/graph.py", - "comfy/model_management.py", - "comfy/model_patcher.py", - "comfy/sampler_helpers.py", - "server.py", - "main.py", - "comfy_execution/graph_utils.py", - "comfy_execution/cache_provider.py", - "nodes.py", - "comfy/sample.py", - "comfy/samplers.py", - "tests/execution/test_execution.py", - "tests-unit/execution_test/test_execution_list.py", - "tests-unit/comfy_test/model_management_test.py", - "tests/inference/test_inference.py", - ".github/workflows/test-execution.yml", - "LICENSE", - "pyproject.toml", - "tests-unit/execution_test/test_cache_provider.py", - "AGENTS.md" - ], - "readset": "references/repo-provenance.md" - } - ], - "applicability": { - "scope": "Source-grounded ComfyUI workflow-output cache and model-weight ownership audit; confirm the actual revision, node registry, cache caller and static/dynamic patcher before transfer.", - "prerequisites": "Access to the pinned provenance and the actual node registry/cache caller; residency questions additionally require the concrete patcher and effective dynamic/static mode. Native memory conclusions need the actual backend.", - "input_contract": "Exact revision, API-format prompt/output targets, relevant node classes/constants/links/hidden inputs and external state, selected cache caller/mode and patcher settings, plus available execution/load/memory/timing observations.", - "produced_artifact": "A state/owner/key/lifetime/consumer ledger with invalidation or release events, evidence and unresolved boundaries, followed by a discriminating check and its actual result.", - "failure_modes": "Treating identical JSON as proof of unchanged external inputs, confusing eviction with key validity, or equating workflow cache hits with device-resident model weights can explain the wrong state contract.", - "refusal_point": "Stop when the cause/action is supported or required environment, backend or model resources are unavailable; do not manufacture a hit, hardware result or performance threshold." - }, - "limitations": [ - "Source-derived candidate; no local executor-cache, device-residency or performance qualification.", - "Seven self-authored cases exercised only the unchanged pure upstream graph_utils module; they do not verify PromptExecutor, cache signatures or model behavior.", - "The dynamic native allocator and third-party custom-node behavior remain outside the verified scope." - ], - "provenance_readsets": [ - "references/repo-provenance.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Draw the affected dependency cone with declared, lazy and dynamically expanded links, then mark unfingerprinted external state unresolved.", - "Separate output/object cache validity and retention from patch identity, partial unload, full detach and allocator-visible release.", - "Change one test variable while fixing prompt, output target, model, backend, precision, seed and cache mode; check values and executed nodes before timing." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json deleted file mode 100644 index 806dac0f3..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json +++ /dev/null @@ -1,66 +0,0 @@ -{ - "tags": [ - "deepinfer", - "sliding-window-attention", - "prefix-replay", - "kv-cache", - "metadata", - "correctness" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/deepseek-ai/DeepInfer", - "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", - "paths": [ - "csrc/engine/input_metadata/prepare_tensor_input_v4.cpp", - "deepinfer/engine/llm_engine.py", - "deepinfer/model_executor/input_metadata/yoco.py", - "deepinfer/model_executor/model_metadata_utils.py", - "deepinfer/model_executor/models/decoder_layer.py", - "deepinfer/model_executor/models/v4_modules/attn.py", - "deepinfer/model_executor/models/v4_modules/yoco.py", - "deepinfer/worker/cache_engine.py", - "deepinfer/worker/model_runner.py", - "deepinfer/environ.py", - "LICENSE" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Decoders with an explicit producer/consumer boundary and a demonstrated dependency closure for compact query rows, including separately preserved historical state", - "prerequisites": "A derived dependency closure; a full-length KV write at the boundary; known rollback depth; control over metadata construction", - "hardware": "Backend-independent retain arithmetic; window sizes, boundary index and ring lengths come from the model config and do not transfer", - "input_contract": "The actual producer/consumer split and transitive dependency closure, boundary index, window and rollback depth, consumer KV sharing/compression layout and separately preserved historical keys.", - "produced_artifact": "A dependency listing, enforced retain length, compact query/position/store-slot/visibility metadata, a separate historical-key retention statement and a diagnostic equivalence-reference mode.", - "failure_modes": "Applying window-plus-rollback without dependency closure, discarding full-layout state, clipping away the sampling row, or comparing across preemption can invalidate replay; compact queries do not imply historical keys were dropped.", - "refusal_point": "Stop if required consumer state is unavailable without a preserved full-length copy, the boundary representation cannot be captured, or comparison preemption moves the anchor. Do not use for clipped-region outputs or required bit-identical preemption replay." - }, - "limitations": [ - "Source-derived candidate; no equivalence run was performed.", - "Applies to architectures with a demonstrated boundary dependency, not to arbitrary sliding-window stacks.", - "Window sizes, boundary-layer index and ring lengths are model-config specific and are not transferable.", - "Query-row compaction does not imply dropping historical keys; the two decisions are separate.", - "Prompt logprobs are unrecoverable for the clipped region; the reference forces them off with a warning.", - "The truncation path is off by default in the reference source." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/validation.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Derive every consumer dependency before computing keep; preserve the required full-layout representation separately from the compact query rows.", - "Use the visibility-only diagnostic mode to isolate equivalence from row gathering, and retain one sampling row for a fully clipped chunk.", - "Check metadata at chunk/store boundaries, actual prompt-logprob handling and zero preemption during equivalence comparison." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json deleted file mode 100644 index 50d9822fa..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json +++ /dev/null @@ -1,65 +0,0 @@ -{ - "tags": [ - "deepinfer", - "speculative-decoding", - "mtp", - "verification", - "cost-model", - "scheduling" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/deepseek-ai/DeepInfer", - "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", - "paths": [ - "csrc/engine/mtp_varlen.hpp", - "csrc/engine/scheduler.hpp", - "deepinfer/worker/worker.py", - "deepinfer/environ.py", - "deepinfer/sampling_params.py", - "deepinfer/model_executor/models/mtp/dflash.py", - "deepinfer/model_executor/layers/sampler.py", - "deepinfer/model_executor/layers/tilelang_ops/fused_sample.py", - "LICENSE" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Engines with multi-token speculative decoding whose verification forward cost grows with query rows", - "prerequisites": "A calibrated per-position acceptance signal; a measured (rows, step_time) curve; a handle on metadata construction", - "hardware": "Backend-independent procedure; the reference implementation targets NVIDIA SM100 and Ascend 950 only", - "input_contract": "Draft width and row representation, the defined individual/conditional versus cumulative acceptance semantics, verify rows/time curve, sampler cost and acceptance round actually available at schedule time.", - "produced_artifact": "A per-step verification row budget, per-request query lengths from one through draft_width plus one, cumulative acceptance vectors and predicted-versus-realized acceptance error logs.", - "failure_modes": "Confusing verification savings with reduced drafting, misreading confidence semantics or available rounds, allowing later-position confidence to select an earlier token, or generating non-prefix grants can invalidate the budget or sampling claim.", - "refusal_point": "Stop if unthrottled drafting or sampler cost dominates, the signal cannot be made cumulative, or no round supplies acceptance estimates; the procedure also does not apply without a per-step cost model." - }, - "limitations": [ - "Source-derived candidate; no local efficacy validation and no reproduced gain.", - "The reference budgets verification rows only; draft compute is unchanged.", - "The budgeted path is off by default in the reference source.", - "Budgeting can lower accepted length; equal accepted length is not an acceptance criterion.", - "Per-position independence of the confidence used for verification is a correctness requirement, not only a scheduling convenience.", - "Cross-backend rejection-comparison parity is unproven; the reference has no equality assertion.", - "Numeric acceptance rates and warmup curves are deployment-specific and are not carried here." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/validation.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Measure draft, verify and sampler costs separately and fit the verify curve for graph sizes that will actually replay.", - "Preserve the one-row floor and prefix grants, and audit CPU/GPU allocation timing against the per-position decision-independence contract.", - "Report verification rows, accepted tokens, total step time and throughput separately; use an independently specified sampling oracle rather than requiring unchanged accepted length." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/SKILL.md deleted file mode 100644 index e06a3425e..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/SKILL.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -name: dwarfstar -description: "Investigate DwarfStar model fit, expert SSD streaming, session and vision cache correctness, speculative sampling and distributed execution using pinned source contracts and explicit native verification boundaries." -license: NO_LICENSE -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# DwarfStar inference engineering - -Use this candidate to turn a DwarfStar capacity, state-correctness or execution -question into a bounded hypothesis and discriminating check. It distills -`antirez/ds4`, not an arbitrary GGUF runner. Start with the -[source and method contract](references/repo-provenance.md). - -The operating guidance is source-derived. A local CPU build or bookkeeping test -does not validate Metal/CUDA/ROCm kernels, model output, a real service, physical -tensor parallelism or a performance gain. Required backend evidence remains -**BLOCKED_REQUIRED_BACKEND**; this candidate is not fully backend-verified. - -## Route the question - -| Question | Read next | Expected output | -| --- | --- | --- | -| A model fits on disk but not in RAM/VRAM; cache grows; more sessions fail | [runtime-memory](sub-skills/runtime-memory/SKILL.md) | Representation and phase/owner memory ledger, one causal hypothesis | -| A repeated prompt misses, reuses wrong image/tool history or corrupts a continuation | [serving-state](sub-skills/serving-state/SKILL.md) | Token/image/state identity and invalidation ledger | -| A draft is faster but changes results; a TP parser test passes; a backend falls back | [qualification](sub-skills/qualification/SKILL.md) | Matching correctness test, actual execution path and remaining native gap | - -Use the bundled [request builder](scripts/request_template.py) when preparing a -minimal Chat Completions request. It prints JSON only: no engine, network, -credential, model download or service is started. Its options are deliberately -smaller than the upstream API. - -## Establish the execution tuple - -Record the engine revision and patches, model family/checkpoint/quantization and -artifact identity, selected native backend/build, device and transport topology, -RAM/VRAM/storage budget, context/prefill chunk/session count, image inputs, -sampling/seed and speculation policy. Add cold/warm/live/disk-cache state and -the workload's quality and service requirements. Unknown fields remain unknown. - -Do not infer model support from a flag, shared quantization name, successful file -mapping or API model alias. The actual loaded GGUF and model-specific graph -select semantics. Flash, V4.1, PRO, GLM and Qwen routes differ. - -## Native preparation, not automatic installation - -DwarfStar is a native C engine, not a Python distribution. A prepared CPU -inspection build can expose CLI help without weights; accelerated builds require -their matching compiler/runtime. An operator with a compatible installed build -can inspect `ds4 --help` and `ds4-server --help` before choosing options. -For an authorized source installation, use the public repository at the -reviewed source revision and the native `make cpu` target for CPU inspection; -Metal/CUDA/ROCm require the matching native build described by this graph. -Building, downloading weights, exposing a listener and changing system memory -limits are separate actions requiring task authorization. - -This extraction adapted AREX's Python-only environment fields to native C build -and CLI identity. It did not invent an installed Python package, run a DisCo -model backend or install into a managed DisCo skill collection. - -## Work one bounded hypothesis - -1. Select the relevant route and read its bundled mechanism and failure notes. -2. Separate source observations, upstream reports and proposed experiments. -3. Identify the owner, lifetime, validity key and release/rollback boundary. -4. Freeze controls before a comparison; change one mechanism or explicitly - declare a coupled change. Numeric gates and campaign revisions belong in - the acceptance issue, not in this reusable guidance. -5. Verify the smallest invariant before measuring performance; retain failures, - refusals, fallback messages, cancellations and unavailable fixtures. -6. For a service claim, run the accepted model/workload/quality/concurrency - protocol on the actual backend. Microtests are not service qualification. - -## Stop and report - -Return the executing tuple, observed signal, source-backed mechanism, alternate -explanation, proposed falsifier, actual check outcome and unresolved gap. Stop -at unavailable hardware/weights, unsafe exposure or the authorized budget. -Do not disable a memory guard to turn a refusal into a capacity claim. - -Read [source notices](LICENSE) when redistributing adapted material. -`NO_LICENSE` is the AREX resolver's unauthenticated-CLI warning fallback, not a -claim that DwarfStar lacks a legal license; the pinned source license is MIT. -Model weights and retained third-party code have independent obligations. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/metadata.json deleted file mode 100644 index a8f11e400..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/metadata.json +++ /dev/null @@ -1,99 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "partial_cpu_only", - "license": "NO_LICENSE", - "tags": [ - "dwarfstar", - "ds4", - "moe", - "ssd-streaming", - "session-state", - "kv-cache", - "speculation", - "tensor-parallelism" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/antirez/ds4", - "commit": "ee7e8405232c8971000cc7ba2b945b00bf6d20d3", - "paths": [ - "README.md", - "LICENSE", - "Makefile", - "ds4.h", - "ds4.c", - "ds4_ssd.c", - "ds4_ssd.h", - "ds4_kvstore.c", - "ds4_kvstore.h", - "ds4_server.c", - "ds4_tp.c", - "ds4_tp.h", - "docs/SSD_STREAMING.md", - "docs/SERVER.md", - "docs/SPECULATIVE_DECODING.md", - "docs/DISTRIBUTED.md", - "docs/CUDA_MULTI_GPU.md", - "docs/TESTING.md", - "tests/test_session_state.c", - "tests/test_sampling.c", - "tests/test_tp_commands.c", - "tests/test_engine_mgpu_placement.c", - "ds4_cuda.cu", - "QA_BEFORE_RELEASES.md" - ], - "readset": "references/source-identity.json", - "license": "MIT" - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Portable instructions executed by compatible agent with native C adaptation; no DisCo CLI/model run or managed import." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "User-approved native C adaptation and partial drafting; manual stable-ID subskill orchestration and integrated create/verify stages. Required-backend blocks disable automatic admission." - }, - "applicability": { - "scope": "Pinned DwarfStar source investigation and experiment planning; not arbitrary GGUF support or cross-backend qualification.", - "prerequisites": "Access to the pinned native-source contract and relevant memory/state/qualification route; native verification needs a compatible build, matching hardware/weights and task-specific execution authority.", - "input_contract": "Engine revision/patches, actual model artifact/representation, selected backend/build and topology, memory/storage budget, context/chunk/session count, image inputs, sampling/speculation policy, cache state and quality/service requirements.", - "produced_artifact": "A bounded executing-tuple report with observed signal, source-backed mechanism, alternate explanation, proposed falsifier, actual check outcome and unresolved backend/model/service gap.", - "failure_modes": "Inferring model support from flags, aliases, file mapping or quantization names can select the wrong graph; treating CPU bookkeeping or parser checks as native backend/service qualification can overstate the result.", - "refusal_point": "Stop at unavailable hardware/weights, unsafe exposure or the authorized budget; retain memory-guard refusals rather than disabling the guard to claim fit." - }, - "limitations": [ - "Metal/CUDA/ROCm inference, actual model output, SSD runtime I/O, GPU recurrent rollback, physical distributed behavior, service quality/performance and skill efficacy remain unverified.", - "CPU build/help and selected bookkeeping tests do not clear BLOCKED_REQUIRED_BACKEND.", - "NO_LICENSE is the actual unauthenticated GitHub CLI resolver fallback, not a legal conclusion; pinned source MIT notice is retained.", - "Only selected source bodies were inspected; full kernels, conversion/imatrix tools and every workflow are not covered." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Choose the memory, serving-state or qualification route by the observed symptom and identify the owner, validity key and release/rollback boundary.", - "Freeze comparison controls, test the smallest invariant first and retain fallback messages, cancellations, refusals and unavailable fixtures.", - "Base model support on the actual loaded artifact and graph; use native backend evidence for model, tensor-parallel or service conclusions." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/metadata.json deleted file mode 100644 index f433264e6..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/metadata.json +++ /dev/null @@ -1,90 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "Apache-2.0", - "tags": [ - "Flare", - "flare-local-inference", - "local-inference", - "cuda", - "session-state", - "kv-cache", - "recurrent-state", - "convolution", - "restore-admission", - "transfer-ticket", - "fences", - "offload" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/NVIDIA/flare-local-inference", - "resolved_url": "https://github.com/NVIDIA/local-inference", - "commit": "bce032c014bae34502ba22d3304fa084d5b40afe", - "paths": [ - "README.md", - "LICENSE", - "docs/runtime-package-design.md", - "docs/prefix-cache-offload.md", - "helios/src/runtime_package/format.rs", - "helios/src/runtime_package/model.rs", - "helios/src/execution/manifest.rs", - "helios/src/model/contract.rs", - "helios/src/model/catalog.rs", - "helios/src/cache.rs", - "helios/src/offload/mod.rs", - "helios/src/offload/geometry.rs", - "helios/src/offload/index.rs", - "helios/src/offload/record.rs", - "helios/src/offload/transfer.rs", - "helios/src/scheduler.rs", - "helios/src/server/runtime.rs", - "tests/server/test_prefix_cache_offload.py", - "qualify/performance.py", - "qualify/provenance.py" - ], - "readset": "references/source-notes.md", - "identity": "references/source-ledger.json", - "license": "Apache-2.0" - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "task-agnostic", - "strategy": "reuse-existing", - "status": "source-derived-candidate", - "method": "AREX scope/ground/construct/verify and repository evidence/usability methods adapted to InferRSI native owners and packaging; no DisCo runtime or managed import." - }, - "applicability": { - "scope": "Pinned Flare complete-session restore and admission audit for its NVIDIA SM120/SM121 research prototype. CUDA catalog scope is content classification, not device or backend qualification.", - "prerequisites": "Access to package/source identity and bounded restore traces, with preparation compatibility established before cache-state diagnosis. Running, freeing resources or changing configuration requires separate authority.", - "input_contract": "Execution ABI and prepared-contract version, ordinary/speculative configuration, model/layout identity, request IDs and consumed-token endpoints, GPU/host prefixes, continuation row, ticket/waiter identities, transfer events, fences and resource counts.", - "produced_artifact": "A checkpoint/ticket ownership table, five-outcome admission ledger and diagnosis identifying the earliest unsupported transition and smallest missing observation.", - "failure_modes": "A KV-only lookup hit can omit continuation state; admission can be mistaken for actual reuse; client cancellation can be mistaken for transfer completion; mixing absolute timestamps from client and native clocks can corrupt the diagnosis.", - "refusal_point": "Stop with an unresolved edge when package identity, layout, transfer completion or clock provenance is unknown." - }, - "limitations": [ - "Source-only candidate; native Flare/model/GPU/service and skill efficacy were not evaluated.", - "GDS/Direct transport is deferred and explicitly rejected by the pinned implementation.", - "Multi-rank source branches do not establish multi-GPU, multi-host or hardware failure qualification.", - "Root Apache-2.0 attribution does not grant model, asset or separately licensed dependency rights.", - "Only the selected symbols/sections were inspected; kernels, exporter implementation and whole-repository behavior are outside coverage." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/source-ledger.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Pair the immutable KV parent chain with the complete continuation row at the same consumed-token endpoint before calling a checkpoint restorable.", - "Distinguish lookup outcomes, destination reservation, ticket creation, transfer completion, publication and actual reused-token evidence.", - "Follow scheduler-owned tickets through waiter cancellation and check write/use fences before allowing storage reuse." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md deleted file mode 100644 index 583d8368f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -name: audit-h3-runtime-residency -description: Audit MiniMax-H3 generation in h3.c on Apple Silicon by tracing conditioning, AdaLN precompute, DiT residency, Metal precision paths, SSD streaming and audio/video decoding. ---- - -# Audit h3.c runtime residency and numerical paths - -Source-derived candidate for `antirez/h3.c` at -`8974cc055ea9c02fcd14cc27dfda3e1027c05153`. Use for MiniMax-H3 -text/image/video/audio conditioning, memory-fit diagnosis, kernel changes or a -dedicated-engine design derived from this C/Metal runtime. Start with the -[source readset and architecture](references/source-notes.md). Native execution, -media quality, performance and behavioral skill efficacy are **not_run**. - -## Inputs and decision - -Record the runtime revision/patches, macOS/SDK/GPU, memory budget, selected FL2VA -or Ref2VA checkpoint and complete component/format inventory. Freeze the prompt, -ordered reference media, requested and aligned frames, output/internal canvas, -steps, seed, layer/reuse/token policies, all precision controls, preview and -interactive-cache state. An installed engine name or a successful metadata load -does not identify this execution configuration. - -Produce a phase ledger and one smallest discriminating check. The useful output -is a supported cause, invariant, counter-explanation and falsifier; model-file -bytes alone do not decide whether a workload fits unified memory. - -## Procedure - -1. **Resolve the executing pipeline.** Follow `h3_generate` through the - [phase ledger](references/phase-residency.md). Ordered references select the - separate Ref2VA DiT; FL2VA first/last-frame conditioning still needs both the - video input encoder and the Qwen vision/language path. Audio conditioning and - audio output decoding are different calls. Count the selected input and - output components without adding alternative checkpoints or repeated shared - files. Use the [retained weight inventory](references/minimax-h3-weight-inventory.json) - for its exact checkpoint selections, not as a universal model-size constant. -2. **Check representation before capacity.** The raw loader validates exact - names, rank, shape and BF16/F32 dtype. Runtime symmetric int8 conversion is - different from loading MLX affine `U32` bundles. A smaller published bundle - needs a compatible engine or an explicit conversion; matching bit width is - insufficient. Keep header validity and payload integrity separate. -3. **Trace ownership across phases and requests.** Distinguish host embeddings, - media/latent/output buffers, shared Metal tensors, file-backed mappings, - AdaLN tables and prepared-model/decoder caches. In the one-shot path the DiT - is freed before decoding; preview and interactive retention can overlap - phases. Audit cache keys and clear/reset boundaries before diagnosing a leak - or assuming a warm request has cold-request memory behavior. -4. **Preserve the schedule contract.** AdaLN tables are indexed by the actual - video/audio schedule, conditioning presence, modality tags and layout. - Each projection is submitted and released before loading the next. Rebuild - when those dependencies change; do not replace conditioning timesteps or - collapse audio/video rows because their step counts match. See the - [schedule and cache invariants](references/phase-residency.md#adaln-and-prepared-state). -5. **Classify every numerical change.** Follow - [precision and streaming](references/precision-and-streaming.md). Separate - BF16-preserving fusions from int8 approximation, BF16 output-head conversion, - layer thinning, velocity/core reuse, token reduction and internal-canvas/RoPE - changes. On M5, 50 layers and `--reuse 1` still allow default int8 paths. - Select the explicit BF16 controls and F32-head diagnostic when that is the - frozen reference. A kernel parity result is not full audio/video quality. -6. **Choose the residency tradeoff that addresses the measured phase.** Qwen - prefetch overlaps current-layer compute with future-layer reads and increases - resident buffers. Mapped persistent DiT weights remain file-backed; explicit - SSD mode instead rotates two BF16 matrix slots and disables int8. It changes - I/O and command boundaries. Reducing DiT residency will not resolve a VAE, - reference-media or host-output peak. Track physical/compressed/swap memory - alongside tensor counters and storage reads. -7. **Run the matching check when resources exist.** The - [validation recipe](references/validation.md) maps native targets, required - fixtures and proposed probes to these invariants. Preserve skipped fixtures, - startup/first/warm timing, both output modalities and all failed trials. - The [Apple feasibility comparison](references/minimax-h3-apple.md) is retained - imported research; neither its upstream measurements nor packaged-reader - tests qualify a new Mac, workload or 64 GiB memory budget. - -## Report and stop - -Return a table with `phase`, `component and format`, `owner`, `resident weights`, -`runtime/host buffers`, `overlap`, `release or reuse boundary`, and -`observation/unknown`; then the exact check, actual outcome and remaining gap. -Stop at unavailable model/fixture/backend resources or the accepted experiment -budget. Reading this skill does not install an engine, download weights, run a -distiller, authorize a campaign or promote this candidate. Missing target -evidence remains unverified/INCONCLUSIVE. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/metadata.json deleted file mode 100644 index b026cebf4..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/metadata.json +++ /dev/null @@ -1,133 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "MIT", - "tags": [ - "h3.c", - "MiniMax-H3", - "apple-silicon", - "metal", - "diffusion", - "video", - "audio", - "multimodal-conditioning", - "adaln", - "model-residency", - "ssd-streaming", - "int8", - "bf16" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/antirez/h3.c", - "commit": "8974cc055ea9c02fcd14cc27dfda3e1027c05153", - "paths": [ - "h3.h", - "h3_internal.h", - "h3.c", - "h3_weights.c", - "h3_weights.h", - "h3_safetensors.c", - "h3_host.c", - "h3_text_encoder.c", - "h3_vision_encoder.c", - "h3_multimodal.c", - "h3_video_encoder.c", - "h3_audio_vae.c", - "h3_video_vae.c", - "h3_dit_schedule.c", - "h3_dit_schedule.h", - "h3_dit.c", - "h3_gpu.m", - "h3_shaders.metal", - "main.c", - "h3_cli.c", - "Makefile", - "README.md", - "LICENSE", - "THIRD_PARTY_NOTICES.md", - "tests/test_h3.c", - "tests/test_bf16.c", - "tests/test_real_dit_schedule.c", - "tests/test_real_prompt.c", - "tests/test_real_multimodal_text.c", - "tests/test_semantic_dit.c", - "tests/test_semantic_vae.c", - "tests/test_real_audio_vae.c", - "tests/test_real_video_vae.c", - "tests/bench_dit.c" - ], - "readset": "references/source-notes.md", - "identity": "references/source-identity.json", - "license": "MIT", - "third_party_notices": "references/THIRD_PARTY_NOTICES.md" - }, - { - "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI", - "commit": "f804ec1503e977787c572d5d27d1f3258017b01f", - "paths": [ - "src/inferrsi/data/skill_library/workflows/inference-engine-research/references/minimax-h3-apple.md", - "src/inferrsi/data/skill_library/workflows/inference-engine-research/references/minimax-h3-weight-inventory.json" - ], - "scope": "Prior model/engine comparison and safetensors header observations, reused without reissuing range requests; inventory bytes preserved." - }, - { - "type": "document", - "url": "https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/42ed227ee7df40d41602854ae760620d6eb651fe/FL2VA", - "version": "42ed227ee7df40d41602854ae760620d6eb651fe", - "scope": "29 JSON-header digests and component aggregates retained from prior inspection; payload and complete model license not newly verified.", - "license": "UNKNOWN", - "readset": "references/minimax-h3-weight-inventory.json" - }, - { - "type": "document", - "url": "https://huggingface.co/appautomaton/minimax-h3-base-8bit-mlx/tree/9a08116a349ab469b097775b5a483affcb5ccd34", - "version": "9a08116a349ab469b097775b5a483affcb5ccd34", - "scope": "Four selected export JSON-header digests/packing aggregates retained from prior inspection; payload and model/export license not newly verified.", - "license": "UNKNOWN", - "readset": "references/minimax-h3-weight-inventory.json" - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "capability-specific", - "strategy": "direct", - "status": "portable-candidate", - "native_import": "not-run", - "method": "Manual pinned-source distillation informed by the existing Repo-To-Skill ledger; no AREX/DisCo execution or behavioral efficacy claim." - }, - "applicability": { - "scope": "MiniMax-H3 joint audio/video C/Metal runtime audit: raw checkpoint compatibility, input encoders, schedule precompute, phased/cached residency, numerical controls and SSD streaming on Apple Silicon. Common catalog placement is a discovery classification, not cross-backend qualification.", - "prerequisites": "Access to the pinned source architecture and selected component/format inventory; native checks need matching model, fixture and backend resources within an authorized experiment budget.", - "input_contract": "Runtime revision/patches, macOS/SDK/GPU and memory budget, FL2VA/Ref2VA selection, component formats, prompt and ordered media, requested/aligned frames, canvases, steps/seed, layer/reuse/token/precision policies and preview/interactive-cache state.", - "produced_artifact": "A phase/component/owner ledger with resident weights, runtime/host buffers, overlap, release/reuse boundaries and observations/unknowns, followed by a discriminating check, actual result and remaining gap.", - "failure_modes": "Estimating fit from model-file bytes, confusing runtime int8 with incompatible MLX bundles, ignoring preview/cache overlap, or reducing DiT residency to address a VAE/host-output peak can produce a wrong capacity diagnosis.", - "refusal_point": "Stop at unavailable model/fixture/backend resources or the accepted experiment budget; missing target evidence remains unverified/INCONCLUSIVE." - }, - "limitations": [ - "Native h3.c/Metal execution, target peak memory, audio/video quality, performance, service behavior and skill efficacy were not run.", - "Only the bounded readset was inspected; complete kernel, driver/MPSGraph, tokenizer, FFmpeg, concurrent-caller and failure-path coverage is not established.", - "The retained inventory authenticates only prior JSON header observations, not weight payloads; full official versus selected truncated/packed export inventories are distinct.", - "Upstream 128 GB M5 Max results and tracked DiT bytes do not qualify a 64 GiB target, a different workload or total physical memory.", - "MIT covers the h3.c source; retained BSD-3-Clause notices and independent model/export/tool licenses remain separate. UNKNOWN source-license fields are unresolved." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/source-identity.json", - "references/minimax-h3-weight-inventory.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve the actual conditioning and output components before counting residency; do not add alternative checkpoints or count shared files twice.", - "Trace one-shot, preview and interactive ownership separately, including AdaLN schedule dependencies and prepared-cache reset boundaries.", - "Classify approximation and precision controls before comparison, then measure the relevant phase with tensor, physical/compressed/swap and storage observations." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md deleted file mode 100644 index 0c6643dc7..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -name: audit-h3-metal-fasth3-semantics -description: "Audit h3-metal FastH3 effective configuration, checkpoint tensor and schedule semantics, partial VSA tiles, sparse eligibility and dense-plus-coarse fallback without treating source or header checks as native parity." ---- - -# Audit h3-metal FastH3 semantics - -Use for the fork-specific four-step DiT path, a checkpoint-loader port, confusing -FastH3 flags, or sparse-attention fallback. This is a **source-derived candidate**, -not a generation/benchmark runner. Read the [source and scope](references/source-notes.md) -and [semantic contracts](references/semantic-contracts.md). Native Metal, model, -quality, memory-fit, performance and behavioral skill validation are **not_run**. -General h3.c residency remains with the separately registered -`audit-h3-runtime-residency`; the Apple measurement playbook is context, not new -evidence for these contracts. - -## Inputs and output - -Record source revision/patches, requested parameters, **explicitly captured** -environment keys, checkpoint path/key-family/config identity, requested/aligned -frames, latent dimensions, packed prefix segments and actual sparse-pipeline -availability (unknown is allowed). Keep model payload integrity, licensing and -native execution separate. Do not collect credentials or the complete environment. - -Return a ledger with `requested`, `normalized parameter`, `ambient override`, -`source invariant`, `counterexample`, `observed/unknown`, and `next check`. -Finish with the smallest discriminating check, not an inferred speed/quality claim. - -## Procedure - -1. **Resolve the effective request.** FastH3 normalizes the API parameters to - four steps, 50 layers, denoise/core reuse 1, token-reduction parameter 0 and - SSD streaming 0 before validation. Ordered Ref2VA references are rejected; - FL2VA first/last-frame anchors are a different input route. Capture ambient - token-reduction controls: normalization does **not** erase their later effect. - Record both requested and aligned frame counts; requested 49 aligns to 56. -2. **Check checkpoint identity before mapping.** The DiT's path-substring switch - and schedule's key-presence switch differ. Identify both, reject mixed-family - assumptions and inspect the complete intended config/schema. Matching tensor - names/shapes or an inferred header is not conversion, dtype or payload proof. -3. **Audit transformations, not just names.** Separate Q/K/V tensors must become - per-head Q/K/V groups, not one global Q-then-K-then-V concatenation. Swap FC1 - halves before quantization. Preserve mapped BF16-to-F32 consumers, the BF16 - timestep MLP/materialization order, separate audio/video sigma rows and the - generated RoPE dependency. Use labeled tiny fixtures to falsify the wrong - permutation; do not call such a fixture numerical model parity. -4. **Reconstruct VSA membership.** Merge adjacent same-kind prefix segments, - then split each merged segment into at-most-64-row tiles. Target video uses - ceil-divided 4×4×4 blocks on the patchified latent grid. Pool real boundary - rows only. Prefix queries retain every key tile; video queries retain all - prefix keys plus top-k video keys. Preserve score-descending/index-ascending - ties, row/head-major distinctions and scatter order. -5. **Establish actual dispatch.** Top-k uses double-precision arithmetic; 120 - video tiles yield 12, not the float32 counterexample's 13. Sparse fine requires - an available pipeline, no sparse-disable environment key, and a full sequence. - The disable key's **presence**, even value `0`, disables it. Sparse fine already - includes gated coarse; fallback is **dense fine plus gated coarse**, not base - H3 and not a reason to add coarse twice. Unknown pipeline availability stays - unknown. Check the [failure table](references/troubleshooting.md). -6. **Run only the authorized layer.** The bundled - [offline audit](scripts/audit_fasth3.py) accepts a caller-declared JSON object - and performs selected configuration/tile arithmetic, without probing this - machine, loading weights or running source scripts. Its - [input/output contract](references/preflight.md) includes executable examples - and counterchecks. Native candidates and required resources are classified in - the [source notes](references/source-notes.md#native-candidates-and-script-dispositions). - No CPU helper replaces their missing Apple/Metal/model evidence. - -## Stop and report - -Stop at conflicting configuration, mixed checkpoint identity, missing resources -or the authorized boundary. Name the next native test/fixture needed and report -it as blocked/not_run, not passing. The key-map and benchmark scripts were -inspected but are **not** bundled execution dependencies. Reading this procedure -does not authorize downloading, conversion, generation, benchmarks, upstream -publication or candidate promotion. Missing acceptance evidence is INCONCLUSIVE. - -The [provenance and licensing note](references/repo-provenance.md) explains why -the machine-resolver fallback `NO_LICENSE` does not negate the inspected MIT -source license or the retained BSD-3-Clause component notice. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/metadata.json deleted file mode 100644 index 2a83f4b35..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/metadata.json +++ /dev/null @@ -1,106 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NO_LICENSE", - "tags": [ - "h3-metal", - "FastH3", - "MiniMax-H3", - "apple-silicon", - "metal", - "qkv-layout", - "swiglu", - "bf16", - "vsa", - "sparse-attention", - "fallback", - "effective-configuration" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/zuiho-kai/h3-metal", - "commit": "4ba47a3eabdbebf801ac6664b31f8ec168bc57a8", - "paths": [ - "LICENSE", - "THIRD_PARTY_NOTICES.md", - "Makefile", - "README.md", - "docs/USAGE.md", - "h3.h", - "h3.c", - "main.c", - "h3_host.h", - "h3_host.c", - "h3_dit.c", - "h3_dit_schedule.c", - "h3_gpu.m", - "h3_shaders.metal", - "tests/test_h3.c", - "tests/test_vsa_coarse.c", - "tests/test_real_dit_schedule.c", - "tests/bench_dit.c", - "tools/fasth3_keymap.py", - "tools/fasth3_keymap.md", - "bench/run_fox2.sh" - ], - "readset": "references/source-notes.md", - "identity": "references/source-identity.json", - "license": "MIT; selected shader-derived components separately BSD-3-Clause", - "third_party_notices": "references/THIRD_PARTY_NOTICES.md" - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill", - "cli/packages/coding-agent/src/disco/skills/extend-repo-skill" - ], - "scope": "Portable source/CPU-only preparation, extraction, safe-script disposition, difficult cases and verification adaptation to InferRSI ownership; no DisCo CLI/model run or managed import." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "capability-specific", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Bounded fork-specific source audit, bundled selected-declaration helper and assertion-backed cases. Native Apple/Metal/model evidence remains blocked; no numerical/performance/skill-efficacy claim." - }, - "applicability": { - "scope": "h3-metal FastH3 requested/effective configuration, checkpoint transformation and schedule/RoPE semantics, VSA prefix/partial tiles and dispatch/fallback audit. Common catalog placement is discovery, not CPU inference or Apple qualification.", - "prerequisites": "Access to the fork-specific source and semantic contracts; native checks additionally need their matching Metal/model resources and separate execution authority.", - "input_contract": "Source revision/patches, requested parameters and explicitly captured environment controls, checkpoint path/key-family/config identity, requested/aligned frames, latent dimensions, packed prefix segments and sparse-pipeline availability or unknown.", - "produced_artifact": "A ledger of requested and normalized parameters, ambient overrides, source invariants, counterexamples, observed/unknown facts and the smallest discriminating next check.", - "failure_modes": "Ignoring ambient overrides, mixing checkpoint identity switches, using global rather than per-head Q/K/V concatenation, pooling padded boundary rows, or double-adding coarse attention can invalidate the semantic audit.", - "refusal_point": "Stop on conflicting configuration, mixed checkpoint identity, missing resources or the authorized boundary; name the missing native test/fixture and leave it blocked/not_run." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: native Apple/Metal, actual FastH3 checkpoint/schedule/sparse parity, audio/video quality, target memory fit, performance and service evidence not_run.", - "Offline helper audits caller declarations and selected arithmetic only, not engine parameter validity, payload/dtype/conversion equivalence, attention numerics, actual environment or dispatch.", - "Bounded file/range readset only; full kernels, model/config/weights, encoders/VAEs, device/runtime behavior and external FastVideo conformance are not established.", - "Native coarse VSA and original-H3 schedule test definitions do not cover sparse fine or FastH3 BF16 schedule execution; no native tests were run.", - "Key-map and benchmark source scripts are excluded from execution; checked-in reports, source comments and existing measurement documents are not new local evidence.", - "NO_LICENSE is the actual unauthenticated machine-resolver fallback, not absence of licensing. Inspected MIT source and BSD-3-Clause notices are retained; model rights remain separate/unverified." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/source-identity.json", - "references/repo-provenance.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Record requested parameters, normalized values and later environment overrides separately, including requested versus aligned frame counts.", - "Audit checkpoint transformations and schedule consumers beyond tensor names/shapes; tiny permutation fixtures can falsify mappings but do not establish model parity.", - "Resolve sparse eligibility and the dense-fine-plus-gated-coarse fallback explicitly; preserve unknown pipeline availability." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/husky/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/husky/metadata.json deleted file mode 100644 index 378a5062b..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/husky/metadata.json +++ /dev/null @@ -1,72 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NO_LICENSE", - "tags": [ - "husky", - "document-research", - "evidence-qualification", - "negative-results" - ], - "sources": [ - { - "type": "document", - "url": "https://husky.underdog.ai/", - "displayed_publication_date": "2026-09-20", - "accessed": "2026-10-27", - "license": "UNKNOWN", - "scope": "Public unversioned official-page view; bounded author-report summary, not source code or retained receipts." - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "AREX-informed document-only adaptation explicitly accepted by the user; not complete repo-to-skill or native verification." - } - ], - "construction": { - "anchor_kind": "document", - "distillation_form": "document-research-procedure", - "strategy": "arex-informed-document-adaptation", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Single coherent research workflow, bounded source summary, original verification questions and offline schema checks. Repository/environment/native stages remain unavailable." - }, - "applicability": { - "scope": "Review Husky document claims and specify missing evidence; common catalog scope is not inference backend compatibility.", - "prerequisites": "Access to the bounded official-document inventory and verification protocol; implementation and native reproduction remain unavailable until separately supplied and authenticated.", - "input_contract": "The exact claim, source/view date, workload and comparison intent, supporting evidence classes and available implementation/model/output/measurement artifacts; unknown identity, licensing and availability remain explicit.", - "produced_artifact": "A reviewable claim/evidence request recording source availability, mechanism hypothesis, counter-signal, missing identities/artifacts, proposed falsifying check and actual result or bounded INCONCLUSIVE conclusion.", - "failure_modes": "Treating a public-page reading as code inspection, proposed extensions as shipped behavior, summed independent ratios as one gain, or a structurally valid self-reported card as authentic evidence can overstate qualification.", - "refusal_point": "Stop at missing source/receipts, unsafe or private data, unsupported resources or the authorized budget; unavailable artifacts yield an unverified/INCONCLUSIVE research conclusion." - }, - "limitations": [ - "No canonical public repository, implementation revision, retained receipts or source-code license was identified; unknowns remain unknown.", - "The unversioned official page is author-reported and unreproduced; no immutable source HTML hash is claimed.", - "Native reproduction is unavailable; schema checks do not authenticate supplied identities, artifacts, output equality or performance.", - "Research protocols and ablation questions are original proposals, not claims about uninspected implementation behavior.", - "NO_LICENSE is the unavailable repository-license resolver result, not a legal absence finding or grant to copy publisher material.", - "Final informed candidate admission is separate; no automatic import, native qualification or skill efficacy is claimed." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-availability.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Separate author reports, inspected implementation, executed experiments and new interpretations, including negative findings and future work.", - "Request immutable identities and artifacts, then state one mechanism, competing explanation and smallest ablation with frozen semantic requirements.", - "Use the offline card auditor only to identify structural gaps; it authenticates neither artifacts nor execution and leaves qualification unadjudicated." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/lily/SKILL.md deleted file mode 100644 index 30e096b30..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lily/SKILL.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -name: lily -description: "Investigate Lily's exact Qwen3.6-35B-A3B MLX-affine-Q4 contract, Metal state and kernels, strict greedy text API, token-prefix session cache and source-qualified validation or benchmark plans." -license: NO_LICENSE -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# Lily inference engineering - -Use this source-derived candidate to investigate representation, state or -qualification questions in `perplexityai/pplx-garden/lily`. It is not a generic -Metal/MLX runner. Read [source and method provenance](references/repo-provenance.md) -before applying it to a different revision, model or device. - -## Route the question - -| Question | Route | Expected result | -| --- | --- | --- | -| A nominally 4-bit model fails to load, or memory/sequence capacity is unclear | [model-runtime](sub-skills/model-runtime/SKILL.md) | Exact architecture/layout and ownership/capacity contract | -| A request is rejected, or equal keys/history do not reuse state as expected | [serving-cache](sub-skills/serving-cache/SKILL.md) | Strict API, token-prefix and state-lifetime diagnosis | -| CPU references look green, a golden is ignored, or MLX comparisons disagree | [qualification](sub-skills/qualification/SKILL.md) | Matching native evidence, version and metric boundaries | - -The focused routes bundle safe offline screening/request support where useful. -Those helpers do not invoke Rust, compile shaders, load tensors, tokenize a real -checkpoint or start inference. Their positive result is not native acceptance. - -## Establish the execution tuple - -Record engine/source and Cargo lock/toolchain, macOS/chip/GPU family, exact -checkpoint and tensor payload identities, config/affine layout, tokenizer/chat -template and output-stop identity. Add sequence/output budgets, greedy policy, -input history, prefix-cache state, workload and quality/measurement requirements. - -Equal nominal bit width, file size, API name or `prompt_cache_key` does not -identify equivalent tensors, rendered tokens or valid recurrent/KV state. -Separate reported benchmark identity from the checked-out runner's version. - -## Native preparation boundary - -The source requires Rust 1.92, macOS 26 and Apple GPU family 10/M5 or newer. -For an authorized source build the public command is -`cargo build --release --locked`; shader source is compiled at runtime, not by -an offline metallib step. Inspect an installed binary's help before any model -run. Building a package does not supply its supported checkpoint or qualify it. - -This extraction has neither Cargo/Rust nor a compatible Mac/model environment. -AREX's Python-package preparation was explicitly adapted to native Rust/Metal -identity and small offline helper checks. No installed Python distribution, -Rust test run or native backend success is fabricated. - -## Work one bounded hypothesis - -1. Read the selected bundled contract and failure notes. -2. Resolve exact input representation and actual API semantics before tuning. -3. Trace owner, logical token frontier, allocation, cache key and release or - restore boundary. Matching visible text is weaker than exact token identity. -4. Preserve the source's extra request lookahead slot when budgeting; do not - replace its checked arithmetic with only prompt-plus-visible-output length. -5. Freeze workload, checkpoint/version, numerical/quality controls and metrics - before comparing. Gates and campaign revisions belong in an acceptance issue. -6. Distinguish source inspection, offline helper checks, compiled native tests, - real Metal operators, matched-model goldens and service measurements. - -## Stop and report - -Return exact source/model/device identity, observation, invariant, alternate -cause, smallest falsifier, actual check result and remaining gap. Stop at -unsupported format/API, missing compiler/backend/checkpoint, unfrozen gates or -the authorized budget. Do not install a toolchain, download a model, alter the -API or weaken an ignored/golden test merely to obtain a green result. - -Required native compiler/Metal/model evidence remains blocked. CPU reference -functions and Python screening are not GPU test execution; skipped or ignored -cases do not become passing evidence. No quality, performance, service or -behavioral skill-efficacy qualification is claimed. Final partial-candidate -admission is a separate informed decision, not automatic import. - -Source [Apache-2.0 license](LICENSE) and [NOTICE](NOTICE.md) are retained. -`NO_LICENSE` is the actual AREX unauthenticated-CLI fallback, not a legal finding; -model weights and third-party material retain their own obligations. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/lily/metadata.json deleted file mode 100644 index e986edc1e..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lily/metadata.json +++ /dev/null @@ -1,117 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NO_LICENSE", - "tags": [ - "lily", - "rust", - "metal", - "apple-silicon", - "mlx-affine-q4", - "session-prefix", - "numerical-validation", - "benchmark-provenance" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/perplexityai/pplx-garden", - "commit": "1ed972ed3f0bd5616c997c9507c25616c63394fc", - "paths": [ - "lily/README.md", - "lily/Cargo.toml", - "lily/Cargo.lock", - "lily/rust-toolchain.toml", - "lily/CONTRIBUTING.md", - "lily/LICENSE", - "lily/NOTICE", - "lily/src/lib.rs", - "lily/src/config.rs", - "lily/src/weights.rs", - "lily/src/safetensors.rs", - "lily/src/tensor.rs", - "lily/src/metal.rs", - "lily/src/model.rs", - "lily/src/serve.rs", - "lily/src/serve/session.rs", - "lily/src/chat.rs", - "lily/src/generate.rs", - "lily/src/tokenizer.rs", - "lily/src/kernels/attention.rs", - "lily/src/kernels/gdn.rs", - "lily/src/kernels/gemm.rs", - "lily/src/kernels/skinny.rs", - "lily/src/bin/lily-bench.rs", - "lily/tests/support/cpu_ref.rs", - "lily/tests/test_shader_compile.rs", - "lily/tests/test_e2e_35b.rs", - "lily/tests/test_tokenizer.rs", - "lily/tests/unit/weights.rs", - "lily/tests/unit/serve.rs", - "lily/tests/unit/serve/session.rs", - "lily/tests/unit/generate.rs", - "lily/tests/unit/kernels/gemm.rs", - "lily/tests/unit/kernels/gdn.rs", - "lily/tests/unit/kernels/skinny.rs", - "lily/docs/2026-09-01-performance-mlx-0.31.2.md", - "lily/docs/2026-09-02-performance-mlx-0.32.2.md", - "lily/benchmarks/run_matrix.py", - "lily/benchmarks/bench_mlx.py", - "lily/benchmarks/requirements.txt" - ], - "license": "Apache-2.0", - "scope": "Lily subtree only; independent lily/LICENSE and NOTICE retained.", - "readset": "references/source-identity.json" - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Portable native Rust/Metal adaptation executed by a compatible agent; not a DisCo CLI/model run." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "User-approved native Rust/Metal and partial drafting adaptation; stable-ID lanes, integrated source graph and assertion/helper verification. Missing compiler/backend/checkpoint remains explicit." - }, - "applicability": { - "scope": "Exact Lily source/checkpoint/layout/API/cache/qualification investigation; common catalog classification is not CPU inference or cross-device support.", - "prerequisites": "Access to the exact Lily source/model-layout/API contracts; native checks require the matching Rust/Metal/checkpoint environment and frozen comparison requirements.", - "input_contract": "Source revision, Cargo lock/toolchain, macOS/chip/GPU family, checkpoint and tensor payload identities, affine configuration/layout, tokenizer/template/stops, sequence/output budgets, greedy policy, input history, prefix-cache state and quality/measurement requirements.", - "produced_artifact": "A source/model/device-bound representation, serving-cache or qualification diagnosis with the observation, invariant, alternate cause, smallest falsifier, actual result and remaining native gap.", - "failure_modes": "Equal bit width, file size, API names or prompt_cache_key do not establish tensor/token/state equivalence; ignoring the extra lookahead slot can understate request capacity, and green CPU references do not replace matched-model Metal goldens.", - "refusal_point": "Stop at unsupported formats/APIs, missing compiler/backend/checkpoint, unfrozen gates or the authorized budget; do not weaken ignored/golden tests to obtain a green result." - }, - "limitations": [ - "Cargo/Rust1.92, supported M5/macOS26/Metal and model resources are unavailable; native evidence remains BLOCKED_REQUIRED_BACKEND.", - "Offline Python helper checks are not Rust serde/loader/tokenizer, native kernel/state/cache, model quality, service or performance validation.", - "Current benchmark source and later report have distinct package/harness identities; negative long-context prefill observations remain author-reported.", - "Only selected source bodies are mined; complete kernel/format/platform/failure coverage and behavioral skill efficacy remain unverified.", - "NO_LICENSE is the actual unauthenticated GitHub CLI fallback, not a legal conclusion; lily Apache-2.0 LICENSE and NOTICE are independently retained." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve exact affine representation and strict API semantics, then trace token frontier, cache key and restore/release ownership.", - "Retain the source's extra request lookahead slot in capacity arithmetic rather than counting only prompt and visible output.", - "Separate reported benchmark version from the checked-out runner and distinguish offline screening, native compilation, Metal operators and matched-model goldens." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/metadata.json deleted file mode 100644 index 5c1dc9cf4..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/metadata.json +++ /dev/null @@ -1,73 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "lithos-metal", - "metal", - "apple-silicon", - "repository-guide" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/lithos-ai/lithos-metal", - "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", - "paths": [ - "kernels/common/spec_ops.metal", - "monolith/backends/metal/base.py", - "monolith/backends/metal/registry.py", - "monolith/compiler/arena.py", - "monolith/compiler/barriers.py", - "monolith/compiler/coverage.py", - "monolith/compiler/decoder_fusion.py", - "monolith/compiler/emit.py", - "monolith/compiler/region_fusion.py", - "monolith/compiler/static_fusion.py", - "monolith/core/step_state.py", - "monolith/generate.py", - "monolith/runtime/engine.py", - "monolith/runtime/prefix_cache.py", - "monolith/runtime/prefix_store.py", - "monolith/serve.py", - "monolith/serving/cache.py", - "monolith/serving/lifecycle.py", - "monolith/serving/setup.py", - "monolith/spec/dspark/model.py", - "runtime/src/metal_core.mm" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately.", - "prerequisites": "Access to the pinned package/API and selected compiler, generation or serving contracts; revision/API drift needs a fresh bounded inspection, and efficacy claims need campaign-owned gates and matching native/model evidence.", - "input_contract": "Actual source/package version, backend/configuration, chip/core count and OS, target/draft checkpoint identities, context and prompt-token IDs, sampling/numerical options and the affected recipe/runtime/session/cache boundary.", - "produced_artifact": "A routed region/dependency, committed-state or cache-identity/lifecycle ledger, followed by the smallest supported finding, missing checks and next bounded action.", - "failure_modes": "Treating session adoption/cache hits as generation correctness, fewer dispatches as faster rounds, another chip recipe as the default, CPU cache checks as GPU rollback or health liveness as loaded/warm model state can yield unsupported conclusions." - }, - "limitations": [ - "Source-derived candidate; empirical efficacy validation not_run.", - "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", - "No model downloads, live DisCo deployment or inference backend integration is provided." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "license": "Apache-2.0", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - }, - "recommendations": [ - "Follow session replacement across recipe geometry, committed generation state and serving adoption/cache identity before interpreting a cache hit.", - "Keep an unfused/native or target-only control with matching decoded weights, numerical mode, row bounds and sampling policy; verify dependencies and persistent state before timing.", - "Distinguish package/CPU checks, synthetic Metal execution, model continuation and HTTP workload evidence; chip recipes and target-compute projections do not establish another device or E2E rate." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/metadata.json deleted file mode 100644 index 4a412dfcf..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/metadata.json +++ /dev/null @@ -1,52 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "lithos-metal", - "metal", - "apple-silicon", - "compile-fusion" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/lithos-ai/lithos-metal", - "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", - "paths": [ - "monolith/compiler/emit.py", - "monolith/compiler/region_fusion.py", - "monolith/compiler/static_fusion.py", - "monolith/compiler/decoder_fusion.py", - "monolith/compiler/barriers.py", - "monolith/compiler/arena.py", - "monolith/compiler/coverage.py", - "monolith/backends/metal/base.py", - "monolith/backends/metal/registry.py" - ], - "readset": "../../references/source-notes.md" - } - ], - "applicability": { - "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately." - }, - "limitations": [ - "Source-derived candidate; empirical efficacy validation not_run.", - "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", - "No model downloads, live DisCo deployment or inference backend integration is provided." - ], - "provenance_readsets": [ - "../../references/source-notes.md" - ], - "consumption_mode": "packaged", - "license": "Apache-2.0", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - } -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/metadata.json deleted file mode 100644 index 921695d95..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/metadata.json +++ /dev/null @@ -1,49 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "lithos-metal", - "metal", - "apple-silicon", - "generation-state" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/lithos-ai/lithos-metal", - "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", - "paths": [ - "monolith/core/step_state.py", - "monolith/generate.py", - "monolith/spec/dspark/model.py", - "monolith/runtime/engine.py", - "runtime/src/metal_core.mm", - "kernels/common/spec_ops.metal" - ], - "readset": "../../references/source-notes.md" - } - ], - "applicability": { - "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately." - }, - "limitations": [ - "Source-derived candidate; empirical efficacy validation not_run.", - "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", - "No model downloads, live DisCo deployment or inference backend integration is provided." - ], - "provenance_readsets": [ - "../../references/source-notes.md" - ], - "consumption_mode": "packaged", - "license": "Apache-2.0", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - } -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/metadata.json deleted file mode 100644 index 9868575ca..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/metadata.json +++ /dev/null @@ -1,50 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "lithos-metal", - "metal", - "apple-silicon", - "serving-cache" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/lithos-ai/lithos-metal", - "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", - "paths": [ - "monolith/serving/cache.py", - "monolith/serving/setup.py", - "monolith/runtime/prefix_cache.py", - "monolith/runtime/prefix_store.py", - "monolith/serving/lifecycle.py", - "monolith/serve.py", - "monolith/generate.py" - ], - "readset": "../../references/source-notes.md" - } - ], - "applicability": { - "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately." - }, - "limitations": [ - "Source-derived candidate; empirical efficacy validation not_run.", - "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", - "No model downloads, live DisCo deployment or inference backend integration is provided." - ], - "provenance_readsets": [ - "../../references/source-notes.md" - ], - "consumption_mode": "packaged", - "license": "Apache-2.0", - "disable-model-invocation": true, - "metadata": { - "disco-role": "operating" - } -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/metadata.json deleted file mode 100644 index bd071221d..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/metadata.json +++ /dev/null @@ -1,69 +0,0 @@ -{ - "tags": [ - "llama.cpp", - "context", - "state", - "graph-reuse", - "cancellation", - "correctness" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ggml-org/llama.cpp", - "commit": "707616feb5f379e99ff7442680a55b4a1704c7c9", - "paths": [ - "LICENSE", - "include/llama.h", - "src/llama-context.cpp", - "src/llama-context.h", - "src/llama-memory.h", - "src/llama-memory-recurrent.cpp", - "src/llama-graph.cpp", - "src/llama-graph.h", - "ggml/src/ggml-backend.cpp", - "tools/server/server-context.cpp", - "tools/server/server-queue.cpp", - "tools/server/server-queue.h", - "tests/llama-context/cases-kv.cpp", - "tests/llama-context/cases-rs.cpp", - "tests/CMakeLists.txt", - "tests/test-llama-archs.cpp", - "tests/test-backend-ops.cpp", - ".github/workflows/build-cann.yml", - "tools/server/tests/unit/test_completion.py" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Pinned llama.cpp context-memory, graph-reuse and server task-lifecycle diagnosis; concrete model/backend behavior requires native validation.", - "prerequisites": "Pinned llama.cpp source or a concrete runtime trace, access to the relevant memory/graph/server ownership evidence and an identified caller recovery policy that remains in force.", - "input_contract": "Source/build and model revisions, memory type, actual backend/device and execution mode, context/batch/ubatch parameters, sequence IDs/positions/output flags and return status; graph cases need dispatch evidence, server cases need correlated HTTP/task/slot/cache logs.", - "produced_artifact": "An owner/event ledger of ubatch progress, retained memory, graph-gate decisions, input readiness and slot release/retention, with the first supported broken obligation or smallest missing observation.", - "failure_modes": "Assuming whole-batch rollback after partial decode failure, treating token count as a complete graph-reuse key, or equating posted cancellation with released slots can hide retained state or misdiagnose correct deferred work.", - "refusal_point": "Stop before reset or repair if revision, memory type, actual backend, sequence ownership or partial progress is unknown; escalate unsupported rollback or unresolved in-flight work." - }, - "limitations": [ - "Source-derived candidate; native runtime and efficacy validation not run.", - "No cross-backend preemption, model-quality or performance claim.", - "KV, recurrent and hybrid state contracts must be checked independently." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Reconcile per-sequence retained positions with completed ubatches before retrying a failed logical batch.", - "Check both graph allow_reuse and input can_reuse gates, then establish copy readiness independently of graph identity.", - "Trace cancellation posting, normal task processing, slot release and prompt retention as separate events." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/metadata.json deleted file mode 100644 index 3f383464b..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/metadata.json +++ /dev/null @@ -1,37 +0,0 @@ -{ - "tags": [ - "kv-cache", - "offload", - "l1-l2-tiering", - "eviction", - "prefetch", - "serving", - "lmcache" - ], - "status": "candidate", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [], - "consumption_mode": "packaged", - "applicability": { - "scope": "LMCache multiprocess runtime KV tier policy across engine GPU cache, host L1 and configured L2 adapters, covering store/retention, prefetch admission, eviction and RAM pressure; startup loading and checkpoint recovery have separate owners.", - "prerequisites": "The deployment uses the inspected MP-mode semantics rather than single-process knobs; selected GDS/DAX/raw-block paths need their own platform and adapter prerequisites. Lazy offload additionally requires vLLM prefix caching.", - "input_contract": "MP versus single-process mode, store/retention policy and rationale, L1 manager capacities and eviction settings, L2 adapter capacities/eviction, chunk-size distribution and prefetch in-flight setting, together with token-level hit and pressure counters.", - "produced_artifact": "A mode- and adapter-specific tier-policy choice with retention rationale, an ordered RAM-release plan and a verification record separating L1/L2 token hits, prefetch pressure, store lag and per-manager usage.", - "failure_modes": "Mixing MP and in-process controls, treating key-count eviction as a byte fraction for unequal chunks, omitting fs capacity management, or increasing count-based prefetch admission without tracking L1 pressure can make the policy or memory claim unusable." - }, - "recommendations": [ - "Choose store retention and prefetched-key retention separately; use skip_l1 with noop only for buffer-only operation and account for failed stores remaining in L1.", - "Tune the count-based prefetch in-flight limit against L1 allocation failures and in-flight bytes; eager or warm prefetch can raise pressure before useful reuse.", - "Record per-adapter capacity/eviction and actual chunk-size uniformity; report token-level L1/L2 hits and sum per-l1_tag bytes rather than ratios." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/SKILL.md deleted file mode 100644 index 7bc0b0371..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/SKILL.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -name: local-dream -description: "Audit Local Dream Android desired-versus-serving process state, remote exposure, native backend and DiT packaging/ABI selection, shared generation cancellation and prompt-cache identity without mistaking declarations or source tests for on-device diffusion qualification." -license: NOASSERTION -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# Local Dream Android diffusion boundary audit - -Use this source-derived **candidate** when a model page reports the wrong -backend, a quick reopen races a stop, host mode changes reachability, DiT fails -to load, generation appears stuck, or prompt-cache reuse is suspect. Start with -[source identity and rights](references/repo-provenance.md). This is original -bounded analysis of `xororz/local-dream`, not an Android installer, device -compatibility certification or a managed DisCo import. - -## Route to the failing boundary - -| Symptom / question | Procedure | -| --- | --- | -| Same model but wrong resolution/backend/bind address; stop/reopen/restart; stale Error | [Android desired/serving lifecycle](references/android-lifecycle.md) | -| Remote control says Ready, or comments imply pairing/authentication | [Actual host exposure and readiness](references/android-lifecycle.md); never enable host mode as a diagnostic default | -| CPU/MNN versus QNN versus Anima versus dynamically loaded DiT; SSE disconnect or upscaler contention | [Native dispatch, cancellation and cache](references/native-contracts.md) | -| Missing engine/entry point/ABI/skel, QNN runtime mixing, APK versus DSP packaging | [Packaging and native prerequisites](references/packaging-and-abi.md) | -| Same prompt gives suspicious reuse after export/model/textual-inversion changes | [Cache identity and caller guards](references/native-contracts.md); shape/header acceptance is not model-revision invalidation | -| Only declarations are available | [Offline declaration screen](scripts/screen_declarations.py), whose explicit non-native limits are below | - -## Establish identities before suggesting a change - -1. Record the source revision, installed APK/version/flavor/ABI, actual native - executable and separately loaded DiT library, QNN/Hexagon/MNN build identities, - selected model/export/weight/tokenizer identities, requested configuration and - **serving** configuration. Use anonymized IDs and `` in shared - reports. A source version string or file existence is not installed identity. -2. Follow actual backend dispatch, not the README headline. SD1.5/SDXL MNN and - QNN paths, Anima and `zimage`/`klein`/`qwen21`/`krea2turbo` DiT routing are not - interchangeable. A UI `runOnCpu` flag does not alone specify the executed - MNN CPU/GPU provider or the DiT engine's device/parameter residency. -3. Compare the full process configuration: model, backend type, width, height - and `listenOnAll`. A process still alive in the stop grace window is not - evidence that it serves the newly requested configuration. Follow both - process state and health/config observations. `Running` is published after - process start, before proven native readiness; the local health consumer - checks model identity but not every serving field. Ready is not image - completion. Equality also omits startup preferences and changed model bytes; - an equal tuple is not proof that such changes were applied. -4. Separate startup, model load, prompt encoding, denoising, VAE work, preview, - safety filtering, final response, history persistence and teardown. SSE - progress, a client cancellation request, native completion, buffer release - and saved final output are different events. Native/shared resource locks - can serialize generation and upscaling; do not infer parallel NPU capacity. -5. Treat persistent prompt caches as model-directory-scoped implementation - caches, not authenticated model-revision namespaces. Check mode, version, - dimensions and callers' textual-inversion bypass. Preserve failure signals - rather than deleting caches or model directories without authorization. - -## Safe offline use - -The bundled standard-library helper reads a small explicit declaration object -from stdin or one regular JSON file; it makes no network, subprocess, Android, -model, tensor, native-library, filesystem-discovery or device calls. It compares -the five declared configuration fields and separately declared ABI/SDK labels. -It does **not** reproduce coroutine timing, resolution snapping, Android service -rules, dynamic loading, symbol-table validation, runtime stamping/copy behavior, -prompt-cache formats or native execution. - -```bash -python /scripts/screen_declarations.py --example -python /scripts/screen_declarations.py -``` - -An equal configuration can still be dead, unready or serving stale bytes. Equal -ABI/SDK labels can still describe incompatible or corrupted artifacts. The -helper returns field names and fixed diagnostics, not supplied identifiers; it -always retains `native_execution: not_run` and `qualification: INCONCLUSIVE`. -Its zero exit means valid input and completed screening, not native success. - -## Evidence required for a stronger conclusion - -Required Android/ARM64 and actual selected MNN/QNN/Hexagon/model execution remains -`BLOCKED_REQUIRED_BACKEND / not_run`; there is no full CPU substitute for it. -Source assertions, Gradle configuration, offline helper tests and InferRSI wheel -access are separate evidence classes. No upstream test, APK build/sign/install, -native loader, live HTTP endpoint, model/image quality, memory fit, thermal, -latency, throughput, service or broad skill-efficacy result is claimed here. - -For an authorized future run, freeze a RunSpec and workload first. Require -actual backend identity, model/tokenizer/export hashes, load and readiness -records, final image plus quality validator, cancellation and teardown outcomes, -memory/thermal controls and complete timing boundaries appropriate to the claim. -Source hardware gates and implementation constants are not portable acceptance -thresholds. Return the failed boundary, counter-signal, identities observed vs -declared, smallest missing discriminator and retained `INCONCLUSIVE` scope. - -Upstream's pinned LICENSE is **CC BY-NC 4.0**, while repository API SPDX metadata -reports `NOASSERTION`. The latter is preserved, not permission for commercial -reuse or relicensing. See [rights and exclusions](references/repo-provenance.md). diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/metadata.json deleted file mode 100644 index 4f846892c..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/metadata.json +++ /dev/null @@ -1,111 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NOASSERTION", - "tags": [ - "local-dream", - "android", - "snapdragon", - "qnn", - "hexagon", - "mnn", - "diffusion", - "lifecycle", - "cancellation", - "prompt-cache", - "abi", - "lan-exposure" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/xororz/local-dream", - "commit": "db162df4d9bb6efc6595250423d8df26241f4c27", - "license": "CC-BY-NC-4.0; repository API SPDX NOASSERTION retained separately", - "readset": "references/source-identity.json", - "paths": [ - ".gitmodules", - "LICENSE", - "README.md", - "app/build.gradle.kts", - "app/src/main/cpp/CMakeLists.txt", - "app/src/main/cpp/CMakePresets.json", - "app/src/main/cpp/build.sh", - "app/src/main/cpp/dit/CMakeLists.txt", - "app/src/main/cpp/dit/DitEngine.cpp", - "app/src/main/cpp/dit/build.sh", - "app/src/main/cpp/include/DitEngine.h", - "app/src/main/cpp/src/Config.hpp", - "app/src/main/cpp/src/Pipeline.hpp", - "app/src/main/cpp/src/PipelineAnima.hpp", - "app/src/main/cpp/src/PipelineDit.hpp", - "app/src/main/cpp/src/PipelineSd15Cpu.hpp", - "app/src/main/cpp/src/PipelineSd15Npu.hpp", - "app/src/main/cpp/src/PipelineSdxl.hpp", - "app/src/main/cpp/src/PipelineSdxlMnn.hpp", - "app/src/main/cpp/src/PromptCache.hpp", - "app/src/main/cpp/src/QnnRuntime.hpp", - "app/src/main/cpp/src/main.cpp", - "app/src/main/java/io/github/xororz/localdream/data/DitEngine.kt", - "app/src/main/java/io/github/xororz/localdream/data/DitResolution.kt", - "app/src/main/java/io/github/xororz/localdream/data/Model.kt", - "app/src/main/java/io/github/xororz/localdream/remote/RemoteApiClient.kt", - "app/src/main/java/io/github/xororz/localdream/remote/RemoteHostServer.kt", - "app/src/main/java/io/github/xororz/localdream/remote/RemoteProtocol.kt", - "app/src/main/java/io/github/xororz/localdream/service/BackendService.kt", - "app/src/main/java/io/github/xororz/localdream/service/BackgroundGenerationService.kt", - "app/src/main/java/io/github/xororz/localdream/service/RemoteHostService.kt", - "app/src/main/java/io/github/xororz/localdream/ui/screens/ModelRunScreen.kt", - "app/src/main/java/io/github/xororz/localdream/ui/screens/ModelRunSupport.kt", - "app/src/test/java/io/github/xororz/localdream/ExampleUnitTest.kt", - "app/src/test/java/io/github/xororz/localdream/ModelStorageTest.kt" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "readset": "references/source-identity.json", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill/SKILL.md", - "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/repository-evidence.md", - "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/input-output-and-structure.md", - "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/planning-and-writing.md", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env/SKILL.md", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill/SKILL.md", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill/references/usability-test-cases.md" - ] - } - ], - "applicability": { - "scope": "Selected Android app/process and native diffusion boundary contracts; common is a discovery category, not CPU/device compatibility or native qualification.", - "prerequisites": "Access to the pinned Android lifecycle/native/packaging contracts and identified installed artifacts; stronger conclusions require an authorized, frozen workload on the actual selected Android/ARM64 backend.", - "input_contract": "Source revision, installed APK/version/flavor/ABI, native executable and loaded DiT library, QNN/Hexagon/MNN builds, model/export/weights/tokenizer identities, requested and serving configuration, and lifecycle/readiness/generation/cache observations.", - "produced_artifact": "A bounded failed-boundary report with counter-signals, identities observed versus declared, the smallest missing discriminator and retained INCONCLUSIVE native scope.", - "failure_modes": "Reading UI flags or equal declared configurations as actual dispatch/readiness, treating cancellation/SSE progress as native completion, or interpreting model-directory caches as authenticated model-revision namespaces can misidentify stale state." - }, - "limitations": [ - "Source-derived candidate; required Android ARM64/MNN/QNN/Hexagon/model execution BLOCKED_REQUIRED_BACKEND / not_run; qualification INCONCLUSIVE.", - "Portable AREX-informed source analysis and InferRSI candidate integration, not DisCo CLI execution, managed import or full AREX verification.", - "Original upstream Gradle/JVM/native tests, APK build/sign/install, loader/device/model/media-quality/memory-fit/thermal/service/performance and agent-efficacy checks not_run.", - "Offline helper compares supplied exact configuration and ABI/SDK labels only; it does not authenticate process state, readiness, artifact bytes, cache correctness or runtime behavior.", - "LAN control is unauthenticated in the inspected implementation despite stale comments; audit guidance does not authorize enabling host mode or exposing ports.", - "NOASSERTION is machine metadata, not license absence or commercial permission. Pinned LICENSE is CC BY-NC 4.0; retain restriction and attribution; SDK/model/dependency rights separate.", - "Full UI/navigation/import/download/storage/history/scheduler/kernel/security review and external dependency bodies excluded." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Compare requested and serving model/backend/dimensions/bind address through stop/reopen and health observations; never enable host mode as a diagnostic default.", - "Trace the actual MNN/QNN/Anima/DiT path and separate native library/ABI identity from UI runOnCpu flags and declared SDK labels.", - "Keep progress, cancellation, native completion, release and saved final output separate; check prompt-cache mode/version/dimensions and textual-inversion caller guards before proposing cache changes." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/SKILL.md deleted file mode 100644 index a82df4011..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/SKILL.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -name: mistral-rs -description: "Investigate mistral.rs Metal paged-attention decode layouts, prefill coordinates and workspace, actual-model prewarm and native test evidence without treating static metadata or prepared pipelines as executed GPU correctness or performance." -license: NO_LICENSE -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# mistral.rs Metal paged-attention contracts - -Use this source-derived **candidate** for the Rust/Candle engine's native Metal -paged-attention path. It is a focused repository operating graph, not a generic -model catalog, a deployment guide or a verified engine. Read -[source and method provenance](references/repo-provenance.md) before using it on -a different checkout. Reading it does not authorize installation or execution. - -## Route by the failed contract - -| Question | Canonical owner | -| --- | --- | -| Does HND/packed K/V, FP8 scale, GQA/head geometry, sliding decode or scratch agree? | [decode-contract](sub-skills/decode-contract/SKILL.md) | -| Do packed/padded queries, absolute positions, masks/ranges and prefill workspace agree? | [prefill-contract](sub-skills/prefill-contract/SKILL.md) | -| Did model warmup prepare the intended pipelines, and what did native tests actually prove? | [qualification](sub-skills/qualification/SKILL.md) | -| Which other engine features, dependencies and licenses are outside this extraction? | [scope and notices](references/scope-and-notices.md) | - -The selected source owns Rust shape/configuration/consumer code and actual MSL -attention, rather than merely naming an external backend. Candle supplies the -tensor/device integration. Shared ancestry with other Metal engines does not -make their cache layouts, scratch planners or synchronization interchangeable. - -## Establish identity before proposing a change - -1. Record exact mistral.rs/Candle/native artifact revisions, Rust toolchain, - Apple SDK/OS/device, actual model/cache dtype and representation, shape, - strides/offsets, block table, context and execution limits. Model and asset - identities are separate from source version or advertised support. -2. Identify the producer, view and native consumer of cache/metadata bytes. - Equal element counts, a layout setting and physical alias identity differ. - Decode and prefill deliberately have different supported geometry domains. -3. Select the smallest matching sub-skill and its bundled helper. Helpers use - explicit synthetic/caller-declared JSON and Python standard library only. - They do not import an engine, compile Rust, read weights, allocate device - memory, emulate GPU computation or establish native eligibility. -4. Separate a static source predicate, pipeline preparation, actual kernel - dispatch, numerical correctness, model quality and service performance. - None automatically implies the next. Preserve unsupported cases, failed - observations, skipped/gated tests and required missing evidence. -5. For an actual change or experiment, obtain its separate authority, freeze - controls/acceptance gates and select discriminating native references before - running it. Do not use source crossover constants or test tolerances as - portable performance/quality acceptance bars. - -## Preparation and use boundary - -This snapshot's workspace is version 0.9.5, edition 2021 and requires Rust 1.94. -The Metal feature requires a compatible Apple native stack plus matched Candle -revision and shader/build inputs. The source repository's broad workspace tests -can use model fixtures, tokens and downloads; they are not safe default checks. - -This graph does not ship an installer. Use only an independently reviewed, -versioned upstream release/build procedure under separate installation -authorization. Do not execute remote shell installers or install Rust/SDKs just -to inspect this graph. If an authorized CLI is already installed, a bounded -identity/help check is: - -```bash -mistralrs --version -mistralrs --help -``` - -Those public CLI invocations were **not run** here. Current inspection has no -Cargo/rustc or compatible Apple backend; no Rust compilation or live Rust API -inspection is claimed. The source/manual adaptation is explicitly partial. -Bundled helpers remain usable without the source checkout or inference package. - -## Return a bounded result - -Report source/build identities, actual versus declared facts, route, invariant, -safe support result, counter-signal and smallest next authorized native check. -Required Metal/kernel/model validation remains `BLOCKED_REQUIRED_BACKEND` until -separately executed; compiler checks are `not_run`. Candidate state is not -promoted by helper tests, packaging, independent content review or a merged PR. -Memory fit, quality, service performance and broad skill efficacy are unverified. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/metadata.json deleted file mode 100644 index 6eab1ae7f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/metadata.json +++ /dev/null @@ -1,101 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NO_LICENSE", - "tags": [ - "mistral-rs", - "metal", - "candle", - "paged-attention", - "cache-layout", - "prefill-metadata", - "workspace", - "prewarm", - "native-evidence" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/EricLBuehler/mistral.rs", - "commit": "9459891e7697238225885518119682ac8209a612", - "version": "0.9.5", - "license": "MIT root; identified paged attention portions Apache-2.0 with NOTICE", - "readset": "references/source-identity.json", - "paths": [ - ".github/workflows/ci.yml", - "AGENTS.md", - "Cargo.lock", - "Cargo.toml", - "LICENSE", - "README.md", - "docs/src/content/docs/guides/perf/paged-attention.mdx", - "examples/python/paged_attention.py", - "mistralrs-core/src/paged_attention/cache_engine.rs", - "mistralrs-core/src/paged_attention/metal_hnd_tests.rs", - "mistralrs-core/src/paged_attention/mod.rs", - "mistralrs-core/src/pipeline/metal_warmup.rs", - "mistralrs-paged-attn/Cargo.toml", - "mistralrs-paged-attn/LICENSE-APACHE", - "mistralrs-paged-attn/NOTICE", - "mistralrs-paged-attn/build.rs", - "mistralrs-paged-attn/src/metal/backend/mod.rs", - "mistralrs-paged-attn/src/metal/backend/paged_attention.rs", - "mistralrs-paged-attn/src/metal/backend/paged_prefill.rs", - "mistralrs-paged-attn/src/metal/kernels/mod.rs", - "mistralrs-paged-attn/src/metal/kernels/paged_decode_hnd.metal", - "mistralrs-paged-attn/src/metal/kernels/paged_prefill.metal", - "mistralrs-paged-attn/src/metal/mod.rs", - "mistralrs-paged-attn/tests/metal_prefill.rs" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Portable compatible-agent native Rust/source adaptation; no DisCo CLI/model execution or managed import." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Stable-ID bounded decode/prefill/qualification author lanes, whole-graph reconciliation, hard cases and offline helper/static/scaffold/wheel checks; compiler and actual Apple evidence not_run." - }, - "applicability": { - "scope": "Selected native Metal paged decode/prefill layout/metadata/workspace and preparation/evidence investigation. Common catalog categorization is not CPU inference or hardware certification.", - "prerequisites": "Access to the pinned Rust/Candle/MSL decode, prefill and qualification contracts; native comparisons need the matched Rust/Apple/Candle/shader stack and separately authorized frozen references.", - "input_contract": "mistral.rs/Candle/native artifact revisions, Rust toolchain, Apple SDK/OS/device, actual model/cache dtype and representation, shape/strides/offsets, block table, context and execution limits.", - "produced_artifact": "A source/build-bound route and invariant report with actual versus declared facts, safe support result, counter-signal and smallest next native check; unavailable Metal/kernel/model evidence remains blocked.", - "failure_modes": "Equating element counts with physical alias/layout identity, transferring decode geometry to prefill, or treating prewarm/pipeline preparation as actual dispatch can misclassify eligibility and native correctness." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: actual Rust/Candle/Apple Metal compilation, native tensor/cache/dispatch/parity/model/quality/service/performance evidence not_run.", - "Cargo/rustc and compatible Apple SDK/device unavailable; no Rust package/toolchain or native dependency installation, live Rust API inspection or source test execution claimed.", - "Offline helpers screen caller declarations, source byte formulas or boundary enumeration only; no tensor authentication, GPU simulation, native eligibility, scratch-capacity or physical-fit proof.", - "Generic model/loading/quantization/server/agentic/distributed/CUDA paths and unmined specialized kernels remain explicit long-tail gaps; broad skill efficacy unmeasured.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback, not legal absence; root MIT and identified component Apache-2.0/NOTICE retained independently." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Select decode, prefill or qualification by the failed contract, then follow cache/metadata bytes from producer through view to native consumer.", - "Keep static predicates, pipeline preparation, kernel dispatch, numerical correctness and model/service outcomes as separate evidence steps; retain skipped or gated tests.", - "Use the matched Candle revision and native shader/build inputs; broad workspace tests can require model fixtures, tokens and downloads." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/SKILL.md deleted file mode 100644 index 835ef76a9..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/SKILL.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -name: mlc-llm -description: "Investigate MLC LLM compiled Metal artifact/schema, Apple target and pinned TVM module/storage/stream contracts, and compile/runtime KV limits without confusing canonical metadata, codegen or automatic estimates with runnable native model qualification." -license: NO_LICENSE -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# MLC LLM compiled Metal and runtime contracts - -Use this source-derived **candidate** for the boundary between model-package -metadata, compiled target artifacts and real native runtime behavior. Begin -with [source identity](references/repo-provenance.md). MLC and its exact TVM -fork/gitlink are separate source owners; an arbitrary newer TVM is not an -interchangeable implementation. - -## Route by the failed contract - -| Question | Canonical owner | -| --- | --- | -| Do normalized tasks, parameter schema, executor/adapter roles and parameter-only resources agree? | [artifact-contract](sub-skills/artifact-contract/SKILL.md) | -| Do target/host/SDK/suffix/prefix, payload format, runnable module and selected storage/stream agree? | [metal-build-runtime](sub-skills/metal-build-runtime/SKILL.md) | -| Which compiled/runtime limit or workspace input is actually controlling the KV path? | [runtime-kv-limits](sub-skills/runtime-kv-limits/SKILL.md) | -| What remains outside these source contracts, including rights and native evidence? | [scope and notices](references/scope-and-notices.md) | - -## Establish the actual boundary - -1. Record MLC revision, TVM gitlink/build, model/tokenizer/quantized parameter - identity, target/host/SDK, compiled artifact/format and runtime device. A - canonical interface or parameter-schema hash does not hash tensor bytes, - authenticate exports, select a device or prove native consumer support. -2. Identify which consumer implements the interface. The inspected artifact - manifest is **opt-in**, first consumed by WebLLM. The native engine does not - consume that contract at this pin; a valid sidecar cannot imply native - support for its model/task ABI. -3. Separate metadata checks, IR/codegen, serialization/export, real runtime - factory/load, pipeline creation, dispatch, stream completion, numerical - behavior and model/service results. Each requires distinct evidence. A - serializable kind=metal fallback is explicitly non-runnable. -4. Follow actual compiled metadata and configuration sources. Runtime preference, - compile-time bound, nested workspace shape and explicit engine override are - different values. The Metal working-set attribute and automatic-capacity - constants do not establish free memory, safe allocation or universal limits. -5. Use only the selected bundled standard-library helper on explicit synthetic - or anonymized declarations. Helpers do not import MLC/TVM, run Pydantic, - initialize submodules, inspect weights/binaries, invoke a compiler or probe a - device. Their narrow domains are not complete upstream input validators. - -## Native preparation and evidence limits - -The source pyproject declares 0.20.0.dev0; no installed package/build is implied. -MLC's native Python package loads TVM/serving integration, and original artifact -tests also import TVM DataType/model definitions and Pydantic. Even apparently -pure schema/codegen/IR tests require the actual prepared native stack. A -standalone Python declaration helper is not that stack or a passing source test. - -This graph ships no installer, model-loading recipe or checkout-dependent source -test command. Compiler/native tests require separately prepared, version-matched -MLC/TVM/FFI dependencies and Apple SDK/device where applicable; broad compile or -integration loops can exercise many models and are not small default checks. -No compiler/live API/CLI/original source test or actual Metal/model execution is -claimed. Required native evidence remains `BLOCKED_REQUIRED_BACKEND / not_run`. - -Return identities, actual versus declared facts, route/invariant, negative -signal and smallest missing native discriminator. Candidate empirical validation -is `not_run`. Helper, source hash, packaging or generic compile success does not -qualify cache correctness, physical fit, model quality, latency/service gains or -broad skill efficacy. No portable acceptance bar is derived from source test -constants or implementation crossover/capacity values. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/metadata.json deleted file mode 100644 index bc539b634..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/metadata.json +++ /dev/null @@ -1,106 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NO_LICENSE", - "tags": [ - "mlc-llm", - "tvm", - "metal", - "artifact-contract", - "target", - "sdk", - "module", - "stream", - "kv-cache", - "compiled-limits" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/mlc-ai/mlc-llm", - "commit": "8978ea9aec626d64096aaa92fbc89bda2e3ea802", - "license": "Apache-2.0 with retained NOTICE and source-specific component notices", - "readset": "references/source-identity.json", - "paths": [ - ".github/workflows/lint.yml", - ".gitmodules", - "CONTRIBUTORS.md", - "LICENSE", - "NOTICE", - "README.md", - "cpp/serve/config.cc", - "cpp/serve/config.h", - "docs/community/guideline.rst", - "docs/compilation/model_artifact_manifest.rst", - "pyproject.toml", - "python/mlc_llm/__init__.py", - "python/mlc_llm/base.py", - "python/mlc_llm/compiler_pass/dispatch_kv_cache_creation.py", - "python/mlc_llm/interface/compile.py", - "python/mlc_llm/interface/convert_weight.py", - "python/mlc_llm/interface/gen_config.py", - "python/mlc_llm/model/gemma4/gemma4_model.py", - "python/mlc_llm/model/model.py", - "python/mlc_llm/protocol/__init__.py", - "python/mlc_llm/protocol/artifact_manifest.py", - "python/mlc_llm/serve/__init__.py", - "python/mlc_llm/serve/engine.py", - "python/mlc_llm/support/auto_target.py", - "tests/python/conftest.py", - "tests/python/integration/test_model_compile.py", - "tests/python/model/test_kv_cache.py", - "tests/python/protocol/test_artifact_manifest.py", - "tests/python/support/test_auto_target.py" - ] - }, - { - "type": "git", - "url": "https://github.com/mlc-ai/relax", - "commit": "201fa0001c81431ccce05419b442c47779ea917c", - "license": "Apache-2.0 with retained NOTICE and source-specific component notices", - "readset": "references/source-identity.json", - "paths": [ - "AGENTS.md", - "LICENSE", - "NOTICE", - "src/backend/metal/codegen/codegen_metal.cc", - "src/backend/metal/codegen/metal_fallback_module.cc", - "src/backend/metal/codegen/metal_fallback_module.h", - "src/backend/metal/runtime/metal_common.h", - "src/backend/metal/runtime/metal_device_api.mm", - "src/backend/metal/runtime/metal_module.mm", - "tests/python/codegen/test_target_codegen_metal.py" - ] - } - ], - "applicability": { - "scope": "Selected artifact/Apple target/pinned TVM Metal consumer and KV configuration source contracts. Common catalog categorization is not CPU inference or Apple qualification.", - "prerequisites": "Access to the pinned MLC and exact TVM fork/gitlink contracts; native checks require separately prepared matching MLC/TVM/FFI and the applicable Apple SDK/device stack.", - "input_contract": "MLC revision, TVM gitlink/build, model/tokenizer/quantized-parameter identity, target/host/SDK, compiled artifact/format, runtime device and the actual compiled/runtime configuration sources.", - "produced_artifact": "An identity and actual-versus-declared boundary report for artifact, Metal runtime or KV-limit ownership, with a negative signal and smallest missing native discriminator.", - "failure_modes": "Treating a schema hash as payload authentication, an opt-in WebLLM artifact contract as a native-engine ABI, serializable Metal fallback as runnable, or automatic capacity estimates as free memory can create unsupported qualification claims." - }, - "limitations": [ - "Source-derived candidate; original compiled CPU/native tests, compiler/live API/CLI/model/Metal/fit/quality/service/performance execution not_run; required native BLOCKED_REQUIRED_BACKEND.", - "Opt-in artifact manifest first targets WebLLM; native engine ignoring that ABI is not qualified by canonical metadata agreement.", - "Serializable kind=metal fallback can be non-runnable; codegen/export/pipeline/submission and actual load/dispatch/completion differ.", - "Helpers screen narrow explicit canonical metadata, literal target strings or limit clamping only; no general Pydantic/TVM emulation, actual tensor/weight/export authentication or GPU/memory simulation.", - "Full model/tokenizer/quantization/VLM/RNN/speculation/disaggregation/CUDA/scheduler/dependency implementation and broad skill efficacy remain outside selected contracts.", - "NO_LICENSE is unavailable machine SPDX metadata, not legal absence; both source owners Apache LICENSE/NOTICE bytes are independently retained." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Identify the interface's actual consumer before using a valid artifact manifest as evidence of native model/task support.", - "Separate codegen/export from runtime factory/load, pipeline creation, dispatch and stream completion; a serializable kind=metal fallback can still be non-runnable.", - "Trace runtime preference, compile-time bounds, nested workspace inputs and explicit overrides separately before interpreting a KV or working-set estimate." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/metadata.json deleted file mode 100644 index e334dedd1..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/metadata.json +++ /dev/null @@ -1,58 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "MIT", - "tags": [ - "mlx-lm", - "generation", - "batching", - "kv-cache", - "cancellation" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ml-explore/mlx-lm", - "commit": "ab7f7048dc341ced2d84b1887eeb17d60b0d1b02", - "paths": [ - "mlx_lm/generate.py", - "mlx_lm/models/cache.py", - "mlx_lm/server.py", - "tests/test_generate.py", - "tests/test_prompt_cache.py", - "LICENSE", - "pyproject.toml", - "mlx_lm/cache_prompt.py", - ".github/workflows/pull_request.yml" - ], - "readset": "references/source-map.md" - } - ], - "applicability": { - "scope": "MLX-LM generation, batch-row and prefix-cache diagnosis; model/backend/runtime compatibility requires separate qualification.", - "prerequisites": "Access to the independently pinned MLX-LM generation/cache contracts and the selected entry path; model, backend and runtime compatibility is separately established for any native claim.", - "input_contract": "MLX-LM/MLX revisions and builds, backend/device, model/tokenizer/adapter/draft identities, entry point, tokenized prompts, cache classes, batching/prefill limits, per-request sampling/processors/stops, seed and KV quantization; failure traces include UIDs, stages, lengths, terminal reason and next-request result.", - "produced_artifact": "An entry-path decision, UID/stage/cache-row table, prefix identity and cache-class assumptions, terminal/cleanup trace and actual evidence status with a targeted verification plan.", - "failure_modes": "Ignoring server batching gates, filtering only some per-request row state, reusing a text-only prefix key or treating a stop flag as immediate MLX work cancellation can corrupt lifecycle reasoning.", - "refusal_point": "Stop at the first unsupported contract or missing required fixture/backend; retain a targeted plan without inferring paged allocation, radix sharing or instantaneous prefill interruption." - }, - "limitations": [ - "Source-derived candidate; no local native-backend, model, service or efficacy validation.", - "Static content checks do not promote this candidate or establish installed API conformance." - ], - "provenance_readsets": [ - "references/source-map.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve the library versus server entry and its batching gates before transferring BatchGenerator behavior.", - "Keep UID, caches, tokens, samplers, processors, lengths, stop matchers and pending outputs aligned under the same removal selection, including prompt-stage requests.", - "Check exact token/model/adapter/draft identity and cache-type copy/trim semantics, then trace cooperative removal through survivor and next-request behavior." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/SKILL.md deleted file mode 100644 index ffbb39646..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/SKILL.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -name: mlx-swift-lm -description: "Investigate MLX Swift LM generation ownership, cancellation and settled task completion, and recurrent GatedDelta Metal versus ops contracts without confusing source predicates, final info or offline metadata with executed native correctness." -license: NO_LICENSE -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# MLX Swift LM ownership and recurrent Metal contracts - -Use this source-derived **candidate** for the Swift package's generation and -selected native recurrent-attention boundary. It is not a full model catalog, -a generic MLX tutorial or a verified engine. Begin with -[source and method identity](references/repo-provenance.md); source, resolved -MLX dependency, native artifacts and model assets are separate identities. - -## Route by the broken contract - -| Question | Canonical owner | -| --- | --- | -| Who owns mutable context/arrays/cache, what does cancellation stop, and has this producer actually settled? | [generation-lifecycle](sub-skills/generation-lifecycle/SKILL.md) | -| Does recurrent state/shape/dtype agree with the public GatedDelta gate and actual Metal/ops consumer? | [gated-delta-contract](sub-skills/gated-delta-contract/SKILL.md) | -| Which dependencies, model/runtime surfaces and rights remain outside this extraction? | [scope and notices](references/scope-and-notices.md) | - -The package owns meaningful Swift concurrency and an embedded MSL recurrent -kernel, not merely an example invoking a foreign engine. Its MLX dependency -still owns array evaluation, custom-kernel binding and device/wired-memory -services. Do not invent those internals from a wrapper, or transfer Python -MLX-LM and other engines' cache/stream semantics without examining the caller. - -## Establish identity and evidence before changing behavior - -1. Record the exact source revision, resolved Swift/MLX dependencies, Xcode/SDK/ - OS/device, API family and native artifact identity. A manifest version range - or a source release is not a resolved dependency or compiled artifact. -2. Separate lexical Swift isolation, exclusive access across await, ownership - transfer, queued GPU work and completed task join. They are not synonyms. - Select the lifecycle route before reusing a caller-owned cache or shutting - down a producer after a consumer stops. -3. For recurrent kernels, follow producer shape/dtype/layout to the actual - native consumer and fallback. A public predicate or `useKernel: true` is - not observed dispatch; an ops fallback is not necessarily CPU execution. -4. Use the selected bundled helper with explicit synthetic/caller declarations. - It does not import Swift/MLX, execute tasks or recurrence, authenticate - observations, compile native code, load weights or simulate a device. -5. Retain negative signals, untested feature combinations and source-only - discrepancies. Obtain separate authority and freeze controls/gates before - any actual native, model, quality or service experiment. Source test-local - tolerances and implementation constants are not portable acceptance bars. - -## Preparation boundary - -The inspected main branch documents the 3.x API line; its Package.swift requires -Swift tools 6.2 and declares platform minima macOS 14/iOS 17/tvOS 17/visionOS 1. -FoundationModels integration has distinct SDK/runtime availability constraints; -those minima do not qualify that adapter. MLX Swift is declared with -`.upToNextMinor(from: "0.33.0")`, and swift-syntax has a version range. There is -no tracked Package.resolved at this pin. Record the actual resolution before -native comparison rather than treating an observed tag as a lockfile. - -This portable graph ships no installer or model-loading example. Use a -separately reviewed, authorized and versioned native preparation procedure; -do not run remote installers or fetch models/dependencies to read the skill. -The source explicitly says `swift test` does not work for this package; its -native route uses Xcode macOS testing. Source test fixtures and build steps are -not runtime dependencies of this packaged graph. Native execution plans stay -in external verification reports, not checkout-dependent helper commands. - -Current inspection has no Swift/Xcode/Apple Metal stack. No live Swift API, -compiler, original native tests or CLI execution is claimed. The actual AREX -create/prepare/manual-workflow/verify adaptation is **partial source plus -offline support**, not a DisCo CLI/model run or managed skill installation. - -## Return a bounded result - -Report identities, actual versus declared facts, route/invariant, safe support -result, counter-signal and smallest next separately authorized native check. -Keep source inference, preparation, dispatch, numerical behavior, model quality -and client-observed service measurements distinct. Required native evidence -remains `BLOCKED_REQUIRED_BACKEND / not_run`; candidate empirical validation -is `not_run`. Packaging, independent content review and repository CI do not -promote the engine, establish physical fit, or prove comparative skill efficacy. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/metadata.json deleted file mode 100644 index 9c63f32f5..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/metadata.json +++ /dev/null @@ -1,113 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "NO_LICENSE", - "tags": [ - "mlx-swift-lm", - "swift", - "metal", - "generation", - "ownership", - "cancellation", - "task-join", - "recurrent-state", - "gated-delta", - "native-evidence" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/ml-explore/mlx-swift-lm", - "commit": "3d066d3a0ffdc7e70c476a3d31488ea62dfdd0ac", - "version": "main 3.x API context; Swift tools 6.2", - "license": "MIT; unselected vendored xgrammar separately Apache-2.0", - "readset": "references/source-identity.json", - "paths": [ - ".github/workflows/integration_tests.yml", - ".github/workflows/pull_request.yml", - "AGENTS.md", - "CONTRIBUTING.md", - "LICENSE", - "Libraries/MLXLLM/Models/Qwen35.swift", - "Libraries/MLXLLM/Models/Qwen3Next.swift", - "Libraries/MLXLMCommon/Documentation.docc/upgrade.md", - "Libraries/MLXLMCommon/Documentation.docc/using.md", - "Libraries/MLXLMCommon/Evaluate.swift", - "Libraries/MLXLMCommon/GatedDelta.swift", - "Libraries/MLXLMCommon/KVCache.swift", - "Libraries/MLXLMCommon/ModelContainer.swift", - "Libraries/MLXLMCommon/ModelFactory.swift", - "Libraries/MLXLMCommon/TokenLoopHandler.swift", - "Libraries/MLXLMCommon/Tokenizer.swift", - "Libraries/MLXLMCommon/UserInput.swift", - "Libraries/MLXLMCommon/Utilities/GenerationWorker.swift", - "Libraries/MLXLMCommon/Utilities/SerialAccessContainer.swift", - "Libraries/MLXVLM/Models/Qwen35.swift", - "Package.swift", - "README.md", - "Tests/MLXLMTests/CancellationTests.swift", - "Tests/MLXLMTests/GatedDeltaQKNormTests.swift", - "Tests/MLXLMTests/GatedDeltaTests.swift", - "Tests/MLXLMTests/GenerationExecutionTests.swift", - "Tests/MLXLMTests/Qwen35GDNDecodeBitwiseTests.swift", - "Tests/MLXLMTests/Qwen35MTPTests.swift", - "skills/mlx-swift-lm/SKILL.md", - "skills/mlx-swift-lm/references/concurrency.md", - "skills/mlx-swift-lm/references/generation.md", - "skills/mlx-swift-lm/references/kv-cache.md", - "skills/mlx-swift-lm/references/model-container.md", - "skills/mlx-swift-lm/references/model-porting.md" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Actual portable compatible-agent source/native-Swift adaptation; no DisCo CLI/model run or managed installation." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Stable-ID lifecycle and recurrent-kernel author lanes, integrated root/local references, hard cases and offline support/static/scaffold/package checks; native Swift/Xcode/MLX compiler/tests not_run." - }, - "applicability": { - "scope": "Selected Swift ownership/cancellation/settling and actual GatedDelta Metal/ops contracts. Common categorization is not CPU inference or Apple qualification.", - "prerequisites": "Access to the Swift lifecycle and selected GatedDelta contracts, with resolved MLX/native identities for a comparison; native checks require a prepared matching Swift/Xcode/Apple stack and separate authorization.", - "input_contract": "Exact source revision, resolved Swift/MLX dependencies, Xcode/SDK/OS/device, API family and native artifact identity; the selected question supplies lifecycle ownership/completion observations or recurrent producer shape/dtype/layout and actual consumer/fallback.", - "produced_artifact": "An identity and actual-versus-declared report for generation ownership or recurrent kernel invariants, with safe support result, counter-signal and smallest separately authorized native check.", - "failure_modes": "Conflating lexical isolation with exclusive access across await or completed task join, treating useKernel as observed dispatch, calling an ops fallback CPU execution, or assuming a manifest range is a resolved dependency can invalidate the conclusion." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: native Swift/MLX compiler, host-control tests and required Apple Metal cancellation/backgrounding/kernel/cache/model/fit/quality/service/performance evidence not_run.", - "Swift/Xcode/Apple SDK/MLX native stack unavailable; no live Swift API, package resolution, original source/native tests or CLI execution claimed.", - "Helpers audit caller-declared evidence or selected metadata/byte/source-predicate contracts only; no tasks/recurrence/GPU simulation, authenticated observation, actual dispatch or numerical validation.", - "Inspected masked kernel and ops output handling differ at source level; no masked-output native parity case executed, confirmed numeric bug or fix claimed.", - "Full ChatSession/cache reconciliation, models/loading/quantization/VLM/tools/grammar/LoRA/training/FoundationModels and external MLX manager/provider internals are bounded gaps; broad skill efficacy unmeasured.", - "NO_LICENSE is actual unauthenticated gh resolver fallback, not legal absence; root MIT retained independently." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve mutable-context/cache ownership and task settlement separately from lexical isolation and queued GPU work before cache reuse or producer shutdown.", - "Follow recurrent shape/dtype/layout to the actual Metal or ops consumer; predicate eligibility and useKernel alone do not prove dispatch.", - "Record the actual dependency resolution and use the documented Xcode macOS test route; the source states swift test does not work for this package." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/SKILL.md deleted file mode 100644 index 62eb1bc31..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/SKILL.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -name: project-maya -description: "Investigate Project Maya model-family dispatch, representation and memory tiers, API versus native state, speculative decoding and source-qualified verification without treating safe CPU checks as CUDA/HIP or model evidence." -license: NO_LICENSE -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# Project Maya inference engineering - -Use this source-derived **candidate** for model-specific inference questions in -`mw00/project-maya`. Read [source and method provenance](references/repo-provenance.md) -first. It is neither a generic model loader nor a verified inference engine. -The bundled helpers are offline support; reading this skill does not authorize -installation, engine startup, model downloads, GPU/API calls or host changes. - -## Route by the failing contract - -| Question | Canonical procedure | -| --- | --- | -| Which model family/format, architecture guard, physical memory pool or residency tier? | [model-runtime](sub-skills/model-runtime/SKILL.md) | -| API continuation, ownership, native slots, images, parallel state, cancellation or recovery? | [serving-state](sub-skills/serving-state/SKILL.md) | -| Draft selection, rollback, calibration, quantization quality, benchmark or evidence level? | [qualification](sub-skills/qualification/SKILL.md) | -| Build/setup side effects, missing tools or dependency/license ambiguity? | [troubleshooting](references/troubleshooting.md) and [license boundaries](references/license-boundaries.md) | - -The retained snapshot has GLM, DeepSeek and MiniMax operating paths plus a newly -documented Qwen route. Coverage is intentionally asymmetric: selected core/state/ -qualification bodies for the first three, shallow dispatch/format/guard evidence -for Qwen. Unmined deep kernels and unexecuted native/model paths stay explicit; -source presence or a common Python facade is not cross-family qualification. - -## Establish identity before changing a setting - -1. Record the full source revision, actual executable/family, model and asset - identities, representation, compiler/runtime/device, full rendered workload, - sampling policy and active single/parallel route. A release label, common - quant tier name or nominal bit width is insufficient. -2. Distinguish caller-supplied metadata from inspected bytes and executed results. - Root MIT notices do not license every optional model asset; source code, - weights, drafts, fonts and UI components have separate boundaries. -3. Identify the state/resource owner and lifetime. Shared weights are not private - conversation state; API history is not native KV; unified physical memory - must not be counted independently as both RAM and VRAM. -4. Select the relevant route, its local helper and its stop conditions. Preserve - exact tokens/image identity and completion boundaries before attempting reuse - or tuning. Never repair a slot header merely to make it appear compatible. -5. State a falsifiable hypothesis and the smallest safe check. A policy arithmetic - result, simulated decision or request JSON can guide an investigation but - cannot authenticate a model payload, actual fit, GPU cleanup or speedup. -6. For a native change, freeze correctness and comparison requirements before - experiments, retain failures/negative observations, and distinguish component - throughput from complete service/quality outcomes. Do not transfer a source - tuner margin into a portable campaign acceptance bar. - -## Return a bounded conclusion - -Report the route and identities, evidence class, invariant, proposed check, -actual result, counter-signal, missing resource and next authorized action. -When required CUDA/HIP/model evidence is unavailable, keep -`BLOCKED_REQUIRED_BACKEND` and an unverified/INCONCLUSIVE conclusion. Do not turn -exit-zero skips, CPU simulations, source reports or green scaffold CI into native -PASS. Model quality, E2E service behavior and broad skill efficacy need separate -validation. Native/backend flags remain unverified until separately established. - -## Safe support and source boundaries - -Each sub-skill owns a small bundled helper and detailed contract/troubleshooting -references. They are standard-library utilities, not model runners or copies of -unsafe setup/calibration scripts. Use their documented synthetic inputs; no -helper sends a request or installs dependencies. Source files are immutable -attribution links, not required runtime checkout paths. Keep this entire owner -folder when distributing it so the references, scripts and MIT notice remain. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/metadata.json deleted file mode 100644 index 46142880b..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/metadata.json +++ /dev/null @@ -1,159 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "partial_cpu_only", - "license": "NO_LICENSE", - "tags": [ - "project-maya", - "model-specific-inference", - "cuda", - "rocm", - "expert-tiering", - "native-state", - "speculation", - "evidence-boundaries" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/mw00/project-maya", - "commit": "249f9a87d1a9d72bec2b684d704b0aa9296b1c9c", - "version": "VERSION1.1.34; CMake0.1.27 differs", - "license": "MIT (root source); model/optional assets separate", - "readset": "references/source-identity.json", - "paths": [ - "CMakeLists.txt", - "LICENSE", - "LICENSE.deepseek-v41", - "LICENSE.minimax-m31", - "LICENSE.qwen-community-1.0", - "README.md", - "VERSION", - "bench/results/2026-10-06-glm-spec/README.md", - "bench/results/MAYA-S.md", - "docs/MODELS.md", - "docs/RESPONSES.md", - "docs/SETTINGS.md", - "include/strata/core/glm_model.hpp", - "include/strata/core/mtp_rows.hpp", - "include/strata/program/conv_cache.hpp", - "include/strata/spec/controller.hpp", - "include/strata/spec/draft_policy.hpp", - "include/strata/spec/suffix_drafter.hpp", - "maya.py", - "maya.sh", - "serve/response_config.py", - "serve/response_jobs.py", - "serve/response_schema.py", - "serve/response_store.py", - "serve/responses.py", - "serve/server.py", - "serve/test_response_jobs.py", - "serve/test_response_store.py", - "serve/test_responses.py", - "setup.py", - "setup.sh", - "src/core/glm_fast_path.cu", - "src/core/glm_memory.hpp", - "src/core/glm_model.cu", - "src/core/glm_mtp.cu", - "src/core/glm_mtp_rows.cu", - "src/core/glm_multi.cu", - "src/core/glm_window.cu", - "src/ds4/ds4_main.cpp", - "src/ds4/ds4_model.cu", - "src/ds4/ds4_model.hpp", - "src/mm3/mm3_geo.hpp", - "src/mm3/mm3_lane_plan.hpp", - "src/mm3/mm3_main.cpp", - "src/mm3/mm3_model.cu", - "src/mm3/mm3_model.hpp", - "src/mm3/mm3_pack.hpp", - "src/mm3/mm3_sample.hpp", - "src/program/conv_cache_test.cpp", - "src/program/generate.cpp", - "src/spec/controller.cpp", - "src/spec/controller_test.cpp", - "src/spec/draft_policy.cpp", - "src/spec/draft_policy_test.cpp", - "src/spec/suffix_drafter.cpp", - "src/spec/suffix_drafter_test.cpp", - "tests/core/mtp_rows_test.cpp", - "tests/glm_memory_test.cpp", - "tests/glm_pack_graph_test.cpp", - "tests/glm_rows_attn_test.cu", - "tests/glm_rows_kernels_test.cu", - "tests/mm3_lane_plan_test.cpp", - "third_party/beellama.cpp/LICENSE", - "third_party/exllamav3/LICENSE", - "third_party/ggml/LICENSE", - "tools/calibrate_draft.py", - "tools/calibrate_ds4.py", - "tools/calibrate_ds4_tp.py", - "tools/calibrate_glm.py", - "tools/calibrate_mm3_prefill.py", - "tools/ds4/pack.py", - "tools/ds4/pack_gguf.py", - "tools/exl3_pack.py", - "tools/iq_pack.py", - "tools/maya_names.py", - "tools/maya_quant/kl_eval.py", - "tools/maya_quants.py", - "tools/mm3/calibrate.py", - "tools/mm3/pack.py", - "tools/mm3/pack_gguf.py", - "tools/test_bench.py", - "tools/test_calibrate_glm.py" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Compatible-agent portable native C++ adaptation, not DisCo CLI/model execution." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Stable-ID isolated author lanes, integrated runtime graph, assertion/helper/static and selected direct-compiler CPU verification; required accelerator/model blocks retained." - }, - "applicability": { - "scope": "Model-specific dispatch/residency, API/native state and qualification investigation. Common catalog placement is not CPU inference or cross-device certification.", - "prerequisites": "Access to the selected model-family runtime/state/qualification contract; native changes need frozen correctness/comparison requirements and the matching authorized CUDA/HIP/model resources.", - "input_contract": "Full source revision, actual executable/model family, model and asset identities, representation, compiler/runtime/device, rendered workload, sampling policy, single/parallel route, exact tokens/images and relevant completion observations.", - "produced_artifact": "A route/identity/evidence-class report with invariant, proposed check, actual result, counter-signal, missing resource and next authorized action; missing native requirements remain INCONCLUSIVE.", - "failure_modes": "Transferring coverage through a common facade or quantization name, confusing API history with native KV/private state, double-counting unified RAM/VRAM, or repairing a slot header to appear compatible can invalidate diagnosis.", - "refusal_point": "When required CUDA/HIP/model evidence is unavailable, retain BLOCKED_REQUIRED_BACKEND and an unverified/INCONCLUSIVE conclusion; exit-zero skips or CPU simulations cannot clear it." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: no actual CUDA/HIP/model/device, quantized payload, vision, parallel/state/restore or cancellation-cleanup evidence.", - "Safe CPU slices and offline helpers cannot establish complete engine builds, memory fit, numerical/model quality, service/performance or broad skill efficacy.", - "Qwen coverage is shallow routing/format/guard evidence; its deep runtime/state/kernels remain unmined/unverified. Other family coverage is bounded rather than exhaustive.", - "Source reports and source-specific calibration margins are not local measurements or portable acceptance gates.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback; root MIT is retained independently and optional/model asset licenses remain separate." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Select the family-specific route and preserve the explicitly shallow Qwen coverage rather than transferring deeper GLM/DeepSeek/MiniMax evidence.", - "Separate shared weights, private conversation state, API history and native KV; count unified physical memory once.", - "Preserve exact token/image identities and completion boundaries before reuse, then test one falsifiable hypothesis without modifying slot headers to bypass compatibility." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/metadata.json deleted file mode 100644 index 0b7873ee8..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/metadata.json +++ /dev/null @@ -1,74 +0,0 @@ -{ - "tags": [ - "sglang", - "diffusion", - "stage", - "state", - "residency", - "cancellation" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "license": "Apache-2.0", - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/sgl-project/sglang", - "commit": "df13d5dce31abfeff0e701a62e8c39a136758e6c", - "paths": [ - "python/sglang/multimodal_gen/runtime/entrypoints/control_requests.py", - "python/sglang/multimodal_gen/runtime/entrypoints/openai/utils.py", - "python/sglang/multimodal_gen/runtime/entrypoints/openai/video_api.py", - "python/sglang/multimodal_gen/runtime/managers/gpu_worker.py", - "python/sglang/multimodal_gen/runtime/managers/memory_managers/component_manager.py", - "python/sglang/multimodal_gen/runtime/managers/scheduler.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/composed_pipeline_base.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/diffusion_scheduler_utils.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/executors/parallel_executor.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/executors/pipeline_executor.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/schedule_batch.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/stages/dedup.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/stages/denoising.py", - "python/sglang/multimodal_gen/runtime/pipelines_core/stages/timestep_preparation.py", - "python/sglang/multimodal_gen/runtime/scheduler_client.py", - ".github/workflows/diffusion-ci-gt-gen-npu.yml", - ".github/workflows/diffusion-ci-gt-gen.yml", - "LICENSE", - "python/pyproject.toml", - "python/sglang/multimodal_gen/test/unit/test_component_residency.py", - "python/sglang/multimodal_gen/test/unit/test_pipeline_executor.py", - "python/sglang/multimodal_gen/test/unit/test_pipeline_residency_manager_install.py", - "python/sglang/multimodal_gen/test/unit/test_scheduler_client.py" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Bounded monolithic multimodal_gen composed-pipeline state and lifecycle diagnosis; actual backend/model behavior requires native validation.", - "prerequisites": "Pinned source or a concrete trace of the selected composed-pipeline executor and installed residency manager, with the applicable monolithic route identified.", - "input_contract": "Source/build, backend and executor/pipeline class, model revision, modality/shape, steps/scheduler/seed/output count, grouped/serial route, ranks, placement, warmup/save settings and symptom; timeout cases need correlated request/job and client/scheduler/worker events.", - "produced_artifact": "A version-bound owner/event ledger, first unresolved boundary, causal hypothesis, smallest real discriminating check, exact results and missing dependencies.", - "failure_modes": "Assuming all executors share the SyncExecutor contract, treating closed component use as eviction, assuming grouped schedulers are isolated, or reading socket close/job deletion as stopped device compute can misdiagnose ownership.", - "refusal_point": "Stop performance claims when the executing path, cleanup or output quality is unresolved; retain the failure and a known baseline rather than inferring a blanket leak or safety conclusion." - }, - "limitations": [ - "Source-derived candidate; upstream runtime and efficacy validation not run.", - "Supplier verified one selected source blob, not the whole archive tree.", - "No automatic grouped isolation, device cancellation, cross-rank recovery or performance claim." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Trace the actual executor and separate request, stage, component-use, module, scheduler, finalizer and allocator-cache lifetimes.", - "Check scheduler aliases and declared output-copy policies before introducing overlap; compare real serial/grouped outputs with fixed timesteps and seeds.", - "Correlate worker, finalizer and device completion after client timeout or deletion; preserve an unknown execution outcome until that evidence exists." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/splash/SKILL.md deleted file mode 100644 index 64e57e7ea..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/splash/SKILL.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -name: splash -description: "Investigate Splash model-format compatibility, automatic memory planning, Metal and ANE execution, DFlash2 verification, cache and request identity, and correctness-gated serving measurements." -license: NO_LICENSE -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# Splash inference engineering - -Use this source-derived candidate to diagnose an inference or serving question, -select a bounded change and state the check that could falsify it. It is not a -complete upstream manual or a certified Apple backend recipe. Read -[source and method provenance](references/repo-provenance.md) before reuse. - -## Route the work - -| Question | Route | Expected result | -| --- | --- | --- | -| A format is rejected, a model will not fit or a cache cap is surprising | [runtime-planning](sub-skills/runtime-planning/SKILL.md) | Compatible representation and memory-owner ledger | -| Prefix reuse, persistent cache, request isolation or Responses history is confusing | [serving-cache](sub-skills/serving-cache/SKILL.md) | Identity, state-lifetime, admission and recovery ledger | -| Draft acceptance rises but speed/answers change; an ANE or precision path differs | [qualification](sub-skills/qualification/SKILL.md) | Semantic control, actual path and matching proof ladder | - -Use the bundled [request builder](scripts/request_template.py) to prepare a -small Chat Completions body offline. It prints JSON; it never contacts a server, -installs Splash, launches a client or downloads a model. Its deliberately small -option set is not the complete upstream API. - -## Establish the execution tuple - -Record source/build revision, macOS and Apple device, target checkpoint/format/ -quantization and artifact identity, compatible draft, tokenizer/template and -vision selection. Add context, cache/memory controls, KV and recurrent-state -precision, ANE setting, verifier-row policy, concurrency/priority, sampling, -prompt/output/quality controls and cold/warm/persistent-cache state. - -An API model alias is not a new weight selection. Equal bit width does not -prove loader/kernel compatibility. A seeded request does not freeze cache, -batch/chunk or adaptive-verification scheduling. - -## Native preparation boundary - -The source targets Apple Silicon M3 or newer and macOS 26.4 or later. The public -release installation is `brew install incoai/tap/splash`; an authorized operator -can inspect `splash serve --help` before loading anything. Installation and -serving are separate actions: the normal serve path can download target/draft -weights. Do not run it solely to inspect an option. - -Splash is a native Metal/C++ runtime with Python server/installer modules, not -a pip-installable project. This extraction adapted AREX's package inspection -to isolated, explicit source-module identity/signature checks. Linux Python -logic does not validate the actual Metal/ANE/model path. - -## Work a bounded hypothesis - -1. Read the selected route and nearest bundled mechanism/failure reference. -2. Separate source observation, upstream measurement and proposed experiment. -3. Identify representation, owner, validity key, lifetime and reclaim/rollback - boundary before tuning a numeric control. -4. Freeze workload, semantic reference and measurement boundaries before a - comparison. Keep campaign pins and gates in the acceptance issue. -5. Confirm the path actually activated; retain fallbacks, admission waits, - evictions, cancellations and errors instead of treating them as warmup noise. -6. Correctness and model quality precede ranking. A cache/parser/codec check - cannot establish GPU state, a distributional guarantee or service SLOs. - -## Stop and report - -Return execution identity, observation, source-backed invariant, alternate -explanation, smallest discriminating check, actual result and unresolved gap. -Stop at unsupported formats, unavailable backend/model resources, unsafe service -exposure or the authorized budget. Do not change system settings or launch an -external coding client merely because imported guidance mentions one. - -Required Metal/ANE/model/actual disk-state evidence remains -**BLOCKED_REQUIRED_BACKEND**. Source and CPU checks do not qualify performance, -quality, service operation or behavioral skill efficacy. Final admission of a -partially verified candidate requires an informed decision, not auto-import. - -The source [Apache-2.0 license](LICENSE) and -[third-party notices](THIRD_PARTY_NOTICES) are retained. `NO_LICENSE` is the -actual AREX resolver's unauthenticated-CLI warning fallback, not a legal finding; -models and external tools retain their own licenses. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/splash/metadata.json deleted file mode 100644 index 81cb03e26..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/splash/metadata.json +++ /dev/null @@ -1,128 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "partial_cpu_only", - "license": "NO_LICENSE", - "tags": [ - "splash", - "apple-silicon", - "metal", - "ane", - "dflash2", - "memory-plan", - "prefix-cache", - "persistent-cache" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/incoai/splash", - "commit": "39a97385f8560cf8a93972cec9b0a963c9195250", - "paths": [ - "README.md", - "DEVELOPMENT.md", - "docs/performance.md", - "LICENSE", - "THIRD_PARTY_NOTICES", - "Makefile", - "server/serve_options.py", - "server/frontend.py", - "server/lru.py", - "server/protocol.py", - "server/http_security.py", - "server/origins.py", - "server/runtime.py", - "install/gguf.py", - "runtime/engine/MemoryPlan.hpp", - "runtime/engine/MemoryPlan.cpp", - "runtime/engine/MemoryGovernor.cpp", - "runtime/engine/MemoryControl.cpp", - "runtime/engine/MemoryAudit.hpp", - "runtime/engine/MemoryAudit.cpp", - "runtime/engine/KvCache.hpp", - "runtime/engine/KvCache.cpp", - "runtime/engine/Cache.hpp", - "runtime/engine/Cache.cpp", - "runtime/engine/StateCache.hpp", - "runtime/engine/StateCache.cpp", - "runtime/engine/Scheduler.cpp", - "runtime/engine/WriteBehind.hpp", - "runtime/engine/WriteBehind.cpp", - "runtime/engine/RuntimeResources.mm", - "runtime/engine/Protocol.hpp", - "runtime/engine/Protocol.cpp", - "runtime/ops/DraftSelector.hpp", - "runtime/ops/DraftSelector.cpp", - "runtime/ops/Sampling.hpp", - "runtime/ops/Sampling.cpp", - "runtime/metal/kernels/decode/sampling.metal", - "runtime/model/Runtime.hpp", - "runtime/model/Runtime.cpp", - "runtime/model/QwenState.cpp", - "runtime/model/ModelDescriptor.mm", - "runtime/model/DFlashDraft.hpp", - "runtime/model/VerifyBudget.hpp", - "dev/benchmarks/abba.py", - "dev/benchmarks/backend_regression.py", - "dev/benchmarks/logit_compare.py", - "dev/benchmarks/http_regression.py", - "dev/tests/server/test_serve_options.py", - "dev/tests/server/test_lru_cache.py", - "dev/tests/server/test_protocol_python.py", - "dev/tests/tools/test_abba.py", - "dev/tests/engine/engine_memory_plan_test.cpp", - "dev/tests/engine/memory_audit_test.cpp" - ], - "license": "Apache-2.0", - "readset": "references/source-identity.json" - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Actual portable compatible-agent workflow with native/source-module adaptation, not DisCo CLI/model execution." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "User-approved native Metal/C++ and isolated source-Python adaptation; stable-ID lanes, integrated graph, assertion cases and create/verify stages. Required-backend block disables automatic admission." - }, - "applicability": { - "scope": "Pinned Splash representation, planning, state/serving and verification investigation; common discovery does not qualify a different Mac/model/backend.", - "prerequisites": "Access to the pinned representation/planning, serving-cache or qualification route; native checks need matching Apple/backend/model resources, an authorized workload and frozen semantic/measurement controls.", - "input_contract": "Source/build, macOS/device, target and compatible draft artifacts/formats, tokenizer/template/vision selection, context/cache/memory controls, KV/recurrent precision, ANE and verifier-row policy, concurrency/priority/sampling, quality controls and cache state.", - "produced_artifact": "An execution-identity report with observation, source-backed invariant, alternate explanation, smallest discriminating check, actual result and unresolved Metal/ANE/model/disk-state gap.", - "failure_modes": "Treating an API alias as new weights, equal bit width as loader/kernel compatibility, or a fixed seed as fixed cache/batch/adaptive-verification scheduling can invalidate the comparison; CPU cache/codec checks do not prove GPU state.", - "refusal_point": "Stop at unsupported formats, unavailable backend/model resources, unsafe service exposure or the authorized budget." - }, - "limitations": [ - "Required Metal/ANE/model/actual device and disk-state execution remains BLOCKED_REQUIRED_BACKEND; Python policy/codec checks do not clear it.", - "Service performance, quality, native recurrent-state recovery and skill efficacy remain unverified.", - "Only selected bodies were mined; not all formats, kernels, supported devices or release/client workflows.", - "NO_LICENSE is the actual unauthenticated GitHub CLI resolver fallback, not a legal conclusion; Apache-2.0 and third-party notices are retained." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve representation, memory/cache owner, validity key and reclaim/rollback boundary before tuning a numeric control.", - "Freeze semantic and workload controls beyond the seed, and retain actual-path fallbacks, admission waits, evictions, cancellations and errors.", - "Inspect options without starting the normal serve path, which can download target/draft weights; installation and serving require separate task authority." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/metadata.json deleted file mode 100644 index 4fa47651f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/metadata.json +++ /dev/null @@ -1,85 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "MIT", - "tags": [ - "stable-diffusion-cpp", - "diffusion", - "model-residency", - "workspace", - "prefetch", - "state-ownership", - "weight-loading", - "storage-io" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/leejet/stable-diffusion.cpp", - "commit": "b13a48219d5cdf408edcb93da6cd69b612ed03d3", - "paths": [ - "src/stable-diffusion.cpp", - "src/pipeline/image.cpp", - "src/pipeline/diffusion_engine.h", - "src/pipeline/diffusion_engine.cpp", - "src/core/ggml_runner.cpp", - "src/core/segment_weight_pipeline.cpp", - "src/core/compute_workspace.cpp", - "src/model_manager.cpp", - "src/model_manager_prefetch.cpp", - "src/device_residency_manager.h", - ".github/workflows/build.yml", - ".gitmodules", - "cmake/ggml.cmake", - "CMakeLists.txt", - "examples/cli/main.cpp", - "include/stable-diffusion.h", - "LICENSE", - "src/model_manager.h", - "src/model_loader.h", - "src/model_loader_files.cpp", - "src/model_loader.cpp", - "src/model_manager_files.cpp", - "src/model.h", - "src/core/util.h", - "src/core/util.cpp", - "src/model_io/tensor_storage.h", - "src/model_io/safetensors_io.cpp" - ], - "readset": "references/source-notes.md" - } - ], - "applicability": { - "scope": "Source-grounded audit of native image-generation weight/runtime residency, optional segment prefetch and bounded non-ZIP safetensors catalog/read/staging ownership; confirm the actual source, GGML, backend, model and executing path independently.", - "prerequisites": "Access to the pinned source/storage boundaries and evidence of the actual image runner/backend dispatch; native capacity or prefetch checks require matching GGML, model and backend resources.", - "input_contract": "Runtime/GGML revisions and selection, compiled/loaded backends and devices, model/components and numerical image settings, effective memory/prefetch/placement settings and actual dispatch; retain available phase allocations, cache/workspace, transfer events and completed-image counts.", - "produced_artifact": "A phase-by-runner/backend/device ownership table for stored and resident weights, retained runtime, pending allocations, pin/prefetch ownership and release/eviction, with one discriminating check and its actual status.", - "failure_modes": "Treating max_vram as proof of segmentation, unpin as physical freeing, or configured prefetch as completed useful overlap can misidentify the capacity cause; missing observations cannot be read as zero.", - "refusal_point": "Stop when the cause is resolved or matching GGML/model/backend resources are unavailable; invoking this audit does not authorize dependency installation, model downloads, policy changes or device provisioning." - }, - "limitations": [ - "Source-derived candidate; native inference, allocator behavior, image quality and performance were not run.", - "Eighteen bounded readset paths plus one separately identity-checked README match the pinned Git commit/tree blob IDs and sizes (19 blobs total); header coverage is limited to selected owner/state declarations, and this does not authenticate every archive member or submodule contents.", - "The loader/storage extension separately authenticates twelve bounded C++ source blobs and the retained MIT text; its line-scoped readset does not replace the historical archive receipt. Catalog stamps are not content hashes; read/mapping/backend calls do not establish storage throughput or device completion.", - "GGML and other submodules are not expanded; matching private GGML headers are required even for a system library.", - "CPU bypasses the inspected accelerator capacity gate and segmentation eligibility, so CPU checks cannot qualify accelerator memory budgets or prefetch.", - "Video/audio pipelines, detailed model kernels, approximate denoising caches and full multi-device execution remain outside this readset.", - "Loader rules select ordinary non-ZIP safetensors; full GGUF/Torch/ZIP parsers, special quantization/conversion correctness and native failure injection remain unqualified." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/loader-storage-boundaries.md" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Separate parameter storage, compute-resident weights, retained workspace/output caches and pending allocations for each runner and phase.", - "Verify the actual monolithic, segmented or multi-device path before interpreting a memory budget or prefetch setting.", - "For a policy comparison, hold model, inputs, numerical settings and device fixed; check completed images and quality before latency, and separate startup, first and warm requests." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md deleted file mode 100644 index a4bfc3d01..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/SKILL.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -name: vllm-ascend -description: "Analyze vLLM Ascend v0.27.1rc1 performance and commit evolution for DeepSeek-V4, Kimi K3, MiniMax-M3 and GLM5+, fused operators, DSA-CP, FlashComm1, Fused-MC2 and DSpark, with effective dispatch, rollback and test-oracle guidance." ---- - -# Investigate release-specific Ascend performance - -Start with the plugin revision, upstream vLLM version, actual device profile, model and draft checkpoint revisions, failure phase and the effective configuration on each role. Use the release table to classify the deployment before borrowing a tutorial or test command. - -| Model | Release-reported configuration | First comparison | -| --- | --- | --- | -| MiniMax-M3 | Atlas 800 A3 W8A8 or 950DT W8A8C8, including two-node 1P1D | Separate weight format, sparse-cache storage and dense-cache storage. | -| DeepSeek-V4-Flash | 950DT W4A8C8, two-node 1P1D DSpark and KV Cache Pool | Compare checkpoint draft budget, connector topology and graph mode on each role. | - -The tag pairs with upstream vLLM **v0.27.1**. Its release stack lists HDK 26.1.1, CANN 9.1.0, PyTorch 2.10.0, torch_npu 2.10.0.post4 and Triton Ascend 3.2.2. Images use Python 3.12 and Mooncake 0.3.11.post1. Record the installed stack separately. For A2 or another model, report the combination as outside this RC's declared scope and identify the version that owns any example you use. - -## Select the performance question - -| Question | Load first | Required discriminator | -| --- | --- | --- | -| DeepSeek-V4 support through performance evolution | [DSV4 and DSpark evolution](references/deepseek-v4-evolution.md) | Current attention/indexer/compressor path, metadata lifecycle and target/draft ownership | -| DSA-CP, FlashComm1 or Fused-MC2 enabled but ineffective | [Configuration and communication](references/communication-config.md) | Requested values versus normalized SP state, hardware dispatch, rollback and conflicting options | -| Fusion opportunity or fusion performance regression | [Fused operators](references/fused-operators.md) | Graph match or actual kernel dispatch, dtype/layout constraints and current replacement path | -| Kimi K3, MiniMax-M3 or GLM5-family optimization | [Model performance](references/model-performance.md) | Specific checkpoint quantization, attention family, runner and phase | -| Trace an old optimization or audit history coverage | [History scope and review levels](references/history-scope.md) | Reachable candidate versus inspected mechanism versus retained behavior | - -Kimi K3 and GLM5-family material describes source implementations within the pinned history. The RC release table above retains its separate validation scope. GLM coverage focuses on GLM5 and later variants present at this tag. - -For an optimization proposal, return the relevant commit, mechanism, current eligibility, rollback or superseding change, closest numerical oracle, and a same-workload measurement plan. Check both producer and consumer before recommending a historical optimization. Distinguish operator timing, model quality, speculative acceptance and sustained service results. - -## Select the first failing boundary - -- Engine initialization or unknown architecture → [trace plugin discovery into the engine process](references/plugin-registration.md). -- MiniMax-M3 starts with FP8 requested but some layers allocate BF16, or top-k results differ → [inspect the effective layer policy and its consumers](references/minimax-cache.md). -- DeepSeek-V4-Flash starts on each node but DSpark acceptance or PD generation fails → [reconcile draft, connector and capture settings](references/dspark-pd.md). - -Return the first discrepancy, its source locator, the effective values on the affected worker or rank, the smallest distinguishing check, and the evidence still needed. Keep initialization, token correctness, acceptance length and sustained service throughput as separate observations. This is a source-derived candidate. Consult the [pinned readset](references/sources.md) for provenance and native validation limits. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/metadata.json deleted file mode 100644 index 005de95f1..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/metadata.json +++ /dev/null @@ -1,149 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "tags": [ - "vllm-ascend", - "v0.27.1rc1", - "MiniMax-M3", - "FP8", - "KV cache", - "DeepSeek-V4-Flash", - "DSpark", - "PD", - "plugin registration", - "DeepSeek-V4", - "Kimi K3", - "MiniMax M3", - "GLM5", - "GLM5.1", - "GLM5.2", - "GLM5.3", - "DSA-CP", - "FlashComm1", - "Fused-MC2", - "fusion", - "performance", - "commit-history" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/vllm-ascend", - "commit": "3b31886237ed65c435e0965b10e8002ff5769766", - "license": "Apache-2.0", - "paths": [ - "LICENSE", - "docs/source/tutorials/models/DeepSeek-V4-Flash.md", - "docs/source/tutorials/models/MiniMax-M3.md", - "docs/source/user_guide/release_notes.md", - "setup.py", - "tests/0.27.1rc_vllm-ascend_overall_conclusion_release_notes_EN.md", - "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_sfa_cp_a2a.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_chunk_kda_aclnn.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_sparse_attention_overlap.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_k3_situ_fusion_cases.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_copy_and_expand_dflash_dspark.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_kimi_k3_fusions.py", - "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_tp_rmsnorm_rope.py", - "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", - "tests/e2e/pull_request/eight_card/test_glm5_2.py", - "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", - "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", - "tests/e2e/pull_request/one_card/compile/test_norm_quant_fusion.py", - "tests/e2e/pull_request/one_card/test_minimax_m3_sparse_attn.py", - "tests/e2e/pull_request/sixteen_card/test_kimi_k3.py", - "tests/e2e/pull_request/utils.py", - "tests/ut/attention/test_dsa_v1.py", - "tests/ut/attention/test_dsv4_block_geometry.py", - "tests/ut/kv_transfer/test_layerwise_multi_cache_spec.py", - "tests/ut/models/minimax_m3/test_minimax_m3_eplb.py", - "tests/ut/models/minimax_m3/test_msa_m3.py", - "tests/ut/models/test_kimi_k3_adapter.py", - "tests/ut/ops/test_kimi_kda.py", - "tests/ut/patch/platform/test_patch_parallel_config.py", - "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", - "tests/ut/quantization/methods/test_fp8_block.py", - "tests/ut/spec_decode/test_dspark_speculator.py", - "tests/ut/spec_decode/test_llm_base_proposer.py", - "tests/ut/test_ascend_config.py", - "tests/ut/test_ascend_forward_context.py", - "tests/ut/test_platform.py", - "tests/ut/worker/test_device_metadata.py", - "tests/ut/worker/test_model_runner_v1.py", - "vllm_ascend/__init__.py", - "vllm_ascend/ascend_config.py", - "vllm_ascend/ascend_forward_context.py", - "vllm_ascend/attention/context_parallel/dsa_cp.py", - "vllm_ascend/attention/dsa_attn_kv_plan.py", - "vllm_ascend/attention/mla_v1.py", - "vllm_ascend/attention/sfa_v1.py", - "vllm_ascend/attention/sparse_flash_mla.py", - "vllm_ascend/compilation/graph_fusion_pass_manager.py", - "vllm_ascend/compilation/passes/muls_add_pass.py", - "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", - "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", - "vllm_ascend/core/kv_cache_interface.py", - "vllm_ascend/device/device_op.py", - "vllm_ascend/lora/fused_moe.py", - "vllm_ascend/models/__init__.py", - "vllm_ascend/models/common/ops/sequence_parallel.py", - "vllm_ascend/models/deepseek_v4/compressor.py", - "vllm_ascend/models/deepseek_v4/indexer.py", - "vllm_ascend/models/deepseek_v4/model.py", - "vllm_ascend/models/minimax_m3/minimax_m3.py", - "vllm_ascend/models/minimax_m3/msa_m3.py", - "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py", - "vllm_ascend/ops/fused_moe/moe_comm_method.py", - "vllm_ascend/ops/fused_moe/moe_mlp.py", - "vllm_ascend/ops/fused_moe/token_dispatcher.py", - "vllm_ascend/ops/gdn.py", - "vllm_ascend/ops/kimi_kda.py", - "vllm_ascend/ops/rope_dsv4.py", - "vllm_ascend/ops/triton/docs/sfa_cp_a2a_fused.md", - "vllm_ascend/ops/triton/dsa_cp.py", - "vllm_ascend/patch/platform/patch_parallel_config.py", - "vllm_ascend/patch/worker/patch_deepseek_v2.py", - "vllm_ascend/patch/worker/patch_minimax_m2.py", - "vllm_ascend/platform.py", - "vllm_ascend/quantization/configs/modelslim_config.py", - "vllm_ascend/sample/sampler.py", - "vllm_ascend/spec_decode/dspark_proposer.py", - "vllm_ascend/spec_decode/llm_base_proposer.py", - "vllm_ascend/utils.py", - "vllm_ascend/worker/device_metadata.py", - "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py" - ], - "readset": "references/sources.md" - } - ], - "provenance_readsets": [ - "references/sources.md", - "references/history-scope.md", - "references/history-reviewed.json" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Release-pinned performance diagnosis and historical optimization analysis for DeepSeek-V4, Kimi K3, MiniMax-M3, GLM5+, fusion and communication/speculation paths.", - "prerequisites": "Pinned release evidence or a matching installed stack and role-specific logs.", - "input_contract": "Source and checkpoint revisions, quantization metadata, device and operators, V1/V2 runner, P/D role, TP/DP/PP/PCP/DCP, requested versus derived config, dominant phase and numerical/performance traces.", - "produced_artifact": "First failing boundary with effective values, source locator and smallest discriminating check.", - "refusal_point": "Report an evidence gap when package, role, cache dtype or release identity cannot be established." - }, - "recommendations": [ - "Check the registered general hook in the failing process.", - "Resolve the per-layer cache skip set before interpreting FP8 storage.", - "Compare both DSpark role budgets and connector layouts before changing capture tiers.", - "Check optimization reversals and exact current dispatch before reusing an old performance change.", - "Use commit history for provenance and numerical or serving oracles for validation." - ], - "limitations": [ - "Source-derived guidance for the selected release paths only.", - "Native NPU execution, performance and behavioral skill evaluation not run." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/SKILL.md b/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/SKILL.md deleted file mode 100644 index da83b2213..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/SKILL.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -name: vllm-metal -description: "Investigate vllm-metal Apple inference through platform and native-ABI selection, paged KV/offload ownership, speculative/deferred state and actual dispatch/serving evidence without upgrading source or CPU checks to backend qualification." -license: NO_LICENSE -disable-model-invocation: true -metadata: - disco-role: operating ---- - -# vLLM Metal inference investigation - -Use this source-derived **candidate** for the vLLM Apple Silicon plugin, not -for generic vLLM, MLX or Metal programming. Read -[source/method provenance](references/repo-provenance.md) before applying it to -a checkout. This graph is self-contained operating guidance with offline -support; it does not include an inference engine or authorize executing one. - -## Route by the contract that can fail - -| Question | Owner | -| --- | --- | -| Why does import succeed but Metal registration/build/model selection fail? | [platform-config](sub-skills/platform-config/SKILL.md) | -| Why is a setting ignored, the runner wrong, or a native ABI stale? | Same owner; selected-knob syntax helper and preparation references | -| Does host offload add RAM, which prefix is reusable, or why does wake fail? | [cache-lifecycle](sub-skills/cache-lifecycle/SKILL.md) | -| Are GDN drafts rolled back correctly, pending tokens owned safely, or GQA actually executed? | [qualification](sub-skills/qualification/SKILL.md) | -| Can a reported rate/parity result support a service or quality claim? | Same owner; frozen workload, execution-boundary and failure accounting | -| What is covered, excluded or differently licensed? | [scope and notices](references/scope-and-notices.md) | - -Upstream vLLM supplies API/scheduler/block management, mlx-lm model layers, -and this plugin the request-aware native attention/runtime integrations. -Shared upstream interfaces do not automatically qualify every combination. -The MLX path, alternative MPS path, full-attention versus recurrent state, -cache capacity versus physical residency, and kernel versus HTTP timing are -deliberately separate throughout these procedures. - -## Establish identity and authority - -1. Record exact source/package/runtime/native artifact identities, Python, - OS/chip/device, backend and executor, checkpoint/tokenizer/draft identities, - format/dtype, effective feature flags and workload. Readsets identify source - bytes and selected analysis spans, not native execution. -2. Match the intended claim to its owner. Resolve an unsupported pairing or - stale artifact before tuning capacity or quoting throughput. A feature - switch permits eligibility; it does not prove that the worker took that path. -3. Choose the smallest safe check. The bundled helpers parse explicit JSON - and compute bounded arithmetic/summaries, without native imports, downloads, - services, credentials or host mutation. No helper authenticates model data, - proves memory fit or measures inference. -4. Separate empirical facts, source reports, test definitions and proposals. - Retain unsuccessful paths and missing evidence. A successful Python import - or CPU parser test is not a native accelerator substitute. -5. For actual engine changes or experiments, establish separate authorization, - safe resource ownership, frozen correctness/quality/performance gates and - cleanup. Never erase a user KV store or install toolchains as an implicit - step in answering a diagnostic question. - -## Public preparation boundary - -The source snapshot supports Python3.12/3.13 and requires Apple Silicon/macOS15+ -for engine execution. Stable users can use the project's Homebrew tap/formula; -development source setup additionally builds native artifacts with full Xcode/ -SDK26.2+. Installation is not performed by this skill. After an authorized -installation, a minimal read-only package check is: - -```bash -python -I -c 'import importlib.metadata as m; import vllm_metal; print(m.version("vllm-metal"), vllm_metal.__version__)' -``` - -That deliberately lazy import proves only selected package visibility. Read -the platform route for exact ABI/backend/runner distinctions and failure -handling before a separately authorized startup. Do not execute remote shell -installers, upstream development scripts or model downloads from this graph. - -## Return a bounded conclusion - -Report identity, selected route, source-supported invariant, actual safe check, -result, counter-signal and smallest next authorized native check. This delivery -has partial CPU API/parser evidence only. Required Apple MLX/Metal and MPS -kernel/model, real transfers/sleep/wake, output quality, HTTP serving/performance -and broad skill efficacy remain unverified (`BLOCKED_REQUIRED_BACKEND` where -required). Source-derived candidate status is not promoted by packaging, -independent content review, a merged PR or green scaffold CI. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/metadata.json deleted file mode 100644 index 5df0e0f7e..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/metadata.json +++ /dev/null @@ -1,115 +0,0 @@ -{ - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "partial_cpu_only", - "license": "NO_LICENSE", - "tags": [ - "vllm-metal", - "apple-silicon", - "mlx", - "mps", - "paged-kv", - "kv-offload", - "speculation", - "deferred-execution", - "evidence-boundaries" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/vllm-metal", - "commit": "e7df7a6ff215f99c7842552d230fdbc709835934", - "version": "0.32.0", - "license": "Apache-2.0 (retained source); dependency/model assets separate", - "readset": "references/source-identity.json", - "paths": [ - "LICENSE", - "README.md", - "docs/CONTRIBUTING.md", - "docs/benchmarking-macos.md", - "docs/configuration.md", - "docs/gqa-decode.md", - "docs/installation.md", - "docs/kv_offloading.md", - "docs/pytorch-mps.md", - "docs/speculative_decoding.md", - "pyproject.toml", - "scripts/lint.sh", - "scripts/test.sh", - "setup.py", - "tests/conftest.py", - "tests/test_config.py", - "tests/test_envs.py", - "tests/test_kv_offload_budget.py", - "tests/test_kv_offload_tiering.py", - "tests/test_kv_offload_worker.py", - "tests/test_macos_defaults.py", - "tools/avg_gen_length.py", - "tools/native_sampling_probe.py", - "vllm_metal/__init__.py", - "vllm_metal/config.py", - "vllm_metal/envs.py", - "vllm_metal/metal/build.py", - "vllm_metal/metal/paged_attention.cpp", - "vllm_metal/platform.py", - "vllm_metal/v1/cache_policy.py", - "vllm_metal/v1/decode_pipeline.py", - "vllm_metal/v1/kv_offload/config.py", - "vllm_metal/v1/kv_offload/connector.py", - "vllm_metal/v1/kv_offload/fs_tier.py", - "vllm_metal/v1/kv_offload/shared_region.py", - "vllm_metal/v1/kv_offload/spec.py", - "vllm_metal/v1/kv_offload/worker.py", - "vllm_metal/v1/model_lifecycle.py" - ] - }, - { - "type": "git", - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", - "paths": [ - "cli/packages/coding-agent/src/disco/skills/create-repo-skill", - "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", - "cli/packages/coding-agent/src/disco/skills/workflow-authoring", - "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" - ], - "scope": "Compatible-agent portable source/Python partial adaptation, not DisCo CLI/model execution or managed import." - } - ], - "construction": { - "anchor_kind": "source", - "distillation_form": "repository-operating-graph", - "strategy": "arex-portable-meta-skills", - "status": "partial-candidate", - "native_import": "not-run", - "method": "Stable-ID manual author sequencing plus bounded cache author lane; integrated graph followed by assertion/helper/static and selected pure Python source tests; Apple native blocks retained." - }, - "applicability": { - "scope": "Selected plugin preparation, KV/offload ownership and speculative/deferred/dispatch evidence investigation. Common catalog scope does not certify CPU inference or cross-backend qualification.", - "prerequisites": "Access to the plugin-specific platform/ABI, cache-lifecycle and qualification contracts; actual engine changes require separate authority, resource ownership, frozen gates and cleanup, with a supported artifact pairing.", - "input_contract": "Source/package/runtime/native artifact identities, Python/OS/chip/device, backend/executor, checkpoint/tokenizer/draft identities, format/dtype, effective feature flags and workload.", - "produced_artifact": "An identity/route/invariant report with actual safe-check result, counter-signal and next authorized native check; partial CPU evidence stays separate from unverified transfers, sleep/wake, model and HTTP outcomes.", - "failure_modes": "Using lazy import success as native registration or ABI evidence, treating a feature switch as dispatch, or conflating cache capacity with physical residency and kernel time with HTTP time can overstate plugin behavior." - }, - "limitations": [ - "BLOCKED_REQUIRED_BACKEND: actual Apple MLX/Metal and MPS kernels/model loading, cache transfers/sleep/wake and real worker dispatch are not verified.", - "Installed pure API inspection and selected parser/default tests do not establish a healthy inference dependency set: pip check failed missing requirements, a FastAPI mismatch and unsupported host platform.", - "No model payload authentication, numerical/model quality, physical memory fit, HTTP service/performance or broad skill efficacy validation.", - "Detailed GGUF/AWQ/quantization/MoE, DFlash/DSpark, multimodal, diffusion, STT/pooling/LoRA and distributed paths remain bounded route evidence or explicit long-tail gaps.", - "NO_LICENSE is the actual unauthenticated gh resolver fallback, not legal absence; upstream Apache-2.0 LICENSE retained independently." - ], - "provenance_readsets": [ - "references/repo-provenance.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Resolve platform registration, backend/runner and native ABI pairing before capacity tuning; a package visibility check establishes only its reported software facts.", - "Keep MLX and MPS, full-attention and recurrent state, logical cache capacity and physical residency as separate contracts.", - "Preserve failed paths and distinguish parser/API checks from real transfers, sleep/wake, model output and HTTP performance evidence." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/metadata.json deleted file mode 100644 index 2d8c751ed..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/metadata.json +++ /dev/null @@ -1,79 +0,0 @@ -{ - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "status": "candidate", - "empirical_validation": "not_run", - "license": "Apache-2.0", - "tags": [ - "vllm-rlt", - "recurrent", - "kv-cache", - "early-exit", - "last-exited", - "shared", - "validity", - "completion", - "lifetime" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/InferMatrixLab/vllm-rlt", - "commit": "4a350344027960a48384626aa646f1765fda6d4f", - "paths": [ - "README.md", - "LICENSE", - "NOTICE", - "vllm_rlt/core/kv_cache_manager.py", - "vllm_rlt/worker/model_runner.py", - "tests/test_kv_cache.py", - "pyproject.toml", - "vllm_rlt/config.py", - "vllm_rlt/engine/stage_engine.py", - "vllm_rlt/core/scheduler.py", - "vllm_rlt/worker/executor.py", - "vllm_rlt/attention/__init__.py", - "vllm_rlt/attention/backend.py", - "vllm_rlt/models/ouro.py", - "vllm_rlt/layers/__init__.py", - "vllm_rlt/__init__.py", - "tests/conftest.py", - "vllm_rlt/request.py", - "vllm_rlt/worker/metadata.py", - "vllm_rlt/kernels/paged_attention.py", - "vllm_rlt/worker/buffers.py" - ], - "readset": "references/source-notes.md", - "identity": "references/source-identity.json", - "license": "Apache-2.0", - "third_party_notices": "references/NOTICE.md" - } - ], - "applicability": { - "scope": "Pinned vLLM-RLT native Ouro synchronous scheduling with host KVMetadata: LAST_EXITED propagation versus SHARED, later depth-specific Torch-reference attention and normal release/wait/free order. No speculation, connector, preemption, prefix sharing, split/packed prefill or graph/static-buffer path qualification.", - "prerequisites": "The selected native Ouro synchronous path uses host KVMetadata, no speculation/connector/preemption, atomic prefill and disabled graph/static-buffer/packed-prefill paths; changed execution conditions need a new bounded check.", - "input_contract": "Repository revision, model/configuration, recurrent-layer set, loop count, cache layout and block size, dtype/device, attention backend and effective flags, plus the traced token/depth/layer and its next consumer.", - "produced_artifact": "A bounded token/finalization/consumer/retirement trace separating source call order, observed data and written validity, completion evidence and remaining native model/backend/performance gaps.", - "failure_modes": "Substituting mark_finalized for K/V propagation, accepting allocated pages as a complete written prefix, applying LAST_EXITED copy expectations to SHARED, or treating event recording as completion can invalidate the ownership diagnosis.", - "refusal_point": "On a failed prerequisite or complete-prefix check, retain the first failing token/depth/layer and trace the missing operation. Stop the conclusion when flags or metadata ownership leave the selected path." - }, - "limitations": [ - "Source-derived candidate; native CPU/CUDA, model correctness/quality, performance and agent efficacy are not validated.", - "Allocated capacity, host written-position validity, copied K/V bytes, event recording and completed device work are distinct; host-row checks do not qualify DeviceKVMetadata.", - "Repository Apache-2.0 and retained model/configuration NOTICE attribution do not constitute a whole-tree or model-weight license audit." - ], - "provenance_readsets": [ - "references/source-notes.md", - "references/source-identity.json" - ], - "consumption_mode": "packaged", - "recommendations": [ - "Distinguish copied K/V values, logical written positions and allocated capacity; follow the later depth-specific attention consumer.", - "Preserve executed shallower values during LAST_EXITED propagation and use SHARED as a separate one-plane control.", - "Follow the latest request event through runner release before scheduler cache release; page reuse is not evidence of erased old bytes." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/metadata.json deleted file mode 100644 index 2240a2356..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/metadata.json +++ /dev/null @@ -1,48 +0,0 @@ -{ - "tags": [ - "vllm", - "kv-cache", - "lifecycle", - "correctness" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "sources": [ - { - "type": "git", - "url": "https://github.com/vllm-project/vllm", - "commit": "7f730b4cea9a66520a9200265928ebe016b48a39", - "paths": [ - "vllm/distributed/kv_transfer/kv_connector/v1/base.py", - "vllm/v1/core/sched/scheduler.py" - ], - "readset": "references/source-notes.md" - } - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation." - ], - "provenance_readsets": [ - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Pinned vLLM V1 scheduler/connector ownership audit for asynchronous KV load/save completion, retained blocks, preemption and failed receives; concrete connector conformance is not established by the interface.", - "prerequisites": "A concrete connector and worker/model-runner call trace at identified revisions, with request/block events sufficient to compare actual callers against the pinned scheduler/interface contracts.", - "input_contract": "Engine/connector revision and class, producer/consumer roles, KV-group count/block namespace, failure policy, concurrent-batch and graph modes; correlate bounded request IDs with allocation, matched tokens, load/save events, receive failures, preemption, finish and pool return.", - "produced_artifact": "An ownership/event ledger and bounded contract-conformance report naming the earliest evidenced broken obligation, owner, affected requests/blocks and smallest missing observation, with a proposed targeted regression check.", - "failure_modes": "Equating reserved tokens with usable KV, finished responses with reusable blocks, or send completion with scheduler write-fence completion can hide unsafe reuse; missing failed-receive completion can leave waits unreleased.", - "refusal_point": "Escalate absent concrete caller evidence, ambiguous block namespaces, unsupported layouts or revision drift; stop before inferring a backend fix, freeing retained blocks or claiming a reproduced bug." - }, - "recommendations": [ - "Trace lookup, allocation, worker execution and completion propagation before changing request state; repeated or undecided lookup and ordinary admission waits can be valid.", - "Check delayed-free ownership and scheduler in-flight-write fences separately, including empty-step save/metadata ordering in the real worker.", - "Keep receive failure and finished_recving correlated, and use request-level failure evidence for multi-group layouts while preserving the existing recovery policy." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/metadata.json b/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/metadata.json deleted file mode 100644 index e4470c935..000000000 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/metadata.json +++ /dev/null @@ -1,50 +0,0 @@ -{ - "tags": [ - "vllm", - "scheduler", - "admission", - "kv-cache" - ], - "status": "candidate", - "empirical_validation": "not_run", - "profile": "local-skill-v1", - "schema_version": 1, - "kind": "skill", - "origin": [ - "external" - ], - "limitations": [ - "Source-derived candidate; no local efficacy validation.", - "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." - ], - "provenance_readsets": [ - "references/papers-20261015/cases/migration-is-not-free-memory.md", - "references/papers-20261015/cases/parallelism-change-with-state-cost.md", - "references/papers-20261015/cases/preemption-under-kv-pressure.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/scheduling-state-boundaries.md", - "references/papers-20261015/notes/fastserve.md", - "references/papers-20261015/notes/llumnix.md", - "references/papers-20261015/notes/loongserve.md", - "references/papers-20261015/notes/orca.md", - "references/papers-20261015/references/fastserve-readset.md", - "references/papers-20261015/references/llumnix-readset.md", - "references/papers-20261015/references/loongserve-readset.md", - "references/papers-20261015/references/orca-readset.md", - "references/source-notes.md" - ], - "consumption_mode": "packaged", - "applicability": { - "scope": "Pinned vLLM V1 branch-based diagnosis of queued requests, low scheduled-token counts and KV-pressure preemption, separating admission/token/input/allocation/connector-wait constraints.", - "prerequisites": "The deployed scheduler must be mapped to the pinned branches, with effective configuration and a bounded request/step trace; another revision requires re-establishing that correspondence.", - "input_contract": "Deployed revision, scheduler class/configuration, connector role, KV layout, policy and speculative/encoder/concurrent-batch features; per-step queues, tokens, both budgets, allocation results, victims, transfer/free state and timing.", - "produced_artifact": "A per-step accounting ledger, observed limiting branch and competing explanation, missing instrumentation and one conditional configuration/progress hypothesis with predicted effects and trade-offs.", - "failure_modes": "Inferring memory exhaustion from a queue or high KV usage, conflating token/input budgets, ignoring pending frees or reading async token accounting as load completion can attribute admission stalls to the wrong branch.", - "refusal_point": "When required state is missing, leave the diagnosis unresolved and request the smallest trace; escalate allocation or connector behavior outside the readset and stop before live mutation or performance claims." - }, - "recommendations": [ - "Trace the first actual blocking or skip branch and record running versus admission caps and token versus input budgets separately.", - "Inspect deferred frees and request-free eligibility before blaming preemption victim selection, then reconcile restored budgets for an already-scheduled victim.", - "Reconstruct waiting admission independently, including queue precedence and async prefix states; change no setting without a branch-specific hypothesis." - ] -} diff --git a/src/inferrsi/knowledge/data/skill_library/skill-index.json b/src/inferrsi/knowledge/data/skill_library/skill-index.json index 5eac8d3db..a6f85207e 100644 --- a/src/inferrsi/knowledge/data/skill_library/skill-index.json +++ b/src/inferrsi/knowledge/data/skill_library/skill-index.json @@ -2,323 +2,355 @@ "schema_version": 1, "skills": { "ascend-aclgraph-capture-replay": { - "path": "hardware/ascend/ascend-aclgraph-capture-replay", + "path": "systems/platforms/ascend/ascend-aclgraph-capture-replay", "backend": "ascend" }, "ascend-device-health": { - "path": "hardware/ascend/ascend-device-health", + "path": "systems/platforms/ascend/ascend-device-health", "backend": "ascend" }, "ascend-distributed-parallel": { - "path": "hardware/ascend/ascend-distributed-parallel", + "path": "systems/platforms/ascend/ascend-distributed-parallel", "backend": "ascend" }, "ascend-graph-mode": { - "path": "hardware/ascend/ascend-graph-mode", + "path": "systems/platforms/ascend/ascend-graph-mode", "backend": "ascend" }, "ascend-model-adaptation": { - "path": "hardware/ascend/ascend-model-adaptation", + "path": "systems/platforms/ascend/ascend-model-adaptation", "backend": "ascend" }, "ascend-operator-profiling": { - "path": "hardware/ascend/ascend-operator-profiling", + "path": "systems/platforms/ascend/ascend-operator-profiling", "backend": "ascend" }, "ascend-operator-tuning": { - "path": "hardware/ascend/ascend-operator-tuning", + "path": "systems/platforms/ascend/ascend-operator-tuning", "backend": "ascend" }, "ascend-optimization-casebook": { - "path": "hardware/ascend/ascend-optimization-casebook", + "path": "casebooks/ascend/ascend-optimization-casebook", "backend": "ascend" }, "ascend-performance-analysis": { - "path": "hardware/ascend/ascend-performance-analysis", + "path": "systems/platforms/ascend/ascend-performance-analysis", "backend": "ascend" }, "ascend-video-diffusion-optimization": { - "path": "hardware/ascend/ascend-video-diffusion-optimization", + "path": "systems/platforms/ascend/ascend-video-diffusion-optimization", "backend": "ascend" }, "ascendc-tiling-design": { - "path": "hardware/ascend/ascendc-tiling-design", + "path": "systems/platforms/ascend/ascendc-tiling-design", "backend": "ascend" }, "assess-serving-preemption": { - "path": "methods/assess-serving-preemption", + "path": "playbooks/planning/assess-serving-preemption", "backend": "common" }, "assess-split-backward-schedule": { - "path": "methods/assess-split-backward-schedule", + "path": "playbooks/planning/assess-split-backward-schedule", "backend": "common" }, "audit-afd-p2p-exchanges": { - "path": "repositories/afd-plugin/audit-afd-p2p-exchanges", + "path": "systems/engines/afd-plugin/audit-afd-p2p-exchanges", "backend": "cuda" }, "audit-fixed-workload-claim": { - "path": "methods/audit-fixed-workload-claim", + "path": "playbooks/measurement/audit-fixed-workload-claim", "backend": "common" }, "audit-flare-session-lifecycle": { - "path": "repositories/flare-local-inference/audit-flare-session-lifecycle", + "path": "systems/engines/flare-local-inference/audit-flare-session-lifecycle", "backend": "cuda" }, "audit-h3-metal-fasth3-semantics": { - "path": "repositories/h3-metal/audit-h3-metal-fasth3-semantics", + "path": "systems/engines/h3-metal/audit-h3-metal-fasth3-semantics", "backend": "common" }, "audit-h3-runtime-residency": { - "path": "repositories/h3-c/audit-h3-runtime-residency", + "path": "systems/engines/h3-c/audit-h3-runtime-residency", "backend": "common" }, "audit-llama-runtime-state": { - "path": "repositories/llama-cpp/audit-llama-runtime-state", + "path": "systems/engines/llama-cpp/audit-llama-runtime-state", "backend": "common" }, "audit-recurrent-kv-finalization": { - "path": "repositories/vllm-rlt/audit-recurrent-kv-finalization", + "path": "systems/engines/vllm-rlt/audit-recurrent-kv-finalization", "backend": "common" }, "audit-sd-cpp-residency": { - "path": "repositories/stable-diffusion-cpp/audit-sd-cpp-residency", + "path": "systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency", "backend": "common" }, "audit-sglang-diffusion-lifetimes": { - "path": "repositories/sglang/audit-sglang-diffusion-lifetimes", + "path": "systems/engines/sglang/audit-sglang-diffusion-lifetimes", "backend": "common" }, "audit-vllm-kv-lifetimes": { - "path": "repositories/vllm/audit-vllm-kv-lifetimes", + "path": "systems/engines/vllm/audit-vllm-kv-lifetimes", "backend": "common" }, "bound-windowed-attention-replay": { - "path": "repositories/deepinfer/bound-windowed-attention-replay", + "path": "systems/engines/deepinfer/bound-windowed-attention-replay", "backend": "common" }, "budget-speculative-verification": { - "path": "repositories/deepinfer/budget-speculative-verification", + "path": "systems/engines/deepinfer/budget-speculative-verification", "backend": "common" }, "calibrate-serving-simulation": { - "path": "methods/calibrate-serving-simulation", + "path": "playbooks/measurement/calibrate-serving-simulation", "backend": "common" }, "cann-attribution-overlap": { - "path": "hardware/ascend/cann-attribution-overlap", + "path": "systems/platforms/ascend/cann-attribution-overlap", "backend": "ascend" }, "cgroup-resource-controls": { - "path": "hardware/host/cgroup-resource-controls", + "path": "systems/platforms/host/cgroup-resource-controls", "backend": "common" }, "checkpoint-recovery-consistency": { - "path": "methods/checkpoint-recovery-consistency", + "path": "playbooks/planning/checkpoint-recovery-consistency", "backend": "common" }, "choose-parallel-training-plan": { - "path": "methods/choose-parallel-training-plan", + "path": "playbooks/planning/choose-parallel-training-plan", "backend": "common" }, "cite-check-downstream-pr": { - "path": "workflows/cite-check-downstream-pr", + "path": "workflows/governance/cite-check-downstream-pr", "backend": "common" }, "claim-contract": { - "path": "workflows/claim-contract", + "path": "workflows/governance/claim-contract", "backend": "common" }, "comfy-ui": { - "path": "repositories/comfy-ui", + "path": "systems/engines/comfy-ui", "backend": "common" }, "competitor-scan": { - "path": "workflows/competitor-scan", + "path": "workflows/research/competitor-scan", "backend": "common" }, "dcp-async-save-performance": { - "path": "frameworks/pytorch/dcp-async-save-performance", + "path": "systems/frameworks/pytorch/dcp-async-save-performance", "backend": "common" }, "dense-performance-evidence": { - "path": "methods/dense-performance-evidence", + "path": "playbooks/measurement/dense-performance-evidence", "backend": "common" }, "diagnose-attention-bottlenecks": { - "path": "hardware/cuda/diagnose-attention-bottlenecks", + "path": "systems/platforms/cuda/diagnose-attention-bottlenecks", "backend": "cuda" }, "diagnose-vllm-admission-pressure": { - "path": "repositories/vllm/diagnose-vllm-admission-pressure", + "path": "systems/engines/vllm/diagnose-vllm-admission-pressure", "backend": "common" }, "diffusers-invocation-state": { - "path": "frameworks/diffusers/diffusers-invocation-state", + "path": "systems/frameworks/diffusers/diffusers-invocation-state", "backend": "common" }, "draft-acceptance-issue": { - "path": "workflows/draft-acceptance-issue", + "path": "workflows/governance/draft-acceptance-issue", "backend": "common" }, "dwarfstar": { - "path": "repositories/dwarfstar", + "path": "systems/engines/dwarfstar", "backend": "common" }, "evaluate-elastic-serving-placement": { - "path": "methods/evaluate-elastic-serving-placement", + "path": "playbooks/planning/evaluate-elastic-serving-placement", "backend": "common" }, "generation-cache-lifecycle": { - "path": "repositories/mlx-lm/generation-cache-lifecycle", + "path": "systems/engines/mlx-lm/generation-cache-lifecycle", "backend": "common" }, "host-memory-management": { - "path": "hardware/host/host-memory-management", + "path": "systems/platforms/host/host-memory-management", "backend": "common" }, "host-overhead-optimization": { - "path": "hardware/host/host-overhead-optimization", + "path": "systems/platforms/host/host-overhead-optimization", "backend": "common" }, "host-overhead-profiling": { - "path": "hardware/cuda/host-overhead-profiling", + "path": "systems/platforms/cuda/host-overhead-profiling", "backend": "cuda" }, "husky": { - "path": "repositories/husky", + "path": "systems/engines/husky", "backend": "common" }, "inference-benchmark-design": { - "path": "methods/inference-benchmark-design", + "path": "playbooks/measurement/inference-benchmark-design", "backend": "common" }, "inference-optimization-casebook": { - "path": "methods/inference-optimization-casebook", + "path": "casebooks/inference/inference-optimization-casebook", + "backend": "common" + }, + "inference-task-router": { + "path": "tasks/inference-task-router", "backend": "common" }, "kernel-writing-guide": { - "path": "methods/kernel-writing-guide", + "path": "playbooks/optimization/kernel-writing-guide", "backend": "common" }, "kv-tier-management": { - "path": "repositories/lmcache/kv-tier-management", + "path": "systems/engines/lmcache/kv-tier-management", "backend": "common" }, "lazy-evaluation-stream-lifetimes": { - "path": "frameworks/mlx/lazy-evaluation-stream-lifetimes", + "path": "systems/frameworks/mlx/lazy-evaluation-stream-lifetimes", "backend": "common" }, "lily": { - "path": "repositories/lily", + "path": "systems/engines/lily", + "backend": "common" + }, + "lithos-metal": { + "path": "systems/engines/lithos-metal", "backend": "common" }, "local-dream": { - "path": "repositories/local-dream", + "path": "systems/engines/local-dream", "backend": "common" }, "mistral-rs": { - "path": "repositories/mistral-rs", + "path": "systems/engines/mistral-rs", "backend": "common" }, "mlc-llm": { - "path": "repositories/mlc-llm", + "path": "systems/engines/mlc-llm", "backend": "common" }, "mlx-swift-lm": { - "path": "repositories/mlx-swift-lm", + "path": "systems/engines/mlx-swift-lm", "backend": "common" }, "npu-skill-factory": { - "path": "workflows/npu-skill-factory", + "path": "workflows/maintenance/npu-skill-factory", "backend": "common" }, "optimization-search-policy": { - "path": "methods/optimization-search-policy", + "path": "playbooks/planning/optimization-search-policy", "backend": "common" }, "os-process-placement": { - "path": "hardware/host/os-process-placement", + "path": "systems/platforms/host/os-process-placement", "backend": "common" }, "performance-ceiling-analysis": { - "path": "methods/performance-ceiling-analysis", + "path": "playbooks/measurement/performance-ceiling-analysis", "backend": "common" }, "plan-activation-residency": { - "path": "methods/plan-activation-residency", + "path": "playbooks/planning/plan-activation-residency", "backend": "common" }, "project-maya": { - "path": "repositories/project-maya", + "path": "systems/engines/project-maya", "backend": "common" }, "pytorch": { - "path": "frameworks/pytorch/pytorch", + "path": "systems/frameworks/pytorch/pytorch", "backend": "common" }, "reason-hopper-kernels": { - "path": "hardware/cuda/reason-hopper-kernels", + "path": "systems/platforms/cuda/reason-hopper-kernels", "backend": "cuda" }, "repository-evolution-analysis": { - "path": "methods/repository-evolution-analysis", + "path": "workflows/research/repository-evolution-analysis", "backend": "common" }, "rsi-evidence": { - "path": "workflows/rsi-evidence", + "path": "workflows/governance/rsi-evidence", "backend": "common" }, "skill-evolution-playbook": { - "path": "methods/skill-evolution-playbook", + "path": "workflows/maintenance/skill-evolution-playbook", "backend": "common" }, "splash": { - "path": "repositories/splash", + "path": "systems/engines/splash", "backend": "common" }, "storage-io-measurement": { - "path": "hardware/host/storage-io-measurement", + "path": "systems/platforms/host/storage-io-measurement", + "backend": "common" + }, + "task-accuracy": { + "path": "tasks/task-accuracy", + "backend": "common" + }, + "task-day-0": { + "path": "tasks/task-day-0", + "backend": "common" + }, + "task-debug": { + "path": "tasks/task-debug", + "backend": "common" + }, + "task-derived-engine": { + "path": "tasks/task-derived-engine", + "backend": "common" + }, + "task-independent-engine": { + "path": "tasks/task-independent-engine", + "backend": "common" + }, + "task-performance": { + "path": "tasks/task-performance", + "backend": "common" + }, + "task-version-alignment": { + "path": "tasks/task-version-alignment", "backend": "common" }, "torch-npu": { - "path": "frameworks/torch-npu/torch-npu", + "path": "systems/frameworks/torch-npu/torch-npu", "backend": "ascend" }, "torchnpu-device-memory-lifecycle": { - "path": "hardware/ascend/torchnpu-device-memory-lifecycle", + "path": "systems/platforms/ascend/torchnpu-device-memory-lifecycle", "backend": "ascend" }, "torchnpu-pinned-host-memory-lifecycle": { - "path": "hardware/ascend/torchnpu-pinned-host-memory-lifecycle", + "path": "systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle", "backend": "ascend" }, "training-input-pipeline": { - "path": "methods/training-input-pipeline", + "path": "playbooks/optimization/training-input-pipeline", "backend": "common" }, "training-resource-budget": { - "path": "methods/training-resource-budget", + "path": "playbooks/planning/training-resource-budget", "backend": "common" }, "video-diffusion-optimization": { - "path": "methods/video-diffusion-optimization", + "path": "playbooks/optimization/video-diffusion-optimization", "backend": "common" }, "vllm-ascend": { - "path": "repositories/vllm-ascend", + "path": "systems/engines/vllm-ascend", "backend": "ascend" }, "vllm-metal": { - "path": "repositories/vllm-metal", + "path": "systems/engines/vllm-metal", "backend": "common" }, "weight-loading-optimization": { - "path": "methods/weight-loading-optimization", - "backend": "common" - }, - "lithos-metal": { - "path": "repositories/lithos-metal", + "path": "playbooks/optimization/weight-loading-optimization", "backend": "common" } } diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md similarity index 61% rename from src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md index 0a061e81f..8c6a9400a 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/SKILL.md @@ -1,6 +1,78 @@ --- name: audit-afd-p2p-exchanges description: Audit CUDA P2pNccl Attention/FFN rank namespaces and per-stage, per-peer hidden-state ownership when uneven worker counts or a mismatched return slice make an AFD exchange ambiguous. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "Apache-2.0", + "tags": [ + "afd-plugin", + "cuda", + "p2p", + "rank-mapping", + "hidden-states", + "ownership" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/vllm-project/afd-plugin", + "commit": "df8d6a0c403c3ca1b4ed9cf1687e7204908bebec", + "paths": [ + "README.md", + "LICENSE", + "afd_plugin/config.py", + "afd_plugin/distributed/topology.py", + "afd_plugin/connectors/factory.py", + "afd_plugin/connectors/metadata.py", + "afd_plugin/connectors/gpu/p2p.py", + "afd_plugin/v1/worker/attention_metadata.py", + "afd_plugin/v1/worker/attention_model_runner_v2.py", + "afd_plugin/v1/worker/ffn_worker.py", + "afd_plugin/v1/worker/ffn_model_runner.py", + "afd_plugin/model_executor/models/deepseek_v2.py", + "afd_plugin/model_executor/models/forward_context.py", + "tests/unit/distributed/test_topology_partition.py" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Source-based CUDA P2pNccl rank/payload audit; selected eager dense-FFN path with compute_gate_on_attention=false. Repository identity afd_plugin; runtime/backend compatibility unverified.", + "prerequisites": "Access to the pinned source ledger and caller-level evidence for the selected eager dense FFN path with Attention-side gating disabled; any proposed cross-stream reuse needs an established communicator completion contract.", + "input_contract": "Plugin and dependency revisions, connector and role, A/F worker counts, DP/PCP/TP coordinates, worker/device ranks, subgroup roster, stage, per-peer token counts, payload shape/dtype/device, transfer context and returned slices.", + "produced_artifact": "A bounded rank-namespace and per-peer payload ownership ledger, the earliest evidenced mismatch or missing edge, and a proposed verification plan.", + "failure_modes": "Inferring data peers from control senders, accepting equal total row counts despite a peer permutation, or treating an even-split fallback or Python return as proof of ownership/completion can produce a wrong exchange diagnosis.", + "refusal_point": "Stop on A < F, invalid role ranks, duplicate/missing assignments or an unresolved rank namespace; return the conflicting callers and ledger entries without inventing ranks or attributing the fault to networking." + }, + "limitations": [ + "Source-derived candidate; native exchange, model quality, backend compatibility, performance and agent efficacy not_run.", + "Selected Python readset is Apache-2.0; the retained MIT notice concerns excluded NPU async CAM implementation and adapted bindings.", + "No qualification of graph/DBO, remote-expert MoE, multi-host transport, complete token-count projection or underlying communicator completion semantics." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "references": [ + "../../vllm/audit-vllm-kv-lifetimes/SKILL.md", + "../../../../playbooks/measurement/inference-benchmark-design/SKILL.md", + "../../../../playbooks/measurement/dense-performance-evidence/SKILL.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Record role, world and subgroup-local ranks separately; reconstruct the actual subgroup roster before interpreting send/receive peers.", + "Track distinct row identities and stage-specific peer lengths through concatenation and splitting; use same-total swapped lengths as a countercheck.", + "Establish the last communicator/stream completion obligation before reusing or releasing a receive buffer." + ] + } --- # Audit AFD P2P exchanges @@ -24,8 +96,8 @@ the two rules. **F1–F4** below are bounded source observations; the procedure This owner concerns Attention/FFN hidden-state exchanges, not KV-block release. Use [vLLM KV lifetimes](../../vllm/audit-vllm-kv-lifetimes/SKILL.md) for scheduler/connector KV retention at its separate pin. Use -[benchmark design](../../../methods/inference-benchmark-design/SKILL.md) and -[evidence classification](../../../methods/dense-performance-evidence/SKILL.md) +[benchmark design](../../../../playbooks/measurement/inference-benchmark-design/SKILL.md) and +[evidence classification](../../../../playbooks/measurement/dense-performance-evidence/SKILL.md) for a separately authorized comparison; a source call graph is not a measurement. The existing repository identity is `afd_plugin`. Its runtime descriptor stays diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/afd-plugin-LICENSE.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/references/afd-plugin-LICENSE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/afd-plugin-LICENSE.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/references/afd-plugin-LICENSE.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/SKILL.md similarity index 53% rename from src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/SKILL.md index 8e3374a3c..ca4c272da 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/SKILL.md @@ -1,6 +1,80 @@ --- name: comfy-ui description: Audit ComfyUI workflow-output cache validity, lazy dependencies and model-weight residency as separate state contracts using a pinned source revision and explicit runtime evidence. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "GPL-3.0", + "tags": [ + "comfyui", + "workflow", + "cache-invalidation", + "model-residency", + "state-ownership" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/Comfy-Org/ComfyUI", + "commit": "b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c", + "paths": [ + "execution.py", + "comfy_execution/caching.py", + "comfy_execution/graph.py", + "comfy/model_management.py", + "comfy/model_patcher.py", + "comfy/sampler_helpers.py", + "server.py", + "main.py", + "comfy_execution/graph_utils.py", + "comfy_execution/cache_provider.py", + "nodes.py", + "comfy/sample.py", + "comfy/samplers.py", + "tests/execution/test_execution.py", + "tests-unit/execution_test/test_execution_list.py", + "tests-unit/comfy_test/model_management_test.py", + "tests/inference/test_inference.py", + ".github/workflows/test-execution.yml", + "LICENSE", + "pyproject.toml", + "tests-unit/execution_test/test_cache_provider.py", + "AGENTS.md" + ], + "readset": "references/repo-provenance.md" + } + ], + "applicability": { + "scope": "Source-grounded ComfyUI workflow-output cache and model-weight ownership audit; confirm the actual revision, node registry, cache caller and static/dynamic patcher before transfer.", + "prerequisites": "Access to the pinned provenance and the actual node registry/cache caller; residency questions additionally require the concrete patcher and effective dynamic/static mode. Native memory conclusions need the actual backend.", + "input_contract": "Exact revision, API-format prompt/output targets, relevant node classes/constants/links/hidden inputs and external state, selected cache caller/mode and patcher settings, plus available execution/load/memory/timing observations.", + "produced_artifact": "A state/owner/key/lifetime/consumer ledger with invalidation or release events, evidence and unresolved boundaries, followed by a discriminating check and its actual result.", + "failure_modes": "Treating identical JSON as proof of unchanged external inputs, confusing eviction with key validity, or equating workflow cache hits with device-resident model weights can explain the wrong state contract.", + "refusal_point": "Stop when the cause/action is supported or required environment, backend or model resources are unavailable; do not manufacture a hit, hardware result or performance threshold." + }, + "limitations": [ + "Source-derived candidate; no local executor-cache, device-residency or performance qualification.", + "Seven self-authored cases exercised only the unchanged pure upstream graph_utils module; they do not verify PromptExecutor, cache signatures or model behavior.", + "The dynamic native allocator and third-party custom-node behavior remain outside the verified scope." + ], + "provenance_readsets": [ + "references/repo-provenance.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Draw the affected dependency cone with declared, lazy and dynamically expanded links, then mark unfingerprinted external state unresolved.", + "Separate output/object cache validity and retention from patch identity, partial unload, full detach and allocator-visible release.", + "Change one test variable while fixing prompt, output target, model, backend, precision, seed and cache mode; check values and executed nodes before timing." + ] + } --- # Audit ComfyUI cache and residency diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/graph-cache-invalidation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/graph-cache-invalidation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/graph-cache-invalidation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/graph-cache-invalidation.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/model-residency.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/model-residency.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/model-residency.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/model-residency.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/validation-boundaries.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/validation-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/comfy-ui/references/validation-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/comfy-ui/references/validation-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/SKILL.md similarity index 75% rename from src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/SKILL.md index c33c55f98..c73cd0a2a 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/SKILL.md @@ -1,6 +1,74 @@ --- name: bound-windowed-attention-replay description: Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved historical state satisfy the full dependency closure. Use when a model has shared hidden state or preserved-compressed KV and the retained region must be derived rather than assumed. Derives the retain rule from window plus rollback depth, separates query-row compaction from dropping historical keys, and names what becomes unrecoverable. +metadata: + inferrsi: | + { + "tags": [ + "deepinfer", + "sliding-window-attention", + "prefix-replay", + "kv-cache", + "metadata", + "correctness" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/deepseek-ai/DeepInfer", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "paths": [ + "csrc/engine/input_metadata/prepare_tensor_input_v4.cpp", + "deepinfer/engine/llm_engine.py", + "deepinfer/model_executor/input_metadata/yoco.py", + "deepinfer/model_executor/model_metadata_utils.py", + "deepinfer/model_executor/models/decoder_layer.py", + "deepinfer/model_executor/models/v4_modules/attn.py", + "deepinfer/model_executor/models/v4_modules/yoco.py", + "deepinfer/worker/cache_engine.py", + "deepinfer/worker/model_runner.py", + "deepinfer/environ.py", + "LICENSE" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Decoders with an explicit producer/consumer boundary and a demonstrated dependency closure for compact query rows, including separately preserved historical state", + "prerequisites": "A derived dependency closure; a full-length KV write at the boundary; known rollback depth; control over metadata construction", + "hardware": "Backend-independent retain arithmetic; window sizes, boundary index and ring lengths come from the model config and do not transfer", + "input_contract": "The actual producer/consumer split and transitive dependency closure, boundary index, window and rollback depth, consumer KV sharing/compression layout and separately preserved historical keys.", + "produced_artifact": "A dependency listing, enforced retain length, compact query/position/store-slot/visibility metadata, a separate historical-key retention statement and a diagnostic equivalence-reference mode.", + "failure_modes": "Applying window-plus-rollback without dependency closure, discarding full-layout state, clipping away the sampling row, or comparing across preemption can invalidate replay; compact queries do not imply historical keys were dropped.", + "refusal_point": "Stop if required consumer state is unavailable without a preserved full-length copy, the boundary representation cannot be captured, or comparison preemption moves the anchor. Do not use for clipped-region outputs or required bit-identical preemption replay." + }, + "limitations": [ + "Source-derived candidate; no equivalence run was performed.", + "Applies to architectures with a demonstrated boundary dependency, not to arbitrary sliding-window stacks.", + "Window sizes, boundary-layer index and ring lengths are model-config specific and are not transferable.", + "Query-row compaction does not imply dropping historical keys; the two decisions are separate.", + "Prompt logprobs are unrecoverable for the clipped region; the reference forces them off with a warning.", + "The truncation path is off by default in the reference source." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/validation.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Derive every consumer dependency before computing keep; preserve the required full-layout representation separately from the compact query rows.", + "Use the visibility-only diagnostic mode to isolate equivalence from row gathering, and retain one sampling row for a fully clipped chunk.", + "Check metadata at chunk/store boundaries, actual prompt-logprob handling and zero preemption during equivalence comparison." + ] + } --- # Bound a windowed-attention replay diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/validation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/references/validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/bound-windowed-attention-replay/references/validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/SKILL.md similarity index 74% rename from src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/SKILL.md index 8b88b6a2d..fda422f49 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/SKILL.md @@ -1,6 +1,73 @@ --- name: budget-speculative-verification description: Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an engine already runs a draft head with multi-token speculation and verification cost grows with query rows. Separates draft cost from verify cost, builds a cumulative-acceptance rate vector, fits a cost model, and allocates a per-step row budget. +metadata: + inferrsi: | + { + "tags": [ + "deepinfer", + "speculative-decoding", + "mtp", + "verification", + "cost-model", + "scheduling" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/deepseek-ai/DeepInfer", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "paths": [ + "csrc/engine/mtp_varlen.hpp", + "csrc/engine/scheduler.hpp", + "deepinfer/worker/worker.py", + "deepinfer/environ.py", + "deepinfer/sampling_params.py", + "deepinfer/model_executor/models/mtp/dflash.py", + "deepinfer/model_executor/layers/sampler.py", + "deepinfer/model_executor/layers/tilelang_ops/fused_sample.py", + "LICENSE" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Engines with multi-token speculative decoding whose verification forward cost grows with query rows", + "prerequisites": "A calibrated per-position acceptance signal; a measured (rows, step_time) curve; a handle on metadata construction", + "hardware": "Backend-independent procedure; the reference implementation targets NVIDIA SM100 and Ascend 950 only", + "input_contract": "Draft width and row representation, the defined individual/conditional versus cumulative acceptance semantics, verify rows/time curve, sampler cost and acceptance round actually available at schedule time.", + "produced_artifact": "A per-step verification row budget, per-request query lengths from one through draft_width plus one, cumulative acceptance vectors and predicted-versus-realized acceptance error logs.", + "failure_modes": "Confusing verification savings with reduced drafting, misreading confidence semantics or available rounds, allowing later-position confidence to select an earlier token, or generating non-prefix grants can invalidate the budget or sampling claim.", + "refusal_point": "Stop if unthrottled drafting or sampler cost dominates, the signal cannot be made cumulative, or no round supplies acceptance estimates; the procedure also does not apply without a per-step cost model." + }, + "limitations": [ + "Source-derived candidate; no local efficacy validation and no reproduced gain.", + "The reference budgets verification rows only; draft compute is unchanged.", + "The budgeted path is off by default in the reference source.", + "Budgeting can lower accepted length; equal accepted length is not an acceptance criterion.", + "Per-position independence of the confidence used for verification is a correctness requirement, not only a scheduling convenience.", + "Cross-backend rejection-comparison parity is unproven; the reference has no equality assertion.", + "Numeric acceptance rates and warmup curves are deployment-specific and are not carried here." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/validation.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Measure draft, verify and sampler costs separately and fit the verify curve for graph sizes that will actually replay.", + "Preserve the one-row floor and prefix grants, and audit CPU/GPU allocation timing against the per-position decision-independence contract.", + "Report verification rows, accepted tokens, total step time and throughput separately; use an independently specified sampling oracle rather than requiring unchanged accepted length." + ] + } --- # Budget speculative verification diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/validation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/references/validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/deepinfer/budget-speculative-verification/references/validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/SKILL.md new file mode 100644 index 000000000..ba6a7bde8 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/SKILL.md @@ -0,0 +1,186 @@ +--- +name: dwarfstar +description: "Investigate DwarfStar model fit, expert SSD streaming, session and vision cache correctness, speculative sampling and distributed execution using pinned source contracts and explicit native verification boundaries." +license: NO_LICENSE +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "partial_cpu_only", + "license": "NO_LICENSE", + "tags": [ + "dwarfstar", + "ds4", + "moe", + "ssd-streaming", + "session-state", + "kv-cache", + "speculation", + "tensor-parallelism" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/antirez/ds4", + "commit": "ee7e8405232c8971000cc7ba2b945b00bf6d20d3", + "paths": [ + "README.md", + "LICENSE", + "Makefile", + "ds4.h", + "ds4.c", + "ds4_ssd.c", + "ds4_ssd.h", + "ds4_kvstore.c", + "ds4_kvstore.h", + "ds4_server.c", + "ds4_tp.c", + "ds4_tp.h", + "docs/SSD_STREAMING.md", + "docs/SERVER.md", + "docs/SPECULATIVE_DECODING.md", + "docs/DISTRIBUTED.md", + "docs/CUDA_MULTI_GPU.md", + "docs/TESTING.md", + "tests/test_session_state.c", + "tests/test_sampling.c", + "tests/test_tp_commands.c", + "tests/test_engine_mgpu_placement.c", + "ds4_cuda.cu", + "QA_BEFORE_RELEASES.md" + ], + "readset": "references/source-identity.json", + "license": "MIT" + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/workflow-authoring", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" + ], + "scope": "Portable instructions executed by compatible agent with native C adaptation; no DisCo CLI/model run or managed import." + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "repository-operating-graph", + "strategy": "arex-portable-meta-skills", + "status": "partial-candidate", + "native_import": "not-run", + "method": "User-approved native C adaptation and partial drafting; manual stable-ID subskill orchestration and integrated create/verify stages. Required-backend blocks disable automatic admission." + }, + "applicability": { + "scope": "Pinned DwarfStar source investigation and experiment planning; not arbitrary GGUF support or cross-backend qualification.", + "prerequisites": "Access to the pinned native-source contract and relevant memory/state/qualification route; native verification needs a compatible build, matching hardware/weights and task-specific execution authority.", + "input_contract": "Engine revision/patches, actual model artifact/representation, selected backend/build and topology, memory/storage budget, context/chunk/session count, image inputs, sampling/speculation policy, cache state and quality/service requirements.", + "produced_artifact": "A bounded executing-tuple report with observed signal, source-backed mechanism, alternate explanation, proposed falsifier, actual check outcome and unresolved backend/model/service gap.", + "failure_modes": "Inferring model support from flags, aliases, file mapping or quantization names can select the wrong graph; treating CPU bookkeeping or parser checks as native backend/service qualification can overstate the result.", + "refusal_point": "Stop at unavailable hardware/weights, unsafe exposure or the authorized budget; retain memory-guard refusals rather than disabling the guard to claim fit." + }, + "limitations": [ + "Metal/CUDA/ROCm inference, actual model output, SSD runtime I/O, GPU recurrent rollback, physical distributed behavior, service quality/performance and skill efficacy remain unverified.", + "CPU build/help and selected bookkeeping tests do not clear BLOCKED_REQUIRED_BACKEND.", + "NO_LICENSE is the actual unauthenticated GitHub CLI resolver fallback, not a legal conclusion; pinned source MIT notice is retained.", + "Only selected source bodies were inspected; full kernels, conversion/imatrix tools and every workflow are not covered." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Choose the memory, serving-state or qualification route by the observed symptom and identify the owner, validity key and release/rollback boundary.", + "Freeze comparison controls, test the smallest invariant first and retain fallback messages, cancellations, refusals and unavailable fixtures.", + "Base model support on the actual loaded artifact and graph; use native backend evidence for model, tensor-parallel or service conclusions." + ] + } +--- + +# DwarfStar inference engineering + +Use this candidate to turn a DwarfStar capacity, state-correctness or execution +question into a bounded hypothesis and discriminating check. It distills +`antirez/ds4`, not an arbitrary GGUF runner. Start with the +[source and method contract](references/repo-provenance.md). + +The operating guidance is source-derived. A local CPU build or bookkeeping test +does not validate Metal/CUDA/ROCm kernels, model output, a real service, physical +tensor parallelism or a performance gain. Required backend evidence remains +**BLOCKED_REQUIRED_BACKEND**; this candidate is not fully backend-verified. + +## Route the question + +| Question | Read next | Expected output | +| --- | --- | --- | +| A model fits on disk but not in RAM/VRAM; cache grows; more sessions fail | [runtime-memory](sub-skills/runtime-memory/SKILL.md) | Representation and phase/owner memory ledger, one causal hypothesis | +| A repeated prompt misses, reuses wrong image/tool history or corrupts a continuation | [serving-state](sub-skills/serving-state/SKILL.md) | Token/image/state identity and invalidation ledger | +| A draft is faster but changes results; a TP parser test passes; a backend falls back | [qualification](sub-skills/qualification/SKILL.md) | Matching correctness test, actual execution path and remaining native gap | + +Use the bundled [request builder](scripts/request_template.py) when preparing a +minimal Chat Completions request. It prints JSON only: no engine, network, +credential, model download or service is started. Its options are deliberately +smaller than the upstream API. + +## Establish the execution tuple + +Record the engine revision and patches, model family/checkpoint/quantization and +artifact identity, selected native backend/build, device and transport topology, +RAM/VRAM/storage budget, context/prefill chunk/session count, image inputs, +sampling/seed and speculation policy. Add cold/warm/live/disk-cache state and +the workload's quality and service requirements. Unknown fields remain unknown. + +Do not infer model support from a flag, shared quantization name, successful file +mapping or API model alias. The actual loaded GGUF and model-specific graph +select semantics. Flash, V4.1, PRO, GLM and Qwen routes differ. + +## Native preparation, not automatic installation + +DwarfStar is a native C engine, not a Python distribution. A prepared CPU +inspection build can expose CLI help without weights; accelerated builds require +their matching compiler/runtime. An operator with a compatible installed build +can inspect `ds4 --help` and `ds4-server --help` before choosing options. +For an authorized source installation, use the public repository at the +reviewed source revision and the native `make cpu` target for CPU inspection; +Metal/CUDA/ROCm require the matching native build described by this graph. +Building, downloading weights, exposing a listener and changing system memory +limits are separate actions requiring task authorization. + +This extraction adapted AREX's Python-only environment fields to native C build +and CLI identity. It did not invent an installed Python package, run a DisCo +model backend or install into a managed DisCo skill collection. + +## Work one bounded hypothesis + +1. Select the relevant route and read its bundled mechanism and failure notes. +2. Separate source observations, upstream reports and proposed experiments. +3. Identify the owner, lifetime, validity key and release/rollback boundary. +4. Freeze controls before a comparison; change one mechanism or explicitly + declare a coupled change. Numeric gates and campaign revisions belong in + the acceptance issue, not in this reusable guidance. +5. Verify the smallest invariant before measuring performance; retain failures, + refusals, fallback messages, cancellations and unavailable fixtures. +6. For a service claim, run the accepted model/workload/quality/concurrency + protocol on the actual backend. Microtests are not service qualification. + +## Stop and report + +Return the executing tuple, observed signal, source-backed mechanism, alternate +explanation, proposed falsifier, actual check outcome and unresolved gap. Stop +at unavailable hardware/weights, unsafe exposure or the authorized budget. +Do not disable a memory guard to turn a refusal into a capacity claim. + +Read [source notices](LICENSE) when redistributing adapted material. +`NO_LICENSE` is the AREX resolver's unauthenticated-CLI warning fallback, not a +claim that DwarfStar lacks a legal license; the pinned source license is MIT. +Model weights and retained third-party code have independent obligations. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/scripts/request_template.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/scripts/request_template.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/scripts/request_template.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/scripts/request_template.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/qualification/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/qualification/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/qualification/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/qualification/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/references/verification-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/qualification/references/verification-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/qualification/references/verification-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/qualification/references/verification-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/runtime-memory/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/runtime-memory/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/references/memory-ledger.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/runtime-memory/references/memory-ledger.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/references/memory-ledger.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/runtime-memory/references/memory-ledger.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/runtime-memory/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/runtime-memory/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/runtime-memory/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/serving-state/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/serving-state/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/references/state-contracts.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/serving-state/references/state-contracts.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/references/state-contracts.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/serving-state/references/state-contracts.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/serving-state/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/dwarfstar/sub-skills/serving-state/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/dwarfstar/sub-skills/serving-state/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md similarity index 51% rename from src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md index a2d462528..e78a4a950 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/SKILL.md @@ -1,6 +1,98 @@ --- name: audit-flare-session-lifecycle description: Audit Flare complete prefix/session restoration, inbound transfer tickets, publication/use/retirement fences and host-restore admission outcomes. Use when a lookup hit does not become reused tokens, a cancelled waiter leaves transfers in flight, or a restarted session must recover KV pages together with recurrent, convolution and partial-tail state. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "Apache-2.0", + "tags": [ + "Flare", + "flare-local-inference", + "local-inference", + "cuda", + "session-state", + "kv-cache", + "recurrent-state", + "convolution", + "restore-admission", + "transfer-ticket", + "fences", + "offload" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/NVIDIA/flare-local-inference", + "resolved_url": "https://github.com/NVIDIA/local-inference", + "commit": "bce032c014bae34502ba22d3304fa084d5b40afe", + "paths": [ + "README.md", + "LICENSE", + "docs/runtime-package-design.md", + "docs/prefix-cache-offload.md", + "helios/src/runtime_package/format.rs", + "helios/src/runtime_package/model.rs", + "helios/src/execution/manifest.rs", + "helios/src/model/contract.rs", + "helios/src/model/catalog.rs", + "helios/src/cache.rs", + "helios/src/offload/mod.rs", + "helios/src/offload/geometry.rs", + "helios/src/offload/index.rs", + "helios/src/offload/record.rs", + "helios/src/offload/transfer.rs", + "helios/src/scheduler.rs", + "helios/src/server/runtime.rs", + "tests/server/test_prefix_cache_offload.py", + "qualify/performance.py", + "qualify/provenance.py" + ], + "readset": "references/source-notes.md", + "identity": "references/source-ledger.json", + "license": "Apache-2.0" + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "task-agnostic", + "strategy": "reuse-existing", + "status": "source-derived-candidate", + "method": "AREX scope/ground/construct/verify and repository evidence/usability methods adapted to InferRSI native owners and packaging; no DisCo runtime or managed import." + }, + "applicability": { + "scope": "Pinned Flare complete-session restore and admission audit for its NVIDIA SM120/SM121 research prototype. CUDA catalog scope is content classification, not device or backend qualification.", + "prerequisites": "Access to package/source identity and bounded restore traces, with preparation compatibility established before cache-state diagnosis. Running, freeing resources or changing configuration requires separate authority.", + "input_contract": "Execution ABI and prepared-contract version, ordinary/speculative configuration, model/layout identity, request IDs and consumed-token endpoints, GPU/host prefixes, continuation row, ticket/waiter identities, transfer events, fences and resource counts.", + "produced_artifact": "A checkpoint/ticket ownership table, five-outcome admission ledger and diagnosis identifying the earliest unsupported transition and smallest missing observation.", + "failure_modes": "A KV-only lookup hit can omit continuation state; admission can be mistaken for actual reuse; client cancellation can be mistaken for transfer completion; mixing absolute timestamps from client and native clocks can corrupt the diagnosis.", + "refusal_point": "Stop with an unresolved edge when package identity, layout, transfer completion or clock provenance is unknown." + }, + "limitations": [ + "Source-only candidate; native Flare/model/GPU/service and skill efficacy were not evaluated.", + "GDS/Direct transport is deferred and explicitly rejected by the pinned implementation.", + "Multi-rank source branches do not establish multi-GPU, multi-host or hardware failure qualification.", + "Root Apache-2.0 attribution does not grant model, asset or separately licensed dependency rights.", + "Only the selected symbols/sections were inspected; kernels, exporter implementation and whole-repository behavior are outside coverage." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/source-ledger.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Pair the immutable KV parent chain with the complete continuation row at the same consumed-token endpoint before calling a checkpoint restorable.", + "Distinguish lookup outcomes, destination reservation, ticket creation, transfer completion, publication and actual reused-token evidence.", + "Follow scheduler-owned tickets through waiter cancellation and check write/use fences before allowing storage reuse." + ] + } --- # Audit Flare session lifetimes @@ -21,7 +113,7 @@ is supplied by this knowledge. | Question | Route | |---|---| -| What may leave Python preparation and be reconstructed by Rust? | Existing [weight-loading owner](../../../methods/weight-loading-optimization/SKILL.md#10-separate-preparation-from-serving), including its Flare preparation reference | +| What may leave Python preparation and be reconstructed by Rust? | Existing [weight-loading owner](../../../../playbooks/optimization/weight-loading-optimization/SKILL.md#10-separate-preparation-from-serving), including its Flare preparation reference | | What constitutes a restorable Flare checkpoint? | [Complete state and lifetimes](references/lifecycle.md#complete-state) | | Why did a host match fail to save prefill? | [Admission outcomes](references/diagnostics.md#admission-outcomes) | | What must remain live after cancellation? | [Transfer and fence ledger](references/lifecycle.md#ownership-and-completion) | diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md similarity index 98% rename from src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md index 4ac1c0b50..6af298967 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/diagnostics.md @@ -79,7 +79,7 @@ references. A format/carrier version cannot substitute for either contract. required runtime; re-export with the matching preparation path when authorized. Do not edit a version number to silence validation or claim that Python graph handles can be reused in another process. See the existing loading owner's -[preparation reference](../../../../methods/weight-loading-optimization/references/flare-preparation-boundary.md). +[preparation reference](../../../../../playbooks/optimization/weight-loading-optimization/references/flare-preparation-boundary.md). ## Cancelled request while restore is in flight diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/lifecycle.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/lifecycle.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/lifecycle.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/lifecycle.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-ledger.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/source-ledger.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-ledger.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/source-ledger.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md similarity index 98% rename from src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md index 06364062d..ab0ccc293 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/flare-local-inference/audit-flare-session-lifecycle/references/source-notes.md @@ -22,7 +22,7 @@ means a proposed diagnostic action. None is a native/agent-efficacy result. | F1 / DOC | [`README.md`](https://github.com/NVIDIA/local-inference/blob/bce032c014bae34502ba22d3304fa084d5b40afe/README.md), [`docs/runtime-package-design.md`](https://github.com/NVIDIA/local-inference/blob/bce032c014bae34502ba22d3304fa084d5b40afe/docs/runtime-package-design.md) §§1–2 | Research-prototype SM120/SM121 scope; preparation/export/native ownership; nonportable live handles; independent payload/carrier versions | The generic addition belongs to the -[weight-loading preparation reference](../../../../methods/weight-loading-optimization/references/flare-preparation-boundary.md). +[weight-loading preparation reference](../../../../../playbooks/optimization/weight-loading-optimization/references/flare-preparation-boundary.md). Exporter code, native package reconstruction, signing/security qualification, model recipes and CUDA kernels were not audited. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md new file mode 100644 index 000000000..ba0b62e66 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md @@ -0,0 +1,220 @@ +--- +name: audit-h3-runtime-residency +description: Audit MiniMax-H3 generation in h3.c on Apple Silicon by tracing conditioning, AdaLN precompute, DiT residency, Metal precision paths, SSD streaming and audio/video decoding. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "MIT", + "tags": [ + "h3.c", + "MiniMax-H3", + "apple-silicon", + "metal", + "diffusion", + "video", + "audio", + "multimodal-conditioning", + "adaln", + "model-residency", + "ssd-streaming", + "int8", + "bf16" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/antirez/h3.c", + "commit": "8974cc055ea9c02fcd14cc27dfda3e1027c05153", + "paths": [ + "h3.h", + "h3_internal.h", + "h3.c", + "h3_weights.c", + "h3_weights.h", + "h3_safetensors.c", + "h3_host.c", + "h3_text_encoder.c", + "h3_vision_encoder.c", + "h3_multimodal.c", + "h3_video_encoder.c", + "h3_audio_vae.c", + "h3_video_vae.c", + "h3_dit_schedule.c", + "h3_dit_schedule.h", + "h3_dit.c", + "h3_gpu.m", + "h3_shaders.metal", + "main.c", + "h3_cli.c", + "Makefile", + "README.md", + "LICENSE", + "THIRD_PARTY_NOTICES.md", + "tests/test_h3.c", + "tests/test_bf16.c", + "tests/test_real_dit_schedule.c", + "tests/test_real_prompt.c", + "tests/test_real_multimodal_text.c", + "tests/test_semantic_dit.c", + "tests/test_semantic_vae.c", + "tests/test_real_audio_vae.c", + "tests/test_real_video_vae.c", + "tests/bench_dit.c" + ], + "readset": "references/source-notes.md", + "identity": "references/source-identity.json", + "license": "MIT", + "third_party_notices": "references/THIRD_PARTY_NOTICES.md" + }, + { + "type": "git", + "url": "https://github.com/InferMatrixLab/InferRSI", + "commit": "f804ec1503e977787c572d5d27d1f3258017b01f", + "paths": [ + "src/inferrsi/data/skill_library/workflows/research/inference-engine-research/references/minimax-h3-apple.md", + "src/inferrsi/data/skill_library/workflows/research/inference-engine-research/references/minimax-h3-weight-inventory.json" + ], + "scope": "Prior model/engine comparison and safetensors header observations, reused without reissuing range requests; inventory bytes preserved." + }, + { + "type": "document", + "url": "https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/42ed227ee7df40d41602854ae760620d6eb651fe/FL2VA", + "version": "42ed227ee7df40d41602854ae760620d6eb651fe", + "scope": "29 JSON-header digests and component aggregates retained from prior inspection; payload and complete model license not newly verified.", + "license": "UNKNOWN", + "readset": "references/minimax-h3-weight-inventory.json" + }, + { + "type": "document", + "url": "https://huggingface.co/appautomaton/minimax-h3-base-8bit-mlx/tree/9a08116a349ab469b097775b5a483affcb5ccd34", + "version": "9a08116a349ab469b097775b5a483affcb5ccd34", + "scope": "Four selected export JSON-header digests/packing aggregates retained from prior inspection; payload and model/export license not newly verified.", + "license": "UNKNOWN", + "readset": "references/minimax-h3-weight-inventory.json" + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "capability-specific", + "strategy": "direct", + "status": "portable-candidate", + "native_import": "not-run", + "method": "Manual pinned-source distillation informed by the existing Repo-To-Skill ledger; no AREX/DisCo execution or behavioral efficacy claim." + }, + "applicability": { + "scope": "MiniMax-H3 joint audio/video C/Metal runtime audit: raw checkpoint compatibility, input encoders, schedule precompute, phased/cached residency, numerical controls and SSD streaming on Apple Silicon. Common catalog placement is a discovery classification, not cross-backend qualification.", + "prerequisites": "Access to the pinned source architecture and selected component/format inventory; native checks need matching model, fixture and backend resources within an authorized experiment budget.", + "input_contract": "Runtime revision/patches, macOS/SDK/GPU and memory budget, FL2VA/Ref2VA selection, component formats, prompt and ordered media, requested/aligned frames, canvases, steps/seed, layer/reuse/token/precision policies and preview/interactive-cache state.", + "produced_artifact": "A phase/component/owner ledger with resident weights, runtime/host buffers, overlap, release/reuse boundaries and observations/unknowns, followed by a discriminating check, actual result and remaining gap.", + "failure_modes": "Estimating fit from model-file bytes, confusing runtime int8 with incompatible MLX bundles, ignoring preview/cache overlap, or reducing DiT residency to address a VAE/host-output peak can produce a wrong capacity diagnosis.", + "refusal_point": "Stop at unavailable model/fixture/backend resources or the accepted experiment budget; missing target evidence remains unverified/INCONCLUSIVE." + }, + "limitations": [ + "Native h3.c/Metal execution, target peak memory, audio/video quality, performance, service behavior and skill efficacy were not run.", + "Only the bounded readset was inspected; complete kernel, driver/MPSGraph, tokenizer, FFmpeg, concurrent-caller and failure-path coverage is not established.", + "The retained inventory authenticates only prior JSON header observations, not weight payloads; full official versus selected truncated/packed export inventories are distinct.", + "Upstream 128 GB M5 Max results and tracked DiT bytes do not qualify a 64 GiB target, a different workload or total physical memory.", + "MIT covers the h3.c source; retained BSD-3-Clause notices and independent model/export/tool licenses remain separate. UNKNOWN source-license fields are unresolved." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/source-identity.json", + "references/minimax-h3-weight-inventory.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Resolve the actual conditioning and output components before counting residency; do not add alternative checkpoints or count shared files twice.", + "Trace one-shot, preview and interactive ownership separately, including AdaLN schedule dependencies and prepared-cache reset boundaries.", + "Classify approximation and precision controls before comparison, then measure the relevant phase with tensor, physical/compressed/swap and storage observations." + ] + } +--- + +# Audit h3.c runtime residency and numerical paths + +Source-derived candidate for `antirez/h3.c` at +`8974cc055ea9c02fcd14cc27dfda3e1027c05153`. Use for MiniMax-H3 +text/image/video/audio conditioning, memory-fit diagnosis, kernel changes or a +dedicated-engine design derived from this C/Metal runtime. Start with the +[source readset and architecture](references/source-notes.md). Native execution, +media quality, performance and behavioral skill efficacy are **not_run**. + +## Inputs and decision + +Record the runtime revision/patches, macOS/SDK/GPU, memory budget, selected FL2VA +or Ref2VA checkpoint and complete component/format inventory. Freeze the prompt, +ordered reference media, requested and aligned frames, output/internal canvas, +steps, seed, layer/reuse/token policies, all precision controls, preview and +interactive-cache state. An installed engine name or a successful metadata load +does not identify this execution configuration. + +Produce a phase ledger and one smallest discriminating check. The useful output +is a supported cause, invariant, counter-explanation and falsifier; model-file +bytes alone do not decide whether a workload fits unified memory. + +## Procedure + +1. **Resolve the executing pipeline.** Follow `h3_generate` through the + [phase ledger](references/phase-residency.md). Ordered references select the + separate Ref2VA DiT; FL2VA first/last-frame conditioning still needs both the + video input encoder and the Qwen vision/language path. Audio conditioning and + audio output decoding are different calls. Count the selected input and + output components without adding alternative checkpoints or repeated shared + files. Use the [retained weight inventory](references/minimax-h3-weight-inventory.json) + for its exact checkpoint selections, not as a universal model-size constant. +2. **Check representation before capacity.** The raw loader validates exact + names, rank, shape and BF16/F32 dtype. Runtime symmetric int8 conversion is + different from loading MLX affine `U32` bundles. A smaller published bundle + needs a compatible engine or an explicit conversion; matching bit width is + insufficient. Keep header validity and payload integrity separate. +3. **Trace ownership across phases and requests.** Distinguish host embeddings, + media/latent/output buffers, shared Metal tensors, file-backed mappings, + AdaLN tables and prepared-model/decoder caches. In the one-shot path the DiT + is freed before decoding; preview and interactive retention can overlap + phases. Audit cache keys and clear/reset boundaries before diagnosing a leak + or assuming a warm request has cold-request memory behavior. +4. **Preserve the schedule contract.** AdaLN tables are indexed by the actual + video/audio schedule, conditioning presence, modality tags and layout. + Each projection is submitted and released before loading the next. Rebuild + when those dependencies change; do not replace conditioning timesteps or + collapse audio/video rows because their step counts match. See the + [schedule and cache invariants](references/phase-residency.md#adaln-and-prepared-state). +5. **Classify every numerical change.** Follow + [precision and streaming](references/precision-and-streaming.md). Separate + BF16-preserving fusions from int8 approximation, BF16 output-head conversion, + layer thinning, velocity/core reuse, token reduction and internal-canvas/RoPE + changes. On M5, 50 layers and `--reuse 1` still allow default int8 paths. + Select the explicit BF16 controls and F32-head diagnostic when that is the + frozen reference. A kernel parity result is not full audio/video quality. +6. **Choose the residency tradeoff that addresses the measured phase.** Qwen + prefetch overlaps current-layer compute with future-layer reads and increases + resident buffers. Mapped persistent DiT weights remain file-backed; explicit + SSD mode instead rotates two BF16 matrix slots and disables int8. It changes + I/O and command boundaries. Reducing DiT residency will not resolve a VAE, + reference-media or host-output peak. Track physical/compressed/swap memory + alongside tensor counters and storage reads. +7. **Run the matching check when resources exist.** The + [validation recipe](references/validation.md) maps native targets, required + fixtures and proposed probes to these invariants. Preserve skipped fixtures, + startup/first/warm timing, both output modalities and all failed trials. + The [Apple feasibility comparison](references/minimax-h3-apple.md) is retained + imported research; neither its upstream measurements nor packaged-reader + tests qualify a new Mac, workload or 64 GiB memory budget. + +## Report and stop + +Return a table with `phase`, `component and format`, `owner`, `resident weights`, +`runtime/host buffers`, `overlap`, `release or reuse boundary`, and +`observation/unknown`; then the exact check, actual outcome and remaining gap. +Stop at unavailable model/fixture/backend resources or the accepted experiment +budget. Reading this skill does not install an engine, download weights, run a +distiller, authorize a campaign or promote this candidate. Missing target +evidence remains unverified/INCONCLUSIVE. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/THIRD_PARTY_NOTICES.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/THIRD_PARTY_NOTICES.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/THIRD_PARTY_NOTICES.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/THIRD_PARTY_NOTICES.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md similarity index 99% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md index 96d665dea..771e2dcb4 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/minimax-h3-apple.md @@ -52,7 +52,7 @@ accelerator. To read this packaged example outside the checkout: from importlib.resources import files owner = files("inferrsi").joinpath( - "data/skill_library/repositories/h3-c/audit-h3-runtime-residency" + "data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency" ) print(owner.joinpath("references/minimax-h3-apple.md").read_text(encoding="utf-8")) ``` diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-weight-inventory.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/minimax-h3-weight-inventory.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/minimax-h3-weight-inventory.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/minimax-h3-weight-inventory.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/phase-residency.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/phase-residency.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/phase-residency.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/phase-residency.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/precision-and-streaming.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/validation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/references/validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/references/validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md new file mode 100644 index 000000000..bc0e4c5f5 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md @@ -0,0 +1,188 @@ +--- +name: audit-h3-metal-fasth3-semantics +description: "Audit h3-metal FastH3 effective configuration, checkpoint tensor and schedule semantics, partial VSA tiles, sparse eligibility and dense-plus-coarse fallback without treating source or header checks as native parity." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NO_LICENSE", + "tags": [ + "h3-metal", + "FastH3", + "MiniMax-H3", + "apple-silicon", + "metal", + "qkv-layout", + "swiglu", + "bf16", + "vsa", + "sparse-attention", + "fallback", + "effective-configuration" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/zuiho-kai/h3-metal", + "commit": "4ba47a3eabdbebf801ac6664b31f8ec168bc57a8", + "paths": [ + "LICENSE", + "THIRD_PARTY_NOTICES.md", + "Makefile", + "README.md", + "docs/USAGE.md", + "h3.h", + "h3.c", + "main.c", + "h3_host.h", + "h3_host.c", + "h3_dit.c", + "h3_dit_schedule.c", + "h3_gpu.m", + "h3_shaders.metal", + "tests/test_h3.c", + "tests/test_vsa_coarse.c", + "tests/test_real_dit_schedule.c", + "tests/bench_dit.c", + "tools/fasth3_keymap.py", + "tools/fasth3_keymap.md", + "bench/run_fox2.sh" + ], + "readset": "references/source-notes.md", + "identity": "references/source-identity.json", + "license": "MIT; selected shader-derived components separately BSD-3-Clause", + "third_party_notices": "references/THIRD_PARTY_NOTICES.md" + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill", + "cli/packages/coding-agent/src/disco/skills/extend-repo-skill" + ], + "scope": "Portable source/CPU-only preparation, extraction, safe-script disposition, difficult cases and verification adaptation to InferRSI ownership; no DisCo CLI/model run or managed import." + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "capability-specific", + "strategy": "arex-portable-meta-skills", + "status": "partial-candidate", + "native_import": "not-run", + "method": "Bounded fork-specific source audit, bundled selected-declaration helper and assertion-backed cases. Native Apple/Metal/model evidence remains blocked; no numerical/performance/skill-efficacy claim." + }, + "applicability": { + "scope": "h3-metal FastH3 requested/effective configuration, checkpoint transformation and schedule/RoPE semantics, VSA prefix/partial tiles and dispatch/fallback audit. Common catalog placement is discovery, not CPU inference or Apple qualification.", + "prerequisites": "Access to the fork-specific source and semantic contracts; native checks additionally need their matching Metal/model resources and separate execution authority.", + "input_contract": "Source revision/patches, requested parameters and explicitly captured environment controls, checkpoint path/key-family/config identity, requested/aligned frames, latent dimensions, packed prefix segments and sparse-pipeline availability or unknown.", + "produced_artifact": "A ledger of requested and normalized parameters, ambient overrides, source invariants, counterexamples, observed/unknown facts and the smallest discriminating next check.", + "failure_modes": "Ignoring ambient overrides, mixing checkpoint identity switches, using global rather than per-head Q/K/V concatenation, pooling padded boundary rows, or double-adding coarse attention can invalidate the semantic audit.", + "refusal_point": "Stop on conflicting configuration, mixed checkpoint identity, missing resources or the authorized boundary; name the missing native test/fixture and leave it blocked/not_run." + }, + "limitations": [ + "BLOCKED_REQUIRED_BACKEND: native Apple/Metal, actual FastH3 checkpoint/schedule/sparse parity, audio/video quality, target memory fit, performance and service evidence not_run.", + "Offline helper audits caller declarations and selected arithmetic only, not engine parameter validity, payload/dtype/conversion equivalence, attention numerics, actual environment or dispatch.", + "Bounded file/range readset only; full kernels, model/config/weights, encoders/VAEs, device/runtime behavior and external FastVideo conformance are not established.", + "Native coarse VSA and original-H3 schedule test definitions do not cover sparse fine or FastH3 BF16 schedule execution; no native tests were run.", + "Key-map and benchmark source scripts are excluded from execution; checked-in reports, source comments and existing measurement documents are not new local evidence.", + "NO_LICENSE is the actual unauthenticated machine-resolver fallback, not absence of licensing. Inspected MIT source and BSD-3-Clause notices are retained; model rights remain separate/unverified." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/source-identity.json", + "references/repo-provenance.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Record requested parameters, normalized values and later environment overrides separately, including requested versus aligned frame counts.", + "Audit checkpoint transformations and schedule consumers beyond tensor names/shapes; tiny permutation fixtures can falsify mappings but do not establish model parity.", + "Resolve sparse eligibility and the dense-fine-plus-gated-coarse fallback explicitly; preserve unknown pipeline availability." + ] + } +--- + +# Audit h3-metal FastH3 semantics + +Use for the fork-specific four-step DiT path, a checkpoint-loader port, confusing +FastH3 flags, or sparse-attention fallback. This is a **source-derived candidate**, +not a generation/benchmark runner. Read the [source and scope](references/source-notes.md) +and [semantic contracts](references/semantic-contracts.md). Native Metal, model, +quality, memory-fit, performance and behavioral skill validation are **not_run**. +General h3.c residency remains with the separately registered +`audit-h3-runtime-residency`; the Apple measurement playbook is context, not new +evidence for these contracts. + +## Inputs and output + +Record source revision/patches, requested parameters, **explicitly captured** +environment keys, checkpoint path/key-family/config identity, requested/aligned +frames, latent dimensions, packed prefix segments and actual sparse-pipeline +availability (unknown is allowed). Keep model payload integrity, licensing and +native execution separate. Do not collect credentials or the complete environment. + +Return a ledger with `requested`, `normalized parameter`, `ambient override`, +`source invariant`, `counterexample`, `observed/unknown`, and `next check`. +Finish with the smallest discriminating check, not an inferred speed/quality claim. + +## Procedure + +1. **Resolve the effective request.** FastH3 normalizes the API parameters to + four steps, 50 layers, denoise/core reuse 1, token-reduction parameter 0 and + SSD streaming 0 before validation. Ordered Ref2VA references are rejected; + FL2VA first/last-frame anchors are a different input route. Capture ambient + token-reduction controls: normalization does **not** erase their later effect. + Record both requested and aligned frame counts; requested 49 aligns to 56. +2. **Check checkpoint identity before mapping.** The DiT's path-substring switch + and schedule's key-presence switch differ. Identify both, reject mixed-family + assumptions and inspect the complete intended config/schema. Matching tensor + names/shapes or an inferred header is not conversion, dtype or payload proof. +3. **Audit transformations, not just names.** Separate Q/K/V tensors must become + per-head Q/K/V groups, not one global Q-then-K-then-V concatenation. Swap FC1 + halves before quantization. Preserve mapped BF16-to-F32 consumers, the BF16 + timestep MLP/materialization order, separate audio/video sigma rows and the + generated RoPE dependency. Use labeled tiny fixtures to falsify the wrong + permutation; do not call such a fixture numerical model parity. +4. **Reconstruct VSA membership.** Merge adjacent same-kind prefix segments, + then split each merged segment into at-most-64-row tiles. Target video uses + ceil-divided 4×4×4 blocks on the patchified latent grid. Pool real boundary + rows only. Prefix queries retain every key tile; video queries retain all + prefix keys plus top-k video keys. Preserve score-descending/index-ascending + ties, row/head-major distinctions and scatter order. +5. **Establish actual dispatch.** Top-k uses double-precision arithmetic; 120 + video tiles yield 12, not the float32 counterexample's 13. Sparse fine requires + an available pipeline, no sparse-disable environment key, and a full sequence. + The disable key's **presence**, even value `0`, disables it. Sparse fine already + includes gated coarse; fallback is **dense fine plus gated coarse**, not base + H3 and not a reason to add coarse twice. Unknown pipeline availability stays + unknown. Check the [failure table](references/troubleshooting.md). +6. **Run only the authorized layer.** The bundled + [offline audit](scripts/audit_fasth3.py) accepts a caller-declared JSON object + and performs selected configuration/tile arithmetic, without probing this + machine, loading weights or running source scripts. Its + [input/output contract](references/preflight.md) includes executable examples + and counterchecks. Native candidates and required resources are classified in + the [source notes](references/source-notes.md#native-candidates-and-script-dispositions). + No CPU helper replaces their missing Apple/Metal/model evidence. + +## Stop and report + +Stop at conflicting configuration, mixed checkpoint identity, missing resources +or the authorized boundary. Name the next native test/fixture needed and report +it as blocked/not_run, not passing. The key-map and benchmark scripts were +inspected but are **not** bundled execution dependencies. Reading this procedure +does not authorize downloading, conversion, generation, benchmarks, upstream +publication or candidate promotion. Missing acceptance evidence is INCONCLUSIVE. + +The [provenance and licensing note](references/repo-provenance.md) explains why +the machine-resolver fallback `NO_LICENSE` does not negate the inspected MIT +source license or the retained BSD-3-Clause component notice. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/THIRD_PARTY_NOTICES.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/THIRD_PARTY_NOTICES.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/THIRD_PARTY_NOTICES.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/THIRD_PARTY_NOTICES.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/preflight.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/preflight.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/preflight.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/preflight.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/repo-provenance.md similarity index 95% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/repo-provenance.md index a244cc0a6..ba4582090 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/repo-provenance.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/repo-provenance.md @@ -35,5 +35,5 @@ independently inspected MIT/BSD texts. No authentication or credential changes were made. The check-only resolution record lives outside the runtime skill. InferRSI's `local-skill-v1` contract places name/description in frontmatter and -structured fields, including license, in `metadata.json`; DisCo-only visibility +structured fields, including license, in the SKILL.md metadata.inferrsi block; DisCo-only visibility and managed-import fields are not imposed on this packaged destination. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/semantic-contracts.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/semantic-contracts.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/semantic-contracts.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/semantic-contracts.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/scripts/audit_fasth3.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/scripts/audit_fasth3.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/scripts/audit_fasth3.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/scripts/audit_fasth3.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/husky/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/husky/SKILL.md similarity index 51% rename from src/inferrsi/knowledge/data/skill_library/repositories/husky/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/husky/SKILL.md index 3f249900c..a980662fb 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/husky/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/husky/SKILL.md @@ -5,6 +5,79 @@ license: NO_LICENSE disable-model-invocation: true metadata: disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NO_LICENSE", + "tags": [ + "husky", + "document-research", + "evidence-qualification", + "negative-results" + ], + "sources": [ + { + "type": "document", + "url": "https://husky.underdog.ai/", + "displayed_publication_date": "2026-09-20", + "accessed": "2026-10-27", + "license": "UNKNOWN", + "scope": "Public unversioned official-page view; bounded author-report summary, not source code or retained receipts." + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/workflow-authoring", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" + ], + "scope": "AREX-informed document-only adaptation explicitly accepted by the user; not complete repo-to-skill or native verification." + } + ], + "construction": { + "anchor_kind": "document", + "distillation_form": "document-research-procedure", + "strategy": "arex-informed-document-adaptation", + "status": "partial-candidate", + "native_import": "not-run", + "method": "Single coherent research workflow, bounded source summary, original verification questions and offline schema checks. Repository/environment/native stages remain unavailable." + }, + "applicability": { + "scope": "Review Husky document claims and specify missing evidence; common catalog scope is not inference backend compatibility.", + "prerequisites": "Access to the bounded official-document inventory and verification protocol; implementation and native reproduction remain unavailable until separately supplied and authenticated.", + "input_contract": "The exact claim, source/view date, workload and comparison intent, supporting evidence classes and available implementation/model/output/measurement artifacts; unknown identity, licensing and availability remain explicit.", + "produced_artifact": "A reviewable claim/evidence request recording source availability, mechanism hypothesis, counter-signal, missing identities/artifacts, proposed falsifying check and actual result or bounded INCONCLUSIVE conclusion.", + "failure_modes": "Treating a public-page reading as code inspection, proposed extensions as shipped behavior, summed independent ratios as one gain, or a structurally valid self-reported card as authentic evidence can overstate qualification.", + "refusal_point": "Stop at missing source/receipts, unsafe or private data, unsupported resources or the authorized budget; unavailable artifacts yield an unverified/INCONCLUSIVE research conclusion." + }, + "limitations": [ + "No canonical public repository, implementation revision, retained receipts or source-code license was identified; unknowns remain unknown.", + "The unversioned official page is author-reported and unreproduced; no immutable source HTML hash is claimed.", + "Native reproduction is unavailable; schema checks do not authenticate supplied identities, artifacts, output equality or performance.", + "Research protocols and ablation questions are original proposals, not claims about uninspected implementation behavior.", + "NO_LICENSE is the unavailable repository-license resolver result, not a legal absence finding or grant to copy publisher material.", + "Final informed candidate admission is separate; no automatic import, native qualification or skill efficacy is claimed." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-availability.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Separate author reports, inspected implementation, executed experiments and new interpretations, including negative findings and future work.", + "Request immutable identities and artifacts, then state one mechanism, competing explanation and smallest ablation with frozen semantic requirements.", + "Use the offline card auditor only to identify structural gaps; it authenticates neither artifacts nor execution and leaves qualification unadjudicated." + ] + } --- # Husky document-derived research diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/husky/references/author-report.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/author-report.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/husky/references/author-report.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/author-report.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/husky/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/husky/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/husky/references/source-availability.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/source-availability.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/husky/references/source-availability.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/source-availability.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/husky/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/husky/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/husky/references/verification-protocol.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/verification-protocol.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/husky/references/verification-protocol.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/husky/references/verification-protocol.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/husky/scripts/audit_evidence_card.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/husky/scripts/audit_evidence_card.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/husky/scripts/audit_evidence_card.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/husky/scripts/audit_evidence_card.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/NOTICE.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/NOTICE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/NOTICE.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/NOTICE.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/SKILL.md new file mode 100644 index 000000000..23b40f790 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/SKILL.md @@ -0,0 +1,200 @@ +--- +name: lily +description: "Investigate Lily's exact Qwen3.6-35B-A3B MLX-affine-Q4 contract, Metal state and kernels, strict greedy text API, token-prefix session cache and source-qualified validation or benchmark plans." +license: NO_LICENSE +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NO_LICENSE", + "tags": [ + "lily", + "rust", + "metal", + "apple-silicon", + "mlx-affine-q4", + "session-prefix", + "numerical-validation", + "benchmark-provenance" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/perplexityai/pplx-garden", + "commit": "1ed972ed3f0bd5616c997c9507c25616c63394fc", + "paths": [ + "lily/README.md", + "lily/Cargo.toml", + "lily/Cargo.lock", + "lily/rust-toolchain.toml", + "lily/CONTRIBUTING.md", + "lily/LICENSE", + "lily/NOTICE", + "lily/src/lib.rs", + "lily/src/config.rs", + "lily/src/weights.rs", + "lily/src/safetensors.rs", + "lily/src/tensor.rs", + "lily/src/metal.rs", + "lily/src/model.rs", + "lily/src/serve.rs", + "lily/src/serve/session.rs", + "lily/src/chat.rs", + "lily/src/generate.rs", + "lily/src/tokenizer.rs", + "lily/src/kernels/attention.rs", + "lily/src/kernels/gdn.rs", + "lily/src/kernels/gemm.rs", + "lily/src/kernels/skinny.rs", + "lily/src/bin/lily-bench.rs", + "lily/tests/support/cpu_ref.rs", + "lily/tests/test_shader_compile.rs", + "lily/tests/test_e2e_35b.rs", + "lily/tests/test_tokenizer.rs", + "lily/tests/unit/weights.rs", + "lily/tests/unit/serve.rs", + "lily/tests/unit/serve/session.rs", + "lily/tests/unit/generate.rs", + "lily/tests/unit/kernels/gemm.rs", + "lily/tests/unit/kernels/gdn.rs", + "lily/tests/unit/kernels/skinny.rs", + "lily/docs/2026-09-01-performance-mlx-0.31.2.md", + "lily/docs/2026-09-02-performance-mlx-0.32.2.md", + "lily/benchmarks/run_matrix.py", + "lily/benchmarks/bench_mlx.py", + "lily/benchmarks/requirements.txt" + ], + "license": "Apache-2.0", + "scope": "Lily subtree only; independent lily/LICENSE and NOTICE retained.", + "readset": "references/source-identity.json" + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/workflow-authoring", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" + ], + "scope": "Portable native Rust/Metal adaptation executed by a compatible agent; not a DisCo CLI/model run." + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "repository-operating-graph", + "strategy": "arex-portable-meta-skills", + "status": "partial-candidate", + "native_import": "not-run", + "method": "User-approved native Rust/Metal and partial drafting adaptation; stable-ID lanes, integrated source graph and assertion/helper verification. Missing compiler/backend/checkpoint remains explicit." + }, + "applicability": { + "scope": "Exact Lily source/checkpoint/layout/API/cache/qualification investigation; common catalog classification is not CPU inference or cross-device support.", + "prerequisites": "Access to the exact Lily source/model-layout/API contracts; native checks require the matching Rust/Metal/checkpoint environment and frozen comparison requirements.", + "input_contract": "Source revision, Cargo lock/toolchain, macOS/chip/GPU family, checkpoint and tensor payload identities, affine configuration/layout, tokenizer/template/stops, sequence/output budgets, greedy policy, input history, prefix-cache state and quality/measurement requirements.", + "produced_artifact": "A source/model/device-bound representation, serving-cache or qualification diagnosis with the observation, invariant, alternate cause, smallest falsifier, actual result and remaining native gap.", + "failure_modes": "Equal bit width, file size, API names or prompt_cache_key do not establish tensor/token/state equivalence; ignoring the extra lookahead slot can understate request capacity, and green CPU references do not replace matched-model Metal goldens.", + "refusal_point": "Stop at unsupported formats/APIs, missing compiler/backend/checkpoint, unfrozen gates or the authorized budget; do not weaken ignored/golden tests to obtain a green result." + }, + "limitations": [ + "Cargo/Rust1.92, supported M5/macOS26/Metal and model resources are unavailable; native evidence remains BLOCKED_REQUIRED_BACKEND.", + "Offline Python helper checks are not Rust serde/loader/tokenizer, native kernel/state/cache, model quality, service or performance validation.", + "Current benchmark source and later report have distinct package/harness identities; negative long-context prefill observations remain author-reported.", + "Only selected source bodies are mined; complete kernel/format/platform/failure coverage and behavioral skill efficacy remain unverified.", + "NO_LICENSE is the actual unauthenticated GitHub CLI fallback, not a legal conclusion; lily Apache-2.0 LICENSE and NOTICE are independently retained." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Resolve exact affine representation and strict API semantics, then trace token frontier, cache key and restore/release ownership.", + "Retain the source's extra request lookahead slot in capacity arithmetic rather than counting only prompt and visible output.", + "Separate reported benchmark version from the checked-out runner and distinguish offline screening, native compilation, Metal operators and matched-model goldens." + ] + } +--- + +# Lily inference engineering + +Use this source-derived candidate to investigate representation, state or +qualification questions in `perplexityai/pplx-garden/lily`. It is not a generic +Metal/MLX runner. Read [source and method provenance](references/repo-provenance.md) +before applying it to a different revision, model or device. + +## Route the question + +| Question | Route | Expected result | +| --- | --- | --- | +| A nominally 4-bit model fails to load, or memory/sequence capacity is unclear | [model-runtime](sub-skills/model-runtime/SKILL.md) | Exact architecture/layout and ownership/capacity contract | +| A request is rejected, or equal keys/history do not reuse state as expected | [serving-cache](sub-skills/serving-cache/SKILL.md) | Strict API, token-prefix and state-lifetime diagnosis | +| CPU references look green, a golden is ignored, or MLX comparisons disagree | [qualification](sub-skills/qualification/SKILL.md) | Matching native evidence, version and metric boundaries | + +The focused routes bundle safe offline screening/request support where useful. +Those helpers do not invoke Rust, compile shaders, load tensors, tokenize a real +checkpoint or start inference. Their positive result is not native acceptance. + +## Establish the execution tuple + +Record engine/source and Cargo lock/toolchain, macOS/chip/GPU family, exact +checkpoint and tensor payload identities, config/affine layout, tokenizer/chat +template and output-stop identity. Add sequence/output budgets, greedy policy, +input history, prefix-cache state, workload and quality/measurement requirements. + +Equal nominal bit width, file size, API name or `prompt_cache_key` does not +identify equivalent tensors, rendered tokens or valid recurrent/KV state. +Separate reported benchmark identity from the checked-out runner's version. + +## Native preparation boundary + +The source requires Rust 1.92, macOS 26 and Apple GPU family 10/M5 or newer. +For an authorized source build the public command is +`cargo build --release --locked`; shader source is compiled at runtime, not by +an offline metallib step. Inspect an installed binary's help before any model +run. Building a package does not supply its supported checkpoint or qualify it. + +This extraction has neither Cargo/Rust nor a compatible Mac/model environment. +AREX's Python-package preparation was explicitly adapted to native Rust/Metal +identity and small offline helper checks. No installed Python distribution, +Rust test run or native backend success is fabricated. + +## Work one bounded hypothesis + +1. Read the selected bundled contract and failure notes. +2. Resolve exact input representation and actual API semantics before tuning. +3. Trace owner, logical token frontier, allocation, cache key and release or + restore boundary. Matching visible text is weaker than exact token identity. +4. Preserve the source's extra request lookahead slot when budgeting; do not + replace its checked arithmetic with only prompt-plus-visible-output length. +5. Freeze workload, checkpoint/version, numerical/quality controls and metrics + before comparing. Gates and campaign revisions belong in an acceptance issue. +6. Distinguish source inspection, offline helper checks, compiled native tests, + real Metal operators, matched-model goldens and service measurements. + +## Stop and report + +Return exact source/model/device identity, observation, invariant, alternate +cause, smallest falsifier, actual check result and remaining gap. Stop at +unsupported format/API, missing compiler/backend/checkpoint, unfrozen gates or +the authorized budget. Do not install a toolchain, download a model, alter the +API or weaken an ignored/golden test merely to obtain a green result. + +Required native compiler/Metal/model evidence remains blocked. CPU reference +functions and Python screening are not GPU test execution; skipped or ignored +cases do not become passing evidence. No quality, performance, service or +behavioral skill-efficacy qualification is claimed. Final partial-candidate +admission is a separate informed decision, not automatic import. + +Source [Apache-2.0 license](LICENSE) and [NOTICE](NOTICE.md) are retained. +`NO_LICENSE` is the actual AREX unauthenticated-CLI fallback, not a legal finding; +model weights and third-party material retain their own obligations. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/references/model-and-memory.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/references/model-and-memory.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/references/model-and-memory.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/references/model-and-memory.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/scripts/screen_config.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/scripts/screen_config.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/model-runtime/scripts/screen_config.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/model-runtime/scripts/screen_config.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/references/verification-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/references/verification-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/references/verification-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/references/verification-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/scripts/audit_benchmark_record.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/scripts/audit_benchmark_record.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/qualification/scripts/audit_benchmark_record.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/qualification/scripts/audit_benchmark_record.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/references/api-and-session.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/references/api-and-session.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/references/api-and-session.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/references/api-and-session.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/scripts/request_template.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/scripts/request_template.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lily/sub-skills/serving-cache/scripts/request_template.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lily/sub-skills/serving-cache/scripts/request_template.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/SKILL.md similarity index 57% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/SKILL.md index ef7537608..50d431444 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/SKILL.md @@ -1,6 +1,81 @@ --- name: lithos-metal description: "Inspect Lithos Metal compilation and fusion, GPU-resident generation and DSpark state, or local serving and exact-prefix caches on Apple silicon; use when changing recipes, diagnosing stale state, or qualifying an inference claim." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "tags": [ + "lithos-metal", + "metal", + "apple-silicon", + "repository-guide" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/lithos-ai/lithos-metal", + "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", + "paths": [ + "kernels/common/spec_ops.metal", + "monolith/backends/metal/base.py", + "monolith/backends/metal/registry.py", + "monolith/compiler/arena.py", + "monolith/compiler/barriers.py", + "monolith/compiler/coverage.py", + "monolith/compiler/decoder_fusion.py", + "monolith/compiler/emit.py", + "monolith/compiler/region_fusion.py", + "monolith/compiler/static_fusion.py", + "monolith/core/step_state.py", + "monolith/generate.py", + "monolith/runtime/engine.py", + "monolith/runtime/prefix_cache.py", + "monolith/runtime/prefix_store.py", + "monolith/serve.py", + "monolith/serving/cache.py", + "monolith/serving/lifecycle.py", + "monolith/serving/setup.py", + "monolith/spec/dspark/model.py", + "runtime/src/metal_core.mm" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately.", + "prerequisites": "Access to the pinned package/API and selected compiler, generation or serving contracts; revision/API drift needs a fresh bounded inspection, and efficacy claims need campaign-owned gates and matching native/model evidence.", + "input_contract": "Actual source/package version, backend/configuration, chip/core count and OS, target/draft checkpoint identities, context and prompt-token IDs, sampling/numerical options and the affected recipe/runtime/session/cache boundary.", + "produced_artifact": "A routed region/dependency, committed-state or cache-identity/lifecycle ledger, followed by the smallest supported finding, missing checks and next bounded action.", + "failure_modes": "Treating session adoption/cache hits as generation correctness, fewer dispatches as faster rounds, another chip recipe as the default, CPU cache checks as GPU rollback or health liveness as loaded/warm model state can yield unsupported conclusions." + }, + "limitations": [ + "Source-derived candidate; empirical efficacy validation not_run.", + "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", + "No model downloads, live DisCo deployment or inference backend integration is provided." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "license": "Apache-2.0", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "recommendations": [ + "Follow session replacement across recipe geometry, committed generation state and serving adoption/cache identity before interpreting a cache hit.", + "Keep an unfused/native or target-only control with matching decoded weights, numerical mode, row bounds and sampling policy; verify dependencies and persistent state before timing.", + "Distinguish package/CPU checks, synthetic Metal execution, model continuation and HTTP workload evidence; chip recipes and target-compute projections do not establish another device or E2E rate." + ] + } --- # Lithos Metal operating guide diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/mlx-license.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/mlx-license.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/mlx-license.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/mlx-license.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/notices.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/notices.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/notices.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/notices.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/package-api.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/package-api.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/package-api.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/package-api.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/upstream-notice.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/upstream-notice.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/references/upstream-notice.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/references/upstream-notice.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/scripts/inspect_contract.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/scripts/inspect_contract.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/scripts/inspect_contract.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/scripts/inspect_contract.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/compile-fusion/SKILL.md similarity index 66% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/compile-fusion/SKILL.md index 2206f1a02..2dedcd7f4 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/compile-fusion/SKILL.md @@ -1,6 +1,60 @@ --- name: compile-fusion description: "Audit Lithos Metal task compilation, chip-qualified fusion, normalized controls, ICB barriers and scratch lifetimes when changing a recipe or diagnosing a fused kernel." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "tags": [ + "lithos-metal", + "metal", + "apple-silicon", + "compile-fusion" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/lithos-ai/lithos-metal", + "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", + "paths": [ + "monolith/compiler/emit.py", + "monolith/compiler/region_fusion.py", + "monolith/compiler/static_fusion.py", + "monolith/compiler/decoder_fusion.py", + "monolith/compiler/barriers.py", + "monolith/compiler/arena.py", + "monolith/compiler/coverage.py", + "monolith/backends/metal/base.py", + "monolith/backends/metal/registry.py" + ], + "readset": "../../references/source-notes.md" + } + ], + "applicability": { + "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately." + }, + "limitations": [ + "Source-derived candidate; empirical efficacy validation not_run.", + "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", + "No model downloads, live DisCo deployment or inference backend integration is provided." + ], + "provenance_readsets": [ + "../../references/source-notes.md" + ], + "consumption_mode": "packaged", + "license": "Apache-2.0", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + } --- # Compile and qualify a fusion region diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/references/mechanisms.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/compile-fusion/references/mechanisms.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/references/mechanisms.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/compile-fusion/references/mechanisms.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/compile-fusion/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/compile-fusion/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/compile-fusion/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/generation-state/SKILL.md similarity index 70% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/generation-state/SKILL.md index bd314f773..0807d6e58 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/generation-state/SKILL.md @@ -1,6 +1,57 @@ --- name: generation-state description: "Trace Lithos Metal GPU-resident generation and DSpark verification, commit/rollback, token-ring and cancellation state when continuation, acceptance or stop behavior is wrong." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "tags": [ + "lithos-metal", + "metal", + "apple-silicon", + "generation-state" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/lithos-ai/lithos-metal", + "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", + "paths": [ + "monolith/core/step_state.py", + "monolith/generate.py", + "monolith/spec/dspark/model.py", + "monolith/runtime/engine.py", + "runtime/src/metal_core.mm", + "kernels/common/spec_ops.metal" + ], + "readset": "../../references/source-notes.md" + } + ], + "applicability": { + "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately." + }, + "limitations": [ + "Source-derived candidate; empirical efficacy validation not_run.", + "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", + "No model downloads, live DisCo deployment or inference backend integration is provided." + ], + "provenance_readsets": [ + "../../references/source-notes.md" + ], + "consumption_mode": "packaged", + "license": "Apache-2.0", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + } --- # Trace generation and speculative state diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/references/state-machine.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/generation-state/references/state-machine.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/references/state-machine.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/generation-state/references/state-machine.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/generation-state/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/generation-state/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/generation-state/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/SKILL.md similarity index 69% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/SKILL.md index 104c045de..3460e7561 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/SKILL.md @@ -1,6 +1,58 @@ --- name: serving-cache description: "Audit Lithos Metal local serving, prepared-pack identity, exact target/draft prefix checkpoints, disk integrity, session adoption and idle release when startup, cache reuse or request isolation fails." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "tags": [ + "lithos-metal", + "metal", + "apple-silicon", + "serving-cache" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/lithos-ai/lithos-metal", + "commit": "6653b7df5f2c4392ec7b038960f2796b1499923f", + "paths": [ + "monolith/serving/cache.py", + "monolith/serving/setup.py", + "monolith/runtime/prefix_cache.py", + "monolith/runtime/prefix_store.py", + "monolith/serving/lifecycle.py", + "monolith/serve.py", + "monolith/generate.py" + ], + "readset": "../../references/source-notes.md" + } + ], + "applicability": { + "scope": "Pinned Lithos Metal 0.1.2 source/API contracts; Apple silicon Metal runtime; model and recipe qualification required separately." + }, + "limitations": [ + "Source-derived candidate; empirical efficacy validation not_run.", + "CPU/software and synthetic Metal contracts do not qualify full-model/DSpark numerics, quality or E2E performance.", + "No model downloads, live DisCo deployment or inference backend integration is provided." + ], + "provenance_readsets": [ + "../../references/source-notes.md" + ], + "consumption_mode": "packaged", + "license": "Apache-2.0", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + } + } --- # Validate serving and cache ownership diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/lithos-metal/sub-skills/serving-cache/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lithos-metal/sub-skills/serving-cache/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md similarity index 68% rename from src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md index 7878ce557..e3fe6e3c4 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md @@ -1,6 +1,77 @@ --- name: audit-llama-runtime-state description: "Audit llama.cpp context memory, partial decode progress, graph reuse and server cancellation when a retry, stale prefix, unexpected rebuild or unreleased slot needs an ownership explanation; requires pinned source or a concrete runtime trace." +metadata: + inferrsi: | + { + "tags": [ + "llama.cpp", + "context", + "state", + "graph-reuse", + "cancellation", + "correctness" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/ggml-org/llama.cpp", + "commit": "707616feb5f379e99ff7442680a55b4a1704c7c9", + "paths": [ + "LICENSE", + "include/llama.h", + "src/llama-context.cpp", + "src/llama-context.h", + "src/llama-memory.h", + "src/llama-memory-recurrent.cpp", + "src/llama-graph.cpp", + "src/llama-graph.h", + "ggml/src/ggml-backend.cpp", + "tools/server/server-context.cpp", + "tools/server/server-queue.cpp", + "tools/server/server-queue.h", + "tests/llama-context/cases-kv.cpp", + "tests/llama-context/cases-rs.cpp", + "tests/CMakeLists.txt", + "tests/test-llama-archs.cpp", + "tests/test-backend-ops.cpp", + ".github/workflows/build-cann.yml", + "tools/server/tests/unit/test_completion.py" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Pinned llama.cpp context-memory, graph-reuse and server task-lifecycle diagnosis; concrete model/backend behavior requires native validation.", + "prerequisites": "Pinned llama.cpp source or a concrete runtime trace, access to the relevant memory/graph/server ownership evidence and an identified caller recovery policy that remains in force.", + "input_contract": "Source/build and model revisions, memory type, actual backend/device and execution mode, context/batch/ubatch parameters, sequence IDs/positions/output flags and return status; graph cases need dispatch evidence, server cases need correlated HTTP/task/slot/cache logs.", + "produced_artifact": "An owner/event ledger of ubatch progress, retained memory, graph-gate decisions, input readiness and slot release/retention, with the first supported broken obligation or smallest missing observation.", + "failure_modes": "Assuming whole-batch rollback after partial decode failure, treating token count as a complete graph-reuse key, or equating posted cancellation with released slots can hide retained state or misdiagnose correct deferred work.", + "refusal_point": "Stop before reset or repair if revision, memory type, actual backend, sequence ownership or partial progress is unknown; escalate unsupported rollback or unresolved in-flight work." + }, + "limitations": [ + "Source-derived candidate; native runtime and efficacy validation not run.", + "No cross-backend preemption, model-quality or performance claim.", + "KV, recurrent and hybrid state contracts must be checked independently." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Reconcile per-sequence retained positions with completed ubatches before retrying a failed logical batch.", + "Check both graph allow_reuse and input can_reuse gates, then establish copy readiness independently of graph identity.", + "Trace cancellation posting, normal task processing, slot release and prompt retention as separate events." + ] + } --- # Audit llama.cpp runtime state @@ -21,8 +92,8 @@ Select the owner by the state or configuration you need to explain. Loading and | Question | Existing owner and readset | Fixed source snapshot | Carried static inspection | | --- | --- | --- | --- | -| Loading modes, mapping/locking and lazy tensors | [weight-loading-optimization §6](../../../methods/weight-loading-optimization/SKILL.md); [loading readset](../../../methods/weight-loading-optimization/references/llama-loading-readset.md) | `3e390af18de1131dda27913c00511725b4274c8a` | 2026-10-22 | -| NUMA modes and generation/batch thread placement | [os-process-placement](../../../hardware/host/os-process-placement/SKILL.md); [NUMA/thread readset](../../../hardware/host/os-process-placement/references/llama-numa-thread-readset.md) | `3e390af18de1131dda27913c00511725b4274c8a` | 2026-10-22 | +| Loading modes, mapping/locking and lazy tensors | [weight-loading-optimization §6](../../../../playbooks/optimization/weight-loading-optimization/SKILL.md); [loading readset](../../../../playbooks/optimization/weight-loading-optimization/references/llama-loading-readset.md) | `3e390af18de1131dda27913c00511725b4274c8a` | 2026-10-22 | +| NUMA modes and generation/batch thread placement | [os-process-placement](../../../platforms/host/os-process-placement/SKILL.md); [NUMA/thread readset](../../../platforms/host/os-process-placement/references/llama-numa-thread-readset.md) | `3e390af18de1131dda27913c00511725b4274c8a` | 2026-10-22 | | Decode progress, graph/input reuse and task/slot cancellation | [This runtime audit](#diagnostic-procedure); [runtime source notes](references/source-notes.md) | `707616feb5f379e99ff7442680a55b4a1704c7c9` | 2026-10-21 | The dates are carried source inspections, not native results. Apply each owner's prerequisites and readset at its own pinned revision. These snapshots do not establish an intervening version range or cross-backend compatibility. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/llama-cpp/audit-llama-runtime-state/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/lmcache/kv-tier-management/SKILL.md similarity index 80% rename from src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/lmcache/kv-tier-management/SKILL.md index dd4638994..8e3a244e3 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/lmcache/kv-tier-management/SKILL.md @@ -3,6 +3,45 @@ name: kv-tier-management description: 'Manage runtime KV cache tiers (GPU → CPU RAM → local disk) for LLM serving: decide when KV is stored, retained, prefetched, or evicted; size the concurrency-versus-RAM-pressure trade-off with an explicit in-flight admission dial; order RAM-release levers; and measure with token-level L1/L2 hit rates and prefetch-pressure counters. Use when configuring KV offload for a serving stack. Covers runtime KV offload only — model startup/loading, checkpoint save/restore, and block-device profiling are separate concerns (see weight-loading-optimization). ' +metadata: + inferrsi: | + { + "tags": [ + "kv-cache", + "offload", + "l1-l2-tiering", + "eviction", + "prefetch", + "serving", + "lmcache" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "LMCache multiprocess runtime KV tier policy across engine GPU cache, host L1 and configured L2 adapters, covering store/retention, prefetch admission, eviction and RAM pressure; startup loading and checkpoint recovery have separate owners.", + "prerequisites": "The deployment uses the inspected MP-mode semantics rather than single-process knobs; selected GDS/DAX/raw-block paths need their own platform and adapter prerequisites. Lazy offload additionally requires vLLM prefix caching.", + "input_contract": "MP versus single-process mode, store/retention policy and rationale, L1 manager capacities and eviction settings, L2 adapter capacities/eviction, chunk-size distribution and prefetch in-flight setting, together with token-level hit and pressure counters.", + "produced_artifact": "A mode- and adapter-specific tier-policy choice with retention rationale, an ordered RAM-release plan and a verification record separating L1/L2 token hits, prefetch pressure, store lag and per-manager usage.", + "failure_modes": "Mixing MP and in-process controls, treating key-count eviction as a byte fraction for unequal chunks, omitting fs capacity management, or increasing count-based prefetch admission without tracking L1 pressure can make the policy or memory claim unusable." + }, + "recommendations": [ + "Choose store retention and prefetched-key retention separately; use skip_l1 with noop only for buffer-only operation and account for failed stores remaining in L1.", + "Tune the count-based prefetch in-flight limit against L1 allocation failures and in-flight bytes; eager or warm prefetch can raise pressure before useful reuse.", + "Record per-adapter capacity/eviction and actual chunk-size uniformity; report token-level L1/L2 hits and sum per-l1_tag bytes rather than ratios." + ] + } --- # KV Cache Tier Management diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/SKILL.md new file mode 100644 index 000000000..fa77f6aef --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/SKILL.md @@ -0,0 +1,211 @@ +--- +name: local-dream +description: "Audit Local Dream Android desired-versus-serving process state, remote exposure, native backend and DiT packaging/ABI selection, shared generation cancellation and prompt-cache identity without mistaking declarations or source tests for on-device diffusion qualification." +license: NOASSERTION +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "tags": [ + "local-dream", + "android", + "snapdragon", + "qnn", + "hexagon", + "mnn", + "diffusion", + "lifecycle", + "cancellation", + "prompt-cache", + "abi", + "lan-exposure" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/xororz/local-dream", + "commit": "db162df4d9bb6efc6595250423d8df26241f4c27", + "license": "CC-BY-NC-4.0; repository API SPDX NOASSERTION retained separately", + "readset": "references/source-identity.json", + "paths": [ + ".gitmodules", + "LICENSE", + "README.md", + "app/build.gradle.kts", + "app/src/main/cpp/CMakeLists.txt", + "app/src/main/cpp/CMakePresets.json", + "app/src/main/cpp/build.sh", + "app/src/main/cpp/dit/CMakeLists.txt", + "app/src/main/cpp/dit/DitEngine.cpp", + "app/src/main/cpp/dit/build.sh", + "app/src/main/cpp/include/DitEngine.h", + "app/src/main/cpp/src/Config.hpp", + "app/src/main/cpp/src/Pipeline.hpp", + "app/src/main/cpp/src/PipelineAnima.hpp", + "app/src/main/cpp/src/PipelineDit.hpp", + "app/src/main/cpp/src/PipelineSd15Cpu.hpp", + "app/src/main/cpp/src/PipelineSd15Npu.hpp", + "app/src/main/cpp/src/PipelineSdxl.hpp", + "app/src/main/cpp/src/PipelineSdxlMnn.hpp", + "app/src/main/cpp/src/PromptCache.hpp", + "app/src/main/cpp/src/QnnRuntime.hpp", + "app/src/main/cpp/src/main.cpp", + "app/src/main/java/io/github/xororz/localdream/data/DitEngine.kt", + "app/src/main/java/io/github/xororz/localdream/data/DitResolution.kt", + "app/src/main/java/io/github/xororz/localdream/data/Model.kt", + "app/src/main/java/io/github/xororz/localdream/remote/RemoteApiClient.kt", + "app/src/main/java/io/github/xororz/localdream/remote/RemoteHostServer.kt", + "app/src/main/java/io/github/xororz/localdream/remote/RemoteProtocol.kt", + "app/src/main/java/io/github/xororz/localdream/service/BackendService.kt", + "app/src/main/java/io/github/xororz/localdream/service/BackgroundGenerationService.kt", + "app/src/main/java/io/github/xororz/localdream/service/RemoteHostService.kt", + "app/src/main/java/io/github/xororz/localdream/ui/screens/ModelRunScreen.kt", + "app/src/main/java/io/github/xororz/localdream/ui/screens/ModelRunSupport.kt", + "app/src/test/java/io/github/xororz/localdream/ExampleUnitTest.kt", + "app/src/test/java/io/github/xororz/localdream/ModelStorageTest.kt" + ] + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "readset": "references/source-identity.json", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill/SKILL.md", + "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/repository-evidence.md", + "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/input-output-and-structure.md", + "cli/packages/coding-agent/src/disco/skills/create-repo-skill/references/planning-and-writing.md", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env/SKILL.md", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill/SKILL.md", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill/references/usability-test-cases.md" + ] + } + ], + "applicability": { + "scope": "Selected Android app/process and native diffusion boundary contracts; common is a discovery category, not CPU/device compatibility or native qualification.", + "prerequisites": "Access to the pinned Android lifecycle/native/packaging contracts and identified installed artifacts; stronger conclusions require an authorized, frozen workload on the actual selected Android/ARM64 backend.", + "input_contract": "Source revision, installed APK/version/flavor/ABI, native executable and loaded DiT library, QNN/Hexagon/MNN builds, model/export/weights/tokenizer identities, requested and serving configuration, and lifecycle/readiness/generation/cache observations.", + "produced_artifact": "A bounded failed-boundary report with counter-signals, identities observed versus declared, the smallest missing discriminator and retained INCONCLUSIVE native scope.", + "failure_modes": "Reading UI flags or equal declared configurations as actual dispatch/readiness, treating cancellation/SSE progress as native completion, or interpreting model-directory caches as authenticated model-revision namespaces can misidentify stale state." + }, + "limitations": [ + "Source-derived candidate; required Android ARM64/MNN/QNN/Hexagon/model execution BLOCKED_REQUIRED_BACKEND / not_run; qualification INCONCLUSIVE.", + "Portable AREX-informed source analysis and InferRSI candidate integration, not DisCo CLI execution, managed import or full AREX verification.", + "Original upstream Gradle/JVM/native tests, APK build/sign/install, loader/device/model/media-quality/memory-fit/thermal/service/performance and agent-efficacy checks not_run.", + "Offline helper compares supplied exact configuration and ABI/SDK labels only; it does not authenticate process state, readiness, artifact bytes, cache correctness or runtime behavior.", + "LAN control is unauthenticated in the inspected implementation despite stale comments; audit guidance does not authorize enabling host mode or exposing ports.", + "NOASSERTION is machine metadata, not license absence or commercial permission. Pinned LICENSE is CC BY-NC 4.0; retain restriction and attribution; SDK/model/dependency rights separate.", + "Full UI/navigation/import/download/storage/history/scheduler/kernel/security review and external dependency bodies excluded." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Compare requested and serving model/backend/dimensions/bind address through stop/reopen and health observations; never enable host mode as a diagnostic default.", + "Trace the actual MNN/QNN/Anima/DiT path and separate native library/ABI identity from UI runOnCpu flags and declared SDK labels.", + "Keep progress, cancellation, native completion, release and saved final output separate; check prompt-cache mode/version/dimensions and textual-inversion caller guards before proposing cache changes." + ] + } +--- + +# Local Dream Android diffusion boundary audit + +Use this source-derived **candidate** when a model page reports the wrong +backend, a quick reopen races a stop, host mode changes reachability, DiT fails +to load, generation appears stuck, or prompt-cache reuse is suspect. Start with +[source identity and rights](references/repo-provenance.md). This is original +bounded analysis of `xororz/local-dream`, not an Android installer, device +compatibility certification or a managed DisCo import. + +## Route to the failing boundary + +| Symptom / question | Procedure | +| --- | --- | +| Same model but wrong resolution/backend/bind address; stop/reopen/restart; stale Error | [Android desired/serving lifecycle](references/android-lifecycle.md) | +| Remote control says Ready, or comments imply pairing/authentication | [Actual host exposure and readiness](references/android-lifecycle.md); never enable host mode as a diagnostic default | +| CPU/MNN versus QNN versus Anima versus dynamically loaded DiT; SSE disconnect or upscaler contention | [Native dispatch, cancellation and cache](references/native-contracts.md) | +| Missing engine/entry point/ABI/skel, QNN runtime mixing, APK versus DSP packaging | [Packaging and native prerequisites](references/packaging-and-abi.md) | +| Same prompt gives suspicious reuse after export/model/textual-inversion changes | [Cache identity and caller guards](references/native-contracts.md); shape/header acceptance is not model-revision invalidation | +| Only declarations are available | [Offline declaration screen](scripts/screen_declarations.py), whose explicit non-native limits are below | + +## Establish identities before suggesting a change + +1. Record the source revision, installed APK/version/flavor/ABI, actual native + executable and separately loaded DiT library, QNN/Hexagon/MNN build identities, + selected model/export/weight/tokenizer identities, requested configuration and + **serving** configuration. Use anonymized IDs and `` in shared + reports. A source version string or file existence is not installed identity. +2. Follow actual backend dispatch, not the README headline. SD1.5/SDXL MNN and + QNN paths, Anima and `zimage`/`klein`/`qwen21`/`krea2turbo` DiT routing are not + interchangeable. A UI `runOnCpu` flag does not alone specify the executed + MNN CPU/GPU provider or the DiT engine's device/parameter residency. +3. Compare the full process configuration: model, backend type, width, height + and `listenOnAll`. A process still alive in the stop grace window is not + evidence that it serves the newly requested configuration. Follow both + process state and health/config observations. `Running` is published after + process start, before proven native readiness; the local health consumer + checks model identity but not every serving field. Ready is not image + completion. Equality also omits startup preferences and changed model bytes; + an equal tuple is not proof that such changes were applied. +4. Separate startup, model load, prompt encoding, denoising, VAE work, preview, + safety filtering, final response, history persistence and teardown. SSE + progress, a client cancellation request, native completion, buffer release + and saved final output are different events. Native/shared resource locks + can serialize generation and upscaling; do not infer parallel NPU capacity. +5. Treat persistent prompt caches as model-directory-scoped implementation + caches, not authenticated model-revision namespaces. Check mode, version, + dimensions and callers' textual-inversion bypass. Preserve failure signals + rather than deleting caches or model directories without authorization. + +## Safe offline use + +The bundled standard-library helper reads a small explicit declaration object +from stdin or one regular JSON file; it makes no network, subprocess, Android, +model, tensor, native-library, filesystem-discovery or device calls. It compares +the five declared configuration fields and separately declared ABI/SDK labels. +It does **not** reproduce coroutine timing, resolution snapping, Android service +rules, dynamic loading, symbol-table validation, runtime stamping/copy behavior, +prompt-cache formats or native execution. + +```bash +python /scripts/screen_declarations.py --example +python /scripts/screen_declarations.py +``` + +An equal configuration can still be dead, unready or serving stale bytes. Equal +ABI/SDK labels can still describe incompatible or corrupted artifacts. The +helper returns field names and fixed diagnostics, not supplied identifiers; it +always retains `native_execution: not_run` and `qualification: INCONCLUSIVE`. +Its zero exit means valid input and completed screening, not native success. + +## Evidence required for a stronger conclusion + +Required Android/ARM64 and actual selected MNN/QNN/Hexagon/model execution remains +`BLOCKED_REQUIRED_BACKEND / not_run`; there is no full CPU substitute for it. +Source assertions, Gradle configuration, offline helper tests and InferRSI wheel +access are separate evidence classes. No upstream test, APK build/sign/install, +native loader, live HTTP endpoint, model/image quality, memory fit, thermal, +latency, throughput, service or broad skill-efficacy result is claimed here. + +For an authorized future run, freeze a RunSpec and workload first. Require +actual backend identity, model/tokenizer/export hashes, load and readiness +records, final image plus quality validator, cancellation and teardown outcomes, +memory/thermal controls and complete timing boundaries appropriate to the claim. +Source hardware gates and implementation constants are not portable acceptance +thresholds. Return the failed boundary, counter-signal, identities observed vs +declared, smallest missing discriminator and retained `INCONCLUSIVE` scope. + +Upstream's pinned LICENSE is **CC BY-NC 4.0**, while repository API SPDX metadata +reports `NOASSERTION`. The latter is preserved, not permission for commercial +reuse or relicensing. See [rights and exclusions](references/repo-provenance.md). diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/android-lifecycle.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/android-lifecycle.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/android-lifecycle.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/android-lifecycle.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/native-contracts.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/native-contracts.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/native-contracts.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/native-contracts.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/packaging-and-abi.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/packaging-and-abi.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/packaging-and-abi.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/packaging-and-abi.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/upstream-license.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/upstream-license.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/local-dream/references/upstream-license.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/references/upstream-license.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/local-dream/scripts/screen_declarations.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/scripts/screen_declarations.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/local-dream/scripts/screen_declarations.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream/scripts/screen_declarations.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/SKILL.md new file mode 100644 index 000000000..c85fe27b9 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/SKILL.md @@ -0,0 +1,186 @@ +--- +name: mistral-rs +description: "Investigate mistral.rs Metal paged-attention decode layouts, prefill coordinates and workspace, actual-model prewarm and native test evidence without treating static metadata or prepared pipelines as executed GPU correctness or performance." +license: NO_LICENSE +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NO_LICENSE", + "tags": [ + "mistral-rs", + "metal", + "candle", + "paged-attention", + "cache-layout", + "prefill-metadata", + "workspace", + "prewarm", + "native-evidence" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/EricLBuehler/mistral.rs", + "commit": "9459891e7697238225885518119682ac8209a612", + "version": "0.9.5", + "license": "MIT root; identified paged attention portions Apache-2.0 with NOTICE", + "readset": "references/source-identity.json", + "paths": [ + ".github/workflows/ci.yml", + "AGENTS.md", + "Cargo.lock", + "Cargo.toml", + "LICENSE", + "README.md", + "docs/src/content/docs/guides/perf/paged-attention.mdx", + "examples/python/paged_attention.py", + "mistralrs-core/src/paged_attention/cache_engine.rs", + "mistralrs-core/src/paged_attention/metal_hnd_tests.rs", + "mistralrs-core/src/paged_attention/mod.rs", + "mistralrs-core/src/pipeline/metal_warmup.rs", + "mistralrs-paged-attn/Cargo.toml", + "mistralrs-paged-attn/LICENSE-APACHE", + "mistralrs-paged-attn/NOTICE", + "mistralrs-paged-attn/build.rs", + "mistralrs-paged-attn/src/metal/backend/mod.rs", + "mistralrs-paged-attn/src/metal/backend/paged_attention.rs", + "mistralrs-paged-attn/src/metal/backend/paged_prefill.rs", + "mistralrs-paged-attn/src/metal/kernels/mod.rs", + "mistralrs-paged-attn/src/metal/kernels/paged_decode_hnd.metal", + "mistralrs-paged-attn/src/metal/kernels/paged_prefill.metal", + "mistralrs-paged-attn/src/metal/mod.rs", + "mistralrs-paged-attn/tests/metal_prefill.rs" + ] + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/workflow-authoring", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" + ], + "scope": "Portable compatible-agent native Rust/source adaptation; no DisCo CLI/model execution or managed import." + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "repository-operating-graph", + "strategy": "arex-portable-meta-skills", + "status": "partial-candidate", + "native_import": "not-run", + "method": "Stable-ID bounded decode/prefill/qualification author lanes, whole-graph reconciliation, hard cases and offline helper/static/scaffold/wheel checks; compiler and actual Apple evidence not_run." + }, + "applicability": { + "scope": "Selected native Metal paged decode/prefill layout/metadata/workspace and preparation/evidence investigation. Common catalog categorization is not CPU inference or hardware certification.", + "prerequisites": "Access to the pinned Rust/Candle/MSL decode, prefill and qualification contracts; native comparisons need the matched Rust/Apple/Candle/shader stack and separately authorized frozen references.", + "input_contract": "mistral.rs/Candle/native artifact revisions, Rust toolchain, Apple SDK/OS/device, actual model/cache dtype and representation, shape/strides/offsets, block table, context and execution limits.", + "produced_artifact": "A source/build-bound route and invariant report with actual versus declared facts, safe support result, counter-signal and smallest next native check; unavailable Metal/kernel/model evidence remains blocked.", + "failure_modes": "Equating element counts with physical alias/layout identity, transferring decode geometry to prefill, or treating prewarm/pipeline preparation as actual dispatch can misclassify eligibility and native correctness." + }, + "limitations": [ + "BLOCKED_REQUIRED_BACKEND: actual Rust/Candle/Apple Metal compilation, native tensor/cache/dispatch/parity/model/quality/service/performance evidence not_run.", + "Cargo/rustc and compatible Apple SDK/device unavailable; no Rust package/toolchain or native dependency installation, live Rust API inspection or source test execution claimed.", + "Offline helpers screen caller declarations, source byte formulas or boundary enumeration only; no tensor authentication, GPU simulation, native eligibility, scratch-capacity or physical-fit proof.", + "Generic model/loading/quantization/server/agentic/distributed/CUDA paths and unmined specialized kernels remain explicit long-tail gaps; broad skill efficacy unmeasured.", + "NO_LICENSE is the actual unauthenticated gh resolver fallback, not legal absence; root MIT and identified component Apache-2.0/NOTICE retained independently." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Select decode, prefill or qualification by the failed contract, then follow cache/metadata bytes from producer through view to native consumer.", + "Keep static predicates, pipeline preparation, kernel dispatch, numerical correctness and model/service outcomes as separate evidence steps; retain skipped or gated tests.", + "Use the matched Candle revision and native shader/build inputs; broad workspace tests can require model fixtures, tokens and downloads." + ] + } +--- + +# mistral.rs Metal paged-attention contracts + +Use this source-derived **candidate** for the Rust/Candle engine's native Metal +paged-attention path. It is a focused repository operating graph, not a generic +model catalog, a deployment guide or a verified engine. Read +[source and method provenance](references/repo-provenance.md) before using it on +a different checkout. Reading it does not authorize installation or execution. + +## Route by the failed contract + +| Question | Canonical owner | +| --- | --- | +| Does HND/packed K/V, FP8 scale, GQA/head geometry, sliding decode or scratch agree? | [decode-contract](sub-skills/decode-contract/SKILL.md) | +| Do packed/padded queries, absolute positions, masks/ranges and prefill workspace agree? | [prefill-contract](sub-skills/prefill-contract/SKILL.md) | +| Did model warmup prepare the intended pipelines, and what did native tests actually prove? | [qualification](sub-skills/qualification/SKILL.md) | +| Which other engine features, dependencies and licenses are outside this extraction? | [scope and notices](references/scope-and-notices.md) | + +The selected source owns Rust shape/configuration/consumer code and actual MSL +attention, rather than merely naming an external backend. Candle supplies the +tensor/device integration. Shared ancestry with other Metal engines does not +make their cache layouts, scratch planners or synchronization interchangeable. + +## Establish identity before proposing a change + +1. Record exact mistral.rs/Candle/native artifact revisions, Rust toolchain, + Apple SDK/OS/device, actual model/cache dtype and representation, shape, + strides/offsets, block table, context and execution limits. Model and asset + identities are separate from source version or advertised support. +2. Identify the producer, view and native consumer of cache/metadata bytes. + Equal element counts, a layout setting and physical alias identity differ. + Decode and prefill deliberately have different supported geometry domains. +3. Select the smallest matching sub-skill and its bundled helper. Helpers use + explicit synthetic/caller-declared JSON and Python standard library only. + They do not import an engine, compile Rust, read weights, allocate device + memory, emulate GPU computation or establish native eligibility. +4. Separate a static source predicate, pipeline preparation, actual kernel + dispatch, numerical correctness, model quality and service performance. + None automatically implies the next. Preserve unsupported cases, failed + observations, skipped/gated tests and required missing evidence. +5. For an actual change or experiment, obtain its separate authority, freeze + controls/acceptance gates and select discriminating native references before + running it. Do not use source crossover constants or test tolerances as + portable performance/quality acceptance bars. + +## Preparation and use boundary + +This snapshot's workspace is version 0.9.5, edition 2021 and requires Rust 1.94. +The Metal feature requires a compatible Apple native stack plus matched Candle +revision and shader/build inputs. The source repository's broad workspace tests +can use model fixtures, tokens and downloads; they are not safe default checks. + +This graph does not ship an installer. Use only an independently reviewed, +versioned upstream release/build procedure under separate installation +authorization. Do not execute remote shell installers or install Rust/SDKs just +to inspect this graph. If an authorized CLI is already installed, a bounded +identity/help check is: + +```bash +mistralrs --version +mistralrs --help +``` + +Those public CLI invocations were **not run** here. Current inspection has no +Cargo/rustc or compatible Apple backend; no Rust compilation or live Rust API +inspection is claimed. The source/manual adaptation is explicitly partial. +Bundled helpers remain usable without the source checkout or inference package. + +## Return a bounded result + +Report source/build identities, actual versus declared facts, route, invariant, +safe support result, counter-signal and smallest next authorized native check. +Required Metal/kernel/model validation remains `BLOCKED_REQUIRED_BACKEND` until +separately executed; compiler checks are `not_run`. Candidate state is not +promoted by helper tests, packaging, independent content review or a merged PR. +Memory fit, quality, service performance and broad skill efficacy are unverified. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/apache-component-license.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/apache-component-license.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/apache-component-license.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/apache-component-license.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/paged-component-notice.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/paged-component-notice.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/paged-component-notice.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/paged-component-notice.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/scope-and-notices.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/scope-and-notices.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/scope-and-notices.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/scope-and-notices.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/helper-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/references/helper-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/helper-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/references/helper-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/layout-and-features.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/references/layout-and-features.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/layout-and-features.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/references/layout-and-features.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/workspace-and-evidence.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/references/workspace-and-evidence.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/references/workspace-and-evidence.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/references/workspace-and-evidence.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/scripts/audit_decode_metadata.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/scripts/audit_decode_metadata.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/decode-contract/scripts/audit_decode_metadata.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/decode-contract/scripts/audit_decode_metadata.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/helper-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/references/helper-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/helper-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/references/helper-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/prefill-and-metadata.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/references/prefill-and-metadata.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/prefill-and-metadata.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/references/prefill-and-metadata.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/scripts/screen_prefill_metadata.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/scripts/screen_prefill_metadata.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/prefill-contract/scripts/screen_prefill_metadata.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/prefill-contract/scripts/screen_prefill_metadata.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/prewarm-and-evidence.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/references/prewarm-and-evidence.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/prewarm-and-evidence.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/references/prewarm-and-evidence.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/warmup-inventory.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/references/warmup-inventory.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/references/warmup-inventory.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/references/warmup-inventory.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/scripts/warmup_shape_inventory.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/scripts/warmup_shape_inventory.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs/sub-skills/qualification/scripts/warmup_shape_inventory.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs/sub-skills/qualification/scripts/warmup_shape_inventory.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/SKILL.md new file mode 100644 index 000000000..542b00932 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/SKILL.md @@ -0,0 +1,177 @@ +--- +name: mlc-llm +description: "Investigate MLC LLM compiled Metal artifact/schema, Apple target and pinned TVM module/storage/stream contracts, and compile/runtime KV limits without confusing canonical metadata, codegen or automatic estimates with runnable native model qualification." +license: NO_LICENSE +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NO_LICENSE", + "tags": [ + "mlc-llm", + "tvm", + "metal", + "artifact-contract", + "target", + "sdk", + "module", + "stream", + "kv-cache", + "compiled-limits" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/mlc-ai/mlc-llm", + "commit": "8978ea9aec626d64096aaa92fbc89bda2e3ea802", + "license": "Apache-2.0 with retained NOTICE and source-specific component notices", + "readset": "references/source-identity.json", + "paths": [ + ".github/workflows/lint.yml", + ".gitmodules", + "CONTRIBUTORS.md", + "LICENSE", + "NOTICE", + "README.md", + "cpp/serve/config.cc", + "cpp/serve/config.h", + "docs/community/guideline.rst", + "docs/compilation/model_artifact_manifest.rst", + "pyproject.toml", + "python/mlc_llm/__init__.py", + "python/mlc_llm/base.py", + "python/mlc_llm/compiler_pass/dispatch_kv_cache_creation.py", + "python/mlc_llm/interface/compile.py", + "python/mlc_llm/interface/convert_weight.py", + "python/mlc_llm/interface/gen_config.py", + "python/mlc_llm/model/gemma4/gemma4_model.py", + "python/mlc_llm/model/model.py", + "python/mlc_llm/protocol/__init__.py", + "python/mlc_llm/protocol/artifact_manifest.py", + "python/mlc_llm/serve/__init__.py", + "python/mlc_llm/serve/engine.py", + "python/mlc_llm/support/auto_target.py", + "tests/python/conftest.py", + "tests/python/integration/test_model_compile.py", + "tests/python/model/test_kv_cache.py", + "tests/python/protocol/test_artifact_manifest.py", + "tests/python/support/test_auto_target.py" + ] + }, + { + "type": "git", + "url": "https://github.com/mlc-ai/relax", + "commit": "201fa0001c81431ccce05419b442c47779ea917c", + "license": "Apache-2.0 with retained NOTICE and source-specific component notices", + "readset": "references/source-identity.json", + "paths": [ + "AGENTS.md", + "LICENSE", + "NOTICE", + "src/backend/metal/codegen/codegen_metal.cc", + "src/backend/metal/codegen/metal_fallback_module.cc", + "src/backend/metal/codegen/metal_fallback_module.h", + "src/backend/metal/runtime/metal_common.h", + "src/backend/metal/runtime/metal_device_api.mm", + "src/backend/metal/runtime/metal_module.mm", + "tests/python/codegen/test_target_codegen_metal.py" + ] + } + ], + "applicability": { + "scope": "Selected artifact/Apple target/pinned TVM Metal consumer and KV configuration source contracts. Common catalog categorization is not CPU inference or Apple qualification.", + "prerequisites": "Access to the pinned MLC and exact TVM fork/gitlink contracts; native checks require separately prepared matching MLC/TVM/FFI and the applicable Apple SDK/device stack.", + "input_contract": "MLC revision, TVM gitlink/build, model/tokenizer/quantized-parameter identity, target/host/SDK, compiled artifact/format, runtime device and the actual compiled/runtime configuration sources.", + "produced_artifact": "An identity and actual-versus-declared boundary report for artifact, Metal runtime or KV-limit ownership, with a negative signal and smallest missing native discriminator.", + "failure_modes": "Treating a schema hash as payload authentication, an opt-in WebLLM artifact contract as a native-engine ABI, serializable Metal fallback as runnable, or automatic capacity estimates as free memory can create unsupported qualification claims." + }, + "limitations": [ + "Source-derived candidate; original compiled CPU/native tests, compiler/live API/CLI/model/Metal/fit/quality/service/performance execution not_run; required native BLOCKED_REQUIRED_BACKEND.", + "Opt-in artifact manifest first targets WebLLM; native engine ignoring that ABI is not qualified by canonical metadata agreement.", + "Serializable kind=metal fallback can be non-runnable; codegen/export/pipeline/submission and actual load/dispatch/completion differ.", + "Helpers screen narrow explicit canonical metadata, literal target strings or limit clamping only; no general Pydantic/TVM emulation, actual tensor/weight/export authentication or GPU/memory simulation.", + "Full model/tokenizer/quantization/VLM/RNN/speculation/disaggregation/CUDA/scheduler/dependency implementation and broad skill efficacy remain outside selected contracts.", + "NO_LICENSE is unavailable machine SPDX metadata, not legal absence; both source owners Apache LICENSE/NOTICE bytes are independently retained." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Identify the interface's actual consumer before using a valid artifact manifest as evidence of native model/task support.", + "Separate codegen/export from runtime factory/load, pipeline creation, dispatch and stream completion; a serializable kind=metal fallback can still be non-runnable.", + "Trace runtime preference, compile-time bounds, nested workspace inputs and explicit overrides separately before interpreting a KV or working-set estimate." + ] + } +--- + +# MLC LLM compiled Metal and runtime contracts + +Use this source-derived **candidate** for the boundary between model-package +metadata, compiled target artifacts and real native runtime behavior. Begin +with [source identity](references/repo-provenance.md). MLC and its exact TVM +fork/gitlink are separate source owners; an arbitrary newer TVM is not an +interchangeable implementation. + +## Route by the failed contract + +| Question | Canonical owner | +| --- | --- | +| Do normalized tasks, parameter schema, executor/adapter roles and parameter-only resources agree? | [artifact-contract](sub-skills/artifact-contract/SKILL.md) | +| Do target/host/SDK/suffix/prefix, payload format, runnable module and selected storage/stream agree? | [metal-build-runtime](sub-skills/metal-build-runtime/SKILL.md) | +| Which compiled/runtime limit or workspace input is actually controlling the KV path? | [runtime-kv-limits](sub-skills/runtime-kv-limits/SKILL.md) | +| What remains outside these source contracts, including rights and native evidence? | [scope and notices](references/scope-and-notices.md) | + +## Establish the actual boundary + +1. Record MLC revision, TVM gitlink/build, model/tokenizer/quantized parameter + identity, target/host/SDK, compiled artifact/format and runtime device. A + canonical interface or parameter-schema hash does not hash tensor bytes, + authenticate exports, select a device or prove native consumer support. +2. Identify which consumer implements the interface. The inspected artifact + manifest is **opt-in**, first consumed by WebLLM. The native engine does not + consume that contract at this pin; a valid sidecar cannot imply native + support for its model/task ABI. +3. Separate metadata checks, IR/codegen, serialization/export, real runtime + factory/load, pipeline creation, dispatch, stream completion, numerical + behavior and model/service results. Each requires distinct evidence. A + serializable kind=metal fallback is explicitly non-runnable. +4. Follow actual compiled metadata and configuration sources. Runtime preference, + compile-time bound, nested workspace shape and explicit engine override are + different values. The Metal working-set attribute and automatic-capacity + constants do not establish free memory, safe allocation or universal limits. +5. Use only the selected bundled standard-library helper on explicit synthetic + or anonymized declarations. Helpers do not import MLC/TVM, run Pydantic, + initialize submodules, inspect weights/binaries, invoke a compiler or probe a + device. Their narrow domains are not complete upstream input validators. + +## Native preparation and evidence limits + +The source pyproject declares 0.20.0.dev0; no installed package/build is implied. +MLC's native Python package loads TVM/serving integration, and original artifact +tests also import TVM DataType/model definitions and Pydantic. Even apparently +pure schema/codegen/IR tests require the actual prepared native stack. A +standalone Python declaration helper is not that stack or a passing source test. + +This graph ships no installer, model-loading recipe or checkout-dependent source +test command. Compiler/native tests require separately prepared, version-matched +MLC/TVM/FFI dependencies and Apple SDK/device where applicable; broad compile or +integration loops can exercise many models and are not small default checks. +No compiler/live API/CLI/original source test or actual Metal/model execution is +claimed. Required native evidence remains `BLOCKED_REQUIRED_BACKEND / not_run`. + +Return identities, actual versus declared facts, route/invariant, negative +signal and smallest missing native discriminator. Candidate empirical validation +is `not_run`. Helper, source hash, packaging or generic compile success does not +qualify cache correctness, physical fit, model quality, latency/service gains or +broad skill efficacy. No portable acceptance bar is derived from source test +constants or implementation crossover/capacity values. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/mlc-notice.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/mlc-notice.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/mlc-notice.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/mlc-notice.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/scope-and-notices.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/scope-and-notices.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/scope-and-notices.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/scope-and-notices.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/tvm-license.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/tvm-license.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/tvm-license.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/tvm-license.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/tvm-notice.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/tvm-notice.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/references/tvm-notice.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/references/tvm-notice.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/contract-mechanisms.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/references/contract-mechanisms.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/contract-mechanisms.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/references/contract-mechanisms.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/helper-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/references/helper-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/helper-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/references/helper-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/scripts/audit_artifact_declarations.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/scripts/audit_artifact_declarations.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/artifact-contract/scripts/audit_artifact_declarations.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/artifact-contract/scripts/audit_artifact_declarations.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/helper-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/helper-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/helper-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/helper-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/modules-and-streams.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/modules-and-streams.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/modules-and-streams.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/modules-and-streams.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/native-evidence.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/native-evidence.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/native-evidence.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/native-evidence.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/targets-and-export.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/targets-and-export.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/targets-and-export.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/targets-and-export.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/scripts/audit_apple_target_declarations.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/scripts/audit_apple_target_declarations.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/metal-build-runtime/scripts/audit_apple_target_declarations.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/metal-build-runtime/scripts/audit_apple_target_declarations.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/cache-lowering-and-evidence.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/references/cache-lowering-and-evidence.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/cache-lowering-and-evidence.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/references/cache-lowering-and-evidence.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/helper-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/references/helper-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/helper-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/references/helper-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/limits-and-workspace.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/references/limits-and-workspace.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/limits-and-workspace.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/references/limits-and-workspace.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/scripts/audit_kv_limit_declarations.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/scripts/audit_kv_limit_declarations.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm/sub-skills/runtime-kv-limits/scripts/audit_kv_limit_declarations.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm/sub-skills/runtime-kv-limits/scripts/audit_kv_limit_declarations.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md similarity index 53% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md index 33d81ec99..2af6f12d2 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md @@ -1,6 +1,66 @@ --- name: generation-cache-lifecycle description: Trace MLX-LM prefill, generation batching, prompt-cache reuse and cooperative stopping when request state, cache rows or retained prefixes become inconsistent. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "MIT", + "tags": [ + "mlx-lm", + "generation", + "batching", + "kv-cache", + "cancellation" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/ml-explore/mlx-lm", + "commit": "ab7f7048dc341ced2d84b1887eeb17d60b0d1b02", + "paths": [ + "mlx_lm/generate.py", + "mlx_lm/models/cache.py", + "mlx_lm/server.py", + "tests/test_generate.py", + "tests/test_prompt_cache.py", + "LICENSE", + "pyproject.toml", + "mlx_lm/cache_prompt.py", + ".github/workflows/pull_request.yml" + ], + "readset": "references/source-map.md" + } + ], + "applicability": { + "scope": "MLX-LM generation, batch-row and prefix-cache diagnosis; model/backend/runtime compatibility requires separate qualification.", + "prerequisites": "Access to the independently pinned MLX-LM generation/cache contracts and the selected entry path; model, backend and runtime compatibility is separately established for any native claim.", + "input_contract": "MLX-LM/MLX revisions and builds, backend/device, model/tokenizer/adapter/draft identities, entry point, tokenized prompts, cache classes, batching/prefill limits, per-request sampling/processors/stops, seed and KV quantization; failure traces include UIDs, stages, lengths, terminal reason and next-request result.", + "produced_artifact": "An entry-path decision, UID/stage/cache-row table, prefix identity and cache-class assumptions, terminal/cleanup trace and actual evidence status with a targeted verification plan.", + "failure_modes": "Ignoring server batching gates, filtering only some per-request row state, reusing a text-only prefix key or treating a stop flag as immediate MLX work cancellation can corrupt lifecycle reasoning.", + "refusal_point": "Stop at the first unsupported contract or missing required fixture/backend; retain a targeted plan without inferring paged allocation, radix sharing or instantaneous prefill interruption." + }, + "limitations": [ + "Source-derived candidate; no local native-backend, model, service or efficacy validation.", + "Static content checks do not promote this candidate or establish installed API conformance." + ], + "provenance_readsets": [ + "references/source-map.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Resolve the library versus server entry and its batching gates before transferring BatchGenerator behavior.", + "Keep UID, caches, tokens, samplers, processors, lengths, stop matchers and pending outputs aligned under the same removal selection, including prompt-stage requests.", + "Check exact token/model/adapter/draft identity and cache-type copy/trim semantics, then trace cooperative removal through survivor and next-request behavior." + ] + } --- # Trace MLX-LM generation and cache lifecycle diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/cache-stop.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/cache-stop.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/cache-stop.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/cache-stop.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/generation-batch.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/generation-batch.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/generation-batch.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/generation-batch.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/validation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-lm/generation-cache-lifecycle/references/validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/SKILL.md new file mode 100644 index 000000000..bfa2e2908 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/SKILL.md @@ -0,0 +1,197 @@ +--- +name: mlx-swift-lm +description: "Investigate MLX Swift LM generation ownership, cancellation and settled task completion, and recurrent GatedDelta Metal versus ops contracts without confusing source predicates, final info or offline metadata with executed native correctness." +license: NO_LICENSE +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NO_LICENSE", + "tags": [ + "mlx-swift-lm", + "swift", + "metal", + "generation", + "ownership", + "cancellation", + "task-join", + "recurrent-state", + "gated-delta", + "native-evidence" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/ml-explore/mlx-swift-lm", + "commit": "3d066d3a0ffdc7e70c476a3d31488ea62dfdd0ac", + "version": "main 3.x API context; Swift tools 6.2", + "license": "MIT; unselected vendored xgrammar separately Apache-2.0", + "readset": "references/source-identity.json", + "paths": [ + ".github/workflows/integration_tests.yml", + ".github/workflows/pull_request.yml", + "AGENTS.md", + "CONTRIBUTING.md", + "LICENSE", + "Libraries/MLXLLM/Models/Qwen35.swift", + "Libraries/MLXLLM/Models/Qwen3Next.swift", + "Libraries/MLXLMCommon/Documentation.docc/upgrade.md", + "Libraries/MLXLMCommon/Documentation.docc/using.md", + "Libraries/MLXLMCommon/Evaluate.swift", + "Libraries/MLXLMCommon/GatedDelta.swift", + "Libraries/MLXLMCommon/KVCache.swift", + "Libraries/MLXLMCommon/ModelContainer.swift", + "Libraries/MLXLMCommon/ModelFactory.swift", + "Libraries/MLXLMCommon/TokenLoopHandler.swift", + "Libraries/MLXLMCommon/Tokenizer.swift", + "Libraries/MLXLMCommon/UserInput.swift", + "Libraries/MLXLMCommon/Utilities/GenerationWorker.swift", + "Libraries/MLXLMCommon/Utilities/SerialAccessContainer.swift", + "Libraries/MLXVLM/Models/Qwen35.swift", + "Package.swift", + "README.md", + "Tests/MLXLMTests/CancellationTests.swift", + "Tests/MLXLMTests/GatedDeltaQKNormTests.swift", + "Tests/MLXLMTests/GatedDeltaTests.swift", + "Tests/MLXLMTests/GenerationExecutionTests.swift", + "Tests/MLXLMTests/Qwen35GDNDecodeBitwiseTests.swift", + "Tests/MLXLMTests/Qwen35MTPTests.swift", + "skills/mlx-swift-lm/SKILL.md", + "skills/mlx-swift-lm/references/concurrency.md", + "skills/mlx-swift-lm/references/generation.md", + "skills/mlx-swift-lm/references/kv-cache.md", + "skills/mlx-swift-lm/references/model-container.md", + "skills/mlx-swift-lm/references/model-porting.md" + ] + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/workflow-authoring", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" + ], + "scope": "Actual portable compatible-agent source/native-Swift adaptation; no DisCo CLI/model run or managed installation." + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "repository-operating-graph", + "strategy": "arex-portable-meta-skills", + "status": "partial-candidate", + "native_import": "not-run", + "method": "Stable-ID lifecycle and recurrent-kernel author lanes, integrated root/local references, hard cases and offline support/static/scaffold/package checks; native Swift/Xcode/MLX compiler/tests not_run." + }, + "applicability": { + "scope": "Selected Swift ownership/cancellation/settling and actual GatedDelta Metal/ops contracts. Common categorization is not CPU inference or Apple qualification.", + "prerequisites": "Access to the Swift lifecycle and selected GatedDelta contracts, with resolved MLX/native identities for a comparison; native checks require a prepared matching Swift/Xcode/Apple stack and separate authorization.", + "input_contract": "Exact source revision, resolved Swift/MLX dependencies, Xcode/SDK/OS/device, API family and native artifact identity; the selected question supplies lifecycle ownership/completion observations or recurrent producer shape/dtype/layout and actual consumer/fallback.", + "produced_artifact": "An identity and actual-versus-declared report for generation ownership or recurrent kernel invariants, with safe support result, counter-signal and smallest separately authorized native check.", + "failure_modes": "Conflating lexical isolation with exclusive access across await or completed task join, treating useKernel as observed dispatch, calling an ops fallback CPU execution, or assuming a manifest range is a resolved dependency can invalidate the conclusion." + }, + "limitations": [ + "BLOCKED_REQUIRED_BACKEND: native Swift/MLX compiler, host-control tests and required Apple Metal cancellation/backgrounding/kernel/cache/model/fit/quality/service/performance evidence not_run.", + "Swift/Xcode/Apple SDK/MLX native stack unavailable; no live Swift API, package resolution, original source/native tests or CLI execution claimed.", + "Helpers audit caller-declared evidence or selected metadata/byte/source-predicate contracts only; no tasks/recurrence/GPU simulation, authenticated observation, actual dispatch or numerical validation.", + "Inspected masked kernel and ops output handling differ at source level; no masked-output native parity case executed, confirmed numeric bug or fix claimed.", + "Full ChatSession/cache reconciliation, models/loading/quantization/VLM/tools/grammar/LoRA/training/FoundationModels and external MLX manager/provider internals are bounded gaps; broad skill efficacy unmeasured.", + "NO_LICENSE is actual unauthenticated gh resolver fallback, not legal absence; root MIT retained independently." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Resolve mutable-context/cache ownership and task settlement separately from lexical isolation and queued GPU work before cache reuse or producer shutdown.", + "Follow recurrent shape/dtype/layout to the actual Metal or ops consumer; predicate eligibility and useKernel alone do not prove dispatch.", + "Record the actual dependency resolution and use the documented Xcode macOS test route; the source states swift test does not work for this package." + ] + } +--- + +# MLX Swift LM ownership and recurrent Metal contracts + +Use this source-derived **candidate** for the Swift package's generation and +selected native recurrent-attention boundary. It is not a full model catalog, +a generic MLX tutorial or a verified engine. Begin with +[source and method identity](references/repo-provenance.md); source, resolved +MLX dependency, native artifacts and model assets are separate identities. + +## Route by the broken contract + +| Question | Canonical owner | +| --- | --- | +| Who owns mutable context/arrays/cache, what does cancellation stop, and has this producer actually settled? | [generation-lifecycle](sub-skills/generation-lifecycle/SKILL.md) | +| Does recurrent state/shape/dtype agree with the public GatedDelta gate and actual Metal/ops consumer? | [gated-delta-contract](sub-skills/gated-delta-contract/SKILL.md) | +| Which dependencies, model/runtime surfaces and rights remain outside this extraction? | [scope and notices](references/scope-and-notices.md) | + +The package owns meaningful Swift concurrency and an embedded MSL recurrent +kernel, not merely an example invoking a foreign engine. Its MLX dependency +still owns array evaluation, custom-kernel binding and device/wired-memory +services. Do not invent those internals from a wrapper, or transfer Python +MLX-LM and other engines' cache/stream semantics without examining the caller. + +## Establish identity and evidence before changing behavior + +1. Record the exact source revision, resolved Swift/MLX dependencies, Xcode/SDK/ + OS/device, API family and native artifact identity. A manifest version range + or a source release is not a resolved dependency or compiled artifact. +2. Separate lexical Swift isolation, exclusive access across await, ownership + transfer, queued GPU work and completed task join. They are not synonyms. + Select the lifecycle route before reusing a caller-owned cache or shutting + down a producer after a consumer stops. +3. For recurrent kernels, follow producer shape/dtype/layout to the actual + native consumer and fallback. A public predicate or `useKernel: true` is + not observed dispatch; an ops fallback is not necessarily CPU execution. +4. Use the selected bundled helper with explicit synthetic/caller declarations. + It does not import Swift/MLX, execute tasks or recurrence, authenticate + observations, compile native code, load weights or simulate a device. +5. Retain negative signals, untested feature combinations and source-only + discrepancies. Obtain separate authority and freeze controls/gates before + any actual native, model, quality or service experiment. Source test-local + tolerances and implementation constants are not portable acceptance bars. + +## Preparation boundary + +The inspected main branch documents the 3.x API line; its Package.swift requires +Swift tools 6.2 and declares platform minima macOS 14/iOS 17/tvOS 17/visionOS 1. +FoundationModels integration has distinct SDK/runtime availability constraints; +those minima do not qualify that adapter. MLX Swift is declared with +`.upToNextMinor(from: "0.33.0")`, and swift-syntax has a version range. There is +no tracked Package.resolved at this pin. Record the actual resolution before +native comparison rather than treating an observed tag as a lockfile. + +This portable graph ships no installer or model-loading example. Use a +separately reviewed, authorized and versioned native preparation procedure; +do not run remote installers or fetch models/dependencies to read the skill. +The source explicitly says `swift test` does not work for this package; its +native route uses Xcode macOS testing. Source test fixtures and build steps are +not runtime dependencies of this packaged graph. Native execution plans stay +in external verification reports, not checkout-dependent helper commands. + +Current inspection has no Swift/Xcode/Apple Metal stack. No live Swift API, +compiler, original native tests or CLI execution is claimed. The actual AREX +create/prepare/manual-workflow/verify adaptation is **partial source plus +offline support**, not a DisCo CLI/model run or managed skill installation. + +## Return a bounded result + +Report identities, actual versus declared facts, route/invariant, safe support +result, counter-signal and smallest next separately authorized native check. +Keep source inference, preparation, dispatch, numerical behavior, model quality +and client-observed service measurements distinct. Required native evidence +remains `BLOCKED_REQUIRED_BACKEND / not_run`; candidate empirical validation +is `not_run`. Packaging, independent content review and repository CI do not +promote the engine, establish physical fit, or prove comparative skill efficacy. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/scope-and-notices.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/scope-and-notices.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/scope-and-notices.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/scope-and-notices.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/helper-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/references/helper-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/helper-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/references/helper-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/recurrence-and-state.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/references/recurrence-and-state.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/recurrence-and-state.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/references/recurrence-and-state.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/verification-boundaries.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/references/verification-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/references/verification-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/references/verification-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/scripts/audit_gated_delta_declarations.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/scripts/audit_gated_delta_declarations.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/gated-delta-contract/scripts/audit_gated_delta_declarations.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/gated-delta-contract/scripts/audit_gated_delta_declarations.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/evidence-schema.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/references/evidence-schema.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/evidence-schema.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/references/evidence-schema.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/native-evidence.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/references/native-evidence.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/native-evidence.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/references/native-evidence.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/ownership-and-termination.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/references/ownership-and-termination.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/ownership-and-termination.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/references/ownership-and-termination.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/scripts/audit_generation_evidence.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/scripts/audit_generation_evidence.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm/sub-skills/generation-lifecycle/scripts/audit_generation_evidence.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm/sub-skills/generation-lifecycle/scripts/audit_generation_evidence.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/SKILL.md new file mode 100644 index 000000000..56d9656de --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/SKILL.md @@ -0,0 +1,233 @@ +--- +name: project-maya +description: "Investigate Project Maya model-family dispatch, representation and memory tiers, API versus native state, speculative decoding and source-qualified verification without treating safe CPU checks as CUDA/HIP or model evidence." +license: NO_LICENSE +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "partial_cpu_only", + "license": "NO_LICENSE", + "tags": [ + "project-maya", + "model-specific-inference", + "cuda", + "rocm", + "expert-tiering", + "native-state", + "speculation", + "evidence-boundaries" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/mw00/project-maya", + "commit": "249f9a87d1a9d72bec2b684d704b0aa9296b1c9c", + "version": "VERSION1.1.34; CMake0.1.27 differs", + "license": "MIT (root source); model/optional assets separate", + "readset": "references/source-identity.json", + "paths": [ + "CMakeLists.txt", + "LICENSE", + "LICENSE.deepseek-v41", + "LICENSE.minimax-m31", + "LICENSE.qwen-community-1.0", + "README.md", + "VERSION", + "bench/results/2026-10-06-glm-spec/README.md", + "bench/results/MAYA-S.md", + "docs/MODELS.md", + "docs/RESPONSES.md", + "docs/SETTINGS.md", + "include/strata/core/glm_model.hpp", + "include/strata/core/mtp_rows.hpp", + "include/strata/program/conv_cache.hpp", + "include/strata/spec/controller.hpp", + "include/strata/spec/draft_policy.hpp", + "include/strata/spec/suffix_drafter.hpp", + "maya.py", + "maya.sh", + "serve/response_config.py", + "serve/response_jobs.py", + "serve/response_schema.py", + "serve/response_store.py", + "serve/responses.py", + "serve/server.py", + "serve/test_response_jobs.py", + "serve/test_response_store.py", + "serve/test_responses.py", + "setup.py", + "setup.sh", + "src/core/glm_fast_path.cu", + "src/core/glm_memory.hpp", + "src/core/glm_model.cu", + "src/core/glm_mtp.cu", + "src/core/glm_mtp_rows.cu", + "src/core/glm_multi.cu", + "src/core/glm_window.cu", + "src/ds4/ds4_main.cpp", + "src/ds4/ds4_model.cu", + "src/ds4/ds4_model.hpp", + "src/mm3/mm3_geo.hpp", + "src/mm3/mm3_lane_plan.hpp", + "src/mm3/mm3_main.cpp", + "src/mm3/mm3_model.cu", + "src/mm3/mm3_model.hpp", + "src/mm3/mm3_pack.hpp", + "src/mm3/mm3_sample.hpp", + "src/program/conv_cache_test.cpp", + "src/program/generate.cpp", + "src/spec/controller.cpp", + "src/spec/controller_test.cpp", + "src/spec/draft_policy.cpp", + "src/spec/draft_policy_test.cpp", + "src/spec/suffix_drafter.cpp", + "src/spec/suffix_drafter_test.cpp", + "tests/core/mtp_rows_test.cpp", + "tests/glm_memory_test.cpp", + "tests/glm_pack_graph_test.cpp", + "tests/glm_rows_attn_test.cu", + "tests/glm_rows_kernels_test.cu", + "tests/mm3_lane_plan_test.cpp", + "third_party/beellama.cpp/LICENSE", + "third_party/exllamav3/LICENSE", + "third_party/ggml/LICENSE", + "tools/calibrate_draft.py", + "tools/calibrate_ds4.py", + "tools/calibrate_ds4_tp.py", + "tools/calibrate_glm.py", + "tools/calibrate_mm3_prefill.py", + "tools/ds4/pack.py", + "tools/ds4/pack_gguf.py", + "tools/exl3_pack.py", + "tools/iq_pack.py", + "tools/maya_names.py", + "tools/maya_quant/kl_eval.py", + "tools/maya_quants.py", + "tools/mm3/calibrate.py", + "tools/mm3/pack.py", + "tools/mm3/pack_gguf.py", + "tools/test_bench.py", + "tools/test_calibrate_glm.py" + ] + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/workflow-authoring", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" + ], + "scope": "Compatible-agent portable native C++ adaptation, not DisCo CLI/model execution." + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "repository-operating-graph", + "strategy": "arex-portable-meta-skills", + "status": "partial-candidate", + "native_import": "not-run", + "method": "Stable-ID isolated author lanes, integrated runtime graph, assertion/helper/static and selected direct-compiler CPU verification; required accelerator/model blocks retained." + }, + "applicability": { + "scope": "Model-specific dispatch/residency, API/native state and qualification investigation. Common catalog placement is not CPU inference or cross-device certification.", + "prerequisites": "Access to the selected model-family runtime/state/qualification contract; native changes need frozen correctness/comparison requirements and the matching authorized CUDA/HIP/model resources.", + "input_contract": "Full source revision, actual executable/model family, model and asset identities, representation, compiler/runtime/device, rendered workload, sampling policy, single/parallel route, exact tokens/images and relevant completion observations.", + "produced_artifact": "A route/identity/evidence-class report with invariant, proposed check, actual result, counter-signal, missing resource and next authorized action; missing native requirements remain INCONCLUSIVE.", + "failure_modes": "Transferring coverage through a common facade or quantization name, confusing API history with native KV/private state, double-counting unified RAM/VRAM, or repairing a slot header to appear compatible can invalidate diagnosis.", + "refusal_point": "When required CUDA/HIP/model evidence is unavailable, retain BLOCKED_REQUIRED_BACKEND and an unverified/INCONCLUSIVE conclusion; exit-zero skips or CPU simulations cannot clear it." + }, + "limitations": [ + "BLOCKED_REQUIRED_BACKEND: no actual CUDA/HIP/model/device, quantized payload, vision, parallel/state/restore or cancellation-cleanup evidence.", + "Safe CPU slices and offline helpers cannot establish complete engine builds, memory fit, numerical/model quality, service/performance or broad skill efficacy.", + "Qwen coverage is shallow routing/format/guard evidence; its deep runtime/state/kernels remain unmined/unverified. Other family coverage is bounded rather than exhaustive.", + "Source reports and source-specific calibration margins are not local measurements or portable acceptance gates.", + "NO_LICENSE is the actual unauthenticated gh resolver fallback; root MIT is retained independently and optional/model asset licenses remain separate." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Select the family-specific route and preserve the explicitly shallow Qwen coverage rather than transferring deeper GLM/DeepSeek/MiniMax evidence.", + "Separate shared weights, private conversation state, API history and native KV; count unified physical memory once.", + "Preserve exact token/image identities and completion boundaries before reuse, then test one falsifiable hypothesis without modifying slot headers to bypass compatibility." + ] + } +--- + +# Project Maya inference engineering + +Use this source-derived **candidate** for model-specific inference questions in +`mw00/project-maya`. Read [source and method provenance](references/repo-provenance.md) +first. It is neither a generic model loader nor a verified inference engine. +The bundled helpers are offline support; reading this skill does not authorize +installation, engine startup, model downloads, GPU/API calls or host changes. + +## Route by the failing contract + +| Question | Canonical procedure | +| --- | --- | +| Which model family/format, architecture guard, physical memory pool or residency tier? | [model-runtime](sub-skills/model-runtime/SKILL.md) | +| API continuation, ownership, native slots, images, parallel state, cancellation or recovery? | [serving-state](sub-skills/serving-state/SKILL.md) | +| Draft selection, rollback, calibration, quantization quality, benchmark or evidence level? | [qualification](sub-skills/qualification/SKILL.md) | +| Build/setup side effects, missing tools or dependency/license ambiguity? | [troubleshooting](references/troubleshooting.md) and [license boundaries](references/license-boundaries.md) | + +The retained snapshot has GLM, DeepSeek and MiniMax operating paths plus a newly +documented Qwen route. Coverage is intentionally asymmetric: selected core/state/ +qualification bodies for the first three, shallow dispatch/format/guard evidence +for Qwen. Unmined deep kernels and unexecuted native/model paths stay explicit; +source presence or a common Python facade is not cross-family qualification. + +## Establish identity before changing a setting + +1. Record the full source revision, actual executable/family, model and asset + identities, representation, compiler/runtime/device, full rendered workload, + sampling policy and active single/parallel route. A release label, common + quant tier name or nominal bit width is insufficient. +2. Distinguish caller-supplied metadata from inspected bytes and executed results. + Root MIT notices do not license every optional model asset; source code, + weights, drafts, fonts and UI components have separate boundaries. +3. Identify the state/resource owner and lifetime. Shared weights are not private + conversation state; API history is not native KV; unified physical memory + must not be counted independently as both RAM and VRAM. +4. Select the relevant route, its local helper and its stop conditions. Preserve + exact tokens/image identity and completion boundaries before attempting reuse + or tuning. Never repair a slot header merely to make it appear compatible. +5. State a falsifiable hypothesis and the smallest safe check. A policy arithmetic + result, simulated decision or request JSON can guide an investigation but + cannot authenticate a model payload, actual fit, GPU cleanup or speedup. +6. For a native change, freeze correctness and comparison requirements before + experiments, retain failures/negative observations, and distinguish component + throughput from complete service/quality outcomes. Do not transfer a source + tuner margin into a portable campaign acceptance bar. + +## Return a bounded conclusion + +Report the route and identities, evidence class, invariant, proposed check, +actual result, counter-signal, missing resource and next authorized action. +When required CUDA/HIP/model evidence is unavailable, keep +`BLOCKED_REQUIRED_BACKEND` and an unverified/INCONCLUSIVE conclusion. Do not turn +exit-zero skips, CPU simulations, source reports or green scaffold CI into native +PASS. Model quality, E2E service behavior and broad skill efficacy need separate +validation. Native/backend flags remain unverified until separately established. + +## Safe support and source boundaries + +Each sub-skill owns a small bundled helper and detailed contract/troubleshooting +references. They are standard-library utilities, not model runners or copies of +unsafe setup/calibration scripts. Use their documented synthetic inputs; no +helper sends a request or installs dependencies. Source files are immutable +attribution links, not required runtime checkout paths. Keep this entire owner +folder when distributing it so the references, scripts and MIT notice remain. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/license-boundaries.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/license-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/license-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/license-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/model-and-memory.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/references/model-and-memory.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/model-and-memory.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/references/model-and-memory.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/notices.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/references/notices.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/notices.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/references/notices.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/scripts/audit_runtime_plan.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/scripts/audit_runtime_plan.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/model-runtime/scripts/audit_runtime_plan.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/model-runtime/scripts/audit_runtime_plan.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/references/verification-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/references/verification-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/references/verification-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/references/verification-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/scripts/summarize_decode_records.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/scripts/summarize_decode_records.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/qualification/scripts/summarize_decode_records.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/qualification/scripts/summarize_decode_records.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/api-and-history.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/references/api-and-history.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/api-and-history.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/references/api-and-history.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/native-state.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/references/native-state.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/native-state.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/references/native-state.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/scripts/request_template.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/scripts/request_template.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/project-maya/sub-skills/serving-state/scripts/request_template.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya/sub-skills/serving-state/scripts/request_template.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/references/snapshot.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/references/snapshot.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/references/snapshot.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/references/snapshot.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-architecture/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-architecture/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-architecture/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-architecture/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-architecture/references/runtime.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-architecture/references/runtime.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-architecture/references/runtime.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-architecture/references/runtime.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-extension/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-extension/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-extension/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-extension/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-extension/references/integration.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-extension/references/integration.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-extension/references/integration.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-extension/references/integration.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-serving/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-serving/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-serving/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-serving/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-serving/references/recipes-and-tuning.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-serving/references/recipes-and-tuning.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang-omni/sub-skills/sglang-omni-serving/references/recipes-and-tuning.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang-omni/sub-skills/sglang-omni-serving/references/recipes-and-tuning.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/SKILL.md similarity index 92% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/SKILL.md index 11e127ad2..b60a28c19 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/SKILL.md @@ -18,7 +18,7 @@ are independent; each row's inspection date applies only to its stated scope. | Question | Existing entry | Fixed evidence / original date | Limit | | --- | --- | --- | --- | | LLM admission, Radix/KV ownership, topology or controlled engine tuning | This engine graph; [runtime and memory](sub-skills/sglang-runtime-memory/SKILL.md) and [source map](references/source-map.md) | `16176278d0180f1a1a578e53e6eef432577876c2`; contribution records accessed 2026-10-14, with separate [snapshot provenance](references/repo-provenance.md) | LLM routes retain this snapshot; engine navigation is not a tested combination of all routes/backends | -| Qwen3.5/Qwen3-Next GDN versus convolution dispatch, phase metadata or one-shot prefill transfer | [Kernel caller/backend contract](../../methods/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md) and [source cases](../../methods/kernel-writing-guide/cases/sglang-gdn-source-contract.md) | Same `16176278d0180f1a1a578e53e6eef432577876c2`; bounded source inspection 2026-10-29 | Candidate/not_run; per-runner GDN choices, conv stride/alias branches and state-row contracts; no HF binding, native hardware, model-quality or performance qualification | +| Qwen3.5/Qwen3-Next GDN versus convolution dispatch, phase metadata or one-shot prefill transfer | [Kernel caller/backend contract](../../../playbooks/optimization/kernel-writing-guide/references/sglang-gdn-causal-conv-contract.md) and [source cases](../../../playbooks/optimization/kernel-writing-guide/cases/sglang-gdn-source-contract.md) | Same `16176278d0180f1a1a578e53e6eef432577876c2`; bounded source inspection 2026-10-29 | Candidate/not_run; per-runner GDN choices, conv stride/alias branches and state-row contracts; no HF binding, native hardware, model-quality or performance qualification | | Reusable LLM prefix KV across device, host and optional storage tiers | Separate catalog ID `sglang.cuda.llm.hierarchical_prefix_cache` ([canonical peer record](https://github.com/InferMatrixLab/InferRSI/blob/8233a2ba795a22af9b8f17c8793da0a936a38857/src/inferrsi/data/knowledge_packs/peer/sglang/cuda/llm/hierarchical-prefix-cache.json)) | `f7ad4345b86d147a1c3095a3cebabca9a33ebaa1`; HiCache documentation accessed 2026-10-15 | CUDA/LLM peer card; imported, locally_verified=false; prefix identity, sharing scope and transfer/recompute cost require matching evidence | | multimodal_gen stage state, component-use intervals or timeout/job finalization | [Diffusion lifetimes](audit-sglang-diffusion-lifetimes/SKILL.md) and its [source notes](audit-sglang-diffusion-lifetimes/references/source-notes.md) | `df13d5dce31abfeff0e701a62e8c39a136758e6c`; supplied archive inspected 2026-10-22 | Candidate, empirical validation not_run; bounded monolithic path, separate from LLM srt and SGLang-Omni | diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md similarity index 54% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md index 935bab44c..c654ccf70 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md @@ -1,6 +1,82 @@ --- name: audit-sglang-diffusion-lifetimes description: "Diagnose SGLang multimodal_gen stage state, component residency and request timeout/deletion when grouped outputs interfere, a module remains resident, or stopped waiting is mistaken for stopped execution; requires pinned source or a concrete trace." +metadata: + inferrsi: | + { + "tags": [ + "sglang", + "diffusion", + "stage", + "state", + "residency", + "cancellation" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "license": "Apache-2.0", + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/sgl-project/sglang", + "commit": "df13d5dce31abfeff0e701a62e8c39a136758e6c", + "paths": [ + "python/sglang/multimodal_gen/runtime/entrypoints/control_requests.py", + "python/sglang/multimodal_gen/runtime/entrypoints/openai/utils.py", + "python/sglang/multimodal_gen/runtime/entrypoints/openai/video_api.py", + "python/sglang/multimodal_gen/runtime/managers/gpu_worker.py", + "python/sglang/multimodal_gen/runtime/managers/memory_managers/component_manager.py", + "python/sglang/multimodal_gen/runtime/managers/scheduler.py", + "python/sglang/multimodal_gen/runtime/pipelines_core/composed_pipeline_base.py", + "python/sglang/multimodal_gen/runtime/pipelines_core/diffusion_scheduler_utils.py", + "python/sglang/multimodal_gen/runtime/pipelines_core/executors/parallel_executor.py", + "python/sglang/multimodal_gen/runtime/pipelines_core/executors/pipeline_executor.py", + "python/sglang/multimodal_gen/runtime/pipelines_core/schedule_batch.py", + "python/sglang/multimodal_gen/runtime/pipelines_core/stages/dedup.py", + "python/sglang/multimodal_gen/runtime/pipelines_core/stages/denoising.py", + "python/sglang/multimodal_gen/runtime/pipelines_core/stages/timestep_preparation.py", + "python/sglang/multimodal_gen/runtime/scheduler_client.py", + ".github/workflows/diffusion-ci-gt-gen-npu.yml", + ".github/workflows/diffusion-ci-gt-gen.yml", + "LICENSE", + "python/pyproject.toml", + "python/sglang/multimodal_gen/test/unit/test_component_residency.py", + "python/sglang/multimodal_gen/test/unit/test_pipeline_executor.py", + "python/sglang/multimodal_gen/test/unit/test_pipeline_residency_manager_install.py", + "python/sglang/multimodal_gen/test/unit/test_scheduler_client.py" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Bounded monolithic multimodal_gen composed-pipeline state and lifecycle diagnosis; actual backend/model behavior requires native validation.", + "prerequisites": "Pinned source or a concrete trace of the selected composed-pipeline executor and installed residency manager, with the applicable monolithic route identified.", + "input_contract": "Source/build, backend and executor/pipeline class, model revision, modality/shape, steps/scheduler/seed/output count, grouped/serial route, ranks, placement, warmup/save settings and symptom; timeout cases need correlated request/job and client/scheduler/worker events.", + "produced_artifact": "A version-bound owner/event ledger, first unresolved boundary, causal hypothesis, smallest real discriminating check, exact results and missing dependencies.", + "failure_modes": "Assuming all executors share the SyncExecutor contract, treating closed component use as eviction, assuming grouped schedulers are isolated, or reading socket close/job deletion as stopped device compute can misdiagnose ownership.", + "refusal_point": "Stop performance claims when the executing path, cleanup or output quality is unresolved; retain the failure and a known baseline rather than inferring a blanket leak or safety conclusion." + }, + "limitations": [ + "Source-derived candidate; upstream runtime and efficacy validation not run.", + "Supplier verified one selected source blob, not the whole archive tree.", + "No automatic grouped isolation, device cancellation, cross-rank recovery or performance claim." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Trace the actual executor and separate request, stage, component-use, module, scheduler, finalizer and allocator-cache lifetimes.", + "Check scheduler aliases and declared output-copy policies before introducing overlap; compare real serial/grouped outputs with fixed timesteps and seeds.", + "Correlate worker, finalizer and device completion after client timeout or deletion; preserve an unknown execution outcome until that evidence exists." + ] + } --- # Audit SGLang diffusion lifetimes diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/architecture-and-memory.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/architecture-and-memory.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/architecture-and-memory.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/architecture-and-memory.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/native-candidates.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/native-candidates.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/native-candidates.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/native-candidates.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/parallelism-and-extension.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/parallelism-and-extension.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/parallelism-and-extension.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/parallelism-and-extension.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/tuning-and-validation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/tuning-and-validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/references/tuning-and-validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/references/tuning-and-validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/sub-skills/sglang-parallel-extension/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/sub-skills/sglang-parallel-extension/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/sub-skills/sglang-parallel-extension/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/sub-skills/sglang-parallel-extension/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/sub-skills/sglang-runtime-memory/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/sub-skills/sglang-runtime-memory/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/sub-skills/sglang-runtime-memory/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/sub-skills/sglang-runtime-memory/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/sglang/sub-skills/sglang-tuning-validation/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/sub-skills/sglang-tuning-validation/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/sglang/sub-skills/sglang-tuning-validation/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/sglang/sub-skills/sglang-tuning-validation/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/SKILL.md new file mode 100644 index 000000000..a7d59578c --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/SKILL.md @@ -0,0 +1,213 @@ +--- +name: splash +description: "Investigate Splash model-format compatibility, automatic memory planning, Metal and ANE execution, DFlash2 verification, cache and request identity, and correctness-gated serving measurements." +license: NO_LICENSE +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "partial_cpu_only", + "license": "NO_LICENSE", + "tags": [ + "splash", + "apple-silicon", + "metal", + "ane", + "dflash2", + "memory-plan", + "prefix-cache", + "persistent-cache" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/incoai/splash", + "commit": "39a97385f8560cf8a93972cec9b0a963c9195250", + "paths": [ + "README.md", + "DEVELOPMENT.md", + "docs/performance.md", + "LICENSE", + "THIRD_PARTY_NOTICES", + "Makefile", + "server/serve_options.py", + "server/frontend.py", + "server/lru.py", + "server/protocol.py", + "server/http_security.py", + "server/origins.py", + "server/runtime.py", + "install/gguf.py", + "runtime/engine/MemoryPlan.hpp", + "runtime/engine/MemoryPlan.cpp", + "runtime/engine/MemoryGovernor.cpp", + "runtime/engine/MemoryControl.cpp", + "runtime/engine/MemoryAudit.hpp", + "runtime/engine/MemoryAudit.cpp", + "runtime/engine/KvCache.hpp", + "runtime/engine/KvCache.cpp", + "runtime/engine/Cache.hpp", + "runtime/engine/Cache.cpp", + "runtime/engine/StateCache.hpp", + "runtime/engine/StateCache.cpp", + "runtime/engine/Scheduler.cpp", + "runtime/engine/WriteBehind.hpp", + "runtime/engine/WriteBehind.cpp", + "runtime/engine/RuntimeResources.mm", + "runtime/engine/Protocol.hpp", + "runtime/engine/Protocol.cpp", + "runtime/ops/DraftSelector.hpp", + "runtime/ops/DraftSelector.cpp", + "runtime/ops/Sampling.hpp", + "runtime/ops/Sampling.cpp", + "runtime/metal/kernels/decode/sampling.metal", + "runtime/model/Runtime.hpp", + "runtime/model/Runtime.cpp", + "runtime/model/QwenState.cpp", + "runtime/model/ModelDescriptor.mm", + "runtime/model/DFlashDraft.hpp", + "runtime/model/VerifyBudget.hpp", + "dev/benchmarks/abba.py", + "dev/benchmarks/backend_regression.py", + "dev/benchmarks/logit_compare.py", + "dev/benchmarks/http_regression.py", + "dev/tests/server/test_serve_options.py", + "dev/tests/server/test_lru_cache.py", + "dev/tests/server/test_protocol_python.py", + "dev/tests/tools/test_abba.py", + "dev/tests/engine/engine_memory_plan_test.cpp", + "dev/tests/engine/memory_audit_test.cpp" + ], + "license": "Apache-2.0", + "readset": "references/source-identity.json" + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/workflow-authoring", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" + ], + "scope": "Actual portable compatible-agent workflow with native/source-module adaptation, not DisCo CLI/model execution." + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "repository-operating-graph", + "strategy": "arex-portable-meta-skills", + "status": "partial-candidate", + "native_import": "not-run", + "method": "User-approved native Metal/C++ and isolated source-Python adaptation; stable-ID lanes, integrated graph, assertion cases and create/verify stages. Required-backend block disables automatic admission." + }, + "applicability": { + "scope": "Pinned Splash representation, planning, state/serving and verification investigation; common discovery does not qualify a different Mac/model/backend.", + "prerequisites": "Access to the pinned representation/planning, serving-cache or qualification route; native checks need matching Apple/backend/model resources, an authorized workload and frozen semantic/measurement controls.", + "input_contract": "Source/build, macOS/device, target and compatible draft artifacts/formats, tokenizer/template/vision selection, context/cache/memory controls, KV/recurrent precision, ANE and verifier-row policy, concurrency/priority/sampling, quality controls and cache state.", + "produced_artifact": "An execution-identity report with observation, source-backed invariant, alternate explanation, smallest discriminating check, actual result and unresolved Metal/ANE/model/disk-state gap.", + "failure_modes": "Treating an API alias as new weights, equal bit width as loader/kernel compatibility, or a fixed seed as fixed cache/batch/adaptive-verification scheduling can invalidate the comparison; CPU cache/codec checks do not prove GPU state.", + "refusal_point": "Stop at unsupported formats, unavailable backend/model resources, unsafe service exposure or the authorized budget." + }, + "limitations": [ + "Required Metal/ANE/model/actual device and disk-state execution remains BLOCKED_REQUIRED_BACKEND; Python policy/codec checks do not clear it.", + "Service performance, quality, native recurrent-state recovery and skill efficacy remain unverified.", + "Only selected bodies were mined; not all formats, kernels, supported devices or release/client workflows.", + "NO_LICENSE is the actual unauthenticated GitHub CLI resolver fallback, not a legal conclusion; Apache-2.0 and third-party notices are retained." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Resolve representation, memory/cache owner, validity key and reclaim/rollback boundary before tuning a numeric control.", + "Freeze semantic and workload controls beyond the seed, and retain actual-path fallbacks, admission waits, evictions, cancellations and errors.", + "Inspect options without starting the normal serve path, which can download target/draft weights; installation and serving require separate task authority." + ] + } +--- + +# Splash inference engineering + +Use this source-derived candidate to diagnose an inference or serving question, +select a bounded change and state the check that could falsify it. It is not a +complete upstream manual or a certified Apple backend recipe. Read +[source and method provenance](references/repo-provenance.md) before reuse. + +## Route the work + +| Question | Route | Expected result | +| --- | --- | --- | +| A format is rejected, a model will not fit or a cache cap is surprising | [runtime-planning](sub-skills/runtime-planning/SKILL.md) | Compatible representation and memory-owner ledger | +| Prefix reuse, persistent cache, request isolation or Responses history is confusing | [serving-cache](sub-skills/serving-cache/SKILL.md) | Identity, state-lifetime, admission and recovery ledger | +| Draft acceptance rises but speed/answers change; an ANE or precision path differs | [qualification](sub-skills/qualification/SKILL.md) | Semantic control, actual path and matching proof ladder | + +Use the bundled [request builder](scripts/request_template.py) to prepare a +small Chat Completions body offline. It prints JSON; it never contacts a server, +installs Splash, launches a client or downloads a model. Its deliberately small +option set is not the complete upstream API. + +## Establish the execution tuple + +Record source/build revision, macOS and Apple device, target checkpoint/format/ +quantization and artifact identity, compatible draft, tokenizer/template and +vision selection. Add context, cache/memory controls, KV and recurrent-state +precision, ANE setting, verifier-row policy, concurrency/priority, sampling, +prompt/output/quality controls and cold/warm/persistent-cache state. + +An API model alias is not a new weight selection. Equal bit width does not +prove loader/kernel compatibility. A seeded request does not freeze cache, +batch/chunk or adaptive-verification scheduling. + +## Native preparation boundary + +The source targets Apple Silicon M3 or newer and macOS 26.4 or later. The public +release installation is `brew install incoai/tap/splash`; an authorized operator +can inspect `splash serve --help` before loading anything. Installation and +serving are separate actions: the normal serve path can download target/draft +weights. Do not run it solely to inspect an option. + +Splash is a native Metal/C++ runtime with Python server/installer modules, not +a pip-installable project. This extraction adapted AREX's package inspection +to isolated, explicit source-module identity/signature checks. Linux Python +logic does not validate the actual Metal/ANE/model path. + +## Work a bounded hypothesis + +1. Read the selected route and nearest bundled mechanism/failure reference. +2. Separate source observation, upstream measurement and proposed experiment. +3. Identify representation, owner, validity key, lifetime and reclaim/rollback + boundary before tuning a numeric control. +4. Freeze workload, semantic reference and measurement boundaries before a + comparison. Keep campaign pins and gates in the acceptance issue. +5. Confirm the path actually activated; retain fallbacks, admission waits, + evictions, cancellations and errors instead of treating them as warmup noise. +6. Correctness and model quality precede ranking. A cache/parser/codec check + cannot establish GPU state, a distributional guarantee or service SLOs. + +## Stop and report + +Return execution identity, observation, source-backed invariant, alternate +explanation, smallest discriminating check, actual result and unresolved gap. +Stop at unsupported formats, unavailable backend/model resources, unsafe service +exposure or the authorized budget. Do not change system settings or launch an +external coding client merely because imported guidance mentions one. + +Required Metal/ANE/model/actual disk-state evidence remains +**BLOCKED_REQUIRED_BACKEND**. Source and CPU checks do not qualify performance, +quality, service operation or behavioral skill efficacy. Final admission of a +partially verified candidate requires an informed decision, not auto-import. + +The source [Apache-2.0 license](LICENSE) and +[third-party notices](THIRD_PARTY_NOTICES) are retained. `NO_LICENSE` is the +actual AREX resolver's unauthenticated-CLI warning fallback, not a legal finding; +models and external tools retain their own licenses. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/THIRD_PARTY_NOTICES b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/THIRD_PARTY_NOTICES similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/THIRD_PARTY_NOTICES rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/THIRD_PARTY_NOTICES diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/scripts/request_template.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/scripts/request_template.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/scripts/request_template.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/scripts/request_template.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/references/verification-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/references/verification-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/references/verification-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/references/verification-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/scripts/summarize_rounds.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/scripts/summarize_rounds.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/qualification/scripts/summarize_rounds.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/qualification/scripts/summarize_rounds.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/references/memory-and-formats.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/references/memory-and-formats.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/references/memory-and-formats.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/references/memory-and-formats.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/scripts/parse_budget.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/scripts/parse_budget.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/runtime-planning/scripts/parse_budget.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/runtime-planning/scripts/parse_budget.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/serving-cache/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/serving-cache/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/references/state-and-api.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/serving-cache/references/state-and-api.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/references/state-and-api.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/serving-cache/references/state-and-api.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/serving-cache/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/splash/sub-skills/serving-cache/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/splash/sub-skills/serving-cache/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md similarity index 51% rename from src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md index 83d075b0c..ceb6014f4 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md @@ -1,6 +1,93 @@ --- name: audit-sd-cpp-residency description: Audit stable-diffusion.cpp image-generation memory pressure by separating catalog freshness and weight-byte delivery from model parameter storage, compute weight residency, retained runtime buffers and segment prefetch under the actual backend. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "MIT", + "tags": [ + "stable-diffusion-cpp", + "diffusion", + "model-residency", + "workspace", + "prefetch", + "state-ownership", + "weight-loading", + "storage-io" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/leejet/stable-diffusion.cpp", + "commit": "b13a48219d5cdf408edcb93da6cd69b612ed03d3", + "paths": [ + "src/stable-diffusion.cpp", + "src/pipeline/image.cpp", + "src/pipeline/diffusion_engine.h", + "src/pipeline/diffusion_engine.cpp", + "src/core/ggml_runner.cpp", + "src/core/segment_weight_pipeline.cpp", + "src/core/compute_workspace.cpp", + "src/model_manager.cpp", + "src/model_manager_prefetch.cpp", + "src/device_residency_manager.h", + ".github/workflows/build.yml", + ".gitmodules", + "cmake/ggml.cmake", + "CMakeLists.txt", + "examples/cli/main.cpp", + "include/stable-diffusion.h", + "LICENSE", + "src/model_manager.h", + "src/model_loader.h", + "src/model_loader_files.cpp", + "src/model_loader.cpp", + "src/model_manager_files.cpp", + "src/model.h", + "src/core/util.h", + "src/core/util.cpp", + "src/model_io/tensor_storage.h", + "src/model_io/safetensors_io.cpp" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Source-grounded audit of native image-generation weight/runtime residency, optional segment prefetch and bounded non-ZIP safetensors catalog/read/staging ownership; confirm the actual source, GGML, backend, model and executing path independently.", + "prerequisites": "Access to the pinned source/storage boundaries and evidence of the actual image runner/backend dispatch; native capacity or prefetch checks require matching GGML, model and backend resources.", + "input_contract": "Runtime/GGML revisions and selection, compiled/loaded backends and devices, model/components and numerical image settings, effective memory/prefetch/placement settings and actual dispatch; retain available phase allocations, cache/workspace, transfer events and completed-image counts.", + "produced_artifact": "A phase-by-runner/backend/device ownership table for stored and resident weights, retained runtime, pending allocations, pin/prefetch ownership and release/eviction, with one discriminating check and its actual status.", + "failure_modes": "Treating max_vram as proof of segmentation, unpin as physical freeing, or configured prefetch as completed useful overlap can misidentify the capacity cause; missing observations cannot be read as zero.", + "refusal_point": "Stop when the cause is resolved or matching GGML/model/backend resources are unavailable; invoking this audit does not authorize dependency installation, model downloads, policy changes or device provisioning." + }, + "limitations": [ + "Source-derived candidate; native inference, allocator behavior, image quality and performance were not run.", + "Eighteen bounded readset paths plus one separately identity-checked README match the pinned Git commit/tree blob IDs and sizes (19 blobs total); header coverage is limited to selected owner/state declarations, and this does not authenticate every archive member or submodule contents.", + "The loader/storage extension separately authenticates twelve bounded C++ source blobs and the retained MIT text; its line-scoped readset does not replace the historical archive receipt. Catalog stamps are not content hashes; read/mapping/backend calls do not establish storage throughput or device completion.", + "GGML and other submodules are not expanded; matching private GGML headers are required even for a system library.", + "CPU bypasses the inspected accelerator capacity gate and segmentation eligibility, so CPU checks cannot qualify accelerator memory budgets or prefetch.", + "Video/audio pipelines, detailed model kernels, approximate denoising caches and full multi-device execution remain outside this readset.", + "Loader rules select ordinary non-ZIP safetensors; full GGUF/Torch/ZIP parsers, special quantization/conversion correctness and native failure injection remain unqualified." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/loader-storage-boundaries.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Separate parameter storage, compute-resident weights, retained workspace/output caches and pending allocations for each runner and phase.", + "Verify the actual monolithic, segmented or multi-device path before interpreting a memory budget or prefetch setting.", + "For a policy comparison, hold model, inputs, numerical settings and device fixed; check completed images and quality before latency, and separate startup, first and warm requests." + ] + } --- # Audit diffusion weight and workspace residency diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/loader-storage-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/validation-boundaries.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/validation-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/validation-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/validation-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/references/design-space-analysis.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/design-space-analysis.md similarity index 99% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/references/design-space-analysis.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/design-space-analysis.md index 432bff405..8ae440361 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/strata/references/design-space-analysis.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/design-space-analysis.md @@ -4,7 +4,7 @@ Here, AI generation means proposing configurations or code candidates, filtering them through an explicit evaluator, and retaining a reproducible baseline. Fix the model, hardware, workload and quality contract first. If model or hardware selection is allowed, put it in a separate outer search and redefine comparability. -The [English skill](../SKILL.md) contains design, tuning and validation branches. See the [verification record](../../../workflows/inference-engine-research/references/import.md) for construction and checks. +The [English skill](../SKILL.md) contains design, tuning and validation branches. See the [verification record](../../../../workflows/research/inference-engine-research/references/import.md) for construction and checks. ## Separate three search levels diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/references/trial-format.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/trial-format.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/references/trial-format.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/trial-format.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/scripts/summarize_trials.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/scripts/summarize_trials.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/scripts/summarize_trials.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/scripts/summarize_trials.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/design-space/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/design-space/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/design-space/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/design-space/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/design-space/references/design-space.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/design-space/references/design-space.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/design-space/references/design-space.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/design-space/references/design-space.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/tune-execution/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/tune-execution/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/tune-execution/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/tune-execution/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/tune-execution/references/tuning.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/tune-execution/references/tuning.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/tune-execution/references/tuning.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/tune-execution/references/tuning.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/validate-engine/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/validate-engine/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/validate-engine/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/validate-engine/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/validate-engine/references/validation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/validate-engine/references/validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/strata/sub-skills/validate-engine/references/validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/strata/sub-skills/validate-engine/references/validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/SKILL.md new file mode 100644 index 000000000..981d2a0dd --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/SKILL.md @@ -0,0 +1,188 @@ +--- +name: vllm-ascend +description: "Analyze vLLM Ascend v0.27.1rc1 performance and commit evolution for DeepSeek-V4, Kimi K3, MiniMax-M3 and GLM5+, fused operators, DSA-CP, FlashComm1, Fused-MC2 and DSpark, with effective dispatch, rollback and test-oracle guidance." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "tags": [ + "vllm-ascend", + "v0.27.1rc1", + "MiniMax-M3", + "FP8", + "KV cache", + "DeepSeek-V4-Flash", + "DSpark", + "PD", + "plugin registration", + "DeepSeek-V4", + "Kimi K3", + "MiniMax M3", + "GLM5", + "GLM5.1", + "GLM5.2", + "GLM5.3", + "DSA-CP", + "FlashComm1", + "Fused-MC2", + "fusion", + "performance", + "commit-history" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/vllm-project/vllm-ascend", + "commit": "3b31886237ed65c435e0965b10e8002ff5769766", + "license": "Apache-2.0", + "paths": [ + "LICENSE", + "docs/source/tutorials/models/DeepSeek-V4-Flash.md", + "docs/source/tutorials/models/MiniMax-M3.md", + "docs/source/user_guide/release_notes.md", + "setup.py", + "tests/0.27.1rc_vllm-ascend_overall_conclusion_release_notes_EN.md", + "tests/e2e/nightly/single_node/ops/multicard_ops_a3/test_sfa_cp_a2a.py", + "tests/e2e/nightly/single_node/ops/singlecard_ops/test_chunk_kda_aclnn.py", + "tests/e2e/nightly/single_node/ops/singlecard_ops/test_fused_sparse_attention_overlap.py", + "tests/e2e/nightly/single_node/ops/singlecard_ops/test_kimi_k3_situ_fusion_cases.py", + "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_copy_and_expand_dflash_dspark.py", + "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_kimi_k3_fusions.py", + "tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_split_qkv_tp_rmsnorm_rope.py", + "tests/e2e/pull_request/eight_card/model_runner_v2/test_glm5_2.py", + "tests/e2e/pull_request/eight_card/test_glm5_2.py", + "tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py", + "tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py", + "tests/e2e/pull_request/one_card/compile/test_norm_quant_fusion.py", + "tests/e2e/pull_request/one_card/test_minimax_m3_sparse_attn.py", + "tests/e2e/pull_request/sixteen_card/test_kimi_k3.py", + "tests/e2e/pull_request/utils.py", + "tests/ut/attention/test_dsa_v1.py", + "tests/ut/attention/test_dsv4_block_geometry.py", + "tests/ut/kv_transfer/test_layerwise_multi_cache_spec.py", + "tests/ut/models/minimax_m3/test_minimax_m3_eplb.py", + "tests/ut/models/minimax_m3/test_msa_m3.py", + "tests/ut/models/test_kimi_k3_adapter.py", + "tests/ut/ops/test_kimi_kda.py", + "tests/ut/patch/platform/test_patch_parallel_config.py", + "tests/ut/patch/platform/test_prefix_cache_cp_patches.py", + "tests/ut/quantization/methods/test_fp8_block.py", + "tests/ut/spec_decode/test_dspark_speculator.py", + "tests/ut/spec_decode/test_llm_base_proposer.py", + "tests/ut/test_ascend_config.py", + "tests/ut/test_ascend_forward_context.py", + "tests/ut/test_platform.py", + "tests/ut/worker/test_device_metadata.py", + "tests/ut/worker/test_model_runner_v1.py", + "vllm_ascend/__init__.py", + "vllm_ascend/ascend_config.py", + "vllm_ascend/ascend_forward_context.py", + "vllm_ascend/attention/context_parallel/dsa_cp.py", + "vllm_ascend/attention/dsa_attn_kv_plan.py", + "vllm_ascend/attention/mla_v1.py", + "vllm_ascend/attention/sfa_v1.py", + "vllm_ascend/attention/sparse_flash_mla.py", + "vllm_ascend/compilation/graph_fusion_pass_manager.py", + "vllm_ascend/compilation/passes/muls_add_pass.py", + "vllm_ascend/compilation/passes/norm_quant_fusion_pass.py", + "vllm_ascend/compilation/passes/qknorm_rope_fusion_pass.py", + "vllm_ascend/core/kv_cache_interface.py", + "vllm_ascend/device/device_op.py", + "vllm_ascend/lora/fused_moe.py", + "vllm_ascend/models/__init__.py", + "vllm_ascend/models/common/ops/sequence_parallel.py", + "vllm_ascend/models/deepseek_v4/compressor.py", + "vllm_ascend/models/deepseek_v4/indexer.py", + "vllm_ascend/models/deepseek_v4/model.py", + "vllm_ascend/models/minimax_m3/minimax_m3.py", + "vllm_ascend/models/minimax_m3/msa_m3.py", + "vllm_ascend/models/minimax_m3/ops/msa_m3_npu.py", + "vllm_ascend/ops/fused_moe/moe_comm_method.py", + "vllm_ascend/ops/fused_moe/moe_mlp.py", + "vllm_ascend/ops/fused_moe/token_dispatcher.py", + "vllm_ascend/ops/gdn.py", + "vllm_ascend/ops/kimi_kda.py", + "vllm_ascend/ops/rope_dsv4.py", + "vllm_ascend/ops/triton/docs/sfa_cp_a2a_fused.md", + "vllm_ascend/ops/triton/dsa_cp.py", + "vllm_ascend/patch/platform/patch_parallel_config.py", + "vllm_ascend/patch/worker/patch_deepseek_v2.py", + "vllm_ascend/patch/worker/patch_minimax_m2.py", + "vllm_ascend/platform.py", + "vllm_ascend/quantization/configs/modelslim_config.py", + "vllm_ascend/sample/sampler.py", + "vllm_ascend/spec_decode/dspark_proposer.py", + "vllm_ascend/spec_decode/llm_base_proposer.py", + "vllm_ascend/utils.py", + "vllm_ascend/worker/device_metadata.py", + "vllm_ascend/worker/v2/spec_decode/dspark/speculator.py" + ], + "readset": "references/sources.md" + } + ], + "provenance_readsets": [ + "references/sources.md", + "references/history-scope.md", + "references/history-reviewed.json" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Release-pinned performance diagnosis and historical optimization analysis for DeepSeek-V4, Kimi K3, MiniMax-M3, GLM5+, fusion and communication/speculation paths.", + "prerequisites": "Pinned release evidence or a matching installed stack and role-specific logs.", + "input_contract": "Source and checkpoint revisions, quantization metadata, device and operators, V1/V2 runner, P/D role, TP/DP/PP/PCP/DCP, requested versus derived config, dominant phase and numerical/performance traces.", + "produced_artifact": "First failing boundary with effective values, source locator and smallest discriminating check.", + "refusal_point": "Report an evidence gap when package, role, cache dtype or release identity cannot be established." + }, + "recommendations": [ + "Check the registered general hook in the failing process.", + "Resolve the per-layer cache skip set before interpreting FP8 storage.", + "Compare both DSpark role budgets and connector layouts before changing capture tiers.", + "Check optimization reversals and exact current dispatch before reusing an old performance change.", + "Use commit history for provenance and numerical or serving oracles for validation." + ], + "limitations": [ + "Source-derived guidance for the selected release paths only.", + "Native NPU execution, performance and behavioral skill evaluation not run." + ] + } +--- + +# Investigate release-specific Ascend performance + +Start with the plugin revision, upstream vLLM version, actual device profile, model and draft checkpoint revisions, failure phase and the effective configuration on each role. Use the release table to classify the deployment before borrowing a tutorial or test command. + +| Model | Release-reported configuration | First comparison | +| --- | --- | --- | +| MiniMax-M3 | Atlas 800 A3 W8A8 or 950DT W8A8C8, including two-node 1P1D | Separate weight format, sparse-cache storage and dense-cache storage. | +| DeepSeek-V4-Flash | 950DT W4A8C8, two-node 1P1D DSpark and KV Cache Pool | Compare checkpoint draft budget, connector topology and graph mode on each role. | + +The tag pairs with upstream vLLM **v0.27.1**. Its release stack lists HDK 26.1.1, CANN 9.1.0, PyTorch 2.10.0, torch_npu 2.10.0.post4 and Triton Ascend 3.2.2. Images use Python 3.12 and Mooncake 0.3.11.post1. Record the installed stack separately. For A2 or another model, report the combination as outside this RC's declared scope and identify the version that owns any example you use. + +## Select the performance question + +| Question | Load first | Required discriminator | +| --- | --- | --- | +| DeepSeek-V4 support through performance evolution | [DSV4 and DSpark evolution](references/deepseek-v4-evolution.md) | Current attention/indexer/compressor path, metadata lifecycle and target/draft ownership | +| DSA-CP, FlashComm1 or Fused-MC2 enabled but ineffective | [Configuration and communication](references/communication-config.md) | Requested values versus normalized SP state, hardware dispatch, rollback and conflicting options | +| Fusion opportunity or fusion performance regression | [Fused operators](references/fused-operators.md) | Graph match or actual kernel dispatch, dtype/layout constraints and current replacement path | +| Kimi K3, MiniMax-M3 or GLM5-family optimization | [Model performance](references/model-performance.md) | Specific checkpoint quantization, attention family, runner and phase | +| Trace an old optimization or audit history coverage | [History scope and review levels](references/history-scope.md) | Reachable candidate versus inspected mechanism versus retained behavior | + +Kimi K3 and GLM5-family material describes source implementations within the pinned history. The RC release table above retains its separate validation scope. GLM coverage focuses on GLM5 and later variants present at this tag. + +For an optimization proposal, return the relevant commit, mechanism, current eligibility, rollback or superseding change, closest numerical oracle, and a same-workload measurement plan. Check both producer and consumer before recommending a historical optimization. Distinguish operator timing, model quality, speculative acceptance and sustained service results. + +## Select the first failing boundary + +- Engine initialization or unknown architecture → [trace plugin discovery into the engine process](references/plugin-registration.md). +- MiniMax-M3 starts with FP8 requested but some layers allocate BF16, or top-k results differ → [inspect the effective layer policy and its consumers](references/minimax-cache.md). +- DeepSeek-V4-Flash starts on each node but DSpark acceptance or PD generation fails → [reconcile draft, connector and capture settings](references/dspark-pd.md). + +Return the first discrepancy, its source locator, the effective values on the affected worker or rank, the smallest distinguishing check, and the evidence still needed. Keep initialization, token correctness, acceptance length and sustained service throughput as separate observations. This is a source-derived candidate. Consult the [pinned readset](references/sources.md) for provenance and native validation limits. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/communication-config.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/communication-config.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/communication-config.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/communication-config.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/deepseek-v4-evolution.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/deepseek-v4-evolution.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/deepseek-v4-evolution.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/deepseek-v4-evolution.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/dspark-pd.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/dspark-pd.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/dspark-pd.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/dspark-pd.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/fused-operators.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/fused-operators.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/fused-operators.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/fused-operators.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/history-reviewed.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/history-reviewed.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/history-reviewed.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/history-reviewed.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/history-scope.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/history-scope.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/history-scope.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/history-scope.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/minimax-cache.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/minimax-cache.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/minimax-cache.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/minimax-cache.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/model-performance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/model-performance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/model-performance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/model-performance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/plugin-registration.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/plugin-registration.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/plugin-registration.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/plugin-registration.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/sources.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/sources.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/references/sources.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/references/sources.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/scripts/screen_history.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/scripts/screen_history.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend/scripts/screen_history.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend/scripts/screen_history.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/SKILL.md new file mode 100644 index 000000000..75a4224b8 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/SKILL.md @@ -0,0 +1,198 @@ +--- +name: vllm-metal +description: "Investigate vllm-metal Apple inference through platform and native-ABI selection, paged KV/offload ownership, speculative/deferred state and actual dispatch/serving evidence without upgrading source or CPU checks to backend qualification." +license: NO_LICENSE +disable-model-invocation: true +metadata: + disco-role: operating + inferrsi: | + { + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "partial_cpu_only", + "license": "NO_LICENSE", + "tags": [ + "vllm-metal", + "apple-silicon", + "mlx", + "mps", + "paged-kv", + "kv-offload", + "speculation", + "deferred-execution", + "evidence-boundaries" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/vllm-project/vllm-metal", + "commit": "e7df7a6ff215f99c7842552d230fdbc709835934", + "version": "0.32.0", + "license": "Apache-2.0 (retained source); dependency/model assets separate", + "readset": "references/source-identity.json", + "paths": [ + "LICENSE", + "README.md", + "docs/CONTRIBUTING.md", + "docs/benchmarking-macos.md", + "docs/configuration.md", + "docs/gqa-decode.md", + "docs/installation.md", + "docs/kv_offloading.md", + "docs/pytorch-mps.md", + "docs/speculative_decoding.md", + "pyproject.toml", + "scripts/lint.sh", + "scripts/test.sh", + "setup.py", + "tests/conftest.py", + "tests/test_config.py", + "tests/test_envs.py", + "tests/test_kv_offload_budget.py", + "tests/test_kv_offload_tiering.py", + "tests/test_kv_offload_worker.py", + "tests/test_macos_defaults.py", + "tools/avg_gen_length.py", + "tools/native_sampling_probe.py", + "vllm_metal/__init__.py", + "vllm_metal/config.py", + "vllm_metal/envs.py", + "vllm_metal/metal/build.py", + "vllm_metal/metal/paged_attention.cpp", + "vllm_metal/platform.py", + "vllm_metal/v1/cache_policy.py", + "vllm_metal/v1/decode_pipeline.py", + "vllm_metal/v1/kv_offload/config.py", + "vllm_metal/v1/kv_offload/connector.py", + "vllm_metal/v1/kv_offload/fs_tier.py", + "vllm_metal/v1/kv_offload/shared_region.py", + "vllm_metal/v1/kv_offload/spec.py", + "vllm_metal/v1/kv_offload/worker.py", + "vllm_metal/v1/model_lifecycle.py" + ] + }, + { + "type": "git", + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1afa80fb9a09775ecfb2b6cc3ba850a2db6", + "paths": [ + "cli/packages/coding-agent/src/disco/skills/create-repo-skill", + "cli/packages/coding-agent/src/disco/skills/prepare-repo-skill-env", + "cli/packages/coding-agent/src/disco/skills/workflow-authoring", + "cli/packages/coding-agent/src/disco/skills/verify-repo-skill" + ], + "scope": "Compatible-agent portable source/Python partial adaptation, not DisCo CLI/model execution or managed import." + } + ], + "construction": { + "anchor_kind": "source", + "distillation_form": "repository-operating-graph", + "strategy": "arex-portable-meta-skills", + "status": "partial-candidate", + "native_import": "not-run", + "method": "Stable-ID manual author sequencing plus bounded cache author lane; integrated graph followed by assertion/helper/static and selected pure Python source tests; Apple native blocks retained." + }, + "applicability": { + "scope": "Selected plugin preparation, KV/offload ownership and speculative/deferred/dispatch evidence investigation. Common catalog scope does not certify CPU inference or cross-backend qualification.", + "prerequisites": "Access to the plugin-specific platform/ABI, cache-lifecycle and qualification contracts; actual engine changes require separate authority, resource ownership, frozen gates and cleanup, with a supported artifact pairing.", + "input_contract": "Source/package/runtime/native artifact identities, Python/OS/chip/device, backend/executor, checkpoint/tokenizer/draft identities, format/dtype, effective feature flags and workload.", + "produced_artifact": "An identity/route/invariant report with actual safe-check result, counter-signal and next authorized native check; partial CPU evidence stays separate from unverified transfers, sleep/wake, model and HTTP outcomes.", + "failure_modes": "Using lazy import success as native registration or ABI evidence, treating a feature switch as dispatch, or conflating cache capacity with physical residency and kernel time with HTTP time can overstate plugin behavior." + }, + "limitations": [ + "BLOCKED_REQUIRED_BACKEND: actual Apple MLX/Metal and MPS kernels/model loading, cache transfers/sleep/wake and real worker dispatch are not verified.", + "Installed pure API inspection and selected parser/default tests do not establish a healthy inference dependency set: pip check failed missing requirements, a FastAPI mismatch and unsupported host platform.", + "No model payload authentication, numerical/model quality, physical memory fit, HTTP service/performance or broad skill efficacy validation.", + "Detailed GGUF/AWQ/quantization/MoE, DFlash/DSpark, multimodal, diffusion, STT/pooling/LoRA and distributed paths remain bounded route evidence or explicit long-tail gaps.", + "NO_LICENSE is the actual unauthenticated gh resolver fallback, not legal absence; upstream Apache-2.0 LICENSE retained independently." + ], + "provenance_readsets": [ + "references/repo-provenance.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Resolve platform registration, backend/runner and native ABI pairing before capacity tuning; a package visibility check establishes only its reported software facts.", + "Keep MLX and MPS, full-attention and recurrent state, logical cache capacity and physical residency as separate contracts.", + "Preserve failed paths and distinguish parser/API checks from real transfers, sleep/wake, model output and HTTP performance evidence." + ] + } +--- + +# vLLM Metal inference investigation + +Use this source-derived **candidate** for the vLLM Apple Silicon plugin, not +for generic vLLM, MLX or Metal programming. Read +[source/method provenance](references/repo-provenance.md) before applying it to +a checkout. This graph is self-contained operating guidance with offline +support; it does not include an inference engine or authorize executing one. + +## Route by the contract that can fail + +| Question | Owner | +| --- | --- | +| Why does import succeed but Metal registration/build/model selection fail? | [platform-config](sub-skills/platform-config/SKILL.md) | +| Why is a setting ignored, the runner wrong, or a native ABI stale? | Same owner; selected-knob syntax helper and preparation references | +| Does host offload add RAM, which prefix is reusable, or why does wake fail? | [cache-lifecycle](sub-skills/cache-lifecycle/SKILL.md) | +| Are GDN drafts rolled back correctly, pending tokens owned safely, or GQA actually executed? | [qualification](sub-skills/qualification/SKILL.md) | +| Can a reported rate/parity result support a service or quality claim? | Same owner; frozen workload, execution-boundary and failure accounting | +| What is covered, excluded or differently licensed? | [scope and notices](references/scope-and-notices.md) | + +Upstream vLLM supplies API/scheduler/block management, mlx-lm model layers, +and this plugin the request-aware native attention/runtime integrations. +Shared upstream interfaces do not automatically qualify every combination. +The MLX path, alternative MPS path, full-attention versus recurrent state, +cache capacity versus physical residency, and kernel versus HTTP timing are +deliberately separate throughout these procedures. + +## Establish identity and authority + +1. Record exact source/package/runtime/native artifact identities, Python, + OS/chip/device, backend and executor, checkpoint/tokenizer/draft identities, + format/dtype, effective feature flags and workload. Readsets identify source + bytes and selected analysis spans, not native execution. +2. Match the intended claim to its owner. Resolve an unsupported pairing or + stale artifact before tuning capacity or quoting throughput. A feature + switch permits eligibility; it does not prove that the worker took that path. +3. Choose the smallest safe check. The bundled helpers parse explicit JSON + and compute bounded arithmetic/summaries, without native imports, downloads, + services, credentials or host mutation. No helper authenticates model data, + proves memory fit or measures inference. +4. Separate empirical facts, source reports, test definitions and proposals. + Retain unsuccessful paths and missing evidence. A successful Python import + or CPU parser test is not a native accelerator substitute. +5. For actual engine changes or experiments, establish separate authorization, + safe resource ownership, frozen correctness/quality/performance gates and + cleanup. Never erase a user KV store or install toolchains as an implicit + step in answering a diagnostic question. + +## Public preparation boundary + +The source snapshot supports Python3.12/3.13 and requires Apple Silicon/macOS15+ +for engine execution. Stable users can use the project's Homebrew tap/formula; +development source setup additionally builds native artifacts with full Xcode/ +SDK26.2+. Installation is not performed by this skill. After an authorized +installation, a minimal read-only package check is: + +```bash +python -I -c 'import importlib.metadata as m; import vllm_metal; print(m.version("vllm-metal"), vllm_metal.__version__)' +``` + +That deliberately lazy import proves only selected package visibility. Read +the platform route for exact ABI/backend/runner distinctions and failure +handling before a separately authorized startup. Do not execute remote shell +installers, upstream development scripts or model downloads from this graph. + +## Return a bounded conclusion + +Report identity, selected route, source-supported invariant, actual safe check, +result, counter-signal and smallest next authorized native check. This delivery +has partial CPU API/parser evidence only. Required Apple MLX/Metal and MPS +kernel/model, real transfers/sleep/wake, output quality, HTTP serving/performance +and broad skill efficacy remain unverified (`BLOCKED_REQUIRED_BACKEND` where +required). Source-derived candidate status is not promoted by packaging, +independent content review, a merged PR or green scaffold CI. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/scope-and-notices.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/scope-and-notices.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/scope-and-notices.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/scope-and-notices.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/cache-and-offload.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/references/cache-and-offload.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/cache-and-offload.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/references/cache-and-offload.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/helper-contract.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/references/helper-contract.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/helper-contract.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/references/helper-contract.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/scripts/screen_pool_budget.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/scripts/screen_pool_budget.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/cache-lifecycle/scripts/screen_pool_budget.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/cache-lifecycle/scripts/screen_pool_budget.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/configuration.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/references/configuration.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/configuration.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/references/configuration.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/platform-and-loading.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/references/platform-and-loading.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/platform-and-loading.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/references/platform-and-loading.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/scripts/check_environment_plan.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/scripts/check_environment_plan.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/platform-config/scripts/check_environment_plan.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/platform-config/scripts/check_environment_plan.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/dispatch-and-measurement.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/references/dispatch-and-measurement.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/dispatch-and-measurement.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/references/dispatch-and-measurement.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/record-schema.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/references/record-schema.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/record-schema.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/references/record-schema.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/speculation-and-pipeline.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/references/speculation-and-pipeline.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/speculation-and-pipeline.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/references/speculation-and-pipeline.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/scripts/summarize_serving_records.py b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/scripts/summarize_serving_records.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal/sub-skills/qualification/scripts/summarize_serving_records.py rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal/sub-skills/qualification/scripts/summarize_serving_records.py diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/references/evidence.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/references/evidence.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/references/evidence.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/references/evidence.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-architecture/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-architecture/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-architecture/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-architecture/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-architecture/references/architecture.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-architecture/references/architecture.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-architecture/references/architecture.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-architecture/references/architecture.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-serving/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-serving/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-serving/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-serving/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-serving/references/serving.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-serving/references/serving.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-serving/references/serving.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-serving/references/serving.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/references/validation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/references/validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-omni/sub-skills/omni-validation/references/validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-omni/sub-skills/omni-validation/references/validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md similarity index 62% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md index fdfb7147f..210cf8997 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md @@ -1,6 +1,87 @@ --- name: audit-recurrent-kv-finalization description: Audit vLLM-RLT synchronous early-exit KV propagation, later depth-specific attention reads, and request storage retirement. Use when shallow token exits leave deeper history or release order in question. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "Apache-2.0", + "tags": [ + "vllm-rlt", + "recurrent", + "kv-cache", + "early-exit", + "last-exited", + "shared", + "validity", + "completion", + "lifetime" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/InferMatrixLab/vllm-rlt", + "commit": "4a350344027960a48384626aa646f1765fda6d4f", + "paths": [ + "README.md", + "LICENSE", + "NOTICE", + "vllm_rlt/core/kv_cache_manager.py", + "vllm_rlt/worker/model_runner.py", + "tests/test_kv_cache.py", + "pyproject.toml", + "vllm_rlt/config.py", + "vllm_rlt/engine/stage_engine.py", + "vllm_rlt/core/scheduler.py", + "vllm_rlt/worker/executor.py", + "vllm_rlt/attention/__init__.py", + "vllm_rlt/attention/backend.py", + "vllm_rlt/models/ouro.py", + "vllm_rlt/layers/__init__.py", + "vllm_rlt/__init__.py", + "tests/conftest.py", + "vllm_rlt/request.py", + "vllm_rlt/worker/metadata.py", + "vllm_rlt/kernels/paged_attention.py", + "vllm_rlt/worker/buffers.py" + ], + "readset": "references/source-notes.md", + "identity": "references/source-identity.json", + "license": "Apache-2.0", + "third_party_notices": "references/NOTICE.md" + } + ], + "applicability": { + "scope": "Pinned vLLM-RLT native Ouro synchronous scheduling with host KVMetadata: LAST_EXITED propagation versus SHARED, later depth-specific Torch-reference attention and normal release/wait/free order. No speculation, connector, preemption, prefix sharing, split/packed prefill or graph/static-buffer path qualification.", + "prerequisites": "The selected native Ouro synchronous path uses host KVMetadata, no speculation/connector/preemption, atomic prefill and disabled graph/static-buffer/packed-prefill paths; changed execution conditions need a new bounded check.", + "input_contract": "Repository revision, model/configuration, recurrent-layer set, loop count, cache layout and block size, dtype/device, attention backend and effective flags, plus the traced token/depth/layer and its next consumer.", + "produced_artifact": "A bounded token/finalization/consumer/retirement trace separating source call order, observed data and written validity, completion evidence and remaining native model/backend/performance gaps.", + "failure_modes": "Substituting mark_finalized for K/V propagation, accepting allocated pages as a complete written prefix, applying LAST_EXITED copy expectations to SHARED, or treating event recording as completion can invalidate the ownership diagnosis.", + "refusal_point": "On a failed prerequisite or complete-prefix check, retain the first failing token/depth/layer and trace the missing operation. Stop the conclusion when flags or metadata ownership leave the selected path." + }, + "limitations": [ + "Source-derived candidate; native CPU/CUDA, model correctness/quality, performance and agent efficacy are not validated.", + "Allocated capacity, host written-position validity, copied K/V bytes, event recording and completed device work are distinct; host-row checks do not qualify DeviceKVMetadata.", + "Repository Apache-2.0 and retained model/configuration NOTICE attribution do not constitute a whole-tree or model-weight license audit." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/source-identity.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Distinguish copied K/V values, logical written positions and allocated capacity; follow the later depth-specific attention consumer.", + "Preserve executed shallower values during LAST_EXITED propagation and use SHARED as a separate one-plane control.", + "Follow the latest request event through runner release before scheduler cache release; page reuse is not evidence of erased old bytes." + ] + } --- # Audit recurrent KV finalization diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/NOTICE.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/NOTICE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/NOTICE.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/NOTICE.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md similarity index 69% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md index 678b8c6f7..c0854bdbe 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md @@ -1,6 +1,56 @@ --- name: audit-vllm-kv-lifetimes description: Audit vLLM V1 asynchronous KV load/save completion, block retention, preemption, and receive-failure contracts when requests stall or blocks may be reused too early; requires a concrete connector trace. +metadata: + inferrsi: | + { + "tags": [ + "vllm", + "kv-cache", + "lifecycle", + "correctness" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/vllm-project/vllm", + "commit": "7f730b4cea9a66520a9200265928ebe016b48a39", + "paths": [ + "vllm/distributed/kv_transfer/kv_connector/v1/base.py", + "vllm/v1/core/sched/scheduler.py" + ], + "readset": "references/source-notes.md" + } + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Pinned vLLM V1 scheduler/connector ownership audit for asynchronous KV load/save completion, retained blocks, preemption and failed receives; concrete connector conformance is not established by the interface.", + "prerequisites": "A concrete connector and worker/model-runner call trace at identified revisions, with request/block events sufficient to compare actual callers against the pinned scheduler/interface contracts.", + "input_contract": "Engine/connector revision and class, producer/consumer roles, KV-group count/block namespace, failure policy, concurrent-batch and graph modes; correlate bounded request IDs with allocation, matched tokens, load/save events, receive failures, preemption, finish and pool return.", + "produced_artifact": "An ownership/event ledger and bounded contract-conformance report naming the earliest evidenced broken obligation, owner, affected requests/blocks and smallest missing observation, with a proposed targeted regression check.", + "failure_modes": "Equating reserved tokens with usable KV, finished responses with reusable blocks, or send completion with scheduler write-fence completion can hide unsafe reuse; missing failed-receive completion can leave waits unreleased.", + "refusal_point": "Escalate absent concrete caller evidence, ambiguous block namespaces, unsupported layouts or revision drift; stop before inferring a backend fix, freeing retained blocks or claiming a reproduced bug." + }, + "recommendations": [ + "Trace lookup, allocation, worker execution and completion propagation before changing request state; repeated or undecided lookup and ordinary admission waits can be valid.", + "Check delayed-free ownership and scheduler in-flight-write fences separately, including empty-step save/metadata ordering in the real worker.", + "Keep receive failure and finished_recving correlated, and use request-level failure evidence for multi-group layouts while preserving the existing recovery policy." + ] + } --- # Audit vLLM KV lifetimes @@ -27,7 +77,7 @@ Use [diagnose-vllm-admission-pressure](../diagnose-vllm-admission-pressure/SKILL ## Paper-derived state-placement decisions -[Elastic serving placement](../../../methods/evaluate-elastic-serving-placement/SKILL.md) +[Elastic serving placement](../../../../playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) adds a separate decision about migrating request ownership or changing sequence groups. It requires source/destination reservations, catch-up progress, commit/ abort ownership and class-specific SLO costs. The diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/SKILL.md similarity index 61% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/SKILL.md index 6f8a5d8da..fcbe89a9c 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/SKILL.md @@ -1,6 +1,58 @@ --- name: diagnose-vllm-admission-pressure description: Diagnose vLLM V1 admission stalls and KV-pressure preemption by separating token, input, active-request, cache-allocation, and connector-wait constraints at a pinned scheduler revision. +metadata: + inferrsi: | + { + "tags": [ + "vllm", + "scheduler", + "admission", + "kv-cache" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/papers-20261015/cases/migration-is-not-free-memory.md", + "references/papers-20261015/cases/parallelism-change-with-state-cost.md", + "references/papers-20261015/cases/preemption-under-kv-pressure.md", + "references/papers-20261015/INTEGRATION_NOTES.md", + "references/papers-20261015/knowledge/scheduling-state-boundaries.md", + "references/papers-20261015/notes/fastserve.md", + "references/papers-20261015/notes/llumnix.md", + "references/papers-20261015/notes/loongserve.md", + "references/papers-20261015/notes/orca.md", + "references/papers-20261015/references/fastserve-readset.md", + "references/papers-20261015/references/llumnix-readset.md", + "references/papers-20261015/references/loongserve-readset.md", + "references/papers-20261015/references/orca-readset.md", + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Pinned vLLM V1 branch-based diagnosis of queued requests, low scheduled-token counts and KV-pressure preemption, separating admission/token/input/allocation/connector-wait constraints.", + "prerequisites": "The deployed scheduler must be mapped to the pinned branches, with effective configuration and a bounded request/step trace; another revision requires re-establishing that correspondence.", + "input_contract": "Deployed revision, scheduler class/configuration, connector role, KV layout, policy and speculative/encoder/concurrent-batch features; per-step queues, tokens, both budgets, allocation results, victims, transfer/free state and timing.", + "produced_artifact": "A per-step accounting ledger, observed limiting branch and competing explanation, missing instrumentation and one conditional configuration/progress hypothesis with predicted effects and trade-offs.", + "failure_modes": "Inferring memory exhaustion from a queue or high KV usage, conflating token/input budgets, ignoring pending frees or reading async token accounting as load completion can attribute admission stalls to the wrong branch.", + "refusal_point": "When required state is missing, leave the diagnosis unresolved and request the smallest trace; escalate allocation or connector behavior outside the readset and stop before live mutation or performance claims." + }, + "recommendations": [ + "Trace the first actual blocking or skip branch and record running versus admission caps and token versus input budgets separately.", + "Inspect deferred frees and request-free eligibility before blaming preemption victim selection, then reconcile restored budgets for an already-scheduled victim.", + "Reconstruct waiting admission independently, including queue precedence and async prefix states; change no setting without a branch-specific hypothesis." + ] + } --- # Diagnose vLLM admission pressure @@ -23,13 +75,13 @@ That engine graph retains `2521446ac8c92937b5a32b7dc2a84ef7fa8e99c5` Neither navigation nor shared terminology proves cross-pin compatibility or concrete worker/connector conformance. -Use [audit-vllm-kv-lifetimes](../audit-vllm-kv-lifetimes/SKILL.md) when the blocking branch involves receive completion or retained blocks. [optimization-search-policy](../../../methods/optimization-search-policy/SKILL.md) governs experiment selection, not scheduler admission. [kv-tier-management](../../lmcache/kv-tier-management/SKILL.md) covers LMCache tier policy at an independent pin; it does not establish this scheduler's behavior. +Use [audit-vllm-kv-lifetimes](../audit-vllm-kv-lifetimes/SKILL.md) when the blocking branch involves receive completion or retained blocks. [optimization-search-policy](../../../../playbooks/planning/optimization-search-policy/SKILL.md) governs experiment selection, not scheduler admission. [kv-tier-management](../../lmcache/kv-tier-management/SKILL.md) covers LMCache tier policy at an independent pin; it does not establish this scheduler's behavior. ## Paper-derived scheduling alternatives -[Serving preemption](../../../methods/assess-serving-preemption/SKILL.md) distinguishes iteration +[Serving preemption](../../../../playbooks/planning/assess-serving-preemption/SKILL.md) distinguishes iteration eligibility, service ordering and suspended-KV residency. -[Elastic placement](../../../methods/evaluate-elastic-serving-placement/SKILL.md) distinguishes +[Elastic placement](../../../../playbooks/planning/evaluate-elastic-serving-placement/SKILL.md) distinguishes replica migration from changing a sequence-parallel group. The [boundary comparison](references/papers-20261015/knowledge/scheduling-state-boundaries.md) and [hypothetical pressure case](references/papers-20261015/cases/preemption-under-kv-pressure.md) diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md similarity index 93% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md index fb0c52d06..d7534973c 100644 --- a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/INTEGRATION_NOTES.md @@ -10,8 +10,8 @@ Status: candidate. Empirical validation: not_run. Scope completed: **four papers - [FastServe notes](notes/fastserve.md) and [exact readset](references/fastserve-readset.md): service-priority preemption, starvation mitigation, predicted host-state residency, distributed coordination and same-engine attribution. - [Llumnix notes](notes/llumnix.md) and [exact readset](references/llumnix-readset.md): request relocation, virtual load/headroom, normal-class cost, drain/fill and evaluated-artifact limitations. - [LoongServe notes](notes/loongserve.md) and [exact readset](references/loongserve-readset.md): ESP state placement, paid/unpaid migration boundaries, admission/opportunity costs, batching/DoP predictor and historical baseline limitations. -- [Assess serving preemption](../../../../../methods/assess-serving-preemption/SKILL.md): one candidate justified by the coupled service-order/suspended-KV problem. It is not one generic skill per paper. -- [Evaluate elastic serving placement](../../../../../methods/evaluate-elastic-serving-placement/SKILL.md): one branched candidate justified by the choice between moving request ownership and changing execution groups. It separates state-copy and ongoing attention communication rather than treating both as free migration. +- [Assess serving preemption](../../../../../../playbooks/planning/assess-serving-preemption/SKILL.md): one candidate justified by the coupled service-order/suspended-KV problem. It is not one generic skill per paper. +- [Evaluate elastic serving placement](../../../../../../playbooks/planning/evaluate-elastic-serving-placement/SKILL.md): one branched candidate justified by the choice between moving request ownership and changing execution groups. It separates state-copy and ongoing attention communication rather than treating both as free migration. - Three **hypothetical, unexecuted candidate cases**: [preemption under KV pressure](cases/preemption-under-kv-pressure.md), [cross-instance fragmentation](cases/migration-is-not-free-memory.md), [elastic group changes](cases/parallelism-change-with-state-cost.md). They test different decision traps; they are not observed incidents or benchmark results. - One [knowledge brief](knowledge/scheduling-state-boundaries.md) connects the new mechanisms to prior work and preserves metric/fairness/state boundaries. It is not a new resource-operation matrix. diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/migration-is-not-free-memory.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/migration-is-not-free-memory.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/migration-is-not-free-memory.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/migration-is-not-free-memory.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/parallelism-change-with-state-cost.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/parallelism-change-with-state-cost.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/parallelism-change-with-state-cost.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/parallelism-change-with-state-cost.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/preemption-under-kv-pressure.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/preemption-under-kv-pressure.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/preemption-under-kv-pressure.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/cases/preemption-under-kv-pressure.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/knowledge/scheduling-state-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/fastserve.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/fastserve.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/fastserve.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/fastserve.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/llumnix.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/llumnix.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/llumnix.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/llumnix.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/loongserve.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/loongserve.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/loongserve.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/loongserve.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/orca.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/orca.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/orca.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/notes/orca.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/llumnix-readset.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/llumnix-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/llumnix-readset.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/llumnix-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/diagnose-vllm-admission-pressure/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/references/evidence-map.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/references/evidence-map.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/references/evidence-map.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/references/evidence-map.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/references/repo-provenance.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/references/repo-provenance.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/references/repo-provenance.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/references/repo-provenance.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/architecture-search-space/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/architecture-search-space/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/architecture-search-space/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/architecture-search-space/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/architecture-search-space/references/capability-boundaries.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/architecture-search-space/references/capability-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/architecture-search-space/references/capability-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/architecture-search-space/references/capability-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/architecture-search-space/references/runtime-and-invariants.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/architecture-search-space/references/runtime-and-invariants.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/architecture-search-space/references/runtime-and-invariants.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/architecture-search-space/references/runtime-and-invariants.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/extend-validate/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/extend-validate/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/extend-validate/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/extend-validate/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/extend-validate/references/change-boundaries-and-validation.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/extend-validate/references/change-boundaries-and-validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/extend-validate/references/change-boundaries-and-validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/extend-validate/references/change-boundaries-and-validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/operation-performance/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/operation-performance/SKILL.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/operation-performance/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/operation-performance/SKILL.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/operation-performance/references/commands-and-measurements.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/operation-performance/references/commands-and-measurements.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/operation-performance/references/commands-and-measurements.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/operation-performance/references/commands-and-measurements.md diff --git a/src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/operation-performance/references/experiment-decisions.md b/src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/operation-performance/references/experiment-decisions.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/repositories/vllm/sub-skills/operation-performance/references/experiment-decisions.md rename to src/inferrsi/knowledge/data/skill_library/systems/engines/vllm/sub-skills/operation-performance/references/experiment-decisions.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md similarity index 62% rename from src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md index af5298c59..8d7b875f5 100644 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md @@ -1,6 +1,64 @@ --- name: diffusers-invocation-state description: Trace Diffusers pipeline-wrapper and scheduler invocation state when repeated calls, from_pipe reuse or interruption produce unexpected schedules or retained request state; scoped to FluxPipeline and FlowMatchEulerDiscreteScheduler source evidence. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "Apache-2.0", + "tags": [ + "diffusers", + "flux", + "scheduler", + "state-ownership", + "interruption" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/huggingface/diffusers", + "commit": "aea7b70be0f594bcdc15a6f2677ee2ca85e49e8e", + "paths": [ + "src/diffusers/pipelines/pipeline_utils.py", + "src/diffusers/pipelines/flux/pipeline_flux.py", + "src/diffusers/schedulers/scheduling_flow_match_euler_discrete.py", + "tests/pipelines/flux/test_pipeline_flux.py", + "setup.py", + "LICENSE" + ], + "readset": "references/source-map.md" + } + ], + "applicability": { + "scope": "FluxPipeline and FlowMatchEulerDiscreteScheduler invocation ownership at this source pin; inspect the actual caller contract.", + "prerequisites": "Access to the pinned ownership path and an identified caller admission/serialization contract; other pipeline or scheduler classes need their own contracts.", + "input_contract": "Diffusers/PyTorch revisions, pipeline/scheduler classes and object identities, construction/from_pipe path, actual components, scheduler config and resolved timesteps/sigmas, shape/dtype, callback/interrupt timing, output/cleanup path and overlapping intervals.", + "produced_artifact": "An object-identity/ownership map and call-to-schedule-reset/step/interrupt/output/cleanup trace, the caller admission contract, one next check and actual evidence status.", + "failure_modes": "Assuming from_pipe deep-copies scheduler state, passing an enumeration index instead of a scheduler timestep, or treating cooperative iteration skipping as kernel cancellation/exception cleanup can misidentify invocation ownership.", + "refusal_point": "Stop when the ownership question is resolved or a required fixture/backend is absent; do not infer service-wide race or serialization behavior from this readset." + }, + "limitations": [ + "Source-derived candidate; no installed API, native scheduler, model, concurrency or service validation.", + "Supplied manifest verifies a selected pipeline_utils Git blob, not the whole source tree.", + "Other pipelines and schedulers require separate contracts; common indexing is not backend qualification." + ], + "provenance_readsets": [ + "references/source-map.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Check wrapper and scheduler identities explicitly and follow the effective schedule reset and step-index progression.", + "Trace interruption through its loop-boundary consumer and normal decode/cleanup tail while preserving the caller's existing shared-state serialization.", + "Use a real scheduler CPU check only for its scheduler cycle; keep pipeline reuse/interruption and serving claims tied to matching fixtures and execution evidence." + ] + } --- # Trace Diffusers invocation state diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/ownership.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/references/ownership.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/ownership.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/references/ownership.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/validation.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/references/validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/diffusers/diffusers-invocation-state/references/validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md similarity index 53% rename from src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md index 56db0306e..0b5d04eda 100644 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md @@ -1,6 +1,73 @@ --- name: lazy-evaluation-stream-lifetimes description: Diagnose MLX lazy evaluation, cross-stream ordering and array-buffer lifetimes when timing, memory use or asynchronous results disagree with expectations. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "MIT", + "tags": [ + "mlx", + "lazy-evaluation", + "stream", + "buffer-lifetime", + "synchronization" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/ml-explore/mlx", + "commit": "bff10220c742907f0c68e55ff174aaea56f74e3d", + "paths": [ + "mlx/ops.cpp", + "mlx/transforms.cpp", + "mlx/array.cpp", + "mlx/backend/cpu/eval.cpp", + "mlx/backend/metal/eval.cpp", + "mlx/backend/cuda/eval.cpp", + "python/src/transforms.cpp", + "python/tests/test_eval.py", + "LICENSE", + "mlx/scheduler.cpp", + "python/tests/test_memory.py", + "CMakeLists.txt", + "setup.py", + "python/tests/test_device.py", + "python/tests/mlx_tests.py", + ".github/workflows/build_and_test.yml" + ], + "readset": "references/source-map.md" + } + ], + "applicability": { + "scope": "MLX evaluation/stream/buffer diagnosis at the pinned implementation; qualify the actual backend separately.", + "prerequisites": "Access to the pinned MLX execution/lifetime contract and a bounded reproduction with an identified evaluation/read boundary; device-dependent conclusions need the actual selected backend.", + "input_contract": "MLX revision/build, backend/device, Python/C++ entry, array shapes/dtypes, primitive streams, evaluation/read boundary and transformation context; timing needs warmup/work boundaries, while memory needs live references, in-flight buffers, allocator cache and synchronized observations.", + "produced_artifact": "A producer/evaluation/dependency-wait/backend/availability-cleanup trace, ownership table, observed symptom and one discriminating check with build-bound actual evidence status.", + "failure_modes": "Timing graph construction or submission as execution, assuming evaluated means completed, synchronizing only the default stream, or treating graph detachment/Python deletion as reusable storage can misdiagnose time or memory.", + "refusal_point": "Stop when the trace answers the question; if the required backend is absent, retain the source-supported diagnosis and exact runtime gap rather than substituting CPU import success." + }, + "limitations": [ + "Source-derived candidate; no local native-backend, model, service or efficacy validation.", + "Static content checks do not promote this candidate or establish installed API conformance." + ], + "provenance_readsets": [ + "references/source-map.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Separate graph construction, async_eval submission, eval and host reads, and record which stream the measurement actually synchronizes.", + "Preserve the dependency-wait path and selected backend buffer-retention/donation contract before changing streams or assuming storage can be reused.", + "Distinguish live, in-flight and allocator-cached memory; inspect the real error/cleanup path before proposing a leak fix or hiding the failure." + ] + } --- # Diagnose MLX evaluation and lifetimes diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/validation.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/validation.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/validation.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/validation.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/dcp-async-save-performance/SKILL.md similarity index 73% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/dcp-async-save-performance/SKILL.md index a9b453f38..dca34b59a 100644 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/dcp-async-save-performance/SKILL.md @@ -3,6 +3,47 @@ name: dcp-async-save-performance description: 'Diagnose training-visible checkpoint pauses using PyTorch Distributed Checkpoint asynchronous saving and staging. Separate snapshot capture, staging completion, upload completion and backend durability; bound in-flight requests and cached CPU buffers before proposing overlap. Use for DCP save performance, not distributed recovery validation. ' +metadata: + inferrsi: | + { + "tags": [ + "checkpoint", + "training", + "staging", + "host-memory", + "storage", + "pytorch" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/sources.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Source-supported diagnosis of training-visible PyTorch DCP save pauses, snapshot/staging/upload lifetimes and bounded CPU staging-buffer reuse; distributed recovery validity and backend durability need separate evidence.", + "prerequisites": "Known correspondence between the installed PyTorch API and source/version notes, identified checkpointed-state mutation boundaries and an initial one-outstanding-save policy before overlap is considered.", + "input_contract": "Installed PyTorch revision, stager, storage writer and thread/process executor, checkpoint contents, ranks per host and state-mutation boundaries; performance assessment needs caller/staging/upload waits and host/resource observations.", + "produced_artifact": "A completion/lifetime diagnosis and smallest justified staging/cache/backpressure lever, with separate caller-pause, staging, upload and step-time measurements proposed for authorized validation.", + "failure_modes": "Mutating source state before staging completes, overwriting cached snapshots before prior upload completes, ignoring caller-side state_dict work or treating async return as durability can corrupt the snapshot or misstate the pause.", + "refusal_point": "Reject staging overlap that races checkpointed-state mutation; do not infer backend durability from asynchronous return or writer completion." + }, + "recommendations": [ + "Place the staging wait before the earliest mutation of any checkpointed state, including forward-mutated buffers rather than only optimizer.step.", + "Wait for the previous upload before reusing a stager's CPU storage; observe the final upload before closing caller-owned staging resources.", + "Measure checkpoint and intervening steps with the same state/cadence, retaining first-use and cached-buffer costs plus per-host memory and I/O pressure." + ] + } --- # DCP Asynchronous Save Performance diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/references/sources.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/dcp-async-save-performance/references/sources.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/dcp-async-save-performance/references/sources.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/dcp-async-save-performance/references/sources.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/LICENSE b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/LICENSE rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/SKILL.md new file mode 100644 index 000000000..ee6277ccd --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/SKILL.md @@ -0,0 +1,200 @@ +--- +name: pytorch +description: "Operate and diagnose PyTorch installation, tensors and autograd, training and data, compilation and export, accelerator lifetimes, distributed execution, checkpointing, profiling and custom operators from pinned repository evidence." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "tensor", + "autograd", + "training", + "compiler", + "distributed", + "repository-guide" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "AGENTS.md", + "LICENSE", + "NOTICE", + "README.md", + "SECURITY.md", + "aten/src/ATen/core/dispatch/DispatchKeyExtractor.h", + "aten/src/ATen/core/dispatch/Dispatcher.h", + "aten/src/ATen/native/transformers/cuda/sdp_utils.cpp", + "c10/cuda/CUDACachingAllocator.cpp", + "cmake/EnvVarForwarding.cmake", + "docs/source/distributed/checkpoint.md", + "docs/source/distributed/fsdp.fully_shard.md", + "docs/source/distributed/tensor.md", + "docs/source/func.md", + "docs/source/nested.md", + "docs/source/notes/amp_examples.md", + "docs/source/notes/autograd.md", + "docs/source/notes/cuda.md", + "docs/source/notes/ddp.md", + "docs/source/notes/extending.md", + "docs/source/notes/modules.md", + "docs/source/notes/numerical_accuracy.md", + "docs/source/notes/randomness.md", + "docs/source/notes/serialization.md", + "docs/source/onnx.md", + "docs/source/profiler.md", + "docs/source/quantization.md", + "docs/source/sparse.md", + "docs/source/user_guide/torch_compiler/torch.compiler_troubleshooting.md", + "docs/source/utils/mobile_optimizer.md", + "pyproject.toml", + "setup.py", + "test/export/test_export.py", + "test/profiler/test_profiler.py", + "test/test_autograd.py", + "test/test_cuda.py", + "test/test_custom_ops.py", + "test/test_dataloader.py", + "test/test_serialization.py", + "torch/__init__.py", + "torch/_dynamo/convert_frame.py", + "torch/_dynamo/guards.py", + "torch/_inductor/compile_fx.py", + "torch/_inductor/graph.py", + "torch/_library/custom_ops.py", + "torch/csrc/distributed/c10d/ProcessGroup.hpp", + "torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp", + "torch/distributed/checkpoint/format_utils.py", + "torch/distributed/checkpoint/state_dict_saver.py", + "torch/distributed/distributed_c10d.py", + "torch/distributed/elastic/rendezvous/c10d_rendezvous_backend.py", + "torch/distributed/rendezvous.py", + "torch/distributed/run.py", + "torch/export/__init__.py", + "torch/fx/__init__.py", + "torch/jit/__init__.py", + "torch/library.py", + "torch/nn/functional.py", + "torch/onnx/__init__.py", + "torch/optim/optimizer.py", + "torch/profiler/profiler.py", + "torch/serialization.py", + "torch/utils/benchmark/utils/timer.py", + "torch/utils/checkpoint.py", + "torch/utils/collect_env.py", + "torch/utils/cpp_extension.py", + "torch/utils/data/dataloader.py", + "version.txt" + ], + "readset": "references/source-readset.json" + } + ], + "provenance_readsets": [ + "references/source-readset.json" + ], + "references": [ + "../../../platforms/host/host-memory-management/SKILL.md", + "../dcp-async-save-performance/SKILL.md" + ], + "applicability": { + "scope": "Major user workflows and selected deep mechanisms of pinned pytorch/pytorch source; exact device/operator qualification remains task-specific.", + "prerequisites": [ + "An explicitly selected installed torch environment for runtime helpers.", + "Actual required devices, supported build and trusted ranks/storage for backend/distributed operations." + ], + "input_contract": "A concrete model/tensor/data/environment reproduction and desired operation or expected behavior.", + "produced_artifact": "A route-specific environment, tensor/gradient, graph, memory/stream, rank/state or measurement contract with checked and missing evidence.", + "failure_modes": [ + "Transferring pinned-source behavior to another release/backend without checking compatibility.", + "Confusing source inventory, CPU diagnostics and target backend execution." + ], + "refusal_point": "Withhold backend success and performance conclusions when required runtime or workload evidence is missing." + }, + "limitations": [ + "Source-derived candidate; empirical efficacy not_run.", + "Required CUDA/ROCm/MPS/XPU/external accelerator and NCCL/graph checks unavailable locally retain BLOCKED_REQUIRED_BACKEND.", + "Installed CPU wheel 2.15.0+cpu is separate from pinned source 2.16.0a0; CPU diagnostics are not same-source native verification.", + "No automatic compatibility pairing with independently pinned TorchNPU.", + "Directory inventory does not imply every file/operator/device was semantically read or run." + ], + "consumption_mode": "packaged" + } +--- + +# PyTorch operating graph + +Use this graph to plan a PyTorch workflow or isolate a source, API, numerical, +compiler, device, communication or serialization failure. It covers the major +repository workflows through eight routes. **Candidate** describes knowledge +state; source breadth does not mean every file or backend was executed. + +Read [provenance](references/repo-provenance.md) before transferring a recipe to +another revision. The pinned source is `pytorch/pytorch` at +`9a90a4fefcaafa63ac984621963a6fcc66aee68c` (`2.16.0a0` source version). +Installed-wheel observations are a distinct evidence class. Accelerator and +distributed backend qualification must use the actual target stack. + +| User task | Route | Expected artifact | +| --- | --- | --- | +| Choose a wheel/backend; build Python or LibTorch; repair import/ABI | [installation and build](sub-skills/installation-and-build/SKILL.md) | Environment/build identity and smoke result | +| Tensor layout/device/dtype, missing gradient, sparse/nested, torch.func | [tensor, dispatch and autograd](sub-skills/tensor-dispatch-autograd/SKILL.md) | Tensor/alias/gradient contract | +| nn/optim, training/eval, DataLoader, AMP, SDPA, activation recompute | [training and data](sub-skills/training-and-data/SKILL.md) | Bounded step with state and data checks | +| Dynamo guards, Inductor, FX, export, ONNX, TorchScript/mobile migration | [compiler and export](sub-skills/compiler-and-export/SKILL.md) | Graph/shape/deployment contract | +| OOM, allocated versus reserved bytes, streams, graph replay | [memory, streams and graphs](sub-skills/memory-streams-and-graphs/SKILL.md) | Storage/stream lifetime ledger | +| torchrun, c10d, DDP/FSDP2/DTensor, serialization and DCP | [distributed and checkpoint](sub-skills/distributed-and-checkpoint/SKILL.md) | Rank/collective schedule and restore plan | +| Trace/time/memory, nonfinite results, backend/compiler isolation | [profiling and debugging](sub-skills/profiling-and-debugging/SKILL.md) | Reproduction and bounded measurement | +| torch.library, FakeTensor, gradients, C++/CUDA/SYCL extension | [custom operators and extensions](sub-skills/custom-operators-and-extension/SKILL.md) | Schema, numerical and registration checks | + +Start by identifying source commit or distribution version, wheel backend tag, +Python/OS, device and driver/runtime, workload shapes/dtypes/layouts, grad mode, +state/seed and the smallest failing example. Run the bundled +[environment inspector](scripts/inspect_environment.py) in the chosen environment: + +```bash +python /scripts/inspect_environment.py +python /scripts/inspect_environment.py --require-backend cuda +``` + +`` is this skill's installed directory; helpers run from any current +directory. Inspection reports bounded software facts, never automatic package +repair, model downloads or a performance verdict. Use the installation route +for the source-build recipe and current backend prerequisites. + +For cross-route tasks, preserve the earliest incorrect boundary. A custom op +needs tensor/schema correctness before compiler eligibility; graph replay needs +stream/address lifetimes before timing; distributed resume needs allocation and +state ownership before throughput tuning. Use eager CPU as a control where it +represents the same semantics; preserve a missing device check separately. + +Cross-cutting failures are in [troubleshooting](references/troubleshooting.md). +[Repository coverage](references/source-directory-coverage.md) explicitly maps +all top-level source directories and public module families to their owners or +exclusions. [Readset](references/source-readset.json) distinguishes exact inspected +spans from tree inventory. [Script disposition](references/source-script-map.md) +explains bundled helpers and excluded source tooling. + +Existing owners remain authoritative for their focused procedures: +[host memory management](../../../platforms/host/host-memory-management/SKILL.md) +and [DCP async save performance](../dcp-async-save-performance/SKILL.md). +Their source revisions and evidence stay independent; following a link does not +update their pins or qualify the current PyTorch snapshot. + +Read [notices](references/notices.md) before redistribution. This InferRSI +package preserves the AREX operating-role, visibility and repository-license +fields in local-skill-v1 metadata. It does not imply deployment into a managed +DisCo installation. No TorchNPU/PyTorch compatibility pairing is inferred. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/notices.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/notices.md similarity index 93% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/notices.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/notices.md index 9ecdf593a..591d3ece3 100644 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/notices.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/notices.md @@ -19,6 +19,6 @@ license notices for upstream material are retained unmodified as blob bytes. AREX workflow contracts guided scope, evidence, construction and verification. InferRSI local-skill-v1 owns packaging: name/quoted description are frontmatter; -license, disable-model-invocation and disco-role live in metadata.json. Literal +license, disable-model-invocation and disco-role live in the SKILL.md metadata.inferrsi block. Literal AREX frontmatter validation therefore needs an explicitly labeled temporary projection; it cannot be described as a direct-final-tree validation pass. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/repo-provenance.md similarity index 96% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/repo-provenance.md index dcada3b7b..e9fad837e 100644 Binary files a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/repo-provenance.md and b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/repo-provenance.md differ diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-directory-coverage.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/source-directory-coverage.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-directory-coverage.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/source-directory-coverage.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-readset.json b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/source-readset.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-readset.json rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/source-readset.json diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-script-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/source-script-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/source-script-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/source-script-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/upstream-NOTICE b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/upstream-NOTICE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/references/upstream-NOTICE rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/references/upstream-NOTICE diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/scripts/inspect_environment.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/scripts/inspect_environment.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/scripts/inspect_environment.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/scripts/inspect_environment.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md new file mode 100644 index 000000000..94f5b4bd0 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/SKILL.md @@ -0,0 +1,85 @@ +--- +name: compiler-and-export +description: "Diagnose PyTorch Dynamo guards, graph breaks and Inductor lowering, or export models through FX, torch.export, ONNX and legacy deployment interfaces." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "compiler-and-export" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "torch/__init__.py", + "torch/_dynamo/convert_frame.py", + "torch/_dynamo/guards.py", + "torch/_inductor/compile_fx.py", + "torch/_inductor/graph.py", + "torch/export/__init__.py", + "torch/onnx/__init__.py", + "torch/fx/__init__.py" + ], + "readset": "../../references/source-readset.json" + } + ], + "applicability": { + "scope": "Diagnose PyTorch Dynamo guards, graph breaks and Inductor lowering, or export models through FX, torch.export, ONNX and legacy deployment interfaces.", + "prerequisites": [ + "An explicitly selected installed torch environment; actual devices for backend-specific checks." + ], + "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", + "produced_artifact": "Produce a graph and shape contract, a compiler-stage reproduction and a deployment compatibility check.", + "failure_modes": [ + "Transferring source facts to another wheel or backend without checking drift." + ], + "refusal_point": "Withhold backend success when the required device check cannot execute." + }, + "limitations": [ + "Source-supported procedures are not full-repository execution coverage.", + "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." + ], + "consumption_mode": "packaged" + } +--- + +# Compiler and export + +Use this route for the task named in the description. Produce a graph and shape contract, a compiler-stage reproduction and a deployment compatibility check. +This is a source-derived **candidate**, with native backend checks separate from +source support. Read the root [provenance](../../references/repo-provenance.md) +when package, API or platform identity differs. + +## Operating sequence + +1. Record the concrete input contract, current behavior, desired result, package + version and backend. Preserve a tiny eager or single-process control. +2. Read [workflows and mechanisms](references/workflows.md) and choose its + matching branch. Check prerequisites before launching a build or device job. +3. Apply the smallest change at the first disagreeing boundary; preserve tensor + shape, dtype, layout, device, mutation and gradient assumptions. +4. Check the predicted output and failure recovery using the linked helper or + recipe. A software check cannot establish an unavailable device capability. +5. Return the artifact named above, checks actually run, unresolved conditions + and the next discriminating check. Missing required backend execution stays + **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. + +Read [troubleshooting](references/troubleshooting.md) when a prerequisite or +check fails. The references contain complete procedures and exact source +permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/scripts/export_contract.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/scripts/export_contract.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/scripts/export_contract.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/compiler-and-export/scripts/export_contract.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/SKILL.md new file mode 100644 index 000000000..7f9da8cb1 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/SKILL.md @@ -0,0 +1,82 @@ +--- +name: custom-operators-and-extension +description: "Implement or qualify PyTorch Python custom operators, FakeTensor and autograd registrations, C++ extensions, and backend dispatch integrations." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "custom-operators-and-extension" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "torch/_library/custom_ops.py", + "torch/library.py", + "torch/utils/cpp_extension.py", + "docs/source/notes/extending.md", + "test/test_custom_ops.py" + ], + "readset": "../../references/source-readset.json" + } + ], + "applicability": { + "scope": "Implement or qualify PyTorch Python custom operators, FakeTensor and autograd registrations, C++ extensions, and backend dispatch integrations.", + "prerequisites": [ + "An explicitly selected installed torch environment; actual devices for backend-specific checks." + ], + "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", + "produced_artifact": "Produce a schema, metadata and gradient contract with separate registration, numerical and backend checks.", + "failure_modes": [ + "Transferring source facts to another wheel or backend without checking drift." + ], + "refusal_point": "Withhold backend success when the required device check cannot execute." + }, + "limitations": [ + "Source-supported procedures are not full-repository execution coverage.", + "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." + ], + "consumption_mode": "packaged" + } +--- + +# Custom operators and extension + +Use this route for the task named in the description. Produce a schema, metadata and gradient contract with separate registration, numerical and backend checks. +This is a source-derived **candidate**, with native backend checks separate from +source support. Read the root [provenance](../../references/repo-provenance.md) +when package, API or platform identity differs. + +## Operating sequence + +1. Record the concrete input contract, current behavior, desired result, package + version and backend. Preserve a tiny eager or single-process control. +2. Read [workflows and mechanisms](references/workflows.md) and choose its + matching branch. Check prerequisites before launching a build or device job. +3. Apply the smallest change at the first disagreeing boundary; preserve tensor + shape, dtype, layout, device, mutation and gradient assumptions. +4. Check the predicted output and failure recovery using the linked helper or + recipe. A software check cannot establish an unavailable device capability. +5. Return the artifact named above, checks actually run, unresolved conditions + and the next discriminating check. Missing required backend execution stays + **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. + +Read [troubleshooting](references/troubleshooting.md) when a prerequisite or +check fails. The references contain complete procedures and exact source +permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/scripts/custom_operator.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/scripts/custom_operator.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/scripts/custom_operator.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/custom-operators-and-extension/scripts/custom_operator.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md new file mode 100644 index 000000000..327af7736 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/SKILL.md @@ -0,0 +1,86 @@ +--- +name: distributed-and-checkpoint +description: "Set up or debug PyTorch c10d, DDP, FSDP2, DeviceMesh and DTensor, distributed checkpointing and trusted state serialization." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "distributed-and-checkpoint" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "torch/distributed/distributed_c10d.py", + "torch/csrc/distributed/c10d/ProcessGroup.hpp", + "torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp", + "docs/source/notes/ddp.md", + "docs/source/distributed/fsdp.fully_shard.md", + "docs/source/distributed/tensor.md", + "torch/serialization.py", + "torch/distributed/checkpoint/format_utils.py", + "SECURITY.md" + ], + "readset": "../../references/source-readset.json" + } + ], + "applicability": { + "scope": "Set up or debug PyTorch c10d, DDP, FSDP2, DeviceMesh and DTensor, distributed checkpointing and trusted state serialization.", + "prerequisites": [ + "An explicitly selected installed torch environment; actual devices for backend-specific checks." + ], + "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", + "produced_artifact": "Produce a rank and collective schedule and a state-restore contract with explicit trust and backend boundaries.", + "failure_modes": [ + "Transferring source facts to another wheel or backend without checking drift." + ], + "refusal_point": "Withhold backend success when the required device check cannot execute." + }, + "limitations": [ + "Source-supported procedures are not full-repository execution coverage.", + "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." + ], + "consumption_mode": "packaged" + } +--- + +# Distributed and checkpoint + +Use this route for the task named in the description. Produce a rank and collective schedule and a state-restore contract with explicit trust and backend boundaries. +This is a source-derived **candidate**, with native backend checks separate from +source support. Read the root [provenance](../../references/repo-provenance.md) +when package, API or platform identity differs. + +## Operating sequence + +1. Record the concrete input contract, current behavior, desired result, package + version and backend. Preserve a tiny eager or single-process control. +2. Read [workflows and mechanisms](references/workflows.md) and choose its + matching branch. Check prerequisites before launching a build or device job. +3. Apply the smallest change at the first disagreeing boundary; preserve tensor + shape, dtype, layout, device, mutation and gradient assumptions. +4. Check the predicted output and failure recovery using the linked helper or + recipe. A software check cannot establish an unavailable device capability. +5. Return the artifact named above, checks actually run, unresolved conditions + and the next discriminating check. Missing required backend execution stays + **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. + +Read [troubleshooting](references/troubleshooting.md) when a prerequisite or +check fails. The references contain complete procedures and exact source +permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/rank_contract.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/rank_contract.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/rank_contract.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/rank_contract.py diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/state_roundtrip.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/state_roundtrip.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/state_roundtrip.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/distributed-and-checkpoint/scripts/state_roundtrip.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/installation-and-build/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/installation-and-build/SKILL.md new file mode 100644 index 000000000..87edd0bbf --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/installation-and-build/SKILL.md @@ -0,0 +1,81 @@ +--- +name: installation-and-build +description: "Install, inspect, or build PyTorch for a selected OS, Python ABI and device backend; diagnose import and binary-extension mismatches." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "installation-and-build" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "README.md", + "pyproject.toml", + "setup.py", + "cmake/EnvVarForwarding.cmake" + ], + "readset": "../../references/source-readset.json" + } + ], + "applicability": { + "scope": "Install, inspect, or build PyTorch for a selected OS, Python ABI and device backend; diagnose import and binary-extension mismatches.", + "prerequisites": [ + "An explicitly selected installed torch environment; actual devices for backend-specific checks." + ], + "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", + "produced_artifact": "Build a versioned environment plan and distinguish source, wheel, toolkit and driver identities.", + "failure_modes": [ + "Transferring source facts to another wheel or backend without checking drift." + ], + "refusal_point": "Withhold backend success when the required device check cannot execute." + }, + "limitations": [ + "Source-supported procedures are not full-repository execution coverage.", + "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." + ], + "consumption_mode": "packaged" + } +--- + +# Installation and build + +Use this route for the task named in the description. Build a versioned environment plan and distinguish source, wheel, toolkit and driver identities. +This is a source-derived **candidate**, with native backend checks separate from +source support. Read the root [provenance](../../references/repo-provenance.md) +when package, API or platform identity differs. + +## Operating sequence + +1. Record the concrete input contract, current behavior, desired result, package + version and backend. Preserve a tiny eager or single-process control. +2. Read [workflows and mechanisms](references/workflows.md) and choose its + matching branch. Check prerequisites before launching a build or device job. +3. Apply the smallest change at the first disagreeing boundary; preserve tensor + shape, dtype, layout, device, mutation and gradient assumptions. +4. Check the predicted output and failure recovery using the linked helper or + recipe. A software check cannot establish an unavailable device capability. +5. Return the artifact named above, checks actually run, unresolved conditions + and the next discriminating check. Missing required backend execution stays + **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. + +Read [troubleshooting](references/troubleshooting.md) when a prerequisite or +check fails. The references contain complete procedures and exact source +permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/installation-and-build/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md new file mode 100644 index 000000000..bd1cb4680 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/SKILL.md @@ -0,0 +1,80 @@ +--- +name: memory-streams-and-graphs +description: "Diagnose PyTorch accelerator memory, CUDA caching allocation, asynchronous stream dependencies and CUDA graph replay lifetimes." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "memory-streams-and-graphs" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "docs/source/notes/cuda.md", + "c10/cuda/CUDACachingAllocator.cpp", + "test/test_cuda.py" + ], + "readset": "../../references/source-readset.json" + } + ], + "applicability": { + "scope": "Diagnose PyTorch accelerator memory, CUDA caching allocation, asynchronous stream dependencies and CUDA graph replay lifetimes.", + "prerequisites": [ + "An explicitly selected installed torch environment; actual devices for backend-specific checks." + ], + "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", + "produced_artifact": "Produce a storage and stream lifetime ledger plus a bounded allocator or graph reproduction.", + "failure_modes": [ + "Transferring source facts to another wheel or backend without checking drift." + ], + "refusal_point": "Withhold backend success when the required device check cannot execute." + }, + "limitations": [ + "Source-supported procedures are not full-repository execution coverage.", + "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." + ], + "consumption_mode": "packaged" + } +--- + +# Memory streams and graphs + +Use this route for the task named in the description. Produce a storage and stream lifetime ledger plus a bounded allocator or graph reproduction. +This is a source-derived **candidate**, with native backend checks separate from +source support. Read the root [provenance](../../references/repo-provenance.md) +when package, API or platform identity differs. + +## Operating sequence + +1. Record the concrete input contract, current behavior, desired result, package + version and backend. Preserve a tiny eager or single-process control. +2. Read [workflows and mechanisms](references/workflows.md) and choose its + matching branch. Check prerequisites before launching a build or device job. +3. Apply the smallest change at the first disagreeing boundary; preserve tensor + shape, dtype, layout, device, mutation and gradient assumptions. +4. Check the predicted output and failure recovery using the linked helper or + recipe. A software check cannot establish an unavailable device capability. +5. Return the artifact named above, checks actually run, unresolved conditions + and the next discriminating check. Missing required backend execution stays + **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. + +Read [troubleshooting](references/troubleshooting.md) when a prerequisite or +check fails. The references contain complete procedures and exact source +permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/scripts/cuda_lifetimes.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/scripts/cuda_lifetimes.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/scripts/cuda_lifetimes.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/memory-streams-and-graphs/scripts/cuda_lifetimes.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/SKILL.md new file mode 100644 index 000000000..b9ed4a842 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/SKILL.md @@ -0,0 +1,82 @@ +--- +name: profiling-and-debugging +description: "Profile PyTorch operators and memory, benchmark controlled workloads, diagnose compiler stalls, nonfinite results and asynchronous device failures." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "profiling-and-debugging" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "torch/profiler/profiler.py", + "torch/utils/benchmark/utils/timer.py", + "torch/utils/collect_env.py", + "docs/source/notes/numerical_accuracy.md", + "docs/source/notes/randomness.md" + ], + "readset": "../../references/source-readset.json" + } + ], + "applicability": { + "scope": "Profile PyTorch operators and memory, benchmark controlled workloads, diagnose compiler stalls, nonfinite results and asynchronous device failures.", + "prerequisites": [ + "An explicitly selected installed torch environment; actual devices for backend-specific checks." + ], + "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", + "produced_artifact": "Produce a reproducible trace or timing design and a claim bounded by the measured workload.", + "failure_modes": [ + "Transferring source facts to another wheel or backend without checking drift." + ], + "refusal_point": "Withhold backend success when the required device check cannot execute." + }, + "limitations": [ + "Source-supported procedures are not full-repository execution coverage.", + "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." + ], + "consumption_mode": "packaged" + } +--- + +# Profiling and debugging + +Use this route for the task named in the description. Produce a reproducible trace or timing design and a claim bounded by the measured workload. +This is a source-derived **candidate**, with native backend checks separate from +source support. Read the root [provenance](../../references/repo-provenance.md) +when package, API or platform identity differs. + +## Operating sequence + +1. Record the concrete input contract, current behavior, desired result, package + version and backend. Preserve a tiny eager or single-process control. +2. Read [workflows and mechanisms](references/workflows.md) and choose its + matching branch. Check prerequisites before launching a build or device job. +3. Apply the smallest change at the first disagreeing boundary; preserve tensor + shape, dtype, layout, device, mutation and gradient assumptions. +4. Check the predicted output and failure recovery using the linked helper or + recipe. A software check cannot establish an unavailable device capability. +5. Return the artifact named above, checks actually run, unresolved conditions + and the next discriminating check. Missing required backend execution stays + **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. + +Read [troubleshooting](references/troubleshooting.md) when a prerequisite or +check fails. The references contain complete procedures and exact source +permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/scripts/profile_step.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/scripts/profile_step.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/scripts/profile_step.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/profiling-and-debugging/scripts/profile_step.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/SKILL.md new file mode 100644 index 000000000..9a8eb8b03 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/SKILL.md @@ -0,0 +1,83 @@ +--- +name: tensor-dispatch-autograd +description: "Diagnose PyTorch tensor layouts, dtype and device dispatch, autograd history, function transforms, sparse and nested tensor contracts." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "tensor-dispatch-autograd" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "docs/source/notes/autograd.md", + "aten/src/ATen/core/dispatch/Dispatcher.h", + "aten/src/ATen/core/dispatch/DispatchKeyExtractor.h", + "docs/source/sparse.md", + "docs/source/nested.md", + "docs/source/func.md" + ], + "readset": "../../references/source-readset.json" + } + ], + "applicability": { + "scope": "Diagnose PyTorch tensor layouts, dtype and device dispatch, autograd history, function transforms, sparse and nested tensor contracts.", + "prerequisites": [ + "An explicitly selected installed torch environment; actual devices for backend-specific checks." + ], + "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", + "produced_artifact": "Produce a tensor and gradient contract, then isolate the first wrong operator or dispatch boundary.", + "failure_modes": [ + "Transferring source facts to another wheel or backend without checking drift." + ], + "refusal_point": "Withhold backend success when the required device check cannot execute." + }, + "limitations": [ + "Source-supported procedures are not full-repository execution coverage.", + "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." + ], + "consumption_mode": "packaged" + } +--- + +# Tensor dispatch autograd + +Use this route for the task named in the description. Produce a tensor and gradient contract, then isolate the first wrong operator or dispatch boundary. +This is a source-derived **candidate**, with native backend checks separate from +source support. Read the root [provenance](../../references/repo-provenance.md) +when package, API or platform identity differs. + +## Operating sequence + +1. Record the concrete input contract, current behavior, desired result, package + version and backend. Preserve a tiny eager or single-process control. +2. Read [workflows and mechanisms](references/workflows.md) and choose its + matching branch. Check prerequisites before launching a build or device job. +3. Apply the smallest change at the first disagreeing boundary; preserve tensor + shape, dtype, layout, device, mutation and gradient assumptions. +4. Check the predicted output and failure recovery using the linked helper or + recipe. A software check cannot establish an unavailable device capability. +5. Return the artifact named above, checks actually run, unresolved conditions + and the next discriminating check. Missing required backend execution stays + **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. + +Read [troubleshooting](references/troubleshooting.md) when a prerequisite or +check fails. The references contain complete procedures and exact source +permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/scripts/tensor_contracts.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/scripts/tensor_contracts.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/scripts/tensor_contracts.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/tensor-dispatch-autograd/scripts/tensor_contracts.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/SKILL.md new file mode 100644 index 000000000..c6adceb57 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/SKILL.md @@ -0,0 +1,83 @@ +--- +name: training-and-data +description: "Build or debug PyTorch modules, optimizers, data loaders, mixed precision, activation checkpointing and attention training or evaluation." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "pytorch", + "training-and-data" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "9a90a4fefcaafa63ac984621963a6fcc66aee68c", + "paths": [ + "torch/utils/data/dataloader.py", + "docs/source/notes/modules.md", + "docs/source/notes/amp_examples.md", + "torch/utils/checkpoint.py", + "torch/nn/functional.py", + "aten/src/ATen/native/transformers/cuda/sdp_utils.cpp" + ], + "readset": "../../references/source-readset.json" + } + ], + "applicability": { + "scope": "Build or debug PyTorch modules, optimizers, data loaders, mixed precision, activation checkpointing and attention training or evaluation.", + "prerequisites": [ + "An explicitly selected installed torch environment; actual devices for backend-specific checks." + ], + "input_contract": "A bounded model, tensor, data or environment reproduction and expected behavior.", + "produced_artifact": "Produce a bounded training or evaluation recipe with explicit sample, state, gradient and numerical checks.", + "failure_modes": [ + "Transferring source facts to another wheel or backend without checking drift." + ], + "refusal_point": "Withhold backend success when the required device check cannot execute." + }, + "limitations": [ + "Source-supported procedures are not full-repository execution coverage.", + "Required unavailable backends retain BLOCKED_REQUIRED_BACKEND; no measured efficacy claim." + ], + "consumption_mode": "packaged" + } +--- + +# Training and data + +Use this route for the task named in the description. Produce a bounded training or evaluation recipe with explicit sample, state, gradient and numerical checks. +This is a source-derived **candidate**, with native backend checks separate from +source support. Read the root [provenance](../../references/repo-provenance.md) +when package, API or platform identity differs. + +## Operating sequence + +1. Record the concrete input contract, current behavior, desired result, package + version and backend. Preserve a tiny eager or single-process control. +2. Read [workflows and mechanisms](references/workflows.md) and choose its + matching branch. Check prerequisites before launching a build or device job. +3. Apply the smallest change at the first disagreeing boundary; preserve tensor + shape, dtype, layout, device, mutation and gradient assumptions. +4. Check the predicted output and failure recovery using the linked helper or + recipe. A software check cannot establish an unavailable device capability. +5. Return the artifact named above, checks actually run, unresolved conditions + and the next discriminating check. Missing required backend execution stays + **BLOCKED_REQUIRED_BACKEND**; efficacy stays **not_run** without measurements. + +Read [troubleshooting](references/troubleshooting.md) when a prerequisite or +check fails. The references contain complete procedures and exact source +permalinks; the original source checkout is not a runtime prerequisite. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/references/workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/scripts/tiny_training.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/scripts/tiny_training.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/pytorch/pytorch/sub-skills/training-and-data/scripts/tiny_training.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/pytorch/pytorch/sub-skills/training-and-data/scripts/tiny_training.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/SKILL.md new file mode 100644 index 000000000..c38e286b7 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/SKILL.md @@ -0,0 +1,193 @@ +--- +name: torch-npu +description: "Operate and diagnose TorchNPU on Ascend across installation, PrivateUse1 dispatch, operator development, memory and streams, graphs and compilation, HCCL, precision and profiling, with pinned source evidence and explicit backend limits." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "references": [ + "../../../platforms/ascend/torchnpu-device-memory-lifecycle/SKILL.md", + "../../../platforms/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md" + ], + "provenance_readsets": [ + "references/source-map.md", + "references/source-manifest.json" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/Ascend/pytorch", + "commit": "046934140b96836c3fcef14d0f39ae54dc0a06bf", + "paths": [ + "COMPATIBILITY.en.md", + "README.md", + "docs/en/api/environment_variable/op_execution/ASCEND_LAUNCH_BLOCKING.md", + "docs/en/installation_guide/building_from_source.md", + "fxrt/README.md", + "generate_code.sh", + "setup.py", + "test/distributed/test_hccl_group.py", + "test/npu/test_aclgraph_multi_stream.py", + "test/npu/test_amp.py", + "test/npu/test_cpu_fallback_control.py", + "test/npu/test_import_contract.py", + "test/npu/test_stream.py", + "tools/setup_helpers/version.py", + "torch_npu/__init__.py", + "torch_npu/_inductor/__init__.py", + "torch_npu/_init/core/module_loader.py", + "torch_npu/_init/registry/backend.py", + "torch_npu/_init/registry/distributed.py", + "torch_npu/_init/registry/dynamo.py", + "torch_npu/_init/registry/registry_manager.py", + "torch_npu/contrib/transfer_to_npu.py", + "torch_npu/csrc/aten/AutoCastOps.cpp", + "torch_npu/csrc/aten/VariableFallbackKernel.cpp", + "torch_npu/csrc/core/npu/CachingHostAllocator.cpp", + "torch_npu/csrc/core/npu/NPUCachingAllocator.cpp", + "torch_npu/csrc/core/npu/NPUEvent.cpp", + "torch_npu/csrc/core/npu/NPUGraph.cpp", + "torch_npu/csrc/core/npu/NPUHooksInterface.cpp", + "torch_npu/csrc/core/npu/NPUStream.cpp", + "torch_npu/csrc/distributed/ProcessGroupHCCL.cpp", + "torch_npu/csrc/framework/utils/CpuFallbackUtils.h", + "torch_npu/csrc/profiler/profiler_mgr.cpp", + "torch_npu/distributed/tensor/__init__.py", + "torch_npu/dynamo/__init__.py", + "torch_npu/npu/__init__.py", + "torch_npu/npu/_sanitizer.py", + "torch_npu/npu/amp/autocast_mode.py", + "torch_npu/npu/amp/grad_scaler.py", + "torch_npu/npu/graphs.py", + "torch_npu/npu/streams.py", + "torch_npu/onnx/__init__.py", + "torch_npu/optim/__init__.py", + "torch_npu/profiler/profiler.py", + "torch_npu/profiler/scheduler.py", + "torch_npu/utils/serialization.py", + "torchnpugen/gen_backend_stubs.py", + "version.txt" + ], + "readset": "references/source-manifest.json" + }, + { + "type": "git", + "url": "https://github.com/Ascend/op-plugin", + "commit": "ad946817034b3270e309c81c54f0ee808d0b62cb", + "paths": [ + "examples/cpp_extension_full/torch_lib_impl/cpp_extension_full/ops.py", + "examples/cpp_extension_full/torch_lib_impl/csrc/add_custom.cpp", + "op_plugin/config/op_plugin_functions.yaml", + "op_plugin/ops/aclops/AddKernelNpu.cpp", + "op_plugin/ops/opapi/AddKernelNpuOpApi.cpp", + "torchnpugen/gen_op_plugin_functions.py" + ], + "readset": "references/source-manifest.json" + } + ] + } +--- + +# TorchNPU operating graph + +Use for Ascend/pytorch's TorchNPU adapter. Collect exact torch/torch-npu builds, +CANN/driver/hardware, the selected eager/compile/distributed mode, tensor +shape/dtype/layout and a bounded reproduction. This is a source-derived +candidate, not a validated installed stack. NPU, CANN, HCCL and optional compiler +checks remain `BLOCKED_REQUIRED_BACKEND` until exercised on the actual backend. + +Choose the narrowest route, then compose routes when the failure crosses an +ownership boundary: + +| User goal or symptom | Owning procedure | +|---|---| +| Install, select versions, missing shared library, build/package mismatch | [Installation and compatibility](sub-skills/installation-and-compatibility/SKILL.md) | +| Missing `torch.npu`, import ordering, unsupported op or hidden CPU fallback | [PrivateUse1 and dispatch](sub-skills/privateuse1-and-dispatch/SKILL.md) | +| Add an overload/custom op; fix schema, backward or Meta support | [Operator development](sub-skills/operator-development/SKILL.md) | +| Async corruption, stream waits, allocation reuse, pinned copies, memory retention | [Memory, streams and events](sub-skills/memory-streams-and-events/SKILL.md) | +| NPUGraph replay, `torch.compile`, graph breaks, compiler choice or eager fallback | [Graph and compile](sub-skills/graph-and-compile/SKILL.md) | +| Multi-rank launch, HCCL/P2P, hangs, work completion or sharding boundary | [Distributed and HCCL](sub-skills/distributed-and-hccl/SKILL.md) | +| Training migration, autocast/scaler, overflow, optimizer or checkpoint state | [Training and precision](sub-skills/training-and-precision/SKILL.md) | +| Trace collection, memory timeline, async exception or sanitizer | [Profiling and debugging](sub-skills/profiling-and-debugging/SKILL.md) | + +For migration shims, serialization/export, fused modules, RPC, DTensor/FSDP, +FXRT deployment and fault tooling, read [support workflows](references/support-workflows.md). +The [capability inventory](references/capability-inventory.md) distinguishes +substantive inspection from directory discovery and names remaining boundaries. + +## Operating contract + +1. Qualify the environment before executing a device workflow. Run + `python /scripts/inspect_environment.py` for package metadata only; + use `--device` explicitly for a tiny allocation/matmul/completion check. +2. State the relevant source mechanism and the actual last inspected boundary. + Use [the evidence ledger](references/source-map.md) and + [immutable provenance](references/repo-provenance.md) for revision alignment. +3. Establish correctness before changing precision, compilation, allocator or + overlap policy. Keep producer ordering, storage lifetime and completion + observations distinct. +4. Use [runtime smoke examples](scripts/runtime_smoke.py) only after selecting + their exact case and `--run`; the default is non-executing. The + [operator contract checker](scripts/operator_contract.py) requires only + standard-library Python and checks declarations, not a device kernel. +5. Return exact stack identities, chosen path, observations and pass/fail/not-run + scope. Restore a known-correct configuration if a bounded experiment fails. + +Expected output is an executable proposal or diagnostic trace with explicit +verification state. Reading this graph does not authorize installations, +multi-rank launches, device reset, stress tests or arbitrary upstream scripts. +Use only the operations needed for the caller's authorized task. + +## Boundaries and related owners + +Do not infer device readiness from registration, NPU coverage from CPU fallback, +memory safety from ordering alone, completed HCCL transfer from a host future, +compilation from an eager fallback, or service gains from submission timing. +CPU/helper/static checks cannot substitute for required accelerator execution. +No convergence, benchmark gain, L4 service result or candidate promotion is +established by this graph. + +Existing skill IDs `torchnpu-device-memory-lifecycle` and +`torchnpu-pinned-host-memory-lifecycle` own focused allocator tuning at their +earlier exact source pin. Metadata declares those relationships; their evidence +is not refreshed here. This graph provides newer end-to-end operating routes +and preserves both revision boundaries. + +Full-repository inventory does not mean every operator, CANN kernel, compiler +pass or device family was inspected. The pinned op-plugin readset covers real +adapter/generator/registration mechanisms; other gitlinks remain explicit +external dependencies. See [source-script decisions](references/source-script-inventory.md) +and [attribution and retained licenses](references/NOTICE.md). diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/NOTICE.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/NOTICE.md similarity index 92% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/NOTICE.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/NOTICE.md index c857a0abb..baf7a3341 100644 --- a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/NOTICE.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/NOTICE.md @@ -24,8 +24,8 @@ adapted and reorganized rather than reproduced as a guide dump. Source URLs and revisions are in [repo provenance](repo-provenance.md). The AREX exact-commit GitHub license resolver returned `NOASSERTION` for -Ascend/pytorch. Every native local-skill sidecar preserves that source value. +Ascend/pytorch. Every native local-skill entry preserves that source value. This metadata result does not erase the retained licenses or make a legal conclusion about licensing. InferRSI's local profile owns structured fields in -metadata sidecars; the external verification adapter restores AREX frontmatter +SKILL.md metadata; the external verification adapter restores AREX frontmatter only in a scratch projection. No DisCo managed import is claimed. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/capability-inventory.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/capability-inventory.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/capability-inventory.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/capability-inventory.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-LICENSE.txt b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-LICENSE.txt similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-LICENSE.txt rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-LICENSE.txt diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-THIRD-PARTY.txt b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-THIRD-PARTY.txt similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-THIRD-PARTY.txt rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/op-plugin-THIRD-PARTY.txt diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/source-identities.json b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/source-identities.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/source-identities.json rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/source-identities.json diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-LICENSE.txt b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-LICENSE.txt similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-LICENSE.txt rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-LICENSE.txt diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-THIRD-PARTY.txt b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-THIRD-PARTY.txt similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-THIRD-PARTY.txt rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-THIRD-PARTY.txt diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-docs-LICENSE.txt b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-docs-LICENSE.txt similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-docs-LICENSE.txt rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/licenses/torch-npu-docs-LICENSE.txt diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/repo-provenance.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/repo-provenance.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/repo-provenance.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/repo-provenance.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/repo-routing-metadata.json b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/repo-routing-metadata.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/repo-routing-metadata.json rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/repo-routing-metadata.json diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-manifest.json b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/source-manifest.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-manifest.json rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/source-manifest.json diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-script-inventory.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/source-script-inventory.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/source-script-inventory.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/source-script-inventory.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/support-workflows.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/support-workflows.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/references/support-workflows.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/references/support-workflows.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/inspect_environment.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/scripts/inspect_environment.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/inspect_environment.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/scripts/inspect_environment.py diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/operator_contract.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/scripts/operator_contract.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/operator_contract.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/scripts/operator_contract.py diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/runtime_smoke.py b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/scripts/runtime_smoke.py similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/scripts/runtime_smoke.py rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/scripts/runtime_smoke.py diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md new file mode 100644 index 000000000..ff2974bed --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/SKILL.md @@ -0,0 +1,70 @@ +--- +name: distributed-and-hccl +description: "Launch and diagnose TorchNPU HCCL communication, process groups, asynchronous work, stream ownership and higher-level distributed integration boundaries." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch", + "distributed-and-hccl" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "provenance_readsets": [ + "references/source-map.md" + ] + } +--- + +# Run and diagnose HCCL + +Use for NPU distributed launch, collectives/P2P, hangs, work completion, +communication-stream memory or FSDP/DTensor integration. Read [the HCCL procedure](references/hccl.md) +and [failure recovery](references/troubleshooting.md). + +Inputs: ranks/world/local ranks, one selected device per process, rendezvous, +stack/topology, collective order/tensor metadata and timeouts. Output: rank +mapping, ordered collective trace, correctness observations and exact blocked +runtime/dependency gates. + +1. Qualify one-device execution, then establish rank/device mapping. +2. Initialize HCCL with a finite timeout and identical collective sequence. +3. Distinguish producer dependency, work handle, consumer-stream dependency and + host completion. +4. Use [the bundled case](../../scripts/runtime_smoke.py) under `torchrun` + with `--case hccl --run`; it never launches additional workers itself. +5. Escalate to the named higher-level integration only after collective + correctness. Tie failures to the first differing rank/sequence. + +Single-device checks, Gloo CPU collectives and source inspection cannot clear +HCCL `BLOCKED_REQUIRED_BACKEND`. Read [memory](../memory-streams-and-events/SKILL.md) +before changing communication-memory reuse policy. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/hccl.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/hccl.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/hccl.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/hccl.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/distributed-and-hccl/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md new file mode 100644 index 000000000..d3883ded8 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/SKILL.md @@ -0,0 +1,70 @@ +--- +name: graph-and-compile +description: "Choose and debug TorchNPU manual NPUGraph, npugraphs, npugraph_ex, TorchAir, Inductor and FXRT compilation routes with explicit storage and backend gates." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch", + "graph-and-compile" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "provenance_readsets": [ + "references/source-map.md" + ] + } +--- + +# Choose capture and compilation paths + +Use for graph replay, `torch.compile`, graph breaks, stale results, compiler +dependency failures or unexpected eager fallback. Read [the graph procedure](references/graphs.md) +and [failure recovery](references/troubleshooting.md). + +Inputs: exact stack/revisions, backend name, eager function, input shapes/dtypes/ +strides, mutation and aliasing, state/RNG, capture stream and output lifetime. +Output: selected backend contract, eager-versus-compiled/replayed comparison, +capture assumptions and unresolved compiler/device checks. + +1. Establish a correct eager NPU result before capture. +2. Select manual NPUGraph, built-in graph backend, TorchAir, Inductor or FXRT + explicitly; record optional compiler availability. +3. Preserve stable storage and legal stream/capture operations. +4. Compare updated-input replays, then benchmark after warmup and synchronization. +5. Use [the bundled graph case](../../scripts/runtime_smoke.py) with + `--case graph --run` for a tiny manual replay check, not compiler conformance. + +Use [memory](../memory-streams-and-events/SKILL.md) for pool/lifetime questions +and [operator development](../operator-development/SKILL.md) for missing Meta +or custom-op registrations. TorchAir and CANN internals remain external evidence +boundaries; no backend speedup is established by this source study. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/graphs.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/graphs.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/graphs.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/graphs.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/graph-and-compile/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/SKILL.md new file mode 100644 index 000000000..359421682 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/SKILL.md @@ -0,0 +1,70 @@ +--- +name: installation-and-compatibility +description: "Select a compatible TorchNPU, PyTorch, Python, CANN and driver stack; distinguish released wheels from source builds and diagnose import failures." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch", + "installation-and-compatibility" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "provenance_readsets": [ + "references/source-map.md" + ] + } +--- + +# Install and qualify TorchNPU + +Use for a new Ascend environment, version mismatch, wheel/source build decision, +or missing CANN library. Start with [the stack procedure](references/stack.md). +Use [failure recovery](references/troubleshooting.md) before changing packages. + +Inputs: hardware family, Linux architecture, driver/firmware/CANN versions, +Python ABI, exact torch/torch-npu builds, binary versus source intent, and access +to the device. Output: one explicit compatible stack proposal, installation +commands and separate import/device observations. + +1. Separate the released package matrix from the source branch's build lines. +2. Establish driver, CANN and environment setup before package import. +3. Install matching PyTorch and TorchNPU package lines in a private environment. +4. Run [the bundled probe](../../scripts/inspect_environment.py) without device + execution, then use `--device` only on an authorized NPU host. +5. Record actual versions and all blocked checks; route missing backend + registration to [dispatch](../privateuse1-and-dispatch/SKILL.md). + +Source-derived candidate: no wheel install, source build or NPU operation was +validated by this contribution. A clean pip dependency report is insufficient +to qualify the CANN/driver/device combination. Preserve `BLOCKED_REQUIRED_BACKEND` +when the actual Ascend environment is absent. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/stack.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/stack.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/stack.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/stack.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/installation-and-compatibility/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md new file mode 100644 index 000000000..ad58ed5dd --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/SKILL.md @@ -0,0 +1,70 @@ +--- +name: memory-streams-and-events +description: "Reason about TorchNPU device and pinned-host ownership, stream dependencies, events, asynchronous copies, allocator caches and graph interaction." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch", + "memory-streams-and-events" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "provenance_readsets": [ + "references/source-map.md" + ] + } +--- + +# Trace memory and stream lifetimes + +Use for asynchronous corruption, unexpected memory retention, stream races, +pinned copy lifetime or misleading timings. Read [the ownership procedure](references/lifetimes.md) +and [failure recovery](references/troubleshooting.md). + +Inputs: allocation owner, tensor/storage aliases, allocating/using streams, +event edges, host-copy boundaries, allocator configuration and exact build. +Output: ordering diagram, ownership table, synchronized measurements and one +bounded discriminating check. + +1. Identify producer, consumer and completion boundaries. +2. Establish stream ordering and allocation lifetime separately. +3. Classify live, active, reserved and host-pinned memory independently. +4. Run [the bundled stream case](../../scripts/runtime_smoke.py) with + `--case streams --run` on the qualified NPU host. +5. Route capture to [graphs](../graph-and-compile/SKILL.md) and communication + ownership to [HCCL](../distributed-and-hccl/SKILL.md). + +The existing library owners `torchnpu-device-memory-lifecycle` and +`torchnpu-pinned-host-memory-lifecycle` retain their earlier immutable source +pins. Load them by declared metadata reference for allocator-tuning depth; do +not treat this newer operating graph as an update to their evidence. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/lifetimes.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/lifetimes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/lifetimes.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/lifetimes.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/memory-streams-and-events/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/SKILL.md new file mode 100644 index 000000000..ec292ecc6 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/SKILL.md @@ -0,0 +1,68 @@ +--- +name: operator-development +description: "Develop or diagnose TorchNPU operators across schema/code generation, PrivateUse1, autograd, Meta, ACLNN and legacy ACL adapter paths." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch", + "operator-development" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "provenance_readsets": [ + "references/source-map.md" + ] + } +--- + +# Develop and validate an NPU operator + +Use for a missing overload, custom operator, wrong dtype/shape/aliasing behavior, +or compile/backward support gap. Read [the implementation procedure](references/development.md) +and [failure recovery](references/troubleshooting.md). + +Inputs: exact schema, intended dtype/layout/device/shape range, mutability, +gradient requirements, PyTorch/TorchNPU/op-plugin/CANN revisions, target hardware +and reference outputs. Output: a schema/registration/kernel-boundary plan, +implementation checklist and correctness matrix with unverified entries. + +1. Validate the contract with [the bundled checker](../../scripts/operator_contract.py). +2. Choose built-in operator adaptation versus a namespaced extension. +3. Implement device, autograd and Meta behavior as separate responsibilities. +4. Exercise scalar, broadcast, out/in-place, dtype and layout boundaries before + compilation or graph capture. +5. Report L0 results from the actual accelerator and retain external-kernel + boundaries. Source inspection does not establish custom kernel correctness. + +Use [dispatch](../privateuse1-and-dispatch/SKILL.md) to identify the actual +entry path and [graphs](../graph-and-compile/SKILL.md) for capture constraints. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/development.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/development.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/development.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/development.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/operator-development/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/SKILL.md new file mode 100644 index 000000000..9f0bc8d81 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/SKILL.md @@ -0,0 +1,71 @@ +--- +name: privateuse1-and-dispatch +description: "Trace TorchNPU initialization, PrivateUse1 registration, lazy device setup, operator dispatch and explicit CPU fallback boundaries." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch", + "privateuse1-and-dispatch" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "provenance_readsets": [ + "references/source-map.md" + ] + } +--- + +# Trace registration and dispatch + +Use when `torch.npu` is missing, a tensor unexpectedly executes on CPU, +PrivateUse1 registration conflicts, an autograd path fails, or import loads +unexpected compiler state. Read [registration and dispatch](references/dispatch.md) +and [failure recovery](references/troubleshooting.md). + +Inputs: plugin/torch revision, fresh-process import order, autoload settings, +operator schema/overload, tensor devices/dtypes and gradient context. Output: +an initialization or dispatch trace with the selected kernel/fallback and the +first failing boundary. + +1. Locate failure in core extension, backend registration, patches, lifecycle + barrier or optional feature setup. +2. Separate renamed PrivateUse1/device methods from actual runtime initialization. +3. For one operator, identify schema, dispatch key, implementation and fallback. +4. Check forward, autograd and Meta paths independently; route missing kernels + to [operator development](../operator-development/SKILL.md). +5. Validate with a fresh interpreter and the actual NPU stack. Record CPU fallback + rather than describing a CPU result as NPU coverage. + +For broad CUDA-to-NPU migration, [the support reference](../../references/support-workflows.md) +describes the opt-in transfer shim and its process-wide effects. Importing this +skill never enables that shim. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/dispatch.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/dispatch.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/dispatch.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/dispatch.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/privateuse1-and-dispatch/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/SKILL.md new file mode 100644 index 000000000..4013e2d06 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/SKILL.md @@ -0,0 +1,69 @@ +--- +name: profiling-and-debugging +description: "Capture bounded TorchNPU CPU/NPU traces, memory timelines and synchronization-aware diagnostics; distinguish collection, parsing, completion and performance claims." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch", + "profiling-and-debugging" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "provenance_readsets": [ + "references/source-map.md" + ] + } +--- + +# Profile and debug TorchNPU + +Use for kernel/communication/host gaps, memory traces, async exceptions or stream +sanitizer diagnosis. Read [the profiling procedure](references/profiling.md) +and [failure recovery](references/troubleshooting.md). + +Inputs: tiny correct reproduction, stack/backend, selected activities, bounded +schedule, trace output location and question to discriminate. Output: trace +collection/analysis status, relevant events/counters, completion boundary and +one source-grounded hypothesis. + +1. Reproduce correctness without profiling. +2. Select CPU/NPU activities and minimal trace options. +3. Advance the schedule with `prof.step()` and verify actual artifacts. +4. For memory timelines include the required shape/memory/stack-or-module options. +5. Use [the bundled profile case](../../scripts/runtime_smoke.py) with + `--case profile --run --output ` on a qualified host. + +For async error location use a fresh bounded run with the documented launch +blocking setting, then restore the normal timing mode. Profiling overhead and +microbenchmarks do not establish L4 service performance. Device trace collection +remains `BLOCKED_REQUIRED_BACKEND` without NPU/CANN profiling support. diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/profiling.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/profiling.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/profiling.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/profiling.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/profiling-and-debugging/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/SKILL.md new file mode 100644 index 000000000..f836a18ab --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/SKILL.md @@ -0,0 +1,68 @@ +--- +name: training-and-precision +description: "Migrate a bounded TorchNPU training step, select autocast and gradient-scaling behavior, compare optimizer/checkpoint state and diagnose precision failures." +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "NOASSERTION", + "disable-model-invocation": true, + "metadata": { + "disco-role": "operating" + }, + "tags": [ + "torch-npu", + "ascend", + "pytorch", + "training-and-precision" + ], + "applicability": { + "scope": "Source-grounded TorchNPU operation and diagnosis; qualify the selected installed stack independently.", + "prerequisites": "Compatible Linux, PyTorch, TorchNPU, CANN, driver and authorized Ascend device access for device execution.", + "input_contract": "Exact stack identities, bounded workload, tensor metadata and intended correctness/measurement boundary.", + "produced_artifact": "A source-supported procedure or diagnostic trace with actual pass/fail/not-run observations.", + "failure_modes": "Confusing registration with device readiness, CPU fallback with NPU execution, or source inspection with runtime proof.", + "refusal_point": "Withhold backend correctness/performance conclusions when required device/runtime evidence is missing.", + "hardware": "Ascend NPU required for device, stream, graph, precision, profiling and HCCL checks." + }, + "limitations": [ + "Source-derived candidate; installed API and NPU/CANN execution were not validated.", + "BLOCKED_REQUIRED_BACKEND: Linux Ascend NPU/CANN stack, multi-rank HCCL and optional compiler runtime checks remain unavailable.", + "No model quality, benchmark gain, service/E2E claim or candidate promotion follows from static checks.", + "Representative pinned op-plugin inspection does not cover every operator or external CANN kernel implementation." + ], + "consumption_mode": "packaged", + "provenance_readsets": [ + "references/source-map.md" + ] + } +--- + +# Validate training and precision + +Use for a first NPU training step, autocast/GradScaler, gradient clipping, +overflow, fused optimizer migration or numerical mismatch. Read [the precision procedure](references/training.md) +and [failure recovery](references/troubleshooting.md). + +Inputs: model/loss/optimizer, input dtype/layout, gradient accumulation, scaler, +distributed mode, expected tolerances and exact stack. Output: one-step +correctness/overflow/state comparison with a named validation scope. + +1. Establish FP32 eager forward/backward and optimizer behavior. +2. Introduce autocast and scaling deliberately; check finite gradients and + skipped steps separately from loss values. +3. Unscale before clipping, and update the scale only at the optimizer boundary. +4. Run [the bundled AMP case](../../scripts/runtime_smoke.py) with + `--case amp --run` on an NPU; preserve disabled-scaler and overflow observations. +5. Test optimizer/checkpoint changes separately, then add distributed or graph + execution through their owning routes. + +A finite one-step loss is not convergence, numerical-quality, throughput or +service evidence. Custom-op gradients belong to [operator development](../operator-development/SKILL.md). diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/source-map.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/source-map.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/source-map.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/source-map.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/training.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/training.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/training.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/training.md diff --git a/src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/troubleshooting.md b/src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/troubleshooting.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/troubleshooting.md rename to src/inferrsi/knowledge/data/skill_library/systems/frameworks/torch-npu/torch-npu/sub-skills/training-and-precision/references/troubleshooting.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/README.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/README.md similarity index 79% rename from src/inferrsi/knowledge/data/skill_library/hardware/README.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/README.md index 54ed1b7e2..90697fb11 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/README.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/README.md @@ -1,8 +1,10 @@ # Hardware skill navigation The existing **22 registered owners** comprise 14 Ascend, 3 CUDA and 5 host -skills. Start with the question below, then read the selected procedure, -`metadata.json` and its source references. All 22 remain **candidate** with +skills in this source inventory. The Ascend optimization casebook now lives in +[casebooks/ascend](../../casebooks/ascend/); the other 21 entries live here. +Start with the question below, then read the selected procedure, +its frontmatter and its source references. All 22 remain **candidate** with empirical validation **not_run**. The [source inventory](source-inventory.json) describes the evidence actually @@ -42,7 +44,7 @@ measured gain, compatible combined stack or new empirical coverage. | [Model adaptation](ascend/ascend-model-adaptation/SKILL.md) | Historical vLLM-Ascend lesson documents; separate from the newer ACLGraph wrapper pin | | [Operator profiling](ascend/ascend-operator-profiling/SKILL.md) | Imported mirrored profiling procedure; named summary helper is not bundled | | [Operator tuning](ascend/ascend-operator-tuning/SKILL.md) | Mirrored PyPTO skill/playbook distillation; reported analyzer behavior and cases are not native validation | -| [Optimization casebook](ascend/ascend-optimization-casebook/SKILL.md) | Upstream skill-derived cases and inferences; no independent local reruns | +| [Optimization casebook](../../casebooks/ascend/ascend-optimization-casebook/SKILL.md) | Upstream skill-derived cases and inferences; no independent local reruns | | [Performance analysis](ascend/ascend-performance-analysis/SKILL.md) | MindIE-SD guidance plus direct CannBot workflow/callback/helper reads; callback completion is not numerical evidence | | [Video diffusion](ascend/ascend-video-diffusion-optimization/SKILL.md) | First-hand workflow/field notes, explicitly non-reproducible as published | | [AscendC tiling](ascend/ascendc-tiling-design/SKILL.md) | Mirrored skill/reference distillation; chip-specific formulas and unwritten operator classes retain their limits | @@ -64,7 +66,7 @@ measured gain, compatible combined stack or new empirical coverage. | --- | --- | | [Cgroup resource controls](host/cgroup-resource-controls/SKILL.md) | Supplied Linux interface-document research; rolling-page revision and target-system behavior remain unknown | | [Host memory and transfers](host/host-memory-management/SKILL.md) | Direct pinned PyTorch copy/allocator readset, CPython documentation and separate Ascend descriptor case; historical links retained only as lineage | -| [Host-overhead optimization](host/host-overhead-optimization/SKILL.md) | TensorRT-LLM skill import plus bounded DeepInfer code case and method-owned paper guidance; local case is linked despite an empty sidecar readset list | +| [Host-overhead optimization](host/host-overhead-optimization/SKILL.md) | TensorRT-LLM skill import plus bounded DeepInfer code case and method-owned paper guidance; local case is linked despite an empty metadata readset list | | [OS process placement](host/os-process-placement/SKILL.md) | Historical skill/configuration guidance plus a direct llama.cpp NUMA/thread readset; no optimal affinity or thread count demonstrated | | [Storage I/O measurement](host/storage-io-measurement/SKILL.md) | Supplied fio/sysstat documentation and job-example research; no tool-engine or loader implementation audit | @@ -82,10 +84,10 @@ to the 22-owner inventory. | Need | Existing owner and retained boundary | | --- | --- | -| PyTorch compiler graphs or distributed checkpoint staging | [Compiler/DCP scope routing](../README.md#pytorch-compiler-and-checkpoint-scope-routing) keeps the compiler dependency card and [DCP async-save procedure](../frameworks/pytorch/dcp-async-save-performance/SKILL.md) at separate pins. Their graph-output and staging/upload lifetimes do not substitute for the host-copy owner above. | -| FlashInfer paged-decode dispatch and graph buffers | [FlashInfer dependency card](../../knowledge_packs/dependency/flashinfer/cuda/llm/paged-decode-dispatch.json) retains a bounded README/`decode.py` extraction, imported and locally unverified. It establishes no full kernel audit or diffusion/NPU support. | -| Triton fused-softmax resource limits or kernel implementation practice | [Triton dependency card](../../knowledge_packs/dependency/triton/common/kernel/bandwidth-fusion-resource-limits.json) retains one tutorial extraction; [kernel-writing-guide](../methods/kernel-writing-guide/SKILL.md) owns the separate writing/verification procedure and caller-specific source cases. Neither is whole-compiler or native-backend qualification. | -| Apple Metal and MLX execution, caches or model paths | [MLX evaluation/lifetimes](../frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md), [vLLM Metal](../repositories/vllm-metal/SKILL.md), [Lithos Metal](../repositories/lithos-metal/SKILL.md) and [H3 Metal](../repositories/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) retain their own source and model boundaries. Unified memory does not merge host/device ownership or establish a compatible combined stack; native/model/service checks remain owner-specific. | +| PyTorch compiler graphs or distributed checkpoint staging | [Compiler/DCP scope routing](../../README.md#pytorch-compiler-and-checkpoint-scope-routing) keeps the compiler dependency card and [DCP async-save procedure](../frameworks/pytorch/dcp-async-save-performance/SKILL.md) at separate pins. Their graph-output and staging/upload lifetimes do not substitute for the host-copy owner above. | +| FlashInfer paged-decode dispatch and graph buffers | [FlashInfer dependency card](../../records/flashinfer.cuda.llm.paged_decode_dispatch.json) retains a bounded README/`decode.py` extraction, imported and locally unverified. It establishes no full kernel audit or diffusion/NPU support. | +| Triton fused-softmax resource limits or kernel implementation practice | [Triton dependency card](../../records/triton.common.kernel.bandwidth_fusion_resource_limits.json) retains one tutorial extraction; [kernel-writing-guide](../../playbooks/optimization/kernel-writing-guide/SKILL.md) owns the separate writing/verification procedure and caller-specific source cases. Neither is whole-compiler or native-backend qualification. | +| Apple Metal and MLX execution, caches or model paths | [MLX evaluation/lifetimes](../frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md), [vLLM Metal](../engines/vllm-metal/SKILL.md), [Lithos Metal](../engines/lithos-metal/SKILL.md) and [H3 Metal](../engines/h3-metal/audit-h3-metal-fasth3-semantics/SKILL.md) retain their own source and model boundaries. Unified memory does not merge host/device ownership or establish a compatible combined stack; native/model/service checks remain owner-specific. | ## Maintain this view @@ -93,5 +95,5 @@ Keep complete procedures and source ledgers with their existing owners. Update only the affected inventory row and navigation when evidence changes; preserve original inspection dates, unknowns and candidate states. Do not turn source counts into quality scores, replace the two matrices, or duplicate full -readsets here. Use the [library entry](../README.md) for discovery and packaging +readsets here. Use the [library entry](../../README.md) for discovery and packaging contracts. diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-aclgraph-capture-replay/SKILL.md similarity index 76% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-aclgraph-capture-replay/SKILL.md index 6aec75912..806429371 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-aclgraph-capture-replay/SKILL.md @@ -1,6 +1,47 @@ --- name: ascend-aclgraph-capture-replay description: Diagnose vLLM-Ascend ACLGraph capture/replay eligibility, stable-input and ordering contracts, and capture-resource failures at a pinned implementation. Use for evidence-gated graph-mode proposals, not allocator, compiler-fusion, or network tuning. +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "vllm-ascend", + "aclgraph", + "runtime", + "stream-synchronization" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Diagnose vLLM-Ascend ACLGraph eligibility, batch descriptor matching, capture/replay dispatch, caller-buffer ownership, multi-stream ordering, and capture-resource failures.", + "prerequisites": "An existing eager correctness baseline; authorization before any new run or live configuration change.", + "input_contract": "Installed stack and device, requested/effective graph mode, runner/backend settings, capture sizes and batch descriptors, error phase, replay observations and HBM/stream evidence.", + "produced_artifact": "A bounded diagnosis, smallest reversible proposal, expected tradeoff and correctness/resource verification plan with unresolved compatibility or authorization questions.", + "refusal_point": "Stop on output mismatch, unstable replay addresses, capture exhaustion, stream resource errors without capture-size reduction, unknown ordering or version drift; missing evidence permits only a conditional proposal." + }, + "recommendations": [ + "Diagnose the effective runtime mode and batch descriptor before changing capture sizes; intended warmup/profile bypass is not a cache defect.", + "Verify caller BatchDescriptor stability and dispatch mode before investigating graph replay issues.", + "For stream-resource exhaustion, reduce cudagraph_capture_sizes or max_cudagraph_capture_size.", + "Trace stable replay buffers to the owning caller and preserve synchronization/update ordering; the wrapper does not stage new inputs into persistent buffers.", + "Compare eager and graph execution on the same workload, separating initial capture, repeated replay, compiler changes and memory/stream costs; a replay log alone proves neither coverage nor speedup." + ] + } --- # Diagnose Ascend ACLGraph Capture/Replay @@ -15,7 +56,7 @@ Status: candidate; empirical validation: **not_run**. ## Related capabilities & responsibility routing -For checkpoint onboarding, FP8 loading, KV-head replication and multimodal validation order, use [ascend-model-adaptation](../ascend-model-adaptation/SKILL.md). Its startup guidance is adjacent to this wrapper contract, not evidence that the two source revisions form a validated stack. See the [version and responsibility table](../../../README.md#vllm-ascend-version-and-responsibility-routing) for their separate evidence and date boundaries. +For checkpoint onboarding, FP8 loading, KV-head replication and multimodal validation order, use [ascend-model-adaptation](../ascend-model-adaptation/SKILL.md). Its startup guidance is adjacent to this wrapper contract, not evidence that the two source revisions form a validated stack. See the [version and responsibility table](../../../../README.md#vllm-ascend-version-and-responsibility-routing) for their separate evidence and date boundaries. - **Runtime Wrapper (`ascend-aclgraph-capture-replay`)**: Handles graph pool capture, batch descriptor matching, input pointer assertions, and replay dispatch. - **Compiler Passes (`ascend-graph-mode`)**: Covers TorchAir configuration guidance; this skill adds the independently pinned vLLM-Ascend runtime wrapper. diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-aclgraph-capture-replay/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-aclgraph-capture-replay/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-device-health/SKILL.md similarity index 74% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-device-health/SKILL.md index 1a8b79e5d..51e5c1e35 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-device-health/SKILL.md @@ -3,6 +3,44 @@ name: ascend-device-health description: 'Pre-run health checks and recovery procedures for Ascend NPU devices on shared machines: card-group topology validation, Health status checks, SIGKILL residual-state detection, 4-step smoke verification, process occupancy check, and DVFS warmup discipline. Use before any experiment that requires a trusted performance baseline. ' +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "device-health", + "topology", + "shared-machine", + "os-system" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Pre-run Ascend device-health, topology, occupancy and timing checks before a trusted baseline, multi-card run or graph capture, including after a killed job.", + "hardware": "Ascend NPU card groups with topology and health visible through npu-smi.", + "input_contract": "Intended card group, current topology/health and process occupancy, prior termination history and expected workload step timing.", + "produced_artifact": "A pre-run health/occupancy and step-timing record, with recovery observations and a baseline-remeasurement requirement after a card-group change.", + "failure_modes": "Health=OK can miss degraded post-kill timing; foreign processes, unwarmed clocks or comparing different card groups can contaminate the baseline." + }, + "recommendations": [ + "Read topology each session rather than hardcoding card numbers, then check health and foreign-process occupancy before the workload smoke check.", + "Use the short step-timing check to investigate post-kill degradation; record affected cards and timings instead of treating Health=OK as sufficient evidence.", + "Warm up operator measurements and remeasure the baseline after changing card groups; set only environment variables needed by the run." + ] + } --- # Ascend Device Health and Pre-Run Verification diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/SKILL.md similarity index 76% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/SKILL.md index e1f3fcae6..87ae84ff4 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/SKILL.md @@ -3,6 +3,104 @@ name: ascend-distributed-parallel description: 'Select and validate distributed parallelism strategy on Ascend NPU: topology-aware card-group selection, USP/CP/TP/PP strategy choice, communication masking setup and verification, attribution of parallelism gains, and diagnosis of strategies that appear to activate but have no effect. Covers native HCCL multi-rank AllReduce dispatch, scratch and synchronization boundaries, AlltoAllV defect workaround, and source-scoped communication-test timing and correctness coverage. ' +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "distributed", + "parallelism", + "usp", + "hccl", + "communication", + "allreduce", + "topology-dispatch", + "collective-correctness" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/ascend-ai-coding/awesome-ascend-skills", + "commit": "75ec89ce9d401be8ac95980e1c3f42f8d4315057", + "paths": [ + "README.md", + "skills/training/hccl-test/scripts/quick-verify.sh", + "skills/training/hccl-test/scripts/parse-hccl-result.py", + "skills/training/torch-npu-comm-test/scripts/comm-bench.py", + "skills/training/torch-npu-comm-test/references/supported-ops.md" + ], + "readset": "references/awesome-ascend-communication-tests.md" + }, + { + "type": "git", + "url": "https://gitcode.com/cann/hccl", + "commit": "96fab0a2acb761e7f8ab0f94aeb5138f200a345b", + "paths": [ + "src/common/hccl_common.h", + "src/ops/all_reduce/all_reduce.cc", + "src/ops/op_common/op_common.cc", + "src/ops/op_common/selector/selector_engine.cc", + "src/ops/op_common/selector/cost_model.cc", + "src/ops/all_reduce/selector/all_reduce_auto_selector.cc", + "src/ops/all_reduce/algorithm/executor/ins_v2_all_reduce_sole_executor.cc", + "src/ops/op_common/algorithm/topo_match/topo_match_one_level.cc", + "src/ops/op_common/algorithm/template/alg_template_base.h", + "src/ops/op_common/algorithm/template/aicpu/kernel_launch.cc", + "src/ops/all_reduce/algorithm/template/aicpu/ins_temp_all_reduce_mesh_1D_one_shot.cc", + "src/ops/op_common/algorithm/template/wrapper/alg_data_trans_wrapper.cc", + "test/st/algorithm/testcase/all_reduce_testcase.cc", + "test/st/algorithm/utils/src/hccl_verifier/hccl_verifier.cc", + "test/st/algorithm/utils/src/hccl_verifier/semantics_check/allreduce_semantics_checker.cc", + "src/common/log.h", + "LICENSE" + ], + "source_channel": "official_repository", + "inspected_scope": "selected_native_implementation_and_simulator_test_definitions", + "license": "CANN Open Software License Agreement Version 2.0", + "readset": "references/hccl-allreduce-readset.json" + } + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Only the declared communication-test and native HCCL source slices are normalized; inherited strategy guidance and its source identities remain unchanged.", + "Wrapper inspection does not qualify native HCCL/TorchNPU implementations, transport algorithms, numerical correctness, backend compatibility or measured gains.", + "Native source coverage is one selected multi-rank AICPU mesh one-shot AllReduce path at the HCCL pin; legacy device routes, other engines/collectives and HCOMM transport/runtime implementations are not qualified.", + "Inspected upstream tests use simulated task-graph semantics; neither those tests nor native numerical, memory, bandwidth, model-quality or service experiments were run." + ], + "provenance_readsets": [ + "references/awesome-ascend-communication-tests.md", + "references/hccl-allreduce.md", + "references/hccl-allreduce-readset.json" + ], + "consumption_mode": "packaged", + "references": [ + "references/awesome-ascend-communication-tests.md", + "references/hccl-allreduce.md", + "references/hccl-allreduce-readset.json", + "references/hccl-LICENSE.md" + ], + "applicability": { + "scope": "Select and diagnose Ascend parallel strategies and their activation/overlap, with a separately bounded native multi-rank HCCL AllReduce dispatch and correctness procedure.", + "input_contract": "Installed collective/runtime builds, rank/device map and topology, workload/strategy settings, count/dtype/reduction, effective dispatch, per-rank outputs and completed-collective/application timing evidence.", + "produced_artifact": "A topology- and workload-qualified strategy/dispatch diagnosis, activation and correctness checks, stage-cost attribution and a target-validation proposal; isolated test completion is not a service result.", + "failure_modes": "Requested parallel settings can silently fall back; stale local-head/chunk/shard parameters can disable a path, and changed topology/card count can confound strategy comparisons." + }, + "recommendations": [ + "Confirm the final collective algorithm/engine after fallback and observe sharding/collective activation before interpreting correct outputs or tuning performance.", + "Preserve peer/thread waits and test distinct rank/position data with explicit completion on every rank; keep enqueue time, collective completion and application-stage timing separate.", + "Recompute topology-derived parameters and remeasure the anchor after changing card groups or parallel degree; inherited diffusion topology restrictions are not universal HCCL gates.", + "Require producer/consumer timeline and exposed-communication evidence for overlap; compare the real pipeline wait cost rather than extrapolating small collectives from bytes." + ] + } --- # Selecting and validating distributed parallelism diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/awesome-ascend-communication-tests.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-LICENSE.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-LICENSE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-LICENSE.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-LICENSE.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce-readset.json b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-allreduce-readset.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce-readset.json rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-allreduce-readset.json diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-allreduce.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-distributed-parallel/references/hccl-allreduce.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-distributed-parallel/references/hccl-allreduce.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/SKILL.md similarity index 69% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/SKILL.md index e10127f6a..9c59b8f5e 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/SKILL.md @@ -1,6 +1,85 @@ --- name: ascend-graph-mode description: Diagnose an Ascend TorchAir ACL graph path from versioned configuration through FX compilation, capture-key selection, eager fallback, input staging, replay and output lifetime. Use when repeated inference has host overhead, capture churn, stale outputs or an uncertain backend; require source-matched configuration and separate device validation. +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "graph-mode", + "aclgraph", + "npugraph-ex", + "torchair", + "capture-key", + "input-staging", + "output-lifetime" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Historical awesome-ascend-skills/cannbot import retains its short pin and UNKNOWN license in references/legacy-source-notes.md; it was not reread for this source slice.", + "The new official TorchAir read follows Python configuration/code generation and capture/replay ownership only; native TorchNPU/CANN, GE execution, transformations, static kernel compilation and native communication remain uninspected.", + "Selected stub and NPU smoke test definitions were read, not executed. Independent source pins do not establish stack compatibility, correctness, performance or service evidence." + ], + "provenance_readsets": [ + "references/torchair-graph-readset.json", + "references/torchair-graph-path.md", + "references/legacy-source-notes.md" + ], + "consumption_mode": "packaged", + "sources": [ + { + "type": "git", + "url": "https://github.com/Ascend/torchair", + "commit": "e1118e8b0fc1b95190ab53b732c0872989550e20", + "paths": [ + "python/torchair/configs/compiler_config.py", + "python/torchair/configs/aclgraph_config.py", + "python/torchair/configs/debug_config.py", + "python/torchair/configs/npugraphex_config.py", + "python/torchair/npu_fx_compiler.py", + "python/torchair/_acl_concrete_graph/fx2acl_converter.py", + "python/torchair/_acl_concrete_graph/acl_graph.py", + "docs/zh/npugraph_ex/quick_start.md", + "docs/zh/npugraph_ex/api/npugraph_ex/compiler_config.md", + "tests/st/torchair_aclgraph_st.py", + "tests/smoke/aclgraph_test.py", + "LICENSE" + ], + "source_channel": "official_repository", + "inspected_scope": "selected_python_implementation_documentation_and_test_definitions", + "license": "BSD-3-Clause", + "readset": "references/torchair-graph-readset.json" + } + ], + "references": [ + "references/torchair-graph-path.md", + "references/torchair-graph-readset.json", + "references/torchair-LICENSE.md", + "references/legacy-source-notes.md", + "../ascend-aclgraph-capture-replay/SKILL.md" + ], + "applicability": { + "scope": "Diagnose the TorchAir Python ACL graph path from configuration and FX compilation through capture-key selection, fallback, input staging, replay and output lifetime.", + "prerequisites": "Eager correctness established first; configuration must match the installed backend, with authorization and workload-specific tolerances for any NPU comparison.", + "input_contract": "Installed hardware/runtime versions, actual compile backend/options, shape/dtype/stride and alias/mutation contract, retained outputs, capture/replay/fallback observations and workload mix.", + "produced_artifact": "A branch-specific diagnosis and bounded configuration or lifetime proposal with correctness, cold/warm cost, resource and concurrency checks still required on the actual stack.", + "failure_modes": "Shape equality or clone_input=False does not prove capture reuse or zero-copy; capture limits can trigger persistent fallback and later replay can overwrite retained outputs." + }, + "recommendations": [ + "Use the installed CompilerConfig/backend surface and a fresh compiled instance for option changes; do not mix registered-backend snippets with callable-backend configuration.", + "Observe capture, recapture, replay and persistent fallback separately, including changed addresses and revisiting old keys after a capture-limit event.", + "Preserve earlier results before another replay and check views/input-output aliases; keep cold capture, input/output copying, optional compiler passes and service timing separate." + ] + } --- # Ascend Graph Mode: Configuration, Capture and Replay diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/legacy-source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/legacy-source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/legacy-source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/legacy-source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-LICENSE.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-LICENSE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-LICENSE.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-LICENSE.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-path.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-graph-path.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-path.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-graph-path.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-readset.json b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-graph-readset.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-graph-mode/references/torchair-graph-readset.json rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-graph-mode/references/torchair-graph-readset.json diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-model-adaptation/SKILL.md similarity index 68% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-model-adaptation/SKILL.md index cac67352d..cba519770 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-model-adaptation/SKILL.md @@ -3,6 +3,47 @@ name: ascend-model-adaptation description: 'Patterns for porting LLM/VLM model adapters to Ascend NPU: FP8 checkpoint handling, KV replication with TP, multimodal feature validation order, graph mode startup, and feature-status reporting. Source-grounded from vllm-ascend production experience. Use when adapting a new model checkpoint to run on Ascend via vllm-ascend or a similar framework. ' +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "model-adaptation", + "fp8", + "multimodal", + "vllm-ascend" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Checkpoint onboarding for LLM/VLM adapters on Ascend: weight/scale loading, KV-head replication under tensor parallelism and staged multimodal feature checks.", + "prerequisites": "Check the installed CANN and checkpoint format against the versioned guidance; final sign-off requires real weights and at least one full-layer validation.", + "input_contract": "Checkpoint weights/configuration and scale keys, attention-head/TP configuration, model modalities and expected graph/capacity features.", + "produced_artifact": "A checkpoint/feature report separating dummy and real-weight evidence, applicable feature outcomes, and theoretical versus demonstrated serving capacity.", + "failure_modes": "Dummy-only success misses scale/remapping/sharding defects; blindly sharding replicated KV heads or accepting reduced-layer results can conceal loading or full-model failures.", + "refusal_point": "Stop on missing FP8 paired scale keys, unsupported tensor parallel KV-head split without replication, unverified multimodal visual encoder output, or failed base endpoint validation." + }, + "recommendations": [ + "Gate onboarding in order: dummy architecture check, real-weight loading, model endpoint, text request and then multimodal request when the model supports VL; retain the evidence each stage actually supplies.", + "Check weight/scale pairing and each attention tensor loading path, especially when TP exceeds the KV-head count; verify historical FP8 guidance against the actual stack.", + "Expand capacity/concurrency only after basic modality and expected replay checks; mark MoE-only features inapplicable for non-MoE models and require full-layer validation before sign-off." + ] + } --- # Ascend Model Adaptation @@ -14,11 +55,11 @@ Status: candidate. Source-grounded from vllm-ascend @ `bc8115ed7fdc7e500320b5de7 This entry retains the historical model-adapter documentation pin `bc8115ed`. Use it for checkpoint onboarding, FP8 loading, KV-head replication and multimodal validation order. To prevent duplicate definitions and maintain strict attribution boundaries across repository owners, InferRSI adopts the following routing convention: -1. **Tier 1 — vLLM Generic Mechanism (`repositories/vllm/`)**: PagedAttention block allocation, KV cache indexing, preemption scheduling, worker RPC protocols. Does not assume NPU or CANN specifics. -2. **Tier 2 — vLLM Ascend Plugin Features (`hardware/ascend/` & `hardware/host/`)**: Model checkpoint weight mapping, FP8 load-time dequantization into BF16, TP-aware KV-head replication, runtime ACLGraph model runner, and native host KV offload descriptor pooling. -3. **Tier 3 — Ascend / CANN / TorchNPU Bottom Constraints (`hardware/ascend/`)**: Lower-level driver and device constraints such as `aclmdlExecute`, geExecutor, HCCL mesh scratch, TorchNPU caching allocator, H2D/D2H copy streams. +1. **Tier 1 — vLLM Generic Mechanism (`systems/engines/vllm/`)**: PagedAttention block allocation, KV cache indexing, preemption scheduling, worker RPC protocols. Does not assume NPU or CANN specifics. +2. **Tier 2 — vLLM Ascend Plugin Features (`systems/platforms/ascend/` & `systems/platforms/host/`)**: Model checkpoint weight mapping, FP8 load-time dequantization into BF16, TP-aware KV-head replication, runtime ACLGraph model runner, and native host KV offload descriptor pooling. +3. **Tier 3 — Ascend / CANN / TorchNPU Bottom Constraints (`systems/platforms/ascend/`)**: Lower-level driver and device constraints such as `aclmdlExecute`, geExecutor, HCCL mesh scratch, TorchNPU caching allocator, H2D/D2H copy streams. -For the runtime capture/replay execution contract, follow [ascend-aclgraph-capture-replay](../ascend-aclgraph-capture-replay/SKILL.md). Its wrapper evidence does not revalidate this entry or establish a compatible installed stack. For compiler pass options, follow [ascend-graph-mode](../ascend-graph-mode/SKILL.md). For host offload descriptor pooling, follow [host-memory-management](../../host/host-memory-management/SKILL.md). See the [version and responsibility table](../../../README.md#vllm-ascend-version-and-responsibility-routing) for their separate evidence boundaries. +For the runtime capture/replay execution contract, follow [ascend-aclgraph-capture-replay](../ascend-aclgraph-capture-replay/SKILL.md). Its wrapper evidence does not revalidate this entry or establish a compatible installed stack. For compiler pass options, follow [ascend-graph-mode](../ascend-graph-mode/SKILL.md). For host offload descriptor pooling, follow [host-memory-management](../../host/host-memory-management/SKILL.md). See the [version and responsibility table](../../../../README.md#vllm-ascend-version-and-responsibility-routing) for their separate evidence boundaries. ## Debug order for a new checkpoint diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-model-adaptation/references/source-notes.md similarity index 93% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-model-adaptation/references/source-notes.md index c6a7f83c7..ea46d0c5d 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-model-adaptation/references/source-notes.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-model-adaptation/references/source-notes.md @@ -14,12 +14,12 @@ Evidence origin: Historical model-adapter documentation bundle `.agents/skills/v ## Three-Tier Responsibility (InferRSI Routing Convention) To organize knowledge boundaries and prevent duplicate definitions across skills, InferRSI adopts the following routing convention: -1. **Tier 1 — vLLM Generic Mechanism (`repositories/vllm/`)**: +1. **Tier 1 — vLLM Generic Mechanism (`systems/engines/vllm/`)**: - PagedAttention block abstraction, KV cache indexing, scheduler preemption, worker RPC protocol. - Independent of Ascend or NPU specifics. -2. **Tier 2 — vLLM Ascend Plugin Features (`hardware/ascend/` & `hardware/host/`)**: +2. **Tier 2 — vLLM Ascend Plugin Features (`systems/platforms/ascend/` & `systems/platforms/host/`)**: - Model adapter checkpoint weight conversion, FP8 load-time dequantization into BF16, TP-aware GQA/MQA KV replication, ACLGraph runtime runner, and native host KV offload descriptor pooling. -3. **Tier 3 — Ascend / CANN / TorchNPU Bottom Constraints (`hardware/ascend/`)**: +3. **Tier 3 — Ascend / CANN / TorchNPU Bottom Constraints (`systems/platforms/ascend/`)**: - Lower-level CANN runtime, `aclmdlExecute`, geExecutor, HCCL mesh scratch, TorchNPU caching allocator, and H2D/D2H copy streams. ## Inspected source facts diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-operator-profiling/SKILL.md similarity index 84% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-operator-profiling/SKILL.md index 174f8fc7e..3e6916868 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-operator-profiling/SKILL.md @@ -3,6 +3,48 @@ name: ascend-operator-profiling description: 'Benchmark a single Ascend NPU operator using msprof op: build, collect with warmup, read the 8 CSV metrics files, derive a bottleneck verdict, and iterate. Use when validating operator performance in isolation before system-level profiling. ' +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "operator", + "profiling", + "msprof", + "benchmark" + ], + "status": "candidate", + "synced-from": "https://gitcode.com/cann/cannbot-skills external/cannbot/ops/ops-profiling", + "synced-date": "2026-05-26", + "synced-commit": "ac5bbd2b4cf427d011874e11f8d1e8b1bef66eda", + "license": "UNKNOWN", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Build, measure and diagnose one Ascend operator in isolation using msprof op, before interpreting system-level or multi-kernel performance.", + "prerequisites": "A built executable or test harness and access to real NPU hardware with msprof op for collection.", + "input_contract": "Operator executable/harness, collection settings and OPPROF output directory, plus the operator directory used to archive each measurement round.", + "produced_artifact": "Per-round raw CSV archives and observational min/average/max summaries, followed by a bottleneck interpretation and comparison of operator duration and core balance.", + "failure_modes": "Unwarmed or reduced-frequency runs distort timing; separately budgeting simultaneous GM reads/writes misses shared bandwidth, and tiny workloads can be dominated by launch overhead." + }, + "recommendations": [ + "Collect warmup and repeated measurements into a named round, preserve the raw CSVs, and distinguish the summary statistics from the subsequent bottleneck judgment.", + "Compare task duration, dominant pipeline ratios and per-core balance together; change one optimization point, rebuild as needed and archive the new round before deciding to keep it.", + "Check current versus rated frequency and shared read/write bandwidth before blaming the kernel; aggregate msprof ratios do not provide an instruction-by-instruction timeline." + ] + } --- # Ascend Operator-Level Profiling diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-operator-tuning/SKILL.md similarity index 83% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-operator-tuning/SKILL.md index fc5ff3cd9..964804994 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-operator-tuning/SKILL.md @@ -3,6 +3,46 @@ name: ascend-operator-tuning description: 'Run a staged, accuracy-gated tuning campaign on a single Ascend NPU operator: set a numeric target up front, block tuning until an on-device accuracy pass is logged, escalate code-level then scheduling then instruction-level tuning with per-stage stall counters and kill-switches, and route each change by the metric contract (core utilization, bubble rate, load balance). Use after ascend-operator-profiling has measured a baseline and the operator is confirmed as the bottleneck. ' +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "operator", + "tuning", + "performance", + "pypto", + "tile" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Run a bounded, accuracy-gated tuning campaign for a single Ascend operator already identified as the bottleneck, escalating from code-level to scheduling and instruction-level changes.", + "prerequisites": "A measured operator baseline, an explicit numeric target and a logged on-device accuracy pass before tuning; performance comparisons use real hardware.", + "input_contract": "Operator implementation, fixed shape/dtype/card and warmup policy, accuracy result, per-core work/wait measurements and round logs.", + "produced_artifact": "A bounded campaign record of each change, accuracy and measured timing delta, kept/reverted rounds, current stage and documented stop condition.", + "refusal_point": "Do not start without the baseline and accuracy gate; revert accuracy failures, and stop/escalate repeated failures, timeouts, exhausted stages or the campaign budget.", + "failure_modes": "Scheduler-idle and predecessor-wait bottlenecks need different fixes; output inplace-reshape can corrupt slice writes, and a changed analyzer log format can produce a misleading zero-core report." + }, + "recommendations": [ + "Change one parameter and recheck accuracy each round; keep the failure record when reverting and compare the final configuration against the same baseline workload and warmup policy.", + "Route scheduler bubbles separately from predecessor dependencies and enter instruction-level tuning only after isolating a specific long task.", + "Verify the analyzer parsed real cores before trusting its diagnosis, and rederive PyPTO tile/buffer/scheduling settings for the target SoC rather than copying numeric examples." + ] + } --- # Ascend Operator-Level Tuning diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/SKILL.md similarity index 70% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/SKILL.md index 51e6096b5..d55c02fff 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/SKILL.md @@ -3,6 +3,76 @@ name: ascend-performance-analysis description: 'Diagnose performance bottlenecks on Ascend NPU systems using CANN profiling data. Maps GPU profiling concepts (nsys, ncu, SOL%, roofline) to their NPU equivalents (msprof, torch_npu.profiler, CSV/JSON outputs, AI Core utilization, HCCL communication analysis). Covers measurement discipline on shared NPU machines and evidence acceptance for automated NPU optimization stages. Use when profiling data is available or a stage report needs its execution and validation evidence checked. ' +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "profiling", + "performance", + "cann" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://gitcode.com/cann/cannbot", + "commit": "c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110", + "paths": [ + "LICENSE", + "README.md", + ".gitmodules", + "plugins-official/LICENSE", + "plugins-official/README.md", + "plugins-official/model-infer-optimize/.claude-plugin/plugin.json", + "plugins-official/model-infer-optimize/.codex-plugin/plugin.json", + "plugins-official/model-infer-optimize/AGENTS.md", + "plugins-official/model-infer-optimize/plugin-sources.json", + "plugins-official/model-infer-optimize/plugin-install.json", + "plugins-official/model-infer-optimize/workflows/optimize-workflow.md", + "plugins-official/model-infer-optimize/agents/model-infer-analyzer.md", + "plugins-official/model-infer-optimize/agents/model-infer-implementer.md", + "plugins-official/model-infer-optimize/agents/model-infer-reviewer.md", + "plugins-official/model-infer-optimize/hooks/hooks.json", + "plugins-official/model-infer-optimize/hooks/pre_tool_use.py", + "plugins-official/model-infer-optimize/hooks/post_tool_use.py", + "plugins-official/model-infer-optimize/hooks/subagent_stop.py", + "plugins-official/model-infer-optimize/workflows/scripts/archive_progress.py" + ], + "readset": "references/cannbot-stage-evidence.md" + } + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets.", + "Only the incremental canonical CannBot fixed-stage evidence boundary is normalized; inherited MindIE-SD guidance remains unchanged.", + "CannBot callback predicates do not establish numerical accuracy or performance; dependent skills, agent-host integration and NPU execution are unverified." + ], + "provenance_readsets": [ + "references/cannbot-stage-evidence.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Interpret Ascend CANN profiling evidence from step breakdown through kernel/utilization/communication analysis, and assess execution/validator evidence in automated optimization-stage reports.", + "prerequisites": "Usable warm-state captures with representative steps, required files and non-null utilization columns; healthy cards and a consistent valid topology for multi-card comparisons.", + "input_contract": "CANN step/kernel/utilization/communication outputs with verified column semantics; for stage acceptance, the loaded candidate, baseline, quality-validator output and comparable measurements.", + "produced_artifact": "An evidence-qualified bottleneck classification and optimization direction; stage conclusions retain missing-execution/validator INCONCLUSIVE or observed gate-failure FAIL.", + "failure_modes": "Summing concurrent communication durations double-counts overlap; misidentified columns can drop communication, and a successful export can still lack usable utilization data." + }, + "recommendations": [ + "Start from the step-level computing/communication/overlap/free breakdown and stop deepening the analysis once the bottleneck is located; do not sum concurrent streams as wall time.", + "Inspect actual column headers and enum values on the installed CANN version, then verify files and non-null metrics before interpreting utilization heuristics.", + "Use same-window interleaved baseline/candidate measurements on shared machines; progress markers, hashes and source-derived thresholds do not substitute for quality and execution evidence." + ] + } --- # Ascend NPU Performance Analysis diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-CANN-2.0-LICENSE.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/references/cannbot-CANN-2.0-LICENSE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-CANN-2.0-LICENSE.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/references/cannbot-CANN-2.0-LICENSE.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md diff --git a/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-video-diffusion-optimization/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-video-diffusion-optimization/SKILL.md new file mode 100644 index 000000000..298f25d57 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-video-diffusion-optimization/SKILL.md @@ -0,0 +1,86 @@ +--- +name: ascend-video-diffusion-optimization +description: 'Run and verify goal-driven Ascend video diffusion optimization. Specializes the backend-agnostic video-diffusion-optimization skeleton with Ascend NPU operator, layout and profiler references. Use when the user asks for latency, RTF, throughput, or HBM targets across VLM, DiT, MoE, attention, or VAE on Ascend NPU. Not for text-only LLMs.' +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "status": "candidate", + "origin": [ + "first-party" + ], + "tags": [ + "ascend", + "npu", + "video-diffusion", + "dit", + "moe", + "attention", + "vae", + "performance", + "profiling" + ], + "empirical_validation": "not_run", + "verification": { + "locally_verified": false, + "evidence_level": "guidance", + "note": "Procedure is first-hand workflow guidance. The linked field notes are not reproducible as published. No acceptance campaign under this repository's evidence contract backs this skill, and no public performance claim is made." + }, + "applicability": { + "scope": "Ascend NPU video diffusion inference and serving optimization", + "hardware": "Ascend NPU; the measure-change-verify workflow skeleton is backend-agnostic and lives in playbooks/optimization/video-diffusion-optimization, while the operator, layout and profiler references here are Ascend-specific", + "prerequisites": "A reproducible benchmark, a frozen quality check, and numeric gates defined in the governing acceptance issue", + "input_contract": "Issue-scoped performance and quality gates, timing boundary, representative and held-out workloads, frozen model/runtime/input settings and an authorized optimization budget.", + "produced_artifact": "An experiment ledger and final report of the frozen goal, baseline/final samples, quality/resource checks, accepted or reverted changes and remaining gap; verdicts require the acceptance issue evidence.", + "failure_modes": "Packed-layout conversion from noncontiguous weights can corrupt outputs; missing side-stream lifetime/order handling or omitted layout-copy costs can invalidate an optimization comparison." + }, + "limitations": [ + "Ascend specialization of the backend-agnostic playbooks/optimization/video-diffusion-optimization skeleton; every operator, layout and profiler reference (TransData, FRACTAL_NZ, CANN) is Ascend-specific and must be re-derived per backend.", + "Candidate workflow guidance; not independently validated beyond the field notes.", + "Field notes are non-reproducible as published and must not be cited as evidence.", + "Model identifiers, campaign revisions, and numeric gates are intentionally excluded and remain issue-scoped.", + "Remaining figures are order-of-magnitude recollections." + ], + "provenance_readsets": [ + "references/field-notes-video-diffusion.md" + ], + "references": [ + "../../../../playbooks/optimization/video-diffusion-optimization/SKILL.md", + "../../../../playbooks/optimization/video-diffusion-optimization/scripts/evaluate_target.py", + "../../../../playbooks/optimization/video-diffusion-optimization/assets/campaign.template.json" + ], + "consumption_mode": "packaged", + "recommendations": [ + "Follow the common goal-contract and measure-change-verify loop, including final restarted-artifact acceptance; an operator win does not establish end-to-end improvement.", + "Materialize contiguous weights before FRACTAL_NZ conversion when strides differ from physical order, and attribute TransData/copy costs before proposing fusion.", + "Use field notes only to form hypotheses; derive thresholds, chunking, precision and sparsity from the target model measurements, and preserve stream dependencies and tensor lifetimes." + ] + } +--- + +# Ascend Video Diffusion Optimization + +Status: candidate workflow guidance, Ascend-specific. The procedure is first-hand and has not been independently validated beyond the [field notes](references/field-notes-video-diffusion.md). Keep model identifiers, campaign revisions, and numeric gates out of this file; they belong to an acceptance issue and its frozen evidence. + +This skill is the Ascend specialization of the backend-agnostic [video diffusion optimization skeleton](../../../../playbooks/optimization/video-diffusion-optimization/SKILL.md). Follow that skeleton for the goal contract, baseline discipline, optimization loop, candidate-selection strategy, correctness protection, final acceptance, and final report. This file adds the Ascend-specific operator, layout, and profiler references; it does not repeat the full procedure. + +Own the performance objective through implementation and final verification. Do not stop after profiling, proposing changes, or showing a faster microbenchmark. Continue the measure-change-verify loop until the target is met, the authorized budget is exhausted, or an external blocker prevents execution. + +Read the field notes only for prior failures and starting hypotheses. Treat every observation there as untraceable and non-reproducible. Derive thresholds, chunk sizes, precision policy, and sparsity from the target model's own measurements. + +Verify every stage against the packaged [benchmark design](../../../../playbooks/measurement/inference-benchmark-design/SKILL.md) and [evidence discipline](../../../../workflows/governance/rsi-evidence/SKILL.md). A microbenchmark or operator parity check is not end-to-end evidence. + +## Ascend-Specific Layout And Profiler References + +Where the common skeleton says "operator names, memory layouts, and profiler references are backend-specific", apply these Ascend facts: + +- `FRACTAL_NZ` is the Ascend packed weight layout. Materialize contiguous weights before converting to `FRACTAL_NZ` when source strides can differ from physical order; a physical-layout mismatch corrupts output. +- `TransData` performs layout conversion between formats. Attribute `TransData` and layout-copy cost before designing fusion or format-preserving wrappers. +- Profiling and stream instrumentation follow `CANN` conventions. Establish producer-consumer events, record tensor lifetimes on side streams, and join streams before their first common consumer. +- Sparse-attention and quantization candidate selection remain backend-agnostic decisions; only the kernel names, packed-layout materialization, and profiler attributions are Ascend-specific. + +For the candidate-selection guidance (condition/scheduling, DiT/MoE/quantization, attention, VAE/long-video), follow the common skeleton and read the [field notes](references/field-notes-video-diffusion.md) for the Ascend-specific failures to avoid. + +The common skeleton's `scripts/evaluate_target.py` and `assets/campaign.template.json` are shared and backend-agnostic; use them as documented there. diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/references/field-notes-video-diffusion.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-video-diffusion-optimization/references/field-notes-video-diffusion.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/references/field-notes-video-diffusion.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascend-video-diffusion-optimization/references/field-notes-video-diffusion.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascendc-tiling-design/SKILL.md similarity index 85% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascendc-tiling-design/SKILL.md index c1ad29bd3..d41021292 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/ascendc-tiling-design/SKILL.md @@ -3,6 +3,44 @@ name: ascendc-tiling-design description: 'Design tiling for a custom AscendC operator: normalize the shape first (axis-collapse or flatten), answer the fullload-vs-split question for UB, apply the universal four-element template (multi-core split, UB split, buffer plan, branch matrix), and route per operator class (reduction, broadcast, elementwise, matmul, conversion) using algorithm-selection and chip-generation tables (DAV_2201 vs DAV_3510). Use when writing or restructuring an AscendC operator''s tiling logic, before running any performance tuning campaign. ' +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "npu", + "ascendc", + "tiling", + "operator", + "kernel", + "ub" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Design or restructure AscendC tiling for the covered reduction, broadcast and elementwise classes, plus the bounded mxfp8 fusion and small-channel transpose cases.", + "input_contract": "Operator class and semantics, normalized shapes/axes, dtype, target chip generation, valid/aligned lengths and intermediate-buffer requirements.", + "produced_artifact": "A multi-core split, UB chunk formula, buffer-size/total-usage plan and branch matrix with edge tests; these are design outputs, not benchmark results.", + "failure_modes": "Budgeting only inputs omits intermediate UB storage; mixing valid and aligned lengths or copy units corrupts boundaries, and choosing unsupported generation-specific primitives invalidates the design." + }, + "recommendations": [ + "Normalize axes first, then decide full-load versus chunking from the entire live working set; derive tile size from buffer, axis, API and load-balance constraints.", + "Keep valid compute/copy lengths separate from aligned storage offsets and sizes, and complete dtype, tail/alignment and boundary branches before tuning.", + "Select primitives by the target generation and preserve class-specific coverage limits; re-evaluate sample constants and API semantics instead of treating formulas as tested performance evidence." + ] + } --- # AscendC Tiling Design diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/cann-attribution-overlap/SKILL.md similarity index 78% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/cann-attribution-overlap/SKILL.md index 2856b4c4b..20975949b 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/cann-attribution-overlap/SKILL.md @@ -3,6 +3,46 @@ name: cann-attribution-overlap description: 'Attribute NPU trace kernels to model source and plan bounded multistream overlap: select a representative step, map kernels to source owners via code_ref, build the module/operator dependency DAG (data, state, event, collective-order edges), classify serial/candidate/pending groups, choose minimal-first overlap with an explicit execution-mode synchronization contract, and accept only through correctness-first checks. Keeps the !302 import lineage distinct from later revisions, carries known formula/threshold conflicts as unresolved, and qualifies memory evidence by domain (device HBM/UB counters are not host RAM). Planning procedure only — no formula automation for disputed definitions. ' +metadata: + inferrsi: | + { + "tags": [ + "cann", + "npu", + "attribution", + "dependency-dag", + "multistream", + "overlap", + "trace" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Attribute representative CANN trace kernels to source owners and plan bounded multistream overlap using data, state, event and collective-order dependencies.", + "prerequisites": "Pin the installed CANN/TorchNPU/TorchAir versions before choosing one execution-mode synchronization API family; communication proposals require communication evidence.", + "input_contract": "Representative-step selection and shapes, kernel/stream inventory with code_ref, adjacent wait windows and dependency edges; communication analysis also needs rank/transport evidence.", + "produced_artifact": "An attribution map, serial/candidate/pending dependency groups and a minimal overlap plan with joins, enable/fallback path, resource tradeoffs and acceptance checks.", + "failure_modes": "Kernel names and mechanical coverage can misidentify ownership; logical independence can hide resource contention, and overlap ratios or summed durations can hide unchanged wall time or new tails." + }, + "recommendations": [ + "Join hotspots to source owners and adjacent wait windows before selecting a target; keep compute cost, waiting and host gaps separate.", + "Build explicit data/state/event/collective-order edges, then verify one correct overlap point before increasing overlap or partitioning resources.", + "Require observed overlap, preserved state/order and fallback behavior plus latency and outside-module checks; report wall time, interval unions and duration sums separately.", + "Carry disputed formulas and source-revision distinctions forward; device free time alone does not identify Python, GIL or OS causes without matching host evidence." + ] + } --- # CANN Attribution and Overlap Planning diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-device-memory-lifecycle/SKILL.md similarity index 73% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-device-memory-lifecycle/SKILL.md index 13baba1e6..9008f67b8 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-device-memory-lifecycle/SKILL.md @@ -1,6 +1,44 @@ --- name: torchnpu-device-memory-lifecycle description: Diagnose TorchNPU device-HBM reuse, cross-stream lifetimes, cache retention and fragmentation, then propose version-gated allocator changes. Use for NPU allocator OOM or reuse investigations, not pinned-host RAM or network transport. +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "torchnpu", + "device-memory", + "allocator", + "lifecycle" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Diagnose TorchNPU device-HBM reuse, stream lifetimes, fragmentation and graph/private-pool retention before proposing allocator changes.", + "input_contract": "Allocation failure/size sequence, device and allocator path, installed versions/settings, stream and graph-pool timeline, retained tensor references and allocator statistics with synchronization boundaries.", + "produced_artifact": "A suspected allocator state transition with source evidence IDs, one eligible action, resource cost, counter-signals and a correctness-first verification/rollback plan.", + "refusal_point": "Missing evidence supports only a conditional proposal, not a diagnosed leak; stop when external/custom/shared allocation ownership is untraced and keep baseline settings for unsupported versions or live-tensor capacity pressure." + }, + "recommendations": [ + "Separate stream lifetime protection from producer/consumer ordering, and distinguish live tensors, pending events, inactive splits and private-pool retention before tuning.", + "Match split or expandable-segment proposals to the observed allocation pattern and actual feature gates; do not combine incompatible controls or import comment thresholds as defaults.", + "Treat cache release as synchronization/allocation work at a quiescent boundary; compare identical allocation sequences in fresh processes at the same synchronization points." + ] + } --- # TorchNPU device-memory lifecycle diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-device-memory-lifecycle/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-device-memory-lifecycle/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md similarity index 74% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md index d03f1502b..06b4bc556 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md @@ -1,6 +1,44 @@ --- name: torchnpu-pinned-host-memory-lifecycle description: Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy lifetime, reuse and cache controls, then propose version-gated host-allocator changes. Use for pin_memory pressure or host allocation stalls, not device HBM or network transport. +metadata: + inferrsi: | + { + "tags": [ + "ascend", + "torchnpu", + "host-memory", + "pinned-memory", + "lifecycle" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/source-notes.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Diagnose TorchNPU pinned-host RAM allocation, asynchronous-copy ownership/readiness, reuse and path-specific cache controls.", + "input_contract": "Installed versions/settings, host pressure and allocation/reuse distribution, stream timeline, owned versus registered/pageable buffers, selected host allocator path and host statistics/allocation timings.", + "produced_artifact": "An ownership classification, source evidence IDs, one eligible allocator proposal, benefit/resource cost and rollback criteria for a later fresh-process comparison.", + "refusal_point": "Keep baseline settings for live-buffer growth, pageable copies, stable reuse, unverified versions or unsupported paths; stop tuning on event/runtime errors and escalate external ownership or gate conflicts." + }, + "recommendations": [ + "Do not mutate an H2D source or read a D2H destination before completion; non_blocking=True alone proves neither overlap nor external-buffer lifetime tracking.", + "Interpret host allocated/active bytes using the selected host path, and test round/cache controls only in supported default-path fresh processes; a per-buffer cache threshold is not a total-cache cap.", + "Keep host and device allocator options separate and use an explicit quiescent boundary for host cache release; cache clearing is not a copy-completion barrier." + ] + } --- # TorchNPU pinned-host memory lifecycle diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/references/source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle/references/source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/references/source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/ascend/torchnpu-pinned-host-memory-lifecycle/references/source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/SKILL.md similarity index 73% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/SKILL.md index 4551f390a..dfeb6c110 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/SKILL.md @@ -1,6 +1,46 @@ --- name: diagnose-attention-bottlenecks description: Diagnose the next attention-kernel bottleneck after IO reduction using evidence about HBM traffic, work ownership, parallelism, scalar work, asynchronous scheduling and precision. Produce a bounded optimization hypothesis and correctness-first verification plan; do not infer current library support or NPU portability from GPU papers. +metadata: + inferrsi: | + { + "status": "candidate", + "empirical_validation": "not_run", + "source_pins": "2205.14135v2;2307.08691v1;2407.08608v2", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [ + "references/mechanisms.md", + "references/papers-20261015/cases/io-reduction-can-plateau.md", + "references/papers-20261015/cases/overlap-and-fp8-counter-signals.md", + "references/papers-20261015/cases/partitioning-after-fusion.md", + "references/papers-20261015/INTEGRATION_NOTES.md", + "references/papers-20261015/knowledge/attention-cost-model-boundaries.md", + "references/papers-20261015/NEXT_GATE_LEADS.md", + "references/papers-20261015/README.md", + "references/sources.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Diagnose attention-kernel bottlenecks after IO reduction, tile/pipeline regressions and precision or hardware-port assumptions.", + "input_contract": "Actual kernel/device revisions, attention shapes and semantics, operand/accumulator types, correctness/quality criteria, and existing traffic, resource, parallelism and stall observations.", + "produced_artifact": "A bottleneck diagnosis with confidence, one bounded candidate, resource exchange, counter-signal, missing evidence, source ID and next authorization gate.", + "refusal_point": "Return conditional hypotheses when observations are missing; reject or defer changes with wrong outputs/gradients, unsafe lifetimes/order, unsupported primitives, unacceptable accuracy or gains erased by omitted work." + }, + "recommendations": [ + "Preserve masks, dropout replay, online-softmax normalization and gradient obligations before comparing costs; exact algebra does not imply bitwise equality.", + "After fusion, reclassify the exposed limit using scalar work, independent output tiles and resource ceilings; larger tiles and deeper pipelines can reduce residency or spill.", + "Include precision preprocessing/conversion in the measurement boundary and check generated asynchronous scheduling; validate accelerator-specific primitives separately before a port." + ] + } --- # Diagnose attention bottlenecks diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/mechanisms.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/mechanisms.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/mechanisms.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/mechanisms.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/INTEGRATION_NOTES.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/INTEGRATION_NOTES.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/INTEGRATION_NOTES.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/INTEGRATION_NOTES.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/NEXT_GATE_LEADS.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/NEXT_GATE_LEADS.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/NEXT_GATE_LEADS.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/NEXT_GATE_LEADS.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/README.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/README.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/README.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/README.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/io-reduction-can-plateau.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/io-reduction-can-plateau.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/io-reduction-can-plateau.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/io-reduction-can-plateau.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/overlap-and-fp8-counter-signals.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/partitioning-after-fusion.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/partitioning-after-fusion.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/partitioning-after-fusion.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/partitioning-after-fusion.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/knowledge/attention-cost-model-boundaries.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/knowledge/attention-cost-model-boundaries.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/knowledge/attention-cost-model-boundaries.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/papers-20261015/knowledge/attention-cost-model-boundaries.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/sources.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/sources.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/sources.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/diagnose-attention-bottlenecks/references/sources.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/host-overhead-profiling/SKILL.md similarity index 83% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/host-overhead-profiling/SKILL.md index 4f8a247a7..23be94937 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/host-overhead-profiling/SKILL.md @@ -3,6 +3,43 @@ name: host-overhead-profiling description: 'Detect and diagnose CPU/host overhead in inference serving from nsys traces. Answers: is host overhead the bottleneck? What specific work is causing it? Covers GPU idle ratio, host-prep exposed ratio, NVTX breakdown, inter-kernel gap analysis, and multi-rank straggler detection. ' +metadata: + inferrsi: | + { + "tags": [ + "host", + "cpu", + "profiling", + "nsys", + "inference" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Detect exposed host overhead and diagnose per-step host regressions in CUDA inference serving from existing nsys timelines.", + "prerequisites": "An existing serving nsys trace and Python with sqlite3; comparative root-cause analysis needs baseline and target traces, with nsys export available for report-format input.", + "input_contract": "One trace for detection/breakdown or two comparable traces for regression analysis, including the workload/phase context needed to isolate steady-state forward steps.", + "produced_artifact": "A detection verdict with metric/phase evidence, per-step NVTX/kernel/API breakdown and ranked host-operation deltas for a targeted optimization handoff.", + "failure_modes": "Warmup/loading or mismatched phase mixtures distort per-step comparisons; duplicated rank ranges, unresolved NVTX text IDs or overlapping rank end times can miscompute host gaps." + }, + "recommendations": [ + "Isolate comparable steady-state forward iterations and inspect context/generation separately; aggregate utilization can hide a phase-specific host bottleneck.", + "Confirm exposed preparation using both performance impact and idle attribution, retain communication as a confounder, and triage absolute NVTX time deltas before line-level optimization.", + "Resolve NVTX text IDs, account for TP rank duplication and use the maximum overlapping end time for gaps; do not compare full-trace totals as per-step overhead." + ] + } --- # Host Overhead Profiling diff --git a/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/SKILL.md new file mode 100644 index 000000000..d3b8732c7 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/SKILL.md @@ -0,0 +1,255 @@ +--- +name: reason-hopper-kernels +description: Reason about Hopper SM90 CUDA GEMM kernels using TMA and mbarrier lifetimes, WGMMA warpgroup participation, register and pipeline budgets, layouts, software scheduling and FP8 scaling. Use for an H100/H200-family kernel diagnosis or a Blackwell-to-Hopper applicability check; source-derived candidate guidance with no local GPU validation. +metadata: + inferrsi: | + { + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "status": "candidate", + "empirical_validation": "not_run", + "license": "UNKNOWN", + "tags": [ + "cuda", + "Hopper", + "SM90", + "SM90a", + "H100", + "H200", + "TMA", + "mbarrier", + "WGMMA", + "warp-specialization", + "registers", + "pipeline", + "swizzle", + "persistent-kernels", + "FP8", + "DeepGEMM", + "CUTLASS", + "KernelWiki", + "KDA" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/NVlabs/kda", + "commit": "acdd79343c5e9d0c7cb96289f26fd3f1051ce785", + "paths": [ + ".gitmodules", + "THIRD_PARTY_NOTICES.md", + "LICENSE", + "third_party_licenses/MIT-hanlab.txt", + "skills/KernelWiki" + ], + "readset": "references/sources.md", + "identity": "references/source-inventory.json", + "scope": "KDA-to-KernelWiki gitlink and licensing provenance only; first-party runtime code not mined." + }, + { + "type": "git", + "url": "https://github.com/mit-han-lab/KernelWiki", + "commit": "76d27b56f804e7e7295d4c570e1e5d7eef4b0a75", + "paths": [ + "README.md", + "SKILL.md", + "artifacts/kernels/deepgemm/full/PROVENANCE.yaml", + "artifacts/prs/cutlass/PR-2139/PROVENANCE.yaml", + "artifacts/prs/cutlass/PR-2750/key-files/examples/python/CuTeDSL/blackwell/tutorial_gemm/fp16_gemm_1.py", + "artifacts/prs/cutlass/PR-3091/PROVENANCE.yaml", + "artifacts/prs/cutlass/PR-3091/key-files/examples/python/CuTeDSL/hopper/grouped_gemm.py", + "artifacts/prs/cutlass/PR-3091/key-files/python/CuTeDSL/cutlass/utils/tensormap_manager.py", + "data/refresh-cutoff.yaml", + "data/tool-versions.yaml", + "sources/blogs/deepgemm.md", + "sources/prs/cutlass/PR-2139.md", + "sources/prs/cutlass/PR-2965.md", + "sources/prs/sglang/PR-16014.md", + "sources/prs/sglang/PR-19794.md", + "sources/prs/sglang/PR-9559.md", + "sources/prs/vllm/PR-20396.md", + "sources/prs/vllm/PR-23666.md", + "sources/prs/vllm/PR-33285.md", + "wiki/hardware/mbarrier.md", + "wiki/hardware/tma.md", + "wiki/kernels/deepgemm.md", + "wiki/kernels/fp8-block-scale-gemm.md", + "wiki/kernels/grouped-gemm.md", + "wiki/languages/cuda-cpp.md", + "wiki/migration/register-to-tmem.md", + "wiki/migration/wgmma-to-tcgen05.md", + "wiki/patterns/compute-bound.md", + "wiki/patterns/low-sm-utilization.md", + "wiki/patterns/memory-bound.md", + "wiki/patterns/pipeline-stalls.md", + "wiki/patterns/register-pressure.md", + "wiki/patterns/tail-effect.md", + "wiki/techniques/cache-policy.md", + "wiki/techniques/double-buffering.md", + "wiki/techniques/fine-grained-quantization.md", + "wiki/techniques/persistent-kernels.md", + "wiki/techniques/ping-pong-scheduling.md", + "wiki/techniques/pipeline-stages.md", + "wiki/techniques/register-budgeting.md", + "wiki/techniques/swizzling.md", + "wiki/techniques/tile-scheduling.md", + "wiki/techniques/vectorized-loads.md", + "wiki/techniques/warp-specialization.md" + ], + "readset": "references/sources.md", + "identity": "references/source-inventory.json", + "license": "UNKNOWN", + "scope": "All synthesis and source frontmatters screened; selected Hopper bodies mined with explicit qualifications. MIT assertion inherited from KDA; no root LICENSE at this pin." + }, + { + "type": "git", + "url": "https://github.com/deepseek-ai/DeepGEMM", + "commit": "891d57b4db1071624b5c8fa0d1e51cb317fa709f", + "paths": [ + "LICENSE", + "deep_gemm/include/deep_gemm/impls/sm90_fp8_gemm_1d1d.cuh", + "deep_gemm/include/deep_gemm/mma/sm90.cuh", + "deep_gemm/include/deep_gemm/ptx/wgmma.cuh", + "deep_gemm/include/deep_gemm/scheduler/gemm.cuh" + ], + "readset": "references/sources.md", + "identity": "references/source-inventory.json", + "license": "MIT", + "scope": "Selected code sections directly inspected; original bytes compared with KernelWiki where recorded; no native build or execution." + }, + { + "type": "git", + "url": "https://github.com/NVIDIA/cutlass", + "commit": "ca4fdbea708ad940c905359788372b8add9f85e0", + "paths": [ + "LICENSE.txt", + "examples/67_hopper_fp8_warp_specialized_gemm_with_blockwise_scaling/67_hopper_fp8_warp_specialized_gemm_with_blockwise_scaling.cu", + "include/cutlass/gemm/collective/sm90_mma_tma_gmma_ss_warpspecialized_fp8_blockwise_scaling.hpp", + "include/cutlass/gemm/kernel/sm90_gemm_array_tma_warpspecialized_pingpong.hpp" + ], + "readset": "references/sources.md", + "identity": "references/source-inventory.json", + "license": "BSD-3-Clause", + "scope": "Selected code sections directly inspected; original bytes compared with KernelWiki where recorded; no native build or execution." + }, + { + "type": "document", + "url": "https://docs.nvidia.com/cuda/archive/12.8.0/hopper-tuning-guide/index.html", + "version": "CUDA 12.8.0", + "content_sha256": "0fb0a5dd532f43e04c41112aa07bb3fddfab03fc8e68392509f0bec2c03ca418", + "readset": "references/source-inventory.json", + "scope": "Selected architecture/ISA sections directly inspected; source HTML not packaged." + }, + { + "type": "document", + "url": "https://docs.nvidia.com/cuda/archive/12.8.0/parallel-thread-execution/index.html", + "version": "CUDA 12.8.0 / PTX ISA 8.7", + "content_sha256": "48849aa7851b9ff8893efc750f1a0db46e6023852cb76d001d68d57ffa49fa06", + "readset": "references/source-inventory.json", + "scope": "Selected architecture/ISA sections directly inspected; source HTML not packaged." + } + ], + "applicability": { + "scope": "Source-derived Hopper SM90/SM90a kernel reasoning for TMA/mbarrier, WGMMA, resource/layout/pipeline/software scheduling and selected FP8 GEMM. CUDA categorization does not certify other GPU architectures, library dispatch, or a combined tested stack.", + "architectures": [ + "sm90", + "sm90a" + ], + "excluded_mechanisms": [ + "tcgen05", + "TMEM", + "CLC", + "two-SM MMA", + "native NVFP4/block-scaled tcgen05 MMA" + ], + "input_contract": "Device/compute capability, compiler target and selected implementation, kernel/library revisions, shapes/strides/alignment, numeric and grouping/epilogue semantics, correctness criterion and measurement boundary.", + "produced_artifact": "A source-qualified, reviewable hypothesis with one bounded change, resource/accuracy exchange, counter-signal, stop condition and next verification step.", + "failure_modes": "SM90 tags or import/dispatch guards can hide an unbuilt or unselected SM90a instruction path; extra stages or register caps can increase spills or lower residency." + }, + "limitations": [ + "Native compilation, GPU correctness, numerical accuracy, profiling, performance, model/service behavior and skill efficacy were not run.", + "KernelWiki is Blackwell-first; frontmatter tags, verified labels and snippet claims are not Hopper compatibility proofs.", + "Only selected code sections and nine source summary bodies were mined; remaining source headers and synthesis dispositions are explicit in the inventory.", + "Source numerical claims, fixed tuning defaults and unsafe teaching snippets are not imported as local evidence or recipes.", + "Pinned KernelWiki has no root LICENSE; KDA MIT assertion and its sanitized-distribution discrepancy are retained, with third-party terms kept separate." + ], + "provenance_readsets": [ + "references/sources.md", + "references/source-inventory.json", + "references/notices.md" + ], + "references": [ + "../diagnose-attention-bottlenecks/SKILL.md", + "../../../../playbooks/optimization/kernel-writing-guide/SKILL.md" + ], + "consumption_mode": "packaged", + "recommendations": [ + "For WGMMA advice, confirm the compiled and selected SM90a path; do not transplant Blackwell-only instruction or memory mechanisms.", + "Resolve producer/consumer ownership and completion before tuning stage depth; budget live accumulators, padded shared memory, scales and epilogue storage together.", + "For later validation, check tails, short K, stage wraparound and grouped offsets against the frozen reference, inspect actual PTX/SASS resources, and include setup/conversion/reduction costs." + ] + } +--- + +# Reason about Hopper kernels + +Use this procedure to turn an observed Hopper kernel bottleneck into one bounded +change and a correctness-first verification proposal. It does not implement a +kernel or establish installed library support. All operational steps are +**proposed**; the references distinguish inspected source facts, upstream reports +and unresolved claims. Empirical validation is **not_run**. + +Read [source qualification and coverage](references/sources.md) first when using +KernelWiki material: its SM90 tags do not establish Hopper compatibility. The +[inventory](references/source-inventory.json) accounts for every synthesis page +and every source-page header advertising SM90 at the pinned snapshot. Read the +[retained notices](references/notices.md) before reusing source material. + +## Establish the actual contract + +Record device and compute capability, compiler/toolkit and target flags, kernel +and library revisions, selected implementation, shape/strides/alignment, operand, +scale, accumulation and output types, grouping/masking, epilogue and correctness +criterion. Distinguish kernel steady state from quantization, layout conversion, +JIT/descriptor setup, launch overhead and whole-request time. + +Confirm the actual instruction path. Hopper WGMMA needs an architecture-specific +SM90a compilation path; a device-family label, a package import or a generic +SM90 dispatch guard does not prove that path was built or selected. Do not map +Blackwell tcgen05, TMEM, hardware block-scaled MMA, CLC or two-SM MMA recipes to +Hopper. Hopper thread-block clusters and TMA multicast are separate capabilities. + +## Choose the next discriminating check + +| Signal | Inspect and reason about | Small candidate and counter-signal | +| --- | --- | --- | +| TMA/barrier stalls, stale data, hang or size-sensitive failure | [Synchronization](references/synchronization.md): full/empty stages, transaction bytes, phase on reuse, WGMMA completion and epilogue-store lifetime | Repair the ownership protocol before tuning depth. A wait can be a correct dependency, not evidence of insufficient stages. | +| High registers, spills or low resident work | [Resource/layout/scheduling tradeoffs](references/resources-and-scheduling.md): all live accumulators, role budgets, padded SMEM and scale/epilogue storage | Reduce a live range, tile or unroll factor; compare against extra IO or lost instruction parallelism. A register cap can add spills. | +| Load/compute gaps with a long independent K-loop | Same reference: producer/consumer specialization and stage cost | Consider one extra input stage or a role change only after proving independence. Short K or lower residency can make it slower. | +| Low device utilization, thin/grouped GEMM or a long last wave | Same reference: useful tile count, actual residency, software schedule, locality and reduction cost | Compare tile/raster/grouping choices. Persistence cannot create missing work; split-K adds reduction and numerical obligations. | +| FP8 error or scaling/preprocessing cost | [FP8](references/fp8.md): scale meaning/layout, reduction-chunk boundary, partial-sum lifetime and output contract | Change one scale/layout/promotion choice; recheck error and the full conversion-plus-GEMM boundary. A software E8M0 representation is not native block-scaled WGMMA. | + +For attention, start with +[diagnose-attention-bottlenecks](../diagnose-attention-bottlenecks/SKILL.md) for +online-softmax, mask, forward/backward and quality obligations; use this owner's +references only for the selected Hopper mechanism. General kernel lifecycle and +measurement remain with +[kernel-writing-guide](../../../../playbooks/optimization/kernel-writing-guide/SKILL.md). + +## Return a reviewable hypothesis + +State the source path/revision and evidence class; observed signal and missing +observations; semantic and architecture prerequisites; one proposed change; the +resource or accuracy exchange; a counter-signal and stop condition; and the next +check. Retain a rejected direction with its changed-condition requirement. + +For later authorized validation, first check tails, zero/short K, repeated stage +wraparound, multiple work tiles, grouped offsets and output/scale layout against +the frozen reference. Inspect generated PTX/SASS and compiler resources on the +actual stack. Then compare the same workload and boundary, accounting for setup, +conversion, reduction, synchronization and epilogue costs. A source report or CPU +packaging check establishes no GPU correctness, performance or service gain. diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/fp8.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/fp8.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/fp8.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/fp8.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/cutlass-bsd.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/licenses/cutlass-bsd.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/cutlass-bsd.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/licenses/cutlass-bsd.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/deepgemm-mit.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/licenses/deepgemm-mit.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/deepgemm-mit.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/licenses/deepgemm-mit.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/kernelwiki-kda-mit.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/licenses/kernelwiki-kda-mit.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/licenses/kernelwiki-kda-mit.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/licenses/kernelwiki-kda-mit.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/notices.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/notices.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/notices.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/notices.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/resources-and-scheduling.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/resources-and-scheduling.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/resources-and-scheduling.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/resources-and-scheduling.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/source-inventory.json b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/source-inventory.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/source-inventory.json rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/source-inventory.json diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/sources.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/sources.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/sources.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/sources.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/synchronization.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/synchronization.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/cuda/reason-hopper-kernels/references/synchronization.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/cuda/reason-hopper-kernels/references/synchronization.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/cgroup-resource-controls/SKILL.md similarity index 75% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/cgroup-resource-controls/SKILL.md index 7d1dcd059..cb08fd3a6 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/cgroup-resource-controls/SKILL.md @@ -3,6 +3,47 @@ name: cgroup-resource-controls description: 'Diagnose and configure cgroup v2 resource controls for serving/training workloads: distinguish quota stalls from sibling contention (cpu.max vs cpu.weight, cpu.stat vs cpu.stat.local), audit requested-versus-effective cpuset with migration-cost budgeting, triage memory pressure with PSI plus event deltas (high/max/low semantics), and gate io.max caps on confirmed block-backed traffic. Includes hierarchy/delegation/permission preflight and a rollback contract. Extra resource allocation is treated as a capacity spend, never as an efficiency improvement. Control-plane procedures only — applying settings requires separate authorization. ' +metadata: + inferrsi: | + { + "tags": [ + "cgroup", + "cgroupv2", + "cpu", + "cpuset", + "memory", + "io", + "resource-controls", + "os" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Audit Linux cgroup v2 CPU quota/shares, effective placement, memory pressure/protection and conditional block-device IO controls for serving or training workloads.", + "prerequisites": "Identify workload cgroup membership, kernel/controller support, ancestor limits and the authorized file owner; prepare previous values and stop conditions before a separately authorized trial.", + "input_contract": "Target PID and hierarchy/namespace view, requested/effective cpusets, time-aligned CPU/memory/IO counters and application/sibling observations; IO proposals require resolved block-backed traffic.", + "produced_artifact": "An evidence-qualified control review/proposal with capacity tradeoff, configuration provenance, rollback manifest and matched-window application/sibling checks.", + "failure_modes": "Ancestor constraints can invalidate requested placement; pressure movement alone is not application causality, and unsupported IO accounting or shared-inode attribution can misassign traffic." + }, + "recommendations": [ + "Separate own-quota throttling, ancestor limits and sibling contention before choosing a cpu.max or cpu.weight review.", + "Compare requested with effective CPU/node sets and budget cpuset.mems migration before proposing placement; align memory PSI with local versus descendant event deltas.", + "Treat increased headroom as a shared-capacity spend and verify application outcomes and sibling impact, not just counter improvements.", + "Resolve actual block-backed traffic and supported IO controls before proposing caps; preserve inaccessible-ancestor uncertainty and do not infer container write authority." + ] + } --- # Cgroup v2 Resource Controls diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/SKILL.md similarity index 67% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/SKILL.md index 17b004c7e..c768b60cb 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/SKILL.md @@ -1,6 +1,119 @@ --- name: host-memory-management description: 'Audit pinned host buffers and asynchronous CUDA transfers: distinguish copy ordering, host mutation/reuse and device allocator lifetime; use bounded PyTorch source evidence, a reusable staging-slot example and a hardware validation recipe. Also route Ascend native descriptor retention, Python preprocessing and NUMA placement to their scoped sources.' +metadata: + inferrsi: | + { + "tags": [ + "host", + "cpu", + "memory", + "pinned-memory", + "h2d", + "gil", + "numa", + "inference", + "ascend", + "descriptor-pool", + "cuda-streams", + "allocator-lifetime", + "buffer-reuse" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/vllm-project/vllm-ascend", + "commit": "8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf", + "paths": [ + "LICENSE", + "requirements.txt", + "vllm_ascend/distributed/kv_transfer/__init__.py", + "vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/npu.py", + "vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/cpu_npu.py", + "csrc/torch_binding.cpp", + "tests/ut/kv_offload/test_native_cpu_offload.py" + ], + "readset": "references/vllm-ascend-native-offload-readset.json" + }, + { + "type": "git", + "url": "https://github.com/pytorch/pytorch", + "commit": "ba56102387ef21a3b04b357e5b183d48f0afefc7", + "paths": [ + "aten/src/ATen/core/CachingHostAllocator.h", + "aten/src/ATen/cuda/CachingHostAllocator.cpp", + "aten/src/ATen/native/cuda/Copy.cu", + "aten/src/ATen/native/Memory.cpp", + "c10/cuda/CUDAFunctions.h", + "docs/source/notes/cuda.rst", + "torch/cuda/streams.py", + "torch/_tensor_docs.py", + "LICENSE" + ], + "source_channel": "official_repository", + "inspected_scope": "selected_implementation_and_official_documentation", + "license": "BSD-3-Clause", + "readset": "references/pytorch-host-copy-readset.json" + }, + { + "type": "git", + "url": "https://github.com/python/cpython", + "commit": "60403a5409ff2c3f3b07dd2ca91a7a3e096839c7", + "paths": [ + "Doc/howto/free-threading-python.rst", + "LICENSE" + ], + "source_channel": "official_repository", + "inspected_scope": "official_documentation_only", + "license": "PSF-2.0", + "readset": "references/cpython-free-threading-readset.json" + } + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Legacy live source links remain unnormalized in references/legacy-source-notes.md; only the declared immutable source slices are normalized.", + "Added Ascend case is limited to normal native single-tier CPU descriptor-pool ownership; vLLM compatibility and CANN retention/teardown contracts remain unqualified.", + "PyTorch source-derived CUDA transfer/host allocator candidate; hardware, graph implementation, custom allocators, NUMA and service efficacy are not qualified.", + "CPython v3.13.0 documentation corrects GIL detection only; no free-threaded interpreter or inference-stack compatibility test was run." + ], + "provenance_readsets": [ + "references/vllm-ascend-native-offload.md", + "references/vllm-ascend-native-offload-readset.json", + "references/pytorch-host-copy-readset.json", + "references/cpython-free-threading-readset.json" + ], + "references": [ + "references/vllm-ascend-native-offload.md", + "references/vllm-ascend-native-offload-readset.json", + "references/vllm-ascend-LICENSE.md", + "references/pytorch-host-copy.md", + "references/pytorch-host-copy-readset.json", + "references/pytorch-LICENSE.md", + "references/cpython-free-threading-readset.json", + "references/cpython-PSF-LICENSE.md", + "references/legacy-source-notes.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Audit host preparation, pinned-host pressure and asynchronous CUDA-copy ownership/order/lifetime, with separately scoped routing for Ascend descriptor retention, Python preprocessing and NUMA placement.", + "input_contract": "Host buffer ownership and alias/mutation behavior, copy direction/dtype/layout, producer/consumer streams, live/in-flight/cache budget and the actual interpreter/placement context.", + "produced_artifact": "An ownership and dependency audit with bounded transfer candidates and a correctness-first validation plan separating enqueue, pinning, transfer, model and end-to-end cost.", + "failure_modes": "A stream wait does not prevent CPU mutation and record_stream does not create a data dependency; cached host storage may outlive Python scope, and the eager reusable-slot example is not capture-safe." + }, + "recommendations": [ + "Track copy-before-consumer order, CPU payload reuse and device storage lifetime as three independent obligations; wait for D2H completion before CPU reads.", + "Compare pageable, per-request pinning and bounded reuse on the real workload while budgeting all live and in-flight slots, cache and preprocessing memory.", + "Keep the eager staging-slot preconditions explicit and validate ordering/lifetime under stress before performance; a small passing sample does not establish a hazardous pattern is safe." + ] + } --- # Host Memory Management diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/cpython-PSF-LICENSE.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/cpython-PSF-LICENSE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/cpython-PSF-LICENSE.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/cpython-PSF-LICENSE.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/cpython-free-threading-readset.json b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/cpython-free-threading-readset.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/cpython-free-threading-readset.json rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/cpython-free-threading-readset.json diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/legacy-source-notes.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/legacy-source-notes.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/legacy-source-notes.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/legacy-source-notes.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-LICENSE.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/pytorch-LICENSE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-LICENSE.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/pytorch-LICENSE.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-host-copy-readset.json b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/pytorch-host-copy-readset.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-host-copy-readset.json rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/pytorch-host-copy-readset.json diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-host-copy.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/pytorch-host-copy.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/pytorch-host-copy.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/pytorch-host-copy.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-LICENSE.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-LICENSE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-LICENSE.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-LICENSE.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload-readset.json b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-native-offload-readset.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload-readset.json rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-native-offload-readset.json diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-native-offload.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-memory-management/references/vllm-ascend-native-offload.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-memory-management/references/vllm-ascend-native-offload.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/SKILL.md similarity index 85% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/SKILL.md index 6c7ab365d..d631356f6 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/SKILL.md @@ -3,6 +3,43 @@ name: host-overhead-optimization description: 'Profile and reduce CPU/host overhead in inference serving using line_profiler. Iterative profile-analyze-optimize-validate loop targeting Python execution in request preparation, scheduling, and model execution coordination. ' +metadata: + inferrsi: | + { + "tags": [ + "host", + "cpu", + "optimization", + "line-profiler", + "inference" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Use line-level profiling to reduce exposed Python request-preparation, scheduling and execution-coordination overhead in a bounded serving optimization loop.", + "prerequisites": "Confirm host overhead is the limiting factor with the detection procedure before tuning; choose a representative workload and consistent CPU affinity for comparisons.", + "input_contract": "Detection/NVTX triage, qualified target function paths and line-profiler output, workload/affinity settings, related correctness tests and serving benchmark metrics.", + "produced_artifact": "Per-round hotspot/change/rollback and correctness records plus a final before/after benchmark report, failed attempts, remaining hotspots and scoped follow-up recommendations.", + "failure_modes": "Optimizing the default preparation function can miss a different regressing owner; caching/hoisting/preallocation can violate lifetime or invalidation, and a line-level speedup can leave serving throughput unchanged." + }, + "recommendations": [ + "Profile the actual top regressing function by absolute exposed time, drilling into dominant unprofiled subcalls instead of assuming _prepare_tp_inputs is the owner.", + "Prefer one small evidence-supported change with explicit ownership/lifetime/invalidation checks, run relevant correctness tests and revert failed or regressing candidates.", + "Re-profile identical workloads after each accepted change and compare hotspot time, total function time and serving metrics; reassess the next bottleneck and stay within the authorized budget." + ] + } --- # Host Overhead Optimization @@ -42,8 +79,8 @@ locally measured uncertainty; the paper's noise percentage is not a universal law. Invariant reuse, views/shared buffers and preallocation require explicit immutability, lifetime and ordering guarantees. They are not automatically safe. -The [cross-layer procedure](../../../methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md) -and [FlashVector audit](../../../methods/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md) +The [cross-layer procedure](../../../../playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md) +and [FlashVector audit](../../../../playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md) retain the resource trade-offs and SLO counterexample. A component-level gain needs matched full-stack measurement before a service claim. Re-profile and reconsider batching/instance configuration after a change, under the same budget diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/os-process-placement/SKILL.md similarity index 75% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/os-process-placement/SKILL.md index 71472bd90..f59d61231 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/os-process-placement/SKILL.md @@ -3,6 +3,66 @@ name: os-process-placement description: 'Place and configure inference/training processes at the OS level before measuring or optimizing: check then pin CPU affinity to the accelerator-local NUMA node (taskset/numactl), keep affinity constant across measurement rounds, set thread-count environment variables (OMP_NUM_THREADS, OMP_PROC_BIND, CPU_AFFINITY_CONF) per scenario, run a device-busy preflight (npu-smi/nvidia-smi usages, graceful-then-force PID cleanup), and structure environment checks as ERROR vs WARNING with per-failure remediation. Use before any performance measurement on shared multi-socket or multi-tenant hosts. Does not cover cgroups, cpuset, isolcpus, or scheduler priorities — no source evidence for those. ' +metadata: + inferrsi: | + { + "tags": [ + "os", + "numa", + "cpu-affinity", + "process-placement", + "threads", + "environment", + "preflight" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/ggml-org/llama.cpp", + "commit": "3e390af18de1131dda27913c00511725b4274c8a", + "paths": [ + "common/arg.cpp", + "common/common.cpp", + "src/llama.cpp", + "src/llama-model-loader.cpp", + "src/llama-mmap.cpp", + "ggml/src/ggml-cpu/ggml-cpu.c", + "ggml/src/ggml-cpu/ggml-cpu.cpp", + "LICENSE" + ], + "readset": "references/llama-numa-thread-readset.md" + } + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Only the existing llama.cpp NUMA/thread slice is normalized at 3e390af18de1131dda27913c00511725b4274c8a; other original sources remain unnormalized, with their identities and limits retained in the procedure/readsets.", + "Linux NUMA source paths and separate configuration controls do not establish universally optimal thread counts, affinity combinations or measured placement gains." + ], + "provenance_readsets": [ + "references/llama-numa-thread-readset.md" + ], + "consumption_mode": "packaged", + "applicability": { + "scope": "Prepare and record CPU/NUMA process placement, phase-specific thread settings, device contention and environment prerequisites before shared-host performance measurements.", + "prerequisites": "No outstanding hard environment errors: required devices/driver, planned device count and any container prerequisites must be present before running.", + "input_contract": "Current affinity/NUMA policy, accelerator locality, workload phase/thread controls, device occupancy, runtime environment and relevant container/resource limits.", + "produced_artifact": "A recorded placement/thread/environment configuration and preflight outcome, with errors requiring remediation and warnings retained for the measurement campaign.", + "failure_modes": "Changing affinity or thread settings between rounds confounds timing; combining independent affinity mechanisms is order-sensitive, and small driver memory usage can be mistaken for device occupation." + }, + "recommendations": [ + "Inspect affinity and NUMA policy before placement and hold them constant across comparisons; document and requalify a round when placement changes.", + "Choose one affinity-mask authority and distinguish generation, batch and frontend thread controls; source template thread counts are starting points rather than tuned constants.", + "Record device occupancy and structured ERROR/WARNING results before measuring, using the official environment setup for derived CANN paths." + ] + } --- # OS-Level Process Placement and Runtime Configuration @@ -107,7 +167,7 @@ threads are decoupled from inference threads; (3) parallel-slot count (= server concurrency) drives memory pool sizing, so thread/slot decisions are also memory-placement decisions. -Use the [llama.cpp owner map](../../../repositories/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) +Use the [llama.cpp owner map](../../../engines/llama-cpp/audit-llama-runtime-state/SKILL.md#choose-the-scope) to distinguish this placement scope from loading modes and runtime-state diagnosis, with each owner's source pin and readset. diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/references/llama-numa-thread-readset.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/os-process-placement/references/llama-numa-thread-readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/os-process-placement/references/llama-numa-thread-readset.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/os-process-placement/references/llama-numa-thread-readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/SKILL.md b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/storage-io-measurement/SKILL.md similarity index 78% rename from src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/host/storage-io-measurement/SKILL.md index 9744493c9..cbe736f33 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/host/storage-io-measurement/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/host/storage-io-measurement/SKILL.md @@ -3,6 +3,46 @@ name: storage-io-measurement description: 'Measure storage I/O for LLM/diffusion serving without conflating device metrics with application outcomes: match the fio workload to the observed loader signature, validate requested-versus-achieved I/O depth, keep a cache-state ledger (warm/cold/unknown with method), classify startup vs steady-state runs, read outputs with unit and latency-provenance awareness, attribute traffic per device (iostat) and per process (pidstat), and gate any "storage bottleneck" claim through a saturation-evidence check plus a separate application-outcome layer. Measurement methodology only — running benchmarks or manipulating caches requires separate authorization. ' +metadata: + inferrsi: | + { + "tags": [ + "storage", + "fio", + "iostat", + "pidstat", + "measurement", + "io", + "evidence" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "applicability": { + "scope": "Measure and attribute storage traffic for model loading or serving while separating device/process observations from application outcomes and cache/startup effects.", + "prerequisites": "Resolve backing devices and participating worker PIDs first; active fio benchmarks or cache manipulation require separate authorization, with bounded observation during already-authorized runs.", + "input_contract": "Observed loader access pattern, tool/OS/filesystem versions, file-to-device mapping, worker PIDs, phase timestamps, cache-state method and raw IO/job/application measurements.", + "produced_artifact": "A workload- and cache-qualified IO evidence record with units, achieved depth, errors/bytes/completion and device/process attribution, plus a separately measured application outcome where benefit is claimed.", + "refusal_point": "Reject unspecified or copied-example benchmark workloads; do not name a storage bottleneck from util/iowait alone or infer application gain from synthetic bandwidth." + }, + "recommendations": [ + "Match the fio read pattern to the observed loader and verify achieved rather than requested depth; more queued work can increase latency.", + "Record warm/cold/unknown cache state and its method, separate startup from repeated/steady reads, and check completed bytes for truncated runs.", + "Align per-device and per-process observations with the application phase, retain units and latency provenance, and corroborate saturation using traffic, queues and process activity.", + "Only run a separately authorized bounded read-only job against an explicit existing regular file, then remeasure the real application before accepting a benefit." + ] + } --- # Storage I/O Measurement diff --git a/src/inferrsi/knowledge/data/skill_library/hardware/source-inventory.json b/src/inferrsi/knowledge/data/skill_library/systems/platforms/source-inventory.json similarity index 97% rename from src/inferrsi/knowledge/data/skill_library/hardware/source-inventory.json rename to src/inferrsi/knowledge/data/skill_library/systems/platforms/source-inventory.json index 07bf2eb58..0f32d6a8c 100644 --- a/src/inferrsi/knowledge/data/skill_library/hardware/source-inventory.json +++ b/src/inferrsi/knowledge/data/skill_library/systems/platforms/source-inventory.json @@ -12,15 +12,15 @@ }, "registered_library_roots": 79, "library_skill_documents": 142, - "basis": "Local SKILL.md, metadata.json and retained owner references; library routing and the Reference Matrix only where the local entry omits historical provenance." + "basis": "Local SKILL.md frontmatter and retained owner references; library routing and the Reference Matrix only where the local entry omits historical provenance." }, "conventions": { - "paths": "owner is relative to this hardware directory; evidence paths are relative to that owner. Every owner has SKILL.md and metadata.json.", + "paths": "owner is relative to this hardware directory; evidence paths are relative to that owner. Every owner has one SKILL.md.", "source_depth": "Channel and depth describe each retained source slice separately. A source repository, mirror or supplied pack can contain documents or implementation; none implies a whole-repository audit.", "dates": "inspection_date is the source reading date recorded in the cited evidence, not this inventory review, a commit date or a sync date. null means not recorded.", "identity": "Historical prefixes, release tags and rolling URLs remain distinct from full Git revisions; resolution alone is not a semantic reread.", "license": "License notes report the retained declarations and their scope, including UNKNOWN or absent declarations. They do not grant rights or relicense mixed-source owners.", - "validation": "All 22 owner sidecars remain candidate with empirical_validation=not_run. Static source, packaging and link checks do not establish hardware compatibility, numerical quality, performance or agent efficacy.", + "validation": "All 22 owners remain candidate with empirical_validation=not_run. Static source, packaging and link checks do not establish hardware compatibility, numerical quality, performance or agent efficacy.", "maintenance": "Update the affected row when its owner evidence changes; full source paths, ranges, hashes and notices stay with the linked owner. This file adds no skill ID or runtime interface." }, "owners": [ @@ -184,8 +184,7 @@ "inspection_date": null, "license_note": "UNKNOWN in metadata and procedure; the procedure calls awesome-ascend-skills its source while synced-from names cannbot-skills.", "evidence": [ - "SKILL.md", - "metadata.json" + "SKILL.md" ] } ], @@ -215,7 +214,7 @@ }, { "id": "ascend-optimization-casebook", - "owner": "ascend/ascend-optimization-casebook", + "owner": "../../casebooks/ascend/ascend-optimization-casebook", "source_slices": [ { "source": "Ascend/MindIE-SD and vllm-project/vllm-ascend", @@ -581,13 +580,13 @@ "inspection_date": null, "license_note": "Attributed paper paraphrase; no source-code license assumed for paper text.", "evidence": [ - "../../../methods/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md", - "../../../methods/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md" + "../../../playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/references/flashvector.md", + "../../../playbooks/measurement/performance-ceiling-analysis/references/papers-20261015/redirect-cross-layer-optimization/procedure.md" ] } ], "limits": [ - "metadata.provenance_readsets is empty despite the linked local DeepInfer case; this inventory does not normalize the sidecar. DeepInfer inspection date is not recorded in that case; preemption safety and service gains remain unresolved, not known defects/results." + "metadata.provenance_readsets is empty despite the linked local DeepInfer case; this inventory does not normalize the metadata. DeepInfer inspection date is not recorded in that case; preemption safety and service gains remain unresolved, not known defects/results." ] }, { diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/SKILL.md b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/SKILL.md new file mode 100644 index 000000000..d7ff9c811 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/SKILL.md @@ -0,0 +1,105 @@ +--- +name: inference-task-router +description: Select the starting route for day-0 model support, inference performance optimization, debugging, accuracy repair, version alignment, derived-engine development or independent runtime bring-up. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "discovery_role": "task-entry", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "task-routing", + "model-onboarding", + "performance", + "debugging", + "accuracy", + "version-alignment", + "inference-engine", + "day-0", + "cross-platform" + ], + "empirical_validation": "not_run", + "searchspace_not_applicable": "This entry selects a task and its evidence requirements before resource-layer analysis.", + "applicability": { + "scope": "Classifying an inference engineering request and identifying reusable procedures.", + "input_contract": "Goal, deliverable, known model, backend, OS, engine and workload constraints.", + "produced_artifact": "Selected task cards, scoped references, unknowns and acceptance evidence.", + "refusal_point": "If the execution owner, reference behavior or target compatibility is unknown, record the gap before making a target-specific recommendation." + }, + "limitations": [ + "The cards describe routes and acceptance evidence; hardware support and task success require separate validation." + ], + "provenance_readsets": [] + } +--- + +# Choose an inference task + +Choose one primary skill by the requested deliverable. Load its full procedure +before the first supporting evidence search. Record secondary tasks for follow-up. +The route cards remain available to callers that consume the JSON summaries. + +| Primary skill | Choose it when | Route card | +| --- | --- | --- | +| `task-day-0` | First usable support for a new checkpoint or model architecture in an existing engine. | [day0-model](references/routes/day0-model.json) | +| `task-performance` | A correct baseline misses a specified latency, throughput, memory or cost objective. | [performance](references/routes/performance.json) | +| `task-debug` | A hang, crash, OOM or execution-path failure needs a reproducible cause. | [debug](references/routes/debug.json) | +| `task-accuracy` | Inference completes but output violates a reference, quality metric or tolerance. | [accuracy](references/routes/accuracy.json) | +| `task-version-alignment` | A dependency or engine upgrade must preserve API, ABI, operator or runtime behavior across a stack. | [version-align](references/routes/version-align.json) | +| `task-derived-engine` | The deliverable reuses the execution core of vLLM, vLLM-Omni or another pinned parent, as in a DeepInfer-like extension. | [derived-engine](references/routes/derived-engine.json) | +| `task-independent-engine` | A new runtime must own execution, memory and device integration on a distinct OS, model or hardware stack. | [independent-engine](references/routes/independent-engine.json) | + +## Target conditions + +Record model family as a list (LLM, Omni, TTS, Diffusion, RLT or another +specified family), model/checkpoint revision, hardware backend (CUDA, ROCm, +Ascend NPU, XPU or another specified target), device SKU, OS (Linux, Windows, +Android, iOS or another specified target), engine and revision, dtype, shape, +concurrency and constraints. An Omni pipeline may additionally include its TTS +or diffusion stages. Keep deployment form such as mobile/edge separate from OS. +An absent field is unknown. A routing label or a common-backend registration +does not establish device support. Check the selected owner's source and scope. + +Failures that interrupt execution begin in debug. Correct execution with +unacceptable output begins in accuracy. A regression first observed after an +upgrade may use version-align as the primary route and either as a secondary +route. For engine work, decide who owns the execution core: an upstream fork +uses derived-engine; a new runtime owning its own execution and memory model +uses independent-engine, including on Windows or mobile. OS alone cannot +decide between them. + +For a Windows vLLM fork with a TTS quality regression after an upgrade, +choose accuracy if the requested delivery is restored speech quality, or +version-align if the requested delivery is a validated upgrade matrix. Link +the other route and pin both revisions, the same audio input, and reference +output. Record fork lineage as context unless creating the fork is a deliverable. +If the delivery is unclear, state both routes and ask what must be accepted. + +## Use and handoff + +1. Load the selected primary procedure, for example + `inferrsi skills enter task-version-alignment`. The compatibility card for + machine readers is available through + `inferrsi skills resource inference-task-router references/routes/version-align.json`. +2. Capture the actual run as the primary skill directs. Use observed triggers + to choose supporting skills. Search for the named component or symptom with + `inferrsi find`. Resolve candidate skill IDs via `skills show`, read their + applicability, and retain only the source-backed matches. An exact owner + or resource can also be opened directly. If only cross-platform methodology + matches, return it as such, and list the missing target-specific evidence. +3. Record primary and secondary tasks, target conditions, fixed source references, + gaps and acceptance evidence. A candidate skill is a starting point for + inspection. Execution and promotion require the normal workbench contracts. + +The cards contain seed IDs. For each affected stage and component, search the +full catalog as well. Report a target-validated owner, a methodological analogue +with its limits, or a missing owner. For a missing Windows/TTS or XPU match, +state the missing evidence and the smallest target check that would establish +it. Keep a general benchmarking procedure available as methodology. A project +policy or an unreviewed learning does not establish runtime compatibility. diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/accuracy.json b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/accuracy.json new file mode 100644 index 000000000..005d52b5f --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/accuracy.json @@ -0,0 +1,25 @@ +{ + "task": "accuracy", + "title": "Restore numerical or output quality", + "choose_when": "Inference completes but output violates a reference, quality metric or tolerance.", + "required_inputs": [ + "fixed inputs, weights and reference implementation", + "quality metric and tolerance", + "seed, dtype and preprocessing", + "modality-specific outputs" + ], + "steps": [ + "Confirm preprocessing, sampling and evaluation parity.", + "For a post-upgrade regression, compare the last-good and current revisions on identical inputs. If only the upgraded combination fails, add version-align and bisect the changed boundary. If both fail against the reference, keep tracing the quality divergence.", + "Find the first diverging stage, tensor, frame or token.", + "For TTS, collect reference audio or codec tokens at the target revision. Report target-matched evidence separately from observations on other devices. When target evidence is missing, record where you searched, the missing owner and the smallest target parity check.", + "Verify the correction on the reproducer and held-out inputs before rechecking speed." + ], + "acceptance_evidence": "A localized divergence, reproducible quality recovery and held-out results; include any latency cost.", + "candidate_skill_ids": [ + "audit-fixed-workload-claim", + "diffusers-invocation-state", + "torch-npu", + "claim-contract" + ] +} diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/day0-model.json b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/day0-model.json new file mode 100644 index 000000000..a4a8cf9b9 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/day0-model.json @@ -0,0 +1,22 @@ +{ + "task": "day0-model", + "title": "Integrate a newly released model", + "choose_when": "First usable support for a new checkpoint or model architecture in an existing engine.", + "required_inputs": [ + "checkpoint revision and tokenizer or processor", + "model family and supported modalities", + "upstream reference outputs", + "device, backend and engine revisions" + ], + "steps": [ + "Map the model config and I/O contract to the host engine.", + "Run the smallest reference-parity test for each requested modality.", + "List missing operators or unsupported features before benchmarking." + ], + "acceptance_evidence": "A reproducible load and generation smoke, parity checks and a supported-feature matrix on the named target.", + "candidate_skill_ids": [ + "ascend-model-adaptation", + "diffusers-invocation-state", + "claim-contract" + ] +} diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/debug.json b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/debug.json new file mode 100644 index 000000000..d1de9a8f4 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/debug.json @@ -0,0 +1,23 @@ +{ + "task": "debug", + "title": "Diagnose execution failures", + "choose_when": "A hang, crash, OOM or execution-path failure needs a reproducible cause.", + "required_inputs": [ + "symptom, phase and logs", + "minimal reproducer", + "last working version or configuration", + "hardware and system context" + ], + "steps": [ + "Reduce to a failing and a working control.", + "Localize the first failing boundary and test competing causes.", + "Verify the fix against the reproducer and an unaffected control." + ], + "acceptance_evidence": "A minimal reproducer, cause with evidence and regression check.", + "candidate_skill_ids": [ + "inference-optimization-casebook", + "ascend-device-health", + "audit-vllm-kv-lifetimes", + "audit-afd-p2p-exchanges" + ] +} diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/derived-engine.json b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/derived-engine.json new file mode 100644 index 000000000..8cfa50b60 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/derived-engine.json @@ -0,0 +1,23 @@ +{ + "task": "derived-engine", + "title": "Build an engine derived from an existing runtime", + "choose_when": "The deliverable reuses the execution core of vLLM, vLLM-Omni or another pinned parent, as in a DeepInfer-like extension.", + "required_inputs": [ + "parent repository, revision and retained modules", + "delta in model or scheduling contract", + "target hardware and workload", + "reference parent behavior" + ], + "steps": [ + "Identify inherited execution paths and replacement seams.", + "Define compatibility for configuration, responses and state ownership.", + "Compare outputs and performance to the pinned parent on the same inputs." + ], + "acceptance_evidence": "A runnable derived-engine slice, a parent-versus-fork delta and reproducible parity/benchmark results.", + "candidate_skill_ids": [ + "repository-evolution-analysis", + "bound-windowed-attention-replay", + "budget-speculative-verification", + "inference-benchmark-design" + ] +} diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/independent-engine.json b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/independent-engine.json new file mode 100644 index 000000000..9e992f7ff --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/independent-engine.json @@ -0,0 +1,24 @@ +{ + "task": "independent-engine", + "title": "Build an independent inference engine", + "choose_when": "A new runtime must own execution, memory and device integration on a distinct OS, model or hardware stack.", + "required_inputs": [ + "target OS and device", + "runtime API and driver/toolchain availability", + "model format and memory budget", + "reference inputs and expected outputs" + ], + "steps": [ + "Prove model loading and one operation on the target device.", + "Complete one end-to-end output, including fallback and lifecycle checks.", + "Measure resident memory, correctness and cold/warm behavior on the actual OS and hardware." + ], + "acceptance_evidence": "An independently runnable target-native engine, adapter boundaries and target-device validation results.", + "candidate_skill_ids": [ + "mlc-llm", + "local-dream", + "audit-llama-runtime-state", + "os-process-placement", + "inference-benchmark-design" + ] +} diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/performance.json b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/performance.json new file mode 100644 index 000000000..c9a86c628 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/performance.json @@ -0,0 +1,23 @@ +{ + "task": "performance", + "title": "Optimize inference performance", + "choose_when": "A correct baseline misses a specified latency, throughput, memory or cost objective.", + "required_inputs": [ + "fixed workload, quality oracle and baseline", + "target metric and budget", + "device and concurrency", + "phase timings or profile" + ], + "steps": [ + "Pin the workload and verify baseline quality.", + "Locate the limiting stage and estimate headroom.", + "Measure candidates against the same control and report end-to-end trade-offs." + ], + "acceptance_evidence": "Paired measurements with the quality gate, uncertainty and workload conditions.", + "candidate_skill_ids": [ + "inference-benchmark-design", + "performance-ceiling-analysis", + "optimization-search-policy", + "inference-optimization-casebook" + ] +} diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/version-align.json b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/version-align.json new file mode 100644 index 000000000..e785ccb0b --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/inference-task-router/references/routes/version-align.json @@ -0,0 +1,23 @@ +{ + "task": "version-align", + "title": "Align engine and plugin revisions", + "choose_when": "A dependency or engine upgrade must preserve API, ABI, operator or runtime behavior across a stack.", + "required_inputs": [ + "current and target revisions for each component", + "compatibility or ABI contracts", + "known-good configuration", + "affected integration tests" + ], + "steps": [ + "Build a before/after matrix for vLLM, vLLM-Omni, vLLM-Ascend and AFD Plugin as applicable.", + "Change one boundary at a time; record required pins or adaptations.", + "Run import, operator, integration and parity checks for the requested combination." + ], + "acceptance_evidence": "A pinned compatibility matrix, failing and passing combinations, and a rollback point.", + "candidate_skill_ids": [ + "repository-evolution-analysis", + "vllm-ascend", + "audit-afd-p2p-exchanges", + "claim-contract" + ] +} diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/task-accuracy/SKILL.md b/src/inferrsi/knowledge/data/skill_library/tasks/task-accuracy/SKILL.md new file mode 100644 index 000000000..82bc0f3d3 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/task-accuracy/SKILL.md @@ -0,0 +1,68 @@ +--- +name: task-accuracy +description: Use when inference completes but tokens, audio, frames, tensors or a quality metric diverge from a reference. Trace the first divergence and verify end-to-end tolerance. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "discovery_role": "task-entry", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "inference-task", + "primary-task", + "accuracy" + ], + "empirical_validation": "not_run", + "searchspace_not_applicable": "The procedure selects supporting evidence and tests within the caller's declared search space.", + "applicability": { + "scope": "Modality-specific reference parity and accuracy regression repair.", + "input_contract": "Actual hardware, model, engine, script and flags, benchmark, acceptance criteria, current diagnosis, permitted changes and budget.", + "produced_artifact": "First-divergence trace, before/after oracle results and held-out end-to-end quality checks.", + "refusal_point": "Record missing evidence or an exhausted search budget and identify the smallest discriminating test before declaring success." + }, + "provenance_readsets": [] + } +--- + +# Repair a numerical or output-quality regression + +## Capture the actual run + +Record the hardware vendor, device SKU, backend and driver, OS, model family and +checkpoint revision, engine and plugin versions, and the exact run script and +flags. Attach the benchmark inputs, workload shape, dtype, reference or quality +oracle, and the user's acceptance criteria. Write down the current diagnosis +and its supporting observation, allowed search space, and remaining budget. +Mark missing fields as unknown and request the first one that affects a decision. + +## Run the procedure + +1. Freeze the model revision, reference outputs, inputs, seeds, preprocessing, dtype and sampling flags. Define the user's end-to-end tolerance and how the oracle handles ordinary variation. Replay both the reference and failing target with the supplied script. +2. Compare aligned stage outputs, then the first divergent token, frame or tensor. For TTS, distinguish text normalization, codec tokens and decoded audio. Feed a saved reference intermediate to the target stage to distinguish an upstream handoff error from a local math error. +3. If quality changed after an upgrade, replay the last-good and current combinations on identical inputs. Add version alignment as a secondary task if only the target combination fails. If both fail, continue tracing the divergence against the reference. +4. Change one in-scope implementation or configuration. Re-run the original and held-out inputs, including the complete postprocessing path, and report any latency or resource trade-off. + +## Select supporting skills from observed evidence + +Load the skill whose trigger is observed. Check its revision, prerequisites and +scope before using its output. If none fits, search the observed component and +symptom, then verify any result against the actual model, engine, OS and device. + +| Observation in this run | Load skill ID | Returned evidence and next decision | +| --- | --- | --- | +| A fixed-workload report aggregates the quality or completion results differently between runs. | `audit-fixed-workload-claim` | Audit retained observations, declared quality evidence and comparability. Align the reports before attributing a regression to model math. | +| A Flux pipeline or FlowMatch scheduler differs only on repeated calls. | `diffusers-invocation-state` | Get the call-state and schedule trace. Compare fresh versus reused wrappers at the first differing timestep. | +| The first differing tensor is on an Ascend TorchNPU dispatch or dtype boundary. | `torch-npu` | Identify the actual dispatch path and precision behavior. Replay that operator with fixed inputs and verify on the installed device. | +| The reported tolerance passes for only a subset of requested outputs or targets. | `claim-contract` | Get the supported verdict and evidence gaps. State the covered scope and obtain the missing target result before declaring recovery. | + +## Acceptance and handoff + +Finish when the first divergence is explained, the original and held-out end-to-end outputs meet the user's tolerance on the actual target, and trade-offs are reported. If only an analogue exists (for example Ascend evidence for Windows TTS), identify the missing target sample and the smallest stage replay that tests transfer. +If the allowed search space or budget is exhausted, record the blocker, the +evidence already obtained and the smallest remaining discriminating test. diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/task-day-0/SKILL.md b/src/inferrsi/knowledge/data/skill_library/tasks/task-day-0/SKILL.md new file mode 100644 index 000000000..2f37fe65d --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/task-day-0/SKILL.md @@ -0,0 +1,67 @@ +--- +name: task-day-0 +description: Use for Day-0 onboarding of a new checkpoint or architecture to a named inference engine. Produce a reproducible target run, stage parity evidence and a regression fixture. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "discovery_role": "task-entry", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "inference-task", + "primary-task", + "day-0" + ], + "empirical_validation": "not_run", + "searchspace_not_applicable": "The procedure selects supporting evidence and tests within the caller's declared search space.", + "applicability": { + "scope": "New model support in an existing inference engine.", + "input_contract": "Actual hardware, model, engine, script and flags, benchmark, acceptance criteria, current diagnosis, permitted changes and budget.", + "produced_artifact": "Per-stage parity report, supported-feature matrix, end-to-end output and replayable regression fixture.", + "refusal_point": "Record missing evidence or an exhausted search budget and identify the smallest discriminating test before declaring success." + }, + "provenance_readsets": [] + } +--- + +# Bring up a new model in an existing engine + +## Capture the actual run + +Record the hardware vendor, device SKU, backend and driver, OS, model family and +checkpoint revision, engine and plugin versions, and the exact run script and +flags. Attach the benchmark inputs, workload shape, dtype, reference or quality +oracle, and the user's acceptance criteria. Write down the current diagnosis +and its supporting observation, allowed search space, and remaining budget. +Mark missing fields as unknown and request the first one that affects a decision. + +## Run the procedure + +1. Pin the upstream reference implementation, model revision, weights, tokenizer/processor, input and decoding settings. Save the exact reference command and expected output for each requested modality. Record any stochastic comparison rule before changing the adapter. +2. Map the checkpoint and I/O contract onto the engine's loader, preprocessing, stage boundaries, operators and postprocessing. On the target device, start with the smallest real-weight input. Compare each stage against the pinned reference in execution order. +3. At the first failing stage, replay its saved reference inputs through the target stage alone. If those pass, inspect the previous stage's handoff. Otherwise inspect the implementation or weights at that stage. Change one allowed adaptation at a time. +4. Run the complete user script with the agreed features and flags. Save inputs, revisions, stage checks, output and tolerance as a replayable regression fixture. State which features and devices the fixture covers. + +## Select supporting skills from observed evidence + +Load the skill whose trigger is observed. Check its revision, prerequisites and +scope before using its output. If none fits, search the observed component and +symptom, then verify any result against the actual model, engine, OS and device. + +| Observation in this run | Load skill ID | Returned evidence and next decision | +| --- | --- | --- | +| The target is Ascend and weight scales, KV replication or a multimodal adapter prevent a real-weight run. | `ascend-model-adaptation` | Get the checkpoint and feature report. Replay the first unsupported load or stage with real weights before progressing. | +| A Flux diffusion stage agrees on the first call but diverges on reuse. | `diffusers-invocation-state` | Get the wrapper/scheduler state trace. Replay with a fresh pipeline and compare the first differing timestep. | +| The request includes a support claim and some features have only source or smoke evidence. | `claim-contract` | Map observations to the declared gates. Publish the supported subset and leave the uncovered features open. | + +## Acceptance and handoff + +Finish when each requested stage meets its reference check, the full model returns the accepted result on the specified target, and the saved fixture reproduces it. If a stage still fails, name its first mismatching input/output pair and the single replay that separates a loader error from an operator error. +If the allowed search space or budget is exhausted, record the blocker, the +evidence already obtained and the smallest remaining discriminating test. diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/task-debug/SKILL.md b/src/inferrsi/knowledge/data/skill_library/tasks/task-debug/SKILL.md new file mode 100644 index 000000000..572fab5b7 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/task-debug/SKILL.md @@ -0,0 +1,68 @@ +--- +name: task-debug +description: Use for inference crashes, hangs, out-of-memory failures or execution-path errors. Localize the first failing boundary and verify recovery with the original reproducer. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "discovery_role": "task-entry", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "inference-task", + "primary-task", + "debug" + ], + "empirical_validation": "not_run", + "searchspace_not_applicable": "The procedure selects supporting evidence and tests within the caller's declared search space.", + "applicability": { + "scope": "Run-specific failure reproduction, fault isolation and regression verification.", + "input_contract": "Actual hardware, model, engine, script and flags, benchmark, acceptance criteria, current diagnosis, permitted changes and budget.", + "produced_artifact": "Minimal reproducer, passing control, localized cause and verified original-script replay.", + "refusal_point": "Record missing evidence or an exhausted search budget and identify the smallest discriminating test before declaring success." + }, + "provenance_readsets": [] + } +--- + +# Isolate and repair an execution failure + +## Capture the actual run + +Record the hardware vendor, device SKU, backend and driver, OS, model family and +checkpoint revision, engine and plugin versions, and the exact run script and +flags. Attach the benchmark inputs, workload shape, dtype, reference or quality +oracle, and the user's acceptance criteria. Write down the current diagnosis +and its supporting observation, allowed search space, and remaining budget. +Mark missing fields as unknown and request the first one that affects a decision. + +## Run the procedure + +1. Replay the supplied script with its original flags and capture the first observable error, stage, device, rank and timestamps. Keep a last-working revision or configuration as a control. For a hang, capture progress at each boundary before attributing a cause. +2. Minimize the input or configuration one variable at a time. Preserve the original signature and a closely matched passing control. Follow ownership across stages to locate the earliest difference. +3. State two explanations that predict different observations. Probe the smallest boundary that distinguishes them, such as a recorded handoff with a known-good producer or consumer. Apply one in-scope fix only when the observation implicates that boundary. +4. Repeat the minimal reproducer, then the original unmodified script and an unaffected control. Retain logs tying the original failure and recovery to the same boundary and save a regression check. + +## Select supporting skills from observed evidence + +Load the skill whose trigger is observed. Check its revision, prerequisites and +scope before using its output. If none fits, search the observed component and +symptom, then verify any result against the actual model, engine, OS and device. + +| Observation in this run | Load skill ID | Returned evidence and next decision | +| --- | --- | --- | +| The failure resembles a prior trial or appears only with a particular optimization. | `inference-optimization-casebook` | Get the prior failure conditions and retry rule. Test whether the same switch reproduces the signature on this target. | +| Ascend timing or device behavior changed after a killed job or card-group change. | `ascend-device-health` | Collect device health, occupancy and timing checks. Establish a clean matched control before blaming the engine. | +| A vLLM V1 request stalls during asynchronous KV transfer, preemption or block reuse. | `audit-vllm-kv-lifetimes` | Build the scheduler/connector ownership trace. Probe the earliest missing completion or block-release edge. | +| A CUDA AFD dense FFN P2P exchange hangs or returns the wrong slice. | `audit-afd-p2p-exchanges` | Build the rank and per-peer payload ledger. Check the first mismatched assignment against caller traces before testing transport. | + +## Acceptance and handoff + +Finish when the original failure is reproducible before the fix, its earliest implicated boundary is supported by evidence, and both the original script and the unaffected control pass after the fix. For an intermittent failure, record the attempted repeats and propose the narrowest additional trace that can distinguish the remaining causes. +If the allowed search space or budget is exhausted, record the blocker, the +evidence already obtained and the smallest remaining discriminating test. diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/task-derived-engine/SKILL.md b/src/inferrsi/knowledge/data/skill_library/tasks/task-derived-engine/SKILL.md new file mode 100644 index 000000000..9cacde59e --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/task-derived-engine/SKILL.md @@ -0,0 +1,68 @@ +--- +name: task-derived-engine +description: Use for a fork or extension that retains an upstream inference execution core, including a vLLM or vLLM-Omni derivative. Verify inherited behavior and the intended delta separately. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "discovery_role": "task-entry", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "inference-task", + "primary-task", + "derived-engine" + ], + "empirical_validation": "not_run", + "searchspace_not_applicable": "The procedure selects supporting evidence and tests within the caller's declared search space.", + "applicability": { + "scope": "Development and qualification of an engine that retains an upstream execution core.", + "input_contract": "Actual hardware, model, engine, script and flags, benchmark, acceptance criteria, current diagnosis, permitted changes and budget.", + "produced_artifact": "Parent/fork behavior matrix, bounded extension and matched end-to-end results.", + "refusal_point": "Record missing evidence or an exhausted search budget and identify the smallest discriminating test before declaring success." + }, + "provenance_readsets": [] + } +--- + +# Deliver an engine derived from a pinned parent + +## Capture the actual run + +Record the hardware vendor, device SKU, backend and driver, OS, model family and +checkpoint revision, engine and plugin versions, and the exact run script and +flags. Attach the benchmark inputs, workload shape, dtype, reference or quality +oracle, and the user's acceptance criteria. Write down the current diagnosis +and its supporting observation, allowed search space, and remaining budget. +Mark missing fields as unknown and request the first one that affects a decision. + +## Run the procedure + +1. Pin the parent repository, revision, build and known-good run. Identify inherited modules and the fork-owned seam. State what API, model behavior and state ownership must be preserved and what observable behavior should change. +2. Run the actual model, script and flags on the parent. Capture outputs, state transitions and benchmark results. Build the smallest fork slice at the chosen seam on the same target. +3. Run a matched inherited-path check with the extension disabled or bypassed where supported. Run a separate case that exercises the intended delta. At the first mismatch, replay the same input at the parent/fork boundary before extending the change. +4. Repeat end to end with the user's flags and acceptance workload. Report inherited-path regressions, delta behavior, and performance or memory trade-offs separately. + +## Select supporting skills from observed evidence + +Load the skill whose trigger is observed. Check its revision, prerequisites and +scope before using its output. If none fits, search the observed component and +symptom, then verify any result against the actual model, engine, OS and device. + +| Observation in this run | Load skill ID | Returned evidence and next decision | +| --- | --- | --- | +| The parent and fork changed the same subsystem at different revisions. | `repository-evolution-analysis` | Locate the inherited contract and divergence point in history. Test the last shared behavior and then the fork-owned change. | +| The fork compacts replay or prefill rows across a windowed-attention state boundary. | `bound-windowed-attention-replay` | Derive the retain/dependency rule. Test at the rollback and window boundaries before accepting the reduced state. | +| The fork changes speculative verification and draft acceptance varies by position. | `budget-speculative-verification` | Obtain a per-position acceptance and cost model. Test the proposed row budget against the parent at matched quality. | +| The new path changes scheduling or outputs and current benchmarks exercise only parent behavior. | `inference-benchmark-design` | Define matched controls and a delta-exercising workload. Run both on the same device before accepting the fork. | + +## Acceptance and handoff + +Finish when the inherited-path checks pass, the intended delta is demonstrated by a distinct test, and both are reproducible on the named target under the user's gates. If both paths regress, isolate the first common initialization or dispatch boundary before changing the extension. +If the allowed search space or budget is exhausted, record the blocker, the +evidence already obtained and the smallest remaining discriminating test. diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/task-independent-engine/SKILL.md b/src/inferrsi/knowledge/data/skill_library/tasks/task-independent-engine/SKILL.md new file mode 100644 index 000000000..d60e15d8c --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/task-independent-engine/SKILL.md @@ -0,0 +1,69 @@ +--- +name: task-independent-engine +description: Use when the deliverable owns its execution, memory and device integration on a target OS and hardware stack. Prove a native end-to-end path and measure its resource use. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "discovery_role": "task-entry", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "inference-task", + "primary-task", + "independent-engine" + ], + "empirical_validation": "not_run", + "searchspace_not_applicable": "The procedure selects supporting evidence and tests within the caller's declared search space.", + "applicability": { + "scope": "Independent runtime bring-up across OS, model format, device and backend boundaries.", + "input_contract": "Actual hardware, model, engine, script and flags, benchmark, acceptance criteria, current diagnosis, permitted changes and budget.", + "produced_artifact": "Operator/capability map, minimal native execution path, end-to-end parity and resource measurements.", + "refusal_point": "Record missing evidence or an exhausted search budget and identify the smallest discriminating test before declaring success." + }, + "provenance_readsets": [] + } +--- + +# Bring up an independent inference engine + +## Capture the actual run + +Record the hardware vendor, device SKU, backend and driver, OS, model family and +checkpoint revision, engine and plugin versions, and the exact run script and +flags. Attach the benchmark inputs, workload shape, dtype, reference or quality +oracle, and the user's acceptance criteria. Write down the current diagnosis +and its supporting observation, allowed search space, and remaining budget. +Mark missing fields as unknown and request the first one that affects a decision. + +## Run the procedure + +1. Pin model format, reference inputs and outputs, target OS/ABI, device/driver, available runtime APIs and resource limits. Map required operators, dtypes, layouts and memory lifetimes to target capabilities. Mark fallback, missing implementations and unverified capabilities. +2. On the actual target, load the model and run one reference-checked operator with a dispatch trace. Extend to the shortest connected path that produces an output. Test the first missing operator or allocation boundary before expanding coverage. +3. Run the complete input through preprocessing, execution and postprocessing. Verify device placement or any explicitly permitted fallback. Exercise repeated requests and teardown to expose ownership or retained-memory errors. +4. Replay the user's script on the target OS/device. Check end-to-end output, resident or peak memory, cold/warm behavior and the user-specified resource gates. Record unsupported operators and model features. + +## Select supporting skills from observed evidence + +Load the skill whose trigger is observed. Check its revision, prerequisites and +scope before using its output. If none fits, search the observed component and +symptom, then verify any result against the actual model, engine, OS and device. + +| Observation in this run | Load skill ID | Returned evidence and next decision | +| --- | --- | --- | +| An MLC-derived artifact, Metal dispatch or KV-capacity estimate is relevant to the proposed target. | `mlc-llm` | Obtain the pinned artifact/runtime contract and native discriminator. Verify the operator or allocation locally before using it as target evidence. | +| An Android diffusion path exhibits a process, DiT packaging, ABI or backend mismatch. | `local-dream` | Get the declared-versus-serving state trace. Run the minimal packaged path on the named Android device and verify dispatch. | +| A repeated request retains state and the design uses a llama.cpp-compatible context. | `audit-llama-runtime-state` | Obtain a state ownership ledger. Probe the earliest unmatched allocation, progress or release event on the target runtime. | +| Timings vary on a shared host and CPU affinity or accelerator contention is observable. | `os-process-placement` | Record supported placement, occupancy and thread controls. Repeat the workload under matched conditions using target-native tooling. | +| The engine runs end to end but quality, cold/warm timings or the resource envelope are undefined. | `inference-benchmark-design` | Define reference checks and a target workload. Collect quality and resource measurements on the actual device. | + +## Acceptance and handoff + +Finish when a target-native end-to-end run meets the agreed correctness and resource gates, with native dispatch or permitted fallback accounted for. If a backend or device is unavailable, record the missing access and propose the first on-target operator or allocation check that can resolve the blocker. +If the allowed search space or budget is exhausted, record the blocker, the +evidence already obtained and the smallest remaining discriminating test. diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/task-performance/SKILL.md b/src/inferrsi/knowledge/data/skill_library/tasks/task-performance/SKILL.md new file mode 100644 index 000000000..76b25ae7c --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/task-performance/SKILL.md @@ -0,0 +1,68 @@ +--- +name: task-performance +description: Use when a correct inference baseline misses a latency, throughput, memory or cost target. Compare compatible changes under the user's workload and quality gate. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "discovery_role": "task-entry", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "inference-task", + "primary-task", + "performance" + ], + "empirical_validation": "not_run", + "searchspace_not_applicable": "The procedure selects supporting evidence and tests within the caller's declared search space.", + "applicability": { + "scope": "Optimization of a specified metric under fixed workload, quality and resource constraints.", + "input_contract": "Actual hardware, model, engine, script and flags, benchmark, acceptance criteria, current diagnosis, permitted changes and budget.", + "produced_artifact": "Paired baseline/candidate measurements, quality checks, trade-offs and final replay.", + "refusal_point": "Record missing evidence or an exhausted search budget and identify the smallest discriminating test before declaring success." + }, + "provenance_readsets": [] + } +--- + +# Optimize a measured inference bottleneck + +## Capture the actual run + +Record the hardware vendor, device SKU, backend and driver, OS, model family and +checkpoint revision, engine and plugin versions, and the exact run script and +flags. Attach the benchmark inputs, workload shape, dtype, reference or quality +oracle, and the user's acceptance criteria. Write down the current diagnosis +and its supporting observation, allowed search space, and remaining budget. +Mark missing fields as unknown and request the first one that affects a decision. + +## Run the procedure + +1. Replay the supplied benchmark unchanged. Pin requests, shape/batch mix, concurrency, warmup, timing boundary, repetitions, aggregation rule and quality oracle. Capture the baseline distribution and end-to-end output on the actual engine and device. +2. Profile the stage dominating the target metric. Separate queueing, prefill, decode, transfer or modality generation where observable. Estimate how much changing that stage can move the end-to-end metric. +3. Choose one change inside the allowed search space whose mechanism matches the measured bottleneck and the installed hardware/engine. Record the predicted profile signature. Alternate unchanged baseline and candidate trials with the same workload, quality evaluation and resource conditions. +4. After each change, compare the target metric, uncertainty and quality gate. Restore the incumbent if quality fails or the benefit is indistinguishable. Re-profile when the bottleneck moves. Recheck the selected change end to end with the original script. + +## Select supporting skills from observed evidence + +Load the skill whose trigger is observed. Check its revision, prerequisites and +scope before using its output. If none fits, search the observed component and +symptom, then verify any result against the actual model, engine, OS and device. + +| Observation in this run | Load skill ID | Returned evidence and next decision | +| --- | --- | --- | +| The supplied benchmark lacks an agreed workload boundary or modality quality oracle. | `inference-benchmark-design` | Obtain a workload and parity protocol. Resolve the missing gate and measure the baseline before comparing candidates. | +| A slow stage is visible but its contribution to the requested metric is unclear. | `performance-ceiling-analysis` | Estimate stage share and achievable headroom. Profile the next limiting stage if even eliminating this one cannot meet the target. | +| Several legal changes could address the profiled limit. | `optimization-search-policy` | Obtain a ranked shortlist with mechanisms, expected signals and cost. Test one bounded candidate, then retain or revert it using the paired evidence. | +| A prior trial appears to match the bottleneck or an earlier candidate failed. | `inference-optimization-casebook` | Retrieve the prior workload, decision and retry condition. Reuse its hypothesis only after comparing target conditions and retest on this run. | + +## Acceptance and handoff + +Finish when the selected candidate meets the user's metric and quality criteria on an independent end-to-end replay. If trials disagree, identify a targeted control such as an unchanged repeat to separate run variance from the candidate effect. Retain the best verified baseline if budget expires. +If the allowed search space or budget is exhausted, record the blocker, the +evidence already obtained and the smallest remaining discriminating test. diff --git a/src/inferrsi/knowledge/data/skill_library/tasks/task-version-alignment/SKILL.md b/src/inferrsi/knowledge/data/skill_library/tasks/task-version-alignment/SKILL.md new file mode 100644 index 000000000..10a09f3a3 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/tasks/task-version-alignment/SKILL.md @@ -0,0 +1,68 @@ +--- +name: task-version-alignment +description: Use for upgrades spanning inference engines, plugins, drivers or operator libraries. Isolate the incompatible boundary and qualify the requested version combination on the same workload. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "discovery_role": "task-entry", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "inference-task", + "primary-task", + "version-alignment" + ], + "empirical_validation": "not_run", + "searchspace_not_applicable": "The procedure selects supporting evidence and tests within the caller's declared search space.", + "applicability": { + "scope": "Compatibility of explicit working and target version combinations.", + "input_contract": "Actual hardware, model, engine, script and flags, benchmark, acceptance criteria, current diagnosis, permitted changes and budget.", + "produced_artifact": "Version matrix, first incompatible boundary, matched workload checks and rollback point.", + "refusal_point": "Record missing evidence or an exhausted search budget and identify the smallest discriminating test before declaring success." + }, + "provenance_readsets": [] + } +--- + +# Validate an engine or plugin stack upgrade + +## Capture the actual run + +Record the hardware vendor, device SKU, backend and driver, OS, model family and +checkpoint revision, engine and plugin versions, and the exact run script and +flags. Attach the benchmark inputs, workload shape, dtype, reference or quality +oracle, and the user's acceptance criteria. Write down the current diagnosis +and its supporting observation, allowed search space, and remaining budget. +Mark missing fields as unknown and request the first one that affects a decision. + +## Run the procedure + +1. Inventory the known-good and requested versions of the engine, plugin, operator library, backend and driver. Include vLLM, vLLM-Omni, vLLM-Ascend and AFD Plugin where used. Keep build options, model, device and run script constant. +2. Replay the known-good and complete target stacks on the same workload. Record import/build, operator dispatch, integration and output/parity results for each supported combination. +3. Change one compatible dependency boundary at a time, or bisect a coupled pair when the components must travel together. Record unbuildable combinations explicitly. Compare effective APIs, ABI, flags and dispatch at the earliest failing step. +4. Apply the smallest in-scope pin or adapter, replay the complete target stack, and verify the rollback combination with the same script and quality/performance checks requested by the user. + +## Select supporting skills from observed evidence + +Load the skill whose trigger is observed. Check its revision, prerequisites and +scope before using its output. If none fits, search the observed component and +symptom, then verify any result against the actual model, engine, OS and device. + +| Observation in this run | Load skill ID | Returned evidence and next decision | +| --- | --- | --- | +| Several upstream revisions changed a boundary and the first incompatible change is unknown. | `repository-evolution-analysis` | Trace the history to a candidate transition. Run the same contract test immediately before and after it. | +| The changed component is vLLM-Ascend and its release, role or effective dispatch differs. | `vllm-ascend` | Get the version-specific dispatch and first failing boundary. Verify applicability against the installed stack before choosing a pin or adapter. | +| An AFD Plugin upgrade changes a CUDA P2P rank map or per-peer split. | `audit-afd-p2p-exchanges` | Compare the old and new payload ledgers. Test the first changed rank or slice with the same worker configuration. | +| A release claim spans combinations with missing hardware or parity evidence. | `claim-contract` | Map each tested combination to its gates. Publish the observed matrix and retain untested cells as unknown. | + +## Acceptance and handoff + +Finish when the requested target combination passes the user's original workload and gates, the changed boundary is documented, and the rollback point is verified. If only a coupled upgrade fails, state the next API or dispatch check that separates the two components. +If the allowed search space or budget is exhausted, record the blocker, the +evidence already obtained and the smallest remaining discriminating test. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md deleted file mode 100644 index 1ee5c1f73..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -name: cite-check-downstream-pr -description: Review a downstream PR description for correct acceptance issue links, SHA pins, and artifact citations. ---- - -# Cite-check a downstream PR - -Use when reviewing external PRs that claim performance or quality improvements tied to RSI acceptance work. - -## Checklist - -- [ ] Links an acceptance issue URL (not only a chat thread) -- [ ] States **baseline SHA** and **candidate SHA** matching that issue -- [ ] Points to `evidence//results/` (or equivalent published mirror) -- [ ] Does **not** invent new thresholds or restate bars differently -- [ ] Marks summaries **unverified** if artifacts are missing, stale, or off-SHA -- [ ] Uses verdict vocabulary only: `PASS` / `FAIL` / `INCONCLUSIVE` - -## Red flags - -- “X% faster” without artifact path -- Branch names instead of full SHAs -- Microbench-only support for e2e latency claims - -## Reference - -[docs/contributing.md#evidence-and-claims](https://github.com/InferMatrixLab/InferRSI/blob/b2f60f913a949f9073686a23476f752e11d709a4/docs/contributing.md#evidence-and-claims) diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/metadata.json b/src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/metadata.json deleted file mode 100644 index 1a7fe71e3..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/cite-check-downstream-pr/metadata.json +++ /dev/null @@ -1,43 +0,0 @@ -{ - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "review", - "downstream", - "citation", - "evidence", - "verdict" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Review of a downstream PR description that cites RSI acceptance work for a performance or quality improvement", - "prerequisites": "The PR description, the acceptance issue it cites, and the evidence paths it names; reviewing is read-only and needs no local run", - "input_contract": "One PR description plus the cited acceptance issue and its evidence tree; the reviewer reads claims and citations, not the candidate code", - "produced_artifact": "A per-checklist review result that either accepts the citation or marks the summary unverified, using only PASS/FAIL/INCONCLUSIVE vocabulary", - "failure_modes": [ - "No acceptance issue URL, only a chat thread", - "Branch names instead of full baseline and candidate SHAs", - "Microbenchmark evidence offered for an end-to-end latency claim", - "A threshold restated or invented relative to the acceptance issue", - "Evidence path absent, stale or on a different SHA than the cited candidate" - ], - "refusal_point": "Refuses to restate a performance or quality claim whose artifacts are missing, stale or off-SHA; reports the summary as unverified instead" - }, - "recommendations": [ - "Read the cited acceptance issue before the PR description so an invented threshold is visible immediately.", - "Resolve every listed evidence path and compare its SHA with the candidate SHA the PR names.", - "Report an unverifiable claim as unverified rather than repeating its number." - ], - "limitations": [ - "Checklist-only procedure; it validates citation discipline, not the measurement behind an artifact.", - "A passing review does not establish that the campaign evidence itself is correct." - ], - "provenance_readsets": [], - "searchspace_not_applicable": "SKILL.md §§ Checklist and Red flags check acceptance links, revision/artifact citations and claim wording; they provide no hardware-resource mechanism." -} diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/SKILL.md deleted file mode 100644 index 5cb905bfc..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/SKILL.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -name: claim-contract -description: Enforce ThinkFlowLab claim contract wording and verdict rules without embedding campaign-specific revision pins. ---- - -# Claim contract (portable) - -## Public claim wording - -Use: “on {repo}@{candidate} vs baseline@{baseline}, under {RunSpec}, we observed …” - -Only after acceptance **`PASS`** with artifacts under `evidence/`. - -## Rules (not numbers) - -| Verdict | When | -| --- | --- | -| PASS | Pinned SHAs, artifact tree on candidate tip, gates satisfied | -| FAIL | Gates failed with complete evidence | -| INCONCLUSIVE | Cannot defend comparison (env, partial run, missing validator freeze) | - -## What not to do - -- Do not bake baseline/candidate SHAs into shared repo templates, skills, or default CI env. -- Do not treat microbench alone as e2e campaign completion. -- Do not auto-transfer lessons across unverified repo × revision × backend × device × model × scenario tuples. - -## SIAS - -Respect plane boundaries: `harness/` orchestrates, `model/` defines quality, `infra/` supplies execution. Cross-plane deps belong in RunSpec. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/metadata.json b/src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/metadata.json deleted file mode 100644 index 3ed4469ce..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/claim-contract/metadata.json +++ /dev/null @@ -1,41 +0,0 @@ -{ - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "claim", - "wording", - "verdict", - "evidence", - "portable" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Wording and verdict rules for any public performance or quality claim produced from an RSI campaign", - "prerequisites": "A completed campaign with pinned baseline and candidate SHAs and an evidence tree; the rules themselves carry no numeric threshold", - "input_contract": "A claim draft, the campaign RunSpec and its evidence tree under evidence//", - "produced_artifact": "A claim sentence in the pinned form, or a refusal when the campaign verdict is not PASS", - "failure_modes": [ - "Claim asserted before the campaign reaches PASS with artifacts", - "Baseline or candidate SHA written into a shared template, skill or default CI environment", - "Microbenchmark result presented as end-to-end completion", - "A lesson transferred across an unverified repo x revision x backend x device x model x scenario tuple" - ], - "refusal_point": "Refuses any public claim lacking a PASS verdict and artifacts under evidence/, and refuses numeric thresholds because thresholds belong to the campaign" - }, - "recommendations": [ - "Keep campaign SHAs in the acceptance issue; a shared template that carries them will be wrong for the next campaign.", - "Write the verdict from the evidence tree before writing the claim sentence.", - "Treat each repo x revision x backend x device x model x scenario tuple as an independent claim." - ], - "limitations": [ - "States wording and verdict rules only; it selects no numeric gate and validates no measurement.", - "SIAS plane boundaries constrain where a change belongs, not whether it is correct." - ], - "provenance_readsets": [] -} diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/SKILL.md deleted file mode 100644 index d502f3471..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/SKILL.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -name: competitor-scan -description: Structure a lightweight competitive scan without inventing benchmark numbers or campaign verdicts. ---- - -# Competitor scan (generic) - -Use when positioning a new RSI repo—not to replace acceptance evidence. - -## Frame - -| Question | Answer (qualitative) | -| --- | --- | -| What do others optimize? | _throughput, latency, cost, quality, …_ | -| How do they publish proof? | _blogs, papers, dashboards, none_ | -| Claim contract? | _pinned revisions + reproducible artifacts? often missing_ | -| Our wedge | _smallest proof that unlocks the next bet_ | - -## Rules - -- No numeric comparisons unless sourced from **our** `evidence/.../results/` with `PASS`. -- Label third-party numbers **unverified** if not reproduced under our RunSpec. -- Use the [evidence and claim rules](https://github.com/InferMatrixLab/InferRSI/blob/b2f60f913a949f9073686a23476f752e11d709a4/docs/contributing.md#evidence-and-claims) when writing a comparison. - -## Deliverable - -Short markdown: problem, alternatives, differentiation (claim contract + SIAS), limits, open questions. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/metadata.json b/src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/metadata.json deleted file mode 100644 index 8947144b1..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/competitor-scan/metadata.json +++ /dev/null @@ -1,40 +0,0 @@ -{ - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "competitive-analysis", - "positioning", - "claim", - "unverified" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Qualitative positioning of a new RSI repository against other projects before acceptance evidence exists", - "prerequisites": "A selected repository and readable public material from the alternatives; no benchmark run is required", - "input_contract": "A target repository plus whatever the alternatives publish about what they optimize and how they prove it", - "produced_artifact": "A short Markdown note covering problem, alternatives, differentiation, limits and open questions", - "failure_modes": [ - "Numeric comparison written without a PASS campaign in our own evidence tree", - "Third-party figure repeated as if reproduced under our RunSpec", - "Scan presented as a substitute for acceptance evidence" - ], - "refusal_point": "Refuses any numeric comparison not sourced from our own evidence//results/ with a PASS verdict" - }, - "recommendations": [ - "Answer the four framing questions qualitatively first; the absence of a published claim contract is itself a finding.", - "Label every third-party number unverified unless it was reproduced under our RunSpec.", - "Close the note with open questions rather than a conclusion the evidence cannot support." - ], - "limitations": [ - "Produces positioning material only; it establishes no performance or quality result.", - "Third-party claim contracts are read from public material and may be incomplete." - ], - "provenance_readsets": [], - "searchspace_not_applicable": "SKILL.md §§ Frame, Rules and Deliverable produce a qualitative competitive comparison with evidence limits, not a hardware-resource decision or implementation mechanism." -} diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/SKILL.md deleted file mode 100644 index 387b6702f..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/SKILL.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -name: draft-acceptance-issue -description: Draft a campaign acceptance issue with checklist, RunSpec, evidence paths, and placeholders for baseline/candidate SHAs. ---- - -# Draft acceptance issue - -Use when starting a new campaign in a ThinkFlowLab RSI-style repo. - -## Steps - -1. Pick `campaign_id` (naming: `{model-or-scenario} / {device}×{count}`). -2. Copy `evidence/_template/` → `evidence//` (empty tree with README stubs). -3. Open issue using `.github/ISSUE_TEMPLATE/campaign-acceptance.yml` or [docs/contributing.md#acceptance-checklist](https://github.com/InferMatrixLab/InferRSI/blob/b2f60f913a949f9073686a23476f752e11d709a4/docs/contributing.md#acceptance-checklist). -4. Fill **pinned baseline SHA** and **pinned candidate SHA** in the issue—use placeholders `` / `` until known. -5. Resolve criteria using the [acceptance-criteria procedure](../../methods/performance-ceiling-analysis/references/acceptance-criteria.md): - use user-supplied criteria; otherwise use `performance-ceiling-analysis` to - formulate targets. Preserve partial requirements, label derived fields and - retain unresolved evidence gaps. List supported gates and their provenance in - `plan/` before candidate search. -6. Set initial verdict `INCONCLUSIVE`. - -## Do not - -- Commit campaign SHAs into `recipes/`, `benchmarks/`, skills, or workflow defaults. -- Claim `PASS` without `validator/` freeze and `results/` rationale. - -## Output - -Issue body sections: campaign id, SHAs, RunSpec, criteria source and derivation (including evidence gaps and freeze authority), gates, evidence paths, verdict, downstream citation blurb. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/metadata.json b/src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/metadata.json deleted file mode 100644 index 17b64b4a5..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/draft-acceptance-issue/metadata.json +++ /dev/null @@ -1,42 +0,0 @@ -{ - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "acceptance", - "issue", - "runspec", - "template", - "campaign" - ], - "empirical_validation": "not_run", - "applicability": { - "scope": "Opening a new acceptance campaign in an RSI-style repository, before any baseline or candidate run", - "prerequisites": "A chosen campaign subject and an empty evidence// tree copied from evidence/_template/; SHAs may still be placeholders", - "input_contract": "A campaign id of the form {model-or-scenario}/{device}x{count}, any user-supplied criteria or accepted plan, and available workload/baseline/calibration evidence for deriving missing targets", - "produced_artifact": "An acceptance issue body with campaign id, pinned baseline and candidate SHAs, RunSpec, criteria source and derivation including gaps and freeze authority, gates, evidence paths, initial verdict and a downstream citation blurb", - "failure_modes": [ - "Gates written after the search loop has already run", - "Campaign SHAs committed into recipes/, benchmarks/, skills or workflow defaults", - "PASS claimed without a validator/ freeze and a results/ rationale" - ], - "refusal_point": "Refuses to state a verdict other than INCONCLUSIVE while the validator freeze or results rationale is absent" - }, - "recommendations": [ - "Use user-supplied criteria; when absent, follow the performance-ceiling-analysis acceptance procedure to formulate targets, preserving partial requirements and identifying derived fields and evidence gaps.", - "Write the gates in plan/ before any search loop; a gate chosen afterwards cannot reject the candidate honestly.", - "Use the and placeholders until the campaign fixes them, and keep the real values in the issue.", - "Set the initial verdict to INCONCLUSIVE so the absence of evidence is recorded rather than assumed away." - ], - "limitations": [ - "Drafts an issue only; it freezes no gate and runs no measurement.", - "The placeholder SHA convention depends on the campaign owner filling them in later." - ], - "provenance_readsets": [], - "searchspace_not_applicable": "SKILL.md §§ Steps and Output prepare an acceptance issue, RunSpec, frozen gates and evidence paths; device identifiers are campaign metadata rather than a hardware-resource mechanism." -} diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/governance/cite-check-downstream-pr/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/cite-check-downstream-pr/SKILL.md new file mode 100644 index 000000000..3a7fd5777 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/workflows/governance/cite-check-downstream-pr/SKILL.md @@ -0,0 +1,72 @@ +--- +name: cite-check-downstream-pr +description: Review a downstream PR description for correct acceptance issue links, SHA pins, and artifact citations. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "review", + "downstream", + "citation", + "evidence", + "verdict" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Review of a downstream PR description that cites RSI acceptance work for a performance or quality improvement", + "prerequisites": "The PR description, the acceptance issue it cites, and the evidence paths it names; reviewing is read-only and needs no local run", + "input_contract": "One PR description plus the cited acceptance issue and its evidence tree; the reviewer reads claims and citations, not the candidate code", + "produced_artifact": "A per-checklist review result that either accepts the citation or marks the summary unverified, using only PASS/FAIL/INCONCLUSIVE vocabulary", + "failure_modes": [ + "No acceptance issue URL, only a chat thread", + "Branch names instead of full baseline and candidate SHAs", + "Microbenchmark evidence offered for an end-to-end latency claim", + "A threshold restated or invented relative to the acceptance issue", + "Evidence path absent, stale or on a different SHA than the cited candidate" + ], + "refusal_point": "Refuses to restate a performance or quality claim whose artifacts are missing, stale or off-SHA; reports the summary as unverified instead" + }, + "recommendations": [ + "Read the cited acceptance issue before the PR description so an invented threshold is visible immediately.", + "Resolve every listed evidence path and compare its SHA with the candidate SHA the PR names.", + "Report an unverifiable claim as unverified rather than repeating its number." + ], + "limitations": [ + "Checklist-only procedure; it validates citation discipline, not the measurement behind an artifact.", + "A passing review does not establish that the campaign evidence itself is correct." + ], + "provenance_readsets": [], + "searchspace_not_applicable": "SKILL.md §§ Checklist and Red flags check acceptance links, revision/artifact citations and claim wording; they provide no hardware-resource mechanism." + } +--- + +# Cite-check a downstream PR + +Use when reviewing external PRs that claim performance or quality improvements tied to RSI acceptance work. + +## Checklist + +- [ ] Links an acceptance issue URL (not only a chat thread) +- [ ] States **baseline SHA** and **candidate SHA** matching that issue +- [ ] Points to `evidence//results/` (or equivalent published mirror) +- [ ] Does **not** invent new thresholds or restate bars differently +- [ ] Marks summaries **unverified** if artifacts are missing, stale, or off-SHA +- [ ] Uses verdict vocabulary only: `PASS` / `FAIL` / `INCONCLUSIVE` + +## Red flags + +- “X% faster” without artifact path +- Branch names instead of full SHAs +- Microbench-only support for e2e latency claims + +## Reference + +[docs/contributing.md#evidence-and-claims](https://github.com/InferMatrixLab/InferRSI/blob/b2f60f913a949f9073686a23476f752e11d709a4/docs/contributing.md#evidence-and-claims) diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/governance/claim-contract/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/claim-contract/SKILL.md new file mode 100644 index 000000000..581dd6611 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/workflows/governance/claim-contract/SKILL.md @@ -0,0 +1,73 @@ +--- +name: claim-contract +description: Enforce ThinkFlowLab claim contract wording and verdict rules without embedding campaign-specific revision pins. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "claim", + "wording", + "verdict", + "evidence", + "portable" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Wording and verdict rules for any public performance or quality claim produced from an RSI campaign", + "prerequisites": "A completed campaign with pinned baseline and candidate SHAs and an evidence tree; the rules themselves carry no numeric threshold", + "input_contract": "A claim draft, the campaign RunSpec and its evidence tree under evidence//", + "produced_artifact": "A claim sentence in the pinned form, or a refusal when the campaign verdict is not PASS", + "failure_modes": [ + "Claim asserted before the campaign reaches PASS with artifacts", + "Baseline or candidate SHA written into a shared template, skill or default CI environment", + "Microbenchmark result presented as end-to-end completion", + "A lesson transferred across an unverified repo x revision x backend x device x model x scenario tuple" + ], + "refusal_point": "Refuses any public claim lacking a PASS verdict and artifacts under evidence/, and refuses numeric thresholds because thresholds belong to the campaign" + }, + "recommendations": [ + "Keep campaign SHAs in the acceptance issue; a shared template that carries them will be wrong for the next campaign.", + "Write the verdict from the evidence tree before writing the claim sentence.", + "Treat each repo x revision x backend x device x model x scenario tuple as an independent claim." + ], + "limitations": [ + "States wording and verdict rules only; it selects no numeric gate and validates no measurement.", + "SIAS plane boundaries constrain where a change belongs, not whether it is correct." + ], + "provenance_readsets": [] + } +--- + +# Claim contract (portable) + +## Public claim wording + +Use: “on {repo}@{candidate} vs baseline@{baseline}, under {RunSpec}, we observed …” + +Only after acceptance **`PASS`** with artifacts under `evidence/`. + +## Rules (not numbers) + +| Verdict | When | +| --- | --- | +| PASS | Pinned SHAs, artifact tree on candidate tip, gates satisfied | +| FAIL | Gates failed with complete evidence | +| INCONCLUSIVE | Cannot defend comparison (env, partial run, missing validator freeze) | + +## What not to do + +- Do not bake baseline/candidate SHAs into shared repo templates, skills, or default CI env. +- Do not treat microbench alone as e2e campaign completion. +- Do not auto-transfer lessons across unverified repo × revision × backend × device × model × scenario tuples. + +## SIAS + +Respect plane boundaries: `harness/` orchestrates, `model/` defines quality, `infra/` supplies execution. Cross-plane deps belong in RunSpec. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/governance/draft-acceptance-issue/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/draft-acceptance-issue/SKILL.md new file mode 100644 index 000000000..874096066 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/workflows/governance/draft-acceptance-issue/SKILL.md @@ -0,0 +1,74 @@ +--- +name: draft-acceptance-issue +description: Draft a campaign acceptance issue with checklist, RunSpec, evidence paths, and placeholders for baseline/candidate SHAs. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "acceptance", + "issue", + "runspec", + "template", + "campaign" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Opening a new acceptance campaign in an RSI-style repository, before any baseline or candidate run", + "prerequisites": "A chosen campaign subject and an empty evidence// tree copied from evidence/_template/; SHAs may still be placeholders", + "input_contract": "A campaign id of the form {model-or-scenario}/{device}x{count}, any user-supplied criteria or accepted plan, and available workload/baseline/calibration evidence for deriving missing targets", + "produced_artifact": "An acceptance issue body with campaign id, pinned baseline and candidate SHAs, RunSpec, criteria source and derivation including gaps and freeze authority, gates, evidence paths, initial verdict and a downstream citation blurb", + "failure_modes": [ + "Gates written after the search loop has already run", + "Campaign SHAs committed into recipes/, benchmarks/, skills or workflow defaults", + "PASS claimed without a validator/ freeze and a results/ rationale" + ], + "refusal_point": "Refuses to state a verdict other than INCONCLUSIVE while the validator freeze or results rationale is absent" + }, + "recommendations": [ + "Use user-supplied criteria; when absent, follow the performance-ceiling-analysis acceptance procedure to formulate targets, preserving partial requirements and identifying derived fields and evidence gaps.", + "Write the gates in plan/ before any search loop; a gate chosen afterwards cannot reject the candidate honestly.", + "Use the and placeholders until the campaign fixes them, and keep the real values in the issue.", + "Set the initial verdict to INCONCLUSIVE so the absence of evidence is recorded rather than assumed away." + ], + "limitations": [ + "Drafts an issue only; it freezes no gate and runs no measurement.", + "The placeholder SHA convention depends on the campaign owner filling them in later." + ], + "provenance_readsets": [], + "searchspace_not_applicable": "SKILL.md §§ Steps and Output prepare an acceptance issue, RunSpec, frozen gates and evidence paths; device identifiers are campaign metadata rather than a hardware-resource mechanism." + } +--- + +# Draft acceptance issue + +Use when starting a new campaign in a ThinkFlowLab RSI-style repo. + +## Steps + +1. Pick `campaign_id` (naming: `{model-or-scenario} / {device}×{count}`). +2. Copy `evidence/_template/` → `evidence//` (empty tree with README stubs). +3. Open issue using `.github/ISSUE_TEMPLATE/campaign-acceptance.yml` or [docs/contributing.md#acceptance-checklist](https://github.com/InferMatrixLab/InferRSI/blob/b2f60f913a949f9073686a23476f752e11d709a4/docs/contributing.md#acceptance-checklist). +4. Fill **pinned baseline SHA** and **pinned candidate SHA** in the issue—use placeholders `` / `` until known. +5. Resolve criteria using the [acceptance-criteria procedure](../../../playbooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md): + use user-supplied criteria; otherwise use `performance-ceiling-analysis` to + formulate targets. Preserve partial requirements, label derived fields and + retain unresolved evidence gaps. List supported gates and their provenance in + `plan/` before candidate search. +6. Set initial verdict `INCONCLUSIVE`. + +## Do not + +- Commit campaign SHAs into `recipes/`, `benchmarks/`, skills, or workflow defaults. +- Claim `PASS` without `validator/` freeze and `results/` rationale. + +## Output + +Issue body sections: campaign id, SHAs, RunSpec, criteria source and derivation (including evidence gaps and freeze authority), gates, evidence paths, verdict, downstream citation blurb. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/SKILL.md similarity index 62% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/SKILL.md index bad4ec804..558fca4e8 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/SKILL.md @@ -1,6 +1,56 @@ --- name: rsi-evidence description: Guide agents to establish acceptance criteria, freeze and cite RSI evidence using plan/results/patch/validator paths and PASS/FAIL/INCONCLUSIVE verdicts. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "evidence", + "verdict", + "runspec", + "provenance", + "review" + ], + "empirical_validation": "not_run", + "provenance_readsets": [ + "references/loopx-goal-governance/README.md" + ], + "applicability": { + "scope": "Preparing acceptance criteria, documenting or reviewing an optimization campaign whose verdict will be cited downstream", + "prerequisites": "For preparation, the user requirements and available model/hardware/workload evidence; for verdicts, pinned baseline and candidate SHAs and an evidence tree under evidence//; conclusions about LoopX goal authority additionally need its governance reference", + "input_contract": "Campaign artifacts in plan/, results/, patch/ and validator/, plus the RunSpec that fixes repo x revision x backend x device x model x scenario", + "produced_artifact": "An evidence record that links paths instead of pasting metrics, and states exactly one of PASS, FAIL or INCONCLUSIVE with its rationale", + "failure_modes": [ + "Metrics pasted into an issue comment with no link to an evidence path", + "A verdict vocabulary outside PASS/FAIL/INCONCLUSIVE", + "PASS recorded on a cleanup or integrity-probe failure", + "Output reused across trials, aliased into a reference, or written outside its own trial directory", + "Source fingerprint taken over a directory overlap that excludes changed tracked files", + "Org-wide default SHAs assumed in a skill or CI instead of read from the acceptance issue" + ], + "refusal_point": "Refuses PASS when the validator freeze, the results rationale or the pinned SHAs are missing, and reports INCONCLUSIVE for that campaign instead" + }, + "recommendations": [ + "Use user-supplied acceptance criteria; otherwise follow performance-ceiling-analysis to formulate targets. Preserve partial requirements and record each derived field's evidence before freezing gates.", + "Link evidence paths in the report; a metric quoted only in a comment cannot be re-checked later.", + "Fingerprint tracked source even when the output directory overlaps it, and exclude only the current run's generated files.", + "Treat a missing upstream-cache release proof as INCONCLUSIVE rather than a PASS with a caveat.", + "Read baseline and candidate SHAs from the acceptance issue for that campaign, never from a skill default.", + "Freeze optional required_trial_checks for complete requests, effective graph controls and clean worker exits; require literal true per trial and retain supporting receipts." + ], + "limitations": [ + "Documents evidence discipline; it performs no measurement and certifies no campaign.", + "The LoopX goal-governance reference is a source-derived candidate and adds no execution permission or empirical validation." + ] + } --- # RSI evidence discipline @@ -11,7 +61,7 @@ Use when documenting or reviewing optimization campaigns in ThinkFlowLab-style r Use the user's supplied criteria, including a referenced accepted plan. If none were supplied, use `performance-ceiling-analysis` to formulate evidence-backed -targets under its [acceptance-criteria procedure](../../methods/performance-ceiling-analysis/references/acceptance-criteria.md). +targets under its [acceptance-criteria procedure](../../../playbooks/measurement/performance-ceiling-analysis/references/acceptance-criteria.md). Preserve partial requirements and derive only missing fields. Record the source of each criterion, derivation evidence and freeze authority before search. Unknown inputs remain explicit; estimates cannot relax supplied or frozen gates. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/LICENSE b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/LICENSE similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/LICENSE rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/LICENSE diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/LICENSE-MIT.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/LICENSE-MIT.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/LICENSE-MIT.md rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/LICENSE-MIT.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/NOTICE.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/NOTICE.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/NOTICE.md rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/NOTICE.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/README.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md similarity index 98% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/README.md rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md index d69e22447..30a79f0ff 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/README.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md @@ -60,7 +60,7 @@ With the optional `skills` parser installed: ```bash inferrsi skills resource rsi-evidence references/loopx-goal-governance/mechanisms.md -inferrsi skills reference optimization-search-policy ../../workflows/rsi-evidence/references/loopx-goal-governance/README.md +inferrsi skills reference optimization-search-policy ../../workflows/governance/rsi-evidence/references/loopx-goal-governance/README.md ``` The generated catalog's `inferrsi.common.skills.rsi_evidence` record links the diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/decision-cases.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/decision-cases.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/decision-cases.md rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/decision-cases.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/mechanisms.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/mechanisms.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/mechanisms.md rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/mechanisms.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/source-ledger.json b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/source-ledger.json similarity index 99% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/source-ledger.json rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/source-ledger.json index e32c4868f..bb5dda691 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/loopx-goal-governance/source-ledger.json +++ b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/loopx-goal-governance/source-ledger.json @@ -17,7 +17,7 @@ "scope": { "governing_issue": "https://github.com/InferMatrixLab/InferRSI/issues/149", "tracking_issue": "https://github.com/InferMatrixLab/InferRSI/issues/91", - "owner": "workflows/rsi-evidence/references/loopx-goal-governance", + "owner": "workflows/governance/rsi-evidence/references/loopx-goal-governance", "related_historical_issue": "https://github.com/InferMatrixLab/InferRSI/issues/137", "related_historical_commit_preserved": "a6f28057b49fe03bb234fa6ce1bac5d61df0c58f" }, diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/omni-experiments.md b/src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/omni-experiments.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/references/omni-experiments.md rename to src/inferrsi/knowledge/data/skill_library/workflows/governance/rsi-evidence/references/omni-experiments.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/SKILL.md similarity index 54% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/SKILL.md index 7aa36da54..4e99c29f3 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/SKILL.md @@ -3,6 +3,111 @@ name: npu-skill-factory description: 'Design and operation of the InferRSI NPU skill factory: artifact lifecycle, skill evaluation contract, corpus selection, hide-and-re-distill procedure, and skill evolution cycle. Use when deciding where new skill artifacts belong, how to distill a candidate skill, or how to run the No-Skill vs Human-Skill vs Distilled-Skill comparison. ' +metadata: + inferrsi: | + { + "schema_version": 1, + "status": "candidate", + "sources": [ + { + "url": "https://github.com/VectorSpaceLab/AREX-Skill", + "commit": "ac3fe1af", + "paths": [ + "docs/disco-meta-skills.md", + "docs/disco-workflows.md", + "src/disco/skills/create-paper-module-skill/SKILL.md", + "src/disco/skills/create-paper-module-skill/scripts/create_skill_skeleton.py", + "src/disco/skills/create-paper-module-skill/scripts/validate_skill_tree.py", + "src/disco/modes/skill-policy.ts", + "src/core/repo-skills-library-manager.ts" + ], + "type": "git", + "revision_precision": "prefix" + }, + { + "url": "https://github.com/SkillOpt/SkillOpt", + "commit": "343db229", + "paths": [ + "skillopt/evaluation/gate.py", + "skillopt/optimizer/skill.py", + "skillopt/optimizer/clip.py", + "skillopt_sleep/cycle.py", + "skillopt_sleep/harvest_sources.py" + ], + "type": "git", + "revision_precision": "prefix" + }, + { + "url": "https://github.com/WillowMindIE/MindIE-SD", + "commit": "8481d39b", + "paths": [ + ".agents/skills/profiling-analyze/SKILL.md", + ".agents/skills/dit-perf-opt/SKILL.md", + ".agents/skills/perf-gate/SKILL.md", + ".agents/skills/dit-parallel-opt/SKILL.md" + ], + "type": "git", + "revision_precision": "prefix" + } + ], + "verification": { + "locally_verified": false, + "evidence_level": "source_grounded", + "note": "Historical repository extraction and sibling schema at 82218b4, not a current local factory implementation. Reviewed paper additions and exact versions live in references/papers-20261015/references/readset.md and docs/reference-matrix.md; no efficacy or paper-to-code equivalence established." + }, + "applicability": { + "scope": "InferRSI skill lifecycle management", + "prerequisites": "Current candidate inventory and source/evaluation evidence; historical sibling execution interfaces are not required for read-only artifact admission", + "input_contract": "A candidate inventory plus the source and evaluation evidence for each candidate skill; read-only artifact admission needs no historical sibling execution interface", + "produced_artifact": "An admitted or refused skill candidate with its source, evidence and limitation record, and its stable id and owning path", + "failure_modes": [ + "A candidate admitted on an inventory line without source or evaluation evidence", + "A stable id or path assigned twice across owners", + "Indexing or admission reported as validation or promotion", + "Paper-to-code equivalence asserted without a pinned readset" + ], + "refusal_point": "Refuses admission when a candidate has neither source evidence nor a declared evaluation state, and records the refusal instead of leaving the field blank" + }, + "recommendations": [ + "Read the methodology counterexamples in references/papers-20261015/cases/ before admitting a candidate derived from a paper.", + "Keep the source snapshot revision as research evidence; it is not a campaign default.", + "Record an unvalidated candidate as candidate with not_run rather than omitting the state." + ], + "tags": [ + "skill-factory", + "distillation", + "evaluation", + "npu", + "ascend" + ], + "profile": "local-skill-v1", + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation." + ], + "provenance_readsets": [ + "references/papers-20261015/cases/methodology-counterexamples.md", + "references/papers-20261015/COMPARISON.md", + "references/papers-20261015/INTEGRATION_NOTES.md", + "references/papers-20261015/knowledge/evidence-routing.md", + "references/papers-20261015/QA.md", + "references/papers-20261015/README.md", + "references/papers-20261015/references/learning-to-commit.md", + "references/papers-20261015/references/protocol-details.md", + "references/papers-20261015/references/readset.md", + "references/papers-20261015/references/repo-to-skill.md", + "references/papers-20261015/references/skillopt.md", + "references/papers-20261015/references/trace2skill.md", + "references/papers-20261015/UNREAD_LEADS.md", + "references/papers-20261015/updates/bounded-updates.md" + ], + "consumption_mode": "packaged", + "searchspace_not_applicable": "SKILL.md §§ Choose the evidence product, Artifact lifecycle, and What this skill does not do define skill-distillation/evaluation methodology; NPU task tiers are evaluation prerequisites, not a hardware-resource mechanism." + } --- # NPU Skill Factory Design @@ -64,12 +169,12 @@ authorized evaluation trail remains unvalidated regardless of packaging. | Artifact | Location | Format | |----------|----------|--------| -| Knowledge records (experience) | `src/inferrsi/knowledge/data/repo_packs//knowledge///.json` | Validated JSON (catalog.py schema) | +| Knowledge records (experience) | `src/inferrsi/knowledge/data/skill_library/records/.json` | Validated JSON (catalog.py schema) | | Bilingual skill library | `src/inferrsi/knowledge/data/skill_library/en//` and `zh-CN//` | SKILL.md + references/ + scripts/ | | Skill bundle (distillation output) | `evidence//plan/agent-source-candidates/` | schema-validated JSON bundle | | Human reference skill | `evidence//plan/human-/SKILL.md` | SKILL.md + references/ | | Candidate skill under evaluation | `evidence//plan/agent-source-candidates//` | SKILL.md + references/ | -| Promoted skill | `skills//SKILL.md` + `metadata.json` | InferRSI skill format | +| Promoted skill | `skills//SKILL.md` (including metadata) | InferRSI skill format | | Skill evaluation evidence | `evidence//results/` | per InferRSI evidence tree | Source: PR #9 `skill_library/` layout (source-grounded, pr/9 @ 4ad551a9); diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/COMPARISON.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/COMPARISON.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/COMPARISON.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/COMPARISON.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/INTEGRATION_NOTES.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/INTEGRATION_NOTES.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/INTEGRATION_NOTES.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/INTEGRATION_NOTES.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/QA.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/QA.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/QA.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/QA.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/README.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/README.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/README.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/README.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/UNREAD_LEADS.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/UNREAD_LEADS.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/UNREAD_LEADS.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/UNREAD_LEADS.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/knowledge/evidence-routing.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/knowledge/evidence-routing.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/knowledge/evidence-routing.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/knowledge/evidence-routing.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/protocol-details.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/protocol-details.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/protocol-details.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/protocol-details.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/readset.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/readset.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/readset.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/readset.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/repo-to-skill.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/repo-to-skill.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/repo-to-skill.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/repo-to-skill.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/skillopt.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/skillopt.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/skillopt.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/skillopt.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/source-manifest.json b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/source-manifest.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/source-manifest.json rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/source-manifest.json diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/trace2skill.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/trace2skill.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/references/trace2skill.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/references/trace2skill.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/updates/bounded-updates.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/updates/bounded-updates.md similarity index 90% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/updates/bounded-updates.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/updates/bounded-updates.md index 3a7381560..6c642767d 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/updates/bounded-updates.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/papers-20261015/updates/bounded-updates.md @@ -6,7 +6,7 @@ All entries: **P; candidate; not_run; not applied**. These are reviewable semant ## E-U1 — Repository memory needs scope, contradiction, and expiration -**Target:** `src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md` and its existing evidence references. +**Target:** `src/inferrsi/data/skill_library/workflows/research/repository-evolution-analysis/SKILL.md` and its existing evidence references. **Increment:** extend the already-covered temporal split with a distinction between accepted historical implementation and durable invariant. Use the [Learning to Commit ledger](../references/learning-to-commit.md) and [E-C2](../cases/methodology-counterexamples.md#e-c2--better-repository-familiarity-can-produce-unnecessary-caution). @@ -18,7 +18,7 @@ All entries: **P; candidate; not_run; not applied**. These are reviewable semant ## E-U2 — Factory output and verification are independent choices -**Target:** `src/inferrsi/data/skill_library/workflows/npu-skill-factory/SKILL.md` and its source/admission notes. +**Target:** `src/inferrsi/data/skill_library/workflows/maintenance/npu-skill-factory/SKILL.md` and its source/admission notes. **Increment:** add the [artifact decision](../knowledge/evidence-routing.md), not a second lifecycle or another factory implementation. @@ -30,7 +30,7 @@ All entries: **P; candidate; not_run; not applied**. These are reviewable semant ## E-U3 — Consolidation proposes; validation decides -**Target:** `src/inferrsi/data/skill_library/methods/skill-evolution-playbook/SKILL.md` and existing evolution references. +**Target:** `src/inferrsi/data/skill_library/workflows/maintenance/skill-evolution-playbook/SKILL.md` and existing evolution references. **Increment:** distinguish evidence-pool consolidation from incumbent replacement using [Trace2Skill](../references/trace2skill.md), [SkillOpt](../references/skillopt.md), and [E-C3](../cases/methodology-counterexamples.md#e-c3--fixing-targeted-failures-can-break-unrelated-successes). @@ -54,7 +54,7 @@ All entries: **P; candidate; not_run; not applied**. These are reviewable semant ## E-U5 — Report costs and gains on comparable denominators -**Target:** existing `src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md` or `src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md`, whichever already owns agent-methodology cost accounting; link from evolution notes instead if outside their current purpose. +**Target:** existing `src/inferrsi/data/skill_library/playbooks/measurement/dense-performance-evidence/SKILL.md` or `src/inferrsi/data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md`, whichever already owns agent-methodology cost accounting; link from evolution notes instead if outside their current purpose. **Increment:** one narrowly placed metric-contract note, using the four ledgers rather than forcing methodology into hardware coverage. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/skill-factory-design-20261014.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/skill-factory-design-20261014.md similarity index 97% rename from src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/skill-factory-design-20261014.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/skill-factory-design-20261014.md index d7aca0064..88d1d21e0 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/references/skill-factory-design-20261014.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/npu-skill-factory/references/skill-factory-design-20261014.md @@ -148,8 +148,7 @@ When an accepted skill is subsequently evolved, preserve the revision chain: ``` skills// - SKILL.md (current promoted version) - metadata.json (includes revision.version and revision.parent) + SKILL.md (current version, including revision.version and revision.parent) history/ v0/SKILL.md (initial promoted version) v1/SKILL.md (first accepted evolution) @@ -180,7 +179,7 @@ One evolution cycle: result is not an improvement. Preserve the incumbent and report regressions or inconclusive outcomes without excluding them silently from success estimates. 5. **Revision lineage:** on accept, copy the current `SKILL.md` to `history/v0/` - and write the accepted edit as the new `SKILL.md`. Update `metadata.json` + and write the accepted edit as the new `SKILL.md`. Update its frontmatter with the new `revision.version`. This cycle does not require automatic trace harvesting in Phase 1. The trigger diff --git a/src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/skill-evolution-playbook/SKILL.md similarity index 67% rename from src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/workflows/maintenance/skill-evolution-playbook/SKILL.md index 2106203cc..55135276c 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/skill-evolution-playbook/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/maintenance/skill-evolution-playbook/SKILL.md @@ -3,6 +3,53 @@ name: skill-evolution-playbook description: 'Practical design choices for evolving agent skills from execution traces: when one trajectory should update a skill, how conflicting lessons are handled, how negative evidence is preserved, how large an edit should be, and when skills should split or merge. Drawn from Trace2Skill and SkillOpt. Use when designing or operating a skill evolution pipeline. ' +metadata: + inferrsi: | + { + "tags": [ + "skill-evolution", + "trace2skill", + "skillopt", + "rsi" + ], + "status": "candidate", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Design choices for evolving agent skills from execution traces, drawn from Trace2Skill and SkillOpt: when a trajectory should update a skill, how conflicting lessons are handled, how negative evidence is preserved, how large an edit should be and when skills split or merge; for designing or operating a skill evolution pipeline", + "prerequisites": "A final-test partition kept outside edit selection and the incumbent skill retained for rollback; changing a skill's scope first needs explicit review of its identity and consumers", + "input_contract": "Execution trajectories or trace IDs with the diagnosed causal failure and the successful behavior an amendment must preserve; the incumbent skill and its revision lineage; held-out evaluation results for candidate edits", + "produced_artifact": "A scoped case or amendment proposal, or a consolidated candidate edit, with rejection records and the versioned revision lineage; a candidate for independently evaluated acceptance, not a replacement decision or an efficacy claim", + "failure_modes": [ + "An anecdote, or agreement among analysts, is taken as efficacy", + "Consolidation output is treated as a replacement decision instead of a candidate awaiting independent acceptance", + "A tie or uncertain result is reported as improvement", + "A conflict is resolved by voting for whichever outcome appeared more often in the pool", + "A rejection is applied to every future use of an idea instead of the comparison it constrains", + "Paper-specific section counts or length budgets are used as universal defaults, or a skill is split on length alone", + "A stronger task solver is assumed to be a stronger improver, or replay rankings and private inner scores are read as outer generalization" + ], + "refusal_point": "Records a conflict that conditioning cannot resolve, where the same context produces different outcomes across runs, as INCONCLUSIVE and flags it for human review instead of choosing a lesson; claims no universal minimum trace count, paper-to-code equivalence or local efficacy" + }, + "recommendations": [ + "Condition a conflicting lesson on its context (when X, do A; when Y, avoid A) before choosing one side.", + "Store each rejection record with its evaluation evidence, outside the deployed task procedure, and give it a retirement condition.", + "Bound the target passage and intended behavior before editing, inspect the actual diff and preserve rollback state.", + "Prefer an amendment when the existing skill owns the workflow; create a new candidate only for a genuinely distinct capability." + ], + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "Original typed source inventory is not normalized; retain source identities and limits in the procedure/readsets." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "searchspace_not_applicable": "SKILL.md §§ When should one trajectory update a skill?, How large should a skill edit be?, and When should skills split or merge? govern skill-text evolution and held-out evaluation, not hardware-resource state or control." + } --- # Skill Evolution Playbook @@ -10,7 +57,7 @@ description: 'Practical design choices for evolving agent skills from execution Status: candidate. Repository-derived notes record Trace2Skill @ 3d0b52a and SkillOpt @ 343db229; the earlier Trace2Skill attribution to 7e5a1d77 belongs to LearningToCommit and does not establish that implementation's behavior. -The [reviewed paper ledgers](../../workflows/npu-skill-factory/references/papers-20261015/references/readset.md) +The [reviewed paper ledgers](../npu-skill-factory/references/papers-20261015/references/readset.md) keep Trace2Skill v5 and SkillOpt v2 separate from these repository pins. No paper-to-code equivalence or local efficacy is established. @@ -28,9 +75,9 @@ final-test partitions outside edit selection, check combined effects and unrelated-success regressions, and retain the incumbent for rollback. A tie or uncertain result must not be reported as improvement. -See [Trace2Skill evidence](../../workflows/npu-skill-factory/references/papers-20261015/references/trace2skill.md), -[SkillOpt evidence](../../workflows/npu-skill-factory/references/papers-20261015/references/skillopt.md) -and [interaction counterexamples](../../workflows/npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md). +See [Trace2Skill evidence](../npu-skill-factory/references/papers-20261015/references/trace2skill.md), +[SkillOpt evidence](../npu-skill-factory/references/papers-20261015/references/skillopt.md) +and [interaction counterexamples](../npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md). ## How are conflicting lessons handled? @@ -147,7 +194,7 @@ ledger and its human-skill/small-model counterexample before claiming transfer. ## Task solver and improver are different claims -Use the [adaptation-interface records](../optimization-search-policy/references/papers-20261015/knowledge/adaptation-interfaces.md) +Use the [adaptation-interface records](../../../playbooks/planning/optimization-search-policy/references/papers-20261015/knowledge/adaptation-interfaces.md) for Dream-RSI replay support, AIDE²'s separate inner/private/outer evaluation boundaries and JitRL's candidate-scoring prerequisites. A stronger task-solving agent is not automatically a stronger improver. The AIDE² improver comparison diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/metadata.json b/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/metadata.json deleted file mode 100644 index 1ce0977dc..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/npu-skill-factory/metadata.json +++ /dev/null @@ -1,103 +0,0 @@ -{ - "schema_version": 1, - "status": "candidate", - "sources": [ - { - "url": "https://github.com/VectorSpaceLab/AREX-Skill", - "commit": "ac3fe1af", - "paths": [ - "docs/disco-meta-skills.md", - "docs/disco-workflows.md", - "src/disco/skills/create-paper-module-skill/SKILL.md", - "src/disco/skills/create-paper-module-skill/scripts/create_skill_skeleton.py", - "src/disco/skills/create-paper-module-skill/scripts/validate_skill_tree.py", - "src/disco/modes/skill-policy.ts", - "src/core/repo-skills-library-manager.ts" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/SkillOpt/SkillOpt", - "commit": "343db229", - "paths": [ - "skillopt/evaluation/gate.py", - "skillopt/optimizer/skill.py", - "skillopt/optimizer/clip.py", - "skillopt_sleep/cycle.py", - "skillopt_sleep/harvest_sources.py" - ], - "type": "git", - "revision_precision": "prefix" - }, - { - "url": "https://github.com/WillowMindIE/MindIE-SD", - "commit": "8481d39b", - "paths": [ - ".agents/skills/profiling-analyze/SKILL.md", - ".agents/skills/dit-perf-opt/SKILL.md", - ".agents/skills/perf-gate/SKILL.md", - ".agents/skills/dit-parallel-opt/SKILL.md" - ], - "type": "git", - "revision_precision": "prefix" - } - ], - "verification": { - "locally_verified": false, - "evidence_level": "source_grounded", - "note": "Historical repository extraction and sibling schema at 82218b4, not a current local factory implementation. Reviewed paper additions and exact versions live in references/papers-20261015/references/readset.md and docs/reference-matrix.md; no efficacy or paper-to-code equivalence established." - }, - "applicability": { - "scope": "InferRSI skill lifecycle management", - "prerequisites": "Current candidate inventory and source/evaluation evidence; historical sibling execution interfaces are not required for read-only artifact admission", - "input_contract": "A candidate inventory plus the source and evaluation evidence for each candidate skill; read-only artifact admission needs no historical sibling execution interface", - "produced_artifact": "An admitted or refused skill candidate with its source, evidence and limitation record, and its stable id and owning path", - "failure_modes": [ - "A candidate admitted on an inventory line without source or evaluation evidence", - "A stable id or path assigned twice across owners", - "Indexing or admission reported as validation or promotion", - "Paper-to-code equivalence asserted without a pinned readset" - ], - "refusal_point": "Refuses admission when a candidate has neither source evidence nor a declared evaluation state, and records the refusal instead of leaving the field blank" - }, - "recommendations": [ - "Read the methodology counterexamples in references/papers-20261015/cases/ before admitting a candidate derived from a paper.", - "Keep the source snapshot revision as research evidence; it is not a campaign default.", - "Record an unvalidated candidate as candidate with not_run rather than omitting the state." - ], - "tags": [ - "skill-factory", - "distillation", - "evaluation", - "npu", - "ascend" - ], - "profile": "local-skill-v1", - "kind": "skill", - "origin": [ - "external" - ], - "empirical_validation": "not_run", - "limitations": [ - "Source-derived candidate; no local efficacy validation." - ], - "provenance_readsets": [ - "references/papers-20261015/cases/methodology-counterexamples.md", - "references/papers-20261015/COMPARISON.md", - "references/papers-20261015/INTEGRATION_NOTES.md", - "references/papers-20261015/knowledge/evidence-routing.md", - "references/papers-20261015/QA.md", - "references/papers-20261015/README.md", - "references/papers-20261015/references/learning-to-commit.md", - "references/papers-20261015/references/protocol-details.md", - "references/papers-20261015/references/readset.md", - "references/papers-20261015/references/repo-to-skill.md", - "references/papers-20261015/references/skillopt.md", - "references/papers-20261015/references/trace2skill.md", - "references/papers-20261015/UNREAD_LEADS.md", - "references/papers-20261015/updates/bounded-updates.md" - ], - "consumption_mode": "packaged", - "searchspace_not_applicable": "SKILL.md §§ Choose the evidence product, Artifact lifecycle, and What this skill does not do define skill-distillation/evaluation methodology; NPU task tiers are evaluation prerequisites, not a hardware-resource mechanism." -} diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/research/competitor-scan/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/competitor-scan/SKILL.md new file mode 100644 index 000000000..92376ead4 --- /dev/null +++ b/src/inferrsi/knowledge/data/skill_library/workflows/research/competitor-scan/SKILL.md @@ -0,0 +1,69 @@ +--- +name: competitor-scan +description: Structure a lightweight competitive scan without inventing benchmark numbers or campaign verdicts. +metadata: + inferrsi: | + { + "schema_version": 1, + "profile": "local-skill-v1", + "kind": "skill", + "status": "candidate", + "origin": [ + "first-party" + ], + "consumption_mode": "packaged", + "tags": [ + "competitive-analysis", + "positioning", + "claim", + "unverified" + ], + "empirical_validation": "not_run", + "applicability": { + "scope": "Qualitative positioning of a new RSI repository against other projects before acceptance evidence exists", + "prerequisites": "A selected repository and readable public material from the alternatives; no benchmark run is required", + "input_contract": "A target repository plus whatever the alternatives publish about what they optimize and how they prove it", + "produced_artifact": "A short Markdown note covering problem, alternatives, differentiation, limits and open questions", + "failure_modes": [ + "Numeric comparison written without a PASS campaign in our own evidence tree", + "Third-party figure repeated as if reproduced under our RunSpec", + "Scan presented as a substitute for acceptance evidence" + ], + "refusal_point": "Refuses any numeric comparison not sourced from our own evidence//results/ with a PASS verdict" + }, + "recommendations": [ + "Answer the four framing questions qualitatively first; the absence of a published claim contract is itself a finding.", + "Label every third-party number unverified unless it was reproduced under our RunSpec.", + "Close the note with open questions rather than a conclusion the evidence cannot support." + ], + "limitations": [ + "Produces positioning material only; it establishes no performance or quality result.", + "Third-party claim contracts are read from public material and may be incomplete." + ], + "provenance_readsets": [], + "searchspace_not_applicable": "SKILL.md §§ Frame, Rules and Deliverable produce a qualitative competitive comparison with evidence limits, not a hardware-resource decision or implementation mechanism." + } +--- + +# Competitor scan (generic) + +Use when positioning a new RSI repo—not to replace acceptance evidence. + +## Frame + +| Question | Answer (qualitative) | +| --- | --- | +| What do others optimize? | _throughput, latency, cost, quality, …_ | +| How do they publish proof? | _blogs, papers, dashboards, none_ | +| Claim contract? | _pinned revisions + reproducible artifacts? often missing_ | +| Our wedge | _smallest proof that unlocks the next bet_ | + +## Rules + +- No numeric comparisons unless sourced from **our** `evidence/.../results/` with `PASS`. +- Label third-party numbers **unverified** if not reproduced under our RunSpec. +- Use the [evidence and claim rules](https://github.com/InferMatrixLab/InferRSI/blob/b2f60f913a949f9073686a23476f752e11d709a4/docs/contributing.md#evidence-and-claims) when writing a comparison. + +## Deliverable + +Short markdown: problem, alternatives, differentiation (claim contract + SIAS), limits, open questions. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/SKILL.md similarity index 90% rename from src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/SKILL.md index afd0f6d69..57771fb2b 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/SKILL.md @@ -13,13 +13,13 @@ Use this collection to prepare a dedicated engine, compare architectures or desi | Question | Route | |---|---| | Prepare a new model/hardware pair: discovery, reference selection, full weights and feasibility | [Model-aware preparation](references/model-aware-preparation.md) | -| MiniMax-H3 native execution, component lifetimes and Apple Silicon feasibility | [h3.c residency audit](../../repositories/h3-c/audit-h3-runtime-residency/SKILL.md), including its source inventory and comparisons | +| MiniMax-H3 native execution, component lifetimes and Apple Silicon feasibility | [h3.c residency audit](../../../systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md), including its source inventory and comparisons | | Compare graph, cache, lifetime, stop and completion contracts across source slices | [Cross-repository mechanisms](references/cross-repository-mechanisms.md) and [source scope](references/repository-source-scope.md) | -| Strata heterogeneous MoE, CPU/GPU expert placement, memory and speculative execution | [strata](../../repositories/strata/SKILL.md) | -| vLLM token scheduling, KV/prefix reuse, execution/parallelism or model integration | [vllm](../../repositories/vllm/SKILL.md) | -| SGLang scheduling, Radix/HiCache, prefill/decode, parallel runtime or extensions | [sglang](../../repositories/sglang/SKILL.md) | -| SGLang-Omni pipelines, stage/scheduler ownership, streaming and transport | [sglang-omni](../../repositories/sglang-omni/SKILL.md) | -| vLLM-Omni model/deploy separation, AR/diffusion execution, stage transport and heterogeneous outputs | [vllm-omni](../../repositories/vllm-omni/SKILL.md) | +| Strata heterogeneous MoE, CPU/GPU expert placement, memory and speculative execution | [strata](../../../systems/engines/strata/SKILL.md) | +| vLLM token scheduling, KV/prefix reuse, execution/parallelism or model integration | [vllm](../../../systems/engines/vllm/SKILL.md) | +| SGLang scheduling, Radix/HiCache, prefill/decode, parallel runtime or extensions | [sglang](../../../systems/engines/sglang/SKILL.md) | +| SGLang-Omni pipelines, stage/scheduler ownership, streaming and transport | [sglang-omni](../../../systems/engines/sglang-omni/SKILL.md) | +| vLLM-Omni model/deploy separation, AR/diffusion execution, stage transport and heterogeneous outputs | [vllm-omni](../../../systems/engines/vllm-omni/SKILL.md) | ## Comparison procedure diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/cross-repository-mechanisms.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/cross-repository-mechanisms.md similarity index 83% rename from src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/cross-repository-mechanisms.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/cross-repository-mechanisms.md index 2920f6364..64fac68ff 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/cross-repository-mechanisms.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/cross-repository-mechanisms.md @@ -22,20 +22,20 @@ review, compatibility matrix or comparable performance result is established. | Responsibility | Source-supported examples | Transferable question or invariant | Assumptions that remain local | |---|---|---|---| -| **Loader and component construction** | stable-diffusion.cpp separates model catalog, parameter storage and compute staging. Diffusers can create a new pipeline wrapper while passing existing components through `from_pipe`. MLX-LM's cache command loads a model and writes an artifact. | Establish model/component byte identity, construction ownership and the point at which actual weights become usable. Distinguish constructing a wrapper, registering metadata, loading bytes, staging weights and first-request readiness. | This batch does not establish a complete checkpoint/storage-I/O optimization. Diffusers component reuse is not checkpoint loading; MLX-LM loader internals were not deeply read; DeepInfer's loader was largely navigation. Reuse existing loading/storage owners for those questions. [Stable](../../../repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md), [Diffusers](../../../frameworks/diffusers/diffusers-invocation-state/references/source-map.md), [MLX-LM](../../../repositories/mlx-lm/generation-cache-lifecycle/references/source-map.md). | -| **Graph and execution** | ComfyUI executes a workflow dependency graph with lazy/dynamic dependencies. MLX constructs a lazy primitive graph, then evaluates it with stream dependencies. llama.cpp reuses a compatible frontend graph result after memory preparation and refreshes its inputs. stable-diffusion.cpp selects a monolithic or eligible segmented native graph. | Trace producer → effective dispatch → input readiness → consumer → completion. A reusable plan still needs current inputs and valid resource lifetimes. Observe the actual branch before measuring it. | A ComfyUI node graph, an MLX lazy graph, llama.cpp frontend graph reuse and a backend-native captured graph are different objects. None implies fusion, capture replay or transfer overlap in the others. Segmentation eligibility and backend implementations are source-specific. [ComfyUI](../../../repositories/comfy-ui/references/graph-cache-invalidation.md), [MLX](../../../frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md), [llama.cpp](../../../repositories/llama-cpp/audit-llama-runtime-state/references/source-notes.md), [Stable](../../../repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md). | -| **Allocation and lifetime** | MLX backend work can retain buffers after graph detachment. ComfyUI can partially unload weights while retaining a wrapper. stable-diffusion.cpp separates weights, retained runtime and pending allocations; unpinning is distinct from eviction. SGLang closes component-use intervals while policy may retain modules or prefetch the next request. llama.cpp memory and server prompt state have separate owners. | Name every owner and last consumer. Distinguish a logical state transition, permission to reclaim, actual allocation release, allocator cache and device completion. Account for simultaneously live current/queued work without counting reusable storage twice. | Array buffers, model weights, workflow outputs, attention state and request records have different identities. A logical token/cache offset is not allocated capacity. CPU reroutes or bypasses important ComfyUI/stable accelerator paths. SGLang's concrete residency strategies and MLX allocator internals remain outside the respective readsets. [MLX](../../../frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md), [ComfyUI](../../../repositories/comfy-ui/references/model-residency.md), [Stable](../../../repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md), [SGLang](../../../repositories/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md). | -| **Cache validity and retained state** | ComfyUI output signatures include dependencies/fingerprints, while retention has a separate policy. MLX-LM prompt reuse depends on model/adapter/draft identity, token-prefix state, copying and cache-specific trimming. llama.cpp graph compatibility does not replace memory updates. DeepInfer compacts query rows while preserving required full-layout state. | First establish computation/state identity and dependency closure; then decide retention or reclamation. A hit, retained object or shorter logical view does not by itself prove equivalent future computation. External inputs and model bytes need explicit identity. | Node-output caches, prompt KV state, recurrent state, graph arenas and denoising scheduler state are not one cache schema. DeepInfer's retained query suffix is not the whole historical-key set. Diffusers/SGLang scheduler objects are mutable execution state, not a generic cache entry. [ComfyUI](../../../repositories/comfy-ui/references/graph-cache-invalidation.md), [MLX-LM](../../../repositories/mlx-lm/generation-cache-lifecycle/references/cache-stop.md), [DeepInfer full-state path](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/v4_modules/attn.py#L672-L719). | -| **Scheduling, stopping and cancellation** | DeepInfer pipelines CPU scheduling and device work, with explicit stale-state repair. MLX-LM cooperatively removes a UID while preserving aligned survivor rows. Diffusers Flux skips remaining iterations after interruption and can still reach normal output cleanup. llama.cpp queues cancellation and may retain a stateful prompt. SGLang timeout/socket close or video deletion is not per-generation device abort. vLLM-Omni distinguishes enqueue, correlated logical acknowledgment and queued transfer reclamation. | Follow the same request/task/sequence identity across admission, work, stop observation, output finalization and cleanup. Record which event actually occurred. A caller ceasing to wait does not retract work; a release or acknowledgment must name its owner and completion scope. | Partial llama.cpp ubatches can survive abort/fatal failure; recovery is not whole-batch rollback. An HTTP task, numerical diffusion scheduler, engine queue and device kernel are different lifetimes. Do not introduce retries or concurrency changes without the existing caller contract. [llama.cpp](../../../repositories/llama-cpp/audit-llama-runtime-state/SKILL.md), [MLX-LM](../../../repositories/mlx-lm/generation-cache-lifecycle/SKILL.md), [Diffusers](../../../frameworks/diffusers/diffusers-invocation-state/SKILL.md), [SGLang](../../../repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md), [vLLM-Omni](../../../repositories/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md). | -| **Transport and synchronization** | DeepInfer separates CPU scheduling metadata from worker-side GPU confidence repair. MLX and ggml use backend-specific fences, events, copy buffers or synchronization. SGLang has rank policies and a request transport. vLLM-Omni routes abort/release by request-to-replica binding and correlates results separately. | Distinguish data dependency, device-stream ordering, collective participation, message delivery and resource reclamation. Retain buffers until the real receiver/consumer is done. Verify each affected rank/replica and preserve correlation when a waiter disappears. | H2D/D2H copies and a local IPC socket are not network throughput evidence. A barrier declaration does not qualify cross-rank exception recovery. Source-level async capability does not establish useful overlap. Vendor, connector and driver internals have independent pins and unread boundaries. [DeepInfer budget source notes](../../../repositories/deepinfer/budget-speculative-verification/references/source-notes.md), [llama.cpp](../../../repositories/llama-cpp/audit-llama-runtime-state/references/source-notes.md), [SGLang](../../../repositories/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md), [vLLM-Omni](../../../repositories/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md). | +| **Loader and component construction** | stable-diffusion.cpp separates model catalog, parameter storage and compute staging. Diffusers can create a new pipeline wrapper while passing existing components through `from_pipe`. MLX-LM's cache command loads a model and writes an artifact. | Establish model/component byte identity, construction ownership and the point at which actual weights become usable. Distinguish constructing a wrapper, registering metadata, loading bytes, staging weights and first-request readiness. | This batch does not establish a complete checkpoint/storage-I/O optimization. Diffusers component reuse is not checkpoint loading; MLX-LM loader internals were not deeply read; DeepInfer's loader was largely navigation. Reuse existing loading/storage owners for those questions. [Stable](../../../../systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md), [Diffusers](../../../../systems/frameworks/diffusers/diffusers-invocation-state/references/source-map.md), [MLX-LM](../../../../systems/engines/mlx-lm/generation-cache-lifecycle/references/source-map.md). | +| **Graph and execution** | ComfyUI executes a workflow dependency graph with lazy/dynamic dependencies. MLX constructs a lazy primitive graph, then evaluates it with stream dependencies. llama.cpp reuses a compatible frontend graph result after memory preparation and refreshes its inputs. stable-diffusion.cpp selects a monolithic or eligible segmented native graph. | Trace producer → effective dispatch → input readiness → consumer → completion. A reusable plan still needs current inputs and valid resource lifetimes. Observe the actual branch before measuring it. | A ComfyUI node graph, an MLX lazy graph, llama.cpp frontend graph reuse and a backend-native captured graph are different objects. None implies fusion, capture replay or transfer overlap in the others. Segmentation eligibility and backend implementations are source-specific. [ComfyUI](../../../../systems/engines/comfy-ui/references/graph-cache-invalidation.md), [MLX](../../../../systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md), [llama.cpp](../../../../systems/engines/llama-cpp/audit-llama-runtime-state/references/source-notes.md), [Stable](../../../../systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md). | +| **Allocation and lifetime** | MLX backend work can retain buffers after graph detachment. ComfyUI can partially unload weights while retaining a wrapper. stable-diffusion.cpp separates weights, retained runtime and pending allocations; unpinning is distinct from eviction. SGLang closes component-use intervals while policy may retain modules or prefetch the next request. llama.cpp memory and server prompt state have separate owners. | Name every owner and last consumer. Distinguish a logical state transition, permission to reclaim, actual allocation release, allocator cache and device completion. Account for simultaneously live current/queued work without counting reusable storage twice. | Array buffers, model weights, workflow outputs, attention state and request records have different identities. A logical token/cache offset is not allocated capacity. CPU reroutes or bypasses important ComfyUI/stable accelerator paths. SGLang's concrete residency strategies and MLX allocator internals remain outside the respective readsets. [MLX](../../../../systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md), [ComfyUI](../../../../systems/engines/comfy-ui/references/model-residency.md), [Stable](../../../../systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/references/phase-residency.md), [SGLang](../../../../systems/engines/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md). | +| **Cache validity and retained state** | ComfyUI output signatures include dependencies/fingerprints, while retention has a separate policy. MLX-LM prompt reuse depends on model/adapter/draft identity, token-prefix state, copying and cache-specific trimming. llama.cpp graph compatibility does not replace memory updates. DeepInfer compacts query rows while preserving required full-layout state. | First establish computation/state identity and dependency closure; then decide retention or reclamation. A hit, retained object or shorter logical view does not by itself prove equivalent future computation. External inputs and model bytes need explicit identity. | Node-output caches, prompt KV state, recurrent state, graph arenas and denoising scheduler state are not one cache schema. DeepInfer's retained query suffix is not the whole historical-key set. Diffusers/SGLang scheduler objects are mutable execution state, not a generic cache entry. [ComfyUI](../../../../systems/engines/comfy-ui/references/graph-cache-invalidation.md), [MLX-LM](../../../../systems/engines/mlx-lm/generation-cache-lifecycle/references/cache-stop.md), [DeepInfer full-state path](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/v4_modules/attn.py#L672-L719). | +| **Scheduling, stopping and cancellation** | DeepInfer pipelines CPU scheduling and device work, with explicit stale-state repair. MLX-LM cooperatively removes a UID while preserving aligned survivor rows. Diffusers Flux skips remaining iterations after interruption and can still reach normal output cleanup. llama.cpp queues cancellation and may retain a stateful prompt. SGLang timeout/socket close or video deletion is not per-generation device abort. vLLM-Omni distinguishes enqueue, correlated logical acknowledgment and queued transfer reclamation. | Follow the same request/task/sequence identity across admission, work, stop observation, output finalization and cleanup. Record which event actually occurred. A caller ceasing to wait does not retract work; a release or acknowledgment must name its owner and completion scope. | Partial llama.cpp ubatches can survive abort/fatal failure; recovery is not whole-batch rollback. An HTTP task, numerical diffusion scheduler, engine queue and device kernel are different lifetimes. Do not introduce retries or concurrency changes without the existing caller contract. [llama.cpp](../../../../systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md), [MLX-LM](../../../../systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md), [Diffusers](../../../../systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md), [SGLang](../../../../systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md), [vLLM-Omni](../../../../systems/engines/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md). | +| **Transport and synchronization** | DeepInfer separates CPU scheduling metadata from worker-side GPU confidence repair. MLX and ggml use backend-specific fences, events, copy buffers or synchronization. SGLang has rank policies and a request transport. vLLM-Omni routes abort/release by request-to-replica binding and correlates results separately. | Distinguish data dependency, device-stream ordering, collective participation, message delivery and resource reclamation. Retain buffers until the real receiver/consumer is done. Verify each affected rank/replica and preserve correlation when a waiter disappears. | H2D/D2H copies and a local IPC socket are not network throughput evidence. A barrier declaration does not qualify cross-rank exception recovery. Source-level async capability does not establish useful overlap. Vendor, connector and driver internals have independent pins and unread boundaries. [DeepInfer budget source notes](../../../../systems/engines/deepinfer/budget-speculative-verification/references/source-notes.md), [llama.cpp](../../../../systems/engines/llama-cpp/audit-llama-runtime-state/references/source-notes.md), [SGLang](../../../../systems/engines/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md), [vLLM-Omni](../../../../systems/engines/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md). | | **Measurement and acceptance** | All nine slices identify meaningful work/completion boundaries and unrun native checks. DeepInfer's verification objective separates verification rows from accepted tokens and includes unchanged draft work in total cost. MLX/MLX-LM make submission versus completion important. Diffusion paths distinguish preparation, denoising, decoding and finalization. | Freeze the exact workload, numerical/quality contract, backend, model bytes, device count and effective branch. Verify correctness first; separately record startup, first request, warm work, host/device peaks, transfer events, failure cleanup and service completion as applicable. | No matched cross-repository benchmark exists here. Do not rank throughput, compare different token/media work units, copy source margins into acceptance thresholds or treat package checks as native evidence. Earlier author-reported figures are not reproduced results in this synthesis. [Evidence boundary](repository-source-scope.md#validation-boundary). | ## DeepInfer contracts that constrain transfer These source contracts keep query work, historical state, confidence estimates and sampling outcomes separate. The complete procedures remain with the -[bounded-replay owner](../../../repositories/deepinfer/bound-windowed-attention-replay/SKILL.md) and -[verification-budget owner](../../../repositories/deepinfer/budget-speculative-verification/SKILL.md). +[bounded-replay owner](../../../../systems/engines/deepinfer/bound-windowed-attention-replay/SKILL.md) and +[verification-budget owner](../../../../systems/engines/deepinfer/budget-speculative-verification/SKILL.md). | Question | Source-supported statement | Consequence for another implementation | |---|---|---| diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/import.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/import.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/import.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/import.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/model-aware-preparation.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/model-aware-preparation.md similarity index 99% rename from src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/model-aware-preparation.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/model-aware-preparation.md index 2f49d11fb..99821e856 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/model-aware-preparation.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/model-aware-preparation.md @@ -39,7 +39,7 @@ negative results and verification state alongside their full linked procedures. Known profiles supply only source-grounded discovery terms and component roles; their linked repository skills own the detailed source map and inventories. The -MiniMax-H3 profile points to the [h3.c residency audit](../../../repositories/h3-c/audit-h3-runtime-residency/SKILL.md). +MiniMax-H3 profile points to the [h3.c residency audit](../../../../systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md). For an unknown model, supply `--task` and `--modality` after inspecting its sources; no architecture is guessed. Repeat dimensions to add queries. A text query to `knowledge search` and each research query use the same Find diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/overview.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/overview.md similarity index 86% rename from src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/overview.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/overview.md index dc937582c..06e94be21 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/overview.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/overview.md @@ -10,18 +10,18 @@ For a new model/hardware pair, first follow [model-aware preparation](model-awar for independent model/framework/task/modality/mechanism discovery, native reference selection and complete weight/residency accounting. Model-specific procedures, inventories and comparisons stay with their repository owners; for MiniMax-H3, -use the [h3.c residency audit](../../../repositories/h3-c/audit-h3-runtime-residency/SKILL.md). +use the [h3.c residency audit](../../../../systems/engines/h3-c/audit-h3-runtime-residency/SKILL.md). The five original source snapshots below remain independent imports; the preparation procedure does not change their provenance or establish new runtime compatibility. | Repository | Knowledge emphasized | Skill | |---|---|---| -| Strata | Heterogeneous MoE, CPU/GPU placement, memory and speculation | [Skill](../../../repositories/strata/SKILL.md) | -| vLLM | Token/admission/input budgets, KV ownership and prefix reuse, scheduler/runner/attention, parallel serving | [Skill](../../../repositories/vllm/SKILL.md) | -| SGLang | Scheduler and cache reuse, Radix/HiCache, prefill/decode and parallelism, extension and tuning | [Skill](../../../repositories/sglang/SKILL.md) | -| SGLang-Omni | Pipeline/stage runtime, scheduler IO boundary, stream/backpressure/transport, model integration | [Skill](../../../repositories/sglang-omni/SKILL.md) | -| vLLM-Omni | Fixed topology vs deploy resources, stage affinity/connectors, AR/diffusion modes, output/state contracts | [Skill](../../../repositories/vllm-omni/SKILL.md) | +| Strata | Heterogeneous MoE, CPU/GPU placement, memory and speculation | [Skill](../../../../systems/engines/strata/SKILL.md) | +| vLLM | Token/admission/input budgets, KV ownership and prefix reuse, scheduler/runner/attention, parallel serving | [Skill](../../../../systems/engines/vllm/SKILL.md) | +| SGLang | Scheduler and cache reuse, Radix/HiCache, prefill/decode and parallelism, extension and tuning | [Skill](../../../../systems/engines/sglang/SKILL.md) | +| SGLang-Omni | Pipeline/stage runtime, scheduler IO boundary, stream/backpressure/transport, model integration | [Skill](../../../../systems/engines/sglang-omni/SKILL.md) | +| vLLM-Omni | Fixed topology vs deploy resources, stage affinity/connectors, AR/diffusion modes, output/state contracts | [Skill](../../../../systems/engines/vllm-omni/SKILL.md) | | Collection router | Cross-engine reasoning and a unified constrained search workflow | [Skill](../SKILL.md) | ## What to learn across repositories @@ -63,12 +63,12 @@ Existing AREX skills were inspected as prior work and refreshed against source r Construction follows AREX-Skill `distill-ml-knowledge`: **scope → ground → construct → verify**, using its direct construction strategy, provenance/routing patterns, operating metadata and review criteria. This is a portable candidate preview. The DisCo managed CLI/importer, full environment preparation and GPU acceptance were not executed. See [construction record](import.md) and [verification](import.md). -Each repository is independently readable, with an entry point and progressively loaded references/sub-skills. The optional collection router links from `workflows/inference-engine-research/` to the five engine packages under `repositories/`; preserve that library layout when using the router. No global skill was installed or overwritten. +Each repository is independently readable, with an entry point and progressively loaded references/sub-skills. The optional collection router links from `workflows/research/inference-engine-research/` to the five engine packages under `repositories/`; preserve that library layout when using the router. No global skill was installed or overwritten. ## Try it ```text -From the skill library root, read workflows/inference-engine-research/SKILL.md. For a new inference engine, derive a constrained search space from these repositories. Separate existing settings, implementation changes and research extensions. For each dimension give a source anchor, compatibility constraints, correctness tests and measurements. Hardware: [fill in]; model/revision: [fill in]; workload and SLO: [fill in]. +From the skill library root, read workflows/research/inference-engine-research/SKILL.md. For a new inference engine, derive a constrained search space from these repositories. Separate existing settings, implementation changes and research extensions. For each dimension give a source anchor, compatibility constraints, correctness tests and measurements. Hardware: [fill in]; model/revision: [fill in]; workload and SLO: [fill in]. ``` For one repository, read its SKILL.md directly. The source code and model weights are excluded from the library; immutable source URLs and verification recipes are retained. GPU correctness, native distributed behavior and performance still require the target environment. diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/repository-source-scope.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/repository-source-scope.md similarity index 62% rename from src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/repository-source-scope.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/repository-source-scope.md index a8ff1a2cd..f2e27bb8d 100644 --- a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/repository-source-scope.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/repository-source-scope.md @@ -34,15 +34,15 @@ loading, host-placement and offload references retain their independent pins. | Repository | Canonical owner | Selected mechanism | Deferred or excluded | |---|---|---|---| -| DeepInfer | [DeepInfer owner](../../../repositories/deepinfer/bound-windowed-attention-replay/SKILL.md) and [verification budget](../../../repositories/deepinfer/budget-speculative-verification/SKILL.md) | Dependency-closed bounded query replay with separately preserved full state; gated ring margins; verification-row budgeting, draft/verify cost separation and CPU/GPU confidence-round ownership | No native equivalence or sampling-correctness run; model-specific dimensions, vendor/kernel bodies, in-flight preemption safety and backend parity. | -| ComfyUI | [ComfyUI owner](../../../repositories/comfy-ui/SKILL.md) | Dependency-complete output identity versus object identity/retention; lazy consumers and external fingerprints; wrapper/patch identity versus partial/full weight release | Full executor/cache suites, third-party custom nodes, provider storage, native dynamic allocator, model families, device/quality/performance. No LLM KV or storage-tuning claim. | -| llama.cpp | [llama.cpp owner](../../../repositories/llama-cpp/audit-llama-runtime-state/SKILL.md) | Partial ubatch progress and memory-specific rollback; frontend graph compatibility plus fresh inputs/readiness; task/slot/prompt retention and cancellation boundary | Native context/server replay tests; recurrent/hybrid generalization; backend capture/kernel behavior; loading, quantization, full stream sessions and model quality. | -| MLX | [MLX owner](../../../frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) | Graph construction/evaluation/availability; stream fence/event dependencies; backend buffer retention after graph detach | MLX execution on any backend; allocator internals, compile/fusion, numerical operator coverage, distributed execution and model/service qualification. Does not own LLM request/cache policy. | -| MLX-LM | [MLX-LM owner](../../../repositories/mlx-lm/generation-cache-lifecycle/SKILL.md) | Prefill/decode lookahead boundaries; pending/prompt/generation UID and aligned rows; model/token prefix identity, copy/trim/retention; cooperative stop | Native/model/HTTP tests; model-specific cache families, full speculation/VLM/distributed semantics and combined MLX compatibility. No paged/radix allocator inference. | -| Diffusers | [Diffusers owner](../../../frameworks/diffusers/diffusers-invocation-state/SKILL.md) | Flux wrapper/component identity and `from_pipe` sharing; selected FlowMatchEuler scheduler reset/index progression; interruption and normal output tail | Real scheduler/pipeline test, serving caller serialization, other pipeline/scheduler classes, exception cleanup, offload implementation and image quality. No framework-wide thread-safety verdict. | -| stable-diffusion.cpp | [stable-diffusion.cpp owner](../../../repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | Native image entry path; managed weights/runtime budget versus device headroom; protected segment pin/preparation, optional prefetch and synchronized activation/cleanup | Matching unexpanded GGML/private headers, model-specific numerical runner chain, device memory/async behavior, graph-cut algorithm, complete loader I/O, video/audio/server. CPU bypasses decisive accelerator gates. | -| SGLang | [SGLang owner](../../../repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | Default ParallelExecutor and entry/cleanup intervals; component retention and request-tail prefetch; mutable numerical scheduler aliases/dedup copies; timeout/job deletion versus device completion | Concrete offload strategies, pool/disaggregation, realtime/rollout, complete CUDA IPC exception lifetime, model kernels and native quality/performance. Existing LLM/Radix/HiCache is not repinned. | -| vLLM-Omni | [Existing validation owner](../../../repositories/vllm-omni/sub-skills/omni-validation/SKILL.md) and [abort scope](../../../repositories/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md) | Abort submission versus correlated logical receipt versus queued per-replica transfer reclamation; partial AR output/processor commit; waiter/late-result ownership; post-submit wait timeout excludes blocking submission | Native abort guarantee, connector internals, every serving wrapper, full duplex/session lifecycle, all-device reclaim, model/media semantics and service/performance. | +| DeepInfer | [DeepInfer owner](../../../../systems/engines/deepinfer/bound-windowed-attention-replay/SKILL.md) and [verification budget](../../../../systems/engines/deepinfer/budget-speculative-verification/SKILL.md) | Dependency-closed bounded query replay with separately preserved full state; gated ring margins; verification-row budgeting, draft/verify cost separation and CPU/GPU confidence-round ownership | No native equivalence or sampling-correctness run; model-specific dimensions, vendor/kernel bodies, in-flight preemption safety and backend parity. | +| ComfyUI | [ComfyUI owner](../../../../systems/engines/comfy-ui/SKILL.md) | Dependency-complete output identity versus object identity/retention; lazy consumers and external fingerprints; wrapper/patch identity versus partial/full weight release | Full executor/cache suites, third-party custom nodes, provider storage, native dynamic allocator, model families, device/quality/performance. No LLM KV or storage-tuning claim. | +| llama.cpp | [llama.cpp owner](../../../../systems/engines/llama-cpp/audit-llama-runtime-state/SKILL.md) | Partial ubatch progress and memory-specific rollback; frontend graph compatibility plus fresh inputs/readiness; task/slot/prompt retention and cancellation boundary | Native context/server replay tests; recurrent/hybrid generalization; backend capture/kernel behavior; loading, quantization, full stream sessions and model quality. | +| MLX | [MLX owner](../../../../systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md) | Graph construction/evaluation/availability; stream fence/event dependencies; backend buffer retention after graph detach | MLX execution on any backend; allocator internals, compile/fusion, numerical operator coverage, distributed execution and model/service qualification. Does not own LLM request/cache policy. | +| MLX-LM | [MLX-LM owner](../../../../systems/engines/mlx-lm/generation-cache-lifecycle/SKILL.md) | Prefill/decode lookahead boundaries; pending/prompt/generation UID and aligned rows; model/token prefix identity, copy/trim/retention; cooperative stop | Native/model/HTTP tests; model-specific cache families, full speculation/VLM/distributed semantics and combined MLX compatibility. No paged/radix allocator inference. | +| Diffusers | [Diffusers owner](../../../../systems/frameworks/diffusers/diffusers-invocation-state/SKILL.md) | Flux wrapper/component identity and `from_pipe` sharing; selected FlowMatchEuler scheduler reset/index progression; interruption and normal output tail | Real scheduler/pipeline test, serving caller serialization, other pipeline/scheduler classes, exception cleanup, offload implementation and image quality. No framework-wide thread-safety verdict. | +| stable-diffusion.cpp | [stable-diffusion.cpp owner](../../../../systems/engines/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md) | Native image entry path; managed weights/runtime budget versus device headroom; protected segment pin/preparation, optional prefetch and synchronized activation/cleanup | Matching unexpanded GGML/private headers, model-specific numerical runner chain, device memory/async behavior, graph-cut algorithm, complete loader I/O, video/audio/server. CPU bypasses decisive accelerator gates. | +| SGLang | [SGLang owner](../../../../systems/engines/sglang/audit-sglang-diffusion-lifetimes/SKILL.md) | Default ParallelExecutor and entry/cleanup intervals; component retention and request-tail prefetch; mutable numerical scheduler aliases/dedup copies; timeout/job deletion versus device completion | Concrete offload strategies, pool/disaggregation, realtime/rollout, complete CUDA IPC exception lifetime, model kernels and native quality/performance. Existing LLM/Radix/HiCache is not repinned. | +| vLLM-Omni | [Existing validation owner](../../../../systems/engines/vllm-omni/sub-skills/omni-validation/SKILL.md) and [abort scope](../../../../systems/engines/vllm-omni/sub-skills/omni-validation/references/abort-acknowledgment.md) | Abort submission versus correlated logical receipt versus queued per-replica transfer reclamation; partial AR output/processor commit; waiter/late-result ownership; post-submit wait timeout excludes blocking submission | Native abort guarantee, connector internals, every serving wrapper, full duplex/session lifecycle, all-device reclaim, model/media semantics and service/performance. | ## DeepInfer state and sampling boundary @@ -77,7 +77,7 @@ P = performance or efficacy. | Slice | N | M | S | P | Separate bounded evidence | |---|---|---|---|---|---| | DeepInfer | not_run | not_run | not_run | not_run | Source and unexecuted test assertions | -| ComfyUI | not_run | not_run | not_run | not_run | Seven checks passed on the unchanged complete pure-Python graph_utils module; see the [owner boundary](../../../repositories/comfy-ui/references/validation-boundaries.md). This does not cover executor, cache-signature, model or allocator behavior. | +| ComfyUI | not_run | not_run | not_run | not_run | Seven checks passed on the unchanged complete pure-Python graph_utils module; see the [owner boundary](../../../../systems/engines/comfy-ui/references/validation-boundaries.md). This does not cover executor, cache-signature, model or allocator behavior. | | llama.cpp | not_run | not_run | not_run | not_run | Source and unexecuted test assertions | | MLX | not_run | not_run | not_run | not_run | CPU, Metal and CUDA execution remain separately unqualified | | MLX-LM | not_run | not_run | not_run | not_run | Selected test classes load external model fixtures | diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/source-manifest.json b/src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/source-manifest.json similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/workflows/inference-engine-research/references/source-manifest.json rename to src/inferrsi/knowledge/data/skill_library/workflows/research/inference-engine-research/references/source-manifest.json diff --git a/src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/SKILL.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/SKILL.md similarity index 80% rename from src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/SKILL.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/SKILL.md index e3eceb7b5..1ee25d8ae 100644 --- a/src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/SKILL.md +++ b/src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/SKILL.md @@ -3,6 +3,112 @@ name: repository-evolution-analysis description: 'Analyze a repository''s commit history to build a capability-evolution model: what the system can do, how it got there, and what patterns allow it to iterate quickly. Use to produce history-derived knowledge for agent skill conditioning, and to design temporal-split benchmarks for skill evaluation. ' +metadata: + inferrsi: | + { + "schema_version": 1, + "status": "candidate", + "sources": [ + { + "url": "https://github.com/LearningToCommit/LearningToCommit", + "commit": "7e5a1d77", + "paths": [ + "data_construction/s1_scan_commits.py", + "data_construction/s3_tag.py", + "data_construction/s4_sample.py", + "data_construction/s5_build.py", + "modules/agent.py", + "modules/prompt_builder.py", + "modules/trajectory_to_md.py", + "system_prompts/solve_with_skill.txt", + "system_prompts/learn_reflect.txt", + "evaluation/pairwise_judge.py", + "evaluation/programmatic_metrics.py" + ], + "type": "git", + "revision_precision": "prefix" + }, + { + "url": "https://github.com/yetone/magpie", + "commit": "77800032", + "paths": [ + "internal/", + "docs/" + ], + "type": "git", + "revision_precision": "prefix" + }, + { + "url": "https://github.com/yetone/magpie", + "commit": "4b5d6d725fae3d8a263f6e43c707400e3765b84f", + "paths": [ + "internal/library/rtk.go", + "internal/library/rtk_upgrade.go", + "internal/proc/path_other.go", + "internal/proc/path_windows.go", + "internal/provider/install_windows.go" + ], + "type": "git", + "revision_precision": "full" + }, + { + "url": "https://github.com/yetone/magpie", + "commit": "9848a9062a65c49e074c09770dc7de73dd012bea", + "paths": [ + "internal/library/rtk.go", + "internal/library/rtk_upgrade.go", + "internal/proc/path_other.go", + "internal/proc/path_windows.go", + "internal/provider/install_windows.go", + "internal/library/rtk_path.go", + "internal/library/rtk_path_other.go", + "internal/library/rtk_path_windows.go", + "internal/library/rtk_path_test.go", + "internal/library/rtk_test.go", + "internal/library/rtk_upgrade_test.go", + "internal/library/library_test.go", + "internal/gui/library.go", + "internal/gui/assets/library.js", + "internal/gui/tests/library-rtk-path.test.cjs", + "library_cli.go", + "LICENSE" + ], + "type": "git", + "revision_precision": "full" + } + ], + "verification": { + "locally_verified": false, + "evidence_level": "source_grounded", + "note": "Carried LearningToCommit pipeline and Magpie history analysis, not causal evidence of development speed. The reviewed paper amendment is linked from SKILL.md; paper-to-code equivalence is not established. First-hand Magpie coordination research remains pending Dot delivery and owner review. The RTK worked transition is pinned static source/test inspection only; no upstream execution or heldout efficacy evaluation." + }, + "applicability": { + "scope": "any software repository with a recoverable git history", + "prerequisites": "git history access, optionally LearningToCommit-style benchmark for evaluation" + }, + "tags": [ + "repository-intelligence", + "history", + "skill-distillation", + "evaluation" + ], + "profile": "local-skill-v1", + "kind": "skill", + "origin": [ + "external" + ], + "empirical_validation": "not_run", + "limitations": [ + "Source-derived candidate; no local efficacy validation.", + "The RTK transition is a learning example, not an independent heldout task; upstream tests were inspected, not executed.", + "Existing audit and synthetic KV comparison adapters are not a generic historical coding-task evaluator." + ], + "provenance_readsets": [], + "consumption_mode": "packaged", + "references": [ + "references/rtk-consumer-path-transition.md" + ] + } --- # Repository Evolution Analysis @@ -248,7 +354,7 @@ This is directory-based access, not verbatim insertion of the skill body. ### Paper claims and version boundary -The [existing Learning to Commit paper ledger](../../workflows/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md) +The [existing Learning to Commit paper ledger](../../maintenance/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md) records the v1 method, author-reported pilot outcomes and limitations. Those paper claims are separate from the public-code observations above; this source reconciliation does not reproduce them or establish paper-to-code equivalence. @@ -287,8 +393,8 @@ rule. Revise or retire lessons when later accepted evidence contradicts them. Freeze adaptation inputs before future-task evaluation; audit issue synthesis and source access independently. Report correctness, localization, patch size, style and resource use separately. The -[Learning to Commit paper ledger](../../workflows/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md) -and [convention-expiry counterexample](../../workflows/npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md#e-c2--better-repository-familiarity-can-produce-unnecessary-caution) +[Learning to Commit paper ledger](../../maintenance/npu-skill-factory/references/papers-20261015/references/learning-to-commit.md) +and [convention-expiry counterexample](../../maintenance/npu-skill-factory/references/papers-20261015/cases/methodology-counterexamples.md#e-c2--better-repository-familiarity-can-produce-unnecessary-caution) extend the existing temporal-split method. These methodology papers do not verify Magpie's actual development coordination or authorize another history scan. diff --git a/src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/references/capability-presence-audit.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/references/capability-presence-audit.md diff --git a/src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/references/rtk-consumer-path-transition.md b/src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/references/rtk-consumer-path-transition.md similarity index 100% rename from src/inferrsi/knowledge/data/skill_library/methods/repository-evolution-analysis/references/rtk-consumer-path-transition.md rename to src/inferrsi/knowledge/data/skill_library/workflows/research/repository-evolution-analysis/references/rtk-consumer-path-transition.md diff --git a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/metadata.json b/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/metadata.json deleted file mode 100644 index 682e349e2..000000000 --- a/src/inferrsi/knowledge/data/skill_library/workflows/rsi-evidence/metadata.json +++ /dev/null @@ -1,48 +0,0 @@ -{ - "schema_version": 1, - "profile": "local-skill-v1", - "kind": "skill", - "status": "candidate", - "origin": [ - "first-party" - ], - "consumption_mode": "packaged", - "tags": [ - "evidence", - "verdict", - "runspec", - "provenance", - "review" - ], - "empirical_validation": "not_run", - "provenance_readsets": [ - "references/loopx-goal-governance/README.md" - ], - "applicability": { - "scope": "Preparing acceptance criteria, documenting or reviewing an optimization campaign whose verdict will be cited downstream", - "prerequisites": "For preparation, the user requirements and available model/hardware/workload evidence; for verdicts, pinned baseline and candidate SHAs and an evidence tree under evidence//; conclusions about LoopX goal authority additionally need its governance reference", - "input_contract": "Campaign artifacts in plan/, results/, patch/ and validator/, plus the RunSpec that fixes repo x revision x backend x device x model x scenario", - "produced_artifact": "An evidence record that links paths instead of pasting metrics, and states exactly one of PASS, FAIL or INCONCLUSIVE with its rationale", - "failure_modes": [ - "Metrics pasted into an issue comment with no link to an evidence path", - "A verdict vocabulary outside PASS/FAIL/INCONCLUSIVE", - "PASS recorded on a cleanup or integrity-probe failure", - "Output reused across trials, aliased into a reference, or written outside its own trial directory", - "Source fingerprint taken over a directory overlap that excludes changed tracked files", - "Org-wide default SHAs assumed in a skill or CI instead of read from the acceptance issue" - ], - "refusal_point": "Refuses PASS when the validator freeze, the results rationale or the pinned SHAs are missing, and reports INCONCLUSIVE for that campaign instead" - }, - "recommendations": [ - "Use user-supplied acceptance criteria; otherwise follow performance-ceiling-analysis to formulate targets. Preserve partial requirements and record each derived field's evidence before freezing gates.", - "Link evidence paths in the report; a metric quoted only in a comment cannot be re-checked later.", - "Fingerprint tracked source even when the output directory overlaps it, and exclude only the current run's generated files.", - "Treat a missing upstream-cache release proof as INCONCLUSIVE rather than a PASS with a caveat.", - "Read baseline and candidate SHAs from the acceptance issue for that campaign, never from a skill default.", - "Freeze optional required_trial_checks for complete requests, effective graph controls and clean worker exits; require literal true per trial and retain supporting receipts." - ], - "limitations": [ - "Documents evidence discipline; it performs no measurement and certifies no campaign.", - "The LoopX goal-governance reference is a source-derived candidate and adds no execution permission or empirical validation." - ] -} diff --git a/src/inferrsi/knowledge/knowledge_find.py b/src/inferrsi/knowledge/knowledge_find.py index d2d56b6cb..dce4793bc 100644 --- a/src/inferrsi/knowledge/knowledge_find.py +++ b/src/inferrsi/knowledge/knowledge_find.py @@ -27,7 +27,7 @@ import shlex import sys from collections import Counter -from collections.abc import Iterable, Mapping +from collections.abc import Callable, Iterable, Mapping from dataclasses import dataclass from functools import lru_cache from importlib.resources import files @@ -84,6 +84,8 @@ } _BACKEND_NAME_ALIASES = {"昇腾": "ascend"} _MAX_SNIPPET = 400 +_MAX_LIMITATION_BYTES = 400 +_MAX_SHOWN_LIMITATIONS = 3 # A case can record a dozen directions; listing all of them buries the one the # caller asked about. The rest stay available through the cited file. _MAX_SHOWN_FINDINGS = 3 @@ -302,7 +304,16 @@ def view( if scope: record["applicability"] = str(scope).strip()[:_MAX_SNIPPET] if self.limitations: - record["limitations"] = list(self.limitations) + shown = [] + for limitation in self.limitations[:_MAX_SHOWN_LIMITATIONS]: + preview = [*shown, limitation] + preview_bytes = json.dumps(preview, ensure_ascii=False).encode("utf-8") + if len(preview_bytes) > _MAX_LIMITATION_BYTES: + break + shown = preview + record["limitations"] = shown + if len(shown) < len(self.limitations): + record["limitations_withheld"] = len(self.limitations) - len(shown) if self.unverified(): record["locally_verified"] = False return record @@ -467,17 +478,29 @@ def _skill_documents(root: Path, *, source_location: str) -> list[Document]: and carries no index, so the default search raised on a library whose cases `find` could already rank and `read` could already open. """ - from .skill_catalog import _sidecar, skill_registrations + from .skill_catalog import _sidecar, _skill_search_source, skill_registrations documents = [] for name, (directory, backend) in sorted(skill_registrations(root).items()): core = directory / "SKILL.md" body = _body_text(core) - description = _skill_description(body) try: - metadata = _sidecar(directory) or {} + body, metadata = _skill_search_source(body) + except ValueError: + # Malformed structured data is neither evidence nor instruction text. + continue + if metadata is not None and (directory / "metadata.json").exists(): + continue + try: + if metadata is None: + metadata = _sidecar(directory) or {} except (ValueError, OSError): metadata = {} + # Task entrypoints remain addressable through the registry. Counting + # an orientation page as evidence also changes IDF for every owner. + if metadata.get("discovery_role") == "task-entry": + continue + description = _skill_description(body) applicability = metadata.get("applicability", {}) scope = applicability.get("scope") if isinstance(applicability, dict) else applicability documents.append(Document( @@ -613,14 +636,17 @@ def installed_skill_root() -> Path: return Path(files("inferrsi").joinpath("knowledge", "data", "skill_library")).resolve() -def installed_skill_projection(document: Document, records: Mapping[str, dict]) -> dict | None: +def installed_skill_projection( + document: Document, records: Mapping[str, dict] | Callable[[str], dict | None], +) -> dict | None: """Find the generated record for this registered installed Skill identity.""" if ( document.kind != "skill" or document.source_location != "installed" or document.backend is None or document.root != installed_skill_root() ): return None - record = records.get(packaged_skill_id(document.identity, document.backend)) + lookup = records.get if isinstance(records, Mapping) else records + record = lookup(packaged_skill_id(document.identity, document.backend)) if record is None or record.get("index_type") != "packaged-skill": return None return record @@ -646,7 +672,9 @@ def load_documents( if "record" in kinds: from .catalog import knowledge_records - documents.extend(_record_documents(knowledge_records() if records is None else records)) + if records is None: + records = knowledge_records(include_skill_views=False) + documents.extend(_record_documents(records)) identities = [document.qualified for document in documents] if len(set(identities)) != len(identities): duplicate = next(i for i, c in Counter(identities).items() if c > 1) @@ -669,7 +697,8 @@ def _aliases() -> tuple[tuple[str, tuple[str, ...]], ...]: differently -- a caller asking about "startup" should reach a finding that says "initialization". Both expand one way, at query time only. """ - path = files("inferrsi").joinpath("knowledge", "data", "query_aliases.json") + config = files("inferrsi").joinpath("knowledge", "data", "skill_library", "config") + path = config.joinpath("query_aliases.json") record = json.loads(path.read_text(encoding="utf-8")) if record.get("schema_version") != 1: raise ValueError("Query alias map requires schema_version 1") @@ -720,21 +749,21 @@ def expand(query: str) -> tuple[list[str], dict[str, tuple[str, ...]]]: return [token for token in dict.fromkeys(found) if token not in _QUERY_STOP_WORDS], applied -def _open_args(document: Document, readers: dict) -> tuple[str, ...]: - """Validate material and build the read command for a returned hit.""" +def _read_preview(document: Document, readers: dict) -> tuple[tuple[str, ...], int | None]: + """Validate a hit, returning its command and UTF-8 content size without extra reads.""" from .knowledge_operations import KnowledgeReader if not document.qualified: - return () + return (), None reader = readers.get(document.root) if reader is None: reader = readers[document.root] = KnowledgeReader(skill_root=document.root) try: - reader.read(document.qualified).view(material=True) + material = reader.read(document.qualified).view(material=True) except MissingSkillsExtra: if document.source_location != "installed": raise - record = installed_skill_projection(document, reader.records()) + record = installed_skill_projection(document, reader.record) if record is None: raise package_reader = readers.get(None) @@ -745,16 +774,18 @@ def _open_args(document: Document, readers: dict) -> tuple[str, ...]: view = bundle.view() core, _, _ = indexed_skill(record, {**view, "authority": "package"}, bundle.content) bundle.content[core].decode("utf-8") - return ( + args = ( "read", ref, "--omit-resources", "--resource", core, "--revision", view["revision"], ) + return args, len(bundle.content[core]) parts = ("read", document.qualified, "--material", "--omit-resources") - if document.root is None: - return parts - if document.source_location == "installed": - return (*parts, "--installed-library") - return (*parts, "--skill-root", str(document.root)) + if document.root is not None: + if document.source_location == "installed": + parts += ("--installed-library",) + else: + parts += ("--skill-root", str(document.root)) + return parts, len(material["content"].encode("utf-8")) def rank(query: str, corpus: Corpus) -> tuple[ @@ -797,12 +828,15 @@ def find( for value, _, index, detail in scored[:limit]: document = corpus.documents[index] try: - read_args, unreadable = _open_args(document, readers), None + read_args, size = _read_preview(document, readers) + unreadable = None except (ValueError, KeyError, OSError, TypeError) as exc: - read_args, unreadable = (), str(exc) or type(exc).__name__ + read_args, size, unreadable = (), None, str(exc) or type(exc).__name__ view = document.view( score=value, query_tokens=query_tokens, read_args=read_args, unreadable=unreadable, ) + if size is not None: + view["next_read_content_bytes"] = size if explain: view["matched"] = detail items.append(view) diff --git a/src/inferrsi/knowledge/knowledge_operations.py b/src/inferrsi/knowledge/knowledge_operations.py index 3ac313cd3..36c559a89 100644 --- a/src/inferrsi/knowledge/knowledge_operations.py +++ b/src/inferrsi/knowledge/knowledge_operations.py @@ -16,9 +16,10 @@ context_view, indexed_skill, record_context, + skill_procedure, validate_relevance_method, ) -from .catalog import KNOWLEDGE_SCOPES, SCENARIO_ALIASES, knowledge_records +from .catalog import KNOWLEDGE_SCOPES, SCENARIO_ALIASES, knowledge_records, lookup_record if TYPE_CHECKING: from .knowledge_find import Document @@ -227,7 +228,16 @@ def view(self, *, material: bool = False) -> dict: "dependencies": manifest["dependencies"], } if material: - result["content"] = self.content[self.metadata["entrypoint"]].decode("utf-8") + path = self.metadata["entrypoint"] + data = self.content[path] + if self.metadata["kind"] == "skill": + data, locator = skill_procedure(path, data) + if locator: + result["content_view"] = { + **locator, "sha256": _hash(data), + "source_sha256": manifest["resources"][path], + } + result["content"] = data.decode("utf-8") return result @@ -276,6 +286,9 @@ def _tree( ancestors = ancestors | {directory} result = {} for child in sorted(node.iterdir(), key=lambda p: p.name): + # Record/config/build inputs have independent owners. Only skill resources enter this pin. + if prefix == "data/skill_library" and child.name in {"records", "config", "_compiled"}: + continue if child.name in INCIDENTAL or child.name.endswith(".pyc"): continue path = safe_path(f"{prefix}/{child.name}") @@ -329,6 +342,7 @@ def collect_service_context(self, cue, *, relevance_method, limit, context_bytes else: body = _json(record) path = "record.json" + body, locator = skill_procedure(path, body) if len(contexts) >= limit or used + len(body) > context_bytes: omissions.append( { @@ -347,7 +361,8 @@ def collect_service_context(self, cue, *, relevance_method, limit, context_bytes "sources": sources, "content_bytes": len(body), "score": score, - "resources": [{"path": path, "role": "core", "content": body.decode("utf-8")}], + "resources": [{"path": path, "role": "core", "content": body.decode("utf-8"), + **locator}], "omissions": [ {"path": target, "reason": "linked_document_not_loaded"} for target in record.get("skill_metadata", {}).get("references", []) @@ -384,6 +399,7 @@ def __init__(self, *, skill_root: Path | None = None, store=None): raise ValueError("Store must reserve identities from the same checkout") self.store = store self._records: dict[str, dict] | None = None + self._opened_records: dict[str, dict | None] = {} self._library: tuple[dict[str, bytes], str] | None = None def records(self) -> dict[str, dict]: @@ -392,6 +408,14 @@ def records(self) -> dict[str, dict]: self._records = {record["id"]: record for record in knowledge_records()} return self._records + def record(self, identity: str) -> dict | None: + """Read one validated owner, honoring an already-loaded or supplied catalog.""" + if self._records is not None: + return self._records.get(identity) + if identity not in self._opened_records: + self._opened_records[identity] = lookup_record(identity) + return self._opened_records[identity] + def _skill_library(self) -> tuple[dict[str, bytes], str]: if self._library is None: # K02 moved the physical corpus under the Knowledge module. The bundle @@ -418,10 +442,11 @@ def owners(self) -> dict[str, list[str]]: def _checkout_identities(self) -> list[str]: if self.skill_root is None: return [] - # Enumeration reads registered names. Collect validates sidecars per item. + # Enumeration reads registered names. Collect validates metadata per item. from .skill_catalog import skill_directories - identities = [*skill_directories(self.skill_root), *self._cases()] + directories = skill_directories(self.skill_root) + identities = [*directories, *self._cases(directories)] if len(set(identities)) != len(identities): raise IdentityError("collision", "Duplicate identity inside the checkout") return identities @@ -434,7 +459,18 @@ def resolve(self, reference: str, owners: dict[str, list[str]] | None = None) -> authority, qualified, identity = reference.rpartition(":") if not re.fullmatch(IDENTITY, identity) or (qualified and authority not in AUTHORITIES): raise IdentityError("invalid_identity", f"Invalid identity: {reference}") - found = (self.owners() if owners is None else owners).get(identity, []) + if owners is not None: + found = owners.get(identity, []) + else: + found = [] + if (not qualified or authority == "package") and self.record(identity) is not None: + found.append("package") + if ((not qualified or authority == "checkout") + and identity in self._checkout_identities()): + found.append("checkout") + if (not qualified or authority == "store") and self.store is not None: + if identity in self.store.identities(): + found.append("store") if qualified: if authority not in found: raise IdentityError("unknown_identity", f"Unknown identity: {reference}") @@ -456,12 +492,14 @@ def cases(self) -> dict[str, Path]: """ return self._cases() - def _cases(self) -> dict[str, Path]: + def _cases(self, directories: dict[str, Path] | None = None) -> dict[str, Path]: result = {} if self.skill_root is not None: from .skill_catalog import skill_directories - for directory in skill_directories(self.skill_root).values(): + if directories is None: + directories = skill_directories(self.skill_root) + for directory in directories.values(): for path in sorted(directory.glob("cases/*.json")): if path.is_symlink() or not path.resolve().is_relative_to(self.skill_root): raise ValueError("Case must stay inside the selected skills root") @@ -529,7 +567,7 @@ def _checkout(self, identity: str) -> Bundle: return self._skill(identity) def _knowledge(self, identity: str) -> Bundle: - record = self.records().get(identity) + record = self.record(identity) if record is None: raise IdentityError("unknown_identity", f"Unknown knowledge ID: {identity}") content = {"record.json": _json(record)} @@ -576,17 +614,16 @@ def _knowledge(self, identity: str) -> Bundle: ) def _skill(self, identity: str) -> Bundle: - from .skill_catalog import get_skill, skill_directory, skill_metadata, skill_reference + from .skill_catalog import _effective, _reference_path, skill_directory root = self.skill_root - record = get_skill(root, identity) directory = skill_directory(root, identity) + metadata = _effective(directory) prefix = directory.relative_to(root).as_posix() content = _tree(directory, prefix, root=root) - metadata = skill_metadata(root, identity) dependencies = {} for reference in metadata.get("references", []): - path = skill_reference(root, identity, reference) + path = _reference_path(root, directory, reference) key = path.relative_to(root).as_posix() content[key] = path.read_bytes() dependencies[key] = "sha256:" + _hash(content[key]) @@ -595,7 +632,7 @@ def _skill(self, identity: str) -> Bundle: "id": identity, "kind": "skill", "entrypoint": f"{prefix}/SKILL.md", - "title": record["description"], + "title": metadata["description"], "origin": metadata.get("origin", ["unknown"]), "sources": _sources(metadata.get("sources", [])) or [ @@ -605,7 +642,7 @@ def _skill(self, identity: str) -> Bundle: } ], "applicability": metadata.get("applicability", {}), - "state": record.get("status", "unspecified"), + "state": metadata.get("status", "unspecified"), "limitations": metadata.get("limitations", []) + [ "Checkout-only local skill; no automatic execution or promotion.", diff --git a/src/inferrsi/knowledge/research.py b/src/inferrsi/knowledge/research.py index 214f1c7b6..f3e0b1a02 100644 --- a/src/inferrsi/knowledge/research.py +++ b/src/inferrsi/knowledge/research.py @@ -11,13 +11,14 @@ from .knowledge_find import rank PROCEDURE = ( - "data/skill_library/workflows/inference-engine-research/" + "data/skill_library/workflows/research/inference-engine-research/" "references/model-aware-preparation.md" ) def _profile(model: str) -> dict | None: - root = files("inferrsi").joinpath("knowledge", "data", "research_profiles") + config = files("inferrsi").joinpath("knowledge", "data", "skill_library", "config") + root = config.joinpath("research_profiles") for resource in sorted(root.iterdir(), key=lambda path: path.name): if resource.name.endswith(".json"): profile = json.loads(resource.read_text(encoding="utf-8")) diff --git a/src/inferrsi/knowledge/skill_catalog.py b/src/inferrsi/knowledge/skill_catalog.py index 183a507aa..379189a99 100644 --- a/src/inferrsi/knowledge/skill_catalog.py +++ b/src/inferrsi/knowledge/skill_catalog.py @@ -7,7 +7,7 @@ from typing import BinaryIO from ._knowledge_errors import MissingSkillsExtra -from .catalog import BACKENDS +from .catalog import BACKENDS, _skill_search_source, _split_skill_metadata _SKILLS_EXTRA = ( "Reading checkout skills requires the 'skills' extra (PyYAML); install it " @@ -43,7 +43,7 @@ def _yaml(): return yaml -def _frontmatter(stream: BinaryIO) -> dict: +def _frontmatter_parts(stream: BinaryIO) -> tuple[dict, dict | None]: if stream.readline().strip() != b"---": raise ValueError("SKILL.md must start with YAML frontmatter") lines = [] @@ -53,13 +53,31 @@ def _frontmatter(stream: BinaryIO) -> dict: lines.append(line) else: raise ValueError("Unclosed SKILL.md frontmatter") + header = b"".join(lines).decode("utf-8") + _, metadata = _split_skill_metadata(header) yaml = _yaml() try: - record = yaml.safe_load(b"".join(lines).decode("utf-8")) + record = yaml.safe_load(header) except yaml.YAMLError as exc: raise ValueError(f"Invalid skill YAML: {exc}") from exc if not isinstance(record, dict): raise ValueError("Skill frontmatter must be a mapping") + host = record.get("metadata") + if isinstance(host, dict) and "inferrsi" in host: + literal = host.pop("inferrsi") + if not isinstance(literal, str) or metadata is None or json.loads(literal) != metadata: + raise ValueError("Use a metadata.inferrsi literal block of JSON with 2/4-space indents") + if not host: + del record["metadata"] + elif metadata is not None: + raise ValueError("Embedded metadata disagrees with YAML metadata.inferrsi") + return record, metadata + + +def _frontmatter(stream: BinaryIO) -> dict: + record, metadata = _frontmatter_parts(stream) + if metadata is not None: + record.update(metadata) return record @@ -230,13 +248,17 @@ def _applicability(record: dict, name: str) -> None: def _effective(directory: Path) -> dict: # Read bytes line by line; parse only the frontmatter, never decode the body. with _entry(directory, "SKILL.md").open("rb") as stream: - record = _frontmatter(stream) + record, embedded = _frontmatter_parts(stream) for key in ("name", "description"): if not isinstance(record.get(key), str) or not record[key].strip(): raise ValueError(f"Skill requires {key}: {directory.name}") if record["name"] != directory.name: raise ValueError(f"Skill name disagrees with directory: {directory.name}") metadata = _sidecar(directory) + if embedded is not None: + if metadata is not None: + raise ValueError(f"Skill declares metadata in two files: {directory.name}") + metadata = embedded if metadata is not None: for key in record.keys() & metadata.keys(): if record[key] != metadata[key]: @@ -245,9 +267,9 @@ def _effective(directory: Path) -> dict: if any(key in metadata for key in ("name", "description")): raise ValueError("Local skill name and description belong in SKILL.md") if set(record) != {"name", "description"}: - raise ValueError("Local skill structured fields belong in metadata.json") - # Only sidecars declare these fields, and JSON cannot express a container - # cycle; validating the sidecar leaves the bundle normalizer free to report + raise ValueError("Local skill structured fields belong in metadata.inferrsi") + # Structured JSON cannot express a container cycle. + # Validating these fields leaves the bundle normalizer free to report # a cyclic legacy frontmatter value against the field that contains it. _strings(metadata, "recommendations", directory.name) _applicability(metadata, directory.name) @@ -275,7 +297,7 @@ def _metadata(directory: Path) -> dict: def skill_metadata(root: Path, name: str) -> dict: - """Effective legacy frontmatter plus sidecar fields, after conflict checks.""" + """Effective frontmatter, with sidecar support for historical checkouts.""" return _effective(skill_directory(root, name)) @@ -305,6 +327,25 @@ def skill_instructions(root: Path, name: str) -> str: return stream.read().decode("utf-8").lstrip("\r\n") +def primary_task_instructions(root: Path, name: str) -> str: + """Open the registered primary procedure before looking for supporting evidence.""" + directory = skill_directory(root, name) + instructions = _entry(directory, "SKILL.md").read_text(encoding="utf-8") + procedure, metadata = _skill_search_source(instructions) + if metadata is None: + metadata = _sidecar(directory) or {} + tags = metadata.get("tags") + if (metadata.get("discovery_role") != "task-entry" + or not isinstance(tags, list) or "primary-task" not in tags): + raise ValueError(f"Not a primary inference task skill: {name}") + # Deliver the complete instructions without repeating the structured + # discovery fields. Raw owner bytes stay available through pinned reads. + sections = re.split(r"^---[ \t]*$", procedure, maxsplit=2, flags=re.MULTILINE) + if len(sections) != 3 or sections[0] or not sections[2].strip(): + raise ValueError(f"Primary task skill requires frontmatter and instructions: {name}") + return procedure + + def skill_resource(root: Path, name: str, resource: str) -> Path: """Resolve one explicitly selected resource inside its skill directory.""" directory = skill_directory(root, name) @@ -321,6 +362,11 @@ def skill_reference(root: Path, name: str, reference: str) -> Path: directory = skill_directory(root, name) if reference not in _references(_effective(directory), directory.name): raise ValueError(f"Undeclared skill reference: {reference}") + return _reference_path(root, directory, reference) + + +def _reference_path(root: Path, directory: Path, reference: str) -> Path: + """Resolve a reference already declared by this owner.""" if "\\" in reference or Path(reference).is_absolute(): raise ValueError("Reference must use a relative POSIX path") path = (directory / reference).resolve() diff --git a/tests/check_installed.py b/tests/check_installed.py index 58bb4e996..e124a04c1 100644 --- a/tests/check_installed.py +++ b/tests/check_installed.py @@ -20,7 +20,7 @@ from inferrsi.benchmark.skill_eval_engineering import REQUEST from inferrsi.contracts import RunSpec, claim, evidence, json_v1 from inferrsi.knowledge._source_snapshot import BUNDLE_LIBRARY, LIBRARY -from inferrsi.knowledge.catalog import knowledge_records, search_knowledge +from inferrsi.knowledge.catalog import _split_skill_metadata, knowledge_records, search_knowledge from inferrsi.knowledge.knowledge_operations import KnowledgeReader from inferrsi.resources import repository_packs from inferrsi.resources.repository_packs import REPO_TYPES, repository_pack @@ -41,6 +41,12 @@ def bundle_key(logical): return logical +def owner_fields(skill_bytes): + _, fields = _split_skill_metadata(skill_bytes.decode("utf-8").split("---", 2)[1]) + assert isinstance(fields, dict) + return fields + + def resource(logical): """Resolve a logical knowledge key to its installed physical location.""" relative = bundle_key(logical) @@ -55,8 +61,8 @@ def resource(logical): for resource_name in ("task.json", "TASK.md", "benchmark.py", "LICENSE"): assert root.joinpath(resource_name).is_file(), (benchmark_name, resource_name) records = knowledge_records() -assert len(records) == 128, "Knowledge is missing from the installation" -assert resource("data/repo_packs/vllm_omni/knowledge/README.md").is_file() +assert len(records) == 136, "Knowledge is missing from the installation" +assert resource("data/skill_library/records/README.md").is_file() assert repository_pack("vllm_rlt")["id"] == "vllm_rlt" assert search_knowledge(repo="vllm_omni", backend="ascend") assert all(not item["verification"]["locally_verified"] for item in records) @@ -68,7 +74,7 @@ def resource(logical): mlx_records = search_knowledge(repo="mlx", tier="dependency", backend="mlx", limit=100) assert {record["id"] for record in mlx_records} == mlx_ids assert all(record["backend"] == "mlx" for record in mlx_records) -mlx_document = "data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" +mlx_document = "data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" mlx_reader = KnowledgeReader() for record in mlx_records: bundle = mlx_reader.read("package:" + record["id"]) @@ -76,32 +82,35 @@ def resource(logical): "package:" + record["id"], mlx_document, revision=bundle.revision ) == resource(mlx_document).read_bytes() skill_index = search_knowledge(repo="inferrsi", limit=100) -assert len(skill_index) == 80, "Missing installed skill abstracts" +assert len(skill_index) == 88, "Missing installed skill abstracts" assert all(record["index_type"] == "packaged-skill" for record in skill_index) ascend_rc = KnowledgeReader().read("inferrsi.ascend.skills.vllm_ascend") for note in ("plugin-registration", "minimax-cache", "dspark-pd", "sources", "deepseek-v4-evolution", "communication-config", "model-performance", "fused-operators", "history-scope"): - key = f"data/skill_library/repositories/vllm-ascend/references/{note}.md" + key = f"data/skill_library/systems/engines/vllm-ascend/references/{note}.md" assert ascend_rc.content[key] == resource(key).read_bytes() for relative in ("references/history-reviewed.json", "scripts/screen_history.py"): - key = f"data/skill_library/repositories/vllm-ascend/{relative}" + key = f"data/skill_library/systems/engines/vllm-ascend/{relative}" assert ascend_rc.content[key] == resource(key).read_bytes() assert all("history-inventory.json" not in key for key in ascend_rc.content) assert not resource( - "data/skill_library/repositories/vllm-ascend/references/history-inventory.json" + "data/skill_library/systems/engines/vllm-ascend/references/history-inventory.json" ).is_file() indexed = KnowledgeReader().read("inferrsi.ascend.skills.ascend_operator_profiling") -assert "data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md" in indexed.content +assert ( + "data/skill_library/systems/platforms/ascend/ascend-operator-profiling/SKILL.md" + in indexed.content +) video_diffusion = KnowledgeReader().read( "inferrsi.ascend.skills.ascend_video_diffusion_optimization" ) assert ( - "data/skill_library/hardware/ascend/ascend-video-diffusion-optimization/SKILL.md" + "data/skill_library/systems/platforms/ascend/ascend-video-diffusion-optimization/SKILL.md" in video_diffusion.content ) case_path = ( - "data/skill_library/methods/inference-optimization-casebook/" + "data/skill_library/casebooks/inference/inference-optimization-casebook/" "cases/admission-drain-occupancy.md" ) assert case_path in indexed.content @@ -109,12 +118,12 @@ def resource(logical): assert json.loads(indexed.content["record.json"])["skill_metadata"]["license"] == "UNKNOWN" h3_identity = "inferrsi.common.skills.audit_h3_runtime_residency" h3_bundle = KnowledgeReader().read(h3_identity) -h3_prefix = "data/skill_library/repositories/h3-c/audit-h3-runtime-residency/" +h3_prefix = "data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/" h3_inventory = json.loads(h3_bundle.content[ h3_prefix + "references/minimax-h3-weight-inventory.json" ]) assert h3_inventory["selections"][0]["totals"]["logical_tensor_elements"] == 69_235_580_593 -h3_metadata = json.loads(h3_bundle.content[h3_prefix + "metadata.json"]) +h3_metadata = owner_fields(h3_bundle.content[h3_prefix + "SKILL.md"]) assert h3_metadata["status"] == "candidate" assert h3_metadata["empirical_validation"] == "not_run" assert h3_metadata["sources"][0]["commit"] == "8974cc055ea9c02fcd14cc27dfda3e1027c05153" @@ -128,14 +137,14 @@ def resource(logical): h3_metal_identity = "inferrsi.common.skills.audit_h3_metal_fasth3_semantics" h3_metal_bundle = KnowledgeReader().read(h3_metal_identity) h3_metal_prefix = ( - "data/skill_library/repositories/h3-metal/audit-h3-metal-fasth3-semantics/" + "data/skill_library/systems/engines/h3-metal/audit-h3-metal-fasth3-semantics/" ) -h3_metal_metadata = json.loads(h3_metal_bundle.content[h3_metal_prefix + "metadata.json"]) +h3_metal_metadata = owner_fields(h3_metal_bundle.content[h3_metal_prefix + "SKILL.md"]) assert h3_metal_metadata["status"] == "candidate" assert h3_metal_metadata["empirical_validation"] == "not_run" assert h3_metal_metadata["sources"][0]["commit"] == "4ba47a3eabdbebf801ac6664b31f8ec168bc57a8" for relative in ( - "SKILL.md", "metadata.json", "LICENSE", "references/source-identity.json", + "SKILL.md", "LICENSE", "references/source-identity.json", "references/source-notes.md", "references/semantic-contracts.md", "references/preflight.md", "references/troubleshooting.md", "references/repo-provenance.md", "references/THIRD_PARTY_NOTICES.md", "scripts/audit_fasth3.py", @@ -149,14 +158,14 @@ def resource(logical): assert h3_metal_audit["native_execution"] == "not_run" afd_identity = "inferrsi.cuda.skills.audit_afd_p2p_exchanges" afd_bundle = KnowledgeReader().read(afd_identity) -afd_prefix = "data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/" -for relative in ("SKILL.md", "metadata.json", "references/source-notes.md", +afd_prefix = "data/skill_library/systems/engines/afd-plugin/audit-afd-p2p-exchanges/" +for relative in ("SKILL.md", "references/source-notes.md", "references/afd-plugin-LICENSE.md"): assert afd_prefix + relative in afd_bundle.content hopper_identity = "inferrsi.cuda.skills.reason_hopper_kernels" hopper_bundle = KnowledgeReader().read(hopper_identity) -hopper_prefix = "data/skill_library/hardware/cuda/reason-hopper-kernels/" -hopper_metadata = json.loads(hopper_bundle.content[hopper_prefix + "metadata.json"]) +hopper_prefix = "data/skill_library/systems/platforms/cuda/reason-hopper-kernels/" +hopper_metadata = owner_fields(hopper_bundle.content[hopper_prefix + "SKILL.md"]) assert hopper_metadata["status"] == "candidate" assert hopper_metadata["empirical_validation"] == "not_run" hopper_inventory = json.loads(hopper_bundle.content[ @@ -177,13 +186,13 @@ def resource(logical): # Check installed RLT resources without importing the upstream backend. rlt_identity = "inferrsi.common.skills.audit_recurrent_kv_finalization" rlt_bundle = KnowledgeReader().read(rlt_identity) -rlt_prefix = "data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/" +rlt_prefix = "data/skill_library/systems/engines/vllm-rlt/audit-recurrent-kv-finalization/" for relative in ( - "SKILL.md", "metadata.json", "LICENSE", "references/NOTICE.md", + "SKILL.md", "LICENSE", "references/NOTICE.md", "references/source-notes.md", "references/source-identity.json", ): assert rlt_prefix + relative in rlt_bundle.content -rlt_metadata = json.loads(rlt_bundle.content[rlt_prefix + "metadata.json"]) +rlt_metadata = owner_fields(rlt_bundle.content[rlt_prefix + "SKILL.md"]) assert rlt_metadata["status"] == "candidate" assert rlt_metadata["empirical_validation"] == "not_run" assert rlt_metadata["sources"][0]["commit"] == "4a350344027960a48384626aa646f1765fda6d4f" @@ -195,7 +204,7 @@ def resource(logical): assert "yaml" not in sys.modules, "Installed metadata reads must not require the skills extra" lithos = KnowledgeReader().read("inferrsi.common.skills.lithos_metal") -lithos_owner = "data/skill_library/repositories/lithos-metal/" +lithos_owner = "data/skill_library/systems/engines/lithos-metal/" for relative in ( "LICENSE", "references/upstream-notice.md", "references/mlx-license.md", "references/source-notes.md", "scripts/inspect_contract.py", @@ -223,11 +232,19 @@ def resource(logical): selection_context = selection["contexts"][0] selection_core = selection_context["resources"][0] assert selection_core["path"].endswith("/cgroup-resource-controls/SKILL.md") -assert selection_core["content"].encode() == selection_reader.resource( +selection_source = selection_reader.resource( selection_context["ref"], selection_core["path"], revision=selection_context["revision"], ) +assert selection_source == resource(selection_core["path"]).read_bytes() +assert selection_core["locator"] == "skill-procedure" +assert selection_core["source_sha256"] == hashlib.sha256(selection_source).hexdigest() +delivered = b"".join( + selection_source[start:end] for start, end in selection_core["source_byte_ranges"] +) +assert delivered == selection_core["content"].encode() +assert selection_core["sha256"] == hashlib.sha256(delivered).hexdigest() assert selection["context_bytes"] <= 65536 -assert resource("data/knowledge_packs/README.md").is_file() +assert resource("data/skill_library/_compiled/inputs.json").is_file() for tier, count in (("dependency", 6), ("peer", 5)): imported = search_knowledge(tier=tier, limit=100) assert len(imported) >= count, f"Missing installed {tier} records" @@ -252,20 +269,33 @@ def resource(logical): digest = hashlib.sha256(data.replace(b"\r\n", b"\n")).hexdigest() assert digest == expected_hash, relative registry = json.loads(library.joinpath("skill-index.json").read_text(encoding="utf-8"))["skills"] -assert len(registry) == 80 +assert len(registry) == 88 for entry in registry.values(): assert library.joinpath(*entry["path"].split("/"), "SKILL.md").is_file() +# The default wheel opens every primary procedure before an agent searches for evidence. +for task in ("day-0", "performance", "debug", "accuracy", "version-alignment", + "derived-engine", "independent-engine"): + name = f"task-{task}" + with contextlib.redirect_stdout(io.StringIO()) as stdout: + assert main(["skills", "enter", name]) == 0 + entered = json.loads(stdout.getvalue()) + assert entered["name"] == name + owner = library.joinpath(*registry[name]["path"].split("/"), "SKILL.md") + raw = owner.read_text(encoding="utf-8") + native, fields = _split_skill_metadata(raw.split("---", 2)[1]) + assert fields["discovery_role"] == "task-entry" + assert entered["instructions"] == "---" + native + "---" + raw.split("---", 2)[2] + for framework, backend in (("pytorch", "common"), ("torch-npu", "ascend")): identity = f"inferrsi.{backend}.skills.{framework.replace('-', '_')}" framework_reader = KnowledgeReader() graph = framework_reader.read(identity) - prefix = f"data/skill_library/frameworks/{framework}/{framework}/" + prefix = f"data/skill_library/systems/frameworks/{framework}/{framework}/" documents = [key for key in graph.content if key.startswith(prefix) and key.endswith("/SKILL.md")] assert len(documents) == 9, f"Missing installed {framework} graph documents" for document in documents: - metadata_path = document.removesuffix("SKILL.md") + "metadata.json" - metadata = json.loads(graph.content[metadata_path]) + metadata = owner_fields(graph.content[document]) assert metadata["profile"] == "local-skill-v1" assert metadata["status"] == "candidate" assert metadata["disable-model-invocation"] is True @@ -276,7 +306,7 @@ def resource(logical): ) == graph.content[child_document] assert any(key.startswith(prefix + "scripts/") and key.endswith(".py") for key in graph.content), f"Missing installed {framework} helpers" -router = library.joinpath("workflows", "inference-engine-research") +router = library.joinpath("workflows", "research", "inference-engine-research") manifest = json.loads( router.joinpath("references", "source-manifest.json").read_text(encoding="utf-8") ) @@ -295,7 +325,7 @@ def resource(logical): if "skill_documents" in item ] assert len(imported) == 3, f"Missing indexed knowledge for {repo}" - owner = library.joinpath("repositories", slug) + owner = library.joinpath("systems", "engines", slug) assert owner.joinpath("SKILL.md").is_file() assert owner.joinpath("LICENSE").is_file() assert not owner.joinpath("translations", "zh-CN").is_dir() @@ -311,13 +341,13 @@ def resource(logical): assert router.joinpath("SKILL.md").is_file() assert router.joinpath("references", "overview.md").is_file() assert not router.joinpath("translations", "zh-CN").is_dir() -strata = library.joinpath("repositories", "strata") +strata = library.joinpath("systems", "engines", "strata") assert strata.joinpath("references", "design-space-analysis.md").is_file() helper = strata.joinpath("scripts", "summarize_trials.py") assert helper.is_file() with as_file(helper) as path: compile(path.read_text(encoding="utf-8"), str(path), "exec") -ceiling = library.joinpath("methods", "performance-ceiling-analysis") +ceiling = library.joinpath("playbooks", "measurement", "performance-ceiling-analysis") ceiling_namespace = {} exec(compile(ceiling.joinpath("scripts", "roofline.py").read_text(encoding="utf-8"), "roofline.py", "exec"), ceiling_namespace) @@ -333,7 +363,7 @@ def resource(logical): ) assert ascend_image[0]["id"] == "vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i" assert resource( - "data/repo_packs/vllm_omni/knowledge/ascend/diffusion/qwen-image-21-eager-t2i.json" + "data/skill_library/records/vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i.json" ).is_file() output = io.StringIO() with contextlib.redirect_stdout(output): @@ -389,29 +419,29 @@ def resource(logical): assert not inference_modules & sys.modules.keys() print(f"Installed package: {inferrsi.__file__}; {len(records)} source-grounded records available.") -splash_notices = resource("data/skill_library/repositories/splash/THIRD_PARTY_NOTICES") +splash_notices = resource("data/skill_library/systems/engines/splash/THIRD_PARTY_NOTICES") assert splash_notices.is_file(), "Splash third-party notices are missing from the wheel" assert "The ggml authors" in splash_notices.read_text(encoding="utf-8") # Document-only candidate remains available without implying native verification. -husky = library.joinpath("repositories", "husky") +husky = library.joinpath("systems", "engines", "husky") assert husky.joinpath("references", "source-availability.json").is_file() assert husky.joinpath("scripts", "audit_evidence_card.py").is_file() -assert json.loads(husky.joinpath("metadata.json").read_text())["status"] == "candidate" +assert owner_fields(husky.joinpath("SKILL.md").read_bytes())["status"] == "candidate" # Maya retains safe support and unverified-native candidate state in the wheel. -maya = library.joinpath("repositories", "project-maya") +maya = library.joinpath("systems", "engines", "project-maya") assert maya.joinpath("LICENSE").is_file() assert maya.joinpath("sub-skills", "model-runtime", "scripts", "audit_runtime_plan.py").is_file() -assert json.loads(maya.joinpath("metadata.json").read_text())["status"] == "candidate" +assert owner_fields(maya.joinpath("SKILL.md").read_bytes())["status"] == "candidate" # vLLM Metal remains a source/CPU-only candidate with complete bundled helpers. -vllm_metal_skill = library.joinpath("repositories", "vllm-metal") +vllm_metal_skill = library.joinpath("systems", "engines", "vllm-metal") identity = json.loads(vllm_metal_skill.joinpath("references", "source-identity.json").read_text()) assert hashlib.sha256(vllm_metal_skill.joinpath("LICENSE").read_bytes()).hexdigest() == ( identity["files"]["LICENSE"] ) -assert json.loads(vllm_metal_skill.joinpath("metadata.json").read_text())["status"] == "candidate" +assert owner_fields(vllm_metal_skill.joinpath("SKILL.md").read_bytes())["status"] == "candidate" for area, script in ( ("platform-config", "check_environment_plan.py"), ("cache-lifecycle", "screen_pool_budget.py"), @@ -423,8 +453,8 @@ def resource(logical): assert not inference_modules & sys.modules.keys() # mistral.rs packages source evidence/support without an upstream Rust/Metal import. -mistral = library.joinpath("repositories", "mistral-rs") -mistral_meta = json.loads(mistral.joinpath("metadata.json").read_text()) +mistral = library.joinpath("systems", "engines", "mistral-rs") +mistral_meta = owner_fields(mistral.joinpath("SKILL.md").read_bytes()) assert mistral_meta["status"] == "candidate" assert mistral_meta["empirical_validation"] == "not_run" mistral_identity = json.loads(mistral.joinpath("references", "source-identity.json").read_text()) @@ -447,8 +477,8 @@ def resource(logical): assert not inference_modules & sys.modules.keys() # Swift source/support candidate is packaged without a Swift/MLX runtime import. -swift_lm = library.joinpath("repositories", "mlx-swift-lm") -swift_meta = json.loads(swift_lm.joinpath("metadata.json").read_text()) +swift_lm = library.joinpath("systems", "engines", "mlx-swift-lm") +swift_meta = owner_fields(swift_lm.joinpath("SKILL.md").read_bytes()) assert swift_meta["status"] == "candidate" assert swift_meta["empirical_validation"] == "not_run" swift_identity = json.loads(swift_lm.joinpath("references", "source-identity.json").read_text()) @@ -466,8 +496,8 @@ def resource(logical): assert not inference_modules & sys.modules.keys() # MLC source contracts remain readable without native MLC/TVM imports. -mlc = library.joinpath("repositories", "mlc-llm") -mlc_meta = json.loads(mlc.joinpath("metadata.json").read_text()) +mlc = library.joinpath("systems", "engines", "mlc-llm") +mlc_meta = owner_fields(mlc.joinpath("SKILL.md").read_bytes()) assert mlc_meta["status"] == "candidate" and mlc_meta["empirical_validation"] == "not_run" mlc_identity = json.loads(mlc.joinpath("references", "source-identity.json").read_text()) mlc_sources = {source["repository"]: source for source in mlc_identity["sources"]} @@ -492,8 +522,8 @@ def resource(logical): # Local Dream is packaged source knowledge, not a native Android runtime. dream_bundle = KnowledgeReader().read("inferrsi.common.skills.local_dream") -dream_prefix = "data/skill_library/repositories/local-dream/" -dream_metadata = json.loads(dream_bundle.content[dream_prefix + "metadata.json"]) +dream_prefix = "data/skill_library/systems/engines/local-dream/" +dream_metadata = owner_fields(dream_bundle.content[dream_prefix + "SKILL.md"]) assert dream_metadata["status"] == "candidate" assert dream_metadata["empirical_validation"] == "not_run" assert dream_metadata["license"] == "NOASSERTION" @@ -504,10 +534,14 @@ def resource(logical): assert hashlib.sha256(dream_bundle.content[ dream_prefix + "references/upstream-license.md" ]).hexdigest() == dream_identity["sources"][0]["files"]["LICENSE"] -assert len([name for name in dream_bundle.content if name.startswith(dream_prefix)]) == 9 -for relative in ("SKILL.md", "references/android-lifecycle.md", "references/native-contracts.md", - "references/packaging-and-abi.md", "references/repo-provenance.md"): - assert dream_prefix + relative in dream_bundle.content +dream_files = {name.removeprefix(dream_prefix) for name in dream_bundle.content + if name.startswith(dream_prefix)} +assert dream_files == { + "SKILL.md", "references/android-lifecycle.md", "references/native-contracts.md", + "references/packaging-and-abi.md", "references/repo-provenance.md", + "references/source-identity.json", "references/upstream-license.md", + "scripts/screen_declarations.py", +} dream_namespace = {"__name__": "installed_local_dream_declarations"} exec(compile(dream_bundle.content[dream_prefix + "scripts/screen_declarations.py"], "screen_declarations.py", "exec"), dream_namespace) @@ -541,7 +575,10 @@ def _inventory(root, prefix): **_inventory(files("inferrsi").joinpath("knowledge", "data"), "knowledge/data"), **_inventory(files("inferrsi").joinpath("harness", "data"), "harness/data"), } -assert len(inventory) == 930, f"installed resource count is {len(inventory)}, expected 930" +assert len(inventory) == 761, f"installed resource count is {len(inventory)}, expected 761" +for name in inventory: + if "/skill_library/" in name and name.endswith("/SKILL.md"): + assert name.removesuffix("SKILL.md") + "metadata.json" not in inventory, name assert sum(name.startswith("harness/data/") for name in inventory) == len(REPO_TYPES) # Every pinned record path and every registered skill body resolves inside that # same enumeration, so the inventory and the records describe one resource set. diff --git a/tests/fixtures/find/layout-queries.json b/tests/fixtures/find/layout-queries.json new file mode 100644 index 000000000..c1b79fd9d --- /dev/null +++ b/tests/fixtures/find/layout-queries.json @@ -0,0 +1,22 @@ +{ + "base_sha": "a3e33851981f1ecdc8b73d7b62633cc598e5d5ae", + "gate": {"match_all_ordered_top5": true, "max_lost_readable_hits": 0, "max_candidate_to_base_median_cli_latency_ratio": 1.0}, + "pairs_per_task": 4, + "tasks": { + "day-0": "Day zero port of an omni model with text and audio stages to vLLM Ascend NPU", + "performance": "Single GPU diffusion denoising latency baseline warm requests profiler", + "debug": "ROCm GPU hang during CUDA graph capture first failing operator minimal reproduction", + "accuracy": "TTS 新版本音质下降 对齐参考实现 逐层精度回归", + "version-alignment": "Align vllm-omni vllm-ascend and afd-plugin versions after upgrading vLLM", + "derived-engine": "Bring a vLLM forked inference engine up on DeepInfer and reuse scheduling components", + "independent-engine": "Build an independent inference engine for an on-device model on Windows XPU" + }, + "previous_fixed_queries": [ + "MiniMax H3 Image 4K BF16 single B300 six steps hot image latency", + "MiniMax H3 B300 diffusion VAE tile decode batching overlap memory", + "MiniMax H3 B300 packed queries keys RMS normalization rotary embeddings fusion", + "diffusion T2I redundant conditioning across denoising steps BF16 prompt reuse", + "MiniMax H3 denoising time modulation projections cache timestep schedule", + "inference benchmark design warm requests baseline latency quality control" + ] +} diff --git a/tests/test_apps_context_provider.py b/tests/test_apps_context_provider.py index 81f103b81..32be3109e 100644 --- a/tests/test_apps_context_provider.py +++ b/tests/test_apps_context_provider.py @@ -21,7 +21,9 @@ def _request(**overrides): def test_delivery_uses_exact_fixed_resource_bytes_in_knowledge_order(): reader = knowledge_reader() - revisions = {ref: reader.read(ref).revision for ref in (_H3, _CGROUP)} + # local-dream includes a Python helper that must remain a reported omission. + refs = (_H3, _CGROUP, "package:inferrsi.common.skills.local_dream") + revisions = {ref: reader.read(ref).revision for ref in refs} pins = [{"name": ref, "revision": revision} for ref, revision in revisions.items()] request = _request(ids=list(revisions), pins=pins, max_bytes=262144) provider = ContextProvider(reader) @@ -35,10 +37,15 @@ def test_delivery_uses_exact_fixed_resource_bytes_in_knowledge_order(): expected = [] for context, item in zip(contexts, draft.items, strict=True): resources = context["resources"] - body = b"".join( - reader.resource(context["ref"], r["path"], revision=context["revision"]) - for r in resources - ) + pieces = [] + for resource in resources: + raw = reader.resource(context["ref"], resource["path"], revision=context["revision"]) + if resource["locator"] == "skill-procedure": + assert sha256(raw).hexdigest() == resource["source_sha256"] + raw = b"".join(raw[start:end] for start, end in resource["source_byte_ranges"]) + assert sha256(raw).hexdigest() == resource["sha256"] + pieces.append(raw) + body = b"".join(pieces) expected.append(body) assert item["delivered_roles"] == list(dict.fromkeys(r["role"] for r in resources)) assert item["delivered_bytes"] == len(body) diff --git a/tests/test_apps_knowledge.py b/tests/test_apps_knowledge.py index 5aceaf69c..039c15c01 100644 --- a/tests/test_apps_knowledge.py +++ b/tests/test_apps_knowledge.py @@ -25,7 +25,7 @@ def test_knowledge_read_exposes_fixed_qualified_bundle_and_resource(): read(reader, reference, revision="sha256:" + "0" * 64) -def test_checkout_read_scans_owners_once_with_a_read_only_store(tmp_path, monkeypatch): +def test_checkout_read_resolves_once_without_enumerating_all_owners(tmp_path, monkeypatch): skills = tmp_path / "checkout" / "skills" skill = skills / "checkout-example" skill.mkdir(parents=True) @@ -38,19 +38,23 @@ def test_checkout_read_scans_owners_once_with_a_read_only_store(tmp_path, monkey reader = knowledge_reader(skill_root=skills, store_root=store_root) assert reader.store.read_only assert reader.store.database == store.database - owners = reader.owners - scans = [] + resolve = reader.resolve + resolutions = [] - def count_owners(): - scans.append(None) - return owners() + def track_resolution(reference): + resolutions.append(reference) + return resolve(reference) - monkeypatch.setattr(reader, "owners", count_owners) + def reject_global_scan(): + pytest.fail("An exact read loaded the global owner catalog") + + monkeypatch.setattr(reader, "resolve", track_resolution) + monkeypatch.setattr(reader, "owners", reject_global_scan) reference, revision, view = read(reader, "checkout-example", material=True) assert reference == "checkout:checkout-example" assert revision == view["revision"] assert view["content"].endswith("Example\n") - assert len(scans) == 1 + assert resolutions == ["checkout-example"] def test_apps_store_requires_explicit_write_and_retains_fixed_history(tmp_path): diff --git a/tests/test_ascend_video_diffusion_skill.py b/tests/test_ascend_video_diffusion_skill.py index 62e9e11c9..94bf8199b 100644 --- a/tests/test_ascend_video_diffusion_skill.py +++ b/tests/test_ascend_video_diffusion_skill.py @@ -10,9 +10,9 @@ ROOT = Path(__file__).resolve().parents[1] SKILL_ROOT = ROOT / "src/inferrsi/knowledge/data/skill_library" SKILL_NAME = "ascend-video-diffusion-optimization" -SKILL = SKILL_ROOT / "hardware/ascend" / SKILL_NAME +SKILL = SKILL_ROOT / "systems/platforms/ascend" / SKILL_NAME COMMON_NAME = "video-diffusion-optimization" -COMMON = SKILL_ROOT / "methods" / COMMON_NAME +COMMON = SKILL_ROOT / "playbooks/optimization" / COMMON_NAME TEMPLATE = COMMON / "assets/campaign.template.json" EVALUATOR = COMMON / "scripts/evaluate_target.py" @@ -26,7 +26,6 @@ def test_skills_are_discoverable_and_resources_resolve(): assert record["status"] == "candidate" assert {"ascend", "video-diffusion", "performance"} <= set(record["tags"]) assert set(record["resources"]) == { - "metadata.json", "references/field-notes-video-diffusion.md", } @@ -39,7 +38,6 @@ def test_skills_are_discoverable_and_resources_resolve(): assert common["status"] == "candidate" assert set(common["resources"]) == { "assets/campaign.template.json", - "metadata.json", "scripts/evaluate_target.py", } assert common["description"] @@ -50,19 +48,25 @@ def test_skills_are_discoverable_and_resources_resolve(): assert (SKILL / "references/field-notes-video-diffusion.md").is_file() # The Ascend skill delegates to the backend-agnostic common skeleton. for relative in ( - "methods/video-diffusion-optimization/SKILL.md", - "methods/inference-benchmark-design/SKILL.md", - "workflows/rsi-evidence/SKILL.md", + "playbooks/optimization/video-diffusion-optimization/SKILL.md", + "playbooks/measurement/inference-benchmark-design/SKILL.md", + "workflows/governance/rsi-evidence/SKILL.md", ): assert relative in instructions assert (SKILL_ROOT / relative).is_file() common_instructions = skill_instructions(SKILL_ROOT, COMMON_NAME) # The common skeleton links its sibling method and workflow with relative - # paths from methods/video-diffusion-optimization/. + # paths from playbooks/optimization/video-diffusion-optimization/. for relative, target in ( - ("../inference-benchmark-design/SKILL.md", "methods/inference-benchmark-design/SKILL.md"), - ("../../workflows/rsi-evidence/SKILL.md", "workflows/rsi-evidence/SKILL.md"), + ( + "../../measurement/inference-benchmark-design/SKILL.md", + "playbooks/measurement/inference-benchmark-design/SKILL.md", + ), + ( + "../../../workflows/governance/rsi-evidence/SKILL.md", + "workflows/governance/rsi-evidence/SKILL.md", + ), ): assert relative in common_instructions assert (SKILL_ROOT / target).is_file() diff --git a/tests/test_campaign_task_selection.py b/tests/test_campaign_task_selection.py index fa87fc3c6..cc92a2b3e 100644 --- a/tests/test_campaign_task_selection.py +++ b/tests/test_campaign_task_selection.py @@ -1,3 +1,4 @@ +import hashlib import json import shutil @@ -175,3 +176,39 @@ def test_find_does_not_infer_case_backend_from_applicability(tmp_path): assert counts["considered"] == 1 assert counts["excluded"]["ranking_scope"] == 1 assert counts["excluded"]["filters"] == 0 + + +def test_task_skills_are_explicitly_readable_but_never_ranked_into_campaign(tmp_path): + create_campaign(tmp_path, "routing") + reader = KnowledgeReader() + tasks = ("inference_task_router", "task_day_0", "task_performance", "task_debug", + "task_accuracy", "task_version_alignment", "task_derived_engine", + "task_independent_engine") + task_refs = {f"package:inferrsi.common.skills.{task}" for task in tasks} + cue = "inference-task-router day-0 model onboarding performance accuracy" + options = {"context_bytes": 65536, "limit": 135, "repo": "inferrsi", "kind": "knowledge"} + path, _ = capture_context(tmp_path, context_id="ranked", reader=reader, + relevance=cue, **options) + ranked, state = verify_capture(path.read_bytes()) + assert state == "verified" and ranked["items"] + assert ranked["retrieval"]["counts"]["considered"] == 128 + for group in ("items", "contexts", "context_omissions"): + assert task_refs.isdisjoint(item.get("ref") for item in ranked[group]) + refs = sorted(task_refs) + path, _ = capture_context(tmp_path, context_id="explicit", reader=reader, ids=refs, + **{**options, "context_bytes": 1024 * 1024}) + exact, state = verify_capture(path.read_bytes()) + assert state == "verified" + assert {item["ref"] for item in exact["items"]} == task_refs + assert {item["ref"] for item in exact["contexts"]} == task_refs + for context in exact["contexts"]: + core = next(resource for resource in context["resources"] if resource["role"] == "core") + assert core["locator"] == "skill-procedure" + source = reader.resource(context["ref"], core["path"], revision=context["revision"]) + delivered = core["content"].encode("utf-8") + assert hashlib.sha256(source).hexdigest() == core["source_sha256"] + assert hashlib.sha256(delivered).hexdigest() == core["sha256"] + assert b"".join(source[a:b] for a, b in core["source_byte_ranges"]) == delivered + for item in exact["items"]: + pinned = reader.read(item["ref"], revision=item["revision"]) + assert pinned.metadata["id"] == item["ref"].split(":")[1] diff --git a/tests/test_cli_catalog.py b/tests/test_cli_catalog.py index 962301bcb..36eb59672 100644 --- a/tests/test_cli_catalog.py +++ b/tests/test_cli_catalog.py @@ -3,8 +3,8 @@ import contextlib import io import json -import re import shlex +import shutil import subprocess import sys import tempfile @@ -141,19 +141,48 @@ def test_ascend_image_context_preserves_provenance_and_scope(self): self.assertNotIn("Backend: `cuda`", text) +def test_default_find_skips_incidental_system_files(tmp_path, monkeypatch, capsys): + from tools.plugin_bundle import _io_path + + library = Path("knowledge/data/skill_library") + package = _io_path(tmp_path) + installed = package / library + shutil.copytree(PROJECT / "src/inferrsi" / library, installed) + monkeypatch.setattr(knowledge_catalog, "files", lambda _: package) + identity = "flashinfer.cuda.llm.paged_decode_dispatch" + query = ["find", identity, "--no-external", "--limit", "1", "--json"] + assert main(query) == 0 + previous = json.loads(capsys.readouterr().out)["items"] + assert previous[0]["id"] == identity and previous[0]["open_argv"] + for name in (".DS_Store", "Thumbs.db", "Desktop.ini"): + (installed / "records" / name).write_bytes(b"incidental system data") + assert main(query) == 0 + assert json.loads(capsys.readouterr().out)["items"] == previous + + +@pytest.mark.parametrize(("filename", "error"), [ + ("unexpected.txt", "Unexpected authored knowledge resource"), + (".hidden.json", "Invalid knowledge filename"), + ("flashinfer.cuda.llm.invalid.json", "Expecting value"), +]) +def test_records_still_reject_unrecognized_or_invalid_data(tmp_path, monkeypatch, filename, error): + records = tmp_path / "knowledge/data/skill_library/records" + records.mkdir(parents=True) + (records / filename).write_text("unrecognized data", encoding="utf-8") + monkeypatch.setattr(knowledge_catalog, "files", lambda _: tmp_path) + with pytest.raises(ValueError, match=error): + knowledge_records(include_skill_views=False) + + def test_knowledge_has_one_canonical_record_and_complete_navigation(): - packs = PROJECT / "src/inferrsi/knowledge/data/repo_packs" - canonical_paths = list(packs.glob("*/knowledge/*/*/*.json")) - records = search_knowledge(tier="contribution", limit=len(canonical_paths)) - assert len(records) == len(canonical_paths) - for repo in {record["repo"] for record in records}: - root = packs / repo / "knowledge" - paths = list(root.glob("*/*/*.json")) - assert len(paths) == len([record for record in records if record["repo"] == repo]) - assert list(root.rglob("*.md")) == [root / "README.md"] - index = (root / "README.md").read_text(encoding="utf-8") - targets = set(re.findall(r"\]\(([^)]+\.json)\)", index)) - assert targets == {path.relative_to(root).as_posix() for path in paths} + root = PROJECT / "src/inferrsi/knowledge/data/skill_library/records" + records = search_knowledge(tier="contribution", limit=200) + authored = [r for r in records if r.get("index_type") != "packaged-skill"] + assert {r["id"] for r in authored} == { + path.stem for path in root.glob("*.json") + if json.loads(path.read_bytes()).get("tier", "contribution") == "contribution" + } + assert sum(r.get("index_type") == "packaged-skill" for r in records) == 88 for record in records: assert "/pull/" not in json.dumps(record) assert record["verification"]["evidence_level"] in ("guidance", "upstream_reported") @@ -183,8 +212,8 @@ def test_misfiled_or_overstated_record_cannot_leak_into_context( record[field] = value destination = ( tmp_path - / "knowledge/data/repo_packs/vllm_omni/knowledge/ascend/diffusion" - / "qwen-image-21-eager-t2i.json" + / "knowledge/data/skill_library/records" + / "vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i.json" ) destination.parent.mkdir(parents=True) destination.write_text(json.dumps(record)) diff --git a/tests/test_codex_plugin.py b/tests/test_codex_plugin.py index f04d838e5..416d33e19 100644 --- a/tests/test_codex_plugin.py +++ b/tests/test_codex_plugin.py @@ -16,6 +16,7 @@ import pytest +from inferrsi.knowledge.catalog import _split_skill_metadata from tools.build_codex_plugin import build_plugin from tools.build_workswarm_plugin import build_plugin as build_workswarm_plugin from tools.check_release import check_plugin @@ -83,7 +84,7 @@ def test_bundle_contains_all_declared_package_data(distribution): if path.is_file() } assert ( - "knowledge/data/skill_library/repositories/splash/THIRD_PARTY_NOTICES" in expected + "knowledge/data/skill_library/systems/engines/splash/THIRD_PARTY_NOTICES" in expected ) installed = { name.removeprefix("inferrsi/") @@ -136,6 +137,58 @@ def test_isolated_runtime_and_exact_knowledge_reads(distribution, tmp_path): assert "unavailable" in stale.stderr +def test_entry_loads_the_required_primary_before_search(distribution, tmp_path): + entry = (distribution / "skills/inferrsi/SKILL.md").read_text(encoding="utf-8") + # Inspect the actual distributed entry, including the instructions before the + # task table. Moving a plain Find or Collect example above it must fail. + first_search = re.search(r'(?:^ *|`)(?:(?:python -X utf8 ""|inferrsi) )?' + r'(?:find|collect|knowledge search|campaign knowledge)\b', + entry, flags=re.MULTILINE) + assert first_search + rows = dict(re.findall(r"^\| ([^|]+) \| `(skills enter [\w-]+)` \|$", + entry[:first_search.start()], flags=re.MULTILINE)) + required = {"task-day-0", "task-performance", "task-debug", "task-accuracy", + "task-version-alignment", "task-derived-engine", "task-independent-engine"} + assert len(rows) == len(required) + assert {command.split()[-1] for command in rows.values()} == required + before_search = " ".join(entry[:first_search.start()].split()) + assert "before the first evidence search" in before_search + assert "If it fails, stop" in before_search + bundled = runtime_bytes(distribution) + library = "inferrsi/knowledge/data/skill_library/" + registered = json.loads(bundled[library + "skill-index.json"])["skills"] + for name in required: + owner = library + registered[name]["path"] + header = bundled[owner + "/SKILL.md"].decode().split("---", 2)[1] + _, metadata = _split_skill_metadata(header) + assert "primary-task" in metadata["tags"] + assert owner + "/metadata.json" not in bundled + + # The host makes the natural-language choice. Follow its published accuracy + # row and check the actual base-install CLI returns the owner before Find. + command = rows["Repair an accuracy or output-quality regression"].split() + assert command == ["skills", "enter", "task-accuracy"] + primary = invoke(distribution, tmp_path, *command) + assert primary.returncode == 0, primary.stderr + owner = PROJECT / "src/inferrsi/knowledge/data/skill_library/tasks/task-accuracy/SKILL.md" + _, header, body = owner.read_text(encoding="utf-8").split("---", 2) + native, fields = _split_skill_metadata(header) + assert fields["discovery_role"] == "task-entry" + assert json.loads(primary.stdout) == { + "name": "task-accuracy", "instructions": "---" + native + "---" + body + } + hits = invoke(distribution, tmp_path, "find", "TTS quality regression", + "--limit", "1", "--no-external", "--json") + assert hits.returncode == 0, hits.stderr + assert json.loads(hits.stdout)["items"] + + # Substituting the old router does not satisfy the primary-load requirement. + wrong = invoke(distribution, tmp_path, "skills", "enter", "inference-task-router") + assert wrong.returncode == 2 + assert "Not a primary inference task skill" in wrong.stderr + assert not wrong.stdout.strip() + + def test_entry_workflow_selects_packaged_skill_projections(distribution, tmp_path): result = invoke( distribution, tmp_path, "collect", "--relevance", @@ -149,8 +202,8 @@ def test_entry_workflow_selects_packaged_skill_projections(distribution, tmp_pat @pytest.mark.parametrize(("action", "resource"), [ - ("reference", "../../../hardware/host/host-memory-management/SKILL.md"), - ("resource", "metadata.json"), + ("reference", "../../../platforms/host/host-memory-management/SKILL.md"), + ("resource", "references/notices.md"), ]) def test_returned_skill_paths_survive_process_exit_and_later_invocations( distribution, tmp_path, action, resource, diff --git a/tests/test_collect_selection.py b/tests/test_collect_selection.py index 3d4b18149..b03d82d70 100644 --- a/tests/test_collect_selection.py +++ b/tests/test_collect_selection.py @@ -53,10 +53,7 @@ def test_existing_item_payload_and_material_accounting_are_unchanged(): assert new["material_bytes"] == old["material_bytes"] == len(_json(old["items"][0])) assert json.loads(new["items"][0]["content"])["id"] == CGROUP assert new["contexts"][0]["resources"][0]["path"].endswith("/SKILL.md") - assert new["contexts"][0]["omissions"] == [{ - "path": "data/skill_library/hardware/host/cgroup-resource-controls/metadata.json", - "reason": "not_context_resource", - }] + assert new["contexts"][0]["omissions"] == [] assert "contexts" not in reader.collect(ids=[CGROUP], relevance="unrelated") @@ -122,14 +119,32 @@ def test_context_keeps_complete_core_prerequisites_provenance_and_exact_bytes(): assert (context["ref"], context["revision"]) == (item["ref"], item["revision"]) assert context["sources"] == item["sources"] paths = {r["path"] for r in context["resources"]} - assert "data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md" in paths - assert "data/skill_library/methods/inference-benchmark-design/SKILL.md" in paths + assert "data/skill_library/systems/engines/vllm/audit-vllm-kv-lifetimes/SKILL.md" in paths + assert "data/skill_library/playbooks/measurement/inference-benchmark-design/SKILL.md" in paths assert any(path.endswith("/references/source-notes.md") for path in paths) for resource in context["resources"]: actual = reader.resource(context["ref"], resource["path"], revision=context["revision"]) - assert resource["content"].encode() == actual - assert resource["sha256"] == hashlib.sha256(actual).hexdigest() - assert resource["locator"] == "whole-resource" + delivered = resource["content"].encode() + assert resource["sha256"] == hashlib.sha256(delivered).hexdigest() + if resource["path"].endswith("/SKILL.md") and b" inferrsi: |" in actual: + import yaml + + assert resource["locator"] == "skill-procedure" + assert resource["source_sha256"] == hashlib.sha256(actual).hexdigest() + first, second = resource["source_byte_ranges"] + assert first[0] == 0 < first[1] < second[0] < second[1] == len(actual) + assert actual[:first[1]] + actual[second[0]:] == delivered + original_header, original_body = actual.split(b"---", 2)[1:] + header, body = delivered.split(b"---", 2)[1:] + expected = yaml.safe_load(original_header) + del expected["metadata"]["inferrsi"] + if not expected["metadata"]: + del expected["metadata"] + assert yaml.safe_load(header) == expected + assert body == original_body + else: + assert resource["locator"] == "whole-resource" + assert delivered == actual assert context["content_bytes"] == result["context_bytes"] == sum( len(r["content"].encode()) for r in context["resources"] ) @@ -206,7 +221,7 @@ def test_h3_readsets_are_mandatory_even_without_lexical_overlap(): result = KnowledgeReader().collect(ids=[H3], relevance="orchid", context_bytes=65536) assert not result["context_omissions"] paths = {r["path"] for r in result["contexts"][0]["resources"]} - prefix = "data/skill_library/repositories/h3-c/audit-h3-runtime-residency/" + prefix = "data/skill_library/systems/engines/h3-c/audit-h3-runtime-residency/" for path in ("SKILL.md", "LICENSE", "references/source-notes.md", "references/source-identity.json", "references/minimax-h3-weight-inventory.json"): assert prefix + path in paths @@ -214,7 +229,7 @@ def test_h3_readsets_are_mandatory_even_without_lexical_overlap(): def test_parent_owner_license_is_mandatory_and_budgeted_for_cross_skill_references(): identity = "inferrsi.common.skills.audit_vllm_kv_lifetimes" - license_path = "data/skill_library/repositories/vllm/LICENSE" + license_path = "data/skill_library/systems/engines/vllm/LICENSE" reader = KnowledgeReader() for owner in (identity, AFD): result = reader.collect(ids=[owner], context_bytes=100000) @@ -226,19 +241,22 @@ def test_parent_owner_license_is_mandatory_and_budgeted_for_cross_skill_referenc assert license_resource["sha256"] == hashlib.sha256(expected).hexdigest() bundle = reader.read(identity) core = bundle.metadata["skill_documents"]["original"] - limited = reader.collect(ids=[identity], context_bytes=len(bundle.content[core])) + context, = reader.collect(ids=[identity], context_bytes=100000)["contexts"] + core_resource, = (entry for entry in context["resources"] if entry["path"] == core) + core_size = len(core_resource["content"].encode()) + limited = reader.collect(ids=[identity], context_bytes=core_size) assert not limited["contexts"] omission = limited["context_omissions"][0] assert omission["reason"] == "context_limit" assert omission["required_bytes"] >= ( - len(bundle.content[core]) + len(bundle.content[license_path]) + core_size + len(bundle.content[license_path]) ) @pytest.mark.parametrize(("identity", "cue", "path"), [ - (AFD, "P2pNccl", "data/skill_library/repositories/afd-plugin/" + (AFD, "P2pNccl", "data/skill_library/systems/engines/afd-plugin/" "audit-afd-p2p-exchanges/references/afd-plugin-LICENSE.md"), - (H3, "orchid", "data/skill_library/repositories/h3-c/" + (H3, "orchid", "data/skill_library/systems/engines/h3-c/" "audit-h3-runtime-residency/references/THIRD_PARTY_NOTICES.md"), ]) def test_retained_legal_resources_ignore_relevance_and_use_atomic_budget(identity, cue, path): diff --git a/tests/test_compare_find_layouts.py b/tests/test_compare_find_layouts.py new file mode 100644 index 000000000..010e03014 --- /dev/null +++ b/tests/test_compare_find_layouts.py @@ -0,0 +1,105 @@ +import json +import subprocess +import sys + +import pytest + +from tools import compare_find_layouts as replay + + +@pytest.mark.parametrize("dirty", ["index", "tracked", "untracked", "replacement"]) +def test_second_dirty_arm_stops_before_any_search(tmp_path, monkeypatch, dirty): + try: + replay.check_git_version() + except replay.GitVersionError as error: + pytest.skip(str(error)) + base = tmp_path / "base" + candidate = tmp_path / "candidate" + base.mkdir() + + def git(root, *args): + return subprocess.check_output(["git", *args], cwd=root, text=True).strip() + + git(base, "init", "-q") + (base / "source").write_text("original") + git(base, "add", "source") + git( + base, + "-c", + "user.name=Test", + "-c", + "user.email=test@example.org", + "-c", + "commit.gpgsign=false", + "commit", + "-qm", + "fixture", + ) + sha = git(base, "rev-parse", "HEAD") + git(base, "worktree", "add", "--quiet", "--detach", str(candidate), sha) + replay.validate_worktree(candidate, sha) + (candidate / ("scratch" if dirty == "untracked" else "source")).write_text("modified") + if dirty == "index": + git(candidate, "add", "source") + if dirty == "replacement": + git( + candidate, + "-c", + "user.name=Test", + "-c", + "user.email=test@example.org", + "-c", + "commit.gpgsign=false", + "commit", + "-qam", + "different tree", + ) + replacement = git(candidate, "rev-parse", "HEAD") + git(candidate, "reset", "--soft", sha) + git(candidate, "replace", sha, replacement) + assert not git(candidate, "status", "--porcelain") + + (tmp_path / "layout-queries.json").write_text(json.dumps({"base_sha": sha})) + monkeypatch.setattr(replay, "FIXTURES", tmp_path) + monkeypatch.setattr( + sys, + "argv", + [ + "compare", + "--base", + str(base), + "--candidate", + str(candidate), + "--candidate-sha", + sha, + "--output", + str(tmp_path / "out"), + ], + ) + with pytest.raises(ValueError, match="Dirty worktree"): + replay.main() + assert not (tmp_path / "out").exists() + + +def test_comparison_old_git_stops_before_validating_worktrees(tmp_path, monkeypatch, capsys): + monkeypatch.setattr(subprocess, "check_output", lambda *a, **kw: "git version 2.43.0\n") + monkeypatch.setattr( + sys, + "argv", + [ + "compare", + "--base", + "unused", + "--candidate", + "unused", + "--candidate-sha", + "unused", + "--output", + str(tmp_path / "out"), + ], + ) + with pytest.raises(SystemExit) as stopped: + replay.main() + assert stopped.value.code == 2 + assert "Find layout comparison requires Git >= 2.45" in capsys.readouterr().err + assert not (tmp_path / "out").exists() diff --git a/tests/test_flare_knowledge.py b/tests/test_flare_knowledge.py index f33c7d38f..7bd41456a 100644 --- a/tests/test_flare_knowledge.py +++ b/tests/test_flare_knowledge.py @@ -7,7 +7,7 @@ ROOT = Path(__file__).resolve().parents[1] / "src/inferrsi/knowledge/data/skill_library" NAME = "audit-flare-session-lifecycle" -RELATIVE = f"repositories/flare-local-inference/{NAME}" +RELATIVE = f"systems/engines/flare-local-inference/{NAME}" IDENTITY = "inferrsi.cuda.skills.audit_flare_session_lifecycle" PIN = "bce032c014bae34502ba22d3304fa084d5b40afe" @@ -44,6 +44,6 @@ def test_flare_package_and_checkout_reads_retain_the_complete_owner(): assert (owner / "LICENSE").read_bytes() in package.content.values() preparation = "references/flare-preparation-boundary.md" weight = reader.read("weight-loading-optimization") - assert weight.content[f"methods/weight-loading-optimization/{preparation}"] == ( + assert weight.content[f"playbooks/optimization/weight-loading-optimization/{preparation}"] == ( skill_resource(ROOT, "weight-loading-optimization", preparation).read_bytes() ) diff --git a/tests/test_h3_knowledge.py b/tests/test_h3_knowledge.py index c93c2e45b..f9a120fec 100644 --- a/tests/test_h3_knowledge.py +++ b/tests/test_h3_knowledge.py @@ -12,11 +12,11 @@ ROOT = Path(__file__).resolve().parents[1] / "src/inferrsi/knowledge/data/skill_library" NAME = "audit-h3-runtime-residency" -OWNER = ROOT / "repositories/h3-c" / NAME +OWNER = ROOT / "systems/engines/h3-c" / NAME IDENTITY = "inferrsi.common.skills.audit_h3_runtime_residency" COMMIT = "8974cc055ea9c02fcd14cc27dfda3e1027c05153" -PREFIX = f"data/skill_library/repositories/h3-c/{NAME}/" -CHECKOUT_PREFIX = f"repositories/h3-c/{NAME}/" +PREFIX = f"data/skill_library/systems/engines/h3-c/{NAME}/" +CHECKOUT_PREFIX = f"systems/engines/h3-c/{NAME}/" INVENTORY = "references/minimax-h3-weight-inventory.json" INVENTORY_SHA256 = "fbaacaeec340d3ce893556ee864405a0746af4d13530a38cb3d5aa9d60929271" diff --git a/tests/test_h3_metal_knowledge.py b/tests/test_h3_metal_knowledge.py index d3c9666af..65e5f1ac1 100644 --- a/tests/test_h3_metal_knowledge.py +++ b/tests/test_h3_metal_knowledge.py @@ -25,9 +25,9 @@ PROJECT = Path(__file__).resolve().parents[1] LIBRARY = PROJECT / "src/inferrsi/knowledge/data/skill_library" NAME = "audit-h3-metal-fasth3-semantics" -OWNER = LIBRARY / "repositories/h3-metal" / NAME +OWNER = LIBRARY / "systems/engines/h3-metal" / NAME IDENTITY = "inferrsi.common.skills.audit_h3_metal_fasth3_semantics" -PREFIX = f"data/skill_library/repositories/h3-metal/{NAME}/" +PREFIX = f"data/skill_library/systems/engines/h3-metal/{NAME}/" CASES = PROJECT / "tests/fixtures/repo_to_skill/h3-metal/test-cases" COMMIT = "4ba47a3eabdbebf801ac6664b31f8ec168bc57a8" SOURCE_MAP_SHA256 = "e53bca079cfc318ae82d35c9cb5e70c61bb2f84a67edcb0af5efffe8b2f407d2" @@ -79,7 +79,7 @@ def test_discovery_and_complete_checkout_package_resources(): for relative in set(get_skill(LIBRARY, NAME)["resources"]) | {"SKILL.md"}: expected = (OWNER / relative).read_bytes() assert packaged.content[PREFIX + relative] == expected - assert local.content[f"repositories/h3-metal/{NAME}/{relative}"] == expected + assert local.content[f"systems/engines/h3-metal/{NAME}/{relative}"] == expected assert reader.resource(IDENTITY, PREFIX + relative, revision=packaged.revision) == expected diff --git a/tests/test_hopper_knowledge.py b/tests/test_hopper_knowledge.py index 4baec9002..fb2cf18dc 100644 --- a/tests/test_hopper_knowledge.py +++ b/tests/test_hopper_knowledge.py @@ -16,10 +16,10 @@ ROOT = Path(__file__).resolve().parents[1] / "src/inferrsi/knowledge/data/skill_library" NAME = "reason-hopper-kernels" -OWNER = ROOT / "hardware/cuda" / NAME +OWNER = ROOT / "systems/platforms/cuda" / NAME IDENTITY = "inferrsi.cuda.skills.reason_hopper_kernels" -PREFIX = f"data/skill_library/hardware/cuda/{NAME}/" -CHECKOUT_PREFIX = f"hardware/cuda/{NAME}/" +PREFIX = f"data/skill_library/systems/platforms/cuda/{NAME}/" +CHECKOUT_PREFIX = f"systems/platforms/cuda/{NAME}/" def test_hopper_discovery_keeps_cuda_scope_and_unverified_candidate_state(): diff --git a/tests/test_husky_document_research.py b/tests/test_husky_document_research.py index 8592bd1bd..783144c62 100644 --- a/tests/test_husky_document_research.py +++ b/tests/test_husky_document_research.py @@ -12,8 +12,10 @@ import pytest import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -ROOT = PROJECT / 'src/inferrsi/knowledge/data/skill_library/repositories/husky' +ROOT = PROJECT / 'src/inferrsi/knowledge/data/skill_library/systems/engines/husky' FIXTURES = PROJECT / 'tests/fixtures/repo_to_skill/husky' @@ -39,7 +41,7 @@ def test_husky_document_links_and_unknowns_remain_explicit(): continue selected = (path.parent / unquote(link.path)).resolve() assert selected.is_relative_to(ROOT.resolve()) and selected.is_file() - metadata = json.loads((ROOT / 'metadata.json').read_text()) + metadata = skill_metadata(ROOT.parent, ROOT.name) assert metadata['status'] == 'candidate' and metadata['empirical_validation'] == 'not_run' assert metadata['construction']['native_import'] == 'not-run' availability = json.loads((ROOT / 'references/source-availability.json').read_text()) diff --git a/tests/test_knowledge_cli.py b/tests/test_knowledge_cli.py index 96823239d..8715e474d 100644 --- a/tests/test_knowledge_cli.py +++ b/tests/test_knowledge_cli.py @@ -1,6 +1,7 @@ """CLI contracts for the collect / read / resource knowledge commands.""" import contextlib +import hashlib import io import json import tempfile @@ -34,6 +35,19 @@ def run_cli_error(*argv): return stderr.getvalue() +def test_enter_rejects_a_primary_with_no_instructions(tmp_path): + owner = tmp_path / "task-accuracy" + owner.mkdir() + (owner / "metadata.json").write_text( + '{"discovery_role":"task-entry","tags":["primary-task"]}', encoding="utf-8" + ) + (owner / "SKILL.md").write_text( + "---\nname: task-accuracy\ndescription: Missing body\n---\n", encoding="utf-8" + ) + error = run_cli_error("skills", "--root", str(tmp_path), "enter", "task-accuracy") + assert "requires frontmatter and instructions" in error + + def local_candidate(source: Bundle) -> Bundle: return Bundle( { @@ -122,13 +136,46 @@ def test_read_returns_bundle_view_and_rejects_wrong_revision(): assert "unavailable" in stderr -def test_read_can_omit_inventory_without_changing_material_or_revision(): - args = ("read", "package:vllm_omni.common.diffusion.step_state_and_output_validation", - "--material") +@pytest.mark.parametrize("authority", ["checkout", "store"]) +def test_material_delivers_procedure_and_exact_resource_preserves_raw_source(authority, tmp_path): + identity = f"checkout:{SOURCE}" + source_args = ["--skill-root", str(SKILLS)] + if authority == "store": + source = KnowledgeReader(skill_root=SKILLS).read(identity) + KnowledgeStore(tmp_path, skill_root=SKILLS).create(local_candidate(source)) + identity = f"store:{LOCAL}" + source_args += ["--store", str(tmp_path)] + code, stdout, stderr = run_cli("read", identity, "--material", "--omit-resources", *source_args) + assert code == 0, stderr + view = json.loads(stdout) + code, stdout, stderr = run_cli( + "read", identity, "--revision", view["revision"], "--resource", view["entrypoint"], + "--omit-resources", *source_args, + ) + assert code == 0, stderr + raw = json.loads(stdout)["content"].encode("utf-8") + delivered = view["content"].encode("utf-8") + assert b" inferrsi: |" in raw and b" inferrsi: |" not in delivered + assert raw.split(b"---", 2)[2] == delivered.split(b"---", 2)[2] + proof = view["content_view"] + assert proof["locator"] == "skill-procedure" + assert proof["source_sha256"] == hashlib.sha256(raw).hexdigest() + assert proof["sha256"] == hashlib.sha256(delivered).hexdigest() + assert delivered == b"".join(raw[start:end] for start, end in proof["source_byte_ranges"]) + + +@pytest.mark.parametrize("identity", [ + "package:vllm_omni.common.diffusion.step_state_and_output_validation", + "package:inferrsi.common.skills.calibrate_serving_simulation", +]) +def test_read_can_omit_inventory_without_changing_material_or_revision(identity): + args = ("read", identity, "--material") code, stdout, stderr = run_cli(*args) assert code == 0, stderr full = json.loads(stdout) assert "record.json" in full["resources"] + assert "content_view" not in full + assert full["resources"]["record.json"] == hashlib.sha256(full["content"].encode()).hexdigest() assert full["required_resources"] code, stdout, stderr = run_cli(*args, "--omit-resources", "--revision", full["revision"]) diff --git a/tests/test_knowledge_contract.py b/tests/test_knowledge_contract.py index 014297398..ba9cda2a1 100644 --- a/tests/test_knowledge_contract.py +++ b/tests/test_knowledge_contract.py @@ -1,4 +1,5 @@ import json +import os import re from pathlib import Path @@ -8,6 +9,7 @@ from inferrsi.knowledge.skill_catalog import ( skill_directories, skill_directory, + skill_metadata, skill_records, skill_reference, ) @@ -19,30 +21,28 @@ def test_local_profiles_preserve_core_and_structured_ownership(): records = skill_records(ROOT) normalized = [] for record in records: - sidecar = skill_directory(ROOT, record["name"]) / "metadata.json" - if not sidecar.exists(): - continue - metadata = json.loads(sidecar.read_text(encoding="utf-8")) + directory = skill_directory(ROOT, record["name"]) + assert not (directory / "metadata.json").exists() + metadata = skill_metadata(ROOT, record["name"]) if metadata.get("profile") != "local-skill-v1": continue normalized.append(record["name"]) - assert "name" not in metadata and "description" not in metadata + assert metadata["name"] == record["name"] and metadata["description"] assert metadata["status"] == record["status"] assert metadata["kind"] == "skill" for readset in metadata["provenance_readsets"]: - assert (sidecar.parent / readset).is_file() - assert len(normalized) == 69 + assert (directory / readset).is_file() + assert len(normalized) == 77 -def test_local_sidecars_omit_fields_that_equal_reader_defaults(): +def test_local_profiles_omit_fields_that_equal_reader_defaults(): # Readers default absent sources/references to [] and applicability to {}; # a stored empty or placeholder value would only repeat that default. placeholder = {"scope": "See procedure prerequisites and counter-signals"} for record in skill_records(ROOT): - sidecar = skill_directory(ROOT, record["name"]) / "metadata.json" - if not sidecar.exists(): - continue - metadata = json.loads(sidecar.read_text(encoding="utf-8")) + directory = skill_directory(ROOT, record["name"]) + assert not (directory / "metadata.json").exists() + metadata = skill_metadata(ROOT, record["name"]) assert metadata.get("sources") != [], record["name"] assert metadata.get("references") != [], record["name"] assert metadata.get("applicability") != placeholder, record["name"] @@ -97,9 +97,14 @@ def test_contradictory_legacy_metadata_fails(tmp_path): def test_two_real_consumers_resolve_one_authoritative_case(): - reference = "../inference-optimization-casebook/cases/admission-drain-occupancy.md" - first = skill_reference(ROOT, "inference-benchmark-design", reference) - second = skill_reference(ROOT, "dense-performance-evidence", reference) + def read_case(owner): + directory = skill_directory(ROOT, owner) + case = skill_directory(ROOT, "inference-optimization-casebook") / "cases" + relative = os.path.relpath(case / "admission-drain-occupancy.md", directory) + return skill_reference(ROOT, owner, Path(relative).as_posix()) + + first = read_case("inference-benchmark-design") + second = read_case("dense-performance-evidence") assert first == second assert "## Observation" in first.read_text(encoding="utf-8") assert "## Interpretation" in first.read_text(encoding="utf-8") diff --git a/tests/test_knowledge_find.py b/tests/test_knowledge_find.py index f2cd56150..3b769cf4e 100644 --- a/tests/test_knowledge_find.py +++ b/tests/test_knowledge_find.py @@ -37,6 +37,17 @@ def corpus(): return load_corpus(ROOT) +def test_malformed_inline_metadata_never_becomes_searchable_body(tmp_path): + owner = tmp_path / "invalid" + owner.mkdir() + (owner / "SKILL.md").write_text( + "---\nname: invalid\ndescription: test\nmetadata:\n" + " inferrsi: |\n not-json-topsecret-token\n---\nordinary body\n", + encoding="utf-8", + ) + assert not load_corpus(tmp_path, kinds=("skill",)).documents + + def _ranked(result, identity): for position, item in enumerate(result["items"], 1): if item["id"] == identity: @@ -44,12 +55,30 @@ def _ranked(result, identity): return None +def test_find_loads_canonical_owners_without_compiled_catalog(monkeypatch): + from inferrsi.knowledge import catalog + + def unexpected_view(*args, **kwargs): + pytest.fail("find materialized a duplicate packaged-skill view") + + monkeypatch.setattr(catalog, "skill_views", unexpected_view) + corpus = load_corpus(ROOT) + assert len(corpus.documents) == len({doc.identity for doc in corpus.documents}) == 135 + + def test_corpus_covers_all_three_kinds_without_double_counting(corpus): kinds = {} for document in corpus.documents: kinds[document.kind] = kinds.get(document.kind, 0) + 1 assert kinds["skill"] == 80 + # All task procedures resolve by exact ID, outside the evidence corpus and its IDF. + navigation = {path.parent.name for path in (ROOT / "tasks").glob("*/SKILL.md")} + assert len(navigation) == 8 + assert navigation.isdisjoint(document.identity for document in corpus.documents) + day0 = find("day-0 model integration on Ascend", corpus, limit=3) + assert day0["items"] and _ranked(day0, "inference-task-router") is None assert kinds["case"] == 7 + assert kinds["record"] == 48 # Packaged-skill projections are excluded so one owner is not ranked twice. assert all( document.kind != "record" or "skills." not in document.identity @@ -100,7 +129,10 @@ def test_catalog_ranks_find_documents_over_the_filtered_package_scope(corpus, qu by_find_ref = {record.get("checkout_ref", f"package:{record['id']}"): record for record in inventory} eligible = [document for document in corpus.documents if document.qualified in by_find_ref] - assert len(eligible) == len(inventory) + # Navigation is available in an unranked listing and by exact ID. + unranked = set(by_find_ref) - {doc.qualified for doc in eligible} + navigation = {f"checkout:{path.parent.name}" for path in (ROOT / "tasks").glob("*/SKILL.md")} + assert unranked == navigation & set(by_find_ref) _, _, scored = rank(query, Corpus(eligible)) assert scored @@ -325,14 +357,14 @@ def test_finding_order_is_not_decided_by_corpus_common_terms(corpus): def _plant_case(root: Path, owner: str, *, registered: bool = True) -> None: """Copy one packaged case under `owner`, optionally leaving it unregistered.""" - source = ROOT / "methods/inference-optimization-casebook/cases" + source = ROOT / "casebooks/inference/inference-optimization-casebook/cases" cases = root / owner / "cases" cases.mkdir(parents=True) for suffix in (".json", ".md"): shutil.copy(source / f"qwen-ascend-rejected-directions{suffix}", cases) if registered: skill = root / owner / "SKILL.md" - body = (ROOT / "methods/inference-optimization-casebook/SKILL.md").read_text( + body = (ROOT / "casebooks/inference/inference-optimization-casebook/SKILL.md").read_text( encoding="utf-8" ) skill.write_text( @@ -446,6 +478,7 @@ def test_a_hit_read_refuses_gets_a_reason_not_a_command(tmp_path): load_corpus(tmp_path, kinds=("case",)), limit=1)["items"][0] assert not {"ref", "read_args", "open_argv"} & item.keys() assert "Missing case resource" in item["unreadable"] + assert "next_read_content_bytes" not in item @pytest.mark.parametrize("problem, reason", [ @@ -584,6 +617,9 @@ def test_find_opens_both_authorities_when_ids_overlap(tmp_path, capsys): assert args[:2] == ["read", f"{authority}:{identity}"] assert main(args) == 0 opened = json.loads(capsys.readouterr().out) + assert by_authority[authority]["next_read_content_bytes"] == len( + opened["content"].encode("utf-8") + ) assert opened["kind"] == kind assert opened["revision"].startswith("sha256:") if authority == "checkout": @@ -607,6 +643,56 @@ def test_find_preserves_a_structured_verification_limitation(tmp_path, capsys): assert "not locally verified" in capsys.readouterr().out +@pytest.mark.parametrize("limitations, shown", [ + (["short limit", "界" * 150, {"locally_verified": False}], ["short limit"]), + (["界" * 150, {"locally_verified": False}], []), + (["a", "b", "c", "d", {"locally_verified": False}], ["a", "b", "c"]), +]) +def test_find_bounds_caveats_and_sizes_the_opened_content(tmp_path, capsys, limitations, shown): + _plant_custom_case(tmp_path, { + "id": "case.demo", "kind": "case", "content": "demo.md", + "state": "historical_observation", "applicability": [], + "limitations": limitations, "attribution": "Local demo", + "sources": [{"type": "internal", "issue": "local", "artifact": "fixture"}], + }) + body = "精度\n".encode("utf-8") + (tmp_path / "demo/cases/demo.md").write_bytes(body) + args = ["find", "demo", "--kinds", "case", "--skill-root", str(tmp_path)] + assert main([*args, "--json"]) == 0 + item, = json.loads(capsys.readouterr().out)["items"] + assert item["next_read_content_bytes"] == len(body) + assert item["locally_verified"] is False + assert item["limitations"] == shown + assert item["limitations_withheld"] == len(limitations) - len(shown) + assert main(item["read_args"]) == 0 + material = json.loads(capsys.readouterr().out) + assert material["content"].encode("utf-8") == body + assert material["limitations"] == limitations + assert main(args) == 0 + out = capsys.readouterr().out + assert f"{len(body)} UTF-8 bytes" in out + assert f"{len(limitations) - len(shown)} more recorded limitation(s)" in out + + +def test_find_size_hint_allows_a_later_unpinned_checkout_edit(tmp_path, capsys): + _plant_custom_case(tmp_path, { + "id": "case.demo", "kind": "case", "content": "demo.md", + "state": "historical_observation", "attribution": "Local demo", + "applicability": [], "limitations": [], + "sources": [{"type": "internal", "issue": "local", "artifact": "fixture"}], + }) + assert main(["find", "demo", "--kinds", "case", "--skill-root", str(tmp_path), "--json"]) == 0 + hit, = json.loads(capsys.readouterr().out)["items"] + assert "unreadable" not in hit, hit.get("unreadable") + path = tmp_path / "demo/cases/demo.md" + assert hit["next_read_content_bytes"] == path.stat().st_size + revised = "新修订的较长证据。\n".encode("utf-8") + path.write_bytes(revised) + assert main(hit["read_args"]) == 0 + assert json.loads(capsys.readouterr().out)["content"].encode("utf-8") == revised + assert len(revised) > hit["next_read_content_bytes"] + + def test_unknown_outcome_is_rejected(tmp_path): _plant_custom_case(tmp_path, { "id": "case.demo", "kind": "case", "content": "demo.md", diff --git a/tests/test_knowledge_install.py b/tests/test_knowledge_install.py index 407bbb06d..d1b0ff313 100644 --- a/tests/test_knowledge_install.py +++ b/tests/test_knowledge_install.py @@ -1,5 +1,6 @@ """Clean-install contracts: package-only commands without PyYAML and the real-CLI checkout chain.""" +import hashlib import json import os import subprocess @@ -92,6 +93,7 @@ def test_packaged_skill_search_and_resource_work_without_pyyaml(without_yaml, tm assert read["revision"] == packaged["revision"] assert read["content_resource"] == original assert read["content"].encode("utf-8") == expected + assert hit["next_read_content_bytes"] == len(expected) output = tmp_path / "packaged-skill.md" fetched = _run([ @@ -119,9 +121,14 @@ def test_capture_scope_skill_delivers_packaged_text_without_pyyaml(without_yaml, assert context["revision"] == item["revision"] original = context["resources"][0] assert original["path"] == item["skill_documents"]["original"] - assert original["content"].encode() == ( + source = ( SKILLS / Path(original["path"]).relative_to("data/skill_library") ).read_bytes() + assert original["locator"] == "skill-procedure" + assert hashlib.sha256(source).hexdigest() == original["source_sha256"] + procedure = b"".join(source[start:end] for start, end in original["source_byte_ranges"]) + assert original["content"].encode() == procedure + assert hashlib.sha256(procedure).hexdigest() == original["sha256"] assert capture["omissions"] == [] diff --git a/tests/test_knowledge_operations.py b/tests/test_knowledge_operations.py index 7835a532c..ff2440faf 100644 --- a/tests/test_knowledge_operations.py +++ b/tests/test_knowledge_operations.py @@ -53,7 +53,9 @@ def test_resource_only_change_invalidates_old_snapshot_revision(tmp_path): def test_collect_limits_omissions_and_shared_resource_revision(): reader = KnowledgeReader(skill_root=ROOT) selected = reader.read("inference-benchmark-design") - shared = "methods/inference-optimization-casebook/cases/admission-drain-occupancy.md" + shared = ( + "casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md" + ) assert shared in selected.content and shared in selected.dependencies result = reader.collect( ids=["inference-benchmark-design", "dense-performance-evidence"], limit=1 diff --git a/tests/test_knowledge_store.py b/tests/test_knowledge_store.py index f1daf734a..9faf8d9ee 100644 --- a/tests/test_knowledge_store.py +++ b/tests/test_knowledge_store.py @@ -21,7 +21,7 @@ REPO = Path(__file__).resolve().parents[1] ROOT = REPO / "src/inferrsi/knowledge/data/skill_library" -NOTES = "methods/calibrate-serving-simulation/references/source-notes.md" +NOTES = "playbooks/measurement/calibrate-serving-simulation/references/source-notes.md" LOCAL = "calibrate-serving-simulation-local" @@ -367,7 +367,7 @@ def test_exported_tree_without_git_metadata_stays_protected(tmp_path): ) export = tmp_path / "export" export.mkdir() - subprocess.run(["tar", "-x", "-C", str(export)], input=archive.stdout, check=True) + subprocess.run(["tar", "-xf", "-", "-C", str(export)], input=archive.stdout, check=True) workspace = tmp_path / "workspace" workspace.mkdir() env = {**os.environ, "PYTHONPATH": str(export / "src"), "PYTHONUTF8": "1"} @@ -382,7 +382,9 @@ def test_real_shared_resource_bundle_keeps_old_dependency_bytes(tmp_path): original = adaptation("inference-benchmark-design", "inference-benchmark-design-local") store = store_at(tmp_path) revision = store.create(original) - shared = "methods/inference-optimization-casebook/cases/admission-drain-occupancy.md" + shared = ( + "casebooks/inference/inference-optimization-casebook/cases/admission-drain-occupancy.md" + ) identity = original.metadata["id"] assert store.resource(identity, shared, revision=revision) == original.content[shared] altered = dict(original.content) diff --git a/tests/test_knowledge_tiers.py b/tests/test_knowledge_tiers.py index e80ddb03c..383c2aa56 100644 --- a/tests/test_knowledge_tiers.py +++ b/tests/test_knowledge_tiers.py @@ -12,14 +12,13 @@ from inferrsi.knowledge.catalog import get_knowledge, search_knowledge from inferrsi.resources.repository_packs import repository_pack -ROOT = Path(__file__).resolve().parents[1] / "src/inferrsi/knowledge/data/knowledge_packs" +ROOT = Path(__file__).resolve().parents[1] / "src/inferrsi/knowledge/data/skill_library/records" def test_external_snapshot_has_complete_navigation_and_pinned_sources(): - paths = set(ROOT.glob("*/*/*/*/*.json")) - navigation = (ROOT / "README.md").read_text(encoding="utf-8") - links = re.findall(r"\]\(([^)]+\.json)\)", navigation) - assert {ROOT / link for link in links} == paths + paths = {p.stem for p in ROOT.glob("*.json")} + authored = knowledge_catalog.knowledge_records(include_skill_views=False) + assert paths == {r["id"] for r in authored} for tier in ("dependency", "peer"): records = search_knowledge(tier=tier, limit=100) assert {r["repo"] for r in records} == set(knowledge_catalog.KNOWLEDGE_SCOPES[tier]) @@ -84,8 +83,8 @@ def test_misclassified_external_card_is_rejected_before_retrieval( record["tier"] = bad_tier target = ( tmp_path - / "knowledge/data/knowledge_packs/dependency/flashinfer/cuda/llm" - / "paged-decode-dispatch.json" + / "knowledge/data/skill_library/records" + / "flashinfer.cuda.llm.paged_decode_dispatch.json" ) target.parent.mkdir(parents=True) target.write_text(json.dumps(record)) diff --git a/tests/test_lily_repo_to_skill.py b/tests/test_lily_repo_to_skill.py index f66af6140..678d50904 100644 --- a/tests/test_lily_repo_to_skill.py +++ b/tests/test_lily_repo_to_skill.py @@ -9,8 +9,10 @@ import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -LIBRARY = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories" +LIBRARY = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines" FIXTURES = PROJECT / "tests/fixtures/repo_to_skill" GRAPHS = ("lily",) @@ -47,7 +49,7 @@ def test_arex_runtime_graph_links_frontmatter_and_source_boundaries(): assert selected.exists(), (path, target) assert not list(root.rglob("__pycache__")) assert not list(root.rglob("*.pyc")) - metadata = json.loads((root / "metadata.json").read_text(encoding="utf-8")) + metadata = skill_metadata(root.parent, root.name) assert metadata["status"] == "candidate" assert metadata["construction"]["native_import"] == "not-run" assert "BLOCKED_REQUIRED_BACKEND" in " ".join(metadata["limitations"]) diff --git a/tests/test_local_dream_repo_to_skill.py b/tests/test_local_dream_repo_to_skill.py index 157c23908..425b9ed9d 100644 --- a/tests/test_local_dream_repo_to_skill.py +++ b/tests/test_local_dream_repo_to_skill.py @@ -16,8 +16,10 @@ import pytest import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories/local-dream" +ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines/local-dream" CASES = PROJECT / "tests/fixtures/repo_to_skill/local-dream/test-cases" SCRIPT = ROOT / "scripts/screen_declarations.py" MODULE = runpy.run_path(str(SCRIPT)) @@ -30,7 +32,7 @@ def test_owner_source_identity_rights_and_containment(): assert front["disable-model-invocation"] is True assert front["metadata"]["disco-role"] == "operating" assert re.search(r'^description: ".+"$', header, re.MULTILINE) - meta = json.loads((ROOT / "metadata.json").read_text()) + meta = skill_metadata(ROOT.parent, ROOT.name) assert meta["status"] == "candidate" and meta["empirical_validation"] == "not_run" assert "BLOCKED_REQUIRED_BACKEND" in " ".join(meta["limitations"]) ledger = json.loads((ROOT / "references/source-identity.json").read_text()) @@ -174,11 +176,12 @@ def test_cli_duplicate_nonfinite_invalid_encoding_and_bounds(raw, tmp_path): def test_registration_and_catalog_preserve_candidate_boundary(): - index = json.loads((ROOT.parents[1] / "skill-index.json").read_text()) + library = PROJECT / "src/inferrsi/knowledge/data/skill_library" + index = json.loads((library / "skill-index.json").read_text()) assert index["skills"]["local-dream"] == { - "path": "repositories/local-dream", "backend": "common", + "path": "systems/engines/local-dream", "backend": "common", } from inferrsi.knowledge.skill_catalog import skill_metadata, skill_resource - assert skill_metadata(ROOT.parents[1], "local-dream")["status"] == "candidate" - assert "desired" in skill_resource(ROOT.parents[1], "local-dream", - "references/android-lifecycle.md").read_text().lower() + assert skill_metadata(library, "local-dream")["status"] == "candidate" + lifecycle = skill_resource(library, "local-dream", "references/android-lifecycle.md") + assert "desired" in lifecycle.read_text().lower() diff --git a/tests/test_mistral_rs_repo_to_skill.py b/tests/test_mistral_rs_repo_to_skill.py index 7a42e8b67..dc5aa2678 100644 --- a/tests/test_mistral_rs_repo_to_skill.py +++ b/tests/test_mistral_rs_repo_to_skill.py @@ -16,8 +16,10 @@ import pytest import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories/mistral-rs" +ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines/mistral-rs" CASES = PROJECT / "tests/fixtures/repo_to_skill/mistral-rs/test-cases" SCRIPTS = { "decode-contract": "audit_decode_metadata.py", @@ -58,7 +60,7 @@ def test_mistral_rs_graph_is_contained_candidate_with_exact_source_scope(): continue resolved = (path.parent / unquote(link.path)).resolve() assert resolved.is_relative_to(ROOT.resolve()) and resolved.is_file(), (path, target) - metadata = json.loads((ROOT / "metadata.json").read_text()) + metadata = skill_metadata(ROOT.parent, ROOT.name) assert metadata["status"] == "candidate" assert metadata["empirical_validation"] == "not_run" assert metadata["construction"]["native_import"] == "not-run" diff --git a/tests/test_mlc_llm_repo_to_skill.py b/tests/test_mlc_llm_repo_to_skill.py index d87013382..0e4229064 100644 --- a/tests/test_mlc_llm_repo_to_skill.py +++ b/tests/test_mlc_llm_repo_to_skill.py @@ -17,8 +17,10 @@ import pytest import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories/mlc-llm" +ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines/mlc-llm" CASES = PROJECT / "tests/fixtures/repo_to_skill/mlc-llm/test-cases" SCRIPTS = { "artifact-contract": "audit_artifact_declarations.py", @@ -52,7 +54,7 @@ def test_mlc_graph_source_owners_rights_and_containment(): continue resolved = (path.parent / unquote(link.path)).resolve() assert resolved.is_relative_to(ROOT.resolve()) and resolved.is_file(), (path, target) - meta = json.loads((ROOT / "metadata.json").read_text()) + meta = skill_metadata(ROOT.parent, ROOT.name) assert meta["status"] == "candidate" and meta["empirical_validation"] == "not_run" assert "BLOCKED_REQUIRED_BACKEND" in " ".join(meta["limitations"]) identity = json.loads((ROOT / "references/source-identity.json").read_text()) diff --git a/tests/test_mlx_discovery.py b/tests/test_mlx_discovery.py index 3a76a61b7..73badfce9 100644 --- a/tests/test_mlx_discovery.py +++ b/tests/test_mlx_discovery.py @@ -154,7 +154,7 @@ def test_mlx_cards_are_source_scoped_and_readable(capsys): ]) == 0 assert {item["id"] for item in json.loads(capsys.readouterr().out)} == IDS reader = KnowledgeReader() - document = "data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" + document = "data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" # The bundle key stays logical; K02 moved the physical corpus under Knowledge. expected = (PROJECT / "src/inferrsi/knowledge" / document).read_bytes() for record in records: diff --git a/tests/test_mlx_knowledge.py b/tests/test_mlx_knowledge.py index cf2c3c329..4f82a11cf 100644 --- a/tests/test_mlx_knowledge.py +++ b/tests/test_mlx_knowledge.py @@ -17,7 +17,7 @@ def test_mlx_candidate_preserves_source_state_and_owner_resources(): "bff10220c742907f0c68e55ff174aaea56f74e3d" ) resource = skill_resource(LIBRARY, SKILL, "references/source-map.md") - assert resource.is_relative_to(LIBRARY / "frameworks/mlx" / SKILL) + assert resource.is_relative_to(LIBRARY / "systems/frameworks/mlx" / SKILL) assert "TEST-SOURCE" in resource.read_text(encoding="utf-8") diff --git a/tests/test_mlx_lm_knowledge.py b/tests/test_mlx_lm_knowledge.py index 8845adb78..98de0846d 100644 --- a/tests/test_mlx_lm_knowledge.py +++ b/tests/test_mlx_lm_knowledge.py @@ -17,7 +17,7 @@ def test_mlx_lm_candidate_preserves_source_and_application_owner(): "ab7f7048dc341ced2d84b1887eeb17d60b0d1b02" ) resource = skill_resource(LIBRARY, SKILL, "references/cache-stop.md") - assert resource.is_relative_to(LIBRARY / "repositories/mlx-lm" / SKILL) + assert resource.is_relative_to(LIBRARY / "systems/engines/mlx-lm" / SKILL) assert "LRUPromptCache" in resource.read_text(encoding="utf-8") diff --git a/tests/test_mlx_swift_lm_repo_to_skill.py b/tests/test_mlx_swift_lm_repo_to_skill.py index 70cc63c20..ad02375d7 100644 --- a/tests/test_mlx_swift_lm_repo_to_skill.py +++ b/tests/test_mlx_swift_lm_repo_to_skill.py @@ -17,8 +17,10 @@ import pytest import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories/mlx-swift-lm" +ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines/mlx-swift-lm" CASES = PROJECT / "tests/fixtures/repo_to_skill/mlx-swift-lm/test-cases" SOURCE_COMMIT = "3d066d3a0ffdc7e70c476a3d31488ea62dfdd0ac" SOURCE_FILES_SHA256 = "df10bf7c0523c89c51e37f61eb79323b256879538874b33c03a2e242e3762705" @@ -71,7 +73,7 @@ def test_mlx_swift_graph_is_contained_candidate_with_exact_34_file_scope(): continue resolved = (path.parent / unquote(link.path)).resolve() assert resolved.is_relative_to(ROOT.resolve()) and resolved.is_file(), (path, target) - metadata = json.loads((ROOT / "metadata.json").read_text()) + metadata = skill_metadata(ROOT.parent, ROOT.name) assert metadata["schema_version"] == 1 and metadata["kind"] == "skill" assert metadata["status"] == "candidate" and metadata["empirical_validation"] == "not_run" assert metadata["construction"]["native_import"] == "not-run" diff --git a/tests/test_optimization_story.py b/tests/test_optimization_story.py index ce42f3d45..caf7392f0 100644 --- a/tests/test_optimization_story.py +++ b/tests/test_optimization_story.py @@ -221,7 +221,8 @@ def test_dashboard_preserves_redacted_packaged_context_and_lesson(tmp_path): create_campaign(tmp_path, "camp", noise=1.5) path, digest = capture_context( tmp_path, context_id="r1", reader=KnowledgeReader(), - ids=["inferrsi.common.skills.kv_tier_management"], context_bytes=262144) + ids=["inferrsi.common.skills.kv_tier_management", "inferrsi.common.skills.local_dream"], + context_bytes=262144) original = path.read_bytes() document = json.loads(original) assert "/dev/ng" in document["assembled_text"] diff --git a/tests/test_performance_ceiling.py b/tests/test_performance_ceiling.py index 374c2efdb..7f28fe298 100644 --- a/tests/test_performance_ceiling.py +++ b/tests/test_performance_ceiling.py @@ -10,7 +10,8 @@ from inferrsi.knowledge.knowledge_operations import KnowledgeReader ROOT = Path(__file__).resolve().parents[1] -OWNER = ROOT / "src/inferrsi/knowledge/data/skill_library/methods/performance-ceiling-analysis" +OWNER = (ROOT / "src/inferrsi/knowledge/data/skill_library/playbooks/measurement" + / "performance-ceiling-analysis") CODE = compile((OWNER / "scripts/roofline.py").read_text(encoding="utf-8"), "roofline.py", "exec") NAMESPACE = {} exec(CODE, NAMESPACE) @@ -193,7 +194,9 @@ def test_owner_resources_are_readable_at_exact_checkout_revision(monkeypatch): "references/roofline-contract.md", "references/roofline-example.json", ): - resource = "methods/performance-ceiling-analysis/" + relative + resource = "playbooks/measurement/performance-ceiling-analysis/" + relative assert bundle.content[resource] == (OWNER / relative).read_bytes() - assert reader.resource(reference, "methods/performance-ceiling-analysis/scripts/roofline.py", - revision=bundle.revision) == (OWNER / "scripts/roofline.py").read_bytes() + script = "playbooks/measurement/performance-ceiling-analysis/scripts/roofline.py" + assert reader.resource(reference, script, revision=bundle.revision) == ( + OWNER / "scripts/roofline.py" + ).read_bytes() diff --git a/tests/test_project_maya_repo_to_skill.py b/tests/test_project_maya_repo_to_skill.py index c6eb3f885..3d0f8a380 100644 --- a/tests/test_project_maya_repo_to_skill.py +++ b/tests/test_project_maya_repo_to_skill.py @@ -11,8 +11,10 @@ import pytest import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories/project-maya" +ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines/project-maya" CASES = PROJECT / "tests/fixtures/repo_to_skill/project-maya/test-cases" @@ -34,7 +36,7 @@ def test_maya_graph_links_frontmatter_source_and_unverified_state(): continue selected = (path.parent / unquote(link.path)).resolve() assert selected.is_relative_to(ROOT.resolve()) and selected.is_file(), (path, target) - metadata = json.loads((ROOT / "metadata.json").read_text()) + metadata = skill_metadata(ROOT.parent, ROOT.name) assert metadata["status"] == "candidate" assert metadata["empirical_validation"] == "partial_cpu_only" assert metadata["construction"]["native_import"] == "not-run" diff --git a/tests/test_pytorch_torchnpu_integration.py b/tests/test_pytorch_torchnpu_integration.py index 266c0ade1..4e76aeda8 100644 --- a/tests/test_pytorch_torchnpu_integration.py +++ b/tests/test_pytorch_torchnpu_integration.py @@ -26,8 +26,8 @@ def _pinned(path: Path) -> str: @pytest.mark.parametrize("name,backend", GRAPHS) def test_framework_roots_register_graphs_without_registering_children(name, backend): registry = json.loads((LIBRARY / "skill-index.json").read_text(encoding="utf-8"))["skills"] - owner = LIBRARY / "frameworks" / name / name - assert registry[name] == {"path": f"frameworks/{name}/{name}", "backend": backend} + owner = LIBRARY / "systems/frameworks" / name / name + assert registry[name] == {"path": f"systems/frameworks/{name}/{name}", "backend": backend} children = sorted((owner / "sub-skills").glob("*/SKILL.md")) assert len(children) == 8 assert not {path.parent.name for path in children} & set(registry) @@ -35,24 +35,25 @@ def test_framework_roots_register_graphs_without_registering_children(name, back for document in [owner / "SKILL.md", *children]: frontmatter = document.read_text(encoding="utf-8").split("---", 2)[1] header = yaml.safe_load(frontmatter) - assert set(header) == {"name", "description"} + assert set(header) == {"name", "description", "metadata"} assert header["name"] == document.parent.name assert re.search(r'^description: ".+"$', frontmatter, re.MULTILINE) - sidecar = json.loads(document.with_name("metadata.json").read_text(encoding="utf-8")) - assert sidecar["profile"] == "local-skill-v1" - assert sidecar["status"] == "candidate" - assert sidecar["disable-model-invocation"] is True - assert sidecar["metadata"]["disco-role"] == "operating" - assert isinstance(sidecar["license"], str) and sidecar["license"].strip() - assert sidecar["license"] == repository_license + fields = json.loads(header["metadata"]["inferrsi"]) + assert fields["profile"] == "local-skill-v1" + assert fields["status"] == "candidate" + assert fields["disable-model-invocation"] is True + assert fields["metadata"]["disco-role"] == "operating" + assert isinstance(fields["license"], str) and fields["license"].strip() + assert fields["license"] == repository_license # A child is still a valid local skill when selected from its flat parent. root = LIBRARY if document.parent == owner else owner / "sub-skills" - assert skill_metadata(root, header["name"]) == header | sidecar + native = {key: header[key] for key in ("name", "description")} + assert skill_metadata(root, header["name"]) == native | fields @pytest.mark.parametrize("name,backend", GRAPHS) -def test_framework_graph_resources_and_sidecars_survive_checkout_bundle_reads(name, backend): - owner = LIBRARY / "frameworks" / name / name +def test_framework_graph_resources_survive_checkout_bundle_reads(name, backend): + owner = LIBRARY / "systems/frameworks" / name / name reader = KnowledgeReader(skill_root=LIBRARY) identity = f"checkout:{name}" bundle = reader.read(identity) @@ -73,25 +74,24 @@ def test_framework_graph_resources_and_sidecars_survive_checkout_bundle_reads(na @pytest.mark.parametrize("name,backend", GRAPHS) def test_framework_catalog_preserves_owner_metadata_and_scope_without_promotion(name, backend): - owner = LIBRARY / "frameworks" / name / name + owner = LIBRARY / "systems/frameworks" / name / name record = get_knowledge(f"inferrsi.{backend}.skills.{name.replace('-', '_')}") assert record["backend"] == backend assert record["status"] == "imported" assert record["skill_metadata"] == skill_metadata(LIBRARY, name) assert not record["verification"]["locally_verified"] assert record["skill_documents"] == { - "original": f"data/skill_library/frameworks/{name}/{name}/SKILL.md", + "original": f"data/skill_library/systems/frameworks/{name}/{name}/SKILL.md", } source_paths = set(record["source_files"]) for document in owner.rglob("SKILL.md"): # source_files names each input at the pinned source_revision, where the # corpus still sat directly under src/inferrsi/data. assert _pinned(document) in source_paths - assert _pinned(document.with_name("metadata.json")) in source_paths def test_pytorch_retained_license_and_notice_match_declared_immutable_blob_bytes(): - owner = LIBRARY / "frameworks/pytorch/pytorch" + owner = LIBRARY / "systems/frameworks/pytorch/pytorch" ledger = json.loads((owner / "references/source-readset.json").read_text(encoding="utf-8")) metadata = skill_metadata(LIBRARY, "pytorch") assert ledger["commit"] in {source["commit"] for source in metadata["sources"]} @@ -103,7 +103,7 @@ def test_pytorch_retained_license_and_notice_match_declared_immutable_blob_bytes def test_torchnpu_retained_source_documentation_and_third_party_notices_keep_blob_bytes(): - owner = LIBRARY / "frameworks/torch-npu/torch-npu" + owner = LIBRARY / "systems/frameworks/torch-npu/torch-npu" ledger = json.loads( (owner / "references/licenses/source-identities.json").read_text(encoding="utf-8") ) diff --git a/tests/test_repo_to_skill.py b/tests/test_repo_to_skill.py index 6474956e2..b6a8b770c 100644 --- a/tests/test_repo_to_skill.py +++ b/tests/test_repo_to_skill.py @@ -9,8 +9,10 @@ import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -LIBRARY = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories" +LIBRARY = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines" FIXTURES = PROJECT / "tests/fixtures/repo_to_skill" GRAPHS = ("dwarfstar",) @@ -47,7 +49,7 @@ def test_arex_runtime_graph_links_frontmatter_and_source_boundaries(): assert selected.exists(), (path, target) assert not list(root.rglob("__pycache__")) assert not list(root.rglob("*.pyc")) - metadata = json.loads((root / "metadata.json").read_text(encoding="utf-8")) + metadata = skill_metadata(root.parent, root.name) assert metadata["status"] == "candidate" assert metadata["construction"]["native_import"] == "not-run" assert "BLOCKED_REQUIRED_BACKEND" in " ".join(metadata["limitations"]) diff --git a/tests/test_research.py b/tests/test_research.py index 3beaf8ad6..81dda5291 100644 --- a/tests/test_research.py +++ b/tests/test_research.py @@ -69,7 +69,7 @@ def test_scoped_mlx_research_retains_the_lifecycle_resource(): if item["record"]["id"] == "inferrsi.common.skills.lazy_evaluation_stream_lifetimes" ) assert lifecycle["record"]["skill_documents"]["original"] == ( - "data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" + "data/skill_library/systems/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md" ) assert {"group": "framework", "query": "mlx"} in lifecycle["matched_queries"] assert {"group": "mechanism", "query": "lazy evaluation"} in lifecycle["matched_queries"] @@ -181,7 +181,7 @@ def test_model_profile_links_packaged_resources_at_the_repository_owner(alias): assert result["worked_example_resource"] == profile["worked_example_resource"] for key in ("skill_resource", "worked_example_resource", "inventory_resource"): resource = profile[key] - assert resource.startswith("data/skill_library/repositories/h3-c/") + assert resource.startswith("data/skill_library/systems/engines/h3-c/") assert package.joinpath(resource).read_text(encoding="utf-8") record = get_knowledge(profile["knowledge_ids"][0]) assert record["skill_documents"]["original"] == profile["skill_resource"] diff --git a/tests/test_resource_migration.py b/tests/test_resource_migration.py index d0a5d52c1..602935b1c 100644 --- a/tests/test_resource_migration.py +++ b/tests/test_resource_migration.py @@ -18,7 +18,7 @@ import pytest -from inferrsi.knowledge import _source_snapshot +from inferrsi.knowledge import _source_snapshot, knowledge_operations from inferrsi.knowledge.knowledge_operations import KnowledgeReader from inferrsi.resources.repository_packs import REPO_TYPES, repository_pack @@ -77,8 +77,7 @@ def test_old_resource_tier_is_gone_and_the_two_roots_hold_every_resource(checkou assert sorted(harness) == sorted( f"harness/data/repo_packs/{repo}/pack.json" for repo in REPO_TYPES ) - for group in ("skill_library", "knowledge_packs", "research_profiles", "repo_packs"): - assert any(name.startswith(f"knowledge/data/{group}/") for name in knowledge), group + assert {name.split("/")[2] for name in knowledge} == {"skill_library"} def test_importlib_and_checkout_loaders_resolve_the_same_files_and_bytes(checkout_inventory): @@ -130,5 +129,30 @@ def test_bundle_keys_and_pinned_provenance_survive_the_physical_move(): assert "data/skill_library" in bundle.dependencies # After the explicit re-pin, LIBRARY agrees with today's checkout path. assert _source_snapshot.LIBRARY == "src/inferrsi/knowledge/data/skill_library" - snapshot = KNOWLEDGE_DATA / "repo_packs" / "inferrsi" / "knowledge" / _source_snapshot.MANIFEST + snapshot = KNOWLEDGE_DATA / "skill_library" / "_compiled" / _source_snapshot.MANIFEST assert f'"{_source_snapshot.REGISTRY}"' in snapshot.read_text(encoding="utf-8") + + +def test_build_refresh_preserves_linked_record_pin_but_source_edits_invalidate_it( + tmp_path, monkeypatch, +): + reader = KnowledgeReader() + ref = "package:sglang.common.llm.runtime_radix_hicache" + linked = reader.records()[ref.removeprefix("package:")]["skill_documents"]["en"] + target = tmp_path / "knowledge" / linked + target.parent.mkdir(parents=True) + target.write_bytes((CHECKOUT / "knowledge" / linked).read_bytes()) + root = tmp_path / "knowledge/data/skill_library" + for group in ("_compiled", "records", "config"): + (root / group).mkdir() + monkeypatch.setattr(knowledge_operations, "files", lambda _: tmp_path) + original = reader.read(ref) + assert linked in original.content + (root / "_compiled/_provenance.json").write_text('{"source_revision":"refreshed"}') + (root / "config/other.json").write_text("{}") + (root / "records/other.json").write_text("{}") + replay = KnowledgeReader().read(ref, revision=original.revision) + assert replay.content == original.content and replay.dependencies == original.dependencies + target.write_bytes(target.read_bytes() + b"\nEdited guide\n") + with pytest.raises(knowledge_operations.RevisionConflict): + KnowledgeReader().read(ref, revision=original.revision) diff --git a/tests/test_round_citations.py b/tests/test_round_citations.py index faee2f30b..67f063fa7 100644 --- a/tests/test_round_citations.py +++ b/tests/test_round_citations.py @@ -42,7 +42,7 @@ def test_managed_round_resolves_citations_from_original_tool_outputs( "rather than a launch-overhead guess." ) reason = f"Use host/device timing for the next measurement [use {ref} | {quote}]." - path = "data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md" + path = "data/skill_library/systems/platforms/ascend/ascend-aclgraph-capture-replay/SKILL.md" read_argv, read_output = _cli(tmp_path, "read", ref, "--resource", path, "--omit-resources") else: diff --git a/tests/test_skill_catalog.py b/tests/test_skill_catalog.py index c626ddcf0..7cd3a86a6 100644 --- a/tests/test_skill_catalog.py +++ b/tests/test_skill_catalog.py @@ -2,8 +2,10 @@ from pathlib import Path import pytest +import yaml from inferrsi.apps.cli import main +from inferrsi.knowledge.catalog import _skill_search_source from inferrsi.knowledge.skill_catalog import ( get_skill, skill_directories, @@ -17,6 +19,37 @@ PROJECT = Path(__file__).resolve().parents[1] +@pytest.mark.parametrize(("relative", "children"), [ + ("frameworks/pytorch/pytorch", ( + "compiler-and-export", "custom-operators-and-extension", "distributed-and-checkpoint", + "installation-and-build", "memory-streams-and-graphs", "profiling-and-debugging", + "tensor-dispatch-autograd", "training-and-data", + )), + ("frameworks/torch-npu/torch-npu", ( + "distributed-and-hccl", "graph-and-compile", "installation-and-compatibility", + "memory-streams-and-events", "operator-development", "privateuse1-and-dispatch", + "profiling-and-debugging", "training-and-precision", + )), + ("engines/lithos-metal", ("compile-fusion", "generation-state", "serving-cache")), +]) +def test_imported_invocation_flags_retain_their_pre_layout_visibility(relative, children): + # At the pre-layout base these 22 flags were data in metadata.json. Promoting + # them into a host's YAML would newly disable automatic Claude invocation. + library = PROJECT / "src/inferrsi/knowledge/data/skill_library" + owner = library / "systems" / relative + for directory in (owner, *(owner / "sub-skills" / child for child in children)): + text = (directory / "SKILL.md").read_text(encoding="utf-8") + host = yaml.safe_load(text.split("---", 2)[1]) + assert "disable-model-invocation" not in host + assert "disco-role" not in host["metadata"] + assert isinstance(host["metadata"]["inferrsi"], str) + annotation = json.loads(host["metadata"]["inferrsi"]) + root = library if directory == owner else directory.parent + for fields in (annotation, skill_metadata(root, directory.name)): + assert fields["disable-model-invocation"] is True + assert fields["metadata"]["disco-role"] == "operating" + + def _symlink(target: Path, link: Path) -> None: # Symlink creation needs Developer Mode or admin rights on Windows; # skip instead of failing where the platform refuses. @@ -206,3 +239,45 @@ def test_explicit_empty_recommendations_is_a_statement_not_a_shape_error(tmp_pat assert skill_metadata(tmp_path, "selected")["applicability"] == { "hardware": "not-applicable" } + + +@pytest.mark.parametrize("style", ["|", "|-", "|+"]) +def test_compact_metadata_is_identical_for_packaged_and_checkout_readers(tmp_path, style): + directory = tmp_path / "selected" + directory.mkdir() + text = ( + "---\nname: selected\ndescription: test\nmetadata:\n" + " disco-role: operating\n" + f" inferrsi: {style}\n" + ' {"status": "candidate", "tags": ["unique-tag"]}\n' + " scope: local\n---\nUse the procedure here.\n" + ) + (directory / "SKILL.md").write_text(text, encoding="utf-8") + lexical, fields = _skill_search_source(text) + assert fields == {"status": "candidate", "tags": ["unique-tag"]} + assert "unique-tag" not in lexical + assert "disco-role: operating" in lexical and "scope: local" in lexical + metadata = skill_metadata(tmp_path, "selected") + assert metadata == { + "name": "selected", "description": "test", + "metadata": {"disco-role": "operating", "scope": "local"}, **fields, + } + (directory / "metadata.json").write_text(json.dumps(fields), encoding="utf-8") + with pytest.raises(ValueError): + skill_metadata(tmp_path, "selected") + + +@pytest.mark.parametrize("declaration", [ + """ inferrsi: '{"status":"candidate"}'""", + """ 'inferrsi': '{"status":"candidate"}'""", + """ inferrsi: |\n broken JSON""", +]) +def test_checkout_rejects_unreadable_embedded_declarations(tmp_path, declaration): + directory = tmp_path / "selected" + directory.mkdir() + (directory / "SKILL.md").write_text( + "---\nname: selected\ndescription: test\nmetadata:\n" + declaration + "\n---\n", + encoding="utf-8", + ) + with pytest.raises(ValueError): + skill_metadata(tmp_path, "selected") diff --git a/tests/test_skill_coverage_view.py b/tests/test_skill_coverage_view.py index 9c9d63061..331af0e8a 100644 --- a/tests/test_skill_coverage_view.py +++ b/tests/test_skill_coverage_view.py @@ -27,38 +27,27 @@ def test_view_lists_every_registered_skill_exactly_once(): def test_view_rows_copy_owner_state_without_promotion(): - registry = _registry() rows, _, _, _, _ = coverage.collect(PROJECT) for row in rows: assert row["task"] assert coverage.layer_state(row) != "conflict", row["id"] - if not row["sidecar"]: - # No sidecar means no owner state to copy; it must stay unknown. - assert row["status"] == "unknown" - assert row["validation"] == "unknown" - continue - sidecar = json.loads( - (LIBRARY / registry[row["id"]]["path"] / "metadata.json").read_text(encoding="utf-8") - ) + owner = coverage.skill_metadata(LIBRARY, row["id"]) # The projection equals its owner, including a state other than not_run. - assert row["status"] == sidecar.get("status", "unknown") - assert row["validation"] == sidecar.get("empirical_validation", "unknown") + assert row["status"] == owner.get("status", "unknown") + assert row["validation"] == owner.get("empirical_validation", "unknown") def test_non_default_owner_state_is_projected_not_downgraded(): - """A sidecar may record a validation state other than not_run.""" + """A skill may record a validation state other than not_run.""" registry = _registry() rows, _, _, _, _ = coverage.collect(PROJECT) by_id = {row["id"]: row for row in rows} non_default = {} - for name, entry in registry.items(): - sidecar = LIBRARY / entry["path"] / "metadata.json" - if not sidecar.is_file(): - continue - validation = json.loads(sidecar.read_text(encoding="utf-8")).get("empirical_validation") + for name in registry: + validation = coverage.skill_metadata(LIBRARY, name).get("empirical_validation") if validation not in (None, "not_run"): non_default[name] = validation - assert non_default, "expected at least one non-not_run sidecar at this revision" + assert non_default, "expected at least one non-not_run owner at this revision" for name, validation in non_default.items(): assert by_id[name]["validation"] == validation @@ -152,7 +141,7 @@ def test_absent_and_empty_applicability_are_both_unknown_but_distinct_from_na(): def test_gaps_separate_unknown_applicability_from_a_declared_statement(): - base = {"layers": {}, "layer_exclusion": None, "rm": [], "sidecar": True, "eval": [], + base = {"layers": {}, "layer_exclusion": None, "rm": [], "eval": [], "conditions": coverage.conditions({"applicability": {"scope": "x"}})} assert "no-applicability" not in coverage.gaps(base) # A not-applicable-only declaration is a statement, not a missing field. @@ -167,7 +156,7 @@ def test_gaps_separate_unknown_applicability_from_a_declared_statement(): def test_declared_workflow_conditions_reach_the_rows(): - """The five workflow sidecars expose typed conditions, not only a scope.""" + """The five workflow owners expose typed conditions, not only a scope.""" rows, _, _, _, _ = coverage.collect(PROJECT) by_id = {row["id"]: row for row in rows} for name in ( @@ -190,7 +179,8 @@ def test_rendered_view_shows_the_collected_contract_and_advice_summary(): rendered = coverage.render(rows, [], 0, [], "digest") workflow = next(row for row in rows if row["id"] == "claim-contract") line = next( - text for text in rendered.splitlines() if "workflows/claim-contract/SKILL.md" in text + text for text in rendered.splitlines() + if "workflows/governance/claim-contract/SKILL.md" in text ) assert "failure_modes, input_contract, produced_artifact, refusal_point" in line assert f"{workflow['advice']} advice" in line @@ -208,7 +198,7 @@ def test_na_only_declaration_keeps_its_marker_without_a_scope(): rows = [{ "id": "probe", "path": "methods/probe", "backend": "common", "category": "methods", "task": "probe", "status": "candidate", - "validation": "not_run", "readsets": 0, "sidecar": True, + "validation": "not_run", "readsets": 0, "conditions": coverage.conditions({"applicability": {"hardware": "not-applicable"}}), "typed_conditions": [], "advice": 0, "layers": {}, "layer_exclusion": None, "other_ss": [], @@ -255,15 +245,15 @@ def test_applicability_is_descriptive_and_never_promotes_validation(): rows, _, _, _, _ = coverage.collect(PROJECT) described = [row for row in rows if row["typed_conditions"]] - assert described, "expected at least one workflow sidecar at this revision" + assert described, "expected at least one workflow owner at this revision" for row in described: metadata = skill_metadata(LIBRARY, row["id"]) assert metadata.get("verification", {}).get("locally_verified", False) is False assert row["validation"] == metadata.get("empirical_validation", "unknown") -def test_recommendations_default_only_when_the_sidecar_is_silent(): - from tools import build_skill_index as index_builder +def test_recommendations_default_only_when_the_owner_is_silent(): + from inferrsi.knowledge import catalog as index_builder assert index_builder.recommendations({}) == index_builder.DEFAULT_RECOMMENDATIONS # An explicit empty list is a statement, so it must not fall back. @@ -331,7 +321,12 @@ def test_resource_layer_decisions_preserve_unknown_na_and_conflicts(tmp_path): for path in library.rglob("*") if path.is_file() } outputs = index_builder.render_index(library, "0" * 40, hashes) - record = json.loads(outputs["common/skills/process.json"]) + from inferrsi.knowledge.catalog import COMPILED, skill_views + + (library / COMPILED).mkdir() + for name, content in outputs.items(): + (library / COMPILED / name).write_text(content, encoding="utf-8") + record = next(r for r in skill_views(library) if r["title"] == "process") assert record["skill_metadata"]["searchspace_not_applicable"] == reason assert record["skill_metadata"]["status"] == "candidate" assert record["skill_metadata"]["empirical_validation"] == "not_run" diff --git a/tests/test_skill_eval_planned_guidance.py b/tests/test_skill_eval_planned_guidance.py index a58712be3..670fd9f2b 100644 --- a/tests/test_skill_eval_planned_guidance.py +++ b/tests/test_skill_eval_planned_guidance.py @@ -116,7 +116,7 @@ def test_unusable_projection_pin_never_yields_a_concrete_check(tmp_path, mutatio elif mutation == "mismatch": canonical.write_text("changed canonical input\n") elif mutation == "mode": - readme.chmod(0o755) + git(repo, "update-index", "--chmod=+x", readme.relative_to(repo).as_posix()) elif mutation == "symlink": readme.unlink() readme.symlink_to(canonical) @@ -126,7 +126,8 @@ def test_unusable_projection_pin_never_yields_a_concrete_check(tmp_path, mutatio readme.write_text(readme.read_text().replace(pin, "f" * 40)) else: canonical.unlink() - git(repo, "add", "-A") + if mutation != "mode": + git(repo, "add", "-A") git(repo, "commit", "--quiet", "-m", "Synthetic unavailable source") identity["head"] = git(repo, "rev-parse", "HEAD") check = pr.projection_check(repo, identity, changes) @@ -324,6 +325,21 @@ def commit_projection_fixture(repo, identity, executable=()): identity["head"] = git(repo, "rev-parse", "HEAD") +def test_only_library_overview_readme_is_outside_projection_inputs(tmp_path): + repo, _, canonical, pin, identity, changes, _, _ = manifest_fixture(tmp_path) + commit_projection_fixture(repo, identity) + canonical.with_name("README.md").write_text("Updated library navigation\n") + commit_projection_fixture(repo, identity) + assert pr.projection_check(repo, identity, changes) == { + "status": "ready", "source_revision": pin, + "declared_in": pr.PROJECTION_PATH + "/_provenance.json"} + resource = canonical.parent / "example-skill" / "README.md" + resource.parent.mkdir() + resource.write_text("Updated skill resource\n") + commit_projection_fixture(repo, identity) + assert pr.projection_check(repo, identity, changes)["status"] == "unavailable" + + def test_manifest_projection_pin_is_authoritative_and_reads_raw_git(tmp_path): repo, readme, _, pin, identity, changes, manifest, _ = manifest_fixture(tmp_path) readme.write_text("README no longer carries a source pin.\n") diff --git a/tests/test_skill_eval_pr.py b/tests/test_skill_eval_pr.py index d6cd20509..aa11c8b63 100644 --- a/tests/test_skill_eval_pr.py +++ b/tests/test_skill_eval_pr.py @@ -48,6 +48,7 @@ def test_plan_selects_addition_update_resources_without_execution(tmp_path, mode @pytest.mark.parametrize("skill,mode", [ ("audit-fixed-workload-claim", "addition"), ("inference-benchmark-design", "update"), + ("inference-benchmark-design", "moved"), ]) def test_maintained_mapping_selects_real_acceptance_paths(tmp_path, skill, mode): repo = tmp_path / "repo" @@ -61,14 +62,19 @@ def test_maintained_mapping_selects_real_acceptance_paths(tmp_path, skill, mode) registration = {"schema_version": 1, "skills": { skill: {"path": path.parent.relative_to(index.parent).as_posix(), "backend": "common"}}} body = f"---\nname: {skill}\ndescription: Fixed-workload audit fixture\n---\n" - if mode == "update": - path.parent.mkdir(parents=True) - path.write_text(body + "Old audit guidance\n", encoding="utf-8") - write_json(index, registration) + if mode != "addition": + previous = (index.parent / "old-category" / skill / "SKILL.md" + if mode == "moved" else path) + previous.parent.mkdir(parents=True) + previous.write_text(body + "Old audit guidance\n", encoding="utf-8") + write_json(index, {"schema_version": 1, "skills": {skill: { + "path": previous.parent.relative_to(index.parent).as_posix(), "backend": "common"}}}) git(repo, "add", mapping["skill_root"]) git(repo, "add", "seed.txt") git(repo, "commit", "--quiet", "-m", "Fixture acceptance baseline") before = git(repo, "rev-parse", "HEAD") + if mode == "moved": + previous.unlink() path.parent.mkdir(parents=True, exist_ok=True) path.write_text(body + "Audit every request and retain late outliers\n", encoding="utf-8") write_json(index, registration) @@ -80,14 +86,19 @@ def test_maintained_mapping_selects_real_acceptance_paths(tmp_path, skill, mode) "head": head, "version": head}, tmp_path / "planned") assert len(plan["comparisons"]) == 1 and plan["estimated_calls"] == 2 selected = plan["comparisons"][0] - assert selected["skill"] == skill and selected["mode"] == mode + assert selected["skill"] == skill + assert selected["mode"] == ("update" if mode == "moved" else mode) assert selected["cases"] == [{"id": "audit-threshold", "split": "holdout"}] assert selected["exam"]["grader"] == sha256(ROOT / "tools/skill_eval_verifier.py") - assert {"status": "A" if mode == "addition" else "M", + assert {"status": "M" if mode == "update" else "A", "path": path.relative_to(repo).as_posix(), "coverage": "covered"} in plan["coverage"] - assert [change for change in plan["coverage"] if change["coverage"] != "covered"] == ( - [{"status": "A", "path": index.relative_to(repo).as_posix(), "coverage": "uncovered"}] - if mode == "addition" else []) + if mode == "moved": + frozen = compare.load_plan(tmp_path / "planned" / skill) + assert frozen["arms"]["baseline"]["bundle"] is not None + else: + assert [change for change in plan["coverage"] if change["coverage"] != "covered"] == ( + [{"status": "A", "path": index.relative_to(repo).as_posix(), "coverage": "uncovered"}] + if mode == "addition" else []) assert plan["model"] == {"name": "gpt-6.1-sol", "effort": "xhigh"} assert integration.budget(plan)["comparisons"] == 1 assert not (tmp_path / "planned/execution-claim.json").exists() diff --git a/tests/test_skill_index.py b/tests/test_skill_index.py index 7267ac07c..0793cb099 100644 --- a/tests/test_skill_index.py +++ b/tests/test_skill_index.py @@ -18,6 +18,21 @@ ROOT = PROJECT / LIBRARY +def test_index_cli_reports_unsupported_git_before_reading_objects(monkeypatch, capsys): + def old_git(command, **kwargs): + assert command == ["git", "--version"] + return "git version 2.43.0\n" + + monkeypatch.setattr(subprocess, "check_output", old_git) + monkeypatch.setattr(index_builder, "_git", lambda *args: pytest.fail("read before preflight")) + with pytest.raises(SystemExit) as exit_info: + index_builder.main(["--source-revision", "HEAD", "--check"]) + assert exit_info.value.code == 2 + message = capsys.readouterr().err + assert "requires Git >= 2.45 for --no-lazy-fetch" in message + assert "git version 2.43.0. Upgrade Git and retry." in message + + def _physical(relative: str) -> Path: """Map a pinned input name to today's checkout location. @@ -33,7 +48,7 @@ def test_complete_index_retains_metadata_states_and_unknown_license(): records = search_knowledge(repo="inferrsi", limit=100) names = {record["name"] for record in skill_records(ROOT)} registry = json.loads((ROOT / "skill-index.json").read_text(encoding="utf-8"))["skills"] - assert len(records) == len(names) == 80 + assert len(records) == len(names) == 88 assert {record["skill_metadata"]["name"] for record in records} == names states = [] for record in records: @@ -51,7 +66,7 @@ def test_complete_index_retains_metadata_states_and_unknown_license(): } assert set(record["source_files"]) == set(record["sources"][0]["paths"]) states.append(metadata.get("status")) - assert states.count("candidate") == 80 + assert states.count("candidate") == 88 profiling = next(r for r in records if r["title"] == "ascend-operator-profiling") assert profiling["skill_metadata"]["license"] == "UNKNOWN" assert profiling["skill_metadata"]["synced-commit"] == ( @@ -177,7 +192,7 @@ def git(*args): monkeypatch.setattr(index_builder, "LIBRARY", fixture_library) assert index_builder.main(["--source-revision", "HEAD"]) == 0 assert index_builder.main(["--source-revision", "HEAD", "--check"]) == 0 - generated = tmp_path / DESTINATION / "common/skills/example.json" + generated = tmp_path / DESTINATION / "inputs.json" generated.write_text("changed projection", encoding="utf-8") assert index_builder.main(["--source-revision", "HEAD", "--check"]) == 1 assert generated.read_text(encoding="utf-8") == "changed projection" @@ -227,22 +242,18 @@ def commit(message): monkeypatch.setattr(index_builder, "LIBRARY", LIBRARY) first = build_index(tmp_path, "HEAD") - record = json.loads(first["common/skills/example.json"]) - recorded = set(record["source_files"]) - assert recorded - prefix = LIBRARY.as_posix() - assert all(path.startswith(prefix + "/") for path in recorded), recorded - entry = f"{prefix}/methods/example/SKILL.md" - assert record["source_files"][entry] == hashlib.sha256(original.encode("utf-8")).hexdigest() + record = json.loads(first["inputs.json"])["example"] + assert set(record["files"]) == {"SKILL.md"} + assert record["files"]["SKILL.md"] == hashlib.sha256(original.encode("utf-8")).hexdigest() changed = original + "Changed body\n" (root / "SKILL.md").write_text(changed, encoding="utf-8", newline="\n") git("add", LIBRARY.as_posix()) commit("Edit the skill") second = build_index(tmp_path, "HEAD") - updated = json.loads(second["common/skills/example.json"]) - assert updated["source_files"][entry] == hashlib.sha256(changed.encode("utf-8")).hexdigest() - assert updated["source_files"][entry] != record["source_files"][entry] + updated = json.loads(second["inputs.json"])["example"] + assert updated["files"]["SKILL.md"] == hashlib.sha256(changed.encode("utf-8")).hexdigest() + assert updated["files"]["SKILL.md"] != record["files"]["SKILL.md"] def test_package_index_and_checkout_skill_have_distinct_read_boundaries(): @@ -250,7 +261,7 @@ def test_package_index_and_checkout_skill_have_distinct_read_boundaries(): identity = "inferrsi.common.skills.claim_contract" bundle = reader.read(identity) assert "record.json" in bundle.content - assert "data/skill_library/workflows/claim-contract/SKILL.md" in bundle.content + assert "data/skill_library/workflows/governance/claim-contract/SKILL.md" in bundle.content assert bundle.dependencies assert bundle.metadata["kind"] == "knowledge" # The sidecar now declares an origin, so the package record stops falling @@ -260,7 +271,7 @@ def test_package_index_and_checkout_skill_have_distinct_read_boundaries(): assert reader.resolve("claim-contract") == "checkout:claim-contract" assert reader.read("claim-contract").metadata["kind"] == "skill" collected = reader.collect(repo="inferrsi", kind="knowledge", limit=100, material=True) - assert len(collected["items"]) == 80 + assert len(collected["items"]) == 88 assert not collected["collisions"] and not collected["omissions"] assert all(item["authority"] == "package" for item in collected["items"]) assert all("record.json" in item["resources"] for item in collected["items"]) @@ -290,9 +301,9 @@ def test_index_cli_search_show_context_and_execution_boundary(capsys): def test_hardware_filters_include_only_matching_and_shared_skills(): - for backend, count in (("ascend", 75), ("cuda", 64), ("cpu", 59)): + for backend, count in (("ascend", 83), ("cuda", 72), ("cpu", 67)): records = search_knowledge(repo="inferrsi", backend=backend, limit=100) assert len(records) == count assert all(record["backend"] in (backend, "common") for record in records) assert not (PROJECT / "skills").exists() - assert len(list(ROOT.rglob("SKILL.md"))) == 143 + assert len(list(ROOT.rglob("SKILL.md"))) == 151 diff --git a/tests/test_skill_library.py b/tests/test_skill_library.py index bb139cb32..966370938 100644 --- a/tests/test_skill_library.py +++ b/tests/test_skill_library.py @@ -111,7 +111,7 @@ def test_english_documents_are_packaged_local_and_repository_owned(): assert {record["repo"] for record in records} == set(REPOSITORIES) for record in records: assert set(record["skill_documents"]) == {"en"} - expected = LIBRARY / "repositories" / record["repo"].replace("_", "-") + expected = LIBRARY / "systems/engines" / record["repo"].replace("_", "-") # skill_documents holds the logical "data/..." key; K02 moved the physical # corpus under the Knowledge module without changing that spelling. document = (PACKAGE / "knowledge" / record["skill_documents"]["en"]).resolve() @@ -123,7 +123,7 @@ def test_english_documents_are_packaged_local_and_repository_owned(): def test_portable_library_retains_english_resources_licenses_and_resolving_local_links(): engine_nodes = 0 for repo in REPOSITORIES: - owner = LIBRARY / "repositories" / repo.replace("_", "-") + owner = LIBRARY / "systems/engines" / repo.replace("_", "-") assert (owner / "SKILL.md").is_file() license_text = (owner / "LICENSE").read_text(encoding="utf-8") assert ("MIT License" if repo == "strata" else "Apache License") in license_text @@ -133,12 +133,9 @@ def test_portable_library_retains_english_resources_licenses_and_resolving_local assert len(subskills) == 3 engine_nodes += 1 + len(subskills) assert engine_nodes == 20 - router = LIBRARY / "workflows/inference-engine-research" + router = LIBRARY / "workflows/research/inference-engine-research" assert (router / "SKILL.md").is_file() - documents = list(LIBRARY.rglob("*.md")) + [ - PACKAGE / "knowledge/data/repo_packs" / repo / "knowledge/README.md" - for repo in REPOSITORIES - ] + documents = list(LIBRARY.rglob("*.md")) for document in documents: for target in re.findall(r"\]\(([^)]+)\)", document.read_text(encoding="utf-8")): link = urlsplit(target) @@ -147,24 +144,41 @@ def test_portable_library_retains_english_resources_licenses_and_resolving_local destination = (document.parent / unquote(link.path)).resolve() assert destination.is_relative_to(PACKAGE.resolve()), (document, target) assert destination.exists(), (document, target) - assert (LIBRARY / "repositories/strata/scripts/summarize_trials.py").is_file() + assert (LIBRARY / "systems/engines/strata/scripts/summarize_trials.py").is_file() + + +def test_task_router_cards_are_readable_and_point_to_registered_skills(): + from inferrsi.knowledge.skill_catalog import skill_resource + + registry = json.loads((LIBRARY / "skill-index.json").read_text(encoding="utf-8"))["skills"] + router = "inference-task-router" + root = LIBRARY / registry[router]["path"] + instructions = (root / "SKILL.md").read_text(encoding="utf-8") + tasks = {"day0-model", "performance", "debug", "accuracy", "version-align", + "derived-engine", "independent-engine"} + for task in tasks: + path = f"references/routes/{task}.json" + assert f"]({path})" in instructions + card = json.loads(skill_resource(LIBRARY, router, path).read_text(encoding="utf-8")) + assert card["task"] == task and card["required_inputs"] and card["acceptance_evidence"] + assert card["candidate_skill_ids"] and set(card["candidate_skill_ids"]) <= set(registry) -def test_library_root_exposes_only_owner_categories_and_required_entrypoints(): +def test_library_root_exposes_task_routing_and_supporting_categories(): assert {path.name for path in LIBRARY.iterdir() if path.is_file()} == { "README.md", "skill-index.json", } assert {path.name for path in LIBRARY.iterdir() if path.is_dir()} == { - "hardware", "frameworks", "repositories", "methods", "workflows", + "tasks", "systems", "playbooks", "workflows", "casebooks", "records", "config", "_compiled", } - assert len(list(LIBRARY.rglob("SKILL.md"))) == 143 + assert len(list(LIBRARY.rglob("SKILL.md"))) == 151 assert not any(path.name == "zh-CN" for path in LIBRARY.rglob("*") if path.is_dir()) - references = LIBRARY / "workflows/inference-engine-research/references" + references = LIBRARY / "workflows/research/inference-engine-research/references" assert (references / "import.md").is_file() manifest = json.loads((references / "source-manifest.json").read_text(encoding="utf-8")) assert len(manifest["sources"]) == 5 - router = LIBRARY / "workflows/inference-engine-research" - strata = LIBRARY / "repositories/strata" + router = LIBRARY / "workflows/research/inference-engine-research" + strata = LIBRARY / "systems/engines/strata" assert (router / "references/overview.md").is_file() assert (strata / "references/design-space-analysis.md").is_file() @@ -182,7 +196,9 @@ def test_bundled_trial_helper_excludes_failed_configs_and_rejects_mixed_context( ) csv.write_text(header + rows, encoding="utf-8") command = [ - sys.executable, str(LIBRARY / "repositories/strata/scripts/summarize_trials.py"), str(csv), + sys.executable, + str(LIBRARY / "systems/engines/strata/scripts/summarize_trials.py"), + str(csv), ] result = subprocess.run(command, capture_output=True, text=True, check=True) summary = json.loads(result.stdout) diff --git a/tests/test_source_snapshot.py b/tests/test_source_snapshot.py index 619d810e0..e6116df4b 100644 --- a/tests/test_source_snapshot.py +++ b/tests/test_source_snapshot.py @@ -1,15 +1,15 @@ import copy import hashlib +import inspect import json import subprocess +from fnmatch import fnmatchcase from pathlib import Path import pytest from inferrsi.knowledge import _source_snapshot as snapshot -from inferrsi.knowledge import catalog -from inferrsi.knowledge._source_snapshot import source_entry -from inferrsi.knowledge.catalog import search_knowledge +from inferrsi.knowledge.catalog import INPUTS, recommendations, search_knowledge, skill_views from tools import build_skill_index as index_builder from tools.build_skill_index import DESTINATION @@ -22,32 +22,89 @@ LIBRARY = index_builder.PINNED_LIBRARY -def record_files(): - return sorted(p for p in GENERATED.glob("*/skills/*.json")) - - -def test_packaged_records_store_schema_two_and_resolve_to_schema_one(): - manifest = json.loads((GENERATED / snapshot.MANIFEST).read_text(encoding="utf-8")) - records = {r["id"]: r for r in search_knowledge(repo="inferrsi", limit=100)} - assert len(record_files()) == len(records) == 80 - for path in record_files(): - stored = json.loads(path.read_text(encoding="utf-8")) - assert stored["schema_version"] == 2 and "sources" not in stored - assert snapshot.REGISTRY not in stored["source_files"] - public = records[stored["id"]] - assert public == snapshot.resolve(stored, manifest) - assert public["schema_version"] == 1 - assert public["sources"][0]["commit"] == manifest["source_revision"] - assert public["source_files"][snapshot.REGISTRY] == manifest["registry"]["sha256"] - assert set(public["source_files"]) == set(public["sources"][0]["paths"]) - assert list(public)[list(public).index("origin") + 1] == "sources" - - -def test_compact_round_trip_preserves_every_public_field_and_order(): +def test_public_projection_implementation_is_a_protected_knowledge_path(): + policy = json.loads((PROJECT / ".github/knowledge-model-policy.json").read_bytes()) + for builder in (skill_views, recommendations): + path = Path(inspect.getfile(builder)).resolve().relative_to(PROJECT).as_posix() + assert any(fnmatchcase(path, pattern) for pattern in policy["protected_paths"]), path + + +def test_single_owner_reads_survive_an_unrelated_pending_metadata_edit(tmp_path, monkeypatch): + from inferrsi.knowledge import skill_catalog + from inferrsi.knowledge.catalog import get_knowledge, knowledge_records + from inferrsi.knowledge.knowledge_find import Corpus, find, load_documents + from inferrsi.knowledge.knowledge_operations import KnowledgeReader + + library = GENERATED.parent + owner = "ascend-model-adaptation" + metadata = skill_catalog.skill_directory(library, owner) / "SKILL.md" + edited = tmp_path / "SKILL.md" + original = metadata.read_text(encoding="utf-8") + assert '"tags": [' in original + modified = original.replace('"tags": [', '"tags": ["pending-edit",', 1) + edited.write_text(modified, encoding="utf-8") + original_open = Path.open + + def open_with_edit(path, *args, **kwargs): + return original_open(edited if path == metadata else path, *args, **kwargs) + + monkeypatch.setattr(Path, "open", open_with_edit) + identity = "flashinfer.cuda.llm.paged_decode_dispatch" + expected = json.loads((library / "records" / f"{identity}.json").read_bytes()) + for ref in (identity, f"package:{identity}"): + bundle = KnowledgeReader().read(ref) + assert json.loads(bundle.content["record.json"]) == expected + assert bundle.metadata["attribution"] == expected["sources"] + assert get_knowledge(identity) == expected + with pytest.raises(ValueError, match="Stale skill metadata for ascend-model-adaptation"): + KnowledgeReader().read("package:inferrsi.ascend.skills.ascend_model_adaptation") + with pytest.raises(ValueError, match="Stale skill metadata for ascend-model-adaptation"): + knowledge_records() + + def no_yaml(): + raise skill_catalog.MissingSkillsExtra("no PyYAML") + + monkeypatch.setattr(skill_catalog, "_yaml", no_yaml) + corpus = Corpus(load_documents()) + result = find("FlashInfer paged decode dispatch", corpus, limit=5) + record = next(row for row in result["items"] if row["id"] == identity) + assert record["read_args"][:2] == ["read", f"package:{identity}"] + assert "unreadable" not in record + result = find("weight-loading-optimization", corpus, limit=5) + skill = next(row for row in result["items"] if row["id"] == "weight-loading-optimization") + assert skill["read_args"][:2] == [ + "read", "package:inferrsi.common.skills.weight_loading_optimization", + ] + assert "--resource" in skill["read_args"] and "--revision" in skill["read_args"] + + +def test_compiled_inputs_expand_to_public_records(): + root = GENERATED.parent + inputs = json.loads((GENERATED / INPUTS).read_bytes()) + records = list(skill_views(root)) + manifest = json.loads((GENERATED / snapshot.MANIFEST).read_bytes()) + assert {p.name for p in GENERATED.iterdir()} == {INPUTS, snapshot.MANIFEST} + assert len(records) == len(inputs) == 88 + public = search_knowledge(repo="inferrsi", limit=100) + assert {r["id"] for r in records} == {r["id"] for r in public} + for record in records: + assert set(inputs[record["title"]]) == {"files", "header"} + assert record["schema_version"] == 1 + assert record["sources"][0]["commit"] == manifest["source_revision"] + assert record["source_files"][snapshot.REGISTRY] == manifest["registry"]["sha256"] + assert list(record["source_files"]) == record["sources"][0]["paths"] + + +def test_historical_schema_two_round_trip_preserves_public_field_order(): public = search_knowledge(repo="inferrsi", limit=1)[0] - manifest = json.loads((GENERATED / snapshot.MANIFEST).read_text(encoding="utf-8")) + manifest = json.loads((GENERATED / snapshot.MANIFEST).read_bytes()) stored = snapshot.compact(public) + assert stored["schema_version"] == snapshot.STORED_SCHEMA == 2 + assert "sources" not in stored and snapshot.REGISTRY not in stored["source_files"] assert json.dumps(snapshot.resolve(stored, manifest)) == json.dumps(public) + stored["source_files"].pop(snapshot.source_entry(public["skill_documents"]["original"])) + with pytest.raises(ValueError, match="lacks its entry hash"): + snapshot.resolve(stored, manifest) @pytest.mark.parametrize("mutation", [ @@ -56,7 +113,6 @@ def test_compact_round_trip_preserves_every_public_field_and_order(): ]) def test_invalid_snapshot_never_resolves(mutation): manifest = json.loads((GENERATED / snapshot.MANIFEST).read_text(encoding="utf-8")) - stored = json.loads(record_files()[0].read_text(encoding="utf-8")) bad = copy.deepcopy(manifest) if mutation == "extra": bad["unexpected"] = True @@ -75,49 +131,32 @@ def test_invalid_snapshot_never_resolves(mutation): else: bad = [] with pytest.raises(ValueError): - snapshot.resolve(stored, bad) + snapshot._snapshot(bad) @pytest.mark.parametrize( - "mutation", ["v1", "sources", "registry_hash", "missing_entry", "no_origin"]) -def test_invalid_stored_record_never_resolves(mutation): - manifest = json.loads((GENERATED / snapshot.MANIFEST).read_text(encoding="utf-8")) - stored = json.loads(record_files()[0].read_text(encoding="utf-8")) - if mutation == "v1": - stored["schema_version"] = 1 - elif mutation == "sources": - stored["sources"] = [] - elif mutation == "registry_hash": - stored["source_files"][snapshot.REGISTRY] = "0" * 64 + "mutation", + ["missing_owner", "extra_owner", "missing_entry", "bad_header", "bad_hash", "escape"], +) +def test_invalid_compiled_owner_is_rejected(tmp_path, monkeypatch, mutation): + root = _seed(tmp_path, monkeypatch) / LIBRARY + inputs = root / "_compiled" / INPUTS + data = json.loads(inputs.read_bytes()) + if mutation == "missing_owner": + del data["alpha"] + elif mutation == "extra_owner": + data["other"] = data["alpha"] elif mutation == "missing_entry": - stored["source_files"].pop(source_entry(stored["skill_documents"]["original"])) + del data["alpha"]["files"]["SKILL.md"] + elif mutation == "bad_hash": + data["alpha"]["files"]["SKILL.md"] = "invalid" + elif mutation == "escape": + data["alpha"]["files"]["../outside"] = "f" * 64 else: - del stored["origin"] + data["alpha"]["header"]["name"] = "other" + inputs.write_text(json.dumps(data), encoding="utf-8") with pytest.raises(ValueError): - snapshot.resolve(stored, manifest) - - -def test_snapshot_pack_reads_only_the_root_manifest_and_stored_records(tmp_path, monkeypatch): - pack = tmp_path / "knowledge/data/repo_packs/inferrsi/knowledge" - source = record_files()[0] - target = pack / source.relative_to(GENERATED) - target.parent.mkdir(parents=True) - target.write_bytes(source.read_bytes()) - (pack / snapshot.MANIFEST).write_bytes((GENERATED / snapshot.MANIFEST).read_bytes()) - monkeypatch.setattr(catalog, "files", lambda _: tmp_path) - assert len(catalog.knowledge_records()) == 1 - # Only the root manifest is the snapshot; a nested copy is an invalid record. - nested = target.parent / snapshot.MANIFEST - nested.write_bytes((pack / snapshot.MANIFEST).read_bytes()) - with pytest.raises(ValueError, match="not schema 2"): - catalog.knowledge_records() - nested.unlink() - # Schema-1 and schema-2 records never mix in one snapshot pack. - manifest = json.loads((pack / snapshot.MANIFEST).read_text(encoding="utf-8")) - public = snapshot.resolve(json.loads(target.read_text(encoding="utf-8")), manifest) - target.write_text(json.dumps(public), encoding="utf-8") - with pytest.raises(ValueError, match="not schema 2"): - catalog.knowledge_records() + list(skill_views(root)) # Fixture identity and settings; hosted runners have no global Git identity. @@ -158,10 +197,6 @@ def _generate(root, monkeypatch): _git(root, "commit", "-q", "-m", "Regenerate projections") -def _changed(root, base): - return set(_git(root, "diff", "--name-only", base, "HEAD").splitlines()) - - def _seed(tmp_path, monkeypatch): root = tmp_path / "repo" root.mkdir() @@ -190,9 +225,11 @@ def test_single_owner_edit_rewrites_only_its_record_and_the_snapshot(tmp_path, m _generate(root, monkeypatch) generated = DESTINATION.as_posix() assert _git(root, "diff", "--name-only", edit, "HEAD").splitlines() == sorted([ - f"{generated}/{snapshot.MANIFEST}", f"{generated}/common/skills/alpha.json", + f"{generated}/{snapshot.MANIFEST}", f"{generated}/{INPUTS}", ]) - assert f"{generated}/common/skills/bravo.json" not in _changed(root, base) + def bravo(ref): + return json.loads(_git(root, "show", f"{ref}:{generated}/{INPUTS}"))["bravo"] + assert bravo(base) == bravo("HEAD") # Regeneration from the same source is byte-identical. assert index_builder.main(["--source-revision", edit, "--check"]) == 0 @@ -211,22 +248,20 @@ def test_parallel_owner_edits_integrate_in_either_order(tmp_path, monkeypatch, o # Only the shared snapshot may collide (the fixture has no coverage view); # take either side, then regenerate it from the merged source. conflicted = set(_git(root, "diff", "--name-only", "--diff-filter=U").splitlines()) - assert conflicted == {f"{generated}/{snapshot.MANIFEST}"} - _git(root, "checkout", "--ours", "--", f"{generated}/{snapshot.MANIFEST}") + expected = {f"{generated}/{snapshot.MANIFEST}", f"{generated}/{INPUTS}"} + assert conflicted and conflicted <= expected + _git(root, "checkout", "--ours", "--", *sorted(conflicted)) _git(root, "add", "-A") _git(root, "commit", "-q", "--no-edit") _generate(root, monkeypatch) for owner in ("alpha", "bravo"): body = (root / LIBRARY / "methods" / owner / "SKILL.md").read_text(encoding="utf-8") assert f"{owner} contribution" in body - stored = json.loads((root / generated / f"common/skills/{owner}.json").read_text( - encoding="utf-8")) - entry = f"{LIBRARY.as_posix()}/methods/{owner}/SKILL.md" - assert stored["source_files"][entry] == hashlib.sha256(body.encode("utf-8")).hexdigest() - # The untouched owner's stored record never changes. - charlie = f"{generated}/common/skills/charlie.json" - assert _git(root, "rev-parse", f"{seeded}:{charlie}") == _git( - root, "rev-parse", f"HEAD:{charlie}") + stored = json.loads((root / generated / INPUTS).read_bytes())[owner] + assert stored["files"]["SKILL.md"] == hashlib.sha256(body.encode("utf-8")).hexdigest() + def charlie(ref): + return json.loads(_git(root, "show", f"{ref}:{generated}/{INPUTS}"))["charlie"] + assert charlie(seeded) == charlie("HEAD") def test_same_owner_divergent_edits_remain_an_explicit_conflict(tmp_path, monkeypatch): @@ -246,12 +281,14 @@ def test_check_detects_tampered_or_missing_snapshot(tmp_path, monkeypatch): revision = _git(root, "rev-parse", "HEAD~1") manifest = root / DESTINATION / snapshot.MANIFEST original = manifest.read_text(encoding="utf-8") + manifest.write_bytes(original.replace("\n", "\r\n").encode("utf-8")) + assert index_builder.main(["--source-revision", revision, "--check"]) == 1 manifest.write_text(original.replace(revision, "b" * 40), encoding="utf-8") assert index_builder.main(["--source-revision", revision, "--check"]) == 1 manifest.unlink() assert index_builder.main(["--source-revision", revision, "--check"]) == 1 - manifest.write_text(original, encoding="utf-8") - record = root / DESTINATION / "common/skills/alpha.json" + manifest.write_text(original, encoding="utf-8", newline="\n") + record = root / DESTINATION / INPUTS stored = json.loads(record.read_text(encoding="utf-8")) stored["sources"] = [] record.write_text(json.dumps(stored), encoding="utf-8") diff --git a/tests/test_splash_repo_to_skill.py b/tests/test_splash_repo_to_skill.py index ee198b014..161b5e5ff 100644 --- a/tests/test_splash_repo_to_skill.py +++ b/tests/test_splash_repo_to_skill.py @@ -9,8 +9,10 @@ import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -LIBRARY = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories" +LIBRARY = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines" FIXTURES = PROJECT / "tests/fixtures/repo_to_skill" GRAPHS = ("splash",) @@ -47,7 +49,7 @@ def test_arex_runtime_graph_links_frontmatter_and_source_boundaries(): assert selected.exists(), (path, target) assert not list(root.rglob("__pycache__")) assert not list(root.rglob("*.pyc")) - metadata = json.loads((root / "metadata.json").read_text(encoding="utf-8")) + metadata = skill_metadata(root.parent, root.name) assert metadata["status"] == "candidate" assert metadata["construction"]["native_import"] == "not-run" assert "BLOCKED_REQUIRED_BACKEND" in " ".join(metadata["limitations"]) diff --git a/tests/test_vllm_ascend_rc_skill.py b/tests/test_vllm_ascend_rc_skill.py index 5df28b190..f9faf3441 100644 --- a/tests/test_vllm_ascend_rc_skill.py +++ b/tests/test_vllm_ascend_rc_skill.py @@ -102,7 +102,7 @@ def test_history_screen_rejects_incomplete_history(monkeypatch, tmp_path): def test_runtime_reader_keeps_full_history_outside_the_bundle(): bundle = KnowledgeReader().read("inferrsi.ascend.skills.vllm_ascend") assert all("history-inventory.json" not in key for key in bundle.content) - prefix = "data/skill_library/repositories/vllm-ascend/" + prefix = "data/skill_library/systems/engines/vllm-ascend/" for relative in ("references/history-reviewed.json", "scripts/screen_history.py"): assert bundle.content[prefix + relative] == skill_resource( LIBRARY, NAME, relative diff --git a/tests/test_vllm_ascend_source_anchors.py b/tests/test_vllm_ascend_source_anchors.py index 138d5be90..0338138d9 100644 --- a/tests/test_vllm_ascend_source_anchors.py +++ b/tests/test_vllm_ascend_source_anchors.py @@ -140,6 +140,12 @@ def test_requires_unique_label_and_expected_source_path(source_repo, change): check.verify_anchors(repo, text, pin) +def test_tool_reads_the_current_owner_document(): + pin = check.metadata_pin(check.DEFAULT_METADATA) + assert len(pin) == 40 + assert pin in check.DEFAULT_DOC.read_text(encoding="utf-8") + + def test_metadata_requires_full_source_commit(tmp_path): path = tmp_path / "metadata.json" metadata(path, "main") diff --git a/tests/test_vllm_metal_repo_to_skill.py b/tests/test_vllm_metal_repo_to_skill.py index 3d899fe2a..11add6a6e 100644 --- a/tests/test_vllm_metal_repo_to_skill.py +++ b/tests/test_vllm_metal_repo_to_skill.py @@ -15,8 +15,10 @@ import pytest import yaml +from inferrsi.knowledge.skill_catalog import skill_metadata + PROJECT = Path(__file__).resolve().parents[1] -ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/repositories/vllm-metal" +ROOT = PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-metal" CASES = PROJECT / "tests/fixtures/repo_to_skill/vllm-metal/test-cases" @@ -44,7 +46,7 @@ def test_vllm_metal_graph_is_contained_candidate_with_exact_source_scope(): continue resolved = (path.parent / unquote(link.path)).resolve() assert resolved.is_relative_to(ROOT.resolve()) and resolved.is_file(), (path, target) - metadata = json.loads((ROOT / "metadata.json").read_text()) + metadata = skill_metadata(ROOT.parent, ROOT.name) assert metadata["status"] == "candidate" assert metadata["empirical_validation"] == "partial_cpu_only" assert metadata["construction"]["native_import"] == "not-run" diff --git a/tools/build_skill_coverage.py b/tools/build_skill_coverage.py index e06a8d087..259f39c29 100644 --- a/tools/build_skill_coverage.py +++ b/tools/build_skill_coverage.py @@ -30,7 +30,7 @@ LINK = re.compile(r"\(([^()\s]*/skill_library/[^()\s]+/SKILL\.md)\)") BOLD = re.compile(r"\*\*(.+?)\*\*") -# A sidecar states this for a dimension that genuinely does not apply, so the +# An owner states this for a dimension that genuinely does not apply, so the # view can tell it apart from a field nobody has analyzed yet. NOT_APPLICABLE = "not-applicable" @@ -231,9 +231,8 @@ def collect(root): metadata = skill_metadata(root / LIBRARY, name) relative = registry[name]["path"] directory = root / LIBRARY / relative - has_sidecar = (directory / "metadata.json").is_file() inputs.append(directory / "SKILL.md") - if has_sidecar: + if (directory / "metadata.json").is_file(): # Legacy custom checkout. inputs.append(directory / "metadata.json") declared = metadata.get("applicability") advice = metadata.get("recommendations") @@ -246,7 +245,6 @@ def collect(root): "status": metadata.get("status", "unknown"), "validation": metadata.get("empirical_validation", "unknown"), "readsets": len(metadata.get("provenance_readsets") or []), - "sidecar": has_sidecar, "conditions": conditions(metadata), "typed_conditions": sorted( key for key in (declared if isinstance(declared, dict) else {}) @@ -309,8 +307,6 @@ def gaps(row): found.append("conflicting-SearchSpace-layer") if not row["rm"]: found.append("no-Reference-Matrix-row") - if not row["sidecar"]: - found.append("no-sidecar") if not row["conditions"]["scope"] and not row["conditions"]["dimensions"]: found.append("no-applicability") if not row["eval"]: @@ -344,13 +340,12 @@ def render(rows, unregistered, documents, dead, digest): guidance = [row for row in rows if guidance_only(row)] multi = [row for row in layered if len(row["layers"]) > 1] sourced = [row for row in rows if row["rm"]] - sided = [row for row in rows if row["sidecar"]] planned = [row for row in rows if row["eval"]] out = [ "# Skill coverage view", "", - "Generated from the registry, the skill sidecars and the two maintained", + "Generated from the registry, SKILL.md frontmatter and the two maintained", "matrices. **Do not edit by hand**; regenerate after any registry, skill,", "matrix or frozen-plan change.", "", @@ -373,11 +368,11 @@ def render(rows, unregistered, documents, dead, digest): "| Skill id, canonical path, backend | " "[`skill-index.json`](../src/inferrsi/knowledge/data/skill_library/skill-index.json) |", "| Declared task, status, validation state, provenance readsets | " - "the skill's `SKILL.md` frontmatter and `metadata.json` sidecar |", + "the skill's `SKILL.md` frontmatter |", "| Declared applicability conditions, workflow input/output/failure/refusal | " - "`applicability` in the same `metadata.json` sidecar; descriptive, not evidential |", + "`applicability` in the same frontmatter; descriptive, not evidential |", "| Resource layer, lifecycle stage | [SearchSpace Matrix](searchspace-matrix.md) |", - "| Reasoned resource-layer exclusion | `searchspace_not_applicable` in the sidecar |", + "| Reasoned resource-layer exclusion | `searchspace_not_applicable` in the frontmatter |", "| Source identity and inspected scope | [Reference Matrix](reference-matrix.md) |", "| Evaluation selection | frozen `evidence//plan/protocol-*.json` |", "| Delivery PR and issue | not derived here; see [issue #91]" @@ -403,7 +398,6 @@ def render(rows, unregistered, documents, dead, digest): f"{sum(layer_state(row) == 'conflict' for row in rows)} |", f"| Recorded on two or more resource layers | {len(multi)} |", f"| Recorded in the Reference Matrix | {len(sourced)} |", - f"| With a `metadata.json` sidecar | {len(sided)} |", f"| Selected by a frozen evaluation plan | {len(planned)} |", f"| `SKILL.md` documents in the library | {documents} |", f"| Unregistered documents (excluded from every count above) | " @@ -415,13 +409,15 @@ def render(rows, unregistered, documents, dead, digest): "", ] - for key, title in ( - ("hardware", "Hardware and platform"), - ("frameworks", "Frameworks"), - ("repositories", "Repositories"), - ("methods", "Methods"), - ("workflows", "Workflows"), - ): + category_titles = { + "tasks": "Task routing", + "systems": "Systems and implementations", + "playbooks": "Engineering playbooks", + "workflows": "Cross-stage workflows", + "casebooks": "Casebooks and prior outcomes", + } + for key in sorted(counts): + title = category_titles.get(key, key.title()) group = [row for row in rows if row["category"] == key] if not group: continue @@ -501,7 +497,6 @@ def render(rows, unregistered, documents, dead, digest): "- Duplicate ids or paths: none; `skill_catalog` rejects them before this " "view is built.", f"- Skill links in the two matrices that do not resolve: {len(dead)}.", - f"- Registered skills without a `metadata.json` sidecar: {len(rows) - len(sided)}.", "", ] if dead: @@ -511,7 +506,7 @@ def render(rows, unregistered, documents, dead, digest): out += ["## Recorded gaps", "", "| Gap | Skills |", "|---|---|"] for token in ("no-SearchSpace-layer", "conflicting-SearchSpace-layer", - "no-Reference-Matrix-row", "no-sidecar", + "no-Reference-Matrix-row", "no-applicability", "no-eval"): names = [row["id"] for row in rows if token in gaps(row)] listed = (", ".join(names) if len(names) <= 20 @@ -520,7 +515,7 @@ def render(rows, unregistered, documents, dead, digest): out += [ "", "These are inventory gaps, not efficacy results. Missing layer mappings stay", - "unknown unless the sidecar records a reasoned N/A. N/A is not coverage;", + "unknown unless the owner records a reasoned N/A. N/A is not coverage;", "a conflicting matrix mapping remains visible and is excluded from coverage", "counts until reconciled. Supplementary guidance stays distinct from numbered", "layer coverage. A candidate may legitimately remain untested.", @@ -555,7 +550,7 @@ def render(rows, unregistered, documents, dead, digest): "## Drift and maintenance", "", "The generator is the only writer of this file. After changing the registry,", - "a skill body, a sidecar, either matrix or a frozen plan, regenerate and", + "a skill document, either matrix or a frozen plan, regenerate and", "commit both files together. `--check` rebuilds in memory and diffs against", "the committed file, exiting non-zero on any difference, so CI can detect a", "view that drifted from its sources. Only the affected rows need review;", diff --git a/tools/build_skill_index.py b/tools/build_skill_index.py index 9962d4b88..e096381ab 100644 --- a/tools/build_skill_index.py +++ b/tools/build_skill_index.py @@ -1,4 +1,4 @@ -"""Generate knowledge abstracts for the categorized, packaged skill library. +"""Compile offline catalog inputs from the registered skill library. Run from a development installation with the skills extra. The explicit source revision must contain the indexed inputs; Git handles checkout newline filters. @@ -14,19 +14,22 @@ from pathlib import Path from inferrsi.knowledge import _source_snapshot -from inferrsi.knowledge.skill_catalog import skill_metadata, skill_records +from inferrsi.knowledge.catalog import COMPILED, INPUTS +from inferrsi.knowledge.skill_catalog import _frontmatter_parts, skill_records + +if __package__: + from .git_read import GIT_READ, GitVersionError, check_git_version +else: # Direct invocation: python tools/build_skill_index.py + from git_read import GIT_READ, GitVersionError, check_git_version PROJECT = Path(__file__).resolve().parents[1] -# Where the inputs and generated records live today. +# The runtime reads these build products alongside the canonical owners. LIBRARY = Path("src/inferrsi/knowledge/data/skill_library") -DESTINATION = Path("src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge") +DESTINATION = LIBRARY / COMPILED # How those inputs are named at the pinned source revision. After the explicit # K02 re-pin this agrees with LIBRARY, the current checkout path. PINNED_LIBRARY = Path(_source_snapshot.LIBRARY) REPOSITORY = _source_snapshot.REPOSITORY -# Match the comparison tool: replacement refs and lazy promisor fetches must not -# change the bytes hashed for a pinned revision. -GIT_READ = ("git", "--no-replace-objects", "--no-lazy-fetch") def _git(project: Path, *args: str) -> bytes: @@ -56,6 +59,7 @@ def _source_hashes(project: Path, revision: str, paths: set[str]) -> dict[str, s def build_index(project: Path, source_revision: str) -> dict[str, str]: + check_git_version("Skill index generation") revision = _git(project, "rev-parse", "--verify", f"{source_revision}^{{commit}}") revision = revision.decode().strip() root = project / LIBRARY @@ -100,130 +104,23 @@ def source_inputs(root: Path) -> set[str]: } -DEFAULT_RECOMMENDATIONS = [ - "Read the pinned SKILL.md and its prerequisites before choosing this procedure.", - "Read the packaged references and owner-local cases alongside the procedure.", -] - - -def recommendations(metadata: dict) -> list[str]: - """Owner-supplied procedure advice, or generic navigation when absent. - - A declared empty list means the owner states there is no procedure-specific - advice; it is not the same as an absent field and does not fall back. - """ - if "recommendations" in metadata: - return list(metadata["recommendations"]) - return list(DEFAULT_RECOMMENDATIONS) - - def render_index(root: Path, revision: str, source_files: dict[str, str]) -> dict[str, str]: - """Render the projection without requiring historical Git objects.""" - outputs = {} - links = [] - entries = _registry(root) - snapshot = _source_snapshot.manifest(revision, source_files[_source_snapshot.REGISTRY]) - for selected in skill_records(root): - name = selected["name"] - metadata = skill_metadata(root, name) - backend = entries[name]["backend"] - directory = f"{PINNED_LIBRARY.as_posix()}/{entries[name]['path']}" - hashes = { - path: digest for path, digest in sorted(source_files.items()) - if path.startswith(f"{directory}/") - or path == f"{PINNED_LIBRARY.as_posix()}/skill-index.json" + """Compile only the YAML header and Git fingerprints required by the base install.""" + owners = {} + for name, registration in _registry(root).items(): + directory = registration["path"] + prefix = f"{PINNED_LIBRARY.as_posix()}/{directory}/" + with (root / directory / "SKILL.md").open("rb") as stream: + header, _ = _frontmatter_parts(stream) + owners[name] = { + "header": header, + "files": {p[len(prefix):]: digest for p, digest in sorted(source_files.items()) + if p.startswith(prefix)}, } - paths = list(hashes) - entry = f"{directory}/SKILL.md" - state = metadata.get("status", "unspecified") - record = { - "schema_version": 1, - "id": f"inferrsi.{backend}.skills.{name.replace('-', '_')}", - "title": name, - "repo": "inferrsi", - "tier": "contribution", - "backend": backend, - "component": "skills", - "scenario": "skills.discovery", - "status": "imported", - "summary": metadata["description"], - "tags": metadata.get("tags", []), - "origin": metadata.get("origin", ["unknown"]), - "sources": [{ - "type": "git", - "url": f"{REPOSITORY}/blob/{revision}/{entry}", - "commit": revision, - "paths": paths, - "content_sha256": hashes[entry], - }], - "index_type": "packaged-skill", - "skill_documents": { - "original": f"data/skill_library/{entries[name]['path']}/SKILL.md", - }, - "checkout_ref": f"checkout:{name}", - "source_files": hashes, - "skill_metadata": metadata, - "applicability": metadata.get("applicability", {}), - "recommendations": recommendations(metadata), - "limitations": [ - "This generated abstract points to the complete packaged skill; " - "external evidence and upstream repositories remain source links.", - "The imported status describes this index, not the original skill; " - f"its declared status is {state}.", - "Reading or indexing does not authorize execution, validate efficacy, " - "promote a candidate or grant a license.", - "The backend categorizes the procedure's hardware dependency; it does not " - "certify compatibility, and skills.discovery is a discovery scenario.", - *metadata.get("limitations", []), - ], - "verification": {"locally_verified": False, "evidence_level": "guidance"}, - } - relative = f"{backend}/skills/{name}.json" - stored = _source_snapshot.compact(record) - # The catalog must read back exactly the public record rendered here. - if _source_snapshot.resolve(stored, snapshot) != record: - raise ValueError(f"Stored skill record does not round-trip: {name}") - outputs[relative] = json.dumps(stored, ensure_ascii=False, indent=2) + "\n" - links.append(f"| [{name}]({relative}) | {backend} | {state} |") - outputs[_source_snapshot.MANIFEST] = json.dumps(snapshot, indent=2) + "\n" - outputs["README.md"] = "\n".join([ - "# InferRSI skill discovery index", "", - f"Generated abstracts for all {len(entries)} registered skills in `data/skill_library/`.", - "Their complete procedures, references and cases ship in the same package.", - "`skill-index.json` maps stable skill IDs to categorized directories. The", - "reference and searchspace matrices", - "remain the maintained knowledge views. Do not edit generated files by hand.", "", - "Index status `imported` and `locally_verified: false` describe these discovery", - "records. Original status, provenance, license declarations and limitations", - "are preserved under `skill_metadata`; absent fields remain absent. Unknown", - "licenses remain unknown. Indexing grants no execution permission or validation.", "", - "These are knowledge abstracts (`--kind knowledge`), with namespaced package", - "IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA", - "procedures use their corresponding backend; shared methods use `common`.", "", - f"`{_source_snapshot.MANIFEST}` holds the shared source snapshot: the full source", - "revision and the registry hash. Stored records (schema 2) keep only their own", - "fields and resource hashes; the catalog restores the schema-1 `sources` entry and", - "registry hash from the snapshot, so editing one skill regenerates only its record", - "and the snapshot.", "", - "```bash", - "inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50", - "inferrsi collect --repo-type inferrsi --kind knowledge --limit 50 --material", - "```", "", - "After committing changes to skill bodies, resources or the registry, regenerate from a", - "development installation with the skills extra. The source revision must", - "contain the current input paths and bytes (Git checkout filters are respected).", - "`source_files` hashes identify the canonical Git bytes, not a validation result.", "", - "```bash", - "python tools/build_skill_index.py --source-revision ", - "python tools/build_skill_index.py --source-revision --check", - "```", "", - "`--check` detects missing, stale and unexpected generated records without writing;", - f"use the `source_revision` recorded in `{_source_snapshot.MANIFEST}`.", - "References and cases ship with their owning skill, not as duplicate knowledge abstracts.", - "", - "| Skill | Backend | Original declared status |", "| --- | --- | --- |", *links, "", - ]) - return outputs + snapshot = _source_snapshot.manifest(revision, source_files[_source_snapshot.REGISTRY]) + return {name: json.dumps(obj, ensure_ascii=False, indent=2) + "\n" for name, obj in ( + (_source_snapshot.MANIFEST, snapshot), (INPUTS, owners), + )} def main(argv: list[str] | None = None) -> int: @@ -231,7 +128,10 @@ def main(argv: list[str] | None = None) -> int: parser.add_argument("--source-revision", required=True) parser.add_argument("--check", action="store_true") args = parser.parse_args(argv) - outputs = build_index(PROJECT, args.source_revision) + try: + outputs = build_index(PROJECT, args.source_revision) + except GitVersionError as error: + parser.error(str(error)) destination = PROJECT / DESTINATION existing = { path.relative_to(destination).as_posix() @@ -241,13 +141,13 @@ def main(argv: list[str] | None = None) -> int: changed = [ path for path, content in outputs.items() if not (destination / path).is_file() - or (destination / path).read_text(encoding="utf-8") != content + or (destination / path).read_bytes() != content.encode("utf-8") ] if args.check: if stale or changed: print("Skill index drift: " + ", ".join(sorted(stale | set(changed)))) return 1 - print(f"Skill index is current: {len(outputs) - 2} metadata records") + print("Compiled catalog inputs match the registered skills") return 0 if stale: raise ValueError("Remove obsolete generated files explicitly: " + ", ".join(sorted(stale))) @@ -255,8 +155,7 @@ def main(argv: list[str] | None = None) -> int: path = destination / relative path.parent.mkdir(parents=True, exist_ok=True) path.write_text(outputs[relative], encoding="utf-8", newline="\n") - # Outputs are the records plus README.md and the shared snapshot. - print(f"Generated {len(outputs) - 2} skill metadata records") + print("Wrote compiled catalog inputs") return 0 diff --git a/tools/check_skill_layout_migration.py b/tools/check_skill_layout_migration.py new file mode 100644 index 000000000..3d7015e0e --- /dev/null +++ b/tools/check_skill_layout_migration.py @@ -0,0 +1,79 @@ +"""Validate two materialized source trees. This filesystem-only check does not invoke Git.""" + +import argparse +import csv +import sys +from pathlib import Path + +PROJECT = Path(__file__).resolve().parents[1] +LIBRARY = Path("src/inferrsi/knowledge/data/skill_library") +sys.path.insert(0, str(PROJECT / "src")) + +from inferrsi.knowledge.knowledge_operations import KnowledgeReader, RevisionConflict # noqa: E402 +from inferrsi.knowledge.skill_catalog import skill_registrations # noqa: E402 + + +def verify_snapshot(checkout, rows, side): + root = (checkout / LIBRARY).resolve() + reader = KnowledgeReader(skill_root=root) + locations = { + name: ("skill", directory.relative_to(root).as_posix()) + for name, (directory, _) in skill_registrations(root).items() + } + locations.update({ + name: ("case", path.relative_to(root).as_posix()) for name, path in reader.cases().items() + }) + expected = { + row["stable_id"]: (row["kind"], row[f"{side}_path"]) + for row in rows if row[f"{side}_path"] + } + if locations != expected: + mismatches = [name for name in locations.keys() | expected.keys() + if locations.get(name) != expected.get(name)] + raise ValueError(f"{side}: identity or path disagreement: {sorted(mismatches)}") + + rejections = 0 + for row in rows: + path = row[f"{side}_path"] + if not path: + continue + ref = "checkout:" + row["stable_id"] + revision = row[f"{side}_checkout_revision"] + if revision == "unavailable_at_cutover": + try: + reader.read(ref) + except ValueError as error: + if "Document source requires URL and version or snapshot identity" in str(error): + continue + raise + raise ValueError(f"{side}: unexpectedly readable: {ref} at {path}") + reader.read(ref, revision=revision) + old = row["old_checkout_revision"] + if side == "new" and old.startswith("sha256:") and old != revision: + try: + reader.read(ref, revision=old) + except RevisionConflict: + rejections += 1 + else: + raise ValueError(f"Old revision silently resolved: {ref} at {row['old_path']}") + return len(expected), rejections + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--before", type=Path, required=True, + help="pre-layout checkout corresponding to the migration manifest") + parser.add_argument("--after", type=Path, default=PROJECT, help="cutover source checkout") + args = parser.parse_args() + with (PROJECT / "docs/skill-layout-migration.csv").open(encoding="utf-8", newline="") as stream: + rows = list(csv.DictReader(stream)) + if len({row["stable_id"] for row in rows}) != len(rows): + raise ValueError("Duplicate identities in migration manifest") + before, _ = verify_snapshot(args.before, rows, "old") + after, rejections = verify_snapshot(args.after, rows, "new") + print(f"Verified {before} previous and {after} cutover entries; " + f"{rejections} stale revisions rejected.") + + +if __name__ == "__main__": + main() diff --git a/tools/check_vllm_ascend_source_anchors.py b/tools/check_vllm_ascend_source_anchors.py index 698d70b90..eeb9a4189 100644 --- a/tools/check_vllm_ascend_source_anchors.py +++ b/tools/check_vllm_ascend_source_anchors.py @@ -10,10 +10,14 @@ from collections.abc import Mapping from pathlib import Path -OWNER = (Path(__file__).resolve().parents[1] - / "src/inferrsi/knowledge/data/skill_library/repositories/vllm-ascend") +PROJECT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(PROJECT / "src")) + +from inferrsi.knowledge.catalog import _skill_search_source # noqa: E402 + +OWNER = (PROJECT / "src/inferrsi/knowledge/data/skill_library/systems/engines/vllm-ascend") DEFAULT_DOC = OWNER / "references/communication-config.md" -DEFAULT_METADATA = OWNER / "metadata.json" +DEFAULT_METADATA = OWNER / "SKILL.md" SOURCE_URL = "https://github.com/vllm-project/vllm-ascend" MAX_ANCHOR_LINES = 5 EXPECTED = { @@ -23,7 +27,13 @@ def metadata_pin(metadata: Path, source_url: str = SOURCE_URL) -> str: - sources = json.loads(metadata.read_text(encoding="utf-8"))["sources"] + if metadata.name == "SKILL.md": + _, record = _skill_search_source(metadata.read_text(encoding="utf-8")) + if record is None: + raise ValueError("Skill document must declare its source metadata") + else: + record = json.loads(metadata.read_text(encoding="utf-8")) + sources = record["sources"] matches = [source for source in sources if source.get("type") == "git" and source.get("url") == source_url] if len(matches) != 1: diff --git a/tools/compare_find_layouts.py b/tools/compare_find_layouts.py new file mode 100644 index 000000000..90d93ec46 --- /dev/null +++ b/tools/compare_find_layouts.py @@ -0,0 +1,238 @@ +"""Compare Find quality and fresh-process CLI time on two clean, pinned worktrees. + +Run with --base DIR --candidate DIR --candidate-sha FULL_SHA --output OUTSIDE.json. +Each process uses the same Python executable and only its arm's src on PYTHONPATH. +The seven CLI workloads warm the filesystem, not a resident Python process. +""" + +import argparse +import json +import math +import os +import platform +import random +import statistics +import subprocess +import sys +import time +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path + +if __package__: + from .git_read import GIT_READ, GitVersionError, check_git_version +else: + from git_read import GIT_READ, GitVersionError, check_git_version + +FIXTURES = Path(__file__).resolve().parents[1] / "tests" / "fixtures" / "find" + + +def validate_worktree(root, sha): + def git(*args): + return subprocess.check_output([*GIT_READ, *args], cwd=root).decode().strip() + + if Path(git("rev-parse", "--show-toplevel")).resolve() != root: + raise ValueError("Expected a worktree root") + if git("rev-parse", "HEAD") != sha: + raise ValueError(f"Worktree revision differs from {sha}") + if git("status", "--porcelain=v1", "--untracked-files=all", "--ignore-submodules=none"): + raise ValueError(f"Dirty worktree at {sha}") + + +def workload(protocol): + labels = json.loads((FIXTURES / "labelled-queries.json").read_bytes()) + queries = [(q["query"], q["expect"]) for group in labels for q in group["queries"]] + queries.extend((query, None) for query in protocol["previous_fixed_queries"]) + queries.extend((query, None) for query in protocol["tasks"].values()) + return queries + + +def signature(result): + return [[item["kind"], item["id"], bool(item.get("read_args"))] for item in result["items"]] + + +def inspect_arm(): + # Invoked in a separate interpreter rooted in the selected checkout. + from inferrsi.knowledge import knowledge_find + + if not Path(knowledge_find.__file__).resolve().is_relative_to(Path.cwd() / "src"): + raise ValueError("Find did not import from the pinned worktree") + corpus = knowledge_find.load_corpus() + queries = json.load(sys.stdin) + print(json.dumps([signature(knowledge_find.find(q, corpus, limit=5)) for q in queries])) + + +def ratios(timings): + base, candidate = zip(*timings) + return statistics.median(candidate) / statistics.median(base) + + +def summarize_time(samples): + timings = [pair for pairs in samples.values() for pair in pairs] + # Keep the task mix fixed. Resample adjacent AB/BA blocks within each task. + rng = random.Random(516) + bootstrap = [] + for _ in range(2000): + drawn = [] + for pairs in samples.values(): + blocks = [pairs[i : i + 2] for i in range(0, len(pairs), 2)] + for block in rng.choices(blocks, k=len(blocks)): + drawn.extend(block) + bootstrap.append(ratios(drawn)) + bootstrap.sort() + arms = {} + for col, name in enumerate(("base", "candidate")): + times = sorted(pair[col] for pair in timings) + arms[name] = { + "median_s": statistics.median(times), + "p95_s": times[math.ceil(0.95 * len(times)) - 1], + } + return { + "arms": arms, + "pooled_median_ratio": ratios(timings), + "paired_block_bootstrap_95pct": [bootstrap[49], bootstrap[1949]], + } + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--base", type=Path, required=True) + parser.add_argument("--candidate", type=Path, required=True) + parser.add_argument("--candidate-sha", required=True) + parser.add_argument("--output", type=Path, required=True) + args = parser.parse_args() + try: + check_git_version("Find layout comparison") + except GitVersionError as error: + parser.error(str(error)) + protocol = json.loads((FIXTURES / "layout-queries.json").read_bytes()) + roots = {"base": args.base.resolve(), "candidate": args.candidate.resolve()} + revisions = {"base": protocol["base_sha"], "candidate": args.candidate_sha} + output = args.output.resolve() + for arm, root in roots.items(): + if output.is_relative_to(root): + parser.error("Write the report outside both measured worktrees") + validate_worktree(root, revisions[arm]) + envs = { + arm: dict( + os.environ, + PYTHONPATH=str(root / "src"), + PYTHONUTF8="1", + PYTHONDONTWRITEBYTECODE="1", + PYTHONNOUSERSITE="1", + PYTHONHASHSEED="0", + GIT_NO_REPLACE_OBJECTS="1", + GIT_NO_LAZY_FETCH="1", + ) + for arm, root in roots.items() + } + queries = workload(protocol) + + def rank_arm(arm): + return json.loads( + subprocess.check_output( + [sys.executable, str(Path(__file__).resolve()), "--inspect"], + cwd=roots[arm], + env=envs[arm], + input=json.dumps([q for q, _ in queries]).encode(), + ) + ) + + # Rank only; these runs produce no timing samples. CLI measurements run serially below. + with ThreadPoolExecutor(max_workers=2) as pool: + ranks = dict(zip(roots, pool.map(rank_arm, roots), strict=True)) + details = [ + {"query": q, "expected_kind": kind, "base": old, "candidate": new} + for (q, kind), old, new in zip(queries, ranks["base"], ranks["candidate"], strict=True) + ] + matched = sum( + [item[:2] for item in row["base"]] == [item[:2] for item in row["candidate"]] + for row in details + ) + lost_reads = 0 + for row in details: + readable = {(kind, key) for kind, key, has_read in row["candidate"] if has_read} + lost_reads += sum( + has_read and (kind, key) not in readable for kind, key, has_read in row["base"] + ) + quality = { + "matching_ordered_top5": matched, + "queries": len(queries), + "lost_readable_hits": lost_reads, + } + for arm in roots: + positive = [row for row in details if row["expected_kind"] not in (None, "none")] + gaps = [row for row in details if row["expected_kind"] == "none"] + quality[arm] = { + "hit_at_3_by_expected_kind": sum( + any(item[0] == row["expected_kind"] for item in row[arm][:3]) for row in positive + ), + "labelled_positive": len(positive), + "empty_results_on_labelled_gap": sum(not row[arm] for row in gaps), + "labelled_gap": len(gaps), + } + + def cli(arm, query): + start = time.perf_counter() + raw = subprocess.check_output( + [ + sys.executable, + "-m", + "inferrsi", + "find", + query, + "--limit", + "5", + "--no-external", + "--json", + ], + cwd=roots[arm], + env=envs[arm], + ) + elapsed = time.perf_counter() - start + return signature(json.loads(raw)), elapsed + + print(json.dumps(quality), file=sys.stderr, flush=True) + for arm in roots: + cli(arm, "warmup inference benchmark") + tasks = protocol["tasks"] + samples = {task: [] for task in tasks} + for repeat in range(protocol["pairs_per_task"]): + for i, (task, query) in enumerate(tasks.items()): + order = ("base", "candidate") if (repeat + i) % 2 == 0 else ("candidate", "base") + times = {} + for arm in order: + ids, times[arm] = cli(arm, query) + if ids != ranks[arm][queries.index((query, None))]: + raise ValueError(f"Unstable CLI results for {task} ({arm})") + samples[task].append([times["base"], times["candidate"]]) + print(f"Timed repeat {repeat + 1} complete", file=sys.stderr, flush=True) + for arm, root in roots.items(): + validate_worktree(root, revisions[arm]) + latency = summarize_time(samples) + gate = protocol["gate"] + passed = ( + matched == len(queries) + and lost_reads <= gate["max_lost_readable_hits"] + and latency["pooled_median_ratio"] <= gate["max_candidate_to_base_median_cli_latency_ratio"] + ) + report = { + "revisions": revisions, + "platform": platform.system(), + "python": platform.python_version(), + "gate": gate, + "passed": passed, + "quality": quality, + "latency": latency, + "samples_s_base_candidate": samples, + "rankings": details, + } + output.write_text(json.dumps(report, indent=2, ensure_ascii=False) + "\n", encoding="utf-8") + print(json.dumps({k: report[k] for k in ("passed", "quality", "latency")}, indent=2)) + return 0 if passed else 1 + + +if __name__ == "__main__": + if sys.argv[1:] == ["--inspect"]: + inspect_arm() + else: + sys.exit(main()) diff --git a/tools/git_read.py b/tools/git_read.py new file mode 100644 index 000000000..c9f0ca580 --- /dev/null +++ b/tools/git_read.py @@ -0,0 +1,29 @@ +"""Requirements for tools that inspect Git objects without replacements or lazy fetches.""" + +import re +import subprocess + +GIT_READ = ("git", "--no-replace-objects", "--no-lazy-fetch") + + +class GitVersionError(ValueError): + pass + + +def require_git_version(version: str, purpose: str = "Snapshot preparation") -> None: + parsed = re.match(r"git version (\d+)\.(\d+)(?:\.|$)", version.strip()) + if parsed is None or tuple(map(int, parsed.groups())) < (2, 45): + raise GitVersionError( + f"{purpose} requires Git >= 2.45 for --no-lazy-fetch; found {version.strip()}. " + "Upgrade Git and retry." + ) + + +def check_git_version(purpose: str = "Snapshot preparation") -> None: + try: + version = subprocess.check_output(["git", "--version"], text=True) + except (OSError, subprocess.CalledProcessError) as error: + raise GitVersionError( + f"{purpose} requires Git >= 2.45. Install Git on PATH and retry." + ) from error + require_git_version(version, purpose) diff --git a/tools/knowledge_declaration_paths.py b/tools/knowledge_declaration_paths.py index 1efca5271..ab793297c 100644 --- a/tools/knowledge_declaration_paths.py +++ b/tools/knowledge_declaration_paths.py @@ -15,7 +15,8 @@ PROJECT = Path(__file__).resolve().parents[1] POLICY = ".github/knowledge-model-policy.json" -GENERATED = ("src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/", "docs/skill-coverage.md") +GENERATED = ("src/inferrsi/knowledge/data/skill_library/_compiled/", + "src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/", "docs/skill-coverage.md") GIT = ("git", "--no-replace-objects", "--no-lazy-fetch") diff --git a/tools/skill_eval_pr.py b/tools/skill_eval_pr.py index a28463817..a3320a6b6 100644 --- a/tools/skill_eval_pr.py +++ b/tools/skill_eval_pr.py @@ -36,7 +36,7 @@ MAX_ARCHIVE = 100 * 1024 * 1024 SHA = r"[0-9a-f]{40}" LIBRARY_PATH = "src/inferrsi/knowledge/data/skill_library" -PROJECTION_PATH = "src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge" +PROJECTION_PATH = LIBRARY_PATH + "/_compiled" def canonical(value): @@ -143,8 +143,13 @@ def projection_check(repo, identity, changes): library = git(repo, "ls-tree", identity["head"], "--", LIBRARY_PATH).decode().split() if len(library) != 4 or library[:2] != ["040000", "tree"]: return unavailable + # The library overview sits outside every registered skill directory. + # Nested READMEs are skill resources and must still invalidate a stale pin. changed = git(repo, "diff", "--no-ext-diff", "--no-textconv", "--no-renames", - "--name-only", pin, identity["head"], "--", LIBRARY_PATH) + "--name-only", pin, identity["head"], "--", LIBRARY_PATH, + ":(exclude)" + PROJECTION_PATH, ":(exclude)" + LIBRARY_PATH + "/records", + ":(exclude)" + LIBRARY_PATH + "/config", + ":(exclude,literal)" + LIBRARY_PATH + "/README.md") if changed.strip(): return {"status": "unavailable", "reason": "Declared projection source differs from current canonical library bytes"} @@ -220,8 +225,15 @@ def prepare(repo, mapping_path, identity, output, *, provider=None, planning_blo or entry["path"] == p for p in item["paths"]): entry["coverage"] = "unsupported" continue - config = {"mode": "update" if blob_exists(repo, identity["base"], entrypoint) - else "addition", "skill": item["skill"], "skill_root": mapping["skill_root"], + existing = blob_exists(repo, identity["base"], entrypoint) + registry_path = mapping["skill_root"] + "/skill-index.json" + if not existing and blob_exists(repo, identity["base"], registry_path): + # Snapshot loading resolves each arm by stable ID in its own registry. + registry = json.loads(projection_blob(repo, identity["base"], registry_path), + object_pairs_hook=projection_object) + existing = item["skill"] in registry["skills"] + config = {"mode": "update" if existing else "addition", + "skill": item["skill"], "skill_root": mapping["skill_root"], "cases": str((mapping_path.parent / item["cases"]).resolve()), "snapshots": {"baseline": {"git_revision": identity["base"]}, "candidate": {"git_revision": identity["head"]}}, @@ -634,8 +646,8 @@ def public_preparation_reason(reason): def path_groups(paths): groups = {"Canonical library paths": [], "Generated projection paths": [], "Other paths": []} for path in paths: - key = ("Canonical library paths" if matches_path(path, (LIBRARY_PATH,)) else - "Generated projection paths" if matches_path(path, (PROJECTION_PATH,)) else + key = ("Generated projection paths" if matches_path(path, (PROJECTION_PATH,)) else + "Canonical library paths" if matches_path(path, (LIBRARY_PATH,)) else "Other paths") groups[key].append(path) return groups diff --git a/tools/skill_eval_pr_cases.json b/tools/skill_eval_pr_cases.json index 7d7a69d43..c5c7c4468 100644 --- a/tools/skill_eval_pr_cases.json +++ b/tools/skill_eval_pr_cases.json @@ -14,25 +14,25 @@ "skills": [ { "skill": "inference-benchmark-design", - "entrypoint": "methods/inference-benchmark-design/SKILL.md", + "entrypoint": "playbooks/measurement/inference-benchmark-design/SKILL.md", "paths": [ - "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design" + "src/inferrsi/knowledge/data/skill_library/playbooks/measurement/inference-benchmark-design" ], "cases": "../tests/fixtures/skill-comparison/cases.json" }, { "skill": "audit-fixed-workload-claim", - "entrypoint": "methods/audit-fixed-workload-claim/SKILL.md", + "entrypoint": "playbooks/measurement/audit-fixed-workload-claim/SKILL.md", "paths": [ - "src/inferrsi/knowledge/data/skill_library/methods/audit-fixed-workload-claim" + "src/inferrsi/knowledge/data/skill_library/playbooks/measurement/audit-fixed-workload-claim" ], "cases": "../tests/fixtures/skill-comparison/cases.json" }, { "skill": "optimization-search-policy", - "entrypoint": "methods/optimization-search-policy/SKILL.md", + "entrypoint": "playbooks/planning/optimization-search-policy/SKILL.md", "paths": [ - "src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy" + "src/inferrsi/knowledge/data/skill_library/playbooks/planning/optimization-search-policy" ], "cases": "../tests/fixtures/skill-comparison/pr133-diagnostic-case05.json", "adapter": "kv-admission-diagnostic-v1"