Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/coordination.md
Original file line number Diff line number Diff line change
Expand Up @@ -1617,6 +1617,7 @@ items a-runner/b stay with the async/GDN `runner.cpp` owners.

| `CLAIM-CPU-GCC12-WERROR-PORTABILITY` | `QUANT-GGUF-KEEPQ-LOADER` + `KV-OFFLOAD` (maintenance bugfix; lifecycle states unchanged) | Codex (GPT-5) | isolated worktree `.claude/worktrees/codex-cpu-fix`; CPU Release build `build-cpu`, `-DVLLM_CPP_CUDA=OFF -DVLLM_CPP_SERVER=ON`; no GPU/toolkit/model download | branch `codex/cpu-fix`, base `upstream/main` `557988ed` | Remove the two known GCC 12 production-library blockers under mandatory `-Werror`. RED baseline fails at `qwen3_5_gguf_weights.cpp:49` on deprecated volatile compound assignment; the second production failure is `fs_io.cpp:66` (`-Wrestrict`). Owns ONLY the two expression-level fixes in those production files plus this claim and the same-checkpoint STATUS/BENCHMARKS/roadmap/matrix/spec/ledger/state records. No API, layout, algorithm, default, CUDA TU, model forward, fixture, golden, README, or unrelated test-file change. | `ACTIVE` | 2026-07-31 — production `vllm` library and both focused targets build clean on GCC 12 without suppressions; focused tests pass 2/2. Full all-target build is PARTIAL at 42% on unrelated test-only `-Wrestrict` diagnostics in `test_deepseek_v2_paged_engine.cpp` (actively MLA-owned) and `test_glm4_moe_lite_paged_engine.cpp`; full CTest cannot run until those separate claims repair their test TUs. |

**2026-08-01 — `CLAIM-ENV-DOC-READ-SCANNER` COMPLETED (`ROAD-V1-A6`, no area-row lifecycle transition).** Codex (GPT-5), isolated worktree `/home/mudler/.cache/sdd/localai-org-maint-bot-vllm.cpp/codex-cpu-maintenance-20260801`, branch `codex/cpu-maintenance-20260801`, base `upstream/main` `1448e981`. CPU-only maintenance bugfix: the environment-doc checker now inventories actual environment-reader calls instead of every quoted `VT_*`/`VLLM_*` literal. RED-first mutations cover comments/unrelated strings and the seven local wrappers; the rebased inventory contains 209 reads. Owned only the checker/test and A6 governance records; no production C/C++, API, model, kernel, build, README, env classification, lifecycle, or support claim changed.
## Handoff queue

| Priority | Row/block | Dependency | Next handoff | State |
Expand Down
1 change: 1 addition & 0 deletions .agents/parity-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -886,3 +886,4 @@ Columns:
| 2026-07-31 (`SERVE-C-ABI` W0 contract spike; `CLAIM-SERVE-C-ABI-SPIKE`; CPU-only records/docs) | Accepted `.agents/specs/c-api-library.md` for the already-shipped original C packaging layer: complete scope, vLLM semantic chain/deviation, ABI v10/19-symbol baseline, ownership/error/version/dispatch rules, exact code/test anchors, gates, dependencies, risks, and W1-W5 follow-ons. Also fixes the verified stale public `VLLM_ABI_VERSION 9` labels in README/USAGE to the source-of-truth v10 and adds the missing v10 usage-table entry. No production/test/CMake source changed. | Pinned vLLM `555967922` has no C ABI; behavior beneath the adapter remains owned by its vLLM-derived engine rows. The flat ABI is the recorded llama.cpp-style packaging deviation and may translate, never reimplement, policy. | **CPU/records gate only; benchmark NOT APPLICABLE.** Focused C11/C++/dlopen/export gate passed 3/3 after explicitly building `vllm_shared`; five record checkers pass. `check-agent-record` reports the base tree's same six missing closing-commit objects (`444ea9d7`, `7a3f04b2`, `164453a2`), none in this row/diff. Row stays `ANCHOR-BACKFILL` because all-symbol dlsym coverage (chat symbols currently omitted), historical-layout compatibility, allocation-failure no-throw proof, lifetime sanitizer stress, and a standalone real-model C consumer remain W1-W5. |
| 2026-07-31 (`CLAIM-CPU-GCC12-WERROR-PORTABILITY`; maintenance, rows `QUANT-GGUF-KEEPQ-LOADER` + `KV-OFFLOAD`; lifecycle unchanged) | Removes two GCC 12 production-library `-Werror` blockers without suppressions: the GGUF prefault keeps the same one-byte-per-page volatile XOR but uses simple assignment, and the KV filesystem tier builds the identical `.<pid>.<counter>.tmp` suffix with append operations inside its thread-local initializer. No API, algorithm, default, CUDA, fixture, or golden change. | Behavior remains grounded in the accepted loader and KV-persistence leaf specs: llama.cpp mmap prefault intent and vLLM `tiering/fs/io.py` unique temporary-file publication. This is compiler portability, not a parity-surface change. | RED: GCC 12 failed first at `qwen3_5_gguf_weights.cpp:49` (`-Wvolatile`), then at `fs_io.cpp:66` (`-Wrestrict`). GREEN: production `vllm` and focused test targets build clean; focused CTest 2/2 (`test_gguf_keep_quant`, `test_kv_offload_fs`). Full all-target build is PARTIAL at 42% on unrelated test-only GCC 12 `-Wrestrict` diagnostics in `test_deepseek_v2_paged_engine.cpp` and `test_glm4_moe_lite_paged_engine.cpp`; no full-CTest claim. Benchmark NOT APPLICABLE. |
| 2026-08-01 (`SERVE-CLI-CHAT` W0 contract spike; `CLAIM-SERVE-CLI-CHAT-SPIKE`; CPU-only records/spec) | Accepts `.agents/specs/cli-chat-complete.md`, corrects the inventory from “no direct commands” to the actual pinned `chat`/`complete` surface, and decomposes a dual-mode port: exact remote OpenAI HTTP/SSE commands plus preservation of the existing in-process invocation as a compatibility alias. No production, test, CMake, model, kernel, fixture, or generated file changes. | Pinned vLLM `5559679229`: command registration `vllm/entrypoints/cli/main.py:17-37,73-98`; model/auth resolution and stream shaping `vllm/entrypoints/cli/openai.py:30-100`; chat `:155-234`; complete `:237-312`. The local compatibility baseline is `examples/cli/main.cpp:1-207`. | CPU record/doc gates only; benchmark `NOT APPLICABLE`, `benchmark_binding=false`. Implementation remains absent and the row moves `INVENTORIED` -> `SPIKE`. W1-W5 name parse, transport, complete, chat, and packaging gates, including fake-server request/SSE transcript parity, Release `-Werror`, ASan+UBSan, and TSan. |
| 2026-08-01 (`CLAIM-ENV-DOC-READ-SCANNER`; `ROAD-V1-A6`; maintenance, lifecycle unchanged) | Makes `check-env-doc.py` enforce its stated contract: inventory literal `VT_*` / `VLLM_*` names passed to real environment-reader calls, not comments, diagnostics, or unrelated string constants. The scanner strips C/C++ comments and recognizes `getenv`/`std::getenv` plus the seven shipped local wrappers. | House documentation-governance logic; vLLM has no matching checker. Runtime parity is unaffected because no production source changes. | RED: the new negative mutation reported all three planted non-reads; the wrapper mutation then proved a getenv-only implementation incomplete. GREEN: 10/10 focused tests, rebased checker reports all 209 production reads covered. Benchmark NOT APPLICABLE. |
13 changes: 12 additions & 1 deletion .agents/roadmap_v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -533,7 +533,18 @@ diagnostics recorded in `docs/BENCHMARKS.md`.
| A3 | `test_ops_fused_chain` FMA-contraction fix | ✅ merged bf48edb (`-ffp-contract=off` host-wide) |
| A4 | De-Python the build: vendor Triton AOT artifacts per-arch (`triton_aot_vendored/<arch>/` + MANIFEST; `VLLM_CPP_TRITON_REGEN` = maintainer-only Python) | ✅ **DONE** (54367cc..a432461; reproducibility hardening `09f1d23`) — `sm_121a` now has 48 generated C/H files + MANIFEST, including both bf16 `chunk_o` shapes; normal builds remain Python-free. Regen is explicit-target (`cuda:121:32`), line-info-disabled and byte-reproducible across source paths; the pure checker makes source/contract/artifact drift fatal and mutation-tests missing/extra/changed artifacts. A4 remains closed; fresh current-main CUDA/runtime/performance validation belongs to the two ACTIVE `CLAIM-PR3` kernel rows (evidence: porting-inventory §9). |
| `SERVE-E2E-NIGHTLY` (formerly A5) | e2e suites per gates.md (server conformance nightly on dgx etc.) | ☐ next; leaf spike required |
| A6 | **User-facing surface closure** (audit `CLAIM-DOCS-COVERAGE`, 2026-07-24): make what SHIPPED reachable and truthfully documented. The audit found 2 class-D overstatements, 5 stale doc lines, 4 shipped-but-unreachable features and 153 undocumented env vars | 🚧 Tier 1 (falsehood repair) DONE in the audit commit. **Tier 2 DONE 2026-07-24 (`CLAIM-DOCS-T2-FIXES`)**, 3 of its 4 items: the D1 guard (a per-connector worker-half capability predicate; the disk connector is now REFUSED at engine construction on every device instead of silently shortcutting prefill for KV nothing writes — **its worker half remains unimplemented and is NOT claimed**), server `--tool-call-parser`/`--reasoning-parser` (40 tool names / 7 reasoning parsers now reachable over HTTP; defaults reproduce the old hardcode exactly), and server `--kv-transfer-config` mirroring vLLM's own CLI + JSON shape. Also landed from Tier 3/4: `docs/KV-OFFLOAD.md`, the README flag rows, the three missing Gemma model rows, and the `SERVE-C-ABI` 17→19 record repair. **Comprehensive refresh DONE 2026-07-25 (`CLAIM-DOCS-REFRESH`)**: **Tier 4** `docs/ENVIRONMENT.md` (26 user-facing env vars documented of 153 swept, kernel-internal tail on `scripts/env-doc-allowlist.txt`) + its CI checker `scripts/check-env-doc.py` (mutation-tested, fails on a new undocumented+unallowlisted production env var); the **Tier 3** README refresh to current git history (spec-decode `--speculative-config` now documented as shipped, the "not user-visible" class-B falsehood fixed, C ABI `VLLM_ABI_VERSION 6`, CLI spec-config flag); new usage guide `docs/SPECULATIVE-DECODING.md`. STILL OPEN: `vllm-cli` structured-output/sampling flags (Tier 2 item 4, code, not docs). **C-ABI contract W0 2026-07-31 (`CLAIM-SERVE-C-ABI-SPIKE`):** accepted [c-api-library.md](specs/c-api-library.md), corrected public ABI v9 labels to v10, and named W1-W5 compatibility/no-throw/lifetime gates; `SERVE-C-ABI` honestly remains `ANCHOR-BACKFILL`. **CLI chat/complete W0 2026-08-01 (`CLAIM-SERVE-CLI-CHAT-SPIKE`):** [cli-chat-complete.md](specs/cli-chat-complete.md) corrects the inventory (the pin does ship both remote commands), selects exact OpenAI HTTP/SSE parity plus a compatibility-preserved local invocation, and decomposes CPU-only W1-W5 implementation leaves; `SERVE-CLI-CHAT` is `SPIKE`, no command code has landed. Full table: [specs/docs-coverage-audit.md](specs/docs-coverage-audit.md) |
| A6 | **User-facing surface closure** (audit `CLAIM-DOCS-COVERAGE`, 2026-07-24): make what SHIPPED reachable and truthfully documented. The audit found 2 class-D overstatements, 5 stale doc lines, 4 shipped-but-unreachable features and 153 undocumented env vars | 🚧 Tier 1 (falsehood repair) DONE in the audit commit. **Tier 2 DONE 2026-07-24 (`CLAIM-DOCS-T2-FIXES`)**, 3 of its 4 items: the D1 guard (a per-connector worker-half capability predicate; the disk connector is now REFUSED at engine construction on every device instead of silently shortcutting prefill for KV nothing writes — **its worker half remains unimplemented and is NOT claimed**), server `--tool-call-parser`/`--reasoning-parser` (40 tool names / 7 reasoning parsers now reachable over HTTP; defaults reproduce the old hardcode exactly), and server `--kv-transfer-config` mirroring vLLM's own CLI + JSON shape. Also landed from Tier 3/4: `docs/KV-OFFLOAD.md`, the README flag rows, the three missing Gemma model rows, and the `SERVE-C-ABI` 17→19 record repair. **Comprehensive refresh DONE 2026-07-25 (`CLAIM-DOCS-REFRESH`)**: **Tier 4** `docs/ENVIRONMENT.md` (26 user-facing env vars documented of 153 swept, kernel-internal tail on `scripts/env-doc-allowlist.txt`) + its CI checker `scripts/check-env-doc.py` (mutation-tested, fails on a new undocumented+unallowlisted production env var); the **Tier 3** README refresh to current git history (spec-decode `--speculative-config` now documented as shipped, the "not user-visible" class-B falsehood fixed, C ABI `VLLM_ABI_VERSION 6`, CLI spec-config flag); new usage guide `docs/SPECULATIVE-DECODING.md`. **Scanner hardened 2026-08-01 (`CLAIM-ENV-DOC-READ-SCANNER`)**: it now counts actual environment-reader calls, not quoted names in comments or unrelated literals, while retaining all 196 shipped reads. STILL OPEN: `vllm-cli` structured-output/sampling flags (Tier 2 item 4, code, not docs). **C-ABI contract W0 2026-07-31 (`CLAIM-SERVE-C-ABI-SPIKE`):** accepted [c-api-library.md](specs/c-api-library.md), corrected public ABI v9 labels to v10, and named W1-W5 compatibility/no-throw/lifetime gates; `SERVE-C-ABI` honestly remains `ANCHOR-BACKFILL`. Full table: [specs/docs-coverage-audit.md](specs/docs-coverage-audit.md) |

Current A6 continuation: **CLI chat/complete W0 2026-08-01
(`CLAIM-SERVE-CLI-CHAT-SPIKE`)** accepted
[cli-chat-complete.md](specs/cli-chat-complete.md), which corrects the inventory
(the pin ships both remote commands), selects exact OpenAI HTTP/SSE parity plus
a compatibility-preserved local invocation, and decomposes CPU-only W1-W5
implementation leaves. `SERVE-CLI-CHAT` remains `SPIKE`; no command code has
landed.

Rebase note: the scanner checkpoint originally covered 196 reads; the current
tree has 209, all covered by the same reader-call classification.

## B. Research tracks (complete)

Expand Down
13 changes: 13 additions & 0 deletions .agents/specs/docs-coverage-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,3 +303,16 @@ owns that row.
**Not checked:** runtime behavior. This audit read source and records; nothing
was built or executed, so "implemented" here means "present and reachable in
source", grounded on the matrices' own gate evidence for correctness claims.

## 6. Environment-read scanner maintenance (2026-08-01)

The completeness checker originally harvested every quoted `VT_*` / `VLLM_*`
literal. That contradicted its read-site contract: a quoted name in a comment,
diagnostic, or lookup table was counted as a production environment read. The
scanner now removes C/C++ comments and recognizes only calls through the
repository's environment-reader family (`getenv`, `std::getenv`, and the seven
local wrappers). Its mutation suite covers direct and multiline calls, all
wrappers, comments, block comments, and unrelated string literals. The shipped
inventory is 209 names after rebase, so no real read or classification was lost. This
is a CI-governance correction only; no user-visible capability, production
path, or benchmark axis changes.
2 changes: 2 additions & 0 deletions .agents/state.md
Original file line number Diff line number Diff line change
Expand Up @@ -34505,3 +34505,5 @@ The required Slack selection notification was attempted through the bundled
secret-safe sender to the only conventional target available, `#general`, but
Slack returned `channel_not_found`. No channel ID/name is configured and no
credential was inspected or exposed.

**Environment-doc read scanner corrected (2026-08-01, `CLAIM-ENV-DOC-READ-SCANNER`, `ROAD-V1-A6`).** Reproduced that the checker counted quoted env-looking text in comments and unrelated constants. A getenv-only repair passed the initial negative test but dropped 21 genuine wrapper-mediated reads, so the final design strips C/C++ comments and matches the full shipped reader-call family. RED-first mutations cover both failure modes; GREEN retains the full inventory, now 209 names after rebase. CPU/governance only, no lifecycle change, no production source, GPU, model, or benchmark work.
2 changes: 2 additions & 0 deletions docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -11230,3 +11230,5 @@ steady-state per-step / nsys decode tok/s (eager vs graphed) is OWED.

<!-- framework-routing (2026-08-02) -->
- Laguna-XS-2.1-NVFP4 decode (GB10, vs vLLM 42.46 tok/s): split-K + w13-fusion → 35.3 tok/s (83.2%). Residual is GPU-kernel work inside the graph (measured). PENDING: full KV/attention framework port. Qwen3DenseDecodeGraph ~4.3% e2e (directional; rigorous per-step tok/s OWED). Qwen3VL/DS4 routing = framework-conformance (no speed claim); CUDA re-gate OWED.

**Environment-doc scanner hardening (2026-08-01, `CLAIM-ENV-DOC-READ-SCANNER`) — NOT APPLICABLE.** This maintenance checkpoint changes only Python CI-governance logic and its mutation tests. No production C/C++ path, workload, correctness result, latency, throughput, or memory behavior changed. CPU reproduction: `python3 tests/scripts/test_check_env_doc.py && python3 scripts/check-env-doc.py`; the rebased inventory contains 209 covered reads.
2 changes: 2 additions & 0 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2119,3 +2119,5 @@ _(Laguna decode KV+attention → shared-framework port, bf16-KV slice (2026-08-0

<!-- decode/runtime framework-routing (2026-08-02) -->
- **Decode/runtime framework-routing** — `GATING`. Third "MUST route through" seam codified (AGENTS.md) + CI-enforced (`check-runner-routing-consistency.py`, green: 27 models, 3 allowlisted). Landed: Laguna split-K attention + w13-fusion (82.7→83.2% of vLLM decode); shared `Qwen3DenseDecodeGraph` (5 registrations, GB10 token-exact 16/16); Qwen3VL + DeepSeek-V4 routed to on-device logits + runner entry. Pending: Laguna gate-up fold onto the shared MLP seam (allowlisted) + its full KV/attention port onto `AttnBlock` (the kernel-level residual to vLLM parity); CUDA re-gates for Qwen3VL/DS4 owed.

**Environment-doc scanner hardening (2026-08-01).** `ROAD-V1-A6` keeps its prior lifecycle state. The CI inventory now counts actual environment-reader calls and ignores comments or unrelated quoted literals; all 209 reads in the rebased tree are covered. No runtime capability or support claim changed.
70 changes: 66 additions & 4 deletions scripts/check-env-doc.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,75 @@
ENV_DOC = ROOT / "docs/ENVIRONMENT.md"
ALLOWLIST = ROOT / "scripts/env-doc-allowlist.txt"

# A production env var is read as a quoted string literal, e.g. getenv("VT_FOO").
# Match the quoted name so a bare mention in a comment is not counted as a read.
_QUOTED = re.compile(r'"((?:VT_|VLLM_)[A-Z0-9_]+)"')
# Production env vars in this tree are read through getenv/std::getenv or these
# narrow local wrappers, all with a string-literal name. Restricting the match
# to reader calls keeps comments, diagnostics and lookup-table strings from
# silently expanding the inventory. Add new reader wrappers here with tests.
_ENV_READER = (
r"(?:std::)?getenv|GetEnvNonEmpty|EnvOn|EnvOnOr|EnvironmentValue|"
r"EnvironmentBool|EnvironmentEnabled|GdnTritonEnvOn"
)
_ENV_READ = re.compile(
rf'\b(?:{_ENV_READER})\s*\(\s*"((?:VT_|VLLM_)[A-Z0-9_]+)"'
)
# A name token, used to harvest the documented set from the markdown / allowlist.
_TOKEN = re.compile(r'\b((?:VT_|VLLM_)[A-Z0-9_]+)\b')


def _without_cpp_comments(text: str) -> str:
"""Remove C/C++ comments while preserving strings and line structure."""
out: list[str] = []
i = 0
state = "code"
while i < len(text):
char = text[i]
nxt = text[i + 1] if i + 1 < len(text) else ""
if state == "code":
if char == "/" and nxt == "/":
state = "line_comment"
out.extend(" ")
i += 2
continue
if char == "/" and nxt == "*":
state = "block_comment"
out.extend(" ")
i += 2
continue
if char == '"':
state = "string"
elif char == "'":
state = "char"
out.append(char)
i += 1
continue
if state == "line_comment":
if char == "\n":
state = "code"
out.append(char)
else:
out.append(" ")
i += 1
continue
if state == "block_comment":
if char == "*" and nxt == "/":
state = "code"
out.extend(" ")
i += 2
else:
out.append("\n" if char == "\n" else " ")
i += 1
continue
out.append(char)
if char == "\\" and nxt:
out.append(nxt)
i += 2
continue
if (state == "string" and char == '"') or (state == "char" and char == "'"):
state = "code"
i += 1
return "".join(out)


def scan_env_names(root: Path) -> set[str]:
"""Return every VT_/VLLM_ env name read as a string literal in src/+include/."""
names: set[str] = set()
Expand All @@ -42,7 +104,7 @@ def scan_env_names(root: Path) -> set[str]:
if path.suffix.lower() not in SCAN_SUFFIXES or not path.is_file():
continue
text = path.read_text(encoding="utf-8", errors="ignore")
names.update(_QUOTED.findall(text))
names.update(_ENV_READ.findall(_without_cpp_comments(text)))
return names


Expand Down
Loading
Loading