Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .agents/coordination.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,23 @@ without the selected contention proof for their entire run are discarded.

## Active claims

**README escaped-pipe cell-budget repair (`ROAD-V1-A6`, 2026-08-01,
`CLAIM-README-ESCAPED-PIPE`, LANDED + CPU-GATED).** Codex (GPT-5), isolated worktree
`/home/mudler/.cache/sdd/localai-org-maint-bot-vllm.cpp/codex-readme-escaped-pipe`,
branch `codex/readme-escaped-pipe`, base `upstream/main` `1448e981`. CPU-only
governance bugfix: base `1448e981` split table rows on escaped Markdown pipes,
allowing one oversized prose cell to evade the per-cell budget when `\|` divided
it into shorter fragments. The escaped-pipe-aware splitter and RED-first
mutation close that fail-open path. Owns only NEW
`.agents/specs/readme-escaped-pipe-cell-budget.md`, the parser and mutation
coverage in `scripts/check-readme-structure.py` and
`tests/scripts/test_check_readme_structure.py`, this claim, the minimal
`ROAD-V1-A6` checkpoint note, `docs/STATUS.md`, `docs/BENCHMARKS.md`,
`.agents/parity-ledger.md`, and append-only `.agents/state.md`. No runtime C/C++,
README content, lifecycle, support claim, model, build, GPU, download, or
benchmark change. Verification: focused suite 19/19, full Python script suite
96/96, every standalone repository checker, and Git whitespace checks.

**Interactive CLI chat/complete spike (`SERVE-CLI-CHAT`, 2026-08-01,
`CLAIM-SERVE-CLI-CHAT-SPIKE`).** Codex (GPT-5), isolated worktree
`/home/mudler/.cache/sdd/localai-org-maint-bot-vllm.cpp/codex-serve-cli-chat-spike`,
Expand Down
1 change: 1 addition & 0 deletions .agents/parity-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -886,3 +886,4 @@ Columns:
| 2026-07-31 (`SERVE-C-ABI` W0 contract spike; `CLAIM-SERVE-C-ABI-SPIKE`; CPU-only records/docs) | Accepted `.agents/specs/c-api-library.md` for the already-shipped original C packaging layer: complete scope, vLLM semantic chain/deviation, ABI v10/19-symbol baseline, ownership/error/version/dispatch rules, exact code/test anchors, gates, dependencies, risks, and W1-W5 follow-ons. Also fixes the verified stale public `VLLM_ABI_VERSION 9` labels in README/USAGE to the source-of-truth v10 and adds the missing v10 usage-table entry. No production/test/CMake source changed. | Pinned vLLM `555967922` has no C ABI; behavior beneath the adapter remains owned by its vLLM-derived engine rows. The flat ABI is the recorded llama.cpp-style packaging deviation and may translate, never reimplement, policy. | **CPU/records gate only; benchmark NOT APPLICABLE.** Focused C11/C++/dlopen/export gate passed 3/3 after explicitly building `vllm_shared`; five record checkers pass. `check-agent-record` reports the base tree's same six missing closing-commit objects (`444ea9d7`, `7a3f04b2`, `164453a2`), none in this row/diff. Row stays `ANCHOR-BACKFILL` because all-symbol dlsym coverage (chat symbols currently omitted), historical-layout compatibility, allocation-failure no-throw proof, lifetime sanitizer stress, and a standalone real-model C consumer remain W1-W5. |
| 2026-07-31 (`CLAIM-CPU-GCC12-WERROR-PORTABILITY`; maintenance, rows `QUANT-GGUF-KEEPQ-LOADER` + `KV-OFFLOAD`; lifecycle unchanged) | Removes two GCC 12 production-library `-Werror` blockers without suppressions: the GGUF prefault keeps the same one-byte-per-page volatile XOR but uses simple assignment, and the KV filesystem tier builds the identical `.<pid>.<counter>.tmp` suffix with append operations inside its thread-local initializer. No API, algorithm, default, CUDA, fixture, or golden change. | Behavior remains grounded in the accepted loader and KV-persistence leaf specs: llama.cpp mmap prefault intent and vLLM `tiering/fs/io.py` unique temporary-file publication. This is compiler portability, not a parity-surface change. | RED: GCC 12 failed first at `qwen3_5_gguf_weights.cpp:49` (`-Wvolatile`), then at `fs_io.cpp:66` (`-Wrestrict`). GREEN: production `vllm` and focused test targets build clean; focused CTest 2/2 (`test_gguf_keep_quant`, `test_kv_offload_fs`). Full all-target build is PARTIAL at 42% on unrelated test-only GCC 12 `-Wrestrict` diagnostics in `test_deepseek_v2_paged_engine.cpp` and `test_glm4_moe_lite_paged_engine.cpp`; no full-CTest claim. Benchmark NOT APPLICABLE. |
| 2026-08-01 (`SERVE-CLI-CHAT` W0 contract spike; `CLAIM-SERVE-CLI-CHAT-SPIKE`; CPU-only records/spec) | Accepts `.agents/specs/cli-chat-complete.md`, corrects the inventory from “no direct commands” to the actual pinned `chat`/`complete` surface, and decomposes a dual-mode port: exact remote OpenAI HTTP/SSE commands plus preservation of the existing in-process invocation as a compatibility alias. No production, test, CMake, model, kernel, fixture, or generated file changes. | Pinned vLLM `5559679229`: command registration `vllm/entrypoints/cli/main.py:17-37,73-98`; model/auth resolution and stream shaping `vllm/entrypoints/cli/openai.py:30-100`; chat `:155-234`; complete `:237-312`. The local compatibility baseline is `examples/cli/main.cpp:1-207`. | CPU record/doc gates only; benchmark `NOT APPLICABLE`, `benchmark_binding=false`. Implementation remains absent and the row moves `INVENTORIED` -> `SPIKE`. W1-W5 name parse, transport, complete, chat, and packaging gates, including fake-server request/SSE transcript parity, Release `-Werror`, ASan+UBSan, and TSan. |
| 2026-08-01 (`CLAIM-README-ESCAPED-PIPE`; `ROAD-V1-A6`; closing commit: this change) | Makes the README table-cell budget preserve escaped Markdown pipes as literal cell content, so one oversized prose cell cannot evade the limit by appearing as short fragments. No README, runtime, lifecycle, support, model, or benchmark change. | No vLLM runtime equivalent. This enforces the project-local `AGENTS.md:75-87` landing-page budget contract; design and exact anchors are in `.agents/specs/readme-escaped-pipe-cell-budget.md`. | RED: the new real-behavior mutation returned no errors on base `1448e981`. GREEN: focused README suite 19/19 and full Python script suite 96/96. Full repository checker results are recorded in the closing state entry. Benchmark NOT APPLICABLE, `benchmark_binding=false`. |
2 changes: 1 addition & 1 deletion .agents/roadmap_v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -533,7 +533,7 @@ diagnostics recorded in `docs/BENCHMARKS.md`.
| A3 | `test_ops_fused_chain` FMA-contraction fix | ✅ merged bf48edb (`-ffp-contract=off` host-wide) |
| A4 | De-Python the build: vendor Triton AOT artifacts per-arch (`triton_aot_vendored/<arch>/` + MANIFEST; `VLLM_CPP_TRITON_REGEN` = maintainer-only Python) | ✅ **DONE** (54367cc..a432461; reproducibility hardening `09f1d23`) — `sm_121a` now has 48 generated C/H files + MANIFEST, including both bf16 `chunk_o` shapes; normal builds remain Python-free. Regen is explicit-target (`cuda:121:32`), line-info-disabled and byte-reproducible across source paths; the pure checker makes source/contract/artifact drift fatal and mutation-tests missing/extra/changed artifacts. A4 remains closed; fresh current-main CUDA/runtime/performance validation belongs to the two ACTIVE `CLAIM-PR3` kernel rows (evidence: porting-inventory §9). |
| `SERVE-E2E-NIGHTLY` (formerly A5) | e2e suites per gates.md (server conformance nightly on dgx etc.) | ☐ next; leaf spike required |
| A6 | **User-facing surface closure** (audit `CLAIM-DOCS-COVERAGE`, 2026-07-24): make what SHIPPED reachable and truthfully documented. The audit found 2 class-D overstatements, 5 stale doc lines, 4 shipped-but-unreachable features and 153 undocumented env vars | 🚧 Tier 1 (falsehood repair) DONE in the audit commit. **Tier 2 DONE 2026-07-24 (`CLAIM-DOCS-T2-FIXES`)**, 3 of its 4 items: the D1 guard (a per-connector worker-half capability predicate; the disk connector is now REFUSED at engine construction on every device instead of silently shortcutting prefill for KV nothing writes — **its worker half remains unimplemented and is NOT claimed**), server `--tool-call-parser`/`--reasoning-parser` (40 tool names / 7 reasoning parsers now reachable over HTTP; defaults reproduce the old hardcode exactly), and server `--kv-transfer-config` mirroring vLLM's own CLI + JSON shape. Also landed from Tier 3/4: `docs/KV-OFFLOAD.md`, the README flag rows, the three missing Gemma model rows, and the `SERVE-C-ABI` 17→19 record repair. **Comprehensive refresh DONE 2026-07-25 (`CLAIM-DOCS-REFRESH`)**: **Tier 4** `docs/ENVIRONMENT.md` (26 user-facing env vars documented of 153 swept, kernel-internal tail on `scripts/env-doc-allowlist.txt`) + its CI checker `scripts/check-env-doc.py` (mutation-tested, fails on a new undocumented+unallowlisted production env var); the **Tier 3** README refresh to current git history (spec-decode `--speculative-config` now documented as shipped, the "not user-visible" class-B falsehood fixed, C ABI `VLLM_ABI_VERSION 6`, CLI spec-config flag); new usage guide `docs/SPECULATIVE-DECODING.md`. STILL OPEN: `vllm-cli` structured-output/sampling flags (Tier 2 item 4, code, not docs). **C-ABI contract W0 2026-07-31 (`CLAIM-SERVE-C-ABI-SPIKE`):** accepted [c-api-library.md](specs/c-api-library.md), corrected public ABI v9 labels to v10, and named W1-W5 compatibility/no-throw/lifetime gates; `SERVE-C-ABI` honestly remains `ANCHOR-BACKFILL`. **CLI chat/complete W0 2026-08-01 (`CLAIM-SERVE-CLI-CHAT-SPIKE`):** [cli-chat-complete.md](specs/cli-chat-complete.md) corrects the inventory (the pin does ship both remote commands), selects exact OpenAI HTTP/SSE parity plus a compatibility-preserved local invocation, and decomposes CPU-only W1-W5 implementation leaves; `SERVE-CLI-CHAT` is `SPIKE`, no command code has landed. Full table: [specs/docs-coverage-audit.md](specs/docs-coverage-audit.md) |
| A6 | **User-facing surface closure** (audit `CLAIM-DOCS-COVERAGE`, 2026-07-24): make what SHIPPED reachable and truthfully documented. The audit found 2 class-D overstatements, 5 stale doc lines, 4 shipped-but-unreachable features and 153 undocumented env vars | 🚧 Tier 1 (falsehood repair) DONE in the audit commit. **Tier 2 DONE 2026-07-24 (`CLAIM-DOCS-T2-FIXES`)**, 3 of its 4 items: the D1 guard (a per-connector worker-half capability predicate; the disk connector is now REFUSED at engine construction on every device instead of silently shortcutting prefill for KV nothing writes — **its worker half remains unimplemented and is NOT claimed**), server `--tool-call-parser`/`--reasoning-parser` (40 tool names / 7 reasoning parsers now reachable over HTTP; defaults reproduce the old hardcode exactly), and server `--kv-transfer-config` mirroring vLLM's own CLI + JSON shape. Also landed from Tier 3/4: `docs/KV-OFFLOAD.md`, the README flag rows, the three missing Gemma model rows, and the `SERVE-C-ABI` 17→19 record repair. **Comprehensive refresh DONE 2026-07-25 (`CLAIM-DOCS-REFRESH`)**: **Tier 4** `docs/ENVIRONMENT.md` (26 user-facing env vars documented of 153 swept, kernel-internal tail on `scripts/env-doc-allowlist.txt`) + its CI checker `scripts/check-env-doc.py` (mutation-tested, fails on a new undocumented+unallowlisted production env var); the **Tier 3** README refresh to current git history (spec-decode `--speculative-config` now documented as shipped, the "not user-visible" class-B falsehood fixed, C ABI `VLLM_ABI_VERSION 6`, CLI spec-config flag); new usage guide `docs/SPECULATIVE-DECODING.md`. STILL OPEN: `vllm-cli` structured-output/sampling flags (Tier 2 item 4, code, not docs). **C-ABI contract W0 2026-07-31 (`CLAIM-SERVE-C-ABI-SPIKE`):** accepted [c-api-library.md](specs/c-api-library.md), corrected public ABI v9 labels to v10, and named W1-W5 compatibility/no-throw/lifetime gates; `SERVE-C-ABI` honestly remains `ANCHOR-BACKFILL`. **CLI chat/complete W0 2026-08-01 (`CLAIM-SERVE-CLI-CHAT-SPIKE`):** [cli-chat-complete.md](specs/cli-chat-complete.md) corrects the inventory (the pin does ship both remote commands), selects exact OpenAI HTTP/SSE parity plus a compatibility-preserved local invocation, and decomposes CPU-only W1-W5 implementation leaves; `SERVE-CLI-CHAT` is `SPIKE`, no command code has landed. **Escaped-pipe checker repair 2026-08-01 (`CLAIM-README-ESCAPED-PIPE`):** the README cell budget now treats `\|` as literal cell content, closing a fail-open length-budget bypass; lifecycle and README content unchanged. Full table: [specs/docs-coverage-audit.md](specs/docs-coverage-audit.md) |

## B. Research tracks (complete)

Expand Down
82 changes: 82 additions & 0 deletions .agents/specs/readme-escaped-pipe-cell-budget.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# ROAD-V1-A6 spike: escaped-pipe README cell-budget repair

## Scope

This spike covers `ROAD-V1-A6` governance only. The checker must measure a
Markdown table cell as one cell when its content contains an escaped pipe
(`\|`). It must continue to split on unescaped delimiter pipes, ignore fenced
code, and enforce the existing `MAX_CELL_CHARS` threshold. Runtime code, README
content, lifecycle states, support claims, and benchmark behavior are out of
scope.

Claim: `CLAIM-README-ESCAPED-PIPE`.

## Upstream chain

There is no vLLM runtime surface to port. The binding project contract is
`AGENTS.md:75-87`, which requires the README table-cell budget and its mutation
test. Markdown defines a backslash-escaped pipe as literal cell content rather
than a delimiter. Dynamic dispatch and runtime tracing are not applicable.

## Our baseline

`scripts/check-readme-structure.py:167-186` scans table rows outside fenced
code. Base `1448e981` used `split("|")` at the corresponding parse site, which
treats `\|` as a delimiter. A single cell whose total length exceeds
`MAX_CELL_CHARS` could therefore pass when escaped pipes divided it into
fragments below the threshold. The repair is anchored at
`scripts/check-readme-structure.py:65-72,175-180`; its regression is
`tests/scripts/test_check_readme_structure.py:126-133`.

## Port map

| Contract | Local implementation | Deviation |
|---|---|---|
| Markdown escaped pipe remains cell content | `scripts/check-readme-structure.py` table-cell splitter | A small negative-lookbehind split is sufficient for the README subset already accepted by the checker; a full Markdown parser is intentionally out of scope. |
| Oversized cell is rejected | `tests/scripts/test_check_readme_structure.py` mutation | None. |

## Tests to port

No upstream vLLM test applies. Add one local mutation case containing one
oversized cell split by escaped pipes into individually sub-threshold fragments.
The production change that makes it fail is reverting the escaped-pipe-aware
splitter to raw `str.split("|")`. Keep the existing ordinary oversized-cell,
long-table, and fenced-code cases as regression coverage.

## Gates

- RED: the focused mutation test fails on base `1448e981` because no
wall-of-prose error is returned.
- GREEN: `python3 tests/scripts/test_check_readme_structure.py`.
- Regression: `python3 -m unittest discover -s tests/scripts -p 'test_*.py'`.
- Repository governance: every `scripts/check-*.py` checker exits zero.
- Checkpoint: `python3 scripts/check-doc-checkpoint.py --staged`.
- Hygiene: `git diff --cached --check` before commit and `git diff-tree --check
HEAD` after commit.
- Performance, memory, model, backend, and GPU gates: `NOT APPLICABLE`, because
only a pure Python documentation checker and its mutation suite change.

## Dependencies

Depends only on Python 3 and the repository files at `upstream/main` `1448e981`.
No compiler, model, hardware accelerator, external host, package download, or
license addition is required. Open PRs #2, #6, #7, and #13-#21 do not own this
escaped-pipe parser behavior or these exact test lines.

## Work breakdown

| Work | Files | Verification |
|---|---|---|
| A6-EP1 | `tests/scripts/test_check_readme_structure.py` | Focused RED proves the bypass. |
| A6-EP2 | `scripts/check-readme-structure.py` | Focused GREEN plus full script suite. |
| A6-EP3 | Coordination, roadmap, status, benchmark, ledger, state records | Record and document checkers. |

## Risks/decisions

Three approaches were considered. Special-casing the length calculation after
raw splitting is fragile because it must reconstruct cells. A full Markdown
parser adds an unnecessary dependency and a much larger parsing surface. The
selected approach splits only on pipes not immediately preceded by a backslash,
matching the repository's existing table parser convention. Multiple-backslash
Markdown edge cases are outside the checker's established subset and do not
justify broadening this governance fix.
16 changes: 16 additions & 0 deletions .agents/state.md
Original file line number Diff line number Diff line change
Expand Up @@ -34505,3 +34505,19 @@ The required Slack selection notification was attempted through the bundled
secret-safe sender to the only conventional target available, `#general`, but
Slack returned `channel_not_found`. No channel ID/name is configured and no
credential was inspected or exposed.

## 2026-08-01 - README escaped-pipe cell-budget repair

`CLAIM-README-ESCAPED-PIPE`, `ROAD-V1-A6`, isolated worktree
`/home/mudler/.cache/sdd/localai-org-maint-bot-vllm.cpp/codex-readme-escaped-pipe`,
base `upstream/main` `1448e981`, CPU-only. The README structure checker split
table rows with raw `str.split("|")`; an escaped Markdown pipe therefore divided
one real cell into fragments and let an oversized prose cell pass. The accepted
design uses the repository's existing negative-lookbehind convention to split
only unescaped delimiter pipes.

TDD evidence: the new mutation failed alone on the base behavior with
`errors == []`; after the parser fix, the focused README suite passes 19/19 and
the complete `tests/scripts` suite passes 96/96. No runtime C/C++, README
content, lifecycle, support claim, model, build, GPU, download, or benchmark
changed. Final standalone checker and Git-hygiene results are recorded in the PR.
8 changes: 8 additions & 0 deletions docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Benchmarks

## README escaped-pipe cell-budget repair (2026-08-01, `CLAIM-README-ESCAPED-PIPE`) - NOT APPLICABLE

The documentation checker now keeps `\|` inside its Markdown table cell when
enforcing the existing prose-length budget. RED-first mutation coverage proves
the base checker accepted one oversized cell split into short fragments; the
escaped-pipe-aware splitter rejects it. No runtime, model, build, throughput,
latency, or memory behavior changes. `benchmark_binding=false`.

## Laguna-S-2.1-NVFP4 decode — routed-MoE CastF32 fold (`VT_LAGUNA_TAIL_FUSED`), byte-exact, −39 nodes/step, wall-neutral (2026-08-03, `CLAIM-LAGUNA-TAIL-FUSED`)

Continuation of the byte-exact node-count campaign (glue/preamble/addnorm/onecast) on the GB10 NVFP4 decode graph (`~/laguna-xs-nvfp4`, ids `2,785,9626,377,15360,395`, base env `VT_LAGUNA_RESIDENT_DECODE=1 VT_LAGUNA_MARLIN_MOE=1 VT_LAGUNA_DECODE_GRAPH=1`, origin/main `65f3cdc1`). A fresh `cuda_gpu_kern_sum --cuda-graph-trace=node` 20↔70 diff of the baseline ranked the remaining SMALL decode kernels; the biggest tail items are already folded (`RmsNormRow`+`AddAdd2RmsNorm` = the add_rms_norm forms, `FusedQkNormRope` = the preamble) or unfoldable (`SigmoidTopK`/router GEMV = cuBLAS-adjacent, `DecodeAttnGqaSplitG`/`DecodeAttnCombine` = attention compute, `MoeAlign`/`SiluAndMul`/`MoeCombine` = ported-Marlin pipeline). The one clean byte-exact node reduction left was the MoE-output `CastF32`.
Expand Down
7 changes: 7 additions & 0 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@ citing "vLLM 0.25.0" are the last binding measurement against the prior oracle

## Capability status

`ROAD-V1-A6` documentation governance (2026-08-01,
`CLAIM-README-ESCAPED-PIPE`): the README table-cell budget now treats escaped
Markdown pipes as cell content, closing a fail-open path that could hide one
oversized prose cell behind individually sub-threshold fragments. This is a
checker-only repair; lifecycle states, support claims, and README content are
unchanged.

GCC 12 production-library maintenance (2026-07-31): the two known `-Werror`
blockers in GGUF prefaulting and KV-offload temporary-file naming are fixed
without behavior or lifecycle changes. The production library and focused
Expand Down
Loading
Loading