From 17ca6213e6d9098ee6773470b007bf439b4176f5 Mon Sep 17 00:00:00 2001 From: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> Date: Sat, 1 Aug 2026 05:06:10 +0000 Subject: [PATCH] fix(ci): cover all binding checkpoint records Require STATUS and BENCHMARKS updates when SGLang, environment, or upstream-sync records change, with RED-first mutation coverage. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex] --- .agents/coordination.md | 16 ++++++++++++++++ .agents/parity-ledger.md | 1 + .agents/roadmap_v1.md | 2 +- .agents/state.md | 9 +++++++++ docs/BENCHMARKS.md | 7 +++++++ docs/STATUS.md | 5 +++++ scripts/check-doc-checkpoint.py | 3 +++ tests/scripts/test_doc_checkpoint.py | 12 ++++++++++++ 8 files changed, 54 insertions(+), 1 deletion(-) diff --git a/.agents/coordination.md b/.agents/coordination.md index 96f6f388..6fe57369 100644 --- a/.agents/coordination.md +++ b/.agents/coordination.md @@ -118,6 +118,22 @@ without the selected contention proof for their entire run are discarded. ## Active claims +**Documentation-checkpoint canonical-record coverage (`ROAD-V1-A6`, +2026-08-01, `CLAIM-ROAD-V1-A6-DOC-CHECKPOINT-COVERAGE`).** Codex (GPT-5), +isolated worktree `/home/mudler/_git/vllm.cpp-codex-cpu-20260801`, branch +`codex/cpu-maintenance-20260801b`, base `upstream/main` `1448e981`. CPU-only +governance bugfix under the accepted +`.agents/specs/docs-coverage-audit.md`: `scripts/check-doc-checkpoint.py` omits +the canonical `.agents/sglang-matrix.md`, `.agents/environment.md`, and +`.agents/sync/` record surfaces, allowing binding parity/environment/sync +checkpoints to bypass the required same-commit `docs/STATUS.md` and +`docs/BENCHMARKS.md` update. Owns only the checker, its mutation tests, this +claim, the `ROAD-V1-A6` current-state cell, the two required public checkpoint +notes, and append-only state/ledger evidence. No runtime source, build, model, +kernel, lifecycle, support claim, README, or benchmark number changes. No GPU, +model, compiler, external host, or download is required; verification is the +Python checker mutation suite plus the repository record/document checkers. + **Canonical DONE-owner reachability repair (`KV-PREFIX-CACHE`, `SAMPLE-LOGPROBS`, `SPEC-DFLASH`, `MODEL-SPEC-qwen3-dflash-dflash-qwen3-for-causal-lm`, diff --git a/.agents/parity-ledger.md b/.agents/parity-ledger.md index b8028442..3b74da7d 100644 --- a/.agents/parity-ledger.md +++ b/.agents/parity-ledger.md @@ -885,3 +885,4 @@ Columns: | 2026-07-29 (**Post-rebase RE-VALIDATION of the 4B lever onto `main` `f3ecbe70d`, +139 upstream commits: nothing moved**; `CLAIM-POSTREBASE-REVALIDATE`; local working branch `bench-lever-sampled-token-20260727` rebased from `7f620e74`, build `build-nix-cuda-transplant-triton` clean 925/925 with 0 warnings; ALL GPU under one `flock /tmp/gpu`, all 9 performance legs at 0% utilization; NOT pushed; closing commit: this checkpoint) | **What it does.** Re-measures the Qwen3.5-4B matched comparison after the branch was rebased across 139 upstream commits, against the SAME source-built oracle at the parity pin (`0.23.1rc1.dev1511+g555967922`; pin re-checked and unchanged before the run). Result is a null: 0.9972x total throughput (was 0.9970x), TTFT 0.7701x PASS, TPOT 1.1247x FAIL — every axis inside noise. Records-and-evidence only; no engine change. | vLLM `555967922` (0.26.0.dev0) built from source at the pin, run through the same `run_qwen35_4b_compare.sh` harness as the 2026-07-28 series; no upstream code ported. | **VERIFIED, and the control is the point.** Our output is token-identical 128/128 per repetition to the pre-rebase series on BOTH the direct-ON and direct-OFF arms, which is the semantic check on three rebase conflict resolutions (two append-only record files, plus a real `ModelForwardInput` field collision with upstream's `mm`). All three arms drifted down ~0.13% (ours 0.9988x, OFF 0.9983x, pin 0.9986x) — the pin arm is an UNCHANGED binary, so that drift is ambient, not code. Gates re-run green: `test_qwen35_plain_weights --no-skip` 3/3·1672/1672, `test_input_batch` 25/25·183/183, `test_combine_tokens` 7/7·14/14. Evidence: [docs/bench-evidence/qwen35-4b-postrebase-20260729.md](../docs/bench-evidence/qwen35-4b-postrebase-20260729.md). | | 2026-07-31 (`SERVE-C-ABI` W0 contract spike; `CLAIM-SERVE-C-ABI-SPIKE`; CPU-only records/docs) | Accepted `.agents/specs/c-api-library.md` for the already-shipped original C packaging layer: complete scope, vLLM semantic chain/deviation, ABI v10/19-symbol baseline, ownership/error/version/dispatch rules, exact code/test anchors, gates, dependencies, risks, and W1-W5 follow-ons. Also fixes the verified stale public `VLLM_ABI_VERSION 9` labels in README/USAGE to the source-of-truth v10 and adds the missing v10 usage-table entry. No production/test/CMake source changed. | Pinned vLLM `555967922` has no C ABI; behavior beneath the adapter remains owned by its vLLM-derived engine rows. The flat ABI is the recorded llama.cpp-style packaging deviation and may translate, never reimplement, policy. | **CPU/records gate only; benchmark NOT APPLICABLE.** Focused C11/C++/dlopen/export gate passed 3/3 after explicitly building `vllm_shared`; five record checkers pass. `check-agent-record` reports the base tree's same six missing closing-commit objects (`444ea9d7`, `7a3f04b2`, `164453a2`), none in this row/diff. Row stays `ANCHOR-BACKFILL` because all-symbol dlsym coverage (chat symbols currently omitted), historical-layout compatibility, allocation-failure no-throw proof, lifetime sanitizer stress, and a standalone real-model C consumer remain W1-W5. | | 2026-07-31 (`CLAIM-CPU-GCC12-WERROR-PORTABILITY`; maintenance, rows `QUANT-GGUF-KEEPQ-LOADER` + `KV-OFFLOAD`; lifecycle unchanged) | Removes two GCC 12 production-library `-Werror` blockers without suppressions: the GGUF prefault keeps the same one-byte-per-page volatile XOR but uses simple assignment, and the KV filesystem tier builds the identical `...tmp` suffix with append operations inside its thread-local initializer. No API, algorithm, default, CUDA, fixture, or golden change. | Behavior remains grounded in the accepted loader and KV-persistence leaf specs: llama.cpp mmap prefault intent and vLLM `tiering/fs/io.py` unique temporary-file publication. This is compiler portability, not a parity-surface change. | RED: GCC 12 failed first at `qwen3_5_gguf_weights.cpp:49` (`-Wvolatile`), then at `fs_io.cpp:66` (`-Wrestrict`). GREEN: production `vllm` and focused test targets build clean; focused CTest 2/2 (`test_gguf_keep_quant`, `test_kv_offload_fs`). Full all-target build is PARTIAL at 42% on unrelated test-only GCC 12 `-Wrestrict` diagnostics in `test_deepseek_v2_paged_engine.cpp` and `test_glm4_moe_lite_paged_engine.cpp`; no full-CTest claim. Benchmark NOT APPLICABLE. | +| 2026-08-01 (`ROAD-V1-A6`; `CLAIM-ROAD-V1-A6-DOC-CHECKPOINT-COVERAGE`; CPU-only governance) | Extends the existing same-commit public-document policy to the three binding canonical-record surfaces it omitted: `.agents/sglang-matrix.md`, `.agents/environment.md`, and `.agents/sync/`. No runtime, lifecycle, support, or benchmark behavior changes. | Project governance only, under the accepted `.agents/specs/docs-coverage-audit.md`; no vLLM behavior counterpart. | RED-first mutation proof: all three paths produced zero errors before the fix; after the exact allowlist/prefix additions each produces both required STATUS and BENCHMARKS errors. Python script suite 96/96 green. Benchmark NOT APPLICABLE. | diff --git a/.agents/roadmap_v1.md b/.agents/roadmap_v1.md index e1362df7..623753a8 100644 --- a/.agents/roadmap_v1.md +++ b/.agents/roadmap_v1.md @@ -533,7 +533,7 @@ diagnostics recorded in `docs/BENCHMARKS.md`. | A3 | `test_ops_fused_chain` FMA-contraction fix | ✅ merged bf48edb (`-ffp-contract=off` host-wide) | | A4 | De-Python the build: vendor Triton AOT artifacts per-arch (`triton_aot_vendored//` + MANIFEST; `VLLM_CPP_TRITON_REGEN` = maintainer-only Python) | ✅ **DONE** (54367cc..a432461; reproducibility hardening `09f1d23`) — `sm_121a` now has 48 generated C/H files + MANIFEST, including both bf16 `chunk_o` shapes; normal builds remain Python-free. Regen is explicit-target (`cuda:121:32`), line-info-disabled and byte-reproducible across source paths; the pure checker makes source/contract/artifact drift fatal and mutation-tests missing/extra/changed artifacts. A4 remains closed; fresh current-main CUDA/runtime/performance validation belongs to the two ACTIVE `CLAIM-PR3` kernel rows (evidence: porting-inventory §9). | | `SERVE-E2E-NIGHTLY` (formerly A5) | e2e suites per gates.md (server conformance nightly on dgx etc.) | ☐ next; leaf spike required | -| A6 | **User-facing surface closure** (audit `CLAIM-DOCS-COVERAGE`, 2026-07-24): make what SHIPPED reachable and truthfully documented. The audit found 2 class-D overstatements, 5 stale doc lines, 4 shipped-but-unreachable features and 153 undocumented env vars | 🚧 Tier 1 (falsehood repair) DONE in the audit commit. **Tier 2 DONE 2026-07-24 (`CLAIM-DOCS-T2-FIXES`)**, 3 of its 4 items: the D1 guard (a per-connector worker-half capability predicate; the disk connector is now REFUSED at engine construction on every device instead of silently shortcutting prefill for KV nothing writes — **its worker half remains unimplemented and is NOT claimed**), server `--tool-call-parser`/`--reasoning-parser` (40 tool names / 7 reasoning parsers now reachable over HTTP; defaults reproduce the old hardcode exactly), and server `--kv-transfer-config` mirroring vLLM's own CLI + JSON shape. Also landed from Tier 3/4: `docs/KV-OFFLOAD.md`, the README flag rows, the three missing Gemma model rows, and the `SERVE-C-ABI` 17→19 record repair. **Comprehensive refresh DONE 2026-07-25 (`CLAIM-DOCS-REFRESH`)**: **Tier 4** `docs/ENVIRONMENT.md` (26 user-facing env vars documented of 153 swept, kernel-internal tail on `scripts/env-doc-allowlist.txt`) + its CI checker `scripts/check-env-doc.py` (mutation-tested, fails on a new undocumented+unallowlisted production env var); the **Tier 3** README refresh to current git history (spec-decode `--speculative-config` now documented as shipped, the "not user-visible" class-B falsehood fixed, C ABI `VLLM_ABI_VERSION 6`, CLI spec-config flag); new usage guide `docs/SPECULATIVE-DECODING.md`. STILL OPEN: `vllm-cli` structured-output/sampling flags (Tier 2 item 4, code, not docs). **C-ABI contract W0 2026-07-31 (`CLAIM-SERVE-C-ABI-SPIKE`):** accepted [c-api-library.md](specs/c-api-library.md), corrected public ABI v9 labels to v10, and named W1-W5 compatibility/no-throw/lifetime gates; `SERVE-C-ABI` honestly remains `ANCHOR-BACKFILL`. Full table: [specs/docs-coverage-audit.md](specs/docs-coverage-audit.md) | +| A6 | **User-facing surface closure** (audit `CLAIM-DOCS-COVERAGE`, 2026-07-24): make what SHIPPED reachable and truthfully documented. The audit found 2 class-D overstatements, 5 stale doc lines, 4 shipped-but-unreachable features and 153 undocumented env vars | 🚧 Tier 1 (falsehood repair) DONE in the audit commit. **Tier 2 DONE 2026-07-24 (`CLAIM-DOCS-T2-FIXES`)**, 3 of its 4 items: the D1 guard (a per-connector worker-half capability predicate; the disk connector is now REFUSED at engine construction on every device instead of silently shortcutting prefill for KV nothing writes — **its worker half remains unimplemented and is NOT claimed**), server `--tool-call-parser`/`--reasoning-parser` (40 tool names / 7 reasoning parsers now reachable over HTTP; defaults reproduce the old hardcode exactly), and server `--kv-transfer-config` mirroring vLLM's own CLI + JSON shape. Also landed from Tier 3/4: `docs/KV-OFFLOAD.md`, the README flag rows, the three missing Gemma model rows, and the `SERVE-C-ABI` 17→19 record repair. **Comprehensive refresh DONE 2026-07-25 (`CLAIM-DOCS-REFRESH`)**: **Tier 4** `docs/ENVIRONMENT.md` (26 user-facing env vars documented of 153 swept, kernel-internal tail on `scripts/env-doc-allowlist.txt`) + its CI checker `scripts/check-env-doc.py` (mutation-tested, fails on a new undocumented+unallowlisted production env var); the **Tier 3** README refresh to current git history (spec-decode `--speculative-config` now documented as shipped, the "not user-visible" class-B falsehood fixed, C ABI `VLLM_ABI_VERSION 6`, CLI spec-config flag); new usage guide `docs/SPECULATIVE-DECODING.md`. STILL OPEN: `vllm-cli` structured-output/sampling flags (Tier 2 item 4, code, not docs). **C-ABI contract W0 2026-07-31 (`CLAIM-SERVE-C-ABI-SPIKE`):** accepted [c-api-library.md](specs/c-api-library.md), corrected public ABI v9 labels to v10, and named W1-W5 compatibility/no-throw/lifetime gates; `SERVE-C-ABI` honestly remains `ANCHOR-BACKFILL`. **Checkpoint coverage hardened 2026-08-01:** the public-doc gate now includes the SGLang matrix, environment registry, and upstream-sync reports, with mutation proof. Full table: [specs/docs-coverage-audit.md](specs/docs-coverage-audit.md) | ## B. Research tracks (complete) diff --git a/.agents/state.md b/.agents/state.md index 7dfc8423..1b354d79 100644 --- a/.agents/state.md +++ b/.agents/state.md @@ -34432,3 +34432,12 @@ PARTIAL on two further unrelated test-only GCC 12 `-Wrestrict` diagnostics in CTest cannot run from this incomplete build. No GPU, model, or benchmark ran. **Canonical DONE-owner reachability repaired (2026-07-31, `CLAIM-RECORD-DONE-OWNER-REPAIR`).** Records-only correction: the six checker-reported `DONE` owner cells now name the reachable commits that introduced their binding closures (`KV-PREFIX-CACHE` `a41af480`, `SAMPLE-LOGPROBS` `ecda3ce1`, and the four DFlash rows `489a7544`). Lifecycle states, evidence, support claims, source, tests, and benchmark results are unchanged. + +**Documentation-checkpoint canonical-record coverage (2026-08-01, +`CLAIM-ROAD-V1-A6-DOC-CHECKPOINT-COVERAGE`).** CPU-only governance fix for +`ROAD-V1-A6`: `check-doc-checkpoint.py` previously omitted the binding SGLang +matrix, environment registry, and upstream-sync report directory, so changes to +those records could bypass the required same-commit STATUS/BENCHMARKS update. +Added the three exact policy entries and a RED-first mutation test over each +surface (3 failures before the fix, 6/6 green after). No runtime, lifecycle, +support, README, or benchmark-number change. diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 17c962c1..fc20b324 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -10974,3 +10974,10 @@ Six existing `DONE` matrix rows now point to reachable commits that introduced their binding closure states. This records-only metadata repair changes no lifecycle state, implementation, workload, or accepted benchmark result, so no new performance number is applicable or claimed. + +## Documentation checkpoint coverage (2026-08-01) - NOT APPLICABLE + +The documentation gate now covers the canonical SGLang matrix, environment +registry, and upstream-sync reports. This is a Python CI-policy correction with +mutation tests; it changes no runtime code, workload, latency, throughput, or +memory behavior, so no benchmark is applicable. diff --git a/docs/STATUS.md b/docs/STATUS.md index 102532af..d4f74fd7 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -2061,3 +2061,8 @@ _(Laguna W11 go/no-go: decode is GPU-COMPUTE-bound (GPU-busy 2.56s ≈ sync 2.59 **Fold-plan A3 tier CLOSED (2026-07-31).** Both keep-quant models (Laguna W9 + qwen3_5 GGUF W2/W3a/W3b) route their routed-expert MoE through the shared vt::MatmulBTQuantGrouped descriptor, byte-exact gated. See .agents/specs/arch-fusion-fold-plan-2026-07-30.md A3. **Canonical record-owner repair (2026-07-31).** No lifecycle or support status changed. Six existing `DONE` rows now point to their reachable binding closure commits; this is records-only metadata maintenance. + +**Documentation checkpoint coverage (2026-08-01).** The same-commit public-doc +gate now treats `.agents/sglang-matrix.md`, `.agents/environment.md`, and +`.agents/sync/` reports as binding checkpoint surfaces. This is governance +hardening only; no capability lifecycle or support claim changed. diff --git a/scripts/check-doc-checkpoint.py b/scripts/check-doc-checkpoint.py index 7130fe79..ed6ab38a 100755 --- a/scripts/check-doc-checkpoint.py +++ b/scripts/check-doc-checkpoint.py @@ -19,6 +19,7 @@ CHECKPOINT_PREFIXES = ( ".agents/completed/", ".agents/specs/", + ".agents/sync/", ".github/workflows/", "cmake/", "examples/", @@ -33,6 +34,7 @@ ".agents/backend-matrix.md", ".agents/coordination.md", ".agents/engine-matrix.md", + ".agents/environment.md", ".agents/feature-matrix.md", ".agents/kernel-matrix.md", ".agents/model-matrix.md", @@ -40,6 +42,7 @@ ".agents/porting-inventory.md", ".agents/quantization-matrix.md", ".agents/roadmap_v1.md", + ".agents/sglang-matrix.md", ".agents/state.md", } diff --git a/tests/scripts/test_doc_checkpoint.py b/tests/scripts/test_doc_checkpoint.py index b6e5869b..a5a90d99 100755 --- a/tests/scripts/test_doc_checkpoint.py +++ b/tests/scripts/test_doc_checkpoint.py @@ -42,6 +42,18 @@ def test_lifecycle_and_diagnostic_records_are_checkpoints(self) -> None: with self.subTest(path=path): self.assertTrue(doc_checkpoint.checkpoint_errors({path})) + def test_all_binding_record_surfaces_are_checkpoints(self) -> None: + for path in ( + ".agents/sglang-matrix.md", + ".agents/environment.md", + ".agents/sync/2026-08-01-example.md", + ): + with self.subTest(path=path): + errors = doc_checkpoint.checkpoint_errors({path}) + self.assertEqual(len(errors), 2) + self.assertIn("docs/STATUS.md", errors[0]) + self.assertIn("docs/BENCHMARKS.md", errors[1]) + def test_both_public_documents_satisfy_checkpoint(self) -> None: self.assertEqual( doc_checkpoint.checkpoint_errors(