Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .agents/coordination.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,23 @@ leaves. Owns only NEW `.agents/specs/cli-chat-complete.md`, the
fixture, or GPU/model-download change; verification is the CPU record/doc
checker suite. The row and open-PR list were unclaimed at selection time.

**Device-leakage preprocessor polarity hardening (`ROAD-V1-A6`,
`BACKEND-SEAM-AUDIT` work row S1, 2026-08-01,
`CLAIM-DSR-GUARD-POLARITY`).** Codex (GPT-5), isolated worktree
`/home/mudler/_git/vllm.cpp-wt-dsr-guard-polarity`, branch
`codex/dsr-guard-polarity`, base `upstream/main` `1448e981`. CPU-only bugfix:
the S1 ratchet currently treats negative CUDA guards (`#ifndef
VLLM_CPP_CUDA`, `#if !defined(VLLM_CPP_CUDA)`) as CUDA-only branches and
therefore fails open for CUDA includes in portable code while classifying their
CUDA `#else` arms in reverse. Owns only `scripts/check-device-leakage.py`,
`tests/scripts/test_device_leakage.py`, this claim, the S1 hardening note in
`.agents/specs/accelerator-seam-audit.md`, and the required no-lifecycle-change
checkpoint notes in `docs/STATUS.md`, `docs/BENCHMARKS.md`,
`.agents/parity-ledger.md`, and `.agents/state.md`. No source, header, CMake,
workflow, baseline-count, lifecycle, support, README, GPU, model, download, or
benchmark change. Verification is RED/GREEN mutation evidence, the full Python
mutation suite, and all record/document checkers.

**Canonical DONE-owner reachability repair (`KV-PREFIX-CACHE`,
`SAMPLE-LOGPROBS`, `SPEC-DFLASH`,
`MODEL-SPEC-qwen3-dflash-dflash-qwen3-for-causal-lm`,
Expand Down
1 change: 1 addition & 0 deletions .agents/parity-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -886,3 +886,4 @@ Columns:
| 2026-07-31 (`SERVE-C-ABI` W0 contract spike; `CLAIM-SERVE-C-ABI-SPIKE`; CPU-only records/docs) | Accepted `.agents/specs/c-api-library.md` for the already-shipped original C packaging layer: complete scope, vLLM semantic chain/deviation, ABI v10/19-symbol baseline, ownership/error/version/dispatch rules, exact code/test anchors, gates, dependencies, risks, and W1-W5 follow-ons. Also fixes the verified stale public `VLLM_ABI_VERSION 9` labels in README/USAGE to the source-of-truth v10 and adds the missing v10 usage-table entry. No production/test/CMake source changed. | Pinned vLLM `555967922` has no C ABI; behavior beneath the adapter remains owned by its vLLM-derived engine rows. The flat ABI is the recorded llama.cpp-style packaging deviation and may translate, never reimplement, policy. | **CPU/records gate only; benchmark NOT APPLICABLE.** Focused C11/C++/dlopen/export gate passed 3/3 after explicitly building `vllm_shared`; five record checkers pass. `check-agent-record` reports the base tree's same six missing closing-commit objects (`444ea9d7`, `7a3f04b2`, `164453a2`), none in this row/diff. Row stays `ANCHOR-BACKFILL` because all-symbol dlsym coverage (chat symbols currently omitted), historical-layout compatibility, allocation-failure no-throw proof, lifetime sanitizer stress, and a standalone real-model C consumer remain W1-W5. |
| 2026-07-31 (`CLAIM-CPU-GCC12-WERROR-PORTABILITY`; maintenance, rows `QUANT-GGUF-KEEPQ-LOADER` + `KV-OFFLOAD`; lifecycle unchanged) | Removes two GCC 12 production-library `-Werror` blockers without suppressions: the GGUF prefault keeps the same one-byte-per-page volatile XOR but uses simple assignment, and the KV filesystem tier builds the identical `.<pid>.<counter>.tmp` suffix with append operations inside its thread-local initializer. No API, algorithm, default, CUDA, fixture, or golden change. | Behavior remains grounded in the accepted loader and KV-persistence leaf specs: llama.cpp mmap prefault intent and vLLM `tiering/fs/io.py` unique temporary-file publication. This is compiler portability, not a parity-surface change. | RED: GCC 12 failed first at `qwen3_5_gguf_weights.cpp:49` (`-Wvolatile`), then at `fs_io.cpp:66` (`-Wrestrict`). GREEN: production `vllm` and focused test targets build clean; focused CTest 2/2 (`test_gguf_keep_quant`, `test_kv_offload_fs`). Full all-target build is PARTIAL at 42% on unrelated test-only GCC 12 `-Wrestrict` diagnostics in `test_deepseek_v2_paged_engine.cpp` and `test_glm4_moe_lite_paged_engine.cpp`; no full-CTest claim. Benchmark NOT APPLICABLE. |
| 2026-08-01 (`SERVE-CLI-CHAT` W0 contract spike; `CLAIM-SERVE-CLI-CHAT-SPIKE`; CPU-only records/spec) | Accepts `.agents/specs/cli-chat-complete.md`, corrects the inventory from “no direct commands” to the actual pinned `chat`/`complete` surface, and decomposes a dual-mode port: exact remote OpenAI HTTP/SSE commands plus preservation of the existing in-process invocation as a compatibility alias. No production, test, CMake, model, kernel, fixture, or generated file changes. | Pinned vLLM `5559679229`: command registration `vllm/entrypoints/cli/main.py:17-37,73-98`; model/auth resolution and stream shaping `vllm/entrypoints/cli/openai.py:30-100`; chat `:155-234`; complete `:237-312`. The local compatibility baseline is `examples/cli/main.cpp:1-207`. | CPU record/doc gates only; benchmark `NOT APPLICABLE`, `benchmark_binding=false`. Implementation remains absent and the row moves `INVENTORIED` -> `SPIKE`. W1-W5 name parse, transport, complete, chat, and packaging gates, including fake-server request/SSE transcript parity, Release `-Werror`, ASan+UBSan, and TSan. |
| 2026-08-01 (`CLAIM-DSR-GUARD-POLARITY`; `ROAD-V1-A6` / `BACKEND-SEAM-AUDIT` S1 hardening; lifecycle unchanged) | Makes the CPU-only device-leakage ratchet preserve preprocessor branch polarity: positive CUDA guards are protected, negative CUDA guards are portable, and `#else` inverts either classification. No runtime source, baseline count, workflow, lifecycle, support, or benchmark change. | No upstream vLLM analogue; this is project governance under the accepted accelerator-seam S1 contract. | RED: four mutations proved `#ifndef` and `!defined` arms were classified exactly backward; two more caught positive compound-guard regression; two disjunction mutations caught the CPU-reachable OR arm. GREEN: focused 8/8, full mutation suite 32/32, and real-tree DSR 32 equals baseline 32. Benchmark NOT APPLICABLE (`benchmark_binding=false`). |
10 changes: 9 additions & 1 deletion .agents/specs/accelerator-seam-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,14 @@ benchmark — nothing was implemented. `docs/BENCHMARKS.md` records this as
GPU. *Escape hatch:* an explicit `// DSR-ALLOW(<row-id>): <reason>` comment,
which the checker counts separately and reports on every run, so a justified
exception is visible rather than invisible. **0 in force today.**
**Polarity hardening (2026-08-01, `CLAIM-DSR-GUARD-POLARITY`):** the
preprocessor model now distinguishes CUDA-only, portable-only, and unrelated
branches. This closes the fail-open case where `#ifndef VLLM_CPP_CUDA` or
`#if !defined(VLLM_CPP_CUDA)` incorrectly protected a CUDA include in the
portable arm and classified the CUDA `#else` arm in reverse. Four RED-first
mutations cover both spellings and both arms; two more preserve a positive
CUDA term in a compound guard and its inverse arm; two disjunction cases
prove the arm reachable on CPU is never exempted. The suite is now 32 cases.
Per-row targets, restated against the corrected baseline: `S2` → 86 (its
include/queue half is already done, §8.2), `S3` → 83, `S6` → ~37,
`S4`+`S7` → **< 10, all of it class C/D**.
Expand Down Expand Up @@ -382,7 +390,7 @@ claim; none is claimed by this audit.

| Rank | ID | Work | DSR after | Depends | Why here |
|---:|---|---|---:|---|---|
| **1** | `S1` | **The DSR metric + CI ratchet — `DONE` 2026-07-22 (`CLAIM-BACKEND-SEAM-S1-1`).** [`scripts/check-device-leakage.py`](../../scripts/check-device-leakage.py) + [baseline](../../scripts/device-leakage-baseline.json) + CI job `device-leakage` + [24-case mutation suite](../../tests/scripts/test_device_leakage.py). Per-file/per-bucket allowlist with exact counts and reasons, `DSR-ALLOW` escape hatch, per-bucket AND total monotonic decrease | **86** (baseline, §8) | — | Cheapest row in the plan and it guards every row after it. **The counts already regrew 54→63 and 13→16 in days of well-executed work** — without this, every reduction below decays |
| **1** | `S1` | **The DSR metric + CI ratchet — `DONE` 2026-07-22 (`CLAIM-BACKEND-SEAM-S1-1`), negative-guard polarity hardened 2026-08-01 (`CLAIM-DSR-GUARD-POLARITY`).** [`scripts/check-device-leakage.py`](../../scripts/check-device-leakage.py) + [baseline](../../scripts/device-leakage-baseline.json) + CI job `device-leakage` + [32-case mutation suite](../../tests/scripts/test_device_leakage.py). Per-file/per-bucket allowlist with exact counts and reasons, `DSR-ALLOW` escape hatch, per-bucket AND total monotonic decrease | **86** (baseline, §8) | — | Cheapest row in the plan and it guards every row after it. **The counts already regrew 54→63 and 13→16 in days of well-executed work** — without this, every reduction below decays |
| **2** | `S2` | **Reuse-study `W0b-1` residue** — **the include + hard-coded-queue half is ALREADY DONE on main** (§8.2): `model_loader.cpp` now calls `SelectQueue()`, and all four "unconditional" CUDA includes were in fact already `#ifdef`-guarded (an audit measurement error). Remaining: Metal `get_attn_backend_priority()` | 86 | `S1` | Class E and class F are both empty at `18094ee`; what is left is the Metal priority list, which the metric does not count |
| **3** | `S3` | **Platform capability fields — `DONE` 2026-07-23 (`CLAIM-BACKEND-SEAM-S3-1`).** Mirrors `interface.py:914,933,977,1058` (`is_integrated_gpu`/`supports_fp8`/`opaque_attention_op`/`support_static_graph_mode`) + `nvfp4_utils.py:56 cutlass_fp4_supported` + `is_device_capability_family:441` on our `Platform` (base false; the CudaPlatform leg answers vLLM's values); converts **12** deferred `device==kCUDA`/graph gates in `qwen3_5.cpp` to the capability predicate — 7 fp4-activation → `cutlass_fp4_supported()`, 3 fp8-fused → `supports_fp8()`, 2 decode-graph → `support_static_graph_mode()`. See **§12** | **55** (as-built §12; the S6-deferred class-D residency/stream/FA2-dtype/merged-layout sites stay for `S7`) | `S2` | Pure mirror of upstream, mechanical, and the byte-identical unlock `S6` re-scoped the fp4/fp8 gates onto (a capability answers false off-CUDA exactly as `device==kCUDA` did, where `OpRegistered` answered TRUE on `kCPU`). Makes ROCm a *config* rather than a port |
| **4** | `S5` | **Portable reference tier — `DONE` 2026-07-23 (`CLAIM-BACKEND-SEAM-S5-1`).** On a UNIFIED-MEMORY device the CPU kernel installs LAZILY (on the first `GetOp` miss) as a negative-priority (`-1000`) `vt-cpu-ref` `OpProvider`, so an op the backend lacks falls back to the portable CPU reference instead of throwing. Mirrors `custom_op.py:138 forward_native`. See §10 | **67** (unchanged — vt-runtime infra, no shared-layer edit) | `S3` | **Changes the economics of every future backend**: op count stops being a correctness gate. Uses only the mechanism we already shipped. Gate = OPT on Metal token-exact with today's 10 kernels (Mac demo); hardware-free proof = `vt::Relu` on a fake unified device with zero native kernels, bit-identical to the CPU oracle |
Expand Down
16 changes: 16 additions & 0 deletions .agents/state.md
Original file line number Diff line number Diff line change
Expand Up @@ -34505,3 +34505,19 @@ The required Slack selection notification was attempted through the bundled
secret-safe sender to the only conventional target available, `#general`, but
Slack returned `channel_not_found`. No channel ID/name is configured and no
credential was inspected or exposed.

## 2026-08-01 - DSR negative-guard polarity hardening

`CLAIM-DSR-GUARD-POLARITY`, isolated worktree
`/home/mudler/_git/vllm.cpp-wt-dsr-guard-polarity`, branch
`codex/dsr-guard-polarity`, base `upstream/main` `1448e981`, CPU-only. The S1
device-leakage ratchet stored only whether a conditional mentioned a CUDA macro,
so `#ifndef VLLM_CPP_CUDA` and `#if !defined(VLLM_CPP_CUDA)` were treated as
CUDA-only while their `#else` arms were treated as portable. Four RED-first
mutations reproduced both spellings and both inverted arms; two compound-guard
mutations preserve positive CUDA-term precedence; two disjunction mutations
prove the CPU-reachable arm remains counted. The classifier now
stores CUDA-only / portable-only / unrelated polarity and inverts it at
`#else`; nested positive guards retain their prior behavior. Full mutation suite
32/32 and real-tree DSR 32/32 pass. No GPU, compiler, model, download, runtime,
baseline, lifecycle, support, or benchmark change.
10 changes: 10 additions & 0 deletions docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,16 @@ Two byte-exact fronts on the GB10 NVFP4 decode graph (`~/laguna-xs-nvfp4`, ids `

The 78 removed casts are ~78 µs/step — BELOW the ~0.3 ms nsys run-to-run noise (the dominant `enable_if` projection GEMV drifts ±0.4–0.7% between runs), so GPU-busy reads at parity; the win is the deterministic node-count drop (fewer graph nodes → less ramp/drain the CUDA graph doesn't hide), reflected as a neutral-to-+0.29% `decode_hp`. Lands default-ON on the same "shrink the captured node count, byte-exact" basis as the glue/preamble/addnorm folds (`=0` is a same-binary A/B opt-out). The remaining decode tail (projection GEMV ~55%, lm_head 7.4%, we-win Marlin MoE ~12%) is at/beyond parity; the load-time Marlin repack (`TransposeToInt32`/`gptq_marlin_repack`/`ProcessScales`, 20046 inst) is a one-time cost, correctly zeroed by the 2-length diff (the nsys-aggregate trap).

## DSR negative-guard polarity hardening (2026-08-01, `CLAIM-DSR-GUARD-POLARITY`) - NOT APPLICABLE

This CPU-only governance fix changes no runtime, model, kernel, workload, or
benchmark axis (`benchmark_binding=false`). The device-leakage checker now
classifies `#ifndef VLLM_CPP_CUDA` and `#if !defined(VLLM_CPP_CUDA)` as
portable branches and their `#else` arms as CUDA-only. Four RED-first mutations
failed in the exact inverse direction before the fix and pass afterward; the
full 32-case suite passes and the binding tree remains DSR 32. No GPU or
performance reproduction is applicable.

## GCC 12 production-library portability (2026-07-31, `CLAIM-CPU-GCC12-WERROR-PORTABILITY`) - NOT APPLICABLE / all-target build PARTIAL

This maintenance checkpoint changes no runtime algorithm or benchmark axis.
Expand Down
5 changes: 5 additions & 0 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,11 @@ without behavior or lifecycle changes. The production library and focused
tests build clean; the all-target build remains PARTIAL on two unrelated
test-only `-Wrestrict` diagnostics, recorded in [BENCHMARKS](BENCHMARKS.md).

CI governance (`ROAD-V1-A6`, 2026-08-01): the device-specific-reference
ratchet remains `DONE` at DSR 32, with no lifecycle or support change. Its
preprocessor classifier now handles negative CUDA guards and their `#else`
arms with the correct polarity; 32 CPU-only mutations gate the contract.

vllm.cpp implements an intentionally focused subset of vLLM, held to
token-for-token correctness against the pinned oracle.

Expand Down
61 changes: 49 additions & 12 deletions scripts/check-device-leakage.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,13 @@
# non-CUDA build. `VT_*` gates are CUDA kernel-feature macros, only ever defined
# by the CUDA build (cmake/CudaArchFeatures.cmake).
RE_CUDA_GUARD = re.compile(r"\b(VLLM_CPP_CUDA|VT_\w+|__CUDACC__|CUDA_VERSION)\b")
CUDA_GUARD_NAME = r"(?:VLLM_CPP_CUDA|VT_\w+|__CUDACC__|CUDA_VERSION)"
RE_DEFINED_CUDA_GUARD = re.compile(
rf"defined\s*(?:\(\s*)?{CUDA_GUARD_NAME}\s*\)?"
)
RE_NEGATED_DEFINED_CUDA_GUARD = re.compile(
rf"!\s*defined\s*(?:\(\s*)?{CUDA_GUARD_NAME}\s*\)?"
)

RE_DSR_ALLOW = re.compile(r"//\s*DSR-ALLOW\(\s*([A-Za-z0-9_.\-]+)\s*\)\s*:\s*(\S.*?)\s*$")

Expand Down Expand Up @@ -251,6 +258,39 @@ def strip_comments_and_strings(text: str) -> list[str]:
return "".join(out).splitlines()


def cuda_branch_polarity(kind: str, condition: str) -> int:
"""Classify a preprocessor branch as CUDA-only (1), portable (-1), or neutral."""
if not RE_CUDA_GUARD.search(condition):
return 0
if kind == "ifndef":
return -1

# When the expression contains only CUDA-family predicates, evaluate its
# truth in a portable build, where every such macro is undefined/false.
# False means the arm requires CUDA; true means its #else requires CUDA.
portable_expr = RE_DEFINED_CUDA_GUARD.sub("0", condition)
portable_expr = RE_CUDA_GUARD.sub("0", portable_expr)
if re.fullmatch(r"[\s01!&|()]+", portable_expr):
python_expr = portable_expr.replace("&&", " and ").replace("||", " or ")
python_expr = re.sub(r"!(?!=)", " not ", python_expr)
try:
portable_value = bool(eval(python_expr, {"__builtins__": {}}, {}))
except (SyntaxError, ValueError):
pass
else:
return -1 if portable_value else 1

# Unknown terms make disjunctions ambiguous, so count both arms
# conservatively. In a conjunction, an unnegated CUDA term still proves the
# arm cannot compile in a portable build.
if "||" in condition:
return 0
without_negated_guards = RE_NEGATED_DEFINED_CUDA_GUARD.sub("", condition)
if RE_CUDA_GUARD.search(without_negated_guards):
return 1
return 0


def cuda_guard_depth(raw_lines: list[str]) -> list[bool]:
"""Per line: is it inside a CUDA / VT_* preprocessor conditional?

Expand All @@ -259,31 +299,28 @@ def cuda_guard_depth(raw_lines: list[str]) -> list[bool]:
leakage.
"""
guarded: list[bool] = []
stack: list[bool] = []
stack: list[int] = []
for line in raw_lines:
m = RE_PP_IF.match(line)
if m:
stack.append(bool(RE_CUDA_GUARD.search(m.group(2))))
guarded.append(any(stack))
stack.append(cuda_branch_polarity(m.group(1), m.group(2)))
guarded.append(1 in stack)
continue
m = RE_PP_ELIF.match(line)
if m and stack:
stack[-1] = bool(RE_CUDA_GUARD.search(m.group(1)))
guarded.append(any(stack))
stack[-1] = cuda_branch_polarity("if", m.group(1))
guarded.append(1 in stack)
continue
if RE_PP_ELSE.match(line) and stack:
# The negative arm of a CUDA guard is the PORTABLE arm; a CUDA
# include there would be a genuine break, so stop treating it as
# guarded.
stack[-1] = False
guarded.append(any(stack))
stack[-1] = -stack[-1]
guarded.append(1 in stack)
continue
if RE_PP_ENDIF.match(line):
if stack:
stack.pop()
guarded.append(any(stack))
guarded.append(1 in stack)
continue
guarded.append(any(stack))
guarded.append(1 in stack)
return guarded


Expand Down
Loading
Loading