Skip to content

docs: rerun the README evidence snapshot against v0.12.1 - #561

Merged
wchwawa merged 3 commits into
mainfrom
docs/readme-evidence-v0.12.1
Oct 23, 2026
Merged

wchwawa merged 3 commits into
mainfrom
docs/readme-evidence-v0.12.1

Conversation

@wchwawa

@wchwawa wchwawa commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

TLDR

The README evidence snapshot still described commit 0f1995ebee96 and Holt 0.8.6. This PR reruns its rows against the 0.12.1 release and Holt 0.8.7, and marks the rows that were not rerun.

Problem

The snapshot's test counts and Holt pin came from a 0.11-era commit, and the README did not point to the 0.12.0 performance evidence. No issue: this is a release chore, which the policy exempts.

Consequences

A reader checking what the installed release has tested or qualified had to trust numbers from a different revision. One old row could not be reproduced at all.

Example

Before: "Rust workspace: 1,090 passed; 10 ignored" at 0f1995ebee96 with Holt 0.8.6. After: "1,224 passed, 0 failed; 13 ignored" at tag v0.12.1 (6be80629aa), the same counts Linux CI reports for that commit, with the command behind each rerun row listed under the table.

Solution

Every row that one laptop can run was rerun against v0.12.1 and Holt tag v0.8.7. The native-CLI live row needs etcd, an owner, and S3, and the agent-harness row needs the Pi harness, so both say they were not rerun. Where the README discusses performance, it now links the 0.12.0 release-check notes and raw evidence without adding numbers.

The rerun also corrected the Holt rows. The old "705 listed, 10 ignored" does not reproduce at Holt v0.8.6, which lists 706 tests with 8 ignored; the 10 matched a text search that also hit two comments. The old comparator build failure was libclang failing to load, plus a stale holt entry in the bench lockfile on Holt's release/0.8 line.

Gain

The snapshot matches the release users install, and each rerun row names its command so anyone can reproduce it.

Tradeoffs

This changes only the README; no row moves to a stronger status. The Holt comparator sweep was stopped after 34 measurements and claims no numbers, 0.12.1 performance was not re-measured, and only the macOS arm64 wheel was tested. The bench lockfile fix belongs in Holt and is not part of this PR.

Validation

Host: Apple M3 Pro, macOS 26.6, Rust 1.99.1 (the CI pin), Python 3.12.13.

  • cargo test --workspace --locked at v0.12.1: 1,224 passed, 0 failed, 13 ignored. Linux CI run 39593626381 on the same commit agrees. At v0.12.0: 1,220, 0, 13.
  • python3 scripts/workbench/workbench_contract_test.py: 8 tests OK.
  • Offline process checks on the built nokv: 15 of 15 passed. They cover help, version identity (0.12.1, 6be80629aa, Holt 0.8.7, 18 tools), schema against the frozen contract, rejected routing and arguments, and the absent mount command.
  • Release wheel nokv-0.12.1-cp39-abi3-macosx_11_0_arm64.whl: its SHA-256 matches the release sums, verify-install passed, and pytest -q crates/nokv-python/tests passed 34 tests.
  • Holt v0.8.7, cargo test --workspace --all-features --lib --tests --examples --locked: 713 listed, 705 passed, 8 ignored. Doctests: 1 passed, 3 ignored. The crates.io holt-0.8.7 sources equal the tag.
  • Holt cargo bench --manifest-path benches/Cargo.toml --bench main --locked: built after setting LIBCLANG_PATH and DYLD_FALLBACK_LIBRARY_PATH and refreshing the lockfile entry. It made 34 measurements with no errors before I stopped it.
  • Docs: git diff --check passed, all 69 README links resolve, and pandoc renders the 8-row table.
  • Not run: cargo fmt and cargo clippy (no Rust change); the live gates (they need etcd, RustFS, and owner processes; CI passed the bounded recovery gates on 6be80629aa); the Pi harness (the README links the 0.12.0 evidence).
  • DCO: the single commit carries a Signed-off-by trailer.

The snapshot still described commit 0f1995e and Holt 0.8.6. Rerun
its rows on 2026-10-21 against tag v0.12.1 (6be8062, workspace RPC v14,
System format 14) and Holt tag v0.8.7, list the command behind each rerun
row, and mark the native-CLI live row and the agent-harness performance
row as not rerun.

Link the 0.12.0 release-check evidence where the README discusses
performance, without adding numbers.

The old Holt row does not reproduce at Holt v0.8.6: Holt's PR-CI test
command lists 706 tests there, 8 of them ignored, and the "10 ignored"
matches a text search for "#[ignore" that also hits two comments.

Signed-off-by: wchwawa <wch19961116@gmail.com>
@feichai0017 feichai0017 moved this to In Progress in NoKV Delivery Oct 21, 2026
@wchwawa
wchwawa merged commit 7622127 into main Oct 23, 2026
10 checks passed
@wchwawa
wchwawa deleted the docs/readme-evidence-v0.12.1 branch October 23, 2026 07:25
@github-project-automation github-project-automation Bot moved this from In Progress to Done in NoKV Delivery Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants