Repository navigation
docs: rerun the README evidence snapshot against v0.12.1 - #561
Merged
Merged
Conversation
The snapshot still described commit 0f1995e and Holt 0.8.6. Rerun its rows on 2026-10-21 against tag v0.12.1 (6be8062, workspace RPC v14, System format 14) and Holt tag v0.8.7, list the command behind each rerun row, and mark the native-CLI live row and the agent-harness performance row as not rerun. Link the 0.12.0 release-check evidence where the README discusses performance, without adding numbers. The old Holt row does not reproduce at Holt v0.8.6: Holt's PR-CI test command lists 706 tests there, 8 of them ignored, and the "10 ignored" matches a text search for "#[ignore" that also hits two comments. Signed-off-by: wchwawa <wch19961116@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
The README evidence snapshot still described commit
0f1995ebee96and Holt 0.8.6. This PR reruns its rows against the 0.12.1 release and Holt 0.8.7, and marks the rows that were not rerun.Problem
The snapshot's test counts and Holt pin came from a 0.11-era commit, and the README did not point to the 0.12.0 performance evidence. No issue: this is a release chore, which the policy exempts.
Consequences
A reader checking what the installed release has tested or qualified had to trust numbers from a different revision. One old row could not be reproduced at all.
Example
Before: "Rust workspace: 1,090 passed; 10 ignored" at
0f1995ebee96with Holt 0.8.6. After: "1,224 passed, 0 failed; 13 ignored" at tagv0.12.1(6be80629aa), the same counts Linux CI reports for that commit, with the command behind each rerun row listed under the table.Solution
Every row that one laptop can run was rerun against
v0.12.1and Holt tagv0.8.7. The native-CLI live row needs etcd, an owner, and S3, and the agent-harness row needs the Pi harness, so both say they were not rerun. Where the README discusses performance, it now links the 0.12.0 release-check notes and raw evidence without adding numbers.The rerun also corrected the Holt rows. The old "705 listed, 10 ignored" does not reproduce at Holt v0.8.6, which lists 706 tests with 8 ignored; the 10 matched a text search that also hit two comments. The old comparator build failure was libclang failing to load, plus a stale
holtentry in the bench lockfile on Holt'srelease/0.8line.Gain
The snapshot matches the release users install, and each rerun row names its command so anyone can reproduce it.
Tradeoffs
This changes only the README; no row moves to a stronger status. The Holt comparator sweep was stopped after 34 measurements and claims no numbers, 0.12.1 performance was not re-measured, and only the macOS arm64 wheel was tested. The bench lockfile fix belongs in Holt and is not part of this PR.
Validation
Host: Apple M3 Pro, macOS 26.6, Rust 1.99.1 (the CI pin), Python 3.12.13.
cargo test --workspace --lockedatv0.12.1: 1,224 passed, 0 failed, 13 ignored. Linux CI run 39593626381 on the same commit agrees. Atv0.12.0: 1,220, 0, 13.python3 scripts/workbench/workbench_contract_test.py: 8 tests OK.nokv: 15 of 15 passed. They cover help, version identity (0.12.1,6be80629aa, Holt 0.8.7, 18 tools),schemaagainst the frozen contract, rejected routing and arguments, and the absentmountcommand.nokv-0.12.1-cp39-abi3-macosx_11_0_arm64.whl: its SHA-256 matches the release sums,verify-installpassed, andpytest -q crates/nokv-python/testspassed 34 tests.v0.8.7,cargo test --workspace --all-features --lib --tests --examples --locked: 713 listed, 705 passed, 8 ignored. Doctests: 1 passed, 3 ignored. The crates.ioholt-0.8.7sources equal the tag.cargo bench --manifest-path benches/Cargo.toml --bench main --locked: built after settingLIBCLANG_PATHandDYLD_FALLBACK_LIBRARY_PATHand refreshing the lockfile entry. It made 34 measurements with no errors before I stopped it.git diff --checkpassed, all 69 README links resolve, andpandocrenders the 8-row table.cargo fmtandcargo clippy(no Rust change); the live gates (they need etcd, RustFS, and owner processes; CI passed the bounded recovery gates on6be80629aa); the Pi harness (the README links the 0.12.0 evidence).Signed-off-bytrailer.