Skip to content

fix: prepare 0.8.8 blob integrity checks - #81

Merged
wchwawa merged 3 commits into
release/0.8from
fix/blob-integrity-v087-20261026
Oct 26, 2026
Merged

wchwawa merged 3 commits into
release/0.8from
fix/blob-integrity-v087-20261026

Conversation

@wchwawa

@wchwawa wchwawa commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

TLDR

Prepare Holt 0.8.8 to reject damaged file-backed blob reads on NoKV's current dependency line.

Problem

A changed byte in an active blob could reach a reader without a storage error. This is a private security follow-up; no public issue was opened under SECURITY.md.

Consequences

A caller could consume a wrong value, and a reused slot could expose bytes beyond a compact blob's persisted extent.

Example

A read of a damaged page now returns NodeCorrupt. A range read beyond the authenticated extent returns zeros.

Solution

Bind per-page checksums to each blob slot, sync the checksum record before the manifest mapping, and verify reads before returning them. Version 6 stores migrate on writable open. Compact reads fetch only the authenticated prefix; large Linux batches verify independent blobs in parallel and preserve result order.

Gain

Persistent blob damage fails closed. Sparse blobs avoid full-slot reads, and batch verification no longer runs serially.

Tradeoffs

CRC detects accidental damage, not an attacker who can rewrite storage metadata. Existing damage at migration time cannot be authenticated. Version 6 read-only opens need a prior writable migration. Dense-frame checksums still have material cost, so performance acceptance remains open and this PR stays Draft. NoKV remains locked to published Holt 0.8.7 until 0.8.8 is released and separately integrated.

Validation

  • cargo test --workspace --locked -- --test-threads=2 passed on macOS before a final formatting-only change. Exact committed source passed macOS file-store tests 52/52 and Linux/arm64 Docker io-uring file-store tests 58/58, including a 40-blob damaged-page batch.
  • cargo fmt --all -- --check, macOS workspace Clippy, Linux io-uring Clippy, and git diff --check passed. cargo publish --dry-run --locked packaged and verified 128 files.
  • A published 0.8.7 writer's checkpointed store was migrated and read by 0.8.8; a 0.8.8 writer's store was read by the fixed 0.9.2 line. The reverse 0.9.2-to-0.8.8 WAL path is not qualified.
  • Matched 18-pair synthetic frame benchmarks with crc32fast 1.5.0: sparse full reads changed by -47.1% on macOS and -13.2% in Linux/arm64 Docker; dense full reads changed by +33.6% and +56.6%, respectively. Dense batch reads changed by +8.5% and +35.8%. These are fresh-open microbenchmarks, not NoKV product latency evidence. Linux used relaxed Docker seccomp for io-uring.
  • The existing CI workflow does not trigger on pull requests targeting release/0.8. The soak and benchmark lockfiles now match Holt 0.8.8, and both locked harness checks pass. A manual Nightly Validation run with 60-second soak/fuzz inputs passed on this exact HEAD (Verus was not enabled); it is not a full nightly or a PR check.

This is the 0.8-line backport of #80. No crate was published.

Signed-off-by: wchwawa <wch19961116@gmail.com>
Compact records only authenticate a prefix of a reused slot. Read that prefix, clear the rest, and verify independent blobs in parallel for large Linux batches. Keep the per-blob result order and reject damaged pages before callers receive data.

Signed-off-by: wchwawa <wch19961116@gmail.com>
@wchwawa wchwawa changed the title fix: release blob integrity checks in 0.8.8 fix: prepare 0.8.8 blob integrity checks Oct 26, 2026
The soak and benchmark harnesses use locked path dependencies. Update only their Holt package versions so release validation can run against this candidate without changing third-party packages.

Signed-off-by: wchwawa <wch19961116@gmail.com>
@wchwawa
wchwawa marked this pull request as ready for review October 26, 2026 22:54
@wchwawa
wchwawa merged commit bb69987 into release/0.8 Oct 26, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant