Skip to content

ORE v2 (6/n): chained-prefix variable-length scheme + string encryption - #83

Merged
coderdan merged 11 commits into
feat/ore-v2-bit6from
feat/ore-v2-chained
Oct 12, 2026
Merged

coderdan merged 11 commits into
feat/ore-v2-bit6from
feat/ore-v2-chained

Conversation

@coderdan

Copy link
Copy Markdown
Contributor

Stacked on #82. Implements PR 6 of the ORE v2 program: the chained-prefix, variable-length scheme that enables string encryption (and lifts the fixed-N 14-block packed-prefix cap).

Design (preliminary crypto sign-off given; detailed review in progress)

  • Plan: docs/plans/2026-06-12-ore-v2-architecture.md §5(b)
  • A2 design spec: docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md — the precise, reviewed-against construction (injective encoding, security argument, open questions). (Both live in the base ORE v2 (5/n): 6-bit block scheme + v2 wire format #82 commits; this PR adds the implementation + a benchmark-results doc + the PR-6 status note.)

What

  • primitives/cmac.rs — incremental AES-CMAC accumulator (NIST SP 800-38B). absorb (CBC prefix) / finalize (E_k(S⊕F⊕K1)); injective prefix/final block encoders; K1 reuses the vectorized gf128_double. Validated against the NIST AES-128 CMAC test vectors.
  • LemireFyPrp::from_stream (shape ii) — PRP keyed directly from the accumulator's PRP_STREAM branch, so there is no per-block AES key schedule. new() was refactored to call it, keeping the Bit6 vectors byte-identical.
  • scheme/chained.rs — OreAes128Bit6Chained: encrypt_var / encrypt_left_var / encrypt_str / encrypt_left_str; Vec-backed Var{Left,Right,CipherText} with the v2 header (scheme id 0x03, u16 block count); compare_raw_slices doing a constant-time prefix scan with lexicographic semantics (shorter sorts first), reusing the BHKR σ-MMO H and the oblivious compare read.
    • One secret key (the accumulator key, derived from k1 by a labelled AES call), subsuming the fixed-N prf1/prf2 via branch tags.
    • No total-length binding in the derivation, so prefix-sharing strings of different lengths compare correctly (spec §7); length comparability is enforced at the comparator.

Tests

Cross-length lexicographic order vs str::cmp (incl. shared prefixes and >14-block strings), equality across nonces, serialize round-trip, empty string sorts first, cross-scheme byte rejection; plus the CMAC NIST KAT and accumulator unit tests. All local CI gates green (fmt, clippy -D warnings, full test).

Performance (Apple M1 Max — docs/benchmarks/2026-06-15-chained-results.md)

op input time
encrypt "alice" (7 blocks) ~5.05 µs
encrypt "alice@example.com" (23 blocks) ~15.97 µs
encrypt 43-char sentence (58 blocks) ~40.18 µs
encrypt-left 17 chars ~10.09 µs
compare 17 chars ~402 ns

~0.69 µs/block, scaling linearly — and below fixed-N Bit6's ~0.81 µs/block, because the accumulator is keyed once per ciphertext (the shape-(ii) no-key-schedule win, A3).

Not in this PR

  • u128/i128/Decimal via the const-N + accumulator path (fixed-length, >14 blocks).
  • Pinned wire vectors — await final A2 sign-off (as Bit6 vectors awaited A1).
  • chrono Bit6 impls (independent follow-up).

Review notes

The A2 spec has 5 open questions (§11) — key-derivation slot, width-vs-scheme-id tag, shape-(ii) confirmation, birthday ceiling, u128/Decimal comparability. None block the working implementation; they shape the final freeze.

coderdan added a commit that referenced this pull request Jun 16, 2026
…nit, left-query comparator (#83 review)

Review follow-ups on the chained scheme:

- zeroize: wipe L = E_k(0) in CmacAccumulator::new (K1's source; spec section 9)
  and the per-block `ro` RO-tag buffer in encrypt_var (matches the bit2_w6
  scratch wipe). Existing Drops (k1/state/k_acc/stream) were already correct.
- Replace the BRANCH_* u8 consts + a would-be debug_assert with a non-zero
  enum Branch { RoKey = 1, PrpStream = 2 }; final_block takes Branch, so a final
  block's byte 0 can never be 0x00 and collide with a prefix block — the
  injectivity separator is now a compile-time guarantee.
- count > u16::MAX returns OreError::TooManyBlocks (new variant) in both
  encrypt_var and encrypt_left_var, instead of a debug_assert.
- Single-key init: the chained scheme is single-key by design (the CMAC
  accumulator derives every per-block secret from one key via branch tags), so
  drop the vestigial _k2 parameter — init(k1). (Bit6 still uses two PRF keys.)
- Add compare_left_to_full: the Lewi-Wu query path. The comparator core is
  factored into compare_views(left_view, full); compare_raw_slices (full,full)
  and compare_left_to_full (left,full) both call it, so the left-only artifact
  from encrypt_left_* is usable as a query against a stored ciphertext. Adds
  prop_left_query_matches_full + an artifact-rejection unit test.

No wire-format change.
@coderdan

Copy link
Copy Markdown
Contributor Author

Review: code / constant-time / zeroization

Ran the multi-agent code review plus the Trail of Bits constant-time-testing and zeroize-audit skills. The chained scheme is functionally correct (comparator, injective encoding, accumulator threading, wire format all verified, with property tests as the oracle). Findings and fixes (applied in f498990):

Applied

  • Left-vs-right query path (was missing): compare_raw_slices required both operands to be full ciphertexts, so the encrypt_left_* artifact was uncomparable. Added compare_left_to_full(left, full) — the Lewi-Wu query path — by factoring the comparator core into compare_views(left_view, full); both entry points share it. New prop_left_query_matches_full + artifact-rejection test.
  • Single-key init: the chained scheme is single-key by design (the CMAC accumulator derives every per-block secret from one key via branch tags), so the vestigial _k2 param is gone — init(k1). (Bit6 still uses two PRF keys; the differing key arity between fixed and chained is being discussed separately.)
  • count > u16::MAX now returns OreError::TooManyBlocks (both encrypt paths), not a debug_assert.
  • final_block branch is now a non-zero enum Branch (compile-time guarantee that a final block's byte 0 ≠ 0x00, so it can never collide with a prefix block) — replacing a would-be debug_assert.

Constant-time

Inherits Bit6's posture: CT-1 fixed (from_stream is fixed-draw), CT-2/CT-3 mitigated to MemJam sub-line (align-64, N=64), comparator prefix scan is constant-time; CMAC adds no data-dependent control flow. Two carry-overs: the post-ct_select_byte bit extract still uses a secret-amount shift (resolves on rebase onto #82's ct_bit), and the l-indexed block fetch leaks the common-prefix length (in-model online leakage per plan §5b).

Zeroization

Two new gaps found and fixed: L = E_k(0) in CmacAccumulator::new (K1's source) and the per-block ro RO-tag buffer in encrypt_var are now zeroized. All existing Drops (k1/state/k_acc/stream, from_stream) were already correct. Only other finding is the pre-existing ZA-0001 (fixed on #84). Full write-up: docs/reviews/2026-06-16-zeroize-audit-pr83.md.

Open (not blocking)

No pinned wire vectors yet (deliberately deferred to A2 sign-off — the freeze gate); a few low test-gaps noted in the review.

coderdan added a commit that referenced this pull request Jun 16, 2026
…nit, left-query comparator (#83 review)

Review follow-ups on the chained scheme:

- zeroize: wipe L = E_k(0) in CmacAccumulator::new (K1's source; spec section 9)
  and the per-block `ro` RO-tag buffer in encrypt_var (matches the bit2_w6
  scratch wipe). Existing Drops (k1/state/k_acc/stream) were already correct.
- Replace the BRANCH_* u8 consts + a would-be debug_assert with a non-zero
  enum Branch { RoKey = 1, PrpStream = 2 }; final_block takes Branch, so a final
  block's byte 0 can never be 0x00 and collide with a prefix block — the
  injectivity separator is now a compile-time guarantee.
- count > u16::MAX returns OreError::TooManyBlocks (new variant) in both
  encrypt_var and encrypt_left_var, instead of a debug_assert.
- Single-key init: the chained scheme is single-key by design (the CMAC
  accumulator derives every per-block secret from one key via branch tags), so
  drop the vestigial _k2 parameter — init(k1). (Bit6 still uses two PRF keys.)
- Add compare_left_to_full: the Lewi-Wu query path. The comparator core is
  factored into compare_views(left_view, full); compare_raw_slices (full,full)
  and compare_left_to_full (left,full) both call it, so the left-only artifact
  from encrypt_left_* is usable as a query against a stored ciphertext. Adds
  prop_left_query_matches_full + an artifact-rejection unit test.

No wire-format change.
@coderdan
coderdan force-pushed the feat/ore-v2-chained branch from f498990 to 7ab2529 Compare June 16, 2026 10:02
coderdan added a commit that referenced this pull request Oct 8, 2026
…nit, left-query comparator (#83 review)

Review follow-ups on the chained scheme:

- zeroize: wipe L = E_k(0) in CmacAccumulator::new (K1's source; spec section 9)
  and the per-block `ro` RO-tag buffer in encrypt_var (matches the bit2_w6
  scratch wipe). Existing Drops (k1/state/k_acc/stream) were already correct.
- Replace the BRANCH_* u8 consts + a would-be debug_assert with a non-zero
  enum Branch { RoKey = 1, PrpStream = 2 }; final_block takes Branch, so a final
  block's byte 0 can never be 0x00 and collide with a prefix block — the
  injectivity separator is now a compile-time guarantee.
- count > u16::MAX returns OreError::TooManyBlocks (new variant) in both
  encrypt_var and encrypt_left_var, instead of a debug_assert.
- Single-key init: the chained scheme is single-key by design (the CMAC
  accumulator derives every per-block secret from one key via branch tags), so
  drop the vestigial _k2 parameter — init(k1). (Bit6 still uses two PRF keys.)
- Add compare_left_to_full: the Lewi-Wu query path. The comparator core is
  factored into compare_views(left_view, full); compare_raw_slices (full,full)
  and compare_left_to_full (left,full) both call it, so the left-only artifact
  from encrypt_left_* is usable as a query against a stored ciphertext. Adds
  prop_left_query_matches_full + an artifact-rejection unit test.

No wire-format change.
@coderdan
coderdan force-pushed the feat/ore-v2-chained branch from 7ab2529 to 0316c0c Compare October 8, 2026 08:36
@coderdan
coderdan marked this pull request as ready for review October 8, 2026 08:51
coderdan added a commit that referenced this pull request Oct 8, 2026
…nit, left-query comparator (#83 review)

Review follow-ups on the chained scheme:

- zeroize: wipe L = E_k(0) in CmacAccumulator::new (K1's source; spec section 9)
  and the per-block `ro` RO-tag buffer in encrypt_var (matches the bit2_w6
  scratch wipe). Existing Drops (k1/state/k_acc/stream) were already correct.
- Replace the BRANCH_* u8 consts + a would-be debug_assert with a non-zero
  enum Branch { RoKey = 1, PrpStream = 2 }; final_block takes Branch, so a final
  block's byte 0 can never be 0x00 and collide with a prefix block — the
  injectivity separator is now a compile-time guarantee.
- count > u16::MAX returns OreError::TooManyBlocks (new variant) in both
  encrypt_var and encrypt_left_var, instead of a debug_assert.
- Single-key init: the chained scheme is single-key by design (the CMAC
  accumulator derives every per-block secret from one key via branch tags), so
  drop the vestigial _k2 parameter — init(k1). (Bit6 still uses two PRF keys.)
- Add compare_left_to_full: the Lewi-Wu query path. The comparator core is
  factored into compare_views(left_view, full); compare_raw_slices (full,full)
  and compare_left_to_full (left,full) both call it, so the left-only artifact
  from encrypt_left_* is usable as a query against a stored ciphertext. Adds
  prop_left_query_matches_full + an artifact-rejection unit test.

No wire-format change.
@coderdan
coderdan force-pushed the feat/ore-v2-chained branch from 0316c0c to 778d5c1 Compare October 8, 2026 09:05
coderdan added a commit that referenced this pull request Oct 8, 2026
…nit, left-query comparator (#83 review)

Review follow-ups on the chained scheme:

- zeroize: wipe L = E_k(0) in CmacAccumulator::new (K1's source; spec section 9)
  and the per-block `ro` RO-tag buffer in encrypt_var (matches the bit2_w6
  scratch wipe). Existing Drops (k1/state/k_acc/stream) were already correct.
- Replace the BRANCH_* u8 consts + a would-be debug_assert with a non-zero
  enum Branch { RoKey = 1, PrpStream = 2 }; final_block takes Branch, so a final
  block's byte 0 can never be 0x00 and collide with a prefix block — the
  injectivity separator is now a compile-time guarantee.
- count > u16::MAX returns OreError::TooManyBlocks (new variant) in both
  encrypt_var and encrypt_left_var, instead of a debug_assert.
- Single-key init: the chained scheme is single-key by design (the CMAC
  accumulator derives every per-block secret from one key via branch tags), so
  drop the vestigial _k2 parameter — init(k1). (Bit6 still uses two PRF keys.)
- Add compare_left_to_full: the Lewi-Wu query path. The comparator core is
  factored into compare_views(left_view, full); compare_raw_slices (full,full)
  and compare_left_to_full (left,full) both call it, so the left-only artifact
  from encrypt_left_* is usable as a query against a stored ciphertext. Adds
  prop_left_query_matches_full + an artifact-rejection unit test.

No wire-format change.
coderdan added a commit that referenced this pull request Oct 8, 2026
absorb and finalize each built an owned value from the chain state (and,
in finalize, K1) and returned without wiping it. Both now hold it as a
byte array and zeroize it after the AES call. encrypt takes its block by
value and encrypts it in place, so that copy holds only the output.
Raised in the review of #90 against the #83 zeroize audit.
@coderdan
coderdan force-pushed the feat/ore-v2-chained branch from 778d5c1 to 80f2db2 Compare October 8, 2026 09:52
@coderdan
coderdan requested a balanced review from Copilot October 8, 2026 10:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The storage/query API does not meet the documented at-rest security model.

5 open findings
What changed in this PR

Extends the fixed-length Bit6 ORE foundation with CMAC-based chained prefixes, enabling variable-length string encryption beyond the 14-block cap.

Changes:

  • Adds incremental AES-CMAC and stream-based permutation construction.
  • Adds variable-length ciphertexts, serialization, and lexicographic comparison.
  • Adds correctness tests, string benchmarks, and implementation-status documentation.
File Description
packages/​ore-rs/​src/​scheme/​width.rs Removes the bit-selection helper.
packages/​ore-rs/​src/​scheme/​chained.rs Adds chained encryption, comparison, serialization, and tests.
packages/​ore-rs/​src/​scheme/​bit2/​block_types.rs Uses direct bit-position shifts.
packages/​ore-rs/​src/​scheme/​bit2.rs Adjusts encoding helpers, bit extraction, and Debug.
packages/​ore-rs/​src/​scheme/​bit2_w6/​block_types.rs Uses direct bit-position shifts.
packages/​ore-rs/​src/​scheme/​bit2_w6.rs Adjusts encoding, comparison validation, and Debug.
packages/​ore-rs/​src/​scheme.rs Exposes the chained scheme.
packages/​ore-rs/​src/​primitives/​prp.rs Adds permutation construction from a supplied stream.
packages/​ore-rs/​src/​primitives/​hash.rs Shares field doubling with CMAC.
packages/​ore-rs/​src/​primitives/​cmac.rs Adds the CMAC accumulator and validation tests.
packages/​ore-rs/​src/​primitives.rs Exposes the CMAC module.
packages/​ore-rs/​src/​lib.rs Adds the block-count overflow error.
packages/​ore-rs/​Cargo.toml Registers chained benchmarks.
packages/​ore-rs/​benches/​chained.rs Benchmarks string encryption and comparison.
docs/​plans/​2026-06-12-ore-v2-architecture.md Records implementation status and remaining scope.
docs/​benchmarks/​2026-06-15-chained-results.md Documents performance results and caveats.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread packages/ore-rs/src/scheme/chained.rs Outdated
Comment thread packages/ore-rs/src/scheme/bit2.rs Outdated
Comment thread packages/ore-rs/src/scheme/bit2_w6.rs Outdated
Comment thread packages/ore-rs/src/scheme/chained.rs Outdated
Comment thread packages/ore-rs/src/scheme/chained.rs Outdated
coderdan added a commit that referenced this pull request Oct 8, 2026
…nit, left-query comparator (#83 review)

Review follow-ups on the chained scheme:

- zeroize: wipe L = E_k(0) in CmacAccumulator::new (K1's source; spec section 9)
  and the per-block `ro` RO-tag buffer in encrypt_var (matches the bit2_w6
  scratch wipe). Existing Drops (k1/state/k_acc/stream) were already correct.
- Replace the BRANCH_* u8 consts + a would-be debug_assert with a non-zero
  enum Branch { RoKey = 1, PrpStream = 2 }; final_block takes Branch, so a final
  block's byte 0 can never be 0x00 and collide with a prefix block — the
  injectivity separator is now a compile-time guarantee.
- count > u16::MAX returns OreError::TooManyBlocks (new variant) in both
  encrypt_var and encrypt_left_var, instead of a debug_assert.
- Single-key init: the chained scheme is single-key by design (the CMAC
  accumulator derives every per-block secret from one key via branch tags), so
  drop the vestigial _k2 parameter — init(k1). (Bit6 still uses two PRF keys.)
- Add compare_left_to_full: the Lewi-Wu query path. The comparator core is
  factored into compare_views(left_view, full); compare_raw_slices (full,full)
  and compare_left_to_full (left,full) both call it, so the left-only artifact
  from encrypt_left_* is usable as a query against a stored ciphertext. Adds
  prop_left_query_matches_full + an artifact-rejection unit test.

No wire-format change.
coderdan added a commit that referenced this pull request Oct 8, 2026
absorb and finalize each built an owned value from the chain state (and,
in finalize, K1) and returned without wiping it. Both now hold it as a
byte array and zeroize it after the AES call. encrypt takes its block by
value and encrypts it in place, so that copy holds only the output.
Raised in the review of #90 against the #83 zeroize audit.
coderdan added a commit that referenced this pull request Oct 8, 2026
…mbol checks; correct at-rest leakage docs

Review of #83:
- encrypt_var read the left tag from a 1 KiB table at the secret
  permuted symbol, which spans many cache lines. It now derives the tag
  with acc.finalize at that symbol, exactly as encrypt_left_var does;
  the existing left/full consistency property pins identical bytes.
- The comparator shifted by the secret bit position; it now uses
  width::ct_bit, as the Bit6 and legacy comparators do.
- encrypt_str / encrypt_left_str allocated and decomposed the whole
  input before the block-count check. They now refuse more than
  MAX_STR_BYTES (49,151 UTF-8 bytes, the largest whose block count fits
  the u16) up front; const asserts pin the boundary.
- Not raised by the review, same class as the Bit6 finding: both
  comparator entry points now reject xt symbols outside 0..64, which
  otherwise select no right-block byte and read 0. The regression test
  fails without the check.
- The docs promised right-only storage that reveals nothing at rest. No
  right-only path exists: stored full ciphertexts carry their left half,
  so two stored rows compare offline (order, common-prefix length), as
  legacy rows do. The module docs, compare_left_to_full and plan section
  5(b)/9 now say so; right-only storage is a separate design item.
- Plan section 9: chained key separation confirmed (its only key is
  AES_k1("ORE.v2.chain.acc"), outside legacy inputs and distinct from
  Bit6's labels).
@coderdan
coderdan force-pushed the feat/ore-v2-chained branch from 80f2db2 to 893121d Compare October 8, 2026 11:04
coderdan added a commit that referenced this pull request Oct 9, 2026
…nit, left-query comparator (#83 review)

Review follow-ups on the chained scheme:

- zeroize: wipe L = E_k(0) in CmacAccumulator::new (K1's source; spec section 9)
  and the per-block `ro` RO-tag buffer in encrypt_var (matches the bit2_w6
  scratch wipe). Existing Drops (k1/state/k_acc/stream) were already correct.
- Replace the BRANCH_* u8 consts + a would-be debug_assert with a non-zero
  enum Branch { RoKey = 1, PrpStream = 2 }; final_block takes Branch, so a final
  block's byte 0 can never be 0x00 and collide with a prefix block — the
  injectivity separator is now a compile-time guarantee.
- count > u16::MAX returns OreError::TooManyBlocks (new variant) in both
  encrypt_var and encrypt_left_var, instead of a debug_assert.
- Single-key init: the chained scheme is single-key by design (the CMAC
  accumulator derives every per-block secret from one key via branch tags), so
  drop the vestigial _k2 parameter — init(k1). (Bit6 still uses two PRF keys.)
- Add compare_left_to_full: the Lewi-Wu query path. The comparator core is
  factored into compare_views(left_view, full); compare_raw_slices (full,full)
  and compare_left_to_full (left,full) both call it, so the left-only artifact
  from encrypt_left_* is usable as a query against a stored ciphertext. Adds
  prop_left_query_matches_full + an artifact-rejection unit test.

No wire-format change.
coderdan added a commit that referenced this pull request Oct 9, 2026
…mbol checks; correct at-rest leakage docs

Review of #83:
- encrypt_var read the left tag from a 1 KiB table at the secret
  permuted symbol, which spans many cache lines. It now derives the tag
  with acc.finalize at that symbol, exactly as encrypt_left_var does;
  the existing left/full consistency property pins identical bytes.
- The comparator shifted by the secret bit position; it now uses
  width::ct_bit, as the Bit6 and legacy comparators do.
- The comparator's resolution step loaded xt[l], f[l] and right[l] by
  the latched first-difference index. The scan never touches the right
  blocks, so the right[l] load hit or missed the cache according to l;
  dudect measured it at 2 % of the timing spread on a 23-block
  ciphertext (recorded under docs/reviews). The scan now latches all
  three under the choice "this is the first differing block"
  (width::ct_assign_bytes), the same change made to the Bit6
  comparator, so no load after the scan is indexed by l.
- encrypt_str / encrypt_left_str allocated and decomposed the whole
  input before the block-count check. They now refuse more than
  MAX_STR_BYTES (49,151 UTF-8 bytes, the largest whose block count fits
  the u16) up front; const asserts pin the boundary.
- Not raised by the review, same class as the Bit6 finding: both
  comparator entry points now reject xt symbols outside 0..64, which
  otherwise select no right-block byte and read 0. The regression test
  fails without the check.
- The docs promised right-only storage that reveals nothing at rest. No
  right-only path exists: stored full ciphertexts carry their left half,
  so two stored rows compare offline (order, common-prefix length), as
  legacy rows do. The module docs, compare_left_to_full and plan section
  5(b)/9 now say so; right-only storage is a separate design item.
- Plan section 9: chained key separation confirmed (its only key is
  AES_k1("ORE.v2.chain.acc"), outside legacy inputs and distinct from
  Bit6's labels).
coderdan added a commit that referenced this pull request Oct 9, 2026
…uilder, oblivious builder

Records the same-machine bisect of the #83 review hardening (per-block ro
wipe +19%, per-call mixed wipe +45% more), the fixed figures, and the
chained numbers for the rejected sort-by-random-key PRP builder.

Adds the oblivious PRP builder that from_stream now uses, on mains
power: 7 to 10% faster than the indexed builder on every chained encrypt
with NEON, 66 to 95% slower on the SWAR fallback, output unchanged.
@coderdan
coderdan force-pushed the feat/ore-v2-chained branch from 61d78f6 to d12897e Compare October 9, 2026 10:22
@coderdan

coderdan commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Force-pushed (with lease), docs only: A2 and A3 are recorded as signed off (2026-10-09). The review brief's decision table and Gate 2 checklist are updated, and the architecture plan's PR 6 status no longer says "preliminary sign-off" or that wire vectors await it (they are pinned in tests/compat_chained_vectors.rs). Folded into 13f4fb9 (docs: chained scheme benchmark results + mark PR 6 implemented). No code change.

@freshtonic freshtonic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security review of this layer. Two findings:

  1. Medium: every chained ciphertext shows the exact UTF-8 byte length of its plaintext. The leakage profile does not say so.
  2. Low: the 6-bit plaintext copy in str_to_blocks is freed without zeroize. #100 does not fix this.

I checked these items and found no problem: incremental CMAC as a prefix PRF (injective encoding, K1 only, no length extension), branch-free gf128_double, cross-scheme label collisions, string ordering (for example "a" vs "a\0"), parse length checks, and the comparator latch.

🤖 Generated with Claude Code

Comment thread packages/ore-rs/src/scheme/chained.rs
Comment thread packages/ore-rs/src/scheme/chained.rs Outdated

@freshtonic freshtonic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after the round-2 security re-review. The round-1 findings for this PR are fixed or have no remaining blocking items.

🤖 Generated with Claude Code

…PR 6 foundation)

- primitives/cmac.rs: incremental AES-CMAC (NIST SP 800-38B) accumulator per
  the A2 spec — absorb (CBC prefix) / finalize (E_k(S^F^K1)); prefix/final
  block encoders with the injective byte layout; K1 reuses gf128_double.
  Validated against the NIST AES-128 CMAC test vectors.
- prp.rs: factor LemireFyPrp::from_stream (shape ii) out of new(); new() now
  calls it, so Bit6 vectors stay byte-identical. Lets the chained scheme key
  the PRP from the accumulator stream with no per-block AES key schedule.
  from_stream builds through the oblivious builder (NEON on aarch64, SSSE3
  on x86_64 when present, SWAR elsewhere), so the chained scheme's
  PRP_STREAM path, like Bit6's, uses no secret value as an address.
- hash.rs: gf128_double_u128 -> pub(crate) (shared by H and CMAC subkeys).
…on (PR 6)

OreAes128Bit6Chained: variable-length BlockORE over 6-bit blocks, driving
the AES-CMAC accumulator (A2 spec). Per block: PRP keystream from the
accumulator's PRP_STREAM branch via LemireFyPrp::from_stream (shape ii, no
per-block key schedule); ro_keys from the RO_KEY branch; left tag
f[n]=ro(n,xt[n]); right block = H-mask XOR indicator (reuses the BHKR
sigma-MMO H + oblivious compare read). Vec-backed Var{Left,Right,CipherText}
with the v2 header (scheme id 0x03, u16 block count).

- encrypt_var / encrypt_left_var / encrypt_str / encrypt_left_str.
- compare_raw_slices: constant-time prefix scan, lexicographic (shorter
  sorts first); rejects mismatched scheme/version/length.
- No total-length binding in the derivation, so prefix-sharing strings of
  different lengths compare correctly (spec section 7).
- Tests: cross-length lexicographic order, equality across nonces, >14-block
  plaintexts, roundtrip, empty string, cross-scheme rejection.
- benches/chained.rs.

Perf (M1 Max): ~0.69 us/block (encrypt), below fixed-N Bit6's ~0.81 us/block
-- the shape-ii no-key-schedule win. compare ~402ns.
Add docs/benchmarks/2026-06-15-chained-results.md (~0.69 us/block, below
fixed-N Bit6's ~0.81 -- the shape-ii no-key-schedule win) and a PR 6 status
note in the plan roadmap.
…cy, and CMAC accumulator

Adds quickcheck properties matching the crate's existing idiom:

scheme/chained.rs (6 props, lengths capped at 32 blocks so most cases
exceed the 14-block fixed-N cap):
- block-level order == blocks.cmp over the full 6-bit domain
- shared-prefix order (drives the constant-time prefix scan)
- string order == str::cmp across arbitrary Unicode and lengths
- equality-across-nonces with byte divergence
- left ct is deterministic and byte-identical to the full ct's left half
- serialize round-trip

primitives/cmac.rs (3 props):
- incremental absorb/finalize == from-scratch CMAC for any block count
  (NIST vectors only cover t in {1,4})
- final/prefix block encodings are injective (tuple recoverable)

All pass at 4000 iterations each; clippy -D warnings and fmt clean.
…nit, left-query comparator (#83 review)

Review follow-ups on the chained scheme:

- zeroize: wipe L = E_k(0) in CmacAccumulator::new (K1's source; spec section 9)
  and the per-block `ro` RO-tag buffer in encrypt_var (matches the bit2_w6
  scratch wipe). Existing Drops (k1/state/k_acc/stream) were already correct.
- Replace the BRANCH_* u8 consts + a would-be debug_assert with a non-zero
  enum Branch { RoKey = 1, PrpStream = 2 }; final_block takes Branch, so a final
  block's byte 0 can never be 0x00 and collide with a prefix block — the
  injectivity separator is now a compile-time guarantee.
- count > u16::MAX returns OreError::TooManyBlocks (new variant) in both
  encrypt_var and encrypt_left_var, instead of a debug_assert.
- Single-key init: the chained scheme is single-key by design (the CMAC
  accumulator derives every per-block secret from one key via branch tags), so
  drop the vestigial _k2 parameter — init(k1). (Bit6 still uses two PRF keys.)
- Add compare_left_to_full: the Lewi-Wu query path. The comparator core is
  factored into compare_views(left_view, full); compare_raw_slices (full,full)
  and compare_left_to_full (left,full) both call it, so the left-only artifact
  from encrypt_left_* is usable as a query against a stored ciphertext. Adds
  prop_left_query_matches_full + an artifact-rejection unit test.

No wire-format change.
Review of #90: absorb and finalize built an owned `mixed` value from the
chain state (and, in finalize, K1) and handed a by-value copy to the
cipher, leaving the plaintext input on the stack. Encrypting `mixed` in
place closes it without a temporary: the one buffer that held the input
holds the new state or the tag afterwards. A per-call zeroize was tried
first and cost 45% on chained encryption (about 97 calls per block, each
a volatile store and a fence that stops the AES ops pipelining); the
in-place form costs nothing.
…mbol checks; correct at-rest leakage docs

Review of #83:
- encrypt_var read the left tag from a 1 KiB table at the secret
  permuted symbol, which spans many cache lines. It now derives the tag
  with acc.finalize at that symbol, exactly as encrypt_left_var does;
  the existing left/full consistency property pins identical bytes.
- The comparator shifted by the secret bit position; it now uses
  width::ct_bit, as the Bit6 and legacy comparators do.
- The comparator's resolution step loaded xt[l], f[l] and right[l] by
  the latched first-difference index. The scan never touches the right
  blocks, so the right[l] load hit or missed the cache according to l;
  dudect measured it at 2 % of the timing spread on a 23-block
  ciphertext (recorded under docs/reviews). The scan now latches all
  three under the choice "this is the first differing block"
  (width::ct_assign_bytes), the same change made to the Bit6
  comparator, so no load after the scan is indexed by l.
- encrypt_str / encrypt_left_str allocated and decomposed the whole
  input before the block-count check. They now refuse more than
  MAX_STR_BYTES (49,151 UTF-8 bytes, the largest whose block count fits
  the u16) up front; const asserts pin the boundary.
- Not raised by the review, same class as the Bit6 finding: both
  comparator entry points now reject xt symbols outside 0..64, which
  otherwise select no right-block byte and read 0. The regression test
  fails without the check.
- The docs promised right-only storage that reveals nothing at rest. No
  right-only path exists: stored full ciphertexts carry their left half,
  so two stored rows compare offline (order, common-prefix length), as
  legacy rows do. The module docs, compare_left_to_full and plan section
  5(b)/9 now say so; right-only storage is a separate design item.
- Plan section 9: chained key separation confirmed (its only key is
  AES_k1("ORE.v2.chain.acc"), outside legacy inputs and distinct from
  Bit6's labels).
…uilder, oblivious builder

Records the same-machine bisect of the #83 review hardening (per-block ro
wipe +19%, per-call mixed wipe +45% more), the fixed figures, and the
chained numbers for the rejected sort-by-random-key PRP builder.

Adds the oblivious PRP builder that from_stream now uses, on mains
power: 7 to 10% faster than the indexed builder on every chained encrypt
with NEON, 66 to 95% slower on the SWAR fallback, output unchanged.
Known-answer vectors for the chained scheme, mirroring
compat_w6_vectors: left and full ciphertexts for fixed strings and raw
symbol sequences under a fixed key and a deterministic nonce stream, the
alternate-nonce case, and comparison results over the pinned bytes for
both comparators, including malformed inputs. Until now a change to the
chained bytes was caught by nothing (the sort-builder prototype only
failed the Bit6 vectors).
…mposed string

The chained scheme builds a PRP per block through
LemireFyPrp::from_stream and runs a CMAC accumulator, and both leave
secrets in dead stack after encrypt returns (the defect the #82 review
found in LemireFyPrp::new). encrypt_var, encrypt_left_var, encrypt_str
and encrypt_left_str now run inside stack::scrubbed, as the Bit6 entry
points do. CmacAccumulator and the key derivation in init build
encryption-only AES schedules.

str_to_blocks returned the 6-bit decomposition of the string in a plain
Vec, freed without a wipe. The decomposition inverts, so the freed heap
chunk held the whole plaintext. It is now Zeroizing<Vec<u8>>.

New tests encrypt a string on a fresh thread and search the dead stack
for every block's permutation tables: found without the scrub, absent
with it, in release and debug.

Cost on an M1 Max: 1-4% (chained-encrypt-str-17 20.45 -> 20.99 us).
The header stores the block count, and the ciphertext size follows from
it. For a string of n UTF-8 bytes the count is ceil(8n/6), strictly
increasing in n, so one ciphertext (full, left, or a right half alone)
gives n exactly, with no comparison or query. The module docs, plan
§5(b) and the leakage checklist listed only order and common-prefix
length, and §5(b) said a right ciphertext alone reveals nothing.

Add exact length to the documented leakage profile on the string API,
the module docs and plan §5(b). Hiding it needs padding with a pad
symbol below every real symbol (a 65-value domain and a wire-format
change); that is tracked in #104.
@coderdan
coderdan force-pushed the feat/ore-v2-chained branch from 879ad1e to e05bc57 Compare October 12, 2026 05:19
@coderdan
coderdan merged commit 463e5fa into main Oct 12, 2026
2 checks passed
@coderdan
coderdan deleted the feat/ore-v2-chained branch October 12, 2026 08:34
coderdan added a commit that referenced this pull request Oct 12, 2026
- The MMO validation claimed a pl/pgSQL comparator was validated; only
  the σ-MMO hash was implemented and checked (12 vectors). The doc now
  scopes its result to the hash and lists what a comparator still needs
  end-to-end vectors for.
- "LSB" in both the SQL and Rust is bit 0 of byte 0 of the block (bit
  120 of the big-endian field element), as in the frozen legacy hash.
  The SQL and summary now name that coordinate.
- The #82 and #83 zeroize audits marked items clean that left
  key-derived temporaries unwiped (from_stream's per-draw copy; the CMAC
  mixed values). Both audits are amended, pointing at the fixes on #82
  and #83, and say plainly that the wipes are source-level only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants