diff --git a/docs/benchmarks/2026-06-15-chained-results.md b/docs/benchmarks/2026-06-15-chained-results.md new file mode 100644 index 00000000..7a952a53 --- /dev/null +++ b/docs/benchmarks/2026-06-15-chained-results.md @@ -0,0 +1,96 @@ +# Chained variable-length scheme — benchmark results + +**Date:** 2026-06-15 +**Machine:** Apple M1 Max, hardware AES (`aes_armv8`) +**Scheme:** `OreAes128Bit6Chained` (PR 6) — 6-bit blocks, AES-CMAC accumulator, +shape-(ii) PRP (keystream from the accumulator, no per-block key schedule). +**Command:** `cargo bench -p ore-rs --bench chained` + +| benchmark | input | blocks | time | +|---|---|---:|---:| +| `chained-encrypt-str-5` | `"alice"` | 7 | **5.05 µs** | +| `chained-encrypt-str-17` | `"alice@example.com"` | 23 | **15.97 µs** | +| `chained-encrypt-str-43` | 43-char sentence | 58 | **40.18 µs** | +| `chained-encrypt-left-str-17` | 17 chars | 23 | 10.09 µs | +| `chained-compare-str-17` | 17 chars | 23 | **402 ns** | + +## Reading + +- **~0.69 µs/block** for full encryption, scaling linearly with length + (5.05/7, 15.97/23, 40.18/58 all ≈ 0.69). +- This is *below* fixed-N Bit6's **~0.81 µs/block** (≈8.9 µs for an 11-block + u64) — the shape-(ii) payoff: the accumulator is keyed once per ciphertext, so + there is **no per-block AES key schedule**. Per-block work is + ~32 (`PRP_STREAM`) + 64 (`RO_KEY`) + 1 (`absorb`) ≈ 97 AES ops + the 64 + σ-MMO H evals, all under the already-scheduled accumulator cipher. +- `encrypt_left` is ~63% of full encrypt (it skips the 64 `RO_KEY` finalizes and + the right-block masking per block, keeping only the PRP + the single left tag). +- Comparison (~402 ns at 23 blocks) is a constant-time prefix scan plus one H + eval and one oblivious right-block read. + +## Caveats + +- CMAC subkey `K1` and the per-block doubling reuse the vectorized + `gf128_double`; the accumulator XORs are `u128` word ops. +- Numbers are from a 3 s criterion measurement on a developer machine (some + thermal variance); treat as indicative, not a regression gate. + +## Addendum (2026-10-08) — review hardening cost, and the sort-by-random-key builder + +Apple M4, rustc 1.94.1, criterion means, `--release`. Same bench file. + +**Review hardening.** Two of the #83 review-response changes were first +written as per-call wipes and cost more than everything else in the scheme: + +| commit | str-5 | str-17 | str-43 | left-str-17 | +|---|---:|---:|---:|---:| +| before the review hardening | 4.60 µs | 14.88 µs | 37.00 µs | 11.23 µs | +| + `ro` wiped per block (64 zeroizes, 64 fences, per block) | 5.50 µs | 17.74 µs | 44.87 µs | 11.50 µs | +| + `mixed.zeroize()` per `finalize`/`absorb` (≈97 per block) | 7.89 µs | 25.65 µs | 64.30 µs | 13.94 µs | +| **as merged:** `mixed` encrypted in place; `ro` hoisted, wiped once | **4.76 µs** | **15.09 µs** | **38.13 µs** | **11.43 µs** | + +The per-call `zeroize` is a volatile store plus a compiler fence; ~97 of them +per block stop the independent AES ops from pipelining. Encrypting `mixed` in +place leaves no plaintext temporary, so there is nothing to wipe, and `ro` is +fully overwritten by each block, so wiping it once after the loop is +equivalent. Bytes are unchanged by either. + +**Sort-by-random-key PRP builder** (rejected; see +`docs/benchmarks/2026-06-13-bit6-prp-results.md` and review brief A4). Measured +against the pre-fix tip above (25.93 µs for str-17), swapped into `prp_at`'s +`from_stream`; the accumulator's 32 `PRP_STREAM` tags (512 bytes) already cover +the sort's 64 words, so the cost is the network alone: + +| benchmark | Lemire FY | sort, 56-bit key | sort, 64-bit key | +|---|---:|---:|---:| +| chained-encrypt-str-5 | 7.91 µs | 9.95 µs (+26%) | 11.83 µs (+50%) | +| chained-encrypt-str-17 | 25.93 µs | 32.77 µs (+29%) | 38.27 µs (+49%) | +| chained-encrypt-str-43 | 65.06 µs | 81.27 µs (+26%) | 95.24 µs (+48%) | +| chained-encrypt-left-str-17 | 13.49 µs | 20.71 µs (+52%) | 27.31 µs (+98%) | + +The absolute deltas (≈ +0.30 µs and +0.54 µs per block) carry over to the +fixed baseline unchanged. Chained has no known-answer vectors, so a builder +change here would not be caught by a test; only the Bit6 vectors failed. + +## Addendum (2026-10-09) — oblivious PRP builder + +`from_stream` now builds through the oblivious builder (review brief A4, +"Status"; NEON on aarch64, SSSE3 on x86_64 when present, SWAR elsewhere), +with byte-identical tables, so the chained vectors are unchanged. Apple M4 on +mains power, rustc 1.94.1, criterion medians, `--release`; the same build +with the dispatcher pointed at each builder in turn (the indexed column by a +temporary, uncommitted edit). A second run of the NEON build agreed within 3%. + +| benchmark | indexed | NEON | SWAR | +|---|---:|---:|---:| +| chained-encrypt-str-5 | 4.67 µs | **4.37 µs** (−7%) | 7.86 µs (+68%) | +| chained-encrypt-str-17 | 15.20 µs | **13.70 µs** (−10%) | 25.18 µs (+66%) | +| chained-encrypt-str-43 | 36.88 µs | **34.12 µs** (−7%) | 64.47 µs (+75%) | +| chained-encrypt-left-str-17 | 11.29 µs | **10.21 µs** (−10%) | 22.05 µs (+95%) | +| chained-compare-str-17 | 318 ns | 316 ns | — (builds no PRP) | + +The indexed column agrees with the "as merged" row above (15.09 µs for +str-17, 15.20 µs here). The saving is roughly the builder saving times the +block count: about 50 ns per block (248 ns indexed, 199 ns NEON, per +512-byte stream) × 23 blocks ≈ 1.1 µs predicted for str-17, 1.5 µs +measured. diff --git a/docs/plans/2026-06-12-ore-v2-architecture.md b/docs/plans/2026-06-12-ore-v2-architecture.md index f646caa3..be3881c0 100644 --- a/docs/plans/2026-06-12-ore-v2-architecture.md +++ b/docs/plans/2026-06-12-ore-v2-architecture.md @@ -374,18 +374,36 @@ length of the common prefix**, which is materially more revealing than for fixed numerics. But this leakage is scoped to the **comparison operation**, not to stored data, because Lewi-Wu is a left/right scheme: a comparison is only ever evaluated between a left ciphertext and a right ciphertext, and a right ciphertext in isolation -reveals nothing about order. Three threat tiers follow: +reveals nothing about order. It does reveal **length**, below. + +**Every ciphertext reveals the exact plaintext length** (review of #83). The header +stores the block count, the ciphertext size is `4 + 25*count + 16` bytes (`4 + 17*count` +for a left), and for a string of `n` UTF-8 bytes `count = ⌈8n/6⌉`, strictly increasing +in `n`. So one ciphertext, full, left or right alone (`count * 8` bytes), gives `n` +exactly, with no comparison or query. Short, low-entropy columns (country codes, +statuses, first names) are often unique or nearly unique by length, and length plus +common-prefix length is a strong frequency-analysis signal. Hiding it needs padding to +a length bucket (Padmé, or a fixed maximum per column). Padding must keep order, so the +pad symbol must sort below every real symbol: a 65-value domain, which changes the PRP +domain and the wire format. Tracked in +[#104](https://github.com/cipherstash/ore.rs/issues/104); decide it before ore-v2 freezes +the chained wire format, or it needs a new scheme id. Until then, the string leakage +profile is **exact length, plus order and common-prefix length**. Three threat tiers +follow for the order and prefix leakage (length leaks in all of them): - **Offline / at rest:** right ciphertexts are semantically secure, so right-only storage would make this tier clean: an attacker holding only rights recovers nothing. **But right-only storage is not supported today** ([#99](https://github.com/cipherstash/ore.rs/issues/99)). Every scheme here, legacy Bit8 included, stores full ciphertexts, whose deterministic left half (`xt`, `f`) - sits next to the right half, and the only comparators (`compare_raw_slices`, `Ord` - on `CipherText`) take two full ciphertexts. So until #99 lands, an attacker who + sits next to the right half, and the right half is not usable on its own: the only + comparators (`compare_raw_slices`, `Ord` on `CipherText`) take two full ciphertexts, + and `VarRight` has no serialise/compare API. So until #99 lands, an attacker who exfiltrates the database can compare any two stored rows, with no query, and recovers their **order** and, for strings, their **common-prefix length**. This is - the same exposure as legacy Bit8 today, not a regression. + the same exposure as legacy Bit8 today, not a regression. Right-only storage needs a + left-versus-right comparator, likely a trinary indicator, which is a format change + (#99). - **Query time (legitimate operator):** running a query emits a left ciphertext, and each comparison against the stored rights reveals first-differing-block (= common-prefix length for strings) for exactly the pairs that query touches. @@ -559,6 +577,16 @@ PRs 3/4 and 5 are independent after PR 2, so they can proceed in parallel. Each `main` releasable; release-plz handles version bumps (all semver-minor under 0.x except PR 2's trait change, which should be called out in the changelog). +> **PR 6 status (2026-06-15): IMPLEMENTED** on `feat/ore-v2-chained` against the A2 +> design (crypto sign-off of A2 and A3 given 2026-10-09). Ships the +> AES-CMAC accumulator (`primitives/cmac.rs`, validated vs NIST SP 800-38B), +> `LemireFyPrp::from_stream` (shape ii), and `OreAes128Bit6Chained` (`scheme/chained.rs`) +> with variable-length `Var*` types and string encryption. Lexicographic cross-length +> comparison, equality-across-nonces, and >14-block plaintexts are tested; perf +> `docs/benchmarks/2026-06-15-chained-results.md` (~0.69 µs/block, below fixed-N's ~0.81). +> Not yet in: `u128`/`Decimal` via the const-N + accumulator path. Wire vectors are +> pinned (`tests/compat_chained_vectors.rs`). + ## Testing strategy - **Vectors (PR 1) as the refactor contract:** every later PR runs them unchanged for @@ -583,14 +611,17 @@ PR 2's trait change, which should be called out in the changelog). no code path can write seed material into a ciphertext (PR 2). - [ ] Domain separation between Bit8/Bit6/chained schemes under shared keys (PR 5, 6). Bit8/Bit6: done by key derivation (§4), with a cross-scheme regression test. - Chained: to confirm in PR 6. + Chained: separated, its only key is `AES_{k1}("ORE.v2.chain.acc")`, whose last + byte is outside every legacy PRF input and whose label differs from Bit6's. - [x] H instantiation (§6) selected and signed off (2026-06-15): BHKR σ-MMO, random-permutation model; see §6 and review brief A1. -- [ ] Selected §5(b) accumulator candidate reviewed and signed off (before PR 6). +- [x] Selected §5(b) accumulator candidate reviewed and signed off (2026-10-09; review + brief A2 and A3). - [ ] Accumulator chain state treated as key material: zeroized, never serialized, never reachable from `Left`/`Right` types (PR 6). - [ ] GF(2^128) doubling constant-time, if Candidate C is chosen (PR 6). -- [ ] String leakage profile documented and acknowledged at product level. Stored +- [ ] String leakage profile documented and acknowledged at product level. Every + ciphertext reveals the exact plaintext length (padding tracked in #104). Stored full ciphertexts compare offline, revealing order and common-prefix length, as legacy does, until right-only storage is supported (#99); see §5(b) (PR 6). - [ ] Legacy zero-run seed leakage (Context, "Known leakage of the legacy scheme") diff --git a/docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md b/docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md index 7a8f0ec5..41ed5859 100644 --- a/docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md +++ b/docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md @@ -16,8 +16,8 @@ Four crypto decisions gate the v2 work. Two block a PR that is already open | # | Decision | Model claimed | Status in code | Blocks | |---|----------|---------------|----------------|--------| | **A1** | 1-bit hash `H` instantiation | random-permutation (BHKR σ-MMO) | ✅ **RESOLVED** — `FixedPiZ2Hash` = `LSB(π(σ(x)⊕r)⊕σ(x)⊕r)`, σ(x)=2x | — (was the Bit6-vector gate; now cleared) | -| **A2** | Chained-prefix accumulator = AES-CMAC cached-state | CMAC PRF (standard) + 3 auditable claims | designed, not yet coded | PR 6 (variable-length / strings) | -| **A3** | PRP keystream from the accumulator (shape ii) | statistical (≤2⁻⁵⁵) + branch-family soundness | shape (i) shipped; (ii) deferred | PR 6 perf; couples to A2 | +| **A2** | Chained-prefix accumulator = AES-CMAC cached-state | CMAC PRF (standard) + 3 auditable claims | ✅ **SIGNED OFF** 2026-10-09; implemented as `OreAes128Bit6Chained` | — | +| **A3** | PRP keystream from the accumulator (shape ii) | statistical (≤2⁻⁵⁵) + branch-family soundness | ✅ **SIGNED OFF** 2026-10-09; shape (ii) ships in the chained scheme (`from_stream`), shape (i) in Bit6 | — | | **A4** | Secret-indexed swap in PRP key-gen — ratify alignment + MemJam posture? | ~~constant-time / cache-line (sub-line = oblivious tier)~~ **oblivious key generation, every target** (2026-10-09) | oblivious builder (NEON / SSSE3 / SWAR) is the default, byte-identical tables; oblivious table reads and compare read; `N = 64` guard | nothing — fixes are byte-stable; A1 alone gates vectors | **Recommended sequencing:** **A1 is resolved** (BHKR σ-MMO) — Bit6 vectors can now @@ -26,7 +26,7 @@ applied (none of which alter ciphertexts); the MemJam posture call it carried was overtaken on 2026-10-09, when a measured timing dependence made the oblivious builder the default rather than a tier (§5, "Status"). Then **A2 + A3 as one pass** (they gate PR 6, and A3 only exists inside A2's -accumulator). +accumulator). A2 and A3 were signed off together on 2026-10-09. **What is explicitly *not* in scope:** the legacy Bit8 scheme is wire-frozen and byte-identical to v1 (`tests/compat_vectors`); it keeps all status-quo @@ -611,10 +611,11 @@ None of (a)–(d) changes ciphertexts, so Bit6 vectors are gated only by A1. byte-stable; production comparator adopts `ct_select_byte` separately). **Gate 2 — before PR 6 is written:** -- [ ] **A2** CMAC encoding fully specified and checked injective; many-outputs +- [x] **A2** CMAC encoding fully specified and checked injective; many-outputs / chain-state-from-published-tag interaction cleared; impl-faithfulness + - zeroization test plan agreed. -- [ ] **A3** PRP-stream-as-CMAC-branch-family domain separation approved. + zeroization test plan agreed. Signed off 2026-10-09 (Dan Draper). +- [x] **A3** PRP-stream-as-CMAC-branch-family domain separation approved. + Signed off 2026-10-09 (Dan Draper). --- diff --git a/docs/reviews/2026-06-16-zeroize-audit-pr83.md b/docs/reviews/2026-06-16-zeroize-audit-pr83.md index c57d5a9c..18477905 100644 --- a/docs/reviews/2026-06-16-zeroize-audit-pr83.md +++ b/docs/reviews/2026-06-16-zeroize-audit-pr83.md @@ -9,10 +9,14 @@ > `CmacAccumulator::absorb` and `finalize` each build an owned `mixed` value > from the chain state (and, in `finalize`, `K1`) and returned without > wiping it; and `from_stream`'s per-draw keystream copy (see the #82 audit) -> applies here too. Both are fixed: on #83 in "harden(cmac): wipe the mixed -> state temporaries in absorb and finalize", and on #82. The wipes remove the -> source-level copies; they cannot rule out copies the compiler keeps in -> registers or spills, and no IR evidence was gathered for them. +> applies here too. Both are fixed: on #83 in "harden(cmac): encrypt the +> mixed state in place in absorb and finalize", and on #82. In `absorb` and +> `finalize` the mixed input is now encrypted in place, so the one buffer +> that held it holds the new state or the tag afterwards and there is no +> plaintext temporary to wipe (a per-call `zeroize` was measured at +45% on +> chained encryption and dropped). These fixes remove the source-level +> copies; they cannot rule out copies the compiler keeps in registers or +> spills, and no IR evidence was gathered for them. ## Scope @@ -27,12 +31,12 @@ wiped, on the struct Drop and in the per-block local buffers. | Item | Status | |---|---| | `CmacAccumulator` Drop (`k1`, `state`) | ✅ wiped; `cipher` via aes `ZeroizeOnDrop` | -| **`CmacAccumulator::absorb` / `finalize` `mixed` temporaries** | ❌→✅ **was a gap** (missed by this audit) — state/`K1`-derived values left on the stack. **Fixed** on #83 (see the note above). | +| **`CmacAccumulator::absorb` / `finalize` `mixed` temporaries** | ❌→✅ **was a gap** (missed by this audit) — state/`K1`-derived values left on the stack. **Fixed** on #83: encrypted in place, the tag overwrites the input (see the note above). | | chained `k_acc` Drop | ✅ wiped | | chained `stream` (PRP keystream, `prp_at`) | ✅ `stream.zeroize()` | | `from_stream` consumed stream | ✅ caller-owned; both callers wipe. **Amended:** the per-draw `draw`/`x` copies inside `from_stream` were not wiped; fixed on #82. | | **`L = E_k(0)` in `CmacAccumulator::new`** | ❌→✅ **was a gap** — `K1 = dbl(L)`'s source, left on the stack. **Fixed:** `l.zeroize()` after deriving `k1`. | -| **chained `ro` RO-tag buffer (`encrypt_var`)** | ❌→✅ **was a gap** — 64 key-derived CMAC tags, never wiped (the sibling `bit2_w6` wipes its `template`/`work`). **Fixed:** per-block `ro` wipe after the right block is built. | +| **chained `ro` RO-tag buffer (`encrypt_var`)** | ❌→✅ **was a gap** — 64 key-derived CMAC tags, never wiped (the sibling `bit2_w6` wipes its `template`/`work`). **Fixed:** `ro` is one scratch buffer for the whole ciphertext, wiped once after the block loop; every entry is overwritten by each block's tags, so a per-block wipe (measured at +19%) protected nothing between blocks. | Dangerous-API scan: 0 hits (`mem::forget`/`ManuallyDrop`/`Box::leak`/`transmute`/ `write_bytes`/`mem::take`/secret-across-`.await`), grep-confirmed. No `Copy` on diff --git a/packages/ore-rs/Cargo.toml b/packages/ore-rs/Cargo.toml index 005e6931..c57ba418 100644 --- a/packages/ore-rs/Cargo.toml +++ b/packages/ore-rs/Cargo.toml @@ -65,5 +65,9 @@ name = "chrono" harness = false required-features = ["chrono"] +[[bench]] +name = "chained" +harness = false + [[example]] name = "encrypt" diff --git a/packages/ore-rs/benches/chained.rs b/packages/ore-rs/benches/chained.rs new file mode 100644 index 00000000..a3bfc9dc --- /dev/null +++ b/packages/ore-rs/benches/chained.rs @@ -0,0 +1,41 @@ +use criterion::{black_box, criterion_group, criterion_main, Criterion}; +use hex_literal::hex; +use ore_rs::scheme::chained::{OreAes128Bit6Chained, OreAes128Bit6ChainedChaCha20}; + +fn init_ore() -> OreAes128Bit6ChainedChaCha20 { + let k1: [u8; 16] = hex!("00010203 04050607 08090a0b 0c0d0e0f"); + OreAes128Bit6Chained::init(&k1).unwrap() +} + +fn criterion_benchmark(c: &mut Criterion) { + let ore = init_ore(); + + // Short (~8 chars ≈ 11 blocks) and long (~40 chars ≈ 54 blocks) strings. + let short = "alice"; + let medium = "alice@example.com"; + let long = "the quick brown fox jumps over the lazy dog"; + + c.bench_function("chained-encrypt-str-5", |b| { + b.iter(|| ore.encrypt_str(black_box(short)).unwrap()) + }); + c.bench_function("chained-encrypt-str-17", |b| { + b.iter(|| ore.encrypt_str(black_box(medium)).unwrap()) + }); + c.bench_function("chained-encrypt-str-43", |b| { + b.iter(|| ore.encrypt_str(black_box(long)).unwrap()) + }); + c.bench_function("chained-encrypt-left-str-17", |b| { + b.iter(|| ore.encrypt_left_str(black_box(medium)).unwrap()) + }); + + let a = ore.encrypt_str(medium).unwrap().to_bytes(); + let b_ct = ore.encrypt_str("alice@example.org").unwrap().to_bytes(); + c.bench_function("chained-compare-str-17", |bench| { + bench.iter(|| { + OreAes128Bit6ChainedChaCha20::compare_raw_slices(black_box(&a), black_box(&b_ct)) + }) + }); +} + +criterion_group!(benches, criterion_benchmark); +criterion_main!(benches); diff --git a/packages/ore-rs/src/lib.rs b/packages/ore-rs/src/lib.rs index 27c19f2a..a9d86410 100644 --- a/packages/ore-rs/src/lib.rs +++ b/packages/ore-rs/src/lib.rs @@ -169,6 +169,10 @@ pub enum OreError { /// per-ciphertext nonce). #[error("Randomness Error")] RandError(#[from] rand::Error), + /// The plaintext exceeds the maximum block count the wire format can + /// represent (the chained scheme encodes the block count as a `u16`). + #[error("Too many blocks")] + TooManyBlocks, } /// A BlockORE cipher: a key-bound object that can encrypt fixed-N plaintexts diff --git a/packages/ore-rs/src/primitives.rs b/packages/ore-rs/src/primitives.rs index 7d154cf6..0928ae66 100644 --- a/packages/ore-rs/src/primitives.rs +++ b/packages/ore-rs/src/primitives.rs @@ -1,3 +1,4 @@ +pub mod cmac; pub mod hash; pub mod prf; pub mod prp; diff --git a/packages/ore-rs/src/primitives/cmac.rs b/packages/ore-rs/src/primitives/cmac.rs new file mode 100644 index 00000000..35fec7a2 --- /dev/null +++ b/packages/ore-rs/src/primitives/cmac.rs @@ -0,0 +1,274 @@ +//! Chained-prefix AES-CMAC accumulator for the variable-length ORE scheme. +//! +//! Implements the design in `docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md` +//! (plan §5b, the A2 gate). Every per-block secret of the chained scheme is a +//! `finalize(prefix-state, final_block)` — a bona fide AES-CMAC (NIST SP +//! 800-38B) tag of `P_0 ‖ … ‖ P_{n-1} ‖ final_block`, where the prefix CBC +//! state is cached and extended incrementally (`clone-state-then-finalize` +//! *is* incremental CMAC). +//! +//! All accumulator messages are exact 16-byte multiples, so only subkey `K1` +//! is ever used (the `K2`/padding path never occurs). `K1 = dbl(E_k(0))` reuses +//! the σ-MMO GF(2^128) doubling. + +use crate::primitives::hash::gf128_double_u128; +use aes::cipher::{generic_array::GenericArray, BlockEncrypt, KeyInit}; +use aes::Aes128Enc; +use zeroize::Zeroize; + +/// Block-width tag in the final block (domain separation; see the spec §4). +pub(crate) const WIDTH_BIT6: u8 = 6; + +/// Output-family ("branch") tag carried in byte 0 of a final block. Encoded as +/// a non-zero enum: prefix blocks use byte 0 = `0x00`, so making this a type +/// (rather than a `u8`) removes any way to construct a final block whose byte 0 +/// is `0x00` and collides with a prefix block — the disjointness is the basis +/// of the encoding's injectivity (spec §4), now guaranteed at compile time. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +#[repr(u8)] +pub(crate) enum Branch { + RoKey = 0x01, + PrpStream = 0x02, +} + +/// Prefix block `P_t` carrying symbol `sym` at position `pos` (spec §4): +/// `[0x00 ‖ pos(u16 BE) ‖ sym ‖ 0…]`. +#[inline] +pub(crate) fn prefix_block(pos: u16, sym: u8) -> [u8; 16] { + let mut b = [0u8; 16]; + b[1..3].copy_from_slice(&pos.to_be_bytes()); + b[3] = sym; + b +} + +/// Final block `F(branch, n, s)` (spec §4): +/// `[branch ‖ n(u16 BE) ‖ s(u16 BE) ‖ width ‖ 0…]`. `s` is the domain value +/// `j` (or the permuted symbol `xt[n]` for the left tag) for `RoKey`, and the +/// keystream counter `c` for `PrpStream`. `branch` is a [`Branch`] (always +/// non-zero), so byte 0 can never collide with a prefix block. +#[inline] +pub(crate) fn final_block(branch: Branch, n: u16, s: u16, width: u8) -> [u8; 16] { + let mut b = [0u8; 16]; + b[0] = branch as u8; + b[1..3].copy_from_slice(&n.to_be_bytes()); + b[3..5].copy_from_slice(&s.to_be_bytes()); + b[5] = width; + b +} + +/// Incremental AES-CMAC over a prefix of 16-byte blocks. +/// +/// `state` is the CBC-MAC chain over the prefix blocks absorbed so far (no +/// subkey applied — those are never the last block). [`finalize`](Self::finalize) +/// produces the CMAC tag of `prefix ‖ final_block` without mutating the chain; +/// [`absorb`](Self::absorb) extends the prefix. +pub(crate) struct CmacAccumulator { + cipher: Aes128Enc, + k1: [u8; 16], + state: [u8; 16], +} + +impl CmacAccumulator { + /// Schedule AES under `key`, derive `K1 = dbl(E_k(0))`, and start the chain + /// at the all-zero CBC IV (empty prefix). + pub(crate) fn new(key: &[u8; 16]) -> Self { + // Encryption only: CMAC never decrypts, so build no decryption schedule. + let cipher = Aes128Enc::new(GenericArray::from_slice(key)); + let mut acc = Self { + cipher, + k1: [0u8; 16], + state: [0u8; 16], + }; + let mut l = acc.encrypt([0u8; 16]); // L = E_k(0) + acc.k1 = gf128_double_u128(u128::from_be_bytes(l)).to_be_bytes(); + l.zeroize(); // L is K1's source; don't leave it on the stack (spec §9) + acc + } + + #[inline] + fn encrypt(&self, mut b: [u8; 16]) -> [u8; 16] { + self.cipher + .encrypt_block(GenericArray::from_mut_slice(&mut b)); + b + } + + /// Extend the prefix chain: `S ← E_k(S ⊕ block)` (a CBC step, no subkey — + /// not a published tag). + /// + /// The mixed input is encrypted in place: the one buffer that held + /// `S ⊕ block` holds the new `S` afterwards, so there is no plaintext + /// temporary to wipe (and no per-call fence on the hot path). + #[inline] + pub(crate) fn absorb(&mut self, block: &[u8; 16]) { + let mut mixed = + (u128::from_be_bytes(self.state) ^ u128::from_be_bytes(*block)).to_be_bytes(); + self.cipher + .encrypt_block(GenericArray::from_mut_slice(&mut mixed)); + self.state = mixed; + } + + /// CMAC tag of `prefix-so-far ‖ final_block`: `E_k(S ⊕ final_block ⊕ K1)`. + /// Does not change the chain, so all outputs at a given position finalize + /// from the one cached `S`. + /// + /// Encrypted in place for the same reason as [`absorb`](Self::absorb): + /// the buffer that held `S ⊕ final_block ⊕ K1` is overwritten by the tag. + #[inline] + pub(crate) fn finalize(&self, final_block: &[u8; 16]) -> [u8; 16] { + let mut mixed = (u128::from_be_bytes(self.state) + ^ u128::from_be_bytes(*final_block) + ^ u128::from_be_bytes(self.k1)) + .to_be_bytes(); + self.cipher + .encrypt_block(GenericArray::from_mut_slice(&mut mixed)); + mixed + } + + #[cfg(test)] + pub(crate) fn subkey1(&self) -> [u8; 16] { + self.k1 + } +} + +impl Drop for CmacAccumulator { + fn drop(&mut self) { + // `cipher` zeroizes its own key schedule (aes "zeroize" feature). + self.k1.zeroize(); + self.state.zeroize(); + } +} + +#[cfg(test)] +mod tests { + use super::*; + use hex_literal::hex; + use quickcheck::TestResult; + + // NIST SP 800-38B, AES-128 CMAC examples (Appendix D.1). Validates that + // this is real CMAC: the K1 subkey, the single-full-block tag, and the + // multi-block CBC chain + finalize. + #[test] + fn nist_sp800_38b_aes128() { + let k = hex!("2b7e151628aed2a6abf7158809cf4f3c"); + + let acc = CmacAccumulator::new(&k); + assert_eq!(acc.subkey1(), hex!("fbeed618357133667c85e08f7236a8de")); + + // CMAC of one full block (the block is the final block; empty prefix). + let m1 = hex!("6bc1bee22e409f96e93d7e117393172a"); + assert_eq!(acc.finalize(&m1), hex!("070a16b46b4d4144f79bdd9dd04a287c")); + + // CMAC of four full blocks: absorb the first three, finalize the last. + let mut acc = CmacAccumulator::new(&k); + acc.absorb(&hex!("6bc1bee22e409f96e93d7e117393172a")); + acc.absorb(&hex!("ae2d8a571e03ac9c9eb76fac45af8e51")); + acc.absorb(&hex!("30c81c46a35ce411e5fbc1191a0a52ef")); + assert_eq!( + acc.finalize(&hex!("f69f2445df4f9b17ad2b417be66c3710")), + hex!("51f0bebf7e3b9d92fc49741779363cfe") + ); + } + + #[test] + fn encoding_is_injective_on_block_type() { + // Prefix blocks have byte 0 == 0x00; final blocks have byte 0 != 0x00. + assert_eq!(prefix_block(7, 0x3f)[0], 0x00); + assert_ne!(final_block(Branch::RoKey, 7, 9, WIDTH_BIT6)[0], 0x00); + assert_ne!(final_block(Branch::PrpStream, 7, 9, WIDTH_BIT6)[0], 0x00); + // Distinct branches / positions / sub-indices give distinct blocks. + assert_ne!( + final_block(Branch::RoKey, 1, 2, WIDTH_BIT6), + final_block(Branch::PrpStream, 1, 2, WIDTH_BIT6) + ); + assert_ne!( + final_block(Branch::RoKey, 1, 2, WIDTH_BIT6), + final_block(Branch::RoKey, 1, 3, WIDTH_BIT6) + ); + assert_ne!(prefix_block(1, 2), prefix_block(2, 2)); + } + + // finalize must not disturb the cached chain: many outputs from one state. + #[test] + fn finalize_does_not_mutate_state() { + let mut acc = CmacAccumulator::new(&[0x11u8; 16]); + acc.absorb(&prefix_block(0, 5)); + let a = acc.finalize(&final_block(Branch::RoKey, 1, 0, WIDTH_BIT6)); + let _ = acc.finalize(&final_block(Branch::PrpStream, 1, 0, WIDTH_BIT6)); + let a_again = acc.finalize(&final_block(Branch::RoKey, 1, 0, WIDTH_BIT6)); + assert_eq!(a, a_again); + } + + /// Independent from-scratch CMAC of a whole-block message: CBC-MAC chain + /// over all but the last block, then `E_k(state ⊕ last ⊕ K1)`. Mirrors the + /// NIST construction without reusing the accumulator's state threading, so + /// it can validate that threading for any block count (not just t∈{1,4}). + fn reference_cmac(key: &[u8; 16], blocks: &[[u8; 16]]) -> [u8; 16] { + let cipher = aes::Aes128::new(GenericArray::from_slice(key)); + let enc = |mut b: [u8; 16]| { + cipher.encrypt_block(GenericArray::from_mut_slice(&mut b)); + b + }; + let k1 = gf128_double_u128(u128::from_be_bytes(enc([0u8; 16]))).to_be_bytes(); + let (last, prefix) = blocks.split_last().expect("at least one block"); + let mut state = 0u128; + for blk in prefix { + state = u128::from_be_bytes(enc((state ^ u128::from_be_bytes(*blk)).to_be_bytes())); + } + enc((state ^ u128::from_be_bytes(*last) ^ u128::from_be_bytes(k1)).to_be_bytes()) + } + + quickcheck! { + /// Incremental absorb-then-finalize over an arbitrary number of full + /// blocks equals a from-scratch CMAC. Validates the chain threading and + /// subkey use for every block count, well beyond the two fixed vectors. + fn prop_incremental_matches_reference(key: Vec, msg: Vec) -> TestResult { + if key.len() < 16 { + return TestResult::discard(); + } + let mut k = [0u8; 16]; + k.copy_from_slice(&key[..16]); + + // 1..=8 full blocks, derived from `msg` (deterministic, never empty). + let nblocks = 1 + (msg.len() % 8); + let mut blocks = Vec::with_capacity(nblocks); + for i in 0..nblocks { + let mut b = [0u8; 16]; + for (j, slot) in b.iter_mut().enumerate() { + *slot = msg + .get(i * 16 + j) + .copied() + .unwrap_or((i as u8).wrapping_mul(31).wrapping_add(j as u8)); + } + blocks.push(b); + } + + let mut acc = CmacAccumulator::new(&k); + let (last, prefix) = blocks.split_last().unwrap(); + for blk in prefix { + acc.absorb(blk); + } + TestResult::from_bool(acc.finalize(last) == reference_cmac(&k, &blocks)) + } + + /// Final blocks injectively encode `(branch, n, s, width)`: the tuple is + /// recoverable from the bytes and byte 0 is always non-zero, so distinct + /// inputs give distinct blocks (and none collide with a prefix block). + fn prop_final_block_injective(n: u16, s: u16, width: u8) -> bool { + [Branch::RoKey, Branch::PrpStream].iter().copied().all(|branch| { + let fb = final_block(branch, n, s, width); + fb[0] == branch as u8 + && fb[0] != 0x00 + && u16::from_be_bytes([fb[1], fb[2]]) == n + && u16::from_be_bytes([fb[3], fb[4]]) == s + && fb[5] == width + }) + } + + /// Prefix blocks carry byte0 == 0x00 (never colliding with a final + /// block) and injectively encode `(pos, sym)`. + fn prop_prefix_block_injective(pos: u16, sym: u8) -> bool { + let pb = prefix_block(pos, sym); + pb[0] == 0x00 && u16::from_be_bytes([pb[1], pb[2]]) == pos && pb[3] == sym + } + } +} diff --git a/packages/ore-rs/src/primitives/hash.rs b/packages/ore-rs/src/primitives/hash.rs index 3c602b8e..173f64c7 100644 --- a/packages/ore-rs/src/primitives/hash.rs +++ b/packages/ore-rs/src/primitives/hash.rs @@ -99,9 +99,10 @@ fn pi() -> &'static Aes128 { /// dominated the σ-MMO hot path (≈704 doublings per Bit6 u64 encrypt). /// Constant-time: no secret-dependent control flow. #[inline] -fn gf128_double_u128(x: u128) -> u128 { +pub(crate) fn gf128_double_u128(x: u128) -> u128 { // `x << 1` discards the top bit (the GF reduction trigger); fold 0x87 into - // the low byte iff that bit was set. `x >> 127` is 0 or 1. + // the low byte iff that bit was set. `x >> 127` is 0 or 1. Also the CMAC + // subkey doubling (`crate::primitives::cmac`). (x << 1) ^ ((x >> 127) * 0x87) } diff --git a/packages/ore-rs/src/primitives/prp.rs b/packages/ore-rs/src/primitives/prp.rs index 70d0c1ae..b203785a 100644 --- a/packages/ore-rs/src/primitives/prp.rs +++ b/packages/ore-rs/src/primitives/prp.rs @@ -187,6 +187,36 @@ macro_rules! impl_lemire_fy_prp { "LemireFyPrp: the oblivious builders are written for a 64-entry domain" ); + impl LemireFyPrp<$domain> { + /// Build the permutation directly from a precomputed draw stream + /// (shape (ii)): `stream` must be at least `($domain - 1) * 8` + /// bytes, consumed as `$domain - 1` little-endian u64 draws and + /// Lemire-reduced. The chained scheme feeds the CMAC accumulator's + /// `PRP_STREAM` branch here, avoiding a per-block AES key schedule; + /// [`Prp::new`] feeds it an AES-CTR keystream. Either way the + /// tables come from the oblivious builder ([`oblivious::build`]), + /// so no address depends on the stream. + pub(crate) fn from_stream(stream: &[u8]) -> PrpResult { + if stream.len() < ($domain - 1) * 8 { + return Err(PrpError); + } + + // Fisher–Yates with Lemire-reduced wide draws: draw `d` + // (8 bytes) drives step `$domain - 1 - d`. Fixed trip count, + // branch-free index reduction, and no secret-dependent + // address: the oblivious builder keeps both tables in + // registers (or masked words) and maintains the inverse + // alongside the swaps. It wipes its copies of each draw. + let mut perm = Self { + permutation: [0u8; $domain], + inverse: [0u8; $domain], + }; + oblivious::build(stream, &mut perm.permutation, &mut perm.inverse); + + Ok(perm) + } + } + impl Prp for LemireFyPrp<$domain> { fn new(key: &[u8]) -> PrpResult { if key.len() < 16 { @@ -195,7 +225,8 @@ macro_rules! impl_lemire_fy_prp { // Fixed-count AES-CTR keystream from the seed: counter in // the first 4 bytes (big-endian), matching the existing - // PRNG's counter convention. + // PRNG's counter convention. Shape (i): a fresh key schedule + // per call. Shape (ii) skips this via `from_stream`. // Encryption only: no decryption schedule is built or copied. let cipher = Aes128Enc::new(GenericArray::from_slice(&key[0..16])); let mut blocks = [AesBlock::default(); $stream_blocks]; @@ -209,17 +240,7 @@ macro_rules! impl_lemire_fy_prp { stream[i * 16..(i + 1) * 16].copy_from_slice(b); } - // Fisher–Yates with Lemire-reduced wide draws: draw `d` - // (8 bytes) drives step `$domain - 1 - d`. Fixed trip count, - // branch-free index reduction, and no secret-dependent - // address: the oblivious builder keeps both tables in - // registers (or masked words) and maintains the inverse - // alongside the swaps. It wipes its copies of each draw. - let mut perm = Self { - permutation: [0u8; $domain], - inverse: [0u8; $domain], - }; - oblivious::build(&stream, &mut perm.permutation, &mut perm.inverse); + let perm = Self::from_stream(&stream)?; // The keystream determined the permutation — wipe it. stream.zeroize(); diff --git a/packages/ore-rs/src/scheme.rs b/packages/ore-rs/src/scheme.rs index 25df7c0a..6632fcae 100644 --- a/packages/ore-rs/src/scheme.rs +++ b/packages/ore-rs/src/scheme.rs @@ -12,5 +12,11 @@ pub mod bit2; /// pending crypto review of the Z2 hash (v2 plan, section 6). pub mod bit2_w6; +/// Variable-length / chained-prefix scheme (6-bit blocks): lifts the packed +/// 14-block cap via an AES-CMAC accumulator, enabling string encryption. Wire +/// format v2 (scheme id 0x03). See +/// `docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md`. +pub mod chained; + pub(crate) mod decompose; pub(crate) mod width; diff --git a/packages/ore-rs/src/scheme/chained.rs b/packages/ore-rs/src/scheme/chained.rs new file mode 100644 index 00000000..786809be --- /dev/null +++ b/packages/ore-rs/src/scheme/chained.rs @@ -0,0 +1,819 @@ +//! Variable-length / chained-prefix BlockORE over 6-bit blocks. +//! +//! Lifts the fixed-N packed-prefix cap (≤ 14 blocks) by deriving every +//! per-block secret from an incremental **AES-CMAC accumulator** over the +//! prefix, instead of packing the raw prefix into one AES block. This enables +//! arbitrary-length plaintexts — strings in particular. Design: +//! `docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md` (the A2 gate). +//! +//! Per block `n` (prefix `x[0..n-1]`, accumulator state `S_n`): +//! - **PRP** `π_n` (shape ii): keystream = `STREAM_BLOCKS` CMAC tags on the +//! `PRP_STREAM` branch → `LemireFyPrp::from_stream` (no per-block key +//! schedule). +//! - **left tag** `f[n] = ro(n, xt[n])` — the `RO_KEY` branch at the permuted +//! symbol (so the left/right masks cancel at compare time). +//! - **right block**: `H(ro(n, j), nonce) ⊕ indicator`, as the fixed-N scheme. +//! +//! Comparison is lexicographic: scan `min(len_a, len_b)` blocks; if a prefix +//! matches throughout, the shorter sorts first. A comparison reveals the +//! common-prefix length. +//! +//! **Every ciphertext reveals the exact length of its plaintext.** The header +//! stores the block count, and the size is `4 + 25*count + 16` bytes for a +//! full ciphertext (`4 + 17*count` for a left). For a string of `n` UTF-8 +//! bytes, `count = ⌈8n/6⌉`, which is strictly increasing in `n`, so one +//! ciphertext gives `n` exactly, with no comparison or query. The right half +//! alone is `count * 8` bytes, so right-only storage would not hide it either. +//! Padding to a length bucket needs a pad symbol below every real symbol, a +//! 65-value domain and a wire-format change: +//! . +//! +//! **Stored rows compare offline.** A full ciphertext carries its +//! deterministic left half (`xt`, `f`), so anyone holding two stored full +//! ciphertexts can run `compare_raw_slices` with no +//! query and learn their order and common-prefix length. The legacy scheme +//! has the same exposure. There is no right-only storage path yet (plan +//! §5(b)). + +use std::cell::RefCell; +use std::cmp::Ordering; + +use rand::{Rng, SeedableRng}; +use rand_chacha::ChaCha20Rng; +use subtle_ng::{Choice, ConditionallySelectable, ConstantTimeEq}; +use zeroize::{Zeroize, Zeroizing}; + +use aes::cipher::{generic_array::GenericArray, BlockEncrypt, KeyInit}; +use aes::Aes128Enc; + +use crate::ciphertext::{parse_header, ParseError}; +use crate::primitives::cmac::{final_block, prefix_block, Branch, CmacAccumulator, WIDTH_BIT6}; +use crate::primitives::hash::FixedPiZ2Hash; +use crate::primitives::prp::LemireFyPrp; +use crate::primitives::{stack, AesBlock, Hash, HashKey, Prp}; +use crate::scheme::decompose::{decompose_6bit, num_blocks_6bit}; +use crate::scheme::width::{ct_assign_bytes, ct_bit, ct_select_byte}; +use crate::OreError; + +const VERSION: u8 = 0x02; +const SCHEME_ID: u8 = 0x03; +const DOMAIN: usize = 64; +/// CMAC tags per block for the PRP keystream: ⌈(DOMAIN−1)·8 / 16⌉ = 32. +const STREAM_BLOCKS: usize = 32; +const F_LEN: usize = 16; +const RIGHT_LEN: usize = DOMAIN / 8; // 8 +const HEADER_LEN: usize = 4; +const NONCE_LEN: usize = 16; + +/// Domain-separation label for deriving the accumulator key (spec §3). +const ACC_KEY_LABEL: [u8; 16] = *b"ORE.v2.chain.acc"; + +/// Serialised length of a full ciphertext with `count` blocks. +#[inline] +fn total_len(count: usize) -> usize { + HEADER_LEN + count + count * F_LEN + NONCE_LEN + count * RIGHT_LEN +} + +/// Serialised length of a left-only ciphertext (`header ‖ xt ‖ f`) — the prefix +/// of a full ciphertext, with no nonce/right half. Matches `VarLeft::to_bytes`. +#[inline] +fn left_len(count: usize) -> usize { + HEADER_LEN + count + count * F_LEN +} + +// Raw-byte accessors into a serialised full ciphertext (see `to_bytes`). +#[inline] +fn xt_at(s: &[u8], i: usize) -> u8 { + s[HEADER_LEN + i] +} +#[inline] +fn f_at(s: &[u8], count: usize, i: usize) -> &[u8] { + let o = HEADER_LEN + count + i * F_LEN; + &s[o..o + F_LEN] +} +#[inline] +fn nonce_at(s: &[u8], count: usize) -> &[u8] { + let o = HEADER_LEN + count + count * F_LEN; + &s[o..o + NONCE_LEN] +} +#[inline] +fn right_at(s: &[u8], count: usize, i: usize) -> &[u8] { + let o = HEADER_LEN + count + count * F_LEN + NONCE_LEN + i * RIGHT_LEN; + &s[o..o + RIGHT_LEN] +} + +/// Left half of a chained ciphertext: per-block permuted symbol + tag. +#[derive(Clone, Debug)] +pub struct VarLeft { + xt: Vec, + f: Vec<[u8; F_LEN]>, +} + +/// Right half: per-ciphertext nonce + per-block masked bitvectors. +#[derive(Clone, Debug)] +pub struct VarRight { + nonce: [u8; NONCE_LEN], + blocks: Vec<[u8; RIGHT_LEN]>, +} + +/// Full variable-length ciphertext. +#[derive(Clone, Debug)] +pub struct VarCipherText { + left: VarLeft, + right: VarRight, +} + +impl VarLeft { + /// `header ‖ xt ‖ f` (no nonce/right half). + pub fn to_bytes(&self) -> Vec { + let count = self.xt.len(); + let mut out = Vec::with_capacity(HEADER_LEN + count + count * F_LEN); + out.extend_from_slice(&[VERSION, SCHEME_ID]); + out.extend_from_slice(&(count as u16).to_be_bytes()); + out.extend_from_slice(&self.xt); + for fb in &self.f { + out.extend_from_slice(fb); + } + out + } +} + +impl VarCipherText { + /// `header ‖ xt ‖ f ‖ nonce ‖ right`. + pub fn to_bytes(&self) -> Vec { + let count = self.left.xt.len(); + let mut out = Vec::with_capacity(total_len(count)); + out.extend_from_slice(&[VERSION, SCHEME_ID]); + out.extend_from_slice(&(count as u16).to_be_bytes()); + out.extend_from_slice(&self.left.xt); + for fb in &self.left.f { + out.extend_from_slice(fb); + } + out.extend_from_slice(&self.right.nonce); + for rb in &self.right.blocks { + out.extend_from_slice(rb); + } + out + } + + /// Parse the bytes produced by [`Self::to_bytes`]. + pub fn from_slice(data: &[u8]) -> Result { + let ((ver, sid, count), _body) = parse_header(data)?; + if ver != VERSION || sid != SCHEME_ID { + return Err(ParseError); + } + if data.len() != total_len(count) { + return Err(ParseError); + } + // Same rule as the comparators (and the fixed-width parsers): a symbol + // outside the 6-bit domain is not a ciphertext of this scheme. + if !symbols_in_domain(data, count) { + return Err(ParseError); + } + let xt = data[HEADER_LEN..HEADER_LEN + count].to_vec(); + let mut f = Vec::with_capacity(count); + for n in 0..count { + let mut fb = [0u8; F_LEN]; + fb.copy_from_slice(f_at(data, count, n)); + f.push(fb); + } + let mut nonce = [0u8; NONCE_LEN]; + nonce.copy_from_slice(nonce_at(data, count)); + let mut blocks = Vec::with_capacity(count); + for n in 0..count { + let mut rb = [0u8; RIGHT_LEN]; + rb.copy_from_slice(right_at(data, count, n)); + blocks.push(rb); + } + Ok(VarCipherText { + left: VarLeft { xt, f }, + right: VarRight { nonce, blocks }, + }) + } +} + +/// Variable-length / chained-prefix ORE cipher (6-bit blocks). +pub struct OreAes128Bit6Chained { + /// Dedicated CMAC accumulator key (subsumes the fixed-N `prf1`/`prf2`). + k_acc: [u8; 16], + rng: RefCell, +} + +/// Convenience alias backed by `ChaCha20Rng`. +pub type OreAes128Bit6ChainedChaCha20 = OreAes128Bit6Chained; + +impl OreAes128Bit6Chained { + /// Initialise from a single 16-byte key. Unlike the fixed-N schemes (which + /// key two PRFs), the chained scheme is **single-key by design**: the CMAC + /// accumulator derives every per-block secret from one key via branch tags + /// (spec §3), so there is no second key. The accumulator key is `k1` run + /// through a labelled AES call for domain separation. + pub fn init(k1: &[u8; 16]) -> Result { + let cipher = Aes128Enc::new(GenericArray::from_slice(k1)); + let mut k_acc = ACC_KEY_LABEL; + cipher.encrypt_block(GenericArray::from_mut_slice(&mut k_acc)); + Ok(Self { + k_acc, + rng: RefCell::new(R::from_entropy()), + }) + } + + /// Derive the per-block PRP from the accumulator's `PRP_STREAM` branch. + fn prp_at(&self, acc: &CmacAccumulator, n: u16) -> Result, OreError> { + let mut stream = [0u8; STREAM_BLOCKS * 16]; + for (c, chunk) in stream.chunks_mut(16).enumerate() { + chunk.copy_from_slice(&acc.finalize(&final_block( + Branch::PrpStream, + n, + c as u16, + WIDTH_BIT6, + ))); + } + let prp = LemireFyPrp::::from_stream(&stream)?; + stream.zeroize(); + Ok(prp) + } + + /// Full Left+Right ciphertext for `x` (one 6-bit symbol per element). + pub fn encrypt_var(&self, x: &[u8]) -> Result { + stack::scrubbed(|| self.encrypt_var_unscrubbed(x)) + } + + /// Left-only ciphertext (smaller; for query plaintexts). + pub fn encrypt_left_var(&self, x: &[u8]) -> Result { + stack::scrubbed(|| self.encrypt_left_var_unscrubbed(x)) + } + + /// [`Self::encrypt_var`] without the stack scrub. Every caller runs it + /// inside [`stack::scrubbed`]: building a block's PRP and running the + /// accumulator leave their secrets in dead stack (see + /// `primitives::stack`). + fn encrypt_var_unscrubbed(&self, x: &[u8]) -> Result { + let count = x.len(); + debug_assert!(x.iter().all(|&b| (b as usize) < DOMAIN)); + if count > u16::MAX as usize { + return Err(OreError::TooManyBlocks); + } + + let mut acc = CmacAccumulator::new(&self.k_acc); + let mut nonce = [0u8; NONCE_LEN]; + self.rng.borrow_mut().try_fill(&mut nonce)?; + let hasher = FixedPiZ2Hash::new(HashKey::from_slice(&nonce)); + + let mut xt = Vec::with_capacity(count); + let mut f = Vec::with_capacity(count); + let mut blocks = Vec::with_capacity(count); + + // Scratch for the RO_KEY tags (then the H outputs) of one block. Every + // entry is rewritten by the next block, so it is wiped once after the + // loop rather than once per block. + let mut ro = [AesBlock::default(); DOMAIN]; + + for (n, &sym) in x.iter().enumerate() { + let n16 = n as u16; + let prp = self.prp_at(&acc, n16)?; + let permuted = prp.permute(sym)?; + xt.push(permuted); + + // ro_key for every domain value; f[n] = ro(n, xt[n]). + for (j, blk) in ro.iter_mut().enumerate() { + blk.copy_from_slice(&acc.finalize(&final_block( + Branch::RoKey, + n16, + j as u16, + WIDTH_BIT6, + ))); + } + // f[n] = ro(n, xt[n]), derived directly rather than read from `ro` + // at the secret index `permuted` (a 1 KiB table spans many cache + // lines). Same bytes as `encrypt_left_var`. + f.push(acc.finalize(&final_block( + Branch::RoKey, + n16, + permuted as u16, + WIDTH_BIT6, + ))); + + // right block = H-mask ⊕ indicator (trashes `ro`). + let mut rb = [0u8; RIGHT_LEN]; + hasher.hash_all_into(&mut ro, &mut rb); + prp.indicator_mask_xor(sym, &mut rb); + blocks.push(rb); + + acc.absorb(&prefix_block(n16, sym)); + } + + // `ro` held key-derived RO_KEY tags (then H outputs) for the last block. + for blk in ro.iter_mut() { + blk.as_mut_slice().zeroize(); + } + + Ok(VarCipherText { + left: VarLeft { xt, f }, + right: VarRight { nonce, blocks }, + }) + } + + /// [`Self::encrypt_left_var`] without the stack scrub; see + /// [`Self::encrypt_var_unscrubbed`]. + fn encrypt_left_var_unscrubbed(&self, x: &[u8]) -> Result { + let count = x.len(); + debug_assert!(x.iter().all(|&b| (b as usize) < DOMAIN)); + if count > u16::MAX as usize { + return Err(OreError::TooManyBlocks); + } + + let mut acc = CmacAccumulator::new(&self.k_acc); + let mut xt = Vec::with_capacity(count); + let mut f = Vec::with_capacity(count); + + for (n, &sym) in x.iter().enumerate() { + let n16 = n as u16; + let prp = self.prp_at(&acc, n16)?; + let permuted = prp.permute(sym)?; + xt.push(permuted); + f.push(acc.finalize(&final_block( + Branch::RoKey, + n16, + permuted as u16, + WIDTH_BIT6, + ))); + acc.absorb(&prefix_block(n16, sym)); + } + Ok(VarLeft { xt, f }) + } + + /// Encrypt a string (UTF-8 bytes → MSB-first 6-bit blocks). Strings + /// longer than [`MAX_STR_BYTES`] return [`OreError::TooManyBlocks`]. + /// + /// The ciphertext reveals the string's exact UTF-8 byte length, as well + /// as the order and common-prefix length that comparisons reveal; see + /// the module docs. + pub fn encrypt_str(&self, s: &str) -> Result { + stack::scrubbed(|| self.encrypt_var_unscrubbed(&str_to_blocks(s)?)) + } + + /// Left-only string ciphertext. Strings longer than [`MAX_STR_BYTES`] + /// return [`OreError::TooManyBlocks`]. Like [`Self::encrypt_str`], it + /// reveals the string's exact length. + pub fn encrypt_left_str(&self, s: &str) -> Result { + stack::scrubbed(|| self.encrypt_left_var_unscrubbed(&str_to_blocks(s)?)) + } + + /// Compare two serialised full ciphertexts (lexicographic; shorter prefix + /// sorts first). `None` if either is not a well-formed ciphertext of this + /// scheme. + pub fn compare_raw_slices(a: &[u8], b: &[u8]) -> Option { + let ((va, sa, ca), _) = parse_header(a).ok()?; + let ((vb, sb, cb), _) = parse_header(b).ok()?; + if va != VERSION || sa != SCHEME_ID || vb != VERSION || sb != SCHEME_ID { + return None; + } + if a.len() != total_len(ca) || b.len() != total_len(cb) { + return None; + } + if !symbols_in_domain(a, ca) || !symbols_in_domain(b, cb) { + return None; + } + // Only `a`'s left half and `b`'s right half are read (Lewi-Wu + // asymmetry); the left-only query path is `compare_left_to_full`. + Some(Self::compare_views(a, ca, b, cb)) + } + + /// Compare a left-only (query) ciphertext `left` — produced by + /// [`Self::encrypt_left_var`]/[`Self::encrypt_left_str`] — against a stored + /// full ciphertext `full`. This is the Lewi-Wu query path: a comparison + /// needs only the query's left half (`xt`/`f`) and the stored ciphertext's + /// right half, so the smaller left-only artifact suffices. This does not + /// make stored data safe at rest: a stored *full* ciphertext also carries + /// its left half, so two stored rows compare offline (see the module docs). Returns `left`'s order relative to + /// `full` (`Less` ⇒ the query plaintext sorts before the stored one). + /// `None` if either input is malformed. + pub fn compare_left_to_full(left: &[u8], full: &[u8]) -> Option { + let ((vl, sl, cl), _) = parse_header(left).ok()?; + let ((vf, sf, cf), _) = parse_header(full).ok()?; + if vl != VERSION || sl != SCHEME_ID || vf != VERSION || sf != SCHEME_ID { + return None; + } + // `left` is `header ‖ xt ‖ f` (no nonce/right); `full` is complete. + if left.len() != left_len(cl) || full.len() != total_len(cf) { + return None; + } + if !symbols_in_domain(left, cl) || !symbols_in_domain(full, cf) { + return None; + } + Some(Self::compare_views(left, cl, full, cf)) + } + + /// Core comparator. `a` supplies the **left view** (`header ‖ xt ‖ f` — the + /// layout is identical for a `VarLeft` and the left prefix of a full + /// ciphertext, so the `xt_at`/`f_at` accessors work on either); `b` supplies + /// the **right half**. Constant-time scan of the shared prefix; if it + /// matches throughout, the shorter sorts first; otherwise the first + /// differing block is resolved via the random oracle against `b`'s right + /// block. `a` reads only `xt`/`f`, `b` reads `xt`/`f`/`nonce`/`right`. + /// + /// The scan reads every block of the shared prefix, including `b`'s right + /// blocks, and latches the first differing block's `xt`, `f` and `right` + /// as it goes; the resolution step works from the latched copies, so no + /// load is indexed by the position of the first difference. A direct + /// `right_at(b, cb, l)` after the scan was the one load whose cache state + /// depended on `l` (the scan never needed the right blocks), and dudect + /// measured it as a timing signal; see `docs/reviews/`. + fn compare_views(a: &[u8], ca: usize, b: &[u8], cb: usize) -> Ordering { + let min_count = ca.min(cb); + let mut is_equal = Choice::from(1u8); + let mut sel_xt = 0u8; + let mut sel_f = [0u8; F_LEN]; + let mut sel_right = [0u8; RIGHT_LEN]; + for n in 0..min_count { + let differs = !xt_at(a, n).ct_eq(&xt_at(b, n)) | !f_at(a, ca, n).ct_eq(f_at(b, cb, n)); + // Set for exactly one `n`: the first differing block. + let first = is_equal & differs; + sel_xt.conditional_assign(&xt_at(a, n), first); + ct_assign_bytes(&mut sel_f, f_at(a, ca, n), first); + ct_assign_bytes(&mut sel_right, right_at(b, cb, n), first); + is_equal.conditional_assign(&Choice::from(0u8), first); + } + + if bool::from(is_equal) { + // Shared prefix matches throughout — shorter sorts first. + return ca.cmp(&cb); + } + + let hasher = FixedPiZ2Hash::new(HashKey::from_slice(nonce_at(b, cb))); + let h = hasher.hash(&sel_f); + let byte = ct_select_byte(&sel_right, (sel_xt / 8) as usize); + let test = ct_bit(byte, sel_xt % 8) ^ h; + if test == 1 { + Ordering::Greater + } else { + Ordering::Less + } + } +} + +impl Drop for OreAes128Bit6Chained { + fn drop(&mut self) { + self.k_acc.zeroize(); + } +} + +/// Whether every `xt` symbol of a serialised view is inside the 64-element +/// domain. The comparator reads the right block at `xt[l]`; a symbol of 64 or +/// more would select no byte and silently read 0. `xt` is public, so this may +/// branch. +fn symbols_in_domain(view: &[u8], count: usize) -> bool { + (0..count).all(|i| usize::from(xt_at(view, i)) < DOMAIN) +} + +/// Longest string, in UTF-8 bytes, whose block count fits the wire format's +/// `u16`: `⌈8n/6⌉ ≤ 65_535` gives `n ≤ 49_151`. +pub const MAX_STR_BYTES: usize = u16::MAX as usize * 6 / 8; + +const _: () = assert!(num_blocks_6bit(MAX_STR_BYTES) <= u16::MAX as usize); +const _: () = assert!(num_blocks_6bit(MAX_STR_BYTES + 1) > u16::MAX as usize); + +/// Decompose `s` into 6-bit blocks, refusing a string too long for the wire +/// format *before* allocating or decomposing it. The decomposition is the +/// whole plaintext (it inverts), so the buffer is wiped when dropped. +fn str_to_blocks(s: &str) -> Result>, OreError> { + let bytes = s.as_bytes(); + if bytes.len() > MAX_STR_BYTES { + return Err(OreError::TooManyBlocks); + } + let nb = num_blocks_6bit(bytes.len()); + let mut blocks = Zeroizing::new(vec![0u8; nb]); + if nb > 0 { + decompose_6bit(bytes, &mut blocks); + } + Ok(blocks) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn ore() -> OreAes128Bit6ChainedChaCha20 { + OreAes128Bit6Chained::init(&[0x11; 16]).unwrap() + } + + #[test] + fn strings_over_the_wire_limit_are_rejected() { + let c = ore(); + let too_long = "a".repeat(MAX_STR_BYTES + 1); + assert!(matches!( + c.encrypt_str(&too_long), + Err(OreError::TooManyBlocks) + )); + assert!(matches!( + c.encrypt_left_str(&too_long), + Err(OreError::TooManyBlocks) + )); + } + + #[test] + fn comparators_reject_out_of_domain_symbols() { + let c = ore(); + let a = c.encrypt_str("apple").unwrap().to_bytes(); + let b = c.encrypt_str("apricot").unwrap().to_bytes(); + let q = c.encrypt_left_str("apple").unwrap().to_bytes(); + assert!(OreAes128Bit6Chained::::compare_raw_slices(&a, &b).is_some()); + + // First `xt` byte follows the 4-byte header. + let mut bad = a.clone(); + bad[HEADER_LEN] = DOMAIN as u8; + let mut bad_q = q.clone(); + bad_q[HEADER_LEN] = DOMAIN as u8; + + type C = OreAes128Bit6Chained; + assert_eq!(C::compare_raw_slices(&bad, &b), None); + assert_eq!(C::compare_raw_slices(&b, &bad), None); + assert_eq!(C::compare_left_to_full(&bad_q, &b), None); + assert_eq!(C::compare_left_to_full(&q, &bad), None); + } + + fn cmp(c: &OreAes128Bit6ChainedChaCha20, a: &str, b: &str) -> Ordering { + let ca = c.encrypt_str(a).unwrap().to_bytes(); + let cb = c.encrypt_str(b).unwrap().to_bytes(); + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&ca, &cb).unwrap() + } + + #[test] + fn string_total_order_matches_lexicographic() { + let c = ore(); + // Includes shared prefixes of different lengths and beyond 14 blocks. + let mut words = [ + "", + "a", + "ab", + "abc", + "abd", + "ac", + "b", + "banana", + "bananas", + "the quick brown fox jumps over the lazy dog", + "the quick brown fox jumps over the lazy dog.", + "zzz", + ]; + for &a in &words { + for &b in &words { + assert_eq!(cmp(&c, a, b), a.cmp(b), "order mismatch for {a:?} vs {b:?}"); + } + } + words.sort(); + // sanity: sort is stable and the list is what we expect + assert_eq!(words[0], ""); + } + + #[test] + fn equality_same_plaintext_across_nonces() { + let c = ore(); + let a = c.encrypt_str("hello world").unwrap().to_bytes(); + let b = c.encrypt_str("hello world").unwrap().to_bytes(); + assert_ne!(a, b, "nonces should differ"); // different nonce streams + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&a, &b), + Some(Ordering::Equal) + ); + } + + #[test] + fn serialize_roundtrip() { + let c = ore(); + let ct = c.encrypt_str("roundtrip me").unwrap(); + let bytes = ct.to_bytes(); + let parsed = VarCipherText::from_slice(&bytes).unwrap(); + assert_eq!(parsed.to_bytes(), bytes); + } + + #[test] + fn beyond_packed_cap() { + // > 14 6-bit blocks (the fixed-N cap) must work and order correctly. + let c = ore(); + let long_a = "aaaaaaaaaaaaaaaaaaaaaaaa"; // 24 bytes -> 32 blocks + let long_b = "aaaaaaaaaaaaaaaaaaaaaaab"; + assert_eq!(cmp(&c, long_a, long_b), Ordering::Less); + assert_eq!(cmp(&c, long_a, long_a), Ordering::Equal); + } + + #[test] + fn cross_scheme_bytes_rejected() { + let c = ore(); + let mut bytes = c.encrypt_str("x").unwrap().to_bytes(); + bytes[1] = 0x02; // pretend it's the fixed-N scheme id + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&bytes, &bytes), + None + ); + } + + #[test] + fn left_query_path_and_artifact_rejection() { + let c = ore(); + let left = c.encrypt_left_str("hi").unwrap().to_bytes(); + let full = c.encrypt_str("hi").unwrap().to_bytes(); + // The query path accepts (left, full) and reports the order. + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_left_to_full(&left, &full), + Some(Ordering::Equal) + ); + // A left-only artifact is not a full ciphertext, and vice versa, so the + // length checks reject the swapped/mismatched cases. + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&left, &full), + None + ); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_left_to_full(&full, &full), + None + ); + } + + #[test] + fn empty_string_sorts_first() { + let c = ore(); + assert_eq!(cmp(&c, "", "a"), Ordering::Less); + assert_eq!(cmp(&c, "", ""), Ordering::Equal); + } + + // --- Property tests -------------------------------------------------- + // + // The contract under test is "the comparator reproduces the lexicographic + // order of the plaintext", over arbitrary inputs and (independent) lengths — + // not just the hand-picked words above. Lengths are capped so each case + // stays cheap (~97 AES ops/block) while still routinely exceeding the + // 14-block fixed-N packed-prefix cap. + const PROP_MAX_BLOCKS: usize = 32; + + /// Map arbitrary bytes into the 6-bit block domain, length-capped. + fn to_domain(v: &[u8]) -> Vec { + v.iter().take(PROP_MAX_BLOCKS).map(|b| b & 0x3f).collect() + } + + fn enc(c: &OreAes128Bit6ChainedChaCha20, x: &[u8]) -> Vec { + c.encrypt_var(x).unwrap().to_bytes() + } + + quickcheck! { + /// Headline contract: the comparator reproduces the lexicographic order + /// of the underlying 6-bit block sequences, for arbitrary blocks and + /// arbitrary (independent) lengths — the full domain `0..64`, including + /// values that string inputs never produce. + fn prop_block_order_matches_lex(a: Vec, b: Vec) -> bool { + let c = ore(); + let (a, b) = (to_domain(&a), to_domain(&b)); + let (ea, eb) = (enc(&c, &a), enc(&c, &b)); + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&ea, &eb) == Some(a.cmp(&b)) + } + + /// Lewi-Wu query path: a left-only (query) ciphertext compared against a + /// stored full ciphertext reproduces the plaintext order — + /// `compare_left_to_full(left(a), full(b)) == a.cmp(b)`. + fn prop_left_query_matches_full(a: Vec, b: Vec) -> bool { + let c = ore(); + let (a, b) = (to_domain(&a), to_domain(&b)); + let left_a = c.encrypt_left_var(&a).unwrap().to_bytes(); + let full_b = enc(&c, &b); + OreAes128Bit6ChainedChaCha20::compare_left_to_full(&left_a, &full_b) + == Some(a.cmp(&b)) + } + + /// Same, with a forced shared prefix — exercises the constant-time + /// prefix scan and first-differing-block selection at controlled common + /// lengths (independent random inputs almost never share a prefix). + fn prop_shared_prefix_order(prefix: Vec, sa: Vec, sb: Vec) -> bool { + let c = ore(); + let p = to_domain(&prefix); + let mut a = p.clone(); + a.extend(to_domain(&sa)); + a.truncate(PROP_MAX_BLOCKS); + let mut b = p; + b.extend(to_domain(&sb)); + b.truncate(PROP_MAX_BLOCKS); + let (ea, eb) = (enc(&c, &a), enc(&c, &b)); + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&ea, &eb) == Some(a.cmp(&b)) + } + + /// String comparison matches `str::cmp` across arbitrary Unicode and + /// independent lengths: the MSB-first 6-bit packing is order-preserving + /// even with tail zero-padding (cross-length prefix case included). + fn prop_string_order_matches_str(a: String, b: String) -> bool { + let c = ore(); + let a: String = a.chars().take(12).collect(); + let b: String = b.chars().take(12).collect(); + let ea = c.encrypt_str(&a).unwrap().to_bytes(); + let eb = c.encrypt_str(&b).unwrap().to_bytes(); + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&ea, &eb) == Some(a.cmp(&b)) + } + + /// Two encryptions of one plaintext (fresh nonces each) compare Equal, + /// and never produce identical bytes (the nonce streams differ). + fn prop_equality_across_nonces(x: Vec) -> bool { + let c = ore(); + let x = to_domain(&x); + let (a, b) = (enc(&c, &x), enc(&c, &x)); + let equal = OreAes128Bit6ChainedChaCha20::compare_raw_slices(&a, &b) + == Some(Ordering::Equal); + // Empty plaintexts have no right blocks, so their bytes can collide; + // only require nonce divergence when there is masked material. + let distinct = x.is_empty() || a != b; + equal && distinct + } + + /// The left ciphertext is deterministic (it has no nonce) and is + /// byte-identical to the left half embedded in a full ciphertext — the + /// two encrypt paths must not drift. + fn prop_left_deterministic_and_consistent(x: Vec) -> bool { + let c = ore(); + let x = to_domain(&x); + let l1 = c.encrypt_left_var(&x).unwrap().to_bytes(); + let l2 = c.encrypt_left_var(&x).unwrap().to_bytes(); + let full = c.encrypt_var(&x).unwrap(); + l1 == l2 && full.left.to_bytes() == l1 + } + + /// Serialised full ciphertexts round-trip through `from_slice`. + fn prop_serialize_roundtrip(x: Vec) -> bool { + let c = ore(); + let x = to_domain(&x); + let bytes = enc(&c, &x); + match VarCipherText::from_slice(&bytes) { + Ok(ct) => ct.to_bytes() == bytes, + Err(_) => false, + } + } + } + + /// Every block's PRP tables for `s`, which must not survive an + /// encryption in dead stack (review of #82 and #83). + fn block_tables(ore: &OreAes128Bit6ChainedChaCha20, s: &str) -> Vec> { + let blocks = str_to_blocks(s).unwrap(); + let mut acc = CmacAccumulator::new(&ore.k_acc); + let mut needles = Vec::new(); + for (n, &sym) in blocks.iter().enumerate() { + let prp = ore.prp_at(&acc, n as u16).unwrap(); + needles.push((0..64).map(|i| prp.permute(i).unwrap()).collect()); + needles.push((0..64).map(|i| prp.invert(i).unwrap()).collect()); + acc.absorb(&prefix_block(n as u16, sym)); + } + needles + } + + fn assert_no_block_tables( + encrypt: fn(&OreAes128Bit6ChainedChaCha20, &str), + unscrubbed: fn(&OreAes128Bit6ChainedChaCha20, &str), + ) { + use crate::primitives::stack::{residue, SCRUB_BYTES}; + const S: &str = "a chained residue test string"; + let needles = block_tables(&ore(), S); + + // Positive control and depth, as in the Bit6 residue tests. + let dead = residue::after(move || unscrubbed(&ore(), S)); + assert!(needles.iter().any(|n| residue::contains(&dead, n))); + let depth = residue::depth(&dead); + assert!( + depth < SCRUB_BYTES, + "encryption reached {} bytes of stack, past the {}-byte scrub", + depth, + SCRUB_BYTES + ); + + let dead = residue::after(move || encrypt(&ore(), S)); + for (i, needle) in needles.iter().enumerate() { + assert!( + !residue::contains(&dead, needle), + "block {} {} left in dead stack", + i / 2, + ["permutation", "inverse"][i % 2] + ); + } + } + + #[test] + fn encrypt_str_leaves_no_prp_tables_on_the_stack() { + assert_no_block_tables( + |ore, s| { + core::hint::black_box(ore.encrypt_str(s).unwrap()); + }, + |ore, s| { + let blocks = str_to_blocks(s).unwrap(); + core::hint::black_box(ore.encrypt_var_unscrubbed(&blocks).unwrap()); + }, + ); + } + + #[test] + fn encrypt_left_str_leaves_no_prp_tables_on_the_stack() { + assert_no_block_tables( + |ore, s| { + core::hint::black_box(ore.encrypt_left_str(s).unwrap()); + }, + |ore, s| { + let blocks = str_to_blocks(s).unwrap(); + core::hint::black_box(ore.encrypt_left_var_unscrubbed(&blocks).unwrap()); + }, + ); + } +} diff --git a/packages/ore-rs/tests/compat_chained_vectors.rs b/packages/ore-rs/tests/compat_chained_vectors.rs new file mode 100644 index 00000000..361ab600 --- /dev/null +++ b/packages/ore-rs/tests/compat_chained_vectors.rs @@ -0,0 +1,387 @@ +//! Wire-format vectors for the `OreAes128Bit6Chained` scheme — the v2 +//! variable-length (chained-prefix) scheme with the v2 wire header, the CMAC +//! accumulator (review brief A2/A3) and the BHKR σ-MMO hash `H` (A1). +//! +//! These tests pin the exact serialised bytes produced for a fixed key and +//! fixed inputs, plus comparison results over those bytes. They freeze the v2 +//! chained wire format introduced by this PR: once released, any change that +//! alters these bytes is a wire-format break for stored chained ciphertexts +//! and must fail here. (Mirror of `compat_w6_vectors.rs`, which does the same +//! for the fixed-width Bit6 scheme.) +//! +//! Left ciphertexts are deterministic given the key. Full ciphertexts include +//! a random nonce drawn from the cipher's internal RNG, which `init` seeds via +//! `SeedableRng::from_entropy`; [`TestRng`] overrides `from_entropy` to a +//! fixed seed so full-ciphertext bytes are reproducible. +//! +//! To regenerate (only legitimate if the wire format is *deliberately* changed): +//! +//! ```text +//! cargo test --test compat_chained_vectors -- --ignored --nocapture generate +//! ``` + +use ore_rs::scheme::chained::{OreAes128Bit6Chained, OreAes128Bit6ChainedChaCha20, VarCipherText}; +use rand::{RngCore, SeedableRng}; +use rand_chacha::ChaCha20Rng; +use std::cmp::Ordering; + +const K1: [u8; 16] = [ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, +]; + +/// RNG whose `from_entropy` is deterministic, so that `init` (which calls +/// `from_entropy` internally) produces a reproducible nonce stream. +struct TestRng(ChaCha20Rng); + +impl RngCore for TestRng { + fn next_u32(&mut self) -> u32 { + self.0.next_u32() + } + fn next_u64(&mut self) -> u64 { + self.0.next_u64() + } + fn fill_bytes(&mut self, dest: &mut [u8]) { + self.0.fill_bytes(dest) + } + fn try_fill_bytes(&mut self, dest: &mut [u8]) -> Result<(), rand::Error> { + self.0.try_fill_bytes(dest) + } +} + +impl SeedableRng for TestRng { + type Seed = [u8; 32]; + fn from_seed(seed: Self::Seed) -> Self { + Self(ChaCha20Rng::from_seed(seed)) + } + fn from_entropy() -> Self { + Self::from_seed([SEED; 32]) + } +} + +type OreA = OreAes128Bit6Chained>; +type OreB = OreAes128Bit6Chained>; + +fn cipher_a() -> OreA { + OreAes128Bit6Chained::init(&K1).unwrap() +} + +fn cipher_b() -> OreB { + OreAes128Bit6Chained::init(&K1).unwrap() +} + +fn cipher_left() -> OreAes128Bit6ChainedChaCha20 { + OreAes128Bit6Chained::init(&K1).unwrap() +} + +/// Strings pinned as full ciphertexts. Ordered so that adjacent pairs exercise +/// each comparator path: equal prefix then shorter-first (`""` < `"a"`, +/// `"alice"` < `"alice@example.com"`), a difference in the last, padded +/// block (`"alice"` vs `"alicf"`), and a difference in the first block. +const STRINGS: &[&str] = &["", "a", "alice", "alice@example.com", "alicf", "bob"]; + +/// Raw 6-bit symbol sequences pinned as full ciphertexts: the two domain +/// edges, a repeated symbol (distinct permutations per position), and the +/// longest value of one byte. +const SYMBOLS: &[&[u8]] = &[&[0], &[63], &[0, 0, 0], &[63, 63], &[1, 2, 3, 4, 5]]; + +#[path = "compat_chained_vectors/vectors.rs"] +mod vectors; +use vectors::*; + +// --------------------------------------------------------------------------- +// Left ciphertexts (fully deterministic) +// --------------------------------------------------------------------------- + +#[test] +fn left_str_vectors() { + let ore = cipher_left(); + for (s, expected) in LEFT_STR { + let left = ore.encrypt_left_str(s).unwrap(); + assert_eq!( + hex::encode(left.to_bytes()), + *expected, + "left ciphertext mismatch for {s:?}" + ); + } +} + +#[test] +fn left_symbol_vectors() { + let ore = cipher_left(); + for (syms, expected) in LEFT_SYMBOLS { + let left = ore.encrypt_left_var(syms).unwrap(); + assert_eq!( + hex::encode(left.to_bytes()), + *expected, + "left ciphertext mismatch for symbols {syms:?}" + ); + } +} + +// --------------------------------------------------------------------------- +// Full ciphertexts (deterministic via TestRng nonce stream) +// --------------------------------------------------------------------------- + +#[test] +fn full_str_vectors() { + for (s, expected) in FULL_STR { + let ct = cipher_a().encrypt_str(s).unwrap(); + assert_eq!( + hex::encode(ct.to_bytes()), + *expected, + "full ciphertext mismatch for {s:?}" + ); + } +} + +#[test] +fn full_symbol_vectors() { + for (syms, expected) in FULL_SYMBOLS { + let ct = cipher_a().encrypt_var(syms).unwrap(); + assert_eq!( + hex::encode(ct.to_bytes()), + *expected, + "full ciphertext mismatch for symbols {syms:?}" + ); + } +} + +#[test] +fn full_alice_alternate_nonce() { + let ct = cipher_b().encrypt_str("alice").unwrap(); + assert_eq!(hex::encode(ct.to_bytes()), FULL_ALICE_SEED_B); +} + +/// The left half of a full ciphertext is byte-identical to the left-only +/// artefact: `compare_left_to_full` relies on the shared layout. +#[test] +fn full_ciphertext_starts_with_the_left_ciphertext() { + for ((s, left), (s2, full)) in LEFT_STR.iter().zip(FULL_STR) { + assert_eq!(s, s2); + assert!(full.starts_with(left), "left prefix mismatch for {:?}", s); + } +} + +// --------------------------------------------------------------------------- +// Comparison fixtures over pinned bytes (no RNG involved) +// --------------------------------------------------------------------------- + +fn pinned(hex_str: &str) -> Vec { + hex::decode(hex_str).unwrap() +} + +/// Every pair of pinned strings compares as the plaintexts do (byte-wise, +/// shorter prefix first), in both argument orders. +#[test] +fn compare_raw_slices_matches_plaintext_order() { + for (sa, ha) in FULL_STR { + for (sb, hb) in FULL_STR { + let expected = sa.as_bytes().cmp(sb.as_bytes()); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&pinned(ha), &pinned(hb)), + Some(expected), + "compare_raw_slices({sa:?}, {sb:?})" + ); + } + } +} + +#[test] +fn compare_raw_slices_matches_symbol_order() { + for (xa, ha) in FULL_SYMBOLS { + for (xb, hb) in FULL_SYMBOLS { + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&pinned(ha), &pinned(hb)), + Some(xa.cmp(xb)), + "compare_raw_slices({xa:?}, {xb:?})" + ); + } + } +} + +/// The query path: a pinned left-only ciphertext against every pinned full +/// ciphertext. +#[test] +fn compare_left_to_full_matches_plaintext_order() { + for (sa, hl) in LEFT_STR { + for (sb, hf) in FULL_STR { + let expected = sa.as_bytes().cmp(sb.as_bytes()); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_left_to_full(&pinned(hl), &pinned(hf)), + Some(expected), + "compare_left_to_full({sa:?}, {sb:?})" + ); + } + } +} + +#[test] +fn compare_raw_slices_equality_across_nonces() { + let alice_a = pinned(FULL_STR.iter().find(|(s, _)| *s == "alice").unwrap().1); + let alice_b = pinned(FULL_ALICE_SEED_B); + assert_ne!( + alice_a, alice_b, + "different nonces must give different bytes" + ); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&alice_a, &alice_b), + Some(Ordering::Equal) + ); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&alice_b, &alice_a), + Some(Ordering::Equal) + ); +} + +#[test] +fn pinned_bytes_round_trip_the_parser() { + let strs = FULL_STR.iter().map(|(_, h)| *h); + let syms = FULL_SYMBOLS.iter().map(|(_, h)| *h); + for h in strs.chain(syms) { + let bytes = pinned(h); + let ct = VarCipherText::from_slice(&bytes).unwrap(); + assert_eq!(ct.to_bytes(), bytes); + } +} + +// --------------------------------------------------------------------------- +// Malformed input over pinned bytes +// --------------------------------------------------------------------------- + +/// A pinned ciphertext with its first `xt` byte (just after the 4-byte +/// header) set to 64, one past the Bit6 symbol domain. +fn with_out_of_domain_symbol(hex_str: &str) -> Vec { + let mut bytes = pinned(hex_str); + bytes[4] = 64; + bytes +} + +fn full_alice() -> &'static str { + FULL_STR.iter().find(|(s, _)| *s == "alice").unwrap().1 +} + +fn left_alice() -> &'static str { + LEFT_STR.iter().find(|(s, _)| *s == "alice").unwrap().1 +} + +#[test] +fn parser_rejects_out_of_domain_symbols() { + assert!(VarCipherText::from_slice(&with_out_of_domain_symbol(full_alice())).is_err()); + + // The highest in-domain symbol still parses. + let mut edge = pinned(full_alice()); + edge[4] = 63; + assert!(VarCipherText::from_slice(&edge).is_ok()); +} + +#[test] +fn comparators_reject_out_of_domain_symbols() { + let good = pinned(full_alice()); + let bad = with_out_of_domain_symbol(full_alice()); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&bad, &good), + None + ); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&good, &bad), + None + ); + + let bad_left = with_out_of_domain_symbol(left_alice()); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_left_to_full(&bad_left, &good), + None + ); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_left_to_full(&pinned(left_alice()), &bad), + None + ); +} + +#[test] +fn comparators_reject_wrong_lengths_and_headers() { + let full = pinned(full_alice()); + let left = pinned(left_alice()); + + // A left-only artefact is not a full ciphertext, and vice versa. + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&left, &full), + None + ); + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_left_to_full(&full, &full), + None + ); + assert!(VarCipherText::from_slice(&left).is_err()); + + // Truncation and extension. + assert!(VarCipherText::from_slice(&full[..full.len() - 1]).is_err()); + let mut longer = full.clone(); + longer.push(0); + assert!(VarCipherText::from_slice(&longer).is_err()); + + // Wrong version / scheme id. + let mut v = full.clone(); + v[0] ^= 1; + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&v, &full), + None + ); + let mut s = full.clone(); + s[1] ^= 1; + assert_eq!( + OreAes128Bit6ChainedChaCha20::compare_raw_slices(&s, &full), + None + ); +} + +#[test] +#[ignore = "generator: prints the contents of tests/compat_chained_vectors/vectors.rs"] +fn generate() { + println!("// Contents of tests/compat_chained_vectors/vectors.rs"); + println!( + "// Generated by `cargo test --test compat_chained_vectors -- --ignored --nocapture generate`" + ); + println!("// DO NOT regenerate unless deliberately breaking the wire format (see plan doc)."); + println!(); + + let ore = cipher_left(); + + println!("pub const LEFT_STR: &[(&str, &str)] = &["); + for s in STRINGS { + let left = ore.encrypt_left_str(s).unwrap(); + println!(" ({s:?}, \"{}\"),", hex::encode(left.to_bytes())); + } + println!("];"); + println!(); + + println!("pub const LEFT_SYMBOLS: &[(&[u8], &str)] = &["); + for syms in SYMBOLS { + let left = ore.encrypt_left_var(syms).unwrap(); + println!(" (&{syms:?}, \"{}\"),", hex::encode(left.to_bytes())); + } + println!("];"); + println!(); + + println!("pub const FULL_STR: &[(&str, &str)] = &["); + for s in STRINGS { + let ct = cipher_a().encrypt_str(s).unwrap(); + println!(" ({s:?}, \"{}\"),", hex::encode(ct.to_bytes())); + } + println!("];"); + println!(); + + println!("pub const FULL_SYMBOLS: &[(&[u8], &str)] = &["); + for syms in SYMBOLS { + let ct = cipher_a().encrypt_var(syms).unwrap(); + println!(" (&{syms:?}, \"{}\"),", hex::encode(ct.to_bytes())); + } + println!("];"); + println!(); + + let ct = cipher_b().encrypt_str("alice").unwrap(); + println!( + "pub const FULL_ALICE_SEED_B: &str = \"{}\";", + hex::encode(ct.to_bytes()) + ); +} diff --git a/packages/ore-rs/tests/compat_chained_vectors/vectors.rs b/packages/ore-rs/tests/compat_chained_vectors/vectors.rs new file mode 100644 index 00000000..8343c8c3 --- /dev/null +++ b/packages/ore-rs/tests/compat_chained_vectors/vectors.rs @@ -0,0 +1,39 @@ +// Contents of tests/compat_chained_vectors/vectors.rs +// Generated by `cargo test --test compat_chained_vectors -- --ignored --nocapture generate` +// DO NOT regenerate unless deliberately breaking the wire format (see plan doc). + +pub const LEFT_STR: &[(&str, &str)] = &[ + ("", "02030000"), + ("a", "02030002390a367ae8e098c5628373545e3027b39400358726214f304c632146006fb0b5ace3"), + ("alice", "020300073933281b2b370c367ae8e098c5628373545e3027b3940019889a97dee6bd3b45d3e9df587803f858651a8d94901d58e061adcbe4a2873f98ce0340bbca2f8c50b6037d68324d54934c1d5eeccca87af1a0ddfabb106717331ae99d5a41b641fe7ea107efd8108e1eb8c874c638b793ace396ae840b9689"), + ("alice@example.com", "020300173933281b2b37132a0014160c3e3129120f063a0917001b367ae8e098c5628373545e3027b3940019889a97dee6bd3b45d3e9df587803f858651a8d94901d58e061adcbe4a2873f98ce0340bbca2f8c50b6037d68324d54934c1d5eeccca87af1a0ddfabb106717331ae99d5a41b641fe7ea107efd8108e6231054d42fa30f28f14b0ccc9f11cf34ec1caec247464ff4de3e0f87410d2b026e7e5216966e4a04eedbea4b6fdaab2fbbdb4737c21c0436cecc1ddfd881828ede865ec290494b53d0d5dbe5bdd8dfbcde78e87f683c66e34f8943fc6786578432071b6e7e72a905e270116bc8471b07e3caf279c190af16050e3768b6b37bc1ad1a5a1e1e8d8f190ac02e7ac26001ea5130b6c307eecaf19d746dd6979b67f74d661a2f02fae928555213e951bcebcba4964601ebebdfc2c8bb13b5d899e5afc3da351917c1b8aa09f300d6024ef6d7895a8c5dcaf0ac48fcfea1326da41f2a3858d1226f4507ca6853b3b050f8b22922eda12490f4c38f0d1a8c8fdad4319865233cf05adbd805acdf908a13c2504"), + ("alicf", "020300073933281b2b370f367ae8e098c5628373545e3027b3940019889a97dee6bd3b45d3e9df587803f858651a8d94901d58e061adcbe4a2873f98ce0340bbca2f8c50b6037d68324d54934c1d5eeccca87af1a0ddfabb106717331ae99d5a41b641fe7ea107efd8108e5a463f1ba05316b6c7337631775afaf6"), + ("bob", "0203000439161537367ae8e098c5628373545e3027b394003da2a56461c75d557b49be80ffe925eccb16677fb78c701aec7e9917fcf1ec517ba5f2c16fec4488f1ee9a6d431e2ee6"), +]; + +pub const LEFT_SYMBOLS: &[(&[u8], &str)] = &[ + (&[0], "020300011d5d826fb3dfbb8090bbcde5ed24acacf0"), + (&[63], "020300011cfc9c0ae91c66774477c4a9846a8f65aa"), + (&[0, 0, 0], "020300031d3d175d826fb3dfbb8090bbcde5ed24acacf0fbc06a3f7f2751d6db18b546cc98c3d0aeaaf841bf3cb2fba85d7b1d1b9d4157"), + (&[63, 63], "020300021c01fc9c0ae91c66774477c4a9846a8f65aa7fb660a84a5e7f8872d29acbc1557e78"), + (&[1, 2, 3, 4, 5], "0203000522140f253b9d23835cb5386f80380625d3e8367f426027b0e3de1522c0466e10a5b4359e05afa7dd343bfa153c2e00b46f574eebaf450a0a342b0212db53564cf50b16441632474e5b2cc82a27e5d9e8aefa56fb5d"), +]; + +pub const FULL_STR: &[(&str, &str)] = &[ + ("", "0203000098191f46e5830216445436978803697a"), + ("a", "02030002390a367ae8e098c5628373545e3027b39400358726214f304c632146006fb0b5ace398191f46e5830216445436978803697ad909aecea3e41349e00b9cae7013ec8a"), + ("alice", "020300073933281b2b370c367ae8e098c5628373545e3027b3940019889a97dee6bd3b45d3e9df587803f858651a8d94901d58e061adcbe4a2873f98ce0340bbca2f8c50b6037d68324d54934c1d5eeccca87af1a0ddfabb106717331ae99d5a41b641fe7ea107efd8108e1eb8c874c638b793ace396ae840b968998191f46e5830216445436978803697ad909aecea3e41349e02b95ae7093e40add632becdd226d0b708753fab7a69a596e20b624d50e3853b8aa1c1297938c88fef0cca24a0a06cc"), + ("alice@example.com", "020300173933281b2b37132a0014160c3e3129120f063a0917001b367ae8e098c5628373545e3027b3940019889a97dee6bd3b45d3e9df587803f858651a8d94901d58e061adcbe4a2873f98ce0340bbca2f8c50b6037d68324d54934c1d5eeccca87af1a0ddfabb106717331ae99d5a41b641fe7ea107efd8108e6231054d42fa30f28f14b0ccc9f11cf34ec1caec247464ff4de3e0f87410d2b026e7e5216966e4a04eedbea4b6fdaab2fbbdb4737c21c0436cecc1ddfd881828ede865ec290494b53d0d5dbe5bdd8dfbcde78e87f683c66e34f8943fc6786578432071b6e7e72a905e270116bc8471b07e3caf279c190af16050e3768b6b37bc1ad1a5a1e1e8d8f190ac02e7ac26001ea5130b6c307eecaf19d746dd6979b67f74d661a2f02fae928555213e951bcebcba4964601ebebdfc2c8bb13b5d899e5afc3da351917c1b8aa09f300d6024ef6d7895a8c5dcaf0ac48fcfea1326da41f2a3858d1226f4507ca6853b3b050f8b22922eda12490f4c38f0d1a8c8fdad4319865233cf05adbd805acdf908a13c250498191f46e5830216445436978803697ad909aecea3e41349e02b95ae7093e40add632becdd226d0b708753fab7a69a596e20b624d50e3853b8aa1c1297938c88fef0c4a24a0a06ccf0b58d236c1826fb178bf59f98f204ca04fc00282f549d807f1e7d1be9eda18174bebc786a2081d1df4ec95191152515e4b984912fd288f281d0affcd973b418a04287e5ea17159e056982e610220721676a8491be93fb6fa3c438955fa2292f5b9e27ee548f4e683bcf07d200bfdc8efc7beb0b3ac1e7a3b8885a16beee31d2"), + ("alicf", "020300073933281b2b370f367ae8e098c5628373545e3027b3940019889a97dee6bd3b45d3e9df587803f858651a8d94901d58e061adcbe4a2873f98ce0340bbca2f8c50b6037d68324d54934c1d5eeccca87af1a0ddfabb106717331ae99d5a41b641fe7ea107efd8108e5a463f1ba05316b6c7337631775afaf698191f46e5830216445436978803697ad909aecea3e41349e02b95ae7093e40add632becdd226d0b708753fab7a69a596e20b624d50e3853b8aa1c1297938c88fe70c4aa4a0a06ec"), + ("bob", "0203000439161537367ae8e098c5628373545e3027b394003da2a56461c75d557b49be80ffe925eccb16677fb78c701aec7e9917fcf1ec517ba5f2c16fec4488f1ee9a6d431e2ee698191f46e5830216445436978803697ad909aecea3e41349e42bd7ddb591651a358d939875e2204f27ae45ae6d051c06"), +]; + +pub const FULL_SYMBOLS: &[(&[u8], &str)] = &[ + (&[0], "020300011d5d826fb3dfbb8090bbcde5ed24acacf098191f46e5830216445436978803697a848f024abe74230b"), + (&[63], "020300011cfc9c0ae91c66774477c4a9846a8f65aa98191f46e5830216445436978803697a7b70fd95418bdcf4"), + (&[0, 0, 0], "020300031d3d175d826fb3dfbb8090bbcde5ed24acacf0fbc06a3f7f2751d6db18b546cc98c3d0aeaaf841bf3cb2fba85d7b1d1b9d415798191f46e5830216445436978803697a848f024abe74230b12db314a9699cd84d8ea9b4af271d79c"), + (&[63, 63], "020300021c01fc9c0ae91c66774477c4a9846a8f65aa7fb660a84a5e7f8872d29acbc1557e7898191f46e5830216445436978803697a7b70fd95418bdcf4cfb11b544705455d"), + (&[1, 2, 3, 4, 5], "0203000522140f253b9d23835cb5386f80380625d3e8367f426027b0e3de1522c0466e10a5b4359e05afa7dd343bfa153c2e00b46f574eebaf450a0a342b0212db53564cf50b16441632474e5b2cc82a27e5d9e8aefa56fb5d98191f46e5830216445436978803697a848f024aba74230bdbe6bcc62595978eb8f2fd0bb5413f41473d7c08c71195d35417bd12b0e7dc69"), +]; + +pub const FULL_ALICE_SEED_B: &str = "020300073933281b2b370c367ae8e098c5628373545e3027b3940019889a97dee6bd3b45d3e9df587803f858651a8d94901d58e061adcbe4a2873f98ce0340bbca2f8c50b6037d68324d54934c1d5eeccca87af1a0ddfabb106717331ae99d5a41b641fe7ea107efd8108e1eb8c874c638b793ace396ae840b968952f6f5b7a04d1eeff2d0f12880ed8269ced3e4df6e22425d4271990114d6017124e8369f14274ea363f8d1c6ebe93574f0b98da42a0ef9fe6a9c5135976e4cfb60f2981a5958e65d";