diff --git a/docs/benchmarks/2026-06-13-bit6-prp-results.md b/docs/benchmarks/2026-06-13-bit6-prp-results.md new file mode 100644 index 00000000..0f0d4a47 --- /dev/null +++ b/docs/benchmarks/2026-06-13-bit6-prp-results.md @@ -0,0 +1,120 @@ +# Benchmark results — Bit6 PRP swap to fixed-draw Fisher–Yates + +Apple M1 Max, rustc 1.87.0, hardware AES (`--cfg aes_armv8`), criterion +medians. Change: Bit6's per-block PRP moved from the rejection-sampled +Knuth shuffle to `LemireFyPrp` (fixed-count wide draws + Lemire reduction), +**seed-keyed shape (i)** — the drop-in replacement that fits the existing +`Prp::new(seed)` signature. + +| Bit6 benchmark | before (Knuth) | after (Lemire FY, shape i) | +|---|---:|---:| +| encrypt-u64 (11 blocks) | 11.5 µs | **8.6 µs** | +| encrypt-left-u64 | ~8.9 µs | **5.7 µs** | +| encrypt-u32 (6 blocks) | ~7.8 µs | **4.8 µs** | +| compare-u64 | 182 ns | 183 ns (unchanged) | + +## Why not the spike's 3.3 µs yet + +The spike's headline (153 ns/block, ≈3.3 µs encrypt) is the **pre-scheduled +stream, shape (ii)** — the PRP keystream produced under an already-scheduled +cipher, eliminating the AES key schedule per block (~172 ns × 11 ≈ 1.9 µs). +Shape (i), shipped here, still keys a fresh AES from each block's seed, so it +pays those 11 schedules — hence ~8.6 µs, not 3.3 µs. + +Shape (i) is the right increment for this PR: it fits the `Prp` trait with no +architectural change, and its security story is "identical key-usage +structure to the old PRP, with rejection sampling replaced by fixed-count +Lemire draws." Shape (ii) requires deriving the PRP stream under k2 / as a +PRF branch family, which is exactly the structure the §5(b) CMAC accumulator +introduces — so it lands with PR 6, under the same crypto review, and takes +Bit6 u64 encrypt from 8.6 µs to a projected ~3.3 µs. + +## What this PR's change actually buys now + +1. **Closes the timing channel.** Draw count is fixed and seed-independent; + no rejection loop. Encryption time no longer varies with the (plaintext- + derived) PRP seed. This is the security-relevant part and it ships now. +2. **Removes modulo/​rejection bias** in favour of a provable ≤ 2⁻⁵⁵ + statistical distance from uniform — a clean statistical term in the + Lewi-Wu argument. +3. **25% faster encrypt** even in shape (i), before the larger shape-(ii) + win. + +## End-to-end (u64 encrypt, for context) + +| Build | encrypt-u64 | +|---|---:| +| pre-v2 default (Bit8, software AES) | 381 µs | +| Bit8, hardware AES + bulk encoding (#80) | 25.1 µs | +| Bit6, Knuth PRP (#82 initial) | 11.5 µs | +| **Bit6, Lemire FY shape (i) (this change)** | **8.6 µs** | +| Bit6, Lemire FY shape (ii) (projected, PR 6) | ~3.3 µs | + +## Addendum (2026-10-08) — sort-by-random-key builder, benchmarked and rejected + +vitaminc's `permutation` crate adopted an oblivious sort-by-random-key builder +(one random 64-bit word per index, Batcher odd–even merge network, 543 +compare-exchanges at N = 64). It was prototyped as alternative bodies of +`LemireFyPrp::from_stream`, so the schemes are otherwise unchanged, and +benchmarked as a candidate for the A4 high-assurance tier. Apple M4, +rustc 1.94.1, `--release`, criterion means with 95% intervals. Both callers +already supply 512 bytes of stream; Lemire FY consumes 504, the sorts 512. + +| `from_stream`, N = 64 | mean | vs Lemire FY | +|---|---:|---:| +| Lemire FY (shipped) | 210 ns [209, 212] | — | +| sort u64, `(w<<8)\|i` (56-bit key, vitaminc's packing) | 494 ns [493, 496] | +135% | +| sort u128, `(w<<8)\|i` (64-bit key, index tie-break) | 663 ns [661, 666] | +215% | +| sort `(u64 key, u8 idx)` pair, two-level compare | 647 ns [643, 651] | +208% | + +| Bit6 benchmark | Lemire FY | sort, 56-bit key | sort, 64-bit key | +|---|---:|---:|---:| +| encrypt-u64 (11 blocks) | 8.39 µs | 11.80 µs (+41%) | 14.37 µs (+73%) | +| encrypt-left-u64 | 6.12 µs | 9.82 µs (+61%) | 12.39 µs (+102%) | +| encrypt-u32 (6 blocks) | 4.91 µs | 6.69 µs (+34%) | 8.00 µs (+61%) | + +The end-to-end deltas are the builder delta times the block count (11 × 284 ns +≈ 3.1 µs predicted, 3.4 µs measured for encrypt-u64). Left-only encryption +suffers most because the builder is a larger share of its cost. Masking the +gate indices to remove bounds checks made the sort slower (557 ns) and was +reverted. All three sort variants produce a valid, deterministic permutation +and match a reference stable sort; with either sort 8 of the 14 Bit6 compat +vectors fail, as a different permutation must. Decision and reasoning: review +brief A4, "Alternative considered for the tier". + +## Addendum (2026-10-09) — oblivious builder, now the default + +A dudect run measured the indexed builder's time depending on which +permutation it builds (review brief A4, "Status"), so key generation now goes +through oblivious builders with the same draws and swaps and byte-identical +tables (`primitives/prp/oblivious.rs`): NEON on aarch64, SSSE3 on x86_64 when +present, u64 SWAR elsewhere. Apple M4 on mains power (Low Power Mode off), +rustc 1.94.1, `--release`, hardware AES, criterion medians. + +The builder alone, per 512-byte stream, averaged over 256 random streams +(`benches/prp_build.rs`, `--features ct-bench`; each figure includes building +the struct, one oblivious `permute` and the wipe on drop): + +| builder | median | vs indexed | +|---|---:|---:| +| indexed (textbook FY; now a test-only reference) | 248 ns | — | +| **NEON (dispatched on aarch64)** | **199 ns** | **−20%** | +| SWAR (portable fallback) | 710 ns | +186% | +| byte-at-a-time `subtle_ng` (test reference only) | 12.0 µs | 48× | + +End to end, the same build with the dispatcher pointed at each builder in +turn (the indexed column by a temporary, uncommitted edit). A second run of +the NEON build agreed within 2%. + +| Bit6 benchmark | indexed | NEON | SWAR | +|---|---:|---:|---:| +| encrypt-u64 (11 blocks) | 8.59 µs | **7.89 µs** (−8%) | 13.55 µs (+58%) | +| encrypt-left-u64 | 6.33 µs | **5.77 µs** (−9%) | 11.32 µs (+79%) | +| encrypt-u32 (6 blocks) | 4.75 µs | **4.44 µs** (−7%) | 7.67 µs (+61%) | +| compare-u64 | 151 ns | 149 ns | — (builds no PRP) | + +The saving is the builder saving times the block count (11 × ~50 ns ≈ +0.55 µs, 0.70 µs measured for encrypt-u64). The indexed figure of 8.59 µs +matches the 8.39 µs recorded above for the same code on the same machine. +SSSE3 has not been timed on real x86 hardware; under Rosetta 2 it passes the +equivalence tests only. diff --git a/docs/plans/2026-06-12-ore-v2-architecture.md b/docs/plans/2026-06-12-ore-v2-architecture.md index d87ef563..f646caa3 100644 --- a/docs/plans/2026-06-12-ore-v2-architecture.md +++ b/docs/plans/2026-06-12-ore-v2-architecture.md @@ -238,7 +238,9 @@ A new fixed-N scheme instantiating `(Bit6, Packed, fixed-N)`: `u64` (11 six-bit blocks) packs `11 prefix bytes + xt + index = 13 ≤ 16`. The packed construction caps Bit6 at **14 blocks** (= 10 plaintext bytes), which covers every primitive up to 64-bit and dates, but **not** `u128`/`i128` (22 blocks) or `Decimal` - (19 blocks) — those remain on the 8-bit scheme (or wait for the chained prefix, §5). + (19 blocks). `Decimal` (14 bytes) remains on the 8-bit scheme until the chained prefix + (§5). `u128`/`i128` (16 bytes) fit neither: the 8-bit scheme puts the block index at + byte `N` of its left-tag input, so it takes at most 15 bytes and panics on 16. This constraint must be a compile-time assertion. - **Naming:** new module `scheme::bit2_w6` exposing `OreAes128Bit6ChaCha20` (exact name bikesheddable in the PR). The existing `scheme::bit2` module and types are untouched. @@ -257,11 +259,17 @@ bytes 4.. : left ‖ right (existing layout per block width) mis-order. Legacy v1 slices are only ever handled by the legacy type. (A v1 ciphertext could in principle collide with a v2 length; since each *type* only parses its own format, no runtime ambiguity arises.) -- **Domain separation:** the scheme id byte is also mixed into the PRF input layout - decision space — concretely, Bit6 writes `block_count` into the otherwise-unused byte - 15 of the left-tag input so that identical prefixes under Bit6 and Bit8 can never - produce related tags under the same keys. (Cheap insurance; needs a nod from crypto - review.) +- **Domain separation:** Bit6 writes `block_count` into byte 15 of its PRF inputs, but + **that alone does not separate it from Bit8**, contrary to what this bullet first + claimed. Legacy `N = 15` puts the block index in byte 15 and uses every other byte, + so a legacy left ciphertext can reproduce the PRF₁ input of any Bit6 masking key: a + legacy query under shared keys then acts as a Bit6 query token (found in review of + #82 and demonstrated by a test). No input layout can fix this, because legacy owns + the whole 16-byte input space at `N = 15`. Bit6 therefore runs PRF₁/PRF₂ under + **derived keys**, `AES_{k1}("ORE.v2.bit6.prf1")` and `AES_{k2}("ORE.v2.bit6.prf2")`; + the labels end in a byte no legacy PRF input can have (byte 15 is `0..=14` for + PRF₁ and `0` for PRF₂), so legacy never evaluates them. Bit6 seeds also bind the + block index in byte 14, so zero-padded prefixes no longer collide across positions. ### 5. Variable block counts / string encryption (goal 4) @@ -309,7 +317,8 @@ software on aarch64 (no key-schedule instruction); estimated 5–20% overhead ag Bit6's ~130 batched AES ops per block. **Benchmark gate:** measure NEON key-expansion overhead before committing. -**Candidate B — CMAC with cached prefix state.** Every published value is a *bona +**Candidate B — CMAC with cached prefix state.** *(SELECTED; full design spec: +`docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md`.)* Every published value is a *bona fide* AES-CMAC (NIST SP 800-38B) tag of an injectively encoded message — for block `i`: `enc(x₀‖0) ‖ … ‖ enc(x_{i-1}‖i−1) ‖ final_block(branch, value, i)` — and the per-prefix chaining-state cache is purely an implementation optimization (CMAC is CBC @@ -360,13 +369,140 @@ instance, so right-encryption throughput stays comparable to the packed scheme. **String semantics and leakage.** Strings are encoded as their UTF-8 bytes (optionally case-folded/normalised upstream — out of scope here), decomposed by the chosen width. -Lewi-Wu leaks the index of the first differing block; for strings that is **the length -of the common prefix**, which is materially more revealing than for fixed-width numerics. -This must be documented prominently on the string API, and is a product-level decision -about acceptable leakage, not something the library can engineer away. +A comparison reveals the index of the first differing block; for strings that is **the +length of the common prefix**, which is materially more revealing than for fixed-width +numerics. But this leakage is scoped to the **comparison operation**, not to stored +data, because Lewi-Wu is a left/right scheme: a comparison is only ever evaluated +between a left ciphertext and a right ciphertext, and a right ciphertext in isolation +reveals nothing about order. Three threat tiers follow: + +- **Offline / at rest:** right ciphertexts are semantically secure, so right-only + storage would make this tier clean: an attacker holding only rights recovers + nothing. **But right-only storage is not supported today** + ([#99](https://github.com/cipherstash/ore.rs/issues/99)). Every scheme here, legacy + Bit8 included, stores full ciphertexts, whose deterministic left half (`xt`, `f`) + sits next to the right half, and the only comparators (`compare_raw_slices`, `Ord` + on `CipherText`) take two full ciphertexts. So until #99 lands, an attacker who + exfiltrates the database can compare any two stored rows, with no query, and + recovers their **order** and, for strings, their **common-prefix length**. This is + the same exposure as legacy Bit8 today, not a regression. +- **Query time (legitimate operator):** running a query emits a left ciphertext, and + each comparison against the stored rights reveals first-differing-block (= + common-prefix length for strings) for exactly the pairs that query touches. +- **Online adversary observing queries:** an attacker who can watch enough query + traffic accumulates those per-comparison leakages and can reconstruct prefix + structure across the touched set. + +So, until right-only storage is supported (#99), the common-prefix disclosure applies to data at +rest as well as in use: anyone holding two stored rows learns it. This must be +documented prominently on the string API, +and the residual query-time/online leakage is a product-level decision about acceptable +leakage — but it is a narrower decision than the unscoped framing suggests, and not +something the library can engineer away. + +**Block width is a leakage decision, not just a size/perf one.** In Lewi-Wu, the +comparison leaks the index of the first differing block, so **larger blocks leak less**: +Bit8 (8-bit) < Bit6 (6-bit) < CLWW (1-bit, the full first-differing-*bit*). For a u64, +the first differing bit is localised to an **8-bit window** under Bit8 (8 blocks) but a +**6-bit window** under Bit6 (11 blocks) — Bit6 sharpens an online inference adversary's +divergence-point/density estimation by ~1.33×. It is an incremental sharpening, not the +categorical jump to CLWW, but it is real and it is the one axis the library cannot fix. +This sits opposite the **encrypt-side** advantage of Bit6 (the 64-entry PRP; see +Open Q1 / the A4 review brief): cheap constant-time key generation that Bit8 cannot get +for free. *(Updated 2026-10-09: Bit6 key generation is now oblivious by default, through +a register-resident Fisher–Yates builder that is faster on aarch64 than the indexed one +it replaced; review brief A4, "Status".)* The two pull in opposite directions and live in different threat models: + +- They do **not** net out: the leakage axis is an *online/query-time* property against an + inference adversary; the encrypt-side axis is a *side-channel* property against an + attacker co-resident with the encryptor. At rest, the leakage axis is a **tie** (both + reveal nothing), so there Bit6's win is free. +- The encrypt-side axis is a **cost** difference, not "constant-time vs not": full + oblivious constant-time is available at *both* widths via oblivious-swap Fisher–Yates, + just ~12× cheaper at Bit6 (≈44k ct-ops/u64 vs ≈522k at Bit8, counted for the + byte-at-a-time full-scan form). The leakage axis is the **fundamental, unfixable** one. + *(2026-10-09: at Bit6 the oblivious builder is now the default, not a cost to opt + into. It keeps both tables in vector registers, so a step is a few register ops, not + 64 byte selects: 199 ns per block PRP against 248 ns for the indexed builder on an + Apple M4 (NEON), 710 ns for the portable SWAR fallback. Bit8's legacy Knuth PRP is + wire-frozen and has no oblivious build.)* + +*(Superseded 2026-10-12, see below: the only Bit8 scheme is the legacy one, and it leaks +zero runs from a single ciphertext.)* Because leakage is unfixable and encrypt-side CT is +purchasable at either width, the lower-leakage width (Bit8) was the conservative +**default**, with Bit6 an explicit opt-in. Width was therefore a **per-domain / +per-deployment policy** keyed on the target data and threat model, not a global default: + +| Dominant threat | Encryptor environment | Choose | +|---|---|---| +| At-rest exfiltration (right-only, once supported (#99); today stored rows compare offline, see §5(b)) | any | **Bit6** — leakage tie, take the smaller ciphertext + cheap CT | +| Online inference on the plaintext distribution | trusted / dedicated | **Bit8** — encrypt-side moot, take the lower leakage | +| Online inference | hostile / multi-tenant | **Bit8 + oblivious-swap-FY** (low leakage *and* CT, ~522k ct-ops; not built: the legacy Bit8 PRP has no oblivious form), or Bit6, whose key generation is oblivious by default, if 2 bits of resolution is acceptable | + +~~Default numerics to Bit8 (lower leakage, and it is the wire-frozen compatible scheme); +strings pick width per the size/leakage trade in the PR 6 design. This supersedes the +earlier lean toward "Bit6 as the default" (Open Q3).~~ + +**Decision (2026-10-12): Bit6 for new data.** The comparison above assumed both widths +leak only order and the first differing block. The legacy Bit8 scheme leaks more: its +PRP seeds bind neither the block index nor the block count, so one ciphertext shows its +zero runs and an integer's magnitude (Context, "Known leakage of the legacy scheme"). +That leak needs no query or comparison, and it is larger than Bit6's finer first-differing +block. So the table's "At rest … tie" row is true only for right-only storage, and every +"Bit8" choice in it carries the zero-run leak. Recommend **Bit6 for new data**; Bit8 stays +the compatibility scheme for data already stored. A new Bit8 scheme that binds the block +index into its seeds would restore Bit8's lower comparison leakage, but it is not built. + +**Numeric encodings: fixed-point vs log/scientific (an encoding-layer option, not a +low-entropy mitigation).** Block ORE is parameterised by *how* a value maps to blocks, +not just by block width. For **wide-dynamic-range numerics** — currency, scientific / +sensor measurements, high-range decimals — a **mantissa+exponent (a.k.a. log-domain) +encoding** is a good fit: write `x ≈ f · base^e`, lay the exponent in the high-order +blocks and the mantissa in the low-order blocks (order-preserving), then encode as +usual. This is the same family as the "scientific notation" encoding used with CLWW ORE +and as `round(scale · log_base(x))`. Benefits, all of which compose with the §5 +variable-block machinery: + +- **Bounds block count over a wide range** (cents-to-billions becomes exponent + a + fixed-width mantissa, not a 60-bit fixed-point integer). +- **Uniform relative precision** — small and large values get the same significant-figure + blocks. (IEEE-754 already has this shape; the existing `f64` path exploits the + exponent-first layout.) +- **Makes prefix leakage a deliberate choice** — "magnitude band + N significant + figures" rather than an accident of fixed-point width. + +Caveats to design in: +- **Benford's law:** leading significant digits of natural numeric data are non-uniform + (1 ≈ 30%, 9 ≈ 5%), so the mantissa's top block stays skewed and inference-exposed. + Log encoding makes leakage *relative*, not *flat*. +- **You are electing to leak the magnitude band** — usually acceptable, but a conscious + leakage decision. +- **Parameters are a leakage surface:** per *Parameter-Hiding ORE* (Cash–Liu–O'Neill– + Zhang, ASIACRYPT 2018), the base/precision/scale leak distribution info if chosen + per value or per dataset. Fix them **per domain**, treat them as public constants, and + document them. + +**Explicit scope — do not conflate with the width/leakage decision above.** This encoding +solves *dynamic range + relative precision + block count*. It is **order-preserving**, so +it does **not** touch the order-leakage floor and is **not** a mitigation for low-entropy / +narrow-domain fields (DOB, names, etc.). Those fail because their *high-order bits are +skewed and ORE exposes them first*, plus the order floor (NKW sorting/cumulative); an +order-preserving re-encoding — log or otherwise — cannot help, and for a narrow domain like +DOB the exponent is near-constant (it *increases* high-order skew). The only lever there is +coarsening the plaintext to the granularity actually queried (year / age-band). Keep the +two ideas distinct: log/scientific encoding is for *wide-range numerics*; plaintext +coarsening is for *low-entropy narrow domains*. ### 6. Random-oracle instantiation (the 1-bit hash H) +> **RESOLVED 2026-06-15 (A1).** Keep the fixed public-key AES construction +> (option 3), upgraded with the BHKR orthomorphism: `H(x, r) = LSB(π(σ(x) ⊕ r) ⊕ +> σ(x) ⊕ r)` with `π = AES_{K₀}` (public `K₀`) and `σ(x) = 2x` in GF(2^128) (the +> BHKR/Zahur σ-MMO). Rationale below; full write-up in the crypto review brief +> A1 (`docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md`). Tweak-as-key +> (eprint 2019/1168 Thm 2) was **declined** (rekeying breaks the keyless/fast +> requirement, and fixes a multi-instance degradation ORE doesn't suffer). + Lewi-Wu models the right-ciphertext mask as a random oracle `H(ro_key, nonce) → Z₂`. Today it is instantiated as `LSB(AES_nonce(ro_key))` — the **nonce as the AES key** — and the code has carried a TODO questioning that construction since the beginning @@ -385,14 +521,27 @@ key / left tag, `r` = nonce): |---|---|---|---|---| | 1 | `LSB(AES_r(x))` — status quo | ideal cipher | — | key is public, so AES's standard PRP assumption gives nothing; security is an ideal-cipher assertion | | 2 | `LSB(AES_r(x) ⊕ x)` — MMO feedforward | ideal cipher | +1 XOR | matches the analyzed blockcipher-hashing shape; feedforward removes the invertible-public-permutation structure; the minimal upgrade | -| 3 | `LSB(π(x ⊕ r) ⊕ x)`, `π = AES_{K₀}`, K₀ public constant | random permutation | **faster** — zero key schedules ever | fixed-key-AES hashing (BHKR13); mine GKWY20 for known `x ⊕ r` tweaking pitfalls (our requirements are weaker than garbling's — no circularity, no correlated keys) | +| 3 | **SELECTED:** `LSB(π(σ(x) ⊕ r) ⊕ σ(x) ⊕ r)`, `π = AES_{K₀}`, `σ(x)=2x` | random permutation | **faster** — zero key schedules ever | BHKR/Zahur σ-MMO. Published left tags *are* `H` inputs, so the GKWY multi-instance shape applies to the unrevealed RO keys (secret shared across ciphertexts, public nonces as offsets); security rests on the BHKR bound `≈ p·C/2^128` (review brief A1, restated and signed off 2026-10-09). 2025/792 attacks (collision/preimage) target unused properties and are round-reduced | | 4 | `LSB(AES_x(r))` — RO key as AES key | **standard model** (PRF) | ~2–4× right-encryption: one key schedule per `(i, j)` | what the old TODO was reaching for; the honest price of standard-model security; composes poorly with accumulator Candidate A (both pay per-block schedules) | | 5 | SHA-256 (HW) / Blake3 over `x ‖ r` | random oracle | 2–5× encrypt path | comparator computes H once per comparison, so query latency is unaffected — only encryption throughput pays | -Proposal into review: **#3, with #2 as the conservative fallback**, and #4 written up -with its *measured* cost so the standard-model option is accepted or declined with the -price visible. The 1-bit truncation (LSB of a pseudorandom block) is uncontroversial -in every model. +Decision (2026-06-15): **#3 with the BHKR orthomorphism `σ(x)=2x`** — `H(x,r) = +LSB(π(σ(x)⊕r) ⊕ σ(x)⊕r)`. The known fixed-key-MMO attacks (GKWY; the half-gates +multi-instance attack of eprint 2019/1168) require the adversary to know the hash +inputs and recover a global Free-XOR offset. This decision first said ORE's inputs +are secret, so the `O(p·C/2^k)` degradation does not arise; that was wrong (review of +#82): every published left tag is an `H` input, and each unrevealed RO key is a secret +shared across ciphertexts with public nonces as offsets, which is the multi-instance +setting. Security rests on the BHKR bound, about `p·C/2^128`, negligible for realistic +`p` and `C` (review brief A1, restated and signed off 2026-10-09). The orthomorphism is what +places the construction in that analysed setting. The +tight tweak-as-key variant (2019/1168 Thm 2) is declined: rekeying per evaluation +conflicts with the keyless-comparator / performance requirement and addresses a +degradation absent here. AES-hashing cryptanalysis (eprint 2025/792) targets +collision/preimage/one-wayness — not the 1-bit correlation-robustness we rely on — +and reaches only round-reduced AES (7/10 collision), leaving full AES-128's margin. +The 1-bit truncation is uncontroversial in every model. Options #2/#4/#5 remain in +the table as the considered alternatives. ## PR roadmap @@ -433,13 +582,17 @@ PR 2's trait change, which should be called out in the changelog). - [ ] PRP seeds (PRF₂ outputs) structurally separated from serializable `Left` state; no code path can write seed material into a ciphertext (PR 2). - [ ] Domain separation between Bit8/Bit6/chained schemes under shared keys (PR 5, 6). -- [ ] H instantiation (§6) selected and signed off, with its security model - (ideal-cipher / random-permutation / standard) recorded (before PR 5). + Bit8/Bit6: done by key derivation (§4), with a cross-scheme regression test. + Chained: to confirm in PR 6. +- [x] H instantiation (§6) selected and signed off (2026-06-15): BHKR σ-MMO, + random-permutation model; see §6 and review brief A1. - [ ] Selected §5(b) accumulator candidate reviewed and signed off (before PR 6). - [ ] Accumulator chain state treated as key material: zeroized, never serialized, never reachable from `Left`/`Right` types (PR 6). - [ ] GF(2^128) doubling constant-time, if Candidate C is chosen (PR 6). -- [ ] String leakage profile documented and acknowledged at product level (PR 6). +- [ ] String leakage profile documented and acknowledged at product level. Stored + full ciphertexts compare offline, revealing order and common-prefix length, as + legacy does, until right-only storage is supported (#99); see §5(b) (PR 6). - [ ] Legacy zero-run seed leakage (Context, "Known leakage of the legacy scheme") documented in the README; new schemes bind the block index and count into their PRP seeds (PR 2, 5). @@ -457,24 +610,73 @@ PR 2's trait change, which should be called out in the changelog). `u128`/`Decimal` on Bit6 arrive with PR 6 if wanted. 5. **Strings use the chained-prefix variable-length scheme regardless of width**; width choice (6 vs 8) for strings is a ciphertext-size trade-off left to the PR 6 design. -6. **H decision is pulled forward to before PR 5** (not PR 6): Bit6 is a new scheme and - should ship with the chosen H rather than inherit nonce-as-key for compatibility's - sake. Proposal: fixed-public-permutation MMO (§6 option 3), conservative fallback - MMO-with-nonce-key (option 2). +6. **H = BHKR σ-MMO with fixed public AES key (§6, A1 RESOLVED 2026-06-15):** + `LSB(π(σ(x)⊕r) ⊕ σ(x)⊕r)`, `σ(x)=2x`. Bit6 ships with this rather than inheriting + nonce-as-key. Security rests on the BHKR multi-instance bound `≈ p·C/2^128` + (published left tags are `H` inputs; A1 restated and signed off 2026-10-09); tweak-as-key + declined (rekeying); 2025/792 hits only unused + properties on round-reduced AES. Legacy Bit8 keeps the status quo forever. 7. **Accumulator choice is a decision rule, not a fixed pick:** cascade/GGM if NEON key-expansion overhead measures under ~10–15% on Bit6 strings, else CMAC with cached state; XE only if profiling eliminates both (§5(b)). ## Open questions -1. **Cheaper PRP for new schemes:** for Bit6's 64-element domain, a small-domain - constant-time PRP (e.g. swap-or-not or a sorting network) could beat the Knuth - shuffle and be SIMD-friendly. New schemes have no compatibility constraint — worth a - spike during PR 5, not a blocker. +1. **Cheaper PRP for new schemes — RESOLVED by spike (2026-06-13, M1 Max, hw AES; + code at `/tmp/ore-prp-spike`, full analysis in its RESULTS.md):** + - **Winner: fixed-draw Fisher–Yates with pre-scheduled stream derivation** — + 63 Lemire-reduced 64-bit draws (fixed 32 AES-CTR blocks, zero rejection + sampling, branch-free), with the stream produced under an already-scheduled + cipher instead of keying AES per block. **153 ns per block** (construction + + permute + indicator mask) vs ~1.36 µs for a port of today's Knuth-64. + Projected Bit6 u64 encrypt: **≈3.3 µs** (vs 11.5 µs in PR 5). Security story: + exact statistical distance ≤ 2⁻⁵⁵ from a uniformly random permutation — the + object Lewi-Wu already models — so it adds a pure statistical term, no new + assumption. One review item: secret-indexed swaps, defended by the + one-cache-line argument (64-byte `#[repr(align(64))]` table). *(Resolved + 2026-10-09 differently: a measured timing dependence on the swaps made the + oblivious builder the default; review brief A4, "Status".)* + - **Swap-or-not is REJECTED for this setting**, not on speed but on proof: the + right ciphertext exposes a block PRP's full codebook across encryptions + sharing a prefix, so the honest query budget is q = N, where the HMR bound + `8N^{3/2}/(r+4)` is vacuous for any practical round count at N = 64. The + full-security fix (Morris–Rogaway 2014 sometimes-recurse) introduces + key-dependent recursion depth (a timing channel) and plain r=64 was 3× + slower than the winner anyway. Kept in the spike as the strictly + constant-time fallback if review rejects the cache-line argument. + - **Status-quo deficiency found:** the current rejection-sampled PRNG's draw + count is seed-dependent, and seeds derive from the plaintext prefix — an + encrypt-side plaintext-dependent timing channel in the legacy scheme (wire- + frozen, so document rather than fix there; the new PRP eliminates it). + - The pre-scheduled stream shape saves ~140–165 ns/block for *every* variant + and composes with §5(b): the CMAC accumulator can emit the fixed-count PRP + stream as one more branch family. + - **SHIPPED in PR 5 (`LemireFyPrp<64>`, Bit6 only — Bit8 stays wire-frozen on + the Knuth shuffle):** the **seed-keyed shape (i)** is in, because it is a + drop-in for the existing `Prp::new(seed)` signature and its security story is + "identical key-usage structure, rejection sampling → fixed-count Lemire + draws." This already closes the timing channel, removes the bias, and takes + Bit6 u64 encrypt **11.5 µs → 8.6 µs** (benchmarks: + `docs/benchmarks/2026-06-13-bit6-prp-results.md`). + - **DEFERRED to PR 6:** the **pre-scheduled shape (ii)** — deriving the PRP + keystream under an already-scheduled cipher (no per-block AES key schedule; + the ~1.9 µs that separates 8.6 µs from the projected ≈3.3 µs). It needs the + PRP stream to come from a PRF/branch family rather than a fresh per-seed key + schedule, which is exactly what the §5(b) CMAC accumulator provides — so it + lands there, under the same crypto review, rather than as a bespoke + key-reuse pattern bolted onto PR 5. 2. **`u16` vs `u8` block count in the v2 header:** u16 chosen for strings; confirm no need for >65 535 blocks (≈48 KiB plaintext at Bit6). -3. **Should Bit6 become the default scheme** recommended in the README once shipped, with - Bit8 positioned as the legacy/compat scheme? Affects docs tone in PR 7. +3. **Block width is a per-domain / per-deployment choice, not a global default + (RESOLVED — see §5(b) "Block width is a leakage decision"; REVISED 2026-10-12: + Bit6 for new data, because the legacy Bit8 scheme leaks zero runs from a single + ciphertext, see the §5(b) decision).** Earlier framing asked + "should Bit6 be the default"; the answer is no, because width trades online + prefix-leakage (Bit8 leaks less — the unfixable axis) against encrypt-side + constant-time cost (Bit6 cheaper — a fixable axis). Default numerics to **Bit8** + (lower leakage + wire-compat); position **Bit6** as an opt-in for at-rest-dominated, + size/perf-sensitive, or encryptor-hostile (with CT budget) deployments. PR 7 docs + present the decision rule (the §5(b) table), not a single recommended scheme. 4. **Pending review outcomes:** the §5(b) accumulator and §6 H selections await the NEON benchmark spike (PR 5) and internal crypto review; decision rules and candidate write-ups are inline in those sections. diff --git a/docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md b/docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md new file mode 100644 index 00000000..196a9691 --- /dev/null +++ b/docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md @@ -0,0 +1,309 @@ +# ORE v2 — chained-prefix CMAC accumulator: design spec (A2) + +**Date:** 2026-06-15 +**Status:** DRAFT for crypto review — the **A2 gate** that must pass before PR 6 +(chained prefix / variable-length / strings) is written. +**Companions:** plan §5(b) (`docs/plans/2026-06-12-ore-v2-architecture.md`), +review brief A2/A3 (`docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md`). + +This is the precise specification the review brief flagged as "still a sketch." +It pins the message encoding (the injectivity-critical part), the per-block +algorithm, the security argument, and the test plan. + +--- + +## 1. What this replaces, and why + +The fixed-N Bit6 scheme (#82) derives every per-block secret by packing the +**prefix** `x[0..n-1]` into one 16-byte AES block and encrypting it. It runs +under two derived keys, never the caller's raw keys (so that legacy ciphertexts +under shared keys cannot publish Bit6 masking keys): +`prf1 = E_{k1}("ORE.v2.bit6.prf1")` and `prf2 = E_{k2}("ORE.v2.bit6.prf2")`, each +label exactly 16 ASCII bytes. + +- **PRP seed** (`derive_prp_seeds`, key `prf2`): `seed_n = E_{prf2}( x[0..n-1] ‖ 0… ‖ n@[14] ‖ N@[15] )`. + The block index in byte 14 keeps a prefix padded with zero blocks from + colliding with a shorter prefix (the legacy scheme's zero-run leak). +- **RO key / left tag** (key `prf1`): `ro(n, v) = E_{prf1}( x[0..n-1] ‖ v@[n] ‖ 0… ‖ n@[N] ‖ N@[15] )`, + where `v = j` for the right-vector mask at domain position `j`, and `v = xt[n]` + (the permuted current symbol) for the published left tag `f[n]`. **So + `f[n] = ro(n, xt[n])`** — the same function at the value's permuted position; + this is what makes the masks cancel at compare time. + +Packing the raw prefix into one block caps the plaintext at **≤ 14 blocks** +(`MAX_BLOCKS`). PR 6 needs arbitrary length (strings; also `u128`/`Decimal` +which exceed 14 Bit6 blocks). The fix is an **accumulator**: absorb the prefix +incrementally into a fixed 16-byte state, and derive the per-block secrets by +**CMAC-finalising** that state with a per-output final block — instead of +packing the raw prefix. Candidate B (CMAC, NIST SP 800-38B) was selected by the +§5(b) benchmark gate (cascade/GGM rejected on aarch64 key-expansion cost; XE +held in reserve). + +--- + +## 2. Construction + +Let `E_k` be AES-128 under the accumulator key `k` (§3). Standard CMAC: + +``` +L = E_k(0^128) +K1 = dbl(L) # dbl = GF(2^128) "multiply by x" — the same gf128_double +K2 = dbl(K1) # (reused from the σ-MMO hash; constant 0x87) +``` + +All accumulator messages are an exact multiple of 16 bytes, so **only K1 is ever +used** (K2/padding never occurs). Define, over a running CBC state `S`: + +``` +absorb(S, B) = E_k(S ⊕ B) # extend the prefix chain (no subkey) +finalize(S, F) = E_k(S ⊕ F ⊕ K1) # a published CMAC tag (full last block) +``` + +For a plaintext of blocks `x[0..N-1]`, maintain `S_n` = the CBC chain over the +prefix blocks `P_0 ‖ … ‖ P_{n-1}` (`S_0 = 0^128`). Every per-block secret is a +`finalize(S_n, F)` for a final block `F` that injectively names the output. The +prefix chain is cached and extended incrementally — `clone-state-then-finalize` +*is* incremental CMAC, so each published value is a *bona fide* one-shot CMAC tag +of `P_0 ‖ … ‖ P_{n-1} ‖ F`. + +--- + +## 3. Key derivation & domain separation + +The accumulator uses a **dedicated key `k`**, derived from the master ORE key by +a labelled KDF distinct from every other use (the fixed-N `prf1`/`prf2`, H's +public `π`, the nonce RNG): + +``` +k = E_{k1}( "ORE.v2.chain.acc" ) # the 16 ASCII bytes, no terminator; one AES call +``` + +The label is exactly one AES block: the 16 bytes +`4f 52 45 2e 76 32 2e 63 68 61 69 6e 2e 61 63 63`. The key slot is `k1`; the +chained scheme's `init` takes that one key (`chained.rs`, `ACC_KEY_LABEL`). No +legacy PRF input equals the label: legacy PRF₁ inputs end in a block index +`0..=14` and legacy PRF₂ inputs end in `0x00`, and the label ends in `c` (`0x63`). +It also differs from the Bit6 labels, which are encrypted under the same `k1`. A +dedicated key makes the accumulator's PRF security self-contained: no cross-use +collisions with the fixed-N schemes or with H. The chained scheme therefore +**unifies the old `prf1` and `prf2`** into one key, domain-separated by the +**branch tag** (the `RO_KEY` / `PRP_STREAM` output families defined in §4) — +PRF₂ is subsumed. + +**Key inventory.** The chained scheme has exactly **one secret key — `k`** — +which produces *both* branches. Branch-tag domain separation under a good PRF is +equivalent to independent per-branch keys (the injectivity argument of §4 + the +CMAC-PRF reduction of §8), and is cheaper: one AES key schedule and one CMAC +subkey pair (`L, K1, K2`) rather than two. The only other key-shaped material is +**public** (H's fixed `π` constant `K₀`) or non-key (the per-ciphertext nonce). +Two consequences for review: + +- **vs fixed-N (#82):** that scheme keeps two secret keys (`prf1`/`prf2`); the + unification is a *chained-scheme* choice, not retroactive. +- **vs the `init(k1, k2)` API:** `k` is KDF-derived, so a single master input + suffices, and the chained scheme's `init` takes `k1` only (open question 1, + resolved). The alternative design — two keys, one per branch, no branch tag — + is equivalent in security but costs a second key schedule/subkey pair. + +--- + +## 4. Message encoding (injectivity-critical) + +A **branch** names which output family a `finalize` derives. There are two: + +- **`RO_KEY`** (tag `0x01`) — the right-vector mask values `ro(n, j)`, and the + left tag `f[n] = ro(n, xt[n])`; +- **`PRP_STREAM`** (tag `0x02`) — the Fisher–Yates keystream that builds `π_n`. + +These replace the fixed-N scheme's separate `prf1` (ro/f) and `prf2` (PRP) keys +(§3); the branch is carried as the **byte-0 branch tag** of the final block, and +is the `branch` argument in `F(branch, n, s)` below. + +Every block is exactly 16 bytes. Two block types, distinguished by byte 0. + +**Prefix block `P_t`** (absorbed into the chain; carries symbol `x[t]`): + +| byte | 0 | 1–2 | 3 | 4–15 | +|------|---|-----|---|------| +| value | `0x00` (TYPE_PREFIX) | `t` (u16 BE) | `x[t]` | `0x00` | + +**Final block `F(branch, n, s)`** (the last block of a `finalize` message): + +| byte | 0 | 1–2 | 3–4 | 5 | 6–15 | +|------|---|-----|-----|---|------| +| value | branch tag | `n` (u16 BE) | `s` (u16 BE) | `width` (`0x06`) | `0x00` | + +- **branch tag** ∈ `{ 0x01 = RO_KEY, 0x02 = PRP_STREAM }` (both ≠ `0x00`). +- **`n`** = block position being derived. +- **`s`** = sub-index: for `RO_KEY`, the domain value `j` (or `xt[n]` for the + left tag); for `PRP_STREAM`, the keystream counter `c`. +- **`width`** = block width (6) → domain separation from any future Bit8-chained + scheme. + +**No total-length (`N`) binding** — deliberately, and unlike the fixed-N packed +scheme (§7). + +### Injectivity +Every prefix block has `byte0 = 0x00`; every final block has `byte0 ∈ {0x01, +0x02}`. In a message `P_0 ‖ … ‖ P_{i-1} ‖ F`, the final block is the unique +non-`0x00`-byte0 block (and it is last), so the message parses unambiguously into +`((x[0],0),…,(x[i-1],i-1), (branch,n,s,width))`. Hence the map + +``` +(prefix values x[0..i-1], branch, n, s, width) ⟼ message bytes +``` + +is **injective**: distinct logical inputs ⇒ distinct messages. (Claim 1.) + +--- + +## 5. Per-block algorithm + +`STREAM_BLOCKS` = ⌈(DOMAIN−1)·8 / 16⌉ = 32 for Bit6 (63 wide draws → 504 B → 32 +× 16 B). `nonce` is a fresh per-ciphertext random value (unchanged from fixed-N). + +``` +S ← 0^128 +for n in 0..N: + # PRP for block n (shape (ii), A3): keystream straight from the accumulator + stream ← ‖_{c=0}^{STREAM_BLOCKS-1} finalize(S, F(PRP_STREAM, n, c)) + π_n ← LemireFyPrp::from_stream(stream) # new ctor; FY math unchanged + xt[n] ← π_n.permute(x[n]) + + # left tag f[n] = ro(n, xt[n]) — RO_KEY branch at s = xt[n] + f[n] ← finalize(S, F(RO_KEY, n, xt[n])) + + # right block: ro_key per domain value, then mask exactly as fixed-N + for j in 0..DOMAIN: + ro[j] ← finalize(S, F(RO_KEY, n, j)) + encode_right_block(right[n], π_n, x[n], H_nonce, ro) # unchanged: H-mask ⊕ indicator + + # extend the prefix chain + S ← absorb(S, P_n(x[n])) # S_{n+1} +zeroize(S, stream, ro, K1, K2, L, k) +``` + +- `encode_right_block`, `H` (the BHKR σ-MMO, A1), `indicator_mask_xor`, and the + comparator are **unchanged** — only the derivation of `ro`/`f`/PRP changes from + packed-AES to CMAC. `f[n]` reuses the `RO_KEY` finalize at `s = xt[n]`, so the + left/right masks cancel exactly as today. +- Left-only encryption (queries) runs the `PRP_STREAM` + `f[n]` steps only. +- Per-block AES count ≈ `1 (absorb) + 32 (stream) + DOMAIN (ro) ≈ 97` at Bit6, + comparable to the packed scheme's ~130; **no per-block key schedule** (this is + the shape-(ii) win, A3). + +--- + +## 6. Incremental CMAC ↔ one-shot equivalence (Claim 2) + +`finalize(S_n, F)` with `S_n` the CBC chain of `P_0..P_{n-1}` equals one-shot +`CMAC_k(P_0 ‖ … ‖ P_{n-1} ‖ F)` because `F` is a full 16-byte final block (→ K1) +and `S_n` is the standard CBC state. The chain extension `absorb(S_n, P_n)` is a +CBC step with **no subkey**, so it is *not* a published tag. **Test:** assert the +incremental implementation byte-matches the `cmac` crate's one-shot output over +the exact `P/F` messages, across positions/branches/sub-indices. + +--- + +## 7. No length binding — required for cross-length comparison + +The fixed-N scheme binds `N` (byte 15) into every derivation; that is safe +because only equal-length (same-type) ciphertexts are ever compared. The chained +scheme **must not** bind `N`: two strings sharing a prefix (`"app"` vs `"apple"`) +must produce **identical** per-block secrets for the shared blocks so the +comparator finds the first differing block correctly (shorter sorts first). The +per-block secrets therefore depend on `(prefix values, position n, branch, s)` +but **not** total length. + +Length **comparability** is enforced at the comparator, not in the derivation: + +- **Strings:** different lengths allowed; scan `min(len_a, len_b)` blocks; if + equal throughout, the shorter sorts first (lexicographic). Common-prefix-length + leakage is the intended, query-time-scoped leakage (plan §5(b)). +- **Fixed-length types** (`u128`/`Decimal` via the accumulator): the header + carries the block count; the comparator **rejects** mismatched lengths + (cross-type comparison), exactly as fixed-N does. + +--- + +## 8. Security argument + +1. **`k` is a uniform AES key** used *only* by the accumulator (§3), so its PRF + security is self-contained. +2. **CMAC is a secure variable-input-length PRF** in the PRP model (NIST SP + 800-38B; Iwata–Kurosawa OMAC), advantage `≈ (σ)² / 2^128` for `σ` total + blocks processed under `k`, plus the AES PRP term. +3. **Every published secret is `CMAC_k(msg)`** for an injectively-encoded `msg` + (Claim 1). Distinct logical outputs ⇒ distinct messages ⇒ jointly + indistinguishable from independent uniform values. These are precisely the + independent PRF outputs the Lewi-Wu analysis assumes for `ro`/`f`/PRP; ORE + security then follows from the existing Lewi-Wu argument. +4. **The chain state is never published.** `S_n` is an internal CBC value; the + only outputs are `finalize(S_n, F)` with the K1-treated final block. So the + "published-outputs-vs-chaining-value" interaction that complicates cascade/GGM + does **not** arise — it is subsumed by the CMAC PRF abstraction. In + particular, a chain value and a published tag are separated only by that + bound. `absorb` and `finalize` from the same `S_n` evaluate `E_k` at + `S_n ⊕ P_n` and `S_n ⊕ F ⊕ K1`, which are equal exactly when + `K1 = P_n ⊕ F`. `K1` is secret and uniform, and any of its bytes can take + any value, so the byte-0 tags do not keep the two inputs apart: they collide + with probability `2⁻¹²⁸` per pair, as any two CMAC-internal inputs do. That + collision probability is part of the CMAC PRF advantage `≈ σ²/2¹²⁸` in item + 2, which already covers collisions between chain values and finalised + inputs. The byte-0 tags matter only for the injectivity of the message + encoding (§4). +5. **Many outputs per state** (32 stream + DOMAIN ro per block) is sound for the + same reason: distinct final blocks ⇒ distinct messages ⇒ independent tags. +6. **Birthday budget.** `σ ≈ q · L · (DOMAIN + STREAM + 1)`. For `q = 2^32` + ciphertexts, `L = 14`, Bit6: `σ ≈ 2^44`, term `≈ 2^{88-128} = 2^{-40}` — + comfortable. `log()` the assumption if a deployment expects `q ≫ 2^32`. + +Auditable claims for sign-off: **(1)** encoding injectivity (§4), **(2)** +incremental-vs-one-shot faithfulness (§6), **(3)** zeroization (§9). + +--- + +## 9. State hygiene (Claim 3) + +`k`, `L`, `K1`, `K2`, the cached chain states `S_n`, the `stream` buffer, and the +`ro` buffer are all secret key-equivalent material (anyone holding `S_n` can +derive every output for that prefix). All must be: zeroized on drop; never +serialized; never reachable from `Left`/`Right`/`CipherText` types. (Mirrors the +existing `SeedBuf`/template zeroization in #82.) + +--- + +## 10. Test plan + +- **CMAC faithfulness:** incremental impl == `cmac` crate one-shot, over many + `(prefix, branch, n, s)` (§6). +- **Subkey KAT:** `L/K1/K2` against NIST SP 800-38B AES-128 CMAC test vectors + (also exercises the shared `gf128_double`). +- **Cross-length comparison:** strings sharing a prefix of every length compare + correctly (shorter sorts first); fixed-length mismatches are rejected (§7). +- **Order/roundtrip quickcheck** for the chained scheme (as bit2_w6 has). +- **Shape-(ii) PRP equivalence:** `from_stream(accumulator stream)` yields the + same permutation as the spec's FY over that stream; statistical-distance bound + unchanged (A3). +- **Wire vectors pinned** once the encoding is signed off (mirror + `compat_w6_vectors`), incl. a variable-length string set. +- **Zeroization** assertions / `Drop` coverage. + +--- + +## 11. Open questions for review + +1. **Key slot for `k`** (§3): RESOLVED. `k = E_{k1}("ORE.v2.chain.acc")`, the + 16-byte label with no terminator, one AES call; the chained `init` takes `k1` + only. +2. **`width`/scheme tag** (§4): is a 1-byte width enough domain separation, or + should the final block also carry a scheme id (as the wire header does)? +3. **PRP_STREAM vs a single seed:** spec derives the full FY stream as 32 CMAC + tags (shape ii). Alternative: one `PRP_SEED` tag that keys a fresh AES-CTR + (shape i) — simpler, but reintroduces a per-block key schedule. Confirm shape + (ii) is wanted here (it is the A3 perf rationale). +4. **Birthday budget** (§8.6): acceptable `q` ceiling; whether to document a + re-key guidance for very large datasets. +5. **`u128`/`Decimal`** ride this accumulator purely to exceed 14 blocks — they + are fixed-length; confirm the comparator's length-rejection (§7) is the right + place to keep them non-comparable with strings/other types. diff --git a/docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md b/docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md new file mode 100644 index 00000000..7a8f0ec5 --- /dev/null +++ b/docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md @@ -0,0 +1,631 @@ +# ORE v2 — crypto review brief (decisions A1–A4) + +**Date:** 2026-06-14 +**Author:** Dan Draper (via Claude Code) +**Audience:** crypto sign-off reviewer (internal) +**Source of truth:** `docs/plans/2026-06-12-ore-v2-architecture.md` (§5b, §6, Open Q1). +This brief is self-contained; section/line refs let you drill in. + +--- + +## 0. What you're being asked to sign off (TL;DR) + +Four crypto decisions gate the v2 work. Two block a PR that is already open +(#82); two block PR 6 (not yet written). + +| # | Decision | Model claimed | Status in code | Blocks | +|---|----------|---------------|----------------|--------| +| **A1** | 1-bit hash `H` instantiation | random-permutation (BHKR σ-MMO) | ✅ **RESOLVED** — `FixedPiZ2Hash` = `LSB(π(σ(x)⊕r)⊕σ(x)⊕r)`, σ(x)=2x | — (was the Bit6-vector gate; now cleared) | +| **A2** | Chained-prefix accumulator = AES-CMAC cached-state | CMAC PRF (standard) + 3 auditable claims | designed, not yet coded | PR 6 (variable-length / strings) | +| **A3** | PRP keystream from the accumulator (shape ii) | statistical (≤2⁻⁵⁵) + branch-family soundness | shape (i) shipped; (ii) deferred | PR 6 perf; couples to A2 | +| **A4** | Secret-indexed swap in PRP key-gen — ratify alignment + MemJam posture? | ~~constant-time / cache-line (sub-line = oblivious tier)~~ **oblivious key generation, every target** (2026-10-09) | oblivious builder (NEON / SSSE3 / SWAR) is the default, byte-identical tables; oblivious table reads and compare read; `N = 64` guard | nothing — fixes are byte-stable; A1 alone gates vectors | + +**Recommended sequencing:** **A1 is resolved** (BHKR σ-MMO) — Bit6 vectors can now +be generated and pinned against it. **A4 is a ratification** of changes already +applied (none of which alter ciphertexts); the MemJam posture call it carried +was overtaken on 2026-10-09, when a measured timing dependence made the +oblivious builder the default rather than a tier (§5, "Status"). Then +**A2 + A3 as one pass** (they gate PR 6, and A3 only exists inside A2's +accumulator). + +**What is explicitly *not* in scope:** the legacy Bit8 scheme is wire-frozen and +byte-identical to v1 (`tests/compat_vectors`); it keeps all status-quo +constructions forever and is out of review. Everything below concerns *new, +not-yet-frozen* schemes only. + +--- + +## 1. Construction primer (the parts these decisions touch) + +ORE here is **Lewi-Wu (2016) small-domain "BlockORE"**, a *left/right* scheme: + +- A plaintext is decomposed into blocks. Bit8 → blocks over `DOMAIN = 256`; + the new **Bit6** scheme → `DOMAIN = 64`. +- Per block, a **PRP** `π` permutes the `DOMAIN` symbols (keyed per block from a + prefix-dependent seed — this is **A3/A4**). +- The **right ciphertext** of a block is a length-`DOMAIN` vector: for each + permuted symbol `j`, a comparison result `cmp(π⁻¹(j), x)` masked by a 1-bit + value `H(ro_key(i,j), nonce)` (this is **A1**). +- The **left ciphertext** of a block is the permuted index of the plaintext + symbol plus a published tag. +- **Comparison** is evaluated by a *keyless* comparator (e.g. in Postgres) + between one left and one right ciphertext; it recomputes `H` from public + material only — hence H may use the nonce and published tags but **no + long-term secret** (this constraint drives **A1**). +- For plaintexts longer than one packed AES block (~14 Bit6 blocks), the + per-block secrets must be derived from an **accumulator** over the prefix + (this is **A2**, which also hosts **A3**). + +Leakage (for completeness, not under review): a comparison reveals the index of +the first differing block. For strings that is common-prefix length. This is a +property of *comparison*: right ciphertexts are semantically secure (Lewi-Wu), +so right-only storage at rest reveals nothing. **But right-only storage is not +supported today** ([#99](https://github.com/cipherstash/ore.rs/issues/99)): +every scheme here, legacy Bit8 included, stores full ciphertexts, and the only +comparators (`compare_raw_slices`, `Ord` on `CipherText`) take two full +ciphertexts. Until #99 lands, an attacker who exfiltrates the database can +compare any two stored rows offline and learns their order and first differing +block, with no query. This is the legacy exposure, not a regression (plan §5b, +"String semantics and leakage"). + +--- + +## 2. A1 — the 1-bit hash `H` (§6) — ✅ RESOLVED 2026-06-15 + +### The question +`H(x, r) → Z₂` where `x` = RO key / left tag, `r` = per-ciphertext nonce. It must +be computable by a **keyless** comparator from public ciphertext material (no +long-term secret), so any "PRF under a third key" design is out. + +### Resolution +**Keep option 3 (fixed public-key AES), upgraded with the BHKR orthomorphism.** +Shipped construction (`packages/ore-rs/src/primitives/hash.rs`, `FixedPiZ2Hash`): + +``` +H(x, r) = LSB( π(σ(x) ⊕ r) ⊕ σ(x) ⊕ r ), π = AES-128_{K₀}, σ(x) = 2·x in GF(2^128) +``` + +i.e. the **BHKR/Zahur fixed-key σ-MMO** with output truncated to 1 bit. + +- `K₀ = PI_KEY = b"ORE-rs.v2.H-pi.1"` — nothing-up-my-sleeve, deliberately + public; security rests on AES as a good *public random permutation*, not key + secrecy. Expanded once per process. +- `σ(x) = 2x` is the GF(2^128) doubling orthomorphism (same "multiply by x" as + CMAC subkeys; constant `0x87`), constant-time, branch-free (`gf128_double`). + Both `σ` and `σ⊕id` are permutations. +- `hash` (comparator) and `hash_all_into` (encryptor, scalar + SIMD `lsb_mask`) + verified equivalent (`fixed_pi_scalar_matches_bulk`); doubling checked against + the textbook shift/0x87 rule (`gf128_double_reduction`). + +### Why this is sound for ORE (the core justification) +The known attacks on fixed-key MMO **do not port to ORE**, and we now have the +exact reason from the literature: + +- **eprint 2019/1168** (Guo–Katz–Wang–Weng–Yu, *Better Concrete Security for + Half-Gates*) attacks the fixed-key construction `π(2x⊕i)⊕2x⊕i` in the + multi-instance garbling setting with success `O(p·C/2^k)` — but the attack + works by **recovering a global Free-XOR offset `R` from *known* hash inputs** + (the evaluator holds wire labels `Wa` and gate ids `j`, learns + `H(Wa⊕R, j)`, then meet-in-the-middles over `π`). + + > **Restated 2026-10-08 — signed off 2026-10-09 (Dan Draper)** (review of #82). The original + > text said ORE has neither precondition because its `H` inputs are secret + > and the adversary never learns them. That premise is false: `Left.f[n]` + > *is* the RO key at the published symbol `xt[n]`, and the comparator feeds + > it straight to `H`. Every left ciphertext or query token reveals one `H` + > input per block, by design: those are the bits it exists to unmask. + > + > The right model: for block `n` with prefix `p`, each candidate `j` has an + > RO key `k_{p,j}`, a secret PRF output shared by every ciphertext with that + > prefix; each stored ciphertext `i` has a public nonce `r_i`; the right block + > publishes `ind_j ⊕ H(k_{p,j}, r_i)`. Revealed tags are independent PRF + > outputs, so they say nothing about the *unrevealed* keys. For an + > unrevealed `k`, since `σ` is linear, `σ(k) ⊕ r_i` is a secret shared across + > instances plus known per-instance offsets: exactly the BHKR + > correlation-robustness setting, with `k` in the role of the global offset + > `R`. So GKWY's multi-instance shape **is** present, not structurally + > absent, and the honest statement is their bound: an attacker making `p` + > offline `π` queries against `C` targeted unrevealed keys succeeds with + > probability about `p·C/2^128`. Testing a guess needs known indicator bits + > (known or chosen plaintexts) for that key, since `H` outputs one bit. + > Success against one key unmasks candidate `j` in every stored ciphertext + > with that prefix, the same capability as one extra query token. + > + > The conclusion stands for realistic `p` and `C` (for example `p = 2^80`, + > `C = 2^32` gives `2^-16`, and the per-key verification cost rises with the + > one-bit output), but it rests on that bound, not on the inputs being + > secret. Theorem 2's re-keyed variant would remove the multi-instance + > factor; it is still declined for the comparator-speed reason below, now as + > a trade-off against a stated bound rather than against a non-issue. +- Their tight fix (Theorem 2, `E(i, σ(x))⊕σ(x)` — tweak as the AES *key*) is + **deliberately not adopted**: it requires rekeying per evaluation, which breaks + the keyless-comparator/performance requirement, and it fixes a degradation ORE + doesn't suffer. **Do not chase Theorem 2.** +- **eprint 2025/792** (Chen–Guo–List–Shi–Zhang, *Scrutinizing AES-based Hashing*) + is cryptanalysis of **collision / preimage / one-wayness** — properties this + 1-bit hash does not rely on — and its best AES-128 results are **round-reduced** + (7/10 collision on AES-MMO/MP at 2⁶⁰, 4/10 on AES-DM), never reaching full AES. + Mild evidence *for* the random-permutation assumption at full rounds. + +### Why the orthomorphism (the only change from the original draft) +Plain MMO (`σ = id`) would already be fine for ORE's independent-secret-input +setting. `σ(x)=2x` is adopted as **cheap defense-in-depth** (a few branch-free +ops) so security holds **by matching the named BHKR/Zahur construction** rather +than by a usage argument about input independence — robust-by-construction vs +robust-by-argument. The 1-bit truncation is uncontroversial in every model. + +### Status +Resolved. `FixedPiZ2Hash` updated. This **changes Bit6 right ciphertexts**, so +Bit6 byte vectors are (re)generated against this construction when pinned — A1 is +the gate that was holding that, and it is now cleared. + +--- + +## 3. A2 — chained-prefix accumulator = CMAC cached-state (§5b; blocks PR 6) + +> **Full design spec (2026-06-15):** `docs/plans/2026-06-15-ore-v2-cmac-accumulator-spec.md` +> — pins the injective message encoding, per-block algorithm, security argument, +> and test plan that this section summarises. + +### The question +PR 6 (variable-length / strings) needs an accumulator that absorbs the prefix +incrementally and, at each block position, derives the per-block secrets/outputs +(PRP seed, left tag, RO keys). **Candidate B — AES-CMAC with cached prefix +state** was selected by a benchmark gate; sign off its design before PR 6 is +written. + +### Why B (the gate result, durable record in §5b) +- **Candidate A (Cascade/GGM, chain through the AES key slot)** — cleanest + security story (constrained-PRF/GGM hybrid, no related-key assumptions) but + **rejected on performance**: on Apple M1 Max with hardware AES, an AES-128 key + expansion costs ~172 ns ≈ 160 batched block encryptions (no key-schedule + instruction on aarch64), making per-block overhead **~84%** at Bit6 — ~6× over + the ~10–15% threshold. (Spike at `/tmp/ore-keyexp-spike`.) +- **Candidate C (XE-style GF(2¹²⁸) masked single-key CBC)** — fastest, but the + many-outputs-per-state argument is bespoke (PMAC-lineage analyses publish only + a final tag). Held in reserve. +- **Candidate B** — control measured ~0% overhead and has a citable standards + basis. + +### The design to review +Every published value is a *bona fide* **AES-CMAC (NIST SP 800-38B)** tag of an +injectively encoded message. For block `i`: + +``` +msg(branch, i) = enc(x₀‖0) ‖ enc(x₁‖1) ‖ … ‖ enc(x_{i-1}‖i−1) ‖ final(branch, value, i) +tag = CMAC_k( msg(branch, i) ) +``` + +with separate **branch families**: chain step, `prp_seed` (this hosts A3), +left tag `f`, and `ro_key(i, j)` for `j ∈ 0..DOMAIN`. The per-prefix +chaining-state cache (clone-CBC-state-then-finalize) is **purely an +implementation optimization** — incremental CMAC, not a new primitive. + +Security reduces to **CMAC's PRF security** plus three auditable claims: +1. **Encoding injectivity** — `msg(branch, i)` is injective across blocks, + branches, and block widths (so no two distinct logical outputs share a CMAC + input). +2. **Implementation faithfulness** — the incremental/cached implementation is + bit-identical to one-shot CMAC (will be tested against the `cmac` crate's + vectors). +3. **Zeroization** — cached secret states are zeroized; never serialized; never + reachable from `Left`/`Right` types. + +### What needs scrutiny +1. **The injective encoding is currently a sketch** — it must be specified + exactly (field widths, the `enc(·)` block format, the `final(branch,…)` + block, domain-separation tags between the 4 branch families) and then checked + injective. This is the load-bearing claim. +2. **Many outputs per prefix state.** Standard CMAC analyses publish one tag; + here we publish chain + seed + tag + `DOMAIN` RO keys per position. This is + sound iff all inputs are distinct (claim 1) and CMAC is used purely as a PRF — + confirm there's no subtlety in deriving the *next chain state* from a tag that + is itself published (the GGM design made this vanish structurally; CMAC needs + the chain-state branch domain-separated from published branches). +3. **PRF₂ disposition** — does PRF₂ remain a separate key or become another + branch family? (Either is claimed fine; pick one.) + +### Decision & what it unblocks +Sign off the CMAC design (after the encoding is fully specified) → PR 6 can be +written. Fixed-N schemes never touch the accumulator, so this does **not** block +goals 1–3 or #82. + +--- + +## 4. A3 — PRP keystream from the accumulator, "shape (ii)" (Open Q1; rides with A2) + +### Background: the PRP question is otherwise resolved +Open Q1 (cheaper PRP for new schemes) is **resolved by spike** +(`/tmp/ore-prp-spike`, RESULTS.md). Winner: **fixed-draw Fisher–Yates with +Lemire-reduced wide draws** — `N−1` fixed draws, each a 64-bit value reduced by +multiply-high `((x·m) >> 64)`, zero rejection sampling, branch-free. Security: +exact statistical distance **≤ 2⁻⁵⁵** (for N=64) from a uniformly random +permutation — the object Lewi-Wu already models — so it adds a *pure statistical +term*, no new assumption. + +This is **shipped as `LemireFyPrp<64>`** for Bit6 (PR 5), +`packages/ore-rs/src/primitives/prp.rs:135-229`. It already (a) closes a +plaintext-dependent timing channel in the old rejection-sampled PRNG and (b) +removes a power-of-two modulo bias, and takes Bit6 u64 encrypt 11.5 → 8.6 µs. +**Swap-or-not was rejected** (its `8N^{3/2}/(r+4)` bound is vacuous at q=N, and +ORE exposes a block PRP's full codebook). Bit8 stays on the Knuth shuffle +(wire-frozen). + +### The narrow question for review +Shipped **shape (i)** keys a *fresh AES-128 key schedule per block* to produce +the FY keystream (`prp.rs:164-169`: `Aes128::new(seed)` then 32 CTR blocks). +**Shape (ii)** produces the same keystream from the *already-scheduled +accumulator cipher* (a `prp_seed` branch family of A2's CMAC), eliminating the +per-block key schedule — the ~1.9 µs that separates 8.6 µs from the projected +≈3.3 µs. + +The math (FY + Lemire, ≤2⁻⁵⁵) is unchanged. The **only new question** is whether +sourcing the PRP keystream from the accumulator's PRF output is sound: +1. The `prp_seed` branch must be **domain-separated** from the chain/tag/ro_key + branches (no keystream reuse across roles). +2. The keystream-as-PRP-randomness reuse pattern must not interact badly with + the same cipher's other outputs. + +This is **why A3 is reviewed inside A2** rather than as a bespoke key-reuse hack +bolted onto PR 5. + +### Decision & what it unblocks +Approve (or reject) deriving the PRP stream as a CMAC branch family. Unblocks the +PR 6 PRP perf target (~3.3 µs). No effect on #82 (which ships shape (i)). + +--- + +## 5. A4 — secret-indexed swap in PRP key generation (Open Q1; couples to Bit6 vectors) + +### Status (2026-10-09): superseded — oblivious key generation is the default +The rest of this section records what was believed and decided before +2026-10-09; read it as history. What changed: + +- **Measured.** A dudect run on Apple M4 (2026-10-09, + `docs/reviews/2026-10-09-ore-v2-dynamic-verification.md` §4.2) found the + builder's time depending on *which* permutation it builds, on a core with no + SMT sibling and no co-tenant: fixed stream A against fixed stream B reached + |t| = 65 at 100 k samples. Isolating the two halves of the builder put the + whole signal in the Fisher–Yates swap loop; the inverse fill alone, and both + halves redone at public addresses, showed none. That is consistent with + store-to-load forwarding and memory disambiguation on byte-granular accesses + at secret offsets inside one L1-resident line: whether a load overlaps a + store still in flight depends on the draws. The one-cache-line argument + below is about which *line* is touched and does not cover it, and "ARM: this + mechanism does not apply" in the MemJam analysis is true of MemJam but not + of this: it needs no co-tenant, only a clock. +- **Changed.** Key generation now goes through oblivious builders that never + use a secret value as an address (`primitives/prp/oblivious.rs`): both + tables live in registers, `perm[j]` is fetched by a table lookup on a + broadcast `j` and written back by compare-and-select, and the inverse is kept + by exchanging the *values* `i` and `j` in it at each step, which needs + neither swapped entry and replaces the separate fill. Step `i` is public and + `j ≤ i`, so only the chunks covering `0..=i` are searched. The draws and the + swap sequence are the same, so the tables are **byte-identical** and no + vector moves. Dispatch: **NEON** on aarch64 (baseline, no runtime check), + **SSSE3** (`pshufb`) on x86_64 when the CPU reports it, a **u64 SWAR** form + everywhere else. The textbook builder survives only as a test-only + reference that the others are property-tested (and, for SWAR, Kani-proved) + equal to. +- **Cost.** On aarch64 the oblivious builder is *faster* than the indexed one + it replaced, so there is no tier and nothing for a deployment to opt into. + Apple M4 on mains power, criterion medians, per 512-byte stream: indexed + 248 ns, **NEON 199 ns (−20%)**, SWAR 710 ns (2.9×). End to end the NEON + builder takes 6–10% off every encrypt (Bit6 u64 8.59 → 7.89 µs, left-only + 6.33 → 5.77 µs; full tables in + `docs/benchmarks/2026-06-13-bit6-prp-results.md`, addendum 2026-10-09). + The SWAR fallback costs 58–95% end to end on targets with neither NEON nor + SSSE3. SSSE3 timing on real x86 hardware has not been measured. +- **After.** dudect, same machine and session, fixed A against fixed B with a + fresh pair per 100 k-sample run: the indexed builder |t| = 17 to 83 over + five runs, the NEON builder at most 1.9 (and −0.8 pooled over 92 M samples + in 150 s); the end-to-end left-only Bit6 test that showed |t| = 21.6 now + stays under 2.4. Figures in the verification doc. + +So the recommendation below (default = line-granularity; oblivious-swap FY as +a paranoid tier at ~3×) is **withdrawn**: the oblivious builder is the only +builder, on every target. Sort-by-random-key stays rejected (see its +subsection). The checklist item is resolved accordingly. + +### The question +FY key generation performs a swap whose **address is secret-derived**: + +`prp.rs:189-196`: +```rust +for i in (1..$domain).rev() { + let d = $domain - 1 - i; + let x = u64::from_le_bytes(stream[d*8 .. d*8+8]); + let j = ((x as u128 * (i as u128 + 1)) >> 64) as usize; // secret j + perm.permutation.swap(i, j); // secret-indexed write +} +``` + +`j` depends on the (secret, prefix-derived) keystream, so the *memory address +written* is secret. Do we accept the **one-cache-line argument** (the whole +permutation table fits in a single cache line, so the access pattern leaks +nothing through the cache), or require the strictly-constant-time fallback? + +### Discrepancy found, now fixed (pending scrutiny) +The plan (Open Q1) stated the mitigation is a **64-byte cache-line-aligned +table**, but the originally shipped struct had **no alignment** (default +alignment 1), so the 64-byte `permutation` array could straddle two lines — +weakening, not establishing, the one-cache-line argument. There are in fact +**two** secret-indexed writes during key generation that the argument must +cover, not one: +- the Fisher–Yates `permutation.swap(i, j)` — secret `j` (`prp.rs:195`); +- the inverse fill `inverse[val] = …` — secret `val` (`prp.rs:198-200`). + +**Fix applied** (uncommitted, pending this review): `#[repr(C, align(64))]` on +`LemireFyPrp` (`prp.rs:135-139`). `repr(C)` pins field order so `permutation` is +at offset 0; `align(64)` puts the struct on a line boundary. At `N = 64` each +`[u8; 64]` table is exactly one line — `permutation` → line 0, `inverse` → +line 1 — so both secret-indexed writes are line-uniform. The argument holds only +for **N ≤ 64**; the sole instantiation is `LemireFyPrp<64>`. A `const _ = +assert!(N <= 64)` guard is proposed to make a larger instantiation a compile +error. + +### Context +- The **read** paths were *not* all constant-time, contrary to what this item + first said (review of #82). `indicator_mask_xor` scans the *entire* + permutation table via a branch-free `gt_mask` kernel, but `permute` read + `inverse[x[n]]` indexed by the **plaintext symbol**, and `invert` read + `permutation[j]`. Alignment defends only cache-line granularity, and the + sub-line MemJam channel below would reveal plaintext bits directly through + `permute`. Both now read every entry and select in constant time + (`oblivious_lookup` in `prp.rs`, also applied to the legacy Bit8 PRP with + identical output). The **key-generation writes** (above) remain the + secret-indexed accesses, and the high-assurance tier must cover both the + writes and these reads: an oblivious-swap builder alone would not make the + tier oblivious if the reads were left indexed. +- Same class of issue exists in **vitaminc** (filed cipherstash/vitaminc#198) + and in the legacy Bit8 Knuth path (wire-frozen — documented, not fixed). +- **Two fallback forms, only one is wire-compatible** (this matters — see "MemJam + & the oblivious tier" below): an **oblivious-swap Fisher–Yates** (same FY, same + Lemire draws, constant-time `swap`/inverse-fill via full-scan conditional + select) produces the **identical permutation** → identical ciphertext → + drop-in, vectors unchanged, ~O(N²) swap cost. A **different construction** + (swap-or-not) produces a *different* permutation → incompatible wire format, + and was rejected on the q=N proof anyway. The oblivious-swap form is the one to + reach for. +- **MemJam caveat (full analysis below):** cache-line alignment defends only at + *line* granularity; the sub-line (4-byte) MemJam channel on SMT-enabled Intel + is closed only by an oblivious construction. + +### Block-width asymmetry — the one-cache-line property is Bit6-only +The construction-time argument does **not** scale to a hypothetical +`LemireFyPrp<256>` (8-bit width), and the asymmetry is a positive argument for +Bit6 as the default (open question 3): +- **Construction (encryptor host) — strictly worse at D = 256.** The table is + 256 bytes = **4 cache lines**, so each swap leaks ~2 bits (which-of-4-lines). + And there are more swaps per block (D−1 = **255 vs 63**): a u64 is **8 blocks × + 255 = 2040** secret-indexed swaps at Bit8 vs **11 × 63 = 693** at Bit6. Larger + domain grows swaps faster than it shrinks block count, so the "fewer blocks" + intuition inverts. No offsetting benefit on this axis. +- **Comparison (comparator host) — tied.** The compare-side secret-indexed + access is `get_bit(target_block, a[l])` (next subsection). The right block is + 256 bits = **32 bytes** at Bit8 and 64 bits = **8 bytes** at Bit6; both are + ≤ 64, so both fit within one cache line given alignment. Bit8 is **not** better + here — both widths are line-uniform. +- These two leaks are on **different machines** (encryptor vs comparator) and + different trust domains, so neither offsets the other; each is evaluated + per-host. Net: 8-bit would be worse on construction and tied on compare — Bit6 + has the stronger constant-time story end to end. + +### Compare-side: oblivious `get_bit` — FIXED (uncommitted, pending review) +Comparison has its own secret-indexed access on the comparator host. After the +constant-time scan finds the first differing block `l` (`bit2.rs:235-242`, done +with `Choice`/`conditional_assign`), it reads byte `a[l] / 8` of the right block, +where `a[l]` (the permuted symbol) is sensitive. The block base was at an +arbitrary buffer offset, so the read could straddle a line. + +**Fix applied:** all four `get_bit` sites (`bit2.rs`, `bit2_w6.rs`, and the +`RightBlock32`/`RightBlock8` inherent methods) now route the byte read through a +new oblivious helper `width::ct_select_byte`, which scans the *entire* block and +constant-time-selects the target byte, so the access address is independent of +`a[l]`. This was chosen over mere alignment deliberately: an aligned direct index +still leaks at sub-line (MemJam) granularity, whereas the full scan closes both +line and sub-line channels. Cost is ≤ 32 byte-ops per comparison — negligible +beside the per-comparison AES hash. Results are unchanged (compat + comparison +vectors pass), so it does **not** touch the wire format. + +The block-*selection* index `l` was at first left as a direct index, on the +argument that `l` (first-differing-block) is leaked by ORE's definition anyway. +That argument is about who holds the ciphertexts; it says nothing about who can +only *time* the comparator. A dudect run (2026-10-09, recorded in +`docs/reviews/2026-10-09-ore-v2-dynamic-verification.md`) found that the +post-scan loads at `l` do leak it through timing: the scan reads every left +block, so `f[l]` and `a[l]` are cache-resident whatever `l` is, but it never +touches the right blocks, so whether `right[l]` hits depends on `l`. On a +23-block chained ciphertext that was 2 % of the timing spread; on Bit6, whose +whole ciphertext spans five lines, it was at the edge of detection. + +**Fix applied (2026-10-09):** the scan now *latches* `a[l]`, `f[l]` and +`right[l]` while it runs, with `width::ct_assign_bytes` under the choice "this +block is the first difference" (true for exactly one block). The resolution +step hashes and bit-selects from the latched copies, so no load after the scan +is indexed by `l`. Cost is one 8-byte right-block read and ≈ 25 masked byte +copies per block, on a scan that already compares 17 bytes per block. +- **Severity of both compare-side channels was low** (`l` and `a[l]` are in + the ciphertexts the comparator holds; the channels only matter to an attacker + who can time the comparator but not read its memory), but the oblivious forms + are cheap and remove the question entirely. +- **Scope note:** production comparison runs in the Postgres extension / proxy + (separate codebase) and must adopt the same oblivious read (or `ore.rs`'s + comparator) — tracked separately; this repo's `compare_raw_slices` and typed + `cmp` are now fixed. + +### MemJam & the oblivious tier — risk, CPU scope, and wire compatibility +> **Superseded (2026-10-09).** The analysis below stands as an account of +> MemJam, but it is no longer the deciding argument: a timing dependence on +> the swaps was measured on Apple M4 with no co-tenant, and the oblivious +> builder is now the default everywhere (see "Status" at the top of §5). +> Kept for the record of what was believed and why. + +This is the one open judgement call on the construction-side (encryptor) swap. +The shipped `#[repr(C, align(64))]` fix makes the swap **line-uniform**, which +closes the broad cache-line channel for everyone (AMD, ARM, and non-SMT Intel). +It does **not** close MemJam. + +**What MemJam is.** 4K aliasing: Intel's memory disambiguation predicts +store→load dependencies from only the low address bits, so an attacker who writes +to an aliasing address forces a false read-after-write dependency and times the +victim's load at **4-byte granularity within a cache line**. That is exactly why +alignment (a 64-byte property) is insufficient. + +**CPU scope.** +- **Intel x86 — effectively all generations, incl. modern parts and SGX.** The + MemJam paper's headline is that it applies to "all major Intel processors + including the latest generations," unlike its predecessor. +- **Older Intel (pre-Haswell)** also fall to **CacheBleed** (cache-bank + conflicts, sub-line). Between the two, treat "intra-line is safe on Intel" as + false across the line. +- **AMD** — not the MemJam target; no demonstrated equivalent, but not provably + immune. +- **ARM (Apple Silicon, AWS Graviton)** — this mechanism does not apply. + Relevant: our benchmarks (M1 Max) and likely Graviton production are unaffected. + +**Threat model — co-residence required.** MemJam is **not remote**: the spy must +run on the **sibling hyperthread (same physical core, SMT enabled)** of the +victim, hammering the aliasing address throughout the computation. So the surface +is: Intel **+** SMT on **+** attacker code on the same core as the *encryptor* +**+** an attacker who cannot already read the victim's memory. Disabling SMT or +core-isolation neutralizes it with no code change; running the encryptor on ARM +sidesteps it entirely. Marginal leakage is small regardless — the swap addresses +reveal partial info about a permutation whose codebook the ciphertext already +largely exposes (the same fact that sank swap-or-not). + +**Recommendation (withdrawn 2026-10-09 — the oblivious builder is now the +default on every target, and faster on aarch64 than the indexed one).** +1. **Default:** ship the `#[repr(C, align(64))]` fix; document that the + constant-time guarantee is at **cache-line granularity**, and that sub-line + (MemJam/CacheBleed) resistance on SMT-enabled Intel needs either SMT-off / + core-isolation **or** the oblivious build. +2. **High-assurance tier:** offer the **oblivious-swap Fisher–Yates** builder + (~3× slower, already spiked) for threat models that include a malicious + co-tenant on Intel with SMT. + +This matches field practice — aligned-constant-time is the standard bar, +oblivious is the paranoid tier — and lets the deployment, not the library, pay +the 3× only when it needs to. + +The "~3×" was the byte-at-a-time full-scan form. The register-resident form +that shipped instead (value-side inverse update, public `0..=i` search bound) +costs 199 ns against the indexed builder's 248 ns on the M4, so the trade the +tier existed to offer no longer exists. + +### Alternative considered for the tier: sort-by-random-key — rejected (2026-10-08) +vitaminc's `permutation` crate now builds its keys the djbsort / NTRU Prime +way: tag each index with a random 64-bit word, sort the words through a Batcher +odd–even merge network of branchless compare-exchanges, and read the +permutation off the sorted order. Its instruction and memory trace is a function +of N alone, so it was prototyped inside `LemireFyPrp::from_stream` and +benchmarked as a candidate for the tier in place of oblivious-swap FY (Apple M4, +rustc 1.94.1, criterion, N = 64, 543 gates; full tables in +`docs/benchmarks/2026-06-13-bit6-prp-results.md`, addendum). + +| builder | `from_stream` | bit6-encrypt-u64 | bit6-encrypt-left-u64 | +|---|---:|---:|---:| +| `LemireFyPrp` (shipped) | 210 ns | 8.39 µs | 6.12 µs | +| sort, 56-bit key `(w<<8)\|i` (vitaminc's packing) | 494 ns (+135%) | +41% | +61% | +| sort, 64-bit key + index tie-break | 663 ns (+215%) | +73% | +102% | + +**Still rejected after the oblivious builder shipped (2026-10-09).** The +sort removes a secret address by the same means the oblivious FY builder does, +running every compare-exchange whatever the data, so it buys no stronger +property; and the FY form is now cheaper still (199 ns against the sort's +494–663 ns, both on the M4), keeps the inverse maintained alongside the swaps +rather than needing a second pass, and leaves every ciphertext unchanged. + +Rejected, for three reasons that compound: +1. **It costs what oblivious-swap FY costs and buys less.** 2.3–3.2× on the + builder is the same band as the ~3× spiked for oblivious-swap FY (the + register-resident form that shipped later is faster than either), but the + sort removes only the secret-indexed *swaps*: the inverse-table fill + (`inverse[perm[i]] = i`) is still a secret-indexed write, and making it + oblivious needs a second network or an O(N²) scatter — at which point it is + slower than oblivious-swap FY, which already covers both. +2. **It changes the permutation,** so it changes every Bit6 and chained + ciphertext (8 of 14 Bit6 compat vectors fail). Oblivious-swap FY is + byte-identical and can be enabled per build with no re-pin — the property + "Relationship to vector pinning" below relies on. +3. **Uniformity is no better.** With a 64-bit key the only deviation from + uniform is a key tie broken by index, C(64,2)·2⁻⁶⁴ ≈ 2⁻⁵³ per permutation, + marginally worse than Lemire FY's ≤ 2⁻⁵⁵. vitaminc's 56-bit packing is + ≈ 2⁻⁴⁵, which is why it restarts on a collision. + +One rule from it **is** adopted, for every PRP builder including the +oblivious-swap tier: **no retry path.** vitaminc redraws from its RNG when two +keys collide. Here a query's left half and a stored value's right half are +produced in separate calls from `(key, prefix, n)` alone and must agree, so the +stream→permutation map must be a deterministic function with a trace fixed by N. +A tie, or any other "bad draw", is resolved by a fixed, documented rule +(ascending index), never by consuming more stream, and the resulting deviation is +stated as a statistical term as above. A reimplementation (Go, Node) has to +reproduce that rule exactly, or its left halves will not compare against ours. + +### Relationship to vector pinning (revised) +The MemJam tier does **not** gate Bit6 vector pinning, *provided the oblivious +fallback is the oblivious-swap FY form* (recommended): it yields the identical +permutation, so ciphertexts are byte-stable whether or not it is enabled. Vectors +can be pinned after the A1/H decision, and the oblivious builder added later as a +build option with no re-pin. (Only a *different-construction* fallback such as +swap-or-not would change ciphertexts and force a re-pin — another reason to +prefer oblivious-swap FY.) + +### Decision & what it unblocks +*(As first proposed; (a) and (d) are overtaken by the oblivious builder, see +"Status" at the top of this section. (b) and the guard stand, the guard now +pinning `N = 64`, the width the oblivious builders are written for.)* +Ratify: (a) `#[repr(C, align(64))]` as the default construction-side fix; (b) the +oblivious compare-side read; (c) the `N≤64` compile guard; and (d) the MemJam +posture (document scope + offer oblivious-swap FY as the high-assurance build). +None of (a)–(d) changes ciphertexts, so Bit6 vectors are gated only by A1. + +--- + +## 6. Sign-off checklist + +**Gate 1 — before #82 merges / Bit6 vectors pinned:** +- [x] **A1** H construction resolved (2026-06-15): keep fixed public-key AES, + upgraded to the BHKR σ-MMO `LSB(π(σ(x)⊕r)⊕σ(x)⊕r)`, σ(x)=2x. Argument + restated 2026-10-08 (published left tags are `H` inputs, so the GKWY + multi-instance shape applies; security rests on the `≈ p·C/2^128` bound) + and signed off 2026-10-09; + tweak-as-key (2019/1168 Thm 2) explicitly declined; 2025/792 targets + properties we don't use and is round-reduced. +- [x] **A4** ~~ratify the applied construction-side fix `#[repr(C, align(64))]` + (covers both secret-indexed writes; line-uniform at N ≤ 64).~~ + Superseded 2026-10-09: key generation has no secret-indexed write left + (oblivious builder); the alignment stays as layout only. +- [x] **A4** compile guard (applied), now `assert!(N == 64)`: the oblivious + builders are written for 64 entries. +- [ ] **A4** ratify the oblivious compare-side read `width::ct_select_byte` + (applied; all four `get_bit` sites; results unchanged). +- [x] **A4** MemJam posture — **resolved 2026-10-09, not as proposed.** A + dudect run measured a timing dependence on the swaps with no co-tenant, + so the oblivious builder (NEON / SSSE3 / SWAR, byte-identical tables) + is the default on every target, not a tier; on aarch64 it is faster than + the indexed builder. Sort-by-random-key stays rejected (2026-10-08); its + no-retry rule is adopted — see A4. +- [ ] Bit6 byte vectors regenerated and pinned **after A1** (all A4 fixes are + byte-stable; production comparator adopts `ct_select_byte` separately). + +**Gate 2 — before PR 6 is written:** +- [ ] **A2** CMAC encoding fully specified and checked injective; many-outputs + / chain-state-from-published-tag interaction cleared; impl-faithfulness + + zeroization test plan agreed. +- [ ] **A3** PRP-stream-as-CMAC-branch-family domain separation approved. + +--- + +## 7. References / pointers +- Plan: `docs/plans/2026-06-12-ore-v2-architecture.md` — §5b (accumulator), + §6 (H), Open Q1 (PRP), Security checklist. +- Code: `primitives/hash.rs` (A1), `primitives/prp.rs` (A3/A4), + `scheme/bit2_w6.rs` (Bit6 wiring), `tests/compat_vectors` (Bit8 frozen). +- Spikes: `/tmp/ore-keyexp-spike` (A2 gate), `/tmp/ore-prp-spike` (A3/A4). +- Benchmarks: `docs/benchmarks/2026-06-13-*.md`. +- Lit: Lewi-Wu 2016 (BlockORE); BHKR13 / GKWY20 (fixed-key AES hashing); + NIST SP 800-38B (CMAC); HMR12 + Morris-Rogaway 2014 (swap-or-not); Lemire 2019 + (nearly-divisionless bounded random); Moghimi et al. CT-RSA 2018 (MemJam, + arXiv:1711.08002) + Yarom et al. 2016 (CacheBleed) for the sub-line channels. diff --git a/packages/ore-rs/Cargo.toml b/packages/ore-rs/Cargo.toml index d865b63b..005e6931 100644 --- a/packages/ore-rs/Cargo.toml +++ b/packages/ore-rs/Cargo.toml @@ -51,6 +51,10 @@ thiserror = "1.0.38" name = "oreaes128" harness = false +[[bench]] +name = "bit6" +harness = false + [[bench]] name = "decimal" harness = false diff --git a/packages/ore-rs/README.md b/packages/ore-rs/README.md index f5208751..11626c06 100644 --- a/packages/ore-rs/README.md +++ b/packages/ore-rs/README.md @@ -21,7 +21,7 @@ Reference documentation is on [docs.rs/ore-rs](https://docs.rs/ore-rs). ## Supported plaintext types -`OreEncrypt` is implemented for `bool`, all integer widths (`u8`–`u128`, `i8`–`i128`), `char`, `f32`, and `f64`. Two optional features extend this via the sibling [`orderable-bytes`](https://crates.io/crates/orderable-bytes) crate: +`OreEncrypt` is implemented for `bool`, all integer widths (`u8`–`u128`, `i8`–`i128`), `char`, `f32`, and `f64`. `u128` and `i128` cannot be encrypted yet: `OreAes128` takes at most 15 bytes (its left-tag input puts the block index at byte `N` of one AES block), so encrypting a 16-byte value panics, and `OreAes128Bit6` covers types up to 64 bits. Two optional features extend this via the sibling [`orderable-bytes`](https://crates.io/crates/orderable-bytes) crate: - `chrono` — ORE support for `chrono::NaiveDate` and `chrono::DateTime` - `decimal` — ORE support for `rust_decimal::Decimal` @@ -83,7 +83,7 @@ Beyond the order and the first differing block of a compared pair, an `OreAes128 * the number of leading zero bytes of an integer, which is its order of magnitude, as the same value in every ciphertext under a key; * other runs of zero bytes, such as the trailing zero mantissa bytes of a round float. -The wire format of `OreAes128` is frozen, so this cannot change for existing data. See "Known leakage of the legacy scheme" in [the v2 plan](https://github.com/cipherstash/ore.rs/blob/main/docs/plans/2026-06-12-ore-v2-architecture.md). +The wire format of `OreAes128` is frozen, so this cannot change for existing data. For new data, use `OreAes128Bit6` (`scheme::bit2_w6`, ORE v2), which binds the block index and the block count into its seeds. It covers types up to 64 bits; `Decimal` stays on `OreAes128` for now. `u128` and `i128` are supported by neither scheme (see [Supported plaintext types](#supported-plaintext-types)). See "Known leakage of the legacy scheme" in [the v2 plan](https://github.com/cipherstash/ore.rs/blob/main/docs/plans/2026-06-12-ore-v2-architecture.md). To report a security issue, see [SECURITY.md](https://github.com/cipherstash/ore.rs/blob/main/SECURITY.md) or email security@cipherstash.com. diff --git a/packages/ore-rs/benches/bit6.rs b/packages/ore-rs/benches/bit6.rs new file mode 100644 index 00000000..1d0a8e73 --- /dev/null +++ b/packages/ore-rs/benches/bit6.rs @@ -0,0 +1,42 @@ +use criterion::{black_box, criterion_group, criterion_main, Criterion}; +use hex_literal::hex; +use ore_rs::{scheme::bit2_w6::OreAes128Bit6ChaCha20, OreCipher, OreEncrypt, OreOutput}; + +fn init_ore() -> OreAes128Bit6ChaCha20 { + let k1: [u8; 16] = hex!("00010203 04050607 08090a0b 0c0d0e0f"); + let k2: [u8; 16] = hex!("d0d1d2d3 d4d5d6d7 d8d9dadb dcdddedf"); + OreCipher::init(&k1, &k2).unwrap() +} + +fn criterion_benchmark(c: &mut Criterion) { + let ore = init_ore(); + + c.bench_function("bit6-encrypt-u64", |b| { + b.iter(|| black_box(25u64).encrypt(&ore).unwrap()) + }); + + c.bench_function("bit6-encrypt-left-u64", |b| { + b.iter(|| black_box(25u64).encrypt_left(&ore).unwrap()) + }); + + c.bench_function("bit6-encrypt-u32", |b| { + b.iter(|| black_box(25u32).encrypt(&ore).unwrap()) + }); + + let a = 25u64.encrypt(&ore).unwrap(); + let b_ct = 1025u64.encrypt(&ore).unwrap(); + c.bench_function("bit6-compare-u64", |bench| { + bench.iter(|| black_box(&a).partial_cmp(black_box(&b_ct))) + }); + + let a_bytes = a.to_bytes(); + let b_bytes = b_ct.to_bytes(); + c.bench_function("bit6-compare-u64-slice", |bench| { + bench.iter(|| { + OreAes128Bit6ChaCha20::compare_raw_slices(black_box(&a_bytes), black_box(&b_bytes)) + }) + }); +} + +criterion_group!(benches, criterion_benchmark); +criterion_main!(benches); diff --git a/packages/ore-rs/build.rs b/packages/ore-rs/build.rs new file mode 100644 index 00000000..ee8aa182 --- /dev/null +++ b/packages/ore-rs/build.rs @@ -0,0 +1,11 @@ +//! Tells the crate whether it is built unoptimised, which sets the stack +//! scrub size (`primitives::stack::SCRUB_BYTES`). Stack depth follows the +//! optimisation level, not `debug_assertions`, and no `cfg` exposes the +//! optimisation level directly. +fn main() { + println!("cargo:rustc-check-cfg=cfg(ore_unoptimized)"); + println!("cargo:rerun-if-env-changed=OPT_LEVEL"); + if std::env::var("OPT_LEVEL").as_deref() == Ok("0") { + println!("cargo:rustc-cfg=ore_unoptimized"); + } +} diff --git a/packages/ore-rs/src/chrono.rs b/packages/ore-rs/src/chrono.rs index b6decdd6..f34406eb 100644 --- a/packages/ore-rs/src/chrono.rs +++ b/packages/ore-rs/src/chrono.rs @@ -9,35 +9,37 @@ //! ordering properties. use crate::ciphertext::*; +use crate::scheme::bit2::OreAes128; use crate::{OreCipher, OreEncrypt, OreError}; use ::chrono::{DateTime, NaiveDate, Utc}; use orderable_bytes::{FixedOrderableBytes, OrderableBytes}; +use rand::{Rng, SeedableRng}; const NAIVE_DATE_LEN: usize = ::ENCODED_LEN; const DATETIME_UTC_LEN: usize = as FixedOrderableBytes>::ENCODED_LEN; -impl OreEncrypt for NaiveDate { - type LeftOutput = Left; - type FullOutput = CipherText; +impl OreEncrypt> for NaiveDate { + type LeftOutput = Left, NAIVE_DATE_LEN>; + type FullOutput = CipherText, NAIVE_DATE_LEN>; - fn encrypt_left(&self, cipher: &T) -> Result { + fn encrypt_left(&self, cipher: &OreAes128) -> Result { cipher.encrypt_left(&self.to_orderable_bytes()) } - fn encrypt(&self, cipher: &T) -> Result { + fn encrypt(&self, cipher: &OreAes128) -> Result { cipher.encrypt(&self.to_orderable_bytes()) } } -impl OreEncrypt for DateTime { - type LeftOutput = Left; - type FullOutput = CipherText; +impl OreEncrypt> for DateTime { + type LeftOutput = Left, DATETIME_UTC_LEN>; + type FullOutput = CipherText, DATETIME_UTC_LEN>; - fn encrypt_left(&self, cipher: &T) -> Result { + fn encrypt_left(&self, cipher: &OreAes128) -> Result { cipher.encrypt_left(&self.to_orderable_bytes()) } - fn encrypt(&self, cipher: &T) -> Result { + fn encrypt(&self, cipher: &OreAes128) -> Result { cipher.encrypt(&self.to_orderable_bytes()) } } diff --git a/packages/ore-rs/src/ciphertext.rs b/packages/ore-rs/src/ciphertext.rs index 8b34b632..19f53c66 100644 --- a/packages/ore-rs/src/ciphertext.rs +++ b/packages/ore-rs/src/ciphertext.rs @@ -3,6 +3,70 @@ use thiserror::Error; use crate::primitives::NONCE_SIZE; pub use crate::OreCipher; +/// Wire-format header prepended to every serialised artifact (Left, Right +/// or combined ciphertext) of schemes introduced from ORE v2 onwards. +/// +/// Layout: `version ‖ scheme_id ‖ block_count (u16 BE)` — 4 bytes. The +/// legacy [`crate::scheme::bit2`] scheme predates headers and remains +/// headerless forever ([`OreCipher::WIRE_HEADER`] is `None` for it); +/// ciphertexts of headered schemes can never be confused with each other +/// (version + scheme id are validated on parse and compare), and each +/// *type* only ever parses its own format. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct WireHeader { + /// Wire format version. `0x02` is the first headered format. + pub version: u8, + /// Scheme identifier (encodes block width, prefix mode and cipher + /// suite). See the scheme modules for assigned values. + pub scheme_id: u8, +} + +/// Serialised size of a [`WireHeader`] plus the block count it carries. +pub(crate) const WIRE_HEADER_LEN: usize = 4; + +impl WireHeader { + pub(crate) fn write(&self, num_blocks: usize, out: &mut Vec) { + debug_assert!(num_blocks <= u16::MAX as usize); + out.push(self.version); + out.push(self.scheme_id); + out.extend_from_slice(&(num_blocks as u16).to_be_bytes()); + } + + /// Validate `data`'s header against `self` and an expected block count, + /// returning the payload after the header. + pub(crate) fn strip<'a>( + &self, + num_blocks: usize, + data: &'a [u8], + ) -> Result<&'a [u8], ParseError> { + let (header, body) = parse_header(data)?; + if header != (self.version, self.scheme_id, num_blocks) { + return Err(ParseError); + } + Ok(body) + } +} + +/// `(version, scheme_id, block_count)` as parsed from a wire header. +pub(crate) type ParsedHeader = (u8, u8, usize); + +/// Split a headered slice into `((version, scheme_id, block_count), body)`. +pub(crate) fn parse_header(data: &[u8]) -> Result<(ParsedHeader, &[u8]), ParseError> { + if data.len() < WIRE_HEADER_LEN { + return Err(ParseError); + } + let count = u16::from_be_bytes([data[2], data[3]]) as usize; + Ok(((data[0], data[1], count), &data[WIRE_HEADER_LEN..])) +} + +fn header_len() -> usize { + if S::WIRE_HEADER.is_some() { + WIRE_HEADER_LEN + } else { + 0 + } +} + /// The trait of any encryption output (either Left, Right or combined). pub trait OreOutput: Sized { /// The size (in bytes) of this encrypted value @@ -72,7 +136,7 @@ pub trait CipherTextBlock: Default + Copy + std::fmt::Debug { } /// Error returned when a serialised ciphertext can't be parsed (wrong -/// length, malformed block, etc.). +/// length, malformed block, bad or mismatched wire header, etc.). #[derive(Debug, Error)] #[error("Unable to parse ORE Ciphertext")] pub struct ParseError; @@ -84,25 +148,30 @@ impl Left { f: [S::LeftBlockType::default(); N], } } -} -impl OreOutput for Left { - fn size() -> usize { + /// Serialised size of the headerless body. + pub(crate) fn body_size() -> usize { N * (S::LeftBlockType::BLOCK_SIZE + 1) } - fn to_bytes(&self) -> Vec { - let mut vec = Vec::with_capacity(N * S::LeftBlockType::BLOCK_SIZE); + fn write_body(&self, vec: &mut Vec) { + vec.extend_from_slice(&self.xt); self.f .iter() .for_each(|&block| vec.append(&mut block.to_bytes())); - - [self.xt.to_vec(), vec].concat() } - fn from_slice(data: &[u8]) -> Result { + fn from_body(data: &[u8]) -> Result { + if data.len() != Self::body_size() { + return Err(ParseError); + } let mut out = Self::init(); out.xt.copy_from_slice(&data[0..N]); + // `xt` is public, so this check may branch. A symbol outside the + // scheme's domain would index past the right block in the comparator. + if out.xt.iter().any(|&s| usize::from(s) >= S::SYMBOL_DOMAIN) { + return Err(ParseError); + } for i in 0..N { let block_start_index = N + (i * S::LeftBlockType::BLOCK_SIZE); out.f[i] = S::LeftBlockType::from_bytes( @@ -114,6 +183,29 @@ impl OreOutput for Left { } } +impl OreOutput for Left { + fn size() -> usize { + header_len::() + Self::body_size() + } + + fn to_bytes(&self) -> Vec { + let mut vec = Vec::with_capacity(Self::size()); + if let Some(header) = S::WIRE_HEADER { + header.write(N, &mut vec); + } + self.write_body(&mut vec); + vec + } + + fn from_slice(data: &[u8]) -> Result { + let body = match S::WIRE_HEADER { + Some(header) => header.strip(N, data)?, + None => data, + }; + Self::from_body(body) + } +} + impl Right { pub(crate) fn init() -> Self { Self { @@ -121,23 +213,23 @@ impl Right { data: [Default::default(); N], } } -} -impl OreOutput for Right { - fn size() -> usize { + /// Serialised size of the headerless body. + pub(crate) fn body_size() -> usize { (N * S::RightBlockType::BLOCK_SIZE) + NONCE_SIZE } - fn to_bytes(&self) -> Vec { - let mut vec = Vec::with_capacity(N * S::RightBlockType::BLOCK_SIZE); + fn write_body(&self, vec: &mut Vec) { + vec.extend_from_slice(&self.nonce); self.data .iter() .for_each(|&block| vec.append(&mut block.to_bytes())); - - [self.nonce.to_vec(), vec].concat() } - fn from_slice(data: &[u8]) -> Result { + fn from_body(data: &[u8]) -> Result { + if data.len() != Self::body_size() { + return Err(ParseError); + } let mut out = Self::init(); out.nonce.copy_from_slice(&data[0..NONCE_SIZE]); for i in 0..N { @@ -150,24 +242,59 @@ impl OreOutput for Right { } } +impl OreOutput for Right { + fn size() -> usize { + header_len::() + Self::body_size() + } + + fn to_bytes(&self) -> Vec { + let mut vec = Vec::with_capacity(Self::size()); + if let Some(header) = S::WIRE_HEADER { + header.write(N, &mut vec); + } + self.write_body(&mut vec); + vec + } + + fn from_slice(data: &[u8]) -> Result { + let body = match S::WIRE_HEADER { + Some(header) => header.strip(N, data)?, + None => data, + }; + Self::from_body(body) + } +} + impl OreOutput for CipherText { fn size() -> usize { - Left::::size() + Right::::size() + header_len::() + Left::::body_size() + Right::::body_size() } - /// Serialize the ciphertext into a vector of bytes + /// Serialize the ciphertext into a vector of bytes. Headered schemes + /// emit exactly one header for the combined artifact (not one per + /// half). fn to_bytes(&self) -> Vec { - [self.left.to_bytes(), self.right.to_bytes()].concat() + let mut vec = Vec::with_capacity(Self::size()); + if let Some(header) = S::WIRE_HEADER { + header.write(N, &mut vec); + } + self.left.write_body(&mut vec); + self.right.write_body(&mut vec); + vec } /// Deserialize from a slice of bytes fn from_slice(data: &[u8]) -> Result { - if data.len() != (Left::::size() + Right::::size()) { + let body = match S::WIRE_HEADER { + Some(header) => header.strip(N, data)?, + None => data, + }; + if body.len() != Left::::body_size() + Right::::body_size() { return Err(ParseError); } - let (left, right) = data.split_at(Left::::size()); - let left = Left::::from_slice(left)?; - let right = Right::::from_slice(right)?; + let (left, right) = body.split_at(Left::::body_size()); + let left = Left::::from_body(left)?; + let right = Right::::from_body(right)?; Ok(Self { left, right }) } diff --git a/packages/ore-rs/src/decimal.rs b/packages/ore-rs/src/decimal.rs index 2c52a578..046487f7 100644 --- a/packages/ore-rs/src/decimal.rs +++ b/packages/ore-rs/src/decimal.rs @@ -9,21 +9,23 @@ use crate::ciphertext::{CipherText, Left}; use crate::encrypt::OreEncrypt; +use crate::scheme::bit2::OreAes128; use crate::{OreCipher, OreError}; use orderable_bytes::{FixedOrderableBytes, OrderableBytes}; +use rand::{Rng, SeedableRng}; use rust_decimal::Decimal; const ENCODED_LEN: usize = ::ENCODED_LEN; -impl OreEncrypt for Decimal { - type LeftOutput = Left; - type FullOutput = CipherText; +impl OreEncrypt> for Decimal { + type LeftOutput = Left, ENCODED_LEN>; + type FullOutput = CipherText, ENCODED_LEN>; - fn encrypt_left(&self, cipher: &T) -> Result { + fn encrypt_left(&self, cipher: &OreAes128) -> Result { cipher.encrypt_left(&self.to_orderable_bytes()) } - fn encrypt(&self, cipher: &T) -> Result { + fn encrypt(&self, cipher: &OreAes128) -> Result { cipher.encrypt(&self.to_orderable_bytes()) } } diff --git a/packages/ore-rs/src/encrypt.rs b/packages/ore-rs/src/encrypt.rs index 1485bcb0..3354a033 100644 --- a/packages/ore-rs/src/encrypt.rs +++ b/packages/ore-rs/src/encrypt.rs @@ -1,7 +1,9 @@ use crate::ciphertext::*; +use crate::scheme::bit2::OreAes128; use crate::PlainText; use crate::{OreCipher, OreError}; use orderable_bytes::{FixedOrderableBytes, OrderableBytes}; +use rand::{Rng, SeedableRng}; /// Type-directed entry point for encrypting plaintext values with a given /// [`OreCipher`]. @@ -49,17 +51,24 @@ const CHAR_LEN: usize = ::ENCODED_LEN; const F32_LEN: usize = ::ENCODED_LEN; const F64_LEN: usize = ::ENCODED_LEN; +// These impls are specific to the legacy byte-per-block scheme (one +// plaintext byte = one ORE block, so the orderable-bytes encoding feeds +// the cipher directly). The 6-bit scheme provides its own impls in +// `scheme::bit2_w6`, which interpose the byte->block decomposition. They +// were previously blanket impls over `T: OreCipher`; scheme-specific +// impls keep coherence simple now that block count != byte count for +// some schemes. macro_rules! impl_ore_encrypt_via_orderable_bytes { ($type:ty, $len_const:ident) => { - impl OreEncrypt for $type { - type LeftOutput = Left; - type FullOutput = CipherText; + impl OreEncrypt> for $type { + type LeftOutput = Left, $len_const>; + type FullOutput = CipherText, $len_const>; - fn encrypt_left(&self, cipher: &T) -> Result { + fn encrypt_left(&self, cipher: &OreAes128) -> Result { cipher.encrypt_left(&self.to_orderable_bytes()) } - fn encrypt(&self, cipher: &T) -> Result { + fn encrypt(&self, cipher: &OreAes128) -> Result { cipher.encrypt(&self.to_orderable_bytes()) } } @@ -81,15 +90,15 @@ impl_ore_encrypt_via_orderable_bytes!(char, CHAR_LEN); impl_ore_encrypt_via_orderable_bytes!(f32, F32_LEN); impl_ore_encrypt_via_orderable_bytes!(f64, F64_LEN); -impl OreEncrypt for PlainText { - type LeftOutput = Left; - type FullOutput = CipherText; +impl OreEncrypt> for PlainText { + type LeftOutput = Left, N>; + type FullOutput = CipherText, N>; - fn encrypt_left(&self, cipher: &T) -> Result { + fn encrypt_left(&self, cipher: &OreAes128) -> Result { cipher.encrypt_left(self) } - fn encrypt(&self, cipher: &T) -> Result { + fn encrypt(&self, cipher: &OreAes128) -> Result { cipher.encrypt(self) } } diff --git a/packages/ore-rs/src/lib.rs b/packages/ore-rs/src/lib.rs index c993c155..27c19f2a 100644 --- a/packages/ore-rs/src/lib.rs +++ b/packages/ore-rs/src/lib.rs @@ -189,6 +189,17 @@ pub trait OreCipher: Sized { /// truth-table row). type RightBlockType: CipherTextBlock; + /// Wire-format header for this scheme's serialised artifacts, or + /// `None` for the legacy headerless format. Schemes introduced from + /// ORE v2 onwards must set this; see [`WireHeader`]. + const WIRE_HEADER: Option = None; + + /// Number of distinct per-block symbols (`xt` values) the scheme + /// produces: 256 for the byte-wide legacy scheme, 64 for Bit6. Parsers + /// reject a ciphertext whose `xt` holds a symbol outside `0..SYMBOL_DOMAIN`, + /// since the comparator indexes the right block by it. + const SYMBOL_DOMAIN: usize = 256; + /// Initialise the cipher from two 16-byte keys: `k1` for the /// per-block-tag PRF and `k2` for the per-block PRP seed PRF. fn init(k1: &[u8; 16], k2: &[u8; 16]) -> Result; diff --git a/packages/ore-rs/src/primitives.rs b/packages/ore-rs/src/primitives.rs index eff30941..7d154cf6 100644 --- a/packages/ore-rs/src/primitives.rs +++ b/packages/ore-rs/src/primitives.rs @@ -2,6 +2,7 @@ pub mod hash; pub mod prf; pub mod prp; pub mod simd; +pub(crate) mod stack; use aes::cipher::{consts::U16, generic_array::GenericArray}; use aes::Block; diff --git a/packages/ore-rs/src/primitives/hash.rs b/packages/ore-rs/src/primitives/hash.rs index 829a32a5..3c602b8e 100644 --- a/packages/ore-rs/src/primitives/hash.rs +++ b/packages/ore-rs/src/primitives/hash.rs @@ -53,6 +53,130 @@ impl Hash for Aes128Z2Hash { } } +/// Z2 hash instantiated as the BHKR fixed-key-AES **σ-MMO** construction: +/// `H(x, r) = LSB(π(σ(x) ⊕ r) ⊕ σ(x) ⊕ r)`, where `π` is a *fixed public* +/// AES-128 permutation (public key [`PI_KEY`]), `r` is the per-ciphertext +/// nonce, and `σ(x) = 2·x` is the GF(2^128) doubling orthomorphism (the +/// BHKR/Zahur linear orthomorphism; both `σ` and `σ ⊕ id` are permutations). +/// v2 plan §6 option 3 — **A1 resolved 2026-06-15** — analysed in the +/// random-permutation model (BHKR13; GKWY20; Guo–Katz–Wang–Weng–Yu, eprint +/// 2019/1168). +/// +/// `H`'s first input is not always secret: a left ciphertext publishes the RO +/// key at its own symbol, and the comparator hashes it. For the *unrevealed* +/// RO keys, `σ(k) ⊕ r` is a secret shared across ciphertexts plus public +/// per-ciphertext nonces, which is the BHKR correlation-robustness setting; +/// security rests on its multi-instance bound (about `p·C/2^128` for `p` +/// offline `π` queries against `C` targeted keys), not on the inputs being +/// secret. The orthomorphism `σ` is what puts the construction in that named +/// setting. See review brief A1 (restated, pending sign-off). The cryptanalysis of round-reduced AES hashing (eprint 2025/792) +/// targets collision/preimage/one-wayness — properties this 1-bit hash does +/// not rely on — and never reaches full-round AES-128. +/// +/// The `Hash::new` "key" parameter carries the **nonce** `r`; the AES key is +/// the public constant [`PI_KEY`], expanded once per process. +pub struct FixedPiZ2Hash { + nonce: AesBlock, +} + +/// The public, fixed AES key for `π`. Nothing-up-my-sleeve: the ASCII +/// bytes of `"ORE-rs.v2.H-pi.1"`. This key is deliberately *not* secret — +/// the construction's security rests on AES being a good public random +/// permutation, not on key secrecy (the comparator must be able to +/// evaluate H with no key material). +pub const PI_KEY: [u8; 16] = *b"ORE-rs.v2.H-pi.1"; + +fn pi() -> &'static Aes128 { + use std::sync::OnceLock; + static PI: OnceLock = OnceLock::new(); + PI.get_or_init(|| Aes128::new(GenericArray::from_slice(&PI_KEY))) +} + +/// GF(2^128) doubling `2·x` on a big-endian field element packed in a `u128` +/// — the BHKR/Zahur orthomorphism `σ` ("multiply by x"; reduction polynomial +/// x^128 + x^7 + x^2 + x + 1, constant `0x87`). A single shift plus a +/// branch-free conditional XOR, replacing the per-byte carry loop that +/// dominated the σ-MMO hot path (≈704 doublings per Bit6 u64 encrypt). +/// Constant-time: no secret-dependent control flow. +#[inline] +fn gf128_double_u128(x: u128) -> u128 { + // `x << 1` discards the top bit (the GF reduction trigger); fold 0x87 into + // the low byte iff that bit was set. `x >> 127` is 0 or 1. + (x << 1) ^ ((x >> 127) * 0x87) +} + +/// In-place GF(2^128) doubling `b ← 2·b`; `b` must be exactly 16 bytes +/// (big-endian field element). Used by the scalar comparator path; the bulk +/// encrypt path folds the doubling and nonce XOR into one `u128` pass (see +/// `hash_all_into`). +#[inline] +fn gf128_double(b: &mut [u8]) { + debug_assert_eq!(b.len(), 16); + let mut arr = [0u8; 16]; + arr.copy_from_slice(b); + let doubled = gf128_double_u128(u128::from_be_bytes(arr)); + b.copy_from_slice(&doubled.to_be_bytes()); +} + +impl Hash for FixedPiZ2Hash { + fn new(nonce: &HashKey) -> Self { + Self { nonce: *nonce } + } + + fn hash(&self, data: &[u8]) -> u8 { + assert_eq!(data.len(), 16); + // BHKR σ-MMO: m = σ(x) ⊕ r; return lsb(π(m) ⊕ m). + let mut block = [0u8; 16]; + block.copy_from_slice(data); + gf128_double(&mut block); // σ(x) = 2x + for (slot, &r) in block.iter_mut().zip(self.nonce.iter()) { + *slot ^= r; // m = σ(x) ⊕ r + } + let m_lsb = block[0] & 1u8; + let block = GenericArray::from_mut_slice(&mut block); + pi().encrypt_block(block); + (block[0] & 1u8) ^ m_lsb + } + + fn hash_all_into(&self, data: &mut [AesBlock], out: &mut [u8]) { + debug_assert_eq!(out.len() * 8, data.len()); + + // BHKR σ-MMO: m = σ(x) ⊕ r, then out = lsb(m) ^ lsb(π(m)), with + // σ(x) = 2x in GF(2^128). Form m in place first so the feedforward + // captures lsb(m) rather than lsb(x). Doubling and nonce XOR are fused + // into one u128 pass per block — this is the 704-evals/u64 hot loop. + let mut nonce_arr = [0u8; 16]; + nonce_arr.copy_from_slice(self.nonce.as_slice()); + let nonce = u128::from_be_bytes(nonce_arr); + for block in data.iter_mut() { + let mut arr = [0u8; 16]; + arr.copy_from_slice(block.as_slice()); + let m = gf128_double_u128(u128::from_be_bytes(arr)) ^ nonce; + block.copy_from_slice(&m.to_be_bytes()); + } + + // feedforward lsb(m) + if data.len() == 256 { + crate::primitives::simd::lsb_mask_256(data, out); + } else { + crate::primitives::simd::scalar::lsb_mask(data, out); + } + + pi().encrypt_blocks(data); // π(m) + + let mut pi_mask = [0u8; 32]; + let pi_mask = &mut pi_mask[..out.len()]; + if data.len() == 256 { + crate::primitives::simd::lsb_mask_256(data, pi_mask); + } else { + crate::primitives::simd::scalar::lsb_mask(data, pi_mask); + } + for (slot, &m) in out.iter_mut().zip(pi_mask.iter()) { + *slot ^= m; + } + } +} + #[cfg(test)] mod tests { use super::*; @@ -97,4 +221,56 @@ mod tests { hash.hash(&input); } + + // The comparator uses the scalar `hash`; encryption uses the bulk + // `hash_all_into`. For the BHKR σ-MMO they must agree bit-for-bit, over + // both the scalar (n=64, the Bit6 domain) and SIMD (n=256) `lsb_mask` + // backends. + #[test] + fn fixed_pi_scalar_matches_bulk() { + let nonce: [u8; 16] = hex!("0f0e0d0c 0b0a0908 07060504 03020100"); + let h: FixedPiZ2Hash = Hash::new(GenericArray::from_slice(&nonce)); + + for &n in &[64usize, 256usize] { + let mut blocks: Vec = (0..n) + .map(|i| { + let mut b = [0u8; 16]; + for (j, slot) in b.iter_mut().enumerate() { + *slot = (i.wrapping_mul(31).wrapping_add(j)) as u8; + } + *GenericArray::from_slice(&b) + }) + .collect(); + + let mut expected = vec![0u8; n / 8]; + for (i, b) in blocks.iter().enumerate() { + expected[i / 8] |= h.hash(b.as_slice()) << (i % 8); + } + + let mut out = vec![0u8; n / 8]; + h.hash_all_into(&mut blocks, &mut out); + assert_eq!(out, expected, "scalar vs bulk mismatch for n={}", n); + } + } + + // σ(x) = 2x must be an orthomorphism: both σ and σ⊕id are permutations. + // Spot-check the GF(2^128) doubling against the textbook shift/0x87 rule. + #[test] + fn gf128_double_reduction() { + // High bit clear: pure left shift. + let mut b = [0u8; 16]; + b[15] = 0x01; + gf128_double(&mut b); + let mut want = [0u8; 16]; + want[15] = 0x02; + assert_eq!(b, want); + + // High bit set: shift then XOR 0x87 into the low byte. + let mut b = [0u8; 16]; + b[0] = 0x80; + gf128_double(&mut b); + let mut want = [0u8; 16]; + want[15] = 0x87; + assert_eq!(b, want); + } } diff --git a/packages/ore-rs/src/primitives/prp.rs b/packages/ore-rs/src/primitives/prp.rs index 829d68da..70d0c1ae 100644 --- a/packages/ore-rs/src/primitives/prp.rs +++ b/packages/ore-rs/src/primitives/prp.rs @@ -1,8 +1,30 @@ +pub(crate) mod oblivious; pub mod prng; use crate::primitives::prp::prng::Aes128Prng; -use crate::primitives::{Prp, PrpError, PrpResult}; +use crate::primitives::{AesBlock, Prp, PrpError, PrpResult}; +use aes::cipher::{generic_array::GenericArray, BlockEncrypt, KeyInit}; +use aes::Aes128Enc; use zeroize::{Zeroize, ZeroizeOnDrop}; +/// Read `table[index]` without a secret-dependent memory access: every +/// entry is read and the wanted one selected in constant time. `index` is +/// a plaintext symbol (`permute`) or a candidate (`invert`); an aligned +/// table only defends cache-line granularity, and sub-line timing would +/// otherwise reveal which part of the table was read. +/// +/// Out-of-range `index` is an error, decided by a comparison against the +/// public table length. +#[inline] +fn oblivious_lookup(table: &[u8], index: u8) -> PrpResult { + if usize::from(index) >= table.len() { + return Err(PrpError); + } + Ok(crate::scheme::width::ct_select_byte( + table, + usize::from(index), + )) +} + #[derive(Zeroize)] pub struct KnuthShufflePRP { permutation: [T; N], @@ -19,78 +41,213 @@ impl Drop for KnuthShufflePRP { // Impl the ZeroizeOnDrop marker trait since we're zeroizing above impl ZeroizeOnDrop for KnuthShufflePRP {} -impl Prp for KnuthShufflePRP { - /* - * Initialize an 8-bit (256 element) PRP using a KnuthShuffle - * and a 64-bit random seed - */ - fn new(key: &[u8]) -> PrpResult { - let mut rng = Aes128Prng::init(key); // TODO: Use Result type here, too +/// Implements `Prp` for a Knuth-shuffle PRP over a `$domain`-element +/// space (`$domain ≤ 256`), with `$gt_mask` as the bulk indicator kernel. +/// The shuffle algorithm and PRNG byte consumption are identical across +/// domains; only the iteration bounds change. For the 256 domain this is +/// wire-frozen (the legacy bit2 scheme); other domains are used by new +/// schemes only. +macro_rules! impl_knuth_shuffle_prp { + ($domain:literal, $gt_mask:path) => { + impl Prp for KnuthShufflePRP { + /* + * Initialize a ($domain element) PRP using a KnuthShuffle + * seeded from a 16-byte key + */ + fn new(key: &[u8]) -> PrpResult { + let mut rng = Aes128Prng::init(key); // TODO: Use Result type here, too - let mut perm = Self { - permutation: [0u8; 256], - inverse: [0u8; 256], - }; + let mut perm = Self { + permutation: [0u8; $domain], + inverse: [0u8; $domain], + }; - // Initialize values - for i in 0..=255 { - perm.permutation[i] = i as u8; - } + // Initialize values + for i in 0..$domain { + perm.permutation[i] = i as u8; + } - // Iterations stop at i = 1: the i = 0 step always degenerates to - // `swap(0, 0)` after drawing rejection-sampled bytes until one is - // zero (expected 256 draws — a full PRNG regeneration), and the RNG - // is dropped right after this loop, so skipping it consumes no - // observable state and yields a byte-identical permutation. - (1..=255usize).rev().for_each(|i| { - let j = rng.gen_range(i as u8); - perm.permutation.swap(i, j as usize); - }); - - for (index, val) in perm.permutation.iter().enumerate() { - perm.inverse[*val as usize] = index as u8; - } + // Iterations stop at i = 1: the i = 0 step always + // degenerates to `swap(0, 0)` after drawing + // rejection-sampled bytes until one is zero (expected 256 + // draws — a full PRNG regeneration), and the RNG is dropped + // right after this loop, so skipping it consumes no + // observable state and yields a byte-identical permutation. + (1..$domain).rev().for_each(|i| { + let j = rng.gen_range(i as u8); + perm.permutation.swap(i, j as usize); + }); - Ok(perm) - } + for (index, val) in perm.permutation.iter().enumerate() { + perm.inverse[*val as usize] = index as u8; + } - /* - * Permutes a number under the Pseudo-Random Permutation in constant time. - * - * Forward permutations are only used once in the ORE scheme so this is OK - */ - fn permute(&self, input: u8) -> PrpResult { - let index = usize::from(input); - - match self.inverse.get(index) { - Some(i) => Ok(*i), - None => Err(PrpError), - } - } + Ok(perm) + } + + /* + * Permutes a number under the Pseudo-Random Permutation in constant time. + * + * Forward permutations are only used once in the ORE scheme so this is OK + */ + fn permute(&self, input: u8) -> PrpResult { + oblivious_lookup(&self.inverse, input) + } + + /* + * Performs the inverse permutation in constant time. + */ + fn invert(&self, input: u8) -> PrpResult { + // Forward and inverse permutations are reversed for historical reasons + oblivious_lookup(&self.permutation, input) + } - /* - * Performs the inverse permutation in constant time. - */ - fn invert(&self, input: u8) -> PrpResult { - let index = usize::from(input); + fn indicator_mask_xor(&self, data: u8, out: &mut [u8]) { + debug_assert_eq!(out.len() * 8, $domain); - // Forward an inverse permutations are reversed for historical reasons - match self.permutation.get(index) { - Some(i) => Ok(*i), - None => Err(PrpError), + // `invert(j)` is `self.permutation[j]` (see `invert` + // above), so the mask is one pass over the table: a + // bytewise `> data` compare packed to bits — vectorised + // where the target supports it. + $gt_mask(&self.permutation, data, out); + } } - } + }; +} + +// The legacy bit2 scheme (256) is wire-frozen on the Knuth shuffle. New +// schemes use [`LemireFyPrp`] below; no other domain is instantiated here. +impl_knuth_shuffle_prp!(256, crate::primitives::simd::gt_mask_xor_256); - fn indicator_mask_xor(&self, data: u8, out: &mut [u8]) { - debug_assert_eq!(out.len() * 8, 256); +/// PRP over a small domain via a Fisher–Yates shuffle driven by a fixed +/// number of **wide draws** — full 64-bit values reduced to range by +/// Lemire's multiply-high (`(x * range) >> 64`) — with the randomness +/// produced by AES-CTR under the 16-byte seed. Same field/method +/// semantics as [`KnuthShufflePRP`] (`permute` ↦ `inverse`, `invert` and +/// the indicator mask ↦ `permutation`). +/// +/// Contrast with [`KnuthShufflePRP`], which draws single bytes and uses +/// **rejection sampling** to avoid modulo bias: there the number of draws +/// (and PRNG buffer regenerations, and branches taken) depends on the +/// seed, and the seed is `PRF(plaintext prefix)`, so PRP construction time +/// is weakly plaintext-dependent — a timing side-channel. This construction +/// has a seed-independent, branch-free draw count, closing that channel, +/// and is ~9× faster (no rejection loop, no `%`, just multiply-high). +/// +/// Uniformity: each Lemire reduction to range `m` deviates from uniform by +/// at most `m / 2^64`; over the `N-1` draws the output permutation is within +/// statistical distance `< 2^-55` (for `N = 64`) of a uniformly random +/// permutation — exactly the object Lewi-Wu's analysis assumes. That is a +/// pure statistical term on top of the scheme's existing PRF advantage: no +/// new assumption, no new idealised model. +/// +/// New (non-wire-frozen) schemes only. +/// +/// Constant-time construction: key generation goes through the oblivious +/// builders in [`oblivious`] (NEON on aarch64, SSSE3 on x86_64 when the CPU +/// has it, portable SWAR elsewhere), which never use a secret value as a +/// memory address. Textbook Fisher–Yates swaps at the secret draw-derived +/// index and fills `inverse` at the secret permutation values; those +/// offsets stay within one cache line, but dudect measured a timing +/// dependence on the swap sequence on Apple M4, which the one-cache-line +/// argument does not cover (review brief A4). The oblivious builders apply +/// the same draws and swaps, so the tables, and every ciphertext, are +/// byte-identical to the textbook form's. +/// +/// Layout: `#[repr(C, align(64))]` places `permutation` at offset 0 (line 0) +/// and `inverse` at offset 64 (line 1); `repr(C)` pins field order and +/// `align(64)` puts the struct on a cache-line boundary. The lookups +/// (`permute`, `invert`) read every entry either way. +#[derive(Zeroize)] +#[repr(C, align(64))] +pub struct LemireFyPrp { + permutation: [u8; N], + inverse: [u8; N], +} - // `invert(j)` is `self.permutation[j]` (see `invert` above), so the - // mask is one pass over the table: a bytewise `> data` compare - // packed to bits — vectorised where the target supports it. - crate::primitives::simd::gt_mask_xor_256(&self.permutation, data, out); +impl Drop for LemireFyPrp { + fn drop(&mut self) { + self.zeroize(); } } +impl ZeroizeOnDrop for LemireFyPrp {} + +/// Implements `Prp` for [`LemireFyPrp`] over `$domain` elements, drawing +/// `$domain - 1` wide values from `$stream_blocks` AES-CTR blocks +/// (`$stream_blocks == ((($domain - 1) * 8) + 15) / 16`). `$gt_mask` is the +/// bulk indicator kernel for the domain. +macro_rules! impl_lemire_fy_prp { + ($domain:literal, $stream_blocks:literal, $gt_mask:path) => { + // The oblivious builders are written for exactly 64 entries (four + // 16-byte registers, eight 64-bit words per table); make another + // domain a compile error rather than a type mismatch deep inside. + const _: () = assert!( + $domain == oblivious::DOMAIN, + "LemireFyPrp: the oblivious builders are written for a 64-entry domain" + ); + + impl Prp for LemireFyPrp<$domain> { + fn new(key: &[u8]) -> PrpResult { + if key.len() < 16 { + return Err(PrpError); + } + + // Fixed-count AES-CTR keystream from the seed: counter in + // the first 4 bytes (big-endian), matching the existing + // PRNG's counter convention. + // Encryption only: no decryption schedule is built or copied. + let cipher = Aes128Enc::new(GenericArray::from_slice(&key[0..16])); + let mut blocks = [AesBlock::default(); $stream_blocks]; + for (i, b) in blocks.iter_mut().enumerate() { + b[0..4].copy_from_slice(&(i as u32).to_be_bytes()); + } + cipher.encrypt_blocks(&mut blocks); + + let mut stream = [0u8; $stream_blocks * 16]; + for (i, b) in blocks.iter().enumerate() { + stream[i * 16..(i + 1) * 16].copy_from_slice(b); + } + + // Fisher–Yates with Lemire-reduced wide draws: draw `d` + // (8 bytes) drives step `$domain - 1 - d`. Fixed trip count, + // branch-free index reduction, and no secret-dependent + // address: the oblivious builder keeps both tables in + // registers (or masked words) and maintains the inverse + // alongside the swaps. It wipes its copies of each draw. + let mut perm = Self { + permutation: [0u8; $domain], + inverse: [0u8; $domain], + }; + oblivious::build(&stream, &mut perm.permutation, &mut perm.inverse); + + // The keystream determined the permutation — wipe it. + stream.zeroize(); + for b in blocks.iter_mut() { + b.as_mut_slice().zeroize(); + } + + Ok(perm) + } + + fn permute(&self, input: u8) -> PrpResult { + oblivious_lookup(&self.inverse, input) + } + + fn invert(&self, input: u8) -> PrpResult { + oblivious_lookup(&self.permutation, input) + } + + fn indicator_mask_xor(&self, data: u8, out: &mut [u8]) { + debug_assert_eq!(out.len() * 8, $domain); + $gt_mask(&self.permutation, data, out); + } + } + }; +} + +impl_lemire_fy_prp!(64, 32, crate::primitives::simd::gt_mask_xor_64); + #[cfg(test)] mod tests { use super::*; @@ -138,4 +295,75 @@ mod tests { Ok(()) } + + // ----------------------------------------------------------------- + // LemireFyPrp (Bit6 PRP) + // ----------------------------------------------------------------- + + fn init_fy(seed_byte: u8) -> LemireFyPrp<64> { + Prp::new(&[seed_byte; 16]).unwrap() + } + + #[test] + fn fy_is_a_permutation_and_round_trips() { + for seed in 0u8..32 { + let prp = init_fy(seed); + // Every value 0..64 appears exactly once in `permutation`. + let mut seen = [false; 64]; + for v in 0..64u8 { + let mapped = prp.invert(v).unwrap(); + assert!(mapped < 64); + assert!( + !seen[mapped as usize], + "value {} repeated (seed {})", + mapped, seed + ); + seen[mapped as usize] = true; + } + // Forward/inverse round-trip both directions. + for v in 0..64u8 { + assert_eq!(v, prp.invert(prp.permute(v).unwrap()).unwrap()); + assert_eq!(v, prp.permute(prp.invert(v).unwrap()).unwrap()); + } + } + } + + #[test] + fn fy_is_deterministic() { + let a = init_fy(7); + let b = init_fy(7); + for v in 0..64u8 { + assert_eq!(a.permute(v).unwrap(), b.permute(v).unwrap()); + } + // A different seed gives a different permutation (overwhelmingly). + let c = init_fy(8); + assert!((0..64u8).any(|v| a.permute(v).unwrap() != c.permute(v).unwrap())); + } + + #[test] + fn fy_rejects_short_key() { + assert!( as Prp>::new(&[0u8; 8]).is_err()); + } + + quickcheck! { + /// The bulk indicator mask must agree with the per-bit reference for + /// the Bit6 PRP too (guards the gt_mask_xor_64 kernel path). + fn fy_indicator_mask_matches_reference(key: Vec, x: u8) -> quickcheck::TestResult { + if key.len() < 16 { + return quickcheck::TestResult::discard(); + } + let prp: LemireFyPrp<64> = Prp::new(&key[0..16]).unwrap(); + + let mut mask = [0u8; 8]; + prp.indicator_mask_xor(x, &mut mask); + + let mut reference = [0u8; 8]; + for j in 0..64u8 { + let indicator = u8::from(prp.invert(j).unwrap() > x); + reference[(j / 8) as usize] |= indicator << (j % 8); + } + + quickcheck::TestResult::from_bool(mask == reference) + } + } } diff --git a/packages/ore-rs/src/primitives/prp/oblivious.rs b/packages/ore-rs/src/primitives/prp/oblivious.rs new file mode 100644 index 00000000..986974a2 --- /dev/null +++ b/packages/ore-rs/src/primitives/prp/oblivious.rs @@ -0,0 +1,520 @@ +//! Oblivious Fisher–Yates builders for [`LemireFyPrp<64>`](super::LemireFyPrp). +//! +//! Each builder produces exactly the `permutation` and `inverse` tables of +//! the textbook builder (`reference`, test-only) for the same stream: the +//! same 63 Lemire-reduced draws, the same swap sequence. What changes is the +//! memory access pattern. The textbook builder swaps at the draw-derived +//! index `j` and fills `inverse` at the permutation's values, so the +//! *offsets* of its loads and stores within the table's cache line depend on +//! the key-derived stream. Those offsets stay inside one cache line, so the +//! cache does not see them, but dudect on Apple M4 measured the build time +//! depending on the swap sequence (the swaps carry it, the inverse fill does +//! not), consistent with store-to-load forwarding and memory disambiguation +//! on byte-granular accesses at secret offsets. The builders here never use +//! a secret value as an address: +//! +//! - `swar` holds each table as eight `u64` words and does every +//! secret-indexed read and write as a masked pass over the words, with +//! exact SWAR byte-equality masks. Portable fallback. +//! - `scalar` (tests only) is the same construction a byte at a time with +//! `subtle_ng` choices. +//! - `neon` (aarch64) holds both tables in eight 16-byte registers for the +//! whole build: `tbl` fetches `perm[j]` by a broadcast index, and +//! compare-and-select against a constant index vector writes back. +//! Nothing touches memory but the stream reads until the two final +//! stores. +//! - `ssse3` (x86_64, runtime detected) is the same register-resident shape +//! with `pshufb` over four 16-byte chunks, the chunk chosen by a mask. +//! +//! The swap of `perm[i]` and `perm[j]`, with `a = perm[i]` and +//! `b = perm[j]` before it, moves `a` to position `j` and `b` to position +//! `i`; the inverse therefore changes at exactly two points, +//! `inverse[a] = j` and `inverse[b] = i`. Starting both tables at the +//! identity and applying that update with every swap keeps `inverse` the +//! inverse of `perm` throughout, so no separate fill is needed. When +//! `i == j`, `a == b` and both updates write the same value. `scalar` +//! applies the update in that form. The other builders use the equivalent +//! value-side form: the new inverse is `s ∘ inverse` with `s` the +//! transposition of `i` and `j`, so they exchange the *values* `i` and `j` +//! wherever they occur in `inverse`, which needs neither `a` nor `b`. +//! +//! Step `i` is public and `j <= i`, so `perm[j]` can only be in the part of +//! the table at or below `i`. `swar`, `neon` and `ssse3` search and rewrite +//! only the words or 16-byte chunks that cover `0..=i`, a bound that +//! depends on `i` alone. +//! +//! The draw arithmetic (`umulh` / `mul` on the 64-bit draw) is data +//! independent on the targets this crate ships to; the loop trip count is +//! fixed. The only secret-dependent values are register operands. + +use zeroize::Zeroize; + +/// The domain the oblivious builders support. +pub(crate) const DOMAIN: usize = 64; + +/// Bytes of stream the builders consume: one 8-byte draw per step. +pub(crate) const STREAM_BYTES: usize = (DOMAIN - 1) * 8; + +/// The Lemire-reduced draw for Fisher–Yates step `i` (`1..64`): draw +/// `63 - i`, as a little-endian u64, mapped to `0..=i` by multiply-high. +/// Identical to the indexed builder's reduction. +#[inline(always)] +fn lemire_draw(stream: &[u8], i: usize) -> u8 { + let d = DOMAIN - 1 - i; + let mut draw = [0u8; 8]; + draw.copy_from_slice(&stream[d * 8..d * 8 + 8]); + let mut x = u64::from_le_bytes(draw); + let j = ((x as u128 * (i as u128 + 1)) >> 64) as u8; + // `draw` and `x` are copies of the key-derived keystream; the caller + // wipes `stream`, so wipe these too (as the indexed builder does). + draw.zeroize(); + x.zeroize(); + j +} + +/// Build `perm` and `inverse` from `stream` (at least [`STREAM_BYTES`] +/// bytes) with a data-independent memory access pattern, using the fastest +/// oblivious builder for the target. +#[inline] +pub(crate) fn build(stream: &[u8], perm: &mut [u8; DOMAIN], inverse: &mut [u8; DOMAIN]) { + assert!(stream.len() >= STREAM_BYTES); + + #[cfg(target_arch = "aarch64")] + // SAFETY: NEON is baseline on aarch64; `stream` length asserted above. + unsafe { + neon::build(stream, perm, inverse) + } + + #[cfg(not(target_arch = "aarch64"))] + { + #[cfg(target_arch = "x86_64")] + if is_x86_feature_detected!("ssse3") { + // SAFETY: SSSE3 presence just checked; `stream` length asserted. + unsafe { ssse3::build(stream, perm, inverse) }; + return; + } + + swar::build(stream, perm, inverse); + } +} + +/// The reference builder: textbook Fisher–Yates, swapping at the +/// draw-derived index, then `inverse[perm[k]] = k`. Both use a secret value +/// as an address, which is the timing channel the oblivious builders close, +/// so it is never compiled into the library: it exists only as the +/// specification the oblivious builders are tested against. +#[cfg(test)] +pub(crate) mod reference { + use super::*; + + pub(crate) fn build(stream: &[u8], perm: &mut [u8; DOMAIN], inverse: &mut [u8; DOMAIN]) { + assert!(stream.len() >= STREAM_BYTES); + for (k, p) in perm.iter_mut().enumerate() { + *p = k as u8; + } + for i in (1..DOMAIN).rev() { + perm.swap(i, usize::from(lemire_draw(stream, i))); + } + for (k, v) in perm.iter().enumerate() { + inverse[usize::from(*v)] = k as u8; + } + } +} + +/// Byte-at-a-time oblivious builder with `subtle_ng` choices: full-scan +/// reads and masked full-table writes, the inverse maintained alongside. +/// The most direct form of the construction, kept as a second reference +/// for the tests; `subtle_ng`'s per-choice optimisation barrier makes it +/// about 17x slower than [`swar`], which the dispatcher uses instead. +#[cfg(test)] +pub(crate) mod scalar { + use super::*; + use subtle_ng::{ConditionallySelectable, ConstantTimeEq}; + + pub(crate) fn build(stream: &[u8], perm: &mut [u8; DOMAIN], inverse: &mut [u8; DOMAIN]) { + assert!(stream.len() >= STREAM_BYTES); + for (k, (p, v)) in perm.iter_mut().zip(inverse.iter_mut()).enumerate() { + *p = k as u8; + *v = k as u8; + } + + for i in (1..DOMAIN).rev() { + let j = lemire_draw(stream, i); + let i8_ = i as u8; + // `i` is public: a direct read at it is not secret-indexed. + let a = perm[i]; + let mut b = 0u8; + for (k, p) in perm.iter().enumerate() { + b.conditional_assign(p, (k as u8).ct_eq(&j)); + } + + // perm[j] = a, then perm[i] = b (public address). + for (k, p) in perm.iter_mut().enumerate() { + p.conditional_assign(&a, (k as u8).ct_eq(&j)); + } + perm[i] = b; + + // inverse[a] = j, then inverse[b] = i. + for (k, v) in inverse.iter_mut().enumerate() { + let k = k as u8; + v.conditional_assign(&j, k.ct_eq(&a)); + v.conditional_assign(&i8_, k.ct_eq(&b)); + } + } + } +} + +/// Portable oblivious builder in 64-bit SWAR form: each table is eight +/// `u64` words, byte equality against a broadcast key is exact zero-byte +/// detection, and every secret-indexed read and write is a masked pass +/// over every word that could hold the entry (words `0..=i/8` at step `i`, +/// a public bound). The inverse is updated on the value side, as in the +/// NEON builder. Pure integer arithmetic with no `subtle_ng` optimisation barrier +/// per byte, so the compiler can keep the tables in registers. +#[cfg(any(not(target_arch = "aarch64"), test))] +pub(crate) mod swar { + use super::*; + + const LO7: u64 = 0x7f7f_7f7f_7f7f_7f7f; + const ONES: u64 = 0x0101_0101_0101_0101; + + /// `0xff` in each byte of `x` that equals `key`, `0x00` elsewhere. + /// Exact (no false positives from borrows): the high bit of + /// `(z & 0x7f) + 0x7f` is set iff the low seven bits of `z` are not all + /// zero, and OR-ing `z` adds the eighth. + #[inline(always)] + fn eq_mask(x: u64, key: u64) -> u64 { + let z = x ^ key; + let t = (z & LO7).wrapping_add(LO7); + let m = !(t | z | LO7); + (m >> 7).wrapping_mul(0xff) + } + + /// Read `table[k]` from every word of `table`; `kb` is `k` in every + /// byte. + #[inline(always)] + fn fetch(table: &[u64], index: &[u64], kb: u64) -> u8 { + let mut r = 0u64; + for (w, idx) in table.iter().zip(index) { + r |= w & eq_mask(*idx, kb); + } + r |= r >> 32; + r |= r >> 16; + r |= r >> 8; + r as u8 + } + + /// `table[k] = value` for the one `k` with `index[k] == key`. + #[inline(always)] + fn put(table: &mut [u64], index: &[u64], kb: u64, value: u64) { + for (w, idx) in table.iter_mut().zip(index) { + let m = eq_mask(*idx, kb); + *w = (*w & !m) | (value & m); + } + } + + pub(crate) fn build(stream: &[u8], perm: &mut [u8; DOMAIN], inverse: &mut [u8; DOMAIN]) { + assert!(stream.len() >= STREAM_BYTES); + let mut index = [0u64; 8]; + for (w, word) in index.iter_mut().enumerate() { + let mut b = [0u8; 8]; + for (k, byte) in b.iter_mut().enumerate() { + *byte = (w * 8 + k) as u8; + } + *word = u64::from_le_bytes(b); + } + let mut p = index; + let mut v = index; + + for i in (1..DOMAIN).rev() { + // `i` is public, so are its word `w` and byte `lane`, and so is + // every bound below; `j <= i` puts perm[j] in words `0..=w`. + let (w, lane) = (i / 8, 8 * (i % 8)); + let ib = (i as u64).wrapping_mul(ONES); + let jb = u64::from(lemire_draw(stream, i)).wrapping_mul(ONES); + let a = (p[w] >> lane) & 0xff; + let bb = u64::from(fetch(&p[..=w], &index[..=w], jb)).wrapping_mul(ONES); + // perm[j] = a, then perm[i] = b. + put(&mut p[..=w], &index[..=w], jb, a.wrapping_mul(ONES)); + p[w] = (p[w] & !(0xff << lane)) | ((bb & 0xff) << lane); + // inverse = s ∘ inverse: exchange the values i and j. + let flip = ib ^ jb; + for word in v.iter_mut() { + let m = eq_mask(*word, ib) | eq_mask(*word, jb); + *word ^= flip & m; + } + } + + for (k, (pw, vw)) in p.iter().zip(v.iter()).enumerate() { + perm[k * 8..k * 8 + 8].copy_from_slice(&pw.to_le_bytes()); + inverse[k * 8..k * 8 + 8].copy_from_slice(&vw.to_le_bytes()); + } + } +} + +#[cfg(target_arch = "aarch64")] +mod neon { + use super::{lemire_draw, DOMAIN}; + use core::arch::aarch64::*; + + /// `0, 1, …, 63`: the lane index of each table entry. + const INDEX: [u8; DOMAIN] = { + let mut t = [0u8; DOMAIN]; + let mut k = 0; + while k < DOMAIN { + t[k] = k as u8; + k += 1; + } + t + }; + + /// Exchange the lane values `x` and `y` wherever they occur in `t`. + #[inline(always)] + unsafe fn swap_values(t: uint8x16_t, x: uint8x16_t, y: uint8x16_t) -> uint8x16_t { + let to_y = vceqq_u8(t, x); + let to_x = vceqq_u8(t, y); + vbslq_u8(to_y, y, vbslq_u8(to_x, x, t)) + } + + /// The Fisher–Yates steps whose `i` lies in 16-lane chunk `C` + /// (`i` from `16C + 15` down to `max(16C, 1)`). + /// + /// `C` is public, and so is everything it decides: `perm[i]` is read + /// from chunk `C` by a one-register `tbl` at a public lane, and since + /// `j <= i`, `perm[j]` lies in chunks `0..=C`, so it is fetched with a + /// `(C + 1)`-register `tbl` and written back with a masked select over + /// those chunks only. Chunks above `C` are final and not touched again. + /// + /// The inverse is updated on the value side. The swap moves the value at + /// position `i` to `j` and vice versa, so the new inverse is + /// `s ∘ inverse` with `s` the transposition of `i` and `j`: exchange the + /// *values* `i` and `j` wherever they occur in `inverse`. That needs + /// neither `perm[i]` nor `perm[j]`, so it stays off the loop-carried + /// dependency through `perm`. It covers all four chunks. + #[inline(always)] + unsafe fn steps( + stream: &[u8], + p: &mut [uint8x16_t; 4], + v: &mut [uint8x16_t; 4], + index: &[uint8x16_t; 4], + ) { + let low = if C == 0 { 1 } else { 16 * C }; + for i in (low..16 * (C + 1)).rev() { + let iv = vdupq_n_u8(i as u8); + let jv = vdupq_n_u8(lemire_draw(stream, i)); + // Every lane of `a` holds perm[i], every lane of `b` perm[j]: + // register-table lookups, not memory accesses. + let a = vqtbl1q_u8(p[C], vdupq_n_u8((i - 16 * C) as u8)); + let b = match C { + 0 => vqtbl1q_u8(p[0], jv), + 1 => vqtbl2q_u8(uint8x16x2_t(p[0], p[1]), jv), + 2 => vqtbl3q_u8(uint8x16x3_t(p[0], p[1], p[2]), jv), + _ => vqtbl4q_u8(uint8x16x4_t(p[0], p[1], p[2], p[3]), jv), + }; + // perm[j] = a, then perm[i] = b. + for (chunk, idx) in p.iter_mut().zip(index).take(C + 1) { + *chunk = vbslq_u8(vceqq_u8(*idx, jv), a, *chunk); + } + p[C] = vbslq_u8(vceqq_u8(index[C], iv), b, p[C]); + // inverse = s ∘ inverse. + for chunk in v.iter_mut() { + *chunk = swap_values(*chunk, iv, jv); + } + } + } + + /// # Safety + /// + /// Requires NEON (baseline on aarch64) and `stream.len() >= + /// STREAM_BYTES` (asserted by the caller). The only memory operations + /// are the load of the constant `INDEX`, the bounds-checked stream reads + /// in `lemire_draw`, and the two 64-byte stores into `perm` and + /// `inverse`, which are exclusive references to 64-byte arrays. Both + /// tables live in vector registers in between: every `p[k]` / `v[k]` + /// index is a compile-time constant after inlining. + #[target_feature(enable = "neon")] + pub(super) unsafe fn build(stream: &[u8], perm: &mut [u8; DOMAIN], inverse: &mut [u8; DOMAIN]) { + let ix = vld1q_u8_x4(INDEX.as_ptr()); + let index = [ix.0, ix.1, ix.2, ix.3]; + let mut p = index; + let mut v = index; + + steps::<3>(stream, &mut p, &mut v, &index); + steps::<2>(stream, &mut p, &mut v, &index); + steps::<1>(stream, &mut p, &mut v, &index); + steps::<0>(stream, &mut p, &mut v, &index); + + vst1q_u8_x4(perm.as_mut_ptr(), uint8x16x4_t(p[0], p[1], p[2], p[3])); + vst1q_u8_x4(inverse.as_mut_ptr(), uint8x16x4_t(v[0], v[1], v[2], v[3])); + } +} + +#[cfg(target_arch = "x86_64")] +mod ssse3 { + use super::{lemire_draw, DOMAIN}; + use core::arch::x86_64::*; + + /// `select(m, x, y)`: `x` where the mask lane is all ones, else `y`. + #[inline(always)] + unsafe fn sel(m: __m128i, x: __m128i, y: __m128i) -> __m128i { + _mm_or_si128(_mm_and_si128(m, x), _mm_andnot_si128(m, y)) + } + + /// Broadcast `table[k]` to every lane, `kv` holding `k` in every lane: + /// `pshufb` each 16-byte chunk by `k & 15` and keep the chunk `k >> 4` + /// by mask. Only the chunks in `t` are searched. + #[inline(always)] + unsafe fn fetch(t: &[__m128i], kv: __m128i) -> __m128i { + let lo = _mm_and_si128(kv, _mm_set1_epi8(15)); + let hi = _mm_and_si128(_mm_srli_epi16(kv, 4), _mm_set1_epi8(15)); + let mut r = _mm_setzero_si128(); + for (c, chunk) in t.iter().enumerate() { + let m = _mm_cmpeq_epi8(hi, _mm_set1_epi8(c as i8)); + r = _mm_or_si128(r, _mm_and_si128(m, _mm_shuffle_epi8(*chunk, lo))); + } + r + } + + /// Exchange the lane values `x` and `y` wherever they occur in `t`. + #[inline(always)] + unsafe fn swap_values(t: __m128i, x: __m128i, y: __m128i) -> __m128i { + let m = _mm_or_si128(_mm_cmpeq_epi8(t, x), _mm_cmpeq_epi8(t, y)); + _mm_xor_si128(t, _mm_and_si128(m, _mm_xor_si128(x, y))) + } + + /// The steps whose `i` lies in chunk `C`; the same shape as the NEON + /// builder's `steps` (public chunk bounds, value-side inverse). + #[inline(always)] + unsafe fn steps( + stream: &[u8], + p: &mut [__m128i; 4], + v: &mut [__m128i; 4], + index: &[__m128i; 4], + ) { + let low = if C == 0 { 1 } else { 16 * C }; + for i in (low..16 * (C + 1)).rev() { + let iv = _mm_set1_epi8(i as i8); + let jv = _mm_set1_epi8(lemire_draw(stream, i) as i8); + let a = _mm_shuffle_epi8(p[C], _mm_set1_epi8((i - 16 * C) as i8)); + let b = fetch(&p[..=C], jv); + // perm[j] = a, then perm[i] = b. + for (chunk, idx) in p.iter_mut().zip(index).take(C + 1) { + *chunk = sel(_mm_cmpeq_epi8(*idx, jv), a, *chunk); + } + p[C] = sel(_mm_cmpeq_epi8(index[C], iv), b, p[C]); + // inverse = s ∘ inverse. + for chunk in v.iter_mut() { + *chunk = swap_values(*chunk, iv, jv); + } + } + } + + /// # Safety + /// + /// Requires SSSE3 (checked by the caller) and `stream.len() >= + /// STREAM_BYTES` (asserted by the caller). Unaligned stores cover + /// exactly the 64-byte `perm` and `inverse` arrays. + #[target_feature(enable = "ssse3")] + pub(super) unsafe fn build(stream: &[u8], perm: &mut [u8; DOMAIN], inverse: &mut [u8; DOMAIN]) { + let mut index = [_mm_setzero_si128(); 4]; + for (c, chunk) in index.iter_mut().enumerate() { + *chunk = _mm_add_epi8( + _mm_set1_epi8((c * 16) as i8), + _mm_setr_epi8(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15), + ); + } + let mut p = index; + let mut v = index; + + steps::<3>(stream, &mut p, &mut v, &index); + steps::<2>(stream, &mut p, &mut v, &index); + steps::<1>(stream, &mut p, &mut v, &index); + steps::<0>(stream, &mut p, &mut v, &index); + + for (c, (pc, vc)) in p.iter().zip(v.iter()).enumerate() { + _mm_storeu_si128(perm.as_mut_ptr().add(c * 16) as *mut __m128i, *pc); + _mm_storeu_si128(inverse.as_mut_ptr().add(c * 16) as *mut __m128i, *vc); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + type Tables = ([u8; DOMAIN], [u8; DOMAIN]); + + fn run(builder: fn(&[u8], &mut [u8; DOMAIN], &mut [u8; DOMAIN]), stream: &[u8]) -> Tables { + let mut perm = [0u8; DOMAIN]; + let mut inverse = [0u8; DOMAIN]; + builder(stream, &mut perm, &mut inverse); + (perm, inverse) + } + + /// Spread an arbitrary-length seed over a full stream so every draw is + /// exercised, not only the prefix quickcheck happens to fill. + fn stream_from(seed: &[u8]) -> [u8; STREAM_BYTES] { + let mut s = [0u8; STREAM_BYTES]; + let mut rng = ::seed_from_u64( + seed.iter() + .fold(0u64, |h, &b| h.wrapping_mul(131).wrapping_add(u64::from(b))), + ); + rand::RngCore::fill_bytes(&mut rng, &mut s); + for (d, b) in s.iter_mut().zip(seed) { + *d ^= *b; + } + s + } + + quickcheck! { + /// The dispatched builder (NEON on aarch64, SSSE3 or SWAR on + /// x86_64, SWAR elsewhere) builds the reference tables. + fn dispatched_matches_reference(seed: Vec) -> bool { + let s = stream_from(&seed); + run(build, &s) == run(reference::build, &s) + } + + /// The portable SWAR builder builds the reference tables on every + /// target, not only where it is dispatched. + fn swar_matches_reference(seed: Vec) -> bool { + let s = stream_from(&seed); + run(swar::build, &s) == run(reference::build, &s) + } + + /// So does the byte-at-a-time `subtle_ng` builder. + fn scalar_matches_reference(seed: Vec) -> bool { + let s = stream_from(&seed); + run(scalar::build, &s) == run(reference::build, &s) + } + } + + /// Edge streams: all-zero draws (every `j = 0`), all-ones draws (every + /// `j = i`, the identity swaps), and alternating patterns. + #[test] + fn builders_match_reference_on_edge_streams() { + for fill in [0x00u8, 0xff, 0x55, 0xaa, 0x80] { + let s = [fill; STREAM_BYTES]; + let want = run(reference::build, &s); + assert_eq!(run(build, &s), want, "dispatched, fill {fill:#04x}"); + assert_eq!(run(swar::build, &s), want, "swar, fill {fill:#04x}"); + assert_eq!(run(scalar::build, &s), want, "scalar, fill {fill:#04x}"); + } + } + + /// The reference tables are a permutation of `0..64` and its inverse, + /// so matching them means every builder builds one. + #[test] + fn reference_builds_a_permutation_and_its_inverse() { + let (perm, inverse) = run(reference::build, &stream_from(b"reference")); + for (k, p) in perm.iter().enumerate() { + assert_eq!(usize::from(inverse[usize::from(*p)]), k); + } + } + + /// One byte short of a full stream is refused. + #[test] + #[should_panic] + fn build_rejects_a_short_stream() { + run(build, &[0u8; STREAM_BYTES - 1]); + } +} diff --git a/packages/ore-rs/src/primitives/simd.rs b/packages/ore-rs/src/primitives/simd.rs index 33bbfc9b..8e9ee046 100644 --- a/packages/ore-rs/src/primitives/simd.rs +++ b/packages/ore-rs/src/primitives/simd.rs @@ -44,22 +44,41 @@ pub(crate) fn gt_mask_xor_256(table: &[u8; 256], x: u8, out: &mut [u8]) { #[cfg(target_arch = "aarch64")] // SAFETY: NEON is baseline on aarch64; `out` length asserted above. unsafe { - neon::gt_mask_xor_256(table, x, out); + neon::gt_mask_xor_256(table, x, out) } - #[cfg(target_arch = "x86_64")] - if is_x86_feature_detected!("avx2") { - // SAFETY: AVX2 presence just checked; `out` length asserted above. - unsafe { - avx2::gt_mask_xor_256(table, x, out); + // Excluded on aarch64, where the NEON path above always handles it, so + // the fallback is never unreachable code needing an `allow`. + #[cfg(not(target_arch = "aarch64"))] + { + #[cfg(target_arch = "x86_64")] + if is_x86_feature_detected!("avx2") { + // SAFETY: AVX2 presence just checked; `out` length asserted above. + unsafe { avx2::gt_mask_xor_256(table, x, out) }; return; } + + scalar::gt_mask_xor(table, x, out); + } +} + +/// 64-lane variant of [`gt_mask_xor_256`] for the Bit6 block domain. +/// AVX2 gains little at this size (two 32-lane compares vs the scalar +/// loop's ~9 ns); only NEON is dispatched. +#[inline] +pub(crate) fn gt_mask_xor_64(table: &[u8; 64], x: u8, out: &mut [u8]) { + // A real assert, as for `gt_mask_xor_256`: the same contract on every + // backend in release. + assert_eq!(out.len(), 8); + + #[cfg(target_arch = "aarch64")] + // SAFETY: NEON is baseline on aarch64; `out` length asserted above. + unsafe { + neon::gt_mask_xor_64(table, x, out) } - // Scalar fallback; excluded on aarch64 (the NEON path above always handles - // it) so the dead tail needs no blanket `#[allow(unreachable_code)]`. #[cfg(not(target_arch = "aarch64"))] - scalar::gt_mask_xor_256(table, x, out); + scalar::gt_mask_xor(table, x, out); } /// Pack the LSB of byte 0 of each of 256 AES blocks into `out`: @@ -76,11 +95,9 @@ pub(crate) fn lsb_mask_256(blocks: &[AesBlock], out: &mut [u8]) { #[cfg(target_arch = "aarch64")] // SAFETY: NEON is baseline on aarch64; lengths asserted above. unsafe { - neon::lsb_mask_256(blocks, out); + neon::lsb_mask_256(blocks, out) } - // Scalar fallback; excluded on aarch64 (the NEON path above always handles - // it) so the dead tail needs no blanket `#[allow(unreachable_code)]`. #[cfg(not(target_arch = "aarch64"))] scalar::lsb_mask(blocks, out); } @@ -88,13 +105,16 @@ pub(crate) fn lsb_mask_256(blocks: &[AesBlock], out: &mut [u8]) { pub(crate) mod scalar { use super::AesBlock; - // Used by the non-aarch64 `gt_mask_xor_256` dispatcher and as the test - // oracle; on aarch64 the lib always reaches NEON, so outside `cfg(test)` - // this fn is unreferenced there. Compiling it exactly where it's used keeps - // it from being dead code without an `allow`. (`lsb_mask` below stays - // always compiled — `hash.rs` calls it directly for non-256 inputs.) + /// Length-generic scalar indicator pack: `table.len()` must be a + /// multiple of 8 and equal to `out.len() * 8`. + /// + /// Used by the non-aarch64 dispatchers and as the test oracle; on + /// aarch64 the lib always reaches NEON, so outside `cfg(test)` it is + /// unreferenced there. Compiling it exactly where it is used keeps it + /// from being dead code without an `allow`. (`lsb_mask` below stays + /// always compiled: `hash.rs` calls it directly for non-256 inputs.) #[cfg(any(not(target_arch = "aarch64"), test))] - pub(crate) fn gt_mask_xor_256(table: &[u8; 256], x: u8, out: &mut [u8]) { + pub(crate) fn gt_mask_xor(table: &[u8], x: u8, out: &mut [u8]) { for (slot, chunk) in out.iter_mut().zip(table.chunks_exact(8)) { let mut byte = 0u8; for (bit, &p) in chunk.iter().enumerate() { @@ -150,6 +170,17 @@ mod neon { } } + #[target_feature(enable = "neon")] + pub(super) unsafe fn gt_mask_xor_64(table: &[u8; 64], x: u8, out: &mut [u8]) { + let xv = vdupq_n_u8(x); + for i in 0..4 { + let t = vld1q_u8(table.as_ptr().add(i * 16)); + let (lo, hi) = pack_addv(vcgtq_u8(t, xv)); + out[2 * i] ^= lo; + out[2 * i + 1] ^= hi; + } + } + /// `IDX[g]` places the 4 block-start bytes of 64-byte group `g` /// (offsets 0, 16, 32, 48) at lanes 4g..4g+3; all other lanes are 0xFF /// (out-of-range, yielding 0 in `tbl`). @@ -238,7 +269,26 @@ mod tests { let mut b = a; gt_mask_xor_256(&table, x, &mut a); - scalar::gt_mask_xor_256(&table, x, &mut b); + scalar::gt_mask_xor(&table, x, &mut b); + assert_eq!(a, b); + } + } + + #[test] + fn gt_mask_64_dispatched_matches_scalar() { + let mut rng = ChaCha20Rng::seed_from_u64(13); + for _ in 0..5_000 { + let mut table = [0u8; 64]; + rng.fill(&mut table[..]); + // Block values and table entries are < 64 in real use, but the + // kernels must agree on all byte values. + let x: u8 = rng.gen(); + let mut a = [0u8; 8]; + rng.fill(&mut a[..]); + let mut b = a; + + gt_mask_xor_64(&table, x, &mut a); + scalar::gt_mask_xor(&table, x, &mut b); assert_eq!(a, b); } } diff --git a/packages/ore-rs/src/primitives/stack.rs b/packages/ore-rs/src/primitives/stack.rs new file mode 100644 index 00000000..5dc64117 --- /dev/null +++ b/packages/ore-rs/src/primitives/stack.rs @@ -0,0 +1,155 @@ +//! Stack scrubbing for the encrypt entry points. +//! +//! Wiping a value on drop does not wipe the copies the compiler made of it. +//! Building a PRP leaves its seed-keyed AES schedule (round key 0 is the +//! seed) and its permutation tables in stack slots that moves and returns +//! left behind, and `ZeroizeOnDrop` only reaches the final location. Those +//! slots outlive the call, until something else overwrites them. +//! +//! [`scrubbed`] runs the encryption in a frame of its own and then zeroes +//! [`SCRUB_BYTES`] of the stack below the caller's frame, which covers every +//! frame the encryption used. Each scheme's residue tests measure the depth +//! its encrypt paths reach, check that the scrub covers it, and search the +//! dead stack for the secrets. + +/// Bytes of stack zeroed after each encryption. On aarch64 the Bit6 encrypt +/// paths reach about 49 KiB at `opt-level = 0` and at most about 6.5 KiB at +/// every other level (1, 2, 3, `s`, `z`), with or without debug assertions. +/// The size follows the optimisation level (set by `build.rs`), not +/// `debug_assertions`, which a profile can turn off without optimising. The +/// residue tests in each scheme measure the depth and fail if a change takes +/// it past this. +pub(crate) const SCRUB_BYTES: usize = if cfg!(ore_unoptimized) { + 96 * 1024 +} else { + 16 * 1024 +}; + +/// Run `f` in its own stack frame, then zero the stack it used. +/// +/// The work must happen inside `f`. Secrets the caller holds in its own +/// frame are above the scrubbed region and are not wiped. +#[inline(always)] +pub(crate) fn scrubbed(f: impl FnOnce() -> T) -> T { + let out = isolate(f); + scrub(); + out +} + +/// Not inlined, so that `f` runs below the caller's frame. +#[inline(never)] +fn isolate(f: impl FnOnce() -> T) -> T { + f() +} + +/// Zero `SCRUB_BYTES` of stack below the caller's frame. +/// +/// One `memset` of the whole buffer: per-word volatile stores (`Zeroize`) +/// added 5–12% to a Bit6 encryption on an M1 Max; this adds about 1%. `black_box` takes the buffer's +/// address after the stores, so the compiler must assume it reads them and +/// cannot drop them as dead. +#[inline(never)] +fn scrub() { + let mut buf = core::mem::MaybeUninit::<[u8; SCRUB_BYTES]>::uninit(); + // SAFETY: writes `SCRUB_BYTES` bytes into a buffer of that size. + unsafe { core::ptr::write_bytes(buf.as_mut_ptr() as *mut u8, 0, SCRUB_BYTES) }; + core::hint::black_box(&mut buf); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); +} + +/// Helpers for tests that look for secrets left in dead stack. +#[cfg(test)] +pub(crate) mod residue { + use std::thread; + + /// Bytes of dead stack [`after`] copies and searches. + pub(crate) const SCAN_BYTES: usize = super::SCRUB_BYTES + 64 * 1024; + + /// Fill the stack below the caller with a marker byte, so that the depth + /// a later call reaches can be read from what it overwrote. + #[inline(never)] + fn paint() { + let mut buf = [0xa5u8; SCAN_BYTES + 4096]; + core::hint::black_box(&mut buf); + } + + /// Copy the `SCAN_BYTES` of stack below this frame into `out`, using + /// only volatile reads, so that the copy calls no function that would + /// overwrite the region first. + #[inline(never)] + fn snapshot(out: &mut [u8]) { + let marker = 0u8; + let top = core::hint::black_box(&marker as *const u8); + // Start a little below `marker`, clear of this frame's own slots. + let start = top.wrapping_sub(256 + out.len()); + for (i, slot) in out.iter_mut().enumerate() { + // SAFETY: the region is inside this thread's stack (which + // `after` sizes well beyond `SCAN_BYTES`), below the live + // frames, and mapped. It is read as plain bytes. + *slot = unsafe { core::ptr::read_volatile(start.wrapping_add(i)) }; + } + } + + /// The dead stack left by `f`, run on a fresh thread: `SCAN_BYTES`, + /// ordered from the deepest address up. + pub(crate) fn after(f: F) -> Vec { + thread::Builder::new() + .stack_size(4 * 1024 * 1024) + .spawn(move || { + let mut out = vec![0u8; SCAN_BYTES]; + paint(); + run(f); + snapshot(&mut out); + out + }) + .unwrap() + .join() + .unwrap() + } + + #[inline(never)] + fn run(f: F) { + f() + } + + /// How far below the caller `f` wrote: bytes from the deepest address + /// that no longer holds the paint marker to the top of the scan. + pub(crate) fn depth(dead: &[u8]) -> usize { + dead.windows(16) + .position(|w| w.iter().any(|&b| b != 0xa5)) + .map_or(0, |deepest| dead.len() - deepest) + } + + /// Whether `needle` occurs anywhere in `dead`. + pub(crate) fn contains(dead: &[u8], needle: &[u8]) -> bool { + dead.windows(needle.len()).any(|w| w == needle) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn scrub_zeroes_the_region_a_call_used() { + let dead = residue::after(|| { + scrubbed(|| { + let mut secret = [0x5cu8; 4096]; + core::hint::black_box(&mut secret); + }) + }); + assert!(!residue::contains(&dead, &[0x5cu8; 64])); + } + + #[test] + fn unscrubbed_residue_is_visible() { + // Positive control for the scan: without the scrub the secret stays. + let dead = residue::after(|| { + isolate(|| { + let mut secret = [0x5cu8; 4096]; + core::hint::black_box(&mut secret); + }) + }); + assert!(residue::contains(&dead, &[0x5cu8; 64])); + } +} diff --git a/packages/ore-rs/src/scheme.rs b/packages/ore-rs/src/scheme.rs index 33198b49..25df7c0a 100644 --- a/packages/ore-rs/src/scheme.rs +++ b/packages/ore-rs/src/scheme.rs @@ -7,5 +7,10 @@ /// 2-bit-indicator BlockORE with AES-128 PRF and Knuth-shuffle PRP. pub mod bit2; +/// 6-bit-block variant of [`bit2`]: 4x less AES work per block and 4x +/// smaller right blocks. Wire format v2 (headered). Not yet frozen — +/// pending crypto review of the Z2 hash (v2 plan, section 6). +pub mod bit2_w6; + pub(crate) mod decompose; pub(crate) mod width; diff --git a/packages/ore-rs/src/scheme/bit2.rs b/packages/ore-rs/src/scheme/bit2.rs index e599dc16..a0419f3d 100644 --- a/packages/ore-rs/src/scheme/bit2.rs +++ b/packages/ore-rs/src/scheme/bit2.rs @@ -30,7 +30,7 @@ pub use self::block_types::*; /// nonces. The two PRF instances are keyed at construction; the RNG is held /// in a `RefCell` so encryption can take `&self` while still drawing fresh /// randomness. Keys are zeroised on drop. -#[derive(Debug, ZeroizeOnDrop)] +#[derive(ZeroizeOnDrop)] pub struct OreAes128 { prf1: Aes128Prf, prf2: Aes128Prf, @@ -38,6 +38,14 @@ pub struct OreAes128 { rng: RefCell, } +// Opaque Debug: never render key material. (`Aes128`'s own Debug is already +// opaque, but spell it out so the guarantee can't regress.) +impl std::fmt::Debug for OreAes128 { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("OreAes128").finish_non_exhaustive() + } +} + /// Convenience alias for [`OreAes128`] backed by `ChaCha20Rng` — the RNG /// most callers will want. pub type OreAes128ChaCha20 = OreAes128; @@ -86,11 +94,11 @@ fn derive_prp_seeds(prf2: &Aes128Prf, x: &PlainText) -> SeedB /// one linear pass of its inverse table — no per-bit `invert` lookups, no /// heap allocation. `ro_blocks` holds the PRF₁-encrypted RO keys and is /// trashed by the hash pass. -fn encode_right_block( +pub(crate) fn encode_right_block( block: &mut W::RightBlock, prp: &W::Prp, x: u8, - hasher: &Aes128Z2Hash, + hasher: &H, ro_blocks: &mut [AesBlock], ) { debug_assert_eq!(ro_blocks.len(), W::DOMAIN); @@ -204,7 +212,7 @@ impl OreCipher for OreAes128 { work.copy_from(&template); self.prf1.encrypt_all(work.as_mut_slice()); - encode_right_block::(&mut right.data[n], &prp, x[n], &hasher, &mut work); + encode_right_block::(&mut right.data[n], &prp, x[n], &hasher, &mut work); } self.prf1.encrypt_all(&mut left.f); @@ -285,11 +293,10 @@ fn right_block(input: &[u8], n: usize) -> &[u8] { fn get_bit(block: &[u8], bit: usize) -> u8 { debug_assert!(block.len() == RightBlock32::BLOCK_SIZE); debug_assert!(bit < 256); - let byte_index = bit / 8; - let position = bit % 8; - let v = 1 << position; - - (block[byte_index] & v) >> position + // `bit` is the secret permuted symbol; read the byte obliviously so the + // access address does not depend on it. See `width::ct_select_byte`. + let byte = crate::scheme::width::ct_select_byte(block, bit / 8); + crate::scheme::width::ct_bit(byte, (bit % 8) as u8) } impl PartialEq for CipherText { diff --git a/packages/ore-rs/src/scheme/bit2/block_types.rs b/packages/ore-rs/src/scheme/bit2/block_types.rs index 308dbc22..6012f452 100644 --- a/packages/ore-rs/src/scheme/bit2/block_types.rs +++ b/packages/ore-rs/src/scheme/bit2/block_types.rs @@ -40,11 +40,10 @@ impl RightBlock32 { #[inline] pub fn get_bit(&self, bit: usize) -> u8 { debug_assert!(bit < 256); - let byte_index = bit / 8; - let position = bit % 8; - let v = 1 << position; - - (self.data[byte_index] & v) >> position + // `bit` is the secret permuted symbol; read the byte obliviously so the + // access address does not depend on it. See `width::ct_select_byte`. + let byte = crate::scheme::width::ct_select_byte(&self.data, bit / 8); + crate::scheme::width::ct_bit(byte, (bit % 8) as u8) } } diff --git a/packages/ore-rs/src/scheme/bit2_w6.rs b/packages/ore-rs/src/scheme/bit2_w6.rs new file mode 100644 index 00000000..80b8207d --- /dev/null +++ b/packages/ore-rs/src/scheme/bit2_w6.rs @@ -0,0 +1,793 @@ +//! BlockORE with **6-bit input blocks** (domain 64): the `(Bit6, packed +//! prefix, fixed-N)` scheme from the ORE v2 plan. +//! +//! Plaintext bytes are decomposed MSB-first into 6-bit block values (see +//! [`crate::scheme::decompose`]); `N` in the types below counts *blocks*, +//! not bytes (a `u64` is 11 blocks). Compared to the legacy 8-bit scheme, +//! each block costs 64 random-oracle evaluations instead of 256 and stores +//! an 8-byte right block instead of 32 — a `u64` ciphertext is 295 bytes +//! (vs 408) including the 4-byte v2 wire header. +//! +//! The packed prefix caps `N` at 14 (`prefix ≤ 13 ‖ value ‖ index` plus +//! the block count in byte 15 must fit one AES block; seeds also use byte +//! 14 for the block index), which covers all +//! primitives up to 64 bits. `Decimal` stays on the legacy scheme until the +//! chained-prefix construction lands (plan §5). `u128`/`i128` fit neither: +//! the legacy scheme takes at most 15 bytes and panics on 16. +//! +//! **Status: wire format NOT yet frozen.** The Z2 hash is the fixed-π MMO +//! construction proposed in plan §6 (option 3), pending crypto review; +//! flipping [`Z2Hash`] re-keys the right ciphertexts without any other +//! code change. Do not store ciphertexts produced by this scheme until +//! the review lands and vectors are pinned. +//! +//! **Keys:** PRF₁ and PRF₂ run under keys derived from the caller's +//! (see `PRF1_LABEL`), never the raw keys, so sharing keys with the +//! legacy scheme does not let legacy ciphertexts reveal Bit6 masking keys. + +use crate::{ + ciphertext::*, + primitives::{ + hash::FixedPiZ2Hash, prf::Aes128Prf, stack, AesBlock, Hash, HashKey, Prf, Prp, NONCE_SIZE, + }, + scheme::width::{ct_assign_bytes, AesBlockBuf, Bit6, BlockWidth}, + OreCipher, OreError, PlainText, +}; + +use aes::cipher::generic_array::GenericArray; +use rand::{Rng, SeedableRng}; +use rand_chacha::ChaCha20Rng; +use std::cell::RefCell; +use std::cmp::Ordering; +use subtle_ng::{Choice, ConditionallySelectable, ConstantTimeEq}; +use zeroize::{Zeroize, ZeroizeOnDrop}; + +/// Per-block ciphertext component types for this scheme. +pub mod block_types; +pub use self::block_types::*; + +/// The Z2 random-oracle instantiation for this scheme (plan §6, option 3 — +/// pending review; option 2 fallback is `Aes128Z2Hash` with an extra +/// feedforward, and the legacy construction is `Aes128Z2Hash`). +type Z2Hash = FixedPiZ2Hash; + +/// Maximum number of blocks: the packed prefix (`prefix ‖ value ‖ index`) +/// plus the block count at byte 15 must fit one 16-byte AES input. +pub const MAX_BLOCKS: usize = 14; + +/// AES-128 BlockORE cipher over 6-bit blocks, generic over the RNG used +/// for per-encryption nonces. Keys are zeroised on drop. +#[derive(ZeroizeOnDrop)] +pub struct OreAes128Bit6 { + prf1: Aes128Prf, + prf2: Aes128Prf, + #[zeroize(skip)] + rng: RefCell, +} + +// Opaque Debug: never render key material (OpaqueDebug discipline). +impl std::fmt::Debug for OreAes128Bit6 { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("OreAes128Bit6").finish_non_exhaustive() + } +} + +/// Convenience alias for [`OreAes128Bit6`] backed by `ChaCha20Rng`. +pub type OreAes128Bit6ChaCha20 = OreAes128Bit6; + +type EncryptLeftResult = Result, N>, OreError>; +type EncryptResult = Result, N>, OreError>; + +/// v2 wire header for this scheme: version 2, scheme id 0x02 +/// (= AES-128 suite, 6-bit blocks, packed prefix). +const WIRE: WireHeader = WireHeader { + version: 0x02, + scheme_id: 0x02, +}; + +/// PRP seeds are key-equivalent material; see the bit2 sibling for the +/// full rationale. Zeroized on drop. +struct SeedBuf([AesBlock; N]); + +impl Drop for SeedBuf { + fn drop(&mut self) { + for seed in self.0.iter_mut() { + seed.zeroize(); + } + } +} + +/// Labels for deriving this scheme's PRF keys from the caller's keys: +/// `k1' = AES_{k1}(PRF1_LABEL)` and `k2' = AES_{k2}(PRF2_LABEL)`. +/// +/// Bit6 must not evaluate PRF₁/PRF₂ under the caller's raw keys, because +/// the legacy scheme does, and its inputs cover Bit6's: a legacy `N = 15` +/// left ciphertext publishes exactly the PRF₁ outputs Bit6 uses as its +/// masking keys. Deriving separate keys separates the schemes. +/// +/// The labels themselves must never be a legacy PRF input, or a legacy +/// ciphertext would publish the derived key. Every legacy PRF₁ input has +/// byte 15 in `0..=14` (the block index at `N = 15`, else `0`) and every +/// legacy PRF₂ input has byte 15 `= 0`; these labels end in ASCII `'1'` +/// and `'2'` (`0x31`, `0x32`), so they are outside both. +const PRF1_LABEL: [u8; 16] = *b"ORE.v2.bit6.prf1"; +const PRF2_LABEL: [u8; 16] = *b"ORE.v2.bit6.prf2"; + +/// `Aes128Prf` keyed by `AES_k(label)`, wiping the derived key bytes. +fn derive_prf(k: &[u8; 16], label: &[u8; 16]) -> Aes128Prf { + let kdf: Aes128Prf = Prf::new(GenericArray::from_slice(k)); + let mut derived = [AesBlock::clone_from_slice(label)]; + kdf.encrypt_all(&mut derived); + let prf = Prf::new(&derived[0]); + derived[0].as_mut_slice().zeroize(); + prf +} + +impl OreAes128Bit6 { + /// Per-block PRP seeds: `PRF₂(x[0..n] ‖ 0… ‖ n@14 ‖ N@15)`. The block + /// count in byte 15 separates plaintext shapes (plan §4); the block + /// index in byte 14 separates positions, so a prefix padded with zero + /// blocks no longer collides with a shorter prefix (which gave every + /// position of an all-zero plaintext the same permutation). Prefixes + /// equal up to block `n` still share `seed_n`, as the comparator needs. + /// `N ≤ 14` puts the prefix in bytes `0..=12`, so byte 14 is free. + fn derive_prp_seeds(&self, x: &PlainText) -> SeedBuf { + let mut seeds = [AesBlock::default(); N]; + for (n, block) in seeds.iter_mut().enumerate() { + block[0..n].clone_from_slice(&x[0..n]); + block[14] = n as u8; + block[15] = N as u8; + } + self.prf2.encrypt_all(&mut seeds); + SeedBuf(seeds) + } +} + +/// The encrypt paths without the stack scrub. Every caller runs them inside +/// [`stack::scrubbed`]: building a block's PRP leaves its seed and tables in +/// dead stack (see `primitives::stack`). +impl OreAes128Bit6 { + pub(crate) fn encrypt_left_unscrubbed( + &self, + x: &PlainText, + ) -> EncryptLeftResult { + assert!(N <= MAX_BLOCKS); + debug_assert!(x + .iter() + .all(|&b| (b as usize) < ::DOMAIN)); + + let mut output = Left::::init(); + let seeds = self.derive_prp_seeds(x); + + for n in 0..N { + let prp: ::Prp = Prp::new(&seeds.0[n])?; + output.xt[n] = prp.permute(x[n])?; + + output.f[n][0..n].clone_from_slice(&x[0..n]); + output.f[n][n] = output.xt[n]; + output.f[n][N] = n as u8; + output.f[n][15] = N as u8; + } + self.prf1.encrypt_all(&mut output.f); + + Ok(output) + } + + pub(crate) fn encrypt_unscrubbed( + &self, + x: &PlainText, + ) -> EncryptResult { + assert!(N <= MAX_BLOCKS); + debug_assert!(x + .iter() + .all(|&b| (b as usize) < ::DOMAIN)); + + let mut left = Left::::init(); + let mut right = Right::::init(); + + self.rng.borrow_mut().try_fill(&mut right.nonce)?; + + let seeds = self.derive_prp_seeds(x); + let hasher: Z2Hash = Hash::new(HashKey::from_slice(&right.nonce)); + + // RO key template, maintained incrementally (see the bit2 sibling). + // Entry j for block n is (x[0..n] ‖ j ‖ 0… ‖ n@N ‖ N@15). + let mut template = ::RoKeyBuf::zeroed(); + for (j, entry) in template.iter_mut().enumerate() { + entry[0] = j as u8; + entry[15] = N as u8; + } + let mut work = ::RoKeyBuf::zeroed(); + + for n in 0..N { + let prp: ::Prp = Prp::new(&seeds.0[n])?; + left.xt[n] = prp.permute(x[n])?; + + left.f[n][0..n].clone_from_slice(&x[0..n]); + left.f[n][n] = left.xt[n]; + left.f[n][N] = n as u8; + left.f[n][15] = N as u8; + + if n > 0 { + for (j, entry) in template.iter_mut().enumerate() { + entry[n - 1] = x[n - 1]; + entry[n] = j as u8; + entry[N] = n as u8; + } + } + + work.copy_from(&template); + self.prf1.encrypt_all(work.as_mut_slice()); + + crate::scheme::bit2::encode_right_block::( + &mut right.data[n], + &prp, + x[n], + &hasher, + &mut work, + ); + } + + self.prf1.encrypt_all(&mut left.f); + + for entry in template.iter_mut() { + entry.zeroize(); + } + for entry in work.as_mut_slice() { + entry.zeroize(); + } + + Ok(CipherText { left, right }) + } +} + +// Right-block encoding is the width-/hash-generic helper shared with the +// legacy scheme: `crate::scheme::bit2::encode_right_block::`. + +impl OreCipher for OreAes128Bit6 { + type LeftBlockType = LeftBlock16; + type RightBlockType = RightBlock8; + + const WIRE_HEADER: Option = Some(WIRE); + const SYMBOL_DOMAIN: usize = ::DOMAIN; + + fn init(k1: &[u8; 16], k2: &[u8; 16]) -> Result { + let rng: R = SeedableRng::from_entropy(); + + Ok(OreAes128Bit6 { + prf1: derive_prf(k1, &PRF1_LABEL), + prf2: derive_prf(k2, &PRF2_LABEL), + rng: RefCell::new(rng), + }) + } + + /// Encrypt `x`, whose entries are **6-bit block values** (`< 64`, + /// produced by [`crate::scheme::decompose`]); `N` is the block count + /// (≤ [`MAX_BLOCKS`]). The [`crate::OreEncrypt`] impls handle the + /// byte→block decomposition for primitive types. + fn encrypt_left(&self, x: &PlainText) -> EncryptLeftResult { + stack::scrubbed(|| self.encrypt_left_unscrubbed(x)) + } + + fn encrypt(&self, x: &PlainText) -> EncryptResult { + stack::scrubbed(|| self.encrypt_unscrubbed(x)) + } + + fn compare_raw_slices(a: &[u8], b: &[u8]) -> Option { + if a.len() != b.len() { + return None; + } + let (header_a, a) = parse_header(a).ok()?; + let (header_b, b) = parse_header(b).ok()?; + if header_a != header_b || (header_a.0, header_a.1) != (WIRE.version, WIRE.scheme_id) { + return None; + } + let num_blocks = header_a.2; + // Reject a degenerate count=0 header: no OreEncrypt path produces zero + // blocks, and an empty scan would otherwise return Equal for any pair + // of crafted 0-block ciphertexts. + if num_blocks == 0 || num_blocks > MAX_BLOCKS { + return None; + } + + let left_size = Self::LeftBlockType::BLOCK_SIZE; + let right_size = Self::RightBlockType::BLOCK_SIZE; + if a.len() != num_blocks * (left_size + 1 + right_size) + NONCE_SIZE { + return None; + } + // Reject symbols outside the 64-element domain: the scan below reads + // the right block at `a[l]`, which must index one of its 64 bits. + // `xt` is public, so this may branch. + if a[..num_blocks] + .iter() + .chain(&b[..num_blocks]) + .any(|&s| usize::from(s) >= ::DOMAIN) + { + return None; + } + + let mut is_equal = Choice::from(1); + // What the resolution step needs from the first differing block `l`: + // `a`'s permuted symbol and PRF tag, and `b`'s right bitvector. All + // three are latched *inside* the scan, under the choice "this is the + // first difference", so no load after the loop is indexed by `l`. + // The right blocks are the one region the scan would otherwise never + // touch, so a direct `right[l]` afterwards hits or misses the cache + // according to `l`; measured as a timing signal, see `docs/reviews/`. + let mut sel_xt = 0u8; + let mut sel_f = [0u8; LeftBlock16::BLOCK_SIZE]; + let mut sel_right = [0u8; RightBlock8::BLOCK_SIZE]; + + // Slices for the PRF ("f") blocks and the right half. + let a_f = &a[num_blocks..]; + let b_f = &b[num_blocks..]; + let b_right = &b[num_blocks * (left_size + 1)..]; + let b_right_blocks = &b_right[NONCE_SIZE..]; + + for n in 0..num_blocks { + let prp_eq: Choice = !a[n].ct_eq(&b[n]); + let left_block_comparison: Choice = !left_block(a_f, n).ct_eq(left_block(b_f, n)); + let condition: Choice = prp_eq | left_block_comparison; + // Set for exactly one `n`: the first differing block. + let first = is_equal & condition; + + sel_xt.conditional_assign(&a[n], first); + ct_assign_bytes(&mut sel_f, left_block(a_f, n), first); + ct_assign_bytes(&mut sel_right, right_block(b_right_blocks, n), first); + is_equal.conditional_assign(&Choice::from(0), first); + } + + if bool::from(is_equal) { + return Some(Ordering::Equal); + } + + let hash: Z2Hash = Hash::new(HashKey::from_slice(&b_right[0..NONCE_SIZE])); + let h = hash.hash(&sel_f); + let test = get_bit(&sel_right, sel_xt as usize) ^ h; + + if test == 1 { + return Some(Ordering::Greater); + } + + Some(Ordering::Less) + } +} + +#[inline] +fn left_block(input: &[u8], n: usize) -> &[u8] { + let f_pos = n * LeftBlock16::BLOCK_SIZE; + &input[f_pos..(f_pos + LeftBlock16::BLOCK_SIZE)] +} + +#[inline] +fn right_block(input: &[u8], n: usize) -> &[u8] { + let f_pos = n * RightBlock8::BLOCK_SIZE; + &input[f_pos..(f_pos + RightBlock8::BLOCK_SIZE)] +} + +#[inline] +fn get_bit(block: &[u8], bit: usize) -> u8 { + debug_assert!(block.len() == RightBlock8::BLOCK_SIZE); + debug_assert!(bit < 64); + // `bit` is the secret permuted symbol; read the byte obliviously so the + // access address does not depend on it. See `width::ct_select_byte`. + let byte = crate::scheme::width::ct_select_byte(block, bit / 8); + crate::scheme::width::ct_bit(byte, (bit % 8) as u8) +} + +impl PartialEq for CipherText { + fn eq(&self, b: &Self) -> bool { + matches!(self.cmp(b), Ordering::Equal) + } +} + +impl Ord for CipherText { + fn cmp(&self, b: &Self) -> Ordering { + let mut is_equal = Choice::from(1); + let mut l: u64 = 0; // Unequal block + + for n in 0..N { + let condition: Choice = + !(self.left.xt[n].ct_eq(&b.left.xt[n])) | !(self.left.f[n].ct_eq(&b.left.f[n])); + + l.conditional_assign(&(n as u64), is_equal & condition); + is_equal.conditional_assign(&Choice::from(0), is_equal & condition); + } + + let l: usize = l as usize; + + if bool::from(is_equal) { + return Ordering::Equal; + } + + let hash: Z2Hash = Hash::new(HashKey::from_slice(&b.right.nonce)); + let h = hash.hash(&self.left.f[l]); + + let test = b.right.data[l].get_bit(self.left.xt[l] as usize) ^ h; + if test == 1 { + return Ordering::Greater; + } + + Ordering::Less + } +} + +impl PartialOrd for CipherText { + fn partial_cmp(&self, other: &Self) -> Option { + Some(self.cmp(other)) + } +} + +impl Eq for CipherText {} + +// --------------------------------------------------------------------------- +// OreEncrypt impls: byte → 6-bit-block decomposition per primitive type. +// --------------------------------------------------------------------------- + +mod encrypt_impls { + use super::{OreAes128Bit6, MAX_BLOCKS}; + use crate::primitives::stack; + use crate::scheme::decompose::{decompose_6bit, num_blocks_6bit}; + use crate::{CipherText, Left, OreEncrypt, OreError}; + use orderable_bytes::{FixedOrderableBytes, OrderableBytes}; + use rand::{Rng, SeedableRng}; + + macro_rules! impl_ore_encrypt_bit6 { + ($type:ty, $blocks_const:ident) => { + /// Block count for this type at 6-bit width. + const $blocks_const: usize = + num_blocks_6bit(<$type as FixedOrderableBytes>::ENCODED_LEN); + // The packed prefix caps the block count; types beyond it + // (u128, i128, Decimal) must not get these impls. + const _: () = assert!($blocks_const <= MAX_BLOCKS); + + impl OreEncrypt> for $type { + type LeftOutput = Left, $blocks_const>; + type FullOutput = CipherText, $blocks_const>; + + fn encrypt_left( + &self, + cipher: &OreAes128Bit6, + ) -> Result { + // The plaintext's bytes and blocks are copies in this + // frame, so they go inside the scrub too. + stack::scrubbed(|| { + let bytes = self.to_orderable_bytes(); + let mut blocks = [0u8; $blocks_const]; + decompose_6bit(&bytes, &mut blocks); + cipher.encrypt_left_unscrubbed(&blocks) + }) + } + + fn encrypt(&self, cipher: &OreAes128Bit6) -> Result { + stack::scrubbed(|| { + let bytes = self.to_orderable_bytes(); + let mut blocks = [0u8; $blocks_const]; + decompose_6bit(&bytes, &mut blocks); + cipher.encrypt_unscrubbed(&blocks) + }) + } + } + }; + } + + impl_ore_encrypt_bit6!(bool, BOOL_BLOCKS); + impl_ore_encrypt_bit6!(u8, U8_BLOCKS); + impl_ore_encrypt_bit6!(i8, I8_BLOCKS); + impl_ore_encrypt_bit6!(u16, U16_BLOCKS); + impl_ore_encrypt_bit6!(i16, I16_BLOCKS); + impl_ore_encrypt_bit6!(u32, U32_BLOCKS); + impl_ore_encrypt_bit6!(i32, I32_BLOCKS); + impl_ore_encrypt_bit6!(u64, U64_BLOCKS); + impl_ore_encrypt_bit6!(i64, I64_BLOCKS); + impl_ore_encrypt_bit6!(char, CHAR_BLOCKS); + impl_ore_encrypt_bit6!(f32, F32_BLOCKS); + impl_ore_encrypt_bit6!(f64, F64_BLOCKS); +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::encrypt::OreEncrypt; + use crate::OreOutput; + use quickcheck::TestResult; + + type Ore = OreAes128Bit6ChaCha20; + + fn init_ore() -> Ore { + let mut k1: [u8; 16] = Default::default(); + let mut k2: [u8; 16] = Default::default(); + + let mut rng = ChaCha20Rng::from_entropy(); + + rng.fill(&mut k1); + rng.fill(&mut k2); + + OreCipher::init(&k1, &k2).unwrap() + } + + /// Cross-scheme regression (review of #82): under keys shared with the + /// legacy scheme, a legacy `N = 15` left ciphertext must never publish + /// one of Bit6's PRF₁ tags. Bit6's `f[0]` for an 11-block value has PRF₁ + /// input `xt[0] ‖ 0×10 ‖ 0@11 ‖ 0×3 ‖ 11@15`; legacy block 11 of + /// `(xt[0], 0×10, c, 0×3)` has input `xt[0] ‖ 0×10 ‖ xt'[11] ‖ 0×3 ‖ + /// 11@15`, which matches whenever `c` permutes to 0. Search every `c`. + #[test] + fn legacy_ciphertexts_never_publish_bit6_tags() { + use crate::scheme::bit2::OreAes128ChaCha20; + + let mut k1 = [0u8; 16]; + let mut k2 = [0u8; 16]; + let mut rng = ChaCha20Rng::from_entropy(); + rng.fill(&mut k1); + rng.fill(&mut k2); + let bit6: Ore = OreCipher::init(&k1, &k2).unwrap(); + let legacy: OreAes128ChaCha20 = OreCipher::init(&k1, &k2).unwrap(); + + let target = bit6.encrypt_left(&[0u8; 11]).unwrap(); + let tag = target.f[0]; + + let mut probe = [0u8; 15]; + probe[0] = target.xt[0]; + for c in 0..=255u8 { + probe[11] = c; + let left = legacy.encrypt_left(&probe).unwrap(); + assert!( + !bool::from(left.f[11].ct_eq(&tag)), + "legacy block 11 reproduced a Bit6 tag (c = {})", + c + ); + } + } + + /// Position-binding regression (review of #82): every position of an + /// all-zero plaintext must get its own permutation, so its `xt` bytes + /// are not all equal (they were, when seeds bound only the count). + #[test] + fn zero_blocks_use_distinct_permutations_per_position() { + let ore = init_ore(); + let left = ore.encrypt_left(&[0u8; 11]).unwrap(); + let first = left.xt[0]; + assert!( + left.xt.iter().any(|&s| s != first), + "all 11 positions produced the same symbol" + ); + } + + quickcheck! { + fn compare_u64(x: u64, y: u64) -> bool { + let ore = init_ore(); + let a = x.encrypt(&ore).unwrap(); + let b = y.encrypt(&ore).unwrap(); + + match x.cmp(&y) { + Ordering::Greater => a > b, + Ordering::Less => a < b, + Ordering::Equal => a == b + } + } + + fn compare_u64_raw_slices(x: u64, y: u64) -> bool { + let ore = init_ore(); + let a = x.encrypt(&ore).unwrap().to_bytes(); + let b = y.encrypt(&ore).unwrap().to_bytes(); + + match Ore::compare_raw_slices(&a, &b) { + Some(Ordering::Greater) => x > y, + Some(Ordering::Less) => x < y, + Some(Ordering::Equal) => x == y, + None => false + } + } + + fn equality_u64(x: u64) -> bool { + let ore = init_ore(); + let a = x.encrypt(&ore).unwrap(); + let b = x.encrypt(&ore).unwrap(); + + a == b + } + + fn compare_u32(x: u32, y: u32) -> bool { + let ore = init_ore(); + let a = x.encrypt(&ore).unwrap(); + let b = y.encrypt(&ore).unwrap(); + + match x.cmp(&y) { + Ordering::Greater => a > b, + Ordering::Less => a < b, + Ordering::Equal => a == b + } + } + + fn compare_f64(x: f64, y: f64) -> TestResult { + if x.is_nan() || x.is_infinite() || y.is_nan() || y.is_infinite() { + return TestResult::discard(); + } + + let ore = init_ore(); + let a = x.encrypt(&ore).unwrap(); + let b = y.encrypt(&ore).unwrap(); + + match x.partial_cmp(&y) { + Some(Ordering::Greater) => TestResult::from_bool(a > b), + Some(Ordering::Less) => TestResult::from_bool(a < b), + Some(Ordering::Equal) => TestResult::from_bool(a == b), + None => TestResult::failed() + } + } + + fn serialize_roundtrip_u64(x: u64) -> bool { + let ore = init_ore(); + let a = x.encrypt(&ore).unwrap(); + let bytes = a.to_bytes(); + let b = CipherText::::from_slice(&bytes).unwrap(); + a == b + } + } + + #[test] + fn ciphertext_sizes() { + // u64: 11 blocks. header(4) + xt(11) + f(11*16) + nonce(16) + right(11*8) + assert_eq!(CipherText::::size(), 4 + 11 + 176 + 16 + 88); + let ore = init_ore(); + let ct = 456u64.encrypt(&ore).unwrap(); + assert_eq!(ct.to_bytes().len(), 295); + } + + #[test] + fn header_emitted_and_validated() { + let ore = init_ore(); + let mut bytes = 456u64.encrypt(&ore).unwrap().to_bytes(); + assert_eq!(&bytes[0..4], &[0x02, 0x02, 0x00, 11]); + + // Corrupt each header field; parsing must fail. + for i in 0..4 { + let mut bad = bytes.clone(); + bad[i] ^= 0xff; + assert!(CipherText::::from_slice(&bad).is_err()); + } + + // Truncation fails. + bytes.pop(); + assert!(CipherText::::from_slice(&bytes).is_err()); + } + + #[test] + fn cross_scheme_comparison_rejected() { + use crate::scheme::bit2::OreAes128ChaCha20; + + let k1 = [1u8; 16]; + let k2 = [2u8; 16]; + let legacy: OreAes128ChaCha20 = OreCipher::init(&k1, &k2).unwrap(); + let bit6: Ore = OreCipher::init(&k1, &k2).unwrap(); + + let a = 456u64.encrypt(&legacy).unwrap().to_bytes(); + let b = 456u64.encrypt(&bit6).unwrap().to_bytes(); + + assert_eq!(Ore::compare_raw_slices(&a, &b), None); + assert_eq!(Ore::compare_raw_slices(&b, &a), None); + // The legacy comparator infers block count from length; a Bit6 + // u64 ciphertext (295 bytes) never matches a legacy length for + // equal-length inputs, and unequal lengths return None up front. + assert_eq!(OreAes128ChaCha20::compare_raw_slices(&a, &b), None); + } + + #[test] + fn cross_block_count_comparison_rejected() { + let ore = init_ore(); + let a = 456u64.encrypt(&ore).unwrap().to_bytes(); + let b = 456u32.encrypt(&ore).unwrap().to_bytes(); + assert_eq!(Ore::compare_raw_slices(&a, &b), None); + } + + #[test] + fn smallest_to_largest() { + let ore = init_ore(); + let a = 0u64.encrypt(&ore).unwrap(); + let b = u64::MAX.encrypt(&ore).unwrap(); + + assert!(a < b); + } + + #[test] + fn comparisons_in_last_block() { + let ore = init_ore(); + let a = 10u64.encrypt(&ore).unwrap(); + let b = 73u64.encrypt(&ore).unwrap(); + + assert!(a < b); + assert!(b > a); + } + + #[test] + fn different_keys_not_equal() { + let k1 = [1u8; 16]; + let k2 = [2u8; 16]; + let k3 = [3u8; 16]; + + let ore1: Ore = OreCipher::init(&k1, &k2).unwrap(); + let ore2: Ore = OreCipher::init(&k3, &k2).unwrap(); + + let a = 1000u32.encrypt(&ore1).unwrap().to_bytes(); + let b = 1000u32.encrypt(&ore2).unwrap().to_bytes(); + + assert_ne!(Some(Ordering::Equal), Ore::compare_raw_slices(&a, &b)); + } + + /// The PRP seeds and permutation tables of every block, which must not + /// survive an encryption in dead stack (review of #82). + fn block_secrets(ore: &Ore, blocks: &[u8; 11]) -> Vec> { + let seeds = ore.derive_prp_seeds(blocks); + let mut needles = Vec::new(); + for seed in seeds.0.iter() { + let prp: ::Prp = Prp::new(seed).unwrap(); + needles.push(seed.to_vec()); + needles.push((0..64).map(|i| prp.permute(i).unwrap()).collect()); + needles.push((0..64).map(|i| prp.invert(i).unwrap()).collect()); + } + needles + } + + fn assert_no_block_secrets(encrypt: fn(&Ore, u64), unscrubbed: fn(&Ore, u64)) { + use crate::primitives::stack::{residue, SCRUB_BYTES}; + let k1 = [0x3cu8; 16]; + let k2 = [0xc3u8; 16]; + let x = 0x0123_4567_89ab_cdefu64; + let needles = block_secrets(&Ore::init(&k1, &k2).unwrap(), &u64_blocks(x)); + + // Positive control and depth: without the scrub the secrets are + // there, and the scrub must reach deeper than the encryption did. + let ore = Ore::init(&k1, &k2).unwrap(); + let dead = residue::after(move || unscrubbed(&ore, x)); + assert!(needles.iter().any(|n| residue::contains(&dead, n))); + let depth = residue::depth(&dead); + assert!( + depth < SCRUB_BYTES, + "encryption reached {} bytes of stack, past the {}-byte scrub", + depth, + SCRUB_BYTES + ); + + let ore = Ore::init(&k1, &k2).unwrap(); + let dead = residue::after(move || encrypt(&ore, x)); + for (i, needle) in needles.iter().enumerate() { + assert!( + !residue::contains(&dead, needle), + "block {} {} left in dead stack", + i / 3, + ["seed", "permutation", "inverse"][i % 3] + ); + } + } + + fn u64_blocks(x: u64) -> [u8; 11] { + let mut blocks = [0u8; 11]; + crate::scheme::decompose::decompose_6bit(&x.to_be_bytes(), &mut blocks); + blocks + } + + #[test] + fn encrypt_leaves_no_prp_secrets_on_the_stack() { + assert_no_block_secrets( + |ore, x| { + core::hint::black_box(x.encrypt(ore).unwrap()); + }, + |ore, x| { + core::hint::black_box(ore.encrypt_unscrubbed(&u64_blocks(x)).unwrap()); + }, + ); + } + + #[test] + fn encrypt_left_leaves_no_prp_secrets_on_the_stack() { + assert_no_block_secrets( + |ore, x| { + core::hint::black_box(x.encrypt_left(ore).unwrap()); + }, + |ore, x| { + core::hint::black_box(ore.encrypt_left_unscrubbed(&u64_blocks(x)).unwrap()); + }, + ); + } +} diff --git a/packages/ore-rs/src/scheme/bit2_w6/block_types.rs b/packages/ore-rs/src/scheme/bit2_w6/block_types.rs new file mode 100644 index 00000000..97ced6cf --- /dev/null +++ b/packages/ore-rs/src/scheme/bit2_w6/block_types.rs @@ -0,0 +1,58 @@ +use zeroize::Zeroize; + +use crate::ciphertext::{CipherTextBlock, ParseError}; +use crate::scheme::width::RightBitVec; + +pub use crate::scheme::bit2::block_types::LeftBlock16; + +/// Per-block component of the Right half of a `bit2_w6` ciphertext: an +/// 8-byte (64-bit) bitvector encoding one masked truth-table row, one bit +/// per value in the 6-bit block domain. +#[derive(Debug, Copy, Clone, Default)] +pub struct RightBlock8 { + data: [u8; 8], +} + +impl RightBlock8 { + /// Read bit `bit` (in `0..64`); returns `0` or `1`. + #[inline] + pub fn get_bit(&self, bit: usize) -> u8 { + debug_assert!(bit < 64); + // `bit` is the secret permuted symbol; read the byte obliviously so the + // access address does not depend on it. See `width::ct_select_byte`. + let byte = crate::scheme::width::ct_select_byte(&self.data, bit / 8); + crate::scheme::width::ct_bit(byte, (bit % 8) as u8) + } +} + +impl RightBitVec for RightBlock8 { + fn get_bit(&self, bit: usize) -> u8 { + RightBlock8::get_bit(self, bit) + } + fn as_mut_bytes(&mut self) -> &mut [u8] { + &mut self.data + } +} + +impl CipherTextBlock for RightBlock8 { + const BLOCK_SIZE: usize = 8; + + fn to_bytes(self) -> Vec { + self.data.to_vec() + } + + fn from_bytes(data: &[u8]) -> Result { + if data.len() != Self::BLOCK_SIZE { + Err(ParseError) + } else { + let mut arr = [0; 8]; + arr.clone_from_slice(data); + + Ok(Self { data: arr }) + } + } + + fn default_in_place(&mut self) { + self.data.zeroize() + } +} diff --git a/packages/ore-rs/src/scheme/width.rs b/packages/ore-rs/src/scheme/width.rs index 08013208..1dd841a0 100644 --- a/packages/ore-rs/src/scheme/width.rs +++ b/packages/ore-rs/src/scheme/width.rs @@ -13,11 +13,14 @@ use crate::ciphertext::CipherTextBlock; use crate::primitives::{AesBlock, Prp}; use crate::scheme::bit2::block_types::RightBlock32; +use crate::scheme::bit2_w6::block_types::RightBlock8; mod sealed { pub trait Sealed {} impl Sealed for super::Bit8 {} + impl Sealed for super::Bit6 {} impl Sealed for [super::AesBlock; 256] {} + impl Sealed for [super::AesBlock; 64] {} } /// A domain-sized buffer of AES blocks used for per-block random-oracle @@ -35,18 +38,97 @@ pub trait AesBlockBuf: sealed::Sealed { fn copy_from(&mut self, other: &Self); } -impl AesBlockBuf for [AesBlock; 256] { - fn zeroed() -> Self { - [AesBlock::default(); 256] - } - fn as_mut_slice(&mut self) -> &mut [AesBlock] { - self +macro_rules! impl_aes_block_buf { + ($n:literal) => { + impl AesBlockBuf for [AesBlock; $n] { + fn zeroed() -> Self { + [AesBlock::default(); $n] + } + fn as_mut_slice(&mut self) -> &mut [AesBlock] { + self + } + fn copy_from(&mut self, other: &Self) { + self.clone_from_slice(other); + } + } + }; +} + +impl_aes_block_buf!(256); +impl_aes_block_buf!(64); + +/// Oblivious byte read: returns `block[idx]` while touching **every** byte of +/// `block`, so the memory-access address is independent of `idx`. +/// +/// The comparators read the right-ciphertext block at `bit / 8` where `bit` is +/// the secret permuted symbol (`a[l]`, the left ciphertext's permuted index at +/// the first differing block). A direct `block[bit / 8]` index makes the +/// touched address secret-dependent, which is a cache-line timing channel — and +/// because the block fits within a line, even an aligned direct index would +/// still leak at sub-line (4-byte) granularity to a MemJam-class attacker on +/// SMT-enabled Intel. Scanning the whole block removes the data-dependent +/// address entirely, closing both. The block is ≤ 32 bytes, so the scan is +/// cheap relative to the per-comparison AES hash. +/// +/// See `docs/reviews/2026-06-14-ore-v2-crypto-review-brief.md` (A4, compare +/// side). The bit within the selected byte is then extracted with [`ct_bit`], +/// which avoids a shift by the secret amount. +#[inline] +pub(crate) fn ct_select_byte(block: &[u8], idx: usize) -> u8 { + use subtle_ng::{ConditionallySelectable, ConstantTimeEq}; + let mut acc = 0u8; + for (i, &b) in block.iter().enumerate() { + // `i` is the public loop counter; `idx` is secret. Both index a block + // or table of ≤ 256 bytes, so the u8 cast is lossless. + acc.conditional_assign(&b, (i as u8).ct_eq(&(idx as u8))); } - fn copy_from(&mut self, other: &Self) { - self.clone_from_slice(other); + acc +} + +/// Oblivious conditional copy: `dst = src` when `choice` is set, otherwise +/// `dst` is left as it was; every byte of both is touched either way. +/// +/// The comparators use this to *latch* the first differing block while the +/// constant-time prefix scan is still running: at each block `n` the scan +/// knows whether `n` is the first difference, and copies that block's left +/// tag, permuted symbol and right bitvector under that choice. Nothing after +/// the scan is then loaded at an address derived from the position of the +/// first difference. A direct `right[l]` after the scan would be the one load +/// whose cache state depends on `l` (the scan never touches the right blocks), +/// which a timing attacker can read; `docs/reviews/` records the measurement. +#[inline] +pub(crate) fn ct_assign_bytes(dst: &mut [u8], src: &[u8], choice: subtle_ng::Choice) { + use subtle_ng::ConditionallySelectable; + debug_assert_eq!(dst.len(), src.len()); + for (d, s) in dst.iter_mut().zip(src) { + d.conditional_assign(s, choice); } } +/// Oblivious extraction of bit `pos` (`0..8`) of `byte` — used right after +/// [`ct_select_byte`] to read the target bit of the selected right-block byte. +/// +/// `byte >> pos` would be a shift by a *secret* amount; that is constant-time +/// on x86_64/aarch64 (the targets ore.rs ships to) but not guaranteed so on +/// every architecture. Here every candidate shift is a compile-time constant +/// and the result is chosen with a constant-time select, so the timing is +/// data-independent on all targets — defence-in-depth matching the oblivious +/// byte read above. +#[inline] +pub(crate) fn ct_bit(byte: u8, pos: u8) -> u8 { + use subtle_ng::{ConditionallySelectable, ConstantTimeEq}; + let mut out = 0u8; + out.conditional_assign(&(byte & 1), pos.ct_eq(&0)); + out.conditional_assign(&((byte >> 1) & 1), pos.ct_eq(&1)); + out.conditional_assign(&((byte >> 2) & 1), pos.ct_eq(&2)); + out.conditional_assign(&((byte >> 3) & 1), pos.ct_eq(&3)); + out.conditional_assign(&((byte >> 4) & 1), pos.ct_eq(&4)); + out.conditional_assign(&((byte >> 5) & 1), pos.ct_eq(&5)); + out.conditional_assign(&((byte >> 6) & 1), pos.ct_eq(&6)); + out.conditional_assign(&((byte >> 7) & 1), pos.ct_eq(&7)); + out +} + /// Per-block bitvector operations on a Right ciphertext block, one bit per /// value in the block domain. pub trait RightBitVec { @@ -95,3 +177,19 @@ impl BlockWidth for Bit8 { type Prp = crate::primitives::prp::KnuthShufflePRP; type RoKeyBuf = [AesBlock; 256]; } + +/// The 6-bit block width used by [`crate::scheme::bit2_w6`]: six plaintext +/// bits per block, domain 64. Right blocks are 8 bytes (vs 32) and each +/// block costs 64 RO evaluations (vs 256). +#[derive(Debug)] +pub struct Bit6; + +impl BlockWidth for Bit6 { + const BITS: usize = 6; + type RightBlock = RightBlock8; + // Fixed-draw Fisher–Yates, not the rejection-sampled Knuth shuffle: + // Bit6's wire format is not frozen, so it adopts the constant-time, + // ~9×-faster PRP construction. See `LemireFyPrp`. + type Prp = crate::primitives::prp::LemireFyPrp<64>; + type RoKeyBuf = [AesBlock; 64]; +} diff --git a/packages/ore-rs/tests/compat_w6_vectors.rs b/packages/ore-rs/tests/compat_w6_vectors.rs new file mode 100644 index 00000000..79493640 --- /dev/null +++ b/packages/ore-rs/tests/compat_w6_vectors.rs @@ -0,0 +1,392 @@ +//! Wire-format vectors for the `OreAes128Bit6` (bit2_w6) scheme — the v2 +//! 6-bit-block scheme with the v2 wire header and the BHKR σ-MMO hash `H` +//! (plan §6 / review brief A1, resolved 2026-06-15). +//! +//! These tests pin the exact serialised bytes produced for fixed keys and +//! plaintexts, plus comparison results over those bytes. They freeze the v2 +//! Bit6 wire format introduced by this PR: once released, any change that +//! alters these bytes is a wire-format break for stored Bit6 ciphertexts and +//! must fail here. (Mirror of `compat_vectors.rs`, which does the same for the +//! legacy Bit8 `OreAes128` scheme.) +//! +//! Left ciphertexts are deterministic given the keys. Full ciphertexts include +//! a random nonce drawn from the cipher's internal RNG, which `OreCipher::init` +//! seeds via `SeedableRng::from_entropy`; [`TestRng`] overrides `from_entropy` +//! to a fixed seed so full-ciphertext bytes are reproducible. +//! +//! To regenerate (only legitimate if the wire format is *deliberately* changed): +//! +//! ```text +//! cargo test --test compat_w6_vectors -- --ignored --nocapture generate +//! ``` + +use ore_rs::{ + scheme::bit2_w6::{OreAes128Bit6, OreAes128Bit6ChaCha20}, + CipherText, Left, OreCipher, OreEncrypt, OreOutput, +}; +use rand::{RngCore, SeedableRng}; +use rand_chacha::ChaCha20Rng; +use std::cmp::Ordering; + +const K1: [u8; 16] = [ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, +]; +const K2: [u8; 16] = [ + 0xd0, 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda, 0xdb, 0xdc, 0xdd, 0xde, 0xdf, +]; + +/// RNG whose `from_entropy` is deterministic, so that `OreCipher::init` +/// (which calls `from_entropy` internally) produces a reproducible nonce stream. +struct TestRng(ChaCha20Rng); + +impl RngCore for TestRng { + fn next_u32(&mut self) -> u32 { + self.0.next_u32() + } + fn next_u64(&mut self) -> u64 { + self.0.next_u64() + } + fn fill_bytes(&mut self, dest: &mut [u8]) { + self.0.fill_bytes(dest) + } + fn try_fill_bytes(&mut self, dest: &mut [u8]) -> Result<(), rand::Error> { + self.0.try_fill_bytes(dest) + } +} + +impl SeedableRng for TestRng { + type Seed = [u8; 32]; + fn from_seed(seed: Self::Seed) -> Self { + Self(ChaCha20Rng::from_seed(seed)) + } + fn from_entropy() -> Self { + Self::from_seed([SEED; 32]) + } +} + +type OreA = OreAes128Bit6>; +type OreB = OreAes128Bit6>; +type OreC = OreAes128Bit6>; + +fn cipher_a() -> OreA { + OreCipher::init(&K1, &K2).unwrap() +} + +fn cipher_b() -> OreB { + OreCipher::init(&K1, &K2).unwrap() +} + +fn cipher_c() -> OreC { + OreCipher::init(&K1, &K2).unwrap() +} + +/// Plaintexts for [`FULL_U64_ONE_CIPHER`], in ascending order. Blocks are the +/// 6-bit groups of the big-endian bits, so pairs first differ at the last +/// block (`BASE` and `BASE + 1`), a middle block (`+ (1 << 29)`, block 5) and +/// block 0 (`+ (1 << 58)`). None of the additions carries. +const ONE_CIPHER_U64: [u64; 4] = [ + 0x1020_3040_5060_7080, + 0x1020_3040_5060_7081, + 0x1020_3040_7060_7080, + 0x1420_3040_5060_7080, +]; + +fn cipher_left() -> OreAes128Bit6ChaCha20 { + OreCipher::init(&K1, &K2).unwrap() +} + +#[path = "compat_w6_vectors/vectors.rs"] +mod vectors; +use vectors::*; + +// --------------------------------------------------------------------------- +// Left ciphertexts (fully deterministic) +// --------------------------------------------------------------------------- + +#[test] +fn left_u64_456() { + let ore = cipher_left(); + let left = 456u64.encrypt_left(&ore).unwrap(); + assert_eq!(hex::encode(left.to_bytes()), LEFT_U64_456); +} + +#[test] +fn left_u64_zero() { + let ore = cipher_left(); + let left = 0u64.encrypt_left(&ore).unwrap(); + assert_eq!(hex::encode(left.to_bytes()), LEFT_U64_0); +} + +#[test] +fn left_u32_1000() { + let ore = cipher_left(); + let left = 1000u32.encrypt_left(&ore).unwrap(); + assert_eq!(hex::encode(left.to_bytes()), LEFT_U32_1000); +} + +// --------------------------------------------------------------------------- +// Full ciphertexts (deterministic via TestRng nonce stream) +// --------------------------------------------------------------------------- + +#[test] +fn full_u64_vectors() { + for (value, expected) in FULL_U64 { + let ct = value.encrypt(&cipher_a()).unwrap(); + assert_eq!( + hex::encode(ct.to_bytes()), + *expected, + "full ciphertext mismatch for u64 {value}" + ); + } +} + +#[test] +fn full_u32_vectors() { + for (value, expected) in FULL_U32 { + let ct = value.encrypt(&cipher_a()).unwrap(); + assert_eq!( + hex::encode(ct.to_bytes()), + *expected, + "full ciphertext mismatch for u32 {value}" + ); + } +} + +#[test] +fn full_i64_vectors() { + for (value, expected) in FULL_I64 { + let ct = value.encrypt(&cipher_a()).unwrap(); + assert_eq!( + hex::encode(ct.to_bytes()), + *expected, + "full ciphertext mismatch for i64 {value}" + ); + } +} + +#[test] +fn full_f64_vector() { + let ct = 1.5f64.encrypt(&cipher_a()).unwrap(); + assert_eq!(hex::encode(ct.to_bytes()), FULL_F64_1_5); +} + +#[test] +fn full_u64_one_cipher_vectors() { + // One cipher for every value, so its nonce stream advances and each + // ciphertext has a different nonce. + let ore = cipher_c(); + for (value, (pinned_value, expected)) in ONE_CIPHER_U64.iter().zip(FULL_U64_ONE_CIPHER) { + assert_eq!(value, pinned_value); + let ct = value.encrypt(&ore).unwrap(); + assert_eq!( + hex::encode(ct.to_bytes()), + *expected, + "full ciphertext mismatch for u64 {value}" + ); + } +} + +#[test] +fn full_u64_456_alternate_nonce() { + let ct = 456u64.encrypt(&cipher_b()).unwrap(); + assert_eq!(hex::encode(ct.to_bytes()), FULL_U64_456_SEED_B); +} + +// --------------------------------------------------------------------------- +// Comparison fixtures over pinned bytes (no RNG involved) +// --------------------------------------------------------------------------- + +fn pinned(hex_str: &str) -> Vec { + hex::decode(hex_str).unwrap() +} + +#[test] +fn compare_raw_slices_total_order_u64() { + // FULL_U64 is listed in ascending plaintext order. + for (i, (_, a)) in FULL_U64.iter().enumerate() { + for (j, (_, b)) in FULL_U64.iter().enumerate() { + let expected = i.cmp(&j); + assert_eq!( + OreAes128Bit6ChaCha20::compare_raw_slices(&pinned(a), &pinned(b)), + Some(expected), + "u64 vector order mismatch at ({i}, {j})" + ); + } + } +} + +#[test] +fn compare_raw_slices_total_order_i64() { + // FULL_I64 is listed in ascending plaintext order (incl. negatives). + for (i, (_, a)) in FULL_I64.iter().enumerate() { + for (j, (_, b)) in FULL_I64.iter().enumerate() { + let expected = i.cmp(&j); + assert_eq!( + OreAes128Bit6ChaCha20::compare_raw_slices(&pinned(a), &pinned(b)), + Some(expected), + "i64 vector order mismatch at ({i}, {j})" + ); + } + } +} + +#[test] +fn compare_raw_slices_equality_across_nonces() { + // Same plaintext (456u64) encrypted under two different nonce streams + // must compare equal. + let (_, seed_a) = FULL_U64[2]; + assert_eq!( + OreAes128Bit6ChaCha20::compare_raw_slices(&pinned(seed_a), &pinned(FULL_U64_456_SEED_B)), + Some(Ordering::Equal) + ); +} + +#[test] +fn compare_raw_slices_across_nonces() { + // Every unequal pair here has two different nonces, and the comparator + // must hash under the nonce of its second argument. FULL_U64 cannot pin + // this: its unequal pairs share one nonce. + // The nonce opens the right half, after the 4-byte header and 11 left + // blocks of 17 bytes. + let nonce = |ct: &str| pinned(ct)[4 + 11 * 17..4 + 11 * 17 + 16].to_vec(); + for (i, (_, a)) in FULL_U64_ONE_CIPHER.iter().enumerate() { + for (j, (_, b)) in FULL_U64_ONE_CIPHER.iter().enumerate() { + if i != j { + assert_ne!(nonce(a), nonce(b), "pair ({i}, {j}) shares a nonce"); + } + let expected = i.cmp(&j); + assert_eq!( + OreAes128Bit6ChaCha20::compare_raw_slices(&pinned(a), &pinned(b)), + Some(expected), + "raw order mismatch at ({i}, {j})" + ); + let ct_a = CipherText::::from_slice(&pinned(a)).unwrap(); + let ct_b = CipherText::::from_slice(&pinned(b)).unwrap(); + assert_eq!( + ct_a.cmp(&ct_b), + expected, + "typed order mismatch at ({i}, {j})" + ); + } + } +} + +#[test] +fn typed_comparison_of_pinned_bytes() { + // u64 is 11 blocks at 6-bit width. + let a = CipherText::::from_slice(&pinned(FULL_U64[1].1)).unwrap(); + let b = CipherText::::from_slice(&pinned(FULL_U64[3].1)).unwrap(); + assert!(a < b); + assert!(b > a); + assert_eq!(a.cmp(&a), Ordering::Equal); +} + +// --------------------------------------------------------------------------- +// Generator (run manually; see module docs) +// --------------------------------------------------------------------------- + +// --------------------------------------------------------------------------- +// Malformed symbols: an `xt` byte outside the 64-element domain +// --------------------------------------------------------------------------- + +/// A pinned ciphertext with its first `xt` byte (just after the 4-byte +/// header) set to 64, one past the Bit6 symbol domain. +fn with_out_of_domain_symbol(hex_str: &str) -> Vec { + let mut bytes = pinned(hex_str); + bytes[4] = 64; + bytes +} + +#[test] +fn parsers_reject_out_of_domain_symbols() { + let full = with_out_of_domain_symbol(FULL_U64[0].1); + assert!(CipherText::::from_slice(&full).is_err()); + + let left = with_out_of_domain_symbol(LEFT_U64_456); + assert!(Left::::from_slice(&left).is_err()); + + // The highest in-domain symbol still parses. + let mut edge = pinned(FULL_U64[0].1); + edge[4] = 63; + assert!(CipherText::::from_slice(&edge).is_ok()); +} + +#[test] +fn raw_comparator_rejects_out_of_domain_symbols() { + let good = pinned(FULL_U64[0].1); + let bad = with_out_of_domain_symbol(FULL_U64[0].1); + assert_eq!(OreAes128Bit6ChaCha20::compare_raw_slices(&bad, &good), None); + assert_eq!(OreAes128Bit6ChaCha20::compare_raw_slices(&good, &bad), None); +} + +#[test] +#[ignore = "generator: prints the contents of tests/compat_w6_vectors/vectors.rs"] +fn generate() { + println!("// Contents of tests/compat_w6_vectors/vectors.rs"); + println!( + "// Generated by `cargo test --test compat_w6_vectors -- --ignored --nocapture generate`" + ); + println!("// DO NOT regenerate unless deliberately breaking the wire format (see plan doc)."); + println!(); + + let ore = cipher_left(); + let print_left = |name: &str, bytes: Vec| { + println!("pub const {}: &str = \"{}\";", name, hex::encode(bytes)); + }; + + print_left( + "LEFT_U64_456", + 456u64.encrypt_left(&ore).unwrap().to_bytes(), + ); + print_left("LEFT_U64_0", 0u64.encrypt_left(&ore).unwrap().to_bytes()); + print_left( + "LEFT_U32_1000", + 1000u32.encrypt_left(&ore).unwrap().to_bytes(), + ); + println!(); + + println!("pub const FULL_U64: &[(u64, &str)] = &["); + for value in [0u64, 1, 456, 1 << 32, u64::MAX] { + let ct = value.encrypt(&cipher_a()).unwrap(); + println!(" ({}, \"{}\"),", value, hex::encode(ct.to_bytes())); + } + println!("];"); + println!(); + + println!("pub const FULL_U32: &[(u32, &str)] = &["); + for value in [0u32, 1000, u32::MAX] { + let ct = value.encrypt(&cipher_a()).unwrap(); + println!(" ({}, \"{}\"),", value, hex::encode(ct.to_bytes())); + } + println!("];"); + println!(); + + println!("pub const FULL_I64: &[(i64, &str)] = &["); + for value in [i64::MIN, -1i64, 0, 1, i64::MAX] { + let ct = value.encrypt(&cipher_a()).unwrap(); + println!(" ({}, \"{}\"),", value, hex::encode(ct.to_bytes())); + } + println!("];"); + println!(); + + let f = 1.5f64.encrypt(&cipher_a()).unwrap(); + println!( + "pub const FULL_F64_1_5: &str = \"{}\";", + hex::encode(f.to_bytes()) + ); + + let alt = 456u64.encrypt(&cipher_b()).unwrap(); + println!( + "pub const FULL_U64_456_SEED_B: &str = \"{}\";", + hex::encode(alt.to_bytes()) + ); + println!(); + + let ore = cipher_c(); + println!("pub const FULL_U64_ONE_CIPHER: &[(u64, &str)] = &["); + for value in ONE_CIPHER_U64 { + let ct = value.encrypt(&ore).unwrap(); + println!(" ({}, \"{}\"),", value, hex::encode(ct.to_bytes())); + } + println!("];"); +} diff --git a/packages/ore-rs/tests/compat_w6_vectors/vectors.rs b/packages/ore-rs/tests/compat_w6_vectors/vectors.rs new file mode 100644 index 00000000..307681ea --- /dev/null +++ b/packages/ore-rs/tests/compat_w6_vectors/vectors.rs @@ -0,0 +1,39 @@ +// Contents of tests/compat_w6_vectors/vectors.rs +// Generated by `cargo test --test compat_w6_vectors -- --ignored --nocapture generate` +// DO NOT regenerate unless deliberately breaking the wire format (see plan doc). + +pub const LEFT_U64_456: &str = "0202000b013f1f203f373e0f301b3791ce523fe59cdb69838a28604e1576d6deea8b57c4981ef123ecc99ef95c080da53249a27c9326fa7505466975a806307a72070dc5a2d29e83ba85a85a28a7779534113016af87d2ada8191ee4265f1eacd65ee8e69eb4f4b93f328d6c44593dd1d0c6168fa5272f041fc58cf2f7959678c922a78215e28328d25462ba8c1a98b64e7e009b764aedf1e17ca5c598c620b366d3072e597605d5efc750025b7b3b532b80130be54f871b0bb2b7a8c87ac8"; +pub const LEFT_U64_0: &str = "0202000b013f1f203f373e0f303c1a91ce523fe59cdb69838a28604e1576d6deea8b57c4981ef123ecc99ef95c080da53249a27c9326fa7505466975a806307a72070dc5a2d29e83ba85a85a28a7779534113016af87d2ada8191ee4265f1eacd65ee8e69eb4f4b93f328d6c44593dd1d0c6168fa5272f041fc58cf2f7959678c922a78215e28328d25462ba8c1a98b64e7e009b764aedf1e17ca5c598c620c348ac77eb8878a448bac86e710e790e3014f4cdf81f725819a7081622191078"; +pub const LEFT_U32_1000: &str = "020200061627333f0917f820b21dbfbf0cbcd2217f4c4e68c4474cb07354b2454640fad8e4e11ebced9092b4ae437501723eff46f2b995d4f28f394ea4dbd43735c8b62fe32e5ae755fdf7b7b60185f0d34573a13e26936f1fdcbaa7e0b56ed01e508fbe386b9dfdd8b0"; + +pub const FULL_U64: &[(u64, &str)] = &[ + (0, "0202000b013f1f203f373e0f303c1a91ce523fe59cdb69838a28604e1576d6deea8b57c4981ef123ecc99ef95c080da53249a27c9326fa7505466975a806307a72070dc5a2d29e83ba85a85a28a7779534113016af87d2ada8191ee4265f1eacd65ee8e69eb4f4b93f328d6c44593dd1d0c6168fa5272f041fc58cf2f7959678c922a78215e28328d25462ba8c1a98b64e7e009b764aedf1e17ca5c598c620c348ac77eb8878a448bac86e710e790e3014f4cdf81f725819a708162219107898191f46e5830216445436978803697a59555385a79e997b73fbd4d68ecfa7eb78183fcc7ca6a8adffa32130f37278141330e15432698857a26759d44b6c444fc2e11b031b1a20d94edaa29e4279764dd9de35341e279b93ee1507febba91f53e804ed9dfa718b90"), + (1, "0202000b013f1f203f373e0f303c0691ce523fe59cdb69838a28604e1576d6deea8b57c4981ef123ecc99ef95c080da53249a27c9326fa7505466975a806307a72070dc5a2d29e83ba85a85a28a7779534113016af87d2ada8191ee4265f1eacd65ee8e69eb4f4b93f328d6c44593dd1d0c6168fa5272f041fc58cf2f7959678c922a78215e28328d25462ba8c1a98b64e7e009b764aedf1e17ca5c598c620c348ac77eb8878a448bac86e710e790eb1720e7dfeda90eeae9b8d7e648f65e898191f46e5830216445436978803697a59555385a79e997b73fbd4d68ecfa7eb78183fcc7ca6a8adffa32130f37278141330e15432698857a26759d44b6c444fc2e11b031b1a20d94edaa29e4279764dd9de35341e279b93ee1507febba91f53a804ad9dfe718390"), + (456, "0202000b013f1f203f373e0f301b3791ce523fe59cdb69838a28604e1576d6deea8b57c4981ef123ecc99ef95c080da53249a27c9326fa7505466975a806307a72070dc5a2d29e83ba85a85a28a7779534113016af87d2ada8191ee4265f1eacd65ee8e69eb4f4b93f328d6c44593dd1d0c6168fa5272f041fc58cf2f7959678c922a78215e28328d25462ba8c1a98b64e7e009b764aedf1e17ca5c598c620b366d3072e597605d5efc750025b7b3b532b80130be54f871b0bb2b7a8c87ac898191f46e5830216445436978803697a59555385a79e997b73fbd4d68ecfa7eb78183fcc7ca6a8adffa32130f37278141330e15432698857a26759d44b6c444fc2e11b031b1a20d94edaa29e4279764dd9de35341e279b937497215603df55968f4bdfe0fb7aab52"), + (4294967296, "0202000b013f1f203f1e0b2334351091ce523fe59cdb69838a28604e1576d6deea8b57c4981ef123ecc99ef95c080da53249a27c9326fa7505466975a806307a72070dc5a2d29e83ba85a85a28a7779534113016af87d2ada8191ee4265f1ea76552eedbd7b04136b6748cb1233e55533e0e35b39f75d7cefd398742c63065d15cd06372d6260d3ccc01bd067fe4f47fe0814078d8de92c8df09f94d24ef4dd1af408f6d31ca35dea75c3411c312cc96c0b53fbf52ec34a11557c7068065a298191f46e5830216445436978803697a59555385a79e997b73fbd4d68ecfa7eb78183fcc7ca6a8adffa32130f37278141330e15432698857a6d56984db3c240f1e1bd3e21f17307f66b1f163792c9ab1482e7a747c051afca9a1f4abc0bbbad8be8110aff36d9d71"), + (18446744073709551615, "0202000b253a09372324371b10183ad8b7c3915268800457d42ebb622941859100c1f498dee877198108a1b345b71acd43a0d83d6f928534cb8e930973d1df1839911886aaeee67e7239880642b526c62d5970e0d7b3e4dbc71cc63d33042464626dfd79a16d3d053ec160d47b6b85574b5b33b5b057b1737e5eaa567d27a0244cfc87f0a4652b8b023f7c90d9e765f179d37e1118d09c21ac8660bccc3c31d4401164b05b76fa0096573295ac59e2b95f7940b7cae1c8bd1cea7756a56d5898191f46e5830216445436978803697aa4aaac7a586166844dfcaa5a36224b1e09ff5761f4551a1c13e9638983d5b1421a55607c8cfc489896a85545dc9da2992222dbe6b8d93050a4d4153bb6f28153cea4181dfd107a7c5d85d4af6cc1b13d13e7e1d03a7a9681"), +]; + +pub const FULL_U32: &[(u32, &str)] = &[ + (0, "0202000616273305183df820b21dbfbf0cbcd2217f4c4e68c4474cb07354b2454640fad8e4e11ebced9092b4ae437501723eff46f2b995d4f28f1445a7c8d0bebe7da7ab0eae78801ca14a17fb8426399b89c0cac3b16e1b34ac6f5de1ca872354ff74c3fc7c114c601898191f46e5830216445436978803697a1ce8adcc9f55ef1606c15f81f5403ea3269d45dfda2c5e97912bccad0b4ccb72d03b5a20193e294b299cad0417bfedb2"), + (1000, "020200061627333f0917f820b21dbfbf0cbcd2217f4c4e68c4474cb07354b2454640fad8e4e11ebced9092b4ae437501723eff46f2b995d4f28f394ea4dbd43735c8b62fe32e5ae755fdf7b7b60185f0d34573a13e26936f1fdcbaa7e0b56ed01e508fbe386b9dfdd8b098191f46e5830216445436978803697a1ce8adcc9f55ef1606c15f81f5403ea3269d45dfda2c5e97912bccad0b0c4bf26b9e5becddebbad0e6d25af40984992d"), + (4294967295, "02020006013626122b2050acd7182173d9675252473bc5aba2b58d7a00779a3f8157d945436c788f7f1a8408ce1f28c03f6a7659480d4f4757a19e7459919dbc7ee479b7ca6925b6fd3477f331f73459027e5cbd67ce587a52ba64230890141cdb20b5a5086c7490301598191f46e5830216445436978803697ae317123360aa10e964392ddb5994eff0aa5b2aff60bfa65e7e10550174c74a77970aa2b3c27fc0ab3c299314228fac3f"), +]; + +pub const FULL_I64: &[(i64, &str)] = &[ + (-9223372036854775808, "0202000b013f1f203f373e0f303c1a91ce523fe59cdb69838a28604e1576d6deea8b57c4981ef123ecc99ef95c080da53249a27c9326fa7505466975a806307a72070dc5a2d29e83ba85a85a28a7779534113016af87d2ada8191ee4265f1eacd65ee8e69eb4f4b93f328d6c44593dd1d0c6168fa5272f041fc58cf2f7959678c922a78215e28328d25462ba8c1a98b64e7e009b764aedf1e17ca5c598c620c348ac77eb8878a448bac86e710e790e3014f4cdf81f725819a708162219107898191f46e5830216445436978803697a59555385a79e997b73fbd4d68ecfa7eb78183fcc7ca6a8adffa32130f37278141330e15432698857a26759d44b6c444fc2e11b031b1a20d94edaa29e4279764dd9de35341e279b93ee1507febba91f53e804ed9dfa718b90"), + (-1, "0202000b0812273a2b2d2306132033aee5f4bb885c10491e37c26aa8248524468980d109a4f593b34d72f418b9a635b55d6c3e843f833ef9f7fac285d18a2861d70959512fa184d626d86c168880795f78e5a365f67a4d758c9101d23b2de20ddc4feaea6c48b23b342d039bb3181346a70695b4a01eb43a82c058c611d85d6845a29465096d534f56c992f5ce4a735083ae34fad7366a969f07f4c9709e249b56d82721b1713457df78a0c0b6ef2710034a0cf66b6cb61b7042cfdbc2336098191f46e5830216445436978803697aa92e521fbd353c93c0c806c3205cc2aac50ea9dc9ee6bf4634224d6e56b4a277d2f971f53064f02570af95315642779c833239f076304d1cd077e73a4d5c82b29bf4357d18477ba93cb32a1a37f7d276df28d40ec5765763"), + (0, "0202000b3c0629281c26020a231826de0aae25067a28821fd927c5564f204e9015325cfa69405c5cf25ba949a7653729eb1920bf0f0f9e500749e7795ae2d146e2cb11bcc7f66c4c3912a89d1242cba524d065f23a8ea9f57e73bf29780780e073c62a71f18b328e1f04c841519edc624e8ab956d5a826e283077c6d17edae52a8bbdd8c4ca66821b5b43e331e2d593d59488e49adfc54c44ad5006c051a1d8204b525a7ba5fd0ce2ea2dbcd948f86135ca2217bb282fe7a55a1b53707a48098191f46e5830216445436978803697aa92e521fbd353c8389ed22221b338d37ce480060054f33db4c489d855869cbdd94a1134aeb2ee9fef8df172d5abff1c33d3c13504eeb97d3c42b6e7f88f7333f89889b17ff0261d52c2949e409a5e6edfd6c625a0fdc45a8"), + (1, "0202000b3c0629281c26020a231812de0aae25067a28821fd927c5564f204e9015325cfa69405c5cf25ba949a7653729eb1920bf0f0f9e500749e7795ae2d146e2cb11bcc7f66c4c3912a89d1242cba524d065f23a8ea9f57e73bf29780780e073c62a71f18b328e1f04c841519edc624e8ab956d5a826e283077c6d17edae52a8bbdd8c4ca66821b5b43e331e2d593d59488e49adfc54c44ad5006c051a1d8204b525a7ba5fd0ce2ea2dbcd948f86501b8ea4fe62abe5371bff41f26d154d98191f46e5830216445436978803697aa92e521fbd353c8389ed22221b338d37ce480060054f33db4c489d855869cbdd94a1134aeb2ee9fef8df172d5abff1c33d3c13504eeb97d3c42b6e7f88f7333f89889b17ff0261d52c2949e409a5e6edfd6d667a0fde45a8"), + (9223372036854775807, "0202000b253a09372324371b10183ad8b7c3915268800457d42ebb622941859100c1f498dee877198108a1b345b71acd43a0d83d6f928534cb8e930973d1df1839911886aaeee67e7239880642b526c62d5970e0d7b3e4dbc71cc63d33042464626dfd79a16d3d053ec160d47b6b85574b5b33b5b057b1737e5eaa567d27a0244cfc87f0a4652b8b023f7c90d9e765f179d37e1118d09c21ac8660bccc3c31d4401164b05b76fa0096573295ac59e2b95f7940b7cae1c8bd1cea7756a56d5898191f46e5830216445436978803697aa4aaac7a586166844dfcaa5a36224b1e09ff5761f4551a1c13e9638983d5b1421a55607c8cfc489896a85545dc9da2992222dbe6b8d93050a4d4153bb6f28153cea4181dfd107a7c5d85d4af6cc1b13d13e7e1d03a7a9681"), +]; + +pub const FULL_F64_1_5: &str = "0202000b2a1e22383a0923042d353e7c82ff18a971479018c22f6b0208b8d7ba1cdf13403e7a7807660256dcadea6234f9b1b289dcc49195ea661074b607403b346c13012b44f566569da9a07919f1e31646e726448712f7d9e55c4c7b7c774405edad0e194ab436a9d4cf238145a7139f1b955e69f0085817326571524ca59f5eb3d2ffdb150100e5a4c5b9d8374251d14756d7663a0d7c1dd12f2d0d29d91f7d9b72a0fb14ba7deb340c679d3ef37d819b552b91f1e380f999a396e69fd998191f46e5830216445436978803697aa02e9e5bb8713e86e7684edcd4f6a002e9a3e8b3a8bd8721c1b359490904b1ec6e160f1002ff5da7f5cc4bcaf72bec850810fde5bc24d51ae665a9c99103cd18db694e79947abc877107e7cd55a747b5aeaa9e494e2f9827"; +pub const FULL_U64_456_SEED_B: &str = "0202000b013f1f203f373e0f301b3791ce523fe59cdb69838a28604e1576d6deea8b57c4981ef123ecc99ef95c080da53249a27c9326fa7505466975a806307a72070dc5a2d29e83ba85a85a28a7779534113016af87d2ada8191ee4265f1eacd65ee8e69eb4f4b93f328d6c44593dd1d0c6168fa5272f041fc58cf2f7959678c922a78215e28328d25462ba8c1a98b64e7e009b764aedf1e17ca5c598c620b366d3072e597605d5efc750025b7b3b532b80130be54f871b0bb2b7a8c87ac852f6f5b7a04d1eeff2d0f12880ed8269c283bf3e8b442991558a3288ca2bc5451004b39699c297339e1a6d6ad8628bdc320420f475738241ce18c9641a80f0e477c37fcda695a919a46100d01d5717bcee3a5306a3d16adbf6b000b1a4c6c9c08af3d264c07abbc0"; + +pub const FULL_U64_ONE_CIPHER: &[(u64, &str)] = &[ + (1161981756646125696, "0202000b323e111f0e3238383c1e2b243ebe2adc11163e7fc81576802ee75077e2f0a4bd5b71a9f024627f3bc7c3a17d69da0a561907ef559ce414f630de025f5c63736b93f32eacfd3f81a328a8e22a7952974519f434b8dab43c09a5f437f7bece1db93b7d7586dc30cb5a5015b44ad57b017034e713ba6364db8b2cd158784767284511dc3aa19023b0768ddefc13297cb40a0a5c6dad54a150164a5012bd0298f7ede2b451467a3af25a0a653385eba9b26f903602f52a87e2c34a4bbdb0da5d7f2e11289ffc66e452f8a554a23f866719bbdce738a6c53737d60d18179a4b51ce127def6a900001d12bd2827b21a9137fe1e3f2cc52ce831004b10a96e71ae223dddc51c8f8d9138b34b0e00f5e0884f9f8c3b068c0bc9336246086021ff0b6265e4b847b"), + (1161981756646125697, "0202000b323e111f0e3238383c1e12243ebe2adc11163e7fc81576802ee75077e2f0a4bd5b71a9f024627f3bc7c3a17d69da0a561907ef559ce414f630de025f5c63736b93f32eacfd3f81a328a8e22a7952974519f434b8dab43c09a5f437f7bece1db93b7d7586dc30cb5a5015b44ad57b017034e713ba6364db8b2cd158784767284511dc3aa19023b0768ddefc13297cb40a0a5c6dad54a150164a5012bd0298f7ede2b451467a3af25a0a6533c4c8930a7a37b6b95cd1b96c672283100c172dbb05ba9f2f06af649820e3217f1827c2e13e7311e245ec95481e47cdd96cdfcf4ddd424e3870452fd558150c42cd5ee2e9bdd70ed396f129e4acfb0367df169a909cae90081a8974a8d040f4823750391786c5a4c86fe7d2035cd1bdb682ad2a997896998e"), + (1161981757182996608, "0202000b323e111f0e03010503082e243ebe2adc11163e7fc81576802ee75077e2f0a4bd5b71a9f024627f3bc7c3a17d69da0a561907ef559ce414f630de025f5c63736b93f32eacfd3f81a328a8e22a7952974519f434b8dab43c09a5f437e8044023803898adcc685faa6d9df98e5ed82c90405db57bde891d22ad925b1cfd92c719448ea111cd2e81876002c18fc289ff4b641d9d7fa18943ae87b3539593ce9be9d8a566970abf9b9e94588d81c12e11f50787a14b2af806c25182617684418b90551c8d2c2f15737a3938daec53373debb0eb5c73ed27993113a95088723d64b2a1a0322deaad77ab779ca44e58982e23225c6a1fe1ce210f7b267b4020dc8023c309e2cb5a1767694b6beade68c8cf18eed38a524acd2ecf5581e09157138ddf555aa162"), + (1450212132797837440, "0202000b23142c0a223c0914213720611c553876b3bccf7599a93e1cb1db8d3bda0ce7b5b1f38edfd8d0a12f0255f4d344b5dfd1b86c08c57bd8900ddd0cb7f2c97f6693887816529ae50084964238608e73b25abc8e12ef7bbf74ae917dca5b6796987e331de8dea4be8b54ade6b4911e33e9fcd942f8c11d4d6cec6df4a5572bcc68c2f16eefd8d3b9487fa797957ae19f4eb7f4baacac3a99cdbfe91af88381fff377735b6eebe1574e8e255da36ae58ae5991c7dd497c171556711aa7cb38a3220da7270dfb56e24b5a1892ee057df9e31bb2766a17b0b114fca6fb7a69a58183337e5d2f1f6fcc1109d717b65d3bb4889a93a878cdcfa60aaf3dd4dc38e8ffcbdd730f83bdb36248173163c6e0beadafab37ac633063db44db0f3295f9bdf2a4c33b828dd"), +];