ORE v2: variable-length & string ORE, v2 wire format, SIMD + constant-time hardening (integration branch) by coderdan · Pull Request #90 · cipherstash/ore.rs

coderdan · 2026-06-16T10:17:36Z

ORE v2 — the next generation of order-revealing encryption

This is the integration / forthcoming-release branch for the ORE v2 program. It
is the umbrella PR that the stacked feature PRs land into; once the stack is merged
and the remaining crypto-review gates clear, this branch ships the whole v2 change
set to main.

Architecture & rationale: docs/plans/2026-06-12-ore-v2-architecture.md.

Why v2

ORE v1 is the Lewi–Wu block-ORE scheme at an 8-bit block width (Bit8), fixed to
a single wire format and a single fixed-length numeric domain. v2 keeps that scheme
byte-for-byte (it stays the wire-frozen default) and builds a broader, harder, and
more flexible foundation around it:

Variable-length & string ORE. A new chained-prefix scheme encrypts
arbitrary-length values — strings (lexicographic order matching str::cmp),
and a path to u128/Decimal — not just fixed-width integers.
A real v2 wire format. A versioned header (scheme id + parameters) so multiple
schemes and block widths coexist and evolve without ambiguity. v1 ciphertexts are
untouched; v2 is additive.
Block width as a leakage decision. Bit6 (6-bit blocks) joins Bit8 as an opt-in
width. Width is a per-domain/per-deployment trade-off (leakage vs cost), not a
global default — default numerics stay on Bit8 (lower online prefix leakage +
wire compatibility); Bit6 is opt-in for at-rest-dominated / size-sensitive cases.
Performance. Hardware AES on aarch64, NEON + AVX2 kernels for the
right-ciphertext encoder, an allocation-free encrypt path, and a vectorised
GF(2¹²⁸) σ-doubling in the hash. Numbers below.
Security hardening, with receipts. A plaintext-dependent PRP timing channel
closed, the comparator made constant-time at cache-line (and MemJam sub-line)
granularity, key-derived material zeroized on drop, and a fixed-key-AES MMO hash
adopted for the 1-bit hash — each backed by a written review on this branch.

What's in v2 (delivered via the stacked PRs)

The feature work lands as a stack that merges into this branch:

ORE v2 (1/n): compatibility vectors + bench baselines #78 — compatibility vectors + bench baselines (pins v1 behaviour before refactor)
ORE v2 (2/n): core refactor — width abstraction, seed/tag separation, template RO keys #79 — core refactor: width abstraction, seed/tag separation, template RO keys
ORE v2 (3/n): efficient unary encoding + hardware AES on aarch64 #80 — efficient unary encoding + allocation-free encrypt + hardware AES on aarch64
ORE v2 (4/n): NEON + AVX2 SIMD backends #81 — NEON + AVX2 SIMD backends for the indicator / hash-bit masks
ORE v2 (5/n): 6-bit block scheme + v2 wire format #82 — Bit6 6-bit block scheme + v2 wire header
ORE v2 (6/n): chained-prefix variable-length scheme + string encryption #83 — chained-prefix variable-length scheme + string encryption (CMAC accumulator)

Crypto building blocks introduced along the way:

σ-MMO 1-bit hash (FixedPiZ2Hash): H(x,r) = LSB(π(σ(x)⊕r) ⊕ σ(x)⊕r) with π
a fixed public-key AES-128 permutation (nothing-up-my-sleeve PI_KEY) and σ a
GF(2¹²⁸) doubling — the BHKR/GKWY fixed-key-AES MMO construction. Security rests on
AES as a public random permutation, so the comparator needs no key material.
AES-CMAC accumulator (NIST SP 800-38B validated): incremental prefix-block /
final-block absorption for the variable-length scheme. Encryption-side only — the
comparator never evaluates CMAC.
LemireFyPrp (fixed-draw Fisher–Yates, wide 64-bit draws + Lemire
multiply-high): replaces the rejection-sampled Knuth shuffle for the new schemes,
closing a timing channel where the rejection draw-count depended on the
plaintext-derived seed. (Bit8 stays on the Knuth shuffle — wire-frozen; the channel
is documented, not changed.)

Performance

The headline: encrypting a value is ~1.6× faster on the v1-compatible format, and
up to ~4.6× faster on the new Bit6 scheme. Comparison — the operation that runs
server-side at query time — was already fast and is essentially unchanged.

Same workload (encrypt a u64), same machine (Apple M1 Max), hardware AES on for
both so this isolates the rewrite from the backend:

	encrypt a u64	vs v1
v1 (Bit8)	~39 µs	—
v2, same wire-compatible scheme (Bit8)	~25 µs	~1.6×
v2, new opt-in scheme (Bit6)	~8.6 µs	~4.6×

Separately — hardware AES is now on by default. On Apple Silicon the aes crate
needs --cfg aes_armv8 for ARMv8 hardware AES; without it (the old default) every block
ran in software, ~60× slower per block — worth ~10× end to end. v2 sets it for workspace
builds (downstream crates must set it themselves, announced on release). So "old default
as it actually ran" (≈381 µs) vs v2 is ~15× (Bit8) / ~44× (Bit6) — but most of that first
~10× is the flag, not the rewrite, so it shouldn't be read as a speedup from the new code.

Comparison stays flat and fast: ~230 ns at Bit8, ~180 ns at Bit6, ~400 ns for a
23-block string — a constant-time prefix scan plus one hash eval and one oblivious block
read. The rewrite didn't target it; hardware AES alone took it from ~740 ns to ~230 ns.

Criterion medians on a developer machine — indicative, not a regression gate. Full
baseline, per-PR deltas, the SIMD kernels, and the variable-length string scaling
(~0.69 µs/block) are in docs/benchmarks/.

Security reviews (carried on this branch)

docs/reviews/2026-06-16-*:

Zeroize audits (Trail of Bits zeroize-audit) of ORE v2 (2/n): core refactor — width abstraction, seed/tag separation, template RO keys #79 / ORE v2 (5/n): 6-bit block scheme + v2 wire format #82 / ORE v2 (6/n): chained-prefix variable-length scheme + string encryption #83. ZA-0001
(legacy Aes128Prng keystream not wiped on drop) was found, fixed (fix(prp): zeroize Aes128Prng keystream on drop (ZA-0001) #84, already on
main), and inherited here; the new v2 code is clean.
Constant-time analysis (ORE v2 (2/n): core refactor — width abstraction, seed/tag separation, template RO keys #79 sweep, with v2 applicability): the two genuinely
actionable legacy findings — encrypt-timing (CT-1) and the compare-side cache leak
(CT-4) — do not apply to v2 (the LemireFyPrp switch and ct_select_byte
hardening were built to close exactly those). What remains in v2 (CT-2/CT-3) is
structural-but-mitigated to the MemJam sub-line residual.
pl/pgSQL comparator validation: the v2 σ-MMO comparison is implementable in
pl/pgSQL + pgcrypto (12/12 vectors match Rust) — relevant because the interim
server-side comparator can't depend on CMAC, and it doesn't (CMAC is encryption-side).

CI (test.yml) on this branch runs on every pull request (not only PRs into
main), so the whole stack gets GitHub checks while it targets feature branches.

Backwards compatibility

v1 (Bit8) is the wire-frozen default and is unchanged byte-for-byte (the #78
compatibility vectors pin it). v2 is purely additive — new schemes, new widths, new
header — selected explicitly. (A follow-up will split the scheme trait so wire-format
back-compat is enforced at the type level.)

Status

Draft. This is the aggregation point for #78–#83. Remaining gates before it can go
to main: the stack merges in, the A2/A3 CMAC + PRP-shape crypto review clears, and
the Bit6/chained wire vectors are pinned. Flip to ready once those land.

Copilot

🟡 Changes recommended

The audits miss key-derived temporaries, and the PostgreSQL validation overstates comparator coverage while conflicting with the documented LSB convention.

4 open findings

Correct LSB byte selection or clarify output coordinate · New
Include local draw buffer in keystream wiping · New
Wipe temporary accumulator-state copies · New
Add end-to-end comparator validation · New

What changed in this PR

Adds security-review evidence and CI support for the ORE v2 integration branch.

Changes:

Documents zeroization and constant-time audits.
Adds PostgreSQL σ-MMO validation artifacts.
Enables CI for pull requests targeting any branch.

File	Description
`.github/workflows/test.yml`	Runs CI on all pull requests.
`docs/reviews/2026-06-16-zeroize-audit-pr79.md`	Records the PR #79 zeroization audit.
`docs/reviews/2026-06-16-zeroize-audit-pr82.md`	Records the Bit6 zeroization audit.
`docs/reviews/2026-06-16-zeroize-audit-pr83.md`	Records the chained-scheme zeroization audit.
`docs/reviews/2026-06-16-constant-time-analysis-pr79.md`	Documents constant-time findings and v2 applicability.
`docs/reviews/2026-06-16-mmo-plpgsql-validation.sql`	Implements and checks PostgreSQL σ-MMO hashing.
`docs/reviews/2026-06-16-mmo-plpgsql-validation.md`	Summarizes PostgreSQL validation results.

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

absorb and finalize each built an owned value from the chain state (and, in finalize, K1) and returned without wiping it. Both now hold it as a byte array and zeroize it after the AES call. encrypt takes its block by value and encrypts it in place, so that copy holds only the output. Raised in the review of #90 against the #83 zeroize audit.

Review of #90: absorb and finalize built an owned `mixed` value from the chain state (and, in finalize, K1) and handed a by-value copy to the cipher, leaving the plaintext input on the stack. Encrypting `mixed` in place closes it without a temporary: the one buffer that held the input holds the new state or the tag afterwards. A per-call zeroize was tried first and cost 45% on chained encryption (about 97 calls per block, each a volatile store and a fence that stops the AES ops pipelining); the in-place form costs nothing.

Fisher-Yates swapped at the draw-derived index and filled the inverse at the permutation's values. Both offsets stay inside one 64-byte line, so the cache cannot see them, but dudect on Apple M4 measured the build time depending on which permutation is built, with no co-tenant: the swaps carry the signal and the inverse fill alone shows none. That fits store-to-load forwarding and memory disambiguation on byte-granular accesses at secret offsets, which the one-cache-line argument does not cover. The builder now keeps both tables in registers and never uses a secret value as an address: perm[j] is fetched by a table lookup on a broadcast j and written back by compare-and-select, and the inverse is kept by exchanging the values i and j in it at each step, which needs neither swapped entry. Step i is public and j <= i, so only the chunks covering 0..=i are searched. Same 63 Lemire draws and swap sequence, so the tables, and every ciphertext, are byte-identical; the Bit6 vectors pass unchanged. Dispatch: NEON on aarch64 (baseline, no runtime check), SSSE3 (pshufb) on x86_64 when the CPU reports it, a u64 SWAR form with exact byte-equality masks everywhere else. The textbook builder survives only as a test-only reference, with a byte-at-a-time subtle_ng form; property and edge-stream tests pin every builder's tables to it. Each draw is decoded from an 8-byte copy of the keystream into a u64; both copies are wiped, as the caller wipes the stream itself. Raised in the review of #90 against the #82 zeroize audit.

Review of #90: absorb and finalize built an owned `mixed` value from the chain state (and, in finalize, K1) and handed a by-value copy to the cipher, leaving the plaintext input on the stack. Encrypting `mixed` in place closes it without a temporary: the one buffer that held the input holds the new state or the tag afterwards. A per-call zeroize was tried first and cost 45% on chained encryption (about 97 calls per block, each a volatile store and a fence that stops the AES ops pipelining); the in-place form costs nothing.

coderdan · 2026-10-09T09:31:25Z

Force-pushed (with lease) a two-line addition to docs/reviews/2026-06-16-constant-time-analysis-pr79.md, folded into f22cb12's history at the security review records commit: a note under the status table that the CT-2 and CT-3 residuals are superseded, pointing to the crypto review brief's A4 "Status (2026-10-09)" (#82). The rest of the stack (#78–#83, #100) was rebased onto it with no other change; each branch differs from its previous tip only by those two lines.

- The MMO validation claimed a pl/pgSQL comparator was validated; only the σ-MMO hash was implemented and checked (12 vectors). The doc now scopes its result to the hash and lists what a comparator still needs end-to-end vectors for. - "LSB" in both the SQL and Rust is bit 0 of byte 0 of the block (bit 120 of the big-endian field element), as in the frozen legacy hash. The SQL and summary now name that coordinate. - The #82 and #83 zeroize audits marked items clean that left key-derived temporaries unwiped (from_stream's per-draw copy; the CMAC mixed values). Both audits are amended, pointing at the fixes on #82 and #83, and say plainly that the wipes are source-level only.

Fisher-Yates swapped at the draw-derived index and filled the inverse at the permutation's values. Both offsets stay inside one 64-byte line, so the cache cannot see them, but dudect on Apple M4 measured the build time depending on which permutation is built, with no co-tenant: the swaps carry the signal and the inverse fill alone shows none. That fits store-to-load forwarding and memory disambiguation on byte-granular accesses at secret offsets, which the one-cache-line argument does not cover. The builder now keeps both tables in registers and never uses a secret value as an address: perm[j] is fetched by a table lookup on a broadcast j and written back by compare-and-select, and the inverse is kept by exchanging the values i and j in it at each step, which needs neither swapped entry. Step i is public and j <= i, so only the chunks covering 0..=i are searched. Same 63 Lemire draws and swap sequence, so the tables, and every ciphertext, are byte-identical; the Bit6 vectors pass unchanged. Dispatch: NEON on aarch64 (baseline, no runtime check), SSSE3 (pshufb) on x86_64 when the CPU reports it, a u64 SWAR form with exact byte-equality masks everywhere else. The textbook builder survives only as a test-only reference, with a byte-at-a-time subtle_ng form; property and edge-stream tests pin every builder's tables to it. Each draw is decoded from an 8-byte copy of the keystream into a u64; both copies are wiped, as the caller wipes the stream itself. Raised in the review of #90 against the #82 zeroize audit.

Review of #90: absorb and finalize built an owned `mixed` value from the chain state (and, in finalize, K1) and handed a by-value copy to the cipher, leaving the plaintext input on the stack. Encrypting `mixed` in place closes it without a temporary: the one buffer that held the input holds the new state or the tag afterwards. A per-call zeroize was tried first and cost 45% on chained encryption (about 97 calls per block, each a volatile store and a fence that stops the AES ops pipelining); the in-place form costs nothing.

Review/validation artifacts from the v2 crypto review (zeroize audits for the ZA-0001 fix and PRs 82/83, the constant-time v2-applicability analysis, and the pl/pgSQL + pgcrypto validation of the v2 sigma-MMO comparison). ore-v2 is the forthcoming v2 release/integration branch; the stack rebases on top of this.

Drops the `branches: [main]` filter on the pull_request trigger so stacked PRs that target a feature/integration branch (the ore-v2 stack) run CI too. Lands on ore-v2 first so the whole rebased stack inherits it.

- The MMO validation claimed a pl/pgSQL comparator was validated; only the σ-MMO hash was implemented and checked (12 vectors). The doc now scopes its result to the hash and lists what a comparator still needs end-to-end vectors for. - "LSB" in both the SQL and Rust is bit 0 of byte 0 of the block (bit 120 of the big-endian field element), as in the frozen legacy hash. The SQL and summary now name that coordinate. - The #82 and #83 zeroize audits marked items clean that left key-derived temporaries unwiped (from_stream's per-draw copy; the CMAC mixed values). Both audits are amended, pointing at the fixes on #82 and #83, and say plainly that the wipes are source-level only.

freshtonic

Approved after the round-2 security re-review. The round-1 findings for this PR are fixed or have no remaining blocking items.

🤖 Generated with Claude Code

Review of #90: absorb and finalize built an owned `mixed` value from the chain state (and, in finalize, K1) and handed a by-value copy to the cipher, leaving the plaintext input on the stack. Encrypting `mixed` in place closes it without a temporary: the one buffer that held the input holds the new state or the tag afterwards. A per-call zeroize was tried first and cost 45% on chained encryption (about 97 calls per block, each a volatile store and a fence that stops the AES ops pipelining); the in-place form costs nothing.

Fisher-Yates swapped at the draw-derived index and filled the inverse at the permutation's values. Both offsets stay inside one 64-byte line, so the cache cannot see them, but dudect on Apple M4 measured the build time depending on which permutation is built, with no co-tenant: the swaps carry the signal and the inverse fill alone shows none. That fits store-to-load forwarding and memory disambiguation on byte-granular accesses at secret offsets, which the one-cache-line argument does not cover. The builder now keeps both tables in registers and never uses a secret value as an address: perm[j] is fetched by a table lookup on a broadcast j and written back by compare-and-select, and the inverse is kept by exchanging the values i and j in it at each step, which needs neither swapped entry. Step i is public and j <= i, so only the chunks covering 0..=i are searched. Same 63 Lemire draws and swap sequence, so the tables, and every ciphertext, are byte-identical; the Bit6 vectors pass unchanged. Dispatch: NEON on aarch64 (baseline, no runtime check), SSSE3 (pshufb) on x86_64 when the CPU reports it, a u64 SWAR form with exact byte-equality masks everywhere else. The textbook builder survives only as a test-only reference, with a byte-at-a-time subtle_ng form; property and edge-stream tests pin every builder's tables to it. Each draw is decoded from an 8-byte copy of the keystream into a u64; both copies are wiped, as the caller wipes the stream itself. Raised in the review of #90 against the #82 zeroize audit.

coderdan mentioned this pull request Oct 8, 2026

Tracking: review and release chained block ORE (ore v2) so text can use block ORE #95

Open

3 of 9 tasks

coderdan force-pushed the ore-v2 branch from 8c60539 to d4d2a9b Compare October 8, 2026 08:09

coderdan marked this pull request as ready for review October 8, 2026 08:49

coderdan requested review from cipherstash-bot and a balanced review from Copilot October 8, 2026 08:52

Copilot started reviewing on behalf of coderdan October 8, 2026 09:22 View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

Comment thread docs/reviews/2026-06-16-mmo-plpgsql-validation.sql

Comment thread docs/reviews/2026-06-16-zeroize-audit-pr82.md Outdated

Comment thread docs/reviews/2026-06-16-zeroize-audit-pr83.md

Comment thread docs/reviews/2026-06-16-mmo-plpgsql-validation.md Outdated

coderdan requested a review from freshtonic October 8, 2026 23:40

coderdan force-pushed the ore-v2 branch from f76a4dc to f22cb12 Compare October 9, 2026 09:31

freshtonic mentioned this pull request Oct 11, 2026

verify: Valgrind taint, fuzz targets, Kani harnesses and dudect for ORE v2 #100

Merged

coderdan added 3 commits October 12, 2026 12:00

ci: run tests on all pull requests, not only those targeting main

7b6317f

Drops the `branches: [main]` filter on the pull_request trigger so stacked PRs that target a feature/integration branch (the ore-v2 stack) run CI too. Lands on ore-v2 first so the whole rebased stack inherits it.

coderdan force-pushed the ore-v2 branch from f22cb12 to 85323ea Compare October 12, 2026 01:05

freshtonic approved these changes Oct 12, 2026

View reviewed changes

coderdan merged commit 463e5fa into main Oct 12, 2026
2 checks passed

coderdan deleted the ore-v2 branch October 12, 2026 08:34

Navigation Menu

Repository navigation

ORE v2: variable-length & string ORE, v2 wire format, SIMD + constant-time hardening (integration branch) - #90

ORE v2: variable-length & string ORE, v2 wire format, SIMD + constant-time hardening (integration branch)#90
coderdan merged 3 commits into
mainfrom
ore-v2

coderdan commented Jun 16, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderdan commented Oct 9, 2026

Uh oh!

freshtonic left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

coderdan commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ORE v2 — the next generation of order-revealing encryption

Why v2

What's in v2 (delivered via the stacked PRs)

Performance

Security reviews (carried on this branch)

Backwards compatibility

Status

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderdan commented Oct 9, 2026

Uh oh!

freshtonic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coderdan commented Jun 16, 2026 •

edited

Loading