Repository navigation
ORE v2 (1/n): compatibility vectors + bench baselines #78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
e488ab6
test: pin compatibility vectors and bench baseline for ORE v2 refactor
coderdan 5a8c07e
style: rustfmt
coderdan 97bf089
test(compat_vectors): name the generator's real output path
coderdan d82bd5b
docs(plan): specify unsigned byte compares for the SIMD indicator mask
coderdan aadae59
bench: black-box every result and input, and record the effect
coderdan cad46bb
test(compat): pin cross-nonce comparison fixtures
coderdan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # Benchmark baseline — main @ 3f92c45 | ||
|
|
||
| Recorded as the reference point for the ORE v2 performance work | ||
| (`docs/plans/2026-06-12-ore-v2-architecture.md`, PR 1). Criterion baseline | ||
| name: `main-baseline` (in `target/criterion`, not committed — these numbers | ||
| are the durable record). | ||
|
|
||
| **Machine:** Apple M1 Max (aarch64, NEON + ARMv8 crypto extensions), macOS, | ||
| rustc 1.87.0, `cargo bench --bench oreaes128`. | ||
|
|
||
| | Benchmark | Median | Notes | | ||
| |---|---|---| | ||
| | encrypt-8 (u64) | 381.1 µs | full Left+Right, 8 blocks | | ||
| | encrypt-left-8 (u64) | 91.1 µs | Left only | | ||
| | compare-8 | 738.8 ns | typed `Ord` | | ||
| | compare-8-slice | 743.5 ns | `compare_raw_slices` | | ||
| | serialize-8 | 472.5 ns | | | ||
| | deserialize-8 | 34.0 ns | | | ||
| | encrypt-4 (u32) | 187.2 µs | | | ||
| | encrypt-left-4 (u32) | 45.1 µs | | | ||
| | compare-4 | 671.8 ns | | | ||
|
|
||
| ## Observations | ||
|
|
||
| - Per-block costs are ~11 µs for the Left path (PRP setup dominated: Knuth | ||
| shuffle with rejection-sampled PRNG, 512-byte permutation buffers zeroized | ||
| per drop) and ~36 µs for the Right path on top of that. The raw AES work | ||
| per block (~512 batched encryptions) should be well under 1 µs on this | ||
| hardware, so the overwhelming majority of encrypt time is construction | ||
| overhead around AES, not AES itself — consistent with the v2 plan's cost | ||
| model and a large headroom signal for PRs 3–5. | ||
| - `serialize-8` at ~472 ns reflects the per-block `Vec` allocations in | ||
| `to_bytes`; not a target of this program but cheap to improve in passing. | ||
|
|
||
| ## Addendum (2026-10-08): result elimination check | ||
|
|
||
| The benchmark helpers in `benches/oreaes128.rs` returned `()`, so no result | ||
| reached Criterion's `black_box` and the inputs were literal constants. That | ||
| permits the optimiser to elide work whose result is unused. The helpers now | ||
| return their results and the inputs are black-boxed. | ||
|
|
||
| The M1 Max above is not available to re-record on, so the effect was measured | ||
| as an A/B on one machine instead (Apple M4, same commit, `--warm-up-time 2 | ||
| --measurement-time 5`, old helpers saved as a Criterion baseline, then the | ||
| fixed helpers compared against it): | ||
|
|
||
| | Benchmark | Old helpers | Fixed helpers | Change | | ||
| |---|---|---|---| | ||
| | encrypt-8 | 475.3 µs | 479.8 µs | +0.9% | | ||
| | encrypt-left-8 | 107.2 µs | 111.3 µs | +3.3% | | ||
| | compare-8 | 1.023 µs | 1.023 µs | no change (p = 0.33) | | ||
| | compare-8-slice | 1.028 µs | 1.030 µs | +0.4% | | ||
| | serialize-8 | 658.4 ns | 656.9 ns | +0.5% | | ||
| | deserialize-8 | 72.2 ns | 133.6 ns | **+82.8%** | | ||
| | encrypt-4 | 236.2 µs | 238.3 µs | +0.8% | | ||
| | encrypt-left-4 | 53.6 µs | 55.8 µs | +4.0% | | ||
| | compare-4 | 922.3 ns | 931.6 ns | no change (p = 0.35) | | ||
|
|
||
| The encrypt, compare and serialize figures above are not materially affected: | ||
| the 0–4% moves are the cost of producing and dropping each result inside the | ||
| timed loop. **`deserialize-8` was under-measured by about 1.8×**: scaled by the | ||
| ratio measured here, roughly 60 ns on the M1 Max rather than the 34 ns | ||
| recorded above. Later PRs' benchmark results | ||
| were taken with the old helpers; the same caveat applies to their | ||
| deserialization figures only. | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.