perf(derive): batch runs of adjacent big-endian bit fields - #2
Closed
Simon-Calbert-Aerospacelab wants to merge 13 commits into
Closed
perf(derive): batch runs of adjacent big-endian bit fields#2Simon-Calbert-Aerospacelab wants to merge 13 commits into
Simon-Calbert-Aerospacelab wants to merge 13 commits into
Conversation
Simon-Calbert-Aerospacelab
force-pushed
the
perf/derive-batch-bit-runs
branch
from
August 7, 2026 14:00
175c5d3 to
9ab0207
Compare
Simon-Calbert-Aerospacelab
force-pushed
the
perf/bit-fields-as-integers
branch
2 times, most recently
from
August 10, 2026 07:30
22e816f to
1f9ac63
Compare
Simon-Calbert-Aerospacelab
force-pushed
the
perf/derive-batch-bit-runs
branch
from
August 10, 2026 08:02
9ab0207 to
3fb93a9
Compare
Simon-Calbert-Aerospacelab
force-pushed
the
perf/derive-batch-bit-runs
branch
from
August 11, 2026 11:53
3fb93a9 to
370da83
Compare
Simon-Calbert-Aerospacelab
force-pushed
the
perf/bit-fields-as-integers
branch
from
August 11, 2026 11:53
1f9ac63 to
81113a6
Compare
Updates the requirements on [criterion](https://github.com/criterion-rs/criterion.rs) to permit the latest version. - [Release notes](https://github.com/criterion-rs/criterion.rs/releases) - [Changelog](https://github.com/criterion-rs/criterion.rs/blob/master/CHANGELOG.md) - [Commits](criterion-rs/criterion.rs@criterion-plot-v0.7.0...criterion-v0.8.2) --- updated-dependencies: - dependency-name: criterion dependency-version: 0.8.2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: wcampbell <wcampbell1995@gmail.com>
Simon-Calbert-Aerospacelab
force-pushed
the
perf/derive-batch-bit-runs
branch
from
August 11, 2026 13:06
370da83 to
54def22
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The previous PR #1 made one bit field cheap to read and write. This makes a run
of adjacent bit fields cost one call instead of one call per field.
The derive now detects a maximal run of neighbouring fields that a single read
can serve, emits one
read_bits_uint_msb0for the whole run, and hands eachfield its bits with a shift and a mask. Writing is the mirror: each field is
shifted into place, OR'd together, and sent out in one
write_bits_uint_msb0.Stacked on #1. It calls the helpers that PR introduced, so it needs
that one first.
What the derive emits
Take the same header as the previous PR:
Today the derive emits one read per field. Two reads, each with its own
leftover bookkeeping.
With this PR the two fields are recognised as a run of 2 + 10 = 12 bits:
On the input
[0x2A, 0xB5]:The shift for each field is "how many bits sit below it in the run", and the
mask is its own width. Both are compile-time constants, so each field extraction
is two instructions on a register.
The write side is the same arithmetic backwards:
On a real frame
The CCSDS TM primary header in the benchmark has 11 fields of
2, 10, 3, 1, 8, 8, 1, 1, 1, 2, 11 bits. That sums to exactly 48, so all eleven
become a single run: one 48-bit read, then eleven shift-and-mask pairs. The
same header goes out in one write.
Worth stating plainly: the two 8-bit fields in that header carry no
bitsattribute at all. A plain big-endian integer field is exactly
bits = width, soplain byte fields join runs too. That is deliberate and it is why the
byte-aligned benchmark below speeds up, even though the previous PR left that
path untouched. If you would rather this were restricted to fields that
explicitly declare
bits = N, that is a one-line change torun_field.When a run is not formed
This is the part worth reviewing closely. A field joins a run only if all of
these hold:
bits = Nwith a literalN, or nobitsattribute at all (thenNisthe type's width). An expression is rejected, since the width must be known at
expansion time.
u8,u16,u32oru64attribute). Absent means the target's endianness, which is little on x86, so
absent is rejected.
Msb0(absent is fine, since that is the default)count,until,cond,map,ctx,reader,writer,skip,temp,assert,magic, everypad_*,every
seek_*,read_all,bytes,bits_read,bytes_read,updateThe last rule is the safety argument. Each of those attributes either moves the
cursor, makes the read conditional, or depends on a value read earlier, and any
of the three breaks the "one contiguous read" assumption a run depends on. Rather
than reason about which combinations are safe, anything unusual keeps its own
per-field read.
Two more limits:
read_bits_uint_msb0returnsfield keeps its existing single-field path
Anything a run cannot serve is unchanged, so this is purely additive.
Concretely, per file
deku-derive/src/macros/deku_read.rs:run_field(does this fieldqualify),
plan_bit_runs(group adjacent qualifying fields into maximal runs),emit_bit_run_read(emit the single read plus the shift-and-mask extraction).deku-derive/src/macros/deku_write.rs:emit_bit_run_write, the dual.It emits a
check_bit_sizeper field so the "value too wide forbits = N"error is still raised, and skips that check where a field fills its container
and therefore cannot overflow it.
src/reader.rs/src/writer.rs: the helpers from the previous PR madepubso generated code can call them, pluscheck_bit_size.Numbers
Measured against this PR's base, the previous PR, so this is the incremental
gain. Per-item cost from benches that push 128 frames through a single reader or
writer:
u64, readThe last row is the control. A single field cannot form a run, so it takes the
unchanged path, and criterion agrees: "No change in performance detected"
(p = 0.08).
Cumulative over both PRs, against
masterat 088018f for the same header:read 723 ns to 5.74 ns (126x), write 534 ns to 17.9 ns (30x). A 1-bit field
alone in a
u64goes from 69.1 ns to 1.68 ns (41x), all of that from theprevious PR.
Reproduce with
cargo bench --bench bebits --all-features.Testing
cargo testacross the full CI feature matrix (default,--all-features,--no-default-featuresand each individual feature) plus all examples. No testchanges were needed: a batched run is required to produce bit-identical results
to the per-field reads it replaces, so the existing suite is the check.