perf: read and write bit fields as integers - #1
Closed
Simon-Calbert-Aerospacelab wants to merge 13 commits into
Closed
perf: read and write bit fields as integers#1Simon-Calbert-Aerospacelab wants to merge 13 commits into
Simon-Calbert-Aerospacelab wants to merge 13 commits into
Conversation
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Federico Bergero <fbergero@amazon.com>
Simon-Calbert-Aerospacelab
force-pushed
the
perf/bit-fields-as-integers
branch
2 times, most recently
from
August 10, 2026 07:30
22e816f to
1f9ac63
Compare
Simon-Calbert-Aerospacelab
changed the base branch from
perf/be-bits-load-be
to
master
August 10, 2026 07:34
Co-authored-by: wcampbell <wcampbell1995@gmail.com> Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: wcampbell <wcampbell1995@gmail.com> Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: wcampbell <wcampbell1995@gmail.com> Signed-off-by: dependabot[bot] <support@github.com>
Simon-Calbert-Aerospacelab
force-pushed
the
perf/bit-fields-as-integers
branch
from
August 11, 2026 11:53
1f9ac63 to
81113a6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Reading a
#[deku(bits = N)]field currently goes through an intermediatebit-slice: deku copies the bits out of the input one slice operation at a time,
assembles them into a
BitSlice, and only then turns that into an integer.Writing does the mirror image. That intermediate is where nearly all the time
goes.
This PR skips it. For big-endian,
Order::Msb0fields only, the value isread straight into an integer and written straight out of one. No bit-slice is
built at all.
This does not replace sharksforarms#666
The two changes optimise the same idea on two different public traits, and
both stay reachable:
DekuRead::read(&BitSlice, ctx), the slice-level trait.That is the entry point for anyone who already holds a
BitSlice, and it iswhat deku's own delegating impls in
src/impls/primitive.rscall internally(the
ByteSizeand signed-integer wrappers all funnel into it).DekuReader::from_reader_with_ctx, the reader-leveltrait. Derived
from_readerand derivedfrom_bytesboth go through it,since
from_bytesbuilds aCursorand callsfrom_reader_with_ctx.Where both apply, this one returns first, so a derived big-endian
Msb0read nolonger reaches sharksforarms#666's line. But
DekuRead::readis public and still reached bydirect callers, hand-written impls, and deku's internal delegating impls, so
sharksforarms#666 keeps doing its job there. Same optimisation, two layers, neither made
dead by the other.
What it looks like on real bytes
Take a two-field header:
Given the input bytes
[0x2A, 0xB5], that is the bit stream00 1010101011 0101, soversion = 0andid = 683.Reading
versionconsumes byte0x2A, keeps its top 2 bits, and leaves theother 6 behind as the reader's "leftover". So both traces below start from:
Reading
idtodaySix steps, three bit-slice copies, and two
BoundedBitVecs built and dropped,to move 10 bits. (Step 6 is the one sharksforarms#666 already fixed: before that PR the
zero-extension was 6 separate
insert(0, false)calls, each shifting the whole16-bit array. sharksforarms#666 made it a single
load_be.)Reading
idwith this PRThe whole read is one
u128accumulator,acc, and a count of how many bitsare in it,
have.One byte fetched, one shift, one mask. Steps 1 to 5 of the old path are gone,
and no
BitSliceorBoundedBitVecis ever constructed.Writing
id = 683, afterversionhas queued 2 bitsSame accumulator, run backwards. Writing
versionemitted nothing, it only left2 pending bits (
00) waiting for the rest of the byte.That trailing
1011is the top half of0xB5, which is exactly where the nextfield continues. Read and write are mirror images of each other.
Step 1 is worth calling out on its own. Today that check scans a bit-slice with
first_oneto locate the highest set bit:Identical test. One walks a bit-slice, the other is a single
leading_zerosinstruction.
Why this matters more than it looks
The cost removed here is per field and flat. It does not depend on how many
bits the field holds, because it is the price of building and tearing down the
bit-slice machinery, not of moving the bits.
The measurements say exactly that. The CCSDS header has 11 bit fields and costs
723 ns, which is 66 ns per field. A single 1-bit field in a
u64costs 69 ns.Same number. A field that carries one bit pays what a field carrying eleven
does.
After this PR those become 23.8 ns for 11 fields (2.2 ns each) and 1.66 ns for
the single field. Still flat, just roughly 30 times smaller.
Concretely, per file
src/reader.rs: newReader::read_bits_uint_msb0(amt), the read shown above.src/writer.rs: newWriter::write_bits_uint_msb0(value, amt), the dual.Whole bytes leave in one
write_allrather than one call per byte.src/impls/primitive.rs:DekuReader::from_reader_with_ctxandDekuWriter::to_writerreturn early through those helpers when the field isbig-endian and
Msb0, plus the integer fit check.src/lib.rs: two small helpers to view a one-byteMsb0leftover as(byte, bit count).Little-endian and
Order::Lsb0are untouched and keep their existing code path.Numbers
Measured against
masterat 088018f, so this includes sharksforarms#657, sharksforarms#659, sharksforarms#661 and thesyn v3 upgrade. Per-item cost, from benches that push 128 frames through a
single reader or writer so nothing is overlapped or optimised away:
u64, readThe two control rows matter as much as the wins, since a change scoped to bit
fields should leave byte-aligned ones alone. Criterion agrees on the read
control, reporting "No change in performance detected".
The byte-aligned write control is too noisy on my machine to say anything
with. Re-running the unmodified
mastertwice against its own baseline, thesame benchmark reported +2.6% and then +13.3%, so its noise floor is at least
±13% here and the +8% on this branch sits inside it. Nothing in this PR touches
that path: the
writer.rschange is purely additive, and theprimitive.rschange is behind a big-endian
Msb0bit-field guard. Worth watching on CIhardware.
Reproduce with
cargo bench --bench bebits --all-features.Also in here
One commit improves
benches/bebits.rsitself: itblack_boxes the inputs(without it the compiler can see the buffer contents and fold part of the read
away, which made the old numbers look better than they were) and adds benches
that read or write a stream of frames rather than one struct per iteration.
Testing
cargo testacross the full CI feature matrix (default,--all-features,--no-default-featuresand each individual feature) plus all examples. No testchanges were needed: the fast path is required to produce bit-identical results
to the path it replaces.