Perf/batch enum bit fields - #3
Open
Simon-Calbert-Aerospacelab wants to merge 2 commits into
Open
Conversation
Simon-Calbert-Aerospacelab
force-pushed
the
perf/derive-batch-bit-runs
branch
from
August 11, 2026 11:53
3fb93a9 to
370da83
Compare
Simon-Calbert-Aerospacelab
force-pushed
the
perf/batch-enum-bit-fields
branch
2 times, most recently
from
August 11, 2026 13:06
c0305a8 to
4f20248
Compare
Simon-Calbert-Aerospacelab
force-pushed
the
perf/derive-batch-bit-runs
branch
from
August 11, 2026 13:06
370da83 to
54def22
Compare
Simon-Calbert-Aerospacelab
force-pushed
the
perf/derive-batch-bit-runs
branch
from
September 2, 2026 16:19
6e004b1 to
0ad31f9
Compare
Simon-Calbert-Aerospacelab
force-pushed
the
perf/batch-enum-bit-fields
branch
from
September 3, 2026 14:19
4f20248 to
16088c6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The previous PR made a run of adjacent bit fields cost one call instead of one
per field, but only for plain integer fields. A field typed as an enum split the
run in two, because the derive expanding the struct sees only the token
OcfFlagand cannot know how wide it is.
That matters more than it sounds. Bit-packed headers are mostly flags, and a
protocol crate models a flag as an enum, not as a
u8. So the previous PR didits best work on the fields that occur least.
This PR closes that gap with a new trait,
DekuBitField, which puts the width onthe type. It is derived automatically, so flag enums get it with no annotation.
Stacked on #2.
The trait
DekuReadderives it for any enum whose variants are all unit variants, whoseidvalues are integer literals, and whoseid_typeis an unsigned primitive:Nothing was added to that enum. It already qualified.
Two details worth noting, because they made the implementation much smaller than
expected:
from_bit_runreuses the same match arms the reader already generates, soa batched read accepts and rejects exactly the ids an unbatched one does. The
error for an unassigned id is the same
DekuError::Parse.to_bit_rungoes through the existingDekuEnumExt::deku_id, which the derivehas emitted for a long time. The write side needed no new codegen at all.
The opt-in, and why it is unavoidable
When the derive expands a struct, it sees the token
OcfFlagand cannot askwhether that type implements a trait. Stable Rust has no specialization to fall
back on, and both arms of a
const-foldedifare type-checked, so "batch ifpossible" would fail to compile for anyone with an ordinary field.
So exactly one signal is needed, and this puts it in the cheapest place, the
container:
One attribute per struct. Nothing per field, nothing on the enums. If a field in
a run turns out not to implement the trait, the result is a clear compile error
pointing at the field:
What the derive emits
For a header whose first fields are
version(2 bits),id(10 bits) and anOcfFlag, the read becomes one call plus arithmetic:The widths are associated constants, so the shifts and masks are still
compile-time values; they are simply written as constant expressions rather than
literals.
from_bit_runmasks off the bits above its own, so the shift is allthe caller owes it.
On a real frame
The CCSDS TM primary header has 11 fields, five of them flag enums. Before this
PR the previous one batched it into 8 calls: a run of three, then the
ocfenum alone, then a run of two, then four enums, then a trailing 11-bit field with
no neighbour to pair with.
With
batch_bitsall eleven qualify, they sum to exactly 48 bits, and the headerbecomes one read and one write.
The 64-bit cap
A run cannot exceed the 64 bits one read returns. Where every width is known to
the macro it splits runs to fit. Where a width comes from the trait it cannot, so
it emits a constant assertion beside the impl:
That is emitted as an item rather than a statement in the read body on purpose.
Inside the body it only fired at the first use, because that function is
generic over the reader. As an item it fires at the definition.
Concretely, per file
src/lib.rs: theDekuBitFieldtrait, behind thebitsfeature.deku-derive/src/macros/deku_read.rs:bit_field_width(does this enumqualify),
emit_deku_bit_field(the impl), and the run planner extended so afield's width can come from the trait instead of from the macro.
deku-derive/src/macros/deku_write.rs: the dual, foldingto_bit_runintothe composed integer.
deku-derive/src/lib.rs: thebatch_bitscontainer attribute.src/attributes.rs: documentation with a worked example.Numbers
The same 11-field header is declared twice from one macro, once with the
attribute and once without, and both are benched in the same binary. So this is
a direct A/B, not two runs compared. Per header, from benches that push 128
frames through a single reader or writer:
Controls in the same session are unmoved: six plain byte fields read 4.77 ns
against 4.78 ns before, a lone 1-bit field 1.68 ns against 1.68 ns. A struct
without the attribute generates identical code.
On a production 9-protocol decode pipeline (DVB-S2, CCSDS TMTF, SDLS AES-GCM,
Space Packet, PUS, Encapsulation, IPoC, IPv4, UDP), adding the attribute to four
headers moved throughput from 728.7 to 798.0 MiB/s, +9.5%.
Testing
13 new tests, including a differential test that pushes 20,000 random 6-octet
headers through batched and unbatched versions of the same struct and asserts
every field, both error paths, and the re-encoded bytes agree. Full CI feature
matrix, plus all examples.
Separately, the batched output was checked against an independent set of
hand-written parsers for seven real protocol headers: 8.2M cases, zero
mismatches.