Skip to content

Perf/batch enum bit fields - #3

Open
Simon-Calbert-Aerospacelab wants to merge 2 commits into
perf/derive-batch-bit-runsfrom
perf/batch-enum-bit-fields
Open

Perf/batch enum bit fields#3
Simon-Calbert-Aerospacelab wants to merge 2 commits into
perf/derive-batch-bit-runsfrom
perf/batch-enum-bit-fields

Conversation

@Simon-Calbert-Aerospacelab

@Simon-Calbert-Aerospacelab Simon-Calbert-Aerospacelab commented Aug 10, 2026

Copy link
Copy Markdown
Owner

What this changes

The previous PR made a run of adjacent bit fields cost one call instead of one
per field, but only for plain integer fields. A field typed as an enum split the
run in two, because the derive expanding the struct sees only the token OcfFlag
and cannot know how wide it is.

That matters more than it sounds. Bit-packed headers are mostly flags, and a
protocol crate models a flag as an enum, not as a u8. So the previous PR did
its best work on the fields that occur least.

This PR closes that gap with a new trait, DekuBitField, which puts the width on
the type. It is derived automatically, so flag enums get it with no annotation.

Stacked on #2.

The trait

pub trait DekuBitField: Sized {
    const BITS: usize;
    fn from_bit_run(raw: u64) -> Result<Self, DekuError>;
    fn to_bit_run(&self) -> Result<u64, DekuError>;
}

DekuRead derives it for any enum whose variants are all unit variants, whose
id values are integer literals, and whose id_type is an unsigned primitive:

#[derive(DekuRead, DekuWrite)]
#[deku(id_type = "u8", bits = 1, endian = "endian", ctx = "endian: deku::ctx::Endian")]
enum OcfFlag {
    #[deku(id = 0b0)]
    Absent,
    #[deku(id = 0b1)]
    Present,
}

Nothing was added to that enum. It already qualified.

Two details worth noting, because they made the implementation much smaller than
expected:

  • from_bit_run reuses the same match arms the reader already generates, so
    a batched read accepts and rejects exactly the ids an unbatched one does. The
    error for an unassigned id is the same DekuError::Parse.
  • to_bit_run goes through the existing DekuEnumExt::deku_id, which the derive
    has emitted for a long time. The write side needed no new codegen at all.

The opt-in, and why it is unavoidable

When the derive expands a struct, it sees the token OcfFlag and cannot ask
whether that type implements a trait. Stable Rust has no specialization to fall
back on, and both arms of a const-folded if are type-checked, so "batch if
possible" would fail to compile for anyone with an ordinary field.

So exactly one signal is needed, and this puts it in the cheapest place, the
container:

#[deku(endian = "big", batch_bits)]
pub struct TmtfPrimaryHeader { /* fields unchanged */ }

One attribute per struct. Nothing per field, nothing on the enums. If a field in
a run turns out not to implement the trait, the result is a clear compile error
pointing at the field:

error[E0277]: the trait bound `Sub: DekuBitField` is not satisfied
  --> src/lib.rs:12:10
   |
12 |     sub: Sub,
   |          ^^^ unsatisfied trait bound

What the derive emits

For a header whose first fields are version (2 bits), id (10 bits) and an
OcfFlag, the read becomes one call plus arithmetic:

let run: u64 = __deku_reader.read_bits_uint_msb0(2 + 10 + <OcfFlag as DekuBitField>::BITS)?;
let version = ((run >> (10 + <OcfFlag as DekuBitField>::BITS)) & 0b11) as u8;
let id      = ((run >> <OcfFlag as DekuBitField>::BITS) & 0b11_1111_1111) as u16;
let ocf     = <OcfFlag as DekuBitField>::from_bit_run(run)?;

The widths are associated constants, so the shifts and masks are still
compile-time values; they are simply written as constant expressions rather than
literals. from_bit_run masks off the bits above its own, so the shift is all
the caller owes it.

On a real frame

The CCSDS TM primary header has 11 fields, five of them flag enums. Before this
PR the previous one batched it into 8 calls: a run of three, then the ocf
enum alone, then a run of two, then four enums, then a trailing 11-bit field with
no neighbour to pair with.

With batch_bits all eleven qualify, they sum to exactly 48 bits, and the header
becomes one read and one write.

The 64-bit cap

A run cannot exceed the 64 bits one read returns. Where every width is known to
the macro it splits runs to fit. Where a width comes from the trait it cannot, so
it emits a constant assertion beside the impl:

error[E0080]: evaluation panicked: deku: `batch_bits` grouped adjacent fields
              into a run wider than 64 bits

That is emitted as an item rather than a statement in the read body on purpose.
Inside the body it only fired at the first use, because that function is
generic over the reader. As an item it fires at the definition.

Concretely, per file

  • src/lib.rs: the DekuBitField trait, behind the bits feature.
  • deku-derive/src/macros/deku_read.rs: bit_field_width (does this enum
    qualify), emit_deku_bit_field (the impl), and the run planner extended so a
    field's width can come from the trait instead of from the macro.
  • deku-derive/src/macros/deku_write.rs: the dual, folding to_bit_run into
    the composed integer.
  • deku-derive/src/lib.rs: the batch_bits container attribute.
  • src/attributes.rs: documentation with a worked example.

Numbers

The same 11-field header is declared twice from one macro, once with the
attribute and once without, and both are benched in the same binary. So this is
a direct A/B, not two runs compared. Per header, from benches that push 128
frames through a single reader or writer:

CCSDS TM header, 11 fields, 5 typed as enums without with
write 107 ns 11.2 ns (9.5x)
read 14.5 ns 6.93 ns (2.1x)

Controls in the same session are unmoved: six plain byte fields read 4.77 ns
against 4.78 ns before, a lone 1-bit field 1.68 ns against 1.68 ns. A struct
without the attribute generates identical code.

On a production 9-protocol decode pipeline (DVB-S2, CCSDS TMTF, SDLS AES-GCM,
Space Packet, PUS, Encapsulation, IPoC, IPv4, UDP), adding the attribute to four
headers moved throughput from 728.7 to 798.0 MiB/s, +9.5%.

Testing

13 new tests, including a differential test that pushes 20,000 random 6-octet
headers through batched and unbatched versions of the same struct and asserts
every field, both error paths, and the re-encoded bytes agree. Full CI feature
matrix, plus all examples.

Separately, the batched output was checked against an independent set of
hand-written parsers for seven real protocol headers: 8.2M cases, zero
mismatches
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant