Skip to content

Perf/byte array fields - #4

Open
Simon-Calbert-Aerospacelab wants to merge 2 commits into
perf/batch-enum-bit-fieldsfrom
perf/byte-array-fields
Open

Perf/byte array fields#4
Simon-Calbert-Aerospacelab wants to merge 2 commits into
perf/batch-enum-bit-fieldsfrom
perf/byte-array-fields

Conversation

@Simon-Calbert-Aerospacelab

Copy link
Copy Markdown
Owner

What this changes

A [u8; N] field is read one byte at a time.

[T; N] has a single generic impl, and it has to be general: it loops over
MaybeUninit, calls T::from_reader_with_ctx per element, and carries a
drop-on-error unwind path in case a later element fails after earlier ones were
constructed.

let mut array: [MaybeUninit<T>; N] = [const { MaybeUninit::uninit() }; N];
for (n, item) in array.iter_mut().enumerate() {
    match T::from_reader_with_ctx(reader, ctx) { /* ... */ }
}

For T = u8 none of that is needed. The bytes are the value. Measured on a
real pipeline, an IPv4 header's two 4-byte address fields cost 168 instructions
each
. A hand-written parser does one 4-byte load.

This PR has the derive emit one call when the field is syntactically [u8; N].

Stacked on #3.

What the derive emits

Read:

let mut __deku_bytes = [0u8; 4];
__deku_reader.read_bytes_const_into::<4>(&mut __deku_bytes, Order::Msb0)?;
__deku_bytes

Write:

__deku_writer.write_bytes(&self.source_address[..])

Both methods already exist on Reader and Writer and are already public, and
both already handle the unaligned case. This PR adds no runtime code at all; it
only changes which existing call the derive emits.

When a field does not qualify

The field must be [u8; N] with a literal N, and carry no other deku
attribute
: not bits, bytes, count, bytes_read, until, read_all,
map, ctx, update, reader, writer, skip, cond, assert,
assert_eq, temp, temp_value, magic, any pad_*, or any seek_*. Each of
those either moves the cursor, makes the read conditional, or changes what a
single field means.

Two notes on the gates:

  • No endianness gate, unlike the previous PRs. A byte has no byte order, so a
    endian = "little" container qualifies too.
  • Msb0 only. On an unaligned cursor read_bytes_const_into reverses the
    buffer for Lsb0, which is not what a sequence of byte reads does. Rather than
    reason about that, Lsb0 keeps the existing path.

Anything that does not qualify is untouched, so this is purely additive.

Concretely, per file

  • deku-derive/src/macros/deku_read.rs: byte_array_len, the eligibility
    check, plus one arm in the field-read emitter.
  • deku-derive/src/macros/deku_write.rs: one arm in the field-write emitter,
    reusing the same check.

That is the whole change.

Numbers

Per pair of 4-byte address fields, from a bench that pushes 96 items through a
single reader or writer:

before after
write 35.1 ns 7.23 ns 4.9x
read 17.1 ns 4.80 ns 3.6x

On a production 9-protocol decode pipeline, throughput went from 798.0 to
832.5 MiB/s, +4.3%
.

Callgrind on that pipeline confirms the mechanism rather than just the outcome:
169.11M to 165.90M instructions. The generic [T; N] impl disappears from
the profile entirely, and read_bytes_const_into drops from 2.73M instructions
to 1.01M.

Reproduce with cargo bench --bench bebits --all-features.

Testing

The risk here is the unaligned path, so that is where the tests concentrate.
tests/test_byte_arrays.rs has 13 tests:

  • One per bit offset 1 through 7, each reading a [u8; 4] that starts mid-byte
    and comparing against the generic [T; N] impl invoked directly as an
    oracle
    , over 199 seeds each, plus a byte-exact write round-trip.
  • A 20-byte array at a 3-bit offset, so the multi-byte unaligned path is covered.
  • Truncated input still returns Incomplete rather than a panic or a short read.
  • Arrays inside enum variants, both named and tuple.
  • A little-endian container.
  • Five shapes the shortcut must decline: [u16; 2], a bits attribute,
    pad_bytes_before, and a map.

Full CI feature matrix. Separately, the output was checked against independent
hand-written parsers for seven real protocol headers: 8.2M cases, zero
mismatches
.

@Simon-Calbert-Aerospacelab
Simon-Calbert-Aerospacelab force-pushed the perf/byte-array-fields branch 2 times, most recently from af2540a to 7b11aef Compare August 11, 2026 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant