Skip to content

perf: add a zero-copy pbuf output path for WireGuard - #33

Open
kedimuzafer wants to merge 1 commit into
CamM2325:mainfrom
kedimuzafer:perf/zero-copy-wg-output
Open

perf: add a zero-copy pbuf output path for WireGuard#33
kedimuzafer wants to merge 1 commit into
CamM2325:mainfrom
kedimuzafer:perf/zero-copy-wg-output

Conversation

@kedimuzafer

Copy link
Copy Markdown

Problem

wireguard_udp_output_fn takes a flat byte buffer, so both magicsock output paths in wireguardif.c do this per packet:

uint8_t *data = (uint8_t *)mem_malloc(q->tot_len);
pbuf_copy_partial(q, data, q->tot_len, 0);
device->udp_output_fn(dest_ip, port, data, q->tot_len, ctx);
mem_free(data);

and ml_wg_mgr's callback then has to turn it back into a pbuf for udp_sendto:

struct pbuf *p = pbuf_alloc(PBUF_TRANSPORT, len, PBUF_RAM);
memcpy(p->payload, data, len);
udp_sendto(s_wg_output_pcb, p, &dst, dest_port);
pbuf_free(p);

Two allocations and two ~1.3 KB copies of the same bytes, once per outgoing packet, all on the TCPIP thread.

That thread is the constraint. vTaskGetRunTimeStats on an ESP32-S3 carrying a single 1.4 MB/s TCP stream through the tunnel:

tiT=84% wifi=37% s5_conn=23% IDLE1=6%

The encrypted pbuf is allocated PBUF_TRANSPORT in wireguardif_output_to_peer(), so it already has headroom for the UDP and IP headers — udp_sendto can take it directly.

Change

Adds wireguard_udp_output_pbuf_fn and wireguardif_set_udp_output_pbuf(), and prefers it in both output paths when registered. The existing byte-buffer callback is untouched and still used when no pbuf callback is set, so this is additive — existing integrators are unaffected.

ml_wg_mgr registers one, since it already sends through a raw udp_pcb and never needed the flat buffer.

Ownership is unchanged: udp_sendto does not consume the pbuf, and the callback is synchronous, so wireguardif's existing pbuf_free still applies.

Measured

Single-stream throughput through the tunnel, ESP32-S3 SOCKS5 proxy: 1.44 MB/s → 1.49 MB/s. Modest on its own, but it is 25% of the per-packet work on the thread that is nearest to saturation, so it should matter more on boards or workloads where that thread is the limit.

Independent of #30, #31 and #32.

wireguard_udp_output_fn takes a flat byte buffer, so both magicsock output
paths mem_malloc() a buffer and pbuf_copy_partial() the encrypted pbuf into
it. The integrator then has to build a pbuf again to hand it to udp_sendto,
which is a second allocation and a second copy of the same ~1.3 KB. All four
happen on the TCPIP thread, once per outgoing packet.

The encrypted pbuf is allocated PBUF_TRANSPORT in wireguardif_output_to_peer,
so it already has room for the UDP and IP headers and udp_sendto can take it
as-is.

Adds wireguard_udp_output_pbuf_fn plus wireguardif_set_udp_output_pbuf(), and
prefers it in both output paths when set. The byte-buffer callback is
untouched and still used when no pbuf callback is registered, so this is
additive for existing integrators.

ml_wg_mgr registers one, since it already sends through a raw udp_pcb.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bqLmUG7wJ7p9ya2Vp4A9J
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant