Skip to content

[Security review][L4] Undeliverable datagrams silently strand transfers at large --mtu - #33

Merged
gistrec merged 2 commits into
masterfrom
fix/mtu-max-exceeds-udp-limit
Jul 6, 2026
Merged

[Security review][L4] Undeliverable datagrams silently strand transfers at large --mtu#33
gistrec merged 2 commits into
masterfrom
fix/mtu-max-exceeds-udp-limit

Conversation

@gistrec

@gistrec gistrec commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Problem (L4)

A TRANSFER datagram is TRANSFER_HEADER (18) + chunk bytes. Two compounding issues made a large --mtu silently corrupt transfers:

  1. Arithmetic ceiling. --mtu was accepted up to 65507, so --mtu ∈ [65490, 65507] produced a 65508…65525-byte datagram — over the 65507-byte IPv4 UDP limit.
  2. Root cause: swallowed send failures. sendPart() treated any sendto() error as a warning and returned true:
    if (sent < 0) { std::cerr << "Warning: Failed to send part..."; return true; }
    EMSGSIZE is permanent (RESEND retries hit the same wall), so the sender "finished" successfully while the receiver got a broken/timed-out transfer.

This bites hard on macOS/BSD, where net.inet.udp.maxdgram defaults to 9216 — so any --mtu above ~9198 reproduced the silent-stranding symptom, even after the cap.

Runtime proof (macOS, before the root-cause fix): send --mtu 65489 printed Sent ... at 625 MiB/s (success) while the receiver reported Error: Transfer timed out with 3 part(s) missing; only the small tail part was delivered.

Fix

Cap (arithmetic ceiling):

  • Protocol: MAX_UDP_PAYLOAD = 65507; MAX_CHUNK = MAX_UDP_PAYLOAD - TRANSFER_HEADER (65489).
  • --mtu bounded to [MIN_CHUNK, MAX_CHUNK] via the shared constants.

Root cause (no more silent success):

  • sendPart() now treats EMSGSIZE/WSAEMSGSIZE as fatal (return false, clear error); other, transient errors are unchanged and still recoverable via RESEND.
  • The sender rejects an oversized --mtu at startup, before hashing the whole file, by comparing the datagram size against the platform's UDP send limit (net.inet.udp.maxdgram on macOS/BSD, else the 65507 IPv4 ceiling), with an actionable message (use --mtu <= N).

Docs & tests:

  • README range updated to 64..65489.
  • Unit test boundary updated (AnnounceInRange).
  • New e2e regression test: an oversized --mtu must be rejected (never reported as sent), and the largest deliverable --mtu must still transfer.

Verification (macOS, after)

  • --mtu 65489 → exits 1: Error: --mtu 65489 needs a 65507-byte datagram, over this system's UDP send limit of 9216; use --mtu <= 9198.
  • --mtu 9198 (datagram == 9216 limit) → transfers, sha256 verified; --mtu 9199 → rejected.
  • --mtu 8000 / 1500 → transfer and verify.
  • 35 unit tests pass; new e2e test passes.

Reviewed via an adversarial multi-dimension pass (correctness / portability / edge-cases + tests); no blockers or majors, only cosmetic nits (one of which — a size_t underflow in the hint string under a pathological sub-18-byte sysctl — is addressed).

A TRANSFER datagram is TRANSFER_HEADER (18) + chunk bytes, but --mtu and
MAX_CHUNK both allowed a chunk up to 65507. With --mtu in [65490, 65507]
every non-final part became a 65508..65525-byte datagram, which sendto()
rejects with EMSGSIZE; sendPart swallowed that as a warning and returned
true, so the sender 'finished' while each >64 KiB part silently never left
(and was never re-sent on RESEND).

Derive MAX_CHUNK from MAX_UDP_PAYLOAD - TRANSFER_HEADER (65489) and bound
--mtu by the same constants, so the framing overhead can never push a
datagram past the IPv4 UDP limit. Update the README range and tests.
@gistrec gistrec added bug Something isn't working docs Documentation labels Jul 6, 2026
@gistrec gistrec changed the title Cap --mtu below the UDP payload limit so full chunks stay deliverable [Security review][L4] Cap --mtu below the UDP payload limit so full chunks stay deliverable Jul 6, 2026
Capping --mtu at 65489 fixed only the IPv4 arithmetic ceiling. The real
L4 root cause is that sendPart swallowed every sendto() error as a warning
and returned true, so a permanent EMSGSIZE silently stranded the transfer:
the sender 'completed' while the receiver timed out. On macOS this bites at
any --mtu above ~9198, because net.inet.udp.maxdgram defaults to 9216.

- sendPart: treat EMSGSIZE/WSAEMSGSIZE as fatal (return false, clear error)
  instead of a swallowed warning; other (transient) errors are unchanged and
  still recoverable via RESEND.
- Reject an oversized --mtu at sender startup, before hashing the whole file,
  by comparing the datagram size against the platform's UDP send limit
  (net.inet.udp.maxdgram on macOS/BSD, else the 65507 IPv4 ceiling).
- Add an e2e regression test asserting an oversized --mtu is rejected (never
  reported as sent) and the largest deliverable --mtu still transfers.

Verified on macOS: --mtu 65489 now exits 1 with 'use --mtu <= 9198' instead
of a silent broken transfer; --mtu 9198/8000/1500 transfer and verify.
@gistrec gistrec changed the title [Security review][L4] Cap --mtu below the UDP payload limit so full chunks stay deliverable Fix L4: undeliverable datagrams silently strand transfers at large --mtu Jul 6, 2026
@gistrec gistrec changed the title Fix L4: undeliverable datagrams silently strand transfers at large --mtu [Security review][L4] Undeliverable datagrams silently strand transfers at large --mtu Jul 6, 2026
@gistrec
gistrec merged commit 878a6e3 into master Jul 6, 2026
6 checks passed
@gistrec
gistrec deleted the fix/mtu-max-exceeds-udp-limit branch July 6, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working docs Documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant