Skip to content

add datagram handshake state machine over the UDP transport#39

Merged
pzverkov merged 6 commits into
mainfrom
feat/udp-datagram-handshake
May 30, 2026
Merged

add datagram handshake state machine over the UDP transport#39
pzverkov merged 6 commits into
mainfrom
feat/udp-datagram-handshake

Conversation

@pzverkov
Copy link
Copy Markdown
Member

Completes the datagram handshake on top of the UDP transport foundation (#38).
This adds the piece that drives the CH-KEM handshake to an established session
over a lossy, connectionless link, reusing the existing flight builders
unchanged.

Three tested increments:

  • State machine (dgram_handshake_fsm.go): a transport-agnostic FSM that
    classifies each inbound message as advance, replay, or drop, and caches its
    last outbound message so it answers a peer retransmit from bytes. A bad or
    forged datagram drops rather than failing the handshake; the handshake fails
    only on its deadline. No sockets, timers, or fragmentation.

  • Reliability driver (dgram_handshake_driver.go): retransmission with
    exponential backoff (500ms to 8s), a retry ceiling, and a bounded responder
    linger that replays the cached ServerFinished for a retransmitted
    ClientFinished to recover a lost final flight. Callers inject the clock so tests
    run fast and deterministically.

  • Endpoint wiring (dgram_handshake_wire.go, datagram.go): the receive loop, the
    index-and-source demux, outbound fragmentation, one goroutine per handshake,
    DialDatagram, and responder accept. The responder runs no decapsulation and
    sends no ServerHello until a full ClientHello arrives and the half-open gate
    admits it, so it never sends more than it received from an unvalidated source.

Testing: unit tests for the FSM and driver, plus an end-to-end test over a
seeded transport that drops, duplicates, and reorders datagrams. The sweep runs
24 seeds under -race -count=50 (1200 handshakes) and asserts matching
directional keys through a bidirectional encrypt/decrypt round-trip.

The fault-injection test caught two real bugs, both fixed here:

  • The advance path reset the retransmit timer to the default 500ms constant
    instead of the configured value, so the ClientFinished phase silently
    reverted to a 500ms RTO. The trace showed a 500ms gap where retransmits
    belonged milliseconds apart.

  • The responder linger lasted one RTO, so a lost final ServerFinished timed out
    the initiator. Linger now spans the initiator's full ClientFinished
    retransmit window.

Deferred: the stateless cookie/RETRY anti-amplification exchange, the data-path
AEAD open and authenticated CLOSE, and resumption over datagram.

pzverkov added 4 commits May 31, 2026 00:46
Splits a serialized CH-KEM handshake message into datagram HANDSHAKE frames within the MTU, the inverse of the Reassembler. Each fragment carries a distinct increasing sequence and the same total length, so the peer reassembles in any order. Rejects empty and oversize messages. Round-trip tests reassemble out of order and confirm each frame stays under the MTU. The retransmission state machine and responder flight cache build on this.
Adds the receive loop, index/source demux, outbound fragmentation, the
per-handshake goroutine, DialDatagram, and responder accept. Drivers take
configurable backoff so tests run fast; fixes the advance path resetting the
RTO to the default constant instead of the configured value, and lengthens the
responder linger to cover the initiator's full ClientFinished retransmit window.
Adds a seeded loss/reorder/dup end-to-end test.
@pzverkov pzverkov self-assigned this May 30, 2026
Pins the two bugs the fault-injection sweep surfaced: the advance path must
re-arm at the configured rtoInitial, and the responder linger must span the
initiator's full retransmit window. Adds a third test that the endpoint hands
its configured backoff to every driver, and folds the duplicated driver
construction into a newDriver helper so the wiring has one tested seam.
@pzverkov pzverkov force-pushed the feat/udp-datagram-handshake branch 2 times, most recently from ad6d07c to 6aa1547 Compare May 30, 2026 23:25
…hmark

Updates README, CHANGELOG, the datagram design doc, and the CLI reference to
record that the datagram handshake now works while the encrypted data path does
not yet. Adds quantum-vpn bench --datagram-handshakes N, which measures the
datagram handshake rate over loopback UDP: around 1,300/sec on an M1 Pro versus
1,450/sec for the stream path, and computes the benchmark banner padding so the
box aligns.
@pzverkov pzverkov force-pushed the feat/udp-datagram-handshake branch from 6aa1547 to 2bbfc21 Compare May 30, 2026 23:27
@pzverkov pzverkov merged commit 0b51742 into main May 30, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant