Skip to content

test: end-to-end hybrid transport tests, Python hybrid peer, misbehaving peers and fallback (#184 part 7) - #239

Open
djh58 wants to merge 2 commits into
djh58/pq184-6-rpcfrom
djh58/pq184-7-e2e
Open

djh58 wants to merge 2 commits into
djh58/pq184-6-rpcfrom
djh58/pq184-7-e2e

Conversation

@djh58

@djh58 djh58 commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Summary

Part 7 of #184 (#212): end-to-end tests of the hybrid v2 transport. Stacked on #236 (part 6), which provides -v2pqtransport and the RPCs these tests use.

  • G1. Opt-in hybrid Python v2 peer. test_framework/v2_p2p.py gains the ML-KEM-1024 hybrid handshake: offer, accept, key schedule and key confirmation. It uses part 2's independent Python ML-KEM-1024 and follows part 4's spec (doc/design/pq-transport.md). p2p_v2_pq.py covers hybrid connections both ways between the node and the Python peer, plus legacy peers.
  • G2. Misbehaving peers, key confirmation and fallback.
    • p2p_v2_pq_misbehaving.py: malformed records, bad keys and ciphertexts, and a failed confirmation, each closing with the right outcome.
    • p2p_v2_pq_fallback.py: the 3-strike fallback with its windows, and load shedding.
    • Both run through real node paths, including the socket-thread shedding update and the name-proxy endpoint key, the latter through a SOCKS5 proxy.

G3 isn't in this PR yet. G3 adds the --v2pqtransport mode for the whole functional suite and a pq-functional CI job. Its original commit is on djh58/pq184-7-e2e-g3-original (6212bcb), written against an earlier part 6. Rebased onto this stack, it conflicts with part 3's macos-x86_64-unit job in .github/workflows/ci-pq.yml and ci/checks/test_ci_pq_contract.py. The resolution is a union, keeping both jobs; see #238.

Testing

  • Ran focused unit or functional tests for the changed area: on the real part 6, p2p_v2_pq.py, p2p_v2_pq_misbehaving.py, p2p_v2_pq_fallback.py and the adapted tests pass. Including G3, the whole suite (317 tests) passes with --v2pqtransport, with --v2transport --usecli and with no flag. 11 of 11 deliberate node mutations are caught.
  • Ran lint or formatting checks relevant to this change: ruff, mypy on the new files, vulture, codespell, shellcheck.
  • Not run. Reason: GitHub CI on this stack position. The real-time load-shedding subtest skips on machines too slow to exceed 1,000 offers per second.

Target Branch

Risk / Review Notes

  • Consensus, script, crypto, wallet, P2P, release, CI, or security-sensitive behavior changed.

Notes: tests and test framework only.

Part of #212 · Part of #184

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

djh58 and others added 2 commits October 6, 2026 16:59
Give the functional test framework an independent implementation of the
hybrid post-quantum v2 handshake (doc/design/pq-transport.md), built on
crypto/mlkem.py and the key recipe in crypto/bip324_cipher.py, and use
it for end-to-end tests against the node.

- v2_p2p.py: EncryptedP2PState(supports_v2_pq=True) negotiates in both
  roles. A responder offers an ML-KEM-1024 key from two independent
  32-byte draws; an initiator holds its version packet until the
  responder's arrives, accepts a valid offer and switches; each side's
  first hybrid packet is a key confirmation, and the peer's must
  verify. parse_version_contents() mirrors the node's record grammar.
  Subclasses can override the offer and accept contents, the shared
  secret and the confirmation, and a peer can withhold its
  confirmation.
- Framework peers stay legacy by default (supports_v2_pq=False). The
  unconditional assertion that the node's version contents are empty is
  gone: a legacy peer accepts a non-empty offer and answers with empty
  contents. Received and sent contents are recorded, so tests of
  switched-off nodes assert empty contents explicitly.
- p2p.py: authenticate_handshake() also returns the bytes it produced
  (the held version packet, a confirmation), which the driver sends. The
  handshake completes only after the peer's confirmation verified and
  ours was sent; send_pq_confirmation() releases a withheld one.
- test_node.py: add_p2p_connection() and add_outbound_p2p_connection()
  take supports_v2_pq, and the outbound helper gains
  wait_for_v2_handshake.
- v2_p2p.py unit tests (feature_framework_unit_tests.py) replay all four
  vectors of src/test/data/pq_transport_vectors.json through two states,
  run every role pairing across a rekey, and check withheld and damaged
  confirmations.

p2p_v2_pq.py covers two nodes in both directions (hybrid, the same
session id, more than 224 packets each way), the Python peer in both
roles (the node draws fresh keys and encapsulation randomness per
connection), legacy and v1 peers under each switch setting and each
pair of node settings, and the pending status while a peer withholds its
confirmation.

Work item G1 of #212 (#184 part 7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drive every way a hybrid negotiation can go wrong through a real node,
using scripted variants of the Python hybrid peer, and check the
fallback that follows repeated failures.

p2p_v2_pq_misbehaving.py:
- test_records: offers (the node initiates) and accepts (the node
  responds) of every shape: EK and CT one byte short and long, a
  bad-modulus EK, two own records (the first wins either way), unknown
  and reserved headers, junk after a record, zero, non-canonical,
  past-the-end, above-MAX_SIZE and truncated lengths, contents padded to
  exactly 4,000,013 bytes and one byte more, a damaged version tag, and
  version packets cut short before EOF. Each case checks that the
  connection is kept (with its transport_pq_status) or dropped at once,
  the exact log lines, and the exact per-direction counter deltas.
- test_confirmation: a nonzero length, a bad tag and a non-decoy
  confirmation in both roles, a corrupted CT and a responder that
  derives another secret: each disconnects at once, long before
  -peertimeout.
- test_decoys_and_partial_writes: decoys before the version packet
  (the first one authenticating the garbage), decoys after the
  confirmation, coalesced bursts and a handshake fragmented into 1- to
  7-byte writes, in both roles.
- test_unsolicited_accept: a switched-off responder ignores own-record
  bytes it never asked for.
- test_load_shedding: a threaded flood of 64-byte keys starts load
  shedding; while shed, a hybrid initiator sees a legacy responder, an
  accept is ignored and outbound connections still negotiate; then,
  with no inbound connection at all, the socket handler alone ends
  shedding after the quiet period. Shedding counts seconds of the
  steady clock, so a machine that can't outpace the threshold skips
  this part, and one that could but didn't shed fails.

p2p_v2_pq_fallback.py, with addnode onetry for explicit outbound
attempts (addconnection can't open manual connections):
- test_three_failures: -test=pq_fail_first_packet on one node; three
  failed attempts enter the fallback set (fallback = 1, one warning
  line, the set's entry and ring), and the fourth connection is plain
  v2 with status fallback.
- test_expiry_and_restart: past the window the negotiation is tried
  again with the next window at 4 h; a restart clears everything under
  a new instance_id.
- test_malformed_offer_fallback: against a responder that always
  offers a bad-modulus key, the fourth connection completes, because a
  fallback connection neither parses nor validates offers.
- test_inbound_and_success_exclusion: inbound failures, and outbound
  failures after a hybrid success with the endpoint, never lead to a
  fallback.
- test_endpoint_separation: through a SOCKS5 proxy, hostnames in any
  letter case with or without the root dot, one hostname on two ports
  and one proxied address on two ports keep separate histories. This
  drives ConnectNode's name-proxy endpoint key through the real path.

Mutating the node, each of these is killed: dropping the socket
handler's shedding update, keying name-proxy endpoints by the proxy,
reporting v2 before the confirmation, parsing only up to the first own
record, dropping the early confirmation length check, skipping the
fallback lookup, the success protection or the window escalation,
fixed KEM randomness, counting inbound failures, and parsing contents
with the switch off.

Work item G2 of #212 (#184 part 7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@djh58
djh58 added this pull request to stack #232 October 6, 2026 21:02
@djh58 djh58 added the pq-transport Hybrid post-quantum v2 transport (#184) label Oct 6, 2026
@djh58

djh58 commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-06T21:07:09.865294Z dfd5a22 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dfd5a22aeb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

before = counters(node)
ended = []
deadline = time.monotonic() + FLOOD_SECONDS * self.options.timeout_factor
with node.assert_debug_log([f"v2 pq: load_shedding started threshold_per_s={SHED_THRESHOLD}"]):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Move the slow-host skip outside the log assertion

When a machine cannot exceed the shedding threshold before the deadline, the return still unwinds this assert_debug_log context, whose exit handler waits for the absent load_shedding started message and raises instead of skipping. Thus the documented slow-machine path always fails after the flood; only require this log after confirming that shedding became active, or otherwise avoid entering the assertion for the skip path.

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pq-transport Hybrid post-quantum v2 transport (#184)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant