feat(state-node): NAT 配下のノードも参加できるようにする (AutoNAT v2 + circuit relay v2 + DCUtR) - #68
Open
somasekimoto wants to merge 2 commits into
Open
somasekimoto wants to merge 2 commits into
somasekimoto wants to merge 2 commits into
Conversation
somasekimoto
force-pushed
the
feat/state-node-nat-traversal
branch
from
August 27, 2026 09:46
19352b2 to
d37a898
Compare
somasekimoto
changed the base branch from
fix/state-node-peer-resilience
to
feature/read-response-signing
August 27, 2026 09:46
…lay + DCUtR Until now a node could only join if it was directly dialable, which in practice means a machine with a public address. That is a real limit on "anyone can run a node": most people are behind a home router or a NAT gateway. This adds the three libp2p pieces that together remove it. **AutoNAT v2 decides the role, by measurement.** A node asks peers to dial a specific address of its own back; they answer only for addresses they actually reached. So "am I reachable?" is answered by observation rather than by the node's own assumption — which matters because a node that wrongly believes it is reachable advertises relay service it cannot provide. v2 rather than v1 because v1 takes the server's word for it; there is no external-compatibility reason to prefer v1, since every peer here runs this same binary. **Circuit relay v2 turns every reachable node into a relay.** No node is designated: the role follows from measured reachability, the same way the existing application-level relay role follows from whether a node is a member of a content network. Reservations and circuits are capped (128 / 32) because relaying is work done for strangers. **DCUtR gets the relay back out of the path.** Once a relayed connection exists, both sides hole-punch to a direct one, so a relay carries traffic only until the upgrade lands. Nothing here changes how directly-reachable nodes talk to each other. The existing Monas relay — forwarding *requests* to a content network's members — is a separate, application-level mechanism and is untouched; this is about establishing the *connection* underneath it. `--disable-nat-traversal` turns the machinery off for a deployment that does not need it. Two implementation mistakes worth recording, both caught by writing tests that exercise the code rather than restate it: - The circuit address was built from the relay's peer id alone. Every `listen_on` would have failed with `MissingRelayAddr` — the client transport needs to know *where* to open the circuit — and the failure was logged at debug, so the private-node path would have been silently dead. Candidates now come from the connection table, which has the addresses we reached peers on, filtered to ones a third party could dial. - Releasing a reservation only logged what it would drop: `Swarm::listeners()` yields addresses, not `ListenerId`s, so nothing was removed and a relay kept a slot allocated for a node that no longer needed it. The ids are now kept alongside the peer. Verified on a local 4-node cluster with NAT traversal on and mDNS off: AutoNAT converged all four to Public (correct — they are mutually dialable), no relay reservations were taken (also correct), zero errors, and the reconvergence behaviour from the parent branch still works. **Hole punching itself is not verified.** It needs two nodes behind *different* NATs, which a single host cannot provide — and a local cluster would "pass" for the wrong reason, the way mDNS masked the reconvergence bug on the parent branch. Treat the two-sided NAT path as untested until it is run across real networks. cargo test --workspace: 807 passed, 0 failed binaries. clippy (1.97) clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review of the NAT traversal work found four defects, three of them sharing one root: the relay reservation path was an edge-triggered, one-way state machine that could only ever move forwards. **Relaying for strangers is now opt-in.** The relay *server* was created whenever NAT traversal was on — which is the default — so every node became an open circuit relay for anonymous peers from startup. A comment claimed it was "only ever used once AutoNAT reports us as reachable (see libp2p_network.rs)", but no such gating existed anywhere. That gate cannot be built as described: `Toggle` cannot enable a behaviour after the swarm is constructed and `relay::Behaviour` has no runtime mute. So the decision is made where it can be made honestly — from an operator assertion, `--relay-service`, which requires `--external-address` and is refused alongside `--disable-nat-traversal`. Needing to traverse NAT oneself is not the same as volunteering to carry other people's traffic. `entrypoint.sh` now passes `DISABLE_NAT_TRAVERSAL`, `RELAY_SERVICE` and `EXTERNAL_ADDR`. Neither flag was reachable from a deployment before, so the only way to change either was a code change. **A reservation is recorded when the relay grants it, not when we ask.** `listen_on` returning `Ok` only means the address parsed; the relay answers later and may refuse. Recording the attempt as if it were the reservation let a refusal occupy one of the two slots for the life of the process — and since the entry also filtered that peer out of the candidate list, two refusals left the node permanently unreachable. Entries are now `Pending` until `ReservationReqAccepted`, and `ListenerClosed` drops them so the relay can be replaced. This is the same "the call is not the effect" mistake as releasing a reservation by logging about it. **Reservations are topped up from the maintenance tick**, not only on an AutoNAT transition. A relay can refuse, expire or vanish at any time, and the transition will not fire twice. This is why `maintain_connectivity` exists. **Reachability is the aggregate over addresses.** AutoNAT v2 tests one address at a time, so a multi-homed node legitimately gets a failure for its LAN address and a success for its public one. Collapsing each event into a single last-event-wins verdict made the node flip Public/Private and churn reservations against other people's relays. One reachable address is enough. The redundant `add_external_address` is gone with it: the AutoNAT client already emits `ExternalAddrConfirmed`, and the manual set it was writing to is never expired, so a node would advertise a dead address indefinitely. **Circuit addresses strip any `/p2p/` before appending the relay's.** The connection table stores addresses as dialled and the conventional bootstrap form ends in `/p2p/<id>`, so appending blindly produced `/p2p/<id>/p2p/<id>/p2p-circuit` — rejected as malformed, silently, at debug level. `bootstrap.rs` already strips exactly this way. Failures to open a circuit are now `warn!`: a node that cannot reserve anywhere is unreachable, and burying that is how the private-node path stayed dead the first time. `peer_store` no longer persists circuit addresses. They say where a peer can be reached through a relay right now, not where it lives, so a restart would dial a circuit through a relay that has moved on. Five new tests, each verified to fail when its fix is reverted. That includes the release path, which the previous test could not constrain — it asserted the bookkeeping only, and still passed with `remove_listener` removed. The new one observes the `ListenerClosed` the swarm emits, so it fails on a log-only implementation. cargo test --workspace: 814 passed, 0 failed. clippy (1.97) clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
somasekimoto
force-pushed
the
feat/state-node-nat-traversal
branch
from
August 28, 2026 01:40
3c73f51 to
062d3ae
Compare
somasekimoto
changed the base branch from
feature/read-response-signing
to
main
August 28, 2026 01:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
何のための PR か
現状、ノードは直接 dial できる場合しか参加できない。実質的には公開アドレスを持つマシンに限られるということで、「誰でもノードを立てられる」という前提に対する実際の制約になっている。多くの人は自宅ルータや NAT gateway の内側にいる。
これを外すために libp2p の 3 つの部品を入れる。
設計
AutoNAT v2 が役割を決める(自己申告ではなく実測で)
ノードは自分のアドレスを指定して他ノードに dial back を依頼し、相手は実際に到達できたアドレスについてのみ肯定を返す。つまり「自分は到達可能か」を観測で答える。
これが重要なのは、到達不能なノードが自分を到達可能と誤認すると、提供できない relay サービスを広告してしまうため。#54/#56 が一貫して守ってきた「自己申告を信じない」原則と同じ話。
v1 ではなく v2 を採用した。v1 はサーバの申告をそのまま信じる。外部互換性の制約は無い(通信相手は全て同一バイナリ)ので、v1 を選ぶ理由がない。
circuit relay v2 — 到達可能なノードは全員 relay になりうる
専任 relay ノードは置かない。 役割は測定された到達可能性から従属的に決まる。これは既存の Monas relay(content network の member でないノードが転送役になる)と同じ考え方。
relay は見知らぬ相手のために働くので、予約 128 / 回線 32 で上限を設けている。
DCUtR が relay を経路から外す
relay 経由の接続が確立したら両者が hole punch し、直接接続へ昇格する。relay が実トラフィックを運ぶのは昇格までの間だけ。
既存動作への影響
直接到達可能なノード同士の通信は一切変わらない。
既存の Monas relay(content network の member へリクエストを転送する仕組み)とは別の層であり、そちらには手を入れていない。本 PR はその下で接続を確立する話。
--disable-nat-traversalで機構ごと無効化できる。実装中に自分で作り込んだ誤りが 2 件
どちらも「実装を通るテスト」を書いたことで露見した。テストが実装を再記述しただけなら見逃していた。
1. circuit アドレスを relay の peer id だけで組んでいた
listen_onは毎回MissingRelayAddrで失敗していたはず(client transport はどこに回線を開くか知る必要がある)。しかもエラーは debug ログなので、private ノードの経路が黙って死んだままになる。接続テーブル(実際に到達したアドレスを持つ)から候補を取り、第三者が dial 可能なものだけに絞るよう修正。
2. 予約の解放がログを出すだけだった
Swarm::listeners()はアドレスを返しListenerIdを返さないため、何も削除できていなかった。relay 側は不要になったノードのためにスロットを確保し続ける。peer と一緒に id を保持するよう修正。検証
NAT 有効・mDNS 無効のローカル 4 ノードで実施。
AutoNAT が 4 台とも
Publicに収束(相互に dial 可能なので正しい)relay 予約は 0 件(全台到達可能なのでこれも正しい)
エラー・panic 0 件
親ブランチ(fix(state-node): アドレス変更後にピアが自力で再収束できるようにする #67)の再収束も健在: ブートストラップ停止 → 復帰で全台 peers:3 に戻ることを確認
cargo test --workspace→ 807 passed / FAILED バイナリ 0Rust 1.97 clippy
--deny warningsクリーン未検証(重要)
hole punching 本体は検証できていない。 実証には異なる NAT 配下の 2 ノードが必要で、単一ホストでは用意できない。
ローカルで動かすと「成功」してしまうが、それは #67 で mDNS が再収束バグを覆い隠したのと同じ理由による偽陽性になる。両側 NAT の経路は未テストとして扱ってほしい。
実ネットワークを跨いだ検証(手元 Mac + テザリング等)は環境が用意でき次第、別途行う。
🤖 Generated with Claude Code