Repository navigation
feat(seeder/nfx): the NFX-12 HyperDHT transport — dial, listen, files/1 both sides, pay/1 lane - #32
Closed
SovereignTechnology wants to merge 5 commits into
Closed
SovereignTechnology wants to merge 5 commits into
SovereignTechnology wants to merge 5 commits into
Conversation
…d type the NFX-12 surface - `sodium-universal` 5.0.1 becomes a dependency of @sovit/seeder, for the NFX-12 `bind` signature (Ed25519 by the peer key). `@hyperswarm/secret-stream` 6.9.1 becomes a devDependency, for in-memory secret-stream pairs in the transport's tests. Both were already in the lockfile at those versions (core and hyperdht pull them in): the lockfile diff is the two dependency lines of the workspace entry. No native module moves: sodium-native's shortest chain is still app-desktop's own dependency. - Ambient types, written against the installed sources: hyperdht 6.34.0's server and connect (`createServer`, `listen`, `connect`, `fullyBootstrapped`), secret-stream 6.9.1 (`setKeepAlive`, `setTimeout`, `opened`, pause/resume), protomux 3.11.0 (`from`, `createChannel`, `pair`, channels and messages), and the two sodium members the seeder calls.
…/1 both sides, pay/1 lane The transport of nfx `spec/nfx-12-hyperdht.md` (ADR 0020), free mode only, on the wire layouts of `@sovit/core`'s `nfx` codec. Nothing in the app uses it yet; the worker wires it up later. - `listenNfx` / `dialNfx` on a HyperDHT node the caller owns (bootstrap from local config only): the seeder listens on its per-network peer key, the watcher dials it with its own; the dialer closes a connection whose remote key is not the one it dialed. Local policy caps connections in total and per remote key, in the DHT firewall and again on arrival. - `NfxConnection`: keepalive at 25 s and close after 60 s of silence (secret-stream's timers); at most 16 videos per connection; a remote channel only for a canonical video address the source serves; 64 rejected opens close the peer (protomux 3.11 bounds neither pending nor rejected opens). - `nfx/files/1` seeder side: only files the source lists for THIS channel's video (never an oracle for the rest of the store), `missing` otherwise; at most 8 open requests per channel; malformed or misdirected messages, a reused `req` and trapped unknown types close the channel. - Flow control: one pump per connection writes one frame at a time and waits for `'drain'` after the stream reports full, so at most one 64 KiB frame sits on a buffer under 16 KiB, inside the 256 KiB cap; pay/1 is written directly, behind no more than that. Each frame is read inside the pump's turn, so a connection buffers one chunk however many requests it has. - `nfx/files/1` watcher side: each payload copied out of the receive buffer, capped at the file's size, sha256-checked before `ok`; a mismatch, too-long or too-short file drops the connection (NFX-05 §4). Sink writes are serialised and reads held above 1 MiB waiting. A request with no frame for 60 s is abandoned, unless the silence is the sink's. - `hyperdhtBind` / `hyperdhtEndpoint`: the beacon endpoint and its Ed25519 `bind`. - `PayLane`: `nfx/pay/1` carrying NFX-07 messages under the codec's rules, no payment logic. Protomux drops a message type a channel did not add without telling it, so after the pinned types each channel adds four trap types that close it; indices past those are still dropped unseen (no state changes), short of the close NFX-12 asks for.
…t, and its bind vector - bind: reproduces `beacon-hyperdht.json`'s signature byte for byte from its throwaway peer seed, verifies with the codec only for its own seeder and network, refuses bad arguments. - pump: against a stand-in with streamx's write/drain semantics — never a write on a buffer that asked for a drain, the real queue at most one frame over 16 KiB, transfers interleaved in arrival order, waiters answered `false` and listeners removed on close. - transport (a real secret-stream pair joined in memory): multi-frame and zero-byte files byte for byte; the hash list against a cap; `missing` for a file held only for another video; `unserved`; poison, too-long and too-short each drop the source; bad arguments, a duplicate sha and a ninth request refused on the watcher side; a slow sink with reads held and the seeder's queue within 256 KiB; a stalled transfer abandoned while keepalives hold the connection; a throwing sink; keepalives and the idle close (short intervals); the pay/1 lane both ways, refused in free mode, closed on a newline. Hostile watchers on raw Protomux: a trapped unknown type and a reused `req` close the channel; a ninth request, a 17th video and 65 rejected opens close the connection; a non-canonical channel id is rejected. - endpoint (a local hyperdht testnet, as the swarm tests use): dial by key and fetch a multi-frame file, the seeder seeing the watcher's key; a firewalled key and a second connection from one key refused; a peer key that is not 32 bytes.
…end` - connection.ts: the channel-id check decoded with TextDecoder's default, which strips a leading U+FEFF, so `` + a served address passed the canonical-form comparison although its bytes are not the canonical id (the channel was then opened under the canonical id instead). `ignoreBOM: true` keeps it, and the exact comparison refuses it. - files-client.ts: `data` or a second `end` for a request whose `end` already arrived was taken as more of the file (hashed and written to the sink) while the sha256 check waited for the sink. It is now a protocol error, and the deferred check does nothing once the request is settled (so it can no longer drop a connection for a transfer already failed). - Tests: a BOM-prefixed id is refused; `data` corked behind `end` in one write fails the hash-list fetch `protocol`. holepunch.d.ts declares Protomux `cork`/`uncork` for that test.
…ted function, and closed read after an await tsc: `ch` was narrowed non-null after its check, but not inside the hoisted `serve`, so it is bound once as a ProtomuxChannel. eslint no-unnecessary-condition: `ch.closed` and `this.closed` were re-checked after an await, which TypeScript still narrows to false from the earlier check; both are now read through a function, so the re-check stays (the channel or connection can close while the read is pending).
This was referenced Oct 25, 2026
Closed
SovereignTechnology
added a commit
that referenced
this pull request
Oct 25, 2026
…/1 both sides, pay/1 lane (#32) Phase 1 slice 1 of the NFX interop plan (ADR 0020), in packages/seeder/src/nfx/: HyperDHT dial and listen with per-network key pairs (firewall before the handshake, 64 connections, 2 per remote key), the NFX-12 bind and beacon endpoint, per-connection keepalive and channel caps, a files/1 server that serves only files the source lists for that video (NfxAdmission is where paid mode plugs in) and a client that caps each file at its size and checks its sha256 before ok, a fair one-frame-per-turn pump, and a bytes-only pay/1 lane that free mode rejects. Nothing in the app calls it yet; the worker wiring is slice 5.
Owner
Author
|
Squash-merged as 35de9c5 (main, as sovtech). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1 slice 1 of the NFX interop plan (ADR 0020): the NFX-12 transport, in
packages/seeder/src/nfx/. It runs between JS peers today. Nothing in the app calls it yet; the worker wiring comes in slice 5.What's in it
endpoint.tsbind.tsbind(Ed25519 overnfx/hyperdht/1:<seeder>:<namespace>) and thehyperdhtbeacon endpoint. It round-trips with the codec's verifierconnection.tsfiles-server.tsend; otherwisemissing(free mode). The typedNfxAdmissionhook is where paid mode plugs infiles-client.tsok; a mismatch drops the connection. Every payload is copied. Reads pause while more than 1 MiB waits for the sink. A request that stalls failstimeoutpump.tspay-lane.tsnfx/pay/1carries bytes only (UTF-8, no newline, at most 32,767 B). Free mode rejects itDependencies: none new.
@sovit/seedernow declaressodium-universal5.0.1 and, as a dev dependency,@hyperswarm/secret-stream6.9.1. Each already resolves to a single copy in the lockfile, so the lock diff is just those two lines. The native-module inventory is unchanged.Deviations from NFX-12, documented in the code
Review before push
I read every module, and protomux 3.11's pairing, rejection and send paths, to check the assumptions.
Fixed in
4536a71:dataafterendwas hashed into the file instead of being a protocol error.Both have tests; the second is deterministic because its frames are corked into one write.
Fixed in
ab80176, the first CI findings: a typecheck narrowing error and two re-checks after anawaitthat lint called unnecessary. The re-checks stay.CI
All green on
ab80176:verify,test,e2eandpackage.testincludesendpoint.integration.test.tson a local HyperDHT testnet. That run came before the PR existed, from astage-check/push that's since deleted; this PR's own run will repeat it.Left for slice 5 (worker wiring)
Per-network key storage, a
NfxFileSourcebacked by the planned sha256 store, a disk-spool sink,bindsigning in the beacon publisher, and the IPC methods.Merge
Squash. Expect a CHANGELOG
[Unreleased]conflict with the packager PR, whichever merges second.