[Kilombino]

Lets lnd run with bitcoin.node=neutrino on the BLAKE2b chain. This builds on paulscode/btcd-blake2b#1 (BLAKE2b PoW validation in btcd) and on a neutrino change that stores 164-byte headers.

Changes:

• config.go: neutrino is no longer refused. It is still refused when the network's activation height is unknown.
• config_blake2b.go: on regtest and simnet the configured bitcoin.blake2b-activation-height is copied into the btcd params handed to neutrino. The copy leaves the package-level params untouched. Mainnet and testnet4 use the heights in btcd's chaincfg.
• docs/blake2b.md: the backends section now covers neutrino. It needs peers that serve compact filters (blockfilterindex=1, peerblockfilters=1) set with neutrino.connect/addpeer, since DNS seeds don't know which nodes follow this chain.
• Tests: TestBackendChoicesRefused now refuses only btcd, and a new TestNeutrinoBlake2bParams checks the params handed to neutrino.
• go.mod: points btcd and neutrino at the branches below by pseudo-version. Once blockchain: validate BLAKE2b (header v2) proof of work btcd-blake2b#1 is tagged, and neutrino lives wherever you prefer (fork it or take ours), these should become proper versions:
  • btcd: Kilombino/btcd-blake2b blake2b-pow (same commit as blockchain: validate BLAKE2b (header v2) proof of work btcd-blake2b#1)
  • neutrino: Kilombino/neutrino blake2b, from v0.17.1. The header file now holds 80-byte records below the activation height and 164-byte records from it, so offsets stay a formula. WriteHeaders refuses a header whose format doesn't match its height. Without Blake2bHeight the layout is unchanged, so existing files read the same.

A neutrino node doesn't need the chain-identity check: a peer on the SHA256d chain fails at the activation height, because its headers there are still v1.

Tested on regtest (Knots v29.4.2, testactivationheight=blake2b@150, filters on):

• neutrino lnd syncs headers and cfheaders across the activation;
• opens a 1M sat channel to a bitcoind-backed lnd;
• pays invoices both ways;
• closes cooperatively.

The whole mainnet header chain also validates under the btcd change (see paulscode/btcd-blake2b#1). I haven't run neutrino against mainnet peers yet.

[paulscode]

Thanks for this, and for testing it end to end. I ran it too: the PR build
with the neutrino backend against a BLAKE2b Knots regtest (activation at
200, filters on) synced past activation, found and spent coins, served v2
headers over lncli neutrino getblockheader, and resynced after a restart.
Pointed at a SHA256d regtest node, it refused the chain at height 200 and
stayed at 0. So the core of it works.

I'd like to take it, but a few things need to change first.

Where the code comes from. I don't want a release to build btcd and
neutrino from another account by pseudo-version. go.sum pins the content, but
the repos and their history would still live outside the project. So:

• btcd: your btcd-blake2b#1 is merged and tagged v0.25.1-blake2b.6, and
  blake2b-next already uses it.
• neutrino: I've put your fork at paulscode/neutrino-blake2b (your two
  commits and authorship kept, module path unchanged, btcd pointed at the
  new tag). I'll tag it v0.17.1-blake2b.1 once the two fixes below are in;
  PRs against its blake2b branch are the easiest way. If you'd rather keep
  maintaining it yourself, tell me and we can work out how.

Then this PR, rebased on blake2b-next, would point at those two tags.

Two fixes in the neutrino fork:

1. Ban peers whose headers fail with ErrBadHeaderV2, rather than only
   disconnecting them (blockmanager.go:2492-2496 and :2622-2627).
   Mainnet and testnet4 use Bitcoin's DNS seeds, which return almost only
   SHA256d nodes. Each one is refused at 961640, disconnected, and can be
   dialled again, and startSync favours the peer with the highest
   LastBlock, which is usually one of them. A node with only SHA256d peers
   sits at 961639 with nothing in the log but sanity check failures. Those
   peers also get every transaction from pushtx. Something like
   b.cfg.BanPeer(addr, banman.InvalidBlock) for that error would do it.
2. When the header file ends in a partial record, cut it back to the last
   whole header at open instead of failing (headerfs/store.go:320, the
   check in layout.count). Today the only way out is deleting the files by
   hand. Upstream floors the count and carries on, which misaligns later
   appends, so refusing is safer than upstream, but trimming is better than
   both.

Smaller ones, fine as follow-ups:

• Record the activation height used for the layout in the index bucket and
  refuse to open on a mismatch. On regtest a changed
  bitcoin.blake2b-activation-height would read the file at the wrong
  offsets.
• readHeaderRange could check it parsed exactly the number of headers it
  asked for, and WriteHeaders could check that the write offset matches
  the file size.
• chainimport reads only 80-byte records, so neutrino.blockheaderssource
  can't import past activation. Refusing those options when BLAKE2b is
  configured is enough.

Changes in this PR:

1. Check the chain's identity on mainnet. The docs say a neutrino node
   doesn't need the chain identity check. That's true against SHA256d peers,
   but neutrino would follow any most-work chain that passes the BLAKE2b
   rules from activation, and the mainnet activation hash
   (chainreg.Blake2bMainnetActivationHash) is never compared. Please:
   • add a checkpoint at 961640 with that hash to the params handed to
     neutrino, appending to a copy of Checkpoints so the shared parameters
     stay untouched;
   • once synced, compare GetBlockHash(961640) with the pin and write
     chain-identity.json as the bitcoind path does. My StartOS package's
     health check and the Umbrel app's status page both read that file, so
     without it they'd show the chain as unchecked if they ever offered
     neutrino.
2. Peers. On mainnet and testnet4, require neutrino.connect, or at
   least log a clear warning when only neutrino.addpeer is set. addpeer
   still mixes in DNS seed peers.
3. Channel validation. Under neutrino, routing.assumechanvalid follows
   !neutrino.validatechannels (config_builder.go:1750), so by default
   channel announcements aren't checked against the chain at all. For this
   fork I'd make neutrino.validatechannels default to true, or at least
   document it.
4. Docs (docs/blake2b.md, the backends section). Please add what a light
   client doesn't check: transaction and script validity, so neither the
   unified signature hash rule nor the long coinbase maturity is enforced.
   With this chain's small hashrate, a miner could feed a neutrino node an
   invalid block more cheaply than on Bitcoin, so I'd recommend
   neutrino.connect to your own Knots node. Also say that mainnet needs
   fee.url, and that the public fee estimators follow the SHA256 chain.
5. Smaller fixes:
   • config.go:1481: the default error still says only bitcoind and
     nochainbackend are supported.
   • config_blake2b.go:49: range-check the height before converting to
     int32. A value above 2^31 turns negative and gives a misleading error.
   • Testnet4: neutrino uses btcd's built-in 150308; the docs could say
     bitcoin.blake2b-activation-height overrides it.
6. Tests:
   • TestBackendChoicesRefused only checks that the neutrino error doesn't
     contain "not supported". Please add a case that regtest and simnet
     without a height are refused, and one for the mainnet checkpoint.
   • If you can, a short lab or itest run with v2 headers through neutrino.
     The unit tests run on a SHA256d regtest.

Neither package (StartOS, Umbrel) offers neutrino today, since both set
bitcoin.node=bitcoind, so nothing changes for existing users. Thanks again.
It's a real gain for people who don't run a full node.

[Kilombino]

Thanks for the careful review. I've rebased onto blake2b-next (base changed) and gone through your points.

Where the code comes from

• btcd: v0.25.1-blake2b.6, taken from blake2b-next.
• neutrino: the two fixes, plus the three smaller ones, are in Ban SHA256d header peers, trim a partial header record, and smaller fixes neutrino-blake2b#1. Until it is merged and tagged, go.mod points at that PR's commit by pseudo-version (the last commit). I'll switch it to v0.17.1-blake2b.1 once the tag exists. You maintaining the fork is fine with me.

Changes in this PR

1. Chain identity on mainnet.
   • chainreg.NeutrinoChainParams hands neutrino a copy of the params with a checkpoint at 961640 (Blake2bMainnetActivationHash). The shared Checkpoints are left untouched.
   • Under neutrino, NewPartialChainControl now runs the same chainIdentityChecker as the bitcoind path, through an adapter (neutrinoIdentityRPC) that answers GetBlockChainInfo, GetBlockHash and getblockheader from neutrino's validated headers. So it waits for the activation height, checks the v2 header and the id against the pin, writes chain-identity.json, and re-checks every 5 minutes. getdeploymentinfo is reported as unavailable, so reduced_data is left out.
2. Peers. Mainnet and testnet4 refuse neutrino without neutrino.connect. neutrino.addpeer alone starts with a warning.
3. Channel validation. neutrino.validatechannels defaults to true. routing.assumechanvalid=true turns it off instead of erroring.
4. Docs. The backends section now has a neutrino subsection covering:
   • the ban, the checkpoint and the identity file;
   • peers;
   • what a light client does not check (transaction and script validity, the unified signature hash rule, long coinbase maturity), the low hashrate risk, and the advice to neutrino.connect to your own Knots node;
   • the channel validation default;
   • fee.url on mainnet, and the SHA256d public estimators;
   • the testnet4 built-in height and its override;
   • the header import being unsupported.
5. Smaller fixes:
   • the unknown-backend error lists neutrino;
   • the height is range-checked before int32;
   • on testnet4 the check takes btcd's built-in 150308;
   • neutrino.blockheaderssource / filterheaderssource are refused in LND as well as in neutrino.
6. Tests:
   • regtest and simnet without a height are refused;
   • the checkpoint is on a copy, sorted, and absent on testnet4;
   • the adapter confirms the real activation header and refuses the last SHA256d header in its place (status file checked both ways);
   • the peer and import rules;
   • the testnet4 height and the out-of-range height.

Run on regtest (Knots v29.4.2, testactivationheight=blake2b@150, filters on), this build with the neutrino PR:

• neutrino lnd synced past 150, logged Bitcoin BLAKE2b chain confirmed: block 150 … and wrote chain-identity.json (confirmed);
• it opened a 1M sat channel to a bitcoind-backed lnd (visible in its graph with validation on) and paid invoices both ways;
• after a restart with 3 new blocks it resynced to the tip and confirmed again;
• pointed at a SHA256d Knots regtest (200 blocks, filters on), it logged breaks a BLAKE2b header rule … -- banning peer at 150 and banned it for 48h.

I haven't run it against mainnet peers yet; our Knots node isn't serving filters to peers yet.

[Kilombino]

One more commit, found while testing the SHA256d case. A daemon still waiting for the backend to reach the activation height didn't stop on SIGTERM. The wait only watched the chain control's own quit channel, which is closed after it returns. That was already the case with bitcoind (a node that never gets there). With neutrino it's easier to hit, because a node whose only peers are SHA256d ones bans them all and waits at 0 headers.

The wait now also ends on the interceptor's shutdown (chainreg.Config.Shutdown). After the change, SIGTERM in that state logs shutting down while waiting for the BLAKE2b activation height and exits. There's a test for it.

[Kilombino]

Mainnet run. Our Knots node (v29.4.2) now serves filters (peerblockfilters=1). I synced this PR's build with bitcoin.node=neutrino and neutrino.connect set to that node, using a local fee.url, from an empty data dir:

• block and filter headers from genesis to the tip (976864) in about 35 s over the LAN. On disk: block_headers.bin 76 MB, reg_filter_headers.bin 30 MB, neutrino.db 112 MB;
• Bitcoin BLAKE2b chain confirmed: block 961640 is 0000000000000050c1e5f69672f459293be14f46e5a494e7a8c8541396f18eeb (164-byte header v2), with chain-identity.json written as confirmed and the pinned hash;
• Chain backend is fully synced! end_height=976864.

[paulscode]

Thanks, this addresses the review point by point: the checkpoint at 961640, chain-identity.json under neutrino, the connect rule, validatechannels on by default, and the docs all look right. It also cherry-picks cleanly onto blake2b-next 9cbe408, and the build and the chainreg, lncfg and config tests pass there.

What's left:

1. go.mod still points at github.com/Kilombino/neutrino-blake2b. Once neutrino-blake2b#1 is merged, please switch it to github.com/paulscode/neutrino-blake2b v0.17.1-blake2b.1.
2. Mainnet without fee.url only fails after the node has synced to 961640 (chainregistry.go:748 runs after the wait). Could checkNeutrinoBlake2b refuse it up front?
3. The validatechannels help text (lncfg/neutrino.go:21) still says validation is off by default. It could also say that neutrino.validatechannels=false only works in the config file, and routing.assumechanvalid=true is the way to turn it off from the command line.
4. In the docs, please say "the SHA256 chain's DNS seeds" rather than "Bitcoin's", and that bans last 24 hours rather than for good.

If you can, a short regtest run with v2 headers through the final build would be good before release. Thanks again for the careful work.

[paulscode]

One more small thing from CI: the fork checks fail on formatting only. gofmt -l lists config_blake2b.go (the spacing around + in the neutrino.connect error) and config_builder.go (the field alignment in the neutrino.Config literal). gofmt -w config_blake2b.go config_builder.go fixes both; everything else in that run passed.

[Kilombino]

Thanks. 2 to 4 and the gofmt are in (76607e3, 01a5e6f); 1 and the regtest run follow the neutrino tag:

• gofmt: config_blake2b.go and config_builder.go, nothing else changed in them.
• 2: checkNeutrinoBlake2b refuses mainnet neutrino without fee.url up front, with the same message chainreg uses; TestCheckNeutrinoBlake2b covers it (its mainnet configs now carry a fee URL, and a case without one).
• 3: the validatechannels help (and the matching sample-lnd.conf comment) says it is on by default here, that neutrino.validatechannels=false only takes effect in the config file because a boolean flag can't be set to false on the command line, and that routing.assumechanvalid=true turns it off there.
• 4: "the SHA256 chain's DNS seeds". On the ban length: inside lnd it is 48 hours, not 24. config_builder.go sets neutrino.BanDuration = time.Hour * 48 (upstream does the same, and neutrino.banduration is never read), so the doc says 48 hours and that lnd sets it. It also says a peer named in neutrino.connect/neutrino.addpeer is only disconnected, which holds once go.mod moves to the neutrino release with neutrino-blake2b#1's persistent-peer change.
• 1: neutrino-blake2b#1 has your five points addressed (comment there); once you tag v0.17.1-blake2b.1 I'll switch go.mod to github.com/paulscode/neutrino-blake2b v0.17.1-blake2b.1 and do the regtest run with v2 headers on that final build before asking for the merge.

[paulscode]

Thanks for these. I confirmed the BanDuration point (48 hours in config_builder.go, and neutrino.banduration is never read), and the validatechannels help and sample-lnd.conf wording is right. Testnet4, regtest and simnet are unaffected, and the check runs before the wallet unlock.

One problem: the new check at config_blake2b.go:187 only looks at fee.url, but the deprecated feeurl option is copied into fee.url later, in chainreg. So a mainnet neutrino node that still sets feeurl= is now refused at start even though it has a fee URL. Could you check both (cfg.Fee.URL == "" && cfg.FeeURL == "") and add a test case?

neutrino-blake2b#1 is merged and tagged as v0.17.1-blake2b.1, so go.mod can now point at github.com/paulscode/neutrino-blake2b v0.17.1-blake2b.1. The docs already describe its persistent-peer behaviour. Then the regtest run with v2 headers, and I'll merge.

[Kilombino]

Both points are in (a879f42):

• the mainnet neutrino check now passes when either fee.url or the deprecated feeurl is set (cfg.Fee.URL == "" && cfg.FeeURL == ""), with a test case for feeurl alone;
• go.mod points at github.com/paulscode/neutrino-blake2b v0.17.1-blake2b.1.

The regtest run with v2 headers follows; I'll post the results here.

[Kilombino]

Regtest run with v2 (164-byte BLAKE2b) headers, on the final build (a879f42):

• Build: lnd version 0.21.4-beta-blake2b.20 commit=a879f42e7, with replace github.com/lightninglabs/neutrino => github.com/paulscode/neutrino-blake2b v0.17.1-blake2b.1. Tags: signrpc walletrpc chainrpc invoicesrpc routerrpc neutrinorpc peersrpc.
• Node: Bitcoin Knots v29.4.2.knots20260508 (BLAKE2b), regtest, fresh datadir, testactivationheight=blake2b@150, blockfilterindex=1, peerblockfilters=1.
• Chain: heights 0–149 have 80-byte headers. From 150 on, headers are 164 bytes with bit 31 set in the raw version. Mined to 300.
• Sync: node A on the bitcoind backend, node B on neutrino (neutrino.connect, bitcoin.blake2b-activation-height=150).
  • B synced headers and cfheaders 0→300 in one pass and logged Bitcoin BLAKE2b chain confirmed: block 150 … (164-byte header v2). Both nodes reported synced_to_chain=true.
  • block_headers.bin is exactly 150×80 + 151×164 bytes. neutrino getblockheader matches the node at 149, 150 and 300.
• Funds: on-chain deposits to B's p2wkh and p2tr addresses confirmed and were spendable.
• Channel:
  • B (neutrino) opened 5M sat to A with a 1M push, and the channel went active.
  • Payments B→A (50k) and A→B (30k) succeeded.
  • Cooperative close confirmed; B shows COOPERATIVE_CLOSE with its balance settled and nothing pending.
• Restart: B was restarted twice, once with the channel open and blocks mined while it was down, and once after the close.
  • Both times it reloaded the header store, caught up on headers and cfheaders, and logged the chain confirmation again. The channel came back active.
  • Store sizes stayed exactly 150×80 + n×164.
• Startup check: on regtest B starts without fee.url/feeurl, since the check is mainnet-only. go test -run 'Neutrino|FeeURL|Blake2b' . passes, including TestCheckNeutrinoBlake2b: mainnet without a fee URL is refused, the deprecated feeurl is accepted, neutrino.connect is required, and an 80-byte headers source is refused.
• Unrelated to this PR, as far as I can tell: the first closechannel --sat_per_vbyte 2 from B failed with latest fee proposal exceeds max fee. That came from A's regtest estimator, under the legacy closing_signed negotiation. On reconnect, lnd resumed the shutdown with B's static estimate and settled at about 30 sat/vB. That looks like the upstream behaviour.

[paulscode]

Thanks for the thorough regtest run with v2 headers, and for all the rounds on this. Merged; it goes out in the next release. The cooperative close's first fee proposal is lnd's legacy closing_signed negotiation on regtest, as you say, not this PR.