Skip to content

fix(storage): detect signet_<hash> directories and respect network arg - #23

Merged
secsovereign merged 7 commits into
mainfrom
cursor/fix-bitcoin-core-datadir-detection-efcf
Oct 15, 2026
Merged

secsovereign merged 7 commits into
mainfrom
cursor/fix-bitcoin-core-datadir-detection-efcf

Conversation

@secsovereign

@secsovereign secsovereign commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

Follow-up to PR #22 (testnet4 support, merged as badc23e). Fixes how BLVM decides which network's Bitcoin Core block files to read when it reuses a Core datadir in place (storage.reuse_core_block_files, on by default).

On main, a Core datadir whose network could not be detected silently became mainnet, so a custom signet (signet_<hash>) or an unrecognised folder could get a reader with the wrong network magic. This PR stops that without dropping reuse for real mainnet datadirs with non-standard names.

Network rule for Core block reuse

BitcoinCoreDetection::resolve_reuse_network(core_dir, marker_network, node_network) decides the network from three sources, in this order:

  1. the network in the BLVM migration marker (blvm_meta/migration.json); empty counts as absent
  2. the Core datadir folder name (.bitcoin/bitcoind → mainnet, testnet3, testnet4, signet, regtest)
  3. the network the node is running (open_for_node passes it; library callers pass consensus_network)

Every source that is known must agree, and the first known one wins. Reuse is skipped with a warning, and no reader is opened, when:

  • the folder is a custom signet (signet_<hash>), whatever the marker or node say, because its block magic is not the standard signet magic
  • two known sources disagree (e.g. a mainnet marker on .bitcoin/signet, or a mainnet node on .bitcoin/testnet3)
  • the marker names a network BLVM doesn't know
  • no source is known

So ~/Library/Application Support/Bitcoin, %APPDATA%\Bitcoin, bitcoin or /srv/btc-data opened by a mainnet node keep block reuse, and nothing falls back to mainnet by default.

Changes

src/storage/bitcoin_detection.rs

  • detect_network: custom signet (signet_<hash>) and unknown folders return None instead of mainnet.
  • detect_network_strict (new): an error for custom signets and unknown folders. The custom signet message names the setting that actually applies (storage.reuse_core_block_files = false), not a --network flag.
  • resolve_reuse_network (new): the rule above.
  • is_bitcoin_core_dir: now public, and checks the network's subfolder (testnet3/, testnet4/, signet/, regtest/; mainnet at the root).

src/storage/mod.rs

  • open_core_block_reader_for_store takes the node network and uses resolve_reuse_network. The marker's network is no longer trusted unchecked.
  • open_blockstore and both storage constructors pass the caller's consensus_network through. with_backend_pruning_and_indexing no longer swaps an unknown network for Some(Mainnet) before Core reuse.

Small accessors

  • BitcoinCoreBlockReader::network() and BlockStore::bitcoin_core_reader_network() (rocksdb), used by the tests.

CI

  • New Run tests (rocksdb feature) step: cargo test --features rocksdb. The rocksdb-gated Core import and reuse code was never compiled into a test run before. That's how the E0624 in the previous head went unnoticed.

Tests

  • src/storage/core_reuse_tests.rs (new, #[cfg(all(test, feature = "rocksdb"))]): calls open_core_block_reader_for_store from inside the crate. It replaces the integration test that called this pub(crate) fn and failed to compile with E0624. It covers:
    • non-standard mainnet names (bitcoin, Bitcoin, srv/btc-data) with a mainnet node → mainnet reader
    • the same name with no marker and no node network → no reader
    • a mainnet marker on .bitcoin/signet → no reader
    • a signet marker on signet_<hash> → no reader (with and without a signet node)
    • marker vs folder, marker vs node, and folder vs node disagreements → no reader
    • agreeing sources → reader on that network
    • end to end through Storage::with_backend_pruning_and_indexing
  • bitcoin_detection unit tests for resolve_reuse_network: priority, every disagreement pair, custom signet with every marker/node combination, an unknown marker network, and the wording of the custom signet message. These run with default features too.

Results:

Run locally at b79ba28 (crates.io deps as in CI: [patch.crates-io] stripped, blvm-protocol 0.1.30, blvm-consensus 0.1.54; -j 2):

default features --features rocksdb
--lib 1329 passed, 2 ignored 1346 passed, 2 ignored
--test bitcoin_core_tests 1 passed 21 passed
--test storage_tests 36 passed 36 passed
--test core_drop_in_integration (rocksdb only) 8 passed
cargo clippy -- -D warnings --lib --bins --examples clean --lib clean

cargo fmt --check is clean. One default-features --lib run failed node::parallel_ibd::tip_stage::tests::tip_crawl_supply_healthy_requires_past_body_and_fast_gd_or_stream, which is timing-sensitive and not touched here. It passed 3/3 in isolation and in the full rerun. The other ~195 integration test binaries were not run locally because of disk on the shared box. CI's Run tests and the new Run tests (rocksdb feature) step cover them.

Revert checks (each fix reverted alone, cargo test --features rocksdb):

  • B2, node network not passed to the reader: 5 tests fail (nonstandard_mainnet_names_reuse_with_node_network_mainnet, nonstandard_name_follows_node_network, open_for_node_passes_the_node_network_to_core_reuse, folder_and_node_disagreement_is_refused, marker_and_node_disagreement_is_refused)
  • S3, marker trusted unchecked: 5 tests fail (mainnet_marker_on_standard_signet_folder_is_refused, marker_and_folder_disagreement_is_refused, marker_and_node_disagreement_is_refused, resolve_marker_folder_disagreement_refuses, resolve_node_disagreement_refuses)
  • custom signet guard removed from the resolver: 4 tests fail (custom_signet_core_dir_does_not_produce_a_reader, signet_marker_on_custom_signet_folder_is_refused, resolve_custom_signet_refuses_whatever_the_other_sources_say, custom_signet_error_names_the_real_setting)
  • old --network wording: custom_signet_error_names_the_real_setting fails
  • B1, the old integration test put back: --test bitcoin_core_tests fails to compile (E0624, private fn)

Out of scope

  • The consensus_network.unwrap_or(Mainnet) defaults still used for pruning in with_backend_pruning_and_indexing / with_backend_pruning_indexing_and_compression are unchanged. Core block reuse no longer depends on them.
  • When reuse is refused, the node keeps running without the Core reader and logs a warning. It does not fail startup.

cursoragent and others added 2 commits October 7, 2026 21:27
Follow-up to PR #22 (testnet4 support):

1. detect_network: Add testnet4 detection. Custom signet directories
   (signet_<hash>) now return None instead of silently falling back to
   mainnet. The caller must decide how to handle unknown networks.

2. detect_network_strict: New method that returns an explicit error for
   custom signet directories, with a clear message instructing the user
   to specify --network signet.

3. is_bitcoin_core_dir: Now respects the network argument. Mainnet data
   is expected at the root of base_dir, while other networks use their
   standard subdirectories (testnet3/, testnet4/, signet/, regtest/).

4. Tests: Added comprehensive unit tests and integration-style tests
   for testnet4 datadir detection, custom signet handling, and
   network-specific directory layout validation.

Co-authored-by: Secure Sovereign <secsovereign@users.noreply.github.com>
The production caller in open_core_block_reader_for_store was still
using .unwrap_or(CoreDataNetwork::Mainnet) after detect_network returned
None for custom signet directories. This silently opened a Mainnet reader
for signet_<hash> directories.

Changes:
- Replace detect_network + unwrap_or(Mainnet) with detect_network_strict
- Log a clear warning and return None when network cannot be determined
- Make open_core_block_reader_for_store pub(crate) for testing

Tests:
- test_custom_signet_core_dir_does_not_produce_mainnet_reader: verifies
  that signet_<hash> directories with reuse enabled return None, not a
  Mainnet reader
- test_standard_signet_core_dir_detection_works: verifies standard signet
  directories are still detected correctly

Co-authored-by: Secure Sovereign <secsovereign@users.noreply.github.com>
@cursor
cursor Bot force-pushed the cursor/fix-bitcoin-core-datadir-detection-efcf branch from 98cb229 to 2bdd1d1 Compare October 7, 2026 21:38
@secsovereign
secsovereign marked this pull request as ready for review October 7, 2026 22:06
…er and node

Core block-file reuse now resolves the network with
BitcoinCoreDetection::resolve_reuse_network: migration marker, then the
Core folder name, then the node's configured network. Every known source
must agree; custom signet (signet_<hash>) folders are always refused.

- B2: non-standard mainnet datadirs (Bitcoin, bitcoin, /srv/btc-data)
  keep reuse when the node runs mainnet; nothing defaults to mainnet.
- S3: the marker's network is validated instead of trusted.
- B1: the reader-level tests move into the crate
  (src/storage/core_reuse_tests.rs, cfg(all(test, feature="rocksdb")))
  since open_core_block_reader_for_store is pub(crate).
- The custom signet error names storage.reuse_core_block_files instead
  of a --network flag that never reached this code.
The rocksdb-gated Core import and block-reuse code was never compiled
into a CI test run.
@secsovereign
secsovereign merged commit 8dfc1fd into main Oct 15, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants