Skip to content

Latest commit

 

History

History
165 lines (138 loc) · 20.1 KB

consensus-tests.md

File metadata and controls

165 lines (138 loc) · 20.1 KB

Consensus-rule test matrix

Every consensus rule we implement (not delegated wholesale to rust-bitcoin) has an automated test that would fail if the check were removed or inverted.

Out of scope: secp256k1, which stays the shared library. Script opcode parity against Bitcoin Core is this corpus (crates/rbitcoin-consensus/tests/fixtures/script_corpus/) plus Core's JSON (script_tests.json, tx_valid.json, tx_invalid.json, sighash.json), both checked against Core v31.1 VerifyScript. Not every historical mainnet retarget. One boundary is pinned: height 32256 compact 0x1d00d86a, plus the quarter, 4×, and pow-limit clamp literals (mainnet_retarget_32256_matches_core_compact, retarget_clamps_timespan_and_caps_at_pow_limit).

Running

nix-shell
cargo test -p rbitcoin-consensus --test suite
cargo test -p rbitcoin-test --test suite consensus_rules::
# Hornet spec.h / spec.html mapped subset (same tests, named selector):
./scripts/test-hornet-rules.sh
# Core JSON corpora (script_tests / tx_valid / tx_invalid / sighash / BIP341):
cargo test -p rbitcoin-consensus --test suite core_script_tests_all_rows -- --nocapture
cargo test -p rbitcoin-consensus --test suite core_tx_ -- --nocapture
cargo test -p rbitcoin-consensus --test suite core_sighash -- --nocapture
cargo test -p rbitcoin-consensus --test suite core_bip341 -- --nocapture
cargo test -p rbitcoin-consensus --test suite script_corpus_rows_match_our_interpreter -- --quiet
cargo test --manifest-path script-oracle/Cargo.toml --quiet
cargo test -p rbitcoin-consensus --test suite block_866342 -- --nocapture
cargo test -p rbitcoin-consensus --test suite script_edge
# broader integration still covers connect success paths:
cargo test -p rbitcoin-test --test suite scenarios::consensus_

Core consensus corpora (1:1 surface)

Staged each cargo test run from the Bitcoin Core v31.1 submodule third_party/bitcoin/src/test/data/ (MIT). Offline after ./scripts/core-functional/init-submodule.sh. Bump the gitlink pin when refreshing; do not check copies into tests/fixtures/.

Fixture Path Harness Success criterion
script_tests.json third_party/bitcoin/src/test/data/ (staged to $CARGO_TARGET_DIR/core-data/) script::core_vectors::core_script_tests_all_rows every data row; fail == 0 (no allowlist)
tx_valid.json same script::core_tx_vectors::core_tx_valid_all_rows every data row accept
tx_invalid.json same script::core_tx_vectors::core_tx_invalid_all_rows every data row reject
sighash.json same script::core_sighash::core_sighash_all_rows every data row digest matches Core
bip341_wallet_vectors.json same script::core_bip341::core_bip341_wallet_vectors_all_rows key-path fully-signed + per-input spends accept; unknown-leaf script-path accepts
script_corpus/script_tests.json tests/fixtures/script_corpus/ (ours, Core row form) script::core_vectors::script_corpus_rows_match_our_interpreter and script-oracle every row accept/reject matches the file; the oracle is Core v31.1 VerifyScript
mainnet 866342 + prevouts tests/fixtures/block_866342/ (Floresta zstd) block::block_866342::block_866342_structure_scripts_and_overweight structure at height + every non-coinbase verify_job_all_inputs; extra-NOP clone is 4_000_001 WU

How the harness works

  1. Stage JSON from the submodule (not network, not an in-tree copy).
  2. Parse Core script language / hex txs.
  3. Build Core-style credit/spend txs (script_tests) or deserialize fixture txs (tx_*).
  4. Call shipped verify_job_all_inputs(ScriptCheckJob) (or bare EvalScript+P2SH path when WITNESS flag is off — Core treats v0 programs as bare without that flag).
  5. Compare accept/reject to Core’s expected code. Named error codes only require reject, not exact code string.

No allowlist

  • Soft majority pass rates are not success criteria.
  • There is no row skip inventory. A mismatch fails the test; fix the engine or the fixture interpretation before commit.
  • Status: Core JSON corpora green on the shipped path (script_tests 1222/1222 on Core v31.1, tx_valid 121/121, tx_invalid 93/93, sighash 500/500, bip341_wallet_vectors 9/9).

rust-bitcoin vs Core fixtures

Topic Doc
rust-bitcoin gaps we wrap rust-bitcoin-limitations.md
External consensus findings external_findings/

Origin / update

See crates/rbitcoin-consensus/tests/fixtures/README.md.

Hornet’s published 34-rule table is a gap checklist against this matrix, not a second spec. Full ID-by-ID happy/boundary map (Hornet main spec.h @ 151462fa vs spec.html): peer-clients.md § Hornet block-validation rules. Selector: ./scripts/test-hornet-rules.sh (runs structure_rule_tests, header version/time, finality_tests, sigop_cost_tests, consensus_rules).

A. Block structure — validate_block_structure_hashed

ID Rule Error signal Test
S1 Block has ≥1 tx BadBlock("no transactions") (bad-blk-length) rpc_regtest_from_genesis (tip extend and an equal-work sibling)
S2 First tx is coinbase; without one, any 64-byte tx marks the body mutated (Core IsBlockMutated) BadBlock("first tx not coinbase"), BadBlock("merkle mutated by a 64-byte tx") (both bad-cb-missing) rpc_regtest_from_genesis (tip extend, 64-byte uncached, and an equal-work sibling); no_coinbase_with_a_64_byte_tx_is_mutated
S3 No later coinbase BadBlock("coinbase not first") (bad-cb-multiple) rpc_regtest_mature_chain_ops (chain_ops_submit_repeated_txids) and rpc_regtest_from_genesis (equal-work sibling); tip cache chain::tests::hostile_peer_session
S4 Weight ≤ 4_000_000 WU BadBlock("…weight…") s4_rejects_overweight_block, s4_weight_4_000_000_accepts_4_000_001_rejects
S5 Unique txids BadBlock("duplicate txid") (bad-txns-inputs-missingorspent, as Core rejects the second copy at connect) structure_rule_tests::s5_rejects_duplicate_txid; tip cache chain::tests::hostile_peer_session
S6 Merkle root matches txids, checked before every other body rule; a repeated tail (CVE-2012-2459) is mutated BadBlock("merkle root mismatch"), BadBlock("bad-txns-duplicate"); RPC bad-txnmrklroot structure_rule_tests::s6_rejects_merkle_root_mismatch, body_rules_run_after_the_header_merkle_check (+ merkle_root_bytes_single_and_odd; store merkle_mutation_flags_a_repeated_tail_at_any_level; RPC rpc_regtest_from_genesis)
S7 BIP34 height in coinbase (h≥1) BadBlock("bip34…") s7_rejects_bip34_missing_at_height_1, s7_bip34_not_required_at_height_0, s7_regtest_rejects_bip34_missing_at_height_1, s7_regtest_bip34_activation_height_override
S8 Witness commitment when any witness; the last output of at least 38 bytes starting 6a24aa21a9ed is the commitment; a commitment needs a nonce of exactly 32 bytes; reject witness before SegWit activation missing / mismatch / bad-witness-nonce-size / BadBlock("unexpected witness before segwit") rpc_regtest_from_genesis (missing witness commitment, witness commitment mismatch); s8_last_witness_commitment_output_wins (last ≥38-byte output; a 37-byte trailer does not count); s8_coinbase_witness_without_commitment_is_missing (coinbase witness alone); s8_accepts_witness_commitment_with_reserved_value and s8_rejects_empty_or_multi_item_coinbase_witness_reserved (empty and multi-item nonce); s8_rejects_nonce_lengths_beside_32 (31 and 33 bytes); s8_mainnet_accepts_pre_segwit_commitment_magic_without_nonce (pre-segwit aa21a9ed is data); consensus_rules::header_and_spending_boundaries (connect-path unexpected witness before segwit)
S9 Coinbase scriptSig length 2..=100, after the merkle check bad-cb-length s9_coinbase_scriptsig_length_two_accepts_one_rejects (length 2 accepts; length 1 is bad-cb-length even when the byte is the BIP34 push); rpc_regtest_from_genesis (under 2 and 101 reject; a padded height push and 100 bytes connect); short_coinbase_under_a_real_header_is_merkle_mismatch
S10 Output value / sum ≤ MAX_MONEY toolarge rpc_regtest_from_genesis submit (bad-txns-vout-toolarge above MAX_MONEY; MAX_MONEY is bad-cb-amount; two under-max outputs sum to bad-txns-txouttotal-toolarge)
S11 Legacy sigops cost ≤ 80_000 bad-blk-sigops rpc_regtest_from_genesis (20_000 connect / 20_001 reject)
S12 Connect: P2SH + witness sigops (BIP16/BIP141); P2SH scriptSig opcode > OP_16 → 0; witness sigops whenever the WITNESS script flag is set (every block except the BIP16 exception, as in Core). Empty P2WSH and a v1 witness program cost 0; empty P2WPKH still costs 1 bad-blk-sigops sigop_cost_tests::* + witness_empty_p2wsh_and_v1_program_cost_zero + witness_p2wpkh_counts_one + p2sh_sigops_non_push_scriptsig_is_zero + witness_sigops_gated_on_witness_flag + script_flags_follow_core_exception_table
S13 Every tx including coinbase has ≥1 output no outputs rpc_regtest_from_genesis submit of a coinbase with no outputs (bad-txns-vout-empty); header_and_spending_boundaries (non-coinbase empty vout)
S14 Stripped size ≤ 1_000_000 block stripped size too large s14_stripped_size_1_000_000_accepts_1_000_001_rejects
S15 Every tx has ≥1 input no inputs s15_rejects_empty_vin
S16 Tx stripped size ≤ 1_000_000 bad-txns-oversize s16_tx_stripped_size_1_000_000_accepts_1_000_001_rejects
S17 No duplicate outpoints in a tx bad-txns-inputs-duplicate rpc_regtest_from_genesis
S18 Non-coinbase inputs non-null bad-txns-prevout-null rpc_regtest_from_genesis

The Test column is the witness. Do not copy it into TESTING.md or peer-clients.md.

B. Header — validate_header / helpers

ID Rule Error signal Test
H1 Genesis hash matches params BadHeader("genesis hash mismatch") header_and_spending_boundaries
H2 prev links to height−1 BadPrev header_and_spending_boundaries
H3 time > median_time_past timestamp <= median-time-past header_and_spending_boundaries (time==mtp / mtp+1)
H5 bits == expected_next_bits incorrect proof of work bits header_and_spending_boundaries (regtest: must equal prev). Mainnet compact and clamps: mainnet_retarget_32256_matches_core_compact, retarget_clamps_timespan_and_caps_at_pow_limit
H6 Target ≤ pow_limit target above pow limit header_and_spending_boundaries (validate_header with mainnet pow_limit)
H7 PoW valid for claimed bits InvalidPow h7_rejects_header_hash_above_target + smoke via mine_regtest_block accept
H8 Time not > now + 2h timestamp too far in future h8_rejects_timestamp_too_far_in_future + h8_timestamp_exactly_two_hours_accepts_plus_one_rejects (exact +2h is the header unit; not duplicated on the connect journey)
H9 assemble_run future-time + BIP34/66/65 nVersion on every block time-too-new / bad-version check_header_version_and_future_time_regtest + h9_version_floors_at_bip34_66_65 + assemble_second_block_rejects_stale_nversion
H10 Testnet min-difficulty after 20 min expected_next_bits = powLimit testnet_min_difficulty_after_20_minute_gap

Location: crates/rbitcoin-test/tests/consensus_rules.rs (connect-path header walks) and crates/rbitcoin-consensus/src/header.rs (median_time_past_tests: version floors and exact +2h).

C. Connect — accept_and_connect_block / structural_validate_spends

ID Rule Error signal Test
C15 Core tx_valid / tx_invalid accept / reject at listed flags; valid still accepts with extra implemented flags off (FillFlags-implied bits skipped); invalid still rejects with extra restriction flags on (not P2SH/WITNESS/TAPROOT class changes) script::core_tx_vectors::*
C16 Core script_tests.json accept / reject script::core_vectors::core_script_tests_all_rows
C17 Stack + altstack share MAX_STACK_SIZE stack size stack_and_altstack_share_max_size_on_pushdata (022)
C18 Core sighash.json 32-byte digest via SighashCache::legacy_signature_hash script::core_sighash::core_sighash_all_rows
C19 Core bip341_wallet_vectors.json verify_job_all_inputs accept (taproot+witness) script::core_bip341::core_bip341_wallet_vectors_all_rows
C20 Mainnet 866342 + Floresta prevouts structure + scripts; overweight 4_000_001 WU rejects block::block_866342::*
C21 Tapscript initial witness 1000 / 520 stack size / PUSH_SIZE; OP_SUCCESS overrides script_path_rejects_initial_stack_over_max_size (023)
C24 Signet last 38-byte BIP141 commitment; challenge P2SH WITNESS DERSIG
C25 BIP342 tapscript validation weight tapscript validation weight script_path_rejects_tapscript_validation_weight
C26 P2SH scriptSig eval + IsPushOnly script too large / accept OP_1NEGATE p2sh_legacy_op_1negate_scriptsig_accepted, p2sh_legacy_scriptsig_over_10k_rejected
C22 Subsidy halving interval from params 50 BTC until interval p1_block_subsidy_halvings; rejects_coinbase_excess_value_fast; journey overlay interval=2: header_and_spending_boundaries
C27 Captured signet/mainnet script-edge wire blocks (not Core JSON) hash / opcode presence; detached verify script_edge_fixtures
C28 Coinbase maturity (COINBASE_MATURITY) when the coinbase and its spender are in different blocks of one confirm batch BadTx("coinbase immature") at created+99; created+100 accepts consensus_rules::coinbase_maturity_holds_inside_one_confirm_batch; one-block-per-batch: header_and_spending_boundaries
C31 Script flags: P2SH / WITNESS / TAPROOT on every block; Core's mainnet script_flag_exceptions replace the set (BIP16 170060 → none, Taproot 692261 → P2SH+WITNESS); DERSIG / CLTV / CSV / NULLDUMMY height-gated. Policy flags stay off. The testnet3 BIP16 exception arrives with #884; once both land, witness_active = bip16_active clears WITNESS and TAPROOT for that block too, as in Core ScriptVerifyFlags::consensus_at; script reject below an overlaid segwit height script_flags_follow_core_exception_table; exception_hashes_change_which_scripts_run (taproot hashtype 0x00 accepted only on the exception hash; OP_0 P2SH redeem not executed only on the BIP16 exception); consensus_rules::witness_program_rules_bind_below_segwit_height
C32 P2PKH fast path: scriptSig push > 520 falls back to the interpreter PUSH_SIZE p2pkh_signature_push_over_520_rejected_pre_bip66
C33 P2WPKH fast path (native + nested): strict DER only with DERSIG; witness element ≤ 520 PUSH_SIZE / DER reject p2wpkh_der_follows_bip66_flag_and_push_size
C34 Base CHECKMULTISIG FindAndDelete of an empty sig removes every OP_0 opcode before the other sigs hash scriptCode accept / SIG_DER legacy_multisig_empty_sig_deletes_op_0_from_script_code
C35 P2SH scriptPubKey HASH160 <20> EQUAL runs on the scriptSig stack: 1000 items fail MAX_STACK_SIZE at the <20> push, with or without BIP16 stack size p2sh_script_pubkey_push_counts_against_max_stack_size (086)
C36 A tx is final at the connect height locktime=100 accepts at height 101; locktime==height rejects header_and_spending_boundaries
C37 BIP30: a txid is not created again while an unspent output of that txid still exists. Exceptions 91842 and 91880. The BIP34 ancestry skip resolves the ancestor once per write and remembers the confirmed header until that block is disconnected. A wrong hash keeps the check on every connecting block; is_bip30_repeat_matches_core buried_rules_and_a_lying_header_path, bip30_duplicate_coinbase_in_one_batch, repeated_coinbase_chain_connects_in_any_batch_split, repeated_coinbase_chain_rejects_what_core_rejects, bip34_era_repeat_connects_in_any_batch_split, bip34_era_repeated_spend_rejects_bip30_in_any_batch_split, repeated_coinbase_connects_when_ibd_merges_load_batches, respend_of_a_same_block_spend_rejects_in_any_split, repeated_tx_spent_in_its_own_block_connects_in_any_split, third_copy_of_a_coinbase_connects_in_any_split, third_copy_over_an_unspent_copy_rejects_bip30_in_any_split, repeated_coinbase_archived_retry_connects_the_valid_prefix, bip34_ancestor_in_the_same_write_skips_bip30, bip34_ancestor_already_committed_skips_bip30, wrong_bip34_ancestor_keeps_bip30_on, bip34_ancestor_reorg_drops_the_match
C38 An input's prevout exists and is still unspent height-1 coinbase spend accepts missing txid → MissingPrevout; child-before-parent; a second spend of the same outpoint; two spends in one block. header_and_spending_boundaries
C40 A tx's output sum ≤ its input sum in==out accepts in+1 rejects. header_and_spending_boundaries
C41 BIP68 relative finality. The coinbase is not an input: version ≥ 2 and a relative lock that would fail on a non-coinbase still connect nSequence=10 at height 101 accepts; time-type after MTP nSequence=200 at 101 rejects; bip68_skips_a_coinbase_whose_relative_lock_would_fail; finality_tests 109/110
C42 An input binds only to an output from an earlier block or an earlier tx in its own block, also inside one multi-block confirm batch (Core connects one block at a time). A repeated txid binds the newest copy created before the spend, including when the bodies are already archived MissingPrevout (bad-txns-inputs-missingorspent) consensus_rules::same_batch_spend_of_a_later_block_is_missing; plan_batch_spend_between_repeated_txid_binds_the_earlier_copy; repeated_coinbase_archived_retry_connects_the_valid_prefix; IBD isolation batched_spend_of_a_later_block_is_rejected_alone (087); same block: header_and_spending_boundaries (005)
C43 An output whose script starts with OP_RETURN, or is longer than 10,000 bytes, is not a coin. A 10,000-byte script that does not start with OP_RETURN is a coin. The archive still stores the output MissingPrevout / bad-txns-inputs-missingorspent, with scripts on and under a height-only milestone spending_an_unspendable_output_is_missing_prevout; resolve_prevout_rejects_an_unspendable_script; proposal_spend_of_an_unspendable_output_is_missing
C44 Anchored script skip requires this block's hash and the anchor hash on the header path. A failing script above the anchor, or under a different hash, still runs confirm accepts only when both hashes match anchored_milestone_skips_a_failing_p2pkh_script (empty P2PKH scriptSig)

Adding a new rule

  1. Add a row to the inventory above (or mark rust-bitcoin / lib if delegated).
  2. Prefer a pure unit test in rbitcoin-consensus when no chain state is needed; otherwise consensus_rules or a focused scenario.
  3. For Core corpora: every row must pass; never reintroduce allowlist/skip debt.
  4. Assert on the error signal string/variant so removing the check fails the test.

Dependency gate: cargo tree -i bitcoinconsensus must not resolve (product workspace). The isolated fuzz/ workspace may depend on it as a fuzz-oracle-only interpreter (script_kernel_differential); that crate is not a product consensus path.