diff --git a/TESTING.md b/TESTING.md index c0971daf4..f85174f42 100644 --- a/TESTING.md +++ b/TESTING.md @@ -388,7 +388,7 @@ A cell that names a crate-local test describes the witness that exists today. It | `peer_blocksonly_and_orphan_tx` | P2P (**default**, crate) | One hub, relay off and then on. Blocks-only: a tx or wtx inv from an ordinary peer disconnects, a sendraw INVs once it is unbroadcast (never to block-relay) and serves, a `relay@` whitelisted peer's tx is kept and INV'd to the other inbound, and a type-0 getdata is ignored while the tip block still serves. A block confirms those txs and relay-on purges them. Then forced INVs skip block-relay, seen, feefilter, and isolated local-origin peers. Inbound waits 30s even for unbroadcast, never gets a tx older than its connection, and idle ticks do not clone or rescan. GetData serves only an announced or reorged-back tx, and a sendraw after a +300s jump neither INVs nor serves. An orphan parks on a tokio worker without a reject log and asks only the still-missing parent after NONPREF+TXID. | | `stored_header_resends_walk_once` | IBD (crate) | One 240-header genesis chain. A re-sent stored run walks no ancestors, an unmapped stored run walks once, an unknown parent is not stored, a rejected tail keeps the stored prefix under a lowered walk cap, and a run past that cap walks once and is still accepted. | | `two_peers_reserve_the_header_walk` | IBD (crate) | One hub. The lowest time-to-first-byte peer takes headers and the other takes blocks; a short miss moves the reservation without disconnecting; a heavier inv takes it and a lighter inv restores it; refill under the low-water mark stays on the reserved peer. A second chapter, because that lighter challenger is retired: a quiet header peer is skipped, asked again, then disconnected. A third, because that survivor already missed once: the only header peer is disconnected after two misses. | -| `a_seeded_header_walk_stores_drains_and_refills` | IBD (crate) | One walk from the stored tip. The first locator and the status line start there; the first batch is a checkpoint and the empty queue refills from that tip; with room in the queue the reserved peer keeps walking and a second peer is not asked; an empty queue still checkpoints; a refill stores without moving the walk; an empty queue refills from the confirmed tip; a full batch stays stored while a walk ask inside its window is not stirred. | +| `a_seeded_header_walk_stores_drains_and_refills` | IBD (crate) | One walk from the stored tip. The first locator and the status line start there; the first batch is a checkpoint and the empty queue refills from that tip; with room in the queue the reserved peer keeps walking and a second peer is not asked; an empty queue still checkpoints; a refill stores without moving the walk; an empty queue refills from the confirmed tip; a full batch stays stored while a walk ask inside its window is not stirred. On that hub, two catch-up asks at one published tip share one confirmed-chain tail and the locator still starts at a stored path tip ahead of the confirmed tip; a same-height tip hash and a taller tip each rebuild the tail. | | `a_heavier_header_chain_keeps_its_context` | IBD (crate) | One hub: a challenger keeps median time while the tip is genesis (invalid at the median, valid one second later); that hub then confirms a stale ancestor, a heavier chain from below it replaces the candidate and drops its script skip, and a rewind uses the confirmed base work and height. Retarget is a second hub (period length is not regtest difficulty): a heavier fork across a gap retarget is adopted, rewind restores difficulty before the next retarget, and a challenger keeps its retarget snapshot. Those three checkpoint layouts cannot be one candidate. | | `analog_selector_keeps_each_boundary_rate` | Mempool (crate) | One selector table: the exact band edge, absolute log distance, the requested quantile, and the two-hundredth nearest window each keep their own rate. | | `fee_history_gaps_cache_and_reorg_share_one_history` | IBD (crate) | One history: heights without a hurdle are not observations, rates stay cached until a new height, only the newest hashes are kept, and a reorg drops those above. The RPC backfill stays `fee_history_backfills_from_the_chain_when_relay_starts`. | diff --git a/crates/rbitcoin-net/src/ibd/dial.rs b/crates/rbitcoin-net/src/ibd/dial.rs index 61e0c4f14..3e783cc07 100644 --- a/crates/rbitcoin-net/src/ibd/dial.rs +++ b/crates/rbitcoin-net/src/ibd/dial.rs @@ -415,14 +415,12 @@ pub(crate) fn replacement_available( pub(crate) fn request_headers_from( slots: &[PeerSlot], peer: usize, - hub: &ChainHub, _seq: &mut u32, - work_tips: &[BlockHash], + locator: Vec, ) -> Result { let Some(s) = slots.iter().find(|s| s.id == peer && s.alive) else { return Ok(false); }; - let locator = ibd_header_locator(hub, work_tips)?; Ok(s.cmd_tx.send(PeerCmd::GetHeaders { locator }).is_ok()) } @@ -436,7 +434,8 @@ pub(crate) fn request_headers_from( pub(crate) fn ibd_header_locator( hub: &ChainHub, work_tips: &[BlockHash], -) -> Result, NetError> { + confirmed_tail: &[BlockHash], +) -> Vec { let mut locator = Vec::with_capacity(32); // Eight hashes is the walk's own stop. This only enforces the protocol // maximum, so a stored tip prepended in front does not drop the oldest @@ -449,27 +448,24 @@ pub(crate) fn ibd_header_locator( locator.push(*h); } } + // In-memory snapshot. Not a header-store walk. if let Some(t) = hub.tip_hash() { - if !locator.contains(&t) { + if locator.len() < crate::codec::MAX_LOCATOR_SZ && !locator.contains(&t) { locator.push(t); } } - let rest = hub - .query - .locator_hashes() - .map_err(|e| NetError::Consensus(e.to_string()))?; - for h in rest { - if !locator.contains(&h) { - locator.push(h); - } + for h in confirmed_tail { if locator.len() >= crate::codec::MAX_LOCATOR_SZ { break; } + if !locator.contains(h) { + locator.push(*h); + } } if locator.is_empty() { locator.push(BlockHash::from_byte_array([0u8; 32])); } - Ok(locator) + locator } /// Mark `peer` dead and drop it from every hash it holds. Returns the hashes @@ -1836,9 +1832,9 @@ mod tests { let mut seq = 0u32; let mut dead = dummy_slot(1, addr(1), false); dead.alive = false; - assert!(!request_headers_from(&[dead], 1, &hub, &mut seq, &[]).unwrap()); + assert!(!request_headers_from(&[dead], 1, &mut seq, vec![]).unwrap()); // Locator alone (empty work tips + no tip) still returns genesis zero. - let loc = ibd_header_locator(&hub, &[]).unwrap(); + let loc = ibd_header_locator(&hub, &[], &[]); assert!(!loc.is_empty()); let _ = std::fs::remove_dir_all(dir); } diff --git a/crates/rbitcoin-net/src/ibd/header_walk.rs b/crates/rbitcoin-net/src/ibd/header_walk.rs index 807c0d20d..a288163b5 100644 --- a/crates/rbitcoin-net/src/ibd/header_walk.rs +++ b/crates/rbitcoin-net/src/ibd/header_walk.rs @@ -18,7 +18,7 @@ //! checkpoints is the previous checkpoint plus those stored headers. That //! replay runs only when a competing headers batch forks there. -use super::state::IbdWorkState; +use super::state::{ConfirmedLocatorTail, IbdWorkState}; use super::ORDERED_HEADERS_SOFT_CAP; use crate::chain::ChainHub; use bitcoin::block::Header; @@ -2368,13 +2368,43 @@ fn ask_header_peer( peer: usize, tips: &[BlockHash], ) -> Result { - if !super::dial::request_headers_from(&st.slots, peer, hub, &mut st.header_req_seq, tips)? { + let tail = confirmed_chain_tail(st, hub)?; + let locator = super::dial::ibd_header_locator(hub, tips, &tail); + if !super::dial::request_headers_from(&st.slots, peer, &mut st.header_req_seq, locator)? { return Ok(false); } note_lane(st, peer); Ok(true) } +/// Confirmed-chain tail for `getheaders`. +/// +/// Rebuilt when `published_tip` height or hash changes. The work-path prefix +/// is not part of this vec. +fn confirmed_chain_tail( + st: &mut IbdWorkState, + hub: &ChainHub, +) -> Result, crate::error::NetError> { + let published = hub.query.published_tip(); + let height = published.map(|tip| tip.height); + let hash = published.map(|tip| tip.hash); + if let Some(cached) = &st.confirmed_locator_tail { + if cached.height == height && cached.hash == hash { + return Ok(cached.hashes.clone()); + } + } + let tail = hub + .query + .locator_hashes() + .map_err(|e| crate::error::NetError::Consensus(e.to_string()))?; + st.confirmed_locator_tail = Some(ConfirmedLocatorTail { + height, + hash, + hashes: tail.clone(), + }); + Ok(tail) +} + fn parent_header(st: &IbdWorkState, hub: &ChainHub, hash: BlockHash) -> Option
{ known(st, hash).header.or_else(|| hub.header_of(&hash)) } @@ -3574,6 +3604,9 @@ mod tests { /// One ask from the stored tip. An accepted batch is written and /// checkpointed. A header already indexed is appended and not asked for. + /// On that same hub, a second ask at one confirmed tip reuses the + /// confirmed-chain tail. A same-height tip hash, then a taller tip, each + /// rebuild it. #[test] fn a_seeded_header_walk_stores_drains_and_refills() { let (_dir, hub) = crate::chain::tiny_regtest_hub_labeled("header-walk-seeded"); @@ -3655,6 +3688,135 @@ mod tests { assert!(st.ordered_set.contains(&second.block_hash())); assert!(rx0.try_recv().is_err()); assert!(rx1.try_recv().is_err()); + confirmed_tail_follows_the_published_tip(&hub); + } + + /// Same hub as the seeded walk. Two asks share one confirmed tail. A + /// same-height hash, then a taller tip, each rebuild it. + fn confirmed_tail_follows_the_published_tip(hub: &ChainHub) { + let script = bitcoin::ScriptBuf::from_bytes(vec![0x51]); + hub.generate_to_script(2, script.clone(), vec![]).unwrap(); + let tip = hub.tip_hash().unwrap(); + let tip_h = hub.tip_height().unwrap(); + let genesis = hub.params.genesis_hash; + let ancestor = hub + .query + .locator_hashes() + .unwrap() + .into_iter() + .find(|hash| *hash != tip && *hash != genesis) + .expect("the confirmed chain locator has an ancestor between tip and genesis"); + let (peer, mut rx) = slot(0); + let mut st = IbdWorkState::new(vec![peer], Some(tip), Some(tip_h)); + let ahead = BlockHash::from_byte_array([0xab; 32]); + st.ordered.push_back(ahead); + st.ordered_set.insert(ahead); + st.hash_height.insert(ahead, tip_h.saturating_add(100)); + assert!(st.ordered.len() < ORDERED_HEADERS_SOFT_CAP); + assert!(send_getheaders(&mut st, hub).unwrap()); + let first = take_locator(&mut rx); + assert_eq!( + first.first().copied(), + Some(ahead), + "a queue with room asks from the stored path tip" + ); + assert!( + first.contains(&ancestor), + "the confirmed ancestor stays in the locator" + ); + assert!(first.contains(&tip)); + same_tip_reuses_the_tail(&mut st, hub, &mut rx, &first); + let same_h_tip = same_height_rebuilds_the_tail(&mut st, hub, &mut rx, tip, tip_h, ahead); + taller_tip_rebuilds_the_tail(&mut st, hub, &mut rx, &script, same_h_tip, ahead); + } + + fn same_tip_reuses_the_tail( + st: &mut IbdWorkState, + hub: &ChainHub, + rx: &mut mpsc::UnboundedReceiver, + first: &[BlockHash], + ) { + assert_ne!(take_header_ask(st, 0), HeaderAsk::Unsolicited); + let before = rbitcoin_query::header_lookup_count(); + assert!(send_getheaders(st, hub).unwrap()); + let second_ask = take_locator(rx); + assert_eq!( + rbitcoin_query::header_lookup_count().saturating_sub(before), + 0, + "a second ask at the same confirmed tip does not walk the header store" + ); + assert_eq!(second_ask, first); + } + + fn same_height_rebuilds_the_tail( + st: &mut IbdWorkState, + hub: &ChainHub, + rx: &mut mpsc::UnboundedReceiver, + tip: BlockHash, + tip_h: u32, + ahead: BlockHash, + ) -> BlockHash { + hub.invalidate_block(tip).unwrap(); + assert_eq!(hub.tip_height(), Some(tip_h - 1)); + let alt_script = bitcoin::ScriptBuf::from_bytes(vec![0x52]); + hub.generate_to_script(1, alt_script, vec![]).unwrap(); + let same_h_tip = hub.tip_hash().unwrap(); + let published = hub.query.published_tip().expect("published tip"); + assert_eq!(hub.tip_height(), Some(tip_h)); + assert_eq!(published.height, tip_h); + assert_eq!(published.hash, same_h_tip.to_byte_array()); + assert_ne!(same_h_tip, tip); + assert_ne!(take_header_ask(st, 0), HeaderAsk::Unsolicited); + let before = rbitcoin_query::header_lookup_count(); + assert!(send_getheaders(st, hub).unwrap()); + let reorged = take_locator(rx); + assert!( + rbitcoin_query::header_lookup_count() > before, + "a same-height tip hash rebuilds the tail" + ); + assert_eq!(reorged.first().copied(), Some(ahead)); + assert!( + reorged.contains(&same_h_tip), + "the rebuilt tail names the replacement tip" + ); + let tail = st + .confirmed_locator_tail + .as_ref() + .expect("the same-height ask stored a tail"); + assert_eq!(tail.height, Some(tip_h)); + assert_eq!(tail.hash, Some(*same_h_tip.as_byte_array())); + assert!(tail.hashes.contains(&same_h_tip)); + assert!( + !tail.hashes.contains(&tip), + "the replaced tip hash leaves the confirmed tail" + ); + same_h_tip + } + + fn taller_tip_rebuilds_the_tail( + st: &mut IbdWorkState, + hub: &ChainHub, + rx: &mut mpsc::UnboundedReceiver, + script: &bitcoin::ScriptBuf, + same_h_tip: BlockHash, + ahead: BlockHash, + ) { + hub.generate_to_script(1, script.clone(), vec![]).unwrap(); + let new_tip = hub.tip_hash().unwrap(); + assert_ne!(new_tip, same_h_tip); + assert_ne!(take_header_ask(st, 0), HeaderAsk::Unsolicited); + let before = rbitcoin_query::header_lookup_count(); + assert!(send_getheaders(st, hub).unwrap()); + let taller = take_locator(rx); + assert!( + rbitcoin_query::header_lookup_count() > before, + "a changed confirmed tip rebuilds the tail" + ); + assert_eq!(taller.first().copied(), Some(ahead)); + assert!( + taller.contains(&new_tip), + "the rebuilt tail names the new confirmed tip" + ); } fn locator_front(rx: &mut mpsc::UnboundedReceiver) -> Option { @@ -3671,6 +3833,13 @@ mod tests { } } + fn take_locator(rx: &mut mpsc::UnboundedReceiver) -> Vec { + match rx.try_recv() { + Ok(PeerCmd::GetHeaders { locator }) => locator, + _ => panic!("expected one getheaders"), + } + } + /// Walk tip is `first` (height 1). Queue tail height is 0, so the walk is ahead. fn walk_ahead_of_a_short_queue( label: &str, diff --git a/crates/rbitcoin-net/src/ibd/state.rs b/crates/rbitcoin-net/src/ibd/state.rs index b20e28055..b4e418264 100644 --- a/crates/rbitcoin-net/src/ibd/state.rs +++ b/crates/rbitcoin-net/src/ibd/state.rs @@ -125,6 +125,17 @@ impl InflightReq { } } +/// Confirmed-chain `getheaders` tail for one published tip. +/// +/// Absent until the first catch-up ask. The next ask reuses `hashes` while +/// both `height` and `hash` still match `published_tip`. The work-path +/// prefix is not stored here. +pub(crate) struct ConfirmedLocatorTail { + pub(crate) height: Option, + pub(crate) hash: Option<[u8; 32]>, + pub(crate) hashes: Vec, +} + /// Core mutable state for the IBD event loop. pub(crate) struct IbdWorkState { pub slots: Vec, @@ -198,6 +209,8 @@ pub(crate) struct IbdWorkState { pub(crate) header_cmd: Option, /// Heights below this are already claim-ready. Rewind starts here. pub(crate) densify_ready_through: u32, + /// Confirmed-chain `getheaders` tail for one published tip. + pub(crate) confirmed_locator_tail: Option, } pub(crate) enum HeaderCmd { @@ -289,6 +302,7 @@ impl IbdWorkState { confirm_feed: None, header_cmd: None, densify_ready_through: 0, + confirmed_locator_tail: None, } }