Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions crates/warp-core/src/parallel/shard.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,15 @@ use crate::NodeId;

/// Number of virtual shards (power of two for fast modulo).
///
/// # Protocol Constant
/// # Not part of any hash
///
/// This value is recorded in the commit hash domain via `compute_patch_digest_v2`.
/// Changing it would produce different hashes for the same world state.
///
/// If you ever need to change this, it requires a protocol version bump
/// and explicit migration handling.
/// No digest or state root includes this value, and worker deltas are merged in
/// canonical order (`parallel::merge`), so the committed result does not depend
/// on it. It is still observable: it partitions execution items, and it caps the
/// effective worker count (`parallel::exec`), which decides how many per-worker
/// deltas the parallel execution APIs return (even for empty input) and which
/// execution plan they select. Changing it changes those API-visible shapes and
/// load balance, not hashes.
///
/// # Value Choice
///
Expand Down
23 changes: 13 additions & 10 deletions crates/warp-core/src/scheduler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ use crate::telemetry::TelemetrySink;
use crate::tick_delta::OpOrigin;
use crate::tx::TxId;

/// Active footprint tracking using generation-stamped sets for O(1) conflict detection.
/// Active footprint tracking for one transaction, using per-transaction
/// [`GenSet`]s (`BTreeMap`-backed, O(log n) per lookup).
///
/// All resource keys are warp-scoped: they include both `warp_id` and local identifiers.
/// This ensures rewrites in different warps don't cause false conflicts when they
Expand Down Expand Up @@ -62,7 +63,8 @@ impl ActiveFootprints {
pub(crate) struct RadixScheduler {
/// Pending rewrites per transaction, stored for O(1) enqueue and O(n) drain.
pending: BTreeMap<TxId, PendingTx<PendingRewrite>>,
/// Active footprints per transaction for O(m) independence checking via `GenSets`.
/// Active footprints per transaction, checked through `GenSet`s at O(log n) per
/// resource, so O(m log n) for a footprint of size m.
/// Checks all aspects: nodes (read/write), edges (read/write), and boundary ports.
pub(crate) active: BTreeMap<TxId, ActiveFootprints>,
}
Expand Down Expand Up @@ -118,11 +120,11 @@ impl RadixScheduler {
}

/// Attempts to reserve a rewrite by checking full footprint independence
/// using generation-stamped conflict detection.
/// against the transaction's active sets.
///
/// Checks all aspects of the footprint: node read/write sets, edge read/write
/// sets, and boundary ports. Uses O(1) `GenSet` lookups for each resource,
/// making this O(m) where m is the size of the current footprint.
/// sets, and boundary ports. Each resource is an O(log n) `GenSet` lookup,
/// so this is O(m log n) for a footprint of size m against n reserved keys.
Comment thread
flyingrobots marked this conversation as resolved.
Resolved
///
/// On success, marks all resources in the active `GenSets` and transitions
/// the phase to `Reserved`.
Expand Down Expand Up @@ -475,14 +477,15 @@ fn bucket16(r: &RewriteThin, pass: usize) -> u16 {
}

// ============================================================================
// Generation-stamped conflict set for O(1) independence checks
// Per-transaction conflict sets
// ============================================================================

/// Generation-stamped set for O(1) conflict detection.
/// Set of keys seen by one transaction, for conflict detection.
///
/// This data structure allows O(1) conflict checking without clearing hash tables
/// between transactions by using generation counters. Each transaction gets a new
/// generation, and we track which generation last saw each key.
/// Each transaction gets fresh sets (`ActiveFootprints::new`) and they are
/// dropped with it, so the generation stays at 1: it is never advanced, and
/// nothing is reused across transactions. Lookups go through a `BTreeMap`, so
/// they cost O(log n), not O(1).
Comment thread
flyingrobots marked this conversation as resolved.
Resolved
#[derive(Debug)]
pub(crate) struct GenSet<K> {
gen: u32,
Expand Down
4 changes: 3 additions & 1 deletion docs/spec/scheduler-warp-core.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,9 @@ Pending rewrites drain in lexicographic order derived from `scope_hash`, stable

## Decision 5: Radix reservation is the default implementation

The default scheduler uses generation-stamped active sets for membership checks. Expected reservation cost is proportional to candidate footprint size.
The default scheduler uses per-transaction active sets (`BTreeMap`-backed) for
membership checks. Reservation costs O(log n) per footprint resource, where n
is the number of keys already reserved in the transaction.

## Decision 6: Executable Actions are evaluated only inside Tick construction

Expand Down
Loading