Skip to content

txn: a spilled range-tree node can exceed the spill frame limit for an admissible DeleteRange #349

Description

@smiggleworth

Found in review of #348 (the fix for #249).

Observed

TransactionSpillStore.WriteRangeNodeFrame (src/Pants.Core/Transactions/Internal/TransactionSpillStore.cs) writes a range node's start key, end key and the subtree's MaximumEnd into one frame, capped at DiskFormat.WalMaximumRecordBytes (64 MiB). The frame is therefore about 36 + start + end + maximumEnd bytes, and MaximumEnd can come from a different range in the same run.

A DeleteRange whose bounds pass admission (WAL and SST) can therefore still make spilling fail. For example, DeleteRange([0], new byte[40 MiB]) stays resident in a large pool until an unrelated Put triggers SpillResidentIntents. WriteRun then throws "A transaction spill frame exceeds the 64 MiB limit", and every later staging call that needs to spill fails the same way, blaming the wrong operation.

Midge has the same layout (src/runtime/transaction_spill/range.rs, MAX_FRAME_BYTES = WAL_MAX_RECORD_LEN), so this is not a parity gap. Spill runs are transaction-local and never read by another engine or after restart, so the fix doesn't have to follow Midge.

Options

  • Store MaximumEnd as a reference to the node that owns it, rather than an inline copy (spill-run layout change; the file is private).
  • Or give spill frames their own limit derived from the admission bound (3 × the largest admissible bound plus the header).

Acceptance criteria

  • A transaction holding any mix of admissible DeleteRange operations can spill, and then commit, scan and read through the spilled run.
  • Regression test with a ~40 MiB range end that spills after staging.
  • Admission (EntryAdmission) remains the single bound callers see.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:correctnessTransaction/CF/config behavioral hardeningarea:transactionsTransaction engine internals (spill, intents, MVCC read validation)priority:P3Deferred: blocked or intentionally postponed until an explicit prerequisite is met

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions