[WIP] Backfill Gloas execution payload envelopes alongside blocks - #17394
Draft
satushh wants to merge 4 commits into
Draft
[WIP] Backfill Gloas execution payload envelopes alongside blocks#17394satushh wants to merge 4 commits into
satushh wants to merge 4 commits into
Conversation
- Assign blocks and stage state together in handleBlocks, and nil-guard columnsNeeded: a setup error left b.blocks set with a nil columnSync, which transitionToNext dereferenced on the retry path. - Defer an EL cross-check failure on an unclassified batch tail to import-time classification, so a revealed tail reports el_failed rather than peer_exhausted and a withheld tail stays silent. - Publish skip counters only after the batch status write succeeds, so a retried or expired batch cannot leave phantom or duplicate gap counts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Feature
What does this PR do? Why is it needed?
Adds one stage to the existing backfill service so a checkpoint-synced node also fills in execution payload envelopes for the historical slots it backfills blocks for. Envelopes are fetched from CL peers, verified statelessly, cross-checked against the local EL, and persisted blinded alongside the blocks.
No new services, no coverage state, and no changes to serving or availability advertisement.
Why blocks aren't enough any more
Pre-Gloas a block contains its payload — backfill downloads the block and is done. Gloas splits them into two independently-transmitted objects:
That leaves two questions a block cannot answer about itself.
Should this slot have an envelope at all? The builder may have withheld the payload.
Only the child block's bid testifies:
Whose key signs it?
builder_indexis a reusable registry slot, so the current occupant of index B could validly sign a historical envelope and a passing signature would prove nothing. The origin-state snapshot occupant is trusted only whendeposit_epoch < envelope_epoch; otherwise the slot is unverifiable and is never even requested. Self-built payloads use the historical block's proposer key instead (validator indices are never reused, so a snapshot key is always safe there).Verification is stateless, mirroring the existing block-backfill verifier: all key material comes from the checkpoint origin state, with a dedicated
domainCacheinstance forDOMAIN_BEACON_BUILDER. Backfill only descends below the origin, so a pre-Gloas origin implies no expectations at all and a Gloas origin's registries are cumulative below it.One batch, end to end
Blocks inside a batch are in ascending slot order, so the batch's highest block is the tail and its child sits in the batch above, which descent has already imported.
Gate ordering matters because the EL call is the only one that can fail for our reasons rather than the peer's, so it goes last and its failure never downscores. A page of 16 where one envelope binds to the wrong block root is abandoned at gate (a) with zero BLS and zero EL work; the other 15 stay pending for another peer.
Envelopes are held in memory and written only at import, so a batch is all-or-nothing and a re-run is idempotent — already-stored envelopes are skipped before any request.
Edge cases
el_failedskip; blocks still importdeposit_epoch ≥ envelope_epoch, before any requestsig_unverifiable; never fetchedpeer_exhaustedThe key asymmetry: withheld and empty slots produce no metric; every genuine failure produces a typed skip. So
backfill_envelope_slots_skippedhaving no series at all means "nothing failed", not "nothing known".New metrics:
backfill_envelopes_download_count,backfill_envelopes_verified_count,backfill_envelope_slots_skipped{reason},backfill_envelope_conflicts_kept.Deliberate limitations
backfill_envelope_slots_skipped; it is the only signal, and it is process-lifetime only.develop, where a checkpoint-synced node holds zero envelopes below its origin — this PR strictly increases what is servable. An honest coverage gate that refuses rather than truncating, plus tyingearliest_available_slotto EL availability, is separate work and is needed independently of this PR.Other notes for review
Acknowledgements