Skip to content

kona-engine,kona-node!: drop derived work from before an engine reset - #23859

Open
joshklop wants to merge 1 commit into
joshklop/kona-el-sync-completionfrom
joshklop/kona-reset-epoch
Open

joshklop wants to merge 1 commit into
joshklop/kona-el-sync-completionfrom
joshklop/kona-reset-epoch

Conversation

@joshklop

@joshklop joshklop commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #23883 (base joshklop/kona-el-sync-completion), so the diff shows only the reset-epoch change. Stack, in merge order: #23990 → #23883 → #23859.

Closes https://github.com/ethereum-optimism/protocol-team/issues/587

After an engine reset, kona-node's engine and derivation actors could disagree. Derivation keeps handling the requests queued ahead of the reset signal with its old pipeline. The attributes and finalizations it produces then reach the engine after the reset. Consolidating those attributes moved the safe head past blocks that were never re-derived, and after an L1 reorg it recorded the wrong block in the safe head database. Disagreements stalled derivation instead of recovering. A stale finalization above the rewound safe head stopped the engine.

  • Both actors count engine resets. The engine counts the reset signals it sends, derivation counts the ones it handles, and derivation stamps its attributes and finalizations with its count. The engine drops anything stamped before its latest reset, and drops consolidations and finalizations queued when a reset succeeds.
  • Disagreements recover. Derived attributes whose parent is not the engine's local-safe head now trigger a reset. Derivation ignores a safe head that doesn't confirm the attributes it is waiting for, instead of adopting it and leaving a gap in its database.
  • No lost confirmations. The engine reports its local-safe head before starting a reset or backing off, so a confirmation earlier in the same drain is not lost.

Operators see no new flags. After a reset, a node that would have stalled, recorded the wrong safe head or crashed now recovers.

The two actors' counts stay equal because the channel between them is FIFO, a failed reset send stops the engine, and the node stops every actor when one exits. Derivation counts a reset when the engine's reset signal arrives, not when #23815 retries its pipeline reset; the parity tests check both the retry and the fresh-reset path. A debug assertion catches derivation counting ahead of the engine. A reviewer should check that this still holds for any future path that sends a reset signal.

Reviewed with the rust-code-reviewer and safedb-reviewer agents, including after rebasing onto #23815. Not done:

  • A test that drives both actors through a reset over a real channel. The single-actor tests cover each side of the epoch.
  • The engine's awaited sends to derivation in its drain can, in theory, block while derivation waits on an engine reset with its inbound queue full. The new sends have the same shape as the existing one, and kona-node: stop derivation when pipeline signals fail #23815 handles this only for the reset signal.

BREAKING CHANGE: In kona-engine, EngineActorRequest::ProcessSafeL2Signal is replaced by ProcessDerivedAttributes and ProcessDerivedFinalization, which carry the derivation actor's reset epoch, and by ProcessDelegatedSafeHead. ConsolidateTaskError has a new DoesNotExtendLocalSafe variant, which requests a reset, and Engine::drop_derived_tasks is new. In kona-node-service, DerivationEngineClient::send_safe_l2_signal is replaced by send_derived_attributes, send_derived_finalization and send_delegated_safe_head. Code that builds these requests or implements the trait must switch to the new variants and methods.

🤖 Generated with Claude Code

@joshklop
joshklop requested a review from a team as a code owner November 3, 2026 05:00
@joshklop
joshklop added this pull request to stack #23860 November 3, 2026 05:20
@joshklop
joshklop force-pushed the joshklop/kona-reset-epoch branch from dbfc333 to e96990b Compare November 3, 2026 18:06
@joshklop
joshklop removed this pull request from stack #23860 November 3, 2026 18:06
@joshklop
joshklop changed the base branch from joshklop/kona-derivation-reset-retry to einar/kona-failed-reset November 3, 2026 18:06
Base automatically changed from einar/kona-failed-reset to develop November 4, 2026 00:16
@einar-oplabs
einar-oplabs self-requested a review November 5, 2026 10:09
@joshklop
joshklop force-pushed the joshklop/kona-reset-epoch branch from e96990b to d988d28 Compare November 19, 2026 23:32
@joshklop joshklop self-assigned this Nov 21, 2026
@joshklop
joshklop removed the request for review from einar-oplabs November 21, 2026 21:58
Derivation keeps handling the requests queued ahead of an engine reset with
its old pipeline, so attributes and finalizations derived before the reset
reach the engine after it. Consolidated, those attributes moved the safe
head past blocks that were never re-derived, recorded the wrong block in
the safe head database after an L1 reorg, and could leave derivation
waiting forever for a confirmation; a stale finalization above the rewound
safe head stopped the engine. Now:
- both actors count engine resets: the engine counts the reset signals it
  sends, derivation the ones it handles, and derivation stamps its
  attributes and finalizations with its count. The engine drops work
  stamped before its latest reset, and drops consolidations and
  finalizations queued when a reset succeeds;
- derived attributes whose parent is not the local-safe head mean the
  actors disagree, so the engine resets instead of consolidating them;
- the engine reports the local-safe head before starting a reset or
  backing off, so a confirmation earlier in the same drain is not lost;
- derivation ignores a safe head that does not confirm the attributes it is
  waiting for, instead of adopting it and leaving a gap in the safe head
  database.

Derivation counts a reset when the engine's reset signal arrives, not on
each retry of its pipeline reset.

BREAKING CHANGE: EngineActorRequest::ProcessSafeL2Signal is replaced by
ProcessDerivedAttributes and ProcessDerivedFinalization, which carry the
derivation actor's reset epoch, and ProcessDelegatedSafeHead.
DerivationEngineClient::send_safe_l2_signal is replaced by
send_derived_attributes, send_derived_finalization and
send_delegated_safe_head, and Engine::drop_derived_tasks is new.
ConsolidateTaskError has a new DoesNotExtendLocalSafe variant, which
requests a reset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant