Repository navigation
Description
Part of ethereum-optimism/protocol-team#442 (long-term Kona improvements). Related to #23473; dashboards and alerts are not a completion requirement for the current SafeDB integration. Depends on the node-side metrics, warnings, and failure signals in #23478. Dashboard JSON and alert rules land in ethereum-optimism/k8s.
Scope
Make Kona SafeDB health visible on the kona-node Grafana dashboard and route actionable alerts to team=protocol. Reuse op-node SafeDB panels where available so both clients read consistently.
Panels (one row, lowercase side: metric (window)):
safedb: tip l1 blockbeside the L1 head, with lag as a stat.safedb: tip l2 blockbeside local-safe, with lag as a stat.safedb: writes (5m)andsafedb: persistence failures (5m).safedb: truncations (1h)andsafedb: history clears (1h), split by reason (reset, EL-sync).safedb: tip below resume point, showing reset-below-tip events from kona-node,safedb,gap-detection: expose metrics and gap-risk warnings #23478.- Distinguish disabled, empty, unreadable, and unavailable/scrape-missing states from a healthy zero value.
Alerts:
- SafeDB tip L1 lag above a documented threshold for 15 minutes while derivation is advancing; validate the rule against legitimate periods with no safe-head advancement.
- Any tip-below-resume-point event (the gap-risk condition from ethereum-optimism/protocol-team#402).
- A fatal SafeDB persistence failure, using the error/process-failure signals from kona-node,safedb,gap-detection: expose metrics and gap-risk warnings #23478 and counters where available. The alert must work if the node exits before the failure counter is scraped.
The persistence policy is fail-fast, with no write retries. There is no retry-rate panel or exhausted-retry alert. Interval-based historical gap detection remains separate long-term work in #23659.
Done and testing
- Panels render on the kona-node sequencer devnet (ethereum-optimism/protocol-team#283), including disabled and unavailable states.
- All three alerts route to protocol and include recovery guidance, linking to ethereum-optimism/protocol-team#404 when available.
- Thresholds and their reasoning are recorded here; normal periods without safe-head advancement do not produce false stall alerts.
- Alert-rule tests cover normal operation, lag, reset-below-tip, and fatal persistence failure before the next metrics scrape.
Status — 2026-10-29
The current integration passed its 24-hour persistence soak and #23479 is closed. Final evidence. Grafana log access supported a manual review. This does not mean the proposed SafeDB panels or alert rules were implemented or deliberately tested. This issue remains open as deferred long-term work under ethereum-optimism/protocol-team#442; none of its implementation checkboxes are completed by the soak.
Activity
[-]kona-node: SafeDB dashboard panels and metrics[/-][+]kona-node: SafeDB dashboard panels and alerts[/+][-]kona-node: SafeDB dashboard panels and alerts[/-][+]kona-node,safedb: add dashboard panels and alerts[/+][-]kona-node,safedb: add dashboard panels and alerts[/-][+]kona-node,safedb,grafana: add dashboard panels and alerts[/+]