Skip to content

feat(execution-beacon): add runbook_url to default alerts - #264

Merged
adriantpaez merged 1 commit into
mainfrom
bro-1378-execution-beacon-default-alerts-missing-runbook_url-blocking
Aug 24, 2026
Merged

feat(execution-beacon): add runbook_url to default alerts#264
adriantpaez merged 1 commit into
mainfrom
bro-1378-execution-beacon-default-alerts-missing-runbook_url-blocking

Conversation

@adriantpaez

Copy link
Copy Markdown
Contributor

Closes BRO-1378.

Problem

The chart renders a PrometheusRule by default, but grep -rn runbook templates/ returned 0 hits — no alert set the runbook_url annotation, and no values path could inject one. metrics.prometheusRule exposed only default, namespace, severity, additionalLabels and rules, and additionalLabels lands on the object's metadata rather than on alert annotations.

Deployments behind the bros-alerts-must-have-runbooks Kyverno policy were denied at admission:

admission webhook "vpol.validate.kyverno.svc-fail" denied the request:
Policy bros-alerts-must-have-runbooks failed: All alert rules must have the
runbook_url annotation set to a runbook in Notion (retried 3 times)

The policy denies the whole object, so one non-compliant alert failed all of them and left the ArgoCD Application degraded.

Change

Adds metrics.prometheusRule.runbookUrl, defaulting to the Execution Beacon Alerts runbook, and emits runbook_url on all eight default alerts — EthNodeDown and EthNodeUpdateFailed in _alerts.tpl, plus the client-specific branches in prometheusrules.yaml (Lighthouse ×2, Prysm ×3, Teku ×1).

The annotation is guarded by with, mirroring the existing severity handling, so an empty value omits the key rather than rendering runbook_url: "". Making it a value rather than a literal lets consumers outside Nethermind retarget or drop it.

Custom rules passed via metrics.prometheusRule.rules are untouched — that list is tpl-rendered, so callers already control their own annotations.

values.schema.json is additionalProperties: false at every level and hand-maintained, so the new key needed a matching schema entry; without it every helm template would break.

Version 3.2.0 → 3.3.0.

Verification

  • helm template for lighthouse, prysm and teku — alerts missing runbook_url is [] for all three
  • Full render succeeds, confirming the schema edit; --set metrics.prometheusRule.runbookUrl=https://example.com/rb overrides correctly
  • --set metrics.prometheusRule.runbookUrl="" omits the key entirely
  • helm-unittest: 29 passed, 5 suites, 0 failed (includes a new case covering the empty-value guard)
  • helm lint: clean

Still to do against a live cluster: kubectl apply --dry-run=server on the rendered rule to confirm the Kyverno gate passes, then re-sync the Application.

The chart renders a PrometheusRule by default, but no alert set the
runbook_url annotation and no values path could inject one --
metrics.prometheusRule exposed only default, namespace, severity,
additionalLabels and rules, and additionalLabels lands on the object's
metadata rather than on alert annotations. Deployments behind the
bros-alerts-must-have-runbooks Kyverno policy were denied at admission,
failing the whole PrometheusRule and leaving the Application degraded.

Add metrics.prometheusRule.runbookUrl, defaulting to the Execution Beacon
Alerts runbook, and emit runbook_url on all eight default alerts guarded
by `with` so an empty value omits the annotation rather than rendering an
empty string. Making it a value rather than a literal lets consumers
outside Nethermind retarget or drop it.

Custom rules passed via metrics.prometheusRule.rules are untouched; that
list is tpl-rendered, so callers already control their own annotations.

Refs BRO-1378
@adriantpaez
adriantpaez requested a review from a team as a code owner August 24, 2026 09:52
@linear

linear Bot commented Aug 24, 2026

Copy link
Copy Markdown

BRO-1378

@adriantpaez adriantpaez self-assigned this Aug 24, 2026
@adriantpaez
adriantpaez merged commit 6c8943c into main Aug 24, 2026
6 checks passed
@adriantpaez
adriantpaez deleted the bro-1378-execution-beacon-default-alerts-missing-runbook_url-blocking branch August 24, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants