Skip to content

docs(grug): the enforcement-gap runbook the monitor already linked to - #816

Merged
quadseven merged 1 commit into
mainfrom
docs/runbook-enforcement-gap
Aug 2, 2026
Merged

docs(grug): the enforcement-gap runbook the monitor already linked to#816
quadseven merged 1 commit into
mainfrom
docs/runbook-enforcement-gap

Conversation

@quadseven

@quadseven quadseven commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Why

The enforcement-gap monitor's message has pointed at docs/RUNBOOK.md#enforcement-gap since it was written, and that section did not exist. RUNBOOK.md had no enforcement content at all, so the one link an operator follows mid-alert went nowhere.

Found while fixing #716.

Summary

Documents what the alert means, how to triage it, and the two Datadog traps that cost real time this week.

The alert covers two different situations that need different responses: a real gap (none, 0.0) versus a detection failure (error, -1.0, where you cannot currently tell). The runbook splits them.

Also records that pup monitors get does not return live state for this monitor - it reports state: null with overall_state_modified frozen at creation - so live status has to come from pup monitors search.

Acceptance criteria

  • The #enforcement-gap anchor the monitor links to now resolves
  • Both alert cases have distinct triage paths
  • The Pulumi-vs-opt-out decision is stated, including that a Grug-created ruleset on a Pulumi-managed repo drifts
  • Docs-only - no code paths touched

Size

Size: S

Out of scope

Refs #716

Live verification note

This body was edited at 15:07:24Z while Elder was mid-review, on purpose. Before #814 that cancelled both Cave arms and produced all_failed. The review below should survive it.

The monitor message has pointed at docs/RUNBOOK.md#enforcement-gap since it
was written. That section did not exist - RUNBOOK.md had no enforcement
content at all, so the one link an operator follows mid-alert went nowhere.

Covers what the alert actually means (both cases: a real gap at 0.0 and a
detection failure at -1.0, which need different responses), how to find the
repo, the Pulumi-vs-opt-out decision, and the two Datadog traps worth knowing
before chasing a red monitor: `pup monitors get` does not return live state,
and a silent multi-alert group latches for 24h.

Refs #716
@quadseven

Copy link
Copy Markdown
Owner Author

Live proof of #814, captured on this PR

This PR doubled as the verification vehicle for #814. The body was edited while Elder was mid-review, which is the exact sequence that produced all_failed on quadseven/infra#2157.

15:05:39Z  llm_tiered_escalation          <- Elder generating
15:07:23Z  PR body EDITED (deliberately, mid-review)
15:07:28Z  tpm_publishing                 <- GitHub processed the edit; Chief re-ran
15:07:35Z  persona_disabled_skip
~15:09Z    Grug - Elder: success, "Elder clear - no markings"

Counts over that window:

Signal Before #814 Observed now
elder_review_cancelled_mid_flight 1 0
code_review_llm_degraded 1 (all_failed) 0
Elder conclusion neutral (degraded) success
Board comments created 1 (mailed "eyes cloudy") 0

The tpm_publishing at 15:07:28 is what proves the edit actually landed mid-flight - Chief re-evaluated because the body changed. Elder kept going anyway, which is the fix.

Zero bot comments on this PR is also the #791 rule holding: a clean review creates no board, so no email.

@quadseven
quadseven merged commit 50242f3 into main Aug 2, 2026
11 checks passed
@quadseven
quadseven deleted the docs/runbook-enforcement-gap branch August 2, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant