Skip to content

Global + per-repo alerting/log severity thresholds for the ORB stack #5119

Description

@JSONbored

Context

Every observability capture point in the self-host stack fires unconditionally today, with no notion of a configurable verbosity threshold:

  • captureError/captureReviewFailure (src/selfhost/sentry.ts) always forward to Sentry when called.
  • forwardStructuredLogToSentry turns any console.error/console.log structured line carrying an event field into a Sentry event, regardless of how routine or noisy that event kind is.
  • The one precedent for severity-based gating that exists is the PagerDuty paging integration's own PAGERDUTY_MIN_SEVERITY/PAGERDUTY_REPO_MIN_SEVERITY floor (src/services/notify-pagerduty.ts) — but that's scoped to ONE notification channel (whether an ops_anomaly finding pages), not general Sentry/log verbosity.

A self-host operator running gittensory against many repos of differing importance has no way to say "only error/fatal-grade events from most repos, but give me info-level visibility into repo X while I'm actively debugging it" — every repo gets identical, maximal verbosity.

Requirements

  1. Define a shared severity/level taxonomy (likely debug < info < warn < error < fatal, or reuse the existing critical/error/warning/info shape from PagerDutySeverity for consistency) and a shared resolver: global default threshold + per-repo override, mirroring the PAGERDUTY_MIN_SEVERITY/PAGERDUTY_REPO_MIN_SEVERITY env-var + JSON-map precedent (operator-scoped ops tuning, not a maintainer-facing .gittensory.yml gate setting — this is about how noisy the OPERATOR's own Sentry/logs get, not a per-repo review-behavior knob).
  2. Wire the shared resolver into captureError/captureReviewFailure/forwardStructuredLogToSentry so a below-threshold event for a given repo is suppressed from Sentry (still fine to keep in Workers Logs/stdout — this is about Sentry noise specifically, not losing the data entirely).
  3. Retrofit notify-pagerduty.ts's own bespoke resolvePagerDutyMinSeverity to reuse the shared resolver instead of its own copy, so there's one severity-threshold concept in the codebase, not two parallel ones.
  4. Sensible default that changes nothing for an operator who never touches these vars (e.g. default threshold error, matching today's de facto "everything gets through" behavior as closely as possible, or documented explicitly if the default tightens it).

Deliverables

  • A shared severity-threshold resolver (global env var + per-repo JSON-map override).
  • sentry.ts capture paths respect the resolved threshold per repo.
  • notify-pagerduty.ts reuses the shared resolver instead of its own copy.
  • Documented in the self-host env reference (npm run selfhost:env-reference).
  • Tests: both sides of the threshold comparison, global-vs-per-repo precedence, and the "unset → today's behavior" default case.

Expected outcomes

An operator running gittensory across many repos can quiet Sentry noise from low-priority repos by default while dialing up verbosity on one repo under active investigation — without a code change, following the same global-default + per-repo-override shape every other feature in this codebase already uses.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

Status
Todo

Relationships

None yet

Development

No branches or pull requests

Issue actions