feat(signals): auto-pause scouts that produce nothing

[posthog]

Problem

A scout that produces nothing anyone uses keeps running a sandboxed agent on its cadence forever, because enabled only ever moves by hand. On a strict reading of "no output and no engagement over two weeks", most of the running fleet qualifies, so the wasted lanes dominate fleet spend. Pausing them is what frees budget for the teams that actually engage.

Closes #74866

Changes

This PR was reworked in place onto the scout lifecycle status primitive from #75349, as flagged in the earlier coordination comment. The failure breaker (#74458, merged) is the sibling consumer on the failure axis; this is the waste axis.

flowchart TD
    A{{"Daily sweep"}} --> B["Candidates: enabled, emitting, not exempt, past cold-start grace"]
    B --> C{"Productive in the last 14 days?"}
    C -- "yes" --> D["Clear any pending warning back to active"]
    C -- "no" --> E{"Ran enough times in the window?"}
    E -- "no" --> F["Left alone"]
    E -- "yes" --> G{"Already warned?"}
    G -- "no" --> H["Warn: pending_pause, reason no_output or ignored. Capped per sweep"]
    G -- "yes, a week ago" --> I["Pause: paused_by_system, enabled false"]
    classDef phBlue fill:#1d4aff,stroke:#1d4aff,color:#fff;
    classDef phYellow fill:#f9bd2b,stroke:#f9bd2b,color:#000;
    classDef phGray fill:#e5e7eb,stroke:#c7ccd1,color:#000;
    class A phYellow;
    class H,I phBlue;
    class D,F phGray;

Loading

• A daily Celery sweep (pause_inactive_signal_scouts, 06:15 UTC) drives scout_harness/inactivity.py. A scout counts as productive if any run in the last 14 days recorded output on any of the three emit channels (emitted_finding_ids / emitted_report_ids / edited_report_ids), or a person engaged with a report it wrote earlier (a log artefact attributed to a user, or the report reaching a user-driven status). Pipeline-written artefacts do not count.
• The sweep speaks the lifecycle vocabulary instead of keeping fields of its own. The warning is status=pending_pause (still scheduled), the pause is status=paused_by_system (syncing enabled=false), both through transition_status_by_system with the sweep-owned reasons no_output (surfaced nothing) and ignored (surfaced reports nobody picked up). The helper's reason-scoped ownership rule keeps this sweep and the failure breaker from touching each other's pauses, and evaluated_at makes a racing human edit win over a sweep decision made on stale reads.
• Guards: cold-start grace via the model's in_cold_start_grace() (re-anchored by a resume), a minimum-runs floor so budget-starved or slow-cadence scouts are never judged on absence of opportunity, dry-run scouts skipped, and a per-sweep cap on new warnings. Capping warnings bounds later pauses too, since a pause can only follow a warning by the seven-day grace, and it keeps every warned scout's "pauses in a week" promise honest. Deferrals are counted and logged so a capped sweep reads as partial, not clean.
• New auto_pause_exempt on the config (migration 0083, non-blocking AddField with db_default): the opt-out for watchdog scouts whose whole job is staying quiet. It is also set automatically when a human re-enables a sweep-paused scout, so the sweep never overrules a person twice. An explicit auto_pause_exempt=false in the same re-enable request wins.
• Observability: the sweep emits signals_scout_auto_pause_warned / signals_scout_auto_paused, and a human re-enable of a sweep pause emits signals_scout_auto_pause_reverted with hours-since-pause, the leading false-positive indicator (a revert within a day means the rule paused something someone still wanted). Activity log entries for sweep transitions carry the job trigger so they never read as unattributed human edits.
• UI: the fleet page badge shows "Quiet" while pending_pause and "Paused" once paused (folded into one lifecycle badge with the breaker's), and the config form gains a "Keep running while quiet" switch. status_changed_at and auto_pause_exempt are exposed read-only on the config serializer; generated types and MCP tool schemas regenerated.
• Docs updated in the same PR: scout_harness/AGENTS.md and the authoring-scouts skill.

Note

The pause is one-click reversible (enabled=true), and that re-enable permanently exempts the scout from this sweep. There is no probe on this axis: unlike a repeated_failures pause, an inactivity pause never runs again on its own.

How did you test this code?

Automated only, all run locally by the agent:

• test_scout_inactivity.py (21 tests): warn-then-pause sequencing and the no_output vs ignored classification; each emit channel and each human-engagement shape keeping a scout alive (pipeline artefacts don't); recovery clearing a warning; exempt / dry-run / user-paused / breaker-paused rows untouched; cold start; the minimum-runs floor; a resumed scout getting a full fresh window; the per-sweep warn cap deferring the overflow; the pause surviving lazy-seed reconciliation; sweep transitions activity-logged with the job trigger and no user.
• 5 new config API tests: re-enabling a sweep pause marks the scout exempt (and an explicit false in the same request wins), emits the revert event, a breaker pause does not mark exempt, and exempting clears a pending warning.
• Full scout config API suite (58), scout status + harness suites (92), 12 Jest tests across the touched frontend files, repo-wide mypy clean, makemigrations --check clean, sqlmigrate confirms a single non-blocking ADD COLUMN ... DEFAULT false NOT NULL.
• Not done: no live sweep against a real fleet. Day-one rollout leans on the warn-then-pause sequencing, the per-sweep cap, and the revert metric as the read-back.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

scout_harness/AGENTS.md and products/signals/skills/authoring-scouts/SKILL.md updated in this PR.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Reworked in a Claude Code session directed by the assignee, replacing the original bot-authored implementation in place (force-push, single commit). Skills invoked: /django-migrations, /writing-tests, /writing-user-facing-copy, /merging-prs.

Decisions along the way:

• Dropped the draft's five auto_pause_* columns for the feat(signals): add scout lifecycle status with reason-scoped pauses #75349 primitive; only auto_pause_exempt survives as a column. The warning timestamp is now just status_changed_at on a pending_pause row, and the "fresh window on resume" stamp is the grace helper's re-anchor.
• Kept both reasons in v1 rather than holding ignored back: scouts whose output nobody acts on are the point of the sweep, and a fleet quietly shrinking to its engaged core is the intended outcome, not a failure mode.
• The blast-radius cap lands on warnings rather than pauses, so the cap composes with the warn promise instead of silently deferring pauses past it.
• "Never re-pause after a human re-enable" is implemented by auto-setting the visible exemption toggle rather than a hidden marker, so the state is inspectable and reversible in the UI.

[trunk-io]

✨ Submitted to Merge by Andy Maguire (a GitHub user). It will be added to the merge queue once all branch protection rules pass and there are no merge conflicts with the target branch. See more details here.

[github-actions]

🤖 CI report

⚠️ Bundle size — 🔺 +1.5 KiB (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 65.44 MiB · 🔺 +1.5 KiB (+0.0%)

File	Size	Δ vs base
posthog-app/src/scenes/inbox/InboxScene.js	713.8 KiB	🔺 +1.5 KiB (+0.2%)

_{Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report}

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root	Eager (shipped)	Δ vs base	Budget
entry (logged-out pages, app bootstrap) src/index.tsx	1.25 MiB · 22 files	no change	███░░░░░░░ 27.7% of 4.51 MiB
authenticated shell (every logged-in page) src/scenes/AuthenticatedShell.tsx	8.12 MiB · 3,023 files	no change	████████░░ 83.6% of 9.71 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx

Size	File
126.8 KiB	../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB	../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB	../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB	../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB	../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB	../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB	src/RootErrorBoundary.tsx
912 B	../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B	src/scenes/ChunkLoadErrorBoundary.tsx
762 B	src/index.tsx

Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx

Size	File
285.3 KiB	../node_modules/.pnpm/posthog-js@1.409.2/node_modules/posthog-js/dist/rrweb.js
267.7 KiB	../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
235.5 KiB	src/taxonomy/core-filter-definitions-by-group.json
231.4 KiB	../node_modules/.pnpm/posthog-js@1.409.2/node_modules/posthog-js/dist/module.js
154.3 KiB	../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB	../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.2 KiB	src/lib/api.ts
94.7 KiB	../packages/quill/packages/quill/dist/index.js
93.3 KiB	../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.6 KiB	../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js

_{Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479}

✅ Toolbar bundle — eager 2.19 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric	Size	Δ vs base	Budget
Eager (shipped) entry + static imports	2.19 MiB · 17 files	no change	████░░░░░░ 38.3% of 5.72 MiB
Deferred (lazy)	2.08 MiB · 33 files	no change	n/a — loads on demand
Loader dist/toolbar.js	1.1 KiB	no change	█░░░░░░░░░ 5.8% of 19.5 KiB

Largest eagerly-shipped chunks

Size	File
717.7 KiB	dist/toolbar/toolbar-app-3ZUOLZ25.css
551.1 KiB	dist/toolbar/chunk-chunk-HCVXQK4K.js
484.6 KiB	dist/toolbar/chunk-chunk-I33HJPDC.js
133.6 KiB	dist/toolbar/chunk-chunk-LD5GQHAS.js
131.8 KiB	dist/toolbar/chunk-chunk-T5KY5WYR.js
71.0 KiB	dist/toolbar/toolbar-app-DUPY5K74.js
69.0 KiB	dist/toolbar/chunk-chunk-27JL52RE.js
35.6 KiB	dist/toolbar/chunk-chunk-7YZYKQFX.js
20.9 KiB	dist/toolbar/chunk-chunk-4TTVMURW.js
12.2 KiB	dist/toolbar/chunk-chunk-PIK3PADE.js

_{Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile}

✅ Dist folder size — 🔺 +6.4 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1371.16 MiB · 🔺 +6.4 KiB (+0.0%)

✅ Playwright — all passed

All tests passed.

View test results →

⚠️ Backend coverage — 94.0% of changed backend lines covered — 20 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ███████████████████░ 94.0% (321 / 341)

File	Patch	Uncovered changed lines
products/signals/backend/tasks.py	23.5%	518–519, 527–528, 531–532, 536–537, 541–545
products/signals/backend/scout_harness/serializers.py	92.0%	2077–2078
products/signals/backend/scout_harness/inactivity.py	95.0%	153, 155–156, 225, 286

🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 30630511121 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)

Product	Coverage	Lines
platform_features	██░░░░░░░░░░░░░░░░░░ 12.1%	7 / 58
batch_exports	████████░░░░░░░░░░░░ 39.4%	8,782 / 22,290
demo	███████████░░░░░░░░░ 56.3%	1,497 / 2,661
warehouse_sources_queue	████████████░░░░░░░░ 59.2%	148 / 250
data_tools	██████████████░░░░░░ 70.0%	63 / 90
tasks	██████████████░░░░░░ 70.1%	32,865 / 46,867
ai_gateway	███████████████░░░░░ 75.0%	9 / 12
signals	████████████████░░░░ 81.4%	25,264 / 31,030
cdp	████████████████░░░░ 82.1%	3,285 / 3,999
data_modeling	█████████████████░░░ 85.7%	7,716 / 9,008
notebooks	█████████████████░░░ 86.0%	7,794 / 9,060
wizard	█████████████████░░░ 86.4%	1,060 / 1,227
actions	█████████████████░░░ 86.6%	717 / 828
cohorts	██████████████████░░ 87.8%	6,181 / 7,040
product_tours	██████████████████░░ 87.9%	1,303 / 1,482
exports	██████████████████░░ 88.2%	7,046 / 7,986
data_warehouse	██████████████████░░ 88.4%	12,192 / 13,798
business_knowledge	██████████████████░░ 89.0%	4,391 / 4,936
engineering_analytics	██████████████████░░ 89.3%	6,529 / 7,309
conversations	██████████████████░░ 89.3%	17,600 / 19,701
dashboards	██████████████████░░ 89.4%	5,983 / 6,693
visual_review	██████████████████░░ 89.5%	5,870 / 6,558
alerts	██████████████████░░ 90.2%	4,458 / 4,942
links	██████████████████░░ 90.6%	183 / 202
streamlit_apps	██████████████████░░ 90.7%	2,630 / 2,901
mcp_analytics	██████████████████░░ 90.9%	3,191 / 3,511
error_tracking	██████████████████░░ 91.0%	10,926 / 12,008
slack_app	██████████████████░░ 91.1%	9,664 / 10,610
marketing_analytics	██████████████████░░ 91.2%	12,092 / 13,265
stamphog	██████████████████░░ 91.3%	4,505 / 4,936
mcp_store	██████████████████░░ 91.9%	4,257 / 4,634
product_analytics	███████████████████░ 92.5%	5,849 / 6,321
managed_migrations	███████████████████░ 92.6%	1,556 / 1,681
early_access_features	███████████████████░ 92.6%	1,287 / 1,390
notifications	███████████████████░ 92.6%	1,017 / 1,098
ai_observability	███████████████████░ 92.8%	15,618 / 16,821
surveys	███████████████████░ 93.1%	5,771 / 6,197
posthog_ai	███████████████████░ 93.2%	1,326 / 1,422
web_analytics	███████████████████░ 93.3%	14,911 / 15,976
approvals	███████████████████░ 93.3%	3,437 / 3,682
reminders	███████████████████░ 93.4%	468 / 501
legal_documents	███████████████████░ 93.8%	1,628 / 1,736
workflows	███████████████████░ 94.1%	7,241 / 7,698
endpoints	███████████████████░ 94.2%	8,655 / 9,192
tracing	███████████████████░ 94.5%	2,671 / 2,827
review_hog	███████████████████░ 94.6%	8,101 / 8,563
skills	███████████████████░ 94.6%	3,158 / 3,337
messaging	███████████████████░ 94.7%	2,885 / 3,048
experiments	███████████████████░ 95.4%	25,938 / 27,177
logs	███████████████████░ 95.5%	10,440 / 10,937
growth	███████████████████░ 96.1%	3,245 / 3,376
annotations	███████████████████░ 96.2%	732 / 761
revenue_analytics	███████████████████░ 96.3%	1,887 / 1,960
feature_flags	███████████████████░ 96.4%	17,488 / 18,144
replay_vision	███████████████████░ 96.4%	16,081 / 16,679
user_interviews	███████████████████░ 96.5%	2,638 / 2,734
access_control	███████████████████░ 96.9%	870 / 898
customer_analytics	███████████████████░ 97.1%	10,444 / 10,758
warehouse_sources	███████████████████░ 97.3%	359,280 / 369,280
data_catalog	████████████████████ 97.7%	2,588 / 2,648
analytics_platform	████████████████████ 98.0%	2,153 / 2,197
metrics	████████████████████ 98.2%	2,491 / 2,536
pulse	████████████████████ 98.4%	2,017 / 2,049
live_debugger	████████████████████ 99.2%	613 / 618
field_notes	████████████████████ 99.4%	158 / 159

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

⚠️ Django migration SQL — 1 new migration to review

We've detected new migrations on this PR. Review the SQL output for each migration:

products/signals/backend/migrations/0083_signalscoutconfig_auto_pause_exempt.py

BEGIN;
--
-- Add field auto_pause_exempt to signalscoutconfig
--
ALTER TABLE "signals_signalscoutconfig" ADD COLUMN "auto_pause_exempt" boolean DEFAULT false NOT NULL;
COMMIT;

Last updated: 2026-07-31 12:29 UTC (8834cce)

✅ Django migration risk — migration analysis complete

We've analyzed your migrations for potential risks.

Summary: 1 Safe | 0 Needs Review | 0 Blocked

✅ Safe

Brief or no lock, backwards compatible

signals.0083_signalscoutconfig_auto_pause_exempt
  └─ #1 ✅ AddField
     Adding NOT NULL field with constant default (safe in PG11+)
     model: signalscoutconfig, field: auto_pause_exempt

Last updated: 2026-07-31 12:29 UTC (8834cce)

[pr-assigner-resolver-posthog]

👀 Auto-assigned reviewers

These soft owners were skipped because they only have minor changes here. Nothing blocks merge, so self-assign if you'd like a look:

• @PostHog/team-platform-features (posthog/models/owners.yaml)

Soft owners come from each directory's owners.yaml and each product's product.yaml (resolved nearest-file-wins). The locator after each owner is the file that decided it. Generated files and lockfiles are ignored when deciding ownership.

[andrewm4894]

Heads-up on direction: the lifecycle primitive this PR needs has now landed separately in #75349 (approved), and the plan agreed in the linked design discussion is for this PR to rebase onto it as a consumer rather than introduce its own pause fields.

What #75349 gives you, replacing the auto_pause_warned_at / auto_paused_at / auto_pause_reason set here:

• status enum (active / pending_pause / paused_by_system / paused_by_user) + a separate pause_reason (no_output / ignored / repeated_failures). The warn step becomes the pending_pause state, which makes the sweep idempotent for free and gives any human config edit something concrete to clear (the update serializer already clears it).
• transition_status_by_system(new_status, pause_reason=..., evaluated_at=...) — the one entry point for system writers. It enforces the reason-scoped ownership rule (this sweep can never touch a paused_by_user row or another writer's pause), and evaluated_at is a compare-and-set: pass the time you evaluated the scout and a human touch that lands after it wins instead of being overwritten.
• in_cold_start_grace() for the cold-start guard (time-based; keep your ≥N-runs floor as a sweep-side check on top), and one-click re-enable + keep-alive already implemented in the config serializer.
• enabled stays dual-written, so the coordinator and warehouse are unaffected.

Scope requests for the rebase, from the same design discussion:

1. v1 should pause on no_output only and hold ignored back. no_output involves no judgment about humans; ignored is where the false positives live. On a strict reading of the current rule, the large majority of currently-running configs qualify on day one — and mostly because the team is dormant, not because the scout is bad — so the conservative half first.
2. Add a per-sweep cap on pauses applied per run, given that candidate-set size.
3. Never auto-pause a scout a human re-enabled (the re-enable is a human overruling the rule).
4. Emit a re-enable analytics event and treat re-enable-within-24h-of-pause as a false positive — the leading indicator, since complaints are lagging and lossy.

Mechanical notes: migrations 0075 and 0076 are taken by #75349, so the migration here needs renumbering; the auto_pause_exempt opt-out is consumer-specific and fine to keep. Happy to help with the rebase.

[posthog]

🦔 ReviewHog is reviewing this pull request

Step 5/6 · Validating findings · 3/14

Specialist review skills read the changed code in parallel each from their own perspective, a blind-spot sweep catches what they missed, and only validated findings are published back to this pull request.

_{This comment updates as the review progresses.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 78c1957a28

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[andrewm4894]

/trunk merge