ttl-reaper: no alarm detects the reaper NOT RUNNING — all four #469 alarms are notBreaching, so absence is silent

[scttfrdmn]

#469 made the reaper's failures observable. It did not make its absence observable — a reaper that never runs still pages nobody.

The hole

All four alarms from #469 use TreatMissingData: notBreaching. That is correct per-alarm: no sentinel datapoint genuinely is good news, and the alternative would page on every quiet run. But it means the composite condition

the function is not being invoked at all

produces zero breaching datapoints across every alarm, so all four sit contentedly in OK while nothing is being enforced.

Ways to reach that state, none of which the current alarms detect:

• the EventBridge rule is disabled (manually, or by a failed deploy)
• the schedule is deleted, or its target detached
• reserved concurrency set to 0
• the function is deleted outright

Verified today: spawn-ttl-reaper-producti-TTLReaperFunctionSchedule-vkVA3mfgcUkk is ENABLED at rate(10 minutes), so this is latent, not live.

Why it matters here specifically

This is #469's own thesis one level up, and #65's two levels up. The reaper exists because spored can die silently; #469 exists because the reaper could fail silently; this exists because the reaper can be absent silently. Each layer was added to remove an assumption, and this is the last unmonitored one in the chain: "something is running the reaper."

The …-invocation-errors alarm does not cover it — AWS/Lambda Errors requires an invocation to produce a datapoint. No invocation, no error, no alarm.

Proposed fix

One more alarm in lambda/ttl-reaper/template.yaml, gated on the existing ObservabilityEnabled condition:

• Metric AWS/Lambda Invocations, Sum, dimension FunctionName
• ComparisonOperator: LessThanThreshold, Threshold: 1
• Period 1800s / 1 datapoint (≈3 missed ticks at rate(10 minutes))
• TreatMissingData: breaching — the inversion is the entire point, and it is the one alarm in this template where notBreaching would be wrong

Period must be derived loosely enough to tolerate a single skipped tick, but the threshold should not be widened so far that hours of silence pass unnoticed. Worth a comment saying why this alarm's TreatMissingData differs from its four neighbours, or someone will "fix" the inconsistency later.

Acceptance

• Disabling the EventBridge rule causes an alarm within ~30 min.
• A normal healthy run keeps it in OK.
• Documented in the README's sentinel/alarm table alongside the other four.

Follow-up to #469 / PR #471, noticed while reviewing the deployed alarm set.