Skip to content

plan: Stage 0 for the monitoring re-architecture - #19

Merged
sam-aydlette merged 5 commits into
mainfrom
plan/monitoring-rearchitecture
Aug 31, 2026
Merged

sam-aydlette merged 5 commits into
mainfrom
plan/monitoring-rearchitecture

Conversation

@sam-aydlette

Copy link
Copy Markdown
Owner

Stage 0 only — audit, reconciliation, twelve task files, parked questions. No implementation.

Where the brief's assumptions were wrong

The brief was written from README.md, CLAUDE.md, and the essay, without the source tree. Five things:

  1. src/context/ cannot be moved wholesale — it holds both products. Of its 4,843 lines, ~2,415 are briefing machinery (synthesizer 915, topic_matcher 611, curator 461, frame_manager 324, cross_cluster_reconciler 104). The rest is machinery the new product keeps or ports. The cut is file-by-file. This is the finding that most changed the plan.
  2. The archive boundary is otherwise clean — exactly one import crosses it (orchestrator.py:13).
  3. There are two migration directories. A tracked top-level migrations/ (001, 002, Jan 2026) and src/database/migrations/ (11 files), which is what produced the live 21-table schema. The brief points at the former.
  4. An smtplib send path exists today at src/render/email.py:124, reachable from brief --format email. Invariant 1 is violated in the current tree, not just at risk. Task 012 deletes rather than archives it.
  5. .claude/skills exists (add-feature, review-code, review-tests) and .github/workflows is one ci.yml with five jobs. Both survive the rewrite unchanged.

The finding that most affects the schema work

The repository already models most of Watch, and has never used it:

brief's concept existing column rows
trigger decision_factors.what_would_update_me 0
belief state decision_factors.current_state_note 0
so_what target decisions.name 0
evidence direction decision_evidence.direction 0

Not an argument for reusing it as-is — current_state_note is prose where belief is a float. It is an argument for deciding deliberately between evolving Decision+Factor and building Watch alongside, rather than discovering the overlap mid-implementation.

The plan

task size
012 Archive the briefing product M
013 Position + Watch, so_what enforced in schema M
014 Observations immutable, Evidence derived, replay harness M
015 Tier 1 deterministic routing M
016 Tier 2 adjudication, single LLM tier M
017 Tier 3 state machine, hysteresis, debounce L
018 Silence: staleness, dead-man, weekly receipt M
019 Tier 4 SES notification M
020 v0 deployment L
021 Tier 5 slow loops M
023 The state feed M
024 Setup interview + quarterly review loop M

Two sequencing calls worth arguing about:

018 lands alongside 017, not after. The moment hysteresis exists, silence becomes ambiguous. Invariant 5 was flagged as the requirement most likely to be dropped under schedule pressure — sequencing it last is how that happens.

020 cannot be marked DONE at merge. Its acceptance is a three-week measurement of real email volume, and the three possible outcomes are written into the task in advance so the result cannot be reinterpreted afterward.

On Prometheus/Alertmanager

Not running the binary — for one user and fewer than two emails a week, a persistent server is more surface than the thing it manages. Adopting the vocabulary exactly (group_by, group_wait, repeat_interval, inhibit_rules, silences, and for: as hysteresis) means behaviour is documented by Prometheus's own docs, and a later decision to POST to a real Alertmanager becomes a transport swap rather than a rewrite.

Decisions recorded

  • Position lives in a separate private repo — not mainly for safety, but because the quarterly review's value is the diff, and gitignoring the file means no history.
  • Quarterly review degrades visibly, it does not suppress. A stale Position costs a wrongly-prioritised alert; suppression costs a missed one, and those are not symmetric. Suppression would also reintroduce exactly the ambiguity invariant 5 exists to remove.

Still parked

Persistence pricing — I could not verify it and did not estimate. The RDS page renders figures in interactive elements that do not survive extraction, and pricing:GetProducts returns AccessDeniedException for user/saydlette-dev. Grant that permission for real numbers, or choose on shape.

Q5 — what happens to beats, questions, predictions and the calibration loop. Roughly 2,400 lines with 844 tests behind them. The plan neither archives nor ports it, deliberately: coverage_probes is a staleness check by another name and the calibration loop is watch resolution scoring, but institutional activity is a briefing feature. Deciding needs your view on whether the beat survives as a concept.

Audit, reconciliation note, ten staged task files, and six parked questions.
No implementation.

The audit corrected several assumptions the brief was written under, the
largest being that src/context/ mixes both products and cannot be moved
wholesale: roughly 2,400 of its 4,843 lines are briefing machinery and the
rest is machinery the new product keeps or ports.

Also found: two migration directories, an existing smtplib send path that
violates invariant 1 today, and an existing Decision+Factor model that
already expresses most of Watch and holds zero rows.
023 -- a password-protected feed carrying current state, on the principle
that email is good at transitions and bad at state. Non-addictive by
construction rather than restraint: bounded card list, no scroll, no counts,
no history stream, and an acceptance criterion that it shows nothing the
email does not. Auth fails closed, reusing the pattern corrected in
silk-reeling-mirror PR #5.

024 -- an interview that drafts Position to a scratch path for human editing,
and a quarterly review loop. The review is the outermost loop and the only
one that corrects the inputs to all the others; if it lapses nothing breaks
and everything gradually stops being about anything that matters, which
nothing inside the system can detect.

Review degrades visibly rather than suppressing alerts. A stale Position
costs a wrongly-prioritised alert; suppression costs a missed one, and those
are not symmetric. Suppression would also reintroduce exactly the ambiguity
invariant 5 exists to remove.
The task said to reuse the corrected Basic-Auth pattern from
silk-reeling-mirror PR #5. That repo had already replaced Basic Auth with
Cognito JWT, and carried the fail-open fallback into the new implementation.
Both are fixed now, but the lesson the landmine should carry is the second
one: a full re-implementation preserved the hole, so a rewrite is not a fix
and this gate must be verified behaviourally with the config absent.
…s beats and questions

The operator's decisions: beats and questions are removed and Position
replaces them; the old code is deleted rather than archived because git
already holds it; persistence stops blocking.

012 is rewritten from an archive task to a deletion and resized to large --
roughly 8,900 lines of source plus most of 844 tests, which is most of the
product built to date.

Five modules are ported rather than rewritten, and the argument is specific
rather than sentimental. entity_matcher encodes that word boundaries are the
difference between 73 and 5,364 matches on one term in this corpus, and that
its own boundary test passed with the anchors deleted because case-sensitivity
rejected the collisions first. claude_client encodes a retired model that
failed silently for ten weeks and a response-shape change that discarded a
completed synthesis. Rewriting either from a blank file reintroduces bugs
that took a day to find.

An archive/ directory is rejected on the operator's reasoning: dead code in
the tree reads as current to every future reader and drags the old product's
shape forward, which is the outcome the rewrite exists to avoid.
@sam-aydlette
sam-aydlette merged commit 673a8dd into main Aug 31, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant