Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1 +1,6 @@
ANTHROPIC_API_KEY=sk-ant-your-api-key-here

# Position and the watch set are hand-authored and live in your private repo,
# not this one. Defaults: ~/.config/insightweaver/{position,watches}.yaml
POSITION_PATH=~/.config/insightweaver/position.yaml
WATCHES_PATH=~/.config/insightweaver/watches.yaml
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
# Environment variables
.env

# Position and the watch set (backlog task 013, 2026-08-31).
# These name real decisions, deadlines, exposures and beliefs. They belong in
# the operator's PRIVATE repo -- under version control there, because the
# quarterly review's value is the diff -- and never in this public one. Point
# POSITION_PATH / WATCHES_PATH at them; the defaults live outside any checkout.
# The *.example.yaml files in config/ are committed and are not matched here.
/position.yaml
/watches.yaml
config/position.yaml
config/watches.yaml

# User profile and personal configuration (contains personal information)
config/user_profile.json
config/context_modules/core/decision_context.json
Expand Down
9 changes: 8 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: help install install-dev test lint format fmt typecheck check pre-commit clean clean-all coverage db-drop-briefing update-deps
.PHONY: help install install-dev test lint format fmt typecheck check pre-commit clean clean-all coverage db-add-watches db-drop-briefing update-deps

# Tool resolution (added 2026-08-24).
# Before this, every recipe called bare `pytest` / `ruff` / `mypy`, which only
Expand Down Expand Up @@ -44,6 +44,7 @@ help:
@echo " make check Run all checks (lint + typecheck + test)"
@echo ""
@echo "Database:"
@echo " make db-add-watches Create the watches table (additive, safe)"
@echo " make db-drop-briefing Drop the deleted briefing product's tables"
@echo " (DESTRUCTIVE; captures to a dump first)"
@echo ""
Expand Down Expand Up @@ -113,6 +114,12 @@ clean-all: clean
rm -rf venv/
rm -rf .venv/

# Additive: creates the watches table if it is absent and prints that it did
# nothing if it is not. No --confirm, because nothing here can lose data; the
# --down direction can, and requires one. (2026-08-31, backlog task 013.)
db-add-watches:
$(PYTHON) -m src.database.migrations.add_watches_table

# Deliberately does NOT pass --confirm. The migration refuses without it, so
# running this target prints what it would destroy and stops; the operator types
# the real command themselves. A Makefile target that drops tables on `make` is
Expand Down
2 changes: 1 addition & 1 deletion backlog/017-tier3-state-machine.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ STATUS: QUEUED # QUEUED | IN_PROGRESS | PARKED | DONE | FAILED
SIZE: large
ACCEPTANCE: `make check` passes, plus: belief updates and trigger evaluation are deterministic with **no LLM call**, proved the same way as Tier 1; hysteresis requires N independent sources or one source on a per-watch pre-declared authoritative list, and `N` is per-watch configuration, not a global constant; a debounce window collapses the same story arriving from many outlets into **one** transition, with a test driving twelve near-duplicate observations through and asserting exactly one Alert; a belief drift of 0.35 to 0.41 produces **no** Alert, asserted directly as the brief specifies; an Alert is created only on a state transition, never on evidence arrival; transitions are append-only and a Watch's belief history is reconstructable; and a test asserts that replaying the same Evidence twice produces the same transitions and no duplicate Alerts.
OUT OF SCOPE: Sending anything -- Alerts are created here and delivered by task 019. Silencing or grouping UI. Resolution scoring; that is task 021. Auto-adjusting hysteresis from observed alert volume, which would let the system tune away its own noise complaint. Belief models more elaborate than what the acceptance requires -- no Bayesian update framework unless a specific watch demonstrably needs one, and if one does, park and say so.
LANDMINES: **This is the task where the rewrite's whole property lives or dies, and every failure mode here is silent.** Hysteresis too weak floods; too strong means the system never fires and looks identical to a broken pipeline -- which is why task 018 must land alongside, not after. **"N independent sources" needs a definition of independent that survives contact with reality**: three outlets syndicating one wire story are not three sources, and the near-duplicate grouping from task 014 is what makes the distinction available -- if it is not wired in here, "independent" degrades to "distinct feed URL" and the debounce is defeated by exactly the case it exists for. **The state machine must be a pure function of (prior state, evidence) or the append-only history is not reconstructable.** Reading current belief from a mutable column and writing back in place will pass tests and lose the audit trail. `main` is protected -- open a PR.
LANDMINES: **`sync_watches` currently DELETES the row for a watch removed from the file** (task 013, `src/position/watches.py`). That is correct while the table holds nothing derived. The moment this task hangs belief history off those rows, a delete destroys the record of what the operator believed and when -- which is the calibration data the system exists to accumulate. Change it to a soft retire before wiring history, or the first watch someone removes takes its own history with it and nothing surfaces the loss. **This is the task where the rewrite's whole property lives or dies, and every failure mode here is silent.** Hysteresis too weak floods; too strong means the system never fires and looks identical to a broken pipeline -- which is why task 018 must land alongside, not after. **"N independent sources" needs a definition of independent that survives contact with reality**: three outlets syndicating one wire story are not three sources, and the near-duplicate grouping from task 014 is what makes the distinction available -- if it is not wired in here, "independent" degrades to "distinct feed URL" and the debounce is defeated by exactly the case it exists for. **The state machine must be a pure function of (prior state, evidence) or the append-only history is not reconstructable.** Reading current belief from a mutable column and writing back in place will pass tests and lose the audit trail. `main` is protected -- open a PR.
---
Written 2026-08-31. **The largest and most load-bearing task in this plan.**

Expand Down
22 changes: 20 additions & 2 deletions backlog/022-open-questions.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,17 @@ has no partial-failure story, and that migrating off it later is real work.
**To answer:** either grant `pricing:GetProducts` and I will produce the figures, or choose on
shape and accept that the cost comparison is unverified.

## Q2 -- Watch versus the existing Decision + Factor model. Blocks task 013.
## Q2 -- Watch versus the existing Decision + Factor model. Answered by events; confirm or overrule.

**What task 013 built, 2026-08-31.** Task 012 landed first and dropped `decisions`,
`decision_factors` and `decision_evidence`, all at 0 rows -- so there was nothing left to evolve
and the question became "clean shape or resurrection". 013 built `watches` as a new table and put
Decision in **Position**, not in the database: a decision is an operator commitment held in a
hand-edited file under git, not a row the system writes. That leaves one model of the idea, not
two. `belief` is a float and `triggers` is structured, which is the part of the old model worth
not repeating. Overrule this and 013 is the task to revisit.

### Original question

The audit found the repository already models most of Watch, and has never used it:
`decision_factors.what_would_update_me` is a trigger, `.current_state_note` is a belief state,
Expand All @@ -81,7 +91,15 @@ choice on its merits. But it is a schema decision with migration consequences an
in six months). I have no recommendation strong enough to make silently; the task file requires the
implementer to state the rule, but if you have a preference it belongs here first.

## Q4 -- The `so_what` -> decision link. Blocks task 013.
## Q4 -- The `so_what` -> decision link. Built as the key; confirm or overrule.

**What task 013 built, 2026-08-31.** The key, with the prose as an annotation, as the acceptance
assumed: `so_what` is a mapping of `decision` (a key into the Position's `decisions`) and
`because` (the sentence naming what changes). A watch whose `decision` names nothing in the
Position is rejected at load and not stored. The prose is still required and still non-empty --
the key alone would make the invariant checkable and unreadable.

### Original question

Invariant 2 requires every Watch to name a specific decision. Should that be a **free-text
`so_what` string**, or a **foreign key into the Position's decisions** with the prose as an
Expand Down
52 changes: 52 additions & 0 deletions config/position.example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Position -- EXAMPLE ONLY.
#
# Your real Position does not live in this repository. It names real decisions,
# real deadlines and real exposures, and this repo is public. Keep it in a
# private repo under git -- under git because the quarterly review's value is
# the diff -- and point POSITION_PATH at it:
#
# export POSITION_PATH=~/src/position/position.yaml
#
# The default, if you set nothing, is ~/.config/insightweaver/position.yaml.
# .gitignore in this repo refuses a real position.yaml, but the gitignore is a
# seatbelt, not the seat.
#
# The rule this file is written against: a Position holds *stakes*, not
# interests. A stake has a date by which you must have decided, and something
# that is worse if you decide wrong. If you cannot write the deadline, it is an
# interest -- interesting, possibly important, and not what this system is for.
# The loader refuses a decision with no deadline and warns, loudly, when the
# whole file runs past two pages, because that is what drift looks like.

version: 1

# When you last sat down with this file. Task 024 turns this into the quarterly
# review loop; for now it is a date you can read.
reviewed: 2026-08-31

decisions:
- key: fedramp-authorization-renewal
name: Renew the platform's FedRAMP Moderate authorization, or let it lapse
deadline: 2027-03-31
stake: >
Renewal costs roughly two engineer-quarters of continuous-monitoring work.
Letting it lapse closes the federal pipeline, which was 40% of last year's
revenue. The decision has to be made before the annual assessment window
opens, not after the invoice arrives.

- key: hire-compliance-engineer
name: Hire a dedicated compliance engineer, or keep the work distributed
deadline: 2026-12-15
stake: >
If continuous-monitoring scope grows, the distributed arrangement stops
working and the failure mode is a missed POA&M deadline nobody owns.
Hiring ahead of that costs a salary against uncertain need; hiring after
it costs an authorization.

- key: second-cloud-region
name: Commit to a second cloud region for the government workload
deadline: 2027-01-31
stake: >
A second region is the only way to meet the availability commitment in the
renewal, and standing one up takes a quarter. Deciding in February means
deciding not to.
94 changes: 94 additions & 0 deletions config/watches.example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Watches -- EXAMPLE ONLY.
#
# Like Position, the real file lives in your private repo. Point WATCHES_PATH at
# it; the default is ~/.config/insightweaver/watches.yaml.
#
# A watch is one claim you have pre-registered, the decision it serves, and what
# would count as evidence. Nothing but this file creates a watch. There is no
# `watch add` command and there is no path from a model response to a row in the
# watches table -- the system never authors its own watches (invariant 6).
#
# Every field is required and none of them defaults:
#
# id slug; the name an alert will carry
# claim the thing that is or is not true, written so you could
# tell afterwards which it was
# belief your probability now, 0.0 to 1.0
# so_what.decision a key from position.yaml -- not prose, a key, so that
# "which watches serve this decision" is a query
# so_what.because what changes for that decision if the claim resolves
# triggers structured; see below
# expires when the question stops mattering
# staleness_alert_days after how much silence the silence is itself the
# finding. Accepts a whole number of days or a cadence
# string ("30d", "2w", "3m", "1y").
#
# ON TRIGGERS, which is the field people get wrong:
#
# `triggers` is a list of clauses, and a clause is a mapping over `terms`,
# `entities` and `sources`. It is NOT a sentence. Within a clause, every field
# you populate must match (AND) and any value within a field will do (OR);
# across clauses, any clause firing fires the watch. So the shape is a
# disjunction of conjunctions, which Tier 1 compiles into word-boundary regexes
# and a source-membership test with no model involved anywhere.
#
# The reason the loader rejects prose here rather than storing it: the deleted
# prediction ledger had a free-text `trigger_condition`, 33 rows were written
# against it, and none of them were ever graded, because nothing downstream
# could evaluate a sentence. A trigger you cannot compile is a trigger that
# never fires.

version: 1

watches:
- id: fedramp-conmon-scope-expands
claim: >
FedRAMP expands continuous-monitoring requirements for Moderate systems in
a way that adds recurring work, before the 2027 assessment window.
belief: 0.35
so_what:
decision: fedramp-authorization-renewal
because: >
Scope expansion moves renewal from two engineer-quarters to three or
more, which is the point at which lapsing becomes the cheaper option.
triggers:
- entities: [FedRAMP PMO, GSA]
terms: [continuous monitoring, ConMon, significant change]
- sources: [Federal Register]
terms: [FedRAMP]
expires: 2027-03-31
staleness_alert_days: 30

- id: fedramp-20x-supersedes-rev5
claim: >
The FedRAMP 20x program replaces the Rev 5 baseline for new
authorizations, rather than running alongside it.
belief: 0.45
so_what:
decision: fedramp-authorization-renewal
because: >
If 20x supersedes rather than supplements, the renewal is a
re-authorization against a different baseline and the estimate is wrong
by a factor, not a margin.
triggers:
- terms: [FedRAMP 20x, Rev 5, revision 5]
entities: [FedRAMP PMO]
expires: 2027-06-30
staleness_alert_days: 3w

- id: compliance-hiring-market-tightens
claim: >
Median time-to-hire for a compliance engineer in the DC market exceeds 90
days through the end of 2026.
belief: 0.6
so_what:
decision: hire-compliance-engineer
because: >
A 90-day search started in December lands after the assessment window
opens, which converts "hire later if needed" into "do not hire".
triggers:
- terms: [time to hire, compliance engineer, GRC hiring]
- entities: [OPM]
terms: [cyber workforce, hiring authority]
expires: 2026-12-15
staleness_alert_days: 45
22 changes: 19 additions & 3 deletions src/cli/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
from .colors import accent, header, muted
from .output import set_debug_mode
from .sources import sources_command
from .watch import watch_command

# Maps command prefix to its Click command object.
#
Expand All @@ -29,25 +30,40 @@
# the ordering constraint still applies to whatever is added next.
COMMAND_DISPATCH = {
"sources": sources_command,
"watch": watch_command,
}


def print_command_refresher():
"""Print a short refresher of available commands."""
click.echo(f"\n{header('Commands:')} {accent('sources')} | help | exit\n")
click.echo(f"\n{header('Commands:')} {accent('sources')} | {accent('watch')} | help | exit\n")


def print_help():
"""Print full help text for interactive mode."""
click.echo(header("Available commands:"))
click.echo(f" {accent('sources')} - Inspect configured feeds and what they hold")
click.echo(
f" {accent('watch')} - Pre-registered watches and the decisions they serve"
)
click.echo(f" {accent('help')} - Show this help message")
click.echo(f" {accent('exit')} - Exit InsightWeaver")
click.echo()
click.echo(header("Sources command:"))
click.echo(f" {accent('sources list')} - All feeds with stored-article counts")
click.echo(f" {accent('sources show')} <name> - Detailed view for one feed")
click.echo()
click.echo(header("Watch command:"))
click.echo(
f" {accent('watch list')} - Each watch, belief, decision, days to expiry"
)
click.echo(
f" {accent('watch sync')} - Load the hand-authored Position and watch files"
)
click.echo(
muted(" Watches are only ever created from those files -- there is no 'watch add'.")
)
click.echo()
click.echo(muted("Tip: Add --debug to any command to see detailed logs"))
click.echo()

Expand Down Expand Up @@ -154,10 +170,10 @@ def interactive_mode():
@click.version_option(version="1.0.0", prog_name="InsightWeaver")
def cli(ctx, debug):
"""
InsightWeaver - RSS feed ingestion and source inspection.
InsightWeaver - monitoring against pre-registered watches.

The briefing product was removed in backlog task 012. What remains is
ingestion and the source layer.
ingestion, the source layer, and the Position/Watch units added by task 013.
"""
set_debug_mode(debug)
ctx.ensure_object(dict)
Expand Down
Loading
Loading