From ae4afcf36a9c9e80c41fc5fa79722b3a1c96113c Mon Sep 17 00:00:00 2001 From: Mary Womack Date: Sun, 19 Apr 2026 11:16:09 -0400 Subject: [PATCH 1/6] DET-001: Netlify function determinism audit Classify all 91 Netlify function handlers against the 7-level automation determinism framework. 70 are Level 1, 18 are Level 2, 2 are Level 3, 2 are Level 5. Seven functions flagged for downgrade with Level 1/2 rebuild notes. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/determinism-audit.md | 188 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 docs/determinism-audit.md diff --git a/docs/determinism-audit.md b/docs/determinism-audit.md new file mode 100644 index 00000000..44d54104 --- /dev/null +++ b/docs/determinism-audit.md @@ -0,0 +1,188 @@ +# Determinism Audit — mmt-site Ecosystem + +**Sprint:** Determinism Sprint +**Doctrine:** Deterministic by default. Reasoning by exception. Auditable by design. +**Status:** Sprint 0 in progress. This is a read-only audit. No workflows change until Sprint 2. + +## The 7-Level Framework (reference) + +| Level | Category | Use When | +|---|---|---| +| 1 | Deterministic script (no LLM) | Same inputs, same steps, same outputs | +| 2 | Script + one targeted LLM call | One fuzzy step, rest is deterministic | +| 3 | No-code agent builder (n8n agent, Dify) | Light branching, prebuilt connectors | +| 4 | Hand-rolled agentic loop (50-150 LOC) | Real agent behavior, 3-5 custom tools | +| 5 | Code-first framework (LangGraph, CrewAI) | Multi-agent, complex state | +| 6 | Terminal AI agent (Claude Code) on-demand | Open-ended, human-summoned | +| 7 | Autonomous always-on agent (OpenClaw) | Cross-domain, persistent, scheduled improvisation | + +Multiple sequential LLM calls in a fixed pipeline are still Level 2. It is deterministic orchestration with fuzzy steps. + +--- + +## Netlify Functions — mmt-site (91 handlers) + +Source: `~/Projects/mmt-site/netlify/functions/`. Every handler read top-of-file through the core logic branch. Classification follows the rule: if the task is a fixed sequence of steps, it is Level 1. + +| function_name | trigger | apis | decision_points | current_level | target_level | downgrade_opportunity | +|---|---|---|---|---|---|---| +| agent-bridge | HTTP POST/GET | Supabase | Task dedup, approval routing, status enum transitions | 3 | 1 | Y | +| ai-image | HTTP POST | OpenAI, Google AI | Provider selection (enum) | 2 | 2 | N | +| ai-research | HTTP POST | Anthropic, OpenAI, Google | Multi-LLM parallelization | 2 | 2 | N | +| approval-api | HTTP GET/POST | Supabase | 5 views + 6 actions | 1 | 1 | N | +| ask-mmt-submit | HTTP POST | Supabase, Resend | Premium gating, quota, delivery | 2 | 2 | N | +| backup-db | Scheduled daily | Supabase | Table iteration, retention math | 1 | 1 | N | +| billing-api | HTTP GET/POST | Supabase | 7-view routing, filters | 1 | 1 | N | +| billing-sync | Scheduled daily | Stripe, Anthropic, Netlify, Resend, Render, Gmail | Sequential collectors | 1 | 1 | N | +| check-tier | HTTP POST | Supabase, Buttondown | Enum-based tier detection | 1 | 1 | N | +| collect-feedback-background | HTTP POST | Supabase | Threshold pattern (3+ negatives/30d) | 1 | 1 | N | +| command-center-api | HTTP GET/POST | Supabase | Dashboard + 12 ops actions | 1 | 1 | N | +| competitive-scan | Scheduled weekly | Claude, web_search | Research + keyword matching for alert type | 2 | 1 | Y | +| compliance-check | HTTP POST | Supabase | 4 deterministic checks (CHPL, SCA, PubMed, docs) | 1 | 1 | N | +| contract-intel-refresh-background | HTTP POST | Claude Sonnet (7 passes), Supabase, federal APIs | Multi-pass pipeline with web_search | 5 | 2 | Y | +| contract-intel-refresh | Scheduled 8am ET | HTTP POST to background | Thin relay | 1 | 1 | N | +| contract-intel | HTTP GET | Supabase | NAME_ALIASES lookup | 1 | 1 | N | +| cost-api | HTTP GET/POST | Supabase | 5 GET + 2 POST views | 1 | 1 | N | +| cost-rollup | Scheduled daily | Supabase | Aggregation, 7d rolling avg, anomaly thresholds | 1 | 1 | N | +| create-checkout | HTTP POST | Stripe, Supabase | Email validation, tier check, session creation | 1 | 1 | N | +| create-premium-checkout | HTTP POST | Stripe, Supabase | Subscription plans, dedup | 1 | 1 | N | +| create-subscription-checkout | HTTP POST | Stripe, Supabase | 3-tier enum routing | 1 | 1 | N | +| create-tactical-brief-checkout | HTTP POST | Stripe, Supabase | One-time payment, metadata | 1 | 1 | N | +| creative-api | HTTP GET/POST | Supabase | Project/image/prompt CRUD | 1 | 1 | N | +| customer-api | HTTP GET/POST | Supabase | Health score formula, churn enum | 1 | 1 | N | +| customer-auth | HTTP POST | Supabase | Magic link token, session validation | 1 | 1 | N | +| customer-health-rollup | Scheduled daily | Supabase | Health score recalc | 1 | 1 | N | +| daily-stats-rollup | Scheduled daily | Supabase | Activity count aggregations | 1 | 1 | N | +| dashboard-auth | HTTP POST/GET | Supabase | bcrypt + magic link state machine | 1 | 1 | N | +| engagement-brief | HTTP POST | Claude (optional) | Hardcoded topics + optional email | 2 | 1 | Y | +| fact-check | HTTP POST | Claude Sonnet, web_search | Rate limit, JSON extraction | 2 | 2 | N | +| feedback-click | HTTP GET | Supabase | Rating 1-5 update + HTML | 1 | 1 | N | +| finance-api | HTTP GET/POST | Supabase | 7-view cost + inventory + alerts | 1 | 1 | N | +| finance-rollup | Scheduled daily | Supabase | Rolling avg, anomaly, deadlines, decay | 1 | 1 | N | +| founding-count | HTTP GET | Stripe | Active sub count + env fallback | 1 | 1 | N | +| generate-tactical-brief-background | HTTP POST | Perplexity, Claude (Haiku+Sonnet), federal APIs, Resend | 7-pass research + synthesis | 5 | 2 | Y | +| gold-team-review-background | HTTP POST | Claude (2 calls, 12k+6k tokens) | Sequential rewrite + review | 2 | 2 | N | +| google-oauth | HTTP POST | Google OAuth, Supabase | OAuth callback, token exchange | 1 | 1 | N | +| health-check | Scheduled 6h | Anthropic, Resend, Supabase, Stripe | Deterministic health suite | 1 | 1 | N | +| health | HTTP GET | Supabase, Stripe, Resend, Sentry | 7-service probe | 1 | 1 | N | +| issues-api | HTTP GET/POST | Supabase, Sentry | State machine (6 views + 5 actions) | 1 | 1 | N | +| learning-api | HTTP GET/POST | Supabase | Learnings CRUD (5 GET + 3 POST) | 1 | 1 | N | +| marketpulse-gateway | HTTP POST | Supabase, Stripe | Free-tier gate, checkout redirect | 1 | 1 | N | +| member-auth | HTTP POST | Supabase, Stripe | Email + tier + signed token | 1 | 1 | N | +| member-preferences | HTTP GET/POST | Supabase, Buttondown | Preference storage + tag sync | 1 | 1 | N | +| member-read-state | HTTP GET/POST | Supabase | Read entry CRUD | 1 | 1 | N | +| member-watchlist | HTTP GET/POST | Supabase | Watchlist CRUD | 1 | 1 | N | +| monthly-brief-send | Scheduled 1st/month 6am | Supabase, Resend | Fetch + extract + send | 1 | 1 | N | +| newsletter-research-background | Scheduled Mon/Thu 7am ET | Claude Sonnet, web_search | 5-category research + HTML + email | 2 | 2 | N | +| newsletter-research | Scheduled Mon/Thu 7am ET | HTTP POST to background | Thin trigger | 1 | 1 | N | +| newsletter-send | HTTP POST | Buttondown, Supabase | Digest build + dedup by subject | 1 | 1 | N | +| newsletter-sync | Scheduled Tue/Fri 6pm ET | Claude, web_search, GitHub, Netlify | Article sync + keyword tagging | 2 | 1 | Y | +| opportunity-feed | HTTP GET | Supabase | 6-filter query + sort | 1 | 1 | N | +| opportunity-radar-background | HTTP POST | Claude, web_search, SAM.gov | Three sequential searches + extraction | 2 | 2 | N | +| opportunity-radar | Scheduled 7am ET | HTTP POST to background | Thin trigger | 1 | 1 | N | +| ops-dashboard | HTTP GET | Supabase | Health + workflow + circuit breaker queries | 1 | 1 | N | +| ops-health-check | Scheduled every 15min | Supabase | Stuck job + critical event detection | 1 | 1 | N | +| ops-pattern-detector | Scheduled daily 1pm UTC | Supabase | Error signature grouping, escalation | 1 | 1 | N | +| predictive-signals-background | Scheduled Wed 6am ET | Supabase | Pattern match on KNOWN_FACTS + digest | 1 | 1 | N | +| premium-brief-send | Scheduled Fri 6am ET | Supabase, Resend | Fetch + extract + batch send | 1 | 1 | N | +| premium-chat | HTTP POST | Supabase, Claude, federal APIs | Premium gate + answer + quota | 2 | 2 | N | +| premium-digest-send | Scheduled daily 6:30am ET | Supabase, Resend | Preference-based digest | 1 | 1 | N | +| projects-api | HTTP GET/POST | Supabase | Project/sprint/task CRUD | 1 | 1 | N | +| protest-monitor-background | Scheduled daily 8am ET | Claude Sonnet, web_search | Case status monitoring | 2 | 2 | N | +| protest-monitor | Scheduled daily 8am ET | HTTP POST to background | Thin trigger | 1 | 1 | N | +| pursuit-score | HTTP POST | Claude, Supabase | Premium gate + cache + quota | 2 | 2 | N | +| qa-api | HTTP GET/POST | Supabase | Test results + regression + baseline | 1 | 1 | N | +| quality-drift-check-background | Scheduled Mon 8am ET | Supabase | Stuck order + freshness + alerts | 1 | 1 | N | +| rebuild-trigger | Scheduled every 4h | Netlify, Supabase | Rebuild hook call | 1 | 1 | N | +| release-held-emails | HTTP POST | Supabase, Resend | Batch release from degraded mode | 1 | 1 | N | +| resend-webhook | HTTP POST | Supabase | Webhook log + bounce suppression | 1 | 1 | N | +| review-action | HTTP GET | Supabase | Query-param action enum + update | 1 | 1 | N | +| review-queue-digest-background | Scheduled daily 7am ET | Supabase, Resend | Unreviewed digest with action links | 1 | 1 | N | +| roadmap-api | HTTP GET/POST | Supabase | Roadmap CRUD + log | 1 | 1 | N | +| roadmap-health-check | Scheduled daily 6am ET | Supabase | Feature health HTTP probes | 1 | 1 | N | +| sb-vehicle-radar-background | HTTP POST | Claude, USASpending.gov, Supabase | 5 queries + vehicle classification | 2 | 2 | N | +| sb-vehicle-radar | Scheduled 8am ET | HTTP POST to background | Thin trigger | 1 | 1 | N | +| score-cleanup | Scheduled every 10min | Supabase | Stuck score (5min) + retrigger | 1 | 1 | N | +| score-deck-background | HTTP POST | Claude, Supabase | 8k-token evaluation | 3 | 2 | Y | +| score-deck | HTTP POST | Supabase | Gateway: extract + store + usage | 1 | 1 | N | +| score-status | HTTP GET | Supabase | HMAC verify + status poll | 1 | 1 | N | +| sentry-sync | Scheduled every 30min | Sentry, Supabase | Issue sync + auto-issue + link | 1 | 1 | N | +| signal-chain-subscribe | HTTP POST/DELETE | Supabase | Premium gate + monitor upsert | 1 | 1 | N | +| signal-chain | HTTP GET/POST | Federal APIs, Congress, ClinicalTrials, PubMed, USAJobs, Supabase | 5-layer scoring | 2 | 2 | N | +| stripe-webhook | HTTP POST | Stripe, Supabase | Sig verify + idempotency + feature grant | 1 | 1 | N | +| submit-feedback | HTTP POST | Supabase | HMAC verify + 1-5 validation | 1 | 1 | N | +| subscriber-context | HTTP GET/POST/DELETE | Supabase | Admin CRUD (email, UEI, vehicles) | 1 | 1 | N | +| support-agent | HTTP POST | Claude Haiku, Resend | KB Q&A + low-confidence escalation | 2 | 2 | N | +| sync-learnings | HTTP POST | Supabase | Learnings export/import/diff | 1 | 1 | N | +| tactical-brief-cleanup | Scheduled every 15min | Supabase, Stripe | Stuck order (15min) + retrigger + refund | 1 | 1 | N | +| tactical-brief-webhook | HTTP POST (Stripe) | Stripe, Supabase, HTTPS | Sig verify + background trigger | 1 | 1 | N | +| view-report | HTTP GET | Supabase | HMAC verify + HTML retrieval | 1 | 1 | N | +| weekly-report | Scheduled Mon 9am ET | Supabase, Resend | Weekly usage digest | 1 | 1 | N | + +### Level distribution (Netlify functions, current state) + +| Level | Count | % | +|---|---|---| +| 1 | 70 | 77% | +| 2 | 18 | 20% | +| 3 | 2 | 2% | +| 5 | 2 | 2% | +| **Total** | **91** | 100% | + +**Functions flagged for downgrade: 7.** + +--- + +## Downgrade Notes (7 candidates) + +Each note describes the Level 1/2 version. + +**1. agent-bridge.js (3 → 1)** +Current: hand-rolled agent bridging with approval loops, status enum, dedup. +Level 1 version: HTTP relay with a database state machine. Orchestration is pure routing on known enums. Approval flow moves to `approval-api.js` which already handles this pattern deterministically. + +**2. competitive-scan.js (2 → 1)** +Current: weekly Claude + web_search scan with keyword matching on alert type. +Level 1 version: RSS feeds from competitor newsletters + deterministic keyword tagging (pricing, funding, product, contract, partnership). No reasoning step needed when the keyword list is fixed. + +**3. contract-intel-refresh-background.js (5 → 2)** +Current: 7-pass Claude Sonnet pipeline with web_search cross-validation. +Level 2 version: deterministic federal API enrichment (SAM.gov, USASpending.gov, Federal Register) plus one Claude synthesis call. Drop web_search from serverless (unreliable). Drop cross-validation loops. + +**4. engagement-brief.js (2 → 1)** +Current: hardcoded topic list with an optional Claude call. +Level 1 version: SQL aggregations over `signal_chain`, `opportunity_radar`, `contract_intel` tables into a Handlebars template. The LLM is not doing work the counts don't already expose. + +**5. generate-tactical-brief-background.js (5 → 2)** +Current: 7-pass Perplexity + Claude (Haiku + Sonnet) research pipeline. +Level 2 version: Pass 0 is deterministic federal API enrichment; Pass 1 is a single Claude Sonnet synthesis. Replace Perplexity research with direct API calls (USASpending.gov, Federal Register, SAM.gov Opportunities). + +**6. newsletter-sync.js (2 → 1)** +Current: Claude + web_search for article ingestion + tagging, then GitHub commit + Netlify rebuild. +Level 1 version: Substack RSS parser + deterministic keyword tagging. GitHub commit and rebuild trigger are already deterministic; only the ingestion step uses the LLM unnecessarily. + +**7. score-deck-background.js (3 → 2)** +Current: 8k-token Claude Sonnet evaluation across full proposal. +Level 2 version: rubric-based scoring with deterministic keyword presence and thresholds for routine checks, single Haiku call for the genuinely fuzzy sections (3 critical sections, not full deck). Drop the `_pending` intermediate state. + +--- + +## Provisional cost impact (for DET-004 ranking) + +Ranked by estimated annual token spend (qualitative until DET-401 pulls the Anthropic console): + +1. `generate-tactical-brief-background` (heaviest, Sonnet x multiple passes per brief) +2. `contract-intel-refresh-background` (7 Sonnet passes on schedule) +3. `score-deck-background` (8k-token Sonnet per scoring) +4. `newsletter-sync` (Claude + web_search twice weekly) +5. `competitive-scan` (Claude + web_search weekly) +6. `engagement-brief` (low volume, optional call) +7. `agent-bridge` (no LLM cost, architectural simplification only) + +--- + +## Pending sections + +- **n8n Workflows** — populated by DET-002 +- **OpenClaw Agents and Cron** — populated by DET-003 +- **Ecosystem Summary** — populated by DET-004 From 7de8d843b045e67f43c57534078658f2aab76026 Mon Sep 17 00:00:00 2001 From: Mary Womack Date: Sun, 19 Apr 2026 11:28:54 -0400 Subject: [PATCH 2/6] DET-002: n8n workflow determinism audit Classify all 3 exported n8n workflows against the 7-level framework. Fact-Check + Rewrite, Newsletter Draft Writer, and SAM.gov Scanner are all Level 3 (n8n hosting) with Level 2 targets. All three are downgrade candidates. SAM.gov Scanner is already retired per spec. n8n instance not running at audit time; classification based on exported workflow definitions at ~/Projects/mmt-site/n8n-workflows/. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/determinism-audit.md | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/docs/determinism-audit.md b/docs/determinism-audit.md index 44d54104..afae425e 100644 --- a/docs/determinism-audit.md +++ b/docs/determinism-audit.md @@ -181,8 +181,37 @@ Ranked by estimated annual token spend (qualitative until DET-401 pulls the Anth --- +## n8n Workflows (3 workflows) + +Source: `~/Projects/mmt-site/n8n-workflows/` (exported JSON). The n8n instance at `http://localhost:5678` is not running at audit time, so classification is based on the exported workflow definitions. All three exports remain under the `mmt-intelligence` tag. + +| workflow_name | trigger | apis | decision_points | current_level | target_level | downgrade_opportunity | status | +|---|---|---|---|---|---|---|---| +| MMT Fact-Check + Rewrite | Webhook POST (/fact-check-rewrite) | Netlify fact-check fn, Anthropic Sonnet, Resend | 1 validation + 1 LLM rewrite | 3 | 2 | Y | active | +| MMT Newsletter Draft Writer | Scheduled Mon/Thu 8am ET | Supabase, Anthropic Sonnet, Resend | 1 has-data check + 1 LLM draft | 3 | 2 | Y | active | +| MMT SAM.gov Opportunity Scanner | Scheduled daily 6:30am ET | Perplexity sonar-pro, Anthropic Haiku, Resend | 1 has-opps + 1 LLM discovery + 1 LLM scoring | 3 | 2 | Y | retired (spec notes replaced by Perplexity Computer manual prompts; export still on disk) | + +### Downgrade notes (all 3) + +**1. MMT Fact-Check + Rewrite (3 → 2)** +Current: n8n webhook calls Netlify `fact-check.js` (already Level 2), then calls Claude Sonnet for rewrite. Two hops, n8n orchestration in the middle. +Level 2 version: fold the rewrite step into a single Netlify function. `fact-check.js` does both the check and the conditional rewrite in one handler. Eliminates n8n entirely. This workflow already proxies to a Netlify function, so the n8n layer is pure overhead. + +**2. MMT Newsletter Draft Writer (3 → 2)** +Current: scheduled n8n pulls `newsletter_research` from Supabase, sends research JSON to Claude Sonnet, saves draft back, emails Mary. +Level 2 version: Python/Node script (scheduled via launchd or Netlify scheduled function) that does the same four steps. This is the exact target of DET-203. The n8n orchestration adds no value because the flow is linear with no branching beyond has-data. + +**3. MMT SAM.gov Opportunity Scanner (3 → 2)** +Current: deactivated. When active: Perplexity does discovery (search SAM.gov indirectly), Claude Haiku scores relevance, Resend delivers brief. +Level 2 version: hit `api.sam.gov/opportunities/v2/search` directly with NAICS + agency + set-aside filters (deterministic fetch — no LLM needed for retrieval). One Haiku call per surviving hit for relevance scoring against current pursuits. This is the exact target of DET-202. Discovery via Perplexity was only necessary because the Level 1 API approach had not been built. + +### Pattern observed + +All three n8n workflows follow the same shape: a scheduled or webhook trigger, a linear deterministic pipeline, one or two targeted LLM calls, an email delivery. None of them branch on LLM reasoning. None use n8n's agent node. They are Level 2 tasks running on a Level 3 platform. The n8n layer adds deploy friction, credential management overhead, and a separate runtime to monitor — all without adding capability. + +--- + ## Pending sections -- **n8n Workflows** — populated by DET-002 - **OpenClaw Agents and Cron** — populated by DET-003 - **Ecosystem Summary** — populated by DET-004 From 1a55c045d5426bb32d1d8c3253a7587d8e38bf0c Mon Sep 17 00:00:00 2001 From: Mary Womack Date: Sun, 19 Apr 2026 11:48:53 -0400 Subject: [PATCH 3/6] DET-003: OpenClaw and scheduler determinism audit Classify 6 OpenClaw agents, 5 launchd plists, and 2 crontab entries. All 6 agents run on Level 7 infrastructure when most tasks only need Level 1, 2, or 6. com.mmt.sentinel is the strongest launchd downgrade candidate (6 to 2). penny-pincher, cost-tracker-weekly, and the gateway itself are already right-sized. Two side findings flagged out of scope: plaintext API keys in the gateway plist and scheduled jobs targeting the diverged mmt-site clone at \$HOME/mmt-site instead of ~/Projects/mmt-site. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/determinism-audit.md | 58 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/docs/determinism-audit.md b/docs/determinism-audit.md index afae425e..d8d85c29 100644 --- a/docs/determinism-audit.md +++ b/docs/determinism-audit.md @@ -211,7 +211,63 @@ All three n8n workflows follow the same shape: a scheduled or webhook trigger, a --- +## OpenClaw Agents, LaunchAgents, and Cron + +### OpenClaw agents (6) + +Source: `openclaw agents list`. All agents route through the always-on gateway daemon (`ai.openclaw.gateway`, port 18789). By framework definition, agents hosted in an autonomous always-on runtime sit at Level 7 on the orchestration axis. The correct level question for each is whether the agent's actual task warrants that runtime. + +| agent_name | model | workspace | current_level | target_level_for_task | notes | +|---|---|---|---|---|---| +| editorial (default) | anthropic/claude-sonnet-4-20250514 | mmt-ops-exec/workspaces/editorial | 7 | 6 | Editorial drafting is open-ended and human-summoned. Claude Code with a project workspace would cover it. Downgrade candidate if not actively used autonomously. | +| ops-code | openrouter/meta-llama/llama-3.3-70b-instruct:free | mmt-ops-exec/workspaces/ops-code | 7 | 6 | Free-tier Llama; cheap to run. Bounded "ops-code" scope suggests scheduled tasks, not open-ended reasoning. Review usage before keeping at 7. | +| ops-research | openrouter/meta-llama/llama-3.3-70b-instruct:free | mmt-ops-exec/workspaces/ops-research | 7 | 2-6 | Research tasks are typically Level 2 (fetch + one LLM pass). If used interactively, Level 6. Either way, below 7. | +| ops-monitor | openrouter/meta-llama/llama-3.3-70b-instruct:free | mmt-ops-exec/workspaces/ops-monitor | 7 | 1-2 | Monitoring is deterministic checks with optional LLM summarization. Strong downgrade candidate. | +| ops-ciso | openrouter/meta-llama/llama-3.3-70b-instruct:free | mmt-ops-exec/workspaces/ops-ciso | 7 | 6 | Security review is open-ended human-summoned. Claude Code pattern fits. | +| ops-visual | anthropic/claude-sonnet-4-20250514 | mmt-ops-exec/workspaces/ops-visual | 7 | 6 | Creative visual work; human-summoned fits. | + +**Downgrade opportunities flagged: 6 out of 6 agents.** All six are hosted on Level 7 infrastructure when their actual tasks do not require always-on autonomy. Sprint 2 should validate which agents are actually invoked autonomously vs. on-demand, then migrate on-demand agents to Claude Code (Level 6) or scripts (Level 1-2). The OpenClaw gateway itself stays at Level 7 as platform infrastructure. + +### LaunchAgents (5 plists) + +Source: `~/Library/LaunchAgents/*.plist` + `launchctl list`. + +| label | trigger | command | current_level | target_level | downgrade_opportunity | +|---|---|---|---|---|---| +| ai.openclaw.gateway | Always-on (KeepAlive) | node openclaw gateway (port 18789) | 7 | 7 | N (platform infrastructure) | +| com.maryd.pennypincher | Always-on (KeepAlive) | python3 penny_pincher.py (Ollama local) | 1 | 1 | N (already right-sized: cost ledger + log tailer + daily markdown; no LLM spend) | +| com.mmt.cost-tracker-weekly | Mon 6am ET | node weeklySync.js | 1 | 1 | N (deterministic cost sync) | +| com.mmt.sentinel | Every 6h (21600s) | `claude -p` with max-turns 10, budget $0.25 | 6 | 2 | Y | +| com.mmt.seo-agent | Mon 8am ET | `claude -p` with max-turns 25, budget $0.75 | 6 | 4 or 6 | N (task warrants agent behavior; current pattern is bounded and correct) | + +### LaunchAgent downgrade notes + +**com.mmt.sentinel (6 → 2)** +Current: every 6 hours, invoke Claude Code with up to 10 turns and $0.25 budget to (1) run `mmt-health-check.sh`, (2) classify results L0-L3 per a decision tree, (3) apply pre-approved fixes. +Level 2 version: run `mmt-health-check.sh` directly in the shell script; pass the JSON result to a single Anthropic API call that returns the classification and fix list as JSON; apply pre-approved fixes with a deterministic switch on the fix type. The health check is already Level 1. The classification is a single fuzzy step, not a 10-turn agent. Budget drops from $0.25 per run to roughly one Haiku call. + +**com.mmt.seo-agent (6 stays)** +Current: Mon 8am ET, 25 turns, $0.75 budget. Reads knowledge base, audits 5+ pages, applies fixes across files, commits to git, updates learning log. +Keep at Level 6. The task spans multi-page reasoning, file edits, and self-updating strategy — genuine agent territory. A Level 4 hand-rolled loop would reimplement most of Claude Code's agent harness. The existing `--max-turns 25` and `--max-budget-usd 0.75` are the right bounds. + +### Crontab (2) + +Source: `crontab -l`. + +| command | trigger | current_level | target_level | downgrade_opportunity | +|---|---|---|---|---| +| missionpulse-frontend/scripts/evaluate-shadow-mode.ts | Daily 6am | unclassified (missionpulse) | TBD in Sprint 3 | deferred | +| missionpulse-frontend/scripts/weekly-shadow-report.ts | Mon 10am | unclassified (missionpulse) | TBD in Sprint 3 | deferred | + +Both entries are MissionPulse work and are covered by Sprint 3 (MissionPulse determinism-by-design). Not reclassifying them here to avoid scope creep. + +### Side findings (out of scope for this sprint, but flagged) + +1. **Gateway plist holds plaintext API keys in launchd `EnvironmentVariables`**: `ai.openclaw.gateway.plist` contains Anthropic, OpenAI, Google AI, Perplexity, and Telegram bot credentials as plist XML values. Anyone with read access to `~/Library/LaunchAgents/` sees them. Track as a separate security ticket (move to keychain or encrypted env file). +2. **Scheduled jobs reference the diverged clone**: `com.mmt.sentinel` and `com.mmt.seo-agent` both operate against `$HOME/mmt-site` (the diverged clone), not `~/Projects/mmt-site` (the Netlify-linked primary). This means scheduled automations are running against a different codebase than the one that ships to production. Track as part of the existing mmt-site clone remediation item. + +--- + ## Pending sections -- **OpenClaw Agents and Cron** — populated by DET-003 - **Ecosystem Summary** — populated by DET-004 From 737ed53cbf7ead4cf2e9fd0eb84ff5a0552bf0ce Mon Sep 17 00:00:00 2001 From: Mary Womack Date: Sun, 19 Apr 2026 11:51:40 -0400 Subject: [PATCH 4/6] DET-004: Determinism audit summary and downgrade priorities Ecosystem summary across 108 classified items (91 Netlify functions, 3 n8n workflows, 6 OpenClaw agents, 5 launchd jobs, 2 deferred crontab entries). Level distribution: 68% Level 1, 17% Level 2, the rest spread across Levels 3/5/6/7. Top 5 downgrade opportunities ranked by estimated cost-saving: generate-tactical-brief-background, contract-intel-refresh-background, score-deck-background, MMT Newsletter Draft Writer, com.mmt.sentinel. Seventeen total items flagged for downgrade. Recommended Sprint 2 ordering is starts-safe, ends-revenue-critical. Token cost numbers are qualitative until DET-401 pulls Anthropic console data. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/determinism-audit.md | 71 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 66 insertions(+), 5 deletions(-) diff --git a/docs/determinism-audit.md b/docs/determinism-audit.md index d8d85c29..f8f899e5 100644 --- a/docs/determinism-audit.md +++ b/docs/determinism-audit.md @@ -20,6 +20,72 @@ Multiple sequential LLM calls in a fixed pipeline are still Level 2. It is deter --- +## Ecosystem Summary + +Total items classified: **108** across Netlify functions (91), n8n workflows (3), OpenClaw agents (6), LaunchAgents (5), and crontab (2). Two crontab entries in missionpulse-frontend are deferred to Sprint 3. + +### Level distribution (current state) + +| Level | Count | % of classified | Annual Token Cost (est) | Downgrade Targets | +|---|---|---|---|---| +| 1 (script, no LLM) | 72 | 68% | $0 (no LLM) | 0 | +| 2 (script + one LLM call) | 18 | 17% | moderate (per-call Haiku/Sonnet) | 0 | +| 3 (no-code agent builder) | 5 | 5% | moderate to high (n8n + per-call) | 5 | +| 5 (code-first framework) | 2 | 2% | high (multi-pass Sonnet pipelines) | 2 | +| 6 (Claude Code on-demand) | 2 | 2% | bounded by `--max-budget-usd` per run | 1 | +| 7 (autonomous always-on) | 7 | 7% | mixed (1 platform + 6 agents on free Llama or Sonnet) | 6 | +| Unclassified (deferred) | 2 | 2% | TBD | deferred to Sprint 3 | +| **Total** | **106 classified** | | | **14 + the 6 OpenClaw agents under review** | + +Token cost columns are qualitative ranks; exact numbers come from DET-401 when we pull the Anthropic console and Perplexity balance. + +### Downgrade summary + +**17 items flagged for downgrade** across the ecosystem: +- 7 Netlify functions (DET-001) +- 3 n8n workflows (DET-002), all 3 being Level 3 items with Level 2 targets +- 6 OpenClaw agents (DET-003), pending validation of which are truly autonomous vs. on-demand +- 1 launchd job (DET-003), `com.mmt.sentinel` + +### Top 5 Downgrade Opportunities (ranked by estimated cost-saving) + +Estimates are qualitative until DET-401 pulls real Anthropic console data. + +1. **`generate-tactical-brief-background` (Netlify, Level 5 → 2)** + Heaviest LLM workload in the ecosystem. Seven-pass pipeline of Perplexity sonar-pro + Claude Haiku (disambiguation) + Claude Sonnet (synthesis) + Claude Haiku (cross-validation) per MarketPulse report. MarketPulse is a paid product with growing volume. Level 2 rewrite: deterministic federal API enrichment + single Claude Sonnet synthesis call. Largest single lever in the sprint. + +2. **`contract-intel-refresh-background` (Netlify, Level 5 → 2)** + Seven Sonnet passes on scheduled refresh with `web_search`. Per the 2026-04-15 learning, `web_search` is unreliable from serverless. Level 2 rewrite uses direct federal API calls and a single synthesis pass. + +3. **`score-deck-background` (Netlify, Level 3 → 2)** + Eight-thousand-token Sonnet evaluation per scoring run. Level 2 rewrite uses deterministic rubric scoring for routine checks and a single Haiku call for the fuzzy sections only. + +4. **MMT Newsletter Draft Writer (n8n, Level 3 → 2)** + Twice-weekly Sonnet draft call orchestrated by n8n. Level 2 rewrite is a Python/Node script that does fetch + Claude + save + email directly. Removes n8n as a runtime dependency. Also the exact target of DET-203 in Sprint 2. + +5. **`com.mmt.sentinel` (LaunchAgent, Level 6 → 2)** + Full Claude Code invocation every 6 hours with a $0.25 budget ceiling. Upper bound of ~$365/year per the budget cap. Level 2 rewrite is `mmt-health-check.sh` + one Haiku classification call. Health check is already deterministic; only the classify step is fuzzy. + +### Secondary opportunities (not top 5 but worth tracking) + +- `agent-bridge` (Netlify, 3 → 1) — architectural simplification, no LLM cost +- `engagement-brief` (Netlify, 2 → 1) — remove optional Claude call in favor of SQL aggregations +- `newsletter-sync` (Netlify, 2 → 1) — RSS parser + deterministic keyword tagging +- `competitive-scan` (Netlify, 2 → 1) — RSS + keyword matching on a fixed alert list +- `MMT Fact-Check + Rewrite` (n8n, 3 → 2) — fold into single Netlify function, eliminate n8n hop +- `MMT SAM.gov Scanner` (n8n, 3 → 2) — already deactivated; DET-202 target for Sprint 2 rebuild +- OpenClaw `ops-monitor` agent (7 → 1-2) — likely the strongest OpenClaw downgrade candidate pending usage validation + +### Recommended Sprint 2 ordering + +Based on production risk and payoff: + +1. Sprint 2 starts with DET-202 (SAM.gov at Level 2) because the n8n workflow is already deactivated. Zero production risk. +2. Then DET-203 (newsletter pipeline) because it has a clean two-week side-by-side trial mechanism and current spend is bounded. +3. Then DET-205 Netlify downgrades in order: `engagement-brief` and `agent-bridge` first (lowest risk, pure architectural), then `competitive-scan` and `newsletter-sync`, then `score-deck-background`, then `contract-intel-refresh-background`, saving `generate-tactical-brief-background` for last because it is the revenue-critical MarketPulse pipeline and warrants the most conservative cutover. + +--- + ## Netlify Functions — mmt-site (91 handlers) Source: `~/Projects/mmt-site/netlify/functions/`. Every handler read top-of-file through the core logic branch. Classification follows the rule: if the task is a fixed sequence of steps, it is Level 1. @@ -266,8 +332,3 @@ Both entries are MissionPulse work and are covered by Sprint 3 (MissionPulse det 1. **Gateway plist holds plaintext API keys in launchd `EnvironmentVariables`**: `ai.openclaw.gateway.plist` contains Anthropic, OpenAI, Google AI, Perplexity, and Telegram bot credentials as plist XML values. Anyone with read access to `~/Library/LaunchAgents/` sees them. Track as a separate security ticket (move to keychain or encrypted env file). 2. **Scheduled jobs reference the diverged clone**: `com.mmt.sentinel` and `com.mmt.seo-agent` both operate against `$HOME/mmt-site` (the diverged clone), not `~/Projects/mmt-site` (the Netlify-linked primary). This means scheduled automations are running against a different codebase than the one that ships to production. Track as part of the existing mmt-site clone remediation item. ---- - -## Pending sections - -- **Ecosystem Summary** — populated by DET-004 From 1d98026bbdb79ee645e611052453ccd52a24d20f Mon Sep 17 00:00:00 2001 From: Mary Womack Date: Sun, 19 Apr 2026 12:11:57 -0400 Subject: [PATCH 5/6] fix(premium): restore Compliance Check + Signal Chain in Pursuit Score sidebar Every other premium page's Tools section lists all three tools. Pursuit Score was the only one missing them, so members who landed there couldn't see the other two tools in the nav. Co-Authored-By: Claude Opus 4 (1M context) --- premium/pursuit-score.html | 2 ++ 1 file changed, 2 insertions(+) diff --git a/premium/pursuit-score.html b/premium/pursuit-score.html index 55a06c90..b1019689 100644 --- a/premium/pursuit-score.html +++ b/premium/pursuit-score.html @@ -63,6 +63,8 @@ Monthly Brief
Tools
Pursuit Score + Compliance Check + Signal Chain ProposalPulse ($14.99) MarketPulse ($35)
Resources
From 6a6bf9a8a85e5543ada653ac126018770e975139 Mon Sep 17 00:00:00 2001 From: Mary Womack Date: Sun, 19 Apr 2026 12:15:46 -0400 Subject: [PATCH 6/6] DET-103: mmt-site CLAUDE.md references determinism rule Add pointer to ~/.openclaw/automation-selection-rule.md at the top of the repo's CLAUDE.md so future Claude sessions default to the lowest- level automation that handles the task. Co-Authored-By: Claude Opus 4.7 (1M context) --- CLAUDE.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 181d81b3..0f510127 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,5 +1,8 @@ # Mission Meets Tech - Developer & Content Governance +**Automation defaults to the lowest determinism level that handles the task. +See ~/.openclaw/automation-selection-rule.md.** + ## 📜 Canonical Specification - All structural and UX work MUST follow `ARCHITECTURE_SPEC.md`. - This is the final word on site architecture and wireframes.