diff --git a/.claude/agents/planner-migration-auditor.md b/.claude/agents/planner-migration-auditor.md index c49b11d..3069fd9 100644 --- a/.claude/agents/planner-migration-auditor.md +++ b/.claude/agents/planner-migration-auditor.md @@ -5,20 +5,20 @@ tools: Read, Grep, Glob, Bash model: inherit --- -You are the **planner-migration auditor** for AEDIN. The crop planner is **NOT dead code** — it is being **migrated to the separate PolyCrop repo** (`/home/beef/projects/polycrop/`). aedin is academic + bot-facing; PolyCrop owns the consumer planner. Your job is to report the migration's true state across BOTH repos, dependency-aware, so the team neither removes something still depended on nor assumes something migrated that isn't. Read-only — never delete or edit. +You are the **planner-migration auditor** for AEDIN. The crop planner is **NOT dead code** — it is being **migrated to the separate PolyCrop repo** (checked out alongside this one). aedin is academic + bot-facing; PolyCrop owns the consumer planner. Your job is to report the migration's true state across BOTH repos, dependency-aware, so the team neither removes something still depended on nor assumes something migrated that isn't. Read-only — never delete or edit. ## Context (verify, don't trust — this drifts) As of 2026-06-25: aedin's `/api/planner/*` endpoints were ALREADY removed (`grep /api/planner backend/server.js` = 0; server.js comment "PolyCrop owns the planner now"), BUT PolyCrop had **no backend** and its `src/CropPlanner.tsx` still fetched aedin's `/api/planner/*` via `API_BASE` (default `http://localhost:3001`) — i.e. the planner was BROKEN mid-migration. Re-verify; do not rely on this paragraph. ## What to check — BOTH repos -### aedin (source: /home/beef/projects/aedin) +### aedin (source: this repository) 1. `grep -nE "/api/planner" backend/server.js` — endpoints present or removed? 2. Residual artifacts: `migrations/002_planner_schema.js`; tables `tritrophic_chains` / `beneficial_chains` (drop candidates IF empty) / `companion_scores` (KEEP — derived academic data product, not consumer-served). Query row counts read-only via `backend/lib/db-paths.cjs` CORPUS_DB (or the aedin-corpus MCP). 3. Any remaining `*planner*` scripts. -### PolyCrop (destination: /home/beef/projects/polycrop) -4. **Does PolyCrop have a backend at all?** (`ls polycrop/backend`; grep `src` for `express`/`app.listen`/`createServer` — IGNORE `dist/` build bundles, they're compiled frontend). No backend ⇒ it cannot serve the `/api/planner/*` its own frontend calls. +### PolyCrop (destination: the PolyCrop checkout) +4. **Does PolyCrop have a backend at all?** (`ls /backend`; grep `src` for `express`/`app.listen`/`createServer` — IGNORE `dist/` build bundles, they're compiled frontend). No backend ⇒ it cannot serve the `/api/planner/*` its own frontend calls. 5. **Does PolyCrop's frontend still call aedin's planner over HTTP?** grep `src` for `/api/planner` + `API_BASE`/`VITE_API_URL`. If yes AND aedin removed them AND PolyCrop has no backend ⇒ the planner is BROKEN. 6. What planner assets already live in PolyCrop (`CropPlanner.tsx`, `polyculture_planner_schema*.sql`). diff --git a/.gitignore b/.gitignore index 19c8ad6..12fa8d2 100644 --- a/.gitignore +++ b/.gitignore @@ -90,7 +90,7 @@ backend/globi-interactions.tsv.gz .wrangler/ # Re-ingest campaign workspace (regenerated by multi-critic-batch-prepare.js; -# verdicts produced by subscription-mode agents per GUAM-REINGEST-RESUME.md) +# verdicts produced by subscription-mode agents) backend/critic-batches/ backend/critic-verdicts/ backend/critic-batches-recovery/ diff --git a/backend/GUAM-REINGEST-RESUME.md b/backend/GUAM-REINGEST-RESUME.md deleted file mode 100644 index 1b6829c..0000000 --- a/backend/GUAM-REINGEST-RESUME.md +++ /dev/null @@ -1,75 +0,0 @@ -# Guam re-ingest campaign — resume hand-off - -**Last live activity:** 2026-05-30, mid wave-M dispatch. Session-limit cap approaching. - -## State on disk - -- **Batches:** `backend/critic-batches/batch-{000..100}.json` (101 MC batches, durable). -- **Verdicts that already landed:** `backend/critic-verdicts/batch-NNN.json` — varying counts across waves A through L/M, all valid top-level JSON arrays of 16 verdicts each (last batch may be smaller). Treat the directory as source of truth — re-running `multi-critic-batch-import.js` is idempotent (`INSERT OR IGNORE` on `(staging_id, critic_name)`). -- **Patched scripts (committed env-var override):** - - `backend/multi-critic-batch-prepare.js` — honors `BATCH_OUT_DIR` - - `backend/multi-critic-batch-import.js` — honors `BATCH_OUT_DIR` + `VERDICTS_DIR` - -## State in DB (`backend/globi.sqlite`) - -- **Sources 68 + 69** (Guam papers) — original `claims` already deleted by `reset-sources-for-reingest.js` (timestamped JSON backup under `backend/backups/`). -- **`extraction_staging`** rows from the fresh re-ingest are vouched: 802 plausible+uncertain. -- **`claim_critic_verdicts`** has 384 rows from waves 1-3 (192 staging rows × 2 critics) imported earlier. -- Waves A through (wherever the session cap caught us) still need importing — files are on disk under `backend/critic-verdicts/`. - -## Resume checklist - -```bash -cd /home/beef/projects/aedin/backend - -# 1. Import all verdict files into claim_critic_verdicts -BATCH_OUT_DIR=./critic-batches VERDICTS_DIR=./critic-verdicts \ - node multi-critic-batch-import.js - -# 2. Check how many un-verdicted staging rows remain -sqlite3 globi.sqlite " - SELECT COUNT(*) FROM extraction_staging s - WHERE s.source_id IN (68,69) - AND s.ai_vouch_status IN ('plausible','uncertain') - AND NOT EXISTS (SELECT 1 FROM claim_critic_verdicts v WHERE v.staging_id=s.id); -" - -# 3. If un-verdicted rows remain, re-prep + dispatch missing batches -# (the prepare script auto-excludes rows already verdicted via NOT EXISTS) -BATCH_OUT_DIR=./critic-batches node multi-critic-batch-prepare.js \ - --source-id=68 --batch-size=8 -# (and again with --source-id=69 if needed; rename + merge as during this session) - -# 4. Promote consensus-passing rows -node promote-staged-claims.js - -# 5. Validate species-resolution fix in promoted claims -sqlite3 globi.sqlite " - SELECT id, subject_name, object_name, source_quote - FROM claims - WHERE source_id IN (68,69) - AND (subject_name LIKE '%Ralstonia%' OR object_name LIKE '%Ralstonia%') - LIMIT 20; -" -``` - -## Key findings from this campaign - -1. **Species-resolution fix is validated.** Ralstonia normalization confirmed across batches 069, 077, 078 — both critics independently accepted multiple host pairings (soursop, ironwood, banana, pepper). The professor's flagged bug is closed. -2. **Multi-critic gate caught real extraction errors** (would have slipped past single-critic vouch): - - Plutella xylostella (Brassicaceae specialist) mis-attributed to garlic (batch 024) AND lettuce (batches 041, 043). Same systemic extraction artifact across three host crops. - - Phyllocnistis citrella (citrus leafminer, Gracillariidae) mis-classified as Agromyzidae (batches 038, 046, 058) — three independent flags. - - Flea beetles on citrus (batch 046) — both critics implausible (Chrysomelidae specialists target Solanaceae/Brassicaceae). -3. **Router mis-routes** are a known noise source — plant-pathologist returns OOS on arthropod claims, horticulturist returns OOS on pathogen claims. Affected rows simply don't promote (single-plausible doesn't meet ≥2 gate). Fix is in CLAUDE.md backlog. - -## Avoid - -- **Do NOT dispatch more agents until you've confirmed the rate-limit window has reset** (last cap was tied to Pacific/Port_Moresby 11:50). Session-limited dispatches return tokens=0 in tens of seconds — wasteful but not harmful. -- **Do NOT re-run `multi-critic-batch-prepare.js` against a dir that already holds the current batches** — it `unlinkSync`s `batch-*.json` first. The 101 batches were assembled by merging two source dirs; preserve them. -- **Do NOT touch `/tmp/claude/`** — the previous wipe lost ~16 wave-4+5 verdict files. All durable artifacts are now under `backend/`. - -## Outstanding work after wave M completes - -- Commit + push: `backend/render-extractor-prompt.js`, `backend/reset-sources-for-reingest.js`, env-var patches in `multi-critic-batch-prepare.js` and `multi-critic-batch-import.js`, and this resume note. -- Document the campaign outcome in `docs/phase-3-passlog.md` (Pass 12+: Guam re-ingest). -- Begin next regional batch (Pacific Pests after Guam) following the same playbook. diff --git a/backend/prompts/role-agent-interactive.md b/backend/prompts/role-agent-interactive.md index 207a4cc..0071df8 100755 --- a/backend/prompts/role-agent-interactive.md +++ b/backend/prompts/role-agent-interactive.md @@ -4,7 +4,7 @@ You are an ecological role-assignment agent for the AgroEco Explorer database. Y ## Context -**Database location:** `/home/beef/AgroEco/backend/globi.sqlite` +**Database location:** `backend/globi.sqlite` **Key tables:** - `entities` — organisms with `primary_role`, `bio_category`, `family`, `genus` diff --git a/backend/recovery-939-prompt.txt b/backend/recovery-939-prompt.txt deleted file mode 100644 index 480b595..0000000 --- a/backend/recovery-939-prompt.txt +++ /dev/null @@ -1,22 +0,0 @@ -You are an autonomous, UNATTENDED Claude Code session resuming a stalled ingestion recovery in /home/beef/projects/agroeco. No human is watching. Work carefully, log everything, commit + push at the end, and STOP with a written note if anything is genuinely ambiguous rather than guessing. - -## Background -A ~30-book agroecology corpus was extracted + vouched around 2026-06-05 but stalled at the multi-critic consensus stage. ~939 "near-miss" staging rows have exactly 1 plausible verdict (from the agroecologist synthesizer) + 0 implausible + >=1 out_of_scope verdict from a MIS-ROUTED specialist critic. They fail the promote gate (needs >=2 plausible, 0 implausible) only because lib/critic-router.js sent the second critic slot to the wrong specialist, which correctly returned out_of_scope. Confirmed: the OOS comes from specialists (horticulturist/soil-scientist/plant-pathologist/entomologist), almost never the agroecologist. Read docs/phase-3-passlog.md, the CLAUDE.md "Open router-tuning backlog" section, and backend/GUAM-REINGEST-RESUME.md for the pipeline pattern. This is subscription-only — never use the Anthropic API; all LLM work routes through the Agent tool (general-purpose subagents). - -## Recovery steps (do in order) -1. **Fix lib/critic-router.js routing** per the CLAUDE.md backlog: nematode-host claims -> plant-pathologist; above-ground arthropod herbivory -> entomologist; mycorrhizal/rhizobial mutualism -> soil-scientist; bare cereal/oilseed claims -> soil-scientist or agroecologist. Run the router's existing tests (look for lib/critic-router*.test.js or `node --test`) and make them pass. Commit this fix alone first. -2. **Identify the 939** in backend/globi.sqlite: review_status='pending' AND source_id NOT IN (233,234) AND ai_vouch_status IN ('plausible','uncertain') AND the row has exactly 1 plausible + 0 implausible + >=1 out_of_scope in claim_critic_verdicts. -3. **Reset the mis-routed verdicts** with backend/recovery-reset-verdicts.js (it backs up before deleting) — drop ONLY the out_of_scope specialist verdicts on those rows. The agroecologist's plausible verdict MUST remain. -4. **Re-prepare** multi-critic batches scoped to those rows with the corrected router (multi-critic-batch-prepare.js honors BATCH_OUT_DIR + --source-id; it auto-excludes already-verdicted critic slots via NOT EXISTS, so only the now-missing specialist slot re-runs). Batch-size 8. -5. **Dispatch critic subagents** (general-purpose Agents, sonnet) one per batch — each reads its batch JSON (critic_templates + claims), applies each claim's routed critic, writes [{staging_id, critic, verdict, note}] to a verdicts dir. Then multi-critic-batch-import.js (honors BATCH_OUT_DIR + VERDICTS_DIR). Forbid the subagents from running git/push. -6. **Promote**: node promote-staged-claims.js scoped via --source-id to the DISTINCT source_ids of the rows you recovered (compute them; explicitly EXCLUDE 233 and 234, which are already done). Dry-run first, then apply. - -## Safety + hygiene -- Run any bulk DB job under: systemd-run --user --scope -p MemoryMax=4G (better-sqlite3, no huge in-memory arrays). -- Use dedicated scratch dirs for batches/verdicts; clean them up at the end. Do NOT commit scratch dirs, the 43GB globi.sqlite, or backend/reingest/ prompt files. -- Every DB mutation already logs to revision_log via the existing scripts — keep it that way. -- Commit logical units (router fix; then the recovery result) and `git push origin main` after each. Pull/fetch first; other sessions commit to this same main branch — verify `git rev-parse HEAD origin/main` after pushing. -- When done, append a "Pass: book-corpus router-recovery (2026-06-11)" entry to docs/phase-3-passlog.md with: rows reset, re-critiqued, newly-consensus-ready, promoted (interaction vs trait), and any genuine-reject residue. Commit + push it. -- Do NOT publish to D1 (leave that for a coordinated build). Local DB + git only. - -Report a concise final summary to stdout: counts at each step and the commit SHAs. diff --git a/backend/scripts/backup-globi-to-nas.sh b/backend/scripts/backup-globi-to-nas.sh index ca12206..9b325b2 100755 --- a/backend/scripts/backup-globi-to-nas.sh +++ b/backend/scripts/backup-globi-to-nas.sh @@ -12,12 +12,12 @@ # # Schedule via cron: # crontab -e -# 0 3 * * * /home/beef/projects/agroeco/backend/scripts/backup-globi-to-nas.sh >> /home/beef/.local/share/agroeco-backup.log 2>&1 +# 0 3 * * * /path/to/aedin/backend/scripts/backup-globi-to-nas.sh >> "$HOME"/.local/share/aedin-backup.log 2>&1 set -euo pipefail # ─── Defaults (override in ~/.agroeco-backup.conf) ─────────────────────── -SOURCE_DB="/home/beef/projects/agroeco/backend/globi.sqlite" +SOURCE_DB="$(cd "$(dirname "$0")/.." && pwd)/globi.sqlite" NAS_HOST="REPLACE_WITH_NAS_HOSTNAME_OR_IP" # e.g. 192.168.1.100 or yournas.local NAS_USER="REPLACE_WITH_NAS_USERNAME" # your Synology DSM user NAS_PATH="/volume1/backups/agroeco" # path on the NAS diff --git a/backend/scripts/fetch-uog-ceo-pdfs.cjs b/backend/scripts/fetch-uog-ceo-pdfs.cjs index 9ae94dd..9b7f7d6 100644 --- a/backend/scripts/fetch-uog-ceo-pdfs.cjs +++ b/backend/scripts/fetch-uog-ceo-pdfs.cjs @@ -4,7 +4,7 @@ * publications via the publications-press.uog.edu Laravel API, then download * the ag/IPM-scope PDFs to literature/extension/ for the Pass-13+ ingest pipeline. * - * API discovery notes (see GUAM-REINGEST-RESUME.md for the Pass-12 playbook): + * API discovery notes: * - publications-press.uog.edu is a Nuxt SPA backed by a Laravel API. The * SPA viewer URL (/ceo/technicalreport//) is JS-only, but the * underlying API is plain JSON over POST. diff --git a/backend/smoke-provenance-e2e.sh b/backend/smoke-provenance-e2e.sh index 5391d8d..5dee7ab 100755 --- a/backend/smoke-provenance-e2e.sh +++ b/backend/smoke-provenance-e2e.sh @@ -3,7 +3,7 @@ set -euo pipefail cd "$(dirname "$0")" DB="${SMOKE_DB:-${TMPDIR:-/tmp}/globi-provenance-e2e.sqlite}" -PROD_DB="${PROD_DB:-/home/beef/projects/agroeco/backend/globi.sqlite}" +PROD_DB="${PROD_DB:-$(dirname "$0")/globi.sqlite}" if [ ! -f "$PROD_DB" ]; then echo "ERROR: production DB not found at $PROD_DB" diff --git a/backend/smoke-substrate-e2e.sh b/backend/smoke-substrate-e2e.sh index 63d7540..8d6a4e7 100755 --- a/backend/smoke-substrate-e2e.sh +++ b/backend/smoke-substrate-e2e.sh @@ -3,7 +3,7 @@ set -euo pipefail cd "$(dirname "$0")" DB="${SMOKE_DB:-/tmp/globi-substrate-e2e.sqlite}" -PROD_DB=/home/beef/projects/agroeco/backend/globi.sqlite +PROD_DB="${PROD_DB:-$(dirname "$0")/globi.sqlite}" if [ ! -f "$PROD_DB" ]; then echo "ERROR: production DB not found at $PROD_DB" diff --git a/backend/smoke-variety-e2e.sh b/backend/smoke-variety-e2e.sh index f115e6a..24bf116 100755 --- a/backend/smoke-variety-e2e.sh +++ b/backend/smoke-variety-e2e.sh @@ -3,7 +3,7 @@ set -euo pipefail cd "$(dirname "$0")" DB="${SMOKE_DB:-${TMPDIR:-/tmp}/globi-variety-e2e.sqlite}" -PROD_DB="${PROD_DB:-/home/beef/projects/agroeco/backend/globi.sqlite}" +PROD_DB="${PROD_DB:-$(dirname "$0")/globi.sqlite}" if [ ! -f "$PROD_DB" ]; then echo "ERROR: production DB not found at $PROD_DB"