diff --git a/CHANGELOG.md b/CHANGELOG.md index e84dd96d..4c089401 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,43 @@ # Changelog +## Unreleased — feat(memory): `check:` — a checkable fact says how to re-check itself, and `add` will not let it skip (DIVE-3885) + +Item 3 of the memory-janitor plan is a `check:` whose exit code re-derives a fact, plus a pass that +flips it stale. It looked buildable on the existing `--evidence=:`: `cmd:` *is* a check, +`file:` *is* a `test -f`. The fleet census on 2026-09-01 (2,651 atoms, 9 stores) says otherwise — +596 atoms carry an evidence block, and **595 of them are `run:`, stamped automatically +by the consolidate pipeline. Twelve hand-authored refs exist fleet-wide.** The same shape shows in +the lifecycle envelope: `confidence:` (fillable by the pipeline) has heavy adoption; `valid_to:` is +at **zero** and `supersedes:` at ~zero, because both need an author to know something at write time. + +**The rule that yields: a memory field with no write-time enforcement converges to whatever the +autostamp fills in, however well it is documented.** So checkability is a NEW authored field, and +`memory add` enforces it where the fact is written rather than merely offering it. + +- **`memory add --check=''`** — a read-only shell command whose EXIT CODE re-derives the fact. +- **`--no-check=""`** — the opt-out, and it is *recorded*: the reason lands in frontmatter as + `no_check:`, so an unchecked fact is countable rather than merely absent. A bare or stub reason is + refused. `memory consolidate` passes it explicitly, so the pipeline's inability to author a check + is visible in the store instead of silently becoming the norm. +- **`--type=reference` requires one of the two.** That is the class of fact that rots — a claim + about the world outside the store. Every other type, and the wiki store, is unchanged. +- **Two refusals keep the field from degenerating.** A check that cannot go red (`true`, `:`, a bare + `echo`) is refused — an enforcement satisfiable by `--check=true` is the `evidence:` degeneracy + rebuilt. A check that is not read-only (writes, `sudo`, `rm`, `curl | sh`, driving the board, a + redirect to a file) is refused — the pass runs these unattended. `>/dev/null 2>&1` stays legal. +- **`5dive memory check [--write]`** — runs every authored check, dry-run by default. Green is + `fresh`; red is `stale`; and **a checker that could not RUN (timeout, command not found) is + `unknown`, not stale** — an instrument failure and a false fact look identical from the outside, + and folding them together is how an automated janitor starts retiring true facts. +- **Nothing here deletes.** A red check means the fact is wrong OR the checker is, so `stale` is a + flag for a person to adjudicate. Recall demotes a stale atom 0.4x and prints `⚠ check red `; + it still surfaces. There is no `--delete`, and no LLM holds that key. + +`tests/memory_check_field_unit.sh` (54 assertions) is mostly negative controls, including a live +arm through the built binary — that arm is what caught the stale exit code being printed as +"this is a bug in the CLI" by the `lib/output.sh` backstop, which would have buried the digest a +nightly pass exists to produce. + ## Unreleased — fix(supervisor): a lapsed refusal is scrollback, not a wall — `agent info` reads the expiry it was already quoting (DIVE-3880) `agent info` printed **⚠ NOT TRANSACTING (quota-exhausted)** about a seat that was executing the very diff --git a/src/cmd_memory.sh b/src/cmd_memory.sh index 3b0bafea..f781dd97 100644 --- a/src/cmd_memory.sh +++ b/src/cmd_memory.sh @@ -66,7 +66,8 @@ _memory_usage() { [--type=user|feedback|project|reference] [--store=mine|wiki] [--tags=a,b] [--valid-to=YYYY-MM-DD] [--supersedes=] [--confidence=high|medium|low] [--provenance=""] - [--evidence=:]... [--no-dedup] [--force] (body on stdin) + [--evidence=:]... [--check='' | --no-check=""] + [--no-dedup] [--force] (body on stdin) Compile a durable memory: writes a frontmatter markdown file into your own store (default) or the shared team wiki (--store=wiki, the publish path other agents can search), stamps provenance (who/when), appends the @@ -84,6 +85,23 @@ _memory_usage() { sha: url: run: It sits BESIDE --provenance (free text), which is unchanged. A memory with no evidence is not flagged, demoted, or degraded. + CHECKABILITY (DIVE-3885) is a SEPARATE, AUTHORED field — deliberately not + built on --evidence. Measured 2026-09-01 over 2,651 fleet atoms: 596 carry + evidence, but 595 of those are `run:`, stamped by the + consolidate pipeline. TWELVE hand-authored refs exist fleet-wide. A field + the pipeline can fill converges to whatever the pipeline fills in, so + `check:` is one the pipeline CANNOT fill and `add` will not let it skip + silently: + --check='' a read-only shell command whose EXIT CODE re-derives + this fact. 0 = still true. It is run unattended by + `memory check`, so it is refused if it writes, escapes + (sudo/rm/curl|sh), or is trivially green (`true`, `:`, + a bare echo) — a check that cannot go red is not one. + --no-check="" the recorded opt-out. Not free: the reason is written + to frontmatter as `no_check:`, so an unchecked fact is + COUNTABLE rather than merely absent. + WRITE-TIME ENFORCEMENT: --type=reference (a fact about the world, the + class that rots) requires one of the two. Every other type is unchanged. Write-time dedup: `add` WARNS (never refuses) when the body overlaps an existing memory in the same store — silence it with --no-dedup. @@ -127,6 +145,22 @@ _memory_usage() { ledger short-circuits before the distiller is ever invoked). Lower it further with --max-chars, or point --distiller at a cheaper model. + 5dive memory check [--roots=a,b] [--store=all|mine|wiki] [--agent=] + [--slug=]... [--timeout=SEC] [--write] [--json] + Run every atom's authored `check:` command and report which facts no + longer re-derive. Dry-run by default; --write stamps the frontmatter with + `check_status`, `checked_at` and `check_rc`. + exit 0 every check that RAN came back green + exit 1 at least one fact went `stale` (its check ran and went red) + A red check means THE FACT IS WRONG **OR THE CHECKER IS** — so `stale` is + a flag for a human/agent to adjudicate, never a verdict. Recall demotes a + stale fact and says so; it is never hidden, and NOTHING here deletes a + memory. A checker that could not RUN (timeout, command not found) is + `unknown`, NOT stale — an instrument failure and a false fact are the same + shape from the outside, and folding them together is how a janitor starts + deleting true facts. Checks run read-only, with a timeout (default 20s), + one at a time, under your own uid. + 5dive memory doctor [--roots=a,b] [--agent=] [--code-root=] [--json] Hygiene pass over the memory store(s): index drift (MEMORY.md vs files on disk), dangling [[wiki-links]], stale source refs (file:line no longer in @@ -272,7 +306,7 @@ function chunk(file) { const fm = /^---\n([\s\S]*?)\n---\n?/.exec(text); let front = ""; // DIVE-1024 lifecycle envelope, parsed once per file and carried on each chunk. - let meta = { name: path.basename(file).replace(/\.md$/, ""), validTo: "", confidence: "", supersedes: "" }; + let meta = { name: path.basename(file).replace(/\.md$/, ""), validTo: "", confidence: "", supersedes: "", checkStatus: "", checkedAt: "" }; if (fm) { const desc = /^description:\s*["']?(.+?)["']?\s*$/m.exec(fm[1]); if (desc) front = desc[1].replace(/^>\s*/, "").trim(); meta = { @@ -280,6 +314,8 @@ function chunk(file) { validTo: fmField(fm[1], "valid_to"), confidence: fmField(fm[1], "confidence").toLowerCase(), supersedes: fmField(fm[1], "supersedes"), + checkStatus: fmField(fm[1], "check_status").toLowerCase(), + checkedAt: fmField(fm[1], "checked_at"), }; text = text.slice(fm[0].length); } @@ -307,6 +343,12 @@ function lifecycle(m) { if (m.validTo && m.validTo < TODAY) { mult *= 0.3; flags.push(`⚠ expired ${m.validTo}`); } if (superseded.has(m.name)) { mult *= 0.2; flags.push("⤴ superseded"); } const cm = CONF_MULT[m.confidence] ?? 1; if (cm < 1) { mult *= cm; flags.push(`confidence:${m.confidence}`); } + // DIVE-3885: a fact whose authored check went RED. Demoted and flagged, never + // hidden and never deleted — the check may be the thing that broke, not the + // fact. `unknown` (the checker could not run) is flagged at full rank: an + // instrument failure is not evidence about the claim. + if (m.checkStatus === "stale") { mult *= 0.4; flags.push(`⚠ check red${m.checkedAt ? ` ${m.checkedAt}` : ""}`); } + else if (m.checkStatus === "unknown") { flags.push("? check did not run"); } return { mult, flags }; } if (!query) { console.error('usage: memory search ""'); process.exit(2); } @@ -476,9 +518,49 @@ DEDUPPY return 0 } +# _memory_check_validate — DIVE-3885. A `check:` is executed UNATTENDED by +# `memory check`, and its whole value is that it can go RED. Two refusals: +# +# 1. NOT READ-ONLY. The nightly pass runs these with the store owner's uid. A +# check that writes, deletes, escalates or pipes the network into a shell is +# a scheduled command with a memory file for a trigger. +# 2. TRIVIALLY GREEN. `--check=true` satisfies the enforcement and re-derives +# nothing — the exact degeneracy that made `evidence:` 99% autostamp. An +# enforcement with a free-and-silent way to satisfy it enforces nothing. +_memory_check_validate() { + local c="$1" + [ -n "$(printf '%s' "$c" | tr -d '[:space:]')" ] \ + || fail "$E_VALIDATION" "--check is empty — it must be a command whose exit code re-derives the fact" + # 1. read-only + if printf '%s' "$c" | grep -qE '(^|[;&|[:space:]])(sudo|rm|rmdir|mv|dd|mkfs|shutdown|reboot|kill|pkill|truncate|chmod|chown|tee)([[:space:]]|$)'; then + fail "$E_VALIDATION" "--check must be READ-ONLY — it is run unattended by \`memory check\` (refused: writes/escalates)" + fi + # Redirection: silencing is fine, writing a file is not. Strip the shapes that + # produce no file (>/dev/null, >/dev/stderr, 2>&1) and refuse whatever `>` is + # left — that one has a path on the end of it. + local _redir; _redir="$(printf '%s' "$c" | sed -E 's/[0-9]?>>?[[:space:]]*\/dev\/(null|stdout|stderr)//g; s/[0-9]?>>?&[0-9-]//g')" + if printf '%s' "$_redir" | grep -q '>'; then + fail "$E_VALIDATION" "--check must be READ-ONLY — redirecting to a file is a write (>/dev/null 2>&1 is allowed if you only meant to silence it)" + fi + if printf '%s' "$c" | grep -qE '(curl|wget)[^|]*\|[[:space:]]*(ba)?sh'; then + fail "$E_VALIDATION" "--check must be READ-ONLY — piping a download into a shell is not a check" + fi + if printf '%s' "$c" | grep -qE '5dive[[:space:]]+(task|agent|memory)[[:space:]]+(done|add|cancel|start|send|kill|rm|reject|deliver)'; then + fail "$E_VALIDATION" "--check must be READ-ONLY — it must not drive the board or the fleet" + fi + # 2. cannot go red + local t; t="$(printf '%s' "$c" | sed 's/^[[:space:]]*//; s/[[:space:]]*$//')" + case "$t" in + true|:|"exit 0"|true*\;*|echo*|printf*|cat*|"pwd"|"date"|"whoami"|"hostname") + fail "$E_VALIDATION" "--check '$t' can never go red — a check that cannot fail is not a check (it is the degeneracy that made \`evidence:\` 99% autostamp)" ;; + esac + return 0 +} + _memory_add() { local name="" type="" desc="" store="mine" tags="" force=0 no_dedup=0 local valid_to="" supersedes="" confidence="" provenance="" + local check="" no_check="" check_set=0 no_check_set=0 local evidence=() while [ $# -gt 0 ]; do case "$1" in @@ -492,6 +574,8 @@ _memory_add() { --confidence=*) confidence="${1#*=}" ;; --provenance=*) provenance="${1#*=}" ;; --evidence=*) evidence+=("${1#*=}") ;; + --check=*) check="${1#*=}"; check_set=1 ;; + --no-check=*) no_check="${1#*=}"; no_check_set=1 ;; --no-dedup) no_dedup=1 ;; --force) force=1 ;; -h|--help) _memory_usage; return 0 ;; @@ -532,6 +616,31 @@ _memory_add() { || fail "$E_VALIDATION" "--evidence url: wants an http(s) URL, got: ${_ev#url:}" ;; esac done + # DIVE-3885 checkability. A NEW authored field, deliberately not layered on + # --evidence: 595 of the 596 evidence blocks fleet-wide are the pipeline's + # `run:` autostamp, so anything built on that substrate starts from 12 rows. + # The lesson the census yields — a memory field with no WRITE-TIME + # enforcement converges to whatever the autostamp fills in, however well it is + # documented — is why this one refuses instead of merely offering. + if [ "$check_set" -eq 1 ] && [ "$no_check_set" -eq 1 ]; then + fail "$E_USAGE" "--check and --no-check are mutually exclusive — a fact either has a way to re-derive itself or a recorded reason it does not" + fi + [ "$check_set" -eq 1 ] && _memory_check_validate "$check" + if [ "$no_check_set" -eq 1 ]; then + # The opt-out is RECORDED, not free. A bare --no-check would be a second + # budget nobody watches; a reason in frontmatter makes the unchecked fact + # countable, which is the only reason the enforcement survives contact. + [ "${#no_check}" -ge 8 ] \ + || fail "$E_VALIDATION" "--no-check needs a real reason (>= 8 chars) — it is written to frontmatter and counted" + fi + # ENFORCEMENT, scoped to the class of fact that rots: `reference` is a claim + # about the world outside this store (a path, a flag, a service, a limit), and + # it is the class a stale-check pass exists for. user/feedback/project facts + # and wiki pages are unchanged — widening this would just farm --no-check. + if [ "$store" = "mine" ] && [ "$type" = "reference" ] && [ "$check_set" -eq 0 ] && [ "$no_check_set" -eq 0 ]; then + fail "$E_USAGE" "a --type=reference fact needs --check='', or --no-check=\"\" to record the gap (DIVE-3885)" + fi + [ -t 0 ] && fail "$E_USAGE" "memory add reads the body on stdin — pipe or heredoc it" local body; body=$(cat) [ -n "$(printf '%s' "$body" | tr -d '[:space:]')" ] || fail "$E_USAGE" "empty body on stdin — nothing to remember" @@ -593,6 +702,8 @@ _memory_add() { [ -n "$confidence" ] && printf 'confidence: %s\n' "$confidence" [ -n "$valid_to" ] && printf 'valid_to: %s\n' "$valid_to" [ -n "$supersedes" ] && printf 'supersedes: %s\n' "$supersedes" + [ -n "$check" ] && printf 'check: "%s"\n' "$(printf '%s' "$check" | sed 's/\\/\\\\/g; s/"/\\"/g')" + [ -n "$no_check" ] && printf 'no_check: "%s"\n' "$(printf '%s' "$no_check" | sed 's/\\/\\\\/g; s/"/\\"/g')" [ -n "$provenance" ] && printf 'provenance: "%s"\n' "$(printf '%s' "$provenance" | sed 's/"/\\"/g')" _memory_emit_evidence "" ${evidence+"${evidence[@]}"} printf 'updated: %s\ncompiled_by: %s\n---\n\n%s\n' "$today" "$who" "$body" @@ -604,6 +715,8 @@ _memory_add() { [ -n "$confidence" ] && printf ' confidence: %s\n' "$confidence" [ -n "$valid_to" ] && printf ' valid_to: %s\n' "$valid_to" [ -n "$supersedes" ] && printf ' supersedes: %s\n' "$supersedes" + [ -n "$check" ] && printf ' check: "%s"\n' "$(printf '%s' "$check" | sed 's/\\/\\\\/g; s/"/\\"/g')" + [ -n "$no_check" ] && printf ' no_check: "%s"\n' "$(printf '%s' "$no_check" | sed 's/\\/\\\\/g; s/"/\\"/g')" [ -n "$provenance" ] && printf ' provenance: "%s"\n' "$(printf '%s' "$provenance" | sed 's/"/\\"/g')" _memory_emit_evidence " " ${evidence+"${evidence[@]}"} printf -- '---\n\n%s\n' "$body" @@ -625,6 +738,177 @@ _memory_add() { return 0 } +# ---- memory check (DIVE-3885) ---------------------------------------------- +# +# Item 3 of the DIVE-3882 janitor plan: a `check:` whose EXIT CODE re-derives +# the fact, plus a pass that flips `check_status: stale`. +# +# THE ROW'S GUARDRAIL, kept mechanical: a red check means the fact is wrong OR +# the checker is. So this pass flips a FLAG and prints a digest. It never edits +# a body, never deletes a file, and there is no --delete to add later. Recall +# demotes a stale fact and says why; it still surfaces. +# +# And the second half of that guardrail, which is easier to get wrong: a checker +# that could not RUN is `unknown`, not `stale`. A missing binary, a box without +# the tool, a 20s timeout — those are instrument failures, and folding an +# instrument failure into "the fact is false" is exactly how an automated +# janitor starts retiring true facts. +_memory_check() { + local roots="" store="all" agent="" timeout_s=20 write=0 + local slugs=() + while [ $# -gt 0 ]; do + case "$1" in + --roots=*) roots="${1#*=}" ;; + --store=*) store="${1#*=}" ;; + --agent=*) agent="${1#*=}" ;; + --slug=*) slugs+=("${1#*=}") ;; + --timeout=*) timeout_s="${1#*=}" ;; + --write) write=1 ;; + --json) JSON_MODE=1 ;; + -h|--help) _memory_usage; return 0 ;; + *) fail "$E_USAGE" "memory check: unknown argument: $1" ;; + esac + shift + done + case "$store" in all|mine|wiki) : ;; *) fail "$E_VALIDATION" "bad --store '$store' (all | mine | wiki)" ;; esac + printf '%s' "$timeout_s" | grep -qE '^[0-9]+$' || fail "$E_VALIDATION" "--timeout must be whole seconds" + [ "$timeout_s" -gt 0 ] || fail "$E_VALIDATION" "--timeout must be > 0" + command -v python3 >/dev/null 2>&1 || fail "$E_GENERIC" "memory check needs python3" + local resolved; resolved=$(_memory_resolve_roots "$store" "$agent" "$roots") + [ -n "$resolved" ] || fail "$E_NOT_FOUND" "no memory roots to check (--roots/--store/--agent narrowed everything away)" + + local out rc=0 + out=$(FIVEDIVE_MC_TIMEOUT="$timeout_s" FIVEDIVE_MC_WRITE="$write" \ + FIVEDIVE_MC_SLUGS="$(printf '%s\n' ${slugs+"${slugs[@]}"})" \ + python3 - "$resolved" <<'MCPY' +import json, os, re, subprocess, sys, datetime + +roots = [r for r in sys.argv[1].split(",") if r] +timeout = int(os.environ.get("FIVEDIVE_MC_TIMEOUT", "20")) +write = os.environ.get("FIVEDIVE_MC_WRITE") == "1" +want = {s.strip().replace("_", "-") for s in os.environ.get("FIVEDIVE_MC_SLUGS", "").split("\n") if s.strip()} +today = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d") + +FM = re.compile(r"\A---\n(.*?)\n---\n", re.S) + +def fm_field(front, key): + # Both layouts: top-level (wiki) and nested two-space under `metadata:` (own + # store). A quoted value may carry escaped quotes — a check IS shell text. + m = re.search(r'^[ \t]*%s:[ \t]*(.*?)[ \t]*$' % re.escape(key), front, re.M) + if not m: + return "" + v = m.group(1) + if len(v) >= 2 and v[0] == '"' and v[-1] == '"': + v = v[1:-1].replace('\\"', '"').replace("\\\\", "\\") + return v + +def files(): + for root in roots: + if not os.path.isdir(root): + continue + for dirpath, dirnames, names in os.walk(root): + dirnames[:] = [d for d in dirnames if not d.startswith(".")] + for n in sorted(names): + if n.endswith(".md") and n not in ("MEMORY.md", "index.md"): + yield os.path.join(dirpath, n) + +results = [] +for path in files(): + try: + text = open(path, encoding="utf-8", errors="replace").read() + except OSError: + continue + m = FM.match(text) + if not m: + continue + front = m.group(1) + cmd = fm_field(front, "check") + if not cmd: + continue + slug = fm_field(front, "name") or fm_field(front, "title") or os.path.basename(path)[:-3] + if want and slug.replace("_", "-") not in want: + continue + # Read-only by construction at write time; still run with no stdin, in the + # store's dir, so a relative path in a check means something stable. + status, rc, detail = "unknown", None, "" + try: + proc = subprocess.run(["bash", "-c", cmd], stdin=subprocess.DEVNULL, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, + timeout=timeout, cwd=os.path.dirname(path) or ".") + rc = proc.returncode + tail = proc.stdout.decode("utf-8", "replace").strip().splitlines() + detail = tail[-1][:200] if tail else "" + if rc == 0: + status = "fresh" + elif rc == 127: + # The checker itself is missing. That is an instrument failure and + # says nothing about the fact. + status, detail = "unknown", detail or "command not found" + else: + status = "stale" + except subprocess.TimeoutExpired: + status, detail = "unknown", "timed out after %ds" % timeout + except OSError as e: + status, detail = "unknown", str(e) + + if write: + nested = re.search(r'^\s+check:', front, re.M) is not None + ind = " " if nested else "" + new_front = front + for key, val in (("check_status", status), ("checked_at", today), + ("check_rc", "" if rc is None else str(rc))): + line = "%s%s: %s" % (ind, key, val) + pat = re.compile(r'^[ \t]*%s:.*$' % re.escape(key), re.M) + if pat.search(new_front): + new_front = pat.sub(lambda _m, l=line: l, new_front, count=1) + else: + # Immediately after the `check:` line it belongs to, so the + # stamp travels with the field even in the nested layout. + ck = re.search(r'^[ \t]*check:.*$', new_front, re.M) + at = ck.end() + new_front = new_front[:at] + "\n" + line + new_front[at:] + try: + open(path, "w", encoding="utf-8").write("---\n" + new_front + "\n---\n" + text[m.end():]) + except OSError as e: + detail = (detail + " | ").strip() + "could not stamp: %s" % e + results.append({"slug": slug, "file": path, "status": status, + "rc": rc, "cmd": cmd, "detail": detail}) + +print(json.dumps(results)) +MCPY +) || fail "$E_GENERIC" "memory check: the pass failed to run" + + local n_total n_fresh n_stale n_unknown + n_total=$(printf '%s' "$out" | jq 'length') + n_fresh=$(printf '%s' "$out" | jq '[.[]|select(.status=="fresh")]|length') + n_stale=$(printf '%s' "$out" | jq '[.[]|select(.status=="stale")]|length') + n_unknown=$(printf '%s' "$out" | jq '[.[]|select(.status=="unknown")]|length') + + if (( JSON_MODE )); then + printf '%s' "$out" | jq -c --argjson w "$write" \ + '{ok:true, data:{written:($w==1), total:length, fresh:[.[]|select(.status=="fresh")]|length, + stale:[.[]|select(.status=="stale")]|length, unknown:[.[]|select(.status=="unknown")]|length, + results:.}}' + else + if [ "$n_total" -eq 0 ]; then + echo "memory check: no atom in these roots carries a check: — nothing to re-derive" + echo " authored checks are written at compile time: 5dive memory add --check=''" + return 0 + fi + printf '%s' "$out" | jq -r '.[] | (if .status=="fresh" then "✓ fresh " elif .status=="stale" then "✗ STALE " else "? unknown " end) + .slug + (if .detail=="" then "" else " — " + .detail end)' + echo + echo "memory check: $n_total checked · $n_fresh fresh · $n_stale stale · $n_unknown unknown$([ "$write" -eq 1 ] && echo " (stamped)" || echo " (dry-run — --write to stamp)")" + [ "$n_stale" -gt 0 ] && echo " a STALE fact may be wrong, or its CHECK may be. Adjudicate it — nothing here deletes a memory." + [ "$n_unknown" -gt 0 ] && echo " UNKNOWN = the checker could not run. That is an instrument failure, not evidence about the fact." + fi + # A stale fact is a RESULT, not a crash. Without mark_reported the EXIT + # backstop in lib/output.sh prints "exited 1 without reporting a reason … this + # is a bug in the CLI" over a digest that already said exactly what happened — + # and a nightly cron would read that banner instead of the finding. + if [ "$n_stale" -gt 0 ]; then rc=1; mark_reported; fi + return "$rc" +} + # ---- memory hygiene (DIVE-991) --------------------------------------------- # # A runnable hygiene pass over one or more memory stores (per-agent stores + @@ -1290,7 +1574,8 @@ _memory_consolidate() { --name="$a_name" --type="$a_type" --description="$a_desc" \ --confidence="$a_conf" \ --provenance="distilled from session $sid ($(date -u +%F))" \ - --evidence="run:$sid" ) 2>&1) || addrc=$? + --evidence="run:$sid" \ + --no-check="auto-distilled by memory consolidate — no author present to write a check (DIVE-3885)" ) 2>&1) || addrc=$? if [ "$addrc" -eq 0 ]; then n_written=$((n_written+1)) written_files+=("$dir/${a_type}_$(printf '%s' "$a_name" | tr '-' '_').md") @@ -1656,6 +1941,7 @@ cmd_memory() { get|fetch) _memory_get "$@" ;; router) _memory_router "$@" ;; add|compile) _memory_add "$@" ;; + check) _memory_check "$@" ;; doctor|hygiene) _memory_doctor "$@" ;; consolidate|distill) _memory_consolidate "$@" ;; ""|-h|--help) _memory_usage ;; diff --git a/tests/memory_check_field_unit.sh b/tests/memory_check_field_unit.sh new file mode 100644 index 00000000..224a6320 --- /dev/null +++ b/tests/memory_check_field_unit.sh @@ -0,0 +1,191 @@ +#!/usr/bin/env bash +# DIVE-3885 unit harness for MEMORY CHECKABILITY: a NEW authored `check:` field +# with WRITE-TIME ENFORCEMENT, plus the `memory check` pass that flips +# check_status. +# +# The row exists because item 3 of the DIVE-3882 janitor plan looked buildable +# on `--evidence`. Fleet census 2026-09-01: 596 of 2,651 atoms carry evidence, +# 595 of them the pipeline's `run:` autostamp — TWELVE hand-authored +# refs fleet-wide. The lesson: a memory field with no write-time enforcement +# converges to whatever the autostamp fills in. So the assertions that matter +# here are the NEGATIVE ones: +# +# 1. a --type=reference add with NO check REFUSES and writes nothing; +# 2. the escape hatch is RECORDED (a reason lands in frontmatter) and is not +# free (a stub reason is refused); +# 3. a check that CANNOT GO RED is refused (else enforcement is satisfied by +# --check=true and we have rebuilt the evidence: degeneracy); +# 4. a check that is not READ-ONLY is refused (the pass runs these unattended); +# 5. a checker that could not RUN is `unknown`, NOT `stale`; +# 6. NOTHING in the pass deletes or empties a memory file. +# +# Run: bash tests/memory_check_field_unit.sh +set -uo pipefail + +. "$(dirname "${BASH_SOURCE[0]}")/lib/grading_tree.sh" \ + || printf 'grading tree: UNRESOLVED (tests/lib/grading_tree.sh not reachable; no tree named)\n' >&2 +trap 'rc=$?; rm -rf "${TMP:-}"; echo "HARNESS-RC=$rc"' EXIT +cd "$(dirname "$0")/.." +SRC=src + +TMP="$(mktemp -d /tmp/mem-check-unit.XXXXXX)" + +# shellcheck disable=SC1090 +for f in header.sh lib/error_codes.sh lib/output.sh lib/validation.sh; do + source "$SRC/$f" +done +# shellcheck source=/dev/null +source "$SRC/cmd_memory.sh" +JSON_MODE=0 +set +e + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; } +check(){ if [ "$2" = "$3" ]; then ok "$1"; else bad "$1 (want '$3', got '$2')"; fi; } + +STORE="$TMP/home/.claude/projects/proj/memory" +mkdir -p "$STORE"; : > "$STORE/MEMORY.md" +export HOME="$TMP/home" +# _memory_wiki_root checks "$HOME"/projects/5dive/community/wiki FIRST and then +# falls back to a HARDCODED /home/claude/... — so a fake HOME alone does NOT +# isolate --store=wiki. Without this dir the wiki arm below writes a fixture +# into the real shared wiki (and appends an index line to it). Caught the hard +# way; do not delete this mkdir. +mkdir -p "$HOME/projects/5dive/community/wiki" + +add() { ( _memory_add "$@" ) ; } +mcheck(){ ( _memory_check --roots="$STORE" "$@" ) ; } + +BODY='The api vhost caps a proxied request at thirty seconds, so a call that +needs longer has to be moved onto the exec route rather than retried, and the +nginx config is the thing that decides it.' + +echo "── 1. write-time ENFORCEMENT on --type=reference ──" +printf '%s\n' "$BODY" | add --name=enf-none --type=reference --description=d >/dev/null 2>&1 +check "reference with neither --check nor --no-check REFUSES" "$?" "$E_USAGE" +[ -f "$STORE/reference_enf_none.md" ] && bad "refusal wrote no file" || ok "refusal wrote no file" +[ -s "$STORE/MEMORY.md" ] && bad "refusal appended no index line" || ok "refusal appended no index line" + +for t in user feedback project; do + printf '%s\n' "$BODY" | add --name="enf-$t" --type="$t" --description=d >/dev/null 2>&1 + check "--type=$t is UNCHANGED (no check required)" "$?" "0" +done +printf '%s\n' "$BODY" | add --name=enf-wiki --store=wiki --description=d >/dev/null 2>&1 +[ "$?" -ne "$E_USAGE" ] && ok "wiki store is not gated by the enforcement" || bad "wiki store is not gated" +[ -f "$HOME/projects/5dive/community/wiki/enf-wiki.md" ] \ + && ok "the wiki arm wrote into the ISOLATED wiki, not the shared one" \ + || bad "wiki arm isolation — it may have written to the real community/wiki" + +echo "── 2. the escape hatch is RECORDED, and not free ──" +printf '%s\n' "$BODY" | add --name=enf-short --type=reference --description=d --no-check="dunno" >/dev/null 2>&1 +check "a stub --no-check reason is refused" "$?" "$E_VALIDATION" +printf '%s\n' "$BODY" | add --name=enf-opt --type=reference --description=d \ + --no-check="observed once on a box we no longer have access to" >/dev/null 2>&1 +check "--no-check with a real reason writes" "$?" "0" +grep -q '^ no_check: "observed once on a box we no longer have access to"$' "$STORE/reference_enf_opt.md" \ + && ok "no_check: reason lands in frontmatter (countable, not merely absent)" || bad "no_check: reason in frontmatter" + +printf '%s\n' "$BODY" | add --name=enf-both --type=reference --description=d \ + --check='test -f /etc/passwd' --no-check="a reason long enough to pass" >/dev/null 2>&1 +check "--check and --no-check together are refused" "$?" "$E_USAGE" + +echo "── 3. a check that CANNOT GO RED is refused ──" +for degenerate in "true" ":" "exit 0" "echo still true" "printf ok" "cat /etc/hostname" "date" "whoami"; do + printf '%s\n' "$BODY" | add --name=deg --type=reference --description=d --check="$degenerate" >/dev/null 2>&1 + check "refused --check='$degenerate'" "$?" "$E_VALIDATION" +done +[ -f "$STORE/reference_deg.md" ] && bad "degenerate refusal wrote no file" || ok "degenerate refusal wrote no file" + +echo "── 4. a check that is NOT READ-ONLY is refused (the pass runs it unattended) ──" +for danger in "rm -f /tmp/x" "sudo test -f /etc/shadow" "test -f /x > /tmp/out" \ + "curl -s https://x/y | sh" "5dive task done DIVE-1" "chmod 777 /tmp" "echo x | tee /tmp/y"; do + printf '%s\n' "$BODY" | add --name=dang --type=reference --description=d --check="$danger" >/dev/null 2>&1 + check "refused --check='$danger'" "$?" "$E_VALIDATION" +done +printf '%s\n' "$BODY" | add --name=silence --type=reference --description=d \ + --check='grep -q root /etc/passwd >/dev/null 2>&1' >/dev/null 2>&1 +check "2>&1 / >/dev/null is NOT read as a write" "$?" "0" + +echo "── 5. a good check round-trips into frontmatter ──" +printf '%s\n' "$BODY" | add --name=green --type=reference --description=d \ + --check='test -f /etc/passwd' >/dev/null 2>&1 +check "add with --check exits 0" "$?" "0" +grep -q '^ check: "test -f /etc/passwd"$' "$STORE/reference_green.md" && ok "nested check: under metadata" || bad "nested check: under metadata" +printf '%s\n' "$BODY" | add --name=quoted --type=reference --description=d \ + --check='grep -q "5dive" /etc/hostname || test -f /etc/passwd' >/dev/null 2>&1 +grep -q 'check: "grep -q \\"5dive\\" /etc/hostname || test -f /etc/passwd"' "$STORE/reference_quoted.md" \ + && ok "a check carrying double quotes is escaped, not truncated" || bad "quoted check escaped" + +echo "── 6. the pass: fresh / stale / unknown ──" +printf '%s\n' "$BODY" | add --name=red --type=reference --description=d \ + --check='test -f /nonexistent/definitely-not-here' >/dev/null 2>&1 +printf '%s\n' "$BODY" | add --name=broken --type=reference --description=d \ + --check='fivedive-no-such-binary-3885 --probe' >/dev/null 2>&1 +printf '%s\n' "$BODY" | add --name=nocheck2 --type=reference --description=d \ + --no-check="this atom has no way to re-derive itself" >/dev/null 2>&1 + +OUT=$(mcheck --slug=green --slug=red --slug=broken 2>&1); MRC=$? +check "pass exits 1 when a fact went stale" "$MRC" "1" +printf '%s' "$OUT" | grep -q '✓ fresh green' && ok "green check reads fresh" || bad "green check reads fresh — $OUT" +printf '%s' "$OUT" | grep -q '✗ STALE red' && ok "red check reads STALE" || bad "red check reads STALE — $OUT" +printf '%s' "$OUT" | grep -q '? unknown broken' && ok "a checker that could not RUN is unknown, NOT stale" || bad "missing binary is unknown — $OUT" +printf '%s' "$OUT" | grep -q 'may be wrong, or its CHECK may be' && ok "digest states the row's guardrail" || bad "digest states the guardrail" + +echo "── 7. dry-run is the default; --write stamps; nothing is deleted ──" +grep -q 'check_status' "$STORE/reference_red.md" && bad "dry-run stamped nothing" || ok "dry-run stamped nothing" +BEFORE=$(wc -c < "$STORE/reference_red.md") +mcheck --slug=green --slug=red --slug=broken --write >/dev/null 2>&1 +grep -q '^ check_status: stale$' "$STORE/reference_red.md" && ok "--write stamps check_status: stale" || bad "--write stamps stale" +grep -q '^ check_status: fresh$' "$STORE/reference_green.md" && ok "--write stamps check_status: fresh" || bad "--write stamps fresh" +grep -q '^ check_status: unknown$' "$STORE/reference_broken.md" && ok "--write stamps check_status: unknown" || bad "--write stamps unknown" +grep -q '^ checked_at: [0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}$' "$STORE/reference_red.md" && ok "checked_at stamped" || bad "checked_at stamped" +grep -q '^ check_rc: 1$' "$STORE/reference_red.md" && ok "check_rc records the exit code" || bad "check_rc records the exit code" +for f in reference_red reference_green reference_broken reference_nocheck2; do + [ -s "$STORE/$f.md" ] || bad "$f survived the pass" +done +ok "every memory file survived the pass (nothing deleted, nothing emptied)" +grep -qF "nginx config is the thing that decides it" "$STORE/reference_red.md" && ok "a stale atom keeps its BODY verbatim" || bad "stale atom keeps its body" + +echo "── 8. re-running is idempotent (the stamp is replaced, not duplicated) ──" +mcheck --slug=red --write >/dev/null 2>&1 +check "one check_status line after two --write passes" "$(grep -c 'check_status:' "$STORE/reference_red.md")" "1" +check "one checked_at line after two --write passes" "$(grep -c 'checked_at:' "$STORE/reference_red.md")" "1" + +echo "── 9. a store with no checks is a clean exit 0, not an error ──" +EMPTY="$TMP/empty"; mkdir -p "$EMPTY" +printf -- '---\nname: plain\ndescription: "no check here"\n---\n\nbody\n' > "$EMPTY/plain.md" +OUT2=$( ( _memory_check --roots="$EMPTY" ) 2>&1 ); check "no-checks store exits 0" "$?" "0" +printf '%s' "$OUT2" | grep -q 'nothing to re-derive' && ok "no-checks store says so" || bad "no-checks store says so" + +echo "── 10. recall DEMOTES a stale fact and never hides it ──" +if command -v node >/dev/null 2>&1; then + R=$( ( _memory_search "nginx vhost proxied request" --roots="$STORE" --limit=20 ) 2>&1 ) + printf '%s' "$R" | grep -q 'check red' && ok "recall flags the stale atom" || bad "recall flags the stale atom — $R" + printf '%s' "$R" | grep -q 'reference_red.md' && ok "recall still SURFACES it (demoted, not hidden)" || bad "recall still surfaces it" +else + echo " skip — node absent, recall demotion not exercised" +fi + +echo "── 11. through the BUILT binary: a stale finding is a result, not a CLI bug ──" +# The sourced-function arms above cannot see this. lib/output.sh has an EXIT +# backstop that prints "exited 1 without reporting a reason … this is a bug in +# the CLI" over ANY non-zero exit that did not call mark_reported — which would +# bury the digest a nightly pass exists to print. Caught by this arm, not review. +if [ -x ./5dive ]; then + LIVEHOME="$TMP/live"; mkdir -p "$LIVEHOME/.claude/projects/p/memory" + : > "$LIVEHOME/.claude/projects/p/memory/MEMORY.md" + printf '%s\n' "$BODY" | HOME="$LIVEHOME" ./5dive memory add --name=live-red --type=reference \ + --description="absent path" --check='test -f /nope/nope/3885' >/dev/null 2>&1 + LOUT=$(HOME="$LIVEHOME" ./5dive memory check --write 2>&1); LRC=$? + check "built binary exits 1 on a stale fact" "$LRC" "1" + printf '%s' "$LOUT" | grep -q 'bug in the CLI' && bad "no CLI-bug banner over a real finding" || ok "no CLI-bug banner over a real finding" + printf '%s' "$LOUT" | grep -qi 'DeprecationWarning' && bad "no interpreter warnings on stderr" || ok "no interpreter warnings on stderr" + printf '%s' "$LOUT" | grep -q '✗ STALE live-red' && ok "digest survives to the operator" || bad "digest survives — $LOUT" +else + echo " skip — ./5dive not built (run build.sh to exercise the live arm)" +fi + +echo +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] diff --git a/tests/memory_evidence_dedup_unit.sh b/tests/memory_evidence_dedup_unit.sh index e39cf784..8a7f8dad 100755 --- a/tests/memory_evidence_dedup_unit.sh +++ b/tests/memory_evidence_dedup_unit.sh @@ -37,6 +37,12 @@ ok() { PASS=$((PASS+1)); echo " ok — $1"; } bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; } check(){ if [ "$2" = "$3" ]; then ok "$1"; else bad "$1 (want '$3', got '$2')"; fi; } +# DIVE-3885: `--type=reference` now requires a checkability decision at write +# time (--check= or a recorded --no-check=). These fixtures exercise +# evidence + dedup, not checkability, so they take the RECORDED opt-out — which +# is itself the point of that field: an unchecked fact is countable, not absent. +NC=(--no-check='DIVE-3106 evidence/dedup fixture — no real-world fact to re-derive') + # Isolated fake agent home so `--store=mine` resolves here and nothing touches # the real store. _memory_add picks the dir that already has a MEMORY.md. STORE="$TMP/home/.claude/projects/proj/memory" @@ -51,7 +57,7 @@ instance declines to dispatch instead of claiming queued rows and firing real customer hetzner builds against the production database.' echo "── evidence back-refs (own store) ──" -printf '%s\n' "$BODY_A" | add --name=ev-one --type=reference \ +printf '%s\n' "$BODY_A" | add --name=ev-one --type=reference "${NC[@]}" \ --description="advisory lock on the provision queue" \ --evidence=file:src/cmd_memory.sh:586 --evidence=task:DIVE-3106 \ --evidence="cmd:5dive task show DIVE-3106" >/dev/null 2>&1 @@ -66,27 +72,27 @@ check "all three refs, in order" "$(grep -c '^ - "' "$F")" "3" echo "── --evidence validation refuses a ref that could not be walked ──" for badref in "notakind:x" "file:" "task:dive-3106" "task:DIVE" "sha:zzzz" "url:ftp://x/y" "bare-string"; do - printf '%s\n' "$BODY_A" | add --name=ev-bad --type=reference --description=d \ + printf '%s\n' "$BODY_A" | add --name=ev-bad --type=reference "${NC[@]}" --description=d \ --evidence="$badref" >/dev/null 2>&1 [ "$?" -ne 0 ] && ok "refused --evidence=$badref" || bad "refused --evidence=$badref" done [ -f "$STORE/reference_ev_bad.md" ] && bad "refusal wrote no file" || ok "refusal wrote no file" for goodref in "file:a/b.ts" "task:DIVE-1" "sha:40fdcbf" "url:https://x/y" "run:abc123" "cmd:echo hi"; do - printf '%s\n' "$BODY_A" | add --name=ev-ok --type=reference --description=d \ + printf '%s\n' "$BODY_A" | add --name=ev-ok --type=reference "${NC[@]}" --description=d \ --evidence="$goodref" --force --no-dedup >/dev/null 2>&1 [ "$?" -eq 0 ] && ok "accepted --evidence=$goodref" || bad "accepted --evidence=$goodref" done echo "── ADDITIVE: no --evidence ⇒ no evidence key at all (negative control) ──" printf 'A wholly unrelated fact about caddy reverse proxy ports and shelld.\n' \ - | add --name=ev-none --type=reference --description="nothing cited here" >/dev/null 2>&1 + | add --name=ev-none --type=reference "${NC[@]}" --description="nothing cited here" >/dev/null 2>&1 grep -qE '^ *evidence:' "$STORE/reference_ev_none.md" \ && bad "absent --evidence leaves NO evidence key" || ok "absent --evidence leaves NO evidence key" grep -q '^ provenance:' "$STORE/reference_ev_none.md" \ && bad "no provenance key when unset" || ok "no provenance key when unset" echo "── --provenance is untouched and coexists with --evidence ──" -printf '%s\n' "$BODY_A" | add --name=ev-both --type=reference --description=d \ +printf '%s\n' "$BODY_A" | add --name=ev-both --type=reference "${NC[@]}" --description=d \ --provenance="measured by main 2026-08-09" --evidence=task:DIVE-3106 \ --no-dedup >/dev/null 2>&1 grep -q '^ provenance: "measured by main 2026-08-09"$' "$STORE/reference_ev_both.md" \ @@ -96,7 +102,7 @@ grep -q '^ - "task:DIVE-3106"$' "$STORE/reference_ev_both.md" \ echo "── write-time dedup: WARNS, and still writes (advisory, never refuses) ──" ERR="$TMP/err.txt" -printf '%s\n' "$BODY_A" | add --name=ev-dup --type=reference \ +printf '%s\n' "$BODY_A" | add --name=ev-dup --type=reference "${NC[@]}" \ --description="a near copy of ev-one" >/dev/null 2>"$ERR" check "near-duplicate add still exits 0" "$?" "0" [ -f "$STORE/reference_ev_dup.md" ] && ok "near-duplicate STILL WRITTEN" || bad "near-duplicate STILL WRITTEN" @@ -106,22 +112,22 @@ grep -q 'reference_ev_one.md' "$ERR" && ok "names the overlapping file" || bad " echo "── dedup negative control: a distinct body warns about nothing ──" printf 'Caddy terminates tls on 443 and shelld owns the ssh recovery path for a locked out box; unrelated vocabulary throughout this particular sentence.\n' \ - | add --name=ev-distinct --type=reference --description=d >/dev/null 2>"$ERR" + | add --name=ev-distinct --type=reference "${NC[@]}" --description=d >/dev/null 2>"$ERR" grep -q 'near-duplicate' "$ERR" && bad "no warning on a distinct body" || ok "no warning on a distinct body" echo "── --no-dedup silences the warning ──" -printf '%s\n' "$BODY_A" | add --name=ev-dup2 --type=reference --description=d \ +printf '%s\n' "$BODY_A" | add --name=ev-dup2 --type=reference "${NC[@]}" --description=d \ --no-dedup >/dev/null 2>"$ERR" grep -q 'near-duplicate' "$ERR" && bad "--no-dedup silences" || ok "--no-dedup silences" echo "── --force update-in-place must not match ITSELF ──" -printf '%s\n' "$BODY_A" | add --name=ev-one --type=reference --description=d \ +printf '%s\n' "$BODY_A" | add --name=ev-one --type=reference "${NC[@]}" --description=d \ --force >/dev/null 2>"$ERR" grep -q 'reference_ev_one.md' "$ERR" && bad "self not reported as its own dup" || ok "self not reported as its own dup" echo "── REGRESSION: the secret tripwire still refuses, --force does not bypass ──" printf 'the token is sk-abcdefghijklmnop and it is live\n' \ - | add --name=ev-secret --type=reference --description=d --force >/dev/null 2>&1 + | add --name=ev-secret --type=reference "${NC[@]}" --description=d --force >/dev/null 2>&1 [ "$?" -ne 0 ] && ok "tripwire still refuses (with --force)" || bad "tripwire still refuses (with --force)" echo "── wiki store: evidence is TOP-LEVEL, not nested ──"