Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
256 changes: 256 additions & 0 deletions scripts/check-memory-frontmatter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,256 @@
#!/usr/bin/env python3
"""Lint session-memory frontmatter for taxonomy-field schema drift (issue #942).

Background: this repo's on-disk session memories
(`$CLAUDE_CONFIG_DIR/projects/<slug>/memory/*.md`) nest taxonomy fields
(`type`, `originSessionId`, `hookable`, `hookKeywords`, `hookEvents`,
`modified`, `node_type`) under a `metadata:` block by convention — see
`skills/retrospect/references/stage4-execution.md`. Nesting has NO runtime
effect on `hooks/advisory-nudge/memory-hint/impl.py` (its regexes match
either position); the only reason to enforce it is cross-entry consistency,
so a reader never has to check two positions per field.

`node_type` is included in the taxonomy set for *position* checking (if
present, it must be nested and not duplicated, same as any other taxonomy
field) but is never required to be present — no hook in this repo reads it
(grep-verified). A memory missing `node_type` entirely is not drift.

Two retrospect cycles (17 -> 18) flagged the same drift family without it
ever becoming a re-checkable artifact — the finding lived only in a carried
`cycle_note` string in `.omc/state/retrospect-hygiene-cursor.json`, which has
no CI and no issue to close against. This script is the structural fix: it
runs the same check every time instead of relying on a future retrospect
scan to notice again.

Checks (one entry = one `*.md` file in the memory dir, `MEMORY.md` excluded):
1. no taxonomy field at the top level of the frontmatter — each must live
under `metadata:`
2. no taxonomy field duplicated (top-level + nested, or nested twice)
3. `hookKeywords:` / `hookEvents:`, when present, use the single-line
bracket form (`[a, b]`). The multi-line YAML-block form (`hookKeywords:`
followed by `- item` lines) is a FUNCTIONAL bug, not just a style
drift: `hooks/advisory-nudge/memory-hint/impl.py`'s KEYWORDS_RE/EVENTS_RE
read the rest of the `key:` line only, find no `[`, and silently reject
the whole memory — `hookable: true` then never fires. Two entries in the
corpus shipped this way (issue #942 scan): the hint had been dark since
they were authored.
4. `hookable: true` with `hookKeywords` missing entirely, or present as an
empty list (`hookKeywords: []`), is the same silent-drop failure as
check 3's block/scalar form — `impl.py:117-139` returns `None` (no
memory, no hint, ever) the moment `hookKeywords:` has no match or its
bracket content is empty. This check was added in a later revision of
this script (issue #942 F1, caught by an independent codex-review pass
after the first revision shipped without it): the original release only
inspected a field that IS present, so a memory missing `hookKeywords`
outright passed as clean while being just as dark as the block-form
case check 3 catches.

Note on `hookEvents:` specifically: an empty list (`hookEvents: []`) is NOT
flagged the same way — `impl.py:150-169` falls back to the `[Bash]` default
when the bracket content is empty or unparseable, so the memory still gets
indexed (see check 3's own error text distinguishing the two fields' failure
modes — this was also caught in the F1/F2 codex-review pass: prior to that,
`check_file()`'s error message for `hookEvents` incorrectly claimed the same
"entire memory dropped" consequence hookKeywords has).

Resolution: reuses `hooks/_lib/_memory_dir.py::resolve_memory_dir()`, so this
follows `PRAXIS_MEMORY_DIR` / `CLAUDE_CONFIG_DIR` exactly like the memory-hint
hook does — no separate path convention to drift from that one.

Exit codes: 0 clean (or N/A, unless PRAXIS_TESTS_STRICT=1), 1 on any
violation (or on N/A under PRAXIS_TESTS_STRICT=1, for standalone invocation).

CI reality: the memory directory is a local, gitignored, per-user store and
does not exist in CI at all — so in CI this script always prints N/A and
exits 0 (`scripts/run-tests.sh` strips PRAXIS_TESTS_STRICT for this one call,
so N/A can never fail the CI job either). This label is deliberately NOT
"SKIPPED": `scripts/run-tests.sh`'s header explains why a skip is not a pass
(#917) — a SKIPPED line names a *fixable* gap (install the tool) that a
strict rerun can close. This condition can never close; the directory is
structurally absent in CI by design, forever. Printing it as SKIPPED would
plant a permanently-unresolvable entry in exactly the noise this repo already
learned (via #917) drowns out the *actionable* skips — the same signal-loss
failure mode issue #942 itself was opened to stop recurring. N/A keeps it
readable as "nothing to check here", not "something you could fix but
didn't". It has enforcement value locally (a contributor normalizing memory
frontmatter) and as a component this repo's own tests exercise directly
(`tests/test_check_memory_frontmatter.py`, via `PRAXIS_MEMORY_DIR`).
"""
from __future__ import annotations

import os
import re
import sys
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO_ROOT / "hooks" / "_lib"))
from _memory_dir import resolve_memory_dir # type: ignore[import-not-found] # noqa: E402

FENCE_RE = re.compile(r"^---\s*$", re.MULTILINE)
KEY_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\s*:\s*(.*)$")

# Fields this repo's convention requires nested under `metadata:`.
TAXONOMY_FIELDS = (
"node_type",
"type",
"originSessionId",
"hookable",
"hookKeywords",
"hookEvents",
"modified",
)
# Fields that must use single-line `[a, b]` form — the memory-hint parser
# rejects any other shape outright (see module docstring, check 3).
BRACKET_FIELDS = ("hookKeywords", "hookEvents")

# Mirrors hooks/advisory-nudge/memory-hint/impl.py:60 exactly (TRUTHY_VALUES) —
# the runtime's own truthy set for `hookable:`, after the same normalization
# (strip, lowercase, strip quotes) that impl.py:108-113 applies.
HOOKABLE_TRUTHY_VALUES = {"true", "yes"}


def frontmatter_block(raw: str) -> str | None:
"""Return the text between the first two `---` fences, or None."""
matches = list(FENCE_RE.finditer(raw))
if len(matches) < 2:
return None
return raw[matches[0].end() : matches[1].start()]


def _indented_lines(block: str) -> list[tuple[int, str]]:
out = []
for line in block.splitlines():
if not line.strip():
continue
indent = len(line) - len(line.lstrip(" "))
out.append((indent, line.strip()))
return out


def check_file(path: Path) -> list[str]:
"""Return a list of human-readable violations for one memory file."""
raw = path.read_text(encoding="utf-8")
block = frontmatter_block(raw)
if block is None:
return ["missing `---`-fenced frontmatter block"]

errors: list[str] = []
# field -> list of (nested: bool, raw_value: str)
occurrences: dict[str, list[tuple[bool, str]]] = {}
in_metadata = False

for indent, line in _indented_lines(block):
m = KEY_RE.match(line)
if not m:
continue # e.g. a `- item` list line, or free-text
key, value = m.group(1), m.group(2)
if indent == 0:
if key == "metadata":
in_metadata = True
continue
in_metadata = False
nested = in_metadata and indent > 0
if key in TAXONOMY_FIELDS:
occurrences.setdefault(key, []).append((nested, value))

for field, occs in occurrences.items():
if len(occs) > 1:
errors.append(f"`{field}` appears {len(occs)} times — must appear exactly once, nested under `metadata:`")
for nested, value in occs:
if not nested:
errors.append(f"`{field}` is at the top level — must nest under `metadata:`")
if field in BRACKET_FIELDS:
# hookKeywords and hookEvents fail differently at runtime (F2,
# issue #942, caught by an independent codex-review pass): a
# malformed hookKeywords drops the WHOLE memory (impl.py:117-139
# returns None outright), but a malformed hookEvents just falls
# back to the `[Bash]` default (impl.py:150-169) — the memory
# still gets indexed, only the intended non-Bash event is lost.
# An earlier revision of this script used one shared "entire
# memory dropped" message for both fields; that was accurate for
# hookKeywords and wrong for hookEvents.
consequence = (
"silently drops the entire memory from the hint index"
if field == "hookKeywords"
else "silently falls back to the default `[Bash]` event — the memory "
"still gets indexed, but the intended non-Bash event is lost"
)
for _nested, value in occs:
value = value.strip()
if not value:
errors.append(
f"`{field}:` has no inline value (multi-line YAML-block `- item` form) — "
f"the memory-hint parser rejects this shape and {consequence}; "
"use single-line `[a, b]` form"
)
elif not value.startswith("["):
errors.append(
f"`{field}: {value}` is scalar form, not `[a, b]` — the memory-hint parser "
f"rejects this shape and {consequence}"
)
elif field == "hookKeywords":
close_idx = value.find("]")
inner = value[1:close_idx].strip() if close_idx != -1 else value[1:]
if not inner:
errors.append(
f"`{field}: {value}` is an empty list — the memory-hint parser treats "
"an empty hookKeywords the same as absent and silently drops the entire "
"memory from the hint index (issue #942 F1)"
)

# F1 (issue #942): `hookable: true` with `hookKeywords` entirely absent is
# the most direct silent-dark shape and was previously invisible to this
# lint — the BRACKET_FIELDS loop above only inspects a field that IS
# present. hooks/advisory-nudge/memory-hint/impl.py:117-119 returns None
# (drops the memory from the hint index) the moment `hookKeywords:` has no
# match in the frontmatter block at all.
hookable_truthy = False
if "hookable" in occurrences:
_, hookable_value = occurrences["hookable"][0]
hookable_truthy = hookable_value.strip().strip("\"'").lower() in HOOKABLE_TRUTHY_VALUES
if hookable_truthy and "hookKeywords" not in occurrences:
errors.append(
"`hookable: true` but `hookKeywords` is missing — the memory-hint parser requires "
"it and silently drops the entire memory from the hint index when absent (issue #942 F1)"
)

return errors


def iter_memory_files(memory_dir: str) -> list[Path]:
return sorted(p for p in Path(memory_dir).glob("*.md") if p.name != "MEMORY.md")


def main() -> int:
strict = os.environ.get("PRAXIS_TESTS_STRICT", "0") == "1"

memory_dir = resolve_memory_dir()
if not memory_dir:
print("N/A: memory-frontmatter-lint (memory directory not found — expected in CI / a fresh checkout, not a fixable gap, see issue #942)")
return 1 if strict else 0

files = iter_memory_files(memory_dir)
if not files:
print(f"N/A: memory-frontmatter-lint (no memory entries under {memory_dir})")
return 1 if strict else 0

total_errors = 0
for path in files:
errors = check_file(path)
if errors:
total_errors += len(errors)
print(f"FAIL: {path.name}")
for err in errors:
print(f" - {err}")

if total_errors:
print(f"\n{total_errors} frontmatter schema violation(s) across {len(files)} memory entries in {memory_dir}")
return 1

print(f"OK: {len(files)} memory entries in {memory_dir} — frontmatter schema clean")
return 0


if __name__ == "__main__":
sys.exit(main())
54 changes: 46 additions & 8 deletions scripts/run-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,29 @@
# 2. shell — shell tests at tests/hooks/*/test_*.sh and tests/test_*.sh
# 3. manifest — scripts/check-plugin-manifests.py
# 4. invariants — scripts/check-hook-token-invariants.py
# 5. ruff — static Python lint (mirrors the ci.yml `ruff` job)
# 6. shellcheck — static shell lint (mirrors the ci.yml `shellcheck` job)
# 7. markdownlint — advisory markdown lint (mirrors the ci.yml `markdownlint` job)
# 5. memory-frontmatter — scripts/check-memory-frontmatter.py
# 6. ruff — static Python lint (mirrors the ci.yml `ruff` job)
# 7. shellcheck — static shell lint (mirrors the ci.yml `shellcheck` job)
# 8. markdownlint — advisory markdown lint (mirrors the ci.yml `markdownlint` job)
#
# Steps 5-7 mirror CI jobs that used to have no local equivalent, so a change
# Steps 6-8 mirror CI jobs that used to have no local equivalent, so a change
# could pass here and still be flagged on the PR (issue #866). Each skips with
# an explicit SKIPPED line when its tool is absent, so a contributor without
# the toolchain is not blocked — CI remains authoritative either way.
#
# Step 5 is a different repo-internal script, same family as steps 3-4 (no
# external toolchain — nothing to install), but its skip condition is not
# like steps 6-8's: the memory directory it lints is a local, gitignored,
# per-user store that is structurally absent in CI or a fresh checkout,
# always, forever (issue #942) — not a "toolchain not installed" gap a
# contributor can close. It prints "N/A", never "SKIPPED", to keep that
# distinction visible in the log (see its own docstring / #917 below), and
# this call strips PRAXIS_TESTS_STRICT (`env -u`, not passed through like
# steps 6-8) so that permanent N/A can never fail the job. Real drift
# (nonzero exit with violations listed) still counts as FAILED, same as
# steps 3-4 — this is not routed through the SKIPPED_TOOLS/skip_step() path
# at all.
#
# A skip is not a pass (#917). The SKIPPED line used to scroll past and the run
# still ended on a bare "ALL TESTS PASSED", which read as full coverage: PR #912
# shipped 15 markdownlint violations that way, and PR #864 had shipped 23 for the
Expand Down Expand Up @@ -123,7 +137,29 @@ if ! python3 ./scripts/check-hook-token-invariants.py; then
fi

# ---------------------------------------------------------------------------
# 5. Ruff (mirrors ci.yml `ruff` job — blocking there, blocking here)
# 5. Memory frontmatter lint (schema-drift guard, issue #942)
# ---------------------------------------------------------------------------
echo ""
echo "=== memory frontmatter lint ==="
# PRAXIS_TESTS_STRICT is deliberately NOT propagated to this one call: unlike
# steps 6-8 (a tool a contributor could install), the memory dir this checks
# is a local, gitignored, per-user store that structurally never exists in
# CI or a fresh checkout — treating its absence as a strict-mode failure
# would fail every CI run forever, not flag a fixable gap. The script prints
# "N/A", not "SKIPPED", for exactly this reason: #917 exists because a
# scrolling SKIPPED line lost its signal value once contributors stopped
# reading it, and a condition that can never be fixed would sit there as
# permanent unresolvable noise if it wore the same "SKIPPED" label as steps
# 6-8's genuinely-fixable tool-absence skips. An N/A here is always benign;
# only actual detected drift (exit 1 with violations listed) fails this step.
# The script's own PRAXIS_TESTS_STRICT support still works for direct
# standalone invocation (see its tests / docstring).
if ! env -u PRAXIS_TESTS_STRICT python3 ./scripts/check-memory-frontmatter.py; then
FAILED=1
fi

# ---------------------------------------------------------------------------
# 6. Ruff (mirrors ci.yml `ruff` job — blocking there, blocking here)
# ---------------------------------------------------------------------------
echo ""
echo "=== ruff ==="
Expand All @@ -142,7 +178,7 @@ elif ! "${RUFF[@]}" check .; then
fi

# ---------------------------------------------------------------------------
# 6. Shellcheck (mirrors ci.yml `shellcheck` job — same find/severity)
# 7. Shellcheck (mirrors ci.yml `shellcheck` job — same find/severity)
# ---------------------------------------------------------------------------
echo ""
echo "=== shellcheck ==="
Expand All @@ -157,7 +193,7 @@ else
fi

# ---------------------------------------------------------------------------
# 7. Markdownlint (mirrors ci.yml `markdownlint` job)
# 8. Markdownlint (mirrors ci.yml `markdownlint` job)
#
# CI runs this with filter_mode:added and never fails the check, so the repo's
# existing backlog stays untouched. The advisory half is mirrored exactly — a
Expand Down Expand Up @@ -221,7 +257,9 @@ if [[ $FAILED -ne 0 ]]; then
exit 1
fi

# Scope note: this counts the three tool steps this script owns (5-7). Gates
# Scope note: this counts the three tool steps this script owns (6-8). Step 5
# has its own N/A line (deliberately not "SKIPPED") and is excluded from this
# tally on purpose — it is never a missing-toolchain skip. Gates
# that a sub-suite skips internally — e.g. the Darwin-only gate in
# tests/test_codex_broker_reaper.sh — announce themselves in their own summary
# and are not aggregated here; conflating them would make a portable-by-design
Expand Down
Loading
Loading