feat(vocab): LIVE dogfood — run the currency loop on the estate's own vocab (task #13)

.gitignore

@@ -22,3 +22,4 @@ build/
 
 # GitHub API artifacts accidentally fetched locally
 pulls/
+build/vocab-currency-proposals/

CHANGELOG.md

@@ -9,6 +9,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). Thi
 ## [Unreleased]
 
 ### Added
+- Vocab-currency loop — LIVE dogfood on the estate's own vocab (`make vocab-currency-dogfood`): `tools/dogfood_vocab_currency.py` runs the governed loop over this repo's approved `GlossaryTerm`s (fixed/LSA) vs `specs/*.md` (open/LDA). Today only a handful of terms are approved against the full spec corpus, so the governed loop escalates-human (can't reach currency in its bound), writing real draft `GlossaryTerm` proposals to `build/vocab-currency-proposals/` — the artifacts ontogenesis ingests. Reports the outcome (does not gate on it, exit 0); Markdown-cleaned + generic-word stoplisted so domain terms surface. Enforcement stays on fixtures in `validate-vocab-currency-loop`.
 - Vocab-currency loop — draft-term emission (closes the currency→proposal lifecycle): each remediation step now emits a conformant status:**draft** `GlossaryTerm` artifact (`result.proposedTerms`) for the token it connects — the loop PROPOSES drafts for ontogenesis's 3-method alignment pass and never self-approves (`partOfSpeech`/`alignment` are left for downstream; status stays `draft`). The CI harness now REFUSES any proposed term that doesn't conform to `GlossaryTerm.json` or that isn't status:draft, so the loop can't push malformed or self-approved vocab into the glossary. 6 teeth total.
 - Vocab-currency loop v0.1 — the vocabulary-currency GovernedLoop made LIVE (task #13): `tools/vocab_currency_loop.py` instantiates the merged `GovernedLoop` contract and is governed BY it (reads bound + tolerance + `onNonConvergence` + admission from the document). The approved glossary is the FIXED set (LSA), the corpus token distribution is the OPEN set (LDA), and uncovered probability mass is the currency signal (the LSI drive: new fields that became vocab but aren't connected). Remediation connects the highest-mass uncovered term (a proposed `GlossaryTerm`), strictly decreasing divergence. Fail-closed: runs at most `maxIterations` (never spins); escalates-human (non-zero exit) if it can't reach tolerance within the bound rather than declaring currency; refuses to run unadmitted (no `superconsciousRef`). `validate-vocab-currency-loop` asserts all three teeth (converges / escalates / refuses). Emits `candidateNewVocab` for routing into ontogenesis/prophet-ontology. Dependency-free.
 - DataClass v0.1 (OntoDT/OntoDQ data-side governance, task #14; AMG 2.0 deck): a `DataClass` binds a datatype-ontology class (OntoDT), the business `GlossaryTerm` (biz↔data), and a `ValidValues` domain (enum/range/regex), with an optional **TF-Lattice wide-and-deep** classifier that is a first-class CATALOGED model — `modelRef`→`ModelManifest`, `runRef`→`RunRecord`, `compute` on **Ray/TritFabric**, `monotonicFeatures` lattice constraint, and `labels` that are `GlossaryTerm` URNs (assigned in the glossary). `EntityField` gains an optional `dataClassRef`; the drift-guard enforces field↔class domain conformance + classifier integrity (labels are glossary terms, model is cataloged, compute is ray/tritfabric) + ontological typing. 4 negative vectors, teeth-verified. Backward-compatible. Two-level classifier: per-class LOGISTIC head (`DataClass.classifier`, one-vs-rest, required `evalRunRef` — individually testable per class/glossary term) + a per-table SOFTMAX `TableClassifier` aligning an LSA bag-of-words embedding AND a doc2vec sentence-encoder (the n-ary logit→class assignment), a cataloged model on Ray/TritFabric assigning N DataClasses. Drift-guard enforces both.

Makefile

@@ -1,4 +1,4 @@
-.PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-ingestion-pipeline-examples validate-data-acquisition-examples validate-glossary-alignment-examples validate-data-class-examples validate-table-keys validate-dag-loop validate-vocab-currency-loop
+.PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-ingestion-pipeline-examples validate-data-acquisition-examples validate-glossary-alignment-examples validate-data-class-examples validate-table-keys validate-dag-loop validate-vocab-currency-loop vocab-currency-dogfood
 
 validate: validate-data-class-examples validate-glossary-alignment-examples validate-data-acquisition-examples validate-ingestion-pipeline-examples validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-resource-contract validate-table-keys validate-dag-loop validate-vocab-currency-loop
 	@echo "OK: validate"
@@ -23,6 +23,12 @@ validate-vocab-currency-loop:
 	python3 -m pip install --user jsonschema >/dev/null
 	python3 tools/validate_vocab_currency_loop.py
 
+# On-demand LIVE currency scan of THIS repo's approved glossary vs specs/*.md.
+# Informational (always exit 0) — writes draft proposals to build/vocab-currency-proposals/.
+vocab-currency-dogfood:
+	python3 -m pip install --user jsonschema >/dev/null
+	python3 tools/dogfood_vocab_currency.py
+
 validate-dag-loop:
 	python3 -m pip install --user jsonschema >/dev/null
 	python3 tools/validate_dag_loop.py

specs/vocab-currency-loop.md

@@ -28,3 +28,16 @@ ontogenesis / the prophet-ontology as draft terms for the 3-method alignment pas
 ## Remediation artifacts
 
 Each connected token is emitted as a conformant status:`draft` `GlossaryTerm` in `result.proposedTerms` — the loop's output is directly ingestible by ontogenesis for the 3-method alignment pass. The loop only proposes: `partOfSpeech`, `alignment`, and approval are downstream. `make validate-vocab-currency-loop` refuses any proposed term that fails `GlossaryTerm.json` or that is not `draft` (the loop may not self-approve).
+
+## Live dogfood (`make vocab-currency-dogfood`)
+
+`tools/dogfood_vocab_currency.py` runs the governed loop over THIS repo's real vocabulary — every
+approved `GlossaryTerm` in `examples/` (fixed/LSA) vs `specs/*.md` (open/LDA) — answering honestly
+whether our own approved vocab is current with our own specs. Today it is not — only a handful of
+terms are approved against the full spec corpus, so divergence starts high and the governed loop,
+unable to reach currency within its iteration bound, **escalates-human** with concrete draft
+proposals rather than pretending currency. (Exact counts/divergence are point-in-time; run it to
+see the current numbers.) It writes each proposed draft `GlossaryTerm` to `build/vocab-currency-proposals/<slug>.json`
+— the exact artifacts ontogenesis ingests. Informational (exit 0), corpus-configured (Markdown
+cleaned, generic prose words stoplisted so DOMAIN terms surface); the ENFORCEMENT stays in
+`validate-vocab-currency-loop` on known-good fixtures.

tools/dogfood_vocab_currency.py

@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+"""Dogfood the vocab-currency GovernedLoop on THIS repo's own vocabulary (task #13).
+
+Runs the governed loop over the estate's real material — every APPROVED GlossaryTerm in
+`examples/` is the fixed set (LSA); `specs/*.md` is the corpus (LDA) — to answer, honestly: is
+our own approved vocabulary current with our own specifications? On real incomplete vocab the
+governed loop does exactly what it should: it connects what it can within the bound, then
+ESCALATES-human with concrete proposals rather than pretending currency.
+
+This is a REPORT, not a gate: it does not fail on the currency OUTCOME (the estate vocab is
+legitimately incomplete today, so failing CI on a non-current result would be wrong) — it exits 0
+whether the loop reaches currency or escalates. (A genuine IO error still surfaces normally.)
+It writes each proposed draft `GlossaryTerm` to
+`build/vocab-currency-proposals/<slug>.json` — those are the artifacts ontogenesis ingests for
+the 3-method alignment pass. The ENFORCEMENT lives in `validate_vocab_currency_loop.py` (synthetic
+fixtures with a known-good outcome); this shows the same loop on live data.
+"""
+from __future__ import annotations
+
+import glob
+import json
+import re
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+import vocab_currency_loop as vcl  # noqa: E402
+
+ROOT = Path(__file__).resolve().parents[1]
+OUT = ROOT / "build" / "vocab-currency-proposals"
+LOOP = ROOT / "examples" / "governed_loop.vocab_currency.json"
+
+# Real prose carries generic words that aren't estate vocabulary. The core loop stays general;
+# for a live scan of Markdown specs we extend the tokenizer's stoplist here so the surfaced
+# candidates are DOMAIN terms (contract, runtime, attestation) rather than filler (one, every,
+# same). This is corpus configuration, not a change to the loop's semantics.
+GENERIC = {
+    "one", "two", "three", "every", "only", "same", "may", "what", "each", "also", "both",
+    "here", "there", "when", "where", "which", "these", "those", "such", "more", "most",
+    "some", "other", "another", "using", "used", "use", "via", "per", "yet", "still", "now",
+    "does", "done", "make", "makes", "made", "way", "ways", "like", "just", "even", "them",
+    "they", "you", "your", "who", "how", "why", "but", "all", "not", "has", "had", "have",
+}
+
+
+def clean_markdown(text: str) -> str:
+    text = re.sub(r"```.*?```", " ", text, flags=re.S)   # fenced code
+    text = re.sub(r"`[^`]*`", " ", text)                  # inline code
+    text = re.sub(r"https?://\S+", " ", text)             # urls
+    text = re.sub(r"[#>*\-_|]", " ", text)                # md punctuation
+    return text
+
+
+def real_glossary() -> dict:
+    """Every GlossaryTerm found in examples/ (bundle `terms` arrays + standalone term docs),
+    deduped by id. run_loop counts only status==approved as coverage."""
+    by_id: dict[str, dict] = {}
+    for f in sorted(glob.glob(str(ROOT / "examples" / "*.json"))):
+        try:
+            doc = json.loads(Path(f).read_text(encoding="utf-8"))
+        except (json.JSONDecodeError, OSError):
+            continue
+        items = doc["terms"] if isinstance(doc, dict) and isinstance(doc.get("terms"), list) else [doc]
+        for it in items:
+            # A glossary-namespaced id AND (untyped bundle entry OR explicitly a GlossaryTerm) —
+            # so a doc merely carrying such an id but typed as something else isn't miscounted.
+            if (isinstance(it, dict) and str(it.get("id", "")).startswith("urn:srcos:glossary:")
+                    and it.get("type") in (None, "GlossaryTerm")):
+                by_id[it["id"]] = it
+    return {"terms": list(by_id.values())}
+
+
+def real_corpus() -> dict:
+    docs = []
+    for f in sorted(glob.glob(str(ROOT / "specs" / "*.md"))):
+        try:
+            text = Path(f).read_text(encoding="utf-8")
+        except (OSError, UnicodeDecodeError) as exc:
+            print(f"skipping unreadable spec {f}: {exc}", file=sys.stderr)
+            continue
+        docs.append({"id": Path(f).stem, "text": clean_markdown(text)})
+    return {"documents": docs}
+
+
+def main() -> int:
+    vcl.STOP |= GENERIC  # configure the tokenizer for prose (domain terms surface, not filler)
+    glossary = real_glossary()
+    corpus = real_corpus()
+    approved = [t["id"] for t in glossary["terms"] if t.get("status") == "approved"]
+
+    loop = vcl.load(LOOP)
+    result = vcl.run_loop(loop, glossary, corpus)
+
+    # Persist the proposed draft terms — the real artifacts for ontogenesis ingestion.
+    OUT.mkdir(parents=True, exist_ok=True)
+    written = []
+    for term in result.get("proposedTerms", []):
+        slug = term["id"].split(":")[-1]
+        (OUT / f"{slug}.json").write_text(json.dumps(term, indent=2) + "\n", encoding="utf-8")
+        written.append(slug)
+
+    print(json.dumps({
+        "scan": "vocab-currency dogfood (specs/*.md vs approved glossary)",
+        "corpusDocs": len(corpus["documents"]),
+        "approvedTerms": approved,
+        "governedBy": result.get("admittedBy"),
+        "initialDivergence": result.get("trace", [{}])[0].get("divergence"),
+        "finalDivergence": result.get("finalDivergence"),
+        "iterations": result.get("iterations"),
+        "maxIterations": result.get("maxIterations"),
+        "outcome": ("current" if result.get("ok") else result.get("escalated") or result.get("refused")),
+        "detail": result.get("detail"),
+        "proposalsWritten": written,
+        "topRemainingCandidates": [c["term"] for c in result.get("candidateNewVocab", [])],
+    }, indent=2))
+    # Exit 0 regardless of the currency OUTCOME — this reports the state of live vocab, it does
+    # not gate on whether the vocab happens to be current (a genuine IO error still surfaces above).
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())