diff --git a/.devcontainer/memories/MEMORY.md b/.devcontainer/memories/MEMORY.md index 63e6078..2ad0b02 100644 --- a/.devcontainer/memories/MEMORY.md +++ b/.devcontainer/memories/MEMORY.md @@ -2,7 +2,7 @@ SKILL-LOADING RULE: Before ANY GitHub/Git operation in a Codespace, ALWAYS load § WIKI-SKILL SYNC RULE: When updating a skill in `.devcontainer/skills/`, always check if any wiki article in `.devcontainer/wiki/` references the same topic and needs a corresponding update (or cross-reference). User explicitly asked: "check whether the change in this skill deserve some changes in some LM wiki that we have stored." Wiki = reference knowledge; skill = procedural. They should stay in sync on the same topic. § -CI path-filter (user-validated): .devcontainer/memories/** and .devcontainer/skills/** = CONTENT -> 30s lint-check only. Only boot scripts/devcontainer.json/workflows = infrastructure -> full-build. Never move markdown content into infrastructure. Self-check Persistence = 9a+9b only. +CI path-filter (user-validated): .devcontainer/skills/** and .devcontainer/memories/** are CONTENT -> runtime group, 30s lint-check only. Only boot scripts/.devcontainer.json/workflows = infrastructure -> full-build. Never promote content dirs into infrastructure. Mnemon seed import parses `imported` field (not 'added') from mnemon import JSON. PR-ONLY POLICY: Never commit changes to .devcontainer/memories/MEMORY.md or USER.md on a PR branch — memory-automation artifacts create diffs-vs-main that must be reverted. § Mnemon persistence: fresh Codespace spawns re-seed from .devcontainer/mnemon/seed.json (imported by start-hermes.sh every spawn). The LIVE Mnemon DB is ephemeral and does NOT survive a rebuild. To persist a memory/skill across spawns, write a seed.json entry, not just mnemon_remember. Validate: `mnemon import --dry-run .devcontainer/mnemon/seed.json` -> 'validation passed'. § diff --git a/.devcontainer/mnemon/seed.json b/.devcontainer/mnemon/seed.json index 4edb0ba..4581885 100644 --- a/.devcontainer/mnemon/seed.json +++ b/.devcontainer/mnemon/seed.json @@ -584,6 +584,28 @@ "pixelflux" ], "source": "agent" + }, + { + "content": "Skill: mnemon-graph-export \u2014 Export Mnemon knowledge graph to 3D viewer. Location: .devcontainer/skills/mnemon-graph-export/. Trigger: \"export mnemon graph\", \"regenerate knowledge graph\", \"update 3D viewer\", \"new graph from mnemon\". Usage: cd .devcontainer/skills/mnemon-graph-export/scripts && python3 export_graph.py (fresh graph.json + graph-data.js) && python3 -m http.server 8130 (serve viewer at localhost:8130 or file:// mnemon-graph.html). Viewer loads data at runtime \u2014 data refresh never needs rebuild; only template.html changes require python3 build.py (vendors 3d-force-graph@1.80 with SRI pin). Wiki: .devcontainer/wiki/mnemon-graph-viewer.md.", + "category": "fact", + "importance": 4, + "tags": [ + "skill", + "knowledge-graph", + "mnemon", + "export", + "viewer", + "codespace" + ], + "entities": [ + "mnemon-graph-export", + "export_graph.py", + "build.py", + "mnemon-graph.html", + "graph.json", + "graph-data.js" + ], + "source": "agent" } ] } \ No newline at end of file diff --git a/.devcontainer/skills/mnemon-graph-export/SKILL.md b/.devcontainer/skills/mnemon-graph-export/SKILL.md new file mode 100644 index 0000000..6a1853b --- /dev/null +++ b/.devcontainer/skills/mnemon-graph-export/SKILL.md @@ -0,0 +1,143 @@ +--- +name: mnemon-graph-export +description: "Use when exporting/regenerating the Mnemon knowledge graph." +--- + +# Mnemon Knowledge-Graph Export (3D viewer regeneration) + +Regenerate the 3D knowledge-graph viewer from the latest Mnemon data. The tool +lives in `.devcontainer/skills/mnemon-graph-export/scripts/`; the pipeline is +`export_graph.py -> graph.json (+graph-data.js)`, and the STATIC viewer +(`mnemon-graph.html`) loads the data on open — **data refresh never rebuilds +the viewer**. It works both by double-clicking the HTML (file://, via +`graph-data.js`) and over http (fetch). Plus Mnemon's own `viz` command for +the vis.js fallback. Design rationale: see +`scripts/DESIGN.md`; wiki reference: +`.devcontainer/wiki/mnemon-graph-viewer.md`. + +## Trigger + +User says anything like: "export mnemon graph", "regenerate/show my knowledge +graph", "update the 3D viewer", "new graph from mnemon". + +## Steps (verified end-to-end 2026-08) + +```bash +cd .devcontainer/skills/mnemon-graph-export/scripts + +# 1) Fresh snapshot from the live DB (read-only SQLite) -> graph.json +# (+ graph-data.js, the file://-safe sibling — keep both in sync) +python3 export_graph.py +# -> "Exported N nodes, M edges -> graph.json (+graph-data.js)" + +# 2) DONE — the viewer is a fixed asset; it loads the data on open. +# Double-click mnemon-graph.html (file://, uses graph-data.js) or serve: +python3 -m http.server 8130 --bind 0.0.0.0 # then http://localhost:8130/ (index.html forwards) + +# 3) Commit the refreshed data (viewer only changes when template.html does) +git add graph.json graph-data.js +git -c commit.gpgsign=false commit -m "chore(knowledge-graph): refresh graph from latest mnemon export" +``` + +Rebuild the viewer ONLY when the template (`template.html`) changes — re-vendors +the fg2 library into `mnemon-graph.html`; does not touch data: + +```bash +python3 build.py # first run fetches 3d-force-graph into .cache/ +``` + +Non-default DB/store: `python3 export_graph.py -o out.json`. +Point the viewer at any JSON with `mnemon-graph.html?data=other.json`. + +## Verification (MANDATORY — the user's standard) + +Static greps / `node --check` are NOT verification. Prove it renders: + +1. **file:// mode** (the user-reported failure): copy `mnemon-graph.html` + + `graph-data.js` to a fresh dir, open the HTML via `file://` in a headless + browser — subtitle must read "N memories, M connections" with NO server. + + Verified headless invocation (chromium from playwright cache, no module + needed — full render incl. WebGL): + ```bash + CHROME=~/.cache/ms-playwright/chromium-1208/chrome-linux64/chrome + $CHROME --headless=new --no-sandbox --use-angle=swiftshader \ + --enable-unsafe-swiftshader --virtual-time-budget=15000 \ + --dump-dom file:///abs/path/mnemon-graph.html > dom.html 2>/dev/null + grep -oE 'id="subtitle">[^<]*' dom.html # must be "N memories, M connections" + grep -c '=1 + grep -o 'class="nl"' dom.html | wc -l # must equal N (label pills == nodes) + ``` + For http mode: serve a dir containing ONLY `mnemon-graph.html` + + `graph.json` (no graph-data.js) and dump-dom the http:// URL instead. +2. **http mode**: serve a dir with viewer + `graph.json` (no `graph-data.js`), + open `/mnemon-graph.html` — same subtitle (fetch path). +3. Assert category label pills == node count (`.nl` elements in `#labels`). +4. Move the importance slider to 5: node count drops to importance-5 nodes, + edges stay visible between remaining nodes (the linkVisibility bug). +5. Prove portability: swap the data (or use `?data=other.json`) and confirm + the subtitle changes with NO rebuild. + +If any check fails, debug the viewer (see pitfalls), never ship unverified. + +## Pitfalls (all hit and fixed; do not re-derive) + +1. **file:// fetch blocked** (user-reported "Cannot load graph.json"): browsers + block `fetch()` from `file://`. The viewer loads data via a + `") + + # 3) data — deliberately NOT inlined (portable design): the viewer fetches + # graph.json at load time. Refresh = replace the JSON, no rebuild. + + with open(OUT, "w", encoding="utf-8") as f: + f.write(html) + print(f"Built {OUT} ({os.path.getsize(OUT)/1024:.0f} KB)") + + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/.devcontainer/skills/mnemon-graph-export/scripts/export_graph.py b/.devcontainer/skills/mnemon-graph-export/scripts/export_graph.py new file mode 100644 index 0000000..4d5bf1d --- /dev/null +++ b/.devcontainer/skills/mnemon-graph-export/scripts/export_graph.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python3 +""" +export_graph.py — Export Mnemon knowledge graph to graph.json for the 3D viewer. + +Reads the sqlite DB directly (schema-verified) and emits {nodes, edges, meta}. +Usage: python3 export_graph.py [path-to-mnemon.db] [-o out.json] +""" +import json +import os +import re +import sqlite3 +import sys +from datetime import datetime, timezone + +CATEGORY = ["decision", "context", "fact", "insight", "general"] +SHORT_LEN = 42 + + +def short_label(content: str) -> str: + s = " ".join(content.split()) + # Strip common wiki prefixes for cleaner labels + for p in ("Wiki: ", "CI Debugging "): + if s.startswith(p): + s = s[len(p):] + break + return (s[:SHORT_LEN] + "…") if len(s) > SHORT_LEN else s + + +def main(): + args = sys.argv[1:] + db = os.path.expanduser("~/.mnemon/data/default/mnemon.db") + out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "graph.json") + i = 0 + while i < len(args): + if args[i] == "-o" and i + 1 < len(args): + out = args[i + 1] + i += 2 + elif not args[i].startswith("-"): + db = os.path.expanduser(args[i]) + i += 1 + else: + i += 1 + + con = sqlite3.connect(f"file:{db}?mode=ro", uri=True) + con.row_factory = sqlite3.Row + + rows = con.execute(""" + SELECT id, content, category, importance, effective_importance, + tags, entities, source, created_at, access_count + FROM insights WHERE deleted_at IS NULL + """).fetchall() + + nodes = {} + for r in rows: + cat = r["category"] if r["category"] in CATEGORY else "general" + try: + tags = json.loads(r["tags"]) + except (TypeError, ValueError): + tags = [] + try: + entities = json.loads(r["entities"]) + except (TypeError, ValueError): + entities = [] + nodes[r["id"]] = { + "id": r["id"], + "label": short_label(r["content"]), + "content": r["content"], + "category": cat, + "importance": r["importance"], + "eff": round(r["effective_importance"], 3), + "tags": tags[:8], + "entities": entities[:8], + "source": r["source"], + "created": r["created_at"], + } + + edges = [] + for e in con.execute(""" + SELECT source_id, target_id, edge_type, weight FROM edges + """).fetchall(): + # only edges between live nodes + if e["source_id"] in nodes and e["target_id"] in nodes: + edges.append({ + "source": e["source_id"], + "target": e["target_id"], + "type": e["edge_type"], + "weight": round(e["weight"], 3), + }) + + counts = {} + for r in rows: + counts[r["category"] if r["category"] in CATEGORY else "general"] = \ + counts.get(r["category"] if r["category"] in CATEGORY else "general", 0) + 1 + + data = { + "meta": { + "node_count": len(nodes), + "edge_count": len(edges), + "by_category": counts, + "exported_at": datetime.now(timezone.utc).isoformat(), + "db": os.path.basename(db), + }, + "nodes": list(nodes.values()), + "edges": edges, + } + with open(out, "w") as f: + json.dump(data, f, indent=1) + # file://-safe sibling: a + + + +
+
+ +
+

🧠 Mnemon Knowledge Graph

+
loading…
+
+ +
+
Filters▼
+
+
Categories — click to hide
+
+ +
Min importance
+
+ + 1 +
+ +
Force layout
+
+ Link distance + + 30 +
+
+ Repulsion + + -30 +
+
+ Min distance + + 1 +
+ + +
+ + + +
+
Drag = rotate · scroll = zoom · right-drag = pan
click a node for details
+ +
+
+ +
+
decision
+
context
+
fact
+
insight
+
general
+
+ +
⇅ drag · ✦ scroll · right-drag pan
+ +
+ + + + + \ No newline at end of file diff --git a/.devcontainer/skills/mnemon-graph-export/scripts/template.html b/.devcontainer/skills/mnemon-graph-export/scripts/template.html new file mode 100644 index 0000000..1356d79 --- /dev/null +++ b/.devcontainer/skills/mnemon-graph-export/scripts/template.html @@ -0,0 +1,451 @@ + + + + + +Mnemon Knowledge Graph + + + + + +
+
+ +
+

🧠 Mnemon Knowledge Graph

+
loading…
+
+ +
+
Filters▼
+
+
Categories — click to hide
+
+ +
Min importance
+
+ + 1 +
+ +
Force layout
+
+ Link distance + + 30 +
+
+ Repulsion + + -30 +
+
+ Min distance + + 1 +
+ + +
+ + + +
+
Drag = rotate · scroll = zoom · right-drag = pan
click a node for details
+ +
+
+ +
+
decision
+
context
+
fact
+
insight
+
general
+
+ +
⇅ drag · ✦ scroll · right-drag pan
+ +
+ + + + + \ No newline at end of file diff --git a/.devcontainer/wiki/INDEX.md b/.devcontainer/wiki/INDEX.md index 354c054..6cdb82f 100644 --- a/.devcontainer/wiki/INDEX.md +++ b/.devcontainer/wiki/INDEX.md @@ -13,6 +13,7 @@ | [persistent-memory-proposal.md](persistent-memory-proposal.md) | Proposal for versioning Hermes MEMORY.md / USER.md via a symlink architecture (runtime vs tracked) | architecture, memory, persistence, symlink | | [keepalive-proposal.md](keepalive-proposal.md) | Proposal: Codespace keepalive to mimic client activity and avoid idle shutdown (A: terminal heartbeat, B: /delay-shutdown pinger) | codespace, keepalive, idle-timeout, lifecycle, proposal | | [codespace-lifecycle.md](codespace-lifecycle.md) | Reference: how Codespaces detects idle & shuts down, diagnosing container death, keeping a codespace alive | codespace, lifecycle, idle, keep-alive, shutdown, reference | +| [mnemon-graph-viewer.md](mnemon-graph-viewer.md) | Reference: 3D Mnemon knowledge-graph viewer — pipeline, data model, key design decisions, regeneration | mnemon, knowledge-graph, visualization, 3d-force-graph, tool | | [karpathy-coding-guidelines.md](karpathy-coding-guidelines.md) | Reference: Karpathy's LLM coding-pitfall guidelines — four principles, origin, how they map to Hermes skills | coding, discipline, guidelines, karpathy, reference | | [codespace-gh-auth.md](codespace-gh-auth.md) | Extract real GitHub OAuth token from VS Code server process for API and gh CLI in Codespaces | codespace, github, auth, token, vscode, skill | | [codespace-persistent-symlinks.md](codespace-persistent-symlinks.md) | Whole-folder symlink pattern to persist Hermes memories and skills across Codespace rebuilds | codespace, persistence, symlink, memory, skill | diff --git a/.devcontainer/wiki/mnemon-graph-viewer.md b/.devcontainer/wiki/mnemon-graph-viewer.md new file mode 100644 index 0000000..9a49a44 --- /dev/null +++ b/.devcontainer/wiki/mnemon-graph-viewer.md @@ -0,0 +1,92 @@ +# Mnemon Knowledge-Graph Viewer + +> Reference: how the 3D knowledge-graph viewer works and how to regenerate it. +> Procedure: see skill `mnemon-graph-export`. Design detail: +> `scripts/DESIGN.md`. + +## What it is + +A self-contained, dark-themed 3D visualization of the Mnemon knowledge graph +(the memories in `~/.mnemon/data/default/mnemon.db`). **Portable design: the +viewer is a fixed asset that loads `graph.json` at open time** — refresh the +graph by replacing the data files, never by rebuilding HTML. Works two ways: +**double-click the HTML** (file://, data comes from `graph-data.js`) or **serve +it over http** (data fetched from `graph.json`). + +- `mnemon-graph.html` — static 3D viewer (3d-force-graph v1.80.0 / Three.js), + nodes sized by effective importance, colored by category, HTML category-pill + labels overlaid per frame, integer importance slider (1–5), category toggles, + manual auto-rotate. Loads `window.GRAPH_DATA` (from `graph-data.js`) or + `?data=path.json` / `graph.json` via fetch. +- `graph-data.js` — `window.GRAPH_DATA = {…};`, the file://-safe data sibling. + +## Pipeline (data flow) + +``` +~/.mnemon/data/default/mnemon.db + │ export_graph.py (read-only SQLite + enrichment) + ▼ +graph.json ────────────► mnemon-graph.html (STATIC viewer) +graph-data.js ──────────► file:// (script tag, GRAPH_DATA) +``` + +Load priority in the viewer: `GRAPH_DATA` (script tag) → `?data=` → `graph.json`. +`template.html` is the editable **template** (marker `__FORCE_GRAPH__`); +`build.py` vendors the fg2 library into `mnemon-graph.html` — run it once (or +when the template changes), **never for data refresh**. `index.html` is a tiny +meta-refresh forwarder to `mnemon-graph.html`, so the server root URL works +without knowing the artifact filename. The fg2 bundle is fetched once into +`.cache/` (gitignored; override with `KG_CACHE`). + +## Data model + +- Node: id, 42-char label, content, category (decision/context/fact/insight/ + general, unknown → general), importance 1–5, effective importance, tags, + entities, source, created. +- Edge: source/target ids, type (temporal/semantic/causal/entity), weight. +- Only edges between live (non-deleted) nodes are exported. +- Live DB (2026-08): 69 nodes / 1428 edges; committed graph.json may lag — + regenerate to refresh. + +## Key design decisions (why it looks like this) + +| Decision | Rationale | +|---|---| +| **Viewer loads graph.json at runtime (portable)** | Data/viewer decoupling: refresh = replace data files, never rebuild. Same viewer renders any store/snapshot (`?data=`). Dual load path: `graph-data.js` script tag for file:// double-click, fetch for http — both verified | +| Custom 3D viewer (no vis.js fallback) | User asked for 3D; the vis.js fallback (`mnemon viz` output) was dropped as redundant once the 3D viewer was pixel-verified — one renderer, one pipeline | +| No separate three.js inline | Bundle embeds Three r183 (ESM-only, no UMD); mixing a copy = fatal "Multiple instances of Three.js" crash | +| HTML label overlay, not sprite labels | Sprites would need the THREE copy that crashes; `graph2ScreenCoords()` maps graph→screen per frame, crisp DOM text | +| Manual auto-rotate in rAF loop | The vendored fg2 fork exposes no `.autoRotate()` API (internal OrbitControls only); calling it throws and kills `build()` | +| Integer slider 1–5 | Data importance is integer (2–5); `step="0.1"` showed floats | +| linkVisibility accepts object endpoints | Engine resolves link endpoints to node objects; id-only lookup hid all edges on first filter change (user-reported bug, fixed) | +| **Force layout controls exposed in UI** | Dense graphs squish into a ball; sliders for link distance, repulsion strength, min distance + Reheat button let you tune spread live per dataset — no rebuild, no hardcoded values | +| Verification = real browser render + pixels | Static greps proved file contents but missed blank-page bugs; PIL pixel measurement is the ground truth | +| **Auto force-layout on load** | `computeAutoForces()` derives link distance / charge strength / charge min from canvas size + node/edge counts so the graph spreads instead of blob — but see pitfalls 12-13 below: a leading-dot chain after `;` is a JS SyntaxError that kills the whole script, and duplicate function declarations shadow each other | +| **Bubble size** | fg2 radius = `cbrt(nodeVal) * nodeRelSize`; nodeRelSize(3) keeps bubbles small so the graph reads as spread out, not zoomed-in on a giant sphere | +| **Auto-rotate orbits cluster center** | spinCam rotates around `getGraphBbox()` midpoint with `lookAt(tx,ty,tz)`, not the origin — otherwise the framed view drifts during rotation | + +## Regeneration (quick) + +```bash +cd .devcontainer/skills/mnemon-graph-export/scripts +python3 export_graph.py # fresh graph.json + graph-data.js (the ONLY refresh step) +python3 -m http.server 8130 # optional: serve; or just double-click mnemon-graph.html +``` + +`python3 build.py` only when `template.html` (the template) changes. +Full steps + pitfalls: skill `mnemon-graph-export`, or `scripts/DESIGN.md`. + +## Serving + +`index.html` meta-refreshes to `mnemon-graph.html` — so `http://host:8130/` +just works, no need to know the artifact filename. Never serve `template.html` +as the root (unsubstituted markers → blank page). For file:// double-click, +keep `graph-data.js` next to the viewer (browsers block `fetch()` from +file://, so the script tag is the data path there). + +## Related + +- Skill: [mnemon-graph-export](../skills/mnemon-graph-export/SKILL.md) +- Design: `scripts/DESIGN.md` +- [persistent-knowledge-proposal.md](persistent-knowledge-proposal.md) — how + memory/skills/wiki persist across rebuilds