feat: gateway trace-sink adapter (MCP call traces)

.github/workflows/release.yml

@@ -30,10 +30,26 @@ jobs:
 
       # No lockfile + only optional peer deps → a bare install is enough.
       # The test suite runs via node directly (native TS in node 24).
+      # SPEC-1b-3 D5: contract tests import the REAL @getpipher/armory-gateway
+      # (private repo — token-less clone fails). SIBLINGS_PAT is a per-repo secret.
+      - name: Clone armory-gateway (private sibling — contract tests require it)
+        env:
+          SIBLINGS_PAT: ${{ secrets.SIBLINGS_PAT }}
+        run: git clone --depth 1 https://x-access-token:${SIBLINGS_PAT}@github.com/getpipher/armory-gateway.git ../armory-gateway
+
+      # Bare clone has no node_modules — the contract-test import pulls gateway's
+      # full src tree (client.ts → @modelcontextprotocol/sdk …), so its own deps
+      # must be installed before `npm test`.
+      - name: Install gateway deps (bare clone has none)
+        working-directory: ../armory-gateway
+        run: npm install --ignore-scripts
+
       - run: npm install --ignore-scripts
 
       - name: Run tests
         run: npm test
+        env:
+          ARMORY_GATEWAY_PATH: ${{ github.workspace }}/../armory-gateway
 
       - name: Skip if already published
         id: check

README.md

@@ -94,6 +94,20 @@ Store tests: `npm test` (25/25).
 - Imported/memory files are `0600`. Memory is **local only** — never committed, never synced.
 - **Never put secrets in memory.** Memory text is injected into the system prompt and therefore reaches your model provider — same rule as any context file.
 
+## MCP call traces (armory-gateway integration)
+
+When [`@getpipher/armory-gateway`](https://github.com/getpipher/armory-gateway) is installed in the
+same pi, armory-memory registers a trace sink: every executed MCP tool call appends one
+metadata-only JSONL line to `~/.pi/agent/memory/<cwd-slug>/mcp-traces.jsonl`:
+
+    {"ts":…,"server":"github","tool":"create_issue","ok":true,"durationMs":42,"resultSummary":"ok blocks=1 bytes=120"}
+
+- **Args are never persisted** — the file is metadata only (server, tool, ok, duration, content-free
+  result summary, timestamp). It sits outside the `*.md` injection surface and never reaches a
+  system prompt; open it with the `read` tool when you want it.
+- The file keeps the newest 500 lines (compacted automatically).
+- Without armory-gateway installed, memory behaves exactly as before (no traces, no errors).
+
 ## License
 
 MIT.

docs/SPEC-1b-3-gateway-trace-sink.md

@@ -0,0 +1,36 @@
+# SPEC-1b-3 — Gateway trace sink (armory-memory, PR-2 / D2 + D4 + D5)
+
+> Relocated from the SPEC-1b-3 staging spec (`SPEC-1b-3-memory-todo-adapters.md` §2 Q2, §3.2/§3.3, §5, §7) — armory-memory's half of the memory/todo adapter slice.
+
+## Q2 — Locked decision: metadata-only JSONL, invisible to the memory surface
+
+File `~/.pi/agent/memory/<slug>/mcp-traces.jsonl` (non-`.md` → `listMemory`/`renderMemoryBlock` never see it). Per line: `{ts, server, tool, ok, durationMs, resultSummary, agent?, task?}` — `args` NEVER persisted (not even key names; §3.3). `resultSummary` is already content-free at the tap. Bounded: cap 500 entries, compact to newest 500 when file exceeds 1000 lines (append-fast, occasional O(file) rewrite). No new read surface.
+
+## §3.2 — The injection surface makes trace format a prompt-hygiene choice
+
+`renderMemoryBlock` lists all `*.md` newest-first and INLINES the 3 newest (byte-capped) into every system prompt. A `*.md` trace log would pollute every future prompt (the landmine). Non-`.md` files are invisible to `listMemory` — JSONL is outside the injection surface by construction, while remaining openable via the existing `read` tool.
+
+## §3.3 — Memory is a permanent, auto-injected surface — args must be structurally excluded
+
+Persisting `args` (values or key names) would leak tool-call content into a file class that surfaces in prompts indefinitely, and bloat it. The sink's line serializer simply never touches `input.args` — exclusion is structural (impossible, not policy-by-discipline), enforcing the global "never put secrets in memory" rule at the code level.
+
+## §5 — D2: Memory trace sink (as shipped)
+
+- `src/trace-sink.ts` — pure, pi-independent (node:fs + node:path + memory-store only; unit-tested standalone).
+- `tracesFileFor(cwd)` = `memoryDirFor(cwd) + "/mcp-traces.jsonl"`.
+- `traceToLine(input)`: line shape exactly Q2-A's `{ts, server, tool, ok, durationMs, resultSummary, agent?, task?}` — undefined `agent`/`task` keys omitted; `kind` dropped (constant `"mcp_call"` in v1); **`args` never read by the serializer** (§3.3). `GatewayTraceInput` is a structurally-typed LOCAL interface mirroring gateway's `TraceInput` (structural compatibility — never imported from gateway; the registry contract expects exactly this of suite-locked siblings).
+- `compactTraceFile(file)`: parse-per-line over the newest TRACE_CAP lines — torn/invalid lines (partial writes) dropped, never fatal; rewrite at `0600`.
+- `appendTrace(file, input)`: mkdir + append `0600`, compact check per append; **throws propagate** to the pipeline's fail-open catch+warn (SPEC-1b §7.2 — the sink never swallows).
+- Constants: `TRACE_CAP = 500`, `COMPACT_THRESHOLD = 1000`.
+- `src/gateway-adapter.ts` (D2 registration half): `registerGatewayTraceSink({ cwd, importGateway? })` — guarded dynamic import (specifier never statically imported), absent gateway → `{ registered: false }` silent; registered sink closes over `tracesFileFor(cwd)` and propagates throws.
+
+## §7 — D4 + D5: Linkage & release gates (as shipped)
+
+- **D4** — `test/helpers/gateway-link.mts`: `linkGateway(): string | null` reads `ARMORY_GATEWAY_PATH`; unset → `null` (real-module contract test `t.skip`s with a loud notice naming the env var); set → idempotent `node_modules/@getpipher/armory-gateway` symlink → bare-specifier resolution works under plain node 24. No `package.json` dependency changes (Q4-B — public repo, no `file:` devDep).
+- **D5** — `.github/workflows/release.yml`: the armory-gateway clone step sits BEFORE `npm install`, and the test step exports `ARMORY_GATEWAY_PATH: ${{ github.workspace }}/../armory-gateway`. No continue-on-error — a failed clone fails the release. `SIBLINGS_PAT` is a per-repo secret on getpipher/armory-memory (RECTOR sets it; least privilege, same as fleet's).
+
+## As-built notes
+
+- V1/V2 proven 2026-09-03: bare-specifier symlink resolution under plain node 24, and `?dup=1` two-instance symbol-store convergence (real-module contract test passes with `ARMORY_GATEWAY_PATH` set, 3/3, no skip).
+- Two controller-ratified plan-defect fixes landed in the Task 2 test suite: (1) `memoryDirFor` slugs the FULL cwd CC-style (locked by memory-store.test.mts:40) — path assertions assert via the real composition, not basename joins; (2) the torn-line fixture is newline-terminated (real torn-final-write shape) so the next append lands on its own line.
+- Two deferred minors from review: (1) `compactTraceFile` skips silently on read failure (brief-verbatim "nothing to compact"; unreachable in the normal path — a `console.warn` would match repo fail-open style if wanted later); (2) `traceToLine` key order is insertion-ordered — cross-version byte-level line comparison would break on reorder (tests assert parsed keys, not bytes).

extensions/memory.ts

@@ -152,4 +152,16 @@ export default function (pi: ExtensionAPI) {
       }
     },
   });
+
+  // SPEC-1b-3: register the gateway trace sink (silent skip when
+  // @getpipher/armory-gateway is absent — standalone memory unchanged).
+  pi.on("session_start", async (_event, ctx) => {
+    try {
+      const cwd = (ctx as { cwd?: string } | undefined)?.cwd ?? process.cwd();
+      const { registerGatewayTraceSink } = await import("../src/gateway-adapter.ts");
+      await registerGatewayTraceSink({ cwd });
+    } catch {
+      // gateway absent — standalone degradation
+    }
+  });
 }

package.json

@@ -42,7 +42,7 @@
     ]
   },
   "scripts": {
-    "test": "node test/memory-store.test.mts && node --test test/exports.test.mts"
+    "test": "node test/memory-store.test.mts && node test/trace-sink.test.mts && node test/gateway-adapter.test.mts && node --test test/exports.test.mts"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "*",

src/gateway-adapter.ts

@@ -0,0 +1,31 @@
+// Gateway adapter for armory-memory (SPEC-1b-3 D2): registers a TraceSink against
+// @getpipher/armory-gateway's IoC registry. The specifier is NEVER statically
+// imported — guarded dynamic import keeps public-npm installs standalone.
+// Absent gateway → { registered: false }, silent (the normal public state).
+
+import { tracesFileFor, appendTrace, type GatewayTraceInput } from "./trace-sink.ts";
+
+export interface GatewayModuleLike {
+  registerTraceSink(fn: (input: GatewayTraceInput) => Promise<void>): void;
+}
+
+export interface GatewayAdapterDeps {
+  cwd: string;
+  importGateway?: () => Promise<GatewayModuleLike>;
+}
+
+export async function registerGatewayTraceSink(deps: GatewayAdapterDeps): Promise<{ registered: boolean }> {
+  let gw: GatewayModuleLike;
+  try {
+    gw = await (deps.importGateway ?? (() => import("@getpipher/armory-gateway")))();
+  } catch {
+    return { registered: false };
+  }
+  const file = tracesFileFor(deps.cwd);
+  // The sink never swallows: a throw propagates to the pipeline's fail-open
+  // catch+warn (SPEC-1b §7.2 — inheritance rule untouched).
+  gw.registerTraceSink(async (input) => {
+    await appendTrace(file, input);
+  });
+  return { registered: true };
+}

src/trace-sink.ts

@@ -0,0 +1,88 @@
+// Pure, pi-independent MCP call-trace persistence for armory-memory (SPEC-1b-3 D2).
+//
+// Traces land as JSONL INSIDE the cwd-keyed memory dir but OUTSIDE the .md
+// injection surface: listMemory()/renderMemoryBlock() only touch *.md, so
+// mcp-traces.jsonl never reaches a system prompt. Line shape is metadata-only —
+// the serializer never reads input.args (structural exclusion, not policy).
+//
+// Kept free of any pi/typebox imports so it can be unit-tested standalone.
+
+import { appendFileSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { memoryDirFor } from "./memory-store.ts";
+
+/** Structurally mirrors gateway's TraceInput (never imported from gateway).
+ *  agent/task are forward-compat: gateway doesn't populate them yet (1b-2 §15.1),
+ *  and the serializer omits them while undefined. */
+export interface GatewayTraceInput {
+  kind: "mcp_call";
+  server: string;
+  tool: string;
+  args: Record<string, unknown>;
+  ok: boolean;
+  durationMs: number;
+  resultSummary: string;
+  ts: number;
+  agent?: string;
+  task?: string;
+}
+
+/** Max trace lines kept after a compact. */
+export const TRACE_CAP = 500;
+/** Compact triggers when the file exceeds this many lines. */
+export const COMPACT_THRESHOLD = 1000;
+
+/** The JSONL trace file for a cwd (inside the memory dir, NOT *.md). */
+export function tracesFileFor(cwd: string): string {
+  return join(memoryDirFor(cwd), "mcp-traces.jsonl");
+}
+
+/** Serialize one trace to a JSONL line. Metadata-only: args are structurally
+ *  excluded (this function never touches input.args). */
+export function traceToLine(input: GatewayTraceInput): string {
+  const line: Record<string, unknown> = {
+    ts: input.ts,
+    server: input.server,
+    tool: input.tool,
+    ok: input.ok,
+    durationMs: input.durationMs,
+    resultSummary: input.resultSummary,
+  };
+  if (input.agent !== undefined) line.agent = input.agent;
+  if (input.task !== undefined) line.task = input.task;
+  return JSON.stringify(line);
+}
+
+/** Compact an over-threshold trace file to the newest TRACE_CAP VALID lines.
+ *  Parse-per-line: torn/invalid lines (partial writes) are dropped, never fatal.
+ *  O(file) by design — called from appendTrace only past the threshold check. */
+export function compactTraceFile(file: string): void {
+  let raw: string;
+  try {
+    raw = readFileSync(file, "utf8");
+  } catch {
+    return; // nothing to compact
+  }
+  const lines = raw.split("\n").filter((l) => l.length > 0);
+  if (lines.length <= COMPACT_THRESHOLD) return;
+  const valid: string[] = [];
+  for (const line of lines.slice(-TRACE_CAP)) {
+    try {
+      JSON.parse(line);
+      valid.push(line);
+    } catch {
+      // torn/invalid line — drop
+    }
+  }
+  writeFileSync(file, valid.join("\n") + "\n", { encoding: "utf8", mode: 0o600 });
+}
+
+/** Append one trace. Creates the dir/file on miss (0600). Compact check runs
+ *  per append; the read is O(file) but files are bounded (~150B/line, ≤~150KB
+ *  at threshold) — negligible against MCP-call latency. Throws propagate to the
+ *  pipeline's fail-open catch+warn (SPEC-1b §7.2 — the sink never swallows). */
+export function appendTrace(file: string, input: GatewayTraceInput): void {
+  mkdirSync(dirname(file), { recursive: true });
+  appendFileSync(file, traceToLine(input) + "\n", { encoding: "utf8", mode: 0o600 });
+  compactTraceFile(file);
+}

test/gateway-adapter.test.mts

@@ -0,0 +1,64 @@
+// Gateway-adapter contract tests for armory-memory SPEC-1b-3 (node:test for skip).
+// Run: node test/gateway-adapter.test.mts
+// Real-module tests need ARMORY_GATEWAY_PATH (private sibling; see README dev setup).
+
+import assert from "node:assert/strict";
+import { test } from "node:test";
+import { mkdtempSync, readFileSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { linkGateway } from "./helpers/gateway-link.mts";
+import { memoryDirFor } from "../src/memory-store.ts";
+
+const tmp = mkdtempSync(join(tmpdir(), "armory-mem-adapter-"));
+process.env.ARMORY_MEMORY_ROOT = join(tmp, "pi-memory");
+const cwd = join(tmp, "proj");
+
+const { registerGatewayTraceSink } = await import("../src/gateway-adapter.ts");
+
+const fatInput = {
+  kind: "mcp_call" as const,
+  server: "github",
+  tool: "create_issue",
+  args: { secret: "NEVER PERSISTED" },
+  ok: true,
+  durationMs: 7,
+  resultSummary: "ok blocks=1 bytes=9",
+  ts: 1700000000000,
+};
+
+test("injected fake module: registration fires and the sink persists a metadata-only line", async () => {
+  let received: ((input: unknown) => Promise<void>) | undefined;
+  const fake = { registerTraceSink(fn: (input: unknown) => Promise<void>) { received = fn; } };
+  const out = await registerGatewayTraceSink({ cwd, importGateway: async () => fake });
+  assert.deepEqual(out, { registered: true });
+  assert.equal(typeof received, "function");
+  await received!(fatInput);
+  // memoryDirFor slugs the FULL cwd (CC-compatible layout — Task 2 adjudication); assert via the real composition.
+  const line = JSON.parse(readFileSync(join(memoryDirFor(cwd), "mcp-traces.jsonl"), "utf8").trim());
+  assert.equal(line.server, "github");
+  assert.ok(!JSON.stringify(line).includes("NEVER PERSISTED"), "args never persisted");
+  rmSync(tmp, { recursive: true, force: true });
+});
+
+test("import failure → { registered: false }, no throw", async () => {
+  const out = await registerGatewayTraceSink({ cwd, importGateway: async () => { throw new Error("module absent"); } });
+  assert.deepEqual(out, { registered: false });
+});
+
+test("REAL gateway module: registers through the shared symbol store; dup instance sees it", async (t) => {
+  const gwPath = linkGateway();
+  if (!gwPath) {
+    t.skip("ARMORY_GATEWAY_PATH unset — skipping real-module contract tests (set it to the armory-gateway repo)");
+    return;
+  }
+  const out = await registerGatewayTraceSink({ cwd });
+  assert.deepEqual(out, { registered: true });
+  const sym = Symbol.for("@getpipher/armory-gateway:registry");
+  const store = (globalThis as Record<symbol, { trace?: unknown }> | undefined)![sym];
+  assert.ok(store?.trace, "symbol-store trace slot truthy after registration");
+  // dup-instance convergence (plan V2 pattern, re-pinned from the memory side)
+  const resolved = import.meta.resolve("@getpipher/armory-gateway");
+  const dup = (await import(resolved + "?dup=1")) as { registeredKinds(): { trace: boolean } };
+  assert.equal(dup.registeredKinds().trace, true, "distinct module instance sees the same slot");
+});

test/helpers/gateway-link.mts

@@ -0,0 +1,20 @@
+// PLAN-1b-3 D4: env-gated real-module resolution for contract tests.
+// ARMORY_GATEWAY_PATH unset → null (caller t.skip's with a loud notice).
+// Set → idempotently symlink the gateway repo into node_modules/@getpipher/
+// so the adapter's bare-specifier guarded import resolves (verified: plan V1).
+
+import { existsSync, mkdirSync, symlinkSync } from "node:fs";
+import { dirname, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+
+export function linkGateway(): string | null {
+  const target = process.env.ARMORY_GATEWAY_PATH;
+  if (!target) return null;
+  const abs = resolve(target);
+  if (!existsSync(abs)) return null;
+  const pkgDir = join(resolve(dirname(fileURLToPath(import.meta.url))), "..", "..", "node_modules", "@getpipher");
+  mkdirSync(pkgDir, { recursive: true });
+  const link = join(pkgDir, "armory-gateway");
+  if (!existsSync(link)) symlinkSync(abs, link, "dir");
+  return abs;
+}