Skip to content

feat: core kit — trim to the core engine + honest README + tests that run - #21

Merged
totobusnello merged 3 commits into
mainfrom
feat/core-kit
Jun 16, 2026
Merged

totobusnello merged 3 commits into
mainfrom
feat/core-kit

Conversation

@totobusnello

Copy link
Copy Markdown
Owner

Pre-launch "core kit" pass. Trims the public package to the core memory engine, makes the test suite actually run, fixes latent scrub bugs, and aligns the README with reality. Source of truth is the VPS, so trimming here loses nothing.

Trim (D1 — core kit)

Removed enterprise/nicho subsystems (leaves only; reverse-dep scan confirmed no core dependency): viewer, observability, conflict, confidence, OCR (+@google-cloud/documentai), eval harness, hooks/plugins, notion-sync, telemetry, archive/encryption, privacy-br. Kept core: ingest/search (BM25+vec+RRF)/embed, salience, KG, reflect + /api/answer, /api/brief, memory ops, op-audit, generic privacy, regex-extract, CLI/MCP/HTTP.

  • Tarball: 459 → 111 files (−76%), ~597KB → 245KB.

Tests now run (were a silent no-op)

tsconfig excluded tests but the script globbed dist/__tests__ → 0 tests, false-green CI. Fixed with tsconfig.test.json + real test script. Result: 590 tests, 586 pass, 0 fail, 4 skip (skips = E2E key-gated). declaration:false (drops 228 unused .d.ts). npm audit fix (4 high). @xenova/transformers → optionalDependency.

Latent bugs fixed (surfaced by enabling tests)

  • kg_relations.relation_reason + kg_entities.attributes missing on clean install → kg-extract/impact/spo crashed no such table/column. Materialized canonical KG schema in migrateToV8.
  • PRAGMA user_version now always synced in ensureSchema.

README honesty (D2 + D4)

  • Multi-provider: state what's real (Gemini + any OpenAI-compatible, both LLM & embeddings) vs interface-only stubs (anthropic/voyage). Removed the Anthropic example that throws. Added scope note (KG/consolidate/digest/query-expansion still call Gemini directly).
  • Numbers: footnotes qualifying LoCoMo (retrieval vs Mem0 F1), LongMemEval 1.0 (oracle retrieval ceiling, not inference; ~68% standalone), KG-path 769× ($0 SQL path vs Mem0 Cloud incl. inference), <$11/mo (Hostinger + AI Studio free tier), +78.8% (internal baseline).
  • Badges: "oracle validated" → "retrieval ceiling"; tests badge → "100% / tests pass" (real now); de-personalized npm README (NOX_AGENTS default).

Deferred (not blockers): REPRODUCE.md (gate for the HN/Reddit post, done with the campaign), full provider routing of the remaining Gemini paths.

totobusnello and others added 3 commits June 16, 2026 17:43
Ship only the core memory engine. Removes leaf subsystems (confirmed
non-load-bearing by reverse-dep scan) and unwires them from entrypoints.

Removed: viewer/SSE, observability instrumentation dir, conflict, confidence,
OCR, eval harness, hooks/plugins, notion-sync, archive/encryption, privacy-br
(LGPD), telemetry/shadow-tracker, and the server-deps wiring for those.

Entrypoints unwired (core routes/commands/tools preserved):
- api-server.ts: dropped evals/telemetry/shadow routes + static dashboard +
  eval-metrics route; kept obs health/recent-ops/canary-tail + all core routes.
- wire-up.ts: kept POST /api/answer only.
- index.ts: removed sync-notion + notion calls in consolidate/retry-failed.
- mcp-server.ts: unchanged (no removed tools were registered).
- lib/ingest-router.ts: removed OCR probe/enqueue branch (kept entity/markdown
  dispatch); neutralized pdf-scanned kind.

Deps: removed @google-cloud/documentai (OCR-only); moved @xenova/transformers
to optionalDependencies (reranker is opt-in, kept).

Build/test: tsconfig declaration:false; added tsconfig.test.json; test script
now compiles + runs tests (was a silent no-op). Updated OpenAI provider
conformance expectations (now a live provider → MissingKeyError, not
NotImplementedError); fixed .ts->.js test import specifiers; portable tmp dirs;
lazy Gemini provider construction so import is key-free.

Tarball: 597.8kB/459 files -> 245.5kB/111 files.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ser_version

Closes the 5 latent (scrub-induced) test failures surfaced once tests
actually compile/run. Same pattern as the kg_entities.attributes fix.

FIX 1 — kg_relations.relation_reason (edge-typing, 4 fails):
- knowledge-graph.ts: add `relation_reason TEXT DEFAULT 'unknown'` to the
  kg_relations CREATE TABLE; idempotent ALTER + backfill + CREATE INDEX
  idx_kg_relations_reason for pre-existing DBs.
- db.ts migrateToV8: materialize the canonical KG schema (kg_entities +
  kg_relations with relation_reason + indexes) so a clean install's getDb()
  always provides it. Without this, impact.ts / lib/spo-injection.ts / kg-*
  crash with "no such table: kg_relations" on a fresh DB before any KG write
  (ensureGraphTables ran lazily only on KG ops). Restores the column the
  consumers read instead of cutting them.

FIX 2 — pragma-alignment (1 fail):
- db.ts ensureSchema: set `PRAGMA user_version = SCHEMA_VERSION` on every call,
  including the early-return path (meta.schema_version is source of truth).
  Covers recovery DBs whose user_version was manually reset to 0 — re-open
  re-bumps the PRAGMA without reprocessing migrations. Idempotent.

Result: npm test 586 pass / 0 fail / 4 skipped (E2E/key-gated + vec0-absent).
Build clean. No README/claims/version changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Multi-provider: state what's real (gemini + any OpenAI-compatible, LLM & embeddings) vs interface-only (anthropic/voyage). Remove the Anthropic example that throws NotImplementedError. Add scope note: KG/consolidate/digest/query-expansion still call Gemini directly.
- Numbers: add footnotes qualifying LoCoMo (retrieval metric vs Mem0 F1), LongMemEval 1.0 (oracle retrieval ceiling, not inference; ~68% standalone), KG-path 769x ($0 SQL path vs Mem0 Cloud incl. inference), <$11/mo (Hostinger + AI Studio free tier), +78.8% (vs internal baseline).
- Badges: 'oracle validated' -> 'retrieval ceiling'; tests badge label -> 'tests pass' (586 passing/0 failing, now that tests actually run); aria/title updated.
- De-personalize: drop persona names + /root from NOX_AGENTS default in npm README. Genericize schema-init line.
@totobusnello
totobusnello merged commit 05dbfcc into main Jun 16, 2026
1 check passed
@totobusnello
totobusnello deleted the feat/core-kit branch June 16, 2026 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant