GS-MEM

Sovereign, self-hostable memory engine. A SQLite-backed page store with a bi-temporal knowledge graph, full-text search, an MCP server for AI agents, an HTTP REST API, and a web UI — all in pure Rust, offline, with no external API.

Extracted from the NANTAR framework's immortal-gmem crate into a standalone project.

Created by Galo Serrano Abad — NANTAR AI ROBOTICS.

What it is

GS-MEM stores durable pages (compiled markdown knowledge) and SPO triples (a bi-temporal knowledge graph), and exposes them three ways:

• MCP server (gs-mem-mcp) — JSON-RPC 2.0 over stdio with page, source, recall, diary, search, and temporal graph tools. A drop-in memory for Claude Desktop / Claude Code / Codex.
• HTTP API (gs-mem-server) — an axum REST API over the same tools.
• Web UI — a React SPA (pages, search, page editor, knowledge-graph view), embedded directly in the server binary.

"Sovereign" means: pure-Rust inference (bundled MiniLM-L6-v2 weights for the on-device embedder), bundled SQLite (no system dependency), no calls home.

Features

• Expanded memory tools: page CRUD, broad source import, recall, diary, source map/status, maintenance doctor/repair, search/skim, backlinks, and temporal SPO graph operations.
• Repository-local memory corpus: the tracked memory/ tree is the canonical markdown source for this GS-MEM checkout and can be embedded in another project as that project's durable memory source.
• Rust-native source import: project/code/conversation/document text files, gitignore-aware scanning, source fingerprints, chunk provenance, line ranges, wing/room taxonomy, tags, unchanged-file skip gates, stale-source pruning, and per-source import locks with stale-lock recovery.
• Explicit document extraction: .docx, .pptx, .xlsx, .rtf, and .epub are extracted locally when --extract-documents is set; .pdf extraction is controlled behind GS_MEM_ENABLE_EXTERNAL_DOC_EXTRACT=1.
• Recall workflow: recall returns a search-before-answer bundle combining retrieval hits and recent diary entries; Codex hook scripts can auto-capture stdin into diary entries and run wake recall.
• Skim-then-expand (skim): a compact map of the query-relevant corpus region — one-line page summaries plus backlink-graph neighbors that need not match the query lexically — so an agent can orient globally, then expand only the slugs it needs via get_page. Pure-Rust, no model required.
• Graph navigation: bounded traversal from any page/entity across wiki-style links and temporal SPO triples, with incoming/outgoing/both direction, depth caps, closed-edge inclusion, page/entity node metadata, and provenance-rich edges.
• Bi-temporal SPO knowledge graph + wiki-style [[backlinks]], with direction/as-of queries, invalidation, timelines, stats, and provenance.
• Entity registry + fact checker: canonical people/project/org names, aliases, provenance, similar-name warnings, and temporal-graph checks for entity confusion, contradictions, and stale historical facts.
• Operational web UI: pages/search/edit, recall bundles, source import/map reports, diary read/write, graph navigation, temporal triple query, retrieval eval reports, and an Ops control plane for sync, backup, governance, consolidation, temporal history, eval operations, and distribution checks.
• Always-on sync operations: durable sync sources, queued imports, retry bookkeeping, due-run scheduling, per-source health, and a Windows scheduled task installer for gs-mem sync watch.
• Portable backup/restore: .gsmbundle ZIP format with manifest validation for pages, chunks, triples, entities, source versions, and vector identity.
• Governance and consolidation: import/query policies, privacy labels, retention application, redaction queue, mutation audit log, duplicate page detection, entity merge proposals, alias conflicts, and stale source previews.
• Temporal page history: previous page versions are captured before updates; CLI/HTTP/UI can diff or restore those versions and inspect source drift.
• Search: hybrid retrieval — persistent Tantivy BM25 page+chunk index with title/tag/body field boosts, phrase-aware queries, chunk snippets and provenance + HNSW vector search (sovereign MiniLM-L6-v2 embedder), fused with Reciprocal Rank Fusion. Degrades gracefully to SQLite keyword-only when no model or BM25 index is resolvable. Backfill pre-existing pages with gs-mem rebuild-search and vectors with gs-mem-mcp reindex.
• Retrieval Quality Lab: named benchmark runners for LongMemEval, LoCoMo, ConvoMem, MemBench, and custom suites with held-out/regression splits, per-category and per-language recall, persisted run history, and CI recall gates.
• Advanced recall ranking: prompt-contamination sanitizer with telemetry, candidate union across hybrid search and closet pointers, quoted-phrase, proper-name, preference/nostalgia, and temporal boosts, optional deterministic reranking, and failure-category reports.
• Closet pointer index: a compact topic-pointer layer over chunks with exact source slug/chunk/line ranges, matched_via, source purge/rebuild, and neighbor hydration for fast second-pass recall.
• Adapter SDK: Rust-native adapter trait/specs, process/WASM plugin manifest validation, adapter version/privacy/freshness/id declarations, and byte-preserving vs declared-lossy conformance checks.
• Metadata-only source mode: import path descriptions as memory without reading file contents, useful for large binary trees, generated/vendor dirs, and architecture landmarks.
• Deployable storage abstraction: SQLite/Tantivy remains the default; the backend doctor validates explicit opt-in configs for feature-gated pgvector, Qdrant, and LanceDB/object-store style deployments.
• Git intelligence: local git history import creates commit pages and KG triples for commit subjects and changed modules, including branch/remote context.
• Security and recovery hardening: token/secret/PII scanners with redaction proposals and import-time quarantine, backup-before-repair, Unicode/reparse checks, line-ending checks, stale metadata checks, and index invalidation diagnostics.
• Agent registry and release evidence: dedicated agent profiles, scoped recall and diaries, multilingual/model evaluation reports, client ecosystem packaging, and machine-generated release evidence with artifact hashes, benchmark counts, gate commands, and claims backed by tests.
• Memory transaction engine: idempotent transaction begin, append-only mutation log, replay, page rollback, and snapshot checkpoints for memory mutations. put_page can record before/after page+chunk state under a transaction id.
• Project embedding kit: gs-mem init-project-memory creates a portable memory/ profile inside any repo with config, adapter presets, ignore rules, MCP profile, hooks, and an import-proof manifest.
• Provenance receipts: recall can return end-to-end evidence explaining source adapter, source file, chunk span, entity match, closet pointer match, ranking boosts, freshness, privacy label, and stale warnings.
• Privacy-preserving vector lifecycle: privacy-scoped vector namespaces, vector tombstones, embedding-profile migration records, and forced rebuild requests after redaction quarantine.
• Policy DSL, extractor isolation, OpenAPI/SDKs, federated sync, and proof: governance rules can be parsed/dry-run/applied; risky extractors run behind bounded isolation; OpenAPI plus Rust/TypeScript/Python clients are generated; signed .gsmsync bundles support portable peer sync; gs-mem proof emits a machine-readable gate/evidence report.
• One self-contained server binary (UI embedded) plus a Docker image.

Architecture (Cargo workspace)

Crate	Role
crates/gs-mem-core (lib gsmem)	storage, search, graph, embedder, tool logic, optional mcp feature
crates/gs-mem-cli	local CLI for init, import, query, triples, and MCP serving
crates/gs-mem-mcp	stdio MCP server binary
crates/gs-mem-server	axum HTTP API + embedded web UI
crates/immortal-nn, crates/immortal-tokenizer	vendored sovereign NN + WordPiece tokenizer
frontend/	React + Vite web UI

Quickstart

Docker

docker compose -f docker/docker-compose.yml up --build
# open http://localhost:8088

From source

Prerequisites: Rust 1.75+, Node 20+.

# 1. build the web UI (embedded into the server at compile time)
npm ci --prefix frontend && npm run build --prefix frontend
# 2. run the HTTP server + UI
cargo run --release -p gs-mem-server      # http://localhost:8088

As an MCP server (Claude Desktop / Claude Code / Codex)

cargo build --release -p gs-mem-mcp

Then point your MCP client at the built binary:

{
  "mcpServers": {
    "gs-mem": {
      "command": "/abs/path/to/target/release/gs-mem-mcp",
      "env": { "IMMORTAL_GMEM_DATA_DIR": "/abs/path/to/gsmem-data" }
    }
  }
}

Local CLI

cargo run --release -p gs-mem-cli -- init
cargo run --release -p gs-mem-cli -- import memory
cargo run --release -p gs-mem-cli -- import memory --prune
cargo run --release -p gs-mem-cli -- import docs --extract-documents
cargo run --release -p gs-mem-cli -- search "dispatch safety" -n 5
cargo run --release -p gs-mem-cli -- rebuild-search
cargo run --release -p gs-mem-cli -- graph navigate GS-MEM --depth 2 --direction both
cargo run --release -p gs-mem-cli -- recall "recent source import work" -n 8
cargo run --release -p gs-mem-cli -- diary read -n 5
cargo run --release -p gs-mem-cli -- source-adapters
cargo run --release -p gs-mem-cli -- source-map
cargo run --release -p gs-mem-cli -- doctor
cargo run --release -p gs-mem-cli -- repair
cargo run --release -p gs-mem-cli -- validate-index
cargo run --release -p gs-mem-cli -- check-vector-identity
cargo run --release -p gs-mem-cli -- entity upsert GS-MEM project --alias gsmem
cargo run --release -p gs-mem-cli -- entity resolve "GS MEM"
cargo run --release -p gs-mem-cli -- fact-check GS-MEM status active
cargo run --release -p gs-mem-cli -- eval eval-cases.json -k 5
cargo run --release -p gs-mem-cli -- quality run longmem-smoke eval-cases.json --benchmark longmemeval --split heldout --threshold 0.8
cargo run --release -p gs-mem-cli -- quality gate longmem-smoke --benchmark longmemeval --split heldout --threshold 0.8
cargo run --release -p gs-mem-cli -- rank "Where did Alice say \"blue notebook\"?"
cargo run --release -p gs-mem-cli -- closet rebuild
cargo run --release -p gs-mem-cli -- closet search "blue notebook"
cargo run --release -p gs-mem-cli -- adapter conformance markdown_corpus
cargo run --release -p gs-mem-cli -- metadata memory "Canonical project memory corpus"
cargo run --release -p gs-mem-cli -- storage-backends
cargo run --release -p gs-mem-cli -- git-intel .
cargo run --release -p gs-mem-cli -- clients
cargo run --release -p gs-mem-cli -- recovery backup --label before-repair
cargo run --release -p gs-mem-cli -- recovery doctor
cargo run --release -p gs-mem-cli -- secrets scan --path .env --queue
cargo run --release -p gs-mem-cli -- model-eval multilingual-cases.json --profile local-minilm
cargo run --release -p gs-mem-cli -- agent upsert codex-executor "Codex Executor" --scope people --scope projects
cargo run --release -p gs-mem-cli -- release-evidence release-claims.json
cargo run --release -p gs-mem-cli -- transaction begin --actor codex --reason "batch import" --idempotency-key import-1
cargo run --release -p gs-mem-cli -- transaction replay tx-id
cargo run --release -p gs-mem-cli -- init-project-memory . --profile codex
cargo run --release -p gs-mem-cli -- provenance "Where is the quoted evidence?"
cargo run --release -p gs-mem-cli -- vector namespace private-vectors private --encrypted --embedding-profile minilm-l6-v2
cargo run --release -p gs-mem-cli -- policy apply policies.gsmem
cargo run --release -p gs-mem-cli -- extractor builtin-text README.md --max-bytes 65536
cargo run --release -p gs-mem-cli -- consolidation enqueue --stale-root memory
cargo run --release -p gs-mem-cli -- sdk target/gs-mem-sdk
cargo run --release -p gs-mem-cli -- federated export atlas.gsmsync atlas --slug projects/atlas --source-id memory/projects/atlas.md --signing-key local-key
cargo run --release -p gs-mem-cli -- federated validate atlas.gsmsync --signing-key local-key
cargo run --release -p gs-mem-cli -- proof --repo-root . --min-recall 0.8
cargo run --release -p gs-mem-cli -- eval-suite save smoke eval-cases.json
cargo run --release -p gs-mem-cli -- eval-gate smoke --threshold 0.8
cargo run --release -p gs-mem-cli -- sync add project memory --interval-secs 300 --prune --wing project --privacy-label project
cargo run --release -p gs-mem-cli -- sync queue project
cargo run --release -p gs-mem-cli -- sync run-next
cargo run --release -p gs-mem-cli -- backup export gs-mem.gsmbundle
cargo run --release -p gs-mem-cli -- governance policy-upsert private private private --allow-import false --allow-query false
cargo run --release -p gs-mem-cli -- consolidate --stale-root memory
cargo run --release -p gs-mem-cli -- temporal page-history projects/alpha
cargo run --release -p gs-mem-cli -- corpus-origin --text "User: remember this"
cargo run --release -p gs-mem-cli -- line extract --path README.md --line-start 1 --line-end 5
cargo run --release -p gs-mem-cli -- spellcheck user-text --text "pleese rember this"
cargo run --release -p gs-mem-cli -- collision-scan --path proposed-chunks.json
cargo run --release -p gs-mem-cli -- topology rooms memory
cargo run --release -p gs-mem-cli -- topology compute-hallways --min-count 1
cargo run --release -p gs-mem-cli -- wake-bundle --query "project memory" --limit 10
cargo run --release -p gs-mem-cli -- instructions codex --lang en
cargo run --release -p gs-mem-cli -- i18n languages
cargo run --release -p gs-mem-cli -- doctor --profile generic
cargo run --release -p gs-mem-cli -- get users/galo/dispatch-safety-honesty-doctrine

The CLI also exposes put, list, backlinks, triples, graph, skim, corpus-origin, line, spellcheck, collision-scan, topology, wake-bundle, instructions, i18n, status, and serve. It is the replacement for the useful gmem command surface from G-Memory-rs.

doctor inspects real brain.db and vector-index state: orphaned source_versions, source-backed pages missing skip-gate rows, stale vector ids, missing page/chunk vectors, and vector identity drift. repair removes repairable orphan/stale records, backfills missing vectors when the embedder is available, and pins a previously unpinned vector index to the active embedder.

Memory corpus

The memory/ directory is intentionally tracked. scripts/import-memory.ps1 uses it as the default source, and gs-mem import memory indexes it through the same Rust-native source importer used for project/code/document ingestion. To embed GS-MEM in another repository, vendor or copy this directory and import it as that project's single durable memory source.

Use --prune (or .\scripts\import-memory.ps1 -Prune) when the source tree is authoritative and deleted/moved files should remove their old imported pages, chunks, source versions, current links, and vector entries.

Codex-friendly deployment

On machines where CARGO_TARGET_DIR is set, Cargo release binaries may not land under this repo's target/. Use the deployment scripts to produce stable project-local binaries and client config:

.\scripts\deploy-mcp.ps1
.\scripts\import-memory.ps1
bin\gs-mem.exe doctor
.\install\install-codex.ps1     # Codex CLI / IDE extension
.\install\install-codex-hooks.ps1
.\install\install-claude.ps1    # Claude Code (user scope, idempotent)
.\install\install-cursor.ps1
.\install\install-sync-task.ps1 -DeployFirst
.\install\migrate-config.ps1 -ToVersion 0.1.0
bin\gs-mem-mcp.exe reindex      # one-time vector backfill after imports
bin\gs-mem-mcp.exe rebuild-search

The scripts refresh bin\gs-mem-mcp.exe and bin\gs-mem.exe, use %LOCALAPPDATA%\gs-mem as the live data store, and keep the old %LOCALAPPDATA%\gmem store untouched as rollback.

Ready-made per-client snippets (Codex, Claude Code, Claude Desktop, Cursor, Copilot-compatible stdio, and generic MCP) live in clients/. On Windows, scripts/deploy-mcp.ps1 builds the binary, copies it to the stable bin\ path those snippets reference, and smoke-tests it over stdio.

HTTP API

Method & path	Tool
GET /api/health	liveness → {"status":"ok"}
GET /api/pages?tag=&limit=	list_pages
GET /api/pages/{slug}	get_page
PUT /api/pages/{slug}	put_page (body: { body, title?, scope?, frontmatter? })
GET /api/search?q=&limit=	search
POST /api/search/index/rebuild	search_index_rebuild
GET /api/graph/navigate/{start}?depth=&direction=&include_triples=&include_closed=&limit=	graph_navigate
GET /api/recall?q=&limit=	recall
GET /api/skim?q=&limit=&depth=&neighbors=	skim
POST /api/source/import	source_import
GET /api/source/adapters	source_adapters
GET /api/source/map	source_map
GET /api/maintenance/doctor	maintenance_doctor
POST /api/maintenance/repair	maintenance_repair
GET /api/maintenance/vector-identity	check_vector_identity
GET /api/diary?limit= / POST /api/diary	diary_read / diary_write
GET /api/pages/{slug}/backlinks	backlinks
GET /api/triples?subject= | ?object=	triples_find
POST /api/triples	triples_add (body: { subject, predicate, object, confidence? })
POST /api/triples/query	triples_query
POST /api/triples/invalidate	triples_invalidate
GET /api/triples/timeline/{entity}	triples_timeline
GET /api/triples/stats	triples_stats
GET /api/entities?limit= / POST /api/entities	entity_list / entity_upsert
GET /api/entities/resolve?name=	entity_resolve
POST /api/fact-check	fact_check
POST /api/quality/run / GET /api/quality/history / POST /api/quality/gate	quality lab
POST /api/rank / POST /api/query/sanitize	advanced ranking and sanitizer telemetry
POST /api/closet/rebuild / POST /api/closet/search / POST /api/closet/purge	closet pointer index
GET /api/adapters/sdk / POST /api/adapters/conformance	adapter SDK and conformance
POST /api/metadata/import	metadata-only source import
GET /api/storage/backends	storage backend doctor
POST /api/git/import	git intelligence import
GET /api/clients	client ecosystem profile doctor
POST /api/recovery/backup / GET /api/recovery/doctor	recovery hardening
POST /api/secrets/scan	secrets scanner and redaction proposals
POST /api/model/eval	multilingual/model recall evaluation
GET /api/agents / POST /api/agents / POST /api/agents/diary / POST /api/agents/recall	agent registry
POST /api/release/evidence	machine-generated release evidence
POST /api/transactions/begin / GET /api/transactions/{id}/replay / POST /api/transactions/{id}/checkpoint / POST /api/transactions/{id}/rollback	memory transaction engine
POST /api/project-memory/init	embeddable project memory/ profile kit
POST /api/provenance/recall	recall bundle with evidence-chain receipts
POST /api/vectors/namespaces / POST /api/vectors/tombstones / POST /api/vectors/migrate / POST /api/vectors/quarantine-rebuild	vector lifecycle governance
POST /api/policy/dsl/dry-run / POST /api/policy/dsl/apply	governance policy DSL
POST /api/extractors/builtin-text / POST /api/extractors/run	bounded extractor isolation
POST /api/consolidation/proposals/enqueue / GET /api/consolidation/proposals / POST /api/consolidation/proposals/review	active consolidation review queue
GET /api/openapi.json / POST /api/sdk/export	OpenAPI and generated SDK clients
POST /api/federated/export / POST /api/federated/validate / POST /api/federated/restore / POST /api/federated/conflicts	signed portable federated sync
POST /api/proof	schema/adapter/index/recall/non-hollow/docs proof report
POST /api/eval/retrieval	retrieval_eval
GET /api/eval/suites / POST /api/eval/suites	eval_suite_list / eval_suite_save
POST /api/eval/suites/run	eval_suite_run
GET /api/eval/history / POST /api/eval/gate / GET /api/eval/dashboard	eval ops
POST /api/sync/sources / POST /api/sync/queue / POST /api/sync/run-next / POST /api/sync/run-due / GET /api/sync/health	sync operations
POST /api/backup/export / POST /api/backup/validate / POST /api/backup/restore	backup bundle operations
GET /api/governance/policies / POST /api/governance/policies	governance policies
GET /api/governance/redactions / POST /api/governance/redactions / GET /api/governance/audit	redaction queue and audit
POST /api/governance/retention/apply	governance_apply_retention
POST /api/consolidation/analyze	consolidation report
GET /api/temporal/page-history/{slug} / POST /api/temporal/pages/diff / POST /api/temporal/pages/restore	page history
GET /api/temporal/triples/{entity} / GET /api/temporal/source-drift	temporal graph/source drift
GET /api/distribution/doctor?profile=	distribution profile doctor
POST /api/corpus-origin/detect	AI-dialogue corpus origin detection
POST /api/line/render / POST /api/line/extract / POST /api/line/pointer	virtual line numbering and source pointers
POST /api/spellcheck/user-text / POST /api/spellcheck/transcript	conservative user-text/transcript typo correction
POST /api/collision-scan	page/chunk/vector/metadata collision scan
POST /api/topology/rooms / POST /api/topology/hallways/compute / GET /api/topology/hallways	palace room detection and hallway graph
POST /api/topology/hallways/potentiate / POST /api/topology/hallways/decay / POST /api/topology/hallways/delete	hallway strengthening, decay, and deletion
POST /api/memory/wake-bundle	layered identity/importance/scope/recall wake bundle
POST /api/instructions/pack / GET /api/i18n/languages / POST /api/i18n/translate / POST /api/i18n/entity-patterns	localized client instruction packs
GET /api/status	status

Configuration

Env var	Default	Purpose
IMMORTAL_GMEM_DATA_DIR	OS data dir /immortal-gmem	location of brain.db + the search index
IMMORTAL_GMEM_ROOT	$OneDrive or CWD	markdown corpus root (for ingestion)
IMMORTAL_GMEM_MODEL_DIR	path search	sovereign embedder model directory
GS_MEM_PORT	8088	HTTP server port
RUST_LOG	gs_mem_server=info	tracing filter

For this Windows deployment, set IMMORTAL_GMEM_DATA_DIR explicitly to %LOCALAPPDATA%\gs-mem in client configs.

Development

cargo nextest run --workspace        # Rust tests
npm run dev --prefix frontend        # Vite dev server (proxies /api → :8088)

The core crate enforces an "enterprise doctrine" test suite (no anyhow/log, #[non_exhaustive] public enums, no unsafe, no public Result<_, String>). Please keep new core code compliant.

License

GS-MEM is dual-licensed — see LICENSING.md:

• AGPL-3.0-only — free & open source. Self-host, modify, and redistribute under the AGPL's copyleft + network-source-disclosure terms.
• Commercial license — for closed-source/proprietary or SaaS use without AGPL obligations. Contact hello@serragi.com.

Model weights and vendored components are attributed in NOTICE.