perf: SQLite manifest store — replace per-file YAML sidecars (#ALP-912)

CONTRIBUTING_LANGUAGE.md

@@ -160,16 +160,22 @@ formatPort :: Int -> String
 formatPort p = ":" <> show p
 ```
 
-### Step 7 — Generate the sidecar
+### Step 7 — Generate the index
 
 ```bash
-cargo run -- sidecar fixtures/sample.hs
+cargo run -- generate fixtures/
 ```
 
-This creates `fixtures/sample.hs.fmm`. Inspect it and verify:
+Then query the index to verify:
 
-- `exports:` lists the symbols you expected
-- `imports:` and `dependencies:` are correct
+```bash
+cargo run -- outline fixtures/sample.hs
+```
+
+Check that:
+
+- Exports list the symbols you expected
+- Imports and dependencies are correct
 - No obvious omissions or false positives
 
 If the output looks wrong, go back to Step 3 and adjust your extraction logic.
@@ -214,7 +220,7 @@ just test
 [ ] 4. pub mod <lang>; added to src/parser/builtin/mod.rs
 [ ] 5. Parser registered in ParserRegistry::register_builtin() in src/parser/mod.rs
 [ ] 6. fixtures/sample.<ext> created with 50-150 LOC of real code
-[ ] 7. cargo run -- sidecar fixtures/sample.<ext> succeeds and output looks correct
+[ ] 7. cargo run -- generate fixtures/ && cargo run -- outline fixtures/sample.<ext> shows correct exports
 [ ] 8. validate_<lang>_fixture() test added to tests/fixture_validation.rs; just test passes
 ```
 

Cargo.toml

@@ -51,9 +51,11 @@ ctrlc = "3.4"
 # Parallelism
 rayon = "1.10"
 
+# Database
+rusqlite = { version = "0.32", features = ["bundled"] }
+
 # Serialization
 serde = { version = "1.0", features = ["derive"] }
-serde_yaml = "0.9"
 serde_json = "1.0"
 
 # Date/time
@@ -80,6 +82,7 @@ indexmap = { version = "2", features = ["serde"] }
 [dev-dependencies]
 tempfile = "3.14"
 criterion = { version = "0.5", features = ["html_reports"] }
+rusqlite = "0.32"
 
 [[bin]]
 name = "fmm"

README.md

@@ -10,36 +10,22 @@
 npx frontmatter-matters init
 ```
 
-|                          | Without fmm                                 | With fmm                                       |
-| ------------------------ | ------------------------------------------- | ---------------------------------------------- |
-| **How LLM navigates**    | grep → read entire file → summarize, repeat | Read sidecar metadata → open only needed files |
-| **Tokens for 500 files** | ~50,000                                     | ~2,000                                         |
-| **Reduction**            | —                                           | **88-97%**                                     |
+|                          | Without fmm                                 | With fmm                                         |
+| ------------------------ | ------------------------------------------- | ------------------------------------------------ |
+| **How LLM navigates**    | grep → read entire file → summarize, repeat | Query SQLite index → open only needed files      |
+| **Tokens for 500 files** | ~50,000                                     | ~2,000                                           |
+| **Reduction**            | —                                           | **88-97%**                                       |
 
 ## What it does
 
-fmm generates a `.fmm` sidecar file alongside each source file. The sidecar is a tiny YAML file listing exports, imports, dependencies, and line count:
+fmm maintains a single SQLite database (`.fmm.db`) at the project root. Each indexed file contributes its exports, imports, dependencies, and line count. The database supports incremental updates — only changed files are re-parsed.
 
 ```
-src/auth/session.ts      ← 234 lines of source
-src/auth/session.ts.fmm  ← 7 lines of metadata
+src/auth/session.ts   ← 234 lines of source
+.fmm.db               ← single index for the entire project
 ```
 
-```yaml
----
-file: src/auth/session.ts
-fmm: v0.3+0.1.12
-exports:
-  createSession: [12, 45]
-  validateSession: [47, 89]
-  destroySession: [91, 110]
-imports: [jsonwebtoken]
-dependencies: [../config, ../db/users]
-loc: 234
-modified: 2024-01-15
-```
-
-LLMs read sidecars first, then open source files only when they need to edit.
+LLMs query the index first via MCP tools, then open source files only when they need to edit.
 
 ## Installation
 
@@ -56,7 +42,7 @@ npm install -g frontmatter-matters
 ```bash
 fmm init                              # One-command setup
 fmm generate && fmm validate          # CI pipeline
-fmm watch                             # Live sidecar regeneration
+fmm watch                             # Live index updates on file change
 fmm search --export createStore       # O(1) symbol lookup
 fmm search --depends-on src/auth.ts   # Impact analysis
 fmm search --loc ">500"              # Find large files
@@ -67,17 +53,17 @@ That's it. Your AI coding assistant now navigates via metadata instead of brute-
 
 ## Commands
 
-| Command               | Purpose                                                       |
-| --------------------- | ------------------------------------------------------------- |
-| `fmm init`            | Set up config, Claude skill, and MCP server                   |
-| `fmm generate [path]` | Create and update .fmm sidecars (exports, imports, deps, LOC) |
-| `fmm watch [path]`    | Watch source files and regenerate sidecars on change          |
-| `fmm validate [path]` | Check sidecars are current (CI-friendly, exit 1 if stale)     |
-| `fmm search`          | Query the index (O(1) export lookup, dependency graphs)       |
-| `fmm glossary`        | Symbol-level impact analysis — who imports this export?       |
-| `fmm mcp`             | Start MCP server (9 tools for LLM navigation)                 |
-| `fmm status`          | Show config and workspace stats                               |
-| `fmm clean [path]`    | Remove all .fmm sidecars                                      |
+| Command               | Purpose                                                            |
+| --------------------- | ------------------------------------------------------------------ |
+| `fmm init`            | Set up config, Claude skill, and MCP server                        |
+| `fmm generate [path]` | Index source files into `.fmm.db` (exports, imports, deps, LOC)   |
+| `fmm watch [path]`    | Watch source files and update the index on change                  |
+| `fmm validate [path]` | Check the index is current (CI-friendly, exit 1 if stale)          |
+| `fmm search`          | Query the index (O(1) export lookup, dependency graphs)            |
+| `fmm glossary`        | Symbol-level impact analysis — who imports this export?            |
+| `fmm mcp`             | Start MCP server (9 tools for LLM navigation)                      |
+| `fmm status`          | Show config and workspace stats                                    |
+| `fmm clean [path]`    | Clear the fmm index database                                       |
 
 Run `fmm --help` for workflows and examples, or `fmm <command> --help` for detailed per-command help.
 
@@ -113,13 +99,13 @@ fmm includes a built-in MCP server with 9 tools. Configure via `fmm init --mcp`
                         ┌──────────────────────────────────────────────────────┐
                         │                     fmm pipeline                     │
                         │                                                      │
-  Source Files          │   ┌───────-───┐    ┌───────────┐    ┌────────────┐   │    LLM / MCP Client
-  ─────────────────────►│   │  Parser   │───►│ Extractor │───►│  Sidecar   │   │◄──────────────────
+  Source Files          │   ┌───────────┐    ┌───────────┐    ┌────────────┐   │    LLM / MCP Client
+  ─────────────────────►│   │  Parser   │───►│ Extractor │───►│  SQLite    │   │◄──────────────────
   .ts .py .rs .go .c    │   │(tree-sit) │    │           │    │  Writer    │   │   fmm_lookup_export
-  .java .cpp .cs .rb    │   └────────-──┘    └───────────┘    └─────┬──────┘   │   fmm_read_symbol
+  .java .cpp .cs .rb    │   └───────────┘    └───────────┘    └─────┬──────┘   │   fmm_read_symbol
   .dart .lua .zig .sc   │                                     ┌─────▼──────┐   │   fmm_dependency_graph
-                        │                                     │   .fmm     │   │   fmm_file_outline
-                        │                                     │  sidecars  │   │   fmm_list_exports
+                        │                                     │  .fmm.db   │   │   fmm_file_outline
+                        │                                     │  (SQLite)  │   │   fmm_list_exports
                         │                                     └─────┬──────┘   │   fmm_search
                         │                                           │          │   fmm_list_files
                         │                                           │          │   fmm_glossary
@@ -132,8 +118,8 @@ fmm includes a built-in MCP server with 9 tools. Configure via `fmm init --mcp`
 
 1. **Parse** — tree-sitter parses source into AST
 2. **Extract** — identifies exports, imports, dependencies per file
-3. **Generate** — writes `.fmm` sidecar alongside each source file
-4. **Query** — MCP server or CLI reads sidecars on demand, builds in-memory index
+3. **Generate** — upserts file data into `.fmm.db` (incremental, mtime-based)
+4. **Query** — MCP server or CLI loads the index from SQLite in milliseconds
 
 ## Supported Languages
 
@@ -173,7 +159,7 @@ All languages extract: **exports**, **imports**, **dependencies**, **LOC**.
 
 ```yaml
 # GitHub Actions
-- name: Validate fmm sidecars
+- name: Validate fmm index
   run: |
     npx frontmatter-matters validate
 ```

build.rs

@@ -340,22 +340,16 @@ fn generate_skill_md(skill: Option<&SkillConfig>, tools: &IndexMap<String, ToolD
     out.push_str("8. **Read source only when editing** — MCP tools tell you everything you need for navigation\n");
     out.push_str("9. **Trust the index** — it updates automatically after every file write\n\n");
 
-    // Fallback section.
-    out.push_str("## Sidecar Fallback\n\n");
-    out.push_str(
-        "If MCP tools are unavailable, `.fmm` sidecar files exist alongside source files:\n\n",
-    );
-    out.push_str("```yaml\n");
-    out.push_str("file: src/core/pipeline.ts\n");
-    out.push_str("fmm: v0.3+0.1.11\n");
-    out.push_str("exports:\n");
-    out.push_str("  createPipeline: [10, 45]\n");
-    out.push_str("  PipelineConfig: [47, 52]\n");
-    out.push_str("imports: [./engine, ./validators, lodash, zod]\n");
-    out.push_str("loc: 142\n");
-    out.push_str("modified: 2026-03-06\n");
+    // CLI fallback section.
+    out.push_str("## CLI Fallback\n\n");
+    out.push_str("If MCP tools are unavailable, use the CLI directly:\n\n");
+    out.push_str("```bash\n");
+    out.push_str("fmm lookup createPipeline      # Find where a symbol is defined\n");
+    out.push_str("fmm read createPipeline        # Extract exact source\n");
+    out.push_str("fmm deps src/core/pipeline.ts  # Dependency graph\n");
+    out.push_str("fmm outline src/core/pipeline.ts  # Table of contents with line ranges\n");
     out.push_str("```\n\n");
-    out.push_str("Line ranges enable surgical reads: `Read(file, offset=10, limit=36)`.\n");
+    out.push_str("Line ranges from `fmm outline` enable surgical reads: `Read(file, offset=10, limit=36)`.\n");
 
     out
 }

examples/demo-project/WALKTHROUGH.md

@@ -1,6 +1,6 @@
 # Demo Project Walkthrough
 
-This is a small Express.js web application with authentication, database access, and API routes. All 8 source files have pre-generated `.fmm` sidecars.
+This is a small Express.js web application with authentication, database access, and API routes. Run `fmm generate` to build the index, then try the commands below.
 
 ## Project structure
 
@@ -80,7 +80,7 @@ src/auth/session.ts
 
 ## What an LLM sees
 
-Instead of reading 214 lines of source code across 8 files, an LLM reads 8 sidecars totaling ~56 lines of YAML. It knows every export, import, and dependency — enough to navigate to exactly the right file.
+Instead of reading 214 lines of source code across 8 files, an LLM queries the fmm index. It knows every export, import, and dependency — enough to navigate to exactly the right file.
 
 **Without fmm:** ~4,000 tokens to read all source files
-**With fmm:** ~200 tokens to read all sidecars
+**With fmm:** ~200 tokens to query the index