feat(m10.5): graph indexer, RAG pipeline, and Qdrant vector store

.mcp.json

@@ -1,8 +1,11 @@
 {
   "mcpServers": {
     "semantic-code-mcp": {
-      "command": "cargo",
-      "args": ["run", "--manifest-path", "./rust/Cargo.toml"]
+      "command": "./rust/target/release/semantic-code-mcp",
+      "env": {
+        "OXIGRAPH_STORE_PATH": "/tmp/oxigraph-test-store",
+        "QDRANT_URL": "http://localhost:6333"
+      }
     }
   }
 }

PLAN.md

@@ -35,10 +35,10 @@ Plugin system foundation plus the first code loader.
   - Module hierarchy resolution
 - Manual testing: load a Rust project and query its structure via SPARQL
 
-### M3 — Python Loader
+### M3 — Python Loader ✅
 - Implement Python loader (`load_python_code`):
-  - pyproject.toml / setup.py / requirements.txt parsing
-  - `.py` file AST extraction (modules, functions, classes, decorators, imports, docstrings)
+  - pyproject.toml parsing (project metadata, dependencies)
+  - `.py` file AST extraction via `rustpython-parser` (modules, functions, classes, decorators, imports, type annotations, docstrings, async)
 - Manual testing: load a Python project and query its structure via SPARQL
 
 ### M4 — TypeScript Loader ✅
@@ -47,13 +47,13 @@ Plugin system foundation plus the first code loader.
   - `.ts`/`.tsx`/`.js`/`.jsx` file AST extraction via `oxc_parser` (modules, functions, classes, interfaces, type aliases, enums, imports/exports, JSDoc)
 - Manual testing: load a TypeScript project and query its structure via SPARQL
 
-### M5 — Testing and Documentation
+### M5 — Testing and Documentation ✅
 - Integration tests for generic RDF tools
-- Integration tests for each language loader
+- Integration tests for each language loader (Rust, TypeScript, Python)
 - User-facing README with installation and usage instructions
 - Claude Code MCP configuration examples
 
-### M6 — Git History Loader
+### M6 — Git History Loader ✅
 Load git commit history into the RDF knowledge graph, enabling queries that join code structure with change history.
 
 - Add `git2` crate (libgit2 bindings) for native repository access
@@ -74,3 +74,63 @@ Load git commit history into the RDF knowledge graph, enabling queries that join
 - Namespace prefix management (`list_namespaces`, `add_namespace`)
 - Bulk loading with progress reporting
 - Additional language loaders (Go, Java, C/C++, etc.)
+
+### M8 — Pluggable Vector Store ✅
+Introduce the `VectorStore` trait and in-memory default backend as a separate crate.
+
+- Restructure `rust/` as a Cargo workspace
+- Create `crates/vector_store/` crate
+- Define `VectorStore` async trait, `RagChunk`, `SearchHit`, `Filter` types
+- Implement `InMemoryVectorStore` using `hnsw_rs` + `dashmap`
+- Unit tests: upsert, delete, search, filtering
+
+### M9 — RAG Pipeline ✅
+Build the retrieval pipeline connecting embeddings to the vector store.
+
+- Create `crates/rag_pipeline/` crate
+- Implement embedding provider abstraction (`EmbeddingProvider` trait, `MockEmbeddingProvider`, `HttpEmbeddingProvider`)
+- Implement RDF canonicalization for chunk text generation
+- Implement retrieval: embed query → `VectorStore.search()` → return ranked chunks
+- Pass-through reranker (extensible via `Reranker` trait)
+- Context compression via `TruncatingCompressor`
+- Integration tests with in-memory vector store
+
+### M10 — Agent Orchestrator ✅
+Planner/router that dispatches to SPARQL, RAG, and codegen tools.
+
+- Create `crates/agent_orchestrator/` crate
+- Define `AgentTool` trait and `ToolInput` / `ToolOutput` types
+- Implement `SparqlTool`, `RagTool`, `CodegenTool` wrappers
+- Implement planner/router logic (`AgentRouter`)
+- Enforce prompt contract (IRI citation, chunk ID citation, grounding)
+- Wire orchestrator into the MCP server as `agent_query` tool
+- Integration tests with mock LLM
+
+### M10.5 — Graph Indexer & Qdrant Backend ✅
+Bridge the triplestore and the vector store; add production-grade vector DB support.
+
+- Implement `GraphIndexer` (SPARQL → canonicalize → embed → upsert) in `rag_pipeline`
+- Wire `index_graph` as MCP tool
+- Add `HttpEmbeddingProvider` for OpenAI-compatible embedding endpoints
+- Implement `QdrantVectorStore` in `crates/vector_store/src/qdrant.rs` (gRPC via `qdrant-client`)
+- Auto-select Qdrant when `QDRANT_URL` is set, fall back to in-memory
+- Add `docker-compose.yml` for Qdrant (v1.13.2, REST + gRPC)
+- Setup docs in `docs/qdrant-setup.md`
+
+### M11 — External Vector DB Adapters
+Additional production vector database adapters and configuration.
+
+- Implement Milvus adapter (`milvus` feature flag)
+- TOML-based configuration for backend selection
+- Integration tests with containerized Qdrant/Milvus
+- Fallback-to-inmemory on adapter failure
+
+### M12 — Observability & Production Hardening
+Tracing, metrics, security, and multi-tenancy.
+
+- Add `request_id` propagation and structured tracing spans
+- Implement Precision@K and Faithfulness evaluation hooks
+- Implement PII redaction before embedding
+- Implement namespace isolation and graph-level ACL filtering
+- Hash-based chunk deduplication
+- Performance benchmarks (latency, recall, memory)

README.md

@@ -120,6 +120,26 @@ cargo test
 | AST Parsing | syn 2.x |
 | Transport | stdio (JSON-RPC) |
 
+## Roadmap
+
+See [PLAN.md](PLAN.md) for full milestone details and [SPECIFICATIONS.md](SPECIFICATIONS.md) for technical specifications.
+
+| Milestone | Description | Status |
+|---|---|---|
+| M0–M2 | Core server, generic RDF tools, Rust code loader | ✅ Done |
+| M4 | TypeScript code loader | ✅ Done |
+| M3 | Python code loader | Planned |
+| M5 | Testing & documentation | In progress |
+| M6 | Git history loader | Planned |
+| M7 | Advanced features (graph management, store stats) | Planned |
+| M8 | Pluggable vector store (`VectorStore` trait + in-memory backend) | Planned |
+| M9 | RAG pipeline (embedding, retrieval, reranking) | Planned |
+| M10 | Agent orchestrator (planner/router, tool dispatch) | Planned |
+| M11 | External vector DB adapters (Qdrant, Milvus) | Planned |
+| M12 | Observability & production hardening | Planned |
+
+**Future enhancements:** Hybrid lexical + vector retrieval, SHACL-aware scoring, multi-vector per RDF node, incremental embeddings, WASM reranker.
+
 ## License
 
 See [LICENSE](LICENSE) for details.