Modules

Modules

Spector is organized as a multi-module Maven project. Each module has a focused responsibility, clear API boundaries, and minimal cross-module coupling.

---

Architecture

graph LR
    subgraph "🔬 Foundation"
        core["spector-core<br/><i>SIMD kernels</i>"]
        commons["spector-commons<br/><i>Chunkers, tokenizer</i>"]
        config["spector-config<br/><i>SpectorConfig + YAML</i>"]
        storage["spector-storage<br/><i>Panama MemorySegment</i>"]
    end

    subgraph "🧠 Intelligence"
        embedApi["spector-embed-api<br/><i>Embedding SPI</i>"]
        embedOllama["spector-embed-ollama<br/><i>Ollama provider</i>"]
        index["spector-index<br/><i>HNSW + IVF-PQ + BM25</i>"]
        query["spector-query<br/><i>Hybrid + RRF + rerank</i>"]
        gpu["spector-gpu<br/><i>CUDA via Panama FFM</i>"]
    end

    subgraph "⚡ Engine"
        rag["spector-rag<br/><i>RAG pipeline</i>"]
        engine["spector-engine<br/><i>Search facade</i>"]
        ingestion["spector-ingestion<br/><i>File ingest pipeline</i>"]
        memory["spector-memory<br/><i>Cognitive memory 🧠</i>"]
    end

    subgraph "📡 Observability"
        events["spector-events<br/><i>Telemetry event bus</i>"]
        metrics["spector-metrics<br/><i>Micrometer + TelemetryBus</i>"]
        cortex["spector-cortex<br/><i>Moved to enterprise</i>"]
    end

    subgraph "🌐 Runtime & Interfaces"
        runtime["spector-runtime<br/><i>Composition root</i>"]
        node["spector-node<br/><i>Armeria: REST + gRPC + SSE</i>"]
        mcp["spector-mcp<br/><i>MCP Server (stdio)</i>"]
        cli["spector-cli<br/><i>spectorctl</i>"]
        client["spector-client<br/><i>Java SDK</i>"]
        spring["spector-spring<br/><i>Spring AI</i>"]
    end

    subgraph "📦 Distribution"
        bench["spector-bench<br/><i>JMH benchmarks</i>"]
        dist["spector-dist<br/><i>Fat JAR</i>"]
    end

Loading

---

Module Dependency Graph

graph TD
    node["🌐 node"] --> runtime["⚡ runtime"]
    node --> mcp["🤖 mcp"]
    node --> metrics["📈 metrics"]
    node --> events["📡 events"]
    mcp --> runtime
    mcp --> ingestion["📥 ingestion"]
    cli["🖥️ cli"] --> runtime
    cli --> client["📦 client"]

    runtime --> engine["⚡ engine"]
    runtime --> memory["🧠 memory"]
    runtime --> ingestion

    engine --> query["🔍 query"]
    engine --> rag["🤖 rag"]
    engine --> ingestion
    engine --> index["📊 index"]
    engine --> storage["💾 storage"]
    engine --> embedapi["🧬 embed-api"]
    engine -.-> gpu["🎮 gpu"]

    memory --> index
    memory --> storage
    memory --> ingestion
    memory --> embedapi
    memory --> core["🔬 core"]

    metrics --> engine
    metrics --> memory
    metrics --> events

    events --> commons["📄 commons"]

    cortex["🧠 cortex"] -.->|SSE| node

    ingestion --> config["⚙️ config"]
    ingestion --> embedapi

    rag --> query
    rag --> index
    rag --> storage
    rag --> embedapi

    query --> index
    index --> storage
    index --> config
    storage --> config
    storage --> core
    config --> core

    embedapi --> commons
    gpu --> core
    gpu --> storage

    dist["📦 dist"] --> mcp
    dist --> cli
    dist --> runtime

    spring["🌱 spring"] --> engine
    spring --> memory
    spring --> metrics
    bench["🧪 bench"] --> engine
    bench --> memory

Loading

Legend: Solid arrows = compile dependency. Dotted arrows = optional/runtime dependency (gpu = optional Maven dep, cortex = connects via SSE at runtime).

important: Architecture spector-ingestion defines the IngestionPipeline and IngestionTarget interface. Both spector-engine and spector-memory depend on it to implement their IngestionTarget. spector-memory is fully independent of spector-engine — they are peers, wired together only at the SpectorRuntime composition root.

---

Architecture: Entry Points → Runtime → Subsystems

All entry points (MCP, CLI, Server) route through SpectorRuntime:

graph TD
    cli["🖥️ spector-cli<br/><i>SpectorCtl</i>"]
    mcp["🤖 spector-mcp<br/><i>SpectorMcpMain</i>"]
    node["🌐 spector-node<br/><i>SpectorNode (Armeria)</i>"]

    cli --> runtime
    mcp --> runtime
    node --> runtime

    runtime["⚡ SpectorRuntime<br/><i>Composition Root</i>"]

    runtime --> sh["SearchHandler<br/><i>mode-aware search</i>"]
    runtime --> ih["IngestionHandler<br/><i>delegates to IngestionPipeline</i>"]

    sh --> engine["SpectorEngine"]
    sh --> memory["SpectorMemory"]
    ih --> pipeline["IngestionPipeline<br/><i>chunk → embed → store</i>"]
    pipeline --> engineTarget["EngineIngestionTarget<br/><i>SEARCH mode</i>"]
    pipeline --> memTarget["CognitiveIngestionTarget<br/><i>MEMORY mode</i>"]

Loading

SpectorRuntime is a thin composition root — it creates and wires subsystems but contains no business logic. Each handler owns its domain:

Handler	Responsibility	Routes to
SearchHandler	Mode-aware search	Engine (SEARCH mode) or Memory (MEMORY mode)
IngestionHandler	Delegates to unified IngestionPipeline	Pipeline → EngineIngestionTarget or CognitiveIngestionTarget

---

Module Overview

Foundation Layer

Module	Description
[[spector-commons	Modules--Spector-Commons]]
[[spector-core	Modules--Spector-Core]]
[[spector-config	Modules--Spector-Config]]
[[spector-storage	Modules--Spector-Storage]]

Embedding Layer

Module	Description
[[spector-embed-api	Modules--Spector-Embed-Api]]
[[spector-embed-ollama	Modules--Spector-Embed-Ollama]]

Search Layer

Module	Description
[[spector-index	Modules--Spector-Index]]
[[spector-query	Modules--Spector-Query]]
[[spector-gpu	Modules--Spector-Gpu]]

Intelligence Layer

Module	Description
[[spector-rag	Modules--Spector-Rag]]
[[spector-engine	Modules--Spector-Engine]]
[[spector-ingestion	Modules--Spector-Ingestion]]
[[spector-memory	Modules--Spector-Memory]]

Runtime Layer

Module	Description
[[spector-runtime	Modules--Spector-Runtime]]
[[spector-mcp	Modules--Spector-Mcp]]
[[spector-node	Modules--Spector-Node]]

Client Layer

Module	Description
[[spector-cli	Modules--Spector-Cli]]
[[spector-client	Modules--Spector-Client]]
[[spector-spring	Modules--Spector-Spring]]

Infrastructure

Module	Description
[[spector-events	Modules--Spector-Events]]
[[spector-metrics	Modules--Spector-Metrics]]
[[spector-cortex	Modules--Spector-Cortex]]
[[spector-bench	Modules--Spector-Bench]]
[[spector-dist	Modules--Spector-Dist]]