Staging

AGENTS.md

@@ -101,10 +101,10 @@ flowchart TB
     end
 
     subgraph RETRIEVE["⑤ Retrieval (shared)"]
-        Query["GET /v1/retrieval/query"] --> Pipeline["run_retrieval_query"]
+        Query["POST /v1|/v2 retrieval/query"] --> Pipeline["run_retrieval_query"]
         Pipeline --> Classic["classic_topk / small_corpus (use_agentic=False)"]
         Pipeline --> MapNav["mapnav checklist (default / use_agentic≠False)"]
-        Classic --> Channels["3-Channel BM25 (path/content/term)"]
+        Classic --> Channels["map_unit_discovery: path+content BM25 -> RRF"]
         Channels --> Rank["rank_retrieval_candidates"]
         MapNav --> NavSnap["nav_snapshot + run_nav_episode"]
         NavSnap --> Bridge["nav_bridge referenced_chunks"]
@@ -546,7 +546,7 @@ debug CSVs (`preds_*.csv`) are saved alongside for troubleshooting.
 Core retrieval internals are grouped by ownership:
 
 - `execution/`: request shaping, route selection (classic / mapnav / small_corpus), and public response projection.
-- `search/`: lexical channels, scoring, section filters, candidate ranking, and classic `bottom_discovery`.
+- `search/`: `map_unit_discovery` (persisted map-unit BM25 discovery, with a legacy chunk-level PG FTS fallback), scoring, section filters, candidate ranking.
 - `hydration/`: row/path/reference hydration, inline assets, and result assembly.
 - `nav/` + `nav_*.py`: map-nav checklist episode (PLANNER / HARVEST / CONTROL).
 - `trace/`: `DecisionTraceStep` mapping and `TraceRecorder`.
@@ -555,24 +555,33 @@ Core retrieval internals are grouped by ownership:
 
 ### Two Retrieval Modes
 
-Per-request `use_agentic`: `False` → classic 3-channel top-K; `None`/`True` → map-nav (default).
+Per-request `use_agentic`: `False` → classic top-K (map-unit BM25); `None`/`True` → map-nav (default).
 
-#### Classic Mode (3-Channel RRF)
+#### Classic Mode (map-unit BM25 + legacy FTS fallback)
+
+Primary path is `search.map_unit_discovery.map_unit_discovery`: Python BM25Okapi
+over the persisted `document_map_unit_tokens` index, path and content channels
+only, fused by RRF.
 
 ```mermaid
 flowchart LR
-    Q[Query] --> P[Path Channel: BM25 on path_search_text]
-    Q --> C[Content Channel: BM25 on content_search_text]
-    Q --> T[Term Channel: substring on term_search_text]
+    Q[Query] --> P["Path channel: BM25 over document_map_unit_tokens (channel=path)"]
+    Q --> C["Content channel: BM25 over document_map_unit_tokens (channel=content)"]
     P --> RRF["RRF Fusion (k=60)"]
     C --> RRF
-    T --> RRF
     RRF --> Rank[rank_retrieval_candidates]
     Rank --> Assemble[hydration.result_assembly]
 ```
 
-**Channel weights** (default): path=1.0, content=2.0, term=1.5
-**RRF formula**: `score = weight / (k + rank + 1)` per channel, summed across channels.
+**Channel weights** (default): path=1.0, content=2.0. **RRF formula**:
+`score = weight / (k + rank + 1)` per channel, summed across channels, `k=60`.
+
+There is no scored term channel in this primary path. `term_search_text` /
+`term_search_text_lower` are persisted at publish time but are only read by
+the **legacy fallback** (`_legacy_chunk_discovery`), which runs only when a
+revision's map-unit index is missing or incomplete: a single SQL query
+scoring `GREATEST(ts_rank_cd(path_search_tsv), 2 * ts_rank_cd(content_search_tsv))`
+OR `term_search_text LIKE '%query%'`, not three independently-ranked channels.
 
 #### Map-nav Mode (default)
 

README.md

@@ -33,60 +33,76 @@
 
 ## Overview
 
-**Knowhere is the memory layer between complex, dirty documents and AI agents.**
+**Knowhere is a document parsing and retrieval system that turns complex, dirty files into persistent, navigable memory for AI agents—especially across local and offline document collections.**
 
-It ingests unstructured documents and produces persistent, navigable memory: parsing, hierarchy extraction, multi-modal structuring, and graph construction in a single pipeline. Every chunk retains full semantic context, making the output a natural fit for *Agentic RAG*, *vector-based RAG*, or any LLM workflow.
+It ingests unstructured documents and produces persistent, navigable memory: parsing, hierarchy reconstruction, multi-modal structuring, and graph construction in a single pipeline. Every result stays connected to its document, section, source pages, and related assets, making the output a natural fit for *Agentic RAG*, *vector-based RAG*, or any LLM workflow.
+
+Knowhere supports complementary **Vision and Text tracks**. Text-native documents retain precise extracted structure, while complex PDFs and PowerPoint files can be understood directly as pages by frontier vision models. Both tracks converge into the same memory schema, hierarchy, retrieval engine, and citation model.
 
 > [!NOTE]
 > **Get started in seconds with Knowhere Cloud.**
 > Avoid the complexity of self-deployment. Use our managed API at [knowhereto.ai](https://knowhereto.ai) and enjoy **$5 in free credits** upon registration.
 
 ## 📢 News
 
+- **September 2026**: 👁️ **Introducing dual-track Document Parsing 2.0.** Vision Page and Text Track now converge into one hierarchy-native memory schema for retrieval, understanding, and citation.
 - **June 1, 2026**: 📚 **Knowhere now supports ultra-long PDFs and atlas-style documents.** The parsing pipeline can process long-form PDFs with hundreds of pages (for example, 300, 500, or more) and route technical atlases or drawing collections through a dedicated layout-aware parser.
 - **May 7, 2026**: 🚀 **Knowhere is now Open Source!** We have open-sourced our entire stack for document ingestion, parsing, and agentic RAG. You can now self-host the full platform using [knowhere-self-hosted](https://github.com/Ontos-AI/knowhere-self-hosted). Check out our [Contribution Guide](CONTRIBUTING.md) to get involved!
 
-## How it Works
+## Vision + Text: Document Parsing 2.0
 
-Knowhere runs in two steps: build memory from documents, then let agents retrieve from it.
+Traditional OCR and Document Intelligence pipelines try to extract every element before a model can understand the document. On dirty PDFs and slide decks, mistakes in reading order, layout, tables, or hidden text layers can accumulate into unreliable model context.
 
-### Step 1: Parse and Build Memory
+Knowhere does not make perfect element-by-element extraction a prerequisite for retrieval. The Text Track preserves precise text and native structure where they are reliable. The Vision Track uses frontier vision models to understand a page or slide as a whole, so visually complex content can still be recalled and understood without first reconstructing every element.
 
 <p align="center">
-  <img alt="Step 1: Parse and Build Memory" src="docs/assets/step-1-parse-build-memory.png" width="900">
+  <img alt="Vision and Text tracks converge into a unified navigable memory schema" src="docs/assets/step-1-dual-track-memory.png" width="1000">
 </p>
 
-- **Parse**: Route PDFs, Office files, images, tables, Markdown, and text to specialized parsers.
-- **Structure**: Our proprietary Tree-like algorithm reconstructs the full document hierarchy instead of flattening it into a sequence, preventing semantic fragmentation across chunks.
-- **Build Memory**: Store chunks, navigation trees, summaries, and graph links as agent-ready context.
+- **Two tracks, one contract**: Both parsing paths produce the same chunk and metadata schema, so downstream storage, hierarchy, graph construction, and retrieval remain format-independent.
+- **Recall without brittle reconstruction**: Pages can be indexed through summaries, entities, source text, and hierarchy even when OCR or layout extraction cannot reliably recover every component.
+- **One navigable memory**: Text sections and vision-understood pages become compatible hierarchy nodes with source evidence, linked assets, and cross-document relationships.
+
+PDF and `.pptx` uploads through the V2 Jobs API use the Vision Track; other supported formats use the Text Track. The tracks differ in how they understand the source, not in how agents consume the resulting memory.
+
+## How it Works
+
+Knowhere runs in two steps: build memory from documents, then let agents retrieve from it.
+
+### Step 1: Parse and Build Memory
+
+- **Route**: Select the Vision or Text track according to the document format and API generation.
+- **Understand**: Preserve native text structure where it is reliable, or understand complex pages holistically with a vision model.
+- **Normalize**: Convert both tracks into the same hierarchy-native chunk and metadata schema.
+- **Build Memory**: Store navigation trees, linked assets, citations, and cross-document relationships as agent-ready context.
 
 ### Step 2: Agentic Retrieval
 
 <p align="center">
   <img alt="Step 2: Agentic Retrieval" src="docs/assets/step-2-agentic-retrieval.png" width="900">
 </p>
 
-- **Discover**: Fuse keyword, path, content, and semantic signals for broad first-pass coverage.
-- **Navigate**: Walk section trees and graph links to drill into the most relevant document regions.
-- **Cite Evidence**: Return traceable results with source document, section, chunk, and linked assets.
+- **Discover**: Fuse path, content, term, summary, and entity signals for broad first-pass coverage.
+- **Navigate**: Use hierarchy-aware MapNav to move from document overviews into the most relevant sections and evidence.
+- **Cite Evidence**: Return traceable results with source document, section, source pages, and linked assets.
 
 ## FAQ
 
 **Q: What is Knowhere's relationship with MinerU?**
 
-A: Knowhere uses MinerU as its default parser because it performs best in our tests. Any parser only gets you raw Markdown. Knowhere's value is what comes after: hierarchy reconstruction, multi-modal normalization, and cross-document graph construction. Any Markdown-outputting tool works.
+A: MinerU remains the default raw PDF extractor for Knowhere's V1 chunk-based pipeline. PDF and PowerPoint uploads through the V2 API use Vision Page instead: Knowhere renders the source pages, combines their visual interpretation with document profiling and TOC structure, and assembles page-grounded hierarchy nodes. MinerU is still useful, but V2 no longer treats parser-generated Markdown as the only source of truth.
 
 **Q: What LLM / VLM dependencies does Knowhere have?**
 
-A: By default, DeepSeek (`deepseek-chat`) handles text and table summarization, and Qwen-VL (`qwen3.6-flash`) handles image OCR and descriptions. Knowhere is model-agnostic. Swap in OpenAI, DashScope, Zhipu, or Volcengine via environment variables.
+A: We recommend [`deepseek-v4-flash-vision-exp`](https://api-docs.deepseek.com/guides/vision/) as a unified model for both Text and Vision workloads. It accepts text and image input, so the same model can handle summarization, hierarchy reasoning, page understanding, and asset descriptions. The model is currently experimental, and Knowhere remains model-agnostic: you can use another model—or separate Text and Vision models—from OpenAI, Qwen, GLM, Volcengine, or any compatible provider.
 
 **Q: How is Agentic Retrieval different from traditional RAG?**
 
 A: Traditional RAG does a flat vector lookup and returns isolated snippets. Knowhere's agents navigate the document's section tree and cross-document graph, drilling into the most relevant regions the way a human reader would, returning traceable, well-contextualized evidence.
 
 **Q: Does it handle images and tables?**
 
-A: Yes. Knowhere extracts them, runs them through VLMs for summarization and feature extraction, and links them back to their source chunks so agents can retrieve and cite multi-modal assets at inference time.
+A: Yes. Knowhere extracts images and tables, runs them through VLM-assisted summarization and feature extraction, and links them back to their source section nodes. Vision Page also retains rendered page citations, so agents can return both structured context and the visual source evidence.
 
 ## Performance Benchmark
 
@@ -121,22 +137,25 @@ Agents using Knowhere outperform those working from raw documents, Markitdown, U
 
 ## Features
 
-- **Multi-modal Parsing**: High-fidelity extraction from PDF, Office, and images, preserving headings, tables, and hierarchical paths.
-- **Lightweight Memory Graph**: Context-aware organization that links documents and chunks for better relationship understanding.
-- **Agentic RAG**: A hybrid retrieval engine combining traditional search (RRF) with autonomous agent navigation.
-- **Evidence-based Citations**: Every result is backed by traceable source paths, ensuring reliability for AI Agent decision-making.
+- **Dual-track Parsing**: Vision and Text tracks handle different document conditions while producing the same downstream schema.
+- **Vision Page Understanding**: Frontier vision models make complex PDF and PowerPoint content recallable without requiring perfect element-by-element OCR or layout reconstruction.
+- **Hierarchy-native Memory**: Section nodes preserve document paths, page ranges, summaries, entities, and linked assets instead of returning disconnected chunks.
+- **Cross-document Memory Graph**: Page-derived typed entities and keywords connect related documents across a namespace.
+- **Agentic Retrieval**: MapNav navigates document hierarchies, while classic retrieval combines path, content, and term channels through RRF.
+- **Page-grounded Citations**: Results retain source documents, section paths, page numbers, and rendered visual evidence.
 
 ## Supported Formats
 
 **✅ Supported**
 
-- [x] `.pdf` `.docx` `.pptx` `.xlsx` `.csv`
-- [x] `.jpg` `.png`
-- [x] `.md` `.txt` `.json`
+- [x] `.pdf` `.pptx` — Vision Page through the V2 Jobs API
+- [x] `.doc` `.docx` `.xls` `.xlsx`
+- [x] `.jpg` `.jpeg` `.png`
+- [x] `.md` `.txt` `.html` `.htm` `.json`
 
 **⏳ Coming Soon**
 
-- [ ] `.epub` `.html` `.xml`
+- [ ] `.epub` `.xml`
 - [ ] `.mp4` `.mp3`
 - [ ] `.skills.md`
 
@@ -168,8 +187,8 @@ cp apps/worker/.env.example apps/worker/.env
 - database and Redis connection settings
 - S3-compatible storage credentials
 - at least one LLM provider key: `DS_KEY`, `ALI_API_KEYS`, `GPT_API_KEY`, or `GLM_API_KEY`
-- `MINERU_API_KEYS` if you need PDF parsing
-- a vision-capable model provider if you need image summaries, OCR, atlas classification, or image-aware retrieval
+- a vision-capable model provider for V2 PDF/PowerPoint parsing, page understanding, image summaries, OCR, atlas classification, or image-aware retrieval
+- `MINERU_API_KEYS` only if you use the V1 chunk-based PDF/PowerPoint pipeline
 - any optional billing or webhook providers you want to enable
 
 Most parser and retrieval tuning values have code defaults. Start with the

apps/worker/app/services/document_agent/tools/ocr_pages.py

@@ -7,6 +7,10 @@
 
 from app.services.document_agent.manifest import ToolContext, ToolResult
 from app.services.document_agent.pdf_text import PageTextBands
+from app.services.document_parser.formats.pdf.pymupdf_subprocess import (
+    run_in_child_process,
+    worker,
+)
 from app.services.document_agent.registry import has_page_features, register_tool
 from app.services.document_agent.visual import render_pages
 
@@ -32,6 +36,28 @@ def _line_score(item: Any) -> float:
     return 0.0
 
 
+@worker
+def _run_ocr_worker(queue: Any, page_paths: dict[int, str]) -> None:
+    """Run the local OCR model outside the heartbeat-bearing worker process."""
+    from rapidocr_onnxruntime import RapidOCR
+
+    engine = RapidOCR(intra_op_num_threads=2, inter_op_num_threads=1)
+    page_lines: dict[int, list[dict[str, Any]]] = {}
+    for page, image_path in page_paths.items():
+        lines: list[dict[str, Any]] = []
+        result, _elapse = engine(image_path)
+        for item in result or []:
+            lines.append(
+                {
+                    "box": _line_box(item),
+                    "text": _line_text(item),
+                    "score": _line_score(item),
+                }
+            )
+        page_lines[page] = lines
+    queue.put({"ok": True, "page_lines": page_lines})
+
+
 @register_tool(
     name="ocr.pages",
     description="Run RapidOCR on specified pages and return positioned text lines.",
@@ -70,26 +96,18 @@ def ocr_pages(ctx: ToolContext, args: dict[str, Any]) -> ToolResult:
         if item.get("page") is not None and item.get("png_path")
     }
 
-    from rapidocr_onnxruntime import RapidOCR
-
-    engine = RapidOCR()
+    page_paths = {
+        page: image_path for page, image_path in png_by_page.items() if page in pages
+    }
+    result = run_in_child_process(_run_ocr_worker, page_paths, timeout=300)
+    page_lines = {
+        int(page): list(lines)
+        for page, lines in (result.get("page_lines") or {}).items()
+    }
     page_texts: dict[int, str] = {}
     page_bands: dict[int, PageTextBands] = {}
-    page_lines: dict[int, list[dict[str, Any]]] = {}
     for page in pages:
-        image_path = png_by_page.get(page)
-        lines: list[dict[str, Any]] = []
-        if image_path:
-            result, _elapse = engine(image_path)
-            for item in result or []:
-                text = _line_text(item)
-                lines.append(
-                    {
-                        "box": _line_box(item),
-                        "text": text,
-                        "score": _line_score(item),
-                    }
-                )
+        lines = page_lines.get(page, [])
         page_lines[page] = lines
         content = "\n".join(line["text"] for line in lines if line["text"])
         page_texts[page] = content

apps/worker/tests/contract/test_ocr_pages_contract.py

@@ -3,8 +3,6 @@
 from __future__ import annotations
 
 import os
-import sys
-from types import ModuleType
 from unittest.mock import patch
 
 os.environ.setdefault("DATABASE_URL", "postgresql+asyncpg://test:test@localhost/test")
@@ -55,19 +53,29 @@ def test_ocr_pages_requires_pages() -> None:
 def test_ocr_pages_writes_joined_text_to_blackboard() -> None:
     ctx = _ctx()
 
-    class FakeEngine:
-        def __call__(self, _image_path: str):
-            return [[[[0, 0], [1, 0], [1, 1], [0, 1]], "Hello", 0.9]], 0.01
-
-    fake_mod = ModuleType("rapidocr_onnxruntime")
-    fake_mod.RapidOCR = lambda: FakeEngine()  # type: ignore[attr-defined]
-
     def fake_render(*_args, **_kwargs):
         return [{"page": 1, "png_path": "/tmp/ocr_page_1.png"}]
 
+    def fake_child(worker_fn, page_paths, *, timeout):
+        assert worker_fn.__name__ == "_run_ocr_worker"
+        assert page_paths == {1: "/tmp/ocr_page_1.png"}
+        assert timeout == 300
+        return {
+            "ok": True,
+            "page_lines": {
+                1: [
+                    {
+                        "box": [[0, 0], [1, 0], [1, 1], [0, 1]],
+                        "text": "Hello",
+                        "score": 0.9,
+                    }
+                ]
+            },
+        }
+
     with (
         patch.dict(ocr_pages.__globals__, {"render_pages": fake_render}),
-        patch.dict(sys.modules, {"rapidocr_onnxruntime": fake_mod}),
+        patch.dict(ocr_pages.__globals__, {"run_in_child_process": fake_child}),
     ):
         result = ocr_pages(ctx, {"pages": [1]})
 

docs/design/agent-corpus-schema.md

@@ -0,0 +1,10 @@
+# Agent Corpus Schema
+
+**Status:** Design in progress
+The agent-facing corpus schema and tool-usage guidance has a single source
+of truth. The same text is intended to be shipped verbatim as both the API
+`/mcp` server `instructions` and the `agent_explore` system prompt:
+
+[`packages/shared-python/shared/services/retrieval/agent_tools/CORPUS_SCHEMA.md`](../../packages/shared-python/shared/services/retrieval/agent_tools/CORPUS_SCHEMA.md)
+
+Do not copy its content here — edit that file, not this pointer.