DocuMix

Tool to compile documents from a folder into a single Markdown file, similar to Repomix.

Features

• Compiles various document types (PDF, EPUB, DOCX, DOC, XLSX, PPTX, RTF, TXT, MD, etc.) into a single Markdown file
• Supports ZIP files by automatically extracting and processing their contents
• Email Mode: Process .eml files with automatic attachment handling
• Smart Output Formats: Automatically detects and uses email-specific format for .eml files
• Converter Control: Choose and reorder converters per format (--pdf-converters, --docx-converters, --rtf-converters)
• PDF Table Extraction: ML-based (MinerU, PaddleOCR) and rule-based (pdfplumber) table detection with markdown output
• Built-in Benchmarking: Compare converter speed and accuracy on your system
• Estimates token counts and provides statistics
• Security checks for potentially suspicious files
• Colorful console output with emojis for better readability

Installation

You can install documix directly from GitHub:

pip install git+https://github.com/lchojnowski/documix.git

Optional extras

Install with PDF support (markitdown):

pip install "documix[pdf] @ git+https://github.com/lchojnowski/documix.git"

Install with PDF table extraction (pdfplumber):

pip install "documix[tables] @ git+https://github.com/lchojnowski/documix.git"

Install with XLSX spreadsheet and PPTX presentation support (Python 3.10 or newer for MarkItDown):

pip install "documix[office] @ git+https://github.com/lchojnowski/documix.git"

DocuMix itself still supports Python 3.8. Install the Office converter in a Python 3.10+ virtual environment or pipx environment and make its markitdown command available on PATH. Its resolved installation must have an executable python beside that command; shell wrappers and other layouts are refused. DocuMix uses that interpreter for the selected Office parser, so its own Python 3.8 process does not import newer dependencies.

Install with PaddleOCR support (ML-based document analysis; heavy dependencies ~1 GB, models downloaded on first run):

pip install "documix[paddleocr] @ git+https://github.com/lchojnowski/documix.git"

You can combine extras:

pip install "documix[tables,paddleocr] @ git+https://github.com/lchojnowski/documix.git"

Quick Setup (macOS)

Install all dependencies (converters, ML tools, Python packages) in one step:

git clone https://github.com/lchojnowski/documix.git
cd documix
bin/setup

The script installs everything via Homebrew and pip. If mise is available, it will be used for supported tools. Run it again anytime to check for missing dependencies.

Quick Start with uvx

Run documix without installing using uvx:

uvx --from git+https://github.com/lchojnowski/documix.git documix /path/to/folder -r

This always uses the latest version from GitHub without requiring installation.

uvx Examples

Process documents:

uvx --from git+https://github.com/lchojnowski/documix.git documix /path/to/documents -r -o output.md

Process a single email:

uvx --from git+https://github.com/lchojnowski/documix.git documix email.eml -o email_output.md

With table support via uvx:

uvx --from 'documix[tables] @ git+https://github.com/lchojnowski/documix.git' documix /path/to/folder -r

Usage

documix /path/to/folder -r -o output.md

Options

Flag	Description
-o, --output	Path to the output file (default: documix-output.md)
-r, --recursive	Search folders recursively
-e, --extensions	File extensions to process, comma-separated (default: pdf, epub, docx, doc, rtf, txt, md, py, rb, js, html, css, json, yml, yaml, zip, eml)
-x, --exclude	File exclusion patterns, comma-separated (regular expressions)
-v, --version	Display program version and available converters
--standard-format	Force standard output format (even for emails)
--pdf-converters	PDF converters to try, in order (comma-separated)
--docx-converters	DOCX converters to try, in order (comma-separated)
--rtf-converters	RTF converters to try, in order (comma-separated)
--ocr-languages	Installed Tesseract languages, e.g. pol+eng (default: eng; requires explicit tesseract)
--ocr-max-pages	Maximum pages per OCR document (default: 20; range: 1–100)
--ocr-dpi	OCR rendering resolution (default: 150; range: 1–200)
--ocr-timeout	Total OCR time limit in seconds (default: 180; range: 1–300)
--office-require-text-only	Reject XLSX/PPTX with unassessed embedded or linked images (opt-in; independent of PDF OCR)

Converter Control

You can control which converters are used for each format and in what order they are tried. When a flag is omitted, the default order is used. When provided, only the listed converters are tried, in the listed order.

An empty PDF result now advances to the next selected converter, including results containing only whitespace or invisible control characters. The first result containing text is kept unchanged. Existing default converter order and Markdown output format are preserved.

Available converters

Format	Converters (default order)
PDF	mineru, pdfplumber, markitdown-uvx, markitdown, pdftotext, paddleocr
DOCX	pandoc, docx2txt
RTF	pandoc, unrtf, striprtf

PDF also supports tesseract as an explicit opt-in converter; it is never added to the default order or automatic benchmarks. Selecting it requires OCR of the complete document, even when another selected converter returns text.

For non-PDF formats, pandoc is always the default first converter due to its high-quality markdown output. Remaining converters are ordered by speed as fallbacks.

Examples

Use only pdfplumber and pdftotext (skip ML converters for speed):

documix /path/to/pdfs -r --pdf-converters pdfplumber,pdftotext

Prioritize PaddleOCR for best OCR quality:

documix /path/to/pdfs -r --pdf-converters paddleocr,mineru,pdfplumber

Use only pdftotext for plain text extraction:

documix /path/to/pdfs -r --pdf-converters pdftotext

Use only docx2txt for DOCX files (skip pandoc):

documix /path/to/docs -r --docx-converters docx2txt

Use only striprtf for RTF files:

documix /path/to/docs -r --rtf-converters striprtf

Check which converters are installed on your system:

$ documix -v
DocuMix v1.20260316.0-dev (main)

Converter Configuration:
----------------------------------------
   PDF: pdfplumber, markitdown-uvx, markitdown, pdftotext
        missing: mineru, paddleocr
  DOCX: pandoc, docx2txt
   RTF: pandoc
        missing: unrtf, striprtf

Local OCR for scanned and mixed PDFs

To extract text from scans without downloading models during conversion, install Tesseract and Poppler locally, including the language packs you need. On Debian/Ubuntu, for Polish and English:

sudo apt-get install tesseract-ocr tesseract-ocr-pol tesseract-ocr-eng poppler-utils

On macOS, install tesseract, tesseract-lang, and poppler with Homebrew. Check the languages already installed with tesseract --list-langs. This converter currently supports POSIX systems (Linux and macOS).

Select local text extractors together with mandatory complete-document OCR:

documix /path/to/pdfs -r --pdf-converters markitdown,pdftotext,tesseract \
  --ocr-languages pol+eng --ocr-max-pages 20 --ocr-dpi 150 --ocr-timeout 180

markitdown uses an already installed command; markitdown-uvx is a separate converter that may install packages. The example above does not select MinerU, PaddleOCR, or markitdown-uvx. Tesseract does not download tools or language packs, and an unavailable language makes the PDF conversion fail rather than silently switching languages. OCR flags alone do not enable OCR.

When tesseract appears anywhere in your PDF converter list, DocuMix runs it once on every page before trying the other selected converters. It retains the first successful native extraction verbatim and appends a labeled OCR rendition; the reported method is, for example, markitdown+tesseract. This includes scanned content after a text cover or beneath a text heading on the same page. If native extractors return no text, the output is the OCR rendition alone. Without tesseract, the existing first-success converter behavior is unchanged.

The library accepts the same settings without changing existing calls:

from documix.documix import DocumentCompiler

compiler = DocumentCompiler(
    source_path="/path/to/pdfs",
    output_file="output.md",
    converter_config={"pdf": ["markitdown", "pdftotext", "tesseract"]},
    ocr_config={"languages": "pol+eng", "max_pages": 20, "dpi": 150, "timeout": 180},
)
compiler.compile()

OCR processes pages in order and keeps the original PDF unchanged. A completely white rendered page is retained as a page separator. An all-white document, an unreadable nonwhite page, or a failure on any page is not reported as successful extraction of the whole document. A failed or over-budget OCR pass fails the PDF conversion regardless of converter order; native text alone cannot replace the missing OCR coverage. OCR can misread or overlook text and does not reconstruct table layout; review important results against the source. A completed pass describes processing coverage, not a guarantee that every word was recognized.

OCR work is bounded per document: defaults are 20 pages, 150 DPI, and 180 seconds, with maximum settings of 100 pages, 200 DPI, and 300 seconds. Rendering also limits each side to 4,000 pixels and each page to 16 million pixels; oversized pages are scaled down. Each OCR native tool call has a 60-second ceiling within the total OCR time budget. Input PDFs are limited to 128 MiB and recognized text to 4 MiB; the combined native text, label and OCR rendition must also fit within 4 MiB, with no truncation. Temporary page images are removed on success or failure. These limits do not replace an operating-system sandbox and memory limits when processing untrusted documents, and do not change the separate native converters' time limits.

This changes the earlier opt-in fallback behavior: even a fully selectable PDF above your OCR page budget now fails under this profile. Automated pipelines should keep such documents visibly incomplete for review or a separately authorized retry. The default 20-page budget and all hard limits remain unchanged; this mode does not promise automatic coverage of every PDF.

Interrupting OCR cancels the active tool and waits for its cleanup. If cleanup cannot be confirmed, compilation stops, including when the PDF is an email or ZIP attachment. Library callers receive documix.local_ocr.OcrCleanupError, a fatal BaseException rather than an ordinary conversion failure; do not catch it to continue processing other files.

The three-second cleanup acknowledgement budget does not guarantee termination of a process stuck inside the operating system. Use an external process supervisor with a wall-clock limit for unattended or untrusted inputs; a fatal cleanup error must not be treated as proof that every process has stopped.

Examples

Process all documents in a folder recursively:

documix /path/to/documents -r

Process only specific file types:

documix /path/to/documents -e pdf,docx,zip

Exclude certain files:

documix /path/to/documents -x "temp.*,backup.*"

Email Mode

DocuMix provides specialized handling for email files (.eml) with intelligent output formatting.

Email Processing Features

1. Automatic Attachment Detection: If an "attachments" folder exists next to the .eml file, DocuMix will use those files instead of extracting from the email
2. Email Parsing: Extracts metadata (From, To, Subject, Date, etc.) and converts HTML content to Markdown
3. Attachment Processing: All supported attachment types (PDF, DOCX, etc.) are automatically processed and included
4. Smart Output Format: Automatically uses email-specific format that includes:
   • Email metadata display (sender, recipients, date, subject)
   • Authentication information (SPF, DKIM, DMARC)
   • Attachment summaries with file types and sizes
   • Clean, email-focused presentation

Output Format Detection

DocuMix automatically detects and uses the appropriate output format:

• Single Email: Always uses dedicated email analysis format
• Multiple Emails: Treats each email as a separate document in standard format
• Mixed Content: Uses standard format when processing emails with other documents

Email Processing Examples

Process a single email file:

documix email.eml -o email_output.md

Process an email directory structure:

# Directory structure:
# emails/
# ├── message.eml
# └── attachments/
#     ├── document.pdf
#     └── report.docx

documix emails/ -o consolidated_email.md

Process multiple emails:

documix /path/to/emails -r -e eml

Force standard format for email processing:

documix email.eml -o output.md --standard-format

PDF Table Extraction

DocuMix can detect tables in PDFs and output them as markdown tables, preserving structure that would otherwise be lost with plain text extraction. This is especially useful for invoices, reports, and other documents with tabular data.

MinerU (best quality — ML-based)

If MinerU is installed, documix uses it as the top-priority PDF converter. MinerU uses ML models (YOLO layout detection, SLANet table recognition, OCR) to correctly detect complex tables — including borderless tables in invoices — and produces structured markdown output.

pip install magic-pdf[full]

MinerU is called via CLI only (subprocess) to keep its AGPL licence separate from documix's MIT licence.

PaddleOCR (ML-based OCR and document analysis)

PaddleOCR PP-StructureV3 provides ML-based document analysis with OCR, table recognition, and layout detection. It is Apache 2.0 licensed. PaddlePaddle requires Python 3.12 or earlier, so PaddleOCR runs in its own uv-managed venv via subprocess.

Install via the paddleocr extra:

pip install "documix[paddleocr] @ git+https://github.com/lchojnowski/documix.git"

Or install PaddleOCR standalone and it will be auto-detected:

uv tool install paddleocr[doc-parser]

PaddleOCR renders tables as HTML internally; documix automatically converts them to markdown tables.

pdfplumber (rule-based)

Install with table support for rule-based table detection as a fallback:

pip install 'documix[tables] @ git+https://github.com/lchojnowski/documix.git'

Or with uvx:

uvx --from 'documix[tables] @ git+https://github.com/lchojnowski/documix.git' documix /path/to/folder -r

PDF conversion priority

The default order tries each converter until one succeeds:

1. MinerU — ML-based layout/table detection (if installed)
2. pdfplumber — rule-based table detection (if installed)
3. markitdown (via uvx or direct) — plain text
4. pdftotext — plain text with layout preservation
5. PaddleOCR — ML-based OCR and document analysis (if installed)

You can override this order with --pdf-converters. Without any table-aware tool installed, documix falls back to markitdown or pdftotext for PDF conversion (plain text, no table structure).

Benchmarking

DocuMix includes a built-in benchmark command to compare converter speed and accuracy on your system.

documix benchmark

This automatically discovers and benchmarks all files in the resources/ directory using every available converter. Results are saved to the benchmark/ directory.

Benchmark Options

Flag	Description
files (positional)	Additional files to benchmark (on top of resources/)
--runs N	Number of timing iterations per converter (default: 3)
--output-dir DIR	Results directory (default: benchmark/)
--formats FMT	Which formats to benchmark: pdf, docx, rtf, or all (default: all)

Examples

Benchmark only PDF converters with 5 runs:

documix benchmark --formats pdf --runs 5

Benchmark with an additional file:

documix benchmark /path/to/my/document.pdf

Output

The benchmark produces:

• benchmark/results.json — Full timing, accuracy, and system info
• benchmark/converter_ranking.json — Converters ranked by combined speed/accuracy score per format
• benchmark/outputs/ — Raw converter outputs for manual comparison

The ranking file is informational only — it does not override the default converter order at runtime.

Supported File Types

Extension	Converter	External dependency
.pdf	MinerU, pdfplumber, markitdown, pdftotext, PaddleOCR; opt-in Tesseract	See PDF sections above
.epub	ebook-convert	Calibre
.docx	pandoc, docx2txt	pandoc (optional, recommended)
.doc	soffice	LibreOffice
.xlsx, .pptx	installed markitdown	MarkItDown with spreadsheet and presentation support
.rtf	pandoc, unrtf, striprtf	pandoc (optional), unrtf, or pip install striprtf
.eml	built-in	—
.zip	built-in (auto-extract)	—
.txt, .md, .py, .js, etc.	direct read	—

XLSX spreadsheets and PPTX presentations are included by default and use the selected parser from the locally installed MarkItDown environment. Missing dependencies, malformed archives, empty output and exceeded limits produce explicit conversion failures; compressed Office bytes are never treated as document text or passed to a generic ZIP fallback. Conversion does not install packages or download models.

For unattended document screening, select --office-require-text-only. This refuses Office documents with embedded or linked images, even when a heading or other selectable text is available, because the image might contain the only relevant requirement. The result is an explicit failed conversion with reason unassessed_media; keep it incomplete for review. This conservative check can also reject ordinary logos. It does not perform Office image OCR and does not claim general visual or layout coverage. It is independent of PDF OCR and does not change default Office text extraction.

documix /path/to/documents -r --office-require-text-only

The library exposes the same policy with DocumentCompiler(..., office_require_text_only=True); its default is False.

Office conversion checks the ZIP contents without extracting archive paths and passes a private read-only copy to the parser. Limits are 128 MiB input, 10,000 archive members, 32 MiB per expanded member, 256 MiB total expanded content, a 1,000:1 compression ratio, 4 MiB text output and 60 seconds overall. Temporary copies are removed, originals remain unchanged, and unconfirmed cleanup stops compilation. Use an external sandbox, memory limits and supervisor for untrusted documents. Default text extraction does not assess Office images; use the strict option when their omission must leave the document visibly incomplete.

Dependencies

Core Dependencies (installed automatically)

• docx2txt: For DOCX file processing (fallback method)
• html2text: For converting HTML email content to Markdown

Optional Extras

• documix[pdf] — markitdown for PDF text extraction
• documix[office] — MarkItDown with XLSX and PPTX parsing dependencies (Python 3.10+)
• documix[tables] — pdfplumber for rule-based PDF table detection
• documix[paddleocr] — PaddleOCR + PaddlePaddle for ML-based document analysis

Optional External Tools

• MinerU (pip install magic-pdf[full]) — ML-based PDF layout/table detection via CLI
• pandoc — high-quality DOCX and RTF conversion (primary converter)
• Calibre (ebook-convert command) — EPUB conversion
• LibreOffice (soffice command) — DOC conversion
• poppler-utils (pdftotext command) — plain text PDF extraction; pdfinfo and pdftoppm are also required for opt-in OCR
• Tesseract with local language packs — opt-in scanned PDF text extraction
• unrtf — RTF conversion fallback

License

This project is licensed under the MIT License - see the LICENSE file for details.