pdf-toolkit

Inspect a PDF, compare how four Python PDF libraries see it, shrink it, split it into chapters, dissect its forms, or make one from an office document — from one package, one CLI.

What it does

• Inspect — metadata, page count, scanned-vs-digital analysis, and a structural X-Ray explorer, with a unified adapter for pypdf, pdfminer.six, PyMuPDF, and pdfplumber.
• Compare libraries — per-library examples in examples/ that extract text, pages, and metadata the same way, so you can see where the libraries disagree.
• Find a table of contents — outline extraction plus a text heuristic for manuals whose TOC is just printed text, with a hint page for where it starts.
• Split — one PDF per chapter, by bookmarks, a title,page CSV, located headings, or the printed TOC text. Dry-run by default; existing outputs are never silently overwritten.
• Dissect — find templated form pages by keyword or image similarity, detect their field boxes, label them with OCR, and rebuild each page as an interactive HTML overlay, a fillable PDF, and a DOCX, with a local preview server for editing fieldmaps. Needs pdf-toolkit[dissect].
• Export — convert a PDF to Markdown, HTML, DOCX, or a structured JSON report.
• Compress — three methods in one engine:
  • structural (default): PyMuPDF garbage collection + stream deflation + clean. Text stays selectable, vectors stay sharp, nothing is rasterized.
  • pypdf: rewrite with content-stream compression, optionally dropping metadata.
  • ghostscript: Ghostscript pdfwrite presets (screen, ebook, printer, prepress). Ghostscript is an external binary, not a pip dependency.
• GUI — an optional PyQt6 drag-and-drop window with first-page preview, for the compressor.
• Convert to PDF — pdf-toolkit convert turns a .docx or .xlsx file into a PDF. The .docx path uses docx2pdf (needs Word); the .xlsx path drives Excel through COM, so it needs Windows with Excel. Wide sheets go landscape automatically, very wide sheets fit to one page wide.
• Check .docx formatting — pdf-toolkit check-format checks a Word document against formatting rules (font, body/caption/footnote sizes, line spacing, margins — all overridable on the command line) and reports every issue. With an output path it can also save an annotated copy with Word comments, which needs Windows with Word.
• Check citations — pdf-toolkit check-citations flags DOCX manuscript text similar to a PDF source, for manual citation review. Needs pdf-toolkit[citations] for the DOCX reader. Similarity alone never proves a passage is uncited; matches are reported with the source page, snippet, and context for a human to judge.

Quick start

pip install pdf-toolkit
# or, with the GUI:
pip install "pdf-toolkit[gui]"

# Inspect a PDF
pdf-toolkit inspect document.pdf
pdf-toolkit inspect document.pdf --json
pdf-toolkit inspect document.pdf --toc --hint 6

# Compress it (interactive: inspect, then confirm each file)
pdf-toolkit compress document.pdf
# Batch a directory, no prompts, into ./compressed/
pdf-toolkit compress ./pdfs/ -o ./compressed/ -y --method structural
# pypdf, or Ghostscript at ebook quality
pdf-toolkit compress document.pdf -o ./out/ -y --method pypdf
pdf-toolkit compress document.pdf -o ./out/ -y --method ghostscript --gs-quality ebook

# GUI
pdf-toolkit gui
# or the browser frontend (loopback only by default; --demo serves seeded data)
pdf-toolkit web
pdf-toolkit web --demo
# Freeze the demo to static HTML for a GitHub Pages demo (view-only)
pdf-toolkit web --freeze-dir ./_site

# Split into chapters (dry-run first: prints the proposed ranges)
pdf-toolkit split document.pdf
pdf-toolkit split document.pdf -o ./chapters/ --yes
# From an explicit chapter list, or by locating headings in the text
pdf-toolkit split document.pdf --csv chapters.csv --yes
pdf-toolkit split document.pdf --headings headings.txt --yes
# No bookmarks? Fall back to the printed TOC (--offset shifts printed
# page numbers to physical pages when there is front matter)
pdf-toolkit split document.pdf --toc-text --hint 2 --offset 4 --yes
# Just dump the resolved chapter list
pdf-toolkit split document.pdf --export-csv chapters.csv

# Dissect form pages (needs pdf-toolkit[dissect])
pdf-toolkit dissect detect document.pdf --keyword "Form Title" --keyword Signature
# Rebuild matched pages: images/, html/ (overlay + fieldmap JSON),
# fillable_pdfs/, extracted_pdfs/ under ./recon/
pdf-toolkit dissect reconstruct document.pdf --keyword "Form Title" --out-dir ./recon/
# Fill from a fieldmap and data JSON (per-page keys, or a "global" mapping)
pdf-toolkit dissect fill document.pdf --fieldmap ./recon/ --data data.json --out-dir ./filled/
# Export pages to DOCX, optionally filled
pdf-toolkit dissect export-docx --fieldmap ./recon/ --out-dir ./docx/ --fill --data data.json
# Preview and edit fieldmaps locally (loopback only, confined to --root; a non-loopback --host needs --allow-remote)
pdf-toolkit dissect serve --root ./recon/

# Convert to PDF (needs pdf-toolkit[office]; see limitations below)
pdf-toolkit convert document.docx
pdf-toolkit convert spreadsheet.xlsx -o report.pdf
pdf-toolkit convert ./docs/ -o ./pdfs/

# Check .docx formatting (exit 0 = compliant, 1 = issues found)
pdf-toolkit check-format document.docx
pdf-toolkit check-format document.docx --font Calibri --body-size 12 --margin 1.0
pdf-toolkit check-format document.docx -o document-annotated.docx
pdf-toolkit check-format ./docs/ -o ./format-check/ --annotate

# Flag DOCX text similar to a PDF source (needs pdf-toolkit[citations];
# exit 0 = no matches, 1 = matches found, 2 = usage/file error)
pdf-toolkit check-citations manuscript.docx source.pdf --json-out report.json
pdf-toolkit check-citations manuscript.docx source.pdf --threshold 90 --skip-cited

From a checkout, the same commands work after pip install -e ".[dev]", and the library API is importable directly:

from pathlib import Path
from pdf_toolkit.compressor import compress_pdf, quick_inspect
from pdf_toolkit.inspector.adapters.pymupdf import PyMuPDFAdapter

info = quick_inspect(Path("document.pdf"))
print(info["page_count"], info["image_count"])

result = compress_pdf(Path("document.pdf"), Path("document_compressed.pdf"), "structural")
print(result.success, result.reduction_pct)

with PyMuPDFAdapter("document.pdf") as adapter:
    print(adapter.get_metadata()["document_type"])
    print(adapter.get_toc(hint_page=6)[:3])

from pdf_toolkit.splitter import extract_outline_entries, split_pdf

entries, _warnings = extract_outline_entries("document.pdf")
split_pdf("document.pdf", entries, "./chapters/")

from pdf_toolkit.dissect import find_template_pages, reconstruct_pdf_pages

result = find_template_pages("document.pdf", ["Form Title", "Signature"])
reconstruct_pdf_pages("document.pdf", result.matched_pages, "./recon/")

Install options

Extra	What it adds
(none)	Core: PyMuPDF, pypdf, pdfminer.six, pdfplumber, rich
pdf-toolkit[gui]	PyQt6 GUI
pdf-toolkit[web]	Flask browser frontend (web, --demo, --freeze-dir)
pdf-toolkit[tables]	Camelot + tabula-py for --tables (tabula-py also needs Java)
pdf-toolkit[ocr]	pytesseract, pdf2image, Pillow, numpy for OCR paths
pdf-toolkit[export]	pymupdf4llm + pdf2docx for --export md / --export docx
pdf-toolkit[office]	python-docx + docx2pdf + openpyxl for convert and check-format (pywin32 on Windows)
pdf-toolkit[dissect]	OpenCV + NumPy + Pillow + ReportLab + Flask + python-docx + docxtpl + pytesseract for dissect (Tesseract binary still needed separately for OCR labels)
pdf-toolkit[citations]	python-docx for check-citations (RapidFuzz is a core dependency)
pdf-toolkit[dev]	pytest + ruff

Optional features fail with a plain "install this extra" message instead of an import error at startup: the CLI imports Camelot, tabula, pytesseract, the export converters, and the office converters lazily, only when that feature is used.

Examples and scripts

• examples/ — one folder per library (pypdf, pdfminer, pdfplumber, pymupdf), each with text/page/metadata examples and its own tests, all run against the generated examples/sample_example.pdf. No generated outputs are shipped; run an example and its outputs land in examples/<lib>/outputs/.
• scripts/generate_sample_pdf.py — regenerate the sample PDF.
• scripts/collect_metadata.py — collect per-library metadata JSON into examples/outputs/aggregated/combined_metadata.{json,csv}.
• scripts/dashboard.py — build the aggregated HTML dashboard from that JSON.
• scripts/clean_examples.py — delete generated example outputs (keeps .gitkeep).

python scripts/generate_sample_pdf.py
python examples/pypdf/extract_text.py examples/sample_example.pdf
python scripts/collect_metadata.py
python scripts/dashboard.py
python -m pytest -q

Project layout

pdf-toolkit/
├── src/pdf_toolkit/
│   ├── cli.py                 # pdf-toolkit inspect / compress / split / dissect / gui / convert / check-format / check-citations
│   ├── splitter/              # chapter splitting by bookmarks, CSV, headings, or TOC text
│   ├── dissect/               # form-page detection, field detection, reconstruction, preview server
│   ├── citations/             # DOCX-vs-PDF similarity flagging for citation review
│   ├── to_pdf/                # docx/xlsx to PDF, .docx formatting checker
│   ├── inspector/             # adapters, TOC parsers, analysis, extraction
│   │   ├── adapters/          # pypdf, pdfminer, pymupdf, pdfplumber
│   │   ├── toc/               # outline + text-TOC heuristics
│   │   ├── analysis/          # scanned-document detection, X-Ray explorer
│   │   └── extraction/        # tables (optional), converters, OCR (optional)
│   ├── compressor/            # structural / pypdf / ghostscript engines
│   └── gui/                   # PyQt6 GUI (optional)
├── examples/                  # per-library examples + sample PDF
├── scripts/                   # sample generator, aggregator, dashboard, cleaner
└── tests/                     # package tests (examples carry their own too)

Known limitations

• Split chunks keep the source document's metadata but not its bookmarks; each chunk starts at its chapter's first page. Printed TOC numbers are used as physical pages unless --offset is given, so verify the proposed ranges in the dry run before --yes — that check is the point of the dry run. Heading location is text-based and needs extractable text; scanned PDFs need an OCR pass first.
• The text-TOC heuristic was tuned on printed manuals; it needs a sensible --hint page for documents whose contents page is not near page 6.
• Ghostscript and tabula-py depend on external runtimes (Ghostscript, Java). If they are not installed, those paths report a clear error and the rest of the toolkit still works.
• The GUI is not covered by automated tests (it needs a display); CI covers the CLI, the library, and the examples.
• Office conversion is not covered by automated tests either: .xlsx conversion needs Windows with Excel, and .docx conversion needs Word. CI covers the dispatch, the Excel page-setup rules, and the formatting checker (which runs anywhere python-docx runs); the actual conversions are verified by hand on a Windows machine. Word-comment annotation likewise needs Windows with Word — the report itself works everywhere.
• The formatting checker's defaults (Calibri, 12pt body / 10pt captions and footnotes, 1.0 spacing, 1-inch margins) come from the original project's style guide; they are defaults, not a standard, so pass your own values if your documents use different rules.
• Table extraction with Camelot works best on bordered tables; unbordered tables may need the stream fallback or pdfplumber's own table finder in the examples.
• Dissect field detection is a contour heuristic: it finds rectangular boxes, so nested boxes and unboxed fields are missed or merged, and OCR labels are one word per field. The preview server has no authentication and is confined to its --root directory — keep it on loopback; a non-loopback --host is refused unless --allow-remote is passed. The original's WeasyPrint HTML-to-PDF path is not ported (it needs system Cairo/Pango libraries); filled PDFs come from dissect fill instead.
• Citation checking is a similarity flag, not a plagiarism or citation verdict: it cannot tell a properly quoted or cited passage from an uncited one except for the explicit markers --skip-cited recognizes, it uses a lightweight sentence splitter (abbreviations can split early), and it does not read DOCX headers, footers, or footnotes. Scanned PDFs need an OCR pass first, as with the other text paths.

History

pdf-toolkit merges seven earlier projects into one package, with a single CLI and a single dependency story:

• ss-py-pdf-inspector — the inspector, its adapters, TOC parsers, dashboard, and examples. Its old tests that required a private, unshipped manual PDF were replaced with tests against a synthetic TOC fixture built in the test, and its generated example outputs were dropped.
• ss-pdf-small — the CLI batch compressor. Its PyMuPDF structural pass is now the default structural method, and its inspect/estimate step is the compressor's preflight.
• ss-pdf-tool — the PyQt6 GUI compressor. Its pypdf and Ghostscript methods are now compressor methods, and the GUI imports the package instead of reaching for a sibling file via sys.path.
• ss-toPDF (merged in 0.2.0) — DOCX/XLSX to PDF conversion plus the .docx formatting checker. Its converters are now pdf-toolkit convert, and its checker is now pdf-toolkit check-format with the rules as parameters instead of module constants. Its example documents were real work files and are not shipped; the checker tests build synthetic documents in-test instead. The merge also fixed the original batch script, which imported three checker functions that did not exist, and removed hardcoded local paths from the checker.
• ss-HESI (merged in 0.3.0) — scripts for splitting a PDF by bookmarks/TOC text, locating headings, and cleaning split filenames. Its logic is now pdf-toolkit split and the pdf_toolkit.splitter library, generalized off the one book it was built for: no hardcoded paths, one shared CSV parser and filename sanitizer, heading location in a single pass over the PDF, and the outline page resolution the inspector's TOC extractor was missing (it previously returned titles with no pages). The original's bugs are fixed, not carried over: a duplicated output write, a documented-but-unimplemented --preview-only flag, a TOC scan that treated every numbered line on the first ten pages as an entry, and printed page numbers silently used as physical ones. Splitting is verified against a synthetic fixture built in-test; no book content is shipped, and the source PDF is never part of this repo.
• ss-documentDissection (merged in 0.4.0) — a pipeline for detecting templated form pages and rebuilding them as fillable HTML, PDF, and DOCX, with a Flask preview server. Its logic is now pdf-toolkit dissect and the pdf_toolkit.dissect library. The original's bugs are fixed, not carried over: fieldmaps written with str() and read back with eval() (now JSON throughout), a --keywords option that swallowed subcommand names (now repeatable --keyword), --to-html-dir handling nested inside the wrong subcommand and crashing without --reconstruct-dir, a preview server that served and wrote files relative to the working directory with no confinement (now confined to --root, with absolute and escaping paths rejected), a /fill-document endpoint that passed a filename where bytes were expected (now fills and writes), and PDF filling that only ever filled page 1. The WeasyPrint HTML-to-PDF path is not ported; no sample documents are shipped, and the original's history, which carried a personal email address, was not carried over.
• ss-missedCitations (merged in 0.5.0) — a CLI comparing a DOCX manuscript against a PDF source with thefuzz partial_ratio sliding windows. Its logic is now pdf-toolkit check-citations and the pdf_toolkit.citations library, generalized off the one RFP/checklist pair it was built for: no sample documents are shipped (the original's test documents were a real RFP and a real proposal checklist), and the original's bugs are fixed, not carried over: raw-text scoring with no normalization or token prefilter, whose shipped reports were all score-100 footnote fragments (1., P.) and whose current code finds nothing on its own sample documents; DOCX tables silently ignored; two divergent sentence splitters; an unvalidated threshold; a missing file crashing with a traceback; and tests that only ran with PYTHONPATH=src. The thefuzz/Levenshtein stack is replaced with RapidFuzz, and the full-page comparison fallback (which made the "best snippet" a page of unrelated text) is dropped.
• 0.6.0 (staff review fixes, no new commands) — the version is now single-sourced from pdf_toolkit.__version__ (0.5.0 shipped with pyproject at 0.5.0 and __init__ at 0.4.0, and license = "MIT" in pyproject, which current setuptools rejects; both are fixed and the wheel builds as 0.6.0). Compression and splitting are atomic: a failed run leaves no partial output, and a failed split leaves no chapter files. An invalid page spec in the export converters now raises instead of silently exporting different pages per converter. The preview server refuses a non-loopback host unless --allow-remote is passed. import fitz is replaced with import pymupdf throughout (PyMuPDF 2.0 still ships fitz only as a deprecated shim that warns on import, and it will be removed in a future release). The CLI is split into a cli/ package (one module per command family; from pdf_toolkit.cli import main still works), optional imports go through one shared helper, shared defaults live in _defaults.py, citations window scoring reuses precomputed normalized text, and mypy now runs in CI with zero errors on src/pdf_toolkit.
• 0.6.1 (mypy CI fix, no behaviour change) — the 0.6.0 mypy gate failed in CI on ocr_utils.py: mypy infers a module variable's type from its first assignment, so cv2 = None in the optional-import fallback fails as soon as OpenCV is installed (CI installs the dissect extra; a checkout without it sees Any and passes silently, which is why the failure was invisible locally). The fallback assignments now carry # type: ignore[assignment], the same treatment applied to the export converters' optional imports, and the image array in the OCR path is annotated as a bare ndarray so its preprocessing reassignment typechecks. CI now also runs mypy in the extras-smoke job, where every extra is installed, so this class of error is caught even for modules the main job's install set doesn't cover.
• 0.6.2 (mypy CI fix, no behaviour change) — the 0.6.1 extras-smoke mypy step failed in CI on PyQt6's own QtGui.pyi stubs, which contain invalid syntax (*args, /). The first fix attempt, --exclude 'gui', did not work, and the reason matters: the chain is not our gui package at all. Pillow's Image module imports ImageQt, which imports PyQt6, so mypy parses the broken stubs whenever Pillow and PyQt6 are both installed, and mypy follows imports regardless of --exclude (and follow_imports = "skip" does not help either, since mypy still parses the stub to resolve the import). The mypy step therefore moved out of extras-smoke into its own job that installs every extra except gui; with PyQt6 absent its stubs are never parsed, and the gui files are still checked with Qt names as Any. That job also caught a real error the previous runs had hidden: ocr_engine.py passed None for first_page/last_page, which pdf2image's typed signature rejects, so it now only passes those arguments when a page range is given (its runtime defaults are already None, so behaviour is unchanged).
• 0.7.0 (Flask web frontend, PyQt GUI kept) — new pdf-toolkit web command (needs pdf-toolkit[web]): a local Flask app for uploading a PDF, inspecting it, compressing it with any method, and downloading the result, with the same loopback-only host guard as the dissect preview server. pdf-toolkit web --demo serves seeded fixture data instead of processing uploads, and pdf-toolkit web --freeze-dir renders that demo to static HTML for a GitHub Pages demo; the frozen pages render their forms disabled and labelled view-only, because Pages cannot run the server, and there is deliberately no demo download. Demo numbers are real measurements of examples/sample_example.pdf (3 pages, 3,784 bytes, structural compression to 1,756 bytes), frozen in web/demo_data.py, not invented. A Pages workflow deploys the frozen demo on push to main (one-time owner step: Settings → Pages → Source: GitHub Actions).
• 0.7.1 (frozen demo link fix) — the 0.7.0 frozen demo 404'd on the sample page: its templates render absolute links (/demo/sample), which on a project Pages site served under /pdf-toolkit/ resolve off-site. freeze_demo now rewrites the frozen routes' links to paths relative to each page, so the frozen site works at any subpath; the live server is unchanged. Also fixed in the same release: the Pages workflow pairs upload-pages-artifact@v3 with deploy-pages@v4 (v2 could not find the v3 artifact) plus configure-pages@v5.
• 0.7.2 (web frontend: dark mode, sample PDF on the demo page) — the web templates are now theme-aware: they follow the system light/dark theme via prefers-color-scheme, with a toggle button that remembers an explicit choice in localStorage (nothing is sent anywhere). The demo sample page now shows the source next to the output: it embeds the real sample PDF above the inspection and compression results. The PDF is bundled with the web package (web/demo_sample.pdf, shipped via package data) and copied into the frozen build next to the sample page, so the embed works on Pages at any subpath too.

Two fixes from the merge are worth knowing about: the old inspector's example metadata scripts imported a normalize_pdf_metadata name the package never exported (it is now an alias for normalize_metadata), and the old tool's module created a pdf_compressor.log file in the working directory on import (logging is now the caller's job, and importing the compressor has no side effects). The old projects' single-commit histories, which carried personal email addresses, were not carried over; this repo starts with a clean history under the enonymous1 identity.

License

MIT — see LICENSE.