Skip to content

Scanned PDFs report 0 tokens and no error #54

Description

@Phantom-VK

Problem

The worst silent failure in the codebase. pypdf's extract_text() returns an empty string for image-only pages, so a scanned 90-page contract flows through the happy path and produces char_count: 0, token_count: 0, error: None. The UI shows a completed analysis of an empty document. A user checking whether a scanned contract fits in context gets "yes, 0 tokens" and acts on it.

Where

  • parsing.py:32"\n".join(page.extract_text() or "" ...)
  • service.py:56 — builds a normal AnalysisResult, error=None

Fix

Detect a PDF that parsed with pages but near-zero extractable text, and surface it as a distinct warning state ("this PDF appears to be scanned images, no text layer found"), not as a result. Deliberately not an error, since the file did parse. OCR is a separate enhancement.

Impact

Silent undercounting is the one failure mode this tool cannot afford, since avoiding a surprise bill is the whole point.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:parsingcore/parsing.py — PDF/PPTX/DOCX/TXT extractionbugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions