Skip to content

get_paper silently caps chunk_count at 10,000 #542

Open

Description

@chris-dare-dev

Severity low · Subsystem mcp-surface

Problem

handle_get_paper (server/handlers/paper.py, the chunks query, around line 169 on main at d9ddaa8) fetches a paper's rows with .limit(10000), then reports:

  • chunk_count = arrow.num_rows;
  • section_count from the same rows;
  • chunker_version and embedder_version from the first row.

A paper with more than 10,000 chunks reports exactly 10,000, and its section count covers only those rows. Nothing in the response says the numbers were truncated, which breaks the CLAUDE.md §4.9 rule that a degraded answer must never look complete.

10,000 chunks is not reachable by an ordinary arXiv paper, but textbook ingest puts a whole book under one paper_id (textbook:<slug>), and a long monograph or a multi-volume text could approach it.

Found while fixing #353 in #538. That PR changed which columns the query reads, not the limit.

Suggested remedy

Take the count from chunks_table.count_rows(filter) rather than from the rows fetched. Fetch the summary columns without a row cap, or derive section_count from a projection of section_path alone. If any cap stays, the response should say when it was hit rather than return a number that looks exact.

Add a test against a real LanceDB table with more rows than the cap (or with the cap patched down) that expects the true count. tests/test_handlers_paper.py from #538 has the fixture.

Activity

  1. added
    type:findingAdversary/critique finding to remediate
    area:mcp-surfaceMCP tool/resource surface, handlers, shim, session
    on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:mcp-surfaceMCP tool/resource surface, handlers, shim, sessionsev:lowLow severitytype:findingAdversary/critique finding to remediate

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions