Skip to content

Name an oversized CSV cell as an inventory error - #920

Merged
d-chambers merged 1 commit into
devfrom
csv-error-boundary
Aug 16, 2026
Merged

Name an oversized CSV cell as an inventory error#920
d-chambers merged 1 commit into
devfrom
csv-error-boundary

Conversation

@d-chambers

Copy link
Copy Markdown
Contributor

Description

Follow-up to a review comment on #911, which was deliberately deferred there because that PR was a pure refactor and this behavior predates it on dev.

read_table builds a csv.reader to check the header and the row widths before handing the file to pandas. A cell longer than csv.field_size_limit() (131072 by default) stops that scan with _csv.Error, which the handler did not catch. The error therefore left the function as a standard library exception rather than as the caller's own, and dc.inventory(path) escaped its contract that anything malformed raises InvalidInventoryError:

>>> dc.inventory(root)
_csv.Error: field larger than field limit (131072)

csv.Error now joins OSError and UnicodeDecodeError in that handler, so the loader's boundary in _load_table translates it like any other unreadable table.

Refusing rather than raising the limit is deliberate: pandas has no such limit and would read the file, but the csv scan exists only to check the header and row widths, and raising a process-wide limit to accommodate a pathological cell trades a clear error for unbounded memory. The inventory format is meant to be hand-authored spreadsheets, where a single cell over 128 KB is not a thing to accept quietly.

Two tests, one at each level: read_table raises ParameterError, and dc.inventory on a directory holding such a table raises InvalidInventoryError. Both were confirmed to fail without the one-line change.

Changelog

  • fixed: An inventory table with a cell larger than the CSV field size limit now raises InvalidInventoryError rather than a bare _csv.Error.

A cell longer than csv.field_size_limit stops the header and row-width
scan, and read_table did not catch that, so it left the function as a
bare _csv.Error rather than as the caller's own error. Loading an
inventory holding one therefore escaped the contract that anything
malformed raises InvalidInventoryError.

Pandas would read such a file; the csv scan which refuses it runs first
and exists only to check the header and the row widths. What matters is
that it stops as an inventory error rather than as a standard library
one, which is what the boundary in _load_table now sees.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@d-chambers, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 7 minutes

Limit details: You’ve used all 2 included reviews currently available under your plan.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a3f24215-7542-4578-9e3a-69af2a897fd5

📥 Commits

Reviewing files that changed from the base of the PR and between 3f552b8 and 82c3b5c.

📒 Files selected for processing (3)
  • dascore/utils/tables.py
  • tests/test_core/test_inventory_loader.py
  • tests/test_utils/test_tables.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@d-chambers d-chambers added the ready_for_review PR is ready for review label Aug 16, 2026
@codecov

codecov Bot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (54f85a6) to head (82c3b5c).
⚠️ Report is 3 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff            @@
##               dev      #920   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          185       185           
  Lines        22315     22387   +72     
=========================================
+ Hits         22315     22387   +72     
Flag Coverage Δ
network 45.73% <0.00%> (-0.09%) ⬇️
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@d-chambers
d-chambers merged commit 619de33 into dev Aug 16, 2026
31 checks passed
@d-chambers
d-chambers deleted the csv-error-boundary branch August 16, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready_for_review PR is ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant