Name an oversized CSV cell as an inventory error - #920
Conversation
A cell longer than csv.field_size_limit stops the header and row-width scan, and read_table did not catch that, so it left the function as a bare _csv.Error rather than as the caller's own error. Loading an inventory holding one therefore escaped the contract that anything malformed raises InvalidInventoryError. Pandas would read such a file; the csv scan which refuses it runs first and exists only to check the header and the row widths. What matters is that it stops as an inventory error rather than as a standard library one, which is what the boundary in _load_table now sees.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 7 minutes Limit details: You’ve used all 2 included reviews currently available under your plan. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #920 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 185 185
Lines 22315 22387 +72
=========================================
+ Hits 22315 22387 +72
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Description
Follow-up to a review comment on #911, which was deliberately deferred there because that PR was a pure refactor and this behavior predates it on
dev.read_tablebuilds acsv.readerto check the header and the row widths before handing the file to pandas. A cell longer thancsv.field_size_limit()(131072 by default) stops that scan with_csv.Error, which the handler did not catch. The error therefore left the function as a standard library exception rather than as the caller's own, anddc.inventory(path)escaped its contract that anything malformed raisesInvalidInventoryError:csv.Errornow joinsOSErrorandUnicodeDecodeErrorin that handler, so the loader's boundary in_load_tabletranslates it like any other unreadable table.Refusing rather than raising the limit is deliberate: pandas has no such limit and would read the file, but the csv scan exists only to check the header and row widths, and raising a process-wide limit to accommodate a pathological cell trades a clear error for unbounded memory. The inventory format is meant to be hand-authored spreadsheets, where a single cell over 128 KB is not a thing to accept quietly.
Two tests, one at each level:
read_tableraisesParameterError, anddc.inventoryon a directory holding such a table raisesInvalidInventoryError. Both were confirmed to fail without the one-line change.Changelog
InvalidInventoryErrorrather than a bare_csv.Error.