Skip to content

fix: check HTTP status before parsing JSON in multipart upload#146

Merged
Joffref merged 1 commit into
mainfrom
devin/1779327288-fix-multipart-upload-json-error
May 21, 2026
Merged

fix: check HTTP status before parsing JSON in multipart upload#146
Joffref merged 1 commit into
mainfrom
devin/1779327288-fix-multipart-upload-json-error

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented May 21, 2026

Summary

Fixes JSONDecodeError: Expecting value: line 1 column 1 (char 0) when uploading large files via write_binary() / write() with multipart upload.

Root cause: In the async multipart upload methods (_upload_part, _initiate_multipart_upload, _complete_multipart_upload), json.loads(await response.aread()) was called before handle_response_error(response). When the server returned an error with an empty body (e.g., 502 from the gateway under load), json.loads blew up with a cryptic JSONDecodeError instead of a clear ResponseError with status code info.

Changes:

  1. Reordered handle_response_error() before json.loads() in all three async multipart methods so HTTP errors surface as ResponseError with status code/body info.
  2. Reduced MAX_PARALLEL_UPLOADS from 20 → 3 (matching the TypeScript SDK) to reduce the likelihood of overwhelming the sandbox API.
  3. Replaced print() with logger.warning() in _abort_multipart_upload (per repo conventions: no print() in library code).

The sync version already had the correct ordering (handle_response_error before response.json()); only the parallel upload limit was updated there.

Review & Testing Checklist for Human

  • Upload a file >5MB via sandbox.fs.write_binary() to confirm multipart upload succeeds end-to-end
  • Verify that a server error during upload now raises a ResponseError with status info instead of JSONDecodeError
  • Check that reducing parallel uploads from 20 to 3 doesn't noticeably regress upload throughput for very large files

Notes

  • The non-multipart methods (mkdir, write for small files, etc.) have the same json.loads-before-error-check pattern but are much less likely to hit empty-body errors since they're simpler single-request operations. Could be a follow-up cleanup.
  • The sync version's _upload_part already had the correct ordering; only the MAX_PARALLEL_UPLOADS constant was changed there.

Link to Devin session: https://app.devin.ai/sessions/dd4c5d3cf8d74415a761f7b614cd5184
Requested by: @Joffref


Note

Fixes a bug where HTTP error responses with empty bodies (e.g., 502s) during multipart upload caused a cryptic JSONDecodeError instead of a proper ResponseError. The fix reorders handle_response_error() before json.loads() in the three async multipart methods, reduces MAX_PARALLEL_UPLOADS from 20 to 3 to match the TypeScript SDK, and replaces a print() with logger.warning().

Written by Mendral for commit 9c53176.

- Reorder handle_response_error() before json.loads() in _upload_part,
  _initiate_multipart_upload, and _complete_multipart_upload so that
  server errors (empty body, 502, etc.) raise a clear ResponseError
  instead of a cryptic JSONDecodeError.
- Reduce MAX_PARALLEL_UPLOADS from 20 to 3 (matching the TypeScript
  SDK) to avoid overwhelming the sandbox API.
- Replace print() with logger.warning() in _abort_multipart_upload.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Copy link
Copy Markdown
Contributor

@mendral-app mendral-app Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The fix is correct. handle_response_error() raises on error status codes, preventing json.loads() from running on empty/error bodies — the finally block still closes the response regardless. The dataresult rename in _upload_part also eliminates a variable shadowing issue with the data: bytes parameter. No correctness, security, or data-loss issues.

Tag @mendral-app with feedback or questions. View session

Copy link
Copy Markdown
Contributor

@Joffref Joffref left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Joffref Joffref merged commit 28b95c8 into main May 21, 2026
18 checks passed
@Joffref Joffref deleted the devin/1779327288-fix-multipart-upload-json-error branch May 21, 2026 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant