Skip to content

Gracefully handle a request body that can't be deepcopied - #1024

Open
agu2347 wants to merge 1 commit into
kevin1024:masterfrom
agu2347:fix-undeepcopyable-request-body
Open

Gracefully handle a request body that can't be deepcopied#1024
agu2347 wants to merge 1 commit into
kevin1024:masterfrom
agu2347:fix-undeepcopyable-request-body

Conversation

@agu2347

@agu2347 agu2347 commented Aug 4, 2026

Copy link
Copy Markdown

Fixes #737.

before_record_request() unconditionally called copy.deepcopy(request) before running any filter functions. A request's body can contain something that isn't deepcopy-able, most commonly a raw, open file handle -- e.g. aiohttp file uploads, which are commonly passed directly as part of the request body (data={"file": open(path, "rb")}) rather than pre-read into bytes. deepcopy()ing an open file handle raises TypeError: cannot pickle '...' instances, crashing the entire recording attempt for any request containing one, exactly as described in the issue.

The request object here is VCR's own internal representation (named vcr_request throughout the aiohttp stub, for example), not a reference to the live, actual network request -- so if the deepcopy fails, falling back to using the original object directly (letting filter functions mutate it in place instead of a copy) is safe. It's still far better than crashing and preventing the upload/recording from working at all.

Testing: verified directly: confirmed deepcopy() on a real open file handle reproduces the exact reported TypeError, and confirmed _build_before_record_request()'s returned function crashes on a request whose body contains one, with the original code. With the fix, the same request is handled correctly, and header filtering still takes effect. Also verified the normal case (a request with an ordinary bytes/str body) is completely unaffected: it's still properly deep-copied and isolated from the original request object, exactly as before.

Added a regression test using a real open file handle as part of the request body, going through the actual use_cassette()/filter_request() path. I confirmed the test fails with the original code (raises TypeError) and passes with the fix. Ran the full existing test_vcr.py suite (26 passed: 25 baseline + 1 new), no regressions.

before_record_request() unconditionally called copy.deepcopy(request)
before running any filter functions. A request's body can contain
something that isn't deepcopy-able, most commonly a raw, open file
handle -- e.g. aiohttp file uploads, which are commonly passed
directly as part of the request body (data={"file": open(path, "rb")})
rather than pre-read into bytes. deepcopy()ing an open file handle
raises TypeError: cannot pickle '...' instances, crashing the entire
recording attempt for any request containing one, exactly as
described in the issue.

The `request` object here is VCR's own internal representation
(named vcr_request throughout the aiohttp stub, for example), not a
reference to the live, actual network request -- so if the deepcopy
fails, falling back to using the original object directly (letting
filter functions mutate it in place instead of a copy) is safe. It's
still far better than crashing and preventing the upload/recording
from working at all.

Verified directly: confirmed deepcopy() on a real open file handle
reproduces the exact reported TypeError, and confirmed
_build_before_record_request()'s returned function crashes on a
request whose body contains one, with the original code. With the
fix, the same request is handled correctly, and header filtering
still takes effect. Also verified the normal case (a request with an
ordinary bytes/str body) is completely unaffected: it's still
properly deep-copied and isolated from the original request object,
exactly as before.

Added a regression test using a real open file handle as part of the
request body, going through the actual use_cassette()/filter_request()
path. Confirmed the test fails with the original code (raises
TypeError) and passes with the fix. Ran the full existing
test_vcr.py suite (26 passed: 25 baseline + 1 new), no regressions.

Fixes kevin1024#737
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[>=4.4.0] Uploading file with aiohttp seems broken: TypeError: cannot pickle '_io.BufferedReader' object

1 participant