Skip to content

Answer a cut that leaves no whole file part with the limit, not "Field required" - #480

Open
SamikaPerera wants to merge 2 commits into
mainfrom
fix/oversized-decoy-says-limit
Open

SamikaPerera wants to merge 2 commits into
mainfrom
fix/oversized-decoy-says-limit

Conversation

@SamikaPerera

Copy link
Copy Markdown
Collaborator

Closes #479. Follows Shubham's note on #470 (review of Mon 16 Nov 13:56).

The change. After UploadBodyLimit cuts a body, a 422 means the endpoint never ran: FastAPI found no whole file part in what it was shown. That answer is replaced with the 400 the endpoints already use for an oversized file, carrying this message:

The upload is larger than the 2 MB limit, so it was not read. Upload the file on its own.

  • The middleware does not know which part was meant to be the file, so the message does not name one.
  • 400 rather than 413, because that is what the endpoints give for a file over the limit (upload_config's docstring explains why).
  • The response keeps its headers, so the session cookie still arrives on a first request.
  • Only a 422 after a cut is replaced. The endpoints' own refusals pass through untouched, and so does any response to a body that was not cut.

Before and after, through main.app:

Request main this branch
decoy 2,096,152 B, then a policy (fits) 200 200
decoy 2,197,152 B, then a policy 422 "Field required" 400, the limit message
decoy 6 MB, then a policy 422 "Field required" 400, the limit message
decoy 1.5 MB, then a 1 MB config 422 "Field required" 400, the limit message
one part, headers over 64 KB 400 "There was an error parsing the body" unchanged
one 200 MB file 400 "'big.txt' is larger than the 2 MB limit…" unchanged

Unchanged, compared case by case on all three endpoints. Status, message and cookie came out identical on main and on this branch:

  • a legal file of exactly 2 MB
  • 2 MB + 1 byte
  • one 200 MB file
  • a 200 MB file with the wrong extension
  • a small good file
  • a 3 MB body to /api/ask (not limited, read in full)
  • a GET to /api/upload

Tests. Two commits, red first. The new test is parametrised over the three endpoints and two decoy sizes: 6 cases, each failing on main and passing here.

Mutant of the fix test_upload_body_limit.py
replace any status >= 400, not only 422 9 failed (the existing oversized and wrong-extension tests: their own messages would be lost)
drop the app's headers 6 failed (the cookie assertion)
replace only when NOT cut 6 failed
keep the endpoint's body 6 failed

Counts.

  • test_upload_body_limit.py: 45 passed.
  • Full suite, with Batfish not running: 2459 passed, 63 skipped.
  • ruff check --select E4,E7,E9,F is clean on both files.

Not changed.

  • A part whose headers alone exceed what Starlette's parser accepts still gets Starlette's own 400, "There was an error parsing the body". That refusal comes before any cut, so it is not this middleware's to rewrite.
  • A cut that still leaves a whole file part is answered by the endpoint as before.
  • Nothing about what is read or staged changes: the body is cut at the same byte.

Not checked. The frontend's display of the new message. app.js never sends a second part, so it cannot produce this case.

This is a follow-up to Shubham's note on #470 with no rush and is not needed for the week 7 deadline.

…it (red)

With a decoy part first, the 2 MB cut falls inside the decoy and the
`file` part never arrives. FastAPI then answers 422 "Field required",
which tells the user their file is missing when the request was too
big. Shubham noted this in his review of #470. Fails on main: 6 failed.
…d required"

After UploadBodyLimit cuts a body, a 422 means the endpoint never ran:
FastAPI found no whole `file` part in the shortened body. That answer
is replaced with the same 400 the endpoints give for an oversized file,
"The upload is larger than the 2 MB limit, so it was not read. Upload
the file on its own." The endpoints' own refusals are left alone, and
the response keeps its headers, so the session cookie still arrives.

@ARSH871-bot ARSH871-bot left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed fd8a3da, merged onto main cceffec. Approving.

Mutation. Your four, plus one more, each run against your test file and against main's copy:

mutant                             this PR       main's copy
replace any status >= 400           9 failed       9 failed   (the existing oversized/extension tests)
replace only when NOT cut           6 failed       all pass
drop the app's headers (cookie)     6 failed       all pass
keep the endpoint's body            6 failed       all pass
no replacement at all               6 failed       all pass

Nothing else moved. The same 21 boundary requests I used on #470 (each endpoint at 2 MB − 1, 2 MB, 2 MB + 1, + 1 KB, + 63 KB, + 70 KB, + 5 MB) give identical status and message on main and on this branch, so only the after-a-cut 422 changes.

CI-style suite on the merge: 2459 passed, 63 skipped (the 6 new cases over main). Ruff clean. I didn't re-run your real-uvicorn measurements; the table above is through main.app.

@shubhamkataria2005 shubhamkataria2005 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at fd8a3da.

I sent uploads to /api/policy with a decoy part placed before file, so the cut lands where this PR is meant to handle it:

decoy size         result
2,096,152 B        200, policy staged (the whole body fits, so nothing is cut)
2,197,152 B        400 "The upload is larger than the 2 MB limit, so it was not read. Upload the file on its own."
6 MB               400, same message

Cases that must not change:

  • A small body with no file part is never cut, so it still gets FastAPI's 422 "Field required".
  • A single file part over the limit still gets the endpoint's own 400, which names the file.

The swapped response keeps its other headers. I checked that the netwise_session Set-Cookie from attach_session is still there and that content-length (103) matches the new body (103 bytes).

On the merged result: 2509 passed, 13 skipped, and ruff is clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

An upload cut before its file part says "Field required" instead of the 2 MB limit

3 participants