Repository navigation
Name the uploaded file, not the server temp path, in upload refusals - #475
Conversation
…(red) /api/policy and /api/business-context validate the upload through a temp file, and the loaders' refusals for an unreadable file start with that path. Measured on main, four kinds of bad file on each endpoint (invalid JSON, a 5,000-digit integer, nesting 100,000 deep, bytes that are not UTF-8) put the full temp path, Windows user folder included, into the 400 detail. These tests assert the detail names the uploaded file, keeps the loader's reason, and contains no temp folder, no AppData or Users folder and no drive-letter path. A second test covers a path quoted with doubled backslashes, as an OSError prints it. Two controls show a refusal with no path and a good upload are unchanged. Red on purpose: 10 fail on main, the 3 controls pass. The fix is the next commit.
/api/policy and /api/business-context now pass a loader's refusal through _refusal_without_temp_path(), which replaces every spelling of this upload's temp path (as written, as repr() quotes it with doubled backslashes, and with forward slashes) by the quoted name of the file the user uploaded. Every other word of the message is kept. Only the exact temp path is replaced. A pattern matching any path was measured and rejected: the loaders echo the user's own values, and a drive-path regex rewrote "unknown tier 'D:\shares\gold'". The loaders in analysis/ are unchanged, so the command line still names the file it was given. The 10 tests from the previous commit now pass, with the 3 controls.
Mutation testing the previous commit: replacing only the first occurrence of each spelling passed every test, because each test named the path once per spelling. This test has a refusal naming the same path twice and asserts the exact message, so that mutant now fails.
shubhamkataria2005
left a comment
There was a problem hiding this comment.
Approving d4c82d3. Same leak as my #462, same cure: replace only this exact path, and you've thought through all its spellings.
Checked:
- Merged onto current
main(ee769b4): 2340 passed, 13 skipped, ruff clean on CI's selection. - Both endpoints are pinned: unwrapping
/api/policyfails 6 and unwrapping/api/business-contextfails 5. Replacing only the plain spelling fails 2. - Echoing the filename is safe:
display_nameis reduced to its basename, andshowPolicyMessage/showBusinessContextMessagesettextContent, neverinnerHTML, so a hostile filename is shown as text. - Rejecting a drive-path regex was right. Your
D:\\shares\\goldtier example is exactly why.
One gap, not blocking: the same one Arsh found on my #463. Dropping the doubled, the repr() or the as_posix() spelling, one at a time, survives all 14 tests. On Linux (CI, and these tests) a temp path has no backslashes, so those spellings are identical to the plain one, and "a refusal quoting the path with doubled backslashes" doesn't actually contain a doubled backslash there. Arsh's trick on #463 fixes that: put a backslash in a folder name you control (e.g. a tmp_path / "up\\load" temp directory), so the doubled form differs on every OS. Then the Windows case you designed for is tested where CI runs.
Closes #474.
web/is mine, so this stays inweb/main.pyand leaves the loaders inanalysis/alone.What changed
web/main.py:2043adds_refusal_without_temp_path(message, temp_path, display_name). It replaces every spelling of this upload's temp path in a loader's refusal with the quoted name of the file the user uploaded: as written, asrepr()quotes it (an OSError doubles every backslash), and with forward slashes. Every other word of the message is kept.web/main.py:2156(/api/policy) and:2405(/api/business-context) use it in place ofdetail=str(error).Before and after
Measured with the real app and
TEMPandTMPpointed at an empty folder. The temp path is written as<temp path>.<temp path> is not valid JSON: line 1, column 25: Expecting value'mine.json' is not valid JSON: line 1, column 25: Expecting value<temp path> holds a value that cannot be read: ...'mine.json' holds a value that cannot be read: ...<temp path> is nested too deeply to be a policy'mine.json' is nested too deeply to be a policy<temp path> is not UTF-8 text: byte 23 cannot be decoded. ...'mine.json' is not UTF-8 text: byte 23 cannot be decoded. ...<temp path> ...'ctx.json' ..., same wordsWhy the exact path and not a pattern
Three options were measured:
D:\shares\goldgivesunknown tier 'D:\\shares\\gold', and a drive-path regex rewrote that, swallowing the closing'.too.os.chdir, which is process-wide and unsafe with concurrent requests.web/would duplicate the loaders' messages, which the comment above the policy upload explicitly avoids.Tests, red then green on purpose
tests/test_upload_refusal_hides_temp_path.py, 14 tests:AppData, noUsers, no drive-letter path, and the detail keeps the uploaded name and the loader's reasonThe temp folder is a canary whose name contains a backslash, so the doubled spelling differs from the plain one on Linux as well as on Windows.
The commits are in order on purpose:
main(the temp folder is in the refusal), and the 3 controls pass.Mutation check
Each mutant ran in a scratch copy, one at a time:
/api/policyonlyCounts
What this does not change
analysis/: the command line still names the real file it was given./api/upload: it already names the uploaded file and never showed a path./api/askand/api/policy/draft, which passdescribe_error()text that can name the session's config folder. That is not reached by a bad upload and is noted on the issue for a separate look.