Repository navigation
fix(projects): keep attach-to-thread cleanup from replacing the caller's outcome - #7229
Conversation
willem-bd
left a comment
There was a problem hiding this comment.
Verified at head 325cd58: the fix correctly makes the staged-copy unlink best-effort (except OSError + warning, matching uploads.py::_remove_staged_file) and moves aclose() into a nested finally so it can no longer be skipped — and aclose itself already swallows lease-release failures internally, so the cleanup path can no longer replace the caller's outcome. I reproduced the verification claim independently: the two new tests fail with PermissionError on main and pass at head, and the cited sibling suites are green (110 passed). On #7200 (still open): its cleanup_failed_atomic_write helper is sync/atomic-write-specific, so not reusing it here is right, and mirroring _remove_staged_file is the better fit — though the error-preservation family keeps growing (#6871, #7106, #7141, #7157, #7164, #7200, this), so a shared best-effort-unlink helper plus a sweep of the remaining sites would make a good consolidation follow-up. Two same-class gaps remain, in the comments below.
| # this endpoint owns. Leave the leftover to the staging sweep instead. | ||
| try: | ||
| await run_file_io(staged_path.unlink, True) | ||
| except OSError: |
There was a problem hiding this comment.
Same class, two more sites in this same file that this PR leaves unguarded: upload_document_to_shelf and promote_thread_file_to_shelf both call await run_file_io(staged.staging_path.unlink, True) on the empty-file rejection path (lines 220 and 336 at head) immediately before raising the 400. An OSError there — Windows handle still held by a worker or an AV scanner, the exact scenario in the PR description — replaces the intended 400 with a raw 500 and leaves the staged file behind. Since the best-effort pattern is now written, covering both (or filing them as the next family follow-up in the same change set) would close out the file.
| # masked — and would skip ``aclose()`` below, which is the only teardown | ||
| # this endpoint owns. Leave the leftover to the staging sweep instead. | ||
| try: | ||
| await run_file_io(staged_path.unlink, True) |
There was a problem hiding this comment.
Completeness check inside this same endpoint: the await service.cleanup_written() calls in the except blocks above (lines 412, 415, 418, 422) sit before each raise HTTPException(...), and cleanup_written can itself raise. _cleanup_uploaded_paths is per-path best-effort, but the companion leg goes through unregister_companion → record.unlink(missing_ok=True) (deerflow/uploads/companions.py), which only tolerates FileNotFoundError — a PermissionError there would replace the mapped 4xx/500 detail with a raw 500 and leave the remaining companions registered. Same masking class this PR fixes; consider making that leg best-effort too (here or in a tracked follow-up).
| try: | ||
| await run_file_io(staged_path.unlink, True) | ||
| except OSError: | ||
| with suppress(Exception): |
There was a problem hiding this comment.
Nit: suppress(Exception) around a logger.warning call is redundant — logging already contains handler emit failures — and diverges from the mirrored uploads.py::_remove_staged_file, which logs bare. It would also silently hide a genuinely broken logging setup. logger.warning(..., exc_info=True) alone is sufficient here.
6c84ee5 to
7161ea8
Compare
7161ea8 to
ba8b934
Compare
|
Thanks for the close read — all four addressed at
Validation at |
|
One correction on point 3, with the code at the reviewed head await run_file_io(uploads._cleanup_uploaded_paths, self._written_paths)
for original, _ in self._companion_pairs:
try:
await run_file_io(unregister_companion, original)
except Exception:
with suppress(Exception):
logger.warning("Failed to remove upload companion record for %s", original, exc_info=True)So a There is an unguarded caller, though, and it is the one that can mask a success: I'll take that as the follow-up instead of touching |
|
Since #7200 just landed — quick status on this one: all four points from your review are addressed at Workflows don't start for fork PRs here, so there are no green checks to show: the only runs registered on this head are Happy to adjust anything else. |
Why
attach_project_document_to_threadstages a copy of a shelf document and removes it in afinally. When that removal fails — a worker still holds the handle on Windows, a scanner intercepts it, the filesystem is read-only — two things go wrong at once:PermissionErrorreplaces whatever the endpoint was about to return: a successful 200 becomes a 500, and an in-flight 4xx is masked;service.aclose()is skipped, because it sits after the failing call in the samefinally, so the only teardown this endpoint owns never runs.The leftover staged file is the harmless part — it is what the staging sweep exists for. The masked outcome and the unclosed ingestion service are not.
Same class as #7200, at a site that PR does not touch.
What changed
Attach-to-thread now treats staged-copy removal as best-effort: an
OSErrorthere is logged as a warning and the endpoint still returns its real outcome, whileaclose()moves into a nestedfinallyso it is never skipped. This mirrorsrouters/uploads.py::_remove_staged_file, which already handles the identical situation for ordinary uploads.No request or response shape changes.
Surface area
POST /api/projects/{project_id}/documents/{document_id}/attach-to-thread/{thread_id}Bug fix verification
backend/tests/test_project_document_attach_cleanup.pymainand green on this branch? yes — 2 failed → 2 passed. Both cases inject aPermissionErrorfrom the staged path'sunlink; the second also asserts the originalHTTPExceptioninstance still propagates (is, not equality) and thatcleanup_written()andaclose()both ran.Validation
AI assistance
Tool(s) used: DeepSeek Harness (agentic coding agent)
How you used it: the agent located the site with a grep pass over
finally-block cleanup patterns, wrote the two failing tests first, then the fix. I reviewed the diff and ran the checks above.