Skip to content

fix(extensions,persistence): keep staged-write cleanup best-effort - #7485

Merged
WillemJiang merged 1 commit into
bytedance:mainfrom
llb555:fix/census-atomic-cleanup
Oct 25, 2026
Merged

WillemJiang merged 1 commit into
bytedance:mainfrom
llb555:fix/census-atomic-cleanup

Conversation

@llb555

@llb555 llb555 commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

This is the census follow-up from the #7279 review. Four writers in this tree still publish a staged file and then discard it with an unguarded unlink on an exit path that may already be reporting a failure, so a failing discard replaces the error the caller needed:

  • extensions/manager.py::_write_bytes_atomically — the except BaseException: Path(temporary).unlink(missing_ok=True); raise handler (the exact masking fix(config): keep atomic-write cleanup from replacing the caller's outcome #7279 fixes elsewhere), plus the with os.fdopen(fd, ...) that leaks fd when fdopen itself fails;
  • extensions/manager.py — the finally: if temporary is not None: temporary.unlink(...) in the block writers;
  • persistence/agents/file.py::_write — the finally that walks the still-staged paths;
  • persistence/managed_subagents/file.py::_atomic_write — the same finally shape.

All four now go through deerflow.utils.atomic_write.cleanup_failed_atomic_write, which is already the house helper here (it closes an untransferred descriptor, then unlinks best-effort, and logs instead of raising). No behaviour changes beyond the exit path that used to raise.

tests/test_atomic_cleanup_census.py pins the two shapes with a stand-in unlink that fails the way a held handle does:

  • test_extension_restore_keeps_the_replace_error_while_cleanup_fails — the restore path must surface replace failed, and the previous file content must survive;
  • test_managed_subagent_write_keeps_the_replace_error_when_cleanup_fails — same for the subagent store, with no target file left behind.

Both fail on the base revision with PermissionError: temporary unlink failed replacing the original error, and pass here.

Scope note: this does not touch the skills-storage writers (local_skill_storage.py:118, user_scoped_skill_storage.py:141/:423, projection.py:243/:323/:857), which were named in the same census. They are a separate call shape — several are deliberate deletions rather than staged-write cleanup — so I would rather review them on their own than widen this diff.

@github-actions github-actions Bot added area:backend Gateway / runtime / core backend under backend/ risk:medium Medium risk: regular code changes size/S PR changes 20-100 lines labels Oct 25, 2026
@llb555

llb555 commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Lint failure fixed at 1261c6b — it was mine: the new cleanup_failed_atomic_write import landed between the deerflow imports instead of sorted into that block, and ruff check flagged I001 at manager.py:3.

The import now sits ahead of retry_until_locked, ruff check and ruff format --check are clean on all four files, and I re-ran pytest tests/test_atomic_cleanup_census.py tests/test_configured_extensions.py → 19 passed.

Worth noting for future rounds on my side: I verified the tests but ran lint on the wrong path scope, so this should have been caught before pushing.

@llb555
llb555 force-pushed the fix/census-atomic-cleanup branch from 1261c6b to 533bccb Compare October 25, 2026 08:55
@llb555

llb555 commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Reworked at 533bccb, and the reason matters: while this PR was open, extensions/manager.py was fixed upstream by someone else — _write_bytes_atomically now calls cleanup_failed_atomic_write(temporary, fd) and the plugins-block finally does the same, and tests/test_extension_atomic_write_cleanup.py pins both. My version of that file was built on an older tree, so it reverted their structure and broke their tests (the Unit Tests shards 2 and 4 failures on 8b2eec8 came from exactly that, not from a flake).

So I dropped the extensions/manager.py change entirely rather than reworking it against the new file — there is nothing left to fix there — and this PR now covers only the two sites that are still unguarded on main:

  • persistence/agents/file.py::FileAgentStore._write — the finally that walks the still-staged paths;
  • persistence/managed_subagents/file.py::FileManagedSubagentStore._atomic_write — the same finally shape.

Both now go through deerflow.utils.atomic_write.cleanup_failed_atomic_write, matching the house helper already used by the extension writers.

tests/test_atomic_cleanup_census.py pins both with a stand-in unlink that fails like a held handle. They fail on the base revision with PermissionError: temporary unlink failed replacing the original error, and pass here.

Verification: pytest tests/test_atomic_cleanup_census.py tests/test_agent_storage_backend.py → 30 passed; ruff check and ruff format --check clean on all three files.

A note on the census itself: the skills-storage writers named in the original list are also covered upstream now (tests/test_skill_storage_atomic_write_cleanup.py), so the long tail is down to these two persistence sites.

@WillemJiang WillemJiang added this to the 2.2.0 milestone Oct 25, 2026
@WillemJiang
WillemJiang merged commit 6b918bb into bytedance:main Oct 25, 2026
17 checks passed

@willem-bd willem-bd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 533bccb. The fix is correct and verified end-to-end:

  • Both writers now discard staged temps through the shared deerflow.utils.atomic_write.cleanup_failed_atomic_write, matching the #7200 family convention — helper adopted, no duplicated cleanup logic.
  • Double-fault battery: forced replace failure with a simultaneously failing unlink. On this head the original OSError("replace failed") propagates and cleanup failure is log-only; with the fix reverted to the base shape, both new tests fail with PermissionError: temporary unlink failed masking the primary error — so the red-check is genuine and both tests exercise the double fault, not just single-failure cleanup.
  • Portability: in both call sites the temp handle is closed before cleanup can run (NamedTemporaryFile context exits before replace), so the Windows unlink-of-open-file failure mode doesn't apply; the helper itself stays portable.
  • Success paths unaffected: staged is empty / tmp_path is None on success, so the helper never runs on the happy path.
  • 484 tests across the two stores' suites pass; ruff check and ruff format --check clean.

Two non-blocking notes inline: the PR description is stale relative to this push, and one pre-existing staging-block leak the fixed finally still cannot see.

@@ -0,0 +1,71 @@
"""Staged-write cleanup at the two persistence writers must not replace the outcome.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description doesn't match this push. It claims four writers — including extensions/manager.py::_write_bytes_atomically and the block writers — but manager.py at this base (a1b89d0) already routes through cleanup_failed_atomic_write (landed via #7279), so those hunks are gone. It also names test_extension_restore_keeps_the_replace_error_while_cleanup_fails, which doesn't exist here; the actual tests are test_agent_store_write_keeps_the_publish_error_when_cleanup_fails and test_managed_subagent_write_keeps_the_replace_error_when_cleanup_fails. Worth updating the description so it describes the two-writer delta this PR actually ships.

@@ -97,4 +98,4 @@ def _atomic_write(path: Path, definition: ManagedSubagentDefinition) -> None:
tmp_path = None
finally:
if tmp_path is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking, pre-existing and unchanged by this PR: tmp_path = Path(handle.name) is only assigned after flush/fsync inside the NamedTemporaryFile block (with delete=False), so a failure in write/fsync leaves it None, this finally skips cleanup entirely, and the staged temp leaks with no cleanup attempt and no log — the masking fix on the next line can't see that path. The same shape exists in _stage_temp in persistence/agents/file.py. Fine to defer, but it is the one gap left in this writer's staged-write lifecycle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:backend Gateway / runtime / core backend under backend/ risk:medium Medium risk: regular code changes size/S PR changes 20-100 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants