Repository navigation
fix(main): big background deletes no longer stall chat sends and other file I/O - #23746
brennanb2025 wants to merge 15 commits into
Conversation
…ad pool A background recursive delete of a trashed worktree queued every entry on libuv's 4-thread pool, so every other async fs call in the main process (the agent-session store, journals, locks) waited for the whole tree. Chat sends stalled for minutes after two worktree removals. removeHostTree now runs a native rmSync on a worker thread. The native walk never touches the pool and never sees Electron's JS asar shim, so the original-fs wrapper is no longer needed.
About 40 main-process sites still deleted folders with an async recursive rm (file-explorer deletes from remote clients, import/download rollback, failed-clone cleanup, plugin/skill/speech staging, browser storage), each able to fill the shared fs thread pool the same way the worktree trash delete did. They now all call removeHostTree. A boundary test scans src/main and src/shared and fails on any async recursive rm/rmdir, including options held in a constant or built by a call. The AppImage payload removal no longer toggles process.noAsar, since the native walk is asar-transparent.
…e-delete-off-fs-pool
Each removeHostTree call now spawns a worker isolate. The terminal-history tombstone drains start many removals at once (the main process admits up to 64, the daemon's drain is unbounded), which spawned that many isolates together: 64 simultaneous workers measured ~700 MB RSS under Node. Deletes are disk-bound, so running more than the old pool's four at once buys nothing. Reuse PrioritySemaphore (FIFO at one priority) to cap them.
The four-slot worker cap was shared by every delete, so fire-and-forget bulk work (worktree trash, the startup trash sweep, history-tombstone drains) could hold all slots and make a user-awaited folder delete wait behind it. Only the background lane is capped now; awaited deletes start their worker immediately.
Quitting joins every worker thread, and termination only takes effect between JS calls. One native rmSync of a whole worktree therefore kept the quitting process alive, after its windows closed, until the entire tree was deleted. The worker now walks the tree itself and removes one entry per rmSync, so termination lands within one entry. The walk sets noAsar on the worker's own process so Electron's shim never treats an archive as a directory. Also updates the local-worktree-filesystem expectations for the removal lane argument added in the previous commit.
…orker removeHostTree now walks the tree in the main process: lstat each entry, descend only into real directories, and remove each entry with Node's JS `rm`. Every fs call any tree delete makes holds one of two process-wide slots (awaited deletes are served ahead of background ones), so a large delete can no longer take over libuv's thread pool. Why not the worker's native rmSync: on Windows it follows directory junctions out of the tree, can report success while leaving most of a pnpm checkout behind, and fails on read-only files such as git objects. Node's JS rm unlinks junctions and clears the read-only flag. The walk reads through Electron's unpatched fs (asar-transparent-fs, restored from main and widened to lstat/readdir/rm) so a *.asar inside a tree is removed as the file it is.
…ck to rm `rm` costs an lstat plus the removal for every entry. Try `unlink` or `rmdir` first and fall back to `rm` (Windows read-only fix, lock retries) only when that fails. Deleting a cloned pnpm store took 3.9-7.5 s instead of 5.6-26.5 s, with the same store-transaction latency while it ran.
…ole walk After the first failure, lanes in every directory stop taking entries, so the error surfaces once in-flight calls settle rather than after unrelated subtrees finish.
The pool-freedom test asserted only that a 20-stat chain finished before the delete. Queued behind a pool-backed recursive rm the chain ends within a millisecond of the delete, so that ordering passed 2 of 3 times with the old engine. It now requires the chain to take under half the delete's time (old engine: ~1.0; walk: ~0.02-0.15). The asar test's fixture is now removed by a plain unlink, which the shim never sees. A second case refuses every unlink so the rm fallback (what a read-only or locked file on Windows takes) must see the archive as a file; it fails with the shimmed fs and passes with original-fs.
The macOS permissions spec and the desktop-script provider harness stub `fs/promises` with only the calls their subject used to make. The tree walk now reads through that module outside Electron, so they failed on the missing `lstat`. Mock the delete helper instead, as the sibling permission-status spec already does. The provider exec-timeout spec, which failed on CI at f6f7693, passes locally with this change; the CI failure itself was not reproduced on that head.
…on checks Electron's original-fs always carries the full promises API; the only case to handle is the require failing outside Electron.
The static-analysis lane rejects a broad `object` parameter type.
The walk exists because Windows junctions and read-only files broke the previous engine, yet no CI job ran it on Windows. Its junction cases use real junctions there.
The walk stopped the whole delete at the first failure, so a single stuck entry (e.g. a root-owned folder in a worktree) left nearly the entire tree behind, and each startup trash sweep stopped at the same entry again. Siblings now keep going, as with Node's rm; only the failed entry's ancestors are kept, and the walk still rejects only after every call settles.
Review status: ready for review (head 0af34fd)Verdict: the final review loop came back clean, CI is green (42 pass, 6 skipped, 0 failing), and live QA passed. Nothing a user would find surprising remains. Not merged. What changed during review: the PR's first version deleted each tree with Node's native Commits added in review:
Live QA, M4Air Mac, 138,993-entry workspace, two runs per side (main bd5dca4, PR fbce1b2): while the removed workspace's trash folder was being deleted, the worst file read was 8.1 s and 9.0 s on main and 7 ms and 17 ms on the PR. The workspace left the sidebar at once and the trash was empty at the end. Quitting right after starting a delete exited in about 155 ms (914 ms on main), and the next launch swept the leftover. A folder delete from the file explorer works. Windows (real machine, 654f717): junctions pointing outside the tree, a target that is itself a junction, junction loops, dangling junctions, read-only files and folders, a git repo and long paths were all removed with the files outside the tree byte-identical. A real pnpm checkout (86,428 files, 4,010 junctions) was removed in 26.7 s with nothing left (27.1 s before this PR). The walk tests pass on Windows. Not verified: chat-send timing in the live app (it needs a logged-in agent; the file reads above go through the same shared threads); true symbolic links on Windows (they need admin or Developer Mode); Bun 1.4.2 (1.3.14 checked). Accepted low risks:
Follow-ups, not caused by this PR:
|
|
| isDirectory = (await withFsCallSlot(lane, () => fs.lstat(path))).isDirectory() | ||
| if (isDirectory) { | ||
| children = await withFsCallSlot(lane, () => fs.readdir(path, { withFileTypes: true })) |
There was a problem hiding this comment.
Deletion can escape the tree If a concurrent process replaces a directory in a worktree or file-explorer delete with a symlink or junction after
lstat but before readdir, the listing follows the replacement. The walk then removes the listed children through that link, potentially deleting files outside the requested tree. How this was verified: The walk classifies a directory, reopens it by path, and deletes children through paths joined beneath it.
| const removeEntry = (path: string, isDirectory: boolean): Promise<void> => | ||
| withFsCallSlot(lane, () => | ||
| (isDirectory ? fs.rmdir(path) : fs.unlink(path)).catch(() => fs.rm(path, options)) | ||
| ) |
There was a problem hiding this comment.
Fallback bypasses deletion limit If another process adds entries after a directory is listed,
rmdir can find it non-empty. The fallback then runs recursive fs.rm while holding just one semaphore slot. A large subtree can therefore queue file operations outside the per-entry limit, bringing back delays for chat persistence. Keeping this fallback bounded would preserve the protection provided by the new walk.
There was a problem hiding this comment.
Important
The new walk lists a directory's children and deletes them before it attempts rmdir — the reverse of Node's fs.rm, which does rmdir first and recurses only on ENOTEMPTY/EEXIST/EPERM. On a busy mount point that empties the mounted filesystem before rmdir fails EBUSY; fs.rm would have refused without touching the contents. This PR introduces that divergence, and user-driven deletes reach it.
Reviewed changes
- New bounded walk —
src/main/tree-removal-walk.tswalks a tree itself, gating everylstat/readdir/unlink/rmdir/rmcall behind a 2-slot process-widePrioritySemaphore(interactivebeforebackground), with a per-entryunlink/rmdirand anfs.rmfallback. removeHostTreerouted through it — gains an{ lane }option; every async recursive delete insrc/mainandsrc/sharednow calls it. Worktree trash and terminal-history cleanup are taggedbackground; everything else defaults tointeractive.- Unpatched fs —
asar-transparent-fs.tsnow exposes theoriginal-fscall set, so a tree containing a.asaris deleted as files. The AppImage payload removal drops its process-wideprocess.noAsartoggle. - Guards — a no-allowlist boundary test scans
src/main+src/sharedfor async recursiverm/rmdir; new walk/pool/asar tests, with the walk tests added to the Windows CI lane. - Test mocks migrated — specs that stubbed
fs/promises.rmnow stubremoveHostTreeorremoveTreeWithBoundedFsCalls.
I read the full diff (75 files) and ran the new and modified suites; all pass. src/main/terminal-history.test.ts fails locally with No "mkdtempSync" export on the "fs" mock, but that reproduces identically on origin/main, so it is pre-existing and unrelated.
⚠️ A mount point inside the deleted tree is emptied instead of refused
Node's fs.rm tries rmdir first and only enumerates on ENOTEMPTY/EEXIST/EPERM, so an EBUSY from a busy mount point aborts without touching the mounted filesystem. The walk does the opposite: any path lstat calls a directory is readdir'd and its children deleted, and only then is rmdir attempted. On POSIX a bind/overlay/network mount is an ordinary directory to lstat, and on Windows a volume mount point can be reported as one, so the walk descends and deletes the mounted volume's contents before failing. removeHostTree is reached from user-driven deletes (the file-explorer permanent delete and worktree trash), so this is a data-loss path that did not exist before this PR.
Technical details
# Mount point emptied before rmdir
## Affected sites
- `src/main/tree-removal-walk.ts:59-62` — every `lstat`-directory is `readdir`'d before any removal attempt
- `src/main/tree-removal-walk.ts:90` — `rmdir`/`unlink` runs only after the children are gone
- `src/main/host-tree-removal.ts:60` — previously delegated to Node's `fs.rm` (rmdir-first), now to the walk
- Reached from user deletes: `src/main/runtime/runtime-file-commands-create-file-explorer-dir-no-clobber.ts:164`, `src/main/worktree-trash.ts:87`
## Required outcome
A delete must not remove entries inside a mount point that Node's `fs.rm` refused to enter. Either
probe the directory removal before enumerating it, or detect a mount point (e.g. `stat(dir).dev`
vs `stat(dirname(dir)).dev`) and stop instead of descending.
## Suggested approach
Mirror Node's `_rmdir`: call `rmdir(path)` first; only on `ENOTEMPTY`/`EEXIST`/`EPERM` `readdir`
and recurse; let `EBUSY`/`EXDEV` propagate without touching contents. Costs one extra `rmdir` per
directory and restores the old guarantee. The existing per-entry `fs.rm` fallback already refuses
the mount point, so the regression is purely the pre-emptive `readdir`.ℹ️ The relay's remote folder delete still uses an unbounded recursive rm
deleteRelayPath in src/relay/fs-path-mutation-requests.ts:32 still calls rm(targetPath, { recursive, force: true }). The PR body lists "folder deletes from the remote and mobile file explorer" in scope while also stating "the relay is untouched", and the new boundary guard scans only src/main and src/shared, so this path can neither be fixed nor caught. Because the relay is a separate process it does not share the main pool, so it is not the reported chat-send stall — but the stated scope is wider than the change, and a large remote delete still occupies the relay's own async fs work.
Technical details
# Relay recursive delete outside the bounded walk
## Affected sites
- `src/relay/fs-path-mutation-requests.ts:32` — `rm(targetPath, { recursive: !!recursive, force: true })`, the remote file-explorer folder delete
- `src/main/host-tree-removal-boundary.test.ts:14` — `SCANNED_ROOTS = ['src/main', 'src/shared']` excludes `src/relay`
## Required outcome
Either route the relay's tree deletes through the same bounded walk, or narrow the PR's stated scope
so remote file-explorer deletes are not claimed as fixed.
## Open questions for the human
Is the relay intentionally deferred to a follow-up (separate process, separate pool), or should the
boundary guard at least cover `src/relay`?DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
| isDirectory = (await withFsCallSlot(lane, () => fs.lstat(path))).isDirectory() | ||
| if (isDirectory) { | ||
| children = await withFsCallSlot(lane, () => fs.readdir(path, { withFileTypes: true })) | ||
| } |
There was a problem hiding this comment.
This enumerates and deletes a directory's children before attempting rmdir (:90), whereas Node's fs.rm calls rmdir first and recurses only on ENOTEMPTY/EEXIST/EPERM. On a busy mount point (lstat reports a POSIX bind mount, and can report a Windows volume mount point, as an ordinary directory) the walk therefore deletes the mounted filesystem's contents and only then fails with EBUSY — content fs.rm would have left untouched.
Technical details
# Mount point emptied before rmdir
## Affected sites
- `src/main/tree-removal-walk.ts:59-62` — every `lstat`-directory is `readdir`'d before any removal attempt
- `src/main/tree-removal-walk.ts:90` — `rmdir`/`unlink` runs only after the children are gone
- `src/main/host-tree-removal.ts:60` — was Node's `fs.rm` (rmdir-first), now the walk
## Required outcome
A delete must not remove entries inside a mount point that Node's `fs.rm` refused to enter.
## Suggested approach
Call `rmdir(path)` first and `readdir` only on `ENOTEMPTY`/`EEXIST`/`EPERM`, letting `EBUSY`/`EXDEV`
propagate without touching contents (Node's `_rmdir`). The existing per-entry `fs.rm` fallback
already refuses the mount point, so only the pre-emptive `readdir` needs to change.|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. 📝 WalkthroughWalkthroughThe change adds a bounded tree-removal walk and a shared host-tree removal helper. Recursive directory cleanup across application workflows now uses the helper instead of direct recursive Priority: ⬆️ High Merge Risk: 🔵 Low · up to A directory changed during deletion can briefly defeat the responsiveness improvement. This is a narrow case, but the fallback should be bounded before merge if strict responsiveness is required. Security Architecture ReviewSecurity architecture risk: 🟡 Moderate · up to Most deletions should leave capacity for other file operations, but a folder that changes during deletion can still trigger the long-running removal behavior this change aims to prevent. The shared change affects many cleanup paths; no expanded deletion permission was established. Retained concerns
Security review detailsSecurity Blast Radius
Security Findings and Attack Paths
Trust Boundaries and Controls
Resilience and Maintainability Implications
Hardening Proposals
🚥 Pre-merge checks | ✅ 3 | ❌ 1 | ❓ 1❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
Full details: Description checkExplanation The description is thorough and covers the required change summary, rationale, testing, visual proof, review, cross-platform impact, and checklist. However, the Linked Issue section states "No ticket" instead of providing the required issue link. Full details: Docstring CoverageExplanation Docstring coverage is 29.55% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 44 functions across 50 files. (25 skipped: 1 unsupported, 24 over the file limit.)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: 007235db-4e5a-4ee4-af81-cb01a2089e24
📒 Files selected for processing (75)
.github/workflows/pr.ymlconfig/scripts/pr-code-change-scope.mjssrc/main/ai-vault-search/session-search-index-test-fixture.tssrc/main/asar-transparent-fs.test.tssrc/main/asar-transparent-fs.tssrc/main/browser/browser-client-download-relay.tssrc/main/browser/browser-client-upload-staging.test.tssrc/main/browser/browser-client-upload-staging.tssrc/main/browser/browser-route-partition-storage-dependencies.tssrc/main/cli/appimage-payload-removal.reentrancy.test.tssrc/main/cli/appimage-payload-removal.test.tssrc/main/cli/appimage-payload-removal.tssrc/main/computer/desktop-script-provider-client.tssrc/main/computer/desktop-script-provider-test-harness.tssrc/main/computer/macos-computer-use-permission-status.test.tssrc/main/computer/macos-computer-use-permission-status.tssrc/main/computer/macos-computer-use-permissions.test.tssrc/main/daemon/terminal-history-session-tombstone-retry.test.tssrc/main/daemon/terminal-history-session-tombstone.tssrc/main/git/repo-clone-path.tssrc/main/github/client/create/create-github-pull-request.tssrc/main/host-tree-removal-asar.electron.test.tssrc/main/host-tree-removal-boundary.test.tssrc/main/host-tree-removal.test.tssrc/main/host-tree-removal.tssrc/main/ipc/filesystem-download-folder.tssrc/main/ipc/filesystem-download-transfers.test.tssrc/main/ipc/filesystem-import-local-rollback.test.tssrc/main/ipc/filesystem-import-local-tree-copy.tssrc/main/ipc/filesystem-import.test.tssrc/main/ipc/filesystem-test-harness.tssrc/main/ipc/pet-bundle-import.tssrc/main/ipc/pet.tssrc/main/ipc/repos-create.test.tssrc/main/ipc/repos/repo-creation-handlers.tssrc/main/ipc/shallow-watch-delivery-probe-lifetime.test.tssrc/main/ipc/shallow-watch-delivery-probe.tssrc/main/ipc/terminal-render-desync-evidence.tssrc/main/local-downloaded-folder-promotion.tssrc/main/local-worktree-filesystem.test.tssrc/main/orcad/electron-serve-browser-process.tssrc/main/orcad/orcad-bun-native-preflight.test.tssrc/main/orcad/orcad-bun-native-preflight.tssrc/main/persistence/profile-state/profile-state-runtime-preflight.tssrc/main/plugins/plugin-install-publication.tssrc/main/plugins/plugin-install-staging.tssrc/main/plugins/plugin-install.tssrc/main/plugins/plugin-marketplace-fetch.tssrc/main/plugins/plugin-marketplace-installer.tssrc/main/runtime/runtime-file-commands-create-file-explorer-dir-no-clobber.tssrc/main/runtime/runtime-repository-registration-controller.tssrc/main/skills/skill-bundle-creation.tssrc/main/skills/skill-bundle-extraction.tssrc/main/skills/skill-install-filesystem.tssrc/main/skills/skill-package-creation.tssrc/main/skills/skill-package-download.tssrc/main/skills/skill-package-extraction.tssrc/main/skills/skill-share-preparation-service.tssrc/main/skills/skill-upload-staging-ownership.tssrc/main/speech/model-manager.tssrc/main/ssh/orcad-artifact-materializer.tssrc/main/ssh/orcad-bun-runtime-materializer.tssrc/main/ssh/ssh-relay-opencode-runtime.tssrc/main/ssh/system-ssh-file-binary-transfer.tssrc/main/terminal-history-deletion.tssrc/main/terminal-history-gc.test.tssrc/main/terminal-history-tombstone-retry.test.tssrc/main/terminal-history.test.tssrc/main/tree-removal-walk.test.tssrc/main/tree-removal-walk.tssrc/main/window/clipboard-ipc-handlers.test.tssrc/main/window/clipboard-remote-file-staging.test.tssrc/main/window/clipboard-remote-file-staging.tssrc/main/worktree-trash.tssrc/shared/windows-transient-lock-removal.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.
| const removeEntry = (path: string, isDirectory: boolean): Promise<void> => | ||
| withFsCallSlot(lane, () => | ||
| (isDirectory ? fs.rmdir(path) : fs.unlink(path)).catch(() => fs.rm(path, options)) | ||
| ) |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
sed -n '1,100p' src/main/tree-removal-walk.ts
sed -n '1,84p' src/main/host-tree-removal.tsRepository: stablyai/orca
Length of output: 7495
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- tree-removal-walk references ---'
rg -n -C 3 'removeEntry|fs\\.rmdir|fs\\.rm|ENOTEMPTY|EEXIST|EPERM|transientLockRemovalOptions' src/main/tree-removal-walk.ts src/main/asar-transparent-fs.ts src/shared/windows-transient-lock-removal.ts src/main/host-tree-removal.ts
printf '%s\n' '--- asar wrapper ---'
cat -n src/main/asar-transparent-fs.ts
printf '%s\n' '--- removal options ---'
cat -n src/shared/windows-transient-lock-removal.ts
printf '%s\n' '--- callers and tests ---'
rg -n -C 3 'removeHostTree|removeTreeWithBoundedFsCalls' src test tests 2>/dev/null || trueRepository: stablyai/orca
Length of output: 42333
Keep recursive fallback for recovery errors, but reject directory-not-empty errors on Windows.
This is not a deadlock. removeTree releases its slot before recursing. The actual issue is that rmdir can fail with ENOTEMPTY, causing recursive fs.rm to queue the subtree while a slot is held.
On Windows, let ENOTEMPTY and EEXIST reach removeHostTree so its finite retry loop handles transient additions. Keep recursive fs.rm for EPERM and other errors because it provides read-only recovery. Do not reject these errors on non-Windows, where removeHostTree has no retry loop.
Suggested fix
+function isDirectoryNotEmptyError(error: unknown): boolean {
+ const code = typeof error === 'object' && error !== null && 'code' in error ? error.code : null
+ return code === 'ENOTEMPTY' || code === 'EEXIST'
+}
+
- const removeEntry = (path: string, isDirectory: boolean): Promise<void> =>
- withFsCallSlot(lane, () =>
- (isDirectory ? fs.rmdir(path) : fs.unlink(path)).catch(() => fs.rm(path, options))
- )
+ const removeEntry = async (path: string, isDirectory: boolean): Promise<void> => {
+ try {
+ await withFsCallSlot(lane, () => (isDirectory ? fs.rmdir(path) : fs.unlink(path)))
+ } catch (error) {
+ if (process.platform === 'win32' && isDirectory && isDirectoryNotEmptyError(error)) {
+ throw error
+ }
+ await withFsCallSlot(lane, () => fs.rm(path, options))
+ }
+ }Add EEXIST to the transient error codes in isTransientWindowsRemovalError.
|
Superseded by #23837. This PR kept Orca deleting removed worktrees itself and limited how much of Orca's file I/O the delete could use. A run on a real Windows machine showed Node's native recursive delete follows directory junctions out of the tree, and the approach matched no comparable app. #23837 instead lets git delete the checkout in its own process, as before #12416, with the card showing Deleting until git finishes. Closing this one. |



What this fixes
A structured chat stopped accepting messages for about four minutes right after two worktrees were removed: the message appeared in the chat, but the agent only received it once Orca finished deleting the removed worktrees in the background.
Scope: every Orca user on macOS, Linux and Windows whose app deletes a large folder in the background or on request: worktree removal (the trash delete), plugin, skill and speech-model staging cleanup, failed-clone and import rollback, folder deletes from the remote and mobile file explorer, and terminal-history cleanup. While such a delete ran, anything else in the main process that reads or writes a file (chat sends and chat creation through the agent-session store, journals, settings) waited behind it. Git commands and terminals were not affected, because they run in their own processes. For worktree removal this started with #12416, which moved the checkout delete out of git's own process (
git worktree remove) into Orca's main process so removal returns immediately; the other folder deletes always worked this way.ELI5
Node gives each app 4 shared helper threads for all file reads and writes. Deleting a folder the old way put one job per file on that shared line, and a worktree has about 100,000 files, so the line filled up for minutes and every other file read or write, including the one that delivers a chat message, waited at the back. Now a big delete may only use two of those helper threads at a time, so everything else keeps moving while it runs. A test stops new code from going around this.
What Changed
Before:
fs.promises.rm(dir, { recursive: true })walks the tree and queues one request per file and folder on Node's 4 shared file threads, first in, first out. Anything else that asked for file I/O afterwards (the agent-session store's lock, read, write and fsync, the journal files, settings) waited behind the whole tree. Worktree removal renames the checkout into a trash folder and deletes it in the background this way, and about 40 other places in the main process deleted folders the same way.Observed on a real profile: two worktree removals at 14:00:53–14:01:14 were followed by a 4-minute window in which no chat send on the machine was accepted. Sends clicked at 14:01:54, 14:02:18 and 14:03:17 reached the agent only at 14:06:13, 14:06:15 and 14:06:18, and another chat's action clicked at 14:01:33 was not admitted until 14:06:13. Git commands kept completing throughout, because child processes do not use the shared threads, which is the signature of thread starvation rather than a frozen process.
After:
removeHostTreeis the one way the main process and the terminal daemon delete a folder tree. It walks the tree itself (src/main/tree-removal-walk.ts), and every file call any tree delete makes has to hold one of two process-wide slots, so two of the four shared file threads always stay free for everything else.lstatand only goes into real folders. Symbolic links and Windows junctions are removed as links and never followed out of the tree.unlinkorrmdir. Only when that fails does it fall back to Node's ownrmfor that entry, which clears a read-only flag and retries a briefly locked file on Windows.asar-transparent-fs.ts), so a.asararchive inside a tree (every pnpm-installed checkout has one) is removed as the file it is.src/mainandsrc/sharednow callsremoveHostTree. The file-explorer delete keeps a plain single-filermfor files and usesremoveHostTreeonly for folders.src/main/host-tree-removal-boundary.test.tsscanssrc/mainandsrc/sharedand fails on any async recursiverm/rmdir, whether the options are inline, split across lines, held in a constant, or built by a call. There is no allowlist.process.noAsartoggle is removed; it usesremoveHostTreelike everything else.For the user: removing worktrees, clearing plugin or skill staging, or deleting a folder no longer makes chat sends, chat creation or other file-backed actions wait. Large deletes themselves can take somewhat longer, because they no longer take over every file thread.
Why
The failure class is "a bulk background job and latency-critical persistence share one first-in-first-out resource". Capping the bulk job's share of that resource removes it for every folder delete. A timeout or a priority around the send path would only have hidden it, and the next bulk delete would still have stalled everything else that touches a file. It does not remove every possible bulk user of those threads (for example, a large recursive copy would behave the same way); none is known to stall the app today.
Measured: during a 102,000-entry delete, a store-style transaction (read 2 MB, write, fsync, rename) took about 9 ms median and 14–19 ms at the 95th percentile; with the old
fs.promises.rma single transaction waited 3.9–5.1 s, the length of the whole delete. A two-file delete issued during the bulk delete finished in under 1 ms median.It works the same in Electron's main process, the terminal daemon (Electron running as Node), plain Node and Bun, keeps each error's
code, and names the entry that could not be removed. Quitting while a delete runs is not held up: there is no thread to wait for and at most two file calls are in flight. The Windows transient-lock retry loop inremoveHostTreeis unchanged.Alternatives rejected:
rmSyncon a worker thread (this PR's first version): on Windows, Node 24's native recursive delete follows directory junctions and deletes files outside the tree, can report success while leaving most of a pnpm checkout on disk (77,000 of 86,000 files remained on a real checkout), and fails on read-only files such as git objects. Quitting the app also had to wait for the whole tree, and each delete cost a separate JavaScript engine instance.fs.promises.rminto a worker thread: a worker's async file calls use the same process-wide threads; the store still waited about 3 s.git worktree remove --force): Orca renames the checkout into trash first so removal returns immediately instead of holding a spinner for 8–35 s, and the trash folder is no longer a git worktree. It also would not cover the other ~40 folder deletes.UV_THREADPOOL_SIZE: the threads are first in, first out, so a larger pool still runs every queued entry of the tree before later requests.Differences from the common pattern
git worktree remove --forcedelete the checkout inside git's own process and then removes any leftovers with an async recursive delete. Orca keeps its rename-to-trash step for fast removal and deletes the trash folder itself.Known weaknesses
removeHostTreeretries only on Windows, where held handles are the known cause.rmSync) block the event loop rather than the shared threads. That is a different failure and is not changed here.Linked Issue
No ticket. Found while diagnosing a chat that stopped responding for about four minutes.
Visual Proof
Live Electron app on the M4Air Mac, 138,993-entry workspace removed from the sidebar, file reads (
fs:readDir,fs:readFile) timed from the app window every ~250 ms. main = bd5dca4, PR = fbce1b2 (0af34fd only changes what happens when an entry cannot be deleted).Testing
Regression test (
src/main/host-tree-removal.test.ts): a chain of 20 asyncstatcalls issued while a 2,000-file tree is deleted must take less than half the delete's time. With the oldfs.promises.rmswapped back in it fails every run.Walk tests (
src/main/tree-removal-walk.test.ts), each confirmed to fail when its protection is removed: never more than two file calls in flight across ten concurrent deletes; an awaited delete gets the next free slot ahead of queued background calls; a link that the directory listing reports as a folder is unlinked, not followed; a target that is itself a link removes only the link; a link loop completes; a refusedunlink/rmdirfalls back torm; a permanent failure removes everything else and rejects with itscodeand the stuck entry.Real Electron binary (
host-tree-removal-asar.electron.test.ts): removes a tree holding a realdefault_app.asar, also when every entry has to go through thermfallback; that second case fails with Electron's patched file system.Boundary test (
src/main/host-tree-removal-boundary.test.ts): putting one old recursivermback fails it and names the file.Existing tests: every test that imports a changed module or stubs the file-system module passes, including the CLI bundle-parity and plain-Node entry guards.
Windows (real machine, Windows 11, Electron 43 as Node): the bundled
removeHostTreeremoved junctions pointing outside the tree without touching their targets (target files byte-identical before and after), a target that is itself a junction, junction loops, dangling junctions, read-only files and folders, a fresh git repo, 633-character paths, and a real pnpm checkout with 86,428 files and 4,010 junctions (26.7 s, nothing left). A read-only file held open by another process is retried and then reported asEBUSYnaming that file; a second call after release removes the rest. The walk tests pass on Windows.Live Electron app on macOS: on the M4Air, a 138,993-entry workspace removed from the sidebar, two runs per side (main bd5dca4, PR fbce1b2). While the trash folder was deleted, the worst file read took 8.1 s and 9.0 s on main (only 4-5 reads finished in about 20 s) and 7 ms and 17 ms on the PR (median about 0.5 ms, the same as idle). The workspace left the sidebar at once, the trash folder was empty at the end, and
git worktree listwas clean. Quitting withapp.quit()right after the delete started: the app exited in about 155 ms on the PR (914 ms on main), and the next launch swept the leftover trash. A folder delete from the file explorer still works. Chat sends were not timed because that needs a logged-in agent; the file reads above go through the same shared file threads.Runtime checks: Electron 43 as Node, plain Node 26 and Bun 1.3.14.
I manually tested these changes (live Electron QA on macOS,
removeHostTreeon a real Windows 11 machine)Automated tests added/updated
Review
Reviewed in six loops. The first design (Node's native
rmSyncon a worker thread) was replaced after a Windows run showed it follows junctions out of the tree; the last loop was clean. CI is green.Agent skill upstream boundary
Notes
Security: no new dependencies, no code evaluation, no network. Cross-platform: macOS live QA, Windows real-machine fixtures and the Windows CI lane, Linux CI. Remote SSH: only local deletes change; the relay is untouched. Mobile: unaffected. Backwards compatibility: no persisted state or wire changes. Performance: covered above.
Checklist
N/Awith reasonpnpm lint,pnpm typecheck,pnpm test, andpnpm buildpass (or CI will cover; local preferred)