Skip to content

Parallelize VCS calls in list_workspace_entries via thread scope#10

Merged
drivasperez merged 6 commits into
mainfrom
claude/investigate-monorepo-hangs-ZDzrn
May 21, 2026
Merged

Parallelize VCS calls in list_workspace_entries via thread scope#10
drivasperez merged 6 commits into
mainfrom
claude/investigate-monorepo-hangs-ZDzrn

Conversation

@drivasperez

Copy link
Copy Markdown
Owner

Summary

Refactor list_workspace_entries_inner to parallelize per-workspace VCS operations using std::thread::scope. This reduces latency when listing workspaces by running independent VCS subprocesses (diff_stat_vs_trunk, latest_description, is_merged_into_trunk) concurrently across workspaces.

Key Changes

  • Introduce EntryCandidate and ComputedEntry structs to separate candidate collection from parallel computation:

    • EntryCandidate holds workspace metadata and VCS info collected upfront
    • ComputedEntry holds per-workspace fields computed in parallel (diff_stat, description, modified time, is_stale)
  • Parallelize VCS calls via std::thread::scope: Each workspace candidate spawns a thread that independently runs up to 3 VCS subprocesses. These are independent across workspaces and dwarf the cost of thread spawning.

  • Optimize trunk branch detection: Add OnceLock-based caching to GitBackend to memoize detect_trunk() result. Since trunk is invariant for a single dwm invocation, this avoids redundant git subprocess calls across multiple workspaces.

  • Extract stale_by_mtime() helper: Separate mtime-based staleness check from compute_is_stale() to allow skipping the expensive is_merged_into_trunk VCS call when inactivity alone marks a workspace stale.

  • Make VcsBackend trait Send + Sync: Required to pass backend references across thread boundaries in the scope.

  • Update GitBackend to Default: Change from unit struct to #[derive(Default)] struct to support the trunk cache field.

Implementation Details

  • The thread scope collects all workspace candidates first (main workspace + subdirs under .dwm/worktrees/), then fans out VCS calls in parallel
  • Each thread captures immutable references to the backend and main repo path, avoiding data races
  • Staleness computation is optimized: if a workspace is already stale by mtime (>30 days), skip the merged probe subprocess
  • All test instantiations of GitBackend updated to use GitBackend::default()
  • Added unit tests for stale_by_mtime() and trunk cache behavior

https://claude.ai/code/session_011a8xba7hxCYzTrxA5LULBQ

claude added 4 commits May 21, 2026 10:13
detect_trunk runs up to 3 git subprocesses (rev-parse refs/heads/main,
then master, then symbolic-ref refs/remotes/origin/HEAD) and is called
from diff_stat_vs_trunk, is_merged_into_trunk, and preview_diff_stat —
once per workspace per dwm list. Trunk is invariant within a single dwm
invocation, so cache the result on the backend instance.

On a small test repo (10 worktrees), dwm status drops from ~189ms to
~155ms (~18%); on a large monorepo the win scales with workspace count.

https://claude.ai/code/session_011a8xba7hxCYzTrxA5LULBQ
Each non-main workspace serially issued up to 3 cold VCS subprocesses
(diff_stat_vs_trunk, optional latest_description, is_merged_into_trunk).
With N workspaces this serial chain dominates dwm list / dwm status wall
time. Fan the per-workspace work out over std::thread::scope — no new
dependency, one OS thread per workspace (cheap; threads block on
subprocess IO anyway). Display order is preserved via positional zip.

Required adding Send + Sync to the VcsBackend trait; both impls satisfy
it automatically.

On a small test repo (10 worktrees), dwm status drops from ~155ms to
~69ms (~55%) on top of the previous trunk memoisation; the TUI 10-second
background refresh hits the same path and benefits identically.

https://claude.ai/code/session_011a8xba7hxCYzTrxA5LULBQ
GitBackend::workspace_list eagerly ran `git log --format=%s -1` per
worktree to populate WorkspaceInfo::description. The list path in
workspace.rs then called latest_description (which is the same
`git log -1`) whenever the description came back empty — so for any
worktree with commits we paid for the same data twice. Setting
description to empty here makes the lazy path the sole source.

On the test repo dwm status drops from ~69ms to ~47ms (~32%);
cumulative reduction from the original baseline is ~75% (189ms → 47ms).

https://claude.ai/code/session_011a8xba7hxCYzTrxA5LULBQ
compute_is_stale returns true if the workspace is merged into trunk OR
its mtime is older than STALE_DAYS. The list path unconditionally paid
for the is_merged_into_trunk VCS subprocess even when mtime alone
already determined staleness. Short-circuit: if mtime > 30d, mark
stale and skip the probe.

Visible behaviour preserved: stale-by-age is still stale,
recently-merged is still probed and still marked stale.

No measurable change on the test repo (all worktrees have fresh mtime
so the short-circuit doesn't trigger); the win shows on real
long-lived workspaces with stale ones.

https://claude.ai/code/session_011a8xba7hxCYzTrxA5LULBQ
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented May 21, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
dwm e9fb68d May 21 2026, 10:33 AM

claude added 2 commits May 21, 2026 10:32
- Remove redundant comment in GitBackend::workspace_list; the empty
  String::new() in the description field is self-explanatory.
- Drop the .into_iter() call clippy 1.95 flags as useless_conversion
  in list_workspace_entries_inner.

https://claude.ai/code/session_011a8xba7hxCYzTrxA5LULBQ
Clippy in 1.95 promoted unnecessary_sort_by into the default lint set;
the pre-existing sort_by in sort_entries fits sort_by_key cleanly.

https://claude.ai/code/session_011a8xba7hxCYzTrxA5LULBQ

Copy link
Copy Markdown
Owner Author

Pushed two fixes for the review and CI:

  • src/git.rs:143 "We don't need this comment" — removed in 427a53d.
  • CI clippy failure — CI is on Rust 1.95 which promoted unnecessary_sort_by into the default lint set. The pre-existing sort_by in src/tui.rs:166 (from PR Feat/dwm extensions #7) and a useless_conversion in my new parallelised list path were tripping it. Both fixed in 427a53d / e9fb68d.

Re. src/git.rs:108 "Can this use LazyLock?"LazyLock's initializer is fixed at struct-construction time, but detect_trunk(dir) needs the worktree path which only becomes available when the methods are called later (each GitBackend is built by VcsType::to_backend / vcs::detect with no dir argument). I could plumb the repo root onto the struct at construction and use a custom-closure LazyLock, but that means restructuring vcs::detect and VcsType::to_backend — a noticeably bigger diff. OnceLock keeps the change localised to GitBackend while still being thread-safe and lazy. Happy to do the bigger refactor if you'd prefer it.


Generated by Claude Code

@drivasperez
drivasperez merged commit aec10ff into main May 21, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants