Skip to content

mount-sync: bootstrap full-tree pull + recursive fsnotify watch are not provider/path-scoped → fd-storm (~tens of thousands of watches) on large github mirrors #319

Description

@khaliqgant

Problem

On a large workspace (esp. GitHub with deep repos/**/contents trees), the mount mirror storms file descriptors / inotify watches, degrading stability. Two unscoped surfaces drive it:

  1. Recursive watcher add — internal/mountsync/watcher.go:45,78 addDirRecursive walks the entire local mirror and adds one fsnotify watch per directory (fsnotify watches dirs, not files). reservedTopLevel (watcher.go:122) only excludes top-level internal entries (.git/.relay/.skills/digests/node_modules) — it does not scope provider content, so the full github/<owner>/<repo>/contents/** tree gets a watch per dir → tens of thousands of fds.
  2. Bootstrap full-tree pull — internal/mountsync/syncer.go ListTree(ws, path, depth, cursor) has no provider param; only ListEvents(..., provider, ...) does. So the existing --provider flag filters incremental events, not the bootstrap walk — the whole provider tree is pulled + mirrored on bootstrap regardless.

Not covered by recent work

Distinct from #210/#232 (write-path scope case), #315 (connect/sync-readiness decouple), #316 (nested-.relay runtime-state filtering), #312 (self-host SDK). None scope the bootstrap walk or the watcher fan-out.

Proposed fix

  • Extend the existing LazyRepos mechanism (syncer.go:828 LazyRepos *bool, gated at :3317 s.lazyRepos / isUnderLazyGithubRepoSubtree) so the bootstrap full-tree walk is provider/path-scoped, not just lazy github-subtree hydration.
  • Gate watcher.addDirRecursive by the same scope so out-of-scope provider subtrees are neither walked nor watched.
  • A relayfile start scope flag to bound which provider paths are mirrored/watched.

Acceptance

Regression must assert the watched-directory / fd set is bounded by scope (not merely the mirrored tree), e.g. a scoped start does not add watches under out-of-scope github/**/contents.

Provenance

Surfaced via a factory "AR-309" triage; the auto-triage flip-flopped and ultimately mis-labeled it a duplicate. Root cause independently verified against current main source (loci above) before filing. Adjacent to scaling issues #161 / #102.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions