Skip to content

feat: evict idle superseded snapshots during sync - #28

Merged
yuezengwu merged 9 commits into
mainfrom
feat/snapshot-gc
Nov 4, 2026
Merged

yuezengwu merged 9 commits into
mainfrom
feat/snapshot-gc

Conversation

@liuchao-001

Copy link
Copy Markdown
Contributor

Problem

Every fetching sync that lands on a new commit creates a snapshot at snapshots/<key>-<sha>, and nothing ever removes the old ones. On one machine this reached 676 MB, almost all superseded snapshots.

Change

  • Each resolution (read, sync, write prepare, ...) records a use by setting the snapshot directory's mtime.
  • A fetching sync removes snapshots registered in the same bare repository cache that are not the snapshot it just resolved and have not been used for 24 hours.
  • Kept: modified or untracked content (including ignored files), snapshots whose lock is held, locked Git worktrees, anything outside snapshots/ (write-session worktrees), and non-snapshot names.
  • The use time is re-checked under the snapshot lock, so a resolve that finished just before eviction keeps it.
  • Eviction is best effort: any failure leaves that snapshot in place and never fails the sync. sync --offline never evicts.
  • An evicted snapshot is recreated at the same path from the bare cache when resolved again.
  • Repository caches are still never evicted. The specification is updated.

Why 24 hours, not immediately

Agents receive a snapshot path and may keep reading it for a whole task without calling the CLI again. Removing a snapshot as soon as a newer commit appears would delete files under a running reader.

Tests

New tests/snapshot-eviction.test.ts: retained inside the window and removed after it; modified, locked, and write-session worktrees preserved; offline sync does not evict. pnpm check, pnpm typecheck, pnpm test (119 tests) pass.

This touches src/sources/resolve.ts like #27; whichever lands second needs a small rebase.

🤖 Generated with Claude Code

liuchao-001 and others added 4 commits November 4, 2026 17:38
Use a blobless partial clone (--filter=blob:none) for the bare repository
cache and an anchored, literal non-cone sparse pattern for subtree
snapshots and write worktrees, so only the selected subtree's file
contents are downloaded and materialized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every fetching sync that lands on a new commit creates a snapshot, and
nothing removed the old ones, so the snapshot directory grew without
bound. A fetching sync now removes snapshots of the same repository cache
that are superseded and unused for 24 hours. Each resolution records a
use on its snapshot. Modified, locked, or busy snapshots and write-session
worktrees are kept, and eviction never fails the sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
liuchao-001 and others added 3 commits November 4, 2026 20:38
The blobless cache still downloaded every commit's full directory
structure (12 MB for a repository whose docs subtree is 1 MB). A
treeless clone stores commits only and downloads trees and contents
when a snapshot or write worktree checks them out. Covers commit,
publish, and rejected-push recovery through a treeless cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@yuezengwu yuezengwu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduced a concurrent-use eviction defect on 5ed9c2d. The four existing eviction tests, repository check and typecheck pass; an independent regression demonstrates that a just-read snapshot can still be removed. Details are inline. Integration with current main also has conflicts in specification and sparse-worktree tests; no author branch was modified.

Comment thread src/sources/eviction.ts Resolved
liuchao-001 and others added 2 commits November 4, 2026 22:12
…s through removal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@liuchao-001
liuchao-001 requested a review from yuezengwu November 4, 2026 14:17

@yuezengwu yuezengwu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed 41318ce integrated with current main ce8789e. The explicit-use timestamp and final eviction check/removal now share the snapshot lock; status preflight occurs before that final check. The previously reported interleaving is covered by a regression and passes. Git non-force removal still protects modifications made after the status check.

All 123 tests passed with HTTPS Git transport prohibited, along with repository checks, typecheck and real package smoke. The prior finding is resolved; no remaining defect found in this change.

@yuezengwu
yuezengwu merged commit 997a68c into main Nov 4, 2026
3 checks passed
@yuezengwu
yuezengwu deleted the feat/snapshot-gc branch November 4, 2026 14:42
@liuchao-001 liuchao-001 mentioned this pull request Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants