Skip to content

fix: fetch only the selected subtree with a treeless partial clone - #27

Merged
yuezengwu merged 4 commits into
mainfrom
fix/subtree-partial-clone
Nov 4, 2026
Merged

yuezengwu merged 4 commits into
mainfrom
fix/subtree-partial-clone

Conversation

@liuchao-001

@liuchao-001 liuchao-001 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Part of #29.

Summary

Managed Git sources downloaded far more than the selected tree:

  • The repository cache was a full git clone --bare. For first-tree-ai/opentag (tree at docs) this is 76 MB and 51 s.
  • Cone-mode sparse snapshots also materialized every file at the repository root.

This change:

  • Clones the cache as a treeless partial clone (--filter=tree:0). It stores commits only; directory trees and file contents are downloaded when a snapshot or write worktree checks them out. Later fetches inherit the filter via remote.origin.partialclonefilter.
  • Uses non-cone sparse checkout with one anchored, escaped literal pattern (/<subtree>/) for snapshots and write worktrees, so only the selected subtree is materialized. Commits and publishing keep a full index and preserve outside paths.
  • Tests against a filtering remote: only the selected subtree is downloaded; commit, publish, rejected push (WRITE_OUTDATED, session preserved), and a fresh retry all work through a treeless cache.
  • Updates docs/specification.md.

Measurements (real first-tree-ai/opentag, docs, one Local Computer)

Cache Clone Size
full bare 51 s 76 MB
blob:none 6–24 s 12 MB
tree:0 2.5 s 2.3 MB

With a warm treeless cache: a fetch is about 1.2 s per repository; checking out a new commit of docs is about 3 s.

Validation

pnpm check, pnpm typecheck, pnpm test (118 tests), pnpm check:package pass.

Trade-offs

  • sync --offline never fetches refs, but creating a snapshot whose trees or contents are not cached needs the remote.
  • Existing full caches are left as-is; they shrink only if removed and re-synced.

🤖 Generated with Claude Code

Use a blobless partial clone (--filter=blob:none) for the bare repository
cache and an anchored, literal non-cone sparse pattern for subtree
snapshots and write worktrees, so only the selected subtree's file
contents are downloaded and materialized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
liuchao-001 and others added 2 commits November 4, 2026 20:38
The blobless cache still downloaded every commit's full directory
structure (12 MB for a repository whose docs subtree is 1 MB). A
treeless clone stores commits only and downloads trees and contents
when a snapshot or write worktree checks them out. Covers commit,
publish, and rejected-push recovery through a treeless cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@liuchao-001 liuchao-001 changed the title fix: fetch only the selected subtree for managed Git sources fix: fetch only the selected subtree with a treeless partial clone Nov 4, 2026
@liuchao-001
liuchao-001 requested a review from yuezengwu November 4, 2026 12:43

@yuezengwu yuezengwu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The treeless-cache and literal sparse-checkout behavior passed the full118-test suite, typecheck and repository check. One reproducible test-isolation defect remains: the object-presence assertions perform implicit external promisor fetches instead of staying local. Details are inline.

Comment thread tests/sparse-worktrees.test.ts Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@liuchao-001
liuchao-001 requested a review from yuezengwu November 4, 2026 13:12

@yuezengwu yuezengwu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed f89dc02 against current main2e4be21f. The object-presence probe now disables lazy fetching and has a local-only fallback origin. Independent Git tracing found zero fetch children across all four presence probes with HTTPS prohibited.

The full118-test suite, typecheck and repository check passed. The prior test-isolation finding is resolved; no remaining code defect found in the treeless cache, literal sparse checkout or write/publish behavior.

@yuezengwu
yuezengwu merged commit ce8789e into main Nov 4, 2026
3 checks passed
@yuezengwu
yuezengwu deleted the fix/subtree-partial-clone branch November 4, 2026 13:34
@liuchao-001 liuchao-001 mentioned this pull request Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants