Skip to content

feat(miner-governor): freeze/snapshot mechanism for historical replay targets (#3010)#5113

Merged
JSONbored merged 2 commits into
mainfrom
feat/miner-replay-snapshot-3010
Jul 11, 2026
Merged

feat(miner-governor): freeze/snapshot mechanism for historical replay targets (#3010)#5113
JSONbored merged 2 commits into
mainfrom
feat/miner-replay-snapshot-3010

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Closes #3010.

Adds exportReplaySnapshot (packages/gittensory-miner/lib/replay-snapshot.js): given a repo and commit T, produces a detached git worktree checkout at T (never mutates the caller's own checkout/branch) plus a context bundle: commit history up to and including T by ancestry, tags reachable from T, and the README as it existed at T.

Note on this PR's history: this issue was previously closed by #3782, an unrelated PR (Rust protobuf stub classification) that cited this issue number to satisfy the linked-issue gate requirement without implementing the actual feature. Reopened and implemented for real here.

Design notes

  • Ancestry, not date, bounds "up to T": git log T walks the DAG, which is tamper-resistant since a commit's committer date is user-controlled and can't be trusted alone.
  • README matched by search, not guess: git ls-tree lists the root, matched case-insensitively against readme(\.\w+)?, then git show T:<name> — handles README, Readme.md, README.rst, etc. without a fixed spelling list.
  • Fail-fast validation is genuinely meaningful, not just defensive: ancestry-walking already excludes anything unreachable from T by construction, but a tag can point at an ancestor of T while the tag's own creation date is later than T (e.g. a tag added long after the commit it targets), and committer dates aren't strictly monotonic along the DAG in general (rebases, clock skew). validateSnapshotFreshness checks every commit and tag date against T's own commit date and throws, listing every violation, if any postdate T.
  • Commit-keyed persistence: UNIQUE (repo_full_name, commit_sha) in the local store — re-exporting the same (repo, T) pair returns the cached row without touching git again, which is both how "byte-reproducible" holds trivially and avoids redundant work on repeat replay runs.
  • Reuse, verified not assumed: the issue's own text points at "existing discovery/analyze git-history utilities" as a starting point. I grepped both packages/gittensory-engine and packages/gittensory-miner for git-log/tag-reading logic before writing anything and found none — opportunity-fanout.js reads the GitHub API's issue updated_at, not git commit/tag history at all. The one piece that is genuinely reusable — worktree-allocator.ts's injected-exec convention (WorktreeExecFn) and its removeWorktree — is imported directly from @jsonbored/gittensory-engine rather than inventing a third "inject the git subprocess" abstraction alongside cli-subprocess-driver.ts's and worktree-allocator.ts's own. Documented inline at the call site per the issue's own requirement.

Scope

Validation

  • npm run typecheck — clean.
  • node --check on the new file + full packages/gittensory-miner build script (57 files) — all pass.
  • npx vitest run test/unit/miner-replay-snapshot.test.ts — 31/31 passing, covering every deliverable: a repo with tags (single and multiple), a repo with no tags, a commit at the very first commit of history, README present/absent/differently-cased, the freshness-violation fail-fast path, cache-hit short-circuit (verified via a second exec with zero scripts — any call would throw), every git-subcommand failure path, and fail-closed validation on malformed input.
  • Scoped coverage: npx vitest run test/unit/miner-replay-snapshot.test.ts --coverage --coverage.include="packages/gittensory-miner/lib/replay-snapshot.js" → 100% stmts/branches/funcs/lines (104/104, 60/60, 27/27, 91/91).
  • npm audit --audit-level=moderate — no dependency changes.

Safety

  • No secrets/wallets/hotkeys/trust-scores exposed.
  • Fail-closed on every malformed-input and git-subcommand-failure path (tested).
  • Not an API/OpenAPI/MCP/UI surface change.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 11, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui fd6aac9 Commit Preview URL

Branch Preview URL
Jul 11 2026, 04:45 PM

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.32%. Comparing base (46d63b9) to head (fd6aac9).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5113   +/-   ##
=======================================
  Coverage   94.32%   94.32%           
=======================================
  Files         471      471           
  Lines       39821    39821           
  Branches    14533    14533           
=======================================
  Hits        37560    37560           
  Misses       1583     1583           
  Partials      678      678           
Flag Coverage Δ
shard-1 46.34% <ø> (-0.01%) ⬇️
shard-2 34.40% <ø> (-0.26%) ⬇️
shard-3 30.85% <ø> (-1.25%) ⬇️
shard-4 33.11% <ø> (+1.09%) ⬆️
shard-5 33.42% <ø> (-0.17%) ⬇️
shard-6 45.17% <ø> (+0.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gittensory-orb gittensory-orb Bot added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. labels Jul 11, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 11, 2026

Copy link
Copy Markdown

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-11 16:46:14 UTC

4 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
Adds exportReplaySnapshot: a detached git worktree checkout at commit T plus a context bundle (ancestry-walked commit history, merged tags, README-at-commit), with fail-fast freshness validation and a commit-keyed SQLite cache. The git-plumbing choices (ancestry over date, ls-tree-based README discovery, --merged for tags) are sound and well-reasoned, and the test suite scripts exec calls thoroughly across happy-path and failure branches. The notable gap: exportReplaySnapshot creates the worktree (step 2) before any of the subsequent steps that can fail (date lookup, history, tags, README, freshness check) — none of which clean up the worktree on failure, so a partial failure (including the deliberately-designed freshness violation) leaves an on-disk worktree at the deterministic planReplaySnapshotPath location while the store row is never saved, and every retry for that same (repo, commit) then fails at `git worktree add` with an unrelated 'path already exists' style error instead of the real problem.

Blockers

  • packages/gittensory-miner/lib/replay-snapshot.js exportReplaySnapshot (around lines 236-254): if any step after addDetachedWorktree throws — readTargetCommitDate, readCommitHistory, readReachableTags, readReadmeAtCommit, or validateSnapshotFreshness — the worktree created at the deterministic `planReplaySnapshotPath` path is left on disk with no store row saved; a retry for the same (repo, commit) then calls `addDetachedWorktree` again at the same path, which `git worktree add` rejects since the directory already exists, permanently masking the real error and blocking recovery until someone manually removes the worktree.
  • packages/gittensory-miner/lib/replay-snapshot.js:105 uses `%(creatordate)` from `git tag --merged`, but lightweight tags report the tagged commit's date rather than the tag creation time, so a lightweight tag created after T but pointing at an ancestor of T is exported as historical context instead of being rejected; change this to inspect tag object type and either require annotated tags with `%(taggerdate:iso-strict)` bounded by `targetDate` or deliberately omit/reject lightweight tags with a clear error.
  • packages/gittensory-miner/lib/replay-snapshot.js:230 creates the worktree before the later git reads and freshness validation, but no cleanup runs when `readTargetCommitDate`, `readCommitHistory`, `readReachableTags`, `readReadmeAtCommit`, or `validateSnapshotFreshness` throws, so a reachable failure leaves `.gittensory-replay-snapshots/<sha>` behind and the next export for the same uncached pair fails at `git worktree add`; wrap the post-add export in `try/catch` and call `removeReplaySnapshotWorktree(exec, repoPath, worktreePath)` before rethrowing.
Nits — 6 non-blocking
  • The external brief flags depth-5 nesting in replay-snapshot.js around line 70 (readCommitHistory/readReachableTags parsing) — could flatten with early returns or a shared line-parsing helper.
  • No test exercises the partial-failure-then-retry scenario (e.g. freshness violation followed by a second export call for the same commit), which is exactly the path that would have caught the worktree-orphaning issue above.
  • validateSnapshotFreshness parses `input.targetDate`/`commit.date`/`tag.date` via `Date.parse` without validating the strings are well-formed dates first; a malformed date would silently produce `NaN` and never compare as `>`, silently skipping the freshness check for that entry rather than failing loudly.
  • Wrap the post-worktree-creation steps in a try/catch that calls `removeReplaySnapshotWorktree` (or bare `git worktree remove --force`) on failure before rethrowing, so a failed export doesn't permanently block retries at the same commit.
  • Consider validating that `Date.parse(...)` did not return `NaN` in validateSnapshotFreshness and throwing on unparseable dates rather than silently letting them pass the freshness check.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #3010
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 45 registered-repo PR(s), 37 merged, 414 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 45 PR(s), 414 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 45 PR(s), 414 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@gittensory-orb

Copy link
Copy Markdown

An AI reviewer flagged a likely defect, but its confidence was below this repository's configured close-confidence floor, so this is held for a maintainer to confirm instead of closing automatically. Resolve the flagged defect (see the review notes), or ask a maintainer to override.

… targets (#3010)

Adds exportReplaySnapshot: given a repo and commit T, produces a
detached git-worktree checkout at T (never mutates the caller's own
checkout/branch) plus a context bundle (commit history up to and
including T by ancestry, tags reachable from T, and the README as it
existed at T, matched case-insensitively via git ls-tree rather than
a guessed filename list).

Fail-fast validation: every exported commit's and tag's date is
checked against T's own commit date, since ancestry-walking excludes
unreachable commits by construction but does not guarantee dates are
monotonic (a tag can be created long after the commit it points to;
committer dates aren't strictly increasing along the DAG in general).

Persistence is commit-keyed (UNIQUE on repo_full_name, commit_sha) in
the local store -- re-exporting the same (repo, T) pair returns the
cached row rather than re-running git, which is both how "byte-
reproducible" holds trivially and avoids redundant work.

Reuse: this issue's own text points at "existing discovery/analyze
git-history utilities" as the starting point. Grepped both packages
before writing this and found no such utility (opportunity-fanout.js
reads GitHub API issue updated_at, not git commit/tag history at
all). The genuinely reusable piece -- worktree-allocator.ts's
injected-exec convention and its removeWorktree -- is reused directly
via @jsonbored/gittensory-engine rather than inventing a third
"inject the git subprocess" abstraction.

Note on this issue's history: previously closed by #3782, an
unrelated PR (Rust protobuf stub classification) that cited this
issue number to satisfy the linked-issue gate without implementing
the actual feature. Reopened and implemented for real here.

Closes #3010.
…nverifiable lightweight tags

Two real defects found by AI review on the original PR:

- exportReplaySnapshot created the worktree, then ran 4 more git reads
  plus the freshness validation with no cleanup on failure. A thrown
  error after the worktree existed left it on disk at the deterministic
  planReplaySnapshotPath location with no store row saved, so a retry
  for the same (repo, commit) pair hit git worktree add's own "path
  already exists" refusal instead of the real underlying error,
  permanently masking it. Wrapped the post-add steps in try/catch:
  clean up (best-effort) before rethrowing the ORIGINAL error, never a
  cleanup error.

- readReachableTags used %(creatordate) from git tag --merged, but a
  lightweight tag has no tag object of its own, so %(creatordate) for
  one silently falls back to the POINTED-TO commit's date -- git has no
  record of when a lightweight tag was actually created at all. That
  meant a lightweight tag added long after T, but pointing at an
  ancestor of T, could never trigger validateSnapshotFreshness's check
  (its reported date is always <= T's by construction of --merged).
  Since this can't be verified, lightweight tags are now excluded from
  the export entirely -- %(objecttype) is "tag" only for an annotated
  tag's own tag object, "commit" for a lightweight tag's direct target.
@JSONbored JSONbored force-pushed the feat/miner-replay-snapshot-3010 branch from 6178578 to fd6aac9 Compare July 11, 2026 16:40
@JSONbored JSONbored merged commit aacf718 into main Jul 11, 2026
20 checks passed
@JSONbored JSONbored deleted the feat/miner-replay-snapshot-3010 branch July 11, 2026 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. manual-review Gittensor contributor context

Projects

Development

Successfully merging this pull request may close these issues.

feat(miner): freeze/snapshot mechanism for historical replay targets

1 participant