feat: large repo scaling — dedup, proportional budget, time-decay, package detection, context-augmented training - #29
Open
bearmug wants to merge 1 commit into
Open
feat: large repo scaling — dedup, proportional budget, time-decay, package detection, context-augmented training#29bearmug wants to merge 1 commit into
bearmug wants to merge 1 commit into
Conversation
- git-history.ts: SHA-based dedup in mineHistory, package detection (packages/|apps/|libs/ workspaces), dedupKey utility - cards.ts: time-decayed scoring (half-life: 180d fixes/churn, 365d arch), proportional card budget scaling (20 + n/200 commits, max 200), bounded suggestion text (max 3 example subjects), package tagging, min decay floor to prevent cards from vanishing - training.ts: contextFormat option for context-augmented training data, formatCardContext helper for structured card rendering The combination: large repos (10k+ commits) now get proportionally more cards, time-decayed scores favor recent activity, suggestion text is compact, and training data can be generated in context+question format.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Improves repo-arch for large monorepos (validated on
fiale-plus/piwith 4089 commits across 1542 files).Changes
git-history.ts (+20/-3)
mineHistory(): prevents duplicate commits when history is re-minedpackages/*,apps/*,libs/*workspaces): each commit gets apackageIdfileDedupKey()utility for stable dedup across filescards.ts (+123/-19)
timeDecay()applies exponential decay (half-life: 180d fixes/churn, 365d arch). Minimum decay floor of 0.5 prevents cards from vanishing.computeCardBudget()scales with repo size —20 + n/200 commits, max 200. Large repos get proportionally more cards.boundedSuggestion()limits to 3 example commit subjects (was: all subjects, producing 50KB+ card text).packageIdanddecayedScorefor filtering/grouping.training.ts (+36/-7)
--context-formatflag wraps examples in card context (context + question -> answerinstead of plainquestion -> answer).formatCardContext()renders structured card metadata as context blocks.Testing
All 79 existing tests pass. Verified on
fiale-plus/pi(monorepo, 4089 commits):