chore(data): refresh contest training data - #162
Closed
github-actions[bot] wants to merge 1 commit into
Closed
github-actions[bot] wants to merge 1 commit into
github-actions[bot] wants to merge 1 commit into
Conversation
|
Sagargupta16
added a commit
that referenced
this pull request
Sep 3, 2026
… set (#163) * fix(data): stop the weekly refresh from silently halving the training set PR #162 was the refresh job's first successful run since 2026-08-17, and it proposed replacing data/data.json with 123,513 records against the 244,950 already committed. Every check passed. Three independent causes: 1. update_data.py prompted for a user count, so under CI input() raised EOFError and it fell back to min(5000, len(usernames)). The committed dataset came from 6,830 contributing users, so the cron could never reproduce it: both refs average 35.9 records per user, meaning even a flawless 5,000-user run lands about 27% short. --users now defaults to 8,000. 2. A throttled fetch was indistinguishable from an account with no contests, because any non-200 returned []. 1,560 of 5,000 users "failed" that way. Retryable statuses (429, 5xx) and network errors now retry with exponential backoff honouring Retry-After, and a failed fetch returns None so the run summary counts failures apart from genuinely empty accounts. 3. Nothing compared the new dataset against the old before overwriting. A run retaining less than --min-retention (default 95%) now aborts with a non-zero exit, which fails the workflow instead of opening a PR that deletes data. --force overrides it. The user cap is bounded by GitHub's 100 MB per-file hard limit, since data/data.json is committed. At ~8.3 KB per contributing user, all 43,158 usernames would yield roughly a 307 MB file that could never be pushed, so "just fetch everyone" is not available. 8,000 lands near 57 MB. The script logs the written size and warns past 90 MB. Also writes through a temp file plus os.replace so an interrupted run cannot truncate the dataset, and drops a time.sleep(0.05) in the result loop that throttled nothing (futures are all submitted up front) while adding 250s of pure latency per 5,000 users. Verified with 25 offline checks against a fake session (429 exhaustion returns None not [], 404 is not retried, Retry-After honoured and capped, tally separates failed from empty) and 18 end-to-end checks driving main() against a throwaway repo layout: the guard blocks a shrinking run and leaves data.json byte-identical, --force overrides, growth is allowed, and an all-failed run never writes an empty file. Repo ruff clean, 59/59 pytest passing. * fix(deps): relock uv.lock for the 2.3.2 version bump `uv sync --locked` fails when pyproject.toml's version differs from the one recorded in uv.lock, which broke the python CI job. Only the project version entry changed; no dependency was re-resolved.
Owner
|
Closing rather than merging: this run would have deleted 121,437 training records (49.6% of the dataset). Counted directly on both refs:
Records-per-user is identical, so per-user data quality was fine. The whole shortfall was fewer users getting through: Fixed in #163, now on
The next scheduled run on 2026-09-08 regenerates this PR against the same branch, with the guards in place. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated weekly refresh of
data/data.jsonviascripts/update_data.py.Retraining is a separate, manual step: see CONTRIBUTING.md. Merging
this only updates the training inputs, not the served model.