Two run tests assumed timings a hosted runner does not give (0.6.8) - #60
Merged
Merged
Conversation
DataVault's downstream job failed 3 times in 15 identical runs, each on one of these assertions: - test_run_loop_busy.jl: `elapsed >= 3.0` (stale_after) failed at 2.85 s and 2.90 s. DataVault writes `heartbeat=` truncated to the second, so a lock reads up to 1 s older than it is and is reclaimed up to 1 s before stale_after. Measured: a lock written at .95 s is reclaimable 2.07 s later. Now `>= stale_after - 1`. With the busy wait disabled (the bug this pins), run_loop! returns in 0.96-1.01 s with done == 0, so the new bound still catches it. - test_run_deadline.jl: a deadline 0.3 s ahead passed before `run!` handed out its first key (started == 0); a run-start observation takes most of a second. The deadline is now 5 s ahead and the first key runs until it has passed, so the test no longer depends on how fast `run!` starts. Same cause as e207f53. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
📚 Docs preview: https://qatlashub.github.io/SweepRunner.jl/previews/PR60/ (updates on each push to this PR) |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DataVault.jl's downstream
SweepRunnerjob failed 3 times out of 15 identical CompatHelper PRs (#60, #62, #72). Each failure was one of these two assertions, and the contents of those PRs were identical to the 12 that passed. TheEOFError/Error encountered while load … manifest.jld2lines in those logs appear in passing runs too, the same number of times, and are not the failure.test_run_loop_busy.jl:51—elapsed >= 3.0(failed at 2.85 s and 2.90 s)DataVault writes
heartbeat=truncated to the second ("yyyy-mm-ddTHH:MM:SS"), so a lock reads up to 1 s older than it is and is reclaimed up to 1 s beforestale_after.Measured locally:
When
elapsedlands between 2 and 3 s depends on where the round boundaries fall, which is why the test failed only sometimes. The bound is nowstale_after − 1.Does it still catch the bug it pins? With the busy wait disabled (
if false && result.busy > 0),run_loop!returned in 0.96 / 0.99 / 1.01 s withdone == 0after warm-up, so>= 2.0fails as it should.r.done == 1andis_donealso catch it. The first cold run took 9.7 s because of compilation and reclaimed the key; the old>= 3.0misses that case too, so the new bound is no weaker.With
stale_after = 600in production, reclaiming up to 1 s early does not matter. The.runningformat is a contract, so DataVault is not changed.test_run_deadline.jl:58—started[] >= 1(was 0)A deadline 0.3 s ahead passed before
run!handed out its first key: a run-start observation takes most of a second. This is the same cause as e207f53.RunOptsis immutable anddeadlineis absolute, so the fix cannot trigger on the first key the way that commit did. Instead the deadline is 5 s ahead and the first key runs until it has passed. The test still checks the same thing (at least one key started, not every key, returned after the deadline,stopped_by === :deadline) and no longer depends on how fastrun!starts. It takes about 5 s longer.Checked
test_run_loop_busy.jl23 s,test_run_deadline.jl19 s). JuliaFormatter 2 reports them already formatted.🤖 Generated with Claude Code