feat: schedule around artifacts — defer a key whose artifact is mid-build, group keys by artifact

[sotashimozono]

Why

This is the scheduling half of intermediate-artifact reuse. The other halves are QAtlasHub/ParamIO.jl#40, which defines what an artifact is, and QAtlasHub/DataVault.jl#52, which provides artifact! to store it once and reuse it. With those alone, a worker that needs an artifact another worker is still building can only block. That is fine when there are no more keys than workers. When keys outnumber workers, the worker should take a different key instead.

What

• Deferral. If work_fn throws DataVault.ArtifactBusy (from artifact!(...; wait=false)), the key is treated as deferred, not failed:

  • the event log records :artifact_busy;
  • no attempt is spent, and the key is handed back;
  • run! re-dispatches the deferred keys once the pass drains, logging :deferred_round;
  • after a pass that finished nothing, run! first waits RunOpts(defer_poll=30.0), because the builder is then another job;
  • stop and deadline are honoured, and a key still deferred at stop is counted with busy.

  run! handles this itself, so a driver that calls run! once (not run_loop!) still completes the deferred keys.

• artifact_affinity(vault, name): an affinity that groups keys by the artifact they need. Keys that share an artifact stay on one worker, and distinct artifacts start on distinct workers.

• The README gets a "Shared setup without a barrier" section, and the event-kind table in EventLog.jl gets the two new kinds.

Compat floors are DataVault 0.8.2 and ParamIO 0.4.11. CI needs both registered first. Version: 0.6.3 → 0.6.4, which is additive (0.6.3 is #52, which this PR is rebased onto); RunOpts gains a keyword-only field.

Tests

test/run/test_artifacts.jl has 3 testsets. All are deterministic: they use counters, not timers. The first draft used a Timer and passed vacuously because the first compile took longer than the timer.

• Deferral. The build lock is held as another job would hold it, and released after the third ArtifactBusy. With max_attempts=1, all 6 keys finish with err == 0 and busy == 0. An implementation that spent an attempt would end them as :error. The test also checks that the log contains artifact_busy ≥ 3 times and a deferred_round.
• Stop. The flag is raised at the first deferral. The U = 0.0 keys finish, the deferred key counts as busy, nothing is an error, and no U = 0.2 key is marked done.
• Affinity. 6 keys fall into 2 groups by U.

Existing test_run_minimal, test_affinity, test_run_stop_flag, test_prerequisite and test_run_adversarial pass unchanged locally with -t 2.

🤖 Generated with Claude Code

Rebased onto #52

This PR now sits on #52 (d8a6e5e). Two conflicts and one API change:

• Project.toml: the DataVault floor is 0.8.2. The comment fix: complete the keys a killed sibling held, and stop waiting to find out it is dead #52 wrote about why 0.8.1 was the floor (the owner-stamped .running API) is kept alongside the new reason.
• EventLog.jl: fix: complete the keys a killed sibling held, and stop waiting to find out it is dead #52's worker_died / worker_lost rows are kept, with artifact_busy / deferred_round added after them.
• _is_stopped no longer exists. fix: complete the keys a killed sibling held, and stop waiting to find out it is dead #52 folded it into _stop_reason, so the deferral loop now breaks on _stop_reason(opts) !== nothing.

A deferred key is counted with busy. #52's run_loop! rule, which does not count a busy round as empty until stale_after, therefore also keeps a loop alive while an artifact is being built elsewhere.

Run locally after the rebase with -t 2, all passing:

• test_artifacts, test_liveness, test_run_loop_busy and test_run_deadline
• test_run_stop_flag, test_run_minimal, test_prerequisite and test_affinity
• test_run_adversarial and test_eventlog

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

📚 Docs preview: https://qatlashub.github.io/SweepRunner.jl/previews/PR53/

(updates on each push to this PR)