Add post-reset spread to _github_csma_sleep_after_rate_limit to prevent thundering herd on retries

[mysticgohan1]

What happened

PR #2304 added a post-reset spread (up to 60s via GITHUB_CSMA_SPREAD_MAX_SEC) to github_csma_sense in internal/scaffold/fullsend-repo/scripts/lib/github-api-csma.sh to prevent multiple runners from waking simultaneously after a rate-limit reset. The review agent correctly identified that _github_csma_sleep_after_rate_limit (lines 142-164) has the same thundering-herd problem: when multiple runners hit a 429/secondary-rate-limit, they all sleep until the same reset timestamp and wake together. The exponential backoff component adds some randomness but does not fully desynchronize runners on the same attempt count. The finding was not addressed before the PR was approved and merged on 2026-06-16.

What could go better

The fix should have been applied to both sleep-until-reset paths. This is a straightforward omission — the same spread logic that was added to github_csma_sense should also be added after the sleep "${delay}" call in _github_csma_sleep_after_rate_limit. Confidence: high — the review agent's analysis is correct and the two code paths are structurally parallel.

Proposed change

In internal/scaffold/fullsend-repo/scripts/lib/github-api-csma.sh, add the same post-reset spread logic after the sleep "${delay}" call in _github_csma_sleep_after_rate_limit (around line 160). Either extract the spread logic into a shared helper function (e.g., _github_csma_post_reset_spread) called from both github_csma_sense and _github_csma_sleep_after_rate_limit, or inline the same spread_max / RANDOM % spread_max / sleep pattern. The helper approach is preferred to avoid duplication.

Validation criteria

After the fix: (1) _github_csma_sleep_after_rate_limit includes a random spread delay after sleeping until the reset timestamp, using the same GITHUB_CSMA_SPREAD_MAX_SEC configuration. (2) Existing tests (post-prioritize-test.sh and any CSMA unit tests) continue to pass. (3) In a batch of 5+ concurrent runners hitting a 429, runners should wake at staggered times rather than simultaneously.

---

Generated by retro agent from fullsend-ai#2304

[mysticgohan1]

/fs-triage

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 6:43 AM UTC · Completed 6:45 AM UTC
Commit: bb5c185 · View workflow run →

[fullsend-ai-triage]

Triage Summary

Severity: Medium · Category: Bug

Problem

_github_csma_sleep_after_rate_limit (lines 142-164 in github-api-csma.sh) lacks the post-reset spread that was added to github_csma_sense in upstream PR fullsend-ai#2304. When multiple runners hit a 429, they all compute the same sleep-until-reset delay and wake simultaneously, causing a thundering herd that re-triggers rate limits.

Root Cause

The function sleeps until the rate-limit reset timestamp but does not add any random spread afterward. The exponential backoff adds some jitter but doesn't fully desynchronize runners on the same attempt count.

Recommended Fix

Extract the spread logic into a shared helper (e.g., _github_csma_post_reset_spread) that reads GITHUB_CSMA_SPREAD_MAX_SEC and sleeps a random duration up to that max. Call it from both github_csma_sense (after line 87) and _github_csma_sleep_after_rate_limit (after line 163).

Proposed Test

# With GITHUB_CSMA_SPREAD_MAX_SEC=5, verify that
# _github_csma_sleep_after_rate_limit adds a random spread
# (0-5s) after the base delay. With SPREAD_MAX_SEC=0,
# verify no spread is added. Run from parallel subshells
# to confirm wake times are staggered.

Validation Criteria

1. _github_csma_sleep_after_rate_limit includes a random spread delay using GITHUB_CSMA_SPREAD_MAX_SEC
2. Existing tests continue to pass
3. Concurrent runners wake at staggered times rather than simultaneously

---

Labels: This is a confirmed defect in the CSMA rate-limit handling within dispatch-related scaffold scripts.

[fullsend-ai-coder]

🤖 Finished Code · ✅ Success · Started 6:48 AM UTC · Completed 6:56 AM UTC
Commit: bb5c185 · View workflow run →