Skip to content

ci: retry a git 403 or 429 on ls-remote or fetch of the default branch instead of failing the job #18056

Open

Description

@kyle-sexton

Problem

Workflow steps that read the default branch over git HTTPS (git ls-remote / git fetch of main) fail their job on a refusal from GitHub, when they could retry. On 2026-11-21 this happened in the hour the REST pool was exhausted:

  1. work-resolve-issue run 44944183413, job "Resolve Issue Handoff", 20:04:21 to 20:07:31Z: step "Resolve main's tip" failed with the tip of main did not resolve, after its three git ls-remote attempts. That threw away the lane run's work on the issue.
  2. Merge-group run 44940480692, job "Check Skills", step "Fetch base" at 19:24:35Z: fatal: unable to access 'https://github.com/melodic-software/claude-code-plugins/': The requested URL returned error: 403, exit 128. It ejected fix(intake-triage): split an issue whose earlier refinement has no split record #17961 from the merge queue. Re-queued, the PR merged with no change.
  3. Relayed from the hub's metrics, not read by this session: merge_group "Fetch base" failures at 19:41Z and 19:57Z.

Cause, UNVERIFIED: GitHub's limit on git requests, or a token-scoped refusal. Both refusals are git over HTTPS, not REST calls, so this is separate from #16642, which covers REST-pool steps that fail at their floor instead of waiting.

Done when

  • A 403 or 429 from git ls-remote or git fetch of the default branch in these steps is retried with backoff, until the attempt fits no longer in the job's remaining time; any other error fails as before.
  • The retry lives in one shared helper that both the "Resolve main's tip" step and the "Fetch base" step call, not copied into each.
  • Tests cover a refusal followed by success, a refusal that outlasts the time left, and a non-retryable error.

Related

Refs: #16642 (REST-pool floor; separate cause).

session: GitHub Work

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 21, 2026
  2. github-actions commented on Nov 21, 2026

    @github-actions
    Contributor

    Size: Medium
    Complexity: Medium
    Uncertainty: Knowable
    Flags: none
    Blocked by: none
    Blast radius: the "Resolve main's tip" step in work-resolve-issue, the merge-group "Fetch base" step, a new shared retry helper and its tests; read by CI runs and the issue lane.
    Reversible: yes; revert the helper and the two call sites.
    Open questions: none
    Understood: git ls-remote/fetch of the default branch fails the job on a 403 or 429, which has ejected a PR from the merge queue and discarded a lane run. You ask for a retry with backoff bounded by the job's remaining time, one shared helper used by both steps, and tests for success after a refusal, a refusal that outlasts the time left, and a non-retryable error. The done-when is exact. The cause is marked unverified, and the helper does not depend on it. Related but not duplicates: #16642 (REST-pool floor, which you already cite) and #17460 (pacing GITHUB_TOKEN spend). No merged pull request that fixes this was found.

    Automated triage. A maintainer reviews it.

  3. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 21, 2026
  4. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"b148021476d807f6fe0d75212323ca5ce94012bd0ffc3295824da927b2ed32cf","body_sha256":"b10425053316ef69b5035c7493efbbaf2cbc373ee6d7e4ee46f8ada0736bd718"}

  5. kyle-sexton commented on Nov 21, 2026

    @kyle-sexton
    MemberAuthor

    Another occurrence: 44955798864, attempt 1, Test Bash (5), step "Fetch base" (the download-full-history action), 2026-11-21 22:16:59Z:

    git fetch --prune --unshallow --no-tags --filter=blob:none
    fatal: unable to access 'https://github.com/melodic-software/claude-code-plugins/': The requested URL returned error: 403
    ##[error]Process completed with exit code 128.
    

    It is the only git 403 among the attempt-1 failures of the 306 full runs on main since 2026-11-14. Attempt 2 was green.

  6. melodic-automation-lanes commented on Nov 21, 2026

    @melodic-automation-lanes
    Contributor

    This records a re-fire of an earlier stop, not a new one. The start gate stopped run 44954521237, the work-resolve-issue run that agent-ready started, before any model turn, because the GITHUB_TOKEN budget was under its start-gate share, or the window was projected to have under its start-project share of the header's limit left by the run's expected end (github-token start-gate, start-project and start-run-minutes in rate-limits.json; the run's start-gate notice names which); that run ended at 2026-11-21T22:32:18Z. work-unblock has now removed and added agent-ready back to start a fresh run. A budget stop is the lanes' load, not a fault in this issue: it never counts toward limits.limit-stops-per-issue or the kill-switch wait.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Work status

    agent-ready

    Priority

    high

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions