Skip to content

fix(ci): retry action-dist network ops and auto-heal failed releases - #596

Merged
goosewobbler merged 2 commits into
mainfrom
fix/action-dist-retry-and-autoheal
Jul 21, 2026
Merged

goosewobbler merged 2 commits into
mainfrom
fix/action-dist-retry-and-autoheal

Conversation

@goosewobbler

@goosewobbler goosewobbler commented Jul 20, 2026 •

Copy link
Copy Markdown
Owner

Context

Follow-up to #593. That PR made the in-run action-dist build reliable (git-derived tag) and added the manual/scheduled heal. This adds resilience for the two remaining failure classes, distinguished by one question: is the workflow still alive to fix itself?

  • Class 1 — a command inside the action-dist job flakes, but the job is alive (git push rejected, gh release create API blip). The workflow is running, so it can retry.
  • Class 2 — the job or the whole run dies (cancellation, runner crash, GitHub incident) after the tag was pushed but before dist was committed. Nothing in-workflow can catch this — the thing that would retry is the thing that died. Only an out-of-band trigger can heal it.

Changes

Retry the network/API ops in _action-release (Class 1)
git push -f (version tag + major alias) and gh release create now retry up to 3× with backoff. These are the genuinely transient operations. Local git ops and pnpm build are deliberately not retried — they fail deterministically and should surface as a red run.

Auto-heal on a non-success Release run (Class 2)
action-dist-heal.yml now also triggers via workflow_run on the Release workflow, gated to a non-success conclusion:

workflow_run:
  workflows: [Release]
  types: [completed]
# resolve job:
if: >-
  github.event_name != 'workflow_run' ||
  (github.event.workflow_run.conclusion != 'success' &&
   github.event.workflow_run.conclusion != 'skipped')

A successful Release run already committed dist in-run (guaranteed by #593's git-derived tag), so healing there would be pure no-op noise on every push to main. A failed/cancelled run is exactly where the in-run build may not have completed — so the heal fires within minutes and targets the most recently created version tag (the guard no-ops if it already carries dist). The daily schedule stays as the cause-agnostic backstop for any unknown mode that somehow leaves a green run dist-less — deliberately not dropping it, since "assume failures are red" is the exact assumption that caused the original incident.

Why keep the heal separate rather than fold it into the release workflow

You can't self-heal from inside the run that died. An if: always() job in release.yml would cover "action-dist job failed but run continues", but not cancellation or infra death (both take the always() job down too). The heal's value is being in a different failure domain — the watchdog can't live inside the supervised process.

No loop

Heal is not tag-triggered, and _action-release commits with [skip ci] then force-pushes the tag, so a heal never re-triggers CI or Release. Confirmed Release triggers only on workflow_run: [CI] + dispatch.

Validation

  • actionlint clean on both files.
  • The heal path is exercised by the same safe idempotency check as before (dispatch against an already-good tag → no-op); the workflow_run path will first exercise on the next non-success Release run, else stays dormant.

🤖 Generated with Claude Code

Greptile Summary

This PR adds two resilience layers to the action-dist release pipeline: transient network retries inside the running job, and an out-of-band auto-heal triggered by any non-success Release run.

  • _action-release.reusable.yml: Introduces a retry() shell function (≤3 attempts, linear backoff) wrapping only the genuinely transient operations — gh release create and both git push -f calls. Deterministic steps (pnpm build, local git ops) are deliberately excluded so real failures surface immediately.
  • action-dist-heal.yml: Adds a workflow_run trigger on the Release workflow, gated to conclusion != 'success' && != 'skipped', so the heal fires within minutes of a failed/cancelled Release run without producing noise on every successful push. Switches the resolve job's tag-picking to a case statement; the new workflow_run arm selects the most recently created version tag via --sort=-creatordate.

Confidence Score: 5/5

Safe to merge — the retry logic is idempotent (force-push replays the same tag; the release-exists guard prevents double-creation), and the heal's idempotency guard in _action-release ensures re-runs on an already-healed tag are always no-ops.

Both changed files make purely additive, well-guarded changes. The retry() function correctly limits to 3 total attempts, uses "$@" for proper argument passing, and exits non-zero after the final failure so the step still turns red on a real issue. The workflow_run gate in the heal workflow correctly narrows to failure/cancellation conclusions and the case-based tag selection handles all three trigger paths without ambiguity. No logic path introduced here can cause data loss or a runaway loop.

No files require special attention.

Important Files Changed

Filename Overview
.github/workflows/_action-release.reusable.yml Adds a 3-attempt retry shell function wrapping git push -f (version tag + major alias) and gh release create; local git ops and build are intentionally not retried
.github/workflows/action-dist-heal.yml Adds a workflow_run trigger on the Release workflow (non-success/non-skipped gate) and switches the tag-pick step to a case statement that selects the most recently created version tag for the new trigger path

Reviews (2): Last reviewed commit: "chore(ci): add trailing newline to _acti..." | Re-trigger Greptile

Two complementary layers on top of the git-derived action tag (#593),
covering the two remaining failure classes:

- Retry the network/API operations in _action-release (git push, gh release
  create) up to 3x. A transient push rejection or API blip is recoverable
  while the job is alive, so it shouldn't strand a dist-less tag. Local git
  ops and pnpm build are not retried — those fail deterministically and
  should surface.
- Trigger the heal on a non-success Release run (workflow_run), not just the
  daily schedule. The realistic remaining failure is the release run dying
  mid-build, which nothing in-workflow can catch — you can't self-heal from
  inside a run that died. Success/skipped runs are ignored (the in-run build
  already committed dist there). The daily schedule stays as the
  cause-agnostic backstop.

No loop: heal is not tag-triggered, and _action-release commits [skip ci]
then force-pushes, so a heal never re-triggers CI/Release.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CmYRXTZQYs4MNo6AETxT
Comment thread .github/workflows/_action-release.reusable.yml Outdated
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CmYRXTZQYs4MNo6AETxT
@github-actions

Copy link
Copy Markdown
Contributor

Standing release PR: #595 · v0.41.2 queued (4 packages) · open 34m · ✅ ready to merge

Release Preview — v0.41.1 → v0.41.2

Note: Labels on this PR are advisory in standing-pr mode. Bumps come from conventional commits in the standing PR; override by editing labels on the standing PR itself. Add release:immediate to bypass the standing PR and release this PR directly.

These changes will be added to the release PR (#595) when merged:

Changelog

monorepo 0.41.1 → 0.41.2

Changed

  • add trailing newline to _action-release.reusable.yml (ci)

Fixed

  • retry action-dist network ops and auto-heal failed releases (ci)

All packages → 0.41.2 (sync versioning)

  • @releasekit/version
  • @releasekit/notes
  • @releasekit/publish
  • @releasekit/release

After merge — predicted release

No version escalation — this PR's changes will be included in the queued release without affecting the projected versions.

Package Standing PR This PR After merge
monorepo 0.41.2 0.41.2 0.41.2

Updated automatically by ReleaseKit

@goosewobbler
goosewobbler merged commit 9806626 into main Jul 21, 2026
26 checks passed
@goosewobbler
goosewobbler deleted the fix/action-dist-retry-and-autoheal branch July 21, 2026 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant