Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions .github/workflows/main-red-alert.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
name: Main Red Alert

# A red main is invisible, and it silently stops the whole repo.
#
# 2026-08-06: a regression reddened CI on main at 17:40 the previous day. The
# fix was pushed to a PR branch, that branch went green, and the fix was called
# done — but nothing re-runs a base-branch run, so main's own run stayed red.
# Auto-merge's green-base guard then refused every merge for ~16 hours, holding
# back 17 PRs while logging "refusing to merge onto a broken base" into a
# workflow nobody opens. It presented as "the agent can't merge" rather than
# "the base is broken", which is the expensive part: the symptom points away
# from the cause.
#
# This files (or updates) ONE issue the moment CI fails on main, and closes it
# when main is green again — so "is main broken?" is answerable without
# watching Actions, and the merge train never stalls unnoticed again.
#
# Fixing a red main means re-running MAIN's run (`gh run rerun <id> --failed`),
# not just your branch's. A green branch on a red base merges nothing.

on:
workflow_run:
workflows: ['CI Pipeline']
branches: [main]
types: [completed]

permissions:
issues: write
actions: read

jobs:
alert:
runs-on: ubuntu-latest
steps:
- name: File or resolve the main-red issue
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CONCLUSION: ${{ github.event.workflow_run.conclusion }}
RUN_URL: ${{ github.event.workflow_run.html_url }}
RUN_SHA: ${{ github.event.workflow_run.head_sha }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
TITLE="🔴 main is red — CI failing on main"

# A cancelled run means a newer push superseded this one (the CI
# concurrency group cancels in-progress runs). That is not a failure,
# and treating it as one is how "cancelled" gets misread as "broken".
if [ "$CONCLUSION" != "failure" ]; then
existing=$(gh issue list -R "$REPO" --state open --search "$TITLE in:title" --json number --jq '.[0].number // empty')
if [ "$CONCLUSION" = "success" ] && [ -n "$existing" ]; then
gh issue close "$existing" -R "$REPO" \
--comment "main is green again as of ${RUN_SHA:0:8} — $RUN_URL"
fi
exit 0
fi

BODY=$(printf '%s\n' \
"CI failed on \`main\` at commit \`${RUN_SHA:0:8}\`." \
"" \
"Run: $RUN_URL" \
"" \
"**Every PR branched from this commit inherits the failure**, and auto-merge refuses to merge onto a broken base — so nothing ships until this is green." \
"" \
"Fix or revert, then re-run **main's** run (\`gh run rerun <run-id> --failed\`). Re-running your own branch does not clear this." \
"" \
"This issue closes itself when a CI run on main succeeds.")

existing=$(gh issue list -R "$REPO" --state open --search "$TITLE in:title" --json number --jq '.[0].number // empty')
if [ -n "$existing" ]; then
gh issue comment "$existing" -R "$REPO" --body "$BODY"
else
gh issue create -R "$REPO" --title "$TITLE" --body "$BODY"
fi
Loading