Auto-apply ok-to-test label for trusted bot-authored PRs

[fullsend-ai-retro]

What happened

PR #2492 was authored by fullsend-ai-coder[bot] and modified e2e test infrastructure (e2e/admin/admin_test.go, e2e/admin/testutil.go). The e2e test gate skipped execution with the message: "E2E tests did not run. [...] For other contributors, a maintainer must add the ok-to-test label after the latest push." The review agent approved the PR and applied ready-for-merge without e2e tests having run. The human reviewer approved and merged. The PR's changes — retry logic for flaky e2e tests — were never validated by actually running the e2e suite before merge. See PR #2492 and review workflow run.

What could go better

The ok-to-test gate exists to prevent untrusted external contributors from running e2e tests. However, the org's own code agent (fullsend-ai-coder[bot]) is a trusted author — its PRs originate from a controlled pipeline with pre-commit security scanning. Treating bot-authored PRs the same as external contributor PRs means e2e-modifying changes merge without e2e validation. In this case, the fix was correct and simple, so no harm was done. But a subtler bug in retry logic or test helpers could have merged undetected. Confidence: medium-high — this is a clear gap in the pipeline, though the risk is partially mitigated by human review.

Proposed change

Add automation (likely in the fullsend.yaml shim workflow or a dedicated workflow) that auto-applies the ok-to-test label when a PR is opened or updated by a trusted bot actor (e.g., fullsend-ai-coder[bot], fullsend-ai-fix[bot]). This could be a simple conditional step: if github.actor matches the trusted bot list and the label isn't already present, apply it. Alternatively, modify the e2e gate workflow to implicitly trust these actors without requiring the label. Note: issue #1424 covers the related but distinct problem of the review agent applying ready-for-merge without checking CI status — that issue should also be addressed but is already tracked.

Validation criteria

The next 3 bot-authored PRs that touch e2e/ paths should have e2e tests run automatically without manual ok-to-test label application. Verify by checking the PR checks tab for e2e job execution.

---

Generated by retro agent from #2492

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 10:58 AM UTC · Completed 11:01 AM UTC
Commit: 4e21a60 · View workflow run →

[fullsend-ai-triage]

Triage Summary

Severity: Medium · Category: Feature

The e2e gate authorization script (scripts/check-e2e-authorization.sh) checks PR author association against OWNER MEMBER COLLABORATOR, but GitHub App bot accounts like fullsend-ai-coder[bot] typically have association NONE, so they fail the trusted-author check. This means bot-authored PRs touching e2e paths require manual ok-to-test label application before e2e tests will run.

Recommended Approach

The cleanest fix is to add a trusted bot actor check directly in scripts/check-e2e-authorization.sh, alongside the existing trusted-author association check. This avoids a label roundtrip and keeps all authorization logic in one place. The bot list (e.g., fullsend-ai-coder[bot], fullsend-ai-fix[bot]) should be explicitly enumerated rather than pattern-matched to maintain security.

Related: Issue #1424 tracks the separate problem of the review agent applying ready-for-merge without checking CI status.

Proposed Test Case

# Extend scripts/check-e2e-authorization-test.sh:
# 1. PR authored by fullsend-ai-coder[bot] (association=NONE)
#    → should be authorized via trusted bot path
# 2. PR authored by unknown bot (e.g., dependabot[bot])
#    → should remain unauthorized
# 3. Trusted bot PR not touching e2e paths
#    → should still skip e2e (file-relevance filter unaffected)

---

Labels: Feature request targeting e2e gate and CI pipeline automation for trusted bot actors, filed by the retro agent.