The Tests workflow has an e2e job that can't currently pass, but it shows green on every run, and ci-success (the check our main/dev ruleset requires) depends on it. So a broken e2e never blocks a merge, and nobody notices it's broken.
Evidence
Run https://github.com/peanutprotocol/peanut-ui/actions/runs/27425406519/job/81061724408. The "Run E2E tests" step logs TEST_HARNESS_SECRET is not set and exits 1, but the step and the job both report success.
Why it's green
- In
.github/workflows/tests.yml the step has continue-on-error: true, so the job passes no matter what happens.
- The step never passes
TEST_HARNESS_SECRET (or API_BASE_URL / UI_BASE_URL) into the env, and nothing starts the API or UI, so e2e/global-setup.ts throws at getHarnessSecret() before any test runs. It cannot pass as written.
e2e is listed in ci-success.needs, and the ruleset "Require ci-success on api + ui (main, dev)" makes ci-success the required check. So the required gate is satisfied by a job that is guaranteed green.
- The
report job lists e2e in needs but only parses the unit JUnit, so its "all green" summary never reflects e2e anyway.
Options
- Wire the secret and stand up the API/UI, then remove
continue-on-error so real failures block.
- Or, until it's wired, skip the job cleanly with an
if: guard and take it out of ci-success.needs (the way eslint is already handled), so we stop claiming a passing check that never actually ran.
Either way, stop counting a can't-fail job as a required check.
The Tests workflow has an
e2ejob that can't currently pass, but it shows green on every run, andci-success(the check our main/dev ruleset requires) depends on it. So a broken e2e never blocks a merge, and nobody notices it's broken.Evidence
Run https://github.com/peanutprotocol/peanut-ui/actions/runs/27425406519/job/81061724408. The "Run E2E tests" step logs
TEST_HARNESS_SECRET is not setand exits 1, but the step and the job both report success.Why it's green
.github/workflows/tests.ymlthe step hascontinue-on-error: true, so the job passes no matter what happens.TEST_HARNESS_SECRET(orAPI_BASE_URL/UI_BASE_URL) into the env, and nothing starts the API or UI, soe2e/global-setup.tsthrows atgetHarnessSecret()before any test runs. It cannot pass as written.e2eis listed inci-success.needs, and the ruleset "Require ci-success on api + ui (main, dev)" makesci-successthe required check. So the required gate is satisfied by a job that is guaranteed green.reportjob listse2einneedsbut only parses the unit JUnit, so its "all green" summary never reflects e2e anyway.Options
continue-on-errorso real failures block.if:guard and take it out ofci-success.needs(the wayeslintis already handled), so we stop claiming a passing check that never actually ran.Either way, stop counting a can't-fail job as a required check.