test: harden 3 load-induced release-gate flakes (solid fixes)#229
Merged
Conversation
- common.sh: add api_get_with_retry + create_test_user_with_retry — bounded retry-with-backoff on TRANSIENT class only (5xx/000); 4xx never retried so real client/auth errors are not masked. - test-sbt.sh: artifacts GET uses api_get_with_retry (idempotent GET that hit a transient 5xx under fleet worker-starvation). - test-regression-security.sh: 3 setup user-creates use create_test_user_with_retry. - test-scan-dedup-short-circuit-1373.sh: assert exactly-1-completed-row PER scan_type (backend registers dependency+grype scanners) and stable scan_id per type across re-trigger — STRONGER + deterministic vs the old total-count==1. (Test serializes its own triggers; see filed backend race for the concurrent case.) - test-webhook-retry-recover.sh: isolate measured deliveries by repo entity_id so concurrent suites' sibling deliveries to a global webhook no longer contaminate the timing (root cause of attempt_delta=0s); oldest-first; windows account for the 30s scheduler tick + 20% jitter; per-PID receiver port.
brandonrc
added a commit
that referenced
this pull request
Jun 21, 2026
…fits the 300s gate timeout (#230) The #229 rewrite correctly fixed the deltas=0s contamination (entity_id isolation) but set EXPECT_ATTEMPTS=3, which waits for backoff 30+60+120s + 30s ticks + jitter (~330s) and overruns the run-suite per-test TEST_TIMEOUT=300s (exit 124). Default EXPECT_ATTEMPTS to 1: reject attempt 1, recover on attempt 2, validate the first real backoff interval (~30s) and the recover path — fits well inside 300s and stays deterministic. Full 3-interval ladder still available via EXPECT_ATTEMPTS=3 + larger timeouts. Keeps the entity_id isolation fix intact. Co-authored-by: brandonrc <brandonrc@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the 3 flaky release-gate suites SOLID (not retry-roulette). All load-induced flakes from concurrent gate execution; root-caused at the source.
Verified locally under concurrent load where feasible (security 9/9 under load); webhook timing logic validated on synthetic logs (live delivery needs the kind pod network).