Repository navigation
fix(live-alerts): retry a failed Twitch/Kick poll once and log a run of failures once - #374
Conversation
…of failures once
A single network blip ("fetch failed") no longer logs an error. A failed
request is retried once after a short delay for network errors, 5xx and 429
(honouring Retry-After). Failed polls are counted per platform: one warning
after 3 in a row, naming the cause code (e.g. ECONNRESET), and one line when
polling works again. Programming errors and 4xx answers are still logged as
errors.
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; no formal proof of the changed code.
Graphify review — findings
Adds a shared pollErrors helper so transient Twitch and Kick poll failures stop spamming error logs. withRetry repeats a request once after a short delay (3s by default) on network errors, 5xx and 429, honouring Retry-After up to 15s. Other 4xx answers are not retried and fail immediately. createPollHealth replaces the per-poll log.error in both pollers: transient failures log a single warning after 3 consecutive failed polls and one more when polling recovers. Non-transient errors, such as bad credentials or a bug in our own code, are still logged as errors on every poll. Failure messages now come from httpError, which carries the status and Retry-After, and describeError adds the underlying cause code (e.g. ECONNRESET) to Node's generic "fetch failed" and strips URLs so credentials stay out of the logs. Kick's tick is now exported for the new tests.
No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 55 functions depend on the 50 functions this change touches.
Health — grade A; 2 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):
tick()— 1 callers, 18 callees (high)tick()— 1 callers, 16 callees (high)
Verification — 55 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 55 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
4 of 673 test file(s) selected (1%) via static blast radius.
test/configUndo.test.js— impacttest/kickAlerts.test.js— impacttest/liveAlertsPollErrors.test.js— impact, changed-testtest/twitchAlerts.test.js— impact
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Return the tick promise to the scheduler so a slow tick is not started twice. Read Ratelimit-Reset only for a 429. Count only the connection and timeout codes of undici as network errors. Refresh a rejected token and retry once at once. Log an error that is not transient the first time, when it changes and then once an hour, with its stack, and remind hourly during a long network outage. Mark token endpoint errors so a refused token is never read as a rejected value.
… the error throttle by kind
A TypeError('fetch failed') with a cause is now a network error only when the
cause code is a connection or timeout code; UND_ERR_INVALID_ARG and similar
stay errors. The hourly throttle for errors that are not transient keys on
name, status and the message without numbers, so alternating errors or a
changing value in the message are not logged on every poll.
What
Production 5.2.0 logs
Twitch alerts — poll failed: fetch failed(and the same for Kick) every now and then. "fetch failed" is Node's generic network error, a single transient blip, and each one is alog.error.Retry-Afteris honoured, up to 15 s). 4xx answers other than 429 are not retried.log.warnafter 3 in a row and onelog.warnwhen polling works again. The line names the cause code fromerr.cause, e.g.fetch failed (ECONNRESET). It holds no URL or credential.log.errorstays for what needs a fix: programming errors (aTypeErrorin our code) and 4xx answers such as bad credentials.The shared part is
src/modules/lib/pollErrors.js(classification,withRetry,createPollHealth), used bytwitchAlerts.jsandkickAlerts.js.tick()is exported from both for the tests, like on the 5.3 branch.Tests
test/liveAlertsPollErrors.test.js, for both Twitch and Kick: a blip is retried silently and the stream is announced; 3 failed polls give one warning with the cause code, none while it stays down, one recovery line; 503 and 429 (Retry-After) are retried; a programming error and a 403 are logged as errors and not retried. Plus unit tests of the helper.Notes
Review round 2
every()callbacks now return the tick promise, so the scheduler's no-overlap guard applies (a tick that waits on a retry is not started twice).UND_ERR_*codes: only the connection and timeout ones count as network errors.Ratelimit-Resetis only read for a 429.tokenError).