Skip to content

fix: log WARN on transient Railway token-refresh failures and continue with stored token #SUPERLOG - #531

Open
superlog-app[bot] wants to merge 1 commit into
mainfrom
superlog/railway-token-refresh-warn-retry-991474e0
Open

superlog-app[bot] wants to merge 1 commit into
mainfrom
superlog/railway-token-refresh-warn-retry-991474e0

Conversation

@superlog-app

@superlog-app superlog-app Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

When Railway's OAuth token endpoint returns a transient non-JSON response (invalid_response) or a server_error, the worker was logging ERROR and skipping the entire installation for that pass — even though the stored access token is still valid for up to 5 more minutes (refresh fires when expiresAt - now() < 5 min).

This is a third recurrence of the same defect (incidents yeasty-skunk, wandering-porcupine, and now quiet-walrus). Two previous fix PRs (#440, #471) prescribed the same change but were both closed without merging.

Root cause

In apps/worker/src/railway/puller.ts, the token-refresh failure path called deps.log.error(...) and return'd unconditionally, discarding the remaining valid-token window:

// Before
if (!refreshed.ok) {
  stats.errors += 1;
  deps.log.error({ installation_id, error: refreshed.error }, "railway token refresh failed");
  return;   // ← skips even when token still valid
}
accessToken = refreshed.accessToken;
await deps.store.saveTokens(...);

Remediation

  • Log WARN (not ERROR) on a transient refresh failure — Railway's token endpoint sometimes returns non-JSON bodies under load.
  • Only skip the installation if expiresAt <= now() (token is truly expired); otherwise continue the pass with the stored access token.
  • Move accessToken = refreshed.accessToken and saveTokens(...) into the else branch so they only run on success.
  • Two new tests cover both branches: (a) transient failure, token still valid → continues and logs WARN; (b) failure with expired token → skips.

All 8 tests pass: pnpm tsx --test apps/worker/src/railway/puller.test.ts

Fixes incident quiet-walrus (22394ecc-55dd-4554-af1b-d51d7689b80b).


Was this PR helpful? Leave feedback — goes straight to the Superlog team.


Summary by cubic

Fixes the Railway puller skipping installations when a token refresh fails transiently. Previously any refresh failure logged ERROR and aborted the pass, even though the stored access token is valid for up to 5 more minutes. The puller now logs WARN and continues with the stored token when it hasn't expired. This is the third occurrence of this defect, after two earlier fix PRs were closed without merging.

  • Only skips the installation when expiresAt has passed; token assignment and saveTokens now run only on successful refresh.
  • Adds tests for both the still-valid and truly-expired paths.

Written for commit 63ad3af. Summary will update on new commits.

View guided diff Turn on auto-fix

…e with stored token #SUPERLOG

Delivery-Id: 991474e0af8c48e457b92283fcd6ae37a5c8556eec5e2be359902ebfbcb38ebe
Delivery-Base: main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant