Skip to content
This repository was archived by the owner on Jun 8, 2026. It is now read-only.

retry transient ghcr pull failures in watcher - #52

Merged
kualta merged 1 commit into
mainfrom
watcher-pull-retry
May 26, 2026
Merged

retry transient ghcr pull failures in watcher#52
kualta merged 1 commit into
mainfrom
watcher-pull-retry

Conversation

@kualta

@kualta kualta commented May 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Wrap docker pull in the embedded watcher script with a 3-attempt retry (5s, 15s backoff) so transient ghcr.io 502s and TCP timeouts no longer escalate to Telegram notifications.
  • Failure notification now fires only after all three attempts fail, and the message is prefixed with after 3 attempts: so a real persistent failure is still clearly distinguishable from the old transient pattern.
  • Each failed attempt is logged via the existing log() helper, visible through iron tail.

Test plan

  • cargo build --release, cargo fmt --check, cargo clippy -- -D warnings, cargo test all clean
  • Rendered watcher script passes sh -n POSIX syntax check
  • After merge: iron deploy (or iron check) to push the updated watch.sh to fleet servers, then iron tail --app watcher -f on the next ghcr blip to confirm WARN: pull attempt … lines appear and no Pull failed notification fires unless three consecutive attempts fail in a single tick

@kualta
kualta merged commit b8a51db into main May 26, 2026
2 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant