feat: Flickr rate limit retry with backoff#84
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> ci: add tests workflow and update ty check command in CLAUDE.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements a retry mechanism for handling HTTP 429 (Too Many Requests) errors in the FlickrBot using a backoff strategy. It also introduces the RateLimitExhausted exception, refactors DryRunStop to be public, updates project dependencies to include pytest, and adds comprehensive unit tests for the new rate-limiting logic. The review feedback suggests improving the retry logic by checking for the Retry-After header and expanding retry coverage to include other transient errors like connection timeouts or 500-series server responses.
| except httpx.HTTPStatusError as e: | ||
| if e.response.status_code != 429: | ||
| logger.error(f"[{flickr_photo_id}] {e}") | ||
| return | ||
| delay = next(delays, None) | ||
| if delay is None: | ||
| logger.critical(f"[{flickr_photo_id}] Rate limit exhausted after all retries") | ||
| raise RateLimitExhausted | ||
| logger.warning(f"[{flickr_photo_id}] Rate limited, retrying in {delay}s") | ||
| time.sleep(delay) |
There was a problem hiding this comment.
The retry logic for HTTP 429 (Too Many Requests) uses hardcoded delays. While this is a good start, the Flickr API typically returns a Retry-After header indicating how long the client should wait. It would be more efficient to respect this header if present, falling back to the hardcoded delays only if the header is missing.
| except Exception as e: | ||
| logger.error(f"[{flickr_photo_id}] {e}") | ||
| return |
There was a problem hiding this comment.
This catch-all Exception block silently skips the photo on any error that isn't a 429, 404, or 403. While logging the error is good, consider if certain other exceptions (like connection timeouts or 5xx server errors) should also trigger a retry with backoff, as they are often transient.
1 participant
When the Flickr API returns HTTP 429,
get_flickr_photonow retries with delays of 1, 3, and 5 minutes before giving up. On exhaustion it raisesRateLimitExhaustedwhich stops the run loop cleanly (same pattern asDryRunStop).Also sets up pytest and adds a Tests GitHub Actions workflow, and fixes the
ty checkcommand to targetsrc/only.