Repository navigation
Wait out Tolgee's rate limit instead of failing the import - #3
Merged
Merged
Conversation
Tolgee allows 100 translation requests per user every 5 minutes by default, so a 160k-key project (320 pages) hit 429 around page 98. The client now waits as long as Tolgee's retryAfter asks (up to 2 hours per request) instead of retrying blindly, which also added strikes that get connections dropped. The Tasks page and the project strip show the wait.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A real 160k-key import stopped at 49,000 keys with
429 rate_limited, retryAfter: 11730. Self-hosted Tolgee allows 100 translation requests per user every 5 minutes (tolgee.rate-limits.translation-request-limit), so at 500 keys per page the limit hits around page 98, and LowLex's six blind retries (up to 32 s) were not enough and also counted as strikes.retryAfterin the body, orRetry-After), up to 2 hours per request, then continues. Server errors and network errors keep the short backoff.retryAfterin the middle of the import; the import finishes without refetching pages.With Tolgee's default limits a 160k-key project takes about 16 minutes (320 pages at 100 per 5 minutes); raising or disabling Tolgee's endpoint limits makes it as fast as Tolgee can answer. Deployed to the dev server, where the stopped import can be retried from key 49,000.