Repository navigation
fix(proxies): start a copy the cap released even when its first start fails - #271
Merged
Merged
Conversation
…tarts A copy whose start failed when the cap lifted kept its mark and waited for the next reset, up to a month. The once-a-minute pass now retries every marked copy of an uncapped proxy, recording the failure only once. The mid-stop path marks the copy when its start back fails, so it is retried too instead of staying stopped with no mark.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 59 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #271 +/- ##
==========================================
+ Coverage 77.29% 77.34% +0.04%
==========================================
Files 81 81
Lines 12565 12565
==========================================
+ Hits 9712 9718 +6
+ Misses 2048 2044 -4
+ Partials 805 803 -2
🚀 New features to boost your workflow:
|
#268 on main already retries a cap-stopped copy every pass, under the copy lock. Keep that implementation and carry over what this branch adds to it: a retry that keeps failing is not an event every minute, and a start back that fails after the cap lifted mid-stop marks the copy so the next pass starts it. The branch's two tests run against main's code; the fake provider keeps both sides' hooks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a proxy's cap lifts (a reset, or the cap raised or turned off), each copy the cap stopped gets one start. If that start fails, for example because Docker is restarting or the proxy is not answering yet, the copy keeps its "stopped by the cap" mark and nothing tries again until the next reset, which can be a month away.
A second path had the same problem. The cap lifts while a stop is on its way and the start back fails. That copy was left stopped with no mark, so even the next reset would not start it.
The fix follows the idea of GeiserX/CashPilot#637 (retry on the next heartbeat), adapted to Desktop, which has one machine and a once-a-minute pass instead of box heartbeats:
ProxyTicknow retries every copy that still carries the cap's mark on each pass where its proxy is not capped. A copy that starts loses the mark, so it is not started again. A Start or Stop by hand clears the mark, as before.*_errorevent takes 20 points off the service's health score.docs/proxies.mdsays the app retries every minute instead of waiting for the next reset.Tests, in
internal/services/proxies_test.go:TestACappedCopyThatDidNotStartIsRetriedEveryPassandTestACopyWhoseStartBackFailedMidStopIsRetriedboth failed on main (the copy stayed stopped) and pass here.go test -race ./internal/services/ ./internal/runtime/passes.Web: #637 covers the same gap there.