Skip to content

feat(download): make the torrent stall window a setting - #1426

Open
splitsec2 wants to merge 5 commits into
calibrain:mainfrom
splitsec2:feat/torrent-stall-timeout-setting
Open

splitsec2 wants to merge 5 commits into
calibrain:mainfrom
splitsec2:feat/torrent-stall-timeout-setting

Conversation

@splitsec2

Copy link
Copy Markdown
Contributor

#1420 gave a torrent 15 minutes to get going and 15 minutes after each step forward. That suits my swarms, but other people will want it shorter or longer, so this makes the number a setting.

TORRENT_STALL_TIMEOUT_MINUTES sits in Download Clients with the other torrent options and takes 5 to 60. The default is 15, so nothing changes unless someone sets it. 5 is the orchestrator's own stall timeout, so choosing it asks for no extra time and gives the old behaviour. The value can arrive as text from an env var, so anything that isn't a number falls back to the default and anything out of range is clamped.

The orchestrator caps one activity grace at 16 minutes, so I raised that to an hour to fit the top of the range. As far as I can tell the protection bypass is the only caller that asks for more than a few minutes, but I haven't audited every caller. If you'd rather keep the cap, I can lower the maximum to 16 minutes instead.

Tests cover the default, 5 and 30, text values, out of range values and NaN, the start and movement windows following the setting, and the setting showing up in the Download Clients tab. I broke each rule on purpose to check a test catches it. The full suite fails the same tests with and without this change (the bypass and seleniumbase tests don't run in my environment).

calibrain#1420 gave a torrent 15 minutes to start and 15 minutes after each step
forward. That suits a slow swarm, but someone else will want less or more.
TORRENT_STALL_TIMEOUT_MINUTES (Download Clients, 5 to 60) sets it.

The default is 15, so nothing changes. Five is the orchestrator's own stall
timeout, so choosing it asks for nothing extra. The orchestrator's cap on a
single activity grace goes from 16 minutes to an hour to leave room for the
maximum.
grace_requested_at started at 0.0 and was compared against time.monotonic().
On a host that booted less than 10 minutes ago the first queued poll saw
now - 0.0 < 600 and sent no grace. Start from None instead, and make the
queue grace test run on a small clock so it no longer depends on uptime.
…the queue

Leaving the client's queue releases the grace, which puts the stall deadline
back to the orchestrator's 5 minutes. A torrent that had already used its start
window, or was moving before the client requeued it, got no new window until its
progress next went up. Reset the start window on release so the configured
window applies again.
…rator timeout

STALL_TIMEOUT_MIN_MINUTES was a second copy of 300 seconds. Put the number in
download/activity.py, which has no imports, and have both the orchestrator and the
settings module read it.
…he real one

The stall setting is documented for torrents, but _poll_and_complete asked for
the window on every protocol, so usenet downloads got it too. Only torrents ask
for it now.

The stall message always said 300s. Remember the size of the last grace a task
asked for and report that, falling back to the default once it is released.
@splitsec2

Copy link
Copy Markdown
Contributor Author

I pushed four small follow-up commits to this branch after going back over it.

  • The first queue grace was skipped on a host that had been up for under 10 minutes. grace_requested_at started at 0.0 and was compared against time.monotonic(). This is what made test_a_queued_torrent_asks_for_a_grace_once fail in CI, so it now starts as None and the test runs on a small fake clock.
  • Leaving the client's queue dropped a torrent back to the orchestrator's 5 minute window regardless of the setting. The start window now resets when the grace is released.
  • The 5 minute value is one constant (STALL_TIMEOUT_SECONDS) instead of two copies of 300.
  • Only torrents ask for the setting's window now, since I assume usenet picked it up by accident with fix(download): do not cancel a queued torrent as stalled #1420, and the cancel message reports the real window instead of always saying 300s. The torrent-only part is a judgment call, so it's its own commit if you'd rather keep usenet at 15 minutes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant