Repository navigation
Continue paused and interrupted downloads where they stopped - #24
Merged
Merged
Conversation
LoadMedia lists a VOD media playlist's segments with their durations, initialization sections and discontinuities, resolving URIs against the playlist's URL after redirects. Playlists whose segments can't simply be fetched one by one return ErrUnsupported: live and encrypted ones, and those with byte ranges, gaps or variables. Neither BBC nor TVP uses these. Playlists may now be up to 16 MiB, as hours of short segments with tokenized URIs exceed the old 1 MiB, and longer ones fail instead of being cut short.
Pausing a download, restarting magnetowid or retrying a failed attempt started the download over: ffmpeg wrote the MP4 in one pass, and the queue deleted the partial file. magnetowid now fetches HLS segments itself, four at a time, into the job's folder in .incomplete, and ffmpeg only remuxes the saved files. A resumed download looks the stream up again, since its URLs expire, and skips the segments already saved. A segment appears under its final name only once it is complete. Segments are saved by position, because their URLs carry tokens that change with every lookup. The local playlists ffmpeg reads name the rendition, its bandwidth and a hash of the segments' file names, so if the site serves a different stream, the saved segments are discarded and the download starts over. A segment that fails with a server error, a rate limit, a stall or an error page is fetched up to five times. Other client errors, such as an expired token, fail the attempt, and the queue's retry resolves fresh URLs and continues. If ffmpeg finds a saved file damaged or unreadable, the segments are discarded so that retries fetch them again; other remux failures, such as a full disk, keep them. The queue keeps a job's folder through pauses, restarts and retries, and removes it when the job finishes, fails for good or is removed. Paused jobs keep their progress, shown as "Paused at 40%", and the time left counts only the current attempt's progress. Live, encrypted and non-HLS streams still go straight to ffmpeg and start over. Finishing a download needs free space for about twice the video's size.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pausing a download, restarting magnetowid or retrying a failed attempt used to start the download over. magnetowid now fetches HLS segments itself, four at a time, into
.incomplete/<job id>, and ffmpeg only remuxes the saved files into the MP4 (same language tag and+faststartas before). A resumed download looks the stream up again, since its URLs expire, and skips the segments already saved.Testing
make testandmake integrationpass.