Enrich Music with Metadata#82
Open
alexanderadam wants to merge 22 commits into
Open
Conversation
problem was: open "my playlists", click any list, get items from a totally different one. happens because mine=true was kept in the per-item passthrough. when LMS then asked playlistItems.list for one specific playlist, mine=true was still attached and YT returned whatever first matched the auth context, not the requested playlistId. mine is only valid on playlists.list. keep it there, drop it from the per-item bag, and also delete it again in playlistHandler so we cant trip on it later.
EBML element bigger than MAX_EBML used to return WEBM_ERROR. callers ignore the return value so playback just stops a few seconds in. people see it as "song cuts off early". new behaviour: scan the buffer for the next CLUSTER id and pick up there. if no cluster boundary is in the buffer yet, drop most of the buffer and wait for more bytes. we might lose a fragment but the song keeps going, which is what users want.
prep work. next commits filter My Playlists to only music ones, like the youtube android app does.
probe the first item of each unknown playlist, check its videoCategoryId, and only keep category 10 (Music). per-playlist decision goes into the cache so navigating back is free. if the video lookup fails (eg. bad api key or quota) we show all playlists. better than silently hiding everything.
getMetadataFor runs on every UI poll. for each uncached track in the playlist we collected ids and called the data api. when the call failed (bad key, quota, private video, network blip, whatever) nothing was cached, so the next poll did the same thing again. a long playlist turns this into a request storm that drags the whole server down. cache a tiny fail marker per id. 5 min for hard errors, 1 day for ids the api never returns (most likely deleted or private). next poll skips them so we stop hammering.
only checking the first item was too aggressive. workout playlists with a non-music intro got hidden. now we treat a playlist as music if any of the first 5 items is in category 10. also skip _pagedCall for the small sample fetch, and bump the cache key so old single-item decisions dont stick around.
without the token, playlistItems.list gets 404 playlistNotFound for any non-public playlist. only the public ones (EDM, Konzentration, Lounge) survived. everything else got treated as non-music. bump the cache key again, old broken decisions still sit there for an hour.
up to 50 playlists times 5 sampled ids each = 250 video ids. one big videos.list call hits the api 50-id cap and 400's. the whole batch fails so we fall back to "show all playlists", and Kochen, 3D Druck etc come back into the menu. split into chunks of 50, fire them in parallel, merge categories from each chunk. only treat as full failure if every chunk fails. also add a small log line so this is easier to verify next time.
OPML level-2 only fetches one row at a given index. with the filter on we pulled the first 50 raw playlists, filtered down to 18, then indexed into that. position 12 ended up pointing at a different playlist than the menu showed, so Workout opened raining tracks. position 20 fell off the end, so 💔 looked empty. now we ask for max_items upstream whenever the filter is on, so the filtered set is stable no matter which index OPML is asking for.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For things that are declared as music (10) we can do another call to get the artist and such.
It works for me.
The Regex can work wonders in some cases as the fallback.
However, the feature needs the $requestedIds parameter and the yt:meta-fail-$id and it also adds the rich_metadata so it's based on #79 and #81.
The key and the parsing logic is stolen from ytmusicapi.