Restore import data-loss fixes that never landed on master - #75
Merged
Conversation
Four separate defects found by grouping scrobblify_error events in PostHog and tracing each back through the import path. 1. A track whose Last.fm entry has a missing, empty or non-numeric duration was silently thrown away. parseInt returned NaN, and because every comparison against NaN is false it slipped past the "pretend it is 2 minutes" fallback *and* the minimum-length check, only to fail msListened > NaN at the end. The stated intent of that code is to err on the side of giving the user the scrobble. 2. Track durations were looked up once per *play* rather than once per distinct track, each with 250ms of enforced rate buffer. A listening history is mostly repeat plays, so this multiplied validation time by the user's average play count. Now cached per (artist, track), including permanent "not found" answers; rate limits and network errors stay uncached. 3. user.getrecenttracks was sent fractional timestamps (from=1784563202.848). Last.fm documents UNIX timestamps. Same class of bug as the one already fixed for track.scrobble. Window ends now round outwards, so a duplicate check can only ever look too far, never miss. 4. One unreadable or malformed history file abandoned the entire import. Large exports genuinely do come back truncated or corrupted on memory-constrained mobile browsers, seen across three users. Damaged files are now skipped with a visible warning and the rest is imported; only a ZIP where every file fails is an error. Each of the five new tests was confirmed to fail with the source changes stashed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f0df23dc-ed73-4b56-9b7c-4d2846ddf2a2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR #73 was merged into a branch that had already been merged, so its fixes never reached master. This restores them.
fix/import-data-losstargetedfix/scrobble-timestamp-collapse(PR #72). #72 merged to master first, capturing its branch head atcce7b82. #73 then merged into that now-stale branch, producing589e9ea— which is not an ancestor of master:Both PRs show as merged, so this is easy to miss. That is the failure mode of stacking, and it was my call to stack them — apologies.
This branch cherry-picks
874dba3onto master. The result is byte-identical to the stranded merged state:Content is unchanged from what was reviewed and approved in #73: NaN durations no longer discarding plays, per-track duration caching, integer timestamps, and one bad history file no longer aborting the whole import.