Repository navigation
Fix/traewelling full sweep - #231
Merged
Merged
Conversation
A check-in that never reached the inbox stayed unreachable once newer ones had
been imported. The sweep walks /user/{username}/statuses newest-first and breaks
on the first page where nothing is new, and "new" is measured against everything
known: imported, ignored, and already pending. Once the newest page is entirely
known - the steady state with live sync on - the walk ends at page 1, and
anything missed that has scrolled past it is never asked for again. The refresh
button did not help: it only bypassed the one-hour staleness check, not the
break. Nor did paging the list, which pages the local inbox rather than upstream.
So the sweep gains a mode. WhenStale and Force keep the break and now read up to
ten pages instead of five; Full ignores it and follows links.next to the end,
behind a new history button next to refresh and POST /api/traewelling/resync.
Its page limit is a safety valve at 1000, not a depth to tune - a rerun starts at
page 1 again, so there is nothing to page past - and the result says whether the
walk ran out of pages or stopped short, which the frontend passes on.
Two things make a long walk survivable. Each page saves before the next is asked
for, so a walk cut off midway still leaves the inbox fuller than it was, and the
endpoint deliberately does not pass the request's cancellation token: a proxy
timeout or an impatient browser should not throw away half an hour of paging.
The delete-healing needed no change. It is bounded by the oldest departure the
sweep actually saw, and Traewelling orders this listing by departure descending
(statusesForUser: orderByDesc on train_checkins.departure, 15 per page), so the
bound covers exactly the pages that were read whether the walk was cut short or
not.
The tests pin the behaviour that was wrong: against a fixture whose first page is
entirely known and whose second holds an older missed check-in, Force stops after
one request and finds nothing, Full reads both and picks it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Walking a whole history takes minutes, well past any reverse proxy's read timeout. Awaiting it in the request meant the browser got a 504 and a generic error toast for a sweep that was in fact still running and still saving pages - and the natural response to that toast is to press the button again. So the endpoint only queues now and answers 202. TraewellingResyncService does the walking off a channel, one walk per user at a time, and the page hears about it over the hub it already had open for webhook events: ResyncProgress every five pages, ResyncFinished with the totals. The page connects to the hub whenever the account is connected rather than only with live sync on, since without it there would be nothing to hear the resync on. GET /status gained resyncRunning, so a page opened or reloaded midway shows the walk rather than an idle button, and its finish still arrives over the hub. Progress comes out of the sweep as IProgress, reported after each page. Neither end uses Progress<T>: it posts reports to the thread pool, which reorders them and races the counter the service throttles on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.