Repository navigation
Stop saving a trip rebuilding every route's geometry - #232
Merged
Merged
Conversation
Saving a trip imported from Traewelling took seconds before the list came back with its delay display, and almost none of it was the save. The station suggestions the import computes go through the matcher, whose index was one object fingerprinted on the route count *and* the station count. Importing onto a new route bumps the route count, so the very next save found the index stale and rebuilt it - streaming all 12,809 routes' geometry, 382 MB and 9.9M coordinates, through Douglas-Peucker, inside the request. The lookup that triggered it only ever reads the station tree. So the index is now two, counted and cached apart: matching an operator's calling pattern builds the station points and never touches route geometry. Finding a trip's stations reads its endpoints off the geometry already on the stack rather than asking the route index for them, which leaves FindStationTripsAsync as the only caller that builds the expensive half. Three more costs on the same path: - Matching stops asked the database whether the names agreed once per stop, and a calling pattern runs to dozens of stops. Geometry first, then one query for the names. - The save loaded the route with every trip it has ever had, joined across properties and maps, to find the one trip being edited - repeating the route's geometry column on every row. It now checks ownership, loads that one trip, and reads the geometry only when a duration needs computing. - The reload the screen waits on had the same joined shape. Split queries, so the geometry is fetched once instead of once per row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012EGxqy7LB2FpCSns4JiNgZ
Opening the dating queue used to fire the first station request straight into a cold route index. The build reads every route's geometry and takes seconds, so the page sat on an unexplained spinner, and the request was long enough to be at a proxy's mercy. Opening the page is the moment we know the build is coming, so it is started there instead: POST /api/stationbackfill/prepare answers whether the index is warm and, if not, queues the build on MatcherWarmupService. The page waits on StationsHub rather than on a blocked request, showing a determinate bar - routes indexed out of routes to index - because the builder streams rows and the count is known up front. The queue/hosted-service/hub arrangement mirrors the Traewelling resync, which solves the same problem: an endpoint that can only queue, and the work reporting to the page it outlived. Details worth knowing: - The page subscribes before it asks, because asking is what starts the build; connecting after would lose the event of a build that finished in between, and it would then wait out the watchdog for something already done. - Starting the queue is a one-shot claim, so the finished event, the watchdog and a ready answer cannot each load the first station. - Preparing is an optimisation, never a gate: any failure falls through to the request that builds the index itself, which is what this page did before. - IMatcherIndexCache.HasRoutes answers outside the gate, so asking whether a build is needed cannot block behind the build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012EGxqy7LB2FpCSns4JiNgZ
…fetch The save is committed before suggestions are computed, so the one upstream call that follows it was holding the response open for something the user had already got. Rate limited, that call sleeps its whole window - up to a minute of a screen that will not finish. It is now bounded twice over. A limiter that is already blocked is not worth trying: the call would sleep the window out before it left, and we know that up front. Otherwise the fetch gets five seconds, cancelled through the same token the limiter's delay and the HTTP call already honour. What that costs is the suggestions for that trip, and they do not come back - they are computed at import and never stored. So the result now says which happened: no stations because there were none to offer, or because the pattern could not be read. The page says so in a snackbar with no action button, which clears itself; there is nothing to act on, so there is nothing to dismiss. Also moved the inbox-row delete ahead of the fetch. With the request's own token now reaching that call, a client that goes away mid-fetch would have skipped the delete and left an imported trip sitting in the inbox looking unimported. The trip is saved either way, so the row stops being pending either way. GetTripStopoversAsync passes cancellation on rather than logging it as a failed fetch - giving up is a decision, not a fault. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012EGxqy7LB2FpCSns4JiNgZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Saving a RouteInstance imported from Träwelling took seconds before the list came back with its delay display. Almost none of it was the save.
What was happening
PUT /api/routes/instancescommits the trip and then, because an import carries aTraewellingStatusId, computes station suggestions before it answers. The page only repaints after that answer, so everything below sat between the click and the delay display.The dominant cost: the matcher's spatial index was one object fingerprinted on the route count and the station count. Importing a trip onto a new route bumps the route count, so the very next save found the index stale and rebuilt it — streaming every route's geometry out of the database (12,809 routes, 9.9M coordinates, 382 MB, per the measurements already recorded in
MatcherIndex.cs) and running Douglas-Peucker over it, inside the request.MatchStopsAsync, the lookup that triggered this, reads only the station tree and never touches a route segment.The 30-minute idle timeout meant the first import of any session paid it too.
Changes
461530b— split the index. Route segments and station points are now counted and cached separately, so matching a calling pattern builds only the station tree.FindStationsForTripAsynctakes the route's endpoints off the geometry already on its stack instead of asking the route index, which leavesFindStationTripsAsync(the visit-dating and backfill lookups) as the only caller that builds the expensive half.Three smaller costs on the same path:
MatchStopsAsyncasked the database whether names agreed once per stop, and a calling pattern runs to dozens of stops. Geometry pass first, then one batched query — which also replaces the separate name lookup it did afterwards.AsSplitQuery()on both instance reads.c2355ad— build the index before the backfill queue needs it. Opening the dating queue used to fire the first station request into a cold index: an unexplained spinner, and a request long enough to be at a proxy's mercy.POST /api/stationbackfill/preparenow answers whether the index is warm and otherwise queues the build onMatcherWarmupService, with the page waiting onStationsHub. Because the builder streams rows and the count is known up front, the page shows a determinate bar rather than a spinner.The queue / hosted-service / hub arrangement mirrors the Träwelling resync, which solves the same problem. Two details worth knowing: the page subscribes before it asks, because asking is what starts the build and connecting after would lose the event of a build that finished in between; and starting the queue is a one-shot claim, so the finished event, the watchdog and a ready answer cannot each load the first station. Preparing is an optimisation, never a gate — any failure falls through to the request that builds the index itself.
511d803— bound the Träwelling call. The one upstream call after the save was holding the response open for something the user had already got, and rate limited it sleeps its whole window — up to a minute. It is now skipped outright when the limiter is already blocked, and given five seconds otherwise, cancelled through the token the limiter's delay and the HTTP call already honour.That costs the suggestions for that trip, and they do not come back — they are computed at import and never stored. So the result distinguishes "nothing to offer" from "could not ask", and the page says so in a snackbar with no action button, which clears itself.
The inbox-row delete moved ahead of the fetch: with the request's own token now reaching that call, a client going away mid-fetch would have skipped the delete and left an imported trip sitting in the inbox looking unimported.
Verification
CI green on each commit — backend builds clean and 248 tests pass (243 before; seven added across the three commits). The production frontend build was also run locally against Node 24, so
strictTemplatescovers the new component and template code.Not verified: actual timings. This environment has no production data and no .NET SDK, so the size of the improvement is reasoned from the code and the measurements already recorded in it, not from a stopwatch. The mechanism is certain; the magnitude is inference.
Known gaps, left deliberately
GET /api/station/{id}/candidates(the single-station visit dialog) still hits a cold build unannounced. Rarer path — one station at a time rather than a queue — so it was scoped out.Routes.LineString. Precomputing route↔station proximity into a join table, the wayRouteRegionsServicealready derivesRoute.Regionsfrom geometry, would remove it entirely. Larger change, not attempted here.🤖 Generated with Claude Code
https://claude.ai/code/session_012EGxqy7LB2FpCSns4JiNgZ
Generated by Claude Code