fix(data): paging a sorted read is a partition of the result set, not five queries that share a WHERE clause (objectui#3106) - #4367
Merged
Conversation
… five queries that share a WHERE clause (objectui#3106) `ORDER BY status LIMIT 50 OFFSET 50` names a sort key that does not identify a row, and no backend promises that rows with equal keys keep the same relative arrangement between two queries. MongoDB documents this outright: `sort` + `skip`/`limit` on a non-unique key may return the same document more than once. Page 2 then repeats a row page 1 already showed and skips one nobody ever sees — with every page full, every row real, and the two halves of the symptom several screens apart. SqlDriver and MongoDBDriver append a unique tie-breaker to any non-empty `orderBy`, in the last requested key's direction: determinism holds either way, but a same-direction suffix is the one an index can still walk in a single pass. SqlDriver applies it only to objects it created itself (`initObjects` records those in `managedObjectFields`). A federated table (ADR-0015) may carry no `id` column, and guessing there would be worse than doing nothing — the resulting unknown-column error is answered by #3821's recovery ladder retrying with NO ORDER BY at all, trading a reshuffle among ties for the loss of the caller's whole sort. driver-memory needed no change: `Array#sort` is stable and the backing table's order does not move between reads. It gets a suite anyway, because that guarantee is implicit and is exactly what a refactor that looks like a speed-up (a hand-rolled sort, or sorting the array in place) would silently remove. The obligation is normative on `IDataDriver.find` and the cases are shared (`PAGINATION_CASES` in `@objectstack/spec/data`), so a future driver is held to it by a gate rather than by remembering. A paged read with NO `orderBy` is deliberately out of scope and filed as #4363. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 3 package(s): 109 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
59 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
ORDER BY status LIMIT 50 OFFSET 50names a sort key that does not identify a row. No backend promises that rows with equal keys keep the same relative arrangement between two queries — MongoDB documents the opposite:sort+skip/limiton a non-unique key may return the same document more than once.So paging a sorted list was never a partition of the collection:
Every page is full. Every row is real and belongs. The duplicate sits several screens from the omission — which is why this is found by a user counting records, never by reading a response.
How
SqlDriverandMongoDBDriverappend a unique tie-breaker to any non-emptyorderBy, in the last requested key's direction. Determinism holds either way, but a same-direction suffix is the one a compound index can still walk in a single pass.Two deliberate restrictions:
SqlDriverapplies it only to objects it created itself (initObjectsrecords those inmanagedObjectFields). A federated table (ADR-0015) may carry noidcolumn, and guessing there is worse than doing nothing: the unknown-column error is answered by fix(sharing): 共享规则新建页 — 自定义 widget 未国际化，且「接收方」永远无可选项 #3821's recovery ladder retrying with no ORDER BY at all — trading a reshuffle among ties for the loss of the caller's whole sort.driver-memoryneeded no change.Array#sortis stable and the backing table's order does not move between reads, so it already conformed. It gets a suite anyway: that guarantee is implicit, and a refactor that looks like a speed-up (a hand-rolled sort, or sorting the array in place) would remove it without failing any other test in the package.MongoDBDriver's two sort sites (findand_findStream) were identical copies; they now share onebuildSortSpec, so the tie-breaker cannot be added to one and forgotten in the other.The contract
The obligation is now normative on
IDataDriver.find, and the cases are shared —PAGINATION_CASESin@objectstack/spec/data, following theTEMPORAL_CASESprecedent. A future driver is held to this by a gate rather than by remembering it.Each driver runs the shared cases;
driver-sqlanddriver-mongodbadditionally assert the emitted clause. That second half matters here: SQLite over a twelve-row table returns ties in rowid order every time, so the property test alone would pass the day someone deletes the feature. On MongoDB the property test is the real one.Tests
driver-sql(full)driver-memory(full)driver-mongodb(full, real MongoDB viamongodb-memory-server)pnpm --filter @objectstack/spec check:generatedapi-surface.jsonregenerated for the 2 new exports)Out of scope
A paged read with no
orderByat all is non-deterministic on every backend by definition, and imposing an order on callers who asked for none changes plan selection far more broadly than this does. Filed as #4363 per Prime Directive #10 rather than folded in here.Context
This is the server-side half of objectui#3106 (column-header sort under server pagination). The client half lands in
objectui; that work makes low-cardinality columns likestatusthe most common sort key, which turns this from a latent property violation into a daily one.🤖 Generated with Claude Code