Skip to content

Fix nested collection take/skip to paginate per record - #32

Open
kac-axelor wants to merge 2 commits into
axelor:devfrom
kac-axelor:nested-collection-pagination
Open

kac-axelor wants to merge 2 commits into
axelor:devfrom
kac-axelor:nested-collection-pagination

Conversation

@kac-axelor

@kac-axelor kac-axelor commented Jun 10, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #31.

A take/skip inside a collection select is meant to limit each record's
own list
— "every contact with its newest address". Instead the limit was
shared by all the records of the page combined, so some records silently came
back with incomplete or empty lists. This PR makes the nested limit mean what
callers expect, for every relation kind, including negative take ("the last
N of each list").

How

Related records are loaded with one batched query for the whole page of
parents, so a plain LIMIT there can never be per-record. Instead:

  • Nested take/skip are stripped from the batch query and enforced by a
    ranked filter: a row_number() window partitioned by the parent ranks each
    record's items by the nested ordering, and only ids inside the requested
    window survive. The filter matches (parent, item) pairs, so a shared
    many-to-many item inside one record's window cannot leak into another's.
  • The take/skip arithmetic is a single shared rule (takeSkipBounds): the
    top-level query and the in-memory fallback compute the same [start, end)
    window from it, and the ranked filter realizes the identical rule in SQL —
    a negative take anchors its window with the partition's count(*), clamped
    at the list start exactly like the in-memory rule.
  • When the nested where reaches through another collection, the multiplied
    join rows are deduplicated inside the ranked subquery (DISTINCT over the
    (parent, item, order values) tuples) — valid whenever the ordering refers
    only to the item's own columns. Only an orderBy that itself traverses a
    collection (per-item ambiguous) falls back to fetching the filtered lists
    and slicing per record in memory. Multi-valued join detection walks the
    whole join chain rather than only the first level.
  • A page with a single parent record (findOne, and the GraphQL nested
    connection resolver) keeps the full top-level semantics for its collections
    — nested cursor pagination and _count/_cursor metadata included —
    since the batch is exactly that record's list.
  • Any take/skip now gets the unique-ordering tiebreaker from the parser
    (previously only page queries did), so limited windows are deterministic
    at every nesting level even without an explicit orderBy.

Behavior notes

  • On multi-record pages a nested cursor is ignored when nested
    take/skip is present, and nested items carry no _count/_cursor
    metadata; single-record pages keep both (relay connections rely on this).
  • _hasPrev/_hasNext are now cursor-aware: the old count-based formula
    ignored the cursor filter and could report hasNext at the end of a list.
  • A nested take: null from untyped callers is treated as "no limit" instead
    of producing an empty window.
  • README documents nested pagination, including negative take.

Tests

17 e2e tests in the nested-pagination suite — per-record take / skip /
negative take (including window clamping past the list start, on both query
paths) / ordering / default id ordering (with physical row order shuffled to
prove it), filters through single-valued and collection relations, ordering
through a collection, two-level pagination, shared many-to-many adversarial
cases, take: null, and single-record page metadata with nested cursor
paging — plus a GraphQL relay round-trip (edge cursors, totalCount,
hasNextPage, paging with after). Full e2e suite green.


Stacked on #24 — this branch builds on orderby-null-sorting and consumes
its ordering primitives; until #24 merges the diff shows its commits too.
Review only the last commit (9235775). Will rebase onto dev after #24
lands.

Extend OrderBy with ASC_NULLS_FIRST/LAST and DESC_NULLS_FIRST/LAST,
exposed through the GraphQL OrderBy enum and translated to SQL
NULLS FIRST/LAST.

Fix cursor pagination over nullable columns:
- Tie-break uses IS NOT DISTINCT FROM so multi-key cursors do not
  drop rows when a key is NULL.
- Keyset predicate includes trailing NULLs when they sit after the
  cursor, and handles a NULL cursor value explicitly.
- Plain ASC/DESC fall back to the Postgres default (ASC -> NULLS LAST,
  DESC -> NULLS FIRST); backward pagination flips null position with
  direction.
A take/skip inside a collection select was applied as a LIMIT on the
single batched query that loads the collection for the whole page of
parents, capping the combined batch instead of each record's list.

Nested take/skip are now stripped from the batch query and applied per
record: a row_number() window partitioned by the parent filters the ids
(matched as (parent, item) pairs so shared many-to-many items cannot
leak across windows); when the nested where/order traverses another
collection, the filtered lists are sliced in memory instead, since the
multiplied rows would corrupt the ranking. Negative take selects from
the end of each list, mirroring the top-level semantics via the
inverted NULL-aware ordering. A nested cursor is dropped alongside, and
multi-valued join detection now walks the whole join chain.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant