Skip to content

Expose bounded read-only paging over KvDirectory primary rows #36

Description

@smiggleworth

Need

KvDirectory<T, TKey> stores rows under ordered primary keys but exposes direct GetAsync and secondary-index QueryAsync only. A caller cannot page existing primary rows through a bounded read-only query. In Bdgrz Compliance, a platform operator needs a paginated tenant metadata inventory from the global tenant directory. Adding a secondary index for this exact primary ordering requires a historical backfill and cutover, although the primary rows already contain the complete dataset.

KvDirectory.BackfillAsync scans those primary keys internally as a resumable write migration. Calling it from the GET path would write the full directory on every request, contend with the projector, and make pagination proportional to all tenants. That prototype is intentionally unmerged (bdgrz/compliance#238).

Proposed API

Add a bounded read-only primary-key page operation to KvDirectory<T, TKey> (for example QueryPrimaryAsync) that:

  • Uses a caller-owned read transaction and the directory's existing primary route/serialization.
  • Returns Page<T> with a stable, route-bound opaque keyset cursor and a caller-supplied limit, consistent with secondary QueryAsync.
  • Validates malformed or mismatched cursors, supports AOT, and preserves cancellation and tenant/workload isolation.
  • Proves paging across rows written before and after the API is introduced, without index backfill or request-path writes.

This would let consumers use the existing primary ordering without creating a duplicate index or a worker migration. If a primary scan cannot be exposed safely, please document the supported migration/cutover pattern and durability ordering for KvDirectory.BackfillAsync plus its completion marker.

Relevant source: src/Fitz.Extensions/KvDirectory.cs (QueryAsync, BackfillAsync, and primary route construction). No matching open or closed issue was found by searching the repository's issues for KvDirectory or primary paging.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions