Keyset cursor over a field-policy-redacted sort field encodes the redaction marker — page 2 silently returns no rows

[palermo-git]

Describe the bug

Keyset cursors are computed from the record after field policies redact it (Ash.Page.Keyset.data_with_keyset/3 reads Map.get(record, field)). When the sort field is protected by a field policy the actor fails, the cursor encodes the redaction marker instead of the sort value:

cursor |> Base.decode64!() |> :erlang.binary_to_term()
#=> [#Ash.NotLoaded<:attribute, field: :secret_score, ...>, "<pk-uuid>"]

Requesting the next page with that cursor then builds a keyset filter comparing rows against the marker term — on the ETS data layer page 2 silently returns 0 results while more rows exist (no error, pagination just ends early). On a data layer that pushes the comparison down, it surfaces as a confusing filter error carrying the %Ash.ForbiddenField{} struct. Either way a non-admin actor sorting by a field-policy-protected field cannot paginate past page 1, and the failure mode on the reference data layer is silent.

(There's also a small information-exposure angle: the cursor is term_to_binary+Base64, so whatever gets encoded — marker or value — is caller-decodable. Computing cursors from pre-redaction values would fix pagination but would leak the protected value through the cursor; rejecting a keyset sort over a field the actor can't read, or omitting the redacted field from the cursor with a loud error, avoids both.)

To Reproduce

Mix.install([{:ash, "~> 3.29"}, {:simple_sat, "~> 0.1"}])

defmodule Repro.Doc do
  use Ash.Resource,
    domain: Repro.Domain,
    data_layer: Ash.DataLayer.Ets,
    authorizers: [Ash.Policy.Authorizer]

  attributes do
    uuid_primary_key :id
    attribute :title, :string, public?: true
    attribute :secret_score, :integer, public?: true
  end

  policies do
    policy always() do
      authorize_if always()
    end
  end

  field_policies do
    field_policy :secret_score do
      authorize_if actor_attribute_equals(:admin, true)
    end

    field_policy :* do
      authorize_if always()
    end
  end

  actions do
    default_accept [:title, :secret_score]
    defaults [:create]

    read :read do
      primary? true
      pagination keyset?: true, required?: false
    end
  end
end

defmodule Repro.Domain do
  use Ash.Domain, validate_config_inclusion?: false

  resources do
    resource Repro.Doc
  end
end

for i <- 1..4 do
  Repro.Doc
  |> Ash.Changeset.for_create(:create, %{title: "t#{i}", secret_score: i * 10},
    authorize?: false
  )
  |> Ash.create!()
end

# Non-admin actor, sorted by the protected field:
page1 =
  Repro.Doc
  |> Ash.Query.sort(secret_score: :asc)
  |> Ash.read!(actor: %{admin: false}, page: [limit: 2])

cursor = List.last(page1.results).__metadata__.keyset

IO.inspect(cursor |> Base.decode64!() |> :erlang.binary_to_term())
#=> [#Ash.NotLoaded<:attribute, field: :secret_score, ...>, "<pk>"]

page2 =
  Repro.Doc
  |> Ash.Query.sort(secret_score: :asc)
  |> Ash.read!(actor: %{admin: false}, page: [limit: 2, after: cursor])

IO.inspect(length(page2.results))
#=> 0        (4 rows exist; 2 were returned on page 1 — pagination silently ends)

Observed on ash 3.29.3 / Elixir 1.19.5 / OTP 28.

Expected behavior

One of (maintainers' call):

1. A keyset-paginated read sorted by a field the actor cannot read is rejected loudly (clear error at page 1), or
2. the redacted field is excluded from the cursor with a loud error when the cursor can't be built —

rather than a cursor that encodes the redaction marker and silently terminates pagination.

Context

Found while building keyset support for an Ash data layer with fail-closed field-classification semantics; happy to help with a PR once a direction is picked.

[zachdaniel]

What interface where you using that you observed this? In general if a sort comes from user-input, you should be using Ash.Query.sort_input, which if you're exposing over a pagination interface then they sort is effectively user input. Encoding %Ash.NotLoaded{} is something I'll look into, but I don't think this class of problem exists if using the proper sort_input, which transparently includes policy information in the sort.

[nseaSeb]

I tried it — encoding nil on the sort path won't work, pagination needs the real value to seek. With sort_input the sort itself is nil for everyone so nil lines up, but plain sort still orders by the real values (non-admin gets t1,t2,t3,t4 for scores 10/20/30/40), so a nil cursor doesn't match the ordering.

I'd rather raise an error at the developer sorting on a secret column. Sorting on it leaks anyway — page 1 already comes back ordered by the hidden values, so you can deduce the ranking. WDYT?

[zachdaniel]

I'm not sure I understand what you mean yet sorry. If you use Ash.Query.sort, you're opting into this behavior. You're saying "I'm sorting by this thing, not the end-user".

[nseaSeb]

Yeah, agreed, using sort is opting in, I'm not arguing that part.

What I mean is separate from the opt-in: even when you do opt in, the actor still can't read secret_score, so the field policy redacts it on the returned rows. The cursor gets built from those redacted rows, so it holds %Ash.NotLoaded{} instead of the value — and page 2 seeks against that marker and comes back empty. Non-admin, 4 rows sorted by secret_score: page 1 is fine (2 rows, right order), page 2 returns 0.

So it's less "should this be allowed" and more "the cursor gets built after redaction, so pagination can't seek". WDYT?

[zachdaniel]

Ah. Yes okay you are right. And if it was built before redaction, then it would be a problem because it's exploding the values. Okay 🤔 yeah. We should raise an error indeed 👍

[nseaSeb]

My pleasure! Want me to submit a fix?

[zachdaniel]

Yes please 🙇