Repository navigation
perf: flip eql_v2_encrypted infix operator implementations to inlinable SQL (RFC Phase 1) #193
Description
Summary
The =, <>, ~~ (LIKE), ~~* (ILIKE), @>, and <@ operators on
eql_v2_encrypted are registered today, but their implementation
functions are either LANGUAGE plpgsql (which Postgres can never
inline) or LANGUAGE sql VOLATILE (which the planner refuses to
inline into index matching). As a result, queries like
WHERE col = val on encrypted columns silently fall back to a
sequential scan on every install where the catch-all
eql_v2.encrypted_operator_class btree isn't usable — most
prominently Supabase and any --exclude-operator-family build.
This issue tracks the smallest possible change that fixes the
problem at the EQL layer: rewrite the implementation functions as
LANGUAGE sql IMMUTABLE single-statement SELECTs. Postgres inlines
them during planning, and the resulting expressions match the
documented functional indexes (bench_text_hmac_idx,
bench_text_bloom_idx, bench_jsonb_stevec_idx).
This is Phase 1 of the broader predicate/extractor RFC
(docs/plans/uniform-predicate-extractor-pairs-rfc.md in this repo).
Phase 1 is self-contained and shippable independently — it does not
require the new _sem extractor surface or the predicate function
namespace.
Empirical confirmation
Tested against the cipherstash/stack#424 bench fixture (10k encrypted
rows in local-postgres-1):
-- Custom inlinable function:
CREATE FUNCTION eql_v2.eq_inlinable(a eql_v2_encrypted, b eql_v2_encrypted)
RETURNS boolean LANGUAGE sql IMMUTABLE PARALLEL SAFE
AS $$ SELECT eql_v2.hmac_256(a) = eql_v2.hmac_256(b) $$;
CREATE OPERATOR === (LEFTARG = eql_v2_encrypted, RIGHTARG = eql_v2_encrypted,
FUNCTION = eql_v2.eq_inlinable);
EXPLAIN (ANALYZE) SELECT id FROM bench WHERE enc_text === <encrypted_value>;
-- Result:
-- Index Scan using bench_text_hmac_idx on bench
-- Index Cond: ((eql_v2.hmac_256(enc_text))::text = (eql_v2.hmac_256(...))::text)
-- Execution Time: 0.628 msSame plan as the function-call form WHERE eql_v2.eq_inlinable(col, val).
The planner inlines the operator's SQL function during index matching;
this is the same pattern pg_trgm uses internally.
Operators in scope
| Operator | Current impl | Replace with (LANGUAGE sql IMMUTABLE PARALLEL SAFE) |
|---|---|---|
= (eql_v2_encrypted, eql_v2_encrypted) |
plpgsql, dispatches via eql_v2.compare |
SELECT eql_v2.hmac_256(a) = eql_v2.hmac_256(b) |
<> |
plpgsql | <> form of the same |
~~ (LIKE) |
sql VOLATILE | SELECT eql_v2.bloom_filter(a) @> eql_v2.bloom_filter(b) |
~~* (ILIKE) |
sql VOLATILE | same body — case sensitivity is at encryption time |
@> (jsonb containment) |
sql VOLATILE | flip to IMMUTABLE; existing body shape OK |
<@ |
sql VOLATILE | flip to IMMUTABLE |
Out of scope for Phase 1 (deferred to Phase 2 of the RFC, blocked on
cmp_sem extractor design):
<,<=,>,>=— currently VOLATILE plpgsql, no Supabase
index path defined yet.ORDER BY— same.jsonb_path_query_first/jsonb_path_exists(the function-form
variants) — needs predicate vs extractor split per the RFC.
What this fixes
- perf: encryptedSupabase emits bare-column query forms that don't engage Supabase functional indexes stack#420 — encryptedSupabase silent seq-scan.
Solved at source:?col=eq.valuefrom PostgREST inlines through
to the wrapped form and engages the hmac functional index. No
changes needed inencryptedSupabaseitself. - perf: Drizzle eq/ne/inArray emit bare equality, bypassing hmac functional indexes stack#421 + perf: legacy @cipherstash/drizzle has the same bare-equality bug as @cipherstash/stack/drizzle stack#426 — Drizzle
bare-equality bypass and the same bug in@cipherstash/drizzle.
Both are workarounds for the same root cause. After this lands,
the per-ORM wraps (perf(stack/drizzle): wrap eq/ne/inArray/notInArray in eql_v2.hmac_256(...) stack#425) become defense-in-depth
rather than required. - perf: eql_v2.like / eql_v2.ilike are VOLATILE — blocks planner inlining and bloom_filter index match #189 —
like/ilikevolatility flip. Subsumed by the
broader scope here.
What's not changed
eql_v2.encrypted_operator_classcontinues to exist. Customers with
direct-on-column btree indexes via the operator class keep working.- The function-form predicates (
eql_v2.eq(a, b),eql_v2.compare,
etc.) keep their current implementations. This is purely an
operator-implementation swap. - No data migration. Existing encrypted payloads contain the index
components the new operator implementations extract (hm,bf).
Implementation notes
- The new operator implementation functions should live alongside the
existing dispatcher functions, with names likeeql_v2.eq_inlinable
(or whatever convention fits the codebase). The existing
eql_v2.compare-based functions remain available as
eql_v2.eq/eql_v2.neqetc. for direct invocation; only the
operator binding changes. - Verify all six operators with EXPLAIN against the bench fixture
before/after the change. - Update tests in
tests/sqlx/to assert on plan shape (or use a
separate bench harness — see perf(bench): add @cipherstash/bench for index-engagement validation stack#424 for the JS-side
reference implementation).
Related
- RFC:
docs/plans/uniform-predicate-extractor-pairs-rfc.md—
Phase 1 description. - perf: encryptedSupabase emits bare-column query forms that don't engage Supabase functional indexes stack#420 — primary downstream fix target.
- perf(bench): add @cipherstash/bench for index-engagement validation stack#424 — bench fixture used to verify.
- perf(stack/drizzle): wrap eq/ne/inArray/notInArray in eql_v2.hmac_256(...) stack#425 — Drizzle hmac_256 wrap (becomes redundant).
- chore(deps): bump hono from 4.6.16 to 4.9.7 stack#187 — Supabase install bench CI (this issue's fix
should be exercised there once landed).
Activity
perf: flip eql_v2_encrypted infix operator implementations to inlinab…
feat: add eql_v2.lints() inlinability lint (#194)
perf: flip eql_v2_encrypted infix operator implementations to inlinab…
chore(splinter): allowlist new inlinable operators from #193 phase 1
perf: flip eql_v2_encrypted infix operator implementations to inlinab…
perf: flip eql_v2_encrypted infix operator implementations to inlinab…
refactor(eql_v2): collapse to single canonical SEM term per query pur…
docs: add CHANGELOG and v2.3 upgrade guide
fix(ste_vec): match by Blake3 in ste_vec_contains for b3-only elements
Merge pull request #196 from cipherstash/dan/phase-1-operator-inlining
test(lint): include int4 family in operator lint sweep
feat: add eql_v2.lints() inlinability lint (#194)
perf: flip eql_v2_encrypted infix operator implementations to inlinab…
refactor(eql_v2): collapse to single canonical SEM term per query pur…
docs: add CHANGELOG and v2.3 upgrade guide
fix(ste_vec): match by Blake3 in ste_vec_contains for b3-only elements
Merge pull request #196 from cipherstash/dan/phase-1-operator-inlining