Skip to content

Add sql_query data type for pre-analyzed SQL query trees - #1

Open
dimitri wants to merge 1 commit into
masterfrom
sql_query_type
Open

Add sql_query data type for pre-analyzed SQL query trees#1
dimitri wants to merge 1 commit into
masterfrom
sql_query_type

Conversation

@dimitri

@dimitri dimitri commented Jun 25, 2026

Copy link
Copy Markdown
Owner

This introduces a new built-in type, sql_query (OID 8794), that stores
a fully analyzed SQL query. The input function runs SQL text through the
standard parse and analyze pipeline, resolving all object names to OIDs
at assignment time. The output function deparses the stored Query node
using pg_get_querydef(), which is search_path-sensitive and emits
schema-qualified names when needed.

Background

Since commit 2af07e2 (PG17), REFRESH MATERIALIZED VIEW runs under a
restricted search_path of pg_catalog, pg_temp, so user tables are
unreachable by unqualified name. Since commit 4b74ebf (also PG17),
CREATE MATERIALIZED VIEW ... WITH DATA reuses the same REFRESH logic,
meaning a matview containing ts_stat($$SELECT ... FROM articles$$) now
fails at creation time — the problem is immediately visible to the
developer, rather than only surfacing during pg_restore.

For databases migrated from PG16 or earlier, existing matview definitions
that compiled successfully under the old regime still fail when restored
onto PG17+, because pg_restore REFRESH runs under the same restricted
search_path.

Primary motivation: dependency tracking and referential integrity

The deeper problem — unaddressed by search_path hardening — is that
pg_depend has no record of tables referenced inside a ts_stat(text)
argument. The relation name is hidden inside an opaque string constant.
This has two consequences:

Parallel restore ordering. pg_restore -j N has no ordering
constraint between the materialized view REFRESH and the COPY that loads
the referenced table, and may schedule them in the wrong order.

No referential integrity. ts_stat(text) gives none of the
protections that regular views enjoy:

Operation ts_stat(text) ts_stat(sql_query)
ALTER TABLE articles RENAME TO r matview silently uses stale name, REFRESH fails transparent — OID stored, deparser emits current name
ALTER TABLE articles SET SCHEMA s same failure transparent
DROP TABLE articles silently succeeds, matview broken blocked (same error as dropping a table referenced by a view)
DROP TABLE articles CASCADE succeeds, matview silently broken cascades to drop the matview (NOTICE shown)
DROP OWNED BY table owner drops table, matview silently broken cascades to drop the matview

By extending find_expr_references_walker() to walk inside sql_query
constants, this patch creates pg_depend rows for every relation,
function, and type referenced by the stored query tree. The stored Query
holds relation OIDs, so pg_get_querydef() always emits the current
qualified name — rename and schema changes are reflected automatically in
both pg_get_viewdef() output and in subsequent refreshes.

Type design and backward compatibility

ts_stat() now accepts sql_query exclusively. Untyped string literals
such as

ts_stat('SELECT to_tsvector(''english'', body) FROM articles')

continue to work without modification. PostgreSQL function type
resolution (step 3g of parse_func.c) coerces unknown-type literals to
sql_query via sql_query_in(), so no explicit cast is required at
existing call sites. The text → sql_query cast is implicit;
sql_query → text is also implicit (deparsing produces canonical SQL).

With ts_stat(sql_query), object names are resolved to OIDs at query
parse time (when the user has a normal search_path), and the deparser
emits schema-qualified names at execution time regardless of the active
search_path.

-- Works without any explicit cast — string literal coerces to sql_query
-- via function type resolution.  OIDs are captured at parse time under
-- the caller's normal search_path.
CREATE MATERIALIZED VIEW word_stats AS
  SELECT word, ndoc, nentry
  FROM ts_stat($$SELECT to_tsvector('english', body) FROM articles$$);
-- REFRESH under restricted search_path emits public.articles -> OK
-- pg_restore -j N orders REFRESH after COPY articles -> OK
-- ALTER TABLE articles RENAME TO r -> pg_get_viewdef() shows new name -> OK
-- DROP TABLE articles -> ERROR, blocked -> OK

Implementation notes

  • Internal format: nodeToString() of the analyzed Query node,
    same wire format as pg_node_tree. Not stable across major versions,
    but pg_dump writes canonical SQL via sql_query_out(), re-analyzed
    on restore.
  • Input safety: unlike pg_node_tree (which rejects all input),
    sql_query_in() uses the SQL parser — the same path as CREATE VIEW.
  • Volatility: sql_query_in and sql_query_recv are marked stable
    (provolatile = 's'): they perform catalog lookups but return the same
    result for the same input within a transaction.
  • Not collatable (typcollation=0): equality means identical
    nodeToString bytes; comparison uses C collation; hash uses raw bytes.
  • Category Z (matching pg_node_tree): no automatic implicit casts
    from other categories.
  • B-tree opclass (=, <>, <, <=, >, >=) with btequalimage.
  • Hash opclass for GROUP BY, hash joins, and hash indexes.

Files

Area Change
src/backend/utils/adt/sqlquery.c New: I/O, comparison, hash functions
src/backend/utils/adt/tsvector_op.c ts_stat(sql_query[, text]) — replaces former ts_stat(text[, text])
src/backend/catalog/dependency.c SQL_QUERYOID case in find_expr_references_walker
src/include/catalog/pg_type.dat Type OID 8794, array OID 8795
src/include/catalog/pg_proc.dat I/O (8796–8799), cmp (8800–8806), hash (8816–8817), ts_stat (8807–8808)
src/include/catalog/pg_{operator,opfamily,opclass,amop,amproc,cast}.dat Operators, btree+hash opclasses, casts
doc/src/sgml/datatype.sgml New §8.X "sql_query Type"
doc/src/sgml/func/func-textsearch.sgml ts_stat(sql_query) overload docs
src/test/regress/ New sqlquery test; type_sanity and opr_sanity updates
src/include/catalog/catversion.h Bumped to 202606253

A new built-in type sql_query stores a fully analyzed SQL query as a
nodeToString() serialization of a Query node.  On input, the query string
is parsed and analyzed, resolving all referenced object names to their OIDs.
On output, pg_get_querydef() deparses the stored Query back to canonical SQL.
Because deparsing is search_path-sensitive (it calls generate_relation_name()
which invokes RelationIsVisible()), the output is schema-qualified whenever
the referenced relation is not visible under the current search_path.

This solves a long-standing problem with ts_stat() inside materialized views.
pg_restore runs REFRESH MATERIALIZED VIEW with search_path = '' for security;
if the mat-view definition calls ts_stat('SELECT ... FROM articles'), the
unqualified table name cannot be resolved in that context.  With sql_query,
the deparser emits "SELECT ... FROM public.articles" at refresh time, so the
SPI call succeeds regardless of search_path.

ts_stat() now accepts sql_query exclusively.  Untyped string literals such as

    ts_stat('SELECT to_tsvector(''english'', body) FROM articles')

continue to work without modification: PostgreSQL function type resolution
(step 3g of parse_func.c) coerces unknown-type literals to sql_query via
sql_query_in(), so no explicit cast is required at existing call sites.
The text → sql_query cast is implicit, and sql_query → text is implicit
(deparsing the stored query to canonical SQL).

Dependency tracking is extended in find_expr_references_walker() to walk
inside sql_query constants.  When a materialized view is created, pg_depend
rows are recorded for every relation, function, type, and operator referenced
inside the sql_query argument of ts_stat().  This gives the same referential
integrity as plain SQL views: DROP TABLE on a referenced relation is blocked
without CASCADE; DROP TABLE CASCADE drops the materialized view; renames and
schema changes are transparent because the stored representation is OID-based,
not name-based.

The type is non-collatable (typcollation = 0).  Comparison uses C collation
because nodeToString() output is always ASCII.  B-tree and hash opclasses are
provided, enabling ORDER BY, DISTINCT, and index access on sql_query columns.

sql_query_in and sql_query_recv are marked provolatile 's' (stable): they
perform catalog lookups to resolve names to OIDs but return the same result
for the same input within a transaction, consistent with the type_sanity
requirement for type I/O functions.

Bump catalog version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant