Add sql_query data type for pre-analyzed SQL query trees - #1
Open
dimitri wants to merge 1 commit into
Open
Conversation
A new built-in type sql_query stores a fully analyzed SQL query as a
nodeToString() serialization of a Query node. On input, the query string
is parsed and analyzed, resolving all referenced object names to their OIDs.
On output, pg_get_querydef() deparses the stored Query back to canonical SQL.
Because deparsing is search_path-sensitive (it calls generate_relation_name()
which invokes RelationIsVisible()), the output is schema-qualified whenever
the referenced relation is not visible under the current search_path.
This solves a long-standing problem with ts_stat() inside materialized views.
pg_restore runs REFRESH MATERIALIZED VIEW with search_path = '' for security;
if the mat-view definition calls ts_stat('SELECT ... FROM articles'), the
unqualified table name cannot be resolved in that context. With sql_query,
the deparser emits "SELECT ... FROM public.articles" at refresh time, so the
SPI call succeeds regardless of search_path.
ts_stat() now accepts sql_query exclusively. Untyped string literals such as
ts_stat('SELECT to_tsvector(''english'', body) FROM articles')
continue to work without modification: PostgreSQL function type resolution
(step 3g of parse_func.c) coerces unknown-type literals to sql_query via
sql_query_in(), so no explicit cast is required at existing call sites.
The text → sql_query cast is implicit, and sql_query → text is implicit
(deparsing the stored query to canonical SQL).
Dependency tracking is extended in find_expr_references_walker() to walk
inside sql_query constants. When a materialized view is created, pg_depend
rows are recorded for every relation, function, type, and operator referenced
inside the sql_query argument of ts_stat(). This gives the same referential
integrity as plain SQL views: DROP TABLE on a referenced relation is blocked
without CASCADE; DROP TABLE CASCADE drops the materialized view; renames and
schema changes are transparent because the stored representation is OID-based,
not name-based.
The type is non-collatable (typcollation = 0). Comparison uses C collation
because nodeToString() output is always ASCII. B-tree and hash opclasses are
provided, enabling ORDER BY, DISTINCT, and index access on sql_query columns.
sql_query_in and sql_query_recv are marked provolatile 's' (stable): they
perform catalog lookups to resolve names to OIDs but return the same result
for the same input within a transaction, consistent with the type_sanity
requirement for type I/O functions.
Bump catalog version.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This introduces a new built-in type,
sql_query(OID 8794), that storesa fully analyzed SQL query. The input function runs SQL text through the
standard parse and analyze pipeline, resolving all object names to OIDs
at assignment time. The output function deparses the stored Query node
using
pg_get_querydef(), which issearch_path-sensitive and emitsschema-qualified names when needed.
Background
Since commit 2af07e2 (PG17),
REFRESH MATERIALIZED VIEWruns under arestricted
search_pathofpg_catalog, pg_temp, so user tables areunreachable by unqualified name. Since commit 4b74ebf (also PG17),
CREATE MATERIALIZED VIEW ... WITH DATAreuses the same REFRESH logic,meaning a matview containing
ts_stat($$SELECT ... FROM articles$$)nowfails at creation time — the problem is immediately visible to the
developer, rather than only surfacing during
pg_restore.For databases migrated from PG16 or earlier, existing matview definitions
that compiled successfully under the old regime still fail when restored
onto PG17+, because
pg_restoreREFRESH runs under the same restrictedsearch_path.Primary motivation: dependency tracking and referential integrity
The deeper problem — unaddressed by
search_pathhardening — is thatpg_dependhas no record of tables referenced inside ats_stat(text)argument. The relation name is hidden inside an opaque string constant.
This has two consequences:
Parallel restore ordering.
pg_restore -j Nhas no orderingconstraint between the materialized view REFRESH and the COPY that loads
the referenced table, and may schedule them in the wrong order.
No referential integrity.
ts_stat(text)gives none of theprotections that regular views enjoy:
ts_stat(text)ts_stat(sql_query)ALTER TABLE articles RENAME TO rALTER TABLE articles SET SCHEMA sDROP TABLE articlesDROP TABLE articles CASCADEDROP OWNED BYtable ownerBy extending
find_expr_references_walker()to walk insidesql_queryconstants, this patch creates
pg_dependrows for every relation,function, and type referenced by the stored query tree. The stored Query
holds relation OIDs, so
pg_get_querydef()always emits the currentqualified name — rename and schema changes are reflected automatically in
both
pg_get_viewdef()output and in subsequent refreshes.Type design and backward compatibility
ts_stat()now acceptssql_queryexclusively. Untyped string literalssuch as
ts_stat('SELECT to_tsvector(''english'', body) FROM articles')continue to work without modification. PostgreSQL function type
resolution (step 3g of
parse_func.c) coerces unknown-type literals tosql_queryviasql_query_in(), so no explicit cast is required atexisting call sites. The text →
sql_querycast is implicit;sql_query→ text is also implicit (deparsing produces canonical SQL).With
ts_stat(sql_query), object names are resolved to OIDs at queryparse time (when the user has a normal
search_path), and the deparseremits schema-qualified names at execution time regardless of the active
search_path.Implementation notes
nodeToString()of the analyzedQuerynode,same wire format as
pg_node_tree. Not stable across major versions,but
pg_dumpwrites canonical SQL viasql_query_out(), re-analyzedon restore.
pg_node_tree(which rejects all input),sql_query_in()uses the SQL parser — the same path asCREATE VIEW.sql_query_inandsql_query_recvare marked stable(
provolatile = 's'): they perform catalog lookups but return the sameresult for the same input within a transaction.
typcollation=0): equality means identicalnodeToStringbytes; comparison uses C collation; hash uses raw bytes.Z(matchingpg_node_tree): no automatic implicit castsfrom other categories.
=,<>,<,<=,>,>=) withbtequalimage.GROUP BY, hash joins, and hash indexes.Files
src/backend/utils/adt/sqlquery.csrc/backend/utils/adt/tsvector_op.cts_stat(sql_query[, text])— replaces formerts_stat(text[, text])src/backend/catalog/dependency.cSQL_QUERYOIDcase infind_expr_references_walkersrc/include/catalog/pg_type.datsrc/include/catalog/pg_proc.datsrc/include/catalog/pg_{operator,opfamily,opclass,amop,amproc,cast}.datdoc/src/sgml/datatype.sgmldoc/src/sgml/func/func-textsearch.sgmlts_stat(sql_query)overload docssrc/test/regress/sqlquerytest;type_sanityandopr_sanityupdatessrc/include/catalog/catversion.h