The curation gate parses every query twice, and the second parse refuses nothing

[kstonekuan]

reject_non_single_select runs three checks in order (src/hflow/curation.py:604):

1. exactly one statement, of type SELECT (:637)
2. the leading keyword is not pragma / describe / show / summarize (:643)
3. the SQL parses inside SELECT * FROM (<sql>), the shape preview interpolates (:651)

The third one refuses nothing that the first two have not already refused.

Measured

Deleting the whole third block leaves every curation test green:

### wrapper parse removed
    100 passed in 6.24s
    (nothing noticed)

So I went looking for an input it catches on its own: one that parses standalone as a single StatementType.SELECT, is not headed by one of the four keywords, and still fails to parse inside the wrapper. Across 27 shapes there is none. Every case where the wrapped parse fails, something earlier has already refused it:

sql	standalone	wrapped
PRAGMA database_list	1xSELECT	ERR (already refused at step 2)
(PRAGMA database_list)	ERR ParserException	ERR (already refused at step 1)
COPY (SELECT 1) TO 'x.csv'	1xCOPY	ERR (already refused at step 1)
SELECT 1 /* unterminated	ERR ParserException	ERR (already refused at step 1)
everything else tried	1xSELECT	parses

Shapes tried that all parse both ways: SELECT 1;, SELECT 1;;, trailing line comment, ORDER BY, LIMIT, UNION, UNION ALL ... ORDER BY ... LIMIT, CTE, FROM episodes, VALUES, (SELECT 1), (DESCRIBE SELECT 1), SELECT * FROM (SHOW TABLES), SELECT * FROM pragma_version(), TABLE episodes, * EXCLUDE, QUALIFY, unnest, UNPIVOT, GROUP BY ALL, SELECT 1 UNION (SELECT 2).

What it costs

It parses every query a second time, and its only demonstrated behaviour is almost refusing valid SQL. Two workarounds exist purely to stop that: the appended \n at :651, so a trailing -- comment cannot swallow the wrapper's ), and the .rstrip(';'), so SELECT 1; survives. Both are load-bearing only for the block itself. Neither is needed once it goes.

That is the same failure direction as the regression this gate already had once: refusing legal read-only SELECTs.

What to do

Delete the block at :647-659 and the wrapped_input line, keeping steps 1 and 2. Then delete whichever of the two supporting tests no longer describe anything real, and keep the ones that still pin behaviour:

• test_reject_non_single_select_accepts_trailing_line_comment_without_newline keeps its assertion but its comment stops being true, so rewrite the comment.
• test_reject_non_single_select_accepts_a_trailing_semicolon stays as-is; the behaviour it pins must survive.

If you find an input the wrapper parse refuses correctly, say what it is and close this instead. That is the more useful outcome and I would rather be wrong here.

Definition of done

1. The wrapper parse is gone, and PRAGMA/DESCRIBE/SHOW/SUMMARIZE are still refused by both /curation/preview and /curation/pin with the same message.
2. FROM-first, parenthesized, VALUES, trailing-semicolon and trailing-comment queries are all still accepted.
3. Mutation: delete the leading-keyword refusal, confirm a test goes red. It is now the only thing standing between the endpoints and what they advertise.
4. Both suites green, including packages/hflow-server.

Validation

uv sync --locked --all-extras
uv run ruff check
uv run ruff format --check
uv run ty check
uv run pytest -q
cd packages/hflow-server && uv run --project . pytest -q

[Bortlesboat]

I'd like to take this.

I went looking for the input you asked about and found some, but I don't think they argue for keeping the block as it is.

On main (1a23159, duckdb 1.5.5) these pass steps 1 and 2, fail only the wrapper parse, and run fine standalone:

SELECT 1; -- note, SELECT 1;-- c, SELECT 1; /* c */, SELECT 1;\n-- c, SELECT 1 ; ; -- c, SELECT 1\n;\n;, /* lead */ SELECT 1; /* tail */

.rstrip(';') can't reach a semicolon that has a comment after it, so the block refuses legal SQL, same direction as the rest of the issue. It is also, though, what keeps preview and pin agreeing on some of them. Through the API against the test catalog:

sql	main: preview / pin	block deleted: preview / pin
SELECT episode_id FROM episodes -- note	400 syntax error at end of input / 200	400 / 200
SELECT episode_id FROM episodes; -- note	400 gate message / 400 gate message	400 LINE 1: DESCRIBE SELECT * FROM (SELECT episode_id FROM episodes; -- / 200
SELECT episode_id FROM episodes;/* c */	400 gate message / 400 gate message	400 same leak / 200
SELECT episode_id FROM episodes\n;\n;	400 LINE 2: ;) / 200	400 / 200

Two rows already disagree on main: the newline workaround lives in the gate, but the wrappers in run_preview and _described_columns interpolate the server's _stripped_sql_or_refuse output without one. Deleting the block adds the other two, and the second of those is the #450 leak again. test_catalog_curation.py (110) and the preview/pin suites stay green in every column, which matches what you measured.

What I'd do: delete the block as you describe, and fix the text where it is actually wrapped. In _stripped_sql_or_refuse, cut at the first top-level ; only when nothing but semicolons follows it (duckdb.tokenize skips comments and reads ';' in a literal as a string), so SELECT 1; DROP TABLE x stays whole and the gate still refuses it. Then put the newline before each wrapper's closing paren instead of in the gate. Pinned and saved query text stays what the user wrote, minus the trailing semicolons.

I have that working locally: every row above returns 200 from both endpoints, SELECT 1 AS one; CREATE TABLE pwned AS SELECT 1, ; and PRAGMA database_list still get the same 400 from both, and the root suite, the server suite, ruff and ty are green. I'd add a test that posts these shapes to both endpoints and asserts they agree, since nothing notices today.

If you'd rather keep this issue to the deletion and track the server half separately, happy to split it.