-
Notifications
You must be signed in to change notification settings - Fork 5
Description
Superset has the ability to set a time range in its Chart Graph functionality. When set, the resulting SQL has a WHERE clause of this form:
WHERE time >= 2023-01-08 00:00:00.000000
AND time < 2023-01-15 00:00:00.000000
This results in: executing query: failed to create Flight record reader: arrow/flight: could not create flight reader: arrow/ipc: could not read schema from stream: arrow/ipc: could not read message schema: rpc error: code = InvalidArgument desc = Error while planning query: SQL error: ParserError("Expected end of statement, found: 00").
A simple reproducer that generates an error without superset is:
SELECT * FROM "some_table"
WHERE time >= 2023-01-08 00:00:00.000000
AND time < 2023-01-15 00:00:00.000000
LIMIT 10;
We can make the error go away if we quote the date strings:
SELECT * FROM "some_table"
WHERE time >= '2023-01-08 00:00:00.000000'
AND time < '2023-01-15 00:00:00.000000'
LIMIT 10;
https://arrow.apache.org/datafusion/user-guide/sql/select.html#where-clause is pretty lean on details, but AIUI, it is using sqlparser which is ANSI SQL:2011. I tried to find the grammar for that but apparently it is behind a paywall. Postgres is mostly ANSI SQL:2016 compliant and section 8.5.1.3 of https://www.postgresql.org/docs/15/datatype-datetime.html speaks of TIMESTAMP '2004-10-19 10:23:54' being a valid SQL standard timestamp.
As such, this is likely most correct (ie, flightsql-dbapi likely should generate this form of WHERE):
SELECT * FROM "some_table"
WHERE time >= timestamp '2023-01-08 00:00:00.000000'
AND time < timestamp '2023-01-15 00:00:00.000000'
LIMIT 10;