Repository navigation
Answer version, range and change-data mistakes with specific errors - #190
littlegrasscao wants to merge 2 commits into
Conversation
|
🔍 OpenCodeReview found 3 issue(s) in this PR.
📄
|
7614b50 to
07cdf3a
Compare
07cdf3a to
4c24439
Compare
…ame the data provider A shared table whose Delta log is missing, starts after version 0, has a gap, or is otherwise invalid, or whose version Kernel cannot read, returned 500 INTERNAL_ERROR or an error code and message of this server's own. The recipient cannot fix any of these, so the answer now says what is wrong and, when the shared table itself is at fault, whom to contact. DeltaKernel classifies every Kernel failure once, by Kernel's exception type rather than its message text: - A missing log (TableNotFoundException), or a file that is missing or denied (FileNotFoundException, AccessDeniedException), is the shared table's own state: 400 BAD_REQUEST "Unable to access table <share>.<schema>.<table>: something is wrong with the table. Please contact your data provider." The table is named as the share names it, and its storage path stays in the log. - An InvalidTableException, such as a gap in the log, is reported in Kernel's own words, with every spelling of the table's path and any other location removed, and the recipient is asked to contact the data provider: 400 BAD_REQUEST. - Any other Kernel report, such as a version or timestamp the table does not have, is the request's mistake in Kernel's words, without the path: 400 BAD_REQUEST. - A failure of the storage layer for a reason of its own stays a server error. - A reader feature or version Kernel cannot read is named in a fixed order: 400 INVALID_PARAMETER_VALUE "Table features <features> are found in table[ version: <n>]. This server cannot read them. Please contact your data provider." The version endpoint no longer pre-checks a timestamp against the latest commit and leaves that to Kernel, and a snapshot is opened straight at the version or timestamp asked for. Adds ErrorCodes.BAD_REQUEST, ApiException.badRequest and ApiException.unsupportedFeatures, and fixtures for a log with in-commit timestamps but no CommitInfo (in a directory with a space in its name) and a table that is readable at its latest version but not at version 0. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
4c24439 to
72bd79c
Compare
A request with a version or timestamp the table does not have, a malformed or contradictory range, or a Change Data Feed the source table does not record got errors of this server's own wording, and some leaked the table's storage path. Each now has one specific answer: - A non-numeric version, startingVersion or endingVersion is 400 INVALID_PARAMETER_VALUE "<name> is not a valid number.", and a negative one "<name> cannot be negative." - More than one of version, timestamp and startingVersion, or more than one starting or ending bound of a Change Data Feed, names the parameters to choose from. A reversed range names both versions. - A Change Data Feed start or end after the table's latest version, or an end timestamp after its latest commit, is rejected with the latest version or commit time to use, rather than capped. - A table that did not record change data for a version in the range is the source table's own state: 403 PERMISSION_DENIED "Unable to access change data feed for <share>.<schema>.<table>. CDF is not enabled on the original delta table for version <n>. Please contact your data provider." - A version or timestamp Kernel rejects keeps Kernel's own message, in 400 BAD_REQUEST, without the table's storage path. A Change Data Feed range is read whole for its protocols and metaData before the response begins, so a version that stops recording change data, or needs reader features the client did not advertise, answers with its status instead of resetting a stream that has begun. The checks and their messages live in TableRequestChecks, and run once, in ProtocolTables, before the table is read. The snapshot and metadata readers no longer check for a version together with a timestamp. Stacked on the change that classifies Kernel's failures. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
72bd79c to
0c9e105
Compare
Depends on #186.
A request with a version or timestamp the table does not have, a malformed or contradictory range, or a Change Data Feed the source table does not record got errors in this server's own wording, and some leaked the table's storage path. Each now has one specific answer.
version,startingVersion,endingVersionINVALID_PARAMETER_VALUE"<name>is not a valid number."version/startingVersion/endingVersion<name>cannot be negative." (query body: "table version cannot be negative.")endingTimestampafter the latest commit<ts>) is after the latest table version available to this table (<latest>). Please use a timestamp less than or equal to<latest>"PERMISSION_DENIED"Unable to access change data feed for<share>.<schema>.<table>. CDF is not enabled on the original delta table for version<n>. Please contact your data provider."BAD_REQUEST, Kernel's message without the storage path (from #186)A Change Data Feed range is read whole for its protocols and metaData before the response begins, so a version that stops recording change data, or needs reader features the client did not advertise, answers with its status (for example 403) instead of resetting a stream that has begun. A start timestamp before the first commit starts at version 0, as Kernel resolves it.
The checks and their messages live in
TableRequestChecksand run once, inProtocolTables, before the table is read; the snapshot and metadata readers no longer check for a version together with a timestamp.Test plan
opensharing-server-coreandopensharing-servertest runs pass locally (Java 17, Maven). S3-backed integration classes skip without cloud credentials; their expectations were updated but not run.This pull request and its description were written by Isaac.