Skip to content

Refuse tables whose Delta properties the client did not advertise - #189

Closed
littlegrasscao wants to merge 5 commits into
OpenSharing-IO:mainfrom
littlegrasscao:fix/unsupported-table-properties
Closed

littlegrasscao wants to merge 5 commits into
OpenSharing-IO:mainfrom
littlegrasscao:fix/unsupported-table-properties

Conversation

@littlegrasscao

@littlegrasscao littlegrasscao commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Depends on #190 (which depends on #186).

A table whose Delta properties enable a reader feature that the recipient's client did not advertise cannot be served to that client: column mapping by name (delta.columnMapping.mode) and deletion vectors (delta.enableDeletionVectors) are only readable by a client that declares them in delta-sharing-capabilities. Such a request now answers 400 INVALID_PARAMETER_VALUE:

Table features delta.columnMapping.mode are found in table version: 1. Query with responseFormat=delta and advertise the features in delta-sharing-capabilities, or ask your data provider to disable them.

It applies to every snapshot the server serves: metadata, query and changes, for the latest and for a historical version, and for the metadata of each commit of a change feed, which is read whole before the response begins. A property at its off value never blocks a request, and an unrelated property is ignored.

The same answer now reports reader features the client did not advertise, naming all of them in a fixed order with the table version. Deletion vectors stay in a protocol after they are turned off, so only their property decides. A parquet-only client on a table that needs reader features gets the same answer.

The advertised features are parsed once per request, and the checked properties are kept in an ordered list.

Test plan

  • opensharing-server-core and opensharing-server test runs pass locally (Java 17, Maven). S3-backed integration classes skip without cloud credentials.
  • Unit tests: off values, an unrelated property, case of advertised features, and message order.
  • Protocol tests: metadata, query (delta and parquet format), a historical version, and the changes endpoint, including a commit's own metadata.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🔍 OpenCodeReview found 5 issue(s) in this PR.

  • ✅ Successfully posted inline: 2 comment(s)
  • 📋 Routed to summary by policy: 2 comment(s)
  • ⏭️ Skipped (overlap with history): 1 comment(s)

bug · low

📄 server/opensharing-server-core/src/main/java/io/opensharing/asset/table/delta/DeltaKernel.java (L155-L155)

⚠️ GitHub could not post this as an inline comment: Routed to summary (severity low · category bug)

This matches the exception class by simple name. Any AccessDeniedException cause, including one from this server's own expired or misconfigured vended credential, is reported to the recipient as a table fault. The log.error is the only signal operators get. Consider matching the concrete exception classes explicitly, and tell the recipient apart from the operator when the credential has expired.


performance · low

📄 server/opensharing-server-core/src/main/java/io/opensharing/asset/table/delta/DeltaTableQueryReader.java (L344-L344)

⚠️ GitHub could not post this as an inline comment: Routed to summary (severity low · category performance)

checkCommits reads and parses every commit file in the range a second time, before the output pass. For large ranges this doubles the storage I/O. Consider validating during the first pass into a buffer, or limiting the pre-scan to the protocol and metaData actions.

@littlegrasscao
littlegrasscao force-pushed the fix/unsupported-table-properties branch from 4ad8fa3 to 9773bdb Compare November 13, 2026 01:57
…ame the data provider

A shared table whose Delta log is missing, starts after version 0, has a
gap, or is otherwise invalid, or whose version Kernel cannot read,
returned 500 INTERNAL_ERROR or an error code and message of this
server's own. The recipient cannot fix any of these, so the answer now
says what is wrong and, when the shared table itself is at fault, whom
to contact.

DeltaKernel classifies every Kernel failure once, by Kernel's exception
type rather than its message text:

- A missing log (TableNotFoundException), or a file that is missing or
  denied (FileNotFoundException, AccessDeniedException), is the shared
  table's own state: 400 BAD_REQUEST "Unable to access table
  <share>.<schema>.<table>: something is wrong with the table. Please
  contact your data provider." The table is named as the share names
  it, and its storage path stays in the log.
- An InvalidTableException, such as a gap in the log, is reported in
  Kernel's own words, with every spelling of the table's path and any
  other location removed, and the recipient is asked to contact the
  data provider: 400 BAD_REQUEST.
- Any other Kernel report, such as a version or timestamp the table does
  not have, is the request's mistake in Kernel's words, without the
  path: 400 BAD_REQUEST.
- A failure of the storage layer for a reason of its own stays a server
  error.
- A reader feature or version Kernel cannot read is named in a fixed
  order: 400 INVALID_PARAMETER_VALUE "Table features <features> are
  found in table[ version: <n>]. This server cannot read them. Please
  contact your data provider."

The version endpoint no longer pre-checks a timestamp against the
latest commit and leaves that to Kernel, and a snapshot is opened
straight at the version or timestamp asked for.

Adds ErrorCodes.BAD_REQUEST, ApiException.badRequest and
ApiException.unsupportedFeatures, and fixtures for a log with in-commit
timestamps but no CommitInfo (in a directory with a space in its name)
and a table that is readable at its latest version but not at version 0.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
@littlegrasscao
littlegrasscao force-pushed the fix/unsupported-table-properties branch from 9773bdb to 255e713 Compare November 13, 2026 02:02
Sun Cao and others added 2 commits November 13, 2026 02:24
A request with a version or timestamp the table does not have, a
malformed or contradictory range, or a Change Data Feed the source table
does not record got errors of this server's own wording, and some
leaked the table's storage path. Each now has one specific answer:

- A non-numeric version, startingVersion or endingVersion is 400
  INVALID_PARAMETER_VALUE "<name> is not a valid number.", and a
  negative one "<name> cannot be negative."
- More than one of version, timestamp and startingVersion, or more than
  one starting or ending bound of a Change Data Feed, names the
  parameters to choose from. A reversed range names both versions.
- A Change Data Feed start or end after the table's latest version, or
  an end timestamp after its latest commit, is rejected with the latest
  version or commit time to use, rather than capped.
- A table that did not record change data for a version in the range is
  the source table's own state: 403 PERMISSION_DENIED "Unable to access
  change data feed for <share>.<schema>.<table>. CDF is not enabled on
  the original delta table for version <n>. Please contact your data
  provider."
- A version or timestamp Kernel rejects keeps Kernel's own message, in
  400 BAD_REQUEST, without the table's storage path.

A Change Data Feed range is read whole for its protocols and metaData
before the response begins, so a version that stops recording change
data, or needs reader features the client did not advertise, answers
with its status instead of resetting a stream that has begun.

The checks and their messages live in TableRequestChecks, and run once,
in ProtocolTables, before the table is read. The snapshot and metadata
readers no longer check for a version together with a timestamp.

Stacked on the change that classifies Kernel's failures.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
A table whose Delta properties enable a reader feature that the
recipient's client did not advertise cannot be served to that client:
column mapping by name (delta.columnMapping.mode) and deletion vectors
(delta.enableDeletionVectors) are only readable by a client that declares
them in delta-sharing-capabilities. Such a request now answers 400
INVALID_PARAMETER_VALUE "Table features <properties> are found in table
version: <n>. Query with responseFormat=delta and advertise the features
in delta-sharing-capabilities, or ask your data provider to disable
them." on every snapshot the server serves: the metadata, query and
change answers, for the latest and for a historical version, and for the
metadata of each commit of a change feed, before the response is
written; a change feed's commits are read whole first. A property set to its off value never blocks a
request, and a property no feature depends on is ignored.

The same answer, with the same wording, now reports reader features the
client did not advertise, naming every one of them in a fixed order
with the table version, where it named the first. Deletion vectors stay
in a protocol after they are turned off, so only their property decides.

The advertised reader features are parsed once per request, and
DeltaSharingCapabilities keeps its properties in an ordered list.

Adds column-mapping, deletion-vector, properties-off and later-column-
mapping tables to the local fixtures.

Stacked on the change that answers version, range and change-data
mistakes.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
@littlegrasscao
littlegrasscao force-pushed the fix/unsupported-table-properties branch from 255e713 to 519ffbd Compare November 13, 2026 02:25
Delta reads delta.columnMapping.mode and delta.enableDeletionVectors
without regard to case, so "NONE" and "False" turn their features off.
The check no longer refuses a table that sets them so.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
A denied read of the table's files can be this server's own expired or
misconfigured credential, so it is logged at error level for operators,
while the recipient is still told to contact the data provider.

The table path is also removed when Kernel's message spells it fully
percent-encoded, and tests cover s3, s3a, abfss, gs and file locations
in each spelling.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
@littlegrasscao

Copy link
Copy Markdown
Collaborator Author

Superseded by #197, stacked on #196 from an OpenSharing-IO branch. Review threads here were answered before the move.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant