Skip to content

Send metaData.version only where a Delta Sharing response defines it - #193

Merged
littlegrasscao merged 1 commit into
mainfrom
fix/response-shape
Nov 13, 2026
Merged

littlegrasscao merged 1 commit into
mainfrom
fix/response-shape

Conversation

@littlegrasscao

@littlegrasscao littlegrasscao commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Depends on #192.

Problem

Query Table and Query Table Metadata responses put the queried version into metaData.version on version and timestamp queries, and carried the latest snapshot's size and file count in the metaData of a Change Data Feed. Delta Sharing sends metaData.version only on a Change Data Feed response with includeHistoricalMetadata=true, and never a size or file count on a Change Data Feed; the version of a version or timestamp query travels only in the Delta-Table-Version header. Clients that compare whole metadata objects fail on the extra fields.

Change

  • DeltaTableMetadataReader and DeltaTableQueryReader write metaData.version as null for metadata and snapshot query responses; the version stays in the Delta-Table-Version header.
  • A Change Data Feed's leading metaData carries a version only with includeHistoricalMetadata, and never size or numFiles. The metaData of a startingVersion query keeps the starting version and the checksum's size and file count.

Test plan

  • mvn -pl opensharing-server-core,opensharing-server clean test (Java 17) passes; cloud-gated integration classes need credentials and run in CI.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

✅ OpenCodeReview: Review complete: 0 finding(s) across 2 selected item(s).

@littlegrasscao littlegrasscao changed the title Send metaData.version only for historical CDF metadata Send metaData.version only where a Delta Sharing response defines it Nov 13, 2026
Base automatically changed from fix/cdc-data-change to main November 13, 2026 20:46
A snapshot or metadata response carries no version in its metaData, even
when it reads a given version or timestamp: the version is in the
Delta-Table-Version header. A Change Data Feed's leading metaData carries
a version only with includeHistoricalMetadata and never a size or file
count.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
@littlegrasscao
littlegrasscao merged commit 7b3adb7 into main Nov 13, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants