Skip to content

Relay Kernel's report on a version, timestamp, or log it cannot read - #195

Merged
littlegrasscao merged 1 commit into
mainfrom
fix/incomplete-delta-log-catalog-error
Nov 13, 2026
Merged

littlegrasscao merged 1 commit into
mainfrom
fix/incomplete-delta-log-catalog-error

Conversation

@littlegrasscao

@littlegrasscao littlegrasscao commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

A version, timestamp, or log that Delta Kernel rejects was answered with this server's own wording, or with 500 INTERNAL_ERROR for a gapped or invalid log. Delta Sharing relays Kernel's report instead.

Case Answer
A version after the latest, a timestamp before the first commit or after the latest, a gapped or invalid log, or another Kernel report 400 BAD_REQUEST, Kernel's message as it is

Change

  • ApiFailure maps a KernelException to 400 BAD_REQUEST with its message; ErrorCodes.BAD_REQUEST is new. Requests that fail after a response began use the same mapping.
  • DeltaSnapshots and the version endpoint no longer check a timestamp against the latest commit before asking Kernel, so Kernel's message is what the recipient gets. A protocol Kernel cannot read keeps its existing answer.
  • A version after the latest is answered with Kernel's message without asking Kernel, which otherwise searches back through the log for a checkpoint for minutes on object storage for a version such as 999999.

Test plan

  • mvn -pl opensharing-server-core,opensharing-server clean test (Java 17) passes; cloud-gated integration classes need credentials, and their expectations were updated for CI.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

🔍 OpenCodeReview found 3 issue(s) in this PR.

  • ✅ Successfully posted inline: 0 comment(s)
  • ⏭️ Skipped (overlap with history): 3 comment(s)

ℹ️ All inline comments overlapped with existing reviews; nothing new was posted.

@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch from 5922607 to b449678 Compare November 13, 2026 04:34
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch 2 times, most recently from c956d68 to 616ca34 Compare November 13, 2026 05:08
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch from 616ca34 to a15be76 Compare November 13, 2026 05:20
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch from a15be76 to 8132582 Compare November 13, 2026 05:41
@littlegrasscao littlegrasscao changed the title Answer a missing or invalid shared-table Delta log with errors that name the data provider Answer a missing or failing shared-table Delta log as the Delta Sharing server does Nov 13, 2026
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch from 8132582 to fcd5e7a Compare November 13, 2026 05:51
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch 2 times, most recently from 92613ac to a163a33 Compare November 13, 2026 07:18
@littlegrasscao littlegrasscao changed the title Answer a missing or failing shared-table Delta log as the Delta Sharing server does Relay Kernel's report on a version, timestamp, or log it cannot read Nov 13, 2026
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch from a163a33 to 5d176b1 Compare November 13, 2026 07:50
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch from 5d176b1 to d3af967 Compare November 13, 2026 17:10
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch from d3af967 to c107c50 Compare November 13, 2026 17:38
A version, timestamp, or log that Delta Kernel rejects, such as a version
after the latest, a timestamp before the first commit, or a missing log
file, is answered with 400 BAD_REQUEST and Kernel's message, instead of
the server's own wording for the cases it checked first.

A version after the latest is answered with Kernel's message without
asking Kernel: for a version far ahead, such as 999999, Kernel searches
back through the log for a checkpoint, which takes minutes on object
storage.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Sun Cao <sun.cao+data@databricks.com>
@littlegrasscao
littlegrasscao force-pushed the fix/incomplete-delta-log-catalog-error branch from c107c50 to b393ef0 Compare November 13, 2026 21:46
@littlegrasscao
littlegrasscao merged commit e9769f0 into main Nov 13, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants