Skip to content

[CI only] OpenSharing stack against published 0.1.1 - #4

Closed
littlegrasscao wants to merge 7 commits into
mainfrom
opensharing-embedded
Closed

littlegrasscao wants to merge 7 commits into
mainfrom
opensharing-embedded

Conversation

@littlegrasscao

Copy link
Copy Markdown
Owner

CI-only draft. It runs the fork's workflows on the whole OpenSharing stack (unitycatalog#2145 through unitycatalog#1825), merged with upstream main at 2f41e34. The point is to resolve io.opensharing:opensharing-server-core:0.1.1 from Maven Central. Upstream runs on the real PRs are waiting for maintainer approval. Don't merge; close this once the upstream runs pass.

This pull request and its description were written by Isaac.

Sun Cao and others added 7 commits November 16, 2026 21:18
The connector is how embedded OpenSharing reads Unity Catalog: it
resolves a table only for a user UC lets read its data, and vends that
table's read-only credentials with UC's credential vendor, refusing any
other operation instead of downgrading it. Nothing serves OpenSharing
yet.

opensharing-server-core is an ordinary dependency of the server module,
like Iceberg, and the connector lives in the server's service/sharing
package, so every packaging path that collects the server's classpath
carries it. Its Hadoop modules stay on hadoopVersion, Delta Kernel's
defaults stay on the server's Delta version, and the published
unitycatalog-client and unitycatalog-hadoop that delta-storage brings in
are excluded.

hadoop-client-runtime registers its relocated Woodstox for StAX, which
breaks Jackson's XML mapper (S3Mock, the Azure SDK), so the server sets
the StAX factory system properties to Woodstox's own. The shaded server
that the Spark connector tests use leaves out the modules that only
OpenSharing adds, by module name.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Consume OpenSharing 0.1.1

Bump opensharing-server-core to 0.1.1. OpenSharing now matches the AWS
SDK and Delta Kernel versions this server uses, so those pins are gone.
It builds against a newer Hadoop, so the server keeps overriding the
Hadoop modules to hadoopVersion. Its core excludes the published Unity
Catalog jars from delta-storage itself, so the exclusion here is gone.
The connector reports whether the caller owns the metastore in the
user context's admin flag, which lets them transfer the ownership of
any share or recipient.

Co-authored-by: Isaac <no-reply@databricks.com>
Keep OpenSharing's Delta and Hadoop versions in the POM and initialize Woodstox for builder-started servers.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Co-authored-by: Isaac <no-reply@databricks.com>
OpenSharing's six tables (os_shares, os_share_properties, os_shared_data_objects, os_recipients, os_share_permissions and os_recipient_tokens) are Unity Catalog schema steps, V3 through V10, for H2, PostgreSQL and MySQL: one statement per step, the tables in foreign-key order, then the owner_id indexes. They are created whether or not embedded OpenSharing is enabled, and the steps build the schema Hibernate builds from OpenSharing's entities, which UC's own session factory now maps. Each step ships its H2 effect file; the PostgreSQL and MySQL ones follow from a run with Docker.

HibernateTransactions runs OpenSharing's store work in sessions of that factory, and joins a transaction the thread already has, so a caller can make several OpenSharing calls and its own steps one unit of work. The schema migrator and the schema snapshot treat os_* tables as Unity Catalog's own, so the steps' index names are not blamed on another application and the snapshots see the tables the steps create.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Match OpenSharing 0.1.1's schema

OpenSharing 0.1.1 stores ids as the database's native uuid (binary(16) on MySQL), its enums as plain varchar through converters, and names its foreign keys. The steps build the same columns, types and names its V1 does on each dialect, and the owner_id indexes its entities declare.

Co-authored-by: Isaac <no-reply@databricks.com>
Shares and recipients become securable types, with CREATE SHARE, USE
SHARE, CREATE RECIPIENT and USE RECIPIENT as privileges on the
metastore. UC grants nothing on a share or recipient itself, so
validation of grants, the authorization hierarchy and the OPA input
treat them as roots, and the permissions API refuses them.

The key mapper resolves a share or recipient by name from OpenSharing's
tables, and answers a missing one with the NOT_FOUND that UC gives for
any missing object, not OpenSharing's own exception. It reads those
tables only when asked for such a key, which only OpenSharing's provider
API, once enabled, produces. The result filter reads the ids of
OpenSharing's share and recipient responses.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Read OpenSharing 0.1.1's UUID ids

OpenSharing 0.1.1 keys its entities by UUID, so the OPA input and the
key mapper look them up directly instead of by their string form, and
the JSON still carries the string form.

Co-authored-by: Isaac <no-reply@databricks.com>
With server.opensharing.enabled=true, the server serves OpenSharing's
provider API for shares at /api/2.1/unity-catalog/opensharing/provider,
behind UC's authentication and authorization. It is registered as the
OPENSHARING protocol of ArmeriaServerBuilder next to Iceberg and Delta,
with OpenSharing's JSON mapper and error dialect. UC records each
share's owner in its authorizer, in the same transaction as the share
itself. A share that does not exist is OpenSharing's 404 on these
routes. With the flag off, none of this is registered and none of
OpenSharing's tables is read or written.

A sharing docs page describes the provider API, its privileges, how
shared tables are read, and the database tables.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Consume OpenSharing 0.1.1 and support admin ownership transfers

The provider service answers with OpenSharing's JSON, so its methods
carry ProducesJson: without it Armeria found no response converter and
answered 500. The caller's context now says whether they own the
metastore, and PATCH /shares/{share} takes a catalog admin transferring
the ownership to the body's ownerId, on its own, with UC's owner record
moving to the new owner in the same transaction.

Co-authored-by: Isaac <no-reply@databricks.com>
The provider API now also creates, reads, changes and deletes
recipients, rotates their tokens, and grants shares to them. UC records
each recipient's owner in its authorizer, in the same transaction as the
recipient itself. Granting a share to a recipient takes owning the
share, and owning the recipient or USE RECIPIENT, checked by annotation
with the recipient read from the typed body, so a request grants to one
recipient at most. Activation links are built on
server.opensharing.public-url, which embedded OpenSharing now requires.
The sharing docs page covers the new endpoints and privileges.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Support admin ownership transfers of recipients

PATCH /recipients/{recipient} takes a catalog admin transferring the
ownership to the body's ownerId, on its own, with UC's owner record
moving to the new owner in the same transaction.

Co-authored-by: Isaac <no-reply@databricks.com>

Document ownership transfers in the sharing docs

Co-authored-by: Isaac <no-reply@databricks.com>
…Catalog

Recipients activate at /api/1.0/opensharing/activations and read shared
tables over the sharing protocol at /api/1.0/opensharing, on UC's port.
They authenticate with OpenSharing's bearer tokens, so these routes sit
outside UC's security decorators: ArmeriaServerBuilder.selfAuthenticated
registers them on the API virtual host. Profiles name the public URL.
The sharing docs page describes recipient access.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Co-authored-by: Isaac <no-reply@databricks.com>
@littlegrasscao
littlegrasscao changed the base branch from branch-ci-opensharing to main November 16, 2026 22:00
@littlegrasscao

Copy link
Copy Markdown
Owner Author

Closing: the real PRs now run CI upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant