Skip to content

[Server] Serve OpenSharing's share API from Unity Catalog - #1

Draft
littlegrasscao wants to merge 3 commits into
sharing/connectorfrom
sharing/provider-shares
Draft

littlegrasscao wants to merge 3 commits into
sharing/connectorfrom
sharing/provider-shares

Conversation

@littlegrasscao

@littlegrasscao littlegrasscao commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Review copy of unitycatalog#2146, part 2 of 4 of embedded OpenSharing in Unity Catalog. It targets the previous PR's branch, so its diff shows only this part. It won't be merged here: the change merges through unitycatalog#2146, which has the full description and CI.

The four PRs, each building on the one before:

  1. [Server][OpenSharing 1/6] Add the Unity Catalog connector for OpenSharing unitycatalog/unitycatalog#2145: the Unity Catalog connector for OpenSharing. It targets main, so it has no review copy.
  2. [Server][OpenSharing 4/6] Serve OpenSharing's share API from Unity Catalog unitycatalog/unitycatalog#2146: the share API. Review copy: this PR.
  3. [Server][OpenSharing 5/6] Serve OpenSharing's recipient API and share grants from Unity Catalog unitycatalog/unitycatalog#2147: the recipient API and share grants. Review copy: [Server] Serve OpenSharing's recipient API and share grants from Unity Catalog #2.
  4. [Server][OpenSharing 6/6] Serve the Delta Sharing protocol and recipient activation from Unity Catalog unitycatalog/unitycatalog#1825: the sharing protocol and recipient activation. Review copy: [Server] Serve the Delta Sharing protocol and recipient activation from Unity Catalog #3.

@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch from e83dae1 to c0d1f16 Compare November 7, 2026 01:19
@littlegrasscao
littlegrasscao force-pushed the sharing/connector branch 2 times, most recently from 0e9a704 to 191a789 Compare November 8, 2026 00:02
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch from c0d1f16 to 24fdad2 Compare November 8, 2026 00:02
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch 2 times, most recently from 85ec7f0 to 56656e6 Compare November 9, 2026 23:10
@littlegrasscao
littlegrasscao force-pushed the sharing/connector branch 2 times, most recently from cf91d2a to 9f202d8 Compare November 9, 2026 23:19
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch 3 times, most recently from d070cdf to f8dbeaf Compare November 11, 2026 17:11
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch from f8dbeaf to f89a689 Compare November 11, 2026 18:22
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch from f89a689 to c63edbb Compare November 11, 2026 23:57
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch 2 times, most recently from 428161b to c921c28 Compare November 12, 2026 02:34
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch from c921c28 to 4b19553 Compare November 12, 2026 05:46
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch from 4b19553 to 61e2db2 Compare November 12, 2026 05:57
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch 2 times, most recently from 15c307b to 80d1cfb Compare November 16, 2026 22:43
@littlegrasscao
littlegrasscao force-pushed the sharing/provider-shares branch 7 times, most recently from 3c34540 to 8c978a6 Compare November 17, 2026 22:13
Sun Cao and others added 3 commits November 17, 2026 22:40
OpenSharing's six tables (os_shares, os_share_properties, os_shared_data_objects, os_recipients, os_share_permissions and os_recipient_tokens) are Unity Catalog schema steps, V4 through V11, for H2, PostgreSQL and MySQL: one statement per step, the tables in foreign-key order, then the owner_id indexes. They are created whether or not embedded OpenSharing is enabled, and the steps build the schema Hibernate builds from OpenSharing's entities, which UC's own session factory now maps. Each step ships its H2 effect file; the PostgreSQL and MySQL ones follow from a run with Docker.

HibernateTransactions runs OpenSharing's store work in sessions of that factory, and joins a transaction the thread already has, so a caller can make several OpenSharing calls and its own steps one unit of work. The schema migrator and the schema snapshot treat os_* tables as Unity Catalog's own, so the steps' index names are not blamed on another application and the snapshots see the tables the steps create.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Match OpenSharing 0.1.1's schema

OpenSharing 0.1.1 stores ids as the database's native uuid (binary(16) on MySQL), its enums as plain varchar through converters, and names its foreign keys. The steps build the same columns, types and names its V1 does on each dialect, and the owner_id indexes its entities declare.

Co-authored-by: Isaac <no-reply@databricks.com>
Shares and recipients become securable types, with CREATE SHARE, USE
SHARE, CREATE RECIPIENT and USE RECIPIENT as privileges on the
metastore. UC grants nothing on a share or recipient itself, so
validation of grants, the authorization hierarchy and the OPA input
treat them as roots, and the permissions API refuses them.

The key mapper resolves a share or recipient by name from OpenSharing's
tables, and answers a missing one with the NOT_FOUND that UC gives for
any missing object, not OpenSharing's own exception. It reads those
tables only when asked for such a key, which only OpenSharing's provider
API, once enabled, produces. The result filter reads the ids of
OpenSharing's share and recipient responses.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Read OpenSharing 0.1.1's UUID ids

OpenSharing 0.1.1 keys its entities by UUID, so the OPA input and the
key mapper look them up directly instead of by their string form, and
the JSON still carries the string form.

Co-authored-by: Isaac <no-reply@databricks.com>
With server.opensharing.enabled=true, the server serves OpenSharing's
provider API for shares at /api/2.1/unity-catalog/opensharing/provider,
behind UC's authentication and authorization. It is registered as the
OPENSHARING protocol of ArmeriaServerBuilder next to Iceberg and Delta,
with OpenSharing's JSON mapper and error dialect. UC records each
share's owner in its authorizer, in the same transaction as the share
itself. A share that does not exist is OpenSharing's 404 on these
routes. With the flag off, none of this is registered and none of
OpenSharing's tables is read or written.

A sharing docs page describes the provider API, its privileges, how
shared tables are read, and the database tables.

Signed-off-by: Sun Cao <sun.cao+data@databricks.com>

Consume OpenSharing 0.1.1 and support admin ownership transfers

The provider service answers with OpenSharing's JSON, so its methods
carry ProducesJson: without it Armeria found no response converter and
answered 500. The caller's context now says whether they own the
metastore, and PATCH /shares/{share} takes a catalog admin transferring
the ownership to the body's ownerId, on its own, with UC's owner record
moving to the new owner in the same transaction.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant