Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
c4270b3
CXH-2166: implement PAT (workspace token) authentication
al-conductorone Jul 29, 2026
5d69329
CXH-2166: address review feedback on PAT auth config and logging
al-conductorone Aug 3, 2026
850b2a6
CXH-2166: restore mutual exclusivity between OAuth and workspace tokens
al-conductorone Aug 3, 2026
c08d8b1
CXH-2166: pick auth mode from opts.SelectedAuthMethod, not field pres…
al-conductorone Aug 5, 2026
870c90c
docs: refresh CLI usage dump in README after rebase
al-conductorone Aug 6, 2026
8b12e6a
CXH-2166: fix longest-prefix token match and workspace field docs
al-conductorone Aug 12, 2026
df53744
CXH-2166: fix workspace listing for OAuth installs, address remaining…
al-conductorone Aug 12, 2026
27f217d
CXH-2166: wrap long field-group Fields slice to satisfy line-length lint
al-conductorone Aug 12, 2026
c29d543
CXH-2166: address Warn-nit and resource-ID review comments
al-conductorone Aug 13, 2026
423d12e
CXH-2166: fix group grant parenting under token auth, warn on empty w…
al-conductorone Aug 13, 2026
c6917da
CXH-2166: fix remaining review findings for token auth and account gr…
al-conductorone Aug 13, 2026
622422d
CXH-2166: skip groups the rule-sets API rejects instead of failing th…
al-conductorone Aug 13, 2026
b7158d4
CXH-2166: tighten group-not-found match, warn on skipped roles, drop …
al-conductorone Aug 13, 2026
f1a5ff0
CXH-2166: scope group-not-found skip to workspace-parented groups
al-conductorone Aug 14, 2026
4527918
CXH-2166: make workspaces and exclude-workspaces mutually exclusive
al-conductorone Aug 14, 2026
b691ba6
CXH-2166: remove EOL username/password auth from docs
al-conductorone Aug 14, 2026
b06c054
CXH-2166: require workspaces when workspace-tokens is set
al-conductorone Aug 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 52 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,18 @@ right top corner that will open a dropdown menu with the account ID along other
options.

Another requirement is to have valid credentials to run the connector with. This
will decide how connector will be executed. You can use either OAuth client
credentials flow or Basic auth flow (username and password) or Bearer auth flow.
Both OAuth and Basic can be used across account and all workspaces you have
access to. Bearer auth can be used only for a specific workspace.
will decide how connector will be executed. You can use either the OAuth client
credentials flow or the Bearer auth flow. OAuth can be used across account and
all workspaces you have access to. Bearer auth can be used only for a specific
workspace.

To use the OAuth, you need to create a service principal and add OAuth secret
(client id and secret) to it. You can do that by going to the user management
tab and clicking on the Service Principals tab. Then click on the Add Service
principal button and name it. You then need to add OAuth secret to it by
clicking on the Generate secret button. You can use this secret to authenticate
across all workspaces that service principal has access to. To use basic auth,
you just need to provide a username and password of a user that has access to
the Databricks API. Both methods require admin access to the Databricks account
and each workspace you want to sync.
across all workspaces that service principal has access to. This requires admin
access to the Databricks account and each workspace you want to sync.

To use bearer auth, you need to provide a Databricks workspace access token. You
can create a new token by logging into the workspace and going into user
Expand All @@ -55,14 +53,14 @@ baton-databricks --hostname "azuredatabricks.net"
```
brew install conductorone/baton/baton conductorone/baton/baton-databricks

BATON_ACCOUNT_ID=account_id BATON_USERNAME=username BATON_PASSWORD=password baton-databricks
BATON_ACCOUNT_ID=account_id BATON_DATABRICKS_CLIENT_ID=client_id BATON_DATABRICKS_CLIENT_SECRET=client_secret baton-databricks
baton resources
```

## docker

```
docker run --rm -v $(pwd):/out -e BATON_ACCOUNT_ID=account_id BATON_USERNAME=username BATON_PASSWORD=password ghcr.io/conductorone/baton-databricks:latest -f "/out/sync.c1z"
docker run --rm -v $(pwd):/out -e BATON_ACCOUNT_ID=account_id BATON_DATABRICKS_CLIENT_ID=client_id BATON_DATABRICKS_CLIENT_SECRET=client_secret ghcr.io/conductorone/baton-databricks:latest -f "/out/sync.c1z"
docker run --rm -v $(pwd):/out ghcr.io/conductorone/baton:latest -f "/out/sync.c1z" resources
```

Expand All @@ -72,7 +70,7 @@ docker run --rm -v $(pwd):/out ghcr.io/conductorone/baton:latest -f "/out/sync.c
go install github.com/conductorone/baton/cmd/baton@main
go install github.com/conductorone/baton-databricks/cmd/baton-databricks@main

BATON_ACCOUNT_ID=account_id BATON_USERNAME=username BATON_PASSWORD=password baton-databricks
BATON_ACCOUNT_ID=account_id BATON_DATABRICKS_CLIENT_ID=client_id BATON_DATABRICKS_CLIENT_SECRET=client_secret baton-databricks
baton resources
```

Expand All @@ -97,6 +95,11 @@ both flags at the same time. If you do that, connector will sync with all
workspaces that are associated with provided tokens and all workspaces that are
in the list of workspaces.

When authenticating with `--workspace-tokens` instead of the OAuth client ID and

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Docs] Separate from this PR — "Username and password" auth is still documented but doesn't exist

Not something this PR needs to fix, but since you're already cleaning up the auth docs in this exact file: further up (README.md's "Bearer auth"/basic-auth paragraphs, around the intro section) this README still describes a third auth option — username/password ("Basic auth") — and there's genuinely zero implementation of it anywhere in the codebase (no BasicAuth type in pkg/databricks/auth.go, no username/password fields in pkg/config/config.go). Might be worth a follow-up to either implement it or drop the claim, since a customer following these docs for that option will configure something that silently doesn't exist.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled the username/password bits from the README in this PR. databricks EOL'd basic auth back in 2024 and it was never implemented here, so the docs were just stale.

https://docs.databricks.com/aws/en/security/auth/password-deprecation

secret, also pass `--auth-method workspace-token` (or set

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: while cleaning up these auth docs, the paragraph just above (lines 88-96) is now stale. --workspaces takes deployment names, not "workspace hostnames" (per the field description), and NewTokenAuth pairs --workspaces/--workspace-tokens positionally with ValidateConfig requiring equal lengths — so "use both flags at the same time … sync with all workspaces associated with provided tokens and all workspaces in the list" no longer describes the behavior. The --help dump further down (lines 147/175) is also stale relative to the new "Mutually exclusive with …" descriptions in pkg/config/config.go.

`BATON_AUTH_METHOD=workspace-token`), otherwise the connector validates against
the OAuth fields by default and rejects the config.

To instead exclude specific workspaces from the sync, pass them to the
`--databricks-exclude-workspaces` flag (or the
`BATON_DATABRICKS_EXCLUDE_WORKSPACES` environment variable) as a comma-separated
Expand Down Expand Up @@ -129,25 +132,47 @@ Usage:
Available Commands:
capabilities Get connector capabilities
completion Generate the autocompletion script for the specified shell
config Get the connector config schema
health-check Check the health of a running connector
help Help about any command

Flags:
--account-hostname string The hostname used to connect to the Databricks account API. If not set, it will be calculated from the hostname field. ($BATON_ACCOUNT_HOSTNAME)
--account-id string required: The Databricks account ID used to connect to the Databricks Account and Workspace API ($BATON_ACCOUNT_ID)
--client-id string The client ID used to authenticate with ConductorOne ($BATON_CLIENT_ID)
--client-secret string The client secret used to authenticate with ConductorOne ($BATON_CLIENT_SECRET)
--databricks-client-id string The Databricks service principal's client ID used to connect to the Databricks Account and Workspace API ($BATON_DATABRICKS_CLIENT_ID)
--databricks-client-secret string The Databricks service principal's client secret used to connect to the Databricks Account and Workspace API ($BATON_DATABRICKS_CLIENT_SECRET)
--databricks-exclude-workspaces strings Workspaces to exclude from sync, identified by workspace name, deployment name, or numeric workspace ID ($BATON_DATABRICKS_EXCLUDE_WORKSPACES)
-f, --file string The path to the c1z file to sync with ($BATON_FILE) (default "sync.c1z")
-h, --help help for baton-databricks
--hostname string The Databricks hostname used to connect to the Databricks API ($BATON_HOSTNAME) (default "cloud.databricks.com")
--log-format string The output format for logs: json, console ($BATON_LOG_FORMAT) (default "json")
--log-level string The log level: debug, info, warn, error ($BATON_LOG_LEVEL) (default "info")
-p, --provisioning This must be set in order for provisioning actions to be enabled ($BATON_PROVISIONING)
--skip-full-sync This must be set to skip a full sync ($BATON_SKIP_FULL_SYNC)
--ticketing This must be set to enable ticketing support ($BATON_TICKETING)
-v, --version version for baton-databricks
--account-hostname string The hostname used to connect to the Databricks account API. If not set, it will be calculated from the hostname field. ($BATON_ACCOUNT_HOSTNAME)
--account-id string required: The Databricks account ID used to connect to the Databricks Account and Workspace API ($BATON_ACCOUNT_ID)
--auth-method string ($BATON_AUTH_METHOD)
--client-id string The client ID used to authenticate with ConductorOne ($BATON_CLIENT_ID)
--client-secret string The client secret used to authenticate with ConductorOne ($BATON_CLIENT_SECRET)
--databricks-client-id string required: The Databricks service principal's client ID used to connect to the Databricks Account and Workspace API ($BATON_DATABRICKS_CLIENT_ID)
--databricks-client-secret string required: The Databricks service principal's client secret used to connect to the Databricks Account and Workspace API ($BATON_DATABRICKS_CLIENT_SECRET)
--databricks-exclude-workspaces strings Workspaces to exclude from sync, identified by workspace name, deployment name, or numeric workspace ID ($BATON_DATABRICKS_EXCLUDE_WORKSPACES)
--external-resource-c1z string The path to the c1z file to sync external baton resources with ($BATON_EXTERNAL_RESOURCE_C1Z)
--external-resource-entitlement-id-filter string The entitlement that external users, groups must have access to sync external baton resources ($BATON_EXTERNAL_RESOURCE_ENTITLEMENT_ID_FILTER)
--external-resource-traits strings Resource type traits (e.g. "user", "group", "app") to sync and match from the external resource c1z. When unset the matcher falls back to user and group; passing this flag replaces the full set rather than adding to it. ($BATON_EXTERNAL_RESOURCE_TRAITS)
-f, --file string The path to the c1z file to sync with ($BATON_FILE) (default "sync.c1z")
--health-check Enable the HTTP health check endpoint ($BATON_HEALTH_CHECK)
--health-check-port int Port for the HTTP health check endpoint ($BATON_HEALTH_CHECK_PORT) (default 8081)
-h, --help help for baton-databricks
--hostname string The Databricks hostname used to connect to the Databricks API ($BATON_HOSTNAME) (default "cloud.databricks.com")
--http-timeout-seconds int HTTP client timeout in seconds (max 1800) ($BATON_HTTP_TIMEOUT_SECONDS) (default 300)
--keep-previous-sync-c1z Keep the previously synced c1z on disk to enable ETag replay across service-mode syncs (requires a connector that supports ETag replay; costs one c1z of local disk) ($BATON_KEEP_PREVIOUS_SYNC_C1Z)
--log-format string The output format for logs: json, console ($BATON_LOG_FORMAT) (default "json")
--log-level string The log level: debug, info, warn, error ($BATON_LOG_LEVEL) (default "info")
--log-level-debug-expires-at string The timestamp indicating when debug-level logging should expire ($BATON_LOG_LEVEL_DEBUG_EXPIRES_AT)
--log-path strings The file path to write logs to ($BATON_LOG_PATH)
--otel-collector-endpoint string The endpoint of the OpenTelemetry collector to send observability data to (used for both tracing and logging if specific endpoints are not provided) ($BATON_OTEL_COLLECTOR_ENDPOINT)
--parallel-sync Deprecated: use --workers instead. ($BATON_PARALLEL_SYNC)
-p, --provisioning This must be set in order for provisioning actions to be enabled ($BATON_PROVISIONING)
--skip-entitlements-and-grants This must be set to skip syncing of entitlements and grants ($BATON_SKIP_ENTITLEMENTS_AND_GRANTS)
--skip-full-sync This must be set to skip a full sync ($BATON_SKIP_FULL_SYNC)
--storage-engine string The storage engine to use when opening the sync c1z file: sqlite or pebble. Leave unset to use the baton-sdk default. ($BATON_STORAGE_ENGINE)
--sync-resource-types strings The resource type IDs to sync ($BATON_SYNC_RESOURCE_TYPES)
--sync-resources strings The resource IDs to sync ($BATON_SYNC_RESOURCES)
--task-concurrency int The number of Baton tasks to run concurrently in service mode. Tasks may include sync, grant, revoke, and more. Minimum value is 1, maximum value is 100. ($BATON_TASK_CONCURRENCY) (default 3)
--ticketing This must be set to enable ticketing support ($BATON_TICKETING)
-v, --version version for baton-databricks
--workers int The number of sync workers to use. -1 for auto-detect, 0 for sequential, >0 for parallel ($BATON_WORKERS)
--workspace-tokens strings required: The Databricks personal access tokens scoped to specific workspaces used to connect to the Databricks Workspace API ($BATON_WORKSPACE_TOKENS)
--workspaces strings Limit syncing to the specified workspaces, by deployment name, not workspace ID. Required when using workspace tokens, in the same order as workspace-tokens. ($BATON_WORKSPACES)

Use "baton-databricks [command] --help" for more information about a command.
```
72 changes: 70 additions & 2 deletions config_schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -140,14 +140,82 @@
"defaultValue": "cloud.databricks.com"
}
},
{
"name": "workspaces",
"displayName": "Workspaces",
"description": "Limit syncing to the specified workspaces, by deployment name, not workspace ID. Required when using workspace tokens, in the same order as workspace-tokens. Mutually exclusive with databricks-exclude-workspaces.",
"stringSliceField": {}
},
{
"name": "workspace-tokens",
"displayName": "Workspace Tokens",
"description": "The Databricks personal access tokens scoped to specific workspaces used to connect to the Databricks Workspace API",
"isRequired": true,
"isSecret": true,
"stringSliceField": {
"rules": {
"isRequired": true
}
}
},
{
"name": "databricks-exclude-workspaces",
"displayName": "Exclude Workspaces",
"description": "Workspaces to exclude from sync, identified by workspace name, deployment name, or numeric workspace ID",
"description": "Workspaces to exclude from sync, identified by workspace name, deployment name, or numeric workspace ID. Mutually exclusive with workspaces.",
"stringSliceField": {}
}
],
"constraints": [
{
"kind": "CONSTRAINT_KIND_MUTUALLY_EXCLUSIVE",
"fieldNames": [
"workspaces",
"databricks-exclude-workspaces"
]
},
{
"kind": "CONSTRAINT_KIND_DEPENDENT_ON",
"fieldNames": [
"workspace-tokens"
],
"secondaryFieldNames": [
"workspaces"
]
}
],
"displayName": "Databricks",
"helpUrl": "/docs/baton/databricks",
"iconUrl": "/static/app-icons/databricks.svg"
"iconUrl": "/static/app-icons/databricks.svg",
"fieldGroups": [
{
"name": "oauth2",
"displayName": "OAuth2",
"helpText": "Authenticate as a service principal using an OAuth2 client ID and secret.",
"fields": [
"account-id",
"databricks-client-id",
"databricks-client-secret",
"hostname",
"account-hostname",
"workspaces",
"base-url",
"databricks-exclude-workspaces"
],
"default": true
},
{
"name": "workspace-token",
"displayName": "Workspace token",
"helpText": "Authenticate with a personal access token scoped to each workspace.",
"fields": [
"account-id",
"workspaces",
"workspace-tokens",
"hostname",
"account-hostname",
"base-url",
"databricks-exclude-workspaces"
]
}
]
Comment on lines +189 to +220

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: this generated artifact is stale relative to pkg/config/config.go — both groups there now include ExcludeWorkspacesField, but neither group here lists databricks-exclude-workspaces, and the workspaces description here is the old one (config.go now says "by deployment name, not workspace ID … in the same order as workspace-tokens"). Re-run go generate ./pkg/config so the committed schema matches. Same staleness in the README.md flag dump for --workspaces, which isn't auto-regenerated.

🤖 via Claude Code

}
26 changes: 6 additions & 20 deletions docs/connector.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ A user with the **Account admin** role in each Databricks workspace you want to

### Generate Databricks credentials

You have three authentication choices when setting up the Databricks connector:
You have two authentication choices when setting up the Databricks connector:

- **OAuth** (syncs info from all Databricks workspaces)

Expand All @@ -69,10 +69,6 @@ You have three authentication choices when setting up the Databricks connector:
</Step>
</Steps>

- **Username and password** (syncs info from all Databricks workspaces)

You do not need to generate any additional credentials to use this method.

**Done.** Here's the set of credentials you'll need when setting up the connector:

- Account ID
Expand All @@ -83,13 +79,7 @@ OR

- Account ID
- Personal access token
- Workspace ID for the Databricks workspace you're syncing

OR

- Account ID
- Username
- Password
- Deployment name of the Databricks workspace you're syncing (the subdomain in the workspace URL, not the workspace ID)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Docs] Same "Username and password" gap as README.md

Same note as README.md: this page's auth-choices list (and the Kubernetes secret example further down) also still promises a "Username and password" auth method with BATON_USERNAME/BATON_PASSWORD that has no implementation anywhere in the code. Not this PR's job to fix, just flagging it since you're touching these exact docs for the PAT fix already.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, pulled the username/password bits from connector.mdx too.


Next, move on to the instructions for your chosen setup method.

Expand Down Expand Up @@ -132,13 +122,13 @@ To complete this task, you'll need:
Find the **Settings** area of the page and click **Edit**.
</Step>
<Step>
Select whether you're authenticating with **OAuth**, a **Personal access token**, or your **Username and password**.
Select whether you're authenticating with **OAuth** or a **Personal access token**.
</Step>
<Step>
Paste the account ID you looked up in Step 1 into the **Account ID** field.
</Step>
<Step>
Enter the required OAuth, token, or username and password credentials into the other two fields.
Enter the required OAuth or token credentials into the other two fields.
</Step>
<Step>
**Google Cloud Platform and Azure Databricks customers only:** Enter your Databricks account hostname and hostname in the relevant fields.
Expand Down Expand Up @@ -226,13 +216,9 @@ stringData:

# Databricks credentials, option 2
BATON_ACCOUNT_ID: <Databricks account ID>
BATON_AUTH_METHOD: workspace-token
BATON_WORKSPACE_TOKENS: <Personal access token>
BATON_WORKSPACES: <Workspace ID for the Databricks workspace you're syncing>

# Databricks credentials, option 3
BATON_ACCOUNT_ID: <Databricks account ID>
BATON_USERNAME: <Username for the Databricks account>
BATON_PASSWORD: <Password for the Databricks account>
BATON_WORKSPACES: <Deployment name of the Databricks workspace you're syncing, not the workspace ID>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Option 2 needs BATON_AUTH_METHOD: workspace-token. Without it the SDK falls back to the default oauth2 field group (FieldGroupFields picks the group with Default: true), which marks databricks-client-id/databricks-client-secret required, so this documented config fails at startup. The README was updated with this caveat but connector.mdx wasn't.


# Optional: comma-separated workspaces to exclude from sync (workspace name, deployment name, or numeric ID)
BATON_DATABRICKS_EXCLUDE_WORKSPACES: <workspace-a,workspace-b>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: this commit adds field.WithConstraints(field.FieldsMutuallyExclusive(WorkspacesField, ExcludeWorkspacesField)), and the SDK enforces it globally (pkg/field/validation.govalidateConstraints, present > 1 → hard error), so setting BATON_WORKSPACES and BATON_DATABRICKS_EXCLUDE_WORKSPACES together now fails startup with fields marked as mutually exclusive were set. Option 2 above requires BATON_WORKSPACES, so a workspace-token user who follows this example verbatim can never use this "Optional" line. Reword it to say the exclude list only applies to OAuth syncs that do not set BATON_WORKSPACES.

Expand Down
2 changes: 2 additions & 0 deletions pkg/config/conf.gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading