Skip to content

configuration.md

Latest commit

 

History

History
110 lines (83 loc) · 3.86 KB

configuration.md

File metadata and controls

110 lines (83 loc) · 3.86 KB

Configuration Reference

Environment Variables (.env)

The app is configured via control-plane-app/.env. Copy .env.example to get started:

cp control-plane-app/.env.example control-plane-app/.env

Required

Variable Description Example
LAKEBASE_DNS Lakebase instance hostname ep-xxxx.database.us-east-1.cloud.databricks.com
LAKEBASE_DATABASE Lakebase database name control_plane

Plus exactly one of the following, depending on Lakebase mode:

Variable Mode Example
LAKEBASE_ENDPOINT_PATH Autoscaling (recommended) projects/ai-control-plane-db/branches/production/endpoints/primary
LAKEBASE_INSTANCE Provisioned (legacy) ai-control-plane-db

Optional

Variable Description Default
DATABRICKS_HOST Workspace URL (auto-detected in Databricks Apps) Auto-detected
DATABRICKS_TOKEN PAT for local development Not needed in Apps
DATABRICKS_ACCOUNT_ID Account ID for cross-workspace features None
CORS_ORIGINS Comma-separated allowed origins (local dev only) Empty (disabled)

Workflow Configuration (databricks.yml)

The discovery workflows are configured via workflows/databricks.yml. Each target defines the environment:

Target Variables

Variable Description Example
catalog Unity Catalog name for Delta tables main
schema Schema name control_plane
lakebase_dns Lakebase hostname (same as .env) ep-xxxx.database.us-east-1.cloud.databricks.com
lakebase_endpoint_path Autoscaling endpoint path (set this OR lakebase_instance) projects/<name>/branches/<branch>/endpoints/<endpoint>
lakebase_instance Provisioned instance name (set this OR lakebase_endpoint_path) ai-control-plane-db
warehouse_id SQL warehouse ID for system table queries xxxxxxxxxxxx
account_id Databricks account ID xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Example Target (Autoscaling)

targets:
  dev:
    mode: development
    default: true
    variables:
      catalog: my_catalog
      schema: control_plane
      lakebase_dns: "ep-xxxx.database.us-east-1.cloud.databricks.com"
      lakebase_endpoint_path: "projects/my-lakebase/branches/production/endpoints/primary"
      lakebase_instance: ""
      warehouse_id: "xxxxxxxxxxxx"
      account_id: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

For Provisioned Lakebase, leave lakebase_endpoint_path empty and set lakebase_instance.

Workflow Schedule

By default, the discovery workflow runs every 30 minutes. To change:

resources:
  jobs:
    agent_discovery:
      schedule:
        quartz_cron_expression: "0 0/15 * * * ?"  # every 15 min

App Configuration (app.yaml)

app.yaml is auto-generated by deploy.sh from .env values. Do not edit manually.

Key settings:

  • Uvicorn server on port 8000
  • Lakebase connection via environment variables
  • OBO auth via the resources block with permission: "all-apis"

Finding Your Values

Lakebase DNS & Instance Name

  1. Go to SQL > Lakebase in your workspace
  2. Click your instance
  3. Instance name: shown at the top
  4. DNS: shown in the connection details

SQL Warehouse ID

  1. Go to SQL > SQL Warehouses
  2. Click your warehouse
  3. The ID is in the URL: .../sql/warehouses/<warehouse-id>

Account ID

  • Found in your workspace URL: https://<workspace>.cloud.databricks.com/?o=<org-id>
  • Or in the Databricks Account Console

App Service Principal ID

  • After creating the app, go to the app detail page
  • The SP application ID is listed under the app settings
  • Needed for granting system.mlflow access (see installation guide)