Docs: MCP Server — v2.0.x-SNAPSHOT OAuth, security, tools reference, and connection guide updates

docs.json

@@ -5933,7 +5933,8 @@
                       "v2.0.x-SNAPSHOT/how-to-guides/mcp/claude-code",
                       "v2.0.x-SNAPSHOT/how-to-guides/mcp/goose",
                       "v2.0.x-SNAPSHOT/how-to-guides/mcp/cursor",
-                      "v2.0.x-SNAPSHOT/how-to-guides/mcp/vscode"
+                      "v2.0.x-SNAPSHOT/how-to-guides/mcp/vscode",
+                      "v2.0.x-SNAPSHOT/how-to-guides/mcp/databricks"
                     ]
                   },
                   "v2.0.x-SNAPSHOT/how-to-guides/mcp/semantic-search"

v2.0.x-SNAPSHOT/deployment/semantic-search.mdx

@@ -1,14 +1,14 @@
 ---
 title: Enable Semantic Search | OpenMetadata Deployment Guide
-description: Configure semantic search with vector embeddings in OpenMetadata to enable natural language queries against your metadata catalog using OpenSearch.
+description: Configure semantic search with vector embeddings in OpenMetadata to enable natural language queries against your metadata catalog using OpenSearch or Elasticsearch.
 sidebarTitle: Enable Semantic Search
 ---
 
 # Enable Semantic Search
 
 ## Prerequisites
 
-- **OpenSearch** as your search backend (Elasticsearch is not supported)
+- **OpenSearch or Elasticsearch** as your search backend
 - An external embedding provider: **OpenAI** or **AWS Bedrock**, or **DJL** for HuggingFace models.
 - Network access from the OpenMetadata server to the embedding provider API (unless using DJL)
 
@@ -21,7 +21,7 @@ exact words don't appear in the metadata.
 
 <Info>
 
-Semantic Search is currently supported only with **OpenSearch** as the search backend.
+Semantic Search supports both **OpenSearch** and **Elasticsearch** as the search backend. The OpenSearch-specific hybrid search pipeline (which blends keyword and semantic scores server-side) is not available on Elasticsearch. Core vector/KNN search works the same on both backends.
 
 </Info>
 
@@ -38,7 +38,7 @@ metadata catalog.
   </Step>
   <Step title="Embedding Generation & Vector Indexing">
     The text is sent to the configured embedding provider to generate a numerical vector (embedding), which is stored
-    in a dedicated OpenSearch `vector_search_index` using the HNSW algorithm with cosine similarity. At query time,
+    in a dedicated `dataAssetEmbeddings` index (`<clusterAlias>_dataAssetEmbeddings` if you've set a cluster alias) using the HNSW algorithm with cosine similarity. At query time,
     the search text is also embedded and a KNN (K-Nearest Neighbor) similarity search finds the most relevant results.
   </Step>
   <Step title="Automatic Lifecycle Management">
@@ -260,8 +260,8 @@ multiple text chunks.
 
 ### Semantic Search returns no results
 - Verify that `SEMANTIC_SEARCH_ENABLED` is set to `true` and the server has been restarted.
-- Confirm that OpenSearch is your search backend (Elasticsearch is not supported).
-- Check that the `vector_search_index` exists in OpenSearch.
+- Confirm your search backend (OpenSearch or Elasticsearch) is reachable and correctly configured.
+- Check that the `dataAssetEmbeddings` index (or `<clusterAlias>_dataAssetEmbeddings` if you've set a cluster alias) exists in your search backend.
 - Run a Reindex to generate embeddings for existing entities.
 
 ### Embedding generation fails

v2.0.x-SNAPSHOT/how-to-guides/mcp/connect.mdx

@@ -10,10 +10,10 @@ OpenMetadata provides a Model Context Protocol (MCP) server that allows AI assis
 ## Server Information
 
 - **Server Name**: `openmetadata-mcp-stateless`
-- **Version**: `0.11.2`
+- **Version**: `1.1.0`
 - **Endpoint**: `{OMURL}/mcp`
-- **Protocol**: Server-Sent Events (SSE) over HTTP
-- **Authentication**: JWT Bearer Token
+- **Protocol**: Streamable HTTP (MCP spec `2025-03-26`)
+- **Authentication**: OAuth 2.0 (recommended) or JWT Bearer Token
 
 ## Connection Setup
 
@@ -26,9 +26,12 @@ Your MCP server is available at:
 Replace `{OMURL}` with your OpenMetadata instance URL (e.g., `https://your-openmetadata.com/mcp`)
 
 ### 2. Authentication
-The MCP server requires [JWT authentication](/v2.0.x-SNAPSHOT/how-to-guides/mcp#personal-access-token-pat). Include your token in the Authorization header:
+The MCP server supports two authentication methods:
+
+- **OAuth 2.0 (recommended)**: Sign in with your existing OpenMetadata login. See [OAuth 2.0 Authentication](/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth) for setup instructions.
+- **Personal Access Token (PAT)**: For environments where browser-based login isn't available. Generate a token as described in [Personal Access Token (PAT)](/v2.0.x-SNAPSHOT/how-to-guides/mcp#personal-access-token-pat), then include it in the Authorization header:
 ```http
-Authorization: Bearer <your-jwt-token>
+Authorization: Bearer <your-token>
 ```
 
 ### 3. Content Type
@@ -49,7 +52,7 @@ Content-Type: application/json
   "id": 1,
   "method": "initialize",
   "params": {
-    "protocolVersion": "2024-11-05",
+    "protocolVersion": "2025-03-26",
     "capabilities": {
       "tools": {},
       "prompts": {},
@@ -72,7 +75,7 @@ Content-Type: application/json
   "jsonrpc": "2.0",
   "id": 1,
   "result": {
-    "protocolVersion": "2024-11-05",
+    "protocolVersion": "2025-03-26",
     "capabilities": {
       "tools": true,
       "prompts": true,
@@ -84,7 +87,7 @@ Content-Type: application/json
     },
     "serverInfo": {
       "name": "openmetadata-mcp-stateless",
-      "version": "0.11.2"
+      "version": "1.1.0"
     }
   }
 }
@@ -102,7 +105,8 @@ Content-Type: application/json
 }
 ```
 
-**Sample Response**:
+**Sample Response** (truncated — the server exposes 24 tools; see the [MCP Tools Reference](/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference) for the complete list):
+
 ```json
 {
   "jsonrpc": "2.0",
@@ -111,37 +115,32 @@ Content-Type: application/json
     "tools": [
       {
         "name": "search_metadata",
-        "description": "Find your data and business terms in OpenMetadata. For example if the user asks to 'find tables that contain customers information', then 'customers' should be the query, and the entity_type should be 'table'. Here make sure to use 'Href' is available in result to create a hyperlink to the entity in OpenMetadata.",
+        "description": "Find your data and business terms in OpenMetadata.",
         "inputSchema": {
           "type": "object",
           "properties": {
             "query": {
               "type": "string",
               "description": "Keywords to use for searching."
             },
-            "entity_type": {
+            "entityType": {
               "type": "string",
-              "description": "Optional entity type to filter results..."
+              "description": "Optional entity type to filter results."
             },
-            "limit": {
+            "size": {
               "type": "integer",
               "description": "Maximum number of results to return. Default is 10."
-            },
-            "fields": {
-              "type": "string",
-              "description": "Comma-separated list of additional fields to include..."
             }
-          },
-          "required": ["query"]
+          }
         }
       },
       {
         "name": "get_entity_details",
-        "description": "Get detailed information about a specific entity",
+        "description": "Retrieve full details for a specific entity by FQN.",
         "inputSchema": {
           "type": "object",
           "properties": {
-            "entity_type": {
+            "entityType": {
               "type": "string",
               "description": "Type of entity"
             },
@@ -150,7 +149,7 @@ Content-Type: application/json
               "description": "Fully qualified name of the entity"
             }
           },
-          "required": ["entity_type", "fqn"]
+          "required": ["entityType", "fqn"]
         }
       }
     ]
@@ -171,27 +170,13 @@ Content-Type: application/json
 ```
 
 **Sample Response**:
+
 ```json
 {
   "jsonrpc": "2.0",
   "id": 3,
   "result": {
     "prompts": [
-      {
-        "name": "create-greeting",
-        "description": "Generate a customized greeting message",
-        "arguments": [
-          {
-            "name": "name",
-            "description": "Name of the person to greet",
-            "required": true
-          },
-          {
-            "name": "style",
-            "description": "The style of greeting, such as formal, excited, or casual. If not specified casual will be used"
-          }
-        ]
-      },
       {
         "name": "search_metadata",
         "description": "Creates a prompt for Searching metadata in OpenMetadata.",

v2.0.x-SNAPSHOT/how-to-guides/mcp/databricks.mdx

@@ -0,0 +1,152 @@
+---
+title: Getting Started with Databricks
+description: Register your OpenMetadata MCP Server as a governed Unity Catalog MCP Service so Databricks agents can discover and call its tools.
+sidebarTitle: Databricks
+---
+
+# Getting Started with Databricks
+
+Connect your OpenMetadata instance to Databricks by registering it as an MCP Service in Unity Catalog. Once registered, Databricks agents and AI assistants (including Databricks Assistant and Mosaic AI Agent Framework agents) can discover and call OpenMetadata's MCP tools as a governed, auditable Unity Catalog resource.
+
+Unlike client-side integrations such as Claude or Cursor, Databricks doesn't connect directly to the OpenMetadata MCP endpoint from a local config file. Instead, an admin registers the server once as a Unity Catalog securable, and Databricks proxies and governs every call through its Unity AI Gateway.
+
+## Prerequisites
+
+Before you begin, ensure you have:
+
+- **OpenMetadata v1.8.0+** running.
+- **MCP Server** application installed in OpenMetadata.
+   1. Navigate to **Settings** > **Application** and click **Add Apps**.
+   2. Install **MCP Server** application if not already installed.
+- An OpenMetadata [Personal Access Token](/v2.0.x-SNAPSHOT/how-to-guides/mcp#personal-access-token-pat), or an OpenMetadata user account to sign in with if you want per-user OAuth authentication instead.
+- A **Databricks workspace with Unity Catalog enabled**, in a region that supports Model Serving.
+- Sufficient Unity Catalog privileges to create connections and MCP Services in the target catalog and schema.
+
+## Configure the MCP Service
+
+### Step 1: Create a Unity Catalog HTTP Connection
+
+To create a Unity Catalog HTTP connection in Databricks, follow these steps:
+
+1. Navigate to **Catalog > Connections**.
+2. Click **Create connection**.
+3. Set **Connection type** to `HTTP`.
+4. Set **Host / MCP server URL** to `https://<YOUR-OpenMetadata-SERVER>/mcp`.
+5. Set **Authentication** to `Bearer token`, and paste your OpenMetadata Personal Access Token.
+6. Enable the **Is MCP connection** checkbox so Databricks treats the endpoint as an MCP server rather than a generic HTTP API.
+7. Click **Create** to save the connection.
+
+<Tip>
+If your OpenMetadata deployment authenticates with OAuth 2.0 instead of a PAT, use **OAuth U2M** (per-user login) as the authentication type instead of Bearer token. OpenMetadata's MCP OAuth server only issues `authorization_code` and `refresh_token` grants. It doesn't support `client_credentials` (OAuth M2M), so that option isn't available here. For a shared, non-interactive credential, use a Personal Access Token instead. See [OAuth 2.0 Authentication](/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth) for the values OpenMetadata expects.
+</Tip>
+
+If you selected **OAuth U2M**, each user must authenticate individually before they can call the service:
+
+1. Open the connection's detail page in Catalog Explorer.
+2. Click **Login**.
+3. Complete the OAuth consent flow against your OpenMetadata instance.
+
+Bearer token skips this step: it uses a single shared credential set once, at connection creation, so no per-user login is required.
+
+### Step 2: Register the MCP Service
+
+Register the MCP Service through the Databricks UI or the REST API.
+
+#### Using the Databricks UI
+
+To register the MCP service in Databricks, follow these steps:
+
+1. In the left sidebar, select **AI Gateway > MCPs** > **Register MCP Server**.
+
+    Alternatively, go to **Catalog**, open the target schema, and select **Create > MCP Service**, which opens the same form.
+
+3. Set **Catalog** and **Schema** to where the MCP Service will live as a Unity Catalog securable.
+4. Set **MCP Service name**, for example `openmetadata` (this can't be changed after creation).
+5. Set **Connection** to the HTTP connection you created in Step 1.
+6. Optional: Set **Tools** to the OpenMetadata MCP tools you want to expose. Leave unrestricted to expose all of them. See the [MCP Tools Reference](/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference) for the full list.
+7. Click **Create** to register the service.
+
+#### Using the REST API
+
+Register the service directly with a POST request:
+
+```bash
+databricks api post \
+  "/api/2.1/unity-catalog/mcp-services?parent=schemas/main.default&mcp_service_id=openmetadata" \
+  --json '{
+    "comment": "OpenMetadata MCP Server",
+    "config": {
+      "source_connection": {
+        "name": "connections/main.default.openmetadata_http"
+      },
+      "include_tool_selectors": []
+    }
+  }'
+```
+
+### Step 3: Grant EXECUTE Permission
+
+On the MCP Service's **Permissions** tab, grant **EXECUTE** to the users, groups, or service principals that should be able to call OpenMetadata's tools:
+
+```bash
+databricks api patch \
+  "/api/2.1/unity-catalog/permissions/mcp_service/main.default.openmetadata" \
+  --json '{
+    "changes": [
+      { "principal": "data-team", "add": ["EXECUTE"] }
+    ]
+  }'
+```
+
+<Warning>
+Grant **EXECUTE** on the MCP Service itself. Never grant **USE CONNECTION** on the underlying HTTP connection to end users. That would let them bypass tool selection and call the OpenMetadata MCP endpoint directly with the shared credential.
+</Warning>
+
+### Step 4: Connect an Agent or Client
+
+After getting registered, the MCP Service is reachable at:
+
+```
+https://<YOUR-DATABRICKS-WORKSPACE>/ai-gateway/mcp-services/<catalog>.<schema>.<service-name>
+```
+
+Mosaic AI Agent Framework agents can add it as a tool source directly. To verify the connection manually, authenticate with a Databricks token and list the available tools:
+
+```bash
+TOKEN=$(databricks auth token | jq -r .access_token)
+curl -s -X POST \
+  "https://<YOUR-DATABRICKS-WORKSPACE>/ai-gateway/mcp-services/main.default.openmetadata" \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Accept: application/json, text/event-stream" \
+  -H "Content-Type: application/json" \
+  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'
+```
+
+You should see OpenMetadata's MCP tools (`search_metadata`, `get_entity_details`, and others) in the response. See the [MCP Tools Reference](/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference) for the complete list.
+
+## Try It Out
+
+Once your agent has the OpenMetadata MCP Service attached as a tool source, try prompting it:
+
+> "What tables do you have access to in OpenMetadata?"
+
+> "Find the tables that track customer transactions, engagement metrics, and churn indicators."
+
+## Troubleshooting
+
+If you run into connection issues:
+
+1. **Verify OpenMetadata is running**: `curl <YOUR-OpenMetadata-SERVER>/api/health`.
+2. **Check the MCP endpoint**: `curl <YOUR-OpenMetadata-SERVER>/mcp` (should return `401`).
+3. **Verify MCP Application is installed**: Visit `<YOUR-OpenMetadata-SERVER>/marketplace/apps/McpApplication`.
+4. **Confirm the connection is marked as an MCP connection**: Open the HTTP connection in Catalog Explorer and check that **Is MCP connection** is enabled.
+5. **Confirm EXECUTE is granted**: Check the MCP Service's Permissions tab for the calling user, group, or service principal.
+6. **Token expired**: [Generate a new PAT](/v2.0.x-SNAPSHOT/how-to-guides/guide-for-data-users/personal-access-token) and update the HTTP connection's credentials.
+
+## Additional Resources
+
+- [OpenMetadata MCP Documentation](/v2.0.x-SNAPSHOT/how-to-guides/mcp)
+- [OAuth 2.0 Authentication](/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth)
+- [MCP Tools Reference](/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference)
+- [Databricks: Connect agents to third-party tools with MCP Services](https://docs.databricks.com/aws/en/agents/agent-framework/mcp-services)
+- [Databricks: Register an external MCP server](https://docs.databricks.com/aws/en/ai-gateway/register-mcp-service)