diff --git a/docs.json b/docs.json index 86168884..fce8fc63 100644 --- a/docs.json +++ b/docs.json @@ -5933,7 +5933,8 @@ "v2.0.x-SNAPSHOT/how-to-guides/mcp/claude-code", "v2.0.x-SNAPSHOT/how-to-guides/mcp/goose", "v2.0.x-SNAPSHOT/how-to-guides/mcp/cursor", - "v2.0.x-SNAPSHOT/how-to-guides/mcp/vscode" + "v2.0.x-SNAPSHOT/how-to-guides/mcp/vscode", + "v2.0.x-SNAPSHOT/how-to-guides/mcp/databricks" ] }, "v2.0.x-SNAPSHOT/how-to-guides/mcp/semantic-search" diff --git a/v2.0.x-SNAPSHOT/deployment/semantic-search.mdx b/v2.0.x-SNAPSHOT/deployment/semantic-search.mdx index 7dd44e76..821fc99a 100644 --- a/v2.0.x-SNAPSHOT/deployment/semantic-search.mdx +++ b/v2.0.x-SNAPSHOT/deployment/semantic-search.mdx @@ -1,6 +1,6 @@ --- title: Enable Semantic Search | OpenMetadata Deployment Guide -description: Configure semantic search with vector embeddings in OpenMetadata to enable natural language queries against your metadata catalog using OpenSearch. +description: Configure semantic search with vector embeddings in OpenMetadata to enable natural language queries against your metadata catalog using OpenSearch or Elasticsearch. sidebarTitle: Enable Semantic Search --- @@ -8,7 +8,7 @@ sidebarTitle: Enable Semantic Search ## Prerequisites -- **OpenSearch** as your search backend (Elasticsearch is not supported) +- **OpenSearch or Elasticsearch** as your search backend - An external embedding provider: **OpenAI** or **AWS Bedrock**, or **DJL** for HuggingFace models. - Network access from the OpenMetadata server to the embedding provider API (unless using DJL) @@ -21,7 +21,7 @@ exact words don't appear in the metadata. -Semantic Search is currently supported only with **OpenSearch** as the search backend. +Semantic Search supports both **OpenSearch** and **Elasticsearch** as the search backend. The OpenSearch-specific hybrid search pipeline (which blends keyword and semantic scores server-side) is not available on Elasticsearch. Core vector/KNN search works the same on both backends. @@ -38,7 +38,7 @@ metadata catalog. The text is sent to the configured embedding provider to generate a numerical vector (embedding), which is stored - in a dedicated OpenSearch `vector_search_index` using the HNSW algorithm with cosine similarity. At query time, + in a dedicated `dataAssetEmbeddings` index (`_dataAssetEmbeddings` if you've set a cluster alias) using the HNSW algorithm with cosine similarity. At query time, the search text is also embedded and a KNN (K-Nearest Neighbor) similarity search finds the most relevant results. @@ -260,8 +260,8 @@ multiple text chunks. ### Semantic Search returns no results - Verify that `SEMANTIC_SEARCH_ENABLED` is set to `true` and the server has been restarted. -- Confirm that OpenSearch is your search backend (Elasticsearch is not supported). -- Check that the `vector_search_index` exists in OpenSearch. +- Confirm your search backend (OpenSearch or Elasticsearch) is reachable and correctly configured. +- Check that the `dataAssetEmbeddings` index (or `_dataAssetEmbeddings` if you've set a cluster alias) exists in your search backend. - Run a Reindex to generate embeddings for existing entities. ### Embedding generation fails diff --git a/v2.0.x-SNAPSHOT/how-to-guides/mcp/connect.mdx b/v2.0.x-SNAPSHOT/how-to-guides/mcp/connect.mdx index 10769383..7dafc141 100644 --- a/v2.0.x-SNAPSHOT/how-to-guides/mcp/connect.mdx +++ b/v2.0.x-SNAPSHOT/how-to-guides/mcp/connect.mdx @@ -10,10 +10,10 @@ OpenMetadata provides a Model Context Protocol (MCP) server that allows AI assis ## Server Information - **Server Name**: `openmetadata-mcp-stateless` -- **Version**: `0.11.2` +- **Version**: `1.1.0` - **Endpoint**: `{OMURL}/mcp` -- **Protocol**: Server-Sent Events (SSE) over HTTP -- **Authentication**: JWT Bearer Token +- **Protocol**: Streamable HTTP (MCP spec `2025-03-26`) +- **Authentication**: OAuth 2.0 (recommended) or JWT Bearer Token ## Connection Setup @@ -26,9 +26,12 @@ Your MCP server is available at: Replace `{OMURL}` with your OpenMetadata instance URL (e.g., `https://your-openmetadata.com/mcp`) ### 2. Authentication -The MCP server requires [JWT authentication](/v2.0.x-SNAPSHOT/how-to-guides/mcp#personal-access-token-pat). Include your token in the Authorization header: +The MCP server supports two authentication methods: + +- **OAuth 2.0 (recommended)**: Sign in with your existing OpenMetadata login. See [OAuth 2.0 Authentication](/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth) for setup instructions. +- **Personal Access Token (PAT)**: For environments where browser-based login isn't available. Generate a token as described in [Personal Access Token (PAT)](/v2.0.x-SNAPSHOT/how-to-guides/mcp#personal-access-token-pat), then include it in the Authorization header: ```http -Authorization: Bearer +Authorization: Bearer ``` ### 3. Content Type @@ -49,7 +52,7 @@ Content-Type: application/json "id": 1, "method": "initialize", "params": { - "protocolVersion": "2024-11-05", + "protocolVersion": "2025-03-26", "capabilities": { "tools": {}, "prompts": {}, @@ -72,7 +75,7 @@ Content-Type: application/json "jsonrpc": "2.0", "id": 1, "result": { - "protocolVersion": "2024-11-05", + "protocolVersion": "2025-03-26", "capabilities": { "tools": true, "prompts": true, @@ -84,7 +87,7 @@ Content-Type: application/json }, "serverInfo": { "name": "openmetadata-mcp-stateless", - "version": "0.11.2" + "version": "1.1.0" } } } @@ -102,7 +105,8 @@ Content-Type: application/json } ``` -**Sample Response**: +**Sample Response** (truncated — the server exposes 24 tools; see the [MCP Tools Reference](/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference) for the complete list): + ```json { "jsonrpc": "2.0", @@ -111,7 +115,7 @@ Content-Type: application/json "tools": [ { "name": "search_metadata", - "description": "Find your data and business terms in OpenMetadata. For example if the user asks to 'find tables that contain customers information', then 'customers' should be the query, and the entity_type should be 'table'. Here make sure to use 'Href' is available in result to create a hyperlink to the entity in OpenMetadata.", + "description": "Find your data and business terms in OpenMetadata.", "inputSchema": { "type": "object", "properties": { @@ -119,29 +123,24 @@ Content-Type: application/json "type": "string", "description": "Keywords to use for searching." }, - "entity_type": { + "entityType": { "type": "string", - "description": "Optional entity type to filter results..." + "description": "Optional entity type to filter results." }, - "limit": { + "size": { "type": "integer", "description": "Maximum number of results to return. Default is 10." - }, - "fields": { - "type": "string", - "description": "Comma-separated list of additional fields to include..." } - }, - "required": ["query"] + } } }, { "name": "get_entity_details", - "description": "Get detailed information about a specific entity", + "description": "Retrieve full details for a specific entity by FQN.", "inputSchema": { "type": "object", "properties": { - "entity_type": { + "entityType": { "type": "string", "description": "Type of entity" }, @@ -150,7 +149,7 @@ Content-Type: application/json "description": "Fully qualified name of the entity" } }, - "required": ["entity_type", "fqn"] + "required": ["entityType", "fqn"] } } ] @@ -171,27 +170,13 @@ Content-Type: application/json ``` **Sample Response**: + ```json { "jsonrpc": "2.0", "id": 3, "result": { "prompts": [ - { - "name": "create-greeting", - "description": "Generate a customized greeting message", - "arguments": [ - { - "name": "name", - "description": "Name of the person to greet", - "required": true - }, - { - "name": "style", - "description": "The style of greeting, such as formal, excited, or casual. If not specified casual will be used" - } - ] - }, { "name": "search_metadata", "description": "Creates a prompt for Searching metadata in OpenMetadata.", diff --git a/v2.0.x-SNAPSHOT/how-to-guides/mcp/databricks.mdx b/v2.0.x-SNAPSHOT/how-to-guides/mcp/databricks.mdx new file mode 100644 index 00000000..b4be80ac --- /dev/null +++ b/v2.0.x-SNAPSHOT/how-to-guides/mcp/databricks.mdx @@ -0,0 +1,152 @@ +--- +title: Getting Started with Databricks +description: Register your OpenMetadata MCP Server as a governed Unity Catalog MCP Service so Databricks agents can discover and call its tools. +sidebarTitle: Databricks +--- + +# Getting Started with Databricks + +Connect your OpenMetadata instance to Databricks by registering it as an MCP Service in Unity Catalog. Once registered, Databricks agents and AI assistants (including Databricks Assistant and Mosaic AI Agent Framework agents) can discover and call OpenMetadata's MCP tools as a governed, auditable Unity Catalog resource. + +Unlike client-side integrations such as Claude or Cursor, Databricks doesn't connect directly to the OpenMetadata MCP endpoint from a local config file. Instead, an admin registers the server once as a Unity Catalog securable, and Databricks proxies and governs every call through its Unity AI Gateway. + +## Prerequisites + +Before you begin, ensure you have: + +- **OpenMetadata v1.8.0+** running. +- **MCP Server** application installed in OpenMetadata. + 1. Navigate to **Settings** > **Application** and click **Add Apps**. + 2. Install **MCP Server** application if not already installed. +- An OpenMetadata [Personal Access Token](/v2.0.x-SNAPSHOT/how-to-guides/mcp#personal-access-token-pat), or an OpenMetadata user account to sign in with if you want per-user OAuth authentication instead. +- A **Databricks workspace with Unity Catalog enabled**, in a region that supports Model Serving. +- Sufficient Unity Catalog privileges to create connections and MCP Services in the target catalog and schema. + +## Configure the MCP Service + +### Step 1: Create a Unity Catalog HTTP Connection + +To create a Unity Catalog HTTP connection in Databricks, follow these steps: + +1. Navigate to **Catalog > Connections**. +2. Click **Create connection**. +3. Set **Connection type** to `HTTP`. +4. Set **Host / MCP server URL** to `https:///mcp`. +5. Set **Authentication** to `Bearer token`, and paste your OpenMetadata Personal Access Token. +6. Enable the **Is MCP connection** checkbox so Databricks treats the endpoint as an MCP server rather than a generic HTTP API. +7. Click **Create** to save the connection. + + +If your OpenMetadata deployment authenticates with OAuth 2.0 instead of a PAT, use **OAuth U2M** (per-user login) as the authentication type instead of Bearer token. OpenMetadata's MCP OAuth server only issues `authorization_code` and `refresh_token` grants. It doesn't support `client_credentials` (OAuth M2M), so that option isn't available here. For a shared, non-interactive credential, use a Personal Access Token instead. See [OAuth 2.0 Authentication](/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth) for the values OpenMetadata expects. + + +If you selected **OAuth U2M**, each user must authenticate individually before they can call the service: + +1. Open the connection's detail page in Catalog Explorer. +2. Click **Login**. +3. Complete the OAuth consent flow against your OpenMetadata instance. + +Bearer token skips this step: it uses a single shared credential set once, at connection creation, so no per-user login is required. + +### Step 2: Register the MCP Service + +Register the MCP Service through the Databricks UI or the REST API. + +#### Using the Databricks UI + +To register the MCP service in Databricks, follow these steps: + +1. In the left sidebar, select **AI Gateway > MCPs** > **Register MCP Server**. + + Alternatively, go to **Catalog**, open the target schema, and select **Create > MCP Service**, which opens the same form. + +3. Set **Catalog** and **Schema** to where the MCP Service will live as a Unity Catalog securable. +4. Set **MCP Service name**, for example `openmetadata` (this can't be changed after creation). +5. Set **Connection** to the HTTP connection you created in Step 1. +6. Optional: Set **Tools** to the OpenMetadata MCP tools you want to expose. Leave unrestricted to expose all of them. See the [MCP Tools Reference](/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference) for the full list. +7. Click **Create** to register the service. + +#### Using the REST API + +Register the service directly with a POST request: + +```bash +databricks api post \ + "/api/2.1/unity-catalog/mcp-services?parent=schemas/main.default&mcp_service_id=openmetadata" \ + --json '{ + "comment": "OpenMetadata MCP Server", + "config": { + "source_connection": { + "name": "connections/main.default.openmetadata_http" + }, + "include_tool_selectors": [] + } + }' +``` + +### Step 3: Grant EXECUTE Permission + +On the MCP Service's **Permissions** tab, grant **EXECUTE** to the users, groups, or service principals that should be able to call OpenMetadata's tools: + +```bash +databricks api patch \ + "/api/2.1/unity-catalog/permissions/mcp_service/main.default.openmetadata" \ + --json '{ + "changes": [ + { "principal": "data-team", "add": ["EXECUTE"] } + ] + }' +``` + + +Grant **EXECUTE** on the MCP Service itself. Never grant **USE CONNECTION** on the underlying HTTP connection to end users. That would let them bypass tool selection and call the OpenMetadata MCP endpoint directly with the shared credential. + + +### Step 4: Connect an Agent or Client + +After getting registered, the MCP Service is reachable at: + +``` +https:///ai-gateway/mcp-services/.. +``` + +Mosaic AI Agent Framework agents can add it as a tool source directly. To verify the connection manually, authenticate with a Databricks token and list the available tools: + +```bash +TOKEN=$(databricks auth token | jq -r .access_token) +curl -s -X POST \ + "https:///ai-gateway/mcp-services/main.default.openmetadata" \ + -H "Authorization: Bearer $TOKEN" \ + -H "Accept: application/json, text/event-stream" \ + -H "Content-Type: application/json" \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' +``` + +You should see OpenMetadata's MCP tools (`search_metadata`, `get_entity_details`, and others) in the response. See the [MCP Tools Reference](/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference) for the complete list. + +## Try It Out + +Once your agent has the OpenMetadata MCP Service attached as a tool source, try prompting it: + +> "What tables do you have access to in OpenMetadata?" + +> "Find the tables that track customer transactions, engagement metrics, and churn indicators." + +## Troubleshooting + +If you run into connection issues: + +1. **Verify OpenMetadata is running**: `curl /api/health`. +2. **Check the MCP endpoint**: `curl /mcp` (should return `401`). +3. **Verify MCP Application is installed**: Visit `/marketplace/apps/McpApplication`. +4. **Confirm the connection is marked as an MCP connection**: Open the HTTP connection in Catalog Explorer and check that **Is MCP connection** is enabled. +5. **Confirm EXECUTE is granted**: Check the MCP Service's Permissions tab for the calling user, group, or service principal. +6. **Token expired**: [Generate a new PAT](/v2.0.x-SNAPSHOT/how-to-guides/guide-for-data-users/personal-access-token) and update the HTTP connection's credentials. + +## Additional Resources + +- [OpenMetadata MCP Documentation](/v2.0.x-SNAPSHOT/how-to-guides/mcp) +- [OAuth 2.0 Authentication](/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth) +- [MCP Tools Reference](/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference) +- [Databricks: Connect agents to third-party tools with MCP Services](https://docs.databricks.com/aws/en/agents/agent-framework/mcp-services) +- [Databricks: Register an external MCP server](https://docs.databricks.com/aws/en/ai-gateway/register-mcp-service) diff --git a/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth.mdx b/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth.mdx index 9d9291a3..c47b382b 100644 --- a/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth.mdx +++ b/v2.0.x-SNAPSHOT/how-to-guides/mcp/oauth.mdx @@ -92,9 +92,7 @@ OAuth tokens are handled entirely by your MCP client with no manual management n - **Sessions stay active** as long as you're using the MCP client regularly - **Re-authentication** is only needed if your refresh token expires after an extended period of inactivity (30 days) -To **revoke access** for an MCP client, an administrator can manage active sessions from the OpenMetadata admin settings. - -{/* TODO: Add screenshot of token/session management if available in admin UI */} +To **revoke access** for an MCP client, use the OAuth 2.0 Token Revocation endpoint described under [Revoking Access](#revoking-access) below. ## Security @@ -128,3 +126,76 @@ Set up OAuth authentication with your preferred MCP client: Connect via Block's open-source AI agent. + +## How the Connection Works (Under the Hood) + +OpenMetadata's MCP OAuth implementation uses **OAuth 2.0 Dynamic Client Registration** (RFC 7591), so MCP clients connect without any manual app setup: + +1. The MCP client fetches OpenMetadata's OAuth discovery document at `/.well-known/oauth-authorization-server` to learn the authorization, token, and registration endpoints. +2. The client automatically registers itself by posting its metadata to the registration endpoint. OpenMetadata issues a `client_id` in response — no admin action is required. +3. The client initiates an **Authorization Code flow with PKCE** (SHA-256), opening your browser to sign in via your configured SSO provider or basic auth. +4. After you sign in, OpenMetadata redirects back to the client with an authorization code. +5. The client exchanges the code for an access token and a refresh token. +6. All subsequent MCP tool calls include the access token. When it expires, the client uses the refresh token to get a new one silently. + +### Discovery Endpoints + +| Endpoint | Purpose | +|----------|---------| +| `GET /.well-known/oauth-authorization-server` | OAuth 2.0 authorization server metadata (RFC 8414) | +| `GET /.well-known/openid-configuration` | OAuth metadata compatibility alias — serves the same RFC 8414 metadata as above for clients that only check this OIDC-conventional path (not a true OpenID Connect discovery document) | +| `POST /mcp/register` | Dynamic client registration | + +## Token Lifetimes + +OAuth tokens are handled entirely by your MCP client with no manual management needed. + +| Token type | Lifetime | Notes | +|-----------|---------|-------| +| **Access token** | 10 minutes | Short-lived to limit exposure. Automatically refreshed by the client | +| **Authorization code** | 10 minutes | Single-use. Expires if the login flow stalls | +| **Refresh token** | 30 days | Grants new access tokens without re-authentication. Expires after 30 days of inactivity | + +Re-authentication is only required when the refresh token expires (30-day inactivity window). To **revoke access** for an MCP client, use the OAuth 2.0 Token Revocation endpoint described under [Revoking Access](#revoking-access) below. + +## Rate Limits + +The MCP OAuth endpoints are rate-limited per IP address to prevent abuse: + +| Endpoint | Limit | +|---------|-------| +| `/mcp/register` (client registration) | 10 requests per hour | +| `/mcp/token` (token requests) | 30 requests per minute | + +These limits are per-server-instance. In clustered deployments, the effective limit is multiplied by the number of instances. + +## Allowed Origins (CORS) + +By default, the MCP Server allows CORS requests only from a small set of local development origins (`http://localhost:3000`, `http://localhost:8585`, `http://localhost:9090`) — it does not allow all origins. This allowlist is part of OpenMetadata's MCP settings, stored in the database rather than the static server configuration file. Update it from the MCP application's configuration page (`/marketplace/apps/McpApplication`) or via the `GET`/`PUT /api/v1/system/mcp/config` API (admin only). + +Only requests whose `Origin` header exactly matches an entry in the allowlist receive a valid `Access-Control-Allow-Origin` response header — there is no wildcard or prefix matching. This is relevant for browser-based MCP clients or custom integrations that call the MCP endpoint directly from a web page. + +## Revoking Access + +MCP clients can revoke an access or refresh token directly using the OAuth 2.0 Token Revocation endpoint ([RFC 7009](https://datatracker.ietf.org/doc/html/rfc7009)), advertised as `revocation_endpoint` in the OAuth discovery document. A `POST` request to `{baseUrl}/mcp/revoke` invalidates the token immediately. There is currently no separate admin-side option to revoke an MCP client's access. Revocation is a client-initiated action using its registered credentials. + +The revocation endpoint requires client authentication before it processes the request, using the `client_id` and `client_secret` issued during [Dynamic Client Registration](#how-the-connection-works-under-the-hood). Authenticate using either method: + +**HTTP Basic authentication** (client credentials in the `Authorization` header): +```http +POST {baseUrl}/mcp/revoke +Authorization: Basic +Content-Type: application/x-www-form-urlencoded + +token=&token_type_hint=access_token +``` + +**Form-based authentication** (client credentials in the request body): +```http +POST {baseUrl}/mcp/revoke +Content-Type: application/x-www-form-urlencoded + +token=&token_type_hint=access_token&client_id=&client_secret= +``` + +`token_type_hint` is optional and can be `access_token` or `refresh_token`. A request without valid client credentials returns `invalid_client`. diff --git a/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference.mdx b/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference.mdx index 25a5098f..e410200a 100644 --- a/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference.mdx +++ b/v2.0.x-SNAPSHOT/how-to-guides/mcp/reference.mdx @@ -9,10 +9,13 @@ All OpenMetadata MCP tools, with parameters and examples. ## Available Tools +Tools are labeled **Read** (retrieve data only) or **Write** (create or modify data). + | Category | Tools | |---|---| | **Discover** | [search_metadata](#search_metadata), [semantic_search](#semantic_search), [search_company_context](#search_company_context) | | **Inspect** | [get_entity_details](#get_entity_details), [get_company_context](#get_company_context) | +| **Context** | [get_user_context](#get_user_context), [get_persona_context](#get_persona_context), [get_asset_context](#get_asset_context), [find_context](#find_context), [get_knowledge_content](#get_knowledge_content) | | **Lineage & Impact** | [get_entity_lineage](#get_entity_lineage), [create_lineage](#create_lineage), [root_cause_analysis](#root_cause_analysis) | | **Knowledge** | [create_glossary](#create_glossary), [create_glossary_term](#create_glossary_term), [create_context_memory](#create_context_memory) | | **Govern & Classify** | [create_classification](#create_classification), [create_tag](#create_tag), [create_domain](#create_domain), [create_data_product](#create_data_product), [patch_entity](#patch_entity) | @@ -53,6 +56,7 @@ Search and find data assets across your catalog using keyword, semantic, or natu - **User Entities**: `user`, `team` - **Domain Entities**: `domain`, `dataProduct` - **Governance Entities**: `metric`, `glossary`, `glossaryTerm` +- **Data Quality Entities**: `testCase`, `testSuite`, `testCaseResult` **Examples** @@ -126,6 +130,10 @@ Search and find data assets across your catalog using keyword, semantic, or natu } ``` + +Results include a `similarityScore` field for each entity whenever the search backend returns a relevance score for that hit. This is the backend's raw, opaque score (not a value normalized to a 0.0–1.0 range), so absolute values vary by query and are not comparable across searches — only relative ordering within the same result set is meaningful. Higher scores indicate stronger relevance. + + ### semantic_search **Description**: Find data assets by meaning using vector search ([setup guide](../../deployment/semantic-search#enable-semantic-search)). Use for exploratory queries where you don't know exact names. Returns conceptually related assets even when no keywords match. @@ -219,7 +227,7 @@ Retrieve detailed information about specific entities and company knowledge pill ### get_entity_details -**Description**: Retrieve full details for a specific entity by fully qualified name (FQN). Pass the exact `fullyQualifiedName` value from search results — do not construct FQNs manually. +**Description**: Retrieve full details for a specific entity by fully qualified name (FQN). Pass the exact `fullyQualifiedName` value from search results — do not construct FQNs manually. The response includes an `extension` field containing any custom properties defined for the entity type. **Parameters** @@ -307,6 +315,185 @@ Retrieve detailed information about specific entities and company knowledge pill } ``` +## Context + +Retrieve context knowledge associated with the current user, a persona, or a specific data asset. Context tools are all **Read** operations. + +### get_user_context + +**Type**: Read + +**Description**: Returns context about the currently authenticated user — identity (ID, name, display name, email, admin/bot flags), team memberships, roles (direct and team-inherited), domains, active persona, and lightweight summaries of entities the user owns and follows. Use this to answer identity questions such as "what is my role?" or "what do I own?" before asking the user for details. The user is resolved from the authenticated request — there is no parameter to look up another user. + +**Parameters** + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `includeOwnedSummary` | boolean | No | Include a summary of entities owned by the user and their teams (default: `true`) | +| `includeFollowedSummary` | boolean | No | Include a summary of entities the user follows (default: `true`) | +| `ownedLimit` | integer | No | Max owned entities to return, 1–100 (default: 20) | +| `followedLimit` | integer | No | Max followed entities to return, 1–100 (default: 20) | +| `ownedFilter` | string | No | Narrow owned entities to ones worth surfacing: `NONE` (default, all owned), `MISSING_DESCRIPTION`, `MISSING_TIER`, or `ANY_GAP`. Gap filters apply to data-asset entity types only | + +**Example** + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": { + "name": "get_user_context", + "arguments": { + "ownedFilter": "ANY_GAP" + } + } +} +``` + +**Sample Response** + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": { + "content": [ + { + "type": "text", + "text": "## Current User\n\n**Name**: Jane Doe\n**Email**: jane.doe@company.com\n**Persona**: Data Analyst\n**Teams**: analytics-team, data-platform\n**Roles**: DataSteward" + } + ] + } +} +``` + +### get_persona_context + +**Type**: Read + +**Description**: Returns the shared AI context document curated for a persona — preferences, use cases, and runbook entries scoped to that persona's role. With no `personaName`, returns context for the caller's active persona. The document can span multiple parts. Call again with an incremented `part` while the response's `hasMore` is `true`. Access is limited to persona members, admins, and bots. + +**Parameters** + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `personaName` | string | No | Fully qualified name of the persona. Omit to use the caller's active persona | +| `format` | string | No | `markdown` (default) or `json` | +| `part` | integer | No | One-based document part to fetch. Start at 1 and continue while `hasMore` is `true` (default: 1) | + +**Example** + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": { + "name": "get_persona_context", + "arguments": { + "personaName": "Data Analyst" + } + } +} +``` + +### get_asset_context + +**Type**: Read + +**Description**: Returns the full AI context for a single data asset in one call: the business knowledge attached to it (approved glossary term definitions, Context Center articles) plus type-specific structural context. For tables this includes the column schema, primary key, foreign keys (with the referenced columns and cardinality), and the columns most frequently joined with other tables. Prefer this over `get_entity_details` once you've selected an asset and need its business rules and join keys. Use the `fullyQualifiedName` and `entityType` values directly from search results. + +**Parameters** + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `entityType` | string | Yes | Type of the asset (e.g., `table`, `dashboard`, `pipeline`) | +| `fqn` | string | Yes | Fully qualified name of the asset — use the exact value from search results | +| `format` | string | No | `markdown` (default) returns a compact, LLM-friendly document. `json` returns the structured AIContext object | +| `query` | string | No | The user's question. When set, each truncated knowledge item is excerpted to the passage most relevant to this question instead of its opening | + +**Example** + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": { + "name": "get_asset_context", + "arguments": { + "entityType": "table", + "fqn": "mysql_prod.ecommerce.public.customer_orders" + } + } +} +``` + +### find_context + +**Type**: Read + +**Description**: For a business question where no specific asset has been chosen yet, semantically searches the company-knowledge layer — glossary term definitions, metric definitions, and Context Center articles — and returns the matching definitions plus the candidate data assets each concept points to (glossary terms → the tables tagged with them, metrics → the assets they apply to, and articles → the assets they're about). Use this to bootstrap a data question from business concepts into candidate tables, then call `get_asset_context` on those FQNs. This is a semantic search over company knowledge, not a keyword search over asset or persona memories. + +**Parameters** + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `query` | string | Yes | Natural-language question or business terms to find relevant company knowledge and candidate assets for | +| `size` | integer | No | Number of knowledge items to return (default: 10, max: 50) | +| `format` | string | No | `markdown` (default) returns definitions and candidate assets as an LLM-friendly document. `json` returns the structured result | + +**Example** + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": { + "name": "find_context", + "arguments": { + "query": "PII handling policy", + "size": 5 + } + } +} +``` + +### get_knowledge_content + +**Type**: Read + +**Description**: `get_asset_context` and `find_context` return bounded excerpts of long glossary definitions, metric definitions, and Context Center articles. When you need the full detail of one specific item, call this tool with that item's `entityType` and `fqn`. Without a `query`, the tool returns the full body. With a `query`, it returns only the passages most relevant to that query. + +**Parameters** + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `entityType` | string | Yes | Knowledge item type: `glossaryTerm`, `metric`, `page`, or `contextMemory` | +| `fqn` | string | Yes | Fully qualified name of the knowledge item, as returned by `get_asset_context` or `find_context` | +| `query` | string | No | When set, return only the passages most relevant to this query instead of the full body | +| `size` | integer | No | Number of relevant passages to return when `query` is set (default: 3, max: 10) | +| `format` | string | No | `markdown` (default) or `json` | + +**Example** + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": { + "name": "get_knowledge_content", + "arguments": { + "entityType": "page", + "fqn": "company-wiki.data-policies.pii-handling" + } + } +} +``` + ## Lineage & Impact Explore data dependencies, trace upstream sources, and analyze downstream impact. diff --git a/v2.0.x-SNAPSHOT/how-to-guides/mcp/semantic-search.mdx b/v2.0.x-SNAPSHOT/how-to-guides/mcp/semantic-search.mdx index 24b9456c..f6a8d33d 100644 --- a/v2.0.x-SNAPSHOT/how-to-guides/mcp/semantic-search.mdx +++ b/v2.0.x-SNAPSHOT/how-to-guides/mcp/semantic-search.mdx @@ -18,7 +18,7 @@ or `buyer_activity` that are semantically related but don't share the same keywo -Semantic Search requires **OpenSearch** as the search backend and must be [enabled in your deployment +Semantic Search requires **OpenSearch or Elasticsearch** as the search backend and must be [enabled in your deployment configuration](/v2.0.x-SNAPSHOT/deployment/semantic-search) before it can be used through MCP. @@ -30,7 +30,7 @@ When an AI assistant calls the Semantic Search tool through MCP: 1. The natural language query is sent to the OpenMetadata server's vector search endpoint (`/api/v1/search/vector/query`). 2. The query text is converted into a vector embedding using the configured embedding provider (OpenAI, AWS Bedrock, or DJL). -3. OpenSearch performs a KNN (K-Nearest Neighbor) similarity search against pre-computed entity embeddings. +3. OpenSearch or Elasticsearch performs a KNN (K-Nearest Neighbor) similarity search against pre-computed entity embeddings. 4. Results are deduplicated by entity and returned to the AI assistant with metadata including entity type, fully qualified name, owners, tags, and similarity score. @@ -133,7 +133,7 @@ Under the hood, when an AI assistant calls the semantic search tool, it sends an | Matching | Exact keyword and text matching | Meaning-based similarity matching | | Query style | Specific keywords and filters | Natural language questions | | Results | Documents containing the search terms | Conceptually related documents | -| Search backend | OpenSearch or Elasticsearch | OpenSearch only | +| Search backend | OpenSearch or Elasticsearch | OpenSearch or Elasticsearch | | Configuration | Available by default | Requires [enabling](/v2.0.x-SNAPSHOT/deployment/semantic-search) and an embedding provider | Both search methods are complementary. Keyword search is precise when you know the exact terms, while Semantic Search @@ -153,6 +153,6 @@ excels at discovery when you describe what you're looking for in natural languag - Ensure entity descriptions and metadata are well-populated -- richer metadata produces better embeddings. ### No results returned -- Check that the `vector_search_index` exists in OpenSearch and contains documents. +- Check that the `dataAssetEmbeddings` index (or `_dataAssetEmbeddings` if you've set a cluster alias) exists in your search backend and contains documents. - Verify the embedding provider is correctly configured and accessible. - Review the OpenMetadata server logs for errors related to vector search.