diff --git a/README.md b/README.md index 1573af7..89663a7 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,10 @@ StatGPT bridges the gap between complex statistical databases and everyday users - [Architecture Overview](./architecture/overview.md) - Solution overview and core requirements - [SDMX Compatibility](./architecture/sdmx-compatibility.md) - SDMX integration details +### [Application MCP](./architecture/mcp.md) +Agentic access via the Model Context Protocol — channel tools surfaced to AI agents (Claude, Cursor, MCP-aware DIAL +applications). Covers DIAL Application registration, MCP-spec discovery, and IDP OAuth flow in one page. + ### Guides - [Admin Guide](./guides/admin-guide.md) - System administration and configuration - [GTDC Portal Guide](./guides/gtdc-portal-guide.md) - Instructions for using the GTDC (Global Trusted Data Commons) Portal diff --git a/architecture/README.md b/architecture/README.md index b8326c5..ee715de 100644 --- a/architecture/README.md +++ b/architecture/README.md @@ -12,9 +12,10 @@ design, services, tools, and integration requirements. | **[📋 Overview](./overview.md)** | Complete platform overview with requirements and features | • Natural language querying
• Data accuracy & reliability
• Security & governance
• Performance & scalability | | **[🏭 Services](./services.md)** | Core services and dependencies architecture | • Chat Backend (DIAL app)
• Admin Backend & Frontend
• Portal Frontend
• Third-party integrations | | **[🔧 Tools](./tools.md)** | Agent tools and capabilities documentation | • Data query tools
• Publications RAG
• Glossary management
• Web search integration | +| **[🔌 Application MCP](./mcp.md)** | Agentic access via Model Context Protocol | • Channel tools surfaced to AI agents
• DIAL Application registration
• MCP-spec discovery + IDP OAuth flow | | **[📊 SDMX Compatibility](./sdmx-compatibility.md)** | SDMX standards and requirements guide | • Version support (2.1/3.0)
• Metadata requirements
• Performance standards
• Quality checklist | -### Design Documentation +### Design Documentation/res | Document | Description | Key Topics | |-----------------------------------|-----------------------------------------------|-----------------------------------------------------------------------------------| @@ -100,7 +101,3 @@ All architecture documentation follows these principles: - **Navigation**: Internal links and cross-references for easy browsing - **Completeness**: Comprehensive coverage of technical requirements - **Maintenance**: Regular updates as the platform evolves - ---- - -*Last updated: September 2025* diff --git a/architecture/content/architecture_detailed.svg b/architecture/content/architecture_detailed.svg index 4af69ab..df57389 100644 --- a/architecture/content/architecture_detailed.svg +++ b/architecture/content/architecture_detailed.svg @@ -1,3 +1,4 @@ + -
Common Services (**)
Container Orchestration (Kubernetes, OCP, etc.)
Azure
AI DIAL
StatGPT
Icon-identity-230 BusinessUser
HTTP
HTTP
StatGPT App Backend
TCP
HTTP
HTTP
StatGPT Admin Backend
PostgreSQL (*)
Elasticsearch (*)
Request flow
TCP
HTTP
HTTP
DIAL Core
HTTP
DIAL OpenAI Adapter
HTTP
DIAL Realtime Analytics
HTTP
Reads
Logs
HTTP
Vector
InfluxDBRedis (*)
End user Communication Channels
DIAL Chat StatGPT Portal
TCP
HTTP
DIAL Admin Backend
Admin Communication Channels
Grafana DIALAdmin FE StatGPTAdmin FE
HTTP
HTTP
HTTP
HTTP
Azure OpenAIBlob Storage Icon-identity-230 AdminUser
Logging
Monitoring
(OTEL)
External Services
Secret 
management
SSO
(*) - cloud-managed versions of services can be used
(**) - Common Services can be located either in Container orchestration or in different Clouds
SDMX API (HTTP)
SDMX Registries
(External)
SDMX API (HTTP)
+
Common Services (**)
Container Orchestration (Kubernetes, OCP, etc.)
Azure
AI DIAL
StatGPT
Icon-identity-230 BusinessUser
HTTP
HTTP
StatGPT App Backend
TCP
HTTP
HTTP
StatGPT Admin Backend
PostgreSQL (*)
Elasticsearch (*)
Request flow
TCP
HTTP
HTTP
DIAL Core
HTTP
DIAL OpenAI Adapter
HTTP
DIAL Realtime Analytics
HTTP
Reads
Logs
HTTP
Vector
InfluxDBRedis (*)
End user Communication Channels
DIAL Chat StatGPT PortalMCP Client
TCP
HTTP
DIAL Admin Backend
Admin Communication Channels
Grafana DIALAdmin FE StatGPTAdmin FE
HTTP
HTTP
HTTP
HTTP
Azure OpenAIBlob Storage Icon-identity-230 AdminUser
Logging
Monitoring
(OTEL)
External Services
Secret 
management
SSO
(*) - cloud-managed versions of services can be used
(**) - Common Services can be located either in Container orchestration or in different Clouds
SDMX API (HTTP)
SDMX Registries
(External)
SDMX API (HTTP)
StatGPT App MCP
HTTP
HTTP
diff --git a/architecture/content/architecture_high_level.svg b/architecture/content/architecture_high_level.svg index a9b9f30..7e09332 100644 --- a/architecture/content/architecture_high_level.svg +++ b/architecture/content/architecture_high_level.svg @@ -1,3 +1,4 @@ + -
External Services
Application Services
Icon-identity-230 Business User
StatGPT Application
AI DIAL
Communication Channels
AI DIAL Chat StatGPT Portal
Request flow
SDMX Registries
Icon-identity-230 Admin User
Admin Communication Channels
DIAL Admin StatGPT Admin
Azure
Azure OpenAIGrafana
+
External Services
Application Services
Icon-identity-230 Business User
StatGPT Application
AI DIAL
Communication Channels
AI DIAL Chat StatGPT PortalMCP Client
Request flow
SDMX Registries
Icon-identity-230 Admin User
Admin Communication Channels
DIAL Admin StatGPT Admin
Azure
Azure OpenAIGrafana
diff --git a/architecture/content/diagrams.drawio b/architecture/content/diagrams.drawio index 61394e4..26ee1bf 100644 --- a/architecture/content/diagrams.drawio +++ b/architecture/content/diagrams.drawio @@ -1,11 +1,11 @@ - + - + @@ -35,13 +35,16 @@ - + - + + + + - + @@ -131,38 +134,38 @@ - + - + - + - + - + - + - + - + @@ -170,13 +173,13 @@ - - + + - + @@ -187,30 +190,30 @@ - - + + - + - - + + - + - + - + - + @@ -258,7 +261,7 @@ - + @@ -274,9 +277,10 @@ - + + - + @@ -284,37 +288,45 @@ - + - + - + - + - + - + + + + - + + + + + + - + - + @@ -338,10 +350,11 @@ - + + - - + + @@ -351,9 +364,10 @@ - - - + + + + @@ -367,7 +381,7 @@ - + @@ -376,7 +390,7 @@ - + @@ -385,29 +399,53 @@ - + - - + + - + - - + + + + + + + + + + + + + + + + + + + + + + + + + + - - + + diff --git a/architecture/mcp.md b/architecture/mcp.md new file mode 100644 index 0000000..bd4699e --- /dev/null +++ b/architecture/mcp.md @@ -0,0 +1,127 @@ +# Application MCP + +Application MCP gives AI agents — Claude, Cursor, custom DIAL applications, and other MCP-aware LLM clients — +agentic access to a channel's StatGPT tools (data query, RAG, glossary, web search, …) over the +[Model Context Protocol](https://modelcontextprotocol.io/), so the agent can invoke those tools inside its own +reasoning loop. + +## 🎯 Scope + +- **Does**: Exposes the same primitives the StatGPT supreme agent uses internally. +- **Does not**: Replace the supreme agent's `/chat/completions` endpoint, pick tools, or compose multi-tool answers. +- **Configuration**: Tools surfaced for `deployment_id=foo` come directly from `foo`'s channel YAML — no MCP-specific override. + +## 🔌 Endpoint + +| Property | Value | +|------------|--------------------------------------------------------| +| Path | `POST /api/v1/{deployment_id}/mcp` on Chat Backend | +| Transport | HTTP streaming (`streamable-http`) | + +`{deployment_id}` selects the channel; tools and config differ per channel. The endpoint is part of the Chat Backend +image — there is no separate enable flag. To make it reachable, register StatGPT as a DIAL Application (see +[Authentication & Authorization](#-authentication--authorization) below). + +## 🛠️ Tools + +Application MCP exposes whichever tools the channel YAML enables. The full tool catalog and per-tool semantics live in [Architecture / Tools](./tools.md). + +## 🔐 Authentication & Authorization + +Application MCP is wired into AI DIAL as a **DIAL Application**. StatGPT itself runs no OAuth flow. DIAL Core acts as +the MCP-spec authorization-server discovery layer in front of the MCP endpoint, points clients at the configured IDP, +and forwards the resulting bearer + per-request key on every proxied tool call. + +### Registration: DIAL Application + +StatGPT is registered with DIAL as a **DIAL Application**. A DIAL Application can declare both a chat-completion +endpoint and an MCP endpoint side-by-side; DIAL Core proxies traffic to either through its routing layer. + +Authoritative DIAL docs: + +- Application administration UI / fields → [docs.dialx.ai entities-applications](https://docs.dialx.ai/tutorials/admin/entities-applications). +- Application config schema → [DIAL Core applications.md](https://github.com/epam/ai-dial-core/blob/development/docs/dynamic-settings/applications.md). +- Per-request keys → [docs.dialx.ai per-request-keys](https://docs.dialx.ai/platform/core/per-request-keys). + +Relevant fields on the StatGPT Application: + +| Field | Value | Effect | +|--------------------|--------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| +| `mcpEndpoint` | `https:///api/v1/{deployment_id}/mcp` | The MCP endpoint DIAL Core proxies to. | +| `forwardAuthToken` | `true` | DIAL forwards the caller's `Authorization: Bearer …` header to the application unchanged on every proxied call. | + +There is no per-application OAuth client config (no `client_id`, no `redirect_uri`, no DIAL-side signin endpoint). +DIAL's job is discovery + forwarding, not running OAuth itself. + +### Auth Flow + +The flow follows the [MCP authorization spec](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization) +discovery flow, with two architectural details: + +- **DIAL plays the resource-server-front role.** It returns the 401 + `WWW-Authenticate`, hosts + `/.well-known/oauth-protected-resource` (RFC 9728), and points clients at the IDP via the `authorization_servers` + field. StatGPT MCP itself has no `/.well-known` endpoints and does not return 401s — it never sees a request + without a bearer. +- **DIAL is not in the OAuth path.** The OAuth 2.1 + PKCE handshake happens directly between the MCP client and the + IDP (Keycloak, Entra ID, …). DIAL holds no PKCE verifier, no authorization code, no refresh token, no per-app + client credentials. + +After the client has a bearer: + +1. Client → DIAL: `tools/call` with `Authorization: Bearer `. +2. DIAL → StatGPT MCP: same call, with the bearer forwarded (because `forwardAuthToken=true`) and an additional + `api-key: ` header. +3. StatGPT MCP treats both headers as already-validated input — it does no token validation of its own. If neither + header reaches the server, `tools/list` returns empty and `tools/call` errors. + +### Relationship to the MCP Authorization Specification + +At the wire level StatGPT's MCP authorization is **spec-compliant** — a spec-compliant MCP client (one that +implements the discovery flow described in +[MCP Authorization](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization)) works against the +StatGPT MCP endpoint without modification. The architectural twist is that the OAuth resource-server role is **split +across two HTTP participants**: + +| Role | Who handles it | +|-----------------------------------------------------|---------------------------------| +| Returns `401 Unauthorized` with `WWW-Authenticate` | DIAL | +| Hosts `/.well-known/oauth-protected-resource` (RFC 9728) | DIAL | +| Validates and uses the presented bearer | StatGPT MCP (after DIAL forwards it) | +| Acts as the OAuth authorization server | The IDP (Keycloak, Entra ID, …) | + +This split is invisible to MCP clients — they see one logical resource server that behaves correctly under +[RFC 9728](https://datatracker.ietf.org/doc/html/rfc9728), and run their handshake against whatever +`authorization_servers` URL DIAL advertised. + +**Why this split:** + +- **Single IDP across the platform** — DIAL Core terminates discovery for every DIAL Application; StatGPT MCP + doesn't reimplement RFC 9728 / RFC 8414 endpoints. +- **Smaller StatGPT auth surface** — StatGPT processes never run the OAuth handshake, so they cannot leak codes, + PKCE verifiers, or client secrets. +- **One audit / per-request-key enforcement layer** — DIAL Core handles both, consistent with how it handles every + other DIAL Application. + +### Refresh Tokens & Session Lifecycle + +Refresh tokens are **owned by the MCP client** and live in its OAuth session against the IDP. DIAL is not in the +refresh path: + +- When the client's access token expires, the client runs the refresh-token grant directly against the IDP and gets + a new access token. +- The new bearer is presented to DIAL on the next call; DIAL forwards it, same as before. +- StatGPT MCP never sees refresh tokens — it only ever sees a current access token in the `Authorization` header. +- If the IDP rejects a refresh (session revoked, etc.), the client gets a fresh 401 from DIAL on its next call and + the discovery flow starts over. + +The MCP server is stateless — token freshness is established per request from the forwarded headers, with nothing +cached across calls. + +## 📚 References + +- [MCP Authorization specification](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization) — + the canonical OAuth 2.1 + PKCE discovery flow that DIAL implements in front of the StatGPT MCP endpoint. +- [DIAL Core applications.md](https://github.com/epam/ai-dial-core/blob/development/docs/dynamic-settings/applications.md) — + Application config schema (`mcpEndpoint`, `forwardAuthToken`, …). +- [DIAL per-request keys](https://docs.dialx.ai/platform/core/per-request-keys) — how DIAL Core injects `api-key` + headers on every proxied application call. diff --git a/architecture/overview.md b/architecture/overview.md index a77b853..928d25f 100644 --- a/architecture/overview.md +++ b/architecture/overview.md @@ -199,6 +199,15 @@ The AI agent is explicitly instructed to: → See [AI DIAL Access Control](https://docs.dialx.ai/platform/core/access-control-intro) for configuration. +### Agentic Access via MCP + +In addition to the chat UI, StatGPT exposes a channel's configured tools (data query, RAG, glossary, web search, …) +over the [Model Context Protocol](https://modelcontextprotocol.io/) via **Application MCP**, mounted on the Chat +Backend. AI DIAL fronts the endpoint as the `mcpEndpoint` of a DIAL Application and handles MCP-spec +authorization-server discovery, so AI agents can invoke StatGPT tools inside their own reasoning loops. + +→ See [Application MCP](./mcp.md) for the endpoint, tool catalog, and authentication flow. + ### Security & Privacy **Data Protection**: diff --git a/architecture/services.md b/architecture/services.md index f8bd2c3..f1c9a17 100644 --- a/architecture/services.md +++ b/architecture/services.md @@ -96,6 +96,15 @@ with [DIAL API Specification](https://dialx.ai/dial_api#operation/sendChatComple | **External AI Models** | LLM capabilities (e.g., Azure OpenAI) | | **SDMX Providers** | Statistical data sources (IMF, World Bank, Eurostat) | +#### MCP Endpoint + +The Chat Backend exposes channel tools over the [Model Context Protocol](https://modelcontextprotocol.io/) at +`POST /api/v1/{deployment_id}/mcp`, fronted by AI DIAL as the `mcpEndpoint` of a DIAL Application. DIAL handles +MCP-spec authorization-server discovery and forwards the user's credentials to the endpoint, so AI agents can +invoke channel tools inside their own reasoning loops. + +→ See [Application MCP](./mcp.md) for the tools catalog and request flow. + ### ⚙️ StatGPT Admin Backend #### Technology Stack diff --git a/architecture/tools.md b/architecture/tools.md index 2a17b1b..46012a2 100644 --- a/architecture/tools.md +++ b/architecture/tools.md @@ -223,3 +223,11 @@ Returns: - Provides static content to agent or users **Example Use Case**: Portal guides with dynamic information about available pages and features. + +## 🔌 Agentic Access via MCP + +The same tool catalog above is also exposed over the [Model Context Protocol](https://modelcontextprotocol.io/) by +StatGPT's **Application MCP** server, so AI agents can invoke these tools inside their own reasoning loops. +The MCP tool surface for a given channel is whatever its YAML enables. + +→ See [Application MCP](./mcp.md) for the endpoint and authentication flow.