Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion docs/CONNECTOR-ONBOARDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,9 @@ is the validation.
whose live metadata said so on 2026-09-10 — GitHub (measured first: a
secretless exchange answers `incorrect_client_credentials`, #1006), Slack,
HubSpot, Zoom, Box, Asana, Docusign, Front, Wrike, Xero, AlloyDB and the
eight Google Workspace servers. Left off `amazon-ads` and `doordash`, whose
eight Google Workspace servers — and Sage Intacct, whose metadata the
2026-09-14 catalog audit read the same way (no registration endpoint, no
`none`). Left off `amazon-ads` and `doordash`, whose
metadata lists `none`, and off five of the six `tenant` entries, whose URL
cannot be probed without a tenant. Azure DevOps is the sixth: its
protected-resource metadata is organization-agnostic and names Microsoft
Expand Down Expand Up @@ -157,6 +159,15 @@ Face — and a test caps it at 8–20 entries, never community provenance. The
same pass audited the directory for dead/low-quality listings and added
newly-verified official endpoints (see that PR in the repository history).

**Catalog corrections do not rewrite saved connections**: fleet builds the
remote MCP mount from the saved row (`srv.URL`, via `connFor` in
`internal/remotemcp/resolver.go`). The directory UI marks an entry "added"
by name (`web/src/app/settings/connections/page.tsx`), so a user who connected
an entry before a catalog fix (such as Cartesia, Octagon, or Globalping) keeps
dialing the old URL and auth after upgrading, and cannot pick the corrected
entry without removing the connection first. To adopt a catalog correction,
the operator or user must Remove the connector and re-add it.

## Deviations / honest scope

- **`auth: tenant` semantics narrowed**: it now means "your URL + OAuth". The
Expand Down
182 changes: 182 additions & 0 deletions docs/HOSTED-CONNECTORS-RUNBOOK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
# Hosted MCP connectors — operator runbook

How to connect a hosted (official, per-user OAuth) MCP server to fleet, what
each vendor needs before the Connect button works, how to tell a fleet
problem from a vendor problem, and what to do when a connection stops. Every
statement here was measured against the vendor in the #1006 OAuth pack; the
per-vendor results are in [`MCP-CATALOG-STATUS.md`](MCP-CATALOG-STATUS.md).
The design behind the mechanics is in [ADR-0009](adr/0009-per-user-remote-mcp-oauth.md)
(per-user OAuth), [ADR-0050](adr/0050-remote-mcp-seats.md) (seats),
[`REMOTE-MCP-MULTI-LOGIN.md`](REMOTE-MCP-MULTI-LOGIN.md) and
[`CONNECTION-SHARING.md`](CONNECTION-SHARING.md); the directory itself in
[`MCP-CATALOG.md`](MCP-CATALOG.md) and [`CONNECTOR-ONBOARDING.md`](CONNECTOR-ONBOARDING.md).

## Before the first connection

- **The callback URL is derived, not configured.** fleet advertises
`FLEET_PUBLIC_BASE_URL` + `/api/oauth/mcp/callback` and logs it at boot
(`remote MCP OAuth: ENABLED … redirect …`). `FLEET_PUBLIC_BASE_URL` must be
the browser's origin for the **web tier**, not the chat-server port. Every
vendor that asks for a redirect URI gets exactly that string.
- **Some vendors refuse plain-http callbacks.** Slack requires HTTPS. Microsoft
Entra allows `http://` only for `localhost` (not `127.0.0.1`) and ignores the
port there. Single-box installs sit behind Caddy with TLS by default. A chart
deployment reaches the web tier however the operator exposes it — the chart's
optional Ingress (disabled by default, `ingress.tls` empty) or something in
front of it — and whatever the path, the public callback URL must be HTTPS or
HTTPS-only vendors (Slack) reject it; a local dev rig needs a tunnel for Slack.
- **`FLEET_MCP_OAUTH_ENCRYPTION_KEY` is the connection.** Tokens, client
secrets and registration tokens are sealed with it, bound to (owner, URL).
Lose or rotate it and every hosted connection must be reconnected by its
Comment thread
bradflaugher marked this conversation as resolved.
owner. Keep `FLEET_MCP_OAUTH_ENCRYPTION_KEY` in a separate access-controlled
secret store, never in the same backup set as the database; both artifacts
are required for recovery, so document where each lives.
- **Three shapes of Connect.** *One-click* — the vendor registers fleet as a
client itself (Notion, Linear, Stripe, Grafana Cloud, Uptime Robot, Plaid,
Cartesia, Globalping, …): the user clicks Connect and signs in. *Bring your
own client* — the vendor has no self-registration: an admin creates an OAuth
app at the vendor with fleet's callback URL and users paste the client ID
and, where the directory says **required**, the secret (GitHub, Slack, the
Google Workspace servers, Azure DevOps, Sage Intacct, …). *Tenant* — the
URL carries the customer's org or host; the form asks for it, then one of
the two flows above follows.
- **Why both a client secret and a login.** The secret identifies fleet, the
application, to the vendor; the login identifies the user whose data the
tools will act on. Vendors without self-registration need the first done by
hand; the second is always per user.

## Per-vendor notes (measured)

- **GitHub** — OAuth App (not a GitHub App) with fleet's callback as the
Authorization callback URL; the secret is mandatory — without it GitHub
answers `incorrect_client_credentials` with HTTP 200. Tokens live 8 hours
with a refresh token; `bad_refresh_token` means the user must reconnect.
"Revoke all user tokens" on the app is noticed at the next mount (401 →
*Reconnect needed*). No revocation endpoint: sign-out clears fleet's copy.
- **Google Workspace (Drive, Gmail, …)** — a Google Cloud OAuth client (Web
application) with fleet's callback; secret required. Enable the Workspace
MCP APIs on the project. fleet asks for `access_type=offline` and
`prompt=consent`, without which Google issues a one-hour token and no
refresh token. **The Workspace MCP servers are a Developer Preview**: a
personal Gmail account connects but every tool call answers
`The caller does not have permission`. Test with a Workspace account that is
enrolled.
- **Slack** — create the app from a manifest or by hand; under *Features →
Agents & AI Apps* turn on **Model Context Protocol** (without it the server
answers `App is not enabled for Slack MCP server access`); redirect URL must
be HTTPS; secret required. The user token has no expiry and no refresh
token; no revocation endpoint. The vendor's metadata names its bare origin
as the resource while serving MCP at `/mcp` — fleet keeps the typed URL.
- **Notion, Linear** — one click. Linear publishes a revocation endpoint, so
sign-out revokes at the vendor; Notion does not.
- **Azure DevOps** — the org must be connected to a Microsoft Entra tenant
(a standalone personal-account org is refused by Microsoft). In that
tenant: confirm the *Azure DevOps MCP* enterprise application exists (or
`az ad sp create --id 2a72489c-aab2-4b65-b93a-a91edccf33b8`), create an app
registration on the **Web** platform with fleet's callback, add a client
secret, add the *Azure DevOps MCP* delegated permission and **grant admin
consent**. Sign in with a **work account native to the tenant**: a personal
Microsoft account, even one made a tenant Member during Azure sign-up, loops
at Microsoft's account picker on the `organizations` endpoint fleet is
directed to. That user must have opened `dev.azure.com/<org>` once, or every
tool call answers `Identity … has not been materialized`. Tokens live about
70 minutes with a rotating refresh token; no revocation endpoint.
- **Stripe** — one click. Every API tool needs `stripe_context` (the account
id) and `livemode`; the model gets them from `list_available_accounts_or_orgs`
first. Calls made without them answer HTTP 422.
- **Grafana Cloud, Uptime Robot** — one click. Uptime Robot's server returns a
client secret at registration although fleet asked to be a public client;
fleet stores and uses it.
- **Plaid, Intercom** — no protected-resource metadata; connect worked on the
2026-09-14 build. `main` currently refuses them at Add (their MCP path
answers 401 under every sub-path, which the discovery code reads as a
server failure). Known and deliberately not fixed at the time of writing.
- **Square, Smartlead** — SSE-only endpoints. fleet's hosted-connector
transport is streamable HTTP; these cannot connect today.

## Running with connections

- **Chat.** A connection mounts only when it is enabled in the conversation's
Tools picker. The web UI pre-enables the default set; an API client must
send `enabled_optional` naming the connection, or nothing hosted mounts and
the model reports "no MCP tools".
- **Scheduled tasks** mount every connected connection of the task's owner on
its default seat, subject to the overlay ceiling. Only connections enabled for
that user mount; a disabled connection (own or shared seat alike) is omitted
silently with no notice to the model. The overlay caps mounting at
`maxOverlayServers = 8` connected servers per user (`maxOverlayServers` in
`internal/agent/remote_mcp_overlay.go`), applied uniformly to chat,
scheduled runs, and the broker. Selection order follows connection list order
(`internal/remotemcp/resolver.go`'s `ConnectedServersForUser`: the owner's
own connections newest-first, then the ones shared with them); servers past the
8-server cap are skipped and logged. `mcp_selection` pins choose which seat
account mounts for a connector, but pins do **not** bypass the 8-server cap
or reorder servers ahead of it. A pin to a seat that is not connected skips
that connector and tells the model; a pin to a server the owner never
connected dead-letters the task without a model call.
- **Expired tokens refresh headlessly** in both chat and scheduled runs,
under a row lock. Network errors and 5xx stay transient (the refresh path in
`internal/remotemcp/service.go`): the connection is skipped for that run,
remains connected, and is retried on the next call. Only terminal OAuth
errors (e.g. `invalid_grant`) mark the connection *Reconnect needed*
(`needs_reauth`). In either case, the run completes without the skipped
connector.
- **More than 128 tools (by default)** switches the run to deferred mode:
`disclosureThreshold()` defaults to 128 tools and honours both the
`FLEET_TOOL_DISCLOSURE_THRESHOLD` environment variable and the live
`tool_disclosure_threshold` admin setting. Above that threshold, the model
finds tools with `tool_search`, reads their schema with `tool_describe`
(which lists the required arguments) and calls them with `tool_call`, which
refuses a call missing a required argument and names it.
- **Seats and sharing.** One connection name can hold several logins; each is
a seat with one owner-chosen default. Sharing is per seat, the grantee never
sees the token, tool calls act as the owner at the vendor, and a revoked
share takes effect on the grantee's next turn. Share only what you would
hand over: a grantee gets the seat's full tool surface.

## When something fails

Read `fleet.log` first. The lines that matter:

- `remote-mcp: skipping server "<name>" for <user> — token unavailable` —
refresh failed. For terminal OAuth failures (e.g. `invalid_grant`), the row is
marked *Reconnect needed* (`needs_reauth`); have the owner click Connect. For
transient network errors or vendor 5xx, the same line is logged but the
connection stays connected and the next call retries automatically.
- `remote-mcp: skipping server "<name>" … — failed to connect: … HTTP 404` —
the vendor rejected the tools handshake; usually a wrong URL (check the
vendor's documented MCP path), not a credential problem, although the
model is currently told the connector "needs re-authorization" either way.
- `mcpbroker: tool call failed (masked to the caller): … HTTP 422 …` — the
vendor refused the tool arguments. The model sees only
`credential-owner call failed`; the real text is on this line.
- `scheduled task <id>: wired N remote MCP server(s) for <user>` /
`skipped remote MCP server(s) needing re-auth or a missing pinned seat` —
what a scheduled run mounted and what it could not.
- `Fantasy tools registered: … N MCP tools DEFERRED` — the run is in deferred
mode.

At **Add** time, the error toast names every metadata location fleet tried.
`fetch protected-resource metadata … 404` on every location means the vendor
publishes none; `authorization-server issuer mismatch` means the vendor's
document names another issuer and neither that issuer nor the vendor's host
vouched for its endpoints; `client secret required` means the vendor accepts
no public clients and the form needs the secret.

At **sign-in**, errors starting with `AADSTS` come from Microsoft:
`50011` a redirect URI that does not match the app registration, `65001`
admin consent not granted, `90100` a stale sign-in page (start again in a
private window, or complete the new user's first sign-in at
myaccount.microsoft.com first).

## Adding a vendor to the directory

The catalog rules are in [`MCP-CATALOG.md`](MCP-CATALOG.md). Before listing a
hosted server, run discovery against it (the Connect button does exactly
that) and record: does it publish protected-resource metadata, does its
authorization server allow public clients (`none` in
`token_endpoint_auth_methods_supported`), and does it self-register. For
OAuth entries, verify that the unauthenticated `initialize` POST answers with
401 and protected-resource metadata rather than 404; `open` and `api_key`
entries must complete `initialize` (200) and `tools/list`. A URL that
discovers but 404s tool calls is the most common catalog error.
Loading