feat(mcp): add protected HTTP MCP server support#47
Open
frederikprijck wants to merge 1 commit into
Open
Conversation
Adds OAuth-protected HTTP MCP server support so evals can target servers that require an Authorization: Bearer token (e.g. the Auth0 hosted MCP server, which authenticates with a Management API token). - mintMcpToken: per-job client-credentials token mint for HTTP MCP servers - MCPHttpServerConfig.auth (tokenUrl/clientId/clientSecret/audience) - All four runners forward the minted token: claude-code, copilot, and gemini-cli set an Authorization header on the server config; codex writes a bearer_token_env_var reference into config.toml and injects the token into its process env (an inline bearer_token is rejected by codex) - A failed token mint skips the server with a warning rather than registering it unauthenticated - sandbox.passthroughEnv: forward named host env vars into the Docker sandbox so MCP credentials reach the container - docs/PROTECTED_MCP.md setup guide
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for protected HTTP MCP servers — servers that require an
Authorization: Bearertoken rather than being publicly reachable (e.g. the Auth0 hosted MCP server, which authenticates with a Management API token).This is the auth-plumbing layer only. The
calledTool/calledToolOneOftrace-based grader primitives and thehosted_mcp_list_applicationseval that build on top of it are intentionally left for a follow-up PR.What's included
mintMcpToken— per-job client-credentials token mint for HTTP MCP servers (minted per job so a long matrix run never reuses an expired token).MCPHttpServerConfig.auth— new optionalauthblock (tokenUrl/clientId/clientSecret/audience) typed asMCPOAuthConfig.claude-code,copilot, andgemini-cliset anAuthorization: Bearerheader on the MCP server config;codexwrites abearer_token_env_varreference intoconfig.tomland injects the token into its process env (an inlinebearer_tokenis rejected by Codex, so the secret stays out of the file).logger.warnrather than registering it unauthenticated.sandbox.passthroughEnv— forward named host env vars into the Docker sandbox (names only; values resolved fromprocess.envat launch) so MCP credentials reach the container.docs/PROTECTED_MCP.md— setup guide (prerequisites, env vars, config, how minting/forwarding works, troubleshooting).Test plan
npm run buildpassesnpm testpasses (697 tests)npm run lintpasses--tools mcpagainst a protected server withMCP_TENANT_DOMAIN/MCP_CLIENT_ID/MCP_CLIENT_SECRETset and confirm the token is minted and forwarded for each runner