DevOps Agent — Generic OpenSearch MCP Server

Deploy a generic OpenSearch MCP server on AWS Bedrock AgentCore that can connect to any OpenSearch cluster in your VPC. No hardcoded OPENSEARCH_URL — the calling agent passes opensearch_url per tool call.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                          Your VPC                                   │
│                                                                     │
│  ┌──────────────┐     ┌──────────┐     ┌──────────────────────┐    │
│  │  DevOps Agent │────▶│   VPCE   │────▶│  AgentCore Runtime   │    │
│  │  (Agent Space)│     │ (private)│     │  opensearch_mcp_     │    │
│  └──────────────┘     └──────────┘     │  generic             │    │
│                                         │  (no hardcoded URL)  │    │
│                                         └──────────┬───────────┘    │
│                                                    │                │
│                              ┌─────────────────────┼────────┐      │
│                              │                     │        │      │
│                              ▼                     ▼        ▼      │
│                     ┌──────────────┐  ┌────────┐  ┌────────────┐   │
│                     │ OpenSearch   │  │Cluster │  │  Cluster   │   │
│                     │ Cluster A    │  │   B    │  │     C      │   │
│                     │ (eks-app-logs│  │        │  │            │   │
│                     └──────────────┘  └────────┘  └────────────┘   │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

All traffic stays private within the VPC via VPC Endpoint. The agent decides which cluster to query by passing opensearch_url per tool call.

What Gets Created

Step	Resource	Tool	Purpose
1	AgentCore Runtime	agentcore deploy	MCP server container (opensearch-mcp-server-py)
1	ECR Repository	agentcore deploy	Container image storage
1	IAM Role	agentcore deploy	Runtime execution role
1	CodeBuild Project	agentcore deploy	Builds container remotely (no local Docker needed)
2	IAM Inline Policy	scripts/01-add-iam-policy.sh	Generic OpenSearch access (all domains in region)
3	VPC Endpoint	scripts/02-create-vpce.sh	Private connectivity to AgentCore
4	Resource Policy	scripts/03-apply-resource-policy.sh	Restrict runtime to VPCE-only access
5	Cognito User Pool	scripts/04-setup-cognito-auth.sh	OAuth client credentials auth (required by DevOps Agent)
5	Cognito App Client	scripts/04-setup-cognito-auth.sh	Client ID + Secret for OAuth flow
5	JWT Authorizer	scripts/04-setup-cognito-auth.sh	Runtime auth config update

All resources tagged: Project=devops-poc

---

Prerequisites

• AWS CLI v2 with bedrock-agentcore-control commands
• AgentCore CLI — install with: npm install -g @aws/agentcore
• AWS credentials with Admin access to the target account
• Existing VPC with:
  • Private subnets (where OpenSearch domains live)
  • Security group that allows traffic to OpenSearch (port 443)
  • One or more OpenSearch domains running in VPC mode

---

Deployment Steps

Step 0: Clone and Configure

git clone https://github.com/lillyjohns/devopsagent-opensearch-mcp.git
cd devopsagent-opensearch-mcp

Edit agentcore/agentcore.json — update these fields for your environment:

{
  "runtimes": [{
    "networkConfig": {
      "subnets": ["subnet-YOUR_SUBNET_ID"],        // ← Your private subnet(s)
      "securityGroups": ["sg-YOUR_SG_ID"]           // ← SG with OpenSearch access
    }
  }]
}

Edit agentcore/aws-targets.json — set your account and region:

[
  {
    "name": "default",
    "account": "YOUR_ACCOUNT_ID",
    "region": "us-west-2"
  }
]

Install CDK dependencies:

cd agentcore/cdk && npm install && cd ../..

Step 1: Deploy AgentCore Runtime

agentcore deploy -y -v

This builds the container remotely via CodeBuild, pushes to ECR, and creates the AgentCore runtime. No local Docker needed.

Wait for output like:

✓ Deployed to 'default'

Outputs:
  ...RuntimeArnOutput...: arn:aws:bedrock-agentcore:us-west-2:ACCOUNT:runtime/RUNTIME_ID
  ...RoleArnOutput...:    arn:aws:iam::ACCOUNT:role/ROLE_NAME
  ...RuntimeIdOutput...:  RUNTIME_ID

Save the RUNTIME_ID and ROLE_NAME — you'll need them for the next steps.

Verify the runtime is ready:

agentcore status

Step 2: Add IAM Policy for OpenSearch Access

The agentcore deploy creates an IAM role but it doesn't have OpenSearch permissions. Add them:

./scripts/01-add-iam-policy.sh <ROLE_NAME>

# Example:
./scripts/01-add-iam-policy.sh AgentCore-opensearchmcpge-ApplicationAgentOpensearc-lJCNIOEamBZp

This grants es:ESHttp* access to all OpenSearch domains in the region. To restrict to specific domains, edit the script's Resource ARN.

Step 3: Create VPC Endpoint

Create a new, separate VPC Endpoint for this AgentCore runtime:

./scripts/02-create-vpce.sh

# Or override defaults:
VPC_ID=vpc-xxx SUBNET_IDS=subnet-xxx SECURITY_GROUP_ID=sg-xxx ./scripts/02-create-vpce.sh

Output:

✅ VPC Endpoint created: vpce-0ace990d19cd40585
Save this VPCE ID for the next step:
  export VPCE_ID=vpce-0ace990d19cd40585

Wait ~30 seconds for the VPCE state to change from pending to available.

Step 4: Apply Resource-Based Policy

Restrict the runtime to only accept requests through the VPCE (no public access):

./scripts/03-apply-resource-policy.sh <RUNTIME_ID> <VPCE_ID>

# Example:
./scripts/03-apply-resource-policy.sh opensearchmcpgeneric_opensearch_mcp_generic-yYHc3mFkCR vpce-0ace990d19cd40585

Step 5: Set Up Cognito OAuth Authentication

The DevOps Agent requires OAuth 2.0 authentication to connect to MCP servers. This script creates everything automatically — no pre-existing Cognito setup needed:

• Cognito User Pool + Domain (for token endpoint)
• Resource Server with scopes
• App Client (client_credentials flow)
• Updates the AgentCore runtime with JWT authorizer

./scripts/04-setup-cognito-auth.sh <RUNTIME_ID>

# Example:
./scripts/04-setup-cognito-auth.sh opensearchmcpgeneric_opensearch_mcp_generic-yYHc3mFkCR

The script outputs the OAuth credentials you'll need for DevOps Agent registration:

✅ Cognito OAuth Setup Complete

App Client ID:      xxxxxxxxxxxxxxxxxxxxxxxxxxxx
App Client Secret:  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Token Endpoint:     https://xxx.auth.us-west-2.amazoncognito.com/oauth2/token
Scopes:             opensearch-mcp/invoke

Save these values — you'll need them in the next step.

Step 6: Register with AWS DevOps Agent

Note: This step must be done in the AWS Console — there is no CLI for DevOps Agent service registration yet.

1. Open the AWS DevOps Agent console

2. Navigate to Capability Providers → find MCP Server → click Register

3. MCP server details:

   Field	Value
   Name	opensearch-mcp-generic
   Endpoint URL	Your AgentCore invocation URL (from agentcore status --json → invocationUrl)
   Connect to endpoint using a private connection	✅ Select the VPCE from Step 3

4. Authorization flow: Select OAuth Client Credentials

5. Authorization configuration (values from Step 5 output):

   Field	Value
   Client ID	<from Step 5 output>
   Client Secret	<from Step 5 output>
   Exchange URL	<Token Endpoint from Step 5 output>
   Scopes	opensearch-mcp/invoke

6. Review and Submit — DevOps Agent validates the connection

7. Go to your Agent Space → Capabilities → MCP Servers → Add → select opensearch-mcp-generic → allow tools

Step 7: Test

Start a chat in your Agent Space and try:

List all indices in the OpenSearch cluster at https://vpc-eks-app-logs-xxx.us-west-2.es.amazonaws.com

The agent should use the generic MCP server and pass the opensearch_url dynamically.

---

MCP Tools Available

From opensearch-mcp-server-py:

Core tools (enabled by default):

Tool	Description
ListIndexTool	List all indices with doc counts, store size
IndexMappingTool	Get index mappings and settings
SearchIndexTool	Search with OpenSearch Query DSL
GetShardsTool	Shard information per index
ClusterHealthTool	Cluster health status
CountTool	Document count (with optional query filter)
ExplainTool	Explain why a document matches a query
MsearchTool	Multi-search in one request
GenericOpenSearchApiTool	Call any OpenSearch API endpoint

All tools accept an optional opensearch_url parameter to target different clusters dynamically.

Additional tools (disabled by default, enable via env vars):

• GetClusterStateTool, GetSegmentsTool, CatNodesTool, GetNodesTool
• GetIndexInfoTool, GetIndexStatsTool, GetQueryInsightsTool
• GetNodesHotThreadsTool, GetAllocationTool, GetLongRunningTasksTool

See the opensearch-mcp-server-py User Guide for how to enable additional tools.

---

Configuration

Environment Variables (optional)

Set these in agentcore/agentcore.json under environmentVariables if you want defaults:

{
  "runtimes": [{
    "environmentVariables": {
      "OPENSEARCH_URL": "https://default-cluster.us-west-2.es.amazonaws.com",  // optional default
      "OPENSEARCH_USERNAME": "admin",     // for basic auth
      "OPENSEARCH_PASSWORD": "password",  // for basic auth
      "OPENSEARCH_NO_AUTH": "true"        // skip auth entirely
    }
  }]
}

If no OPENSEARCH_URL env var is set (recommended), the agent must pass opensearch_url per tool call.

Network Requirements

For the runtime to reach an OpenSearch cluster:

1. Same VPC (or peered VPC with proper routing)
2. Security group allows outbound to OpenSearch on port 443
3. OpenSearch access policy allows the runtime's IAM role
4. DNS resolution works for the OpenSearch endpoint from within the VPC

Authentication to OpenSearch

The runtime supports multiple auth methods:

Method	When to use
IAM (default)	AWS-managed OpenSearch domains — uses the runtime's IAM role
Basic auth	Self-managed OpenSearch with username/password
No auth	Development clusters with security disabled

For IAM auth (most common), the IAM policy from Step 2 handles it.

---

Cleanup

Remove everything in reverse order:

# 1. Remove from DevOps Agent (console — delete the MCP server integration)

# 2. Remove resource-based policy
aws bedrock-agentcore-control delete-resource-policy \
  --resource-arn "arn:aws:bedrock-agentcore:us-west-2:ACCOUNT:runtime/RUNTIME_ID" \
  --region us-west-2

# 3. Remove VPC Endpoint
aws ec2 delete-vpc-endpoints --vpc-endpoint-ids vpce-XXXX --region us-west-2

# 4. Remove IAM policy
aws iam delete-role-policy --role-name ROLE_NAME --policy-name opensearch-generic-access

# 5. Destroy AgentCore runtime (removes ECR, CodeBuild, IAM role, runtime)
cd opensearchmcpgeneric
agentcore destroy

---

Project Structure

devopsagent-opensearch-mcp/
├── README.md                           # This file
├── agentcore/
│   ├── agentcore.json                  # AgentCore project config (runtime, VPC, tags)
│   ├── aws-targets.json                # Deployment target (account + region)
│   └── cdk/                            # CDK infrastructure (auto-generated, do not edit)
│       ├── package.json
│       ├── lib/cdk-stack.ts
│       └── ...
├── agents/
│   └── opensearch_mcp_generic/
│       ├── Dockerfile                  # Container definition
│       ├── main.py                     # MCP server entrypoint (streamable HTTP on port 8080)
│       └── requirements.txt            # Python deps: opensearch-mcp-server-py
└── scripts/
    ├── 01-add-iam-policy.sh            # Add ES permissions to runtime role
    ├── 02-create-vpce.sh               # Create VPC Endpoint for AgentCore
    ├── 03-apply-resource-policy.sh     # Restrict runtime to VPCE-only access
    └── 04-setup-cognito-auth.sh        # Create Cognito OAuth + update runtime auth

---

Troubleshooting

Runtime stuck in CREATING

• Check CodeBuild logs: agentcore logs
• Verify the Dockerfile builds correctly
• Check IAM permissions for the CodeBuild execution role

VPCE state is "failed"

• Verify the subnet has available IP addresses
• Check VPC Lattice service quotas in your account
• Ensure no restrictive SCPs blocking VPC Lattice actions

Agent can't reach OpenSearch

• Verify security group allows outbound on port 443
• Check the OpenSearch domain's access policy includes the runtime's IAM role ARN
• Ensure the OpenSearch domain is in the same VPC (or peered)

Resource policy error "invalid action"

• The correct action is bedrock-agentcore:InvokeAgentRuntime (not InvokeRuntime)

---

References

• opensearch-mcp-server-py — The MCP server we deploy
• AWS DevOps Agent — Private Connections — How VPC private connectivity works
• Bedrock AgentCore Documentation — AgentCore runtime management
• AgentCore CLI — CLI for deploying AgentCore projects