From ba97a3a8309d58ffc74a3357b33075ca9d5885dc Mon Sep 17 00:00:00 2001 From: Bean Labs <287763725+beanscg@users.noreply.github.com> Date: Mon, 8 Jun 2026 16:53:25 -0400 Subject: [PATCH] Add rate limits example --- README.md | 7 ++ examples/rate-limits/README.md | 160 ++++++++++++++++++++++++++++++ examples/rate-limits/helio.yaml | 78 +++++++++++++++ examples/rate-limits/package.json | 8 ++ examples/rate-limits/start.mjs | 111 +++++++++++++++++++++ 5 files changed, 364 insertions(+) create mode 100644 examples/rate-limits/README.md create mode 100644 examples/rate-limits/helio.yaml create mode 100644 examples/rate-limits/package.json create mode 100644 examples/rate-limits/start.mjs diff --git a/README.md b/README.md index c18125c..b6cf31d 100644 --- a/README.md +++ b/README.md @@ -141,6 +141,13 @@ That's it. Every tool call now passes through Helio with a full audit trail, rat Want human-in-the-loop approvals for write operations? See [docs/approvals.md](./docs/approvals.md) for the full Slack and dashboard approval flow, or copy [examples/slack-approvals/](./examples/slack-approvals/) as a starting point. +Standalone examples: + +- [Basic](./examples/basic/) - Log, allow, and deny tool calls with a local MCP echo server +- [Rate Limits](./examples/rate-limits/) - Cap repeated tool calls with sliding-window limits +- [Spend Limits](./examples/spend-limits/) - Cap cumulative spend across payment and refund tools +- [Slack Approvals](./examples/slack-approvals/) - Route sensitive actions to Slack approval workflows + ## Features ### Policy Engine diff --git a/examples/rate-limits/README.md b/examples/rate-limits/README.md new file mode 100644 index 0000000..d9f4f78 --- /dev/null +++ b/examples/rate-limits/README.md @@ -0,0 +1,160 @@ +# Rate Limits Example + +A Helio configuration that demonstrates sliding-window rate limits for repeated tool calls. + +## What This Demonstrates + +- Per-tool rate limiting with a shared bucket across all sessions +- The accepted `key: agent` configuration shape, including the current MCP fallback behavior +- Structured self-repair feedback when a request exceeds its limit +- The dashboard Limits page for live bucket state +- Audit trail recording for allowed and rate-limited tool calls + +## Prerequisites + +- Node.js 22+ +- `jq` (optional) for pretty-printing JSON command output. If unavailable, remove `| jq` from curl commands. +- Build the proxy from the repo root: + +```bash +pnpm install && pnpm build +``` + +## Quick Start + +```bash +cd examples/rate-limits +pnpm start +``` + +This starts: + +1. A local MCP echo server on port 8080 (5 demo tools) +2. The Helio proxy on port 3000 +3. The dashboard on port 3100 + +> **Note:** All examples use the same ports (8080, 3000, 3100). Stop any running example before starting another. + +## Try It Out + +> If `jq` is not installed, remove `| jq` from the command snippets below. + +### List available tools + +```bash +curl -s -X POST http://localhost:3000/mcp \ + -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | jq +``` + +### Hit the per-tool limit + +The `limit-weather-lookups` rule allows 3 `get_weather` calls per one-minute window. The fourth call is blocked and returns structured `rate_limited` feedback. + +```bash +for city in London Paris Tokyo Berlin; do + curl -s -X POST http://localhost:3000/mcp \ + -H 'Content-Type: application/json' \ + -d "{\"jsonrpc\":\"2.0\",\"id\":\"$city\",\"method\":\"tools/call\",\"params\":{\"name\":\"get_weather\",\"arguments\":{\"city\":\"$city\"}}}" | jq +done +``` + +This rule uses: + +```yaml +- name: limit-weather-lookups + match: + tool: 'get_weather' + action: rate_limit + limits: + max_calls: 3 + window: 1m + key: tool +``` + +`key: tool` means all `get_weather` calls share one bucket, regardless of session. + +### See the agent-scoped config shape + +The `limit-email-by-agent` rule shows the `key: agent` shape requested by agent-governance deployments: + +```yaml +- name: limit-email-by-agent + match: + tool: 'send_email' + action: rate_limit + limits: + max_calls: 2 + window: 5m + key: agent +``` + +Current MCP requests do not carry agent identity through `McpRequest`, so Helio accepts this config and logs a startup warning that `key: agent` falls back to `key: tool` at runtime. If you need isolation that works today, use `key: session` and pass an MCP session ID with each request. + +## Configuration Walkthrough + +```yaml +version: '1' +``` + +Required. Currently always `"1"`. + +```yaml +upstream: + url: 'http://localhost:8080/mcp' + transport: streamable-http +``` + +The MCP server to govern. All tool calls are forwarded here after policy evaluation. + +```yaml +dashboard: + enabled: true + port: 3100 + allow_open_mode: true +``` + +The dashboard shows live rate-limit buckets on the Limits page. This example intentionally uses local open mode so `pnpm start` works without extra setup; keep this loopback-only. + +```yaml +policies: + default: allow +``` + +Any call that does not match a rate-limit or deny rule is allowed. + +```yaml +- name: block-destructive + match: + annotations: + destructiveHint: true + action: deny +``` + +Destructive tools are denied before rate-limit rules run. + +```yaml +- name: allow-reads + match: + annotations: + readOnlyHint: true + action: allow +``` + +Read-only tools are explicitly allowed after the more specific `get_weather` rate limit has a chance to match. + +```yaml +audit: + storage: sqlite + path: ./helio-audit.db + retention: 90d + include_responses: true +``` + +Allowed and blocked calls are recorded to a local SQLite database for dashboard review and export. + +## Next Steps + +- [Basic](../basic/) - Learn annotation-based allow and deny rules +- [Spend Limits](../spend-limits/) - Cap cumulative spend across payment tools +- [Slack Approvals](../slack-approvals/) - Route sensitive actions to Slack for human approval diff --git a/examples/rate-limits/helio.yaml b/examples/rate-limits/helio.yaml new file mode 100644 index 0000000..ea99b98 --- /dev/null +++ b/examples/rate-limits/helio.yaml @@ -0,0 +1,78 @@ +# Rate Limits configuration +# +# Bound repeated calls with sliding-window tracking. See README.md for +# runnable examples and notes about each key scope. + +version: '1' + +upstream: + url: 'http://localhost:8080/mcp' + transport: streamable-http + +listen: + port: 3000 + host: '127.0.0.1' + +dashboard: + enabled: true + port: 3100 + # Local demo mode: allow unauthenticated dashboard access on loopback only. + allow_open_mode: true + +policies: + default: allow + rules: + - name: block-destructive + match: + annotations: + destructiveHint: true + action: deny + feedback: + message: 'Destructive operations are blocked by policy.' + suggestion: 'Use a non-destructive alternative or request manual action.' + + # Per-tool limit: all get_weather calls share one bucket, across sessions. + - name: limit-weather-lookups + match: + tool: 'get_weather' + action: rate_limit + limits: + max_calls: 3 + window: 1m + key: tool + feedback: + message: 'Weather lookup rate limit exceeded.' + suggestion: 'Wait for the current window to reset before retrying.' + + # Agent-scoped shape: accepted by the config parser today. MCP requests do + # not currently carry agent identity, so Helio warns and falls back to the + # tool bucket at runtime. Use key: session for isolation available today. + - name: limit-email-by-agent + match: + tool: 'send_email' + action: rate_limit + limits: + max_calls: 2 + window: 5m + key: agent + feedback: + message: 'Email send rate limit exceeded.' + suggestion: 'Wait for the current window to reset before retrying.' + + - name: allow-reads + match: + annotations: + readOnlyHint: true + action: allow + +approval: + timeout: 300s + default_on_timeout: deny + channels: + - type: dashboard + +audit: + storage: sqlite + path: ./helio-audit.db + retention: 90d + include_responses: true diff --git a/examples/rate-limits/package.json b/examples/rate-limits/package.json new file mode 100644 index 0000000..2acf68e --- /dev/null +++ b/examples/rate-limits/package.json @@ -0,0 +1,8 @@ +{ + "name": "@gethelio/example-rate-limits", + "private": true, + "type": "module", + "scripts": { + "start": "node start.mjs" + } +} diff --git a/examples/rate-limits/start.mjs b/examples/rate-limits/start.mjs new file mode 100644 index 0000000..158324b --- /dev/null +++ b/examples/rate-limits/start.mjs @@ -0,0 +1,111 @@ +/** + * Start script for the rate limits example. + * + * Spawns the shared MCP echo server, waits for it to be ready, + * then starts the Helio proxy with the local helio.yaml config. + */ + +import { spawn } from 'node:child_process' +import { dirname, resolve } from 'node:path' +import { fileURLToPath } from 'node:url' +import { registerCleanup, waitForHealthcheck } from '../_shared/start-helpers.mjs' + +const __dirname = dirname(fileURLToPath(import.meta.url)) +const echoServer = resolve(__dirname, '..', '_shared', 'mcp-echo-server.mjs') +const proxyCli = resolve(__dirname, '..', '..', 'packages', 'proxy', 'dist', 'cli.js') +const config = resolve(__dirname, 'helio.yaml') + +const children = [] +const state = { exitCode: 0 } +const cleanup = registerCleanup(children, state) + +// Start the echo server +const echo = spawn('node', [echoServer], { + stdio: 'inherit', + env: { ...process.env, HOST: '127.0.0.1', PORT: '8080' }, +}) +children.push(echo) + +echo.on('error', (err) => { + console.error('Failed to start echo server:', err.message) + process.exit(1) +}) + +// Wait for echo server to be ready +try { + await waitForHealthcheck('http://127.0.0.1:8080/healthz') +} catch (err) { + console.error(err.message) + state.exitCode = 1 + cleanup() +} + +// Start the Helio proxy +const proxy = spawn('node', [proxyCli, 'start', '-c', config], { + stdio: 'inherit', +}) +children.push(proxy) + +proxy.on('error', (err) => { + console.error('Failed to start proxy:', err.message) + state.exitCode = 1 + cleanup() +}) + +proxy.on('exit', (code) => { + if (code !== 0) { + console.error(`Proxy exited with code ${code}`) + state.exitCode = code + } + cleanup() +}) + +// Wait for proxy to be ready, then prime the annotation cache +try { + await waitForHealthcheck('http://127.0.0.1:3100/api/health') + await fetch('http://127.0.0.1:3000/mcp', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ jsonrpc: '2.0', id: 0, method: 'tools/list' }), + }) +} catch (err) { + console.error('Failed to connect to proxy:', err.message) + console.error('Hint: ensure the proxy is built (pnpm build from repo root)') + state.exitCode = 1 + cleanup() +} + +console.log(` +───────────────────────────────────────── + Helio Rate Limits Example +───────────────────────────────────────── + + Dashboard: http://localhost:3100 + Proxy: http://localhost:3000/mcp + + Try these commands in sequence to hit the per-tool limit: + + # Weather lookup 1 (1/3 used) + curl -s -X POST http://localhost:3000/mcp \\ + -H 'Content-Type: application/json' \\ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"get_weather","arguments":{"city":"London"}}}' | jq + + # Weather lookup 2 (2/3 used) + curl -s -X POST http://localhost:3000/mcp \\ + -H 'Content-Type: application/json' \\ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_weather","arguments":{"city":"Paris"}}}' | jq + + # Weather lookup 3 (3/3 used) + curl -s -X POST http://localhost:3000/mcp \\ + -H 'Content-Type: application/json' \\ + -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"get_weather","arguments":{"city":"Tokyo"}}}' | jq + + # Weather lookup 4 (blocked until the one-minute window resets) + curl -s -X POST http://localhost:3000/mcp \\ + -H 'Content-Type: application/json' \\ + -d '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"get_weather","arguments":{"city":"Berlin"}}}' | jq + + # Check rate-limit status on the Dashboard Limits page + +───────────────────────────────────────── +`)