From f1a1a43127d40fb19e884b023d7ec55ddea9a623 Mon Sep 17 00:00:00 2001 From: Joshua Gilman Date: Tue, 25 Aug 2026 10:47:27 -0700 Subject: [PATCH] docs: correct worker-boundary claims and align style Post-refactor docs review (session 008 worker cutover left stale claims): - Fix the two systemic accuracy errors: only main's final value is exposed to the caller (native args/results do cross the private worker protocol), and MaxExecutionTime excludes kill/reap overhead. - Fix probe detection and MaxConcurrentExecutions scope in the API reference to match builder.go and limits.go godoc. - Standardize terminology: worker process, native call, top-level source loading. - Replace ungrounded 'production hosts normally' claims with direct conditions; split multi-action steps; format literal identifiers. - Rework index.md into a Diataxis orientation page; add tutorial prerequisites; retitle the security explanation; add functionally verified verification sections to both how-to guides. --- CONTRIBUTING.md | 8 ++- README.md | 11 ++-- SECURITY.md | 14 ++-- docs/docs/explanation/security-model.md | 52 +++++++-------- docs/docs/how-to/disable-capabilities.md | 32 +++++---- docs/docs/how-to/use-rego-authorization.md | 51 ++++++++++++++- docs/docs/index.md | 76 +++++----------------- docs/docs/reference/mcp-tools.md | 17 +++-- docs/docs/reference/public-api.md | 42 ++++++------ docs/docs/tutorials/first-server.md | 19 ++++-- docs/mkdocs.yml | 2 +- 11 files changed, 178 insertions(+), 146 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 921663f..53bc546 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -34,8 +34,10 @@ Keep the root [README](README.md) concise and link to the detailed pages instead mise install ``` -2. Keep the change focused. Add or update behavior tests when a public contract changes. Update the user documentation when behavior visible through the root, `authz`, or `mcpserver` packages changes. -3. Validate compile-checked examples and documentation directly when either changes: +2. Keep the change focused. +3. When a public contract changes, add or update behavior tests. +4. When behavior visible through the root, `authz`, or `mcpserver` packages changes, update the user documentation. +5. When compile-checked examples or documentation change, validate them directly: ```sh go test ./... -count=1 @@ -43,7 +45,7 @@ Keep the root [README](README.md) concise and link to the detailed pages instead ``` `go test` compiles and runs the Go examples with the package tests. `docs:build` runs the strict MkDocs build with the locked documentation environment. -4. Before requesting review, run the complete repository check: +6. Before requesting review, run the complete repository check: ```sh moon run root:check diff --git a/README.md b/README.md index 4b36ca2..2f0efc3 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ The repository also contains shorter, compile-checked examples: - [`example_test.go`](example_test.go) — typed capability registration, explicit `authz.AllowAll()`, default limits, and direct execution - [`mcpserver/example_test.go`](mcpserver/example_test.go) — trusted typed-context resolution and the official in-memory MCP transport -`authz.AllowAll()` is deliberate in the simple examples and permits every validated invocation. Production hosts normally supply an authorization policy. +`authz.AllowAll()` is deliberate in the simple examples and permits every native call whose arguments bind successfully. Use `authz.AllowAll()` only when every resolved subject may call every enabled capability; otherwise supply an `authz.Authorizer`. The final host binary must call `codemode.ServeWorkerAndExit()` as the first statement of `main`, before flag parsing or any other setup. Test binaries that @@ -40,10 +40,13 @@ call `Builder.Build` must do the same from `TestMain` before `m.Run`. ## Security boundary -Each `execute` request runs Starlark in a fresh process created by re-executing +Each `execute` request runs Starlark in a fresh worker process created by re-executing the host binary. The submitted program must define a zero-argument `main()`, -and only its final converted value crosses the execution boundary. Prints, -globals, and intermediate values do not. +and only its final converted value is exposed to the caller. Printed text, +globals, and interpreter-local intermediate values are not returned. During +execution, each native-call argument map and each validated native result +crosses the private worker protocol, and each crossing value is independently +subject to `MaxValueDepth` and `MaxValueBytes`. Capabilities are bound and canonicalized in the parent before authorization, and authorization completes before handler dispatch. Trusted subjects come diff --git a/SECURITY.md b/SECURITY.md index ed58abf..b05fff9 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -23,19 +23,19 @@ Include the following details when available: CodeMode executes each submitted program in a fresh worker process created by re-executing the host binary. Module loading is disabled, native capabilities are limited to the immutable set registered and enabled by the host, and native -calls are rejected during top-level program loading. A program must define a +calls are rejected during top-level source loading. A program must define a zero-argument `main()` function, and only its converted return value is exposed. Configured limits bound source bytes, interpreter steps, elapsed time, -attempted native calls, concurrent worker processes, and the depth and encoded +attempted native calls, concurrent workers, and the depth and encoded size of every value that crosses the worker boundary. Search query bytes and search result counts are bounded separately in the parent. An elapsed deadline -or request cancellation kills and reaps the worker process. Each validated -native call is rebound in the parent and passes through the host-supplied -`authz.Authorizer` before its handler runs. The MCP adapter resolves the subject -from trusted Go context and ignores identity or credential claims in tool -arguments and request `_meta`. +or request cancellation kills and reaps the worker. Each native call whose +arguments bind successfully is rebound in the parent and passes through the +host-supplied `authz.Authorizer` before its handler runs. The MCP adapter +resolves the subject from trusted Go context and ignores identity or +credential claims in tool arguments and request `_meta`. The worker boundary prevents Starlark from sharing interpreter state with the host and permits hard preemption of Starlark execution. It is not a hard tenant diff --git a/docs/docs/explanation/security-model.md b/docs/docs/explanation/security-model.md index e4f4991..3ee6b32 100644 --- a/docs/docs/explanation/security-model.md +++ b/docs/docs/explanation/security-model.md @@ -1,11 +1,11 @@ --- -title: Security model +title: Understanding CodeMode's security model description: Trust boundaries, authorization order, cancellation, and worker-process containment in CodeMode. --- -# Security model +# Understanding CodeMode's security model -CodeMode separates client-controlled Starlark from host-controlled identity, policy, and Go handlers. The separation depends on the host establishing a trusted request context, installing the worker entry point, and ensuring authorizers and handlers follow their contracts. CodeMode does not authenticate clients or provide operating-system tenant quotas. +CodeMode separates client-controlled Starlark from host-controlled identity, policy, and Go handlers. The separation depends on the host establishing a trusted request context, installing the worker process entry point, and ensuring authorizers and handlers follow their contracts. CodeMode does not authenticate clients or provide operating-system tenant quotas. ## The host establishes identity @@ -24,13 +24,13 @@ Credentials stay in the host's authentication layer. They are not fields on `aut ## Validation precedes authority -A native capability call crosses three boundaries in a fixed order: +A native call crosses three boundaries in a fixed order: 1. **Exact binding and canonicalization.** Duplicate keyword syntax is rejected by the Starlark parser as `ErrInvalidProgram` before this step. Positional, missing, unknown, incorrectly typed, and out-of-range arguments reach binding and map to `ErrInvalidArguments`. Successful binding creates the exact registered Go input and a fresh JSON-shaped argument map. 2. **Authorization.** CodeMode passes the trusted subject, stable capability ID, dotted capability name, and canonical arguments to `authz.Authorizer`. 3. **Handler dispatch.** CodeMode calls the typed handler only if authorization returns `nil`. -This order prevents policy from interpreting malformed Starlark values and prevents a handler from running before policy has evaluated the exact input it will receive. The canonical map is separate from the typed handler input, so policy cannot rewrite the handler's arguments by mutating the map. +This order prevents policy from interpreting malformed Starlark values and prevents a handler from running before policy has evaluated the exact input the handler will receive. The canonical map is separate from the typed handler input, so policy cannot rewrite the handler's arguments by mutating the map. An authorizer reports a recognized denial with an error that wraps `authz.ErrDenied`. CodeMode classifies that outcome as `permission denied` and does not dispatch the handler. Any other authorizer error, and an authorizer panic recovered at the boundary, becomes `authorization policy failure`. Policy diagnostic text does not cross the MCP boundary. @@ -40,7 +40,7 @@ Authorization is evaluated for every attempted native call whose arguments bind `authz.AllowAll()` returns an authorizer because the server never treats a missing authorizer as permission. The simple example uses it deliberately so that a minimal server is complete and the absence of policy logic is visible. -`AllowAll` approves every valid native call for every resolved subject. It is not authentication and does not inspect capability identity or arguments. Production hosts normally supply an authorizer that evaluates the resolved subject, the stable capability ID, and the canonical arguments. A production host should use `AllowAll` only when unrestricted access to every enabled capability is the intended policy. +`AllowAll` approves every native call for every resolved subject. It is not authentication and does not inspect capability identity or arguments. Use `authz.AllowAll()` only when every resolved subject may call every enabled capability; otherwise supply an `authz.Authorizer` that evaluates the resolved subject, the stable capability ID, and the canonical arguments. ## Rego policy runs in process @@ -56,7 +56,7 @@ CodeMode installs no schema set and no schema resolver. Metadata `schema["https: The configured decision is one direct, ground `data` reference. Construction validates that reference syntax and prepares the policy; it cannot prove that the decision is defined and Boolean for every future input. A ground decision is either undefined or yields one value. That value must be Boolean. Boolean `true` allows a call. Boolean `false` is a recognized denial. Undefined and non-Boolean decisions are policy failures, as are evaluation and builtin errors. A total decision with `default allow := false` turns unmatched input into an intentional denial while still failing closed when the policy contract is broken. -These controls restrict policy inputs and evaluator capabilities, not resource consumption or process authority. OPA, authorizers, and handlers run inside the host process; moving Starlark to a worker does not isolate Rego policy. A host that does not trust its policy authors needs an external process or container boundary for policy evaluation. +These controls restrict policy inputs and evaluator capabilities, not resource consumption or process authority. OPA, authorizers, and handlers run inside the host process; moving Starlark to a worker process does not isolate Rego policy. A host that does not trust its policy authors needs an external process or container boundary for policy evaluation. See [Use Rego for authorization](../how-to/use-rego-authorization.md) for configuration and the [`authz/rego` API reference](../reference/public-api.md#authzrego) for the exact input and result contracts. @@ -82,20 +82,20 @@ An allowed or denied call proves only the result for that subject, capability, a ## Execution state does not cross calls -Every `Server.Execute` call starts a fresh process by re-executing the host -binary. The child receives only the immutable enabled-capability manifest, -positive execution limits, and one submitted program through CodeMode's private -protocol. It constructs a fresh Starlark interpreter. Module loading is -disabled, and the only predeclared application functions are the enabled -capability namespace. Native calls are rejected while top-level source is -loading and are accepted only while the required zero-argument `main()` -function runs. - -After `main` returns, the worker converts only its final value to -type-preserving wire data under the depth and encoded-size limits. Printed text -is discarded. Globals, source-loading values, intermediate expressions, and -unrelated native results do not cross the boundary. No Starlark globals or -mutable interpreter state carry into the next execute call. +Every `Server.Execute` call starts a fresh worker process by re-executing the +host binary. The worker process is a child of the host process. The worker +receives only the immutable enabled-capability manifest, positive execution +limits, and one submitted program through CodeMode's private protocol. It +constructs a fresh Starlark interpreter. Module loading is disabled, and the +only predeclared application functions are the enabled capability namespace. +Native calls are rejected during top-level source loading and are accepted +only while the required zero-argument `main()` function runs. + +After `main` returns, only the final converted value is exposed to the caller. +Printed text, globals, and interpreter-local intermediate values are not +returned. Unrelated native results are not exposed in that final caller +result. No Starlark globals or mutable interpreter state carry into the next +execute call. Capability handlers do not run in the worker. A native call crosses the private protocol, is rebound to the exact registered input type in the parent, is @@ -111,10 +111,10 @@ gave that process. CodeMode derives one execution context from `MaxExecutionTime` and the request context. The elapsed budget starts before waiting for a worker slot and covers -process startup, protocol exchange, Starlark execution, parent authorization -and dispatch, and worker cleanup. The interpreter separately enforces source, -bytecode-step, attempted-native-call, crossing-value depth, and crossing-value -size limits. +spawn, protocol exchange, Starlark execution, and parent dispatch. Killing and +reaping the worker can add operating-system overhead beyond the budget. The +interpreter separately enforces source, bytecode-step, attempted-native-call, +crossing-value depth, and crossing-value size limits. When the execution context ends, the parent closes the worker pipes, kills the worker if necessary, and reaps it exactly once. This hard-preempts Starlark, @@ -153,7 +153,7 @@ operating-system user and re-executes the host binary. CodeMode supplies an environment containing only its private worker marker and passes no extra file descriptors, but it provides no operating-system CPU or memory quota. Package initialization and any setup placed before the worker entry point still run in -the child with that user's filesystem and network authority. +the worker with that user's filesystem and network authority. The restricted Starlark environment exposes no file, network, environment, or process built-ins. Native access is limited to the enabled capability manifest diff --git a/docs/docs/how-to/disable-capabilities.md b/docs/docs/how-to/disable-capabilities.md index b6b1150..9cc7ef1 100644 --- a/docs/docs/how-to/disable-capabilities.md +++ b/docs/docs/how-to/disable-capabilities.md @@ -25,23 +25,33 @@ Use the stable ID, `records.entry.lookup`, not the model-facing dotted name, `re Continue to call `codemode.Register` for `records.entry.lookup` before `builder.Build()`. CodeMode validates the complete registration set before applying the filter, and `Build` rejects a disabled ID that does not identify a registered capability. Static filtering cannot hide invalid or conflicting registrations. -The explicit `authz.AllowAll()` remains deliberate only for the simple tutorial. Production hosts normally supply an authorization policy. Static filtering and per-call authorization solve different problems: filtering fixes the deployment-wide surface, while the authorizer decides whether a trusted subject may make an enabled native call. +The explicit `authz.AllowAll()` remains deliberate only for the simple tutorial. Use `authz.AllowAll()` only when every resolved subject may call every enabled capability; otherwise supply an `authz.Authorizer`. Static filtering and per-call authorization solve different problems: filtering fixes the deployment-wide surface, while the authorizer decides whether a trusted subject may make an enabled native call. ## Replace the built server -Call `Build` after registering all capabilities, then pass the resulting server to `mcpserver.New` as usual. An existing immutable server does not change when the builder options change. Replace the server instance and its host-owned transport lifecycle to deploy a different enabled set. +After registering all capabilities, call `Build`. Pass the returned server to `mcpserver.New`. To deploy a different enabled set, replace the existing server instance and its host-owned transport lifecycle. -## Check the resulting surface +## Verify the filtered surface -With the tutorial's unchanged call order, `search_api` prints `[]`, then `describe_api` returns the expected tool error and the helper exits. The later `execute` call is not reached in that run. The list below describes the complete contract of the built server, not the output of one sequential client run. +The tutorial's `callTool` helper returns on the first tool error, so `describe_api` would stop the program before `execute`. Replace the `result.IsError` branch in `callTool` with: -After `records.entry.lookup` is disabled: +```go +if result.IsError { + text, ok := result.Content[0].(*mcp.TextContent) + if !ok { + return fmt.Errorf("%s returned a tool error: %v", params.Name, result.Content) + } + fmt.Printf("%s: %s\n", params.Name, text.Text) + return nil +} +``` -- `search_api` does not return `records.lookup`, even when the query matches its name or summary. -- `describe_api` for `records.lookup` returns the same not-found classification used for any unavailable name. -- The generated Starlark namespace has no `records.lookup` callable. If no other enabled capability starts with `records.`, the `records` namespace is absent; otherwise only `lookup` is absent. -- An `execute` program cannot call the disabled capability. The request fails before authorization or handler dispatch. +Run `go run .`. The program prints: -These effects come from one filtered catalog used by discovery and execution. The filter does not leave a hidden execution path. +``` +search_api: [] +describe_api: capability not found +execute: invalid program +``` -See the [public API reference](../reference/public-api.md) for `Options` and build validation, and the [MCP tools reference](../reference/mcp-tools.md) for discovery and execution results. +See the [public API reference](../reference/public-api.md#static-capability-filtering) for the complete filter contract, and the [MCP tools reference](../reference/mcp-tools.md) for discovery and execution results. diff --git a/docs/docs/how-to/use-rego-authorization.md b/docs/docs/how-to/use-rego-authorization.md index 6c79e20..056833e 100644 --- a/docs/docs/how-to/use-rego-authorization.md +++ b/docs/docs/how-to/use-rego-authorization.md @@ -53,7 +53,13 @@ Starting from the [first-server tutorial](../tutorials/first-server.md), import import "github.com/meigma/codemode/authz/rego" ``` -Keep the policy in memory, then replace the tutorial's builder initialization with: +Fetch the adapter's dependencies: + +```sh +go mod tidy +``` + +Define the policy at package scope: ```go const authorizationPolicy = `package codemode.authz @@ -65,7 +71,11 @@ allow if { input.capability.id == "records.entry.lookup" input.arguments.key != "forbidden" }` +``` +Then replace the tutorial's builder initialization with the following code: + +```go authorizer, err := rego.New( ctx, "data.codemode.authz.allow", @@ -83,6 +93,8 @@ builder := codemode.New(codemode.Options{ }) ``` +The replacement declares `err`. In the tutorial's next statement, change `err := codemode.Register(` to `err = codemode.Register(` so the program still compiles. + `rego.New` validates the ground reference syntax, compiles every supplied module, and prepares the policy synchronously. It cannot prove that the decision is defined and Boolean for every future input. Return the error instead of building a server with another authorizer. The returned authorizer is immutable and can serve concurrent calls. To embed the policy, save the same module as `policies/authorization.rego` and replace the string constant with: @@ -107,4 +119,39 @@ CodeMode calls the authorizer after argument binding and before handler dispatch Do not use an undefined decision as denial behavior. Keep `default allow := false` so an unmatched input produces the intentional Boolean `false` result. -See the [`authz/rego` API reference](../reference/public-api.md#authzrego) for constructor validation and exact result semantics. See the [security model](../explanation/security-model.md#rego-policy-runs-in-process) for the policy trust boundary and runtime restrictions. +## Verify the policy + +Run the adapted first-server program with the tutorial's `execute` source, which calls `records.lookup(key="alpha", limit=2)`: + +```sh +go run . +``` + +The `execute` line is: + +``` +execute: {"result":{"count":2,"key":"alpha"}} +``` + +Change the `execute` source to `return records.lookup(key="forbidden", limit=2)`. + +The tutorial's `callTool` helper formats a tool error with `%v` and stops the program, so the denial text is not visible as written. To print it, replace the `result.IsError` branch in `callTool` with: + +```go +if result.IsError { + text, ok := result.Content[0].(*mcp.TextContent) + if !ok { + return fmt.Errorf("%s returned a tool error: %v", params.Name, result.Content) + } + fmt.Printf("%s: %s\n", params.Name, text.Text) + return nil +} +``` + +Run `go run .` again. `search_api` and `describe_api` still succeed, and the `execute` line is: + +``` +execute: permission denied +``` + +See the [`authz/rego` API reference](../reference/public-api.md#authzrego) for constructor validation and exact result semantics. See [Understanding CodeMode's security model](../explanation/security-model.md#rego-policy-runs-in-process) for the policy trust boundary and runtime restrictions. diff --git a/docs/docs/index.md b/docs/docs/index.md index 65ff151..9e21ae5 100644 --- a/docs/docs/index.md +++ b/docs/docs/index.md @@ -1,79 +1,35 @@ --- title: CodeMode slug: / -description: Reference for bounded Starlark access to registered Go capabilities over MCP. +description: Find the tutorial, how-to guides, reference, and explanation for CodeMode. --- # CodeMode -CodeMode is a Go library that exposes registered, typed Go capabilities to Model Context Protocol (MCP) clients through Starlark programs. The current implementation consists of an immutable capability server, a fresh-process Starlark worker, an authorization interface, an optional in-process Rego authorizer, and an adapter for the official MCP Go SDK. +CodeMode is a Go library that exposes registered, typed Go capabilities to Model Context Protocol (MCP) clients through Starlark programs. Clients use three MCP tools: `search_api`, `describe_api`, and `execute`. Each `execute` call runs Starlark in a fresh worker process, and only the final converted value returned by zero-argument `main()` is exposed to the caller. The pages below are organized by reader need. -## Public packages +## Tutorials -CodeMode has four public packages: +Learn the workflow by building a working server. -- `github.com/meigma/codemode` registers typed capabilities, filters disabled capabilities, searches and describes the enabled catalog, and executes bounded Starlark programs. -- `github.com/meigma/codemode/authz` defines the trusted subject, canonical authorization input, authorizer interface, and recognized denial. -- `github.com/meigma/codemode/authz/rego` prepares a static in-memory Rego decision that implements the authorizer interface. -- `github.com/meigma/codemode/mcpserver` binds a CodeMode service and trusted invocation resolver to an official MCP SDK server. +- [Build your first CodeMode server](tutorials/first-server.md) — register one typed capability and call `search_api`, `describe_api`, and `execute` over the official MCP in-memory transport. -See [Use Rego for authorization](how-to/use-rego-authorization.md) to replace `authz.AllowAll()`. The [public API reference](reference/public-api.md#authzrego) defines the exact Go and policy contracts, and the [security model](explanation/security-model.md#rego-policy-runs-in-process) explains the Rego host-process boundary. +## How-to guides -## MCP surface +Complete a specific host-configuration task. -`mcpserver.New` exposes exactly three tools: +- [Use Rego for authorization](how-to/use-rego-authorization.md) — replace `authz.AllowAll()` with a prepared in-process Rego decision. +- [Disable capabilities for a deployment](how-to/disable-capabilities.md) — remove registered capabilities from every live surface by stable capability ID. -| Tool | Required input | Successful structured output | -| --- | --- | --- | -| `search_api` | `{"query": string}` | An array of `{name, signature, summary}` records for matching enabled capabilities | -| `describe_api` | `{"name": string}` with one exact capability name | `{name, signature, summary, description, input, output}` | -| `execute` | `{"source": string}` with one Starlark program | `{"result":
}` | +## Reference -Search is case-normalized over capability names and summaries, returns name-sorted results, and returns an empty array for a blank query. Description uses exact names. A disabled capability is absent from search, description, the Starlark namespace, and execution. +Look up exact contracts, defaults, and wire shapes. -Each program must define a zero-argument `main()` function. CodeMode rejects native capability calls made while top-level source is loading. It discards printed text and returns only the converted value from `main` under the `result` key. +- [Public API reference](reference/public-api.md) — exported Go types, builder lifecycle, limits, and authorization contracts. +- [MCP tool reference](reference/mcp-tools.md) — inputs, listed descriptions, successful outputs, and errors for the three MCP tools. -## Trust model +## Explanation -The host authenticates the request and stores a non-secret `authz.Subject` in typed, host-owned Go context. An `mcpserver.InvocationResolver` reads that context for every tool call. Resolver failure or an empty subject stops the request before discovery or execution. Tool arguments and MCP `_meta` are untrusted and cannot replace the resolved subject or provide credentials, budgets, modules, or capability filters. +Understand why the trust boundaries exist and what they do not cover. -For a native capability call, CodeMode first binds the Starlark arguments to the registered Go input type. It then calls the host's `authz.Authorizer` with: - -- the resolved subject -- the stable capability ID -- the model-facing capability name -- a fresh JSON-shaped copy of the validated arguments - -A recognized denial prevents handler dispatch. Client-facing MCP failures use coarse error classifications rather than forwarding source, arguments, credentials, policy diagnostics, stack traces, or handler error text. - -The host owns transport startup, authentication, listeners, request context, and shutdown. CodeMode has no generic downstream MCP forwarding path. The final host binary must install `codemode.ServeWorkerAndExit()` as the first statement of `main`, and test binaries that build a server must install it before `m.Run` in `TestMain`. - -## Runtime limits - -`codemode.DefaultLimits()` currently sets: - -| Limit | Default | -| --- | ---: | -| Starlark source | 64 KiB | -| Interpreter steps | 1,000,000 | -| Elapsed execution time | 5 seconds | -| Attempted native calls | 100 | -| Crossing-value depth | 32 | -| Encoded crossing value | 1 MiB | -| Search query | 256 bytes | -| Search results | 20 | -| Concurrent execution workers | 8 | - -Every configured limit must be positive; zero does not mean unlimited. Each -execution gets a fresh interpreter in a re-executed worker process and its own -budgets. The elapsed budget includes semaphore waiting, process startup, -protocol exchange, Starlark work, parent dispatch, and worker cleanup. Module -loading is disabled, and only the frozen namespace built from enabled -capabilities is predeclared. - -The parent can kill and reap Starlark workers when a deadline or request -cancellation occurs. This is not a complete tenant boundary: workers have no -CodeMode-provided operating-system CPU or memory quota, and authorizers and -handlers run as Go code with the host process's privileges. Parent Go code must -honor context cancellation and return promptly because CodeMode cannot forcibly -stop it after dispatch. +- [Understanding CodeMode's security model](explanation/security-model.md) — identity, authorization order, worker containment, cancellation, and tenant-isolation limits. diff --git a/docs/docs/reference/mcp-tools.md b/docs/docs/reference/mcp-tools.md index 859d376..bc094d8 100644 --- a/docs/docs/reference/mcp-tools.md +++ b/docs/docs/reference/mcp-tools.md @@ -68,7 +68,7 @@ The structured content itself is the array described above, not an object that w ## `describe_api` -Describe one enabled capability by the exact name returned by search_api, without whitespace or case changes. +Describe one enabled capability by the exact name returned by `search_api`, without whitespace or case changes. ### Input @@ -153,7 +153,7 @@ The sample capability therefore describes input fields `key` (`str`, required) a ## `execute` -Execute one Starlark program that defines def main(): with zero arguments, calls only names confirmed through search_api and describe_api inside main, and returns main's final result. +Execute one Starlark program that defines `def main():` with zero arguments, calls only names confirmed through `search_api` and `describe_api` inside `main`, and returns `main`'s final result. ### Input @@ -170,7 +170,7 @@ Execute one Starlark program that defines def main(): with zero arguments, calls } ``` -The source must define `def main():` as a function with zero arguments. Source loading cannot call native capabilities; calls are accepted only while `main` runs, and only for names confirmed through `search_api` and `describe_api`. Module loading is disabled. +The source must define `def main():` as a function with zero arguments. Top-level source loading cannot make native calls; calls are accepted only while `main` runs, and only for names confirmed through `search_api` and `describe_api`. Module loading is disabled. Capabilities are available by dotted name. The sample native call is `records.lookup(key="alpha", limit=2)`. Native calls accept keyword arguments only. Duplicate keyword syntax is rejected by the Starlark parser as `invalid program` before authorization or handler dispatch. Positional, unknown, missing, incorrectly typed, and out-of-range arguments reach binding and map to `invalid capability arguments`. For the sample, `key` is required and `limit` can be omitted, `None`, or an integer in the signed 64-bit range. @@ -201,7 +201,7 @@ The `result` property is the final converted return value from `main`. Its runti `MaxValueDepth` is inclusive. A scalar or `None` is depth 1. Each tuple, list, or dictionary wrapper adds one. A scalar with limit 1 succeeds, a one-level container with limit 2 succeeds, and one more wrapper with limit 2 fails. Native arguments, native results, and the final value are independently subject to `MaxValueDepth` and `MaxValueBytes`. Starlark tuples and lists become arrays. `None` becomes `null`. Dictionaries must have string keys. -Only the final converted value crosses from the worker into the successful MCP result. `print` output is discarded. Globals, source-loading values, intermediate expressions, and native results that are not included in the final return value are not added to structured output. The successful envelope contains only `result`. +Only the final converted value from the worker process is exposed in the successful MCP result. `print` output is discarded. Globals, values created during top-level source loading, intermediate expressions, and native results that are not included in the final return value are not added to structured output. The successful envelope contains only `result`. ## Authoring and recovery @@ -210,9 +210,12 @@ The listed descriptions above are the model-facing contract. Recovery uses the s - Search with a short literal substring over enabled names and summaries. If the result is empty, retry with a shorter term. - After `capability not found`, search again and pass `describe_api` an exact returned `name`, without whitespace or case changes. - After `invalid capability arguments`, compare the call with the published `signature` and `input` field shapes. -- After `invalid program`, use a zero-argument `def main():`, call only names confirmed through search and description inside `main`, and return the final value. +- After `invalid program`, check the program against these requirements: + - Define `main` with zero arguments. + - Call only names confirmed through `search_api` and `describe_api`, and call them only inside `main`. + - Return the final value from `main`. - Use `describe_api.output` as the result contract. Do not parse a type name from `signature`. -- After `resource limit exceeded`, reduce source or result complexity and retry. +- After `resource limit exceeded`, reduce the applicable bounded quantity: query or source bytes, execution steps or time, native calls, or crossing-value depth or encoded size. Then retry the call. - After `permission denied` or `authorization policy failure`, contact the host if the access was expected. One allowed or denied input cannot establish whether the policy is default-open, default-deny, complete, or incomplete. ## Errors @@ -235,4 +238,4 @@ After a well-formed call reaches the adapter, a resolver or service failure beco Malformed MCP arguments are rejected by the SDK's input-schema validation and do not call the invocation resolver. These errors can identify malformed client-owned fields or values because validation occurs before trusted resolution. For well-formed inputs, resolver failure stops the request before search, description, or execution and becomes `unauthenticated` without resolver detail. -See [Public API reference](public-api.md) for Go contracts and [Security model](../explanation/security-model.md) for the resolver and execution trust boundaries. +See [Public API reference](public-api.md) for Go contracts and [Understanding CodeMode's security model](../explanation/security-model.md) for the resolver and execution trust boundaries. diff --git a/docs/docs/reference/public-api.md b/docs/docs/reference/public-api.md index d51d9d8..7f9810a 100644 --- a/docs/docs/reference/public-api.md +++ b/docs/docs/reference/public-api.md @@ -28,7 +28,7 @@ func main() { ``` `ServeWorkerAndExit` returns immediately in an ordinary host process. In a -CodeMode worker, it serves exactly one private probe or execution request over +worker process, it serves exactly one private probe or execution request over standard input and output and then terminates the process. Because it may call `os.Exit`, deferred functions in the worker do not run. @@ -43,9 +43,9 @@ func TestMain(m *testing.M) { ``` `IsWorker()` reports whether the current process was re-executed as a CodeMode -worker. Most hosts should call `ServeWorkerAndExit` directly. `IsWorker` exists -for frameworks that must perform minimal setup before they can delegate to the -worker service; it is not a substitute for serving worker mode. +worker. Unless a framework requires minimal setup before worker delegation, +call `ServeWorkerAndExit` directly. For such frameworks, use `IsWorker` only +to perform that setup; `IsWorker` does not serve worker mode. ### Capability registration @@ -129,9 +129,12 @@ options, or capability visibility. `Build` rejects a missing authorizer, non-positive limits, namespace collisions, duplicate disabled IDs, and disabled IDs that do not match a registered capability. It also re-executes the current binary and performs a -fixed five-second private worker probe. A missing or incorrectly ordered worker -entry returns `ErrInvalidRegistration` with an actionable host-wiring -diagnostic. The fixed probe deadline is independent of `MaxExecutionTime`. +fixed five-second private worker probe. The probe detects an absent or +nonfunctional worker entry and returns `ErrInvalidRegistration` with a +host-wiring diagnostic that identifies the missing or misplaced +`ServeWorkerAndExit` call. It cannot detect ordinary host work that completes +silently before `ServeWorkerAndExit`; first-statement ordering is a host +obligation. The fixed probe deadline is independent of `MaxExecutionTime`. ### Static capability filtering @@ -152,13 +155,13 @@ The filter is deployment configuration, not a per-request policy. Use an `authz. | --- | ---: | --- | | `MaxSourceBytes int` | 65,536 bytes (64 KiB) | Starlark source before execution. | | `MaxExecutionSteps uint64` | 1,000,000 | Starlark bytecode steps for one execution. | -| `MaxExecutionTime time.Duration` | 5 seconds | Elapsed time from waiting for a worker slot through worker reap. | +| `MaxExecutionTime time.Duration` | 5 seconds | Elapsed time starting before waiting for a worker slot; covers spawn, protocol exchange, Starlark execution, and parent dispatch. | | `MaxNativeCalls uint64` | 100 | Attempted native calls in one execution. | | `MaxValueDepth int` | 32 | Inclusive nesting depth of any value crossing the worker boundary. | | `MaxValueBytes int` | 1,048,576 bytes (1 MiB) | Type-preserving encoding of any value crossing the worker boundary. | | `MaxSearchQueryBytes int` | 256 bytes | Raw search query before trimming or case normalization. Whitespace padding counts. | | `MaxSearchResults int` | 20 | Search results returned. | -| `MaxConcurrentExecutions int` | 8 | Live execution-worker processes. | +| `MaxConcurrentExecutions int` | 8 | Concurrent spawn attempts and live worker processes. | `Limits.Validate()` returns `ErrInvalidRegistration` if any field is zero or otherwise non-positive. Zero never means unlimited. Passing a zero-value @@ -175,16 +178,17 @@ complete protocol frame. The worker frame cap adds the fixed envelope and, for native calls, the longest enabled encoded capability ID. Build rejects a catalog and limit combination whose largest legal frame cannot be represented. -`MaxExecutionTime` includes semaphore waiting, process startup, protocol -exchange, Starlark work, parent authorization and dispatch, and worker cleanup. +`MaxExecutionTime` starts before waiting for a worker slot and covers spawn, +protocol exchange, Starlark execution, and parent dispatch. Killing and +reaping the worker can add operating-system overhead beyond the budget. If the request context ends first, its cancellation or deadline wins. -`MaxConcurrentExecutions` bounds live execution workers only; the one-shot -build probe is outside that semaphore. +`MaxConcurrentExecutions` bounds concurrent spawn attempts and live worker +processes. The one-shot build probe is outside that semaphore. The parent kills and reaps the Starlark worker when the execution context ends. Authorizers and handlers run in the parent. CodeMode can return without waiting for dispatched Go code, but it cannot forcibly stop that code or undo its side -effects. See [Security model](../explanation/security-model.md#cancellation-and-host-code). +effects. See [Understanding CodeMode's security model](../explanation/security-model.md#cancellation-and-host-code). ### Server operations @@ -241,7 +245,7 @@ Execute(ctx context.Context, subject authz.Subject, program Program) (any, error `Program` is Starlark source. The context must be non-nil and the subject ID must be non-empty. A nil context is a caller-contract violation and is currently classified as `ErrInternal`. -Every call creates a fresh interpreter and fresh budgets. Module loading is disabled. The enabled capability names form the predeclared namespace. Source loading must define a function named `main` that accepts no positional parameters, keyword-only parameters, variadic positional parameters, or variadic keyword parameters. Native capability calls are accepted only while `main` is running. +Every call creates a fresh interpreter and fresh budgets. Module loading is disabled. The enabled capability names form the predeclared namespace. Top-level source loading must define a function named `main` that accepts no positional parameters, keyword-only parameters, variadic positional parameters, or variadic keyword parameters. Native calls are accepted only while `main` is running. For each native call, CodeMode performs these operations in order: @@ -252,7 +256,7 @@ For each native call, CodeMode performs these operations in order: After `main` returns, CodeMode converts only its final value. Supported final values are `None`, booleans, strings, signed 64-bit integers, finite floats, tuples, lists, and dictionaries with string keys, recursively within configured limits. `None` becomes JSON `null`, and tuples and lists become JSON arrays. -Printed text, globals, and intermediate values do not cross the execution boundary. +Only that converted return value is exposed to the caller. Printed text, globals, and interpreter-local intermediate values are not returned. During execution, each native-call argument map and each validated native result crosses the private worker protocol, and each crossing value is independently subject to `MaxValueDepth` and `MaxValueBytes`. ### Error classifications @@ -306,7 +310,7 @@ Authorize(context.Context, AuthorizationInput) error Return `nil` to allow dispatch. Return an error that wraps `ErrDenied` for a recognized denial. Any other error is a policy evaluation failure. CodeMode maps these outcomes to `ErrPermissionDenied` and `ErrPolicyFailure`, respectively, without dispatching the handler. -`AllowAllAuthorizer` permits every valid native call. `AllowAll()` constructs it explicitly. This policy is deliberate in the simple example so that the authorization choice is visible; production hosts normally supply policy that evaluates the trusted subject, stable capability ID, and canonical arguments. +`AllowAllAuthorizer` permits every native call whose arguments bind successfully. `AllowAll()` constructs it explicitly. Use `AllowAll()` only when every resolved subject may call every enabled capability; otherwise supply an `authz.Authorizer` that evaluates the trusted subject, stable capability ID, and canonical arguments. ## `authz/rego` @@ -373,7 +377,7 @@ A ground decision is either undefined or yields one value. That value must be Bo Only Boolean `false` is a recognized denial. CodeMode maps it to `ErrPermissionDenied`; it maps the ordinary policy errors to `ErrPolicyFailure`. Define a total Boolean decision with `default allow := false` so that unmatched input is an intentional denial rather than an undefined policy failure. -See [Use Rego for authorization](../how-to/use-rego-authorization.md) for server wiring and [Security model](../explanation/security-model.md#rego-policy-runs-in-process) for the in-process trust boundary. +See [Use Rego for authorization](../how-to/use-rego-authorization.md) for server wiring and [Understanding CodeMode's security model](../explanation/security-model.md#rego-policy-runs-in-process) for the in-process trust boundary. ## `mcpserver` @@ -407,4 +411,4 @@ New(service Service, resolver InvocationResolver) (*mcp.Server, error) `New` rejects nil and typed-nil dependencies with `ErrInvalidRegistration`. The returned official SDK server exposes exactly `search_api`, `describe_api`, and `execute`. It does not own authentication, transport creation, listeners, cancellation, or shutdown. The host connects the returned server to an official MCP transport and owns that lifecycle. -See [MCP tool reference](mcp-tools.md) for the wire contracts and [Security model](../explanation/security-model.md) for the trust boundary. +See [MCP tool reference](mcp-tools.md) for the wire contracts and [Understanding CodeMode's security model](../explanation/security-model.md) for the trust boundary. diff --git a/docs/docs/tutorials/first-server.md b/docs/docs/tutorials/first-server.md index cd5a503..bbfd8e2 100644 --- a/docs/docs/tutorials/first-server.md +++ b/docs/docs/tutorials/first-server.md @@ -7,6 +7,10 @@ description: Register a typed Go capability and call it through the official MCP This tutorial builds an MCP server with one typed capability. You will connect an official MCP client over the SDK's in-memory transport and call `search_api`, `describe_api`, and `execute`. +## Prerequisites + +The repository currently requires Go 1.26.6. + ## Create a module CodeMode has not published a release. To follow the tutorial against the current `master` branch, create a module and add CodeMode and the official MCP Go SDK: @@ -18,8 +22,6 @@ go mod init example.com/codemode-first-server go get github.com/meigma/codemode@master github.com/modelcontextprotocol/go-sdk/mcp ``` -The repository currently requires Go 1.26.6. - ## Add the server Create `main.go` with the following program: @@ -181,16 +183,21 @@ func main() { } ``` -`authz.AllowAll()` is deliberate in this small example: it makes the authorization decision explicit without adding application policy, and it permits every validated invocation. Do not use it in a production host unless that is the intended policy. Production hosts normally supply an `authz.Authorizer` that evaluates the trusted subject, stable capability ID, exposed name, and canonical arguments. +`authz.AllowAll()` is deliberate in this small example: it makes the authorization decision explicit without adding application policy, and it permits every native call whose arguments bind successfully. Use `authz.AllowAll()` only when every resolved subject may call every enabled capability; otherwise supply an `authz.Authorizer` that evaluates the trusted subject, stable capability ID, exposed name, and canonical arguments. The resolver reads only the typed, server-side Go context. In a production host, authentication middleware establishes that context before the MCP server handles a request. Do not accept a subject or credentials from Starlark source, tool arguments, or MCP `_meta`. ## Run the server -Clean up the module metadata, then run the program: +Clean up the module metadata: ```sh go mod tidy +``` + +Run the program: + +```sh go run . ``` @@ -200,10 +207,10 @@ The program prints structured results for all three tools. The final line contai {"result":{"count":2,"key":"alpha"}} ``` -`execute` first binds and canonicalizes the keyword arguments in the parent, then authorizes the native call, and then dispatches `lookupRecords` in the parent. Each `execute` call runs a fresh bounded interpreter in a re-executed worker process. Only the final converted value returned by a zero-argument `main()` crosses the worker boundary; printed text, globals, and intermediate values do not. +`execute` first binds and canonicalizes the keyword arguments in the parent, then authorizes the native call, and then dispatches `lookupRecords` in the parent. Each `execute` call runs a fresh bounded interpreter in a re-executed worker process. Only the final converted value returned by a zero-argument `main()` is exposed in the successful MCP result; printed text, globals, and interpreter-local intermediate values are not returned. During execution, each native-call argument map and each validated native result crosses the private worker protocol, and each crossing value is independently subject to `MaxValueDepth` and `MaxValueBytes`. `ServeWorkerAndExit` must be the first statement in `main`, before flag parsing, credential loading, client construction, or other application setup. It returns immediately in the ordinary host. In a re-executed worker, it serves one private probe or execution request and terminates the process. -The in-memory MCP transport keeps the client and server in one parent process; Starlark still runs in fresh worker processes. In an application, the host owns authentication, transport selection and startup, listeners, request cancellation, and shutdown. A Go authorizer or handler that can block must honor its context. CodeMode can kill the Starlark worker, but it cannot forcibly stop Go code after parent dispatch. +The in-memory MCP transport keeps the client and server in one parent process; Starlark still runs in fresh workers. In an application, the host owns authentication, transport selection and startup, listeners, request cancellation, and shutdown. A Go authorizer or handler that can block must honor its context. CodeMode can kill the Starlark worker, but it cannot forcibly stop Go code after parent dispatch. For shorter compile-checked forms of this setup, see the [typed registration example](https://github.com/meigma/codemode/blob/master/example_test.go) and [official transport example](https://github.com/meigma/codemode/blob/master/mcpserver/example_test.go). diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 2c662ec..c1eb476 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -44,7 +44,7 @@ nav: - Public API: reference/public-api.md - MCP tools: reference/mcp-tools.md - Explanation: - - Security model: explanation/security-model.md + - Understanding CodeMode's security model: explanation/security-model.md plugins: - search