Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
---
name: populate-constitution
description: Draft a complete agent constitution from an instructional prompt or agent brief. Use when Codex needs to create, populate, revise, or normalize CONSTITUTION.md-style governance documents that define an agent's purpose, scope, permissions, safety rules, handoff conditions, scenario-specific operating rules, and change log.
name: populate-charter
description: Draft a complete agent charter from an instructional prompt or agent brief. Use when Codex needs to create, populate, revise, or normalize CHARTER.md-style governance documents that define an agent's purpose, scope, permissions, safety rules, handoff conditions, scenario-specific operating rules, and change log.
---

# Populate Constitution
# Populate Charter

## Workflow

1. Read the user's instructional prompt and identify the agent's job, users, connected systems, recurring cadence, autonomous actions, approval boundaries, and expected outputs.
2. Load `assets/constitution-template.md` when the exact output structure is needed.
2. Load `assets/charter-template.md` when the exact output structure is needed.
3. If the prompt is underspecified, make conservative assumptions and mark them with bracketed placeholders such as `[Owner]`, `[Review cadence]`, or `[Confirm system]`.
4. Populate the foundational sections and add scenario-specific sections where the prompt needs more operational detail. Do not leave generic filler; write concrete rules derived from the prompt.
5. Preserve the constitution style: concise, operational, safety-forward, and written as binding instructions for the future agent and the tool-use governance system mediating it.
5. Preserve the charter style: concise, operational, safety-forward, and written as binding instructions for the future agent and the tool-use governance system mediating it.
6. Add a one-row change log with today's date unless the user provides another date.

## Drafting Rules

- Use a specific title: `# CONSTITUTION.md - {Agent Name}`.
- Use a specific title: `# CHARTER.md - {Agent Name}`.
- Keep `Purpose` to one or two paragraphs plus a concrete `Success looks like` sentence.
- Define `Scope` with three subsections: `In scope`, `Out of scope`, and `Handoff conditions`.
- Define permission tiers as `Auto`, `Notify`, `Approve`, and `Never`, even if some tiers are sparse.
Expand All @@ -29,7 +29,7 @@ description: Draft a complete agent constitution from an instructional prompt or

## Interpreting Prompts

Map the prompt into constitution sections:
Map the prompt into charter sections:

- Agent mission, cadence, and audience -> `Purpose`
- Things the agent may inspect, summarize, change, create, or send -> `In scope`
Expand All @@ -45,7 +45,7 @@ Map the prompt into constitution sections:

Before finalizing, verify that:

- The constitution can stand alone without the original prompt.
- The charter can stand alone without the original prompt.
- Every autonomous action has a matching permission tier and a failure default.
- The `Never` tier contains hard blocks, not preferences.
- Sensitive data handling is explicit.
Expand Down
4 changes: 4 additions & 0 deletions .agents/skills/populate-charter/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Populate Charter"
short_description: "Draft agent charters from prompts"
default_prompt: "Use $populate-charter to draft a charter from this agent brief."
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# CONSTITUTION.md - {Agent Name}
# CHARTER.md - {Agent Name}

**Version:** 1.0
**Owner:** {Owner or team}
Expand Down Expand Up @@ -43,8 +43,8 @@
<!-- The LLM-as-judge will pass evaluation to the next matching approval rule
when any of these conditions are met, rather than making a final decision.
Use sparingly — prefer explicit approve/deny/human-approval tiers. -->
- {Condition where this constitution has no opinion and should defer}
- {Condition where this constitution has no opinion and should defer}
- {Condition where this charter has no opinion and should defer}
- {Condition where this charter has no opinion and should defer}

---

Expand Down
4 changes: 0 additions & 4 deletions .agents/skills/populate-constitution/agents/openai.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ Auth is OIDC-based and supports multiple authorization servers concurrently (Key

For local no-auth MCP runs, when no `[[auth]]` blocks are configured, callers may provide a best-effort agent identity with `?agent_id=` on `/mcp/{server}` and `/api/v1/agent/rules`. For example: `http://localhost:8080/mcp/shortcut?agent_id=hunners-codex`. This ID is ignored as soon as inbound auth is configured.

The Settings UI can also select a default ValidMind agent record. AI Evaluation uses that record when an incoming runtime agent ID is missing or does not map to a synced agent, allowing local no-auth runs to evaluate against a known constitution without adding TOML.
The Settings UI can also select a default ValidMind agent record. AI Evaluation uses that record when an incoming runtime agent ID is missing or does not map to a synced agent, allowing local no-auth runs to evaluate against a known charter without adding TOML.

## HTTP surface

Expand Down
2 changes: 1 addition & 1 deletion atryum.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ api_key = ""
# name = "staging"
# api_key = ""

# Agent sync and AI evaluation settings (org, record type, constitution field)
# Agent sync and AI evaluation settings (org, record type, charter field)
# are managed through the Atryum Settings UI and stored in the database.
# They no longer need to be configured here.

Expand Down
12 changes: 6 additions & 6 deletions cmd/atryum/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -468,15 +468,15 @@ func (a *agentsLookupAdapter) GetByAgentID(ctx context.Context, agentID string)
if err != nil {
return invocation.AgentRecord{}, err
}
return invocation.AgentRecord{ID: rec.ID, VMCUID: rec.VMCUID, VMOrganizationCUID: rec.VMOrganizationCUID, Constitution: rec.Constitution}, nil
return invocation.AgentRecord{ID: rec.ID, VMCUID: rec.VMCUID, VMOrganizationCUID: rec.VMOrganizationCUID, Charter: rec.Charter}, nil
}

func (a *agentsLookupAdapter) GetByVMCUID(ctx context.Context, vmCUID string) (invocation.AgentRecord, error) {
rec, err := a.repo.GetByVMCUID(ctx, vmCUID)
if err != nil {
return invocation.AgentRecord{}, err
}
return invocation.AgentRecord{ID: rec.ID, VMCUID: rec.VMCUID, VMOrganizationCUID: rec.VMOrganizationCUID, Constitution: rec.Constitution}, nil
return invocation.AgentRecord{ID: rec.ID, VMCUID: rec.VMCUID, VMOrganizationCUID: rec.VMOrganizationCUID, Charter: rec.Charter}, nil
}

// llmConfigsLookupAdapter bridges store.LLMConfigsRepo → invocation.LLMConfigProvider.
Expand All @@ -499,15 +499,15 @@ func (a *llmConfigsLookupAdapter) GetLLMConfig(ctx context.Context, id string) (
}

// syncSettingsAdapter bridges store.AgentSyncSettingsRepo → invocation.SyncSettingsProvider.
// ConstitutionFieldKey is read from the DB on every call so that changes saved
// CharterFieldKey is read from the DB on every call so that changes saved
// via the Settings UI take effect immediately without a restart.
type syncSettingsAdapter struct {
repo *store.AgentSyncSettingsRepo
}

func (a *syncSettingsAdapter) ConstitutionFieldKey(ctx context.Context) string {
func (a *syncSettingsAdapter) CharterFieldKey(ctx context.Context) string {
s, _ := a.repo.Get(ctx)
return s.ConstitutionFieldKey
return s.CharterFieldKey
}

func (a *syncSettingsAdapter) DefaultAgentVMCUID(ctx context.Context) string {
Expand All @@ -530,7 +530,7 @@ func (e *evaluatorAdapter) EvaluateToolCall(ctx context.Context, req invocation.
ModelConfigCUID: req.ModelConfigCUID,
OrgCUID: req.OrgCUID,
AgentVMCUID: req.AgentVMCUID,
ConstitutionFieldKey: req.ConstitutionFieldKey,
CharterFieldKey: req.CharterFieldKey,
ServerName: req.ServerName,
ToolName: req.ToolName,
ToolArgs: req.ToolArgs,
Expand Down
34 changes: 17 additions & 17 deletions docs/testing/ai-evaluation.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This guide walks through manually testing the AI Evaluation rule type end-to-end

## Step 1 — Configure `atryum.toml`

`atryum.toml` only needs two sections for AI Evaluation: `[backend]` (ValidMind credentials) and `[[auth]]` (JWT validation). Agent sync and constitution settings have moved to the UI — see **Step 1b** below.
`atryum.toml` only needs two sections for AI Evaluation: `[backend]` (ValidMind credentials) and `[[auth]]` (JWT validation). Agent sync and charter settings have moved to the UI — see **Step 1b** below.

```toml
# ValidMind backend connection (machine-user credentials).
Expand All @@ -42,7 +42,7 @@ agent_id_claim = "sub" # claim that carries the agent's unique identity

## Step 1b — Configure Agent Sync in the UI

The organization, record type, and constitution field are now configured in the Atryum UI under **Settings → Agent Record Sync**. This replaces the old `[agent_sync]` and `[ai_evaluation]` TOML sections.
The organization, record type, and charter field are now configured in the Atryum UI under **Settings → Agent Record Sync**. This replaces the old `[agent_sync]` and `[ai_evaluation]` TOML sections.

1. Open the Atryum UI and navigate to **Settings**.
2. Under **Agent Record Sync**, fill in the three fields:
Expand All @@ -51,7 +51,7 @@ The organization, record type, and constitution field are now configured in the
|---|---|
| **Organization** | Select the ValidMind organization whose agent records should be synced |
| **Record Type** | Select the primary record type that identifies agent inventory models (e.g. `ai-agents`) |
| **Constitution Field** | Select the custom field key on inventory models that stores the constitution text (e.g. `constitution`) |
| **Charter Field** | Select the custom field key on inventory models that stores the charter text (e.g. `charter`) |

The dropdowns are cascading — selecting an organization loads its record types; selecting a record type loads the custom fields available on that type.

Expand Down Expand Up @@ -89,13 +89,13 @@ In your identity provider (e.g. Auth0):
3. Click the agent to open it, then paste the **user ID** (`sub` value) from Step 3 into the **Agent IDs** field.
4. Save.

Atryum will now recognise bearer tokens issued for that user as belonging to this agent, enabling constitution lookup and org cross-validation when an AI Evaluation rule fires.
Atryum will now recognise bearer tokens issued for that user as belonging to this agent, enabling charter lookup and org cross-validation when an AI Evaluation rule fires.

---

## Step 5 — Author the Agent Constitution
## Step 5 — Author the Agent Charter

The constitution is stored as a custom field (key = `constitution_field_key`) on the agent's Inventory Model in ValidMind. It is free-form Markdown, but the LLM-as-judge recognises the following sections to decide its verdict:
The charter is stored as a custom field (key = `charter_field_key`) on the agent's Inventory Model in ValidMind. It is free-form Markdown, but the LLM-as-judge recognises the following sections to decide its verdict:

### Permission tiers (tool-level)

Expand All @@ -112,7 +112,7 @@ List conditions under which the LLM should route the invocation to the human app
```markdown
### Human approval required when
- The SQL query modifies data (INSERT, UPDATE, DELETE, DROP, …)
- The request targets a table not explicitly listed in the constitution
- The request targets a table not explicitly listed in the charter
- The query returns more than 10,000 rows
```

Expand All @@ -125,20 +125,20 @@ List conditions under which the LLM should pass evaluation to the next matching
- The tool is not a database tool (let downstream rules handle it)
```

### Constitution chain and precedence
### Charter chain and precedence

If the agent's Inventory Model has upstream dependencies, Atryum collects constitutions from the entire chain (most-upstream ancestor first, this agent last) and presents them all to the LLM. The judge applies these precedence rules:
If the agent's Inventory Model has upstream dependencies, Atryum collects charters from the entire chain (most-upstream ancestor first, this agent last) and presents them all to the LLM. The judge applies these precedence rules:

1. **Downstream wins**: later constitutions in the chain are more specific and override earlier ones when they conflict.
2. **Explicit delegation is honoured**: if an upstream says "downstream constitutions may override this", a downstream grant (even conditional) replaces the upstream rule.
3. **Most specific match first**: apply the innermost rule that covers the action; fall back to upstream only when no downstream constitution addresses it.
1. **Downstream wins**: later charters in the chain are more specific and override earlier ones when they conflict.
2. **Explicit delegation is honoured**: if an upstream says "downstream charters may override this", a downstream grant (even conditional) replaces the upstream rule.
3. **Most specific match first**: apply the innermost rule that covers the action; fall back to upstream only when no downstream charter addresses it.
4. **Grants override blanket denials**: a downstream "allow with human approval" beats an upstream "deny all".

**Example:**

```
[Upstream agent]
Deny all Postgres queries. Downstream constitutions may override this.
Deny all Postgres queries. Downstream charters may override this.

---

Expand Down Expand Up @@ -178,8 +178,8 @@ Send a tool call through Atryum using a bearer token issued for the agent identi

1. Atryum receives the invocation and matches it to the AI Evaluation rule.
2. It resolves the agent's Inventory Model CUID and org CUID from the stored record.
3. It calls `POST /api/atryum/unstable/evaluate` on the ValidMind backend with the agent details, constitution field key, and tool call context.
4. The backend fetches the constitution chain from the Inventory Model's custom fields, builds a prompt with precedence rules, and asks the configured LLM for a verdict.
3. It calls `POST /api/atryum/unstable/evaluate` on the ValidMind backend with the agent details, charter field key, and tool call context.
4. The backend fetches the charter chain from the Inventory Model's custom fields, builds a prompt with precedence rules, and asks the configured LLM for a verdict.
5. Atryum receives `{ verdict, reason }` and routes the invocation accordingly:

| Verdict | Disposition | UI outcome |
Expand All @@ -199,7 +199,7 @@ In the Atryum UI, open **Invocations** to see the outcome and the disposition re
|---|---|
| No agents in startup log | `org_cuid` or `agent_record_type_slug` is wrong, or backend credentials are incorrect |
| Rule never matches | Agent IDs field is empty, the user's `sub` doesn't match `agent_id_claim`, or the bearer token is not being sent by the agent |
| Evaluation falls back to `human_approval` | `constitution_field_key` doesn't match the custom field name in VM, or the LLM call failed (check logs) |
| Evaluation falls back to `human_approval` | `charter_field_key` doesn't match the custom field name in VM, or the LLM call failed (check logs) |
| 403 from `/evaluate` | `org_cuid` in the request doesn't match the agent's organization in ValidMind |
| Upstream deny overrides downstream grant | Backend not yet redeployed with the precedence-rules prompt update; check that `verdict` (not `approved`) is present in the `/evaluate` response |
| `next_rule` never advances | Rules are not stacked in priority order, or the constitution does not contain a `### Defer to next rule when` section |
| `next_rule` never advances | Rules are not stacked in priority order, or the charter does not contain a `### Defer to next rule when` section |
1 change: 0 additions & 1 deletion integrations/lib/atryum.sh
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ seed_auto_approve_rules() {
"action": "auto_approve",
"server_patterns": ["*"],
"tool_patterns": ["*"],
"agent_id_pattern": "*",
"description": "integration test catch-all auto-approve"
}'
curl -fsS -X POST "$ATRYUM_URL/api/v1/admin/rules" \
Expand Down
10 changes: 3 additions & 7 deletions internal/api/auth_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,7 @@ func TestMCPAcceptsValidTokenAndPlumbsAgentID(t *testing.T) {
func TestAgentRulesRequiresAuthAndUsesTokenAgentID(t *testing.T) {
rig := newAuthTestRig(t)
rules := &stubRulesRepo{rules: []store.Rule{
{ID: "own-rule", Action: invocation.RuleActionAutoApprove, ServerPatterns: []string{"amp"}, ToolPatterns: []string{"Read"}, AgentIDPattern: "agent-007", Enabled: true, Order: 0},
{ID: "other-rule", Action: invocation.RuleActionAutoDeny, ServerPatterns: []string{"amp"}, ToolPatterns: []string{"Read"}, AgentIDPattern: "other", Enabled: true, Order: 1},
{ID: "auto-rule", Action: invocation.RuleActionAutoApprove, ServerPatterns: []string{"amp"}, ToolPatterns: []string{"Read"}, Enabled: true, Order: 0},
}}
h := NewHandler(&stubService{}, stubServerService{}, nil, rules, nil, nil, nil, nil, nil, nil)
h.SetAuthValidator(rig.v)
Expand All @@ -209,13 +208,10 @@ func TestAgentRulesRequiresAuthAndUsesTokenAgentID(t *testing.T) {
t.Fatal(err)
}
if resp.AgentID != "agent-007" {
t.Fatalf("expected token agent_id to win, got %q", resp.AgentID)
t.Fatalf("expected token agent_id to win over query param, got %q", resp.AgentID)
}
if resp.Action != invocation.RuleActionAutoApprove {
t.Fatalf("expected own rule action, got %q", resp.Action)
}
if len(resp.Items) != 1 || resp.Items[0].ID != "own-rule" {
t.Fatalf("expected only own rule, got %#v", resp.Items)
t.Fatalf("expected auto_approve action, got %q", resp.Action)
}
}

Expand Down
Loading
Loading