Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: ${{ matrix.python-version }}
- name: Run 97-test suite
- name: Run test suite
run: PYTHONPATH=src python -m unittest discover -s tests -v
- name: Compile source
run: python -m compileall -q src tests scripts
Expand Down
20 changes: 18 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
# Changelog

All notable public changes to Model Council are recorded here.
All notable public changes to CouncilLogic are recorded here.

## Unreleased

### Added

- xAI's Grok 4.5 as a fifth live CLI provider, using the Responses API,
structured jury output, official-host restriction, external-only
`XAI_API_KEY` resolution, and provider-reported cost metadata.

### Changed

- The default live council now uses five proposals, five juries, and one
synthesis. Existing file-backed four-provider configurations remain
four-provider until they add an explicit `[providers.xai]` section.
- The mock-only `0.2.0a1` service remains frozen at four deterministic
lineages and nine logical calls.

## [0.2.0a1] - 2026-07-26

Expand Down Expand Up @@ -32,4 +48,4 @@ Initial public alpha.
- External pull requests and copyrightable contributions remain closed until
a counsel-reviewed contributor license agreement is available.

[0.2.0a1]: https://github.com/apcar/model-council/releases/tag/v0.2.0a1
[0.2.0a1]: https://github.com/apcar/CouncilLogic/releases/tag/v0.2.0a1
28 changes: 16 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@
</p>

CouncilLogic is a local-first, auditable council of heterogeneous language
models. OpenAI, Anthropic, Gemini, and Mistral independently propose answers,
judge blinded candidates, contribute to a deterministic Borda aggregate, and
produce a final synthesis. The complete run record is stored locally.
models. OpenAI, Anthropic, Gemini, Mistral, and xAI's Grok independently
propose answers, judge blinded candidates, contribute to a deterministic Borda
aggregate, and produce a final synthesis. The complete run record is stored
locally.

This repository is a **public alpha (`0.2.0a1`)**, not a production service or
a truth oracle. Multiple models can share the same error. Verify consequential
Expand Down Expand Up @@ -45,24 +46,26 @@ record.
That is a single-operator origin, not a validation result. Published research
makes heterogeneous model juries and aggregation worth testing, but also
documents correlated errors and inconsistent gains. The current test suite
establishes software behavior; it does not show that nine calls outperform a
establishes software behavior; it does not show that eleven calls outperform a
strong single model. Establishing that would require matched-cost baselines,
protocol ablations, and evaluation across additional users and task domains.
See [Research](docs/RESEARCH.md).

## Four-stage council

1. **Propose:** OpenAI, Anthropic, Gemini, and Mistral answer independently.
1. **Propose:** OpenAI, Anthropic, Gemini, Mistral, and xAI's Grok answer
independently.
2. **Judge:** each lineage ranks relabeled candidates without provider
attribution; structured responses are validated locally.
3. **Aggregate:** deterministic Borda scoring combines valid juries.
4. **Synthesize:** the selected lineage receives the candidates, aggregate,
and jury records and writes the final answer.

A normal run uses nine application-level calls: four proposals, four juries,
and one synthesis. Successful work is persisted and reused on resume. The
application does not give models tools, web access, code execution, or
model-initiated actions.
A default live run uses eleven application-level calls: five proposals, five
juries, and one synthesis. Mock mode and the frozen `0.2.0a1` service reference
topology remain four-lineage, nine-call fixtures. Successful work is persisted
and reused on resume. The application does not give models tools, web access,
code execution, or model-initiated actions.

## Five-minute credential-free quickstart

Expand Down Expand Up @@ -93,15 +96,16 @@ council --mock --data-dir ./work/demo export RUN_ID \

## Live CLI setup

The live CLI sends the question and candidate text to all four configured
providers. Review provider contracts, data handling, model availability, and
cost controls before using sensitive material.
The default live CLI sends the question and candidate text to all five
configured providers. Review provider contracts, data handling, model
availability, and cost controls before using sensitive material.

```bash
export OPENAI_API_KEY="..."
export ANTHROPIC_API_KEY="..."
export GEMINI_API_KEY="..."
export MISTRAL_API_KEY="..."
export XAI_API_KEY="..."

council doctor
council providers
Expand Down
7 changes: 7 additions & 0 deletions council.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,10 @@ secret_name = "MISTRAL_API_KEY"
max_output_tokens = 1800
timeout_seconds = 90
max_attempts = 3

[providers.xai]
model = "grok-4.5"
secret_name = "XAI_API_KEY"
max_output_tokens = 1800
timeout_seconds = 90
max_attempts = 3
21 changes: 11 additions & 10 deletions docs/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ fencing.

## Service-alpha boundary

The service runs four deterministic mock lineages corresponding to the
four-provider topology: OpenAI, Anthropic, Gemini, and Mistral in the separate
The service retains the four deterministic mock lineages from its frozen
`0.2.0a1` reference topology. It does not mirror the separate five-provider
live CLI. It has six fixed principals; both work identities remain disabled.
Enabled principals have durable mandates for `run:create`, `run:read`, and
`provider:invoke`. Run reads and idempotency are owner-scoped.
Expand Down Expand Up @@ -152,7 +152,7 @@ council --mock --data-dir ./work/release-gate list
Before live use:

1. Review `council.example.toml` and select the intended data directory.
2. Configure all four credentials through the environment or a tested external
2. Configure all five credentials through the environment or a tested external
secret command.
3. Run `council --config ./council.toml doctor`.
4. Run `council --config ./council.toml providers` and verify every provider,
Expand Down Expand Up @@ -213,9 +213,10 @@ Treat each idempotency key as permanent within a database. Reusing it with a
different locked request fails. Use a key that contains no secret or personal
data.

The default four-provider path has nine application-level provider calls:
four proposals, four juries, and one synthesis. Proposal and jury stages run in
parallel. Successful stage/provider slots are reused on resume.
The default five-provider live path has eleven application-level provider
calls: five proposals, five juries, and one synthesis. Proposal and jury stages
run in parallel. Successful stage/provider slots are reused on resume. The
mock-only service remains a four-lineage, nine-call fixture.

## Policy controls

Expand Down Expand Up @@ -335,9 +336,9 @@ provider calls.

Run `doctor`. Confirm that the exact logical names are available:
`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, and
`MISTRAL_API_KEY`, unless the TOML uses different `secret_name` values. For an
external resolver, invoke it manually with a logical name and check only the
exit status; do not paste or record its output.
`MISTRAL_API_KEY`, and `XAI_API_KEY`, unless the TOML uses different
`secret_name` values. For an external resolver, invoke it manually with a
logical name and check only the exit status; do not paste or record its output.

### Authentication or permission failure

Expand Down Expand Up @@ -458,7 +459,7 @@ intentionally written to the database.
2. Update the process environment or external secret resolver under the same
logical secret name.
3. Start a new shell/process and run `doctor`.
4. Perform a low-risk live smoke run and confirm all four providers in the
4. Perform a low-risk live smoke run and confirm all five providers in the
audit record and provider consoles. This incurs live usage.
5. Revoke the old key.
6. Clear old shell environment values, terminal scrollback, and any temporary
Expand Down
20 changes: 11 additions & 9 deletions docs/PORTABLE-SERVICE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,16 @@ is authorized, no live provider key is accepted by this surface, and no work
principal can be enabled. The single-user live CLI remains a separate surface.

The service must not be switched from mock providers to OpenAI, Anthropic,
Google, or Mistral until the later release gates in this document pass.
Google, Mistral, or xAI until the later release gates in this document pass.

### Implementation snapshot

The service alpha now implements:

- the four-lineage topology—OpenAI, Anthropic, Gemini, and Mistral in the
separate live CLI—represented by four deterministic mock slots behind a
loopback-only asynchronous HTTP server with bounded workers and queue;
- the frozen four-lineage `0.2.0a1` reference topology, represented by four
deterministic mock slots behind a loopback-only asynchronous HTTP server
with bounded workers and queue; this fixture does not mirror the separate
five-provider live CLI;
- the fixed six-principal catalog, with both work principals forced disabled;
- per-principal bearer authentication with atomic verifier rotation and narrow
operator revocation;
Expand Down Expand Up @@ -71,7 +72,8 @@ source.
The governance wedge—the product differentiation—is the system around the
inference pattern:

- four genuinely different provider lineages, including Mistral;
- five genuinely different live-provider lineages, including Mistral and
xAI's Grok;
- deterministic aggregation and a resumable, inspectable run record;
- separately authenticated human and agent callers;
- personal/work tenant separation;
Expand Down Expand Up @@ -129,8 +131,8 @@ durable Council run + deterministic mock providers

Only the service process can construct provider adapters. In this alpha those
adapters must be the four local deterministic mocks. A client holds one
revocable Council credential, not an OpenAI, Anthropic, Gemini, Mistral, or
Bitwarden credential.
revocable Council credential, not an OpenAI, Anthropic, Gemini, Mistral, xAI,
or Bitwarden credential.

A later personal-access gate may put an authenticated HTTPS front door in
front of the loopback origin. Loopback binding is not permission to expose the
Expand Down Expand Up @@ -328,7 +330,7 @@ database, queue, or observability plan.

That does not mean the eventual service is free. Later gates may incur:

- usage-based charges from the four model providers;
- usage-based charges from the five live model providers;
- a secret-manager tier or additional machine identity;
- an authenticated tunnel or identity plan;
- backup storage and monitoring; and
Expand Down Expand Up @@ -386,7 +388,7 @@ Passing Gate 2 still does not authorize live model keys.
- Prove that no client, log, request, database, package, or Git object contains
an upstream credential.
- Run a non-sensitive, tightly capped canary across OpenAI, Anthropic, Google,
and Mistral and reconcile Council records to provider billing.
Mistral, and xAI's Grok and reconcile Council records to provider billing.
- Rehearse revocation and rollback before removing any legacy secret.

### Gate 4 — personal agents
Expand Down
16 changes: 10 additions & 6 deletions docs/RESEARCH.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,20 +65,24 @@ Related protocol families remain useful comparison targets:
mixture-of-agents protocol needs a distinct version and matched-cost
evaluation before becoming a default.

## Interpreting the fourth lineage
## Interpreting additional lineages

The public alpha adds Mistral to the OpenAI, Anthropic, and Gemini baseline. This
broadens vendor and model-family heterogeneity and permits a three-of-four
default quorum to tolerate one unavailable lineage. It does not establish
statistical independence, eliminate shared training-data effects, or prove that
four providers outperform a three-provider configuration.

Treat the fourth lineage as an engineering hypothesis. Compare three- and
four-lineage configurations on a fixed, blinded evaluation set; report
The current live configuration adds xAI's Grok as a fifth lineage. That further
broadens provider and model-family coverage and gives a three-of-five quorum
tolerance for two unavailable lineages. It does not prove that five providers
outperform four.

Treat each added lineage as an engineering hypothesis. Compare three-, four-,
and five-lineage configurations on a fixed, blinded evaluation set; report
accuracy, valid-jury rate, abstentions, ties, latency, and total provider cost;
and keep the evidence separate. Mistral API compatibility and a passing live
canary are useful
release checks, not research evidence that the council is correct.
and keep the evidence separate. API compatibility and passing live canaries
are useful release checks, not research evidence that the council is correct.

## Useful research and evaluation kits

Expand Down
15 changes: 8 additions & 7 deletions docs/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,19 @@ controls, but it is not a production identity, billing, or Internet-facing
system. Live service, Cloudflare or another remote front door, work use, and
commercial use remain gated.

The four-provider CLI is a separate surface. Local tests and prior live checks
The five-provider CLI is a separate surface. Local tests and prior live checks
do not remove the alpha boundaries below, establish current provider health,
or authorize service deployment.

Before using sensitive material, decide whether sending that material to all
four providers is permitted under the applicable contracts, account settings,
five providers is permitted under the applicable contracts, account settings,
data residency requirements, and law.

## Assets and trust boundaries

The principal assets are:

- OpenAI, Anthropic, Gemini, and Mistral API credentials.
- OpenAI, Anthropic, Gemini, Mistral, and xAI API credentials.
- Questions and contextual information supplied by the operator.
- Provider prompts and raw responses.
- Jury records, aggregate rankings, and synthesized answers.
Expand All @@ -48,8 +48,8 @@ The trust boundaries are:
### Mock-only loopback service

The `0.2.0a1` service accepts only loopback bind addresses and constructs four
deterministic mock lineages corresponding to the Council's four-provider
topology (OpenAI, Anthropic, Gemini, and Mistral in the separate live CLI).
deterministic mock lineages. This frozen reference fixture does not mirror the
five-provider live CLI.
Startup rejects a store containing any live-mode run. This is a fail-closed
separation from the live CLI, not permission to add a tunnel.

Expand Down Expand Up @@ -166,6 +166,7 @@ api.openai.com
api.anthropic.com
generativelanguage.googleapis.com
api.mistral.ai
api.x.ai
```

Embedded URL credentials and non-allowlisted hosts are rejected. This reduces
Expand Down Expand Up @@ -195,7 +196,7 @@ are treated as invalid judgments rather than trusted records.

### Diversity and blinded evaluation

The default configuration uses four separately named lineages. Candidate
The default live configuration uses five separately named lineages. Candidate
provider names are replaced by shuffled labels for jury prompts, and aggregate
input is anonymized for synthesis.

Expand Down Expand Up @@ -239,7 +240,7 @@ request metadata, and configured secret *names*. It is plaintext.
Do not submit:

- API keys, passwords, recovery codes, private keys, or session tokens.
- Material whose disclosure to all four providers is prohibited.
- Material whose disclosure to all five providers is prohibited.
- Personal or regulated data that has not passed the relevant review.
- Unredacted production incidents when a minimum reproduction will do.

Expand Down
13 changes: 12 additions & 1 deletion src/model_council/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,37 +19,43 @@
"v1beta/models/{model}:generateContent"
),
"mistral": "https://api.mistral.ai/v1/chat/completions",
"xai": "https://api.x.ai/v1/responses",
}

DEFAULT_MODELS = {
"openai": "gpt-5.6-sol",
"anthropic": "claude-sonnet-4-6",
"gemini": "gemini-3.6-flash",
"mistral": "mistral-medium-3-5",
"xai": "grok-4.5",
}

DEFAULT_SECRETS = {
"openai": "OPENAI_API_KEY",
"anthropic": "ANTHROPIC_API_KEY",
"gemini": "GEMINI_API_KEY",
"mistral": "MISTRAL_API_KEY",
"xai": "XAI_API_KEY",
}

DEFAULT_LINEAGES = {
"openai": "openai-gpt",
"anthropic": "anthropic-claude",
"gemini": "google-gemini",
"mistral": "mistral",
"xai": "xai-grok",
}

ALLOWED_ENDPOINT_HOSTS = {
"openai": {"api.openai.com"},
"anthropic": {"api.anthropic.com"},
"gemini": {"generativelanguage.googleapis.com"},
"mistral": {"api.mistral.ai"},
"xai": {"api.x.ai"},
"mock": {"localhost", "127.0.0.1"},
}

_PROVIDERS_ADDED_AFTER_V0_2 = frozenset({"xai"})
_SECRET_NAME_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")


Expand Down Expand Up @@ -83,7 +89,7 @@ def default_config() -> AppConfig:
return AppConfig(
providers=tuple(
_default_provider(name)
for name in ("openai", "anthropic", "gemini", "mistral")
for name in ("openai", "anthropic", "gemini", "mistral", "xai")
),
policy=RunPolicy(),
synthesis_provider="openai",
Expand Down Expand Up @@ -172,6 +178,11 @@ def load_config(path: str | Path | None = None) -> AppConfig:
providers: list[ProviderConfig] = []
defaults = {provider.name: provider for provider in base.providers}
for name, default in defaults.items():
# A file-backed configuration written before a provider was added must
# not silently acquire another credential requirement or billable
# participant. New configurations opt in by including its section.
if name in _PROVIDERS_ADDED_AFTER_V0_2 and name not in provider_raw:
continue
override: dict[str, Any] = dict(provider_raw.get(name) or {})
provider = ProviderConfig(
name=name,
Expand Down
Loading
Loading