Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,26 @@ body:
id: version
attributes:
label: MARGINAL version
placeholder: "0.1.0"
placeholder: "0.2.0"
validations:
required: true
- type: input
id: python
attributes:
label: Python version and platform
placeholder: "Python 3.13 on Windows 11"
validations:
required: true
- type: input
id: mode
attributes:
label: Execution mode and policy/estimator identity
placeholder: "shadow; profile:quality-first@2.0.0; historical-mean@2.0.0"
- type: textarea
id: behavior
attributes:
label: What happened?
description: Include the decision reason and smallest reproducible example.
description: Include reason codes and the smallest reproducible example.
validations:
required: true
- type: textarea
Expand All @@ -35,5 +41,5 @@ body:
- type: textarea
id: trace
attributes:
label: Redacted trace or logs
label: Redacted trace or Decision Ledger records
description: Remove prompts, credentials, personal data, and proprietary content.
11 changes: 11 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,17 @@
- [ ] `mypy src/marginal`
- [ ] `pytest -q`
- [ ] Documentation updated
- [ ] Public API, schemas, examples, roadmap, and changelog are consistent
- [ ] Performance claims are reproducible and honestly labeled
- [ ] New persisted fields have an explicit privacy classification
- [ ] Strict telemetry contains no free text or unreviewed metadata
- [ ] Privacy fixtures use synthetic identifiers and no real user data

## Security and compatibility

## Privacy review

- Privacy profile(s) affected:
- New or changed persisted fields:
- Quasi-identifiers considered:
- Key-management or export implications:
39 changes: 10 additions & 29 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,41 +10,22 @@ permissions:

jobs:
test:
name: test (${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"]
steps:
- name: Check out source
uses: actions/checkout@v6

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
- uses: actions/checkout@v6
- uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
cache: pip

- name: Install development dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"

- name: Check formatting
run: ruff format --check src tests examples

- name: Lint
run: ruff check src tests examples

- name: Type check
run: mypy src/marginal

- name: Test
run: pytest -q

- name: Build distributions
run: python -m build

- name: Validate distributions
run: python -m twine check dist/*
- run: python -m pip install --upgrade pip
- run: python -m pip install -e ".[dev]"
- run: ruff format --check .
- run: ruff check .
- run: mypy src/marginal
- run: pytest -q
- run: python -m build
- run: python -m twine check dist/*
35 changes: 0 additions & 35 deletions .github/workflows/pages.yml

This file was deleted.

44 changes: 12 additions & 32 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
name: Release

on:
workflow_dispatch:
push:
tags:
- "v*"
branches: [main]
workflow_dispatch:

permissions:
contents: write
Expand All @@ -17,38 +16,19 @@ jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v6
- uses: actions/checkout@v6
- uses: actions/setup-python@v6
with:
python-version: "3.13"
cache: pip

- name: Install development dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"

- name: Check formatting
run: ruff format --check src tests examples

- name: Lint
run: ruff check src tests examples

- name: Type check
run: mypy src/marginal

- name: Test
run: pytest -q

- name: Build distributions
run: python -m build

- name: Validate distributions
run: python -m twine check dist/*

- run: python -m pip install --upgrade pip
- run: python -m pip install -e ".[dev]"
- run: ruff format --check src tests examples
- run: ruff check src tests examples
- run: mypy src/marginal
- run: pytest -q
- run: python -m build
- run: python -m twine check dist/*
- name: Create release when the version is new
env:
GH_TOKEN: ${{ github.token }}
Expand Down
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,7 @@ build/
.env
*.jsonl
.DS_Store
!demos/killer-demo/trace.jsonl

# MARGINAL local privacy keys
*.privacy.key
.marginal/
11 changes: 6 additions & 5 deletions ACKNOWLEDGMENTS.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
# Acknowledgments

MARGINAL was created independently by SignalLayer Labs as an independent Apache-2.0 open-source reference
implementation for agent compute capital allocation.
MARGINAL was created by SignalLayer Labs as an independent Apache-2.0 open-source
implementation for economically disciplined AI-agent compute allocation.

The project’s economic framing is inspired by Siqi Zhu’s 2026 position paper,
“Agentic AI Systems Should Be Designed as Marginal Token Allocators”:
<https://arxiv.org/abs/2605.01214>.

The paper and this software are separate works. The paper proposes a research agenda;
MARGINAL provides an independently developed runtime API, accounting model, tests,
documentation, and integration layer.
MARGINAL provides an independently developed runtime API, transactional accounting model,
Decision Ledger, tests, documentation, and universal adapter foundation.

We also acknowledge the broader open-source and research community working on agent
budgets, cost observability, model routing, test-time scaling, and reliable orchestration.
budgets, cost observability, model routing, test-time scaling, causal evaluation, and
reliable orchestration.
87 changes: 63 additions & 24 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,75 @@
# Changelog

All notable changes to MARGINAL are documented here. The project follows Semantic
Versioning.
All notable changes to MARGINAL are documented here. The project follows Semantic Versioning.

## [0.2.0] - 2026-08-06

### Added

- `shadow`, `recommend`, and `enforce` execution modes;
- applied-versus-recommended decision fields and stable reason codes;
- additive `TokenUsage` breakdown for uncached input, cached input, output, reasoning, and total tokens;
- common token-breakdown extraction for provider-like usage objects;
- versioned `EstimatorIdentity`, `ValueEstimate`, contextual historical observations, uncertainty, confidence, sample size, provenance, and deterministic training-data fingerprints;
- `EstimatorRegistry` for explicit estimator name/version resolution;
- versioned `PolicyIdentity` and four transparent reference policy profiles;
- non-blocking unchecked reservations and separate internal reservation identities for accurate concurrent Shadow Mode observation;
- measured overrun accounting in non-blocking modes;
- explicit failed-action settlement through sync and async wrappers, while keeping failed work retryable;
- provider-neutral `Outcome` contract separated from action-level realized gain;
- schema-versioned `JsonlDecisionLedger` with strict envelope validation, run/task/trajectory/engine/model correlation, task/outcome consistency checks, and monotonic sequencing;
- `PrivacyProfile` with `local_full`, keyed `safe_telemetry`, and separate `aggregate_export` modes;
- privacy-preserving aggregate export with a configurable minimum group size of five by default;
- field classification for safe-by-default, pseudonymous, and potentially sensitive evidence;
- HMAC-SHA-256 identifier pseudonymization, UTC-day timestamp generalization, strict free-text removal, and local 256-bit key management;
- opaque random local identifier generation for runs, tasks, and other caller-defined namespaces;
- grouped aggregate exports with deterministic cost/gain buckets, no identifiers, and no timestamps;
- `ledger-export` CLI, overwrite protection, aggregate JSON Schema, privacy guide, and executable privacy example;
- Universal Agent Protocol v1 values, strict round-trip parsing, capability negotiation, directives, state-aware deduplication scopes, and `UniversalRuntime`;
- off-policy decision replay with explicit non-causal reporting;
- ledger validation, ledger reporting, and replay CLI commands;
- JSON schemas for events, decisions, capabilities, outcomes, token usage, and ledger records, plus an installed-resource API through `available_schemas()` and `load_schema()`;
- configurable public-benchmark confidence level, non-inferiority margin, random seed, and cost-per-resolved efficiency metrics;
- executable Shadow Mode and universal-runtime examples;
- complete Learning Loop Foundation, universal-runtime, API, architecture, benchmarking, security, and roadmap documentation.

### Changed

- `Decision` is backward compatible but now carries recommendation, mode, reason-code, uncertainty, confidence, and estimator metadata;
- `MarginalPolicy` now has a stable identity and supports both versioned and legacy custom estimators;
- `Treasury.summary()` includes mode, policy, estimator, observed overruns, failed settlements, outcomes, and estimator observations;
- child treasuries inherit the parent execution mode;
- trace events include execution mode plus policy and estimator identity;
- Decision Ledger records now declare their privacy profile; `local_full` remains backward compatible while strict profiles are opt-in;
- actual failed-call spend can be accounted without replacing the original execution exception; extraction failures conservatively settle the reserved estimate and remain chained as secondary errors;
- strict public-benchmark parsing rejects string-like booleans instead of silently coercing them;
- project description and documentation now consistently describe MARGINAL as a learning-loop foundation rather than only a static wrapper.

### Compatibility

- existing v0.1 constructors, enforced execution, `JsonlTraceSink`, synchronous and asynchronous wrappers, demos, and CLI commands remain supported;
- new dataclass fields have backward-compatible defaults;
- the runtime core continues to have zero mandatory dependencies.

### Scientific limitations

- historical estimates are observational and do not establish causal action value;
- policy replay does not simulate unobserved trajectories or prove quality preservation;
- vendor-specific Codex, Claude Code, GitHub Copilot, and OpenCode adapters remain future milestones;
- reference profiles are transparent defaults, not universal calibrations.

## [0.1.0] - 2026-08-04

### Added

- provider-neutral `Action`, `Cost`, `Decision`, and `Allocation` value objects;
- deterministic `fund_best` candidate ranking and reservation;
- deterministic candidate ranking, authorization, reservation, settlement, and abort;
- hard budgets for tokens, direct USD, latency, and risk;
- pending reservations that prevent concurrent and hierarchical oversubscription;
- pending reservations and hierarchical parent/child accounting;
- protected verification reserves;
- marginal-value policy with token, latency, and risk shadow prices;
- expected-gain capping against the remaining success target;
- exact action and callable-input fingerprinting;
- duplicate prevention for pending and completed work;
- hierarchical treasuries with atomic child and parent accounting;
- explicit abort lifecycle and automatic release on callable failure;
- truthful settlement of actual usage with `BudgetOverrun` reporting;
- synchronous and asynchronous guarded-call and funded-allocation adapters;
- common OpenAI-, Anthropic-, and LiteLLM-like usage extraction;
- append-only JSONL traces and trace reporting CLI;
- transactional trace failure handling that preserves budget and primary-error semantics;
- deterministic synthetic benchmark and integration examples;
- end-to-end Killer Demo with a real generated code defect, verified baseline, action
rankings, HTML/Markdown/SVG/JSON artifacts, one-command CLI execution, and a GitHub
Pages deployment workflow;
- Python 3.10–3.13 CI, CodeQL, release automation, and community documentation.

### Limitations

- the bundled savings result is synthetic and is not a production performance claim;
- expected gains are caller-provided or based on transparent observed averages;
- causal value estimation and counterfactual replay are not included in this release.
- synchronous and asynchronous guarded-call adapters;
- append-only JSONL traces and CLI reporting;
- synthetic benchmark, public comparison utility, and Killer Demo;
- Python 3.10–3.13 CI, CodeQL, packaging, and community documentation.
10 changes: 6 additions & 4 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
cff-version: 1.2.0
message: "If you use MARGINAL in research or production evaluation, please cite this software."
title: "MARGINAL: Compute Capital Allocation for AI Agents"
title: "MARGINAL: Economically Disciplined Compute Allocation for AI Agents"
type: software
authors:
- name: SignalLayer Labs
version: 0.1.0
date-released: 2026-08-04
version: 0.2.0
date-released: 2026-08-06
license: Apache-2.0
repository-code: "https://github.com/SignalLayerLabs/Marginal"
url: "https://github.com/SignalLayerLabs/Marginal"
keywords:
- artificial intelligence
- AI agents
- token optimization
- cost optimization
- compute governance
- decision ledger
- agent infrastructure
- AI FinOps
Loading