Guarded Agentic Compaction

Compile the routine. Refuse the uncertain.

Guarded Agentic Compaction is a research library for turning recurrent, read-only regions of tool-using agents into deterministic guarded programs. It learns from execution traces, but recurrence is never enough: every candidate must pass value-provenance, effect, permission, runtime-position, replay, compatibility, and finite-sample evidence checks. Otherwise the original agent remains unchanged.

Documentation · Current paper (ICLR 2027 draft) · Latest evidence review · Long-form article · Adversarial review · Experiment verification

Publication shelf

The public Pages site keeps the release easy to read without requiring a checkout:

Read	Link
Long-form article in HTML	Browser edition
Long-form article as PDF	Download the article
Evidence and benchmark audit	Open the benchmark explorer
Editable presentation	Download the technical deck

The artifact shelf also explains which source, evidence class, and claim boundary belongs to each item.

Which document is current. The ICLR 2027 draft in paper/iclr/ is the current paper. The long-form article (HTML and PDF above) reports the later studies (BIRD, the post-cutoff pull-request certificate, the second-model and second-provider replications, the retired end-to-end issue gate, and the failed guards) in its "Later Evidence" section, but without the ICLR paper's tables and protocol detail. Where the two differ, the ICLR paper and its evidence review govern. Both slide decks follow the ICLR paper.

Why this exists

Mature agents often spend model calls reselecting the same evidence path. A hand-written macro can remove that overhead, but it requires a developer to discover the pattern, reconstruct argument flow, review effects, maintain invalidation rules, and prove it still works. Guarded Agentic Compaction automates that evidence and lifecycle path for a deliberately narrow class of workflows.

The research contribution is admissibility for trace-derived specialization, not the first agent compiler and not a claim that generated programs outperform good manual code.

Core system

flowchart LR
  A[Application traces] --> B[Typed Episode IR]
  B --> C[Execution graph]
  C --> D[Candidate mining]
  D --> E{Provenance and effects}
  E -- unsafe or unknown --> R[Retain baseline]
  E -- qualified --> S[Bounded synthesis]
  S --> V[Replay and perturbation]
  V --> G{Exact admission}
  G -- insufficient evidence --> R
  G -- admitted --> P[Immutable guarded program]
  P --> H[Shadow]
  H --> Cn[Canary]
  Cn --> L[Live with fallback]

Loading

• GRC — guarded region compilation: infers readable programs from a closed 23-operator language and emits an artifact only when its arguments are grounded and its effects are safe to speculate.
• GCS — guarded composite synthesis: projects an already admitted program behind one task-specific interface and may execute it before the first provider request only under an exact continuation-manifest pin.
• TGWS — trace-guided workflow specialization: learns a shallow route from entry state to an existing specialist prompt and minimal tool surface.
• Portfolio selection: compares only paired, measured actions and keeps manual macro recommendations review-gated.

The framework-neutral typed Episode IR separates capture from optimization. OpenAI Agents SDK capture and runtime integration are maintained adapters, not compiler dependencies.

What the evidence shows

Current paper (ICLR 2027 draft)

Result	Evidence	Interpretation
Three GitHub families, 90 held-out records	compiled 90/90 versus unchanged 89/90 (McNemar p=1; one-sided bound on compiled-only discordance 3.3%); requests −66.6%, tokens −63.1%, latency −64.2%, cost −58.7%	a grounded read-only opening preserves the exact contract at roughly a third of the work, within one pinned snapshot; equivalence is not established
Post-cutoff pull requests	0 end-to-end misses on 132 fresh calibration groups; one-sided bound 0.0173 (0.035 under the primary grid); 84 open, 48 merged, no closed-unmerged	the first certificate whose event includes the model's answer; one family, and the closed-unmerged class is not covered
Six held-out GitHub studies	one compiled-only contract failure in 810 episodes	an observed count, not a certified rate
Cross-repository PR outcome	120/120 on four of five repositories; pytorch/pytorch retires at admission; open-class records abstain on the induced hull	compile-or-retire holds on newer records for an exact two-read task
BIRD text-to-SQL	a schema-first opening compiles on all five qualifying dev databases (150 sealed questions: 97 versus 94 correct, McNemar p=0.45; requests −44.9%, latency −29.4%); 21 of 26 training databases compile; per-question table choice is refused	the one public benchmark where GAC helps; no accuracy loss detected, paired 95% interval on the training split [−3.27, +2.96] points under i.i.d. pairs
NESTFUL, API-Bank, BFCL, AppWorld	best family support 26, 8, 15 versus 92 required; AppWorld admits one program but it is entry-eligible in 37.9% of 8,190 trajectories	no measured benefit: thin evidence or admissible but rarely dispatchable
Second model and provider	gpt-6-luna and claude-sonnet-5 re-derive the same programs on two of three families and refuse backlog routing	the compile-or-retire decision transfers on two of three families
Learned gate versus support-only	indistinguishable across five repositories; an amended 474-issue end-to-end study (200 development, 184 calibration, 90 untouched test) admitted no threshold (best bound 0.0546 against 0.05) and retired before the test	no demonstrated risk–coverage frontier: the certificate mostly confirms that enough examples were seen
Corrupted tool outputs	run-time checks prevented no wrong answers	the guards are not shown to protect against drift

The checked-evidence assessment is 79/100, borderline (a diagnostic judgment, not a reviewer score); see the evidence review.

Earlier long-form results (retained; the GCS, portfolio and comparator rows are not in the ICLR paper)

The primary evaluation spans three distinct workflows over real public GitHub records, deterministic tools on a pinned snapshot, and live provider calls. Each family has a different tool vocabulary and exact decision contract.

Result	Evidence	Interpretation
Three-family total	compiled 90/90 versus baseline 89/90 (McNemar p=1; pooled discordance bound 3.3%); requests −66.6%; tokens −63.1%; observed wall latency −64.2%; estimated cost −58.7%	efficiency transfers across issue type, PR outcome, and backlog attention
Conservative GAC	30/30 exact task contracts; requests −50.0%; tokens −39.5%; observed wall latency −51.7%; estimated cost −32.0%	a grounded two-read prefix can remove one model boundary
Hand-written programs	90/90; fair pre-model programs tie learned programs on the two new families	manual code remains the practical runtime baseline
Aggressive GAC	17/18 versus 18/18 for both comparators	clean program replay does not certify the final answer
GCS vs provider-visible macro	12/12 each; GCS uses one versus two requests	pre-model projection removes an interface request
GCS vs fair pre-model manual	6/6 each; tied requests, interfaces, and input tokens	automatic runtime superiority is not established
Cross-repository, time-forward extension	580/580 exact discovery traces over five frozen repositories; four complete 120/120 held-out pairs, pytorch/pytorch retires at compile time; a balanced rerun adds 360/360 discovery and 180/180 held-out on three repositories	the guarded lifecycle survives a new repository and a later window on an exact two-read task
Prospective gate-frontier study	240/300 pooled held-out pairs on four of five repositories (pytorch/pytorch retires); the cohort overlaps the earlier core study; a support-only comparator (alpha=1, otherwise byte-for-byte identical) is statistically indistinguishable from the learned gate on every metric	the exact alpha=.05 gate remains a support threshold against a comparator built to isolate the risk budget specifically — the pre-declared null, not a demonstrated frontier
NESTFUL and API-Bank	every recurrent family retires	recurrence does not imply admissibility
Selective-risk levels	four artifacts at alpha=.05, three at alpha=.10, portfolio at 15%	GCS and comparator results are licensed only at 10%, not the registered 5%
Calibration proof scope	exact Clopper--Pearson plus 11-threshold union bound for one fixed candidate; a frozen-selection corollary closes the compiler-wide multiplicity gap for searches that adopt it	the three primary GitHub families do not freeze and remain per-candidate; two candidates there require 106 rather than 92 zero-violation groups
Cache accounting	macro: −30.9% tokens but only −8.0% cost at 0.0% cache reads vs. 27.8%	token reduction is not a proxy for cost reduction
Amortization	provider-side break-even 411 / 182 / 181 episodes per family	a shallow admitted prefix may never repay its discovery

The artifact also preserves a revision-pinned supplementary audit of eight other agent benchmarks. Those paths verify interoperability or explicit gates but do not demonstrate optimizer value and are excluded from the main comparison.

The three-family headline results are research-prototype measurements on one repository snapshot. The separate PR-outcome extension carries the cross-repository, time-forward evidence, but only for an exact two-read task, so neither line establishes full-workflow cross-repository or time-forward generalization. Nothing here establishes semantic equivalence, production certification, or state-of-the-art quality. See the latest evidence review, the earlier adversarial review, limitations, and claim audit.

Install

git clone https://github.com/rrahimi-uci/guarded-agentic-compaction.git
cd guarded-agentic-compaction
python -m venv .venv
.venv/bin/pip install -e '.[dev,live,figures]'

Python 3.11–3.14 is supported. The core library does not require MLflow or a provider credential.

Start with an estimate

The estimator is designed to produce a fast, inexpensive refusal before synthesis:

guarded-agentic-compaction estimate traces.jsonl \
  --effects configs/effects.example.yaml \
  --entry channel locale product

Compile only after the trace and effect contracts are complete:

guarded-agentic-compaction compile traces.jsonl \
  --effects configs/effects.example.yaml \
  --entry channel locale product \
  --out artifacts/v1

guarded-agentic-compaction explain artifacts/v1
guarded-agentic-compaction promote artifacts/v1 --stage shadow

Python API

import guarded_agentic_compaction as ac

episodes = ac.read_jsonl("traces.jsonl")
catalog = ac.load_catalog("configs/effects.example.yaml")

job = ac.optimize(
    episodes,
    catalog,
    algorithms=["grc", "tgws"],
    mode="offline",
    partition_by=["tenant_partition", "principal", "policy_version"],
    entry_schema=["channel", "locale"],
    sandbox=make_sandbox,
)

print(job.report())
print(job.explain())
ac.validate(job, suites=["replay", "perturbation"])
ac.promote(job, stage="shadow")

The compiler is fail-closed. Unknown effects, writes, approvals, handoffs, missing provenance, unsupported dispatch positions, streaming boundaries, and incompatible manifests retain the baseline.

OpenAI Agents SDK

Capture normalized traces with the maintained tracing processor, then integrate either at the application-owned staging boundary or through the guarded model adapter.

from guarded_agentic_compaction.capture import (
    AgentsTraceProcessor,
    install_agents_trace_processor,
)

processor = install_agents_trace_processor(
    AgentsTraceProcessor(include_sensitive_data=False)
)

from guarded_agentic_compaction.runtime.model_provider import CompactingModel

agent = Agent(
    name="support",
    model=CompactingModel(
        base_model,
        registry=registry,
        catalog=catalog,
        manifest=manifest,
        mode="shadow",
    ),
    tools=tools,
)

Streaming, hosted tools, incomplete handoffs, and server-managed context are rejected rather than silently approximated. The SDK integration guide documents the exact boundary.

Reproduce the checked-in evidence

No API key is needed to rebuild figures, tables, paper checks, and sealed-result analyses:

.venv/bin/python paper/scripts/build_artifacts.py
.venv/bin/coverage run -m pytest -q
.venv/bin/coverage json -o paper/results/coverage.json
.venv/bin/python scripts/verify_release.py
.venv/bin/python paper/scripts/validate_artifacts.py
.venv/bin/python scripts/build_pages.py --output _site

site/article.html is the full manuscript rendered for the web, generated from paper/tex/body.tex by scripts/build_article_page.py. That step needs pandoc and is not part of the loop above, because the Pages job installs no third-party tooling beyond pyyaml. Regenerate it whenever the manuscript changes:

python scripts/build_article_page.py          # rewrite site/article.html
python scripts/build_article_page.py --check  # verify it matches paper/tex/ without writing

build_pages.py recomputes the source digest the page carries and fails the build if the committed article has fallen behind the manuscript, so a stale page cannot deploy.

Paid study commands are documented separately in paper/README.md. Existing raw provider evidence is sufficient to reproduce all reported results. Local credentials are never required for the standard test or documentation path.

Repository map

src/guarded_agentic_compaction/
  capture/      framework adapters and normalized Episode snapshots
  graph/        qualification, provenance, and recurrent-window mining
  grc/          DSL, synthesis, contracts, calibration, guarded composites
  tgws/         route learning and quality-anchored pruning
  portfolio/    paired evidence, exact-risk admission, reviewed selection
  evaluation/   grouped splits, replay, perturbation, metrics, statistics
  runtime/      interpreter, dispatch, staging, fallback, SDK model adapter
  registry/     immutable artifacts, lifecycle, signing, kill switch, rollback
benchmarks/     public-data adapters, manifests, gold, and external benchmark gates
demos/          simulated worlds and effect catalogs used by the offline studies
paper/          LaTeX sources, figures, tables, raw results, scripts, slides
  iclr/         condensed conference submission (own style files and build)
  linkedIn_article/  practitioner write-up and its figure sources
  open_research/     compiled manuscript PDFs
site/           source for the GitHub Pages documentation
team-reviews/   per-reviewer manuscript review records
tests/          unit, property, integration, mutation, and fault-injection tests

docs/ holds deeper engineering records (ADRs, safety model, trace contract, and
library API). It is deliberately untracked -- see .gitignore -- because parts of it
are generated. The public release surface is the Pages site and the tracked paper,
benchmark, and evidence artifacts; use the [artifact shelf](https://rrahimi-uci.github.io/guarded-agentic-compaction/artifacts.html)
for the reader-facing copies.

Contributing and security

Contributions are welcome for trace adapters, compiler checks, evaluation, runtime controls, and documentation. Start with CONTRIBUTING.md. Every behavior change needs focused tests; safety-boundary changes need mutation or fault-injection evidence. Never submit credentials, restricted traces, or customer data.

Please report vulnerabilities privately as described in SECURITY.md. The project follows the code of conduct and is licensed under Apache-2.0. Release-level changes are recorded in CHANGELOG.md.

Citation

The manuscript is From Traces to Guarded Programs: Evidence-Gated Compilation of Recurrent Agent Workflows by Reza Rahimi, Huaxing Wang, Mi Hwangbo, Roberto Borgione, ArunKumar Patange, and Indranil Dutta (JazzX AI, Palo Alto, CA).

Build	Path
Complete technical report, single column (appendix included)	paper/open_research/article.pdf
Two-column conference build (appendix ships separately)	paper/open_research/main.pdf
ICLR discussion draft (10 main-text pp. + appendix)	paper/iclr/ (build with tectonic --outdir build main.tex)

A versioned citation will be added after archival release; until then, cite the repository commit and the paper PDF together.