Skip to content

Repository files navigation

opencode-shrink

Always-on token compression for opencode agent internal reasoning and responses.

Shrink compresses tokens in two independent modes:

  • compress-in: Internal reasoning, chain-of-thought, planning, intermediate state — always active, saving ~40-55% on every reasoning step
  • compress-out: Response style — off by default, toggle with /shrink-out. 4 levels (off/lite/full/ultra) for ~15-50% output savings

Four compression levels (off / lite / full / ultra) let you balance savings against verbosity.

Quick Start

  1. Install Shrink as an opencode skill (does not overwrite existing AGENTS.md):

    mkdir -p ~/.config/opencode/skills/shrink
    cp SKILL.md ~/.config/opencode/skills/shrink/SKILL.md
    cp AGENTS.md ~/.config/opencode/AGENTS.md.bak  # back up existing rules
    
  2. Append Shrink's agent rules to your existing AGENTS.md (or create it):

    cat AGENTS.md >> ~/.config/opencode/AGENTS.md
    
  3. Shrink activates automatically on next epoch start at level full.

  4. Customize at any time using opencode slash commands:

    • /shrink — load Shrink skill reference
    • /shrink level lite|full|ultra — change compression aggressiveness
    • /shrink-out on — enable output compression at default (full, ~30%)
    • /shrink-out level lite|full|ultra — set compress-out level
    • /shrink-out off — disable output compression
    • /shrink status — view current settings

How It Works

Shrink instructs the LLM to compress its own internal reasoning using:

Technique Savings Example
Drop articles/copula/fillers ~15% the error is aterror at
Ternary expressions ~35% if auth then load else 401auth? load : 401✗
Flow chaining ( | ) ~55% then we validate and parsevalidate | parse
State markers (✓ ✗ ⚠) ~40% the check passed successfully
Structural notation (§ λ ctx) ~45% let me look at the bug section§bug:
Quantifiers (∀ ∃ ∈ ¬ ∧ ∨) ~30% for each file in the directory∀file∈dir

Entity Protection — Hard Constraint

File paths, function names, error messages, numeric values, and variable names must survive compression identically. If compression would change any character of an entity, the LLM keeps it in natural language instead. This prevents the 0% fidelity failures seen in prior approaches.

MISTAKE: redisClient.incr → INCR          (lost API reference)
MISTAKE: src/middleware/rateLimiter.ts → middleware   (lost path)
CORRECT: src/middleware/rateLimiter.ts stays verbatim

Compression Levels

Level Savings What's Applied
off 0% No compression
lite ~20-30% Drop articles, copula, fillers, pleasantries
full ~40-55% Lite + flow symbols, ternary, state markers, quantifiers, structural notation
ultra ~55-70% Full + aggressive, all entities in backticks, pure fragments

Slash Commands

Shrink provides opencode slash commands (in .opencode/commands/). Copy them to ~/.config/opencode/commands/ to use:

Command Action
/shrink Load skill reference & show status
/shrink level lite|full|ultra Set compression level (default: full)
/shrink-out on|off Toggle output compression (default: off)
/shrink status Show current settings

Benchmarks

⚠️ Important caveat: These benchmarks measure token savings on hand-authored compressed examples, not actual model output. They show the potential upper bound of compression, not the real-world savings a model achieves. See limitations for details.

Measured across 5 agentic reasoning task examples (bug fix, code review, feature add, investigation, refactor) using tiktoken o200k_base (GPT-4o tokenizer):

Metric Value
Average token savings (hand-authored) 53.3%
Best savings (Bug Fix) 60.5%
Worst savings (Investigation) 40.4%
Entity fidelity (exact match) 100.0% (5/5 tasks)
Encoding compatibility o200k_base (primary), cl100k_base (documented diffs)

Run python3 benchmark.py to reproduce results (requires tiktoken). Task fixtures are in benchmark/tasks/.

Model-Based Tests (Optional)

Requires a local OpenAI-compatible API endpoint (e.g., llama.cpp server):

# A/B comparison: does compression degrade response quality?
python3 benchmark_ab.py --runs 3 --cooldown 5

# Roundtrip: can compressed text be faithfully decompressed?
python3 benchmark_roundtrip.py

# Run all tests together:
python3 benchmark.py --with-model --ab-runs 1 --ab-cooldown 5

Test results are saved to benchmark/results/ (JSON and Markdown format).

Note: The 4B Qwen model used during development OOMs on repeated long-context requests. A larger model (7B+) or GPU with more VRAM is recommended for reliable model-based testing.

Cross-Tool Compatibility

Shrink works with any agent that supports AGENTS.md (or similar per-project instruction files). See adapters/ for per-tool install guides.

Agent Install method Load timing Token cost (tiktoken o200k)
opencode Skill: ~/.config/opencode/skills/shrink/SKILL.md On load ~0 (skill, lazy-loaded)
Pi Coding Agent ~/.pi/agent/AGENTS.md or project root Per session ~790 tok (AGENTS.md) once
Claude Code @AGENTS.md import via CLAUDE.md Per session ~790 tok once
Cursor .cursor/rules/shrink.mdc (Always rule) Every request ~350 tok/req
Hermes Agent ~/.hermes/SOUL.md appendix or project root Per session ~790 tok once
OpenClaw SKILL.md (lazy-loaded, recommended) On demand ~few tok (lazy)
Codex CLI Project root AGENTS.md Per session ~790 tok once
Gemini CLI Project root AGENTS.md Per session ~790 tok once
Aider .aider.conf.ymlread: AGENTS.md Per session ~790 tok once

The same AGENTS.md content works for all tools — it's just markdown instructions. The only differences are file placement and symbol encoding considerations (cl100k vs o200k_base).

Requirements

  • opencode agentic coding environment (or any AGENTS.md-compatible agent — see compatibility table above)
  • No plugins, no hooks, no external dependencies — works purely through AGENTS.md instructions loaded once per epoch

Credits & Resources

This project builds on prior work and ideas in token optimization for LLM reasoning:

  • Shrink benchmark data — Token savings and entity fidelity measurements generated by the included benchmark suite. See the Benchmarks section for current results.
  • tiktoken — OpenAI's tokenizer, used for all token efficiency measurements. Symbol cost tables verified against o200k_base (GPT-4o). Token costs may differ for other providers' tokenizers.
  • opencode — The agentic environment Shrink targets. Shrink leverages opencode's AGENTS.md injection and skill system.
  • Ponytail — YAGNI coding persona. Its philosophy of reducing output footprint informed Shrink's approach to compressing at the reasoning stage.
  • AGL Confidence Scale (●◕◑◔○) — Notation borrowed from AGL specification for expressing confidence levels in compressed reasoning.

Unsupported Claims Removed

Previous versions of this README cited specific arXiv papers and star counts that were inaccurate. The arXiv ID 2502.12065 does not correspond to the cited BabelTele or MetaGlyph research. Star counts for opencode-linked projects were incorrect. This section has been corrected to reference only verified sources. See limitations for discussion of what this project does and does not yet validate.

About

Always-on token compression for opencode agent internal reasoning and responses.

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages