Gamma is a Python toolkit for interactive model exploration, model comparison, benchmarking, and controlled capability-transfer experiments. Its CLI can run a token-prediction game, compare model outputs, benchmark engines, and coordinate multiple active model engines.
Gamma’s mission is to make model and algorithm behavior inspectable, comparable, and reproducible under named evaluation contracts.
The current goal is to give researchers and engineers one place to inspect token choices, compare models and runtimes, run benchmark workloads, and record whether a change improved the named task. The repository also carries domain experiments for translation, embeddings, WGSL, compression, and the SAME-R method.
Gamma serves:
- Researchers testing prompts, datasets, training methods, or routing policies.
- Engineers comparing engines, kernels, latency, and output quality.
- Reviewers checking whether an improvement survives a fixed benchmark and replication contract.
- People learning how tokenization, logits, attention, and decoding affect an output.
Create the environment and run the interactive game:
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python gamma.py gameThe main command is:
python gamma.py [command] [options]| Surface | Command or link | Use |
|---|---|---|
| Game and chat | python gamma.py game |
Explore next-token prediction and model choices. |
| Comparison | python gamma.py comparison |
Compare model outputs and decoding behavior. |
| Benchmarking | python gamma.py benchmark |
Measure throughput, latency, output quality, or code generation. |
| Mind Meld | python gamma.py mind-meld |
Coordinate and route across active model engines. |
| SAME-R | projects/samer/ |
Run matched capability-transfer trials. |
| Domain research | projects/ |
Inspect translation, embedding, WGSL, and compression experiments. |
| Integrations | integration guide |
Connect APIs, MCP, and external backends. |
Use the command help for workload-specific options:
python gamma.py help benchmark
python gamma.py help codegen
python gamma.py game --comparison --help
python gamma.py mind-meld --helpSAME-R means Swappable Approaches under Matched Evaluation and Replication. It changes the approach while holding the evaluation benchmark, success metric, objective, control environment, and replication contract fixed. The method helps attribute a measured change to the approach; it does not make a weak benchmark or an incomplete replication valid.
For each experiment, record the named baseline, changed inputs, evaluation set, metric, runtime, and replication result. The SAME-R protocol defines the experiment contracts. The benchmarking guide defines the comparison and performance workflow.
Gamma is intended to become a shared workbench for model and algorithm behavior research: interactive experiments for inspection, repeatable benchmarks for comparison, and controlled training, routing, or algorithm trials for capability changes. Results should remain tied to the exact model or algorithm, prompt or dataset, runtime, metric, and replay needed to inspect them.
Gamma does not turn a benchmark result into a general model claim. Distillation,
RLVR, routing, custom kernels, and prompt changes remain separate experiment
variables unless a study explicitly combines them. Engine-specific setup may
require local GPU, API, or backend configuration; see
src/engines/README.md. Model weights are not
installed automatically.
src/— runtime, engines, game logic, comparison, and benchmarksprojects/— distillation, SAME-R, embedding, translation, WGSL, and compression worktools/— developer and model-analysis utilitiesrequirements/— dependency manifests by hardware and enginedocs/— architecture, benchmark, integration, and method guidestests/— automated test suitesgamma.py— CLI entrypoint