---

HyTorch is a Python library for composing and training meta-networks of coding agents with a PyTorch-shaped API. Define dynamic graphs of cooperating agents in Python, give them versioned directories, and improve their behavior with plain-language feedback.

Quickstart

Prerequisites

• Python 3.11 or later
• Git
• One supported agent CLI or the packaged Pi Docker runtime

Installation

Install HyTorch with uv or pip:

# uv (recommended)
uv add hytorch

# pip
pip install hytorch

Examples

Research and synthesize an answer

This example runs three research agents in parallel. A fourth agent combines their work into one supported answer.

A model subclasses mn.Module. HyTorch uses mn.Linear(in_features, out_features) to describe its agent layers. The first number is the number of inputs. The second number is the number of agents that run and produce outputs.

import hytorch
import hytorch.mn as mn


class ResearchNetwork(mn.Module):
    def __init__(self):
        super().__init__()
        self.research = mn.Linear(1, 3, bias="Find independent evidence.")
        self.synthesize = mn.Linear(3, 1, bias="Resolve conflicts and cite sources.")

    def forward(self, state, question):
        evidence = self.research(state, task=question)
        return self.synthesize(evidence, task=question)[0]


model = ResearchNetwork().to("pi")

The forward() method defines a 1 → 3 → 1 network. The first layer receives one input and runs three agents. The second layer receives all three results and runs one synthesis agent. Each bias value gives those agents their initial instructions. Each agent owns a persistent working directory that it can improve during training. model.to("pi") selects the built-in Pi agent runtime.

Give the model a task

HyTorch agents work on complete directories. Each directory uses Git so every input and output has an exact version and history. Create a small input directory with one committed question:

mkdir research
git -C research init -b main
echo "Compare the candidate designs." > research/question.md
git -C research add .
git -C research commit -m "Add research question"

Wrap that directory with hytorch.space. A Space is the value that moves between HyTorch layers, just as a tensor moves between PyTorch layers.

state = hytorch.space("research")

Run the model

Run the model in inference mode when you only need an answer:

with hytorch.inference_mode():
    output = model(state, "Which design has the strongest evidence?")

print(output.dir)  # complete output directory
print(output.commit)  # immutable Git identity

The result is another complete Git-backed directory. The agents decide which files to create or change, then commit their work. output.dir is the output directory. output.commit identifies its exact contents. Inference mode closes runtime resources and discards all private agent-state changes.

Improve the network with feedback

An evaluator can inspect an output and return a plain-language direction for improvement. HyTorch sends that direction backward through the executed network. The DFM optimizer manages these agent updates as complete model generations.

optimizer = hytorch.optim.DFM(
    model.parameters(),
    temp=0.7,
    max_tokens=10_000,
)

for state, target in training_data:
    optimizer.zero_feed()
    output = model(state, target.question)
    feedback = evaluate_with_tools(output, target)
    loss = hytorch.Loss(output, feedback=feedback)
    loss.backward()
    optimizer.step()

Useful feedback is specific and imperative:

Preserve source identifiers when you combine the reports.
Test malformed inputs before you select an implementation.
Keep contradictory evidence and explain how you resolved it.

This lifecycle mirrors PyTorch training. zero_feed() clears feed from the previous iteration. Forward runs a disposable episode fork. backward() resumes the episode and accumulates owner mutation feed. step() resumes each persistent owner once with all accumulated feed. It promotes all updates as one model generation.

Save and load model state

Save the complete canonical model state as a self-contained Git directory:

hytorch.save(model.state_dir(), "research-network")

Create the same model structure, then load the saved workspaces:

restored = ResearchNetwork().to("pi")
result = restored.load_state_dir(hytorch.load("research-network"))

This follows the PyTorch state_dict() pattern with a directory-native state:

# PyTorch
torch.save(model.state_dict(), "model.pth")
model.load_state_dict(torch.load("model.pth", weights_only=True))

# HyTorch
hytorch.save(model.state_dir(), "model-state")
model.load_state_dir(hytorch.load("model-state"))

A StateDir identifies one immutable model commit. The saved directory contains MODEL.json, every complete native agent state, and the canonical model Git history. Native state can include transcripts, memories, compaction records, skills, settings, and databases. It never includes credentials or live process state. Loading is strict by default. Pass strict=False to permit missing or unexpected workspace keys with compatible shapes. The save destination must not already exist.

PyTorch-shaped composition

HyTorch follows PyTorch syntax and ownership where a direct agent equivalent exists:

# PyTorch
tensor = torch.tensor(data, requires_grad=True)
layer = torch.nn.Linear(3, 4, bias=True)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

# HyTorch
state = hytorch.space(directory, requires_feed=True)
layer = hytorch.mn.Linear(3, 4, bias="Synthesize the inputs.")
optimizer = hytorch.optim.DFM(model.parameters(), temp=0.7)

Assignment registers child Modules and Parameters. model.parameters() is the normal optimizer input. mn.Linear(in_features, out_features) owns one directory-backed workspace for each output agent. Its physical weight shape is (out_features,). Every output receives every input Space, so the logical layer is dense.

The bias argument initializes each workspace's mutable AGENTS.md. The native harness and agent can later replace or extend the complete state in any format.

How one agent runs

Each output agent receives three sibling directory trees:

node/
├── statespace/    # activation: writable during forward
├── parameter/     # read-only canonical native state
└── workspace/     # writable temporary episode fork

During forward, the agent merges every input statespace, transforms the merged tree, and commits the result. Its native transcript, memory, and other local state can change inside the episode, but forward never changes the Parameter.

During backward, HyTorch resumes the episode. The statespace and Parameter are read-only. The episode returns one owner proposal and one direction per input. HyTorch accumulates these proposals in .feed and discards the episode. step() resumes the persistent owner once and lets it update its complete native state from all accumulated feed.

Harnesses and environment

One executed graph uses one harness:

model.to("pi")
model.to(harness="pi", mtype="gpt-5.6-terra")

The built-in harness identities are pi, codex, claude-code, opencode, hermes, and prime-agent. Pi uses gpt-5.6-terra by default. Each harness uses its native local profile and session format inside the Parameter.

Identity	Runtime	Persisted native state
pi	Packaged Pi SDK runtime	Pi profile and JSONL session
codex	codex CLI	CODEX_HOME, transcript, and project memory
claude-code	claude CLI	Claude config, projects, and JSONL session
opencode	opencode CLI	Isolated home and all XDG state directories
hermes	hermes CLI	HERMES_HOME, state.db, memories, skills, and profile
prime-agent	prime-agent CLI	Profile, JSONL session, RLM children, and session artifacts

You can construct a harness when you need a custom binary, model, provider, or external credential sidecar:

harness = hytorch.harness.CodexHarness(binary="codex")
model.to(harness)

Agent variables come from ~/.config/hytorch/secrets.env, project .hytorch.env, HYTORCH_ENV_FILE, and exported shell variables, in increasing precedence. HyTorch never loads an ordinary .env file. It does not put secret values in prompts or Git state.

Full example

Terminal-Bench trains and evaluates a 1 → 3 → 1 HyTorch network on Terminal-Bench 2.1 tasks.

Project status

HyTorch 0.1.0 is the first public alpha release. Run agents in isolated environments and review agent-created changes before production use.

Version 0.1.0 includes Spaces, native-state Parameters, dynamic Module graphs, dense Linear layers, directional backward feedback, atomic DFM optimizer generations, and six native agent harnesses.

Resources

• Specification — canonical behavior and invariants
• Glossary — complete PyTorch correspondence
• Contributing — development and test workflow
• Security — trust boundaries and private reporting
• Changelog — release history

License

HyTorch is released under the Apache License 2.0.