SPEC — re:factory (remote-factory)

Status: Draft | Auto-generated by re:factory

Normative Language

The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119.

Implementation-defined means the behavior is part of the implementation contract, but this specification does not prescribe one universal policy.

1. Problem Statement

re:factory is a harness that runs autonomous coding-agent subprocesses through a repeatable experiment lifecycle — hypothesize, implement, evaluate, keep-or-revert, archive — so that improving software with LLM agents becomes a measured, auditable process instead of an unstructured chat session.

It solves several concrete operational problems. It replays a fixed detect → route → delegate → verify → decide → archive cycle instead of letting an agent decide its own scope and stopping criteria, so runs converge on completion rather than trailing off at a self-judged "good stopping point." It scores every change against a weighted composite of hygiene, growth, and project-specific dimensions instead of trusting an agent's self-report that a change "looks good," so keep/revert decisions are grounded in eval deltas rather than vibes. It enforces Sacred Rules (immutable eval directories, scope guards, mandatory QA, no PR merges) via subprocess-level guard checks instead of relying on prompt instructions alone, so a misbehaving or compromised agent cannot silently widen its own blast radius. It persists institutional memory (.factory/archive/, ACE playbooks, cross-project insights) between runs instead of starting every session from a blank context, so agents accumulate DO/DON'T rules from real outcomes rather than repeating the same mistakes. It exposes a portable workflow-graph format (nodes, edges, gates, forks/joins) that renders to both a deterministic headless executor and a Claude Code SKILL.md playbook, instead of hand-maintaining two divergent descriptions of the same pipeline. It provides pluggable CLI backends (Claude Code, Bob Shell, Codex, OpenCode) and pluggable execution runtimes (bare host, podman container, OpenShift cluster) behind one command surface, instead of hard-wiring the orchestrator to a single vendor or a single machine.

Important boundary: re:factory is not itself a coding model, a sandbox, or a CI system. It does not execute untrusted agent-authored code inside a security boundary it can vouch for — the contained runtimes explicitly document that neither the local podman target nor the Kubernetes target confines agent-authored code, and both require the same human PR review that any agent-produced diff would need. It does not replace human code review; every kept experiment produces an open PR for a human to merge. It does not train or fine-tune models — "self-improvement" means evolving markdown playbooks and workflow-graph topologies that steer off-the-shelf agent CLIs, not gradient updates.

2. Goals and Non-Goals

2.1 Goals

Detect a target project's factory-management state (no_repo, incomplete, no_factory, evals_pending_review, has_factory) from git and .factory/ filesystem evidence alone, without relying on agent memory.
Parse a project's factory.md into a strict, versionable FactoryConfig (goal, scope, guards, eval command/threshold, constraints, budgets, and optional research/adversarial/parallel sub-configs).
Dispatch specialist agent subprocesses (Researcher, Strategist, Builder, Health Checker, Code Reviewer, Adversarial Tester, Archivist, Refiner, Failure Analyst, CEO) through a single factory agent <role> contract that resolves prompts, injects evolved playbooks, captures output, and emits lifecycle events.
Compute a weighted composite eval score across hygiene (6 dimensions), growth (5+ dimensions), and optional project-defined dimensions, and gate keep/revert decisions on that score plus non-overridable guard/precheck results.
Represent every factory mode as a typed, validated directed graph (Workflow) of AgentNode/FnNode/GateNode/ForkNode/JoinNode primitives that a headless WorkflowExecutor can run deterministically.
Render the same Workflow graph into a Claude Code SKILL.md playbook via a verified templatize → review → guard → split pipeline, so interactive and headless execution never diverge.
Persist per-project experiment history (results.tsv, experiments/NNN/), long-term institutional memory (.factory/archive/), and cross-project statistics (~/.factory/registry.json) so future cycles and other projects can learn from past outcomes.
Evolve per-role behavioral playbooks (~/.factory/playbooks/<role>.md) from real keep/revert outcomes via a deterministic Reflect → Curate → Inject (ACE) pipeline, with no LLM required for the reflection step.
Support at least four interchangeable agent-CLI runners (Claude Code, Bob Shell, Codex, OpenCode) behind one Runner protocol, each independently authenticatable and dry-runnable.
Execute the same CLI command surface unmodified on the host, inside a local podman container, or inside an OpenShift/Kubernetes namespace via factory contained, without the CLI parsing or altering the wrapped command's semantics.
Evolve workflow topologies (not just prompts) via MAP-Elites quality-diversity search in the outer loop, evaluating each candidate graph against a real benchmark inner loop.

2.2 Non-Goals

Sandboxing agent-authored code execution. (factory contained documents explicitly that neither local podman nor the OpenShift target confines agent-authored code — both require human PR review, the same as any other agent output.)
Automatically merging pull requests. (Sacred Rule 6 forbids it; every kept experiment's PR stays open for human merge.)
Training, fine-tuning, or hosting language models. (Self-improvement operates on markdown playbooks and workflow-graph JSON, consumed by off-the-shelf agent CLIs — never on model weights.)
Acting as a general-purpose CI/CD system. (It orchestrates experiment cycles on a schedule or on demand via --loop; it does not replace a project's own CI pipeline, and guard checks assume CI-independent local git state.)
Providing a universal, provider-agnostic credential vault. (Credential handling is FACTORY_-prefixed env vars plus ~/.factory/config.toml profiles with an explicit forward-list — there is no secret-manager integration or automatic rotation.)
Guaranteeing agent output quality on every invocation. (The CEO Review Gate, redirect/abort protocol, and consecutive-failure abort exist precisely because agent output is treated as fallible and reviewed, not trusted by default.)

2.3 Design Philosophy

Every irreversible or trust-sensitive decision is pushed to a checkable artifact — a guard command's exit code, an eval score delta, a must_contain string in a review file — rather than to an LLM's self-assessment, because self-assessment is exactly the failure mode the system exists to catch. The same graph definition MUST produce both the deterministic executor's path and the interactive CEO's playbook, so there is one source of truth for "what a mode does" rather than two documents that drift. State that matters (experiment verdicts, playbooks, registry, adversarial phase) is always a plain file under .factory/ or ~/.factory/, because a crash-resilient, resumable orchestrator cannot depend on in-memory state surviving a subprocess boundary.

3. Project Identity

Name: re:factory (PyPI/package name: remote-factory)
Type: CLI tool (agentic software-evolution harness) with an embedded FastAPI web dashboard and an MCP server surface
Language: Python 3.11+
Framework: None for the CLI itself (stdlib argparse); FastAPI + Uvicorn for the dashboard; Pydantic v2 for all domain models
Package Manager: uv (PEP 621 project, hatchling + hatch-vcs build backend)
Entry Point: factory console script → factory.cli:main (dispatches to factory/cli/_main.py:main)

4. Technical Stack

4.1 Dependencies

pydantic>=2.0 — strict (extra="forbid") runtime validation for every domain model in factory/models.py, factory/outer_loop/models.py, and factory/workflow/primitives.py
structlog>=24.0 — structured logging (log = structlog.get_logger()) used at module level across the CLI, agents, and eval subsystems
fastapi>=0.115 + uvicorn[standard]>=0.34 — the live web dashboard (factory dashboard), SSE event streaming
mcp>=1.27.0 — Model Context Protocol server (factory serve-mcp) exposing factory operations as tools to other Claude Code sessions
pyyaml>=6.0 — YAML for skill annotation sidecars (SKILL.annotations.yaml) and ACE playbook front-matter
filelock>=3.0 — advisory file locking for concurrent-safe .factory/ writes (experiment ID allocation, TSV append, adversarial state)
networkx>=3.6.1 — graph algorithms backing the code knowledge graph, workflow graph validation, and outer-loop MAP-Elites structural analysis
langfuse>=3.0 — LLM tracing/observability, wrapped as a graceful no-op when unconfigured (factory/telemetry.py)
anthropic[vertex]>=0.52 — direct Anthropic/Vertex API client used by in-process LLMNode execution and SkillOpt reflection, distinct from the subprocess-based agent runners
mempalace>=3.6.0 — long-term memory integration for the study and archivist phases (factory/mempalace/)
graphifyy>=0.9 — AST-derived code knowledge graph extraction library that produces graph.json

4.2 External Dependencies

claude CLI — default agent runner (Claude Code); required unless another runner is selected (OPTIONAL if --runner overrides it)
bob CLI — Bob Shell runner, requires BOBSHELL_API_KEY (OPTIONAL)
codex CLI — OpenAI Codex runner, requires CODEX_API_KEY/OPENAI_API_KEY (OPTIONAL)
opencode CLI — OpenCode runner, requires a provider credential env var (OPTIONAL)
git — worktree isolation, guard checks, branch/commit management (REQUIRED)
gh / glab — GitHub/GitLab issue and PR operations, plan-issue detection (OPTIONAL, degrades to skipped checks when absent)
podman — local contained runtime target (OPTIONAL, only for factory contained --target local)
oc (OpenShift CLI) — cluster contained runtime target, isolated to a sidecar container (OPTIONAL, only for --target k8s)
pytest / ruff / mypy (or the target project's own tooling) — hygiene eval dimensions, auto-detected per project by factory discover (OPTIONAL, project-dependent)
tmux — detached long-running sessions (factory tmux) and the contained runtime's PID-1-safe process host (OPTIONAL/REQUIRED inside runtime images)

5. Architecture Overview

5.1 Abstraction Levels

Python CLI (factory/) — Deterministic tools that make no judgment calls: state detection, config parsing, eval scoring, guard checks, the experiment store, the global registry, event logging. Dispatched from factory/cli/_main.py:main through a cmd_* handler dictionary keyed by subcommand.
Workflow Graph Engine (factory/workflow/) — All active factory modes (design, create, spec-generate, plus contributed benchmark workflows) are directed graphs of typed nodes (AgentNode, FnNode, GateNode, ForkNode, JoinNode, Study, LLMNode) connected by conditioned Edge objects. One graph definition renders to two execution surfaces: a deterministic WorkflowExecutor (headless) and a generated SKILL.md (interactive).
Outer Loop (factory/outer_loop/) — A MAP-Elites evolutionary search over workflow topologies themselves, running a full CEO cycle (the "inner loop") per candidate to score it, then mutating the graph structure toward higher fitness.
CEO Agent (factory/agents/prompts/ceo.md + skills/workflow-*/SKILL.md) — The orchestrator persona. Cross-cutting rules (Sacred Rules, FEEC, keep/revert framework, review gates) live in ceo.md; mode-specific step sequences live in the generated SKILL.md files that ceo.md reads at runtime.
Specialist Agents (factory/agents/) — Independent Claude Code (or Bob/Codex/OpenCode) subprocesses, one per role, each with a two-tier prompt resolution (project override → user-global → factory default) and an auto-injected evolved playbook.

5.2 Data Flow Summary

A run begins with factory ceo <path>, which resolves the CEO prompt (resolve_prompt, injecting any ACE playbook and the relevant SKILL.md) and spawns it as a subprocess. The CEO first calls factory detect to classify project state, then — following the design workflow's routing — either runs factory discover (new/unconfigured projects) or reads the existing .factory/config.json. It then triggers the study subgraph (factory graph update → factory study → graph-exploring Researcher → concatenated study-combined.md), fans out three parallel Researcher agents behind a fork/join, and passes their output through a CEO gate to the Strategist, which writes a phased plan to .factory/strategy/current.md. A hard gate (agent-evaluated in build, user-evaluated in design) MUST approve the plan before the Builder runs. The Builder implements one phase on an experiment branch and opens a PR; a CEO gate reviews the diff; a deep-QA fork (Health Checker + Code Reviewer + Adversarial Tester) runs in parallel and joins into a QA gate with a bounded reloop-to-builder budget; a doc-freshness gate and a non-overridable precheck gate follow. Every step's inputs and outputs are plain files under .factory/reviews/ and .factory/strategy/, so state survives CEO respawns and is inspectable outside the LLM session. factory begin/factory finalize bracket each experiment, writing verdict.json and appending a row to results.tsv; the Archivist runs asynchronously after each verdict and synchronously at cycle end, updating .factory/archive/ and .factory/performance_report.json for the next ACE reflection pass.

6. Domain Model

All models are Pydantic v2 with ConfigDict(strict=True, extra="forbid") — unrecognized or type-mismatched fields MUST raise a ValidationError rather than silently coerce or drop data. [[graph:factory.models]]

6.1 ProjectState

A str enum with exactly five members: NO_REPO, REPO_INCOMPLETE (value "incomplete"), NO_FACTORY, EVALS_PENDING_REVIEW, HAS_FACTORY. detect_state() MUST return exactly one of these for any filesystem path, and MUST check EVALS_PENDING_REVIEW before HAS_FACTORY so that a project mid-way through the discover → review → init flow (eval profile written, config not yet initialized) is not misclassified as fully configured.

6.2 FactoryConfig

The machine-readable form of a project's factory.md, persisted at .factory/config.json. Required fields: goal: str, scope: list[str], guards: list[str], eval_command: str, eval_threshold: float, constraints: list[str]. Fields with defaults: hypothesis_budget: HypothesisBudget (min_growth=2, max_new=2), target_branch: str = "main", smoke_test: str = "", project_eval: list[ProjectEvalDimension] = [], eval_weights: EvalWeights (hygiene=0.5, growth=0.5, project=0.0), research_target: ResearchTarget | None = None, inner_loop/outer_loop: ... | None = None, mutable_surfaces/fixed_surfaces/research_constraints: list[str] = [], cost_budget: CostBudgetConfig | None = None, hard_constraints: list[HardConstraint] = [], eval_spec: list[str] = [], hygiene_weights/growth_weights: TierWeights | None = None, adversarial: AdversarialConfig | None = None, parallel: ParallelConfig | None = None, clean_pr: bool = False (+clean_pr_include/clean_pr_exclude: list[str] = []), test_timeout: int = 600 (constrained ge=1). scope and fixed_surfaces MUST be interpreted as fnmatch-style glob patterns (with ** recursive support) by every guard check that consumes them; a changed file matching no scope pattern MUST be treated as a scope violation, and a changed file matching any fixed_surfaces pattern MUST be treated as a fixed-surface violation regardless of scope membership. [[graph:factory.models.FactoryConfig]]

Nested configuration models: HypothesisBudget (backlog-first hypothesis quotas), HardConstraint (name + shell check command + description — a non-zero exit is a mandatory revert the CEO cannot override), ProjectEvalDimension (name, command, parse: "json"|"exit_code", weight, timeout, description — a user-defined project-specific eval), EvalWeights (hygiene/growth/project tier split), ResearchTarget (objective, metric, target value, run command, result path/parser, timeout), AggregateMethod enum (mean/median/max/all_pass), InnerLoopConfig (multi-run-per-cycle aggregation and plateau detection), OuterLoopConfig (max outer cycles, inner/outer mutable-surface lists), CostBudgetConfig (per-cycle/total USD caps), TierWeights (sparse per-dimension weight overrides within a tier — unset fields keep the default), ParallelConfig (parallel_hypotheses: int in [1,8], selection_strategy: "best_score"), and the adversarial trio described in §6.4.

6.3 Eval Domain: EvalResult, CompositeScore, EvalDimension, EvalProfile

EvalResult (name, score, weight, passed, details) is the atomic unit every eval dimension — hygiene, growth, or project-defined — MUST produce. CompositeScore (total, results, guard_violations, passed) is produced by compute_composite(): if results is empty, total MUST be 0.0 and passed MUST require both zero guard violations and 0.0 >= threshold; otherwise weights MUST be renormalized to sum to 1.0 before the weighted sum is taken, and passed MUST require both zero guard violations and total >= threshold. EvalDimension (name, command, weight, parser: "exit_code"|"json"|"regex", optional regex_pattern, description, source: "explicit"|"discovered"|"researched"|"fallback") is the discovery-time representation; EvalProfile (project_type, dimensions, tier, confidence, human_reviewed: bool = False) gates the EVALS_PENDING_REVIEW state until a human (or --auto-approve) flips human_reviewed to True.

6.4 Adversarial Domain

AdversarialComponent (role: "generator"|"discriminator", eval_command, metric_name, threshold, scope, timeout) describes one side of a GAN-style loop; AdversarialConfig pairs a generator and discriminator with hysteresis: int = 3, optional max_rounds, and convergence_window: int = 5. AdversarialState (persisted at .factory/adversarial_state.json) tracks active_role, current_round, per-role consecutive-above-threshold streaks, a converged: bool, and a bounded history: list[AdversarialPhaseRecord]. Convergence MUST require both generator_consecutive_above and discriminator_consecutive_above to independently reach convergence_window — a single side sustaining performance MUST NOT be sufficient.

6.5 Experiment and Cross-Project Domain

ExperimentRecord (id, timestamp, hypothesis, change_summary, issue_number, pr_number, score_before/after, delta, verdict: "keep"|"revert"|"error"|"superseded", cost_usd, notes, research_citations) is the atomic unit of results.tsv and experiments/NNN/verdict.json; delta, when unset, MUST be derived as round(score_after - score_before, 6) at finalize time. CrossProjectInsights (projects, outcomes, category_stats, winning/losing categories, patterns, generated_at) aggregates HypothesisOutcome, ProjectSummary, and Pattern records across every registered project for factory insights. SessionSummary captures an end-of-cycle rollup (kept/reverted/errored experiments, remaining backlog, guard violations, items needing human input, score/cost deltas).

6.6 ACE Pipeline and Registry Domain

AgentVerdict (role, verdict: "PROCEED"|"REDIRECT"|"ABORT", rationale, issues, experiment_id) is the parsed form of a ceo-verdict-*.md file. Observation (source, content, timestamp, project, tags) and PerformanceReport (per-project verdict/observation rollup plus verdict_patterns) feed the ACE reflector. ProjectEntry/ProjectRegistry back ~/.factory/registry.json — a project is registered idempotently by resolved absolute path, and stats (experiment_count, latest_score, last_experiment_at) are updated on every finalize(). CycleState (persisted at .factory/state/cycle.json) preserves mode, initial_prompt, respawns, and the runner/session identity across CEO respawns within one logical cycle. RefinementEntry/RefinementState track sequential post-cycle refinement requests.

6.7 Runner and Workflow Domain

AgentRunRequest (prompt, prompt_core, task, cwd, timeout, model, skip_permissions: bool = True, role, session identifiers, project_path, extras) and AgentRunResult (stdout, return_code, usage, metadata) form the Runner protocol's structured I/O contract — every runner implementation (claude, bob, codex, opencode) MUST accept the former and return the latter. Workflow-graph primitives (Node and its subtypes AgentNode, FnNode, GateNode, ForkNode, JoinNode, SubgraphForkNode, SelectionNode, Study, LLMNode; Edge; Workflow; Verdict/VerdictType) are detailed behaviorally in §7.3 and §8.2. Outer-loop domain models (SwarmConfig, Individual, MutationRecord/MutationType, HyperparameterRecord, GenerationSummary, outer-loop EvalResult, AuditResult, OuterLoopResult, OuterLoopState) are detailed in §8.3. [[graph:factory.outer_loop.models]]

7. State Machines and Lifecycles

7.1 Project State Detection

                 path missing / no .git
        ┌──────────────────────────────────► NO_REPO
        │
        │  eval_profile.json exists AND
        │  human_reviewed == False
        ├──────────────────────────────────► EVALS_PENDING_REVIEW
        │
        │  .factory/config.json exists
        ├──────────────────────────────────► HAS_FACTORY
        │
        │  open GitHub issue labeled "plan"
        ├──────────────────────────────────► REPO_INCOMPLETE
        │
        └──────────────────────────────────► NO_FACTORY

detect_state() MUST evaluate these checks in exactly this order (missing repo → pending eval review → has factory → open plan issues → fallback), because the pending-review check must fire before the has-factory check to correctly classify projects mid-way through discover → review → init. Only a GitHub issue labeled exactly "plan" MUST be treated as evidence of an unbuilt repo; an issue labeled "implementation" is the factory's own Improve-mode backlog convention on already-built repos and MUST NOT be conflated with it. gh CLI failures or timeouts MUST be treated as "no open plan issues" rather than propagated as errors.

7.2 Experiment Lifecycle

factory begin(hypothesis) ──► experiments/NNN/hypothesis.md created, project registered
        │
        ▼
   [Builder implements on experiment branch, opens PR]
        │
        ▼
   [deep-QA: health_checker + code_reviewer + adversarial_tester]
        │
        ▼
   [guard checks + precheck: score direction, scope, fixed surfaces,
    anti-pattern similarity, hard constraints, QA-execution proof]
        │
        ├── all pass ───────────────► factory finalize(verdict="keep")
        ├── score regressed/guard violated ─► factory finalize(verdict="revert")
        └── agent crash / eval crash ─────► factory finalize(verdict="error")

ExperimentStore.begin() MUST be idempotent: re-invoking it for an experiment directory that already exists (e.g. after an interrupted run) MUST return the existing ID rather than raising. ExperimentStore.finalize() MUST recreate the experiment directory if it was removed (e.g. by git clean) before writing verdict.json. A superseded verdict MUST be used only when a later experiment supersedes an earlier one's hypothesis, never as a substitute for revert.

7.3 Workflow Graph Execution

Every workflow is a Workflow(name, nodes, edges, start_node, trigger). Execution MUST proceed node-by-node from start_node, following Edges whose condition (if set) matches the most recent GateNode's Verdict.type. GateNode.evaluator_type MUST be one of "agent" (an LLM, typically the CEO, judges and emits PROCEED/RELOOP/HALT), "fn" (a shell command's stdout/exit code is parsed programmatically), or "user" (execution MUST block for human input and MUST NOT be auto-approved unless the caller explicitly passed an auto-approve flag). A RELOOP verdict MUST carry a target node id and MUST be bounded by max_iterations (default 3) — an executor or CEO exceeding that bound MUST escalate to HALT rather than loop indefinitely. ForkNode.targets MUST all begin execution before the corresponding JoinNode.sources barrier is considered satisfied; a JoinNode MUST NOT proceed until every listed source has completed. An AgentNode with non-empty post_checks (ArtifactCheck: must_exist, min_size, must_contain) MUST have its declared writes path(s) validated against every check before the node is considered successfully completed; a failed check MUST be treated as node failure, not silently ignored.

7.4 Adversarial Eval Loop Phase Machine

        ┌────────────┐  consecutive_above ≥ hysteresis   ┌────────────────┐
   ───► │  generator  │ ─────────────────────────────────►│ discriminator  │
        │   active    │ ◄─────────────────────────────────│    active      │
        └────────────┘  consecutive_above ≥ hysteresis    └────────────────┘
              │                                                    │
              └──────────── both per-role streaks ≥ convergence_window ─────► converged = True

The active role MUST switch when its consecutive_above streak reaches config.hysteresis; a switch MUST reset the streak counters for the newly active role's tracking. detect_convergence() MUST require generator_consecutive_above >= convergence_window AND discriminator_consecutive_above >= convergence_window simultaneously — convergence based on only one side's streak MUST NOT be reported. A corrupt or unreadable adversarial_state.json MUST be treated as "no state" (fresh AdversarialState()) rather than propagated as a fatal error.

7.5 CEO Cycle and Crash Recovery

CycleState at .factory/state/cycle.json records cycle_id, started_at, mode, respawns, and runner/session identity. The CEO completion guard (factory/ceo_completion.py) MUST detect premature exit (the CEO process terminating before all planned work is recorded as complete) and MUST re-spawn the CEO with a continuation task rather than silently ending the cycle. A cycle.json older than CYCLE_STALENESS_HOURS (24) MUST be treated as stale and MUST NOT trigger an auto-resume. Respawns MUST be capped at DEFAULT_MAX_RESPAWNS (5, env-overridable) per cycle to prevent an infinite respawn loop from a systematically-crashing CEO.

8. Module Specifications

8.1 `factory/cli`

Role: The single dispatch point for every factory <command> invocation. factory/cli/_main.py:main() builds the argparse parser, loads .env.local, and routes to one of ~60 cmd_* handlers via a flat dictionary keyed by subcommand string (with nested dispatch for spec, graph, workflow, and outer-loop subcommands). [[graph:factory.cli]]

The dispatcher MUST catch any exception raised by a handler, print Error: {e} to stderr, and return a non-zero exit code rather than propagating a traceback to the caller's shell.
Handlers under factory/cli/*.py (agents.py, ceo.py, run.py, contained*.py, outer_loop.py, etc.) MUST NOT themselves perform business logic beyond argument marshaling — they MUST delegate to library functions in factory/store.py, factory/eval/, factory/agents/runner.py, and peers, so that the same logic is reachable both from the CLI and from tests without a subprocess.
Adding a new top-level subcommand requires both a parser entry (factory/cli/_parser_groups.py or _main.py) and a handlers dict entry; a subcommand present in one but not the other MUST fail closed (Unknown command) rather than silently no-op.

8.2 `factory/workflow`

Role: The typed graph engine — primitives.py (node/edge/verdict types; Workflow now carries knob_values, knob_bounds, knob_expandable for optimizer-tunable parameters), package.py (composable Package primitive with Port, StateContract, OptKnob, MemoryDeclaration; composition operators Sequential, Parallel, Conditional, Loop; compile() lowers to flat Workflow IR), definitions.py (the four registered graphs: design, create, spec-generate, plus reusable subgraph helpers _study_subgraph, _deep_qa_subgraph, _research_subgraph), executor.py (headless WorkflowExecutor), skill_export.py (graph → SKILL.md renderer), and validation.py (NetworkX-backed structural checks). [[graph:factory.workflow]]

build_workflow() is the base graph (fork 3 researchers → join → CEO gate → strategist → CEO hard gate → async archivist → builder → CEO gate → deep-QA fork/join → QA gate (max 3 reloops) → doc-freshness gate → precheck gate → async archivist → non-blocking spec-generate); design_workflow() MUST be built as build_workflow() plus a gate_has_factory conditional entry (existing projects route through the study subgraph, new/partial projects route through factory discover first) and MUST replace the agent-evaluated gate_strategy with a user-evaluated one.
_get_builtin_registry() MUST be treated as the sole authority on which workflow names are constructible; register_all() MUST invoke every entry in that registry and MUST NOT hardcode a parallel list that can drift from it.
Workflow.validate_graph() MUST be run (via factory workflow validate) before a newly authored or edited workflow is trusted — a graph with unreachable nodes, dangling edges, or a fork without a matching join is a defect the executor cannot recover from at run time.
The templatize → review → guard → split SKILL.md pipeline MUST be re-run (factory workflow export-skills) after any change to definitions.py; a regression test (test_annotations_match_source) MUST fail CI if the exported SKILL.md/SKILL.annotations.yaml drift from the source graph.

8.3 `factory/outer_loop`

Role: MAP-Elites evolutionary search over workflow topologies and knob values. engine.py's SwarmEngine orchestrates seeding, tournament selection, mutations.py's eight structured operators (NODE_INSERT, NODE_REMOVE, EDGE_REDIRECT, PARALLELIZE, SERIALIZE, PARAM_MUTATE, PROMPT_MUTATE, KNOB_MUTATE), and convergence detection (plateau, diversity collapse, early stop). KNOB_MUTATE searches OptKnob values on compiled Workflow IR; when bounds are exhausted and the knob is expandable, default_knob_expander (Opus via CLI) invents new values at runtime. evaluator.py's SwarmEvaluator runs a full CEO cycle per candidate as the "inner loop," with FitnessCache/CycleRecordCache deduplicating by structural/content hash. population.py's Population and MAPElitesArchive maintain a quality-diversity grid (base: depth × fork_degree × agent_count × gate_count; extended with hashed knob values when present). mode_registry.py's EphemeralModeRegistry registers candidates as temporary modes (evolve-gen{N}-{id[:8]}) with content-hash integrity checking. [[graph:factory.outer_loop]]

A candidate graph MUST be scored via SwarmEvaluator, which MUST cache by structural hash so that two candidates that are graph-isomorphic MUST NOT be re-evaluated at full cost.
mode_registry.promote() MUST verify the ephemeral mode's content hash before promoting it to a permanent named mode — a hash mismatch (indicating the mode file was mutated after registration) MUST block promotion.
overfit.py's OverfitDetector comparing training vs. holdout scores SHOULD gate any claim that an evolved topology generalizes; a topology that wins only on training instances MUST be flagged, not silently promoted.

8.4 `factory/agents`

Role: Prompt resolution and subprocess invocation for every specialist role. runner.py's resolve_prompt() implements the three-tier lookup (project override → user-global → factory default) with automatic ACE playbook injection and, for the CEO role, automatic SKILL.md injection keyed by workflow_mode. invoke_agent() is the synchronous, blocking call every CEO invocation of factory agent <role> ultimately runs through. [[graph:factory.agents.runner]]

resolve_prompt() MUST check <project>/.factory/agents/<role>.md before ~/.factory/agents/prompts/<role>.md before factory/agents/prompts/<role>.md; whichever tier resolves MUST still receive the evolved-playbook injection — an override MUST NOT opt a role out of ACE learning.
Consecutive agent-spawn failures MUST be counted; reaching _FAILURE_ABORT_THRESHOLD (2) MUST raise ConsecutiveAgentFailureError rather than let the CEO fall back to doing the specialist's work itself (Sacred Rule 8).
Every invocation MUST write its captured stdout to .factory/reviews/<role>-latest.md (or a --review-tag-suffixed variant for parallel invocations) and MUST emit agent.started/agent.completed/agent.failed/agent.timeout events — a caller that bypasses invoke_agent (e.g. the forbidden native Agent tool) breaks both the review-capture and telemetry contracts.

8.5 `factory/eval`

Role: Composite scoring. runner.py's run_eval() computes 6 mandatory hygiene dimensions (hygiene.py) and 5+ mandatory growth dimensions (growth.py) unconditionally, optionally runs the project's own eval/score.py as additive hygiene dimensions and any factory.md-declared ProjectEvalDimensions, then merges all tiers via _merge_all() and scores via scorer.py's compute_composite(). guards.py implements the non-negotiable safety checks. [[graph:factory.eval]]

Weight distribution MUST default to 50% hygiene / 50% growth when no custom project eval exists, and MUST auto-redistribute to 30%/20%/50% (hygiene/growth/project) when project eval dimensions are present and the user has not set explicit eval_weights — an explicit eval_weights.project > 0 MUST override the auto-split by proportional renormalization instead.
TierWeights overrides MUST be applied as a sparse dict (only non-None fields) before within-tier normalization, so an unset override MUST fall back to the dimension's discovered/default weight rather than zero.
check_scope/check_fixed_surfaces MUST use the same glob-matching semantics (_glob_match, supporting **) so that a pattern authored for scope and reused for fixed_surfaces behaves identically; both MUST ignore a fixed allowlist of auto-generated lock files (uv.lock, package-lock.json, etc.) so routine dependency-resolution side effects never trigger a guard violation.
run_eval() MUST write the resulting CompositeScore to .factory/last_eval.json whenever the .factory directory exists, for dashboard consumption — this write MUST be best-effort (an OSError MUST be swallowed, not propagated).

8.6 `factory/discovery`

Role: First-run project introspection. introspect.py detects language, framework, test/lint/type-check commands, and CI presence by pattern-matching known project files; profile.py's build_eval_profile() converts an introspected ProjectProfile into an EvalProfile; generate.py writes a runnable eval/score.py from that profile; eval_spec.py auto-promotes free-text spec bullet points into executable project-eval dimensions where possible. [[graph:factory.discovery]]

A newly generated EvalProfile MUST be written with human_reviewed=False; only an explicit human review step (or --auto-approve) MUST flip it, per §7.1's EVALS_PENDING_REVIEW gate.
classify_eval_spec_item() MUST label each spec bullet as "executable" (can become a real command) or "judgmental" (requires an LLM judge) — only "executable" items MUST be auto-promoted to ProjectEvalDimensions; judgmental items MUST remain advisory text for the Strategist/QA agents.
The generated eval/score.py is itself a fixed surface for the running experiment: guards.check_eval_immutable() compares a git ls-tree snapshot of eval/ taken before the change against one taken after, and any diff MUST be reported as a guard violation.

8.7 `factory/store`

Role: The .factory/ filesystem contract. ExperimentStore owns init(), reparse_config() (factory.md → FactoryConfig), begin()/finalize() (experiment lifecycle), load_history() (TSV → ExperimentRecord list), and eval-profile/strategy read/write helpers. [[graph:factory.store.ExperimentStore]]

reparse_config()'s markdown parser MUST treat any #/##/### heading as a section boundary, MUST map known heading aliases (e.g. "command" → eval_command, "modifiable" → scope) via section_map, and MUST treat indented continuation lines under a - list item as appended (newline-joined) content of that item rather than a new item — this is the mechanism _parse_project_eval/_parse_hard_constraints rely on for multi-line name:/command:/check: blocks.
begin() and finalize() MUST hold self._lock (a FileLock on .store.lock) for their filesystem-mutating sections, because concurrent parallel-hypothesis experiments MUST NOT race on next-ID allocation or TSV append.
read_config() MUST raise FileNotFoundError with a factory init-pointing message when config.json is absent, and MUST raise ValueError with a factory init --reparse-pointing message on malformed JSON or a Pydantic ValidationError — a caller MUST NOT be left to guess the remediation step from a bare traceback.

8.8 `factory/strategy`

Role: The FEEC (Fix > Exploit > Explore > Combine) priority heuristic and 3-tier experiment-history compression. categorize_hypothesis() classifies free text by keyword matching (FIX keywords checked first, then EXPLOIT, then COMBINE, with EXPLORE as the uncategorized default); find_anti_patterns() flags a proposed hypothesis whose Jaccard token similarity to a reverted past hypothesis exceeds a threshold (default 0.6); format_tiered_history() renders the last 3 experiments in full detail, the next 7 as one-liners, and everything older as aggregate stats only. [[graph:factory.strategy]]

detect_research_plateau() MUST require at least threshold + 1 run summaries before declaring a plateau, and MUST compare the best metric value in the most recent threshold-sized window against the best value in everything before that window — a plateau MUST NOT be declared from an under-populated history.
find_anti_patterns() MUST only consider history entries with verdict == "revert"; a keep or error entry with high textual similarity MUST NOT be flagged, since re-attempting a kept idea is not an anti-pattern.

8.9 `factory/state`

Role: detect_state(), the sole implementation of §7.1's state machine. It MUST be the only code path the CLI and CEO use to classify a project — any mode-routing logic that re-derives project state by other means (e.g. only checking for .git) risks disagreeing with factory detect's output. [[graph:factory.state]]

8.10 `factory/registry`

Role: The global ~/.factory/registry.json project index. register_project() MUST be idempotent by resolved absolute path (re-registering an already-known path MUST be a no-op, not a duplicate entry); update_project_stats() MUST silently log-and-skip (not raise) when asked to update a path not present in the registry, since ExperimentStore.finalize() calls it best-effort. discover_projects() MUST identify a factory-managed directory solely by the presence of .factory/results.tsv, independent of the registry file. [[graph:factory.registry]]

8.11 `factory/adversarial`

Role: Persistence and phase logic for the GAN-style adversarial eval loop described in §7.4. load_adversarial_state() MUST return a fresh default AdversarialState on any parse failure (JSON error, type error, validation error) rather than propagate — a corrupt state file MUST NOT block a research cycle. [[graph:factory.adversarial]]

8.12 `factory/user_config`

Role: Layered global configuration (~/.factory/config.toml) and credential profiles. resolve() implements a five-tier precedence: CLI flag > env var > profile credential > config.toml default > hardcoded default. load_config(profile=...) applies a named [credentials.<name>] section by overriding os.environ (not setdefault) and processes an optional [credentials.<name>.unset] vars list before applying sets. [[graph:factory.user_config]]

_PROTECTED_VARS (shell fundamentals like PATH/HOME; code-execution vectors like LD_PRELOAD; language path vars like PYTHONPATH; factory internals like FACTORY_TRACE_ID) MUST NOT be settable or unsettable via any profile — load_config() MUST raise ValueError if a profile attempts either.
[credentials.<name>.unset].vars MUST be validated as a list; a string or other non-list value MUST raise ValueError rather than being silently iterated character-by-character.
Overriding an already-set env var with a different value via a profile MUST emit log.warning("profile_override", key=k, profile=profile) — the value itself MUST NOT be logged, to avoid leaking secrets into structured logs.
show_config() MUST mask sensitive-looking values (per is_sensitive()) unless --reveal is explicitly passed.

8.13 `factory/runners`

Role: The Runner protocol (protocol.py) and its four implementations (claude.py, bob.py, codex.py, opencode.py), each described by a RunnerMeta (binary name, required env vars, capability flags) and each implementing build_command()/headless()/interactive_run(). [[graph:factory.runners]]

RunnerMeta.check_auth() MUST use a supplied custom_auth_check callable when present (e.g. Bob's file-based auth) and MUST otherwise fall back to checking that every required_env_vars entry is a non-empty environment variable.
A runner that does not support a requested capability (e.g. OpenCode's lack of --bg/session events) MUST fail with an explicit, named error at invocation time — it MUST NOT silently degrade to a different behavior that the caller did not request.
Dry-run modes (FACTORY_BOB_DRY_RUN, FACTORY_CODEX_DRY_RUN, FACTORY_OPENCODE_DRY_RUN) MUST return stub AgentRunResults and MUST still log usage, so tests exercising the CEO loop do not require live API credentials.

8.14 `factory/contained`

Role: Running any factory <command> inside a podman container or an OpenShift/Kubernetes namespace via factory contained [flags] -- <command>, with path rewriting only — the wrapped command's semantics are never parsed or altered. Submodules: provenance.py (five pre-flight assertions that the workspace derives from the live working tree, not HEAD), identity.py (UID/ownership probing so a bind mount is never silently read-only), credentials.py (FACTORY_-prefixed-plus---forward-named env policy, redacted wherever printed), k8s.py/k8s_review.py/k8s_setup.py (cluster object diffing, walk-and-apply, namespace/context selection), division.py/k8s_division.py (opt-in build-tooling access, isolated to an unauthenticated local MCP server or a sidecar oc-only container), style.py (cbreak-mode prompt UI), claude_state.py (pre-recorded answers to Claude Code's interactive-only onboarding prompts). [[graph:factory.contained]]

verify (both local and k8s) MUST report credential shape only — it MUST NOT print secret material — and MUST redact any secret-looking value in any command it prints.
On the cluster target, the sidecar container that holds oc and the ServiceAccount token MUST run a distinct image (FACTORY_CONTAINED_SIDECAR_IMAGE) from the agent's own image, and its Role MUST exclude pods/exec; verify MUST assert this via a SubjectAccessReview API object rather than oc auth can-i --as, because the latter collapses pods/exec onto pods and reports a false "yes."
Each cluster object accepted during the interactive review walk MUST be applied at the moment of acceptance, never batched — a WalkResult MUST record exactly what was applied so an aborted walk's message can state precisely how much survives.
Neither the local nor the cluster target MUST be represented anywhere in user-facing output as a security sandbox for agent-authored code; --help text MUST state the boundary and stop there.

8.15 `factory/ace`

Role: Autonomous Context Engineering — the deterministic (no-LLM) Reflect → Curate → Inject pipeline that evolves per-role playbooks from experiment outcomes. reflector.py parses ceo:keep/ceo:revert notes and agent-failure patterns across all managed projects into candidate bullets; curator.py deduplicates (SequenceMatcher, threshold 0.75), prunes net-negative bullets (harmful count exceeding helpful count with sufficient observations), and caps capacity by net score; injector.py resolves and appends the user-local evolved playbook (~/.factory/playbooks/<role>.md) over the factory default (factory/agents/playbooks/<role>.md) into a role's resolved prompt. [[graph:factory.ace]]

Reflection MUST be pure pattern extraction over structured data (parsed notes fields, event logs) — it MUST NOT require an LLM call, so it can run cheaply and deterministically on every ACE cycle.
Curation MUST apply net-negative removal before deduplication before capacity capping, in that order, since removing clearly-harmful bullets first avoids wasting the similarity pass on rules that will be pruned anyway.

8.16 `factory/precheck` and `factory/ceo_completion`

Role: The two non-overridable gates in the system. precheck.py's run_precheck() aggregates: score-direction (no regression, meets threshold), scope guard, fixed-surface guard, anti-pattern similarity, user-defined hard constraints, and — when an exp_id is supplied — proof that QA actually ran (check_qa_execution, matching either legacy monolithic qa events or the new health_checker/code_reviewer/adversarial_tester events after the experiment's experiment.begin timestamp). ceo_completion.py implements the auto-resume behavior of §7.5. [[graph:factory.precheck]]

A single failing check in run_precheck() MUST fail the whole aggregate (passed = len(failures) == 0) — there is no partial-credit precheck pass.
check_qa_execution() MUST default to passed=True when no matching experiment.begin event exists for the given exp_id (nothing to verify against) rather than failing closed on missing telemetry, but MUST fail closed (passed=False, "Sacred Rule 9 violation") when a begin event exists and no subsequent QA-completion event is found before finalize.

8.17 Observability and Cross-Project Learning: `factory/events`, `factory/checkpoint`, `factory/report`, `factory/insights`, `factory/study`, `factory/digest`, `factory/mempalace`, `factory/dashboard`, `factory/notify`

Role: events.py is the append-only JSONL event log at .factory/events.jsonl, written by agents/runner.py and the heartbeat loop; checkpoint.py saves/restores CEO state for crash-resilient resume distinct from the lighter-weight CycleState; report.py's generate_performance_report() consolidates verdicts/observations into .factory/performance_report.json, the ACE reflector's primary input; insights.py computes CrossProjectInsights across every project the registry knows about; study.py mines prior interaction logs for hypothesis-relevant context; digest.py summarizes factory activity from the Obsidian/MemPalace vault; mempalace/ wraps the optional MemPalace long-term-memory integration behind graceful ImportError degradation (mempalace/helpers.py is documented as the only file permitted to import mempalace.*); dashboard/app.py is the FastAPI live web UI with SSE event streaming; notify/ implements the Notifier protocol (currently TelegramNotifier) for out-of-band cycle notifications. [[graph:factory.events]] [[graph:community:observability]]

Every agent invocation MUST emit agent.started and exactly one terminal event (agent.completed, agent.failed, or agent.timeout) to .factory/events.jsonl — a caller reading this log to reconstruct cycle history MUST be able to assume start/terminal pairing holds.
mempalace/helpers.py's import-isolation convention MUST be preserved by any new mempalace-touching code — routing a new mempalace.* import through a different module would defeat the single-point graceful-degradation guarantee.
The dashboard MUST treat .factory/last_eval.json and .factory/events.jsonl as read-only, best-effort inputs — a missing or malformed file MUST degrade the relevant UI panel, not crash the server.

8.18 `factory/graph` and the code knowledge graph CLI (`factory/cli/graph.py`)

Role: Wraps the graphifyy library to extract (factory graph extract), incrementally update (factory graph update), and query (status/query/explain/path) an AST-derived code knowledge graph persisted at graph.json in the project root (deliberately outside .factory/, since it is a build artifact of the source tree, not experiment state). [[graph:factory.graph]]

graph.json MUST be a NetworkX node-link JSON document (directed, multigraph, nodes, links keys) tagged with built_at_commit; any consumer (the Researcher's graph-exploration step, factory spec generate) MUST treat a missing graph.json as "no graph available" and fall back to direct file exploration rather than failing.
factory graph update MUST be incremental where the underlying graphifyy extraction supports it — a full re-extraction on every study cycle would be prohibitively expensive on large repositories.

8.19 `factory/spec`

Role: Behavioral-spec generation and maintenance for a target repository — the same subsystem that produces this document. generate.py orchestrates graphify extraction plus a single Spec Annotator agent invocation (the spec-generate workflow in §8.2); ops.py provides validate/scope/update/impact operations, several of which shell out to further agent calls; apply_diff.py applies a structured "SPEC Diff" produced by the Strategist back onto SPEC.md. [[graph:factory.spec]]

factory spec validate MUST run only after the annotation step has been CEO-approved (per the spec-generate workflow's gate_annotate) — validating an unapproved draft wastes the check's diagnostic value.
A generated SPEC.md MUST NOT contain scoring tables, coupling metrics, or change-impact tables (the Entry Points table in §11 is the sole permitted exception) — relationship detail belongs in [[graph:...]] references, not inline tables.

8.20 `factory/skillopt`

Role: A benchmark-driven SKILL.md optimization loop, structurally parallel to but independent from the ACE playbook pipeline — it mutates the rendered SKILL.md prose itself (via skill.py's structured Edit/Patch application) based on rollout failures (failure_tracker.py), LLM-ranked candidate edits (clip.py), hierarchical patch merging across minibatches (aggregate.py), and a validation gate.py that accepts or rejects a candidate skill by comparing benchmark scores. adapter.py defines the abstract per-benchmark environment interface new benchmarks MUST implement to plug into this loop. [[graph:factory.skillopt]]

gate.py's accept/reject decision MUST be based on a measured benchmark score comparison between the candidate and incumbent SKILL.md, mirroring the eval-driven keep/revert philosophy applied at the workflow-prose level rather than the code level.

8.21 Isolation Primitives: `factory/worktree` and `factory/podman`

Role: worktree.py manages git worktree lifecycle for experiment isolation (used by parallel-hypothesis execution and outer-loop candidate evaluation, each getting an independent working tree rooted at the same base commit); podman.py is the single file that composes (never executes ad hoc) every podman CLI invocation used by the local contained runtime, so that FACTORY_CONTAINED_DRY_RUN=1 can print the exact argv the real path would run. [[graph:factory.worktree]] [[graph:factory.podman]]

All podman-specific knowledge MUST live in podman.py, and all OpenShift/Kubernetes-specific knowledge MUST live in factory/contained/k8s.py — no other module MUST shell out to podman or oc directly, so that dry-run composition and the live path can never drift.

8.22 Research-Mode Support: `factory/inner_loop`, `factory/cycle_analyzer`, `factory/research/`, `factory/baseline`, `factory/precheck` (research-specific checks)

Role: inner_loop.py's InnerLoop is a model-like wrapper pairing a mode with an evaluator for outer-loop optimizer consumption; cycle_analyzer.py's CycleAnalyzer reconstructs a structured, mode-agnostic record of what happened in one inner-loop cycle (agents run, order, outputs, evaluator verdict) purely by reading .factory/ artifacts after the fact; research/leakage.py guards against ground-truth content leaking into hypotheses or code (direct file access or indirect content hints) during research-mode runs; research/runner.py executes research run commands and parses results per ResearchTarget; baseline.py fetches stored eval baselines from a project's eval-data branch for trend comparison. [[graph:factory.inner_loop]] [[graph:factory.research]]

CycleAnalyzer MUST derive its record solely from persisted .factory/ artifacts (events, reviews, verdict files) — it MUST NOT depend on in-memory state from the CEO process that produced them, since outer-loop evaluation may run the analyzer against a cycle from a different process entirely.
Leakage checks MUST run before a research-mode hypothesis or diff is accepted; a detected leak MUST block the experiment regardless of its eval score, since a leaked ground truth invalidates the measurement itself.

9. Shared Contracts

9.1 The `.factory/` Directory Contract

.factory/ is the single shared filesystem namespace between the CLI, every specialist agent, and the CEO. Any producer writing under .factory/reviews/, .factory/strategy/, .factory/archive/, or .factory/experiments/ MUST use the exact relative paths declared in the consuming workflow node's reads/writes sets (§8.2), because the executor's artifact-validation hooks and the CEO's manual Read-then-review protocol both key off those literal paths. A project MUST add .factory/ to its .gitignore, since it contains per-run experiment data, usage logs, and potentially sensitive auth files (e.g. .factory/.bob_auth) that MUST NOT be committed. [[graph:path:factory.store:factory.agents.runner]]

9.2 The Agent Review Contract

Every factory agent <role> invocation MUST write its captured stdout to .factory/reviews/<role>-latest.md (or a --review-tag-qualified variant), and the CEO's review of that output MUST be written to .factory/reviews/ceo-verdict-<role>.md in the Verdict / Rationale / Issues found / Instructions for next step structure the CEO prompt defines. A downstream node that reads a review file MUST treat its absence as a hard failure of the preceding step, not as "nothing to review."

9.3 The Events Contract

.factory/events.jsonl MUST be append-only, one JSON object per line, each carrying at minimum a type and timestamp field in ISO-8601 form. Consumers (ceo_completion.py's staleness checks, precheck.py's check_qa_execution, the dashboard's SSE stream) MUST tolerate unknown event types and MUST NOT assume a fixed total ordering across concurrently-written events from parallel agent invocations beyond timestamp comparison.

9.4 The `factory.md` → `FactoryConfig` Contract

factory.md is the only human-authored config surface; .factory/config.json MUST always be treated as a derived, regeneratable artifact of it (factory init --reparse). A heading in factory.md not present in store.py's section_map MUST still be captured under its literal lower-snake-case heading name, so that future config fields can be added by extending FactoryConfig and the parser together without a breaking migration of already-authored factory.md files.

9.5 The Workflow-Graph-to-SKILL.md Contract

A Workflow object is the sole source of truth for a mode's behavior; the corresponding skills/workflow-<name>/SKILL.md and SKILL.annotations.yaml MUST be regenerated (factory workflow export-skills), never hand-edited, whenever the graph changes. The CI regression test comparing annotations to source MUST be treated as a correctness gate on this contract, not an optional lint.

10. Configuration Specification

10.1 Configuration Sources and Precedence

Two independent configuration surfaces exist and MUST NOT be conflated:

Global factory configuration (~/.factory/config.toml, FACTORY_* env vars, --profile credential sections): five-tier precedence — CLI flag > env var > profile credential (when --profile is passed) > config.toml default > hardcoded default. A --profile selection MUST override pre-existing shell env vars for the keys it sets (explicit opt-in is authoritative), and MUST apply its unset.vars list before applying its sets.
Per-project factory configuration (factory.md → .factory/config.json): authored once by the Strategist/Builder or a human, parsed into FactoryConfig by ExperimentStore.reparse_config(), and persisted as JSON. There is no environment-variable override layer for this surface — a change MUST go through editing factory.md and re-running factory init --reparse.

10.2 Core Config Fields

FactoryConfig's required fields (goal, scope, guards, eval_command, eval_threshold, constraints) MUST be present for .factory/config.json to validate; every other field is optional with the default documented in §6.2. eval_threshold MUST be a float compared directly against the normalized composite score in compute_composite(). test_timeout MUST be clamped to a minimum of 1; a non-numeric or zero/negative value in factory.md MUST fall back to 600 rather than raise a parse error, since a malformed timeout SHOULD NOT block the rest of config parsing.

10.3 Validation and Error Surface

FactoryConfig uses ConfigDict(strict=True, extra="forbid"), so any unrecognized key or type mismatch present in .factory/config.json MUST raise a ValidationError when read back via ExperimentStore.read_config() — except that model_validate(data, strict=False) is deliberately used at read time to permit enum-string coercion (e.g. AggregateMethod values arriving as plain strings from JSON). A read_config() failure (missing file, invalid JSON, or failed validation) MUST raise an exception carrying an explicit remediation command (factory init or factory init --reparse) in its message, per §8.7.

11. Entry Points

Type	Module	Detail
CLI	`factory.cli:main`	Console script `factory`, ~60 subcommands dispatched from `factory/cli/_main.py`
CLI (fallback)	`factory.cli:cmd_refactory`	Invoked with no subcommand in an interactive TTY — launches the interactive re:factory session
Web UI	`factory dashboard` → `factory/dashboard/app.py`	FastAPI + Uvicorn server, default port 8420, SSE event streaming
MCP server	`factory serve-mcp` → `factory/mcp_server.py`	Exposes factory operations as MCP tools for other Claude Code sessions
Module CLI	`python -m factory.skillopt` → `factory/skillopt/__main__.py`	Standalone SkillOpt benchmark-driven optimization entry point
Agent subprocess	`factory agent <role>` → `factory/agents/runner.py:invoke_agent`	Spawns one specialist agent as a blocking subprocess via the selected `Runner`
Orchestrator subprocess	`factory ceo` / `factory run` → CEO agent	Spawns the CEO persona as a subprocess that itself calls `factory agent`
Contained wrapper	`factory contained -- <command>` → `factory/contained/`	Re-executes any factory command inside a podman container or OpenShift namespace

12. Failure Model and Recovery

12.1 Failure Classes

Agent spawn failure — subprocess crash, timeout, or malformed/garbage output from invoke_agent().
Eval crash — the project's eval/score.py or a custom ProjectEvalDimension command times out, is not found, or returns non-JSON/invalid-schema output.
Guard violation — eval/ mutated, working tree dirty, branch not rooted at baseline, changed files outside scope, or changed files touching fixed_surfaces.
Precheck failure — score regression or below-threshold, guard/scope failure, anti-pattern similarity to a reverted hypothesis, a failing user-defined hard constraint, or missing proof of QA execution (Sacred Rule 9).
Workflow node failure — a GateNode returns HALT, an AgentNode's post_checks fail, or a RELOOP budget (max_iterations) is exhausted.
Consecutive agent failure — _FAILURE_ABORT_THRESHOLD (2) consecutive spawn failures, raising ConsecutiveAgentFailureError.
CEO premature exit — the CEO process ends before all planned work for the cycle is recorded complete.

12.2 Recovery Behavior

The CEO Review Gate protocol MUST classify every agent's output as PROCEED, REDIRECT (re-invoke the same agent with corrections, bounded to 2 redirects), or ABORT (finalize the experiment as error and move to the next hypothesis) — the CEO MUST NOT perform the failed agent's work itself under any of these outcomes (Sacred Rule 8). A GateNode RELOOP MUST target a specific upstream node and MUST respect its max_iterations bound; exhausting that bound MUST escalate to a HALT/abort path rather than looping silently forever. A guard violation MUST always resolve to revert — there is no override path, by design (§6.4, §7.2). A failing precheck MUST always resolve to revert or error — the CEO cannot keep an experiment that fails precheck regardless of its raw eval score.

12.3 Restart and Resume Semantics

factory checkpoint/factory resume persist and restore full CEO state for crash-resilient continuation independent of the lighter CycleState respawn mechanism (§7.5). A stale (> 24h) cycle.json MUST NOT trigger auto-resume, to avoid resurrecting an abandoned run against a codebase that has since diverged. Auto-resume respawns MUST be capped (DEFAULT_MAX_RESPAWNS = 5) so a systematically crashing CEO fails loudly instead of consuming an unbounded respawn budget.

13. Security and Safety

13.1 Trust Boundaries

Agent-authored code and agent-authored shell commands (from Builder, Health Checker, Code Reviewer, Adversarial Tester) are treated as untrusted output that must be verified, not as trusted collaborator output — this is the entire rationale for the CEO Review Gate, guard checks, and precheck. Neither the local podman contained runtime nor the OpenShift/Kubernetes contained runtime is a security sandbox for that untrusted code: both explicitly document "not a security sandbox" in user-facing --help text, and both still require the same human PR review any agent-produced diff needs. The OpenShift target is the stronger of the two (restricted SCC, namespace-scoped RBAC, pods/exec excluded from the sidecar's Role and asserted via SubjectAccessReview), but this is a difference of degree, not a claim of containment.

13.2 Filesystem Safety Invariants

ensure_factory_dir() MUST detect and remove a broken or circular symlink at the .factory/ path before creating the directory, rather than following it. Guard checks (check_scope, check_fixed_surfaces) MUST reject path-traversal-style scope escapes by matching against declared glob patterns rather than trusting agent-reported file lists — the changed-files list is always independently derived from git diff --name-only, never from agent self-report. The contained runtime's provenance checks MUST assert the workspace derives from the live working tree (uncommitted changes included) rather than a HEAD checkout, specifically because a HEAD-only checkout silently drops the gitignored .factory/ directory that holds the entire experiment history — a provenance failure MUST abort naming the failing assertion and the likely cause, and MUST leave the runtime up for inspection rather than tearing it down.

13.3 Secret Handling

Credentials MUST flow through FACTORY_-prefixed environment variables and ~/.factory/config.toml [credentials.<name>] profile sections only; there is no secret-manager or vault integration. show_config() MUST mask sensitive-looking values unless --reveal is explicitly passed, and profile-override warnings MUST log the overridden key name but MUST NOT log the value. _PROTECTED_VARS MUST NOT be settable or unsettable via any profile. Contained-runtime credential handling has no gateway by design: the policy is FACTORY_-prefixed vars plus exactly what --forward names, and verify MUST report credential shape (present/absent/well-formed) but MUST NEVER print secret material — any secret-looking value appearing in a command that is printed anywhere (dry-run output, verify output, logs) MUST be redacted. On the Kubernetes target, credential material MUST come from a namespace Secret the user creates out-of-band; the factory MUST reference it by name only and MUST NEVER handle the underlying material directly.

14. Test and Validation Matrix

14.1 Core Conformance Criteria

pytest -v MUST pass with asyncio_mode = "auto" — async test functions run without an explicit @pytest.mark.asyncio decorator.
An autouse _isolate_registry fixture (tests/conftest.py) MUST redirect the global project registry to a temp directory for every test, so test runs MUST NOT pollute ~/.factory/registry.json.
An autouse fixture forcing style._raw_session to None MUST prevent any contained-runtime test from blocking on a raw terminal keypress, since the raw prompt path does not call input() and therefore ignores builtins.input patches.
ruff check . MUST pass at 100-character line length; mypy factory/ MUST pass with no untyped-def leaks into the public surface.
test_annotations_match_source MUST pass, guaranteeing no drift between factory/workflow/definitions.py and its exported SKILL.md/SKILL.annotations.yaml.

14.2 Test Coverage by Subsystem

Contained-runtime tests (tests/test_contained_k8s.py) MUST stub list_contexts/cluster_context/current_namespace rather than shelling out to a real oc binary, since that binary shells out to a real cluster and materially slows the suite. Tests marked real_worktree MUST use genuine git worktree operations instead of mocks, and MUST be run deliberately, not as part of the default fast loop. Tests marked slow (real external API calls) MUST be deselectable via -m "not slow" for routine local iteration. Coverage MUST be measured over factory/ with tests/, eval/, and factory/dashboard/ excluded from the coverage denominator ([tool.coverage.run] omit).

15. Extension Points

Workflow registry (factory/workflow/definitions.py:_get_builtin_registry, plus .factory/workflows/<name>.py auto-discovery) — a new built-in mode MUST be added to the registry dict; a new portable, project-local mode MUST be written to .factory/workflows/<name>.py containing a meta dict (name, description) and a workflow() function, importing only from factory.workflow.primitives and the stdlib.
Contributed benchmark workflows (factory/workflow/contributed/*.py) — each MUST expose a workflow() callable importable via the lazy registry pattern already used for swebench, featurebench, terminalbench, etc.
Runner protocol (factory/runners/protocol.py) — a new CLI backend MUST implement metadata(), build_command(), headless(), and interactive_run(), and MUST register a RunnerMeta describing its binary, required env vars, and capability flags.
Notifier protocol (factory/notify/) — a new notification channel MUST implement the async Notifier protocol referenced from factory/models.py, following the pattern of TelegramNotifier.
Agent prompt overrides (.factory/agents/<role>.md project-local, ~/.factory/agents/prompts/<role>.md user-global) — either MUST be a complete replacement prompt for that role; both tiers still receive automatic ACE playbook injection.
ACE playbooks (~/.factory/playbooks/<role>.md evolved, factory/agents/playbooks/<role>.md default) — a new role's default playbook MUST be seeded under the factory-default path so injector.py has a fallback before any evolution has occurred.
Plugin system (factory/plugins.py) — pip-installable extensions register additional CLI subcommands via Python entry points, surfaced through factory plugins.
MCP server (factory/mcp_server.py) — exposes a subset of factory operations as MCP tools; a new tool MUST be registered here to be reachable from another Claude Code session.
SkillOpt benchmark adapters (factory/skillopt/adapter.py) — a new benchmark MUST implement the abstract adapter interface to plug into the SkillOpt optimization loop.

16. Implementation Checklist

16.1 Required for Conformance

detect_state() MUST implement the exact five-state, ordered-check logic of §7.1.
FactoryConfig and all nested config models MUST remain strict=True, extra="forbid" Pydantic models — no silent coercion or extra-field tolerance.
Every workflow graph registered in _get_builtin_registry() MUST pass validate_graph() and MUST have a corresponding, regenerated SKILL.md.
Guard checks (check_git_clean, check_experiment_branch, check_scope, check_fixed_surfaces, check_eval_immutable) MUST run before any experiment can be finalized as keep.
run_precheck() MUST be invoked, and a failure MUST force revert/error, before any keep verdict is written for an experiment that produced a PR.
Sacred Rules 1–9 (no test deletion, no out-of-scope file changes, no committed secrets, no lowered eval threshold, no skipped eval, no PR merges, mandatory archival, no agent's-job-done-by-CEO, mandatory QA) MUST hold for every experiment cycle without exception.
Every AgentNode invocation MUST produce a captured review file and a matching lifecycle event pair, per §9.2/§9.3.

16.2 Recommended Extensions

New eval dimensions SHOULD be added as additive ProjectEvalDimensions in factory.md rather than by modifying the mandatory hygiene/growth computation in factory/eval/.
New agent-CLI backends SHOULD implement dry-run support from the start, mirroring FACTORY_BOB_DRY_RUN/FACTORY_CODEX_DRY_RUN/FACTORY_OPENCODE_DRY_RUN, so CI can exercise the integration without live credentials.
New workflow modes SHOULD reuse the existing subgraph helpers (_study_subgraph, _deep_qa_subgraph, _research_subgraph) rather than re-implementing fork/join/gate wiring inline.
Outer-loop mutation operators SHOULD be added to factory/outer_loop/mutations.py's weighted strategy rather than as one-off, hand-triggered graph edits.

Appendix A. Reference Algorithms

A.1 — Composite score computation (factory/eval/scorer.py, factory/eval/runner.py)

function compute_composite(results, guard_violations, threshold):
    if results is empty:
        return CompositeScore(total=0.0,
                               passed = (guard_violations is empty) and (0.0 >= threshold))
    weight_sum = sum(r.weight for r in results)
    if weight_sum > 0 and |weight_sum - 1.0| > epsilon:
        results = [r with weight := r.weight / weight_sum for r in results]   # renormalize
    total = sum(r.score * r.weight for r in results)
    passed = (guard_violations is empty) and (total >= threshold)
    return CompositeScore(total, results, guard_violations, passed)

function merge_all(hygiene, project_additions, growth, custom_project, eval_weights):
    all_hygiene = hygiene + [p in project_additions if p.name not in (hygiene ∪ growth names)]
    (h_w, g_w, p_w) = effective_weights(eval_weights, has_custom = custom_project non-empty)
    # effective_weights: no custom -> (0.5, 0.5, 0.0)
    #                    custom + explicit project weight>0 -> proportional renormalization
    #                    custom + no explicit weights -> (0.30, 0.20, 0.50)
    return normalize_tier(all_hygiene, h_w) + normalize_tier(growth, g_w) + normalize_tier(custom_project, p_w)

Invariant: the sum of all returned EvalResult.weight values MUST equal h_w + g_w + p_w (≈1.0), so compute_composite's renormalization step is a no-op in the common case and only activates for hand-constructed result lists.

A.2 — FEEC hypothesis categorization and anti-pattern detection (factory/strategy.py)

function categorize_hypothesis(text):
    if any(fix_keyword in lower(text)):      return FIX
    if any(exploit_keyword in lower(text)):  return EXPLOIT
    if any(combine_keyword in lower(text)):  return COMBINE
    return EXPLORE   # default / catch-all

function find_anti_patterns(hypothesis, history, threshold=0.6):
    matches = []
    for entry in history where entry.verdict == "revert":
        sim = jaccard(tokenize(hypothesis), tokenize(entry.hypothesis))
        if sim >= threshold: matches.append(entry with similarity=sim)
    return matches   # non-empty => the precheck's anti_pattern check fails

Invariant: only revert-verdict history entries are eligible anti-pattern matches; keep/error/superseded entries are never considered.

A.3 — Workflow graph traversal with bounded reloop (factory/workflow/executor.py)

function execute(workflow, start_node):
    current = start_node
    reloop_counts = {}   # node_id -> count
    while current is not terminal:
        node = workflow.nodes[current]
        match type(node):
            ForkNode:  spawn(node.targets) each as independent traversal; await JoinNode barrier
            GateNode:  verdict = evaluate(node)   # agent | fn | user
                       if verdict.type == RELOOP:
                           reloop_counts[verdict.target] += 1
                           if reloop_counts[verdict.target] > verdict.max_iterations:
                               escalate_to_halt(verdict.target)
                           current = verdict.target; continue
                       if verdict.type == HALT: abort(verdict.reason)
            AgentNode: result = invoke_agent(node.role, ...)
                       for check in node.post_checks: assert_artifact(check)  # failure => node failure
            FnNode:    run_command(node.command)
        current = next_node_via(matching Edge for current's outcome)

Invariant: a RELOOP targeting the same node MUST NOT exceed that edge's max_iterations; exceeding it MUST transition to a HALT rather than loop silently.

A.4 — Adversarial phase switching with hysteresis (factory/adversarial.py)

function step(state, config, round_score, metric_name):
    active = get_active_component(config, state)
    above = round_score >= active.threshold
    state.consecutive_above = state.consecutive_above + 1 if above else 0
    if state.active_role == "generator":
        state.generator_consecutive_above = state.consecutive_above
    else:
        state.discriminator_consecutive_above = state.consecutive_above

    switched = False
    if state.consecutive_above >= config.hysteresis:
        state.active_role = other(state.active_role)
        state.consecutive_above = 0
        switched = True

    state.converged = (state.generator_consecutive_above >= config.convergence_window
                        and state.discriminator_consecutive_above >= config.convergence_window)
    state.history.append(AdversarialPhaseRecord(round, active_role, score, metric_name, switched))
    return state

Invariant: converged MUST require both per-role streak counters to independently satisfy convergence_window; a switch resets only the just-vacated role's active streak tracking, not the other role's independently-tracked counter.

A.5 — Project state detection (factory/state.py)

function detect_state(path):
    if not path.exists() or not (path / ".git").exists():
        return NO_REPO
    if eval_profile.json exists and human_reviewed == False:
        return EVALS_PENDING_REVIEW
    if .factory/config.json exists:
        return HAS_FACTORY
    if gh_issue_list(label="plan", state="open") is non-empty:
        return REPO_INCOMPLETE
    return NO_FACTORY

Invariant: the pending-review check MUST be evaluated strictly before the has-factory check; a gh CLI timeout or missing binary MUST be treated as "no open plan issues," not as an error that aborts detection.

How to Read the Knowledge Graph

This spec uses [[graph:...]] reference links to point into a code knowledge graph extracted by graphify. The graph contains AST-derived entities (modules, classes, functions) and their typed relationships (imports, calls, inherits).

Reference Link Types

[[graph:EntityName]] — look up a specific entity (module, class, function)
[[graph:path:A:B]] — find the dependency path between entities A and B
[[graph:query:question]] — run a natural language query against the graph
[[graph:community:subsystem]] — list all entities in a detected subsystem

When to Use

Planning and design: Read the overview sections in this spec
Implementation details: Resolve [[graph:...]] links by reading graph.json directly, or query the graph with graphify explain, graphify path, graphify query

FilesExpand file tree

SPEC.md

Latest commit

History