AI Dev Team

Status: research release from Borge Labs, published as-is. It has no support SLA or committed roadmap. The benchmark record includes experiments that failed as well as those that worked. Reported scores are historical, self-reported observations; the raw run artifacts are not included in this snapshot, so they cannot be independently reproduced from this repository. AI Dev Team was used for real delivery work, but it is no longer Borge Labs' day-to-day delivery harness. It remains a research and benchmark harness; its runner-observed verification and benchmark results, including results that favored simpler configurations, informed the supervised loop that replaced it in daily delivery. That loop is described in A supervised multi-agent delivery loop.

AI Dev Team is a self-hosted runtime for supervised coding agents. It runs a Planner → Orchestrator → Reviewer delivery loop, executes verification itself, and exposes the resulting diffs, test output, review decisions, usage, and cost in a web UI. A model saying “tests pass” is not a green signal; the runner must observe the command result.

The screenshot contains only the synthetic starter repository and a mock-mode session. The same run's inspector recorded npm ci and npm test as passed, runner-observed verification commands.

What is here

• model-per-role configuration with provider, model, API-key environment, optional base URL, tool set, and thinking level in YAML
• runner-observed builds and tests, review/repair loops, and final quality gates
• cross-family role assignment for implementation and review, as in the research-informed example config
• optional tandem review with a second reviewer in blocking or advisory mode
• isolated Git worktrees for configured-repository runs and delegated workers
• live steering, session controls, artifacts, and per-model usage in the web UI
• Postgres-backed jobs, events, approvals, repository memory, and cost history
• approval policy for merge, deploy, paid, and public actions
• optional GitHub App webhooks and pull-request publishing
• headless SWE-bench-style evaluation with provider-reconciled cost accounting
• optional hooks, MCP servers, slash commands, model routing, and an AI Team bridge

The code is reusable, but this repository is principally a transparent research artifact. Start with SUPPORT.md, the threat model, and the research release notes.

Quick start: no model account required

Prerequisites: Docker with Compose, Git, OpenSSL, a POSIX-compatible shell, and a non-root host user. The initializer rejects UID or primary GID 0 so the web and runner cannot create root-owned files in bind-mounted repositories.

./scripts/init-local-env.sh
docker compose up --build

The initializer creates strong local credentials and an ignored, disposable Git repository at workspaces/sample. It prints the generated admin password; it is also stored in .env. It maps the web and runner containers to your non-root host UID/GID so bind-mounted repositories remain writable without broad permissions. Open http://localhost:3000, sign in, select Sample repository, and create a session.

The default AI_DEV_TEAM_ENGINE=mock produces deterministic role output and makes no model-provider calls. It is intended to verify the control plane, UI, queue, and runner before credentials or real source code are introduced.

The default runner is an Alpine/musl image and is browser-free. It includes Git, Python, a C/C++ build toolchain, and PostgreSQL headers, and installs its own musl-native production dependencies. This is the smallest supported default and had no known vulnerabilities in the release scan. If a trusted repository needs a glibc-only binary or toolchain, select the larger Debian compatibility target:

AI_DEV_TEAM_RUNNER_TARGET=compat

The runner keeps the Playwright JavaScript package so repository dependency detection remains deterministic, but the runtime and compat targets omit Chromium and its OS libraries. Visual verification is disabled in the starter. If a trusted job deliberately needs screenshot capture or Playwright E2E, opt in to the Debian-based browser target before building:

AI_DEV_TEAM_RUNNER_TARGET=browser

docker compose build runner
docker compose up

Without the browser target, an enabled visual stage or detected Playwright E2E suite fails with an explicit rebuild instruction; the runner will not download a browser ad hoc. Return the variable to runtime to restore the Alpine default. The compatibility and browser targets carry a materially larger OS advisory surface and are explicit build-on-demand targets, not part of a plain/default build; see the release notes for the exact scan record.

Jobs intentionally run against the broad workspaceRoot are manual-publication workflows: the runner captures a pre-run repository baseline, reports committed and uncommitted changes in child repositories, and does not push them. Managed branch/PR publication requires selecting a registered GitHub repository before the job so the runner can capture and later verify its immutable base SHA.

Stop the stack with:

docker compose down

Add --volumes only when you intentionally want to delete the local Postgres history.

Minimal live starter

The bundled YAML is a single-provider OpenAI configuration using gpt-5.4-mini for a read-only Chat role and the three delivery roles. In .env, set:

AI_DEV_TEAM_ENGINE=pi
OPENAI_API_KEY=your-real-key

Then recreate the runner service so Compose applies the changed environment:

docker compose up -d --force-recreate runner

To use another provider, change the relevant role entries in ai-dev-team.example.yaml (or copy it to an ignored ai-dev-team.yaml and set AI_DEV_TEAM_CONFIG_FILE=./ai-dev-team.yaml). Compose forwards the common OpenAI, Anthropic, DeepSeek, Xiaomi, OpenRouter, Google, Gemini, Mistral, Groq, and Azure OpenAI credential variables to the runner.

The starter deliberately has no MCP server, lifecycle hook, tandem agent, GitHub integration, or Loop Engineering dependency. Those features are opt-in. It is an onboarding configuration, not the configuration used for the benchmark findings. The separately shipped research-informed DSv4/Qwen config preserves that documented posture without making its two-provider setup the first-run path. The exact single DSv4-build + Qwen-review pairing was not a measured arm; the measured Qwen arm also included a MiMo portfolio candidate.

Use it by copying it to the repository root so its prompt paths resolve exactly as they do in Compose:

cp examples/config/research-informed-dsv4-qwen.yaml ai-dev-team.yaml
AI_DEV_TEAM_CONFIG_FILE=./ai-dev-team.yaml docker compose up --build

Configuration

The web and runner read the same YAML from AI_DEV_TEAM_CONFIG. The complete, tested starter is ai-dev-team.example.yaml. The following is an abridged shape; copy the linked file rather than this excerpt so its budget caps, review limits and other safety defaults are preserved:

workspaces:
  - id: sample
    label: Sample repository
    path: /workspaces/sample

workspaceIsolation:
  mode: worktree
  root: /tmp/ai-dev-team-worktrees
  cleanup: on_success
  subagents: worktree

pipeline: [planner, orchestrator, reviewer]

workflow:
  chat: chat
  planner: planner
  orchestrator: orchestrator
  reviewer: reviewer
  planning:
    enabled: true
    primary: planner
  build:
    mode: "off"

loopEngineering:
  enabled: false

github:
  enabled: false
  webhookSecretEnv: GITHUB_WEBHOOK_SECRET

roles:
  chat:
    kind: orchestrator
    provider: openai
    model: gpt-5.4-mini
    apiKeyEnv: OPENAI_API_KEY
    tools: [read, grep, find, ls]
    thinkingLevel: high
    systemPromptPath: config/prompts/chat.md
  planner:
    kind: planner
    provider: openai
    model: gpt-5.4-mini
    apiKeyEnv: OPENAI_API_KEY
    tools: [read, grep, find, ls]
    thinkingLevel: high
    systemPromptPath: config/prompts/planner.md
  orchestrator:
    kind: orchestrator
    provider: openai
    model: gpt-5.4-mini
    apiKeyEnv: OPENAI_API_KEY
    tools: [read, bash, edit, write, grep, find, ls]
    thinkingLevel: high
    systemPromptPath: config/prompts/orchestrator.md
  reviewer:
    kind: reviewer
    provider: openai
    model: gpt-5.4-mini
    apiKeyEnv: OPENAI_API_KEY
    tools: [read, bash, edit, write, grep, find, ls]
    thinkingLevel: high
    systemPromptPath: config/prompts/reviewer.md

Per-workspace verification commands override package-script auto-detection. This is useful for non-Node repositories:

workspaces:
  - id: service
    label: Python service
    path: /workspaces/service
    verification:
      - label: pytest
        command: python
        args: ["-m", "pytest", "tests/unit", "-x", "-q"]
        timeoutMs: 300000

Security boundary

Only mount repositories you trust, or put the entire runner inside a stronger isolation boundary designed for hostile code. A Git worktree separates changes; it does not sandbox code. Tool policy is a guardrail, not a security boundary. Repository tests, build scripts, Git hooks, Pi resources explicitly enabled by an operator, and other subprocesses run with the runner's filesystem, network, and environment access.

The Compose starter reduces accidental exposure: the web port binds only to 127.0.0.1, credentials have no fallback values, documented placeholders are rejected, and web/runner map to the invoking non-root host UID/GID. Those controls do not make an untrusted repository safe. See docs/threat-model.md for assets, trust assumptions, attack paths, and deployment guidance.

For a remote deployment:

• terminate HTTPS at a trusted reverse proxy and set SESSION_COOKIE_SECURE=true
• keep Postgres private and restrict outbound network access where practical
• mount only the repositories and credentials required for the job
• do not mount a Docker socket, host home directory, or broad credential store
• use separate low-privilege GitHub App installations and provider keys
• configure AI_DEV_TEAM_INTERNAL_TOKEN before exposing /internal/*

When github.enabled: true, github.webhookSecretEnv and the referenced secret must both be present. The web process refuses to start otherwise; unsigned webhooks are never accepted as a fallback.

Verification and development

CI uses Node 24.17.0 and pnpm 11.11.0. Local development supports Node 22.19.0 or newer as declared in package.json:

corepack enable
corepack prepare pnpm@11.11.0 --activate
pnpm install --frozen-lockfile
pnpm audit:prod
pnpm typecheck
pnpm build
pnpm test

Python 3 is also required by the small standard-library test suite for the tracked benchmark helpers.

pnpm audit:prod fails CI for high or critical production advisories. Dependabot is configured for pnpm, Docker, and GitHub Actions updates. Container release qualification scans the package-manager-free Alpine web image, the default Alpine runner, and the explicitly optional Debian compatibility and browser targets separately; the exact results for this snapshot are recorded in the release notes.

Historical benchmark observations

The useful historical observation is not that more agents always win. The self-reported record says the opposite: coordination must justify its cost and failure surface. These figures are retained to explain product decisions, not as a reproducible leaderboard submission.

• A historical 50-instance SWE-bench Verified run was recorded as 28/50.
• On one Django-heavy 20-case slice, a cleaned DSv4-primary/MiMo-candidate portfolio scored 17/20, versus 16/20 for DSv4 alone.
• On a fresher mixed 40-case slice, that portfolio scored 27/40, versus 28/40 for DSv4 alone; it produced no portfolio-only solve.
• A later review-enabled portfolio scored 26/40 at a token-reconciled $3.64, remaining a strict subset of the single-model baseline on that slice.

The self-reported implications, limitations, costs, discordant cases, and failed experiments are recorded in docs/benchmark-results.md and docs/experiment-summary-tables.md. The shorter case study explains how those observations shaped the runtime. Raw predictions, evaluator reports, and the original run ledger are not shipped in this release.

Headless entry points include swe, swe-pro, terminal, and estimate:

pnpm bench -- estimate --tasks 100 --profiles 2 --avg-cost-usd 0.75

pnpm bench -- swe \
  --input ./instances.jsonl \
  --output ./predictions.jsonl \
  --workspace-root /tmp/ai-dev-team-bench \
  --artifacts-dir ./.ai-dev-team/bench/artifacts \
  --engine pi

Benchmark mode disables web/product surfaces, hooks, GitHub publishing, and persistent cross-task memory. It captures patches, events, usage, and evaluator output as artifacts.

Optional integrations

GitHub App

Set github.enabled: true, configure GITHUB_WEBHOOK_SECRET, GITHUB_APP_ID, and either GITHUB_PRIVATE_KEY or GITHUB_PRIVATE_KEY_FILE, then point the GitHub App webhook at /api/github/webhook. Install the app only on selected repositories. Commands beginning with /ai-dev-team create approval-gated jobs by default; successful jobs can push a branch and open a pull request according to externalActions policy. If push or PR creation requires a separate action approval, record it before the job reaches publication. This research release fails closed by skipping publication; it does not keep a completed worktree in a delayed-publication queue.

AI Team bridge

AI Dev Team does not own a Slack app. The sibling AI Team service, or another trusted service, can use the authenticated /internal/* API to create, approve, steer, abort, and inspect jobs. Set a strong AI_DEV_TEAM_INTERNAL_TOKEN; without one, the internal API returns 503. Clients must send the exact approvalId returned by create/promote/continue (or the pendingApproval.id returned by job inspection) when approving or denying. A stale or replayed approval ID fails closed.

Extension points

• routing.rules can choose provider/model by role, prompt size, keywords, or purpose.
• hooks.<event> can run lifecycle commands with job metadata in environment variables.
• mcp.servers can expose explicitly configured stdio or HTTP MCP tools to selected roles.
• slashCommands expands repeatable workflows such as /fix-ci.
• roles.<id>.pi can opt into specific Pi extensions, skills, prompts, or themes.
• loopEngineering is a prototype issue-to-job adapter and remains disabled in the starter. Only its internal executor and before-dev human gate are wired; reserved executor and later-gate values fail closed in this release. Its github.allowedRepos list also fails closed: an empty list permits no repositories.

Enabling extensions expands the trusted computing base. Review their code and configuration before use.

Documentation

• Architecture narrative
• Threat model
• Research release notes
• Runner-observed evaluation case study
• Benchmark results
• Experiment summary tables
• Brownfield bug benches
• Multi-repository sessions
• Loop Engineering notes

About Borge Labs

AI Dev Team is built by Borge Labs, an independent Norwegian AI systems lab and product studio. It was used for real delivery work on:

• Politikkradar, source-grounded monitoring of Norwegian politics
• Ordrett, verbatim Norwegian speech-to-text

That provenance is evidence of practical use, not a promise that this research snapshot fits another production environment.

License

AI Dev Team is released under the MIT License. You may self-host, fork, modify, redistribute, or use it commercially under those terms.