[Request]: Harness Loop Mode

[stilero]

Problem or opportunity

Today BanKan helps manage tasks through stages like planning, implementation, and review.

But a single task still depends too much on a model deciding for itself when it is “done”. That creates the same problem we see in normal AI coding workflows:

• agents stop too early
• fixes are made without validation
• the same mistakes repeat across iterations
• there is no strong connection between execution and real completion criteria

Ralph-style looping solves part of this by forcing repeated iterations, but by itself it is too simple. It mainly says “keep going until a magic completion token appears”.

Anthropic’s harness idea is stronger. The harness controls the agent, tool usage, evaluation, and stop conditions.

This creates an opportunity for BanKan to introduce a more reliable execution mode where a task is not considered done because the model says so, but because BanKan validates it.

Proposed solution

BanKan should act as the harness.

That means BanKan owns:

• loop orchestration
• stage transitions
• execution state
• validation rules
• retry logic
• history of failures across iterations

The agent remains responsible for producing work, but BanKan decides whether the work is actually complete.

New task mode

Add a task mode:

• standard
• harness-loop

When harness-loop is enabled, the task runs as an iterative controlled workflow instead of a single pass through stages.

Loop phases

For a task in Harness Loop Mode, BanKan should support these phases:

1. Plan

   • restate problem
   • identify assumptions
   • propose implementation steps

2. Implement

   • make focused code changes for current step

3. Review

   • review own work or send to review agent/model

4. Validate

   • run configured commands such as test, lint, build, typecheck

5. Refine

   • use validation and review feedback to improve implementation

Then repeat until done.

Completion criteria

Each Harness Loop task should have explicit completion criteria.

Example:

• npm test exits successfully
• npm run lint exits successfully
• npm run build exits successfully
• no blocking review findings remain
• optional required files were changed
• optional human approval required before done

BanKan should not rely on a completion phrase like DONE unless the user explicitly wants that as an extra signal.

Validation profiles

Allow reusable validation profiles at repo or task level.

Examples:

• Backend API profile

  • yarn test
  • yarn lint
  • yarn build

• React Native profile

  • yarn test
  • yarn tsc
  • custom bundle or Expo validation

This makes the feature practical across many repos.

Memory between iterations

Each loop should keep structured memory, for example:

• failed validation commands
• review findings
• repeated mistakes
• implementation decisions
• files already touched

This memory is injected into the next iteration so the agent avoids doing the same bad fix again.

Agent roles

Harness Loop Mode should support either:

• single agent mode
• multi-agent mode

Single agent mode

One agent handles all phases.

Multi-agent mode

Different agents can be configured per phase, for example:

• planner agent
• implementer agent
• reviewer agent
• fixer agent

This fits BanKan extremely well and is stronger than raw Ralph looping.

UI changes

On the task card, show:

• current loop iteration, for example Iteration 2/5
• current phase
• validation status
• last failure reason
• whether task is blocked on human input
• summary of what changed in latest iteration

In task details, show a loop timeline:

• Iteration 1
  • Plan created
  • Implementation completed
  • Validation failed: tests
• Iteration 2
  • Fix applied
  • Review found edge case
  • Validation failed: lint
• Iteration 3
  • Fix applied
  • Validation passed

This would make BanKan much more transparent than terminal-only workflows.

Human escalation

If the loop gets stuck, BanKan should escalate.

Examples:

• same validation failure 3 times
• no meaningful file changes across 2 iterations
• reviewer keeps finding blocking issues
• max iterations reached

At that point the task moves to a state like:

• Needs human help

And BanKan should clearly show:

• what failed
• what the agent tried
• suggested next action for the human

Acceptance criteria

• A task can be created with Harness Loop Mode enabled
• A task supports configurable max iterations
• A task supports configurable validation commands
• BanKan runs the loop automatically until success or failure threshold
• Validation results are stored per iteration
• BanKan injects prior failures/findings into the next loop iteration
• UI shows current iteration, phase, and validation outcome
• Task can escalate to human when stuck
• Supports both single-agent and multi-agent configuration
• Final task completion is based on validation outcome, not only model output

Alternatives considered

1. Plain Ralph-style loop only

Simpler to build, but too weak long term.

Problems:

• relies too much on prompt design
• weak state handling
• poor visibility
• no real structured validation model

2. Keep current BanKan flow and add more prompts

This helps a little, but does not solve the core issue that BanKan still trusts the agent too much.

3. Fully custom autonomous agent framework outside task stages

More flexible, but risks making BanKan much more complex too early. Better to extend the current stage model with controlled looping.

Additional context

This feature would be a strong differentiator for BanKan.

A lot of AI coding tools still behave like this:

• prompt agent
• hope it solves the task
• manually inspect result

Harness Loop Mode would let BanKan position itself more like:

A controlled execution system for AI coding tasks, where completion is validated, visible, and repeatable.

That is much stronger than “kanban for agents”.

Suggested MVP scope

Ship a first version with:

• single agent mode
• fixed loop phases
• configurable max iterations
• configurable validation commands
• task timeline showing iteration results
• human escalation when stuck

Hold off for later on:

• multi-agent per phase
• advanced memory store
• reusable validation profiles at org level
• cross-task learning

Suggested future enhancements

• repo-level harness presets
• reviewer model different from implementer model
• branch protection integration
• auto-create PR only when harness passes
• shared memory across tasks in same repo
• “strict mode” requiring zero failing checks before task can move to done