Skip to content

feat: external task integration — intake, overnight scheduling, and result write-back ("cron for Claude Code") #53

Description

@caoergou

External task integration: make owloop the night shift for tasks defined during the day

Vision

Today's flow: user writes spec → owloop run → agent executes → results stay on local disk.

Target flow:

Day:    GitHub Issue / Jira / TAPD / Feishu task labeled `owloop:auto`
        │  owloop sync  (task → spec, with clarify gate + provenance)
Night:  ▼  scheduled `owloop run` (locked, budgeted, verified per #32/#33/#34)
        │  per-spec outcome write-back (comment + label + branch/PR)
Morning:▼  digest report to Slack/Feishu/issue — "here's what shipped, here's what's stuck"

This upgrades owloop from a dev tool you babysit into an automation teammate: "every night at 2 a.m., owloop picks up the tasks marked auto, runs them to green, and posts a report by 8 a.m." The tagline writes itself: cron for Claude Code.

This issue is the design + research record for that track. It deliberately builds on top of the trust core (#32 deterministic gate, #33 terminal states, #34 rollback) — unattended scheduling without those is how you wake up to a wrecked branch.


Part 1 — Where owloop stands today (code audit)

Capability Status Evidence
Task queue ✅ solid, file-based spec_queue.py: priority, ## Depends On, topological get_next_ready_spec() — already a miniature dependency-aware task DB
Task intake from outside 🟡 embryonic spec-from-issue (spec_from_issue.py): GitHub-only, one-shot, one-way; fetches via gh, renders spec template, checklist items → candidate criteria. Jira is a NotImplementedError stub. No provenance stored, no idempotency, no state write-back
Scheduling ❌ none owloop run is a foreground process; "run overnight" currently means the user hand-writes a cron entry with no lock, no budget enforcement, no report delivery
Result output 🟡 local only owloop_summary_latest.json, events.jsonl, HTML report (report.py + report_ai.py). Nothing leaves the machine; a loop that stops on BLOCKED at 2 a.m. wastes the night silently (already noted in #37 Phase 2)
Mid-run steering STEERING.md, run-notes.md, .owloop/learnings.md — good primitives to carry task-source context into iterations
Sessions/resume session_latest.json, --resume, per-session worktrees — the building blocks a scheduler needs to recover from interrupted nights

Key structural observation: the loop engine never needs to know about external systems. specs/ is already the interface. External integration should be an airlock around the loop (sync-in before, write-back after), not a modification of the loop itself. This preserves the "fresh context per iteration, state on disk" principle verbatim.

Part 2 — How comparable systems do it (research survey, verified 2026-07-06)

System Task intake Result output Anti-chaos mechanism
Ralph lineage (fork parent, official plugin) specs/*.md, IMPLEMENTATION_PLAN.md on disk git commits; completion log; Telegram notify (fork) backpressure = tests/lint/build; per-spec attempt counter → stuck after N tries → "decompose the spec"
OpenHands resolver fix-me label on an issue or @openhands-agent comment draft PR on success; on failure: WIP branch + issue comment explaining outcome label removed after processing → trigger consumed exactly once (idempotency); iteration cap; humans review every PR
Copilot coding agent assign an issue to Copilot PR opened immediately; plan posted as a live checklist on the PR, items checked off as it works ephemeral env per issue; egress firewall; CI on its commits needs human approval
Google Jules / OpenAI Codex cloud task pool via UI / issue refs; AGENTS.md standing instructions PRs VM/container per task; Codex: setup-with-network then execute-offline; plan-approval step
claude-code-action GitHub events, @claude mentions, schedule: cron with explicit prompt issue/PR comments with progress checkboxes; branches; PRs --max-turns per invocation; scoped app permissions; workflow timeouts
SWE-agent / mini-SWE-agent batch mode task-list file, one isolated env per instance, N parallel workers predictions/patch file evaluation is a separate process the agent cannot touch; per-instance cost/step budgets
Aider scripting --message / --message-file, looped by shell direct commits git as undo; --test-cmd/--lint-cmd gates
continuous-claude one goal + SHARED_TASK_NOTES.md scratchpad one merged PR per iteration --max-cost USD cap, --max-duration; done-signal required 3 iterations in a row; stall threshold → pause + diagnostics for human
beads (Yegge) JSONL issues in git + SQLite cache; dependency graph with auto-"ready" detection; hash IDs avoid multi-agent collisions git-native queue shared across agents/machines through git itself
Backlog.md / vibe-kanban markdown+frontmatter tasks / Kanban over 10+ agents board, PRs worktree-per-attempt (converges with owloop); note: vibe-kanban is sunsetting — heavy orchestration UIs don't survive; file-based queues do
Spec Kit / Kiro issue-shaped intent → constitution → spec → derived tasks clarify gate before generation; EARS acceptance criteria; dependency waves

Cross-framework patterns that matter here

  1. Label-as-trigger, label-removal-as-ack (OpenHands) is the cleanest idempotency contract for issue-tracker intake.
  2. Failure must still produce an artifact — WIP branch + comment beats silent retry, everywhere it's tried.
  3. Task intake converges on structured files in git; issue trackers are the second channel, synced in, never driving the loop live.
  4. Multiple independent bounds, always: iteration + cost + wall-clock + per-task attempt caps; mature systems layer at least two.
  5. Nobody auto-closes the source task. PRs/comments propose; humans dispose. Copilot, OpenHands, Jules, Codex all keep a human on merge.
  6. A scheduled agent needs a scratchpad and a lock: PID-file locking with stale detection and persisted resume state are standard reliability plumbing for cron-driven agent runs.
  7. Scheduled human intent-review every N tasks/hours — tests verify behavior, humans verify intent; drift that passes CI is still drift.

Part 3 — Proposed design (five layers, each independently shippable)

Layer 1 — Task intake: owloop sync + pluggable TaskSource

Evolve spec_from_issue.py into a TaskSource adapter interface (mirroring adapters.py for agents):

class TaskSource(Protocol):
    def poll(self) -> list[ExternalTask]      # tasks matching the trigger (e.g. label owloop:auto)
    def acknowledge(self, task) -> None       # consume the trigger (remove/flip label) — idempotency
    def write_back(self, task, outcome) -> None  # comment + label transition

Layer 2 — Spec as contract: provenance frontmatter

Synced specs carry a machine-readable provenance block:

---
source: github            # github | jira | tapd | feishu | todo
source_id: "caoergou/owloop#123"
source_url: https://github.com/caoergou/owloop/issues/123
synced_at: 2026-07-06T18:00:00Z
source_hash: sha256:…     # hash of imported title+body
---
  • source_hash makes re-sync idempotent (same issue → same spec, updated only if upstream changed) and makes divergence detectable: if the upstream issue changed after sync, the spec is flagged stale rather than silently running against yesterday's requirements.
  • The loop never reads external systems. Sync-in and write-back are separate commands around the run — an airlock, not a live pipe. Fresh-context-per-iteration stays intact; a network-dead night still runs everything already in specs/.

Layer 3 — Night shift: owloop schedule / owloop night

Layer 4 — Write-back: results flow to where the task came from

Per-spec terminal state (from #33's taxonomy) maps to source-side actions:

Outcome Issue label Comment Git artifact
verified done (#32 gate passed) owloop:done summary + commit links + criteria results pushed branch (or PR with --pr-per-spec)
stalled / stuck owloop:stuck failure classification, attempts, last error tail WIP branch + discarded-diff patch (#34) — failure still produces an artifact
blocked / decide owloop:blocked the BLOCKED:/DECIDE: payload, verbatim — the morning human's first read
  • Never auto-close the source issue. owloop proposes; the human disposes (unanimous across Copilot/OpenHands/Jules/Codex). Optionally auto-close only when a linked PR merges — GitHub's own machinery, not ours.
  • Write-back is at-least-once with dedup: comments carry a <!-- owloop:run-id:spec --> marker so a re-run after a crash updates rather than duplicates.

Layer 5 — Morning report

  • Extend report.py's existing summary/events into a digest: per-spec outcome table, tokens/cost, stuck items with reasons, links to branches/PRs. Deliver via generic webhook (Slack/Feishu/Discord all accept a JSON POST) and/or a pinned "night report" issue comment.
  • Immediate (not morning) notification on BLOCKED/DECIDE/stalledmeta: loop-engineering roadmap — from prompt-level conventions to enforced invariants #37 Phase 2 already lists this; it's required for scheduled mode, since a silently-stopped 2 a.m. loop wastes the whole night.

Part 4 — Why this doesn't produce chaos (guarantee map)

Risk Mitigation Where
Same task imported/run twice label-consumed-on-sync + source_hash idempotency Layer 1/2
Two runs on one repo overnight single-writer lockfile with stale detection Layer 3
Vague day-time task → wrong overnight work clarify gate at intake; unverifiable tasks parked as needs-spec in daylight Layer 1
Agent self-reports success engine-owned verification gate #32 (prerequisite)
Failure loop burns the night hard stop on stall, named terminal states, budget floors mandatory in scheduled mode #33 (prerequisite) + Layer 3
Failed iteration poisons the next rollback + discarded-diff patch #34 (prerequisite)
Upstream issue edited mid-night loop reads only the synced spec (airlock); staleness flagged at next sync Layer 2
Results lost / duplicated on crash at-least-once write-back with comment markers; session resume Layer 4
Silent 2 a.m. stop immediate notify on BLOCKED/DECIDE/stalled Layer 5
Intent drift that passes tests report is designed for a mandatory human morning review; nothing merges or closes itself Layer 4/5

Phased plan

Phase I — intake foundations (no scheduler yet, immediately useful interactively)

  • TaskSource protocol; refactor spec_from_issue.py into GitHubIssueSource
  • Provenance frontmatter written on import; source_hash idempotent re-sync
  • todo.md source (in-repo checklist with @auto tag)
  • Intake clarify gate: linter + runnable-criteria check; needs-spec outcome

Phase II — night shift (gated on #32 + #33 + #34)

  • Lockfile with stale detection
  • owloop night composed command; scheduled mode requires bounds
  • owloop schedule emitting cron/systemd/launchd/GitHub Actions artifacts

Phase III — write-back & report

  • Per-spec outcome → label transition + deduped comment; WIP branch on stuck
  • --pr-per-spec option
  • Webhook digest (Slack/Feishu-compatible JSON POST) + immediate BLOCKED/DECIDE/stalled notify

Phase IV — more sources

  • Jira (replace the stub), TAPD, Feishu tasks behind TaskSource

Non-goals

References

ghuntley.com/ralph · anthropics/claude-code ralph-wiggum plugin · fstandhartinger/ralph-wiggum · OpenHands resolver docs · GitHub Copilot coding agent docs · jules.google · developers.openai.com/codex/cloud · anthropics/claude-code-action · SWE-agent/mini-swe-agent batch mode · aider.chat/docs/scripting · AnandChowdhary/continuous-claude · steveyegge/beads · MrLesk/Backlog.md · BloopAI/vibe-kanban · github/spec-kit · kiro.dev/docs/specs

Relates to: #32 #33 #34 (prerequisites for Phase II) · #35 #15 (budget caps the scheduler depends on) · #36 (clarify gate reused at intake) · #37 (this is the "integration track" alongside its trust track) · #49 (perf track; parallelism explicitly out of scope here) · #9 #11 (linter/dry-run as the intake gate's mechanical checks)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions