Skip to content

[codex] add portable workspace preparation tool - #26

Merged
divo12 merged 3 commits into
freshfrom
codex/prepare-workspace-tool
Aug 28, 2026
Merged

divo12 merged 3 commits into
freshfrom
codex/prepare-workspace-tool

Conversation

@divo12

@divo12 divo12 commented Aug 28, 2026

Copy link
Copy Markdown
Owner

Problem

OpenFlywheel could query Langfuse traces and record authoritative outcomes, but initializing an optimization experiment still required manual branch management, ad-hoc Harbor commands, and machine-specific Codex MCP registration. That made expensive duplicate baselines possible and could evaluate the original harness checkout instead of the candidate workspace.

Root cause

There was no typed preparation use case connecting Git isolation, generated experiment instructions, a pinned harness revision, ITSM-bench execution, and re-entrant status polling. The plugin MCP launcher also depended on local $PLUGIN_ROOT and $OPENFLYWHEEL_ROOT paths, which codex exec does not reliably inject into contributed MCP subprocesses.

Change

prepare_workspace MCP tool

  • Add strict, frozen preparation input/output contracts with typed error and recovery states.
  • Create ofw/<experiment-id> from an explicit base commit in a sibling Git worktree; never switch or reset the original checkout.
  • Generate and commit PROGRAM.md plus experiment_config.yaml before baseline execution.
  • Run the ITSM Harbor baseline sequentially with zero retries and the initialization commit as the Langfuse release.
  • Require the ITSM adapter to consume OFW_HERMES_SOURCE, preventing evaluation of the wrong Hermes checkout.
  • Persist bounded control state under the Git common directory and make identical calls idempotent: running -> ready | failed.
  • Parse bounded Harbor results into pass, fail, unverified, and unsupported-reward counts without inferring outcomes.

Modular preparation package

src/ofw/preparation/
  contracts.py
  service.py
  worktree.py
  harbor.py
  templates/

Every module is under 500 lines and every touched function has cyclomatic complexity 5 or lower.

Portable plugin runtime

  • Add the installable openflywheel-mcp console entry point.
  • Bundle the shared and ITSM program templates as wheel resources and test byte parity with plugin assets.
  • Launch the plugin through uvx from immutable package commit 6319f24.
  • Pass only named credential environment variables; no credentials or machine-specific filesystem paths are embedded.
  • Remove the obsolete placeholder-only initialization path.

Live ITSM evidence

A fresh Codex session invoked the installed plugin and prepared one real ITSM task:

  • Experiment: itsm-hermes-plugin-smoke-20260828
  • Branch: ofw/itsm-hermes-plugin-smoke-20260828
  • Initialization commit: 7e48d58a26c77d85efef4a1c8a12b3bc7989ff9a
  • ITSM verifier: 1 pass / 0 failures / 0 unverified
  • Langfuse trace: fa75f2dc85225f10a5f3802981cf9c87
  • Observations: 70; generations: 22
  • Langfuse cost: $0.252173849995
  • Outcome scores written during preparation: 0

The original Hermes checkout remained unchanged. Replaying the identical request returned the same ready result without launching another baseline. After removing the manual development MCP registration, a fresh codex exec discovered and called the installed plugin's own openflywheel.prepare_workspace server successfully.

Companion integration

The ITSM adapter change and one-task smoke manifest are pushed separately to divo12/ITSMBench@exp/ofw-hermes:

  • b27c58a — consume OFW_HERMES_SOURCE
  • 24cf9b1 — add safe one-task OpenFlywheel smoke configuration

Verification

  • uv run ruff check src tests plugins/openflywheel/scripts/mcp_server.py
  • uv run mypy src tests plugins/openflywheel/scripts/mcp_server.py
  • uv run pytest --cov=ofw --cov-report=term-missing --cov-fail-under=90 -q
    • 135 passed, 2 skipped
    • 95.13% total coverage
    • each new preparation/MCP module >=90%
  • radon cc -s -a src/ofw/mcp.py src/ofw/preparation plugins/openflywheel/scripts/mcp_server.py
    • average complexity A (1.84), maximum 5
  • skill validation passed
  • plugin validation passed
  • built wheel contains openflywheel-mcp and both packaged templates
  • pinned GitHub uvx startup and plugin-only codex exec verified

Scope boundary

This PR prepares and evaluates the baseline workspace. It does not start the optimization loop automatically, edit the harness after initialization, record outcome scores, push the experiment branch, or open a harness pull request.

Summary by cubic

Adds the prepare_workspace tool that isolates harness experiments in a sibling Git worktree, records a baseline via Harbor, and writes PROGRAM.md from packaged templates. Replaces the inline plugin MCP server with an installable openflywheel-mcp console script launched by uvx from a pinned commit, so the plugin no longer runs from the local checkout.

New Features

  • prepare_workspace creates a non-destructive ofw/<experiment-id> branch in a sibling worktree and returns typed preparation status.
  • Baseline trials normalize unsupported rewards and errors as unverified.

Refactors

  • MCP server code moves from plugins/openflywheel/scripts/mcp_server.py to src/ofw/mcp.py with an openflywheel-mcp console script.
  • The plugin .mcp.json now launches via uvx --from git+https://github.com/divo12/OpenFlyWheel.git@6319f24 and forwards the model and Langfuse environment variables.

Written for commit 69177bd. Summary will update on new commits.

Review in cubic

@divo12 divo12 changed the title Codex/prepare workspace tool [codex] add portable workspace preparation tool Aug 28, 2026
@divo12
divo12 merged commit afc6309 into fresh Aug 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant