[codex] add ITSM workspace initialization skill - #25
Merged
Merged
Conversation
divo12
marked this pull request as ready for review
August 27, 2026 21:49
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Installing the OpenFlywheel plugin currently opens with trace-query and outcome-recording prompts, but it does not guide a new user through attaching the primary agent harness they want to evaluate with ITSM-bench. Users have to rediscover the required experiment decisions and can accidentally start an expensive baseline before its goal, verifier, budget, or stopping rule is explicit.
Root cause
The plugin has focused skills for trace navigation and authoritative outcome recording, but no ITSM-bench onboarding skill or managed
PROGRAM.mdinitialization contract. Its default prompt therefore begins after workspace preparation instead of establishing the harness, ITSM task manifest, verifier, observability identity, and optimization goal first.Change
workspace-initskill.PROGRAM.mdplaceholder.program_templates/base.mdfor the invariant evidence-first optimization loop, edit boundaries, gates, and stopping conditions.program_templates/itsm.mdfor verifier-to-outcome mapping, bounded Langfuse trace search, ITSM constraints, and iteration reporting.workspace_preparehandoff, including running, failed, and ready behavior and a duplicate-baseline stop rule.User impact
A newly installed plugin now leads Codex through a consistent ITSM-bench onboarding conversation before the optimization loop. It does not request secrets, overwrite an existing different
PROGRAM.md, rerun an existing baseline, advertise Tau/Terminal-Bench/BIRD support, or improvise shell execution when the preparation tool is unavailable.Scope boundary
This PR adds the ITSM initialization skill, managed placeholder, and the two templates needed to compose the future program. It does not implement
workspace_prepare, baseline execution, candidate execution, or gate tooling. The skill stops safely and reports that preparation is unavailable until that next vertical slice lands.Verification
uv run ruff check src tests plugins/openflywheel/scripts/mcp_server.pyuv run mypy src tests plugins/openflywheel/scripts/mcp_server.pyuv run pytest --cov=ofw --cov-report=term-missing --cov-fail-under=90 -q— 116 passed, 2 skipped, 96.11% coverageuv run pytest tests/test_openflywheel_mcp.py -q— 4 passed after template updatesbase.mdanditsm.mdexist and no unsupported benchmark/template names remain in the pluginquick_validate.py plugins/openflywheel/skills/workspace-initvalidate_plugin.py plugins/openflywheelgit diff --check