Skip to content

[codex] add ITSM workspace initialization skill - #25

Merged
divo12 merged 5 commits into
freshfrom
codex/prepare-workspace-program
Aug 27, 2026
Merged

divo12 merged 5 commits into
freshfrom
codex/prepare-workspace-program

Conversation

@divo12

@divo12 divo12 commented Aug 27, 2026

Copy link
Copy Markdown
Owner

Problem

Installing the OpenFlywheel plugin currently opens with trace-query and outcome-recording prompts, but it does not guide a new user through attaching the primary agent harness they want to evaluate with ITSM-bench. Users have to rediscover the required experiment decisions and can accidentally start an expensive baseline before its goal, verifier, budget, or stopping rule is explicit.

Root cause

The plugin has focused skills for trace navigation and authoritative outcome recording, but no ITSM-bench onboarding skill or managed PROGRAM.md initialization contract. Its default prompt therefore begins after workspace preparation instead of establishing the harness, ITSM task manifest, verifier, observability identity, and optimization goal first.

Change

  • Add an implicitly discoverable, ITSM-bench-only workspace-init skill.
  • Require repository inspection before questions and collect experiment configuration one field at a time.
  • Reject other benchmark kinds instead of implying unsupported adapters or templates.
  • Require confirmation before writing files or starting a potentially costly baseline.
  • Add a byte-stable managed PROGRAM.md placeholder.
  • Add program_templates/base.md for the invariant evidence-first optimization loop, edit boundaries, gates, and stopping conditions.
  • Add program_templates/itsm.md for verifier-to-outcome mapping, bounded Langfuse trace search, ITSM constraints, and iteration reporting.
  • Define the re-entrant workspace_prepare handoff, including running, failed, and ready behavior and a duplicate-baseline stop rule.
  • Define the exact fresh-session handoff after preparation: read the populated program, treat the baseline as already recorded, and begin at failure analysis.
  • Update the plugin opening prompt and metadata so installation starts with ITSM workspace discovery and initialization.

User impact

A newly installed plugin now leads Codex through a consistent ITSM-bench onboarding conversation before the optimization loop. It does not request secrets, overwrite an existing different PROGRAM.md, rerun an existing baseline, advertise Tau/Terminal-Bench/BIRD support, or improvise shell execution when the preparation tool is unavailable.

Scope boundary

This PR adds the ITSM initialization skill, managed placeholder, and the two templates needed to compose the future program. It does not implement workspace_prepare, baseline execution, candidate execution, or gate tooling. The skill stops safely and reports that preparation is unavailable until that next vertical slice lands.

Verification

  • uv run ruff check src tests plugins/openflywheel/scripts/mcp_server.py
  • uv run mypy src tests plugins/openflywheel/scripts/mcp_server.py
  • uv run pytest --cov=ofw --cov-report=term-missing --cov-fail-under=90 -q — 116 passed, 2 skipped, 96.11% coverage
  • uv run pytest tests/test_openflywheel_mcp.py -q — 4 passed after template updates
  • Verified base.md and itsm.md exist and no unsupported benchmark/template names remain in the plugin
  • quick_validate.py plugins/openflywheel/skills/workspace-init
  • validate_plugin.py plugins/openflywheel
  • git diff --check

@divo12
divo12 marked this pull request as ready for review August 27, 2026 21:49
@divo12 divo12 changed the title [codex] add workspace initialization skill [codex] add ITSM workspace initialization skill Aug 27, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread plugins/openflywheel/program_templates/itsm.md Outdated
Comment thread plugins/openflywheel/program_templates/itsm.md Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread plugins/openflywheel/program_templates/itsm.md
@divo12
divo12 merged commit 8ee01c1 into fresh Aug 27, 2026
2 checks passed
@divo12
divo12 deleted the codex/prepare-workspace-program branch August 27, 2026 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant