Reported by: jmrpineda
Requested by: repository maintainer
Priority: P2
Affected surfaces: VS Code execution settings panel, phase routing guidance
Constraints: Warning only; it must not block saving when the configured phase routing is otherwise valid.
Summary
Add a non-blocking warning in execution settings when the configured evaluator phase resolves to the same underlying model as its paired generator phase, so developers are reminded about shared-bias risk before saving phase routing.
Problem / opportunity
The orchestrated workflow allows per-phase model selection, which is one of its strongest practices for separating generation from evaluation. Today a developer can configure different agent or profile names that still resolve to the same underlying model, especially across spec and refinement or implementation and review, without receiving any guidance about the risk of self-confirming output.
Requested behavior
When Refinement and Spec, or Review and Implementation, resolve to the same underlying provider/model combination, the execution settings panel should surface a visible ⚠️ affordance on the evaluator phase. Clicking it should explain the shared-bias risk and recommend using a different evaluator model or provider family when possible. The warning must be advisory only and must not disable save by itself.
Scope
- In scope: Detect shared underlying model identity for evaluator/generator phase pairs, surface a clickable warning in the execution settings UI, and cover detection with automated tests.
- Out of scope: Blocking configuration, redefining repository-access rules, or redesigning the entire execution settings panel.
Acceptance criteria
- The execution settings panel detects when
refinement/spec or review/implementation resolve to the same underlying model even if they use different profile names.
- The evaluator phase shows a clickable
⚠️ button that reveals a recommendation about reducing shared-bias risk.
- The warning remains non-blocking and does not disable saving when no other validation errors exist.
Notes
- This feature should coexist with the current explicit validation rules for invalid phase routing and permission mismatches.
Reported by: jmrpineda
Requested by: repository maintainer
Priority: P2
Affected surfaces: VS Code execution settings panel, phase routing guidance
Constraints: Warning only; it must not block saving when the configured phase routing is otherwise valid.
Summary
Add a non-blocking warning in execution settings when the configured evaluator phase resolves to the same underlying model as its paired generator phase, so developers are reminded about shared-bias risk before saving phase routing.
Problem / opportunity
The orchestrated workflow allows per-phase model selection, which is one of its strongest practices for separating generation from evaluation. Today a developer can configure different agent or profile names that still resolve to the same underlying model, especially across
specandrefinementorimplementationandreview, without receiving any guidance about the risk of self-confirming output.Requested behavior
When
RefinementandSpec, orReviewandImplementation, resolve to the same underlying provider/model combination, the execution settings panel should surface a visible⚠️affordance on the evaluator phase. Clicking it should explain the shared-bias risk and recommend using a different evaluator model or provider family when possible. The warning must be advisory only and must not disable save by itself.Scope
Acceptance criteria
refinement/specorreview/implementationresolve to the same underlying model even if they use different profile names.⚠️button that reveals a recommendation about reducing shared-bias risk.Notes