Skip to content

fix(studio): ask the human to settle open questions and assumptions instead of repairing the model - #207

Merged
yordis merged 1 commit into
TrogonStack:mainfrom
enilsen16:enilsen16/fix-studio-annotation-prompt
Oct 20, 2026
Merged

yordis merged 1 commit into
TrogonStack:mainfrom
enilsen16:enilsen16/fix-studio-annotation-prompt

Conversation

@enilsen16

@enilsen16 enilsen16 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Studio gives the same fix prompt for every validation finding. For OPEN_QUESTION_PRESENT and ASSUMPTION_PRESENT, the prompt tells the agent to "apply a targeted repair". Nothing in the model is broken for these codes. The prompt pushes the agent to change the model to clear the finding, when only a human can answer the question or confirm the assumption.

Change

  • For OPEN_QUESTION_PRESENT and ASSUMPTION_PRESENT, the prompt starts with "Resolve this validation finding" and replaces the repair instructions. It says that nothing in the model is broken and that the agent must not change behavior to clear the finding.
  • The agent asks the human for the answer or the confirmation and its source. Then it edits the annotation or removes it once resolved, as the server rule descriptions say. If the human cannot settle it yet, the annotation stays.
  • All other codes keep the repair prompt without change.

A resolve_annotation MCP tool with outcomes and annotation history is not part of this change.

Test

  • validation-context.test.ts: for both annotation codes, the prompt has the resolve wording and does not contain "targeted repair". For COMMAND_MULTIPLE_ISSUERS, the prompt still contains "targeted repair".
  • pnpm --filter trogon-atlas-studio test, typecheck and lint.

Closes #116


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…nstead of repairing the model

Signed-off-by: Erik Nilsen <enilsen16@live.com>
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1ac3068f-55bd-4604-8a07-e945cf4a454e
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@enilsen16
enilsen16 marked this pull request as ready for review October 20, 2026 15:19
@cursor

cursor Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Prompt-text-only change in Studio’s copy-fix flow; no server validation or model mutation logic.

Overview
Studio’s validationFixPrompt no longer tells agents to “repair” OPEN_QUESTION_PRESENT and ASSUMPTION_PRESENT findings. For those codes the prompt opens with “Resolve” (not “Fix”) and swaps in instructions that the model is fine, the agent must ask the human for an answer or confirmation, then edit or remove the annotation—without changing behavior or references to clear the rule.

Structural and other validation codes still get the existing “Fix” wording and the full targeted repair guidance (fetch missing defs, preserve causality, rerun validation, etc.). Implementation splits shared copy into annotationInstructions vs repairInstructions keyed by annotationCodes.

Tests assert both annotation codes get resolve/human wording and omit “targeted repair”, while codes like COMMAND_MULTIPLE_ISSUERS keep repair instructions.

Reviewed by Cursor Bugbot for commit 6cc98ad. Bugbot is set up for automated code reviews on this repo. Configure here.

@yordis
yordis merged commit 1245e4c into TrogonStack:main Oct 20, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A resolve action for assumption and open-question findings

2 participants