feat: add prompt improvement feature for low-scoring test cases #1266
Conversation
ViaSocket-Git
commented
Nov 19, 2025
ViaSocket-Git
commented
- Added "Better Prompt" action button in test case version table for scores below 0.7
- Integrated improvePrompt API call with conversation history and expected response
- Implemented loading state management for prompt improvement operations
- Added HistoryPagePromptUpdateModal to display and apply improved prompts
- Included current prompt from bridge version configuration in selector
- Added Action column to test case version table with
- Added "Better Prompt" action button in test case version table for scores below 0.7 - Integrated improvePrompt API call with conversation history and expected response - Implemented loading state management for prompt improvement operations - Added HistoryPagePromptUpdateModal to display and apply improved prompts - Included current prompt from bridge version configuration in selector - Added Action column to test case version table with
Deploying ai-middleware with
|
| Latest commit: |
e901301
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://87c8697c.ai-middleware.pages.dev |
| Branch Preview URL: | https://testcaseprompt-update.ai-middleware.pages.dev |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const handleBetterPrompt = async (testCase, versionId) => { | ||
| const promptKey = `${testCase._id}-${versionId}`; | ||
| setImprovingPrompts(prev => new Set([...prev, promptKey])); | ||
|
|
||
| try { | ||
| const variables = {}; | ||
|
|
||
| // Get the conversation from test case | ||
| const conversation = testCase.conversation || []; | ||
|
|
||
| variables['prompt'] = currentPrompt; | ||
| // Add the model output as assistant response | ||
|
|
||
| variables["conversation_history"] = conversation; | ||
| variables["updated_response"] = testCase.expected?.response || ''; | ||
|
|
||
| const data = await improvePrompt(variables); | ||
|
|
||
| if (data?.updated_prompt) { | ||
| setPromptToUpdate(data.updated_prompt); | ||
| openModal(MODAL_TYPE.HISTORY_PAGE_PROMPT_UPDATE_MODAL); | ||
| } |
There was a problem hiding this comment.
handleBetterPrompt receives the row's versionId but never uses it—currentPrompt and resolvedParams.version are used when calling improvePrompt and when showing the modal. Because the action button is rendered for every entry in the version history table, clicking it for a non‑current version will still generate and apply a prompt update to the currently selected version, potentially overwriting the wrong configuration. The prompt and version should be looked up from versionId so each row updates its own version.
Useful? React with 👍 / 👎.
