fix(agent): block interface schema agent to request future schema.#1124
fix(agent): block interface schema agent to request future schema.#1124
Conversation
samchon
added
bug
There was a problem hiding this comment.
Pull request overview
Updates AutoBE agent prompts and configuration to prevent agents from requesting interface schemas for types that do not yet exist (e.g., future $ref targets), and reduces the default validation retry count to lower latency while maintaining correction capability.
Changes:
- Reduced default validation retry attempts from 8 → 5 via
AutoBeConfigConstant.VALIDATION_RETRY. - Added consistent prompt guidance across interface-schema agents:
getInterfaceSchemasreturns only existing schemas; do not request types that are being newly created via$refand do not retry “non-existent schema” failures. - Extended/standardized preliminary-request checklists to explicitly prohibit requesting non-existent schemas.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/agent/src/constants/AutoBeConfigConstant.ts | Lowers default validation retry count to reduce correction-loop latency. |
| packages/agent/prompts/INTERFACE_SCHEMA_SECURITY_REVIEW.md | Adds explicit getInterfaceSchemas usage rules and checklist reminder. |
| packages/agent/prompts/INTERFACE_SCHEMA_RELATION_REVIEW.md | Adds getInterfaceSchemas “existing-only” rules + checklist item. |
| packages/agent/prompts/INTERFACE_SCHEMA_REFINE.md | Adds getInterfaceSchemas “existing-only” rules + checklist item. |
| packages/agent/prompts/INTERFACE_SCHEMA_CONTENT_REVIEW.md | Splits prelim request guidance into bullets and adds “existing-only” rule + checklist item. |
| packages/agent/prompts/INTERFACE_SCHEMA_COMPLEMENT.md | Reworks prelim guidance bullets and adds “existing-only” rule + checklist item. |
| packages/agent/prompts/INTERFACE_SCHEMA_CASTING.md | Splits prelim request guidance into bullets and adds “existing-only” rule + checklist item. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - NEVER re-request already loaded materials — empty array = exhausted | ||
| - `getInterfaceSchemas` only returns existing schemas | ||
| - NEVER request a type you intend to newly create via `$ref` — it does not exist yet | ||
| - If the call fails with "non-existing", the failure is correct — do not retry |
There was a problem hiding this comment.
This bullet mentions the call failing with "non-existing", but the validator’s error message for unknown interface schemas is phrased as "NON-EXISTING interface schema name" (not "non-existing"). Consider updating this wording (or making it substring-agnostic) so agents can reliably map the instruction to the real feedback.
| - If the call fails with "non-existing", the failure is correct — do not retry | |
| - If the call fails with an error containing "NON-EXISTING interface schema name", the failure is correct — do not retry |
| - Never re-request loaded materials | ||
| - `getInterfaceSchemas` only returns existing schemas | ||
| - NEVER request a type you intend to newly create via `$ref` — it does not exist yet | ||
| - If the call fails with "non-existing", the failure is correct — do not retry |
There was a problem hiding this comment.
The prompt anchors on an error string "non-existing" for getInterfaceSchemas, but the system validation error is worded as "NON-EXISTING interface schema name". Adjusting this (or avoiding quoting a specific substring) will keep the prompt aligned with the actual validator feedback.
| - If the call fails with "non-existing", the failure is correct — do not retry | |
| - If the call fails because the schema does not exist, the failure is correct — do not retry |
| - Empty array response → That type exhausted, move to complete | ||
| - `getInterfaceSchemas` only returns existing schemas | ||
| - NEVER request a type you intend to newly create via `$ref` — it does not exist yet | ||
| - If the call fails with "non-existing", the failure is correct — do not retry |
There was a problem hiding this comment.
This instruction references an error string "non-existing". In code, invalid getInterfaceSchemas requests produce a validation error that starts with "You've referenced a NON-EXISTING interface schema name". Consider updating the wording (or generalizing it) to match the actual feedback agents will see.
| - If the call fails with "non-existing", the failure is correct — do not retry | |
| - If the call fails with a validation error about a non-existing interface schema name (e.g., "You've referenced a NON-EXISTING interface schema name"), the failure is correct — do not retry |
| - Never re-request loaded materials | ||
| - `getInterfaceSchemas` only returns existing schemas | ||
| - NEVER request a type you intend to newly create via `$ref` — it does not exist yet | ||
| - If the call fails with "non-existing", the failure is correct — do not retry |
There was a problem hiding this comment.
The prompt says the call fails with "non-existing", but the actual validator message for unknown schema names is "NON-EXISTING interface schema name". Please update the quoted wording (or avoid quoting a specific token) so the instruction matches the feedback agents receive.
| - If the call fails with "non-existing", the failure is correct — do not retry | |
| - If the call fails with a "NON-EXISTING interface schema name" error, the failure is correct — do not retry |
| - Never re-request loaded materials | ||
| - `getInterfaceSchemas` only returns existing schemas | ||
| - NEVER request a type you intend to newly create via `$ref` — it does not exist yet | ||
| - If the call fails with "non-existing", the failure is correct — do not retry |
There was a problem hiding this comment.
The prompt states the call fails with "non-existing", but the underlying validation error for unknown interface schema names is worded as "NON-EXISTING interface schema name". Consider aligning this wording (or making it more general) so agents don’t look for a string that won’t appear verbatim.
| - If the call fails with "non-existing", the failure is correct — do not retry | |
| - If the call fails with an error indicating a non-existing interface schema name (for example, "NON-EXISTING interface schema name"), the failure is correct — do not retry |
1 participant
This pull request updates the agent prompt documentation and configuration to clarify and enforce the correct use of the
getInterfaceSchemasrequest, ensuring agents do not attempt to fetch schemas for types that do not yet exist. It also reduces the validation retry limit for schema validation. The changes improve prompt accuracy, prevent unnecessary or erroneous requests, and clarify the agent workflow for handling$reftargets.Prompt documentation improvements:
Clarified that
getInterfaceSchemasonly returns existing schemas, and agents should never request types they intend to create via$ref; if a type does not exist, the failure is expected and should not be retried. This guidance was added to all relevant prompt markdown files, includingINTERFACE_SCHEMA_CASTING.md,INTERFACE_SCHEMA_COMPLEMENT.md,INTERFACE_SCHEMA_CONTENT_REVIEW.md,INTERFACE_SCHEMA_REFINE.md, andINTERFACE_SCHEMA_RELATION_REVIEW.md. [1] [2] [3] [4] [5]Updated checklist sections in the prompt files to add explicit reminders not to call
getInterfaceSchemasfor types that do not yet exist or are being created. [1] [2] [3] [4] [5]Added a note to
INTERFACE_SCHEMA_SECURITY_REVIEW.mdclarifying that$refdoes not require the target type to exist, as another agent will create missing targets later, and updated the checklist accordingly. [1] [2]Configuration change:
VALIDATION_RETRYconstant from 8 to 5 inAutoBeConfigConstant, reflecting a new balance between validation robustness and latency.