Skip to content

Conversation

@zhichli
Copy link
Member

@zhichli zhichli commented Dec 20, 2025

Introduce additional properties to handle error states in language model tool results. This enhancement allows for better communication of errors and messages from the tool, improving overall robustness and user experience.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds error handling properties (toolResultError and hasError) to language model tool results, enabling better communication of error states between tools and the chat system. The changes span the API definition layer, type conversion utilities, service interfaces, and the main thread implementation.

Key Changes

  • Added toolResultError string property to ExtendedLanguageModelToolResult API
  • Added hasError boolean flag to IToolResult interface and API
  • Updated type converters to handle the new error properties in both directions (to/from extension API)

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

File Description
src/vscode-dts/vscode.proposed.chatParticipantPrivate.d.ts Added toolResultError property to ExtendedLanguageModelToolResult API definition
src/vs/workbench/contrib/chat/common/languageModelToolsService.ts Added hasError property to IToolResult interface
src/vs/workbench/api/common/extHostTypeConverters.ts Updated LanguageModelToolResult and LanguageModelToolResult2 converters to handle toolResultError, toolResultMessage, toolMetadata, and hasError properties
src/vs/workbench/api/browser/mainThreadLanguageModelTools.ts Updated $invokeTool to pass toolResultMessage, toolResultError, toolMetadata, and hasError to extension host

/** Whether to ask the user to confirm these tool results. Overrides {@link IToolConfirmationMessages.confirmResults}. */
confirmResults?: boolean;
/** Whether there was an error calling the tool. The tool may still have partially succeeded. */
hasError?: boolean;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused about how hasError/toolResultError are used today, are those redundant? The tool can also throw.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, hasError might be only used for this but could be useful to cover other tool calls' failure cases later ?

toolResultError is logged here when tool call throws, does it mean the same as you mentioned by The tool can also throw.?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is ok, we can run it past @connor4312 that it is correct and useful to share these props with the extension for logging from eval runs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants