-
Notifications
You must be signed in to change notification settings - Fork 851
Add server tool call support to OpenTelemetryChatClient per semantic conventions #7240
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Per OpenTelemetry semantic conventions, add support for: - CodeInterpreterToolCallContent → server_tool_call (type=code_interpreter) - CodeInterpreterToolResultContent → server_tool_call_response (type=code_interpreter) - ImageGenerationToolCallContent → server_tool_call (type=image_generation) - ImageGenerationToolResultContent → server_tool_call_response (type=image_generation) - McpServerToolCallContent → server_tool_call (type=mcp) - McpServerToolResultContent → server_tool_call_response (type=mcp) - McpServerToolApprovalRequestContent → server_tool_call (type=mcp_approval_request) - McpServerToolApprovalResponseContent → server_tool_call_response (type=mcp_approval_response) Add new Otel types: - OtelServerToolCallPart - OtelServerToolCallResponsePart - OtelGenericServerToolCall - OtelGenericServerToolCallResponse Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
Change OtelGenericServerToolCall.Arguments from IDictionary to IReadOnlyDictionary to avoid potential null results when the source is a read-only dictionary implementation. Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs
Outdated
Show resolved
Hide resolved
…t type Per code review feedback: - Replace generic OtelGenericServerToolCall and OtelGenericServerToolCallResponse with specialized classes for each content type - Create OtelCodeInterpreterToolCall with "code" field (extracts code from DataContent/TextContent) - Create OtelCodeInterpreterToolCallResponse with "output" field - Create OtelImageGenerationToolCall and OtelImageGenerationToolCallResponse - Create OtelMcpToolCall and OtelMcpToolCallResponse with server_name and arguments - Create OtelMcpApprovalRequest and OtelMcpApprovalResponse with approved field - Use generic OtelServerToolCallPart<T> and OtelServerToolCallResponsePart<T> for type safety Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
| // Check for DataContent with code MIME types | ||
| if (input is DataContent dc && | ||
| dc.MediaType is not null && | ||
| (dc.MediaType.StartsWith("text/", StringComparison.OrdinalIgnoreCase) || |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot this can use dc.HasTopLevelMediaType
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 07050f9. Now using dc.HasTopLevelMediaType("text") instead of StartsWith.
| if (input is DataContent dc && | ||
| dc.MediaType is not null && | ||
| (dc.MediaType.StartsWith("text/", StringComparison.OrdinalIgnoreCase) || | ||
| dc.MediaType.Equals("application/x-python-code", StringComparison.OrdinalIgnoreCase))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot this should be text/x-python, but since that would be covered by the first case, you can instead just remove this condition
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed in 07050f9.
| /// </remarks> | ||
| private static string? ExtractCodeFromInputs(IList<AIContent>? inputs) | ||
| { | ||
| if (inputs is null) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot invert this so as to have only one return null at the end
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 07050f9. Inverted to if (inputs is not null) with single return null at end.
| #pragma warning disable EXTEXP0018 // Type is for evaluation purposes only | ||
| #pragma warning disable EXTEXP0019 // Type is for evaluation purposes only |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot why do you need these?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They weren't needed - the test project already suppresses MEAI001 (the actual diagnostic ID for AI experiments) in NoWarn. I mistakenly added EXTEXP0018/EXTEXP0019 which aren't used. Removed in 07050f9.
- Use dc.HasTopLevelMediaType("text") instead of StartsWith
- Remove redundant application/x-python-code condition
- Invert null check to have single return null at end
- Remove unnecessary pragma warnings (MEAI001 already suppressed in test project)
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR extends OpenTelemetryChatClient to support new OpenTelemetry semantic conventions for server tool calls, covering code interpreter, image generation, and MCP server tooling, and updates tests accordingly. It introduces specialized OTEL serialization types and a helper to extract code interpreter input, ensuring both streaming and non-streaming paths emit the expected OTEL JSON.
Changes:
- Map
CodeInterpreterToolCallContent,CodeInterpreterToolResultContent,ImageGenerationToolCallContent,ImageGenerationToolResultContent,McpServerToolCallContent,McpServerToolResultContent,McpServerToolApprovalRequestContent, andMcpServerToolApprovalResponseContentinto OTELserver_tool_call/server_tool_call_responseparts with dedicated internal DTOs. - Add
ExtractCodeFromInputsto derive thecodefield for code interpreter calls fromAIContentinputs and register the new OTEL DTOs withJsonSerializablefor source-generated serialization. - Add tests to verify the new server tool call and MCP approval content types serialize to the expected OTEL JSON for both single-shot and streaming chat flows.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs |
Adds server tool call / response part types, content-to-OTEL mapping for new server tool content types, the ExtractCodeFromInputs helper, and updates OTEL JSON source-generation registrations. |
test/Libraries/Microsoft.Extensions.AI.Tests/ChatCompletion/OpenTelemetryChatClientTests.cs |
Adds tests to assert correct OTEL JSON for server tool calls/results (code interpreter, image generation, MCP) and MCP approval request/response content, for both streaming and non-streaming scenarios. |
Comments suppressed due to low confidence (1)
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs:493
ExtractCodeFromInputshandles bothDataContent(by decodingdc.Datawhen the media type istext/*orapplication/x-python-code) andTextContent, but the new tests exercise only theTextContentcode path. To reduce the risk of regressions and to validate the primaryDataContent-based scenario used by the OpenAI clients (which populateInputswithDataContentusing a"text/x-python"media type), consider adding a test that creates aCodeInterpreterToolCallContentwith aDataContentinput and verifies that the serializedserver_tool_call.server_tool_call.codefield contains the expected decoded code string.
private static string? ExtractCodeFromInputs(IList<AIContent>? inputs)
{
if (inputs is not null)
{
foreach (var input in inputs)
{
// Check for DataContent with text MIME types
if (input is DataContent dc && dc.HasTopLevelMediaType("text"))
{
// Return the data as a string (decode bytes as UTF8)
return Encoding.UTF8.GetString(dc.Data.ToArray());
}
// Check for TextContent
if (input is TextContent tc && !string.IsNullOrEmpty(tc.Text))
{
return tc.Text;
}
}
}
return null;
}
/// <summary>Creates an activity for a chat request, or returns <see langword="null"/> if not enabled.</summary>
private Activity? CreateAndConfigureActivity(ChatOptions? options)
{
Activity? activity = null;
Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.
Microsoft Reviewers: Open in CodeFlow