Repository navigation
feat(mcp): Tune tool descriptions for harness search, hide deprecated tools - #1906
Merged
Merged
Conversation
… tools With EXPOSE_ALL_TOOLS on, harness tool search (Codex BM25, Claude Code ToolSearch, Cursor, VS Code) picks tools by name and description. Hide the deprecated get_ai_conversation_details, search_ai_conversations, and search_events from tools/list so search does not return them next to their replacements. They stay reachable through execute_sentry_tool when the flag is off. Reword first lines that missed common query terms or blurred similar tools: "List or search" for find_* tools, DSN client keys, cron vs. uptime vs. metric monitors, aggregated vs. single profiles, alert notification channels, and Sentry links for get_sentry_resource. Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
dcramer
reviewed
Nov 3, 2026
Replace the DEPRECATED_TOOL_NAMES list in surfaces.ts with a
`deprecated: { replacedBy: [...] }` field on each tool. isTopLevelTool
keeps deprecated tools out of tools/list; execute_sentry_tool still
reaches them when EXPOSE_ALL_TOOLS is off. toolDefinitions.json carries
the field, and a test checks that each replacement exists and is not
itself deprecated.
Co-Authored-By: David Cramer <david@sentry.io>
dcramer
approved these changes
Nov 3, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1868. Now that every tool is listed directly, the harness picks tools with its own search over tool names and descriptions. Codex uses local BM25 and returns 8 matches. VS Code returns its top 5 by embeddings. Claude Code and Cursor show the model only tool names until it searches. So the metadata has to match the words people actually use, and must not return two tools for the same job.
Deprecated tools are no longer listed
get_ai_conversation_details,search_ai_conversationsandsearch_eventsare deprecated, but they were intools/listnext to the tools that replaced them. Each one now has adeprecated: { replacedBy: [...] }field on its tool config that names its replacements, and deprecated tools aren't listed intools/list.toolDefinitions.jsoncarries the same field.They were only ever reachable through
execute_sentry_tool(name: ...), and that tool isn't registered with the flag on. So hiding them doesn't break any working call. With the flag off they stay reachable through the gateway, just as before.Description audit
I checked about 56 realistic queries against a BM25 index of the tool names and descriptions, for example "sentry client key", "list projects", "notify slack when new issue" and "cron job failed check in". Before this change, 7 of them didn't put the right tool in the top 3. After it, 2 still miss the top 3, and both are in the top 8 that Codex returns. This was a scratch approximation of Codex's search, not the real harness, so the results are directional. Only first lines and pointers to sibling tools changed:
find_projects,find_teams,find_organizations,find_releases: they now start with "List or search…", because "list X" is the most common way to ask and none of them contained the word.find_dsns: mentions client keys,SENTRY_DSNand SDK setup.find_monitorsandget_monitor_detailssay "cron / scheduled jobs / check-ins".find_monitorsnow points tofind_uptime_monitorsandfind_metric_monitors, since all three names look alike.create_metric_monitor: now says what a monitor does in plain words: it detects when an error count or latency crosses a threshold.create_alert_rule: names its notification channels: Slack, email and PagerDuty.get_profile/get_profile_details:get_profileis for aggregated flamegraph data across a transaction, andget_profile_detailsis for one profile by ID or URL. Each now points to the other.get_event_stacktrace: says "stack trace" as well as "stacktrace".get_sentry_resource: says to use it when a user shares a Sentry link.docs/contributing/adding-tools.mdnow covers writing descriptions for harness search, plus thedeprecatedfield.Not in this PR
_meta["anthropic/alwaysLoad"]on core tools.Checks:
tsc,lint, and the logger, toolkit-core, local, docs-mcp, mcp-core, mcp-test-client and mcp-cloudflare test suites pass. Thepackages/clisuite didn't finish because the sandbox timed out. This PR doesn't touch CLI code.--
View Session [Sentry]