Cli install sdk evals#85
Merged
ViaDézo1er / cedric (viadezo1er) merged 11 commits intomainfrom Apr 1, 2026
Merged
Conversation
ViaDézo1er / cedric (viadezo1er)
force-pushed
the
cedric/cli-install-sdk-evals
branch
from
8dc2661 to
b68b629
Compare
ViaDézo1er / cedric (viadezo1er)
requested review from
Abhijeet Prasad (AbhiPrasad) and
Olmo Maldonado (ibolmo)
and removed request for
Olmo Maldonado (ibolmo)
Adds an optional, repeatable `--language` flag to `bt setup instrument`
that lets callers specify the target language(s) directly, bypassing the
agent's language auto-detection step.
Accepted values (case-insensitive):
python, typescript, javascript, go, csharp, c#, java, ruby
`typescript` and `javascript` are treated as the same language;
duplicate values are deduplicated before being passed to the agent.
When one or more languages are provided the rendered task prompt
includes a "Language Override" section telling the agent to skip Step 2
(auto-detection) and instrument the specified language(s) directly.
Also fixes a pre-existing compile error in tests where
`render_instrument_task` was already called with a `workflows` argument
that the implementation didn't accept, and adds the `{WORKFLOW_CONTEXT}`
placeholder so non-instrument workflows inject `bt` CLI guidance.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ViaDézo1er / cedric (viadezo1er)
force-pushed
the
cedric/cli-install-sdk-evals
branch
from
b68b629 to
bcec472
Compare
Contributor
Author
|
CI passes after |
|
Latest downloadable build artifacts for this PR commit
Available artifact names
|
ViaDézo1er / cedric (viadezo1er)
requested a review
from Parker Henderson (parkerhendo)
Abhijeet Prasad (AbhiPrasad)
approved these changes
Mar 31, 2026
Member
Abhijeet Prasad (AbhiPrasad)
left a comment
Abhijeet Prasad (AbhiPrasad)
left a comment
There was a problem hiding this comment.
let's give this a try!
Comment on lines
+1056
to
+1061
| 1 => Some(LanguageArg::Python), | ||
| 2 => Some(LanguageArg::TypeScript), | ||
| 3 => Some(LanguageArg::Go), | ||
| 4 => Some(LanguageArg::Java), | ||
| 5 => Some(LanguageArg::Ruby), | ||
| 6 => Some(LanguageArg::CSharp), |
There was a problem hiding this comment.
can we avoid the magic indices?
| If the SDK does not print a URL, construct one manually using the URL format documented in `{SDK_INSTALL_DIR}/braintrust-url-formats.md`: | ||
|
|
||
| ``` | ||
| https://www.braintrust.dev/app/{org}/p/{project_name}/logs?r={root_span_id} |
There was a problem hiding this comment.
The URL might be different for self hosted.
They should use BRAINTRUST_APP_URL to help construct the URL.
Andrew Kent (realark)
approved these changes
Mar 31, 2026
…trument when possible
fixed cargo-clippy warning
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bt setup instrument: interactive mode, language selection, and scoped permissions
Adds three new flags to bt setup instrument and wires them end-to-end through agent invocation and task generation.
--interactive / -iopens the agent in its interactive TUI (Claude Code, etc.) so the user can review and approve each tool use.--yoloruns the agent in the background with bypassPermissions — no approval prompts.--language <LANG>restricts instrumentation to specific languages (python, typescript, go, java, ruby, csharp); repeatable; omit to let the agent auto-detect.Run-mode prompt (interactive terminal, no flags)
When none of the above flags are passed and the terminal is interactive, the user is asked how to run the agent. Background mode uses acceptEdits with --allowedTools scoped to the package managers for the selected language(s) only (e.g. uv for Python, npm/yarn/pnpm for TypeScript, dotnet for C#). Interactive TUI mode opens the agent's terminal UI.
Language selection prompt
A multi-select prompt is shown between the workflow and run-mode prompts. Selecting "All languages" (the default) lets the agent auto-detect; selecting specific languages also narrows the background tool allowlist.
You'll notice since the mcp can be setup (and is the default option) when using bt setup, adding the mcp resources to this repo can make them redondant. However they are needed in case the mcp isn't setup.