Enhance documentation for advanced experiment runs#35320
Enhance documentation for advanced experiment runs#35320CFLJacquet wants to merge 1 commit intomasterfrom
Conversation
Updated description to specify running experiments on a subset of the dataset. Added a new section on running experiments on a subset of the dataset with examples.
Preview links (active after the
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dafdfb8433
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| Example | ||
| ``` | ||
| LLMObs.pull_dataset(dataset_name="my-dataset", tags=["env:prod", "version:1.0"]) |
There was a problem hiding this comment.
Assign filtered dataset before running experiment
The example calls LLMObs.pull_dataset(...) without storing its return value, so following these steps can leave users running LLMObs.experiment(..., dataset=dataset) with an unfiltered dataset variable from earlier code. In the subset-run workflow this defeats the intended tag filter and can unexpectedly execute (and bill for) the full dataset.
Useful? React with 👍 / 👎.
Updated description to specify running experiments on a subset of the dataset. Added a new section on running experiments on a subset of the dataset with examples.
What does this PR do? What is the motivation?
Merge instructions
Merge readiness:
For Datadog employees:
Your branch name MUST follow the
<name>/<description>convention and include the forward slash (/). Without this format, your pull request will not pass CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.If your branch doesn't follow this format, rename it or create a new branch and PR.
[6/5/2025] Merge queue has been disabled on the documentation repo. If you have write access to the repo, the PR has been reviewed by a Documentation team member, and all of the required checks have passed, you can use the Squash and Merge button to merge the PR. If you don't have write access, or you need help, reach out in the #documentation channel in Slack.
AI assistance
Additional notes