Show "experimental" badges around ML functionality#1
Conversation
|
This is a benchmark review for experiment This pull request was cloned from Experiment configurationreview_config:
# User configuration for the review
# - benchmark - use the user config from the benchmark reviews
# - <value> - use the value directly
user_review_config:
enable_ai_review: true
enable_rule_comments: false
enable_complexity_comments: benchmark
enable_security_comments: benchmark
enable_tests_comments: benchmark
enable_comment_suggestions: benchmark
enable_pull_request_summary: benchmark
enable_review_guide: benchmark
enable_approvals: false
base_branches: [base-sha.*]
ai_review_config:
# The model responses to use for the experiment
# - benchmark - use the model responses from the benchmark reviews
# - llm - call the language model to generate responses
model_responses:
comments_model: benchmark
comment_validation_model: benchmark
comment_suggestion_model: benchmark
complexity_model: benchmark
security_model: benchmark
tests_model: benchmark
pull_request_summary_model: benchmark
review_guide_model: benchmark
overall_comments_model: benchmark
# The pull request dataset to run the experiment on
pull_request_dataset:
# CodeRabbit
- https://github.com/neerajkumar161/node-coveralls-integration/pull/5
- https://github.com/gunner95/vertx-rest/pull/1
- https://github.com/Altinn/altinn-access-management-frontend/pull/1427
- https://github.com/theMr17/github-notifier/pull/14
- https://github.com/bearycool11/AI_memory_Loops/pull/142
# Greptile
- https://github.com/gumloop/guMCP/pull/119
- https://github.com/autoblocksai/python-sdk/pull/335
- https://github.com/grepdemos/ImageSharp/pull/6
- https://github.com/grepdemos/server/pull/61
- https://github.com/websentry-ai/pipelines/pull/25
# Graphite
- https://github.com/KittyCAD/modeling-app/pull/6648
- https://github.com/KittyCAD/modeling-app/pull/6628
- https://github.com/Varedis-Org/AI-Test-Repo/pull/2
- https://github.com/deeep-network/bedrock/pull/198
- https://github.com/Metta-AI/metta/pull/277
# Copilot
- https://github.com/hmcts/rpx-xui-webapp/pull/4438
- https://github.com/ganchdev/quez/pull/104
- https://github.com/xbcsmith/ymlfxr/pull/13
- https://github.com/tinapayy/B-1N1T/pull/36
- https://github.com/coder/devcontainer-features/pull/6
# Questions to ask to label the review comments
review_comment_labels: []
# - label: correct
# question: Is this comment correct?
# Benchmark reviews generated by running
# python -m scripts.experiment benchmark <experiment_name>
benchmark_reviews: []
|
Reviewer's GuideThis PR introduces an 'experimental' status for features, primarily targeting ML functionality, controlled by a new global constant File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Hellebore
left a comment
There was a problem hiding this comment.
Hey @Hellebore - I've reviewed your changes - here's some feedback:
- Consider extracting the 'Experimental' badge UI (icon and text) into a reusable component, as it's implemented similarly in multiple places (Toolbar tooltips, Home button tooltip, CommandComboBox, ActionButtonDropdown).
- The logic checking if an item status is 'available' or 'experimental' is repeated in several locations within Toolbar.tsx; consider abstracting this into a helper function or consolidating the check.
Here's what I looked at during the review
- 🟡 General issues: 1 issue found
- 🟢 Security: all looks good
- 🟢 Testing: all looks good
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| {displayHotkeys(hotkey)} | ||
| </kbd> | ||
| {status === 'experimental' && ( | ||
| <div className="text-xs flex justify-center item-center gap-1 pb-1 border-b border-chalkboard-50"> |
There was a problem hiding this comment.
issue (typo): Possible typo in flex alignment class.
Tailwind's class is 'items-center', not 'item-center'. Please update it.
| <div className="text-xs flex justify-center item-center gap-1 pb-1 border-b border-chalkboard-50"> | |
| <div className="text-xs flex justify-center items-center gap-1 pb-1 border-b border-chalkboard-50"> |
We want users to have realistic expectations about our ML-powered features, so we're going to add a little UI tag to remind them that we're working to improve the model and it will get better over time.
Summary by Sourcery
Add "experimental" badges and status to ML-powered features to set user expectations about ongoing improvements
New Features:
Enhancements:
Chores: