Repository navigation
[bot] Merge master/adcbd45f into rel/dev - #1969
Merged
Merged
Conversation
When the agent asked which of several objects to use, the visualization and alert simulated users sometimes named the one the fixture does not expect, and the agent built what it was told: Order date vs Return date, new_customers vs new_customers_monthly, of_orders vs orders_all_time. Visualization: the goal's date filter gave only its period, never its date dataset, so a choice between Order date and Return date had no answer in the prompt. It now says "date filter on the return date". The prompt also tells the simulated user to choose the alternative that matches its goal and repeat it as the agent wrote it, and that a date attribute such as order_date.year belongs to the order date. Objects stay named by title: giving them by reference made the simulated user volunteer references to the agent. Alert: the goal has no filters, and rule 5 says "no filters ... all time", which matched an offered metric described as ignoring filters and counting all orders. The prompt now tells the simulated user to choose exactly the goal's metric and that such a metric is a different one. Replayed the three recorded agent messages 20 times each against the real models: all correct, and no reference volunteered to a chart type, open or period question. Models, scoring and gates are unchanged. jira: QA-29478 risk: low Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(gooddata-eval): sim user prompt names the date dataset and pick
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## rel/dev #1969 +/- ##
===========================================
+ Coverage 87.07% 87.18% +0.10%
===========================================
Files 355 355
Lines 28052 28055 +3
===========================================
+ Hits 24427 24459 +32
+ Misses 3625 3596 -29 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🚀 Automated PR to perform merge from master into rel/dev with changes up to adcbd45 (created by https://github.com/gooddata/gooddata-python-sdk/actions/runs/44474086929).