Repository navigation
[bot] Merge master/a607c7aa into rel/dev - #1964
Merged
Merged
Conversation
A user who asks for a new chart from an open dashboard without naming a
period may get the dashboard's date filter on the chart, and the product
accepts that as well as an unfiltered chart. The strict date comparison
failed it, which made up most failures of the _ctx visualization
datasets.
The item now says so with expected_output.date_filter_from_context:
true, or {"dataset": "<date dataset>"}. Each candidate without a date
filter is then also scored once per date filter in
user_context.view.dashboard.filters, with that filter added; the
comparison itself is check_filters, unchanged. The dataset comes from
the key, else the dashboard filter's `using` (a bare id gets the
dataset/ prefix), else, for the common date filter, the chart's only
date filter when it is on a dataset/ URI. The key's dataset must be a
bare id or a dataset/ URI, so a typo is a load error. A candidate stating a date filter is not
extended.
A per-item key rather than a run-wide flag: the verdict stays a function
of the dataset item, an item that must reject the dashboard's period
leaves the key out, and the Tavern shim gets it through the expectation
it already relays, with no environment to set.
A derived candidate replaces the best stated one only when it ranks
higher, or ties with it and matches the date it misses, so a chart
failing elsewhere is not also blamed for the date. Among stated
candidates that tie, one that matched the date is preferred, so it keeps
winning over a derived one. The candidate reported
is always the stated one; the detail records filter_date_context and
the failure summary marks the date. Items without the key score exactly
as before.
The single-shot runner now calls an optional Evaluator.validate before
asking the agent, and the visualization evaluator checks its expectation
there. A malformed item, or the key on an item whose context has no
dashboard date filter, errors before any request on every path instead
of after a spent chat turn.
jira: QA-30214
risk: low
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(gooddata-eval): per-item dashboard date filter in viz scoring (QA-30214)
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## rel/dev #1964 +/- ##
===========================================
+ Coverage 86.91% 86.96% +0.04%
===========================================
Files 352 352
Lines 27694 27781 +87
===========================================
+ Hits 24070 24159 +89
+ Misses 3624 3622 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🚀 Automated PR to perform merge from master into rel/dev with changes up to a607c7a (created by https://github.com/gooddata/gooddata-python-sdk/actions/runs/44249189296).