There is a desire to have a widget or view that allows for the creation of data aggregation related to flaky test failures.
Using the existing mechanisms for test history aggregation, this view would identify test cases that have a result that flips between passing and not.
The view should first aggregate for test results matching a given set of metrics. For example, component + assignee + environment, or jenkins meta.
Then, isolate test cases that have flipping states within a given window of test result history. For example, tests that within the last 10 executions have more than two failures.
There is a desire to have a widget or view that allows for the creation of data aggregation related to flaky test failures.
Using the existing mechanisms for test history aggregation, this view would identify test cases that have a result that flips between passing and not.
The view should first aggregate for test results matching a given set of metrics. For example, component + assignee + environment, or jenkins meta.
Then, isolate test cases that have flipping states within a given window of test result history. For example, tests that within the last 10 executions have more than two failures.