Success not reported if run with --repeat, uses @dataProvider and failed at first.#1
Open
PieterDC wants to merge 3 commits intoonly-repeat-failedfrom
Open
Conversation
…aratest exposes its test token.
…with --only-repeat-failed on a test with a dataprovider that failed the first run but succeeds a next run. "F.SSS" shows instead of "F.S.SS".
nomanpaddle
force-pushed
the
only-repeat-failed
branch
from
45fff3c to
316afa6
Compare
michel-gr
force-pushed
the
only-repeat-failed
branch
from
316afa6 to
b86e698
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While working on sebastianbergmann#1965 I discovered that when dataProvider is used with repeat, we can not rely on lastTestFailed from this class.
In ResultPrinter, after the test succeeded in the second run, lastTestFailed is on true while it should be false.
In TestResult, which calls the ResultPrinter listener, lastTestFailed is correctly false.
Does dataprovider mangle with that?
Added a test to a separate branch to uncover this bug.
The bug makes me think of sebastianbergmann#872 '--repeat breaks \PHPUnit_Framework_MockObject_Matcher_InvokedCount' because it's also some kind of count ...
Didn't have time to invest this further, our priority is on sebastianbergmann#1965.
The following workaround doesn't suffice although the "failed status" of the test looked more trustworthy than the one from PHPUnit_TextUI_ResultPrinter while step debugging:
It makes a bunch of already existing tests fail. So, it needs work.