Describe the bug
The results being logged into the terminal by the result handler, sometimes misses out on the result of the last run even though the test ran completely. For example:
✔ Param Handling Success.
✔ Test List built
✔ Log dir creation successful.
✔ Environment setup successful.
✔ Running test case : test_arb_to_repl_conversion_with_io-arb
✖ test_arb_to_repl_conversion_with_io-arb Failed
✔ Running test case : test_arb_to_repl_conversion_with_io-dist-arb
✖ test_arb_to_repl_conversion_with_io-dist-arb Failed
Results for afr
Test nature : disruptive
TestName : test_arb_to_repl_conversion_with_io
+-------------+-------------+-----------------------+-------------+
| Volume Type | Test Result | Time Taken (hh:mm:ss) | Skip Reason |
+-------------+-------------+-----------------------+-------------+
| arb | FAIL | 00:29:14 | NA |
+-------------+-------------+-----------------------+-------------+
+--------------------------------------+-------+
| Category | Count |
+--------------------------------------+-------+
| Disruptive Tests | 1 |
| Non Disruptive Tests | 0 |
| Disruptive Tests Skipped | 0 |
| Non Disruptive Tests Skipped | 0 |
| Tests Ran | 1 |
| Tests Skipped | 0 |
| Total Tests | 1 |
| Disruptive Tests Pass Percentage | 0.0 |
| Non Disruptive Tests Pass Percentage | 0 |
| Total Pass Percentage | 0.0 |
+--------------------------------------+-------+
Total Time Taken : 00 hours 58 minutes 57 seconds
✔ Tearing down successful.
In the above test run, the second test dist-arb result was not present in the table even though the framework ran that (which is confirmed from the logs and the time taken). Though, the results are logged sometime like:
✔ Param Handling Success.
✔ Test List built
✔ Log dir creation successful.
✔ Environment setup successful.
✔ Running test case : test_arb_to_repl_conversion_with_io-arb
✖ test_arb_to_repl_conversion_with_io-arb Failed
✔ Running test case : test_arb_to_repl_conversion_with_io-dist-arb
✖ test_arb_to_repl_conversion_with_io-dist-arb Failed
Results for afr
Test nature : disruptive
TestName : test_arb_to_repl_conversion_with_io
+-------------+-------------+-----------------------+-------------+
| Volume Type | Test Result | Time Taken (hh:mm:ss) | Skip Reason |
+-------------+-------------+-----------------------+-------------+
| arb | FAIL | 00:09:32 | NA |
| dist-arb | FAIL | 00:10:17 | NA |
+-------------+-------------+-----------------------+-------------+
+--------------------------------------+-------+
| Category | Count |
+--------------------------------------+-------+
| Disruptive Tests | 1 |
| Non Disruptive Tests | 0 |
| Disruptive Tests Skipped | 0 |
| Non Disruptive Tests Skipped | 0 |
| Tests Ran | 1 |
| Tests Skipped | 0 |
| Total Tests | 1 |
| Disruptive Tests Pass Percentage | 0.0 |
| Non Disruptive Tests Pass Percentage | 0 |
| Total Pass Percentage | 0.0 |
+--------------------------------------+-------+
Total Time Taken : 00 hours 20 minutes 01 seconds
✔ Tearing down successful.
It is being suspected that this issue is occurring due to a race condition in the multiprocessing queue..
Describe the bug
The results being logged into the terminal by the result handler, sometimes misses out on the result of the last run even though the test ran completely. For example:
In the above test run, the second test
dist-arbresult was not present in the table even though the framework ran that (which is confirmed from the logs and the time taken). Though, the results are logged sometime like:It is being suspected that this issue is occurring due to a race condition in the multiprocessing queue..