Some of the ct testcases fail from time to time but not deterministically
This failure is quite common and hopeless to fix because of how timers work in Erlang. (A call to timer:sleep(20) is guaranteed to last at least 20 ms but there is no upper limit)
%%% xprof_tracing_SUITE ==> long_call: FAILED
%%% xprof_tracing_SUITE ==>
Failure/Error: ?assertMatch({ true , _ }, { Min < 22 * 1000 , Min })
expected: = { true , _ }
got: {false,24480} line: 396
Another example when samples are not yet available. (This happens quite often in long_call testcase)
%%% xprof_http_e2e_SUITE ==> capture_data_when_traced_test: FAILED
%%% xprof_http_e2e_SUITE ==>
Failure/Error: ?assertEqual(1, length ( proplists : get_value ( << "items" >> , Data ) ))
expected: 1
got: 0 line: 210
To make snapshots more deterministic (without waiting 1-2 seconds) is to add a function to xprof_core_trace_handler to trigger taking one synchronously (and callerlang:trace_delivered before). But as the key of the snapshots is the timestamp in seconds, there can only be one snapshot per second (and some testcases require two).
Or captured data is not yet available:
%%% xprof_http_e2e_SUITE ==> capture_data_with_formatted_exception_test: FAILED
%%% xprof_http_e2e_SUITE ==>
Failure/Error: ?assertMatch([ << "** exception error: no match of right hand side value ok" >> ], [ proplists : get_value ( << "res" >> , Item ) || Item <- proplists : get_value ( << "items" >> , Data ) ])
expected: = [ << "** exception error: no match of right hand side value ok" >> ]
got: [] line: 244
Some of the ct testcases fail from time to time but not deterministically
This failure is quite common and hopeless to fix because of how timers work in Erlang. (A call to
timer:sleep(20)is guaranteed to last at least 20 ms but there is no upper limit)Another example when samples are not yet available. (This happens quite often in
long_calltestcase)To make snapshots more deterministic (without waiting 1-2 seconds) is to add a function to
xprof_core_trace_handlerto trigger taking one synchronously (and callerlang:trace_deliveredbefore). But as the key of the snapshots is the timestamp in seconds, there can only be one snapshot per second (and some testcases require two).Or captured data is not yet available: