Version
5.x 5.x-stable
Context
I had trouble getting a green CI build of my https proxy PR #6215. In my effort to determine whether the failures I was seeing were due to my changes I did an analysis of the past failures to see if these failure preexisted due to flaky tests.
I decided to publish the report in case it is useful and to serve as a meta ticket to tackle these flaky test failures. I also added a column to link related PRs or issues. I have posted a PR with a suggested fix for NetTest.testListenDomainSocketAddress[Native] #6217
| Frequency |
Test |
Error |
Related issues / PRs |
| 9/60 (15%) |
NetTest.testEndHandlerCalledAfterAllEmissions |
IllegalState "Timed out" |
#6219 |
| 7/60 (11.7%) |
Http2 *HeadersPutAll / testResponseTrailers* |
Illegal 'TE' header value / Stream reset |
|
| 4/60 (6.7%) |
Http2*.testSslHandshakeTimeout |
Timeout Unsatisfied checkpoint |
|
| 4/60 (6.7%) |
NetTest.testListenDomainSocketAddress[Native] |
leftover .sock / native timeout |
#6217 |
| 3/60 (5%) |
it.networklogging.NetTest.testServerLogging / testNoLogging |
Timeout (120 s) |
|
| 2/60 (3.3%) |
WebSocketTest.* (4 methods across 2 runs) |
Timeout Unsatisfied checkpoint |
|
| 2/60 (3.3%) |
Http1xProxyTest.testNonProxyHosts3 |
Bind "Address already in use" |
|
| 2/60 (3.3%) |
Http1xSendFileTest.testSendOpenRangeFileFromClasspath |
|
|
| 2/60 (3.3%) |
MetricsTest.testThreadPoolMetrics* (windows) |
|
|
| 2/60 (3.3%) |
NetTest.testServerWithIdleTimeoutSendChunkedFile |
|
|
| 1/60 (1.7%) |
NetBandwidthLimitingTest.fileUploadIsThrottled |
Timeout (120 s) |
|
| 1/60 (1.7%) |
DNSTest.* (windows) |
Bind "Address already in use" |
|
| 1/60 (1.7%) |
SynchronizationTest.* (4.x, windows) |
|
|
| 1/60 (1.7%) |
NetTest.testCloseUnwritableSocket |
|
|
| 1/60 (1.7%) |
HttpsProxyTest.testHttpsProxy_Http2Origin_tunnelled |
|
|
| 1/60 (1.7%) |
SharedHttpClientTest.testSharedPoolRetainedByOtherDeployment |
|
|
| 1/60 (1.7%) |
Http3Test>HttpTest.testResponseTrailers |
|
|
| 1/60 (1.7%) |
Http2MetricsTest…testHttpClientMetricsQueueClose |
|
|
| 1/60 (1.7%) |
Http1xTest.testInvalidChunkInHttpServerRequest |
Timeout Unsatisfied checkpoint |
|
| 1/60 (1.7%) |
SSLEngineTest.testWebSocketDefaultEngine |
Timeout Unsatisfied checkpoint |
|
Version
5.x 5.x-stable
Context
I had trouble getting a green CI build of my https proxy PR #6215. In my effort to determine whether the failures I was seeing were due to my changes I did an analysis of the past failures to see if these failure preexisted due to flaky tests.
I decided to publish the report in case it is useful and to serve as a meta ticket to tackle these flaky test failures. I also added a column to link related PRs or issues. I have posted a PR with a suggested fix for
NetTest.testListenDomainSocketAddress[Native]#6217NetTest.testEndHandlerCalledAfterAllEmissions*HeadersPutAll/testResponseTrailers*Http2*.testSslHandshakeTimeoutNetTest.testListenDomainSocketAddress[Native].sock/ native timeoutit.networklogging.NetTest.testServerLogging/testNoLoggingWebSocketTest.*(4 methods across 2 runs)Http1xProxyTest.testNonProxyHosts3Http1xSendFileTest.testSendOpenRangeFileFromClasspathMetricsTest.testThreadPoolMetrics*(windows)NetTest.testServerWithIdleTimeoutSendChunkedFileNetBandwidthLimitingTest.fileUploadIsThrottledDNSTest.*(windows)SynchronizationTest.*(4.x, windows)NetTest.testCloseUnwritableSocketHttpsProxyTest.testHttpsProxy_Http2Origin_tunnelledSharedHttpClientTest.testSharedPoolRetainedByOtherDeploymentHttp3Test>HttpTest.testResponseTrailersHttp2MetricsTest…testHttpClientMetricsQueueCloseHttp1xTest.testInvalidChunkInHttpServerRequestSSLEngineTest.testWebSocketDefaultEngine