Update documentation and add xping test fingerprint#88
Open
xping-admin wants to merge 1 commit into
Open
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 🚀 New features to boost your workflow:
|
Xping InsightsInsights #01kmfds8 • Analysis completed at 12:43 UTC • How to read these insights sampleapp-nunit — Confidence Score: 0.77 (Moderately Reliable, Low) • Success: 88% • 7 passed, 1 failed • 14 ms totalDetails
|
| Factor | Score | Confidence Level | Samples |
|---|
Powered by Xping • ver 1.0.0-rc.1
Xping InsightsInsights #01kn7h1h • Analysis completed at 10:03 UTC • How to read these insights sampleapp-nunit — Suite Reliability: 0.84 (Reliable) •
|
| Category | Tests |
|---|---|
| 🟢 Highly Reliable | 5 (62%) |
| 🟡 Reliable | 1 (12%) |
| 🔴 Unreliable | 2 (25%) |
sampleapp-xunit — Suite Reliability: 0.80 (Reliable) • ⚠️ 2 tests need attention • Success: 83%
Details
- 🧪 10 passed, 1 failed, 1 skipped • 64 ms
- 🖥️ Environment: Local • macOS 26.4.0 • .NET 9.0.4 • xUnit
- 🔬 Scored: 100% of tests
⚠️ Data confidence: Low — based on limited execution history
✅ No Flaky Tests Detected
📊 Suite Reliability Distribution
| Category | Tests |
|---|---|
| 🟢 Highly Reliable | 6 (50%) |
| 🟡 Reliable | 4 (33%) |
| ⛔ Highly Unreliable | 2 (17%) |
Powered by Xping • ver 0.1.0-alpha.1
Xping InsightsInsights #01knmbbw • Upload completed at 21:20 UTC • Confidence score analysis in progress... • How to read these insights sampleapp-xunit • Success: 83% • Full analysis in progress...Details
Flaky detection in progress... Full confidence analysis in progress... Powered by Xping • ver 0.1.0-alpha.1 |
Xping InsightsInsights #01knw1em • Analysis completed at 15:24 UTC • How to read these insights sampleapp-xunit — Suite Reliability: 0,71 (Moderately Reliable) • Success: 0%Details
✅ No Flaky Tests Detected📊 Suite Reliability Distribution
Powered by Xping • ver 0.1.0-alpha.1 |
Xping InsightsInsights #01kpx6ab • Analysis completed at 19:35 UTC • How to read these insights sampleapp-xunit — Suite Reliability: 0,70 (Moderately Reliable) •
|
| Test | Category | Severity | Classifier Confidence |
|---|---|---|---|
SampleApp.XUnit.SampleTests.FlakyTest_EnvironmentState_FailsBasedOnSystemState |
Timing | 🔴 Critical | 84% |
Evidence for FlakyTest_EnvironmentState_FailsBasedOnSystemState
Pass Rate
- Pass rate: 68,3% (highly intermittent)
- Passed 41/60, failed 19/60
- Neither consistently passing nor consistently failing
- Classic flaky test signature
Execution Variance
- Coefficient of variation: 0,53 (high)
- Mean duration: 77ms, StdDev: 41ms
- Range: 12ms - 136ms
- High variance suggests race conditions or resource contention
Retries
- Test does not use retries
Environment
- Tested in 2 environments
💡 Fix: Review test for hard-coded delays or sleep statements
📊 Suite Reliability Distribution
| Category | Tests |
|---|---|
| 🟢 Highly Reliable | 8 (67%) |
| 🟡 Reliable | 1 (8%) |
| 🟠 Moderately Reliable | 2 (17%) |
| 🔴 Unreliable | 1 (8%) |
Powered by Xping • ver 0.1.0-alpha.2
Xping InsightsAnalysis completed at 19:52 UTC • How to read these insights sampleapp-xunit — Moderately Reliable •
|
| Test | Category | Details | |
|---|---|---|---|
| 🔴 | ThrowingTestIsTracked | Highly Unreliable | avg 0,2ms · retries: 0 |
| 🟠 | SkippedTestIsTracked | Unreliable | avg 0,1ms · retries: 0 |
| 🟡 | FlakyTest_EnvironmentState_FailsBasedOnSystemState 🐛 | Moderately Reliable | avg 79,4ms · retries: 0 |
Details
- 🧪 9 passed, 2 failed, 1 skipped • 113 ms
- 🖥️ Environment: CI • macOS 26.4.1 • .NET 10.0.5 • xUnit
- 🔬 Scored: 12/12 tests (100%)
📊 Suite Reliability Distribution
| Category | Tests |
|---|---|
| 🟢 Highly Reliable | 8 (67%) |
| 🟢 Reliable | 1 (8%) |
| 🟡 Moderately Reliable | 1 (8%) |
| 🟠 Unreliable | 1 (8%) |
| 🔴 Highly Unreliable | 1 (8%) |
Powered by Xping • ver 0.1.0-alpha.2
Xping InsightsAnalysis completed at 08:31 UTC • How to read these insights sampleapp-xunit — Reliable •
|
| Test | Category | Details | |
|---|---|---|---|
| 🟠 | ThrowingTestIsTracked | Unreliable | avg 0,2ms · retries: 0 |
| 🟠 | SkippedTestIsTracked | Unreliable | avg 0,1ms · retries: 0 |
| 🟡 | FlakyTest_EnvironmentState_FailsBasedOnSystemState 🐛 | Moderately Reliable | avg 66,8ms · retries: 0 |
Details
- 🧪 10 passed, 1 failed, 1 skipped • 53 ms
- 🖥️ Environment: STAGING • macOS 26.4.1 • .NET 10.0.5 • xUnit
- 🔬 Scored: 12/12 tests (100%)
📊 Suite Reliability Distribution
| Category | Tests |
|---|---|
| 🟢 Highly Reliable | 8 (67%) |
| 🟢 Reliable | 1 (8%) |
| 🟡 Moderately Reliable | 1 (8%) |
| 🟠 Unreliable | 2 (17%) |
Powered by Xping • ver 1.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces improvements to the sample NUnit test suite and documentation, focusing on test tracking and clarity for maintaining historical data. The most important changes are grouped below:
Enhancements to test tracking:
[XpingFingerprint("flaky-random-failure-v1")]attribute to theFlakyTest_RandomFailure_FailsProbabilisticallytest inSampleTests.csto ensure consistent tracking of test history across refactors.Xping.Sdk.Core.AttributesinSampleTests.csto support the use of fingerprint attributes.Documentation updates:
quickstart-nunit.mdto emphasize the importance of treating pinned fingerprints as permanent, clarifying the impact of changing attribute values on historical test data.