Allure reporter support#852
Open
DavertMik wants to merge 18 commits into
Open
Conversation
- Update package-lock.json to sync with package.json - Revert build/scripts/edit-js-files.js to working version
Fixes CI build failure caused by optional peer dependencies (detox, @wix-pilot) that were causing npm ci to fail.
Allure results inherently contain steps and stack traces for all tests, so TESTOMATIO_STACK_PASSED and TESTOMATIO_STEPS_PASSED are now enabled by default in AllureReader to prevent silently stripping this data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Accept upstream changes: log utility, simplified batch enabled logic, and command guard in CLI filter. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
XML and Allure readers use sync() to flush tests, so the server can distinguish bulk imports from normal test runner batch uploads. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resolve conflicts and align Allure batching with the new 2.x strategy. - cli.js: keep AllureReader import alongside 2.x's DEBUG_FILE/BATCH_MODE - testomatio.js: adopt 2.x BATCH_MODE.DISABLED check in #batchUpload and drop the allure-only `bulk` flag. sync() is shared by XML and Allure, so keeping `bulk:true` there would have regressed XML upload behavior. - Extract the proven XML chunking (≤1MB per batch) into a shared splitTestsIntoChunks helper in pipe_utils; use it from both readers. - AllureReader now uploads exactly like XmlReader: BATCH_MODE.MANUAL, buffer per chunk, flush one batch per chunk via sync(). No setInterval auto-upload, so every test is sent exactly once (no double-send). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…IntoChunks util - pipe_utils_test.js: direct unit tests for splitTestsIntoChunks — 1000-test suite splits into multiple chunks, count/order preserved, each chunk under the 1MB limit, oversized singletons isolated, custom maxSizeBytes honored. - allure_chunking_test.js: AllureReader.uploadData with 1000 tests splits into multiple batches, uploads every test exactly once (no loss / no double-send), uses sequential batch indices, finishes the run once. Mirrors the existing XmlReader chunking test. Saves/restores TESTOMATIO_STACK_PASSED/STEPS_PASSED so AllureReader's env side effects don't leak into sibling unit suites. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Allure `links` array (which carries @TMslinK values) was ignored entirely, and test_id was only ever derived from source-code annotations. Reports without source access uploaded with no test_id, so Testomat.io created new cases on every run instead of matching — e.g. 1556 tests producing 1767 duplicates. - allureReader.js: read links[type=tms] (the @TMslinK case) and use the link name as test_id; also recognize Testomat.io test-page URLs even when type is null/missing (real exporters omit it). Strip optional @/T markers, matching how XmlReader stores ids. Fall back to the URL's last segment when the link name is empty. - fetchSourceCode no longer overrides a test_id already taken from a TmsLink. - Tests for extractTestId/normalizeTestId and the processAllureResult wiring. - docs/allure.md: document @TMslinK matching. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ink ids Testomat.io test ids are always exactly 8 word-chars (TEST_ID_REGEX). The TmsLink transform now validates this instead of blindly stripping prefixes: - normalizeTestId only removes the @/T markers when the result is still a valid 8-char id, so a real id starting with T (e.g. "Tabcdef0") is no longer corrupted to 7 chars. The full @t<8> marker still normalizes to the bare 8-char id. - Values that aren't valid 8-char ids (numeric Allure TestOps ids like 12345, JIRA keys, 6-digit TMS numbers) are rejected, so we never send an unmatchable test_id. Such tests still import — matched by title/historyId or created fresh. - URL id extraction is anchored to the `/test/<8>` segment and tolerates query strings, instead of naively taking the last path chunk. Verified on real fixtures: ios link (00062226) -> "00062226"; betterme link (102308, non-Testomat) -> ignored. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Step schema (testomat-api-definition.yml) supports a per-step status (passed | failed | none | custom), but convertSteps dropped Allure's step status, so failing steps were not highlighted in the report. - New mapStepStatus: passed -> passed; failed/broken -> failed (broken is an unexpected error, a failure, consistent with mapStatus for tests); skipped/unknown/absent -> none (the step enum has no "skipped"). - convertSteps now sets status on every step, including nested ones. - Tests for passed/failed/broken/skipped/missing and nested propagation; updated the simple-step assertion to include the new field. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an Allure results reporter (
npx @testomatio/reporter allure <pattern>) that parses Allure JSON result/container files and uploads them to Testomat.io, mirroring the existing XML (report-xml) reader.This PR also brings the Allure reader in line with the new 2.x batching strategy (the
BATCH_MODEenum:auto/manual/disabled) so XML and Allure upload identically.What's included
src/allureReader.js— parse Allure*-result.json/*-container.json, dedupe retries byhistoryId, map statuses, extract suites/labels/links/attachments, fetch source code, upload artifacts.allureCLI command insrc/bin/cli.js.src/junit-adapter/kotlin.js) + factory wiring.docs/allure.md.tests/adapter/allure.test.js,tests/adapter/allure-integration.test.js.Batching: same principles as XML, no double-send
Resolved the in-progress
2.xmerge and aligned batching:splitTestsIntoChunkshelper insrc/utils/pipe_utils.js; bothXmlReaderandAllureReadernow use it (single source of truth).AllureReaderuploads inBATCH_MODE.MANUAL:addTestonly buffers,sync()flushes one batch request per chunk. There is nosetIntervalauto-upload, and#batchUploaddrains the buffer withsplice(0), so each test is sent exactly once.bulkflag from#batchUpload/sync(). Sincesync()is shared by both readers, keepingbulk:truethere would have regressed the XML upload path introduced in 2.x.Verification
npm run build✅npm run lint✅xmlReader_chunkingandtestomatio_pipebatch-mode tests).allure-integration.test.jsare pre-existing (missingsample_allure/androidfixture + an unrelated suite-hierarchy assertion) — confirmed identical pass/fail counts on the pre-merge code; untouched by these changes.🤖 Generated with Claude Code