feat: add dedicated mesheryctl BATS report

[Zayden369]

Notes for Reviewers

Fixes #86

Problem

The existing Mesheryctl quality report combines Bash/BATS end-to-end results with Go unit test results. This makes it harder to inspect BATS results independently.

Changes

• Added a dedicated “Mesheryctl: BATS End-to-End Tests” report in allurerc.mjs.
• The report selects results with both project=mesheryctl and framework=bats, excluding Go unit tests.
• Kept the existing combined Mesheryctl report.
• Updated the README with the new report and its publishing process.

The existing workflow already collects and publishes BATS results, so no CI workflow changes are required. After merge and successful deployment, the new report will be available at https://qa.meshery.io/mesheryctl-bats/.

Validation

• node --check allurerc.mjs and git diff --check passed.
• npm run report:generate completed successfully.
• Using six existing BATS results and two existing Go unit results, the dedicated report included all six BATS results and excluded both Go results. The combined report retained all eight.
• The filter also excluded results with missing labels or a different project.

This validation used sampled existing results; the BATS test suite was not rerun. One sampled BATS result was already marked failed upstream and remained failed in the generated report. Generated reports and sample data are not included in this PR.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: afaae811-dd2d-471e-b21b-5784a117e219

📥 Commits

Reviewing files that changed from the base of the PR and between 7cbf6f8 and 4ac2edc.

📒 Files selected for processing (2)

• README.md
• allurerc.mjs

🚧 Files skipped from review as they are similar to previous changes (2)

• allurerc.mjs
• README.md

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds a filtered Allure report for Mesheryctl BATS results. It filters results by project=mesheryctl and framework=bats, and documents the report URL and existing result-generation workflow.

Changes

Mesheryctl BATS report

Layer / File(s)	Summary
Add BATS report and documentation allurerc.mjs, README.md	Adds the Mesheryctl: BATS End-to-End Tests Allure plugin with Mesheryctl and BATS filters. Documents the report URL and existing BATS result workflow.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 4ac2e

This change adds and documents a dedicated filtered Mesheryctl BATS report. No current merge-blocking risk is identified.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes satisfy issue #86. They add a dedicated Allure report filtered by project=mesheryctl and framework=bats, retain the existing combined Mesheryctl report, and document the report in the READ…
Out of Scope Changes check	✅ Passed	All changes are within scope for issue #86. The pull request modifies only the Allure dashboard configuration and README documentation required for the dedicated Mesheryctl BATS report.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding a dedicated Mesheryctl BATS report.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

Warning

Git: CodeRabbit could not clone the repository, so clone-backed analysis was skipped and this review may be incomplete. Verify repository clone access, such as SSH credentials, before requesting another full review. If clone access is intentionally unavailable, use path_filters to narrow the review scope.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[Maanvi212006]

@Zayden369 Please elaborate about the PR in the PR description!

[Zayden369]

@Zayden369 Please elaborate about the PR in the PR description!

Done. U can check

[SuryanshGarg04]

One thing I noticed: bats-to-allure.js seems to let [project=...] or [framework=...] in the test name overwrite the default labels

So if a test has something like [project=CLI-Docs], it may no longer match the new project=mesheryctl AND framework=bats filter and could be missed from the report.

Can we add a test for this case?

[Zayden369]

Good catch @SuryanshGarg04 . The converter lives in meshery/meshery, so I fixed and tested this at the source in meshery/meshery#21863 (commit 84689f170da99bb70c3ca26a1db827daefab8597).

The converter now treats project=mesheryctl, framework=bats, and language=bash as converter-owned labels, so neither ALLURE_LABELS nor title tokens can override them.

The regression test specifically covers [project=CLI-Docs][framework=cucumber] and confirms that the generated result still contains exactly project=mesheryctl and framework=bats.

node --test bats-to-allure.test.js passes all 28 tests. This keeps these results eligible for the new QA report filter. Please take another look.

[MU5A]

Took a look at both the filter addition and the fix in meshery/meshery#21863. The mesheryctl-bats filter mirrors the existing mesheryctl pattern cleanly, no concerns there.

On Suryansh's concern: confirmed the fix addresses it properly, framework/language/project are now locked as converter-owned in bats-to-allure.js so neither ALLURE_LABELS nor a title token can knock a result out of this report's filter. The new regression test covers both override paths (env var and title token) in one case, and confirms the values survive dedup as single entries, not duplicates. Good catch and a solid fix.

Looks ready from here.

[alexquincy]

Thank you, @SuryanshGarg04 @MU5A @Maanvi212006 for your reviews. 💯

[welcome]

Thanks for your contribution to Meshery! 🎉

        Join the community, if you haven't yet and please leave a ⭐ star on the project. 😄

[Zayden369]

Thank you @alexquincy, @SuryanshGarg04, @MU5A, and @Maanvi212006 for reviewing and helping improve this contribution. The feedback about the PR description and possible label overrides helped make the dedicated Mesheryctl BATS report more reliable. I appreciate everyone’s guidance and support in getting it merged!