feat(scorecard): add GitLab module with issues, MRs, pipelines, and jobs metrics#3478
feat(scorecard): add GitLab module with issues, MRs, pipelines, and jobs metrics#3478fullsend-ai-coder[bot] wants to merge 4 commits into
Conversation
fullsend-ai-coder
Bot
requested review from
a team,
Eswaraiahsapram,
christoph-jerolimov and
its-mitesh-kumar
as code owners
|
Important This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior. Changed Packages
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3478 +/- ##
==========================================
+ Coverage 53.62% 53.71% +0.08%
==========================================
Files 2256 2264 +8
Lines 85870 86118 +248
Branches 24186 24202 +16
==========================================
+ Hits 46046 46254 +208
- Misses 38333 38373 +40
Partials 1491 1491
*This pull request uses carry forward flags. Click here to find out more. Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
|
🤖 Finished Review · ✅ Success · Started 1:54 PM UTC · Completed 2:08 PM UTC |
ReviewFindingsMedium
Low
Previous runReviewFindingsHigh
Medium
Low
Info
Previous run (2)ReviewFindingsMedium
Low
Previous run (3)ReviewFindingsHigh
Medium
Low
|
|
/fs-fix regenerate the api reports and commit the changes (or new files) |
|
🤖 Finished Fix · ✅ Success · Started 6:44 AM UTC · Completed 6:57 AM UTC |
fullsend-ai-coder
Bot
force-pushed
the
feat/scorecard-gitlab-module-3475
branch
from
ec0bd06 to
4a05a43
Compare
Generate the report.api.md file for the new GitLab module plugin to match the pattern used by other scorecard backend modules. Addresses review feedback on #3478
🔧 Fix agent — iteration 1 (human-triggered)Generated the missing report.api.md API report file for the new scorecard-backend-module-gitlab plugin by running Fixed (1):
Tests: passed Updated by fullsend fix agent |
|
🤖 Finished Review · ✅ Success · Started 7:00 AM UTC · Completed 7:09 AM UTC |
![fullsend-ai-review[bot]](https://avatars.githubusercontent.com/in/3478501?s=60&v=4){kind=small}
| } { | ||
| // Derive the host from the project slug or default to gitlab.com | ||
| const host = 'gitlab.com'; | ||
| const url = `https://${host}/${projectSlug}`; |
There was a problem hiding this comment.
[medium] logic-error
getApiBaseUrl hardcodes host to gitlab.com. Self-hosted GitLab instances will never resolve to the correct SCM integration, and the wrong token could be selected.
Suggested fix: Extract the host from the entity annotation or use getEntitySourceLocation(entity).target as the existing GitHub module does.
|
|
||
| const total = response.headers.get('x-total'); | ||
| if (total !== null) { | ||
| return parseInt(total, 10); |
There was a problem hiding this comment.
[medium] logic-error
fetchTotalCount falls back to counting items.length with per_page=1 when x-total header is absent. The GitLab Pipelines API omits x-total for large result sets, so pipeline metrics may silently return 0 or 1.
Suggested fix: For endpoints that may omit x-total, use fetchAllPages for accurate counts, or increase per_page in the fallback path.
| since: Date, | ||
| ): Promise<Array<{ created_at: string }>> { | ||
| const allItems: Array<{ created_at: string }> = []; | ||
| let page = 1; |
There was a problem hiding this comment.
[medium] api-contract
scope[] parameter uses comma-joined string instead of repeated query parameters. Latent bug — all current callers pass single-element arrays, but multi-scope filtering will fail.
Suggested fix: Use url.searchParams.append for each scope element instead of joining with commas.
| } | ||
|
|
||
| async getOpenedMergeRequestsCount( | ||
| projectSlug: GitlabProjectSlug, |
There was a problem hiding this comment.
[medium] logic-error
getClosedIssuesCount and getClosedMergeRequestsCount use updated_after to approximate items closed within a time window. This over-counts because updated_after includes items updated for any reason, not just closure.
Suggested fix: Document as a known approximation, or fetch paginated results and filter by closed_at for accurate counts.
| } | ||
|
|
||
| private async fetchAllPages( | ||
| projectSlug: GitlabProjectSlug, |
There was a problem hiding this comment.
[medium] edge-case
fetchAllPages has no upper bound on page count and assumes descending created_at ordering for early termination. A project with high job volume could trigger excessive API calls.
Suggested fix: Add order_by and sort parameters to lock down ordering and a maximum page limit with a warning log when hit.
| type: 'number', | ||
| history: true, | ||
| }, | ||
| { |
There was a problem hiding this comment.
[medium] pattern-violation
PERCENTAGE_THRESHOLDS is defined but never used. All providers return DEFAULT_NUMBER_THRESHOLDS, so ratio metrics will be evaluated against count-oriented thresholds, producing nonsensical health indicators.
Suggested fix: Split ratio metrics into separate providers, or use PERCENTAGE_THRESHOLDS as the return value for ratio-focused providers.
| projectSlug, | ||
| since, | ||
| ); | ||
|
|
There was a problem hiding this comment.
[low] test-inadequate
Test mocks use Map instead of Headers object, and there are no tests for the multi-page pagination logic in fetchAllPages.
Suggested fix: Use new Headers() in mocks and add tests for multi-page pagination, early termination, and empty page edge cases.
| } | ||
|
|
||
| async getOpenIssuesCount(projectSlug: GitlabProjectSlug): Promise<number> { | ||
| return this.fetchTotalCount(projectSlug, '/issues', { |
There was a problem hiding this comment.
[low] edge-case
parseInt(total, 10) can return NaN if x-total header contains a non-numeric string. NaN propagates without validation.
Suggested fix: Add Number.isNaN() guard after parsing; fall through to fallback or throw.
|
/fs-fix
|
|
🤖 Finished Fix · ✅ Success · Started 8:01 AM UTC · Completed 8:05 AM UTC |
…obs metrics Implements a new scorecard-backend-module-gitlab plugin that provides 16 metrics for GitLab projects via the GitLab REST API. The module uses Backstage SCM integration for authentication and filters entities by the gitlab.com/project-slug annotation. Metrics: open/opened/closed issues (7d), open/opened/closed MRs (7d), started/successful/failed pipelines (7d), started/successful/failed jobs (7d), pipeline and job success ratios (7d and 24h). Closes #3475 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generate the report.api.md file for the new GitLab module plugin to match the pattern used by other scorecard backend modules. Addresses review feedback on #3478
fullsend-ai-coder
Bot
force-pushed
the
feat/scorecard-gitlab-module-3475
branch
from
3cbefe0 to
6f8afcc
Compare
- Add README.md with installation instructions, annotation requirements, and a table of all 16 available metrics - Add changeset for the new package - Change package.json version from 1.0.0 to 0.0.0 Addresses review feedback on #3478
🔧 Fix agent — iteration 2 (human-triggered)Addressed all 3 human review instructions: added README.md with full documentation, changed version to 0.0.0, and added a changeset. Fixed (3):
Tests: passed Updated by fullsend fix agent |
|
🤖 Finished Review · ✅ Success · Started 8:08 AM UTC · Completed 8:22 AM UTC |
| SUCCESS_RATIO_7D: 'gitlab.pipeline_success_ratio_7d', | ||
| SUCCESS_RATIO_24H: 'gitlab.pipeline_success_ratio_24h', | ||
| } as const; | ||
|
|
There was a problem hiding this comment.
[high] logic-error
PERCENTAGE_THRESHOLDS is defined with correct semantics for ratio metrics but is never used. getMetricThresholds() returns DEFAULT_NUMBER_THRESHOLDS which classifies values >50 as error, so a healthy 80% success ratio would be classified as error (semantically inverted). Same problem affects GitlabJobsProvider.
Suggested fix: Have getMetricThresholds() return PERCENTAGE_THRESHOLDS in both GitlabPipelinesProvider and GitlabJobsProvider, or split ratio metrics into separate providers that return the correct thresholds.
|
|
||
| private getApiBaseUrl(projectSlug: string): { | ||
| apiBaseUrl: string; | ||
| token: string | undefined; |
There was a problem hiding this comment.
[medium] logic-error
getApiBaseUrl hard-codes host to gitlab.com, making self-hosted GitLab instances inaccessible.
Suggested fix: Accept GitLab host from entity (via additional annotation or source location parsing) rather than hard-coding gitlab.com.
| status?: string, | ||
| ): Promise<number> { | ||
| const params: Record<string, string> = { | ||
| updated_after: since.toISOString(), |
There was a problem hiding this comment.
[low] api-contract
getJobsCount sets params scope[] = scope.join(',') which would produce incorrect query parameters for multi-element arrays. Current callers only pass single-element arrays so the bug is latent.
Suggested fix: Use url.searchParams.append in a loop for each scope value.
| return items.length; | ||
| } | ||
|
|
||
| async getOpenIssuesCount(projectSlug: GitlabProjectSlug): Promise<number> { |
There was a problem hiding this comment.
[low] logic-error
getClosedIssuesCount uses updated_after which returns issues with any update after the date, not issues specifically closed after it. This is an approximation due to GitLab API limitations.
Suggested fix: Document as approximation in metric description, or client-side filter by closed_at field.
| apiBaseUrl: integration.config.apiBaseUrl ?? `https://${host}/api/v4`, | ||
| token: integration.config.token, | ||
| }; | ||
| } |
There was a problem hiding this comment.
[low] fail-open
When GitLab integration has no token configured, requests proceed unauthenticated silently without warning.
Suggested fix: Emit a warning log via the Backstage logger service when operating without a token.
| { key: 'success', expression: '>80' }, | ||
| ], | ||
| }; | ||
|
|
There was a problem hiding this comment.
[low] code-organization
calculateRatio helper is defined in GitlabPipelinesProvider.ts and imported by GitlabJobsProvider.ts, creating cross-provider coupling. Shared utilities should live in src/gitlab/utils.ts.
Suggested fix: Move calculateRatio and PERCENTAGE_THRESHOLDS to src/gitlab/utils.ts.
| if (status) { | ||
| params.status = status; | ||
| } | ||
| return this.fetchTotalCount(projectSlug, '/pipelines', params); |
There was a problem hiding this comment.
[low] edge-case
fetchTotalCount fallback with per_page=1 returns 0 or 1 when x-total header is absent. GitLab omits x-total for large result sets (>10,000).
Suggested fix: Paginate to get accurate count when x-total is absent, or log a warning.
| const jobs = await this.fetchAllPages(projectSlug, '/jobs', params, since); | ||
| return jobs.length; | ||
| } | ||
|
|
There was a problem hiding this comment.
[low] edge-case
fetchAllPages has no upper bound on pages fetched and relies on descending chronological order without explicitly requesting it.
Suggested fix: Add order_by=created_at and sort=desc to API parameters and add a maximum page limit.
|
/fs-fix run prettier:fix |
|
🤖 Finished Fix · ✅ Success · Started 8:56 AM UTC · Completed 9:09 AM UTC |
Addresses formatting issues in README.md table and import statement per human instruction to run prettier:fix. Addresses review feedback on #3478
🔧 Fix agent — iteration 3 (human-triggered)Ran prettier:fix per human instruction. Only README.md required formatting changes (table alignment and import formatting). Fixed (1):
Tests: passed Updated by fullsend fix agent |
|
|
🤖 Finished Review · ✅ Success · Started 9:12 AM UTC · Completed 9:26 AM UTC |
fullsend-ai-review
Bot
added
the
requires-manual-review
4 participants
Implements a new scorecard-backend-module-gitlab plugin that provides 16 metrics for GitLab projects via the GitLab REST API. The module uses Backstage SCM integration for authentication and filters entities by the gitlab.com/project-slug annotation.
Metrics: open/opened/closed issues (7d), open/opened/closed MRs (7d), started/successful/failed pipelines (7d), started/successful/failed jobs (7d), pipeline and job success ratios (7d and 24h).
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
Closes #3475
Post-script verification
feat/scorecard-gitlab-module-3475)7ccaff17753df64c7ab288cdcba34cee5a657254..HEAD)