Extend the scorecard github module with some new metrics

[christoph-jerolimov]

Scorecard is a backstage plugin and the GitHub module for scorecard lives in folder workspaces/scorecard/plugins/scorecard-backend-module-github. For the new metrics only code changes in this folder should be neccessary.

The GitHub module creates already this metric:

1. Currently Open PRs already exist.

Add this metrics for Issues/PRs:

2. Currently Open issues. This is the number of open issues (not PR) at the time the metric is calculated.
3. Opened issues. This is the number of opened issues in the last 7 days.
4. Opened PRs. This is the number of opened PRs in the last 7 days.
5. Closed issues. This is the number of opened issues in the last 7 days.
6. Closed PRs. This is the number of opened PRs in the last 7 days.

All these metrics for PRs:

7. Time to Review is the time that PRs takes from creation (PR create time) until the first pullrequestreview was added, independend if the pullrequestreview status. This metric should fetch all PRs (open and closed) that was updated in the last 7 days. PRs without any pullrequestreview should be ignored.
8. Time to Approve is the time that PRs takes from creation (PR create time) until the first pullrequestreview has the status approved. This metric should fetch all PRs (open and closed) that was updated in the last 7 days. PRs without any pullrequestreview should be ignored.
9. Time to Merge is the time that PRs takes from creation to merged. This metric should fetch all merged PRs that was updated in the last 7 days. Closed PRs without merge should be ignored.
10. First time CI status pass rate for the last 7 days and the last 24 hours. This metrics should check the CI status for PRs that was opened in the last 7 days or last 24 hours. It was succesful when all CI statuses was green for the last commit of the first push to a PR.

And this metrics for GitHub Actions:

11. Number of started actions in the last 7 days.
12. Number of successfully finished actions in the last 7 days. Ignore actions that are pending, running, canceled, etc.
13. Number of failed actions in the last 7 days. Ignore actions that are pending, running, canceled, etc.
14. Two metrics for Ratio of successful vs successful+failed actions, once for the last 7 days and one for the last 24 hours. Ignore actions that are pending, running, canceled, etc.

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 10:47 AM UTC · Completed 10:50 AM UTC
Commit: 7ccaff1 · View workflow run →

[fullsend-ai-triage]

Triage Summary (Updated)

Category: Feature — extend scorecard GitHub module with 13 new metrics

Scope: All changes within workspaces/scorecard/plugins/scorecard-backend-module-github/

Metrics Requested

Issue/PR Counts (5 metrics):

1. Currently open issues (not PRs)
2. Opened issues in last 7 days
3. Opened PRs in last 7 days
4. Closed issues in last 7 days
5. Closed PRs in last 7 days

PR Lifecycle Timing (3 metrics):
6. Time to first review — from PR creation to first pullrequestreview event (any status). PRs updated in last 7 days; skip PRs with no reviews.
7. Time to first approval — from PR creation to first pullrequestreview with status approved. PRs updated in last 7 days; skip PRs with no reviews.
8. Time to merge — from PR creation to merge. Merged PRs updated in last 7 days; skip closed-not-merged PRs.

CI Pass Rate (1 metric, 2 windows):
9. First-time CI status pass rate — check CI statuses on the last commit of the first push to each PR. Report for both 7-day and 24-hour windows. Success = all CI statuses green on that commit.

GitHub Actions (4 metrics):
10. Started actions in last 7 days
11. Successfully finished actions in last 7 days (exclude pending/running/canceled)
12. Failed actions in last 7 days (exclude pending/running/canceled)
13. Success ratio (successful / (successful + failed)) — two metrics: 7-day and 24-hour windows

Recommended Approach

The existing GithubOpenPRsProvider serves as a reference pattern. New providers should follow the same structure, grouped by domain.

Proposed Test Case

For each metric provider:
1. Mock GitHub API responses with deterministic data
2. Verify correct metric values are returned
3. Edge cases:
   - Empty results (no matching issues/PRs/actions)
   - PRs without reviews (excluded from time-to-review/approve)
   - Closed-not-merged PRs (excluded from time-to-merge)
   - Actions in non-terminal states (excluded from counts)
   - Boundary timestamp filtering (items exactly at 7-day cutoff)
   - CI pass rate: mixed statuses on first push's last commit
   - CI pass rate: PRs with no CI checks
   - CI pass rate: 7-day vs 24-hour window separation

---

Updated to include the CI pass rate metric (item 10 from the original issue) which was missing from the prior triage.

---

Labels: The scorecard label applies to this scorecard workspace feature request; workspace/scorecard is already present.

Previous run

Triage Summary

Category: Feature — extend scorecard GitHub module with 13 new metrics

Scope: All changes within workspaces/scorecard/plugins/scorecard-backend-module-github/src/metricProviders/

Metrics Requested

Issue/PR Counts (5 metrics):

• Currently open issues, opened issues (7d), opened PRs (7d), closed issues (7d), closed PRs (7d)

PR Lifecycle Timing (3 metrics):

• Time to first review, time to first approval, time to merge — all using PRs updated in last 7 days

GitHub Actions (5 metrics):

• Started actions (7d), successful actions (7d), failed actions (7d), success ratio (7d), success ratio (24h)

Recommended Approach

The existing GithubOpenPRsProvider serves as a reference pattern. New providers should follow the same structure, grouped by domain. Key implementation considerations:

• PR lifecycle metrics should use pullrequestreview events and filter by updated timestamp
• CI pass rate metric needs to check statuses on the last commit of the first push to each PR
• Actions metrics should exclude pending/running/canceled workflow runs

Proposed Test Case

For each metric provider:
1. Mock GitHub API responses with deterministic data
2. Verify correct metric values are returned
3. Edge cases:
   - Empty results (no matching issues/PRs/actions)
   - PRs without reviews (excluded from time-to-review/approve)
   - Closed-not-merged PRs (excluded from time-to-merge)
   - Actions in non-terminal states (excluded from counts)
   - Boundary timestamp filtering (items exactly at 7-day cutoff)
   - CI pass rate with mixed statuses on first push

---

Labels: This is a feature enhancement request for the scorecard workspace, which already has the workspace label applied.

[christoph-jerolimov]

@fullsend-ai-triage you missed the CI pass rate metric, that should be implemented as well.

10. First time CI status pass rate for the last 7 days and the last 24 hours. This metrics should check the CI status for PRs that was opened in the last 7 days or last 24 hours. It was succesful when all CI statuses was green for the last commit of the first push to a PR.

[christoph-jerolimov]

/fs-triage

you missed the CI pass rate metric, that should be implemented as well.

10. First time CI status pass rate for the last 7 days and the last 24 hours. This metrics should check the CI status for PRs that was opened in the last 7 days or last 24 hours. It was succesful when all CI statuses was green for the last commit of the first push to a PR.

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 11:00 AM UTC · Completed 11:03 AM UTC
Commit: 7ccaff1 · View workflow run →