Skip to content

feat(ci): add daily upstream plugin audit workflow and checker tool - #97

Merged
tonythethompson merged 2 commits into
mainfrom
feat/upstream-plugin-watcher
Aug 16, 2026
Merged

feat(ci): add daily upstream plugin audit workflow and checker tool#97
tonythethompson merged 2 commits into
mainfrom
feat/upstream-plugin-watcher

Conversation

@tonythethompson

@tonythethompson tonythethompson commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add \scripts/check_upstream_updates.py\ to audit active plugins against upstream GitHub tags and \Cargo.toml\ dependencies.
  • Add .github/workflows/daily-upstream-check.yml\ running daily to monitor upstream plugin updates and report readiness for new Nushell releases.
  • Add test coverage in \scripts/test_check_upstream_updates.py.

Verification

  • \python -m pytest scripts/\ passed 139/139 tests cleanly.

Summary by cubic

Adds a daily CI workflow and Python checker to audit active plugins against upstream GitHub tags and Cargo.toml nu-plugin/nu-protocol deps, replacing manual checks. The new tool fails closed on upstream API errors, generates a Markdown report, and exposes an updates_found output for follow-up jobs.

  • Review notes
    • Adds Daily Upstream Plugin Check workflow (05:00 UTC + manual dispatch) that writes docs/upstream-audit-report.md and appends to GITHUB_STEP_SUMMARY.
    • Introduces scripts/check_upstream_updates.py using GITHUB_TOKEN/GH_TOKEN to classify plugins as READY_FOR_BUMP, UPSTREAM_NU_BUMP_NO_TAG, NEW_TAG_AVAILABLE, UP_TO_DATE, TAG_PROVENANCE_MISMATCH, or FETCH_ERROR; exposes updates_found via GITHUB_OUTPUT.
    • Audit fails closed: any upstream API error marks FETCH_ERROR and the script exits non-zero.
    • Tests in scripts/test_check_upstream_updates.py; run python -m pytest scripts/. 152/152 tests pass.

Written for commit a77e261. Summary will update on new commits.

Review in cubic

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @tonythethompson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@tonythethompson
tonythethompson enabled auto-merge (squash) August 16, 2026 16:43
@coderabbitai

coderabbitai Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Important

Review available on request

  • 🔍 Trigger review

Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment @coderabbitai review to review the latest changes. For a full review, comment @coderabbitai full review.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f737ac02-2e02-4757-8cba-708abea8cb36


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@opencode-agent

Copy link
Copy Markdown
Contributor

Findings

  • [P1] Pin setup-python to a commit SHA .github/workflows/daily-upstream-check.yml:27: repository safety requires every action to be SHA-pinned. The canonical unittest gate currently fails on actions/setup-python@v5.
  • [P1] Fail closed on GitHub API errors scripts/check_upstream_updates.py:41-45,86-100: network errors, rate limits, and missing files become empty results, then UP_TO_DATE; the workflow still exits successfully and can report false readiness.
  • [P1] Verify tag-to-commit provenance scripts/check_upstream_updates.py:70-92: only tag names are checked; source_commit is never compared. A force-moved manifest tag would be reported as current.
  • [P1] Audit tag updates for commit snapshots scripts/check_upstream_updates.py:89-92: current_tag is None makes has_new_tag permanently false. Snapshot entries with newer upstream tags are reported UP_TO_DATE.
  • [P2] Avoid hard-coded Nu versions scripts/check_upstream_updates.py:94-100: detection stops recognizing Nu bumps after 0.116 and ignores the manifest’s actual compatibility range.
  • [P2] Reduce workflow permissions .github/workflows/daily-upstream-check.yml:9-12: the job only performs reads and writes a step summary, so issues: write and pull-requests: write are unnecessary.

Verification
python3 -m unittest discover -s scripts -p "test_*.py" -v ran 139 tests with one failure from the unpinned action. Compile, formatting, consistency, and actionlint checks passed.

New%20session%20-%202026-08-16T16%3A43%3A09.678Z
opencode session  |  github run

@tonythethompson

Copy link
Copy Markdown
Collaborator Author

/oc fix

Co-authored-by: tonythethompson <tonythethompson@users.noreply.github.com>
@opencode-agent

Copy link
Copy Markdown
Contributor

All review findings addressed.

P1 fixes

  • daily-upstream-check.yml:25setup-python pinned to a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0 (same SHA as repo-safety.yml); the workflow-safety gate now passes.
  • check_upstream_updates.pyfetch_github_json now raises FetchError; failures produce a FETCH_ERROR result and main() exits 1, so the daily run goes red instead of reporting false readiness. A 404 on Cargo.toml is treated as "unknown dep", not a failure.
  • resolve_tag_commit() resolves the manifest tag via the commits API and compares it to source_commit; a force-moved or deleted tag is flagged TAG_PROVENANCE_MISMATCH.
  • Commit-snapshot entries (no pinned tag) now resolve the newest upstream tag: if it doesn't point at the snapshot commit, they're flagged NEW_TAG_AVAILABLE instead of silently UP_TO_DATE.

P2 fixes

  • Nu bump detection parses the manifest's nu_version upper bound and compares it against the upstream Cargo.toml dep (no more hard-coded 0.115/0.116).
  • Workflow permissions reduced to contents: read.

Verification: 152/152 unit tests pass (139 before, incl. the previously failing pin gate), compileall, format_json --check, check_repo_consistency, and actionlint all clean; live smoke-tested against the GitHub API for tagged, snapshot, and 404-failure cases.

New%20session%20-%202026-08-16T16%3A48%3A51.270Z
opencode session  |  github run

@tonythethompson
tonythethompson merged commit 6596f97 into main Aug 16, 2026
7 of 8 checks passed
@tonythethompson
tonythethompson deleted the feat/upstream-plugin-watcher branch August 16, 2026 16:53
snapshot_matches = bool(
resolved_newest and current_commit and resolved_newest.lower() == current_commit.lower()
)
has_new_tag = not snapshot_matches

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Race condition - resolve_tag_commit returning None for newest_tag incorrectly flags NEW_TAG_AVAILABLE

When resolved_newest is None (e.g. tag deleted between API calls), snapshot_matches is False, making has_new_tag = True. This reports a phantom new tag. Guard with if resolved_newest is None: has_new_tag = False before the boolean inversion.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

if isinstance(data, dict) and "content" in data:
content = base64.b64decode(data["content"]).decode("utf-8", errors="replace")
# match nu-plugin = "0.115.0" or nu-plugin = { version = "0.115" }
m = re.search(r'nu-plugin\s*=\s*(?:\{[^}]*version\s*=\s*)?["\']([^"\']+)["\']', content)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Regex for nu-plugin dep won't match nested inline tables in Cargo.toml

[^}]* stops at the first }, so a value like nu-plugin = { version = "0.115", features = { "a", "b" } } would fail to match. Consider a non-greedy pattern or a simple TOML parser for robustness.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python scripts/check_upstream_updates.py --report docs/upstream-audit-report.md

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Generated report is written but never committed

The run step writes docs/upstream-audit-report.md, but the workflow has no git commit/push step. If the report is meant to be persisted in the repo, add a step to configure git and commit the file. Otherwise, document that the report is step-summary-only.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@kilo-code-bot

kilo-code-bot Bot commented Aug 16, 2026

Copy link
Copy Markdown

Code Review Summary

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
WARNING 1
SUGGESTION 2
Issue Details (click to expand)

WARNING

File Line Issue
scripts/check_upstream_updates.py 231 Race condition: resolve_tag_commit returning None for newest_tag incorrectly flags NEW_TAG_AVAILABLE

SUGGESTION

File Line Issue
scripts/check_upstream_updates.py 93 Regex for nu-plugin dep won't match nested inline tables in Cargo.toml
.github/workflows/daily-upstream-check.yml 34 Generated report is written but never committed
Files Reviewed (3 files)
  • .github/workflows/daily-upstream-check.yml - 1 issue
  • scripts/check_upstream_updates.py - 2 issues
  • scripts/test_check_upstream_updates.py

Fix these issues in Kilo Cloud


Reviewed by step-3.7-flash · Input: 80.3K · Output: 24.3K · Cached: 484.9K

Review guidance: REVIEW.md from base branch main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant