Skip to content

feat(ci): add /explain-ci bot command for PR failure help#60821

Draft
miaulalala wants to merge 1 commit into
masterfrom
feat/noid/explain-ci-bot
Draft

feat(ci): add /explain-ci bot command for PR failure help#60821
miaulalala wants to merge 1 commit into
masterfrom
feat/noid/explain-ci-bot

Conversation

@miaulalala
Copy link
Copy Markdown
Contributor

@miaulalala miaulalala commented May 28, 2026

Summary

  • Adds .github/workflows/ci-bot.yml: a new slash-command bot triggered by commenting /explain-ci on any pull request
  • When invoked, nextcloud-bot reacts to the comment with 👍 and posts a single summary of all currently-failing CI checks, explaining what each one validates and how to fix it locally
  • Covers all ~40 developer-facing PR workflows with specific fix instructions
  • Extends autotest.sh with Docker-backed service helpers for local reproduction of external-service CI failures

How the bot works

The bot uses the issue_comment trigger (same pattern as /compile and /update-3rdparty), so it has access to COMMAND_BOT_PAT and works on fork PRs too. It:

  1. Fetches all completed workflow runs for the PR's current commit SHA via the Actions API
  2. Deduplicates by workflow name (takes the most recent run per workflow)
  3. Looks up each failing workflow in a static explanation map
  4. Posts a formatted summary — or a "no failures found" message if everything is green

Workflows covered

All PHPUnit variants (SQLite, MariaDB, MySQL, PostgreSQL, OCI, nodb, 32bits, memcached, sharding, primary object store, all files_external backends), ESLint, Stylelint, PHP lint, PHP-CS-Fixer, Psalm, Cypress, Node build/tests, REUSE, OpenAPI, Rector, Behat integration tests (DAV, Litmus, SQLite), object storage tests (S3, Azure, Swift), Samba Kerberos SSO, Code checkers, block checks (fixup commits, unconventional commits, outdated 3rdparty/), and CodeQL.

New autotest.sh Docker service helpers

autotest.sh now supports spinning up Docker-backed external services automatically, making it possible to reproduce CI failures locally without any manual container setup:

Variable What it does
EXTERNAL_STORAGE=ftp Starts vsftpd, runs files_external FTP tests
EXTERNAL_STORAGE=sftp Starts atmoz/sftp, runs files_external SFTP tests
EXTERNAL_STORAGE=smb Starts Samba, runs files_external SMB tests
EXTERNAL_STORAGE=webdav Starts Apache WebDAV, runs files_external WebDAV tests
EXTERNAL_STORAGE=amazons3 Starts LocalStack, runs files_external S3 tests
PRIMARY_STORAGE_CONFIG=s3 USEDOCKER=1 Starts MinIO as primary object store
PRIMARY_STORAGE_CONFIG=azure USEDOCKER=1 Starts Azurite as primary object store
ENABLE_MEMCACHE=memcached USEDOCKER=1 Starts Memcached, copies tests/memcached.config.php

Each helper installs Nextcloud, enables the relevant app/config, waits for the service to be healthy, runs the tests, and tears down the container — no manual cleanup needed. Pre-flight checks catch missing system dependencies (e.g. php-smbclient, smbclient binary) before wasting time on an install.

Example usage

Comment /explain-ci on a PR with failing checks → nextcloud-bot replies:

CI failure summary

Found 2 failing check(s) on commit abc1234:


Lint eslint

What this checks: Checks JavaScript/TypeScript/Vue source files for ESLint rule violations.

How to fix locally:

  • Run npm run lint to see all violations.
  • Run npm run lint:fix to auto-fix many of them, then review remaining errors manually.

View failed run

Test plan

  • Comment /explain-ci on a PR with at least one failing check → bot reacts with 👍 and posts summary
  • Comment /explain-ci on a PR where all checks pass → bot replies "no failing checks found"
  • Comment /explain-ci on a fork PR → bot works (issue_comment runs in base repo context)
  • Re-run a failed check, then comment /explain-ci → only still-failing checks appear in summary
  • EXTERNAL_STORAGE=ftp NOCOVERAGE=1 ./autotest.sh → all FTP tests pass
  • EXTERNAL_STORAGE=amazons3 NOCOVERAGE=1 ./autotest.sh → all S3 tests pass
  • PRIMARY_STORAGE_CONFIG=s3 USEDOCKER=1 NOCOVERAGE=1 ./autotest.sh sqlite lib/Files/ObjectStore/S3Test.php → MinIO tests pass

🤖 Generated with Claude Code

@miaulalala miaulalala requested a review from a team as a code owner May 28, 2026 20:42
@miaulalala miaulalala requested review from ArtificialOwl, icewind1991, leftybournes and salmart-dev and removed request for a team May 28, 2026 20:42
@miaulalala miaulalala marked this pull request as draft May 28, 2026 21:18
@miaulalala miaulalala force-pushed the feat/noid/explain-ci-bot branch from 2a82f09 to a893f1f Compare May 28, 2026 21:18
- Adds `.github/workflows/ci-bot.yml`: slash command `/explain-ci` on
  any PR triggers nextcloud-bot to post a summary of all currently-
  failing checks with purpose, fix steps, Docker commands, and
  prerequisites. Covers all ~40 PR-facing workflows. Uses the
  `issue_comment` trigger (works on fork PRs) and posts via
  `COMMAND_BOT_PAT`, matching the `/compile` and `/update-3rdparty`
  pattern.

- Extends `autotest.sh` with Docker-backed service helpers so external-
  service CI failures can be reproduced locally without manual setup:
  - `EXTERNAL_STORAGE=ftp/sftp/smb/webdav/amazons3` — spins up the
    relevant service container, installs Nextcloud, enables
    files_external, runs the storage tests, and tears down the container
  - `PRIMARY_STORAGE_CONFIG=s3/azure USEDOCKER=1` — spins up MinIO or
    Azurite as primary object store
  - `ENABLE_MEMCACHE=memcached USEDOCKER=1` — spins up Memcached
  - Pre-flight checks warn about missing system dependencies before
    wasting time on an install

- Adds `tests/memcached.config.php` for Memcached cache configuration

AI-Assisted-By: claude-sonnet-4-6 <noreply@anthropic.com>
Signed-off-by: Anna Larch <anna@nextcloud.com>
Comment thread autotest.sh

echo "Using database $DATABASENAME"

function execute_external_tests {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're aware of autotest-external.sh?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope?

with:
github-token: ${{ secrets.COMMAND_BOT_PAT }}
script: |
const WORKFLOW_EXPLANATIONS = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fear that the keys will get out of sync and we won't notice. I don't have a good solution for it though.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdym? The token?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants