Skip to content

feat: release pipeline with build, tag, and comparison workflows#74

Open
skjnldsv wants to merge 15 commits into
mainfrom
feature/tag-repos-on-release
Open

feat: release pipeline with build, tag, and comparison workflows#74
skjnldsv wants to merge 15 commits into
mainfrom
feature/tag-repos-on-release

Conversation

@skjnldsv
Copy link
Copy Markdown
Contributor

@skjnldsv skjnldsv commented May 31, 2026

Summary

Adds a release pipeline that builds Nextcloud server archives, tags all bundled repositories, and compares the output against the existing release script.

Runs automatically when a release is published, alongside the existing changelog workflow.

Workflows

release.yml — Orchestrator

Triggers on release: published (tags starting with v only). Calls the tagger and builder in parallel.

release-tag.yml — Tag all repositories

Tags all bundled app repos at their stable branch HEAD.

  • Reads app list from stableXX.json + non-build repos from tag-only.json
  • Uses gh release create + gh release delete to create tags
  • Falls back to repo default branch if stable branch not found
  • Idempotent: skips existing tags
  • Manual dispatch with force option to recreate tags

release-build.yml — Build and compare release

Builds release archives and compares against the release script's assets on the same release.

Pipeline:

  1. Validate version.php (confirms bump PR is merged)
  2. Auto-detect ref: use tag if it exists, fall back to branch HEAD
  3. Detect compatible PHP version from server
  4. Fetch server, 3rdparty, updater, example-files, docs (parallel)
  5. Fetch all apps via matrix (parallel, with smart composer detection)
  6. Assemble, clean dev files, update version.php, sign, generate metadata
  7. Create tar.bz2 + zip + checksums
  8. Compare file list against release script output

Configuration

App configs (stableXX.json, master.json)

[
  {"id": "activity", "repo": "nextcloud/activity"},
  {"id": "suspicious_login", "repo": "nextcloud/suspicious_login", "composer_args": "--no-dev -a --quiet --no-scripts"}
]

Optional composer_args overrides the default composer flags when needed.

tag-only.json

Non-build repos to tag (server, 3rdparty, updater, example-files, documentation).

Composer detection

Only runs if composer.json has actual runtime dependencies. Prevents creating empty vendor/ autoloaders in apps that don't need them.


Migration plan: from release script to full workflow

Step 1: Merge this PR

The workflow runs alongside the release script. Both produce archives. The compare job validates parity.

Step 2: Add workflow_dispatch to orchestrator

The orchestrator becomes the entry point instead of reacting to releases.

Step 3: Add release-exists guard

Before publishing, check that the release doesn't already exist to prevent duplicates.

Step 4: Enable publish job

Create release on nextcloud-releases/server with archives attached. Create reference release on nextcloud/server. Handle --prerelease for alpha/beta/rc, --latest for current stable.

Step 5: Use environment protection

Add a production environment with required reviewers for the publish job. Build runs automatically, publish waits for approval.

Step 6: Remove from release script

  • Archive creation + upload
  • Tag creation (tag.php)
  • GitHub release creation

Step 7: Automate updater server

Generate releases.json entry with updater signatures and create a PR to nextcloud/updater_server.

Step 8: Automate download server

Webhook on release to pull assets, or direct upload from workflow.

@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch from 88e9b2e to 8f706c6 Compare May 31, 2026 14:51
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch 10 times, most recently from 4a0881c to 5ea87fb Compare May 31, 2026 15:24
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch 5 times, most recently from 58176af to 32c51be Compare May 31, 2026 15:53
Replaces the manual tag.php from github_helper/tagger. Runs
automatically when a release is published on this repo, alongside
the existing changelog workflow.

- Reads repo list from per-version JSON configs (stable32-34)
- Uses GitHub API to create tags at stable branch HEAD
- Handles branch overrides for master (text, app_api use main)
- Idempotent: skips repos where tag already exists
- Reports results in step summary table
- Supports workflow_dispatch for re-tagging
- Checks out github_helper for future changelog integration

Repo lists match github_helper/tagger/tag.php:
- stable32/33: 28 repos
- stable34: 30 repos (+files_lock, +office)

Requires TAG_TOKEN secret with contents:write on all target repos.

Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
@skjnldsv skjnldsv changed the title feat: add tagger workflow to tag all repos on release feat: release pipeline with build, tag, and comparison workflows May 31, 2026
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch 10 times, most recently from 771f852 to ca02e62 Compare May 31, 2026 18:37
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch 2 times, most recently from 69e4922 to 084cac6 Compare May 31, 2026 20:12
@skjnldsv skjnldsv self-assigned this May 31, 2026
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch 2 times, most recently from ba0b9b5 to d1579e2 Compare June 1, 2026 18:42
skjnldsv added 5 commits June 2, 2026 06:14
JSON configs listing all apps per major version. Used by both the
tagger (reads .repo) and the builder (reads full config).

- stable32/33: 23 apps
- stable34/master: 25 apps (+files_lock, +office)
- tag-only.json: repos to tag but not build (server, 3rdparty, etc.)

Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
Tags all bundled repos on release via GitHub API.
Uses gh release create + delete to create tags without SSH keys.
Falls back to repo default branch if stable branch not found.
Supports manual dispatch with force option for re-tagging.

Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
Builds Nextcloud release archives and compares against the release
script's output on the same GitHub release.

Pipeline: validate version.php, detect PHP version, fetch all
components in parallel, assemble, clean dev files, sign, metadata,
archive (tar.bz2 + zip), checksum, compare file lists + content hashes.

Smart composer detection: only runs if real non-dev deps exist.
Shared cleanup script for .nextcloudignore + dev file patterns.

Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
Triggers on release:published (tags starting with v).
Calls tagger and builder in parallel.

Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch 2 times, most recently from 3d926c4 to 71bddb5 Compare June 2, 2026 04:34
Move inline bash/PHP from the workflow into reusable scripts that
can be run manually without GitHub Actions.

Scripts:
- fetch-all.sh: clone all components from JSON config
- assemble.sh: place components into nextcloud/ structure
- clean-server-dev-files.sh: server .nextcloudignore/hardcoded cleanup
- clean-dev-files.sh: app dev file cleanup (already existed)
- update-version-php.sh: rewrite version.php with release metadata
- sign-release.sh: occ integrity:sign-core/sign-app
- generate-metadata.sh: install + occ migrations:generate-metadata
- package.sh: permissions, tar.bz2+zip, checksums

README.md in scripts/ documents the manual release process.
Workflow becomes thin orchestration calling scripts with arguments.

Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch 2 times, most recently from 555a85b to 087aa31 Compare June 2, 2026 04:48
- actions/checkout v4.2.2 → v6.0.2
- actions/upload-artifact v4.6.2 → v7.0.1
- actions/download-artifact v4.3.0 → v8.0.1
- shivammathur/setup-php v2.32.0 → v2.37.1

Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch from 087aa31 to c33c5b4 Compare June 2, 2026 05:24
skjnldsv added 7 commits June 2, 2026 07:35
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
- stable22-24: 18 apps (+ files_videoplayer, files_rightclick)
- stable25-27: 21 apps (+ bruteforcesettings, related_resources, suspicious_login, twofactor_totp)
- stable28: 20 apps (files_rightclick removed)
- stable29: 21 apps (+ files_downloadlimit)

Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
Compare step now tries GitHub release assets first, then falls back
to download.nextcloud.com when no archive is attached to the release.

Also fixes:
- Subshell variable scoping: pipe-to-while lost counters, so the
  tagger never reported failures. Use here-string instead.
- Master config selection: use master.json directly instead of
  guessing via ls stable*.json.
- fetch-3rdparty: remove unnecessary serial dependency on fetch-server
  (only uses API call with init outputs).
- Assemble guard: check all critical dependencies (3rdparty, updater,
  example-files), not just server+apps.

Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
Match the release script behavior: clone 3rdparty at the same
branch/tag as server instead of looking up the submodule SHA.

The release script does `getrepo('3rdparty', $branch)` which clones
at branch HEAD, not at the submodule pointer. Using the submodule SHA
would produce different content than the release script.

Applies to both the workflow (fetch-3rdparty job) and the standalone
fetch-all.sh script.

Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
- Sync clean-dev-files.sh with release script's
  getDataToBeRemovedFromAppFolders: use the same cross-product of
  JS config base names × extensions instead of a hand-picked list.
- Add phpDocumentor.sh and testConfiguration.json (circles leftovers).
- Mark vendor/composer autoloader diffs as known in compare step
  (different PHP/composer version produces different generated files).

Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
Switch from upper-bound-minus-one to the lower bound from
versioncheck.php. Using the minimum supported PHP ensures
deterministic autoloader output regardless of what PHP versions
are available on the build machine.

Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
- Skip root-prefixed .nextcloudignore patterns: the release script's
  `git ls-files -ico` silently skips these for tracked files, so
  composer.json, package.json, docs/ etc. are kept in release archives.
  Match that behavior instead of being more aggressive.
- Stop removing *.map.license in assemble.sh (release script keeps them).
- Add vendor-bin/ to DEV_PATTERNS (dev tooling, not runtime).

Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
@skjnldsv skjnldsv force-pushed the feature/tag-repos-on-release branch from c3d0ab0 to 73ea4b5 Compare June 2, 2026 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant