Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 71 additions & 13 deletions .github/workflows/build-release.yaml
Original file line number Diff line number Diff line change
@@ -1,40 +1,85 @@
# (C) 2022 GoodData Corporation
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
# The single workflow that publishes to PyPI -- both stable and dev releases.
#
# * Stable release: triggered by pushing a vX.Y.Z tag (see bump-version.yaml). Builds every
# component, creates the GitHub release, publishes to PyPI and notifies #releases.
# * Dev release: triggered manually (workflow_dispatch). Bumps the checked-out branch to the
# next N.N.N.devN version, builds and publishes to PyPI. No GitHub release, no notification.
#
# Both paths must stay in this one file. PyPI attestations record the workflow file that
# published each release, and PyPI flags a "Change of provenance" whenever it differs from
# the previous release's. With dev and stable releases in separate files, every stable
# release that followed a dev release was flagged.

name: Build Python Package and Create Release

on:
push:
tags:
- v*.*.*
workflow_dispatch:
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
inputs:
BRANCH_NAME:
type: string
description: 'Branch name to dev-release from'
default: "master"

# One release at a time. Each tag is unique, so runs never collide on the tag itself, but
# is_latest is computed once per run: with two releases in flight, the earlier tag can
# finish last and take the "Latest" badge back from the newer one. Serializing keeps the
# badge in release order. Never cancels -- a cancelled run leaves components half-published.
# One release of each kind at a time. Each tag is unique, so runs never collide on the tag
# itself, but is_latest is computed once per run: with two releases in flight, the earlier
# tag can finish last and take the "Latest" badge back from the newer one. Serializing keeps
# the badge in release order. For dev releases it keeps two runs from computing the same
# devN version from PyPI. Never cancels -- a cancelled run leaves components half-published.
#
# Stable and dev runs use separate groups: a group holds only one pending run, and a newer
# pending run cancels it. In a shared group, a dev release dispatched while a tag release
# waits would cancel the tag release.
concurrency:
group: build-release
group: build-release-${{ github.event_name }}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
cancel-in-progress: false

env:
COMPONENTS: '["gooddata-api-client","gooddata-pandas","gooddata-fdw","gooddata-sdk","gooddata-dbt","gooddata-flight-server","gooddata-flexconnect","gooddata-pipelines","gooddata-eval"]'

jobs:
matrix-components:
name: Prepare matrix components
name: Prepare components and version
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
components: ${{ steps.export.outputs.components }}
# Tag push: the tag. Dev release: the branch head, resolved once so every build job
# builds the same commit even if the branch moves mid-run.
ref: ${{ steps.dev.outputs.sha || github.ref }}
# Dev release only. Computed once and shared, so all components get the same version
# and re-running a failed build job reuses it instead of picking the next devN.
dev_version: ${{ steps.dev.outputs.version }}
steps:
- name: Export components JSON
id: export
run: echo "components=${COMPONENTS}" >> "$GITHUB_OUTPUT"
# The script runs from this workflow's revision, not from BRANCH_NAME, so dev releases
# also work from branches that predate it. The branch only supplies the commit to build
# and the version to start from.
- name: Checkout the workflow revision
if: github.event_name == 'workflow_dispatch'
uses: actions/checkout@v5
with:
persist-credentials: false
- name: Compute dev version
id: dev
if: github.event_name == 'workflow_dispatch'
env:
BRANCH_NAME: ${{ inputs.BRANCH_NAME }}
run: |
git fetch --no-tags --depth=1 origin "$BRANCH_NAME"
sha=$(git rev-parse FETCH_HEAD)
git show "$sha:pyproject.toml" > "$RUNNER_TEMP/branch-pyproject.toml"
# shellcheck disable=SC2046 # one argument per component
version=$(python3 scripts/next_dev_version.py --pyproject "$RUNNER_TEMP/branch-pyproject.toml" $(jq -r '.[]' <<< "$COMPONENTS"))
echo "Dev release $version from $sha"
echo "sha=$sha" >> "$GITHUB_OUTPUT"
echo "version=$version" >> "$GITHUB_OUTPUT"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

build:
name: Build all components
Expand All @@ -47,10 +92,20 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: ${{ needs.matrix-components.outputs.ref }}
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@v10.0.1
with:
prune-cache: true
- name: Bump to dev version
if: github.event_name == 'workflow_dispatch'
env:
DEV_VERSION: ${{ needs.matrix-components.outputs.dev_version }}
run: |
uv sync --only-group release --locked
uv run tbump --only-patch --non-interactive "$DEV_VERSION"
- name: Build ${{ matrix.component }}
run: |
if [ "${{ matrix.component }}" = "gooddata-api-client" ]; then
Expand All @@ -69,6 +124,7 @@ jobs:

tag-checks:
name: Check the triggering tag
if: github.event_name == 'push'
runs-on: ubuntu-latest
permissions:
contents: read
Expand All @@ -84,6 +140,7 @@ jobs:

github_release:
name: Create GitHub release
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs:
- build
Expand Down Expand Up @@ -140,6 +197,7 @@ jobs:
packages-dir: dist/${{ matrix.component }}
verbose: true
slack-notification:
if: github.event_name == 'push'
runs-on: ubuntu-latest
needs: publish
steps:
Expand Down
84 changes: 0 additions & 84 deletions .github/workflows/dev-release.yaml

This file was deleted.

2 changes: 2 additions & 0 deletions .github/workflows/rw-collect-changes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ jobs:
- '.docker/**'
- 'packages/**'
- 'scripts/docs/**'
- 'scripts/next_dev_version.py'
- 'scripts/tests/**'
- '*.mk'
- 'Makefile'
- Dockerfile
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,8 +287,8 @@ release. Add a dependency to the package's `pyproject.toml`, then run `uv lock`.

Three entries in the root `pyproject.toml` — `[project].dependencies`,
`[tool.uv.sources]`, and `[tool.uv.workspace].members` (listed explicitly, not globbed, so
a stray directory cannot break every `uv` command). Then add it to `COMPONENTS` in both
`.github/workflows/dev-release.yaml` and `build-release.yaml`, to the Codecov file list in
a stray directory cannot break every `uv` command). Then add it to `COMPONENTS` in
`.github/workflows/build-release.yaml`, to the Codecov file list in
`rw-python-tests.yaml`, and to `scopes` in `.gitlint`.

## Documentation
Expand Down
6 changes: 2 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,9 @@

When adding a new distributable package to this monorepo, update release automation and PyPI configuration together:

1. Add the package name to `COMPONENTS` in:
- `.github/workflows/dev-release.yaml`
- `.github/workflows/build-release.yaml`
1. Add the package name to `COMPONENTS` in `.github/workflows/build-release.yaml` (it publishes both stable and dev releases).
2. Verify the package is built by release workflows and artifacts are uploaded from its `dist/` directory.
3. Configure the package on PyPI to use Trusted Publisher for this repository/workflow combination.
3. Configure the package on PyPI to use Trusted Publisher for this repository and the `build-release.yaml` workflow.
4. Run/observe a release workflow and confirm publishing succeeds via OIDC (no `PYPI_API_TOKEN` required).

### Adding Support for a New Python Version
Expand Down
7 changes: 6 additions & 1 deletion MAINTENANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,12 @@ The docs site keeps the four newest release branches, sorted by `major.minor`, a
appears in the docs.

### How-to dev release
To publish current master as a dev release version, use [Dev release from master](.github/workflows/dev-release.yaml) GitHub workflow.
To publish current master as a dev release version, manually run the
[Build Python Package and Create Release](.github/workflows/build-release.yaml) workflow (optionally with a
different `BRANCH_NAME`). A manual run bumps to the next `N.N.N.devN` version and publishes to PyPI only, with
no GitHub release and no Slack notification. Dev and stable releases share one workflow on purpose: PyPI
shows a "Change of provenance" warning when a release is published from a different workflow file than the
previous one.

### Errors that may appear

Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ remove-cassettes:

.PHONY: test-docs-scripts
test-docs-scripts:
uv run pytest scripts/docs/tests/ -v
uv run pytest scripts/docs/tests/ scripts/tests/ -v

.PHONY: new-docs
new-docs:
Expand Down
30 changes: 0 additions & 30 deletions scripts/next_dev.sh

This file was deleted.

Loading
Loading