Split package publish from Docker and Helm release by ramantehlan · Pull Request #861 · truefoundry/trueforge

ramantehlan · 2026-09-23T11:41:16Z

Summary

release-packages.yml (was release.yml) stays packages-only work plus one extra job: after pack and Windows smoke, npm publish, PyPI publish, and Create release tag (chart/v<packages/trueforge version>) run in parallel. The tag is pushed with the GitHub App token because GITHUB_TOKEN tag pushes do not start other workflows.
release-chart.yml is triggered by that chart/v* tag (or manual dispatch). It builds the from-source image, packages and pushes the Helm chart to OCI, and commits the chart metadata back to main. No chart-release bot PR and no merge gate.
Prod images build from the workspace root Dockerfile (the former Dockerfile.dev); the npm-install image is gone. The Helm controller command matches it (node dist/controller-main.js).
Chart version is locked to the same major.minor as @truefoundry/trueforge, so chart, package, and image tag prefix stay aligned (scripts/resolve-chart-version.sh).

Release flows

1. Version or publish packages - `release-packages.yml`

flowchart LR
  trigger["Trigger<br/>push to main<br/>or manual workflow_dispatch"]

  select["Select mode"]
  versionJob["Version packages"]
  pack["Pack packages"]
  smoke["Windows npx smoke"]
  npmJob["Publish packages (npm, OIDC)"]
  pypi["Publish Python SDK (PyPI, OIDC)"]
  tagJob["Create release tag<br/>chart/v&lt;version&gt; (App token)"]

  trigger --> select
  select -->|"pending .changeset files"| versionJob
  select -->|"no pending changesets,<br/>unpublished versions on main"| pack
  select -->|"no pending changesets,<br/>unpublished versions on main"| smoke

  pack --> npmJob
  smoke --> npmJob
  pack --> pypi
  smoke --> pypi
  pack --> tagJob
  smoke --> tagJob

2. Build server image and publish Helm chart - `release-chart.yml`

flowchart LR
  trigger["Trigger<br/>push of chart/v* tag<br/>or manual workflow_dispatch<br/>(optional app_version)"]

  resolve["Resolve versions<br/>(tag must match package.json)"]
  build["Build and push server image<br/>{version}-{shortSha}"]
  publishChart["Publish Helm chart<br/>bump, lint, package, OCI push,<br/>commit chart files to main"]

  trigger --> resolve
  resolve --> build
  resolve --> publishChart
  build --> publishChart

How they connect

flowchart LR
  packages["Version or publish packages<br/>release-packages.yml"]
  imageChart["Build server image and publish Helm chart<br/>release-chart.yml"]

  packages -->|"chart/v* tag (App token)"| imageChart

Test plan

Update npm and PyPI trusted publishers to workflow filename release-packages.yml before the first publish (they were bound to release.yml).
On a publish-mode run, confirm npm, PyPI, and the chart/v* tag job all start after pack and smoke, in parallel.
Confirm the chart/v* tag push starts release-chart.yml (needs the App token, not GITHUB_TOKEN).
Confirm the image is from-source (root Dockerfile, tag {version}-{shortSha}) and the chart is pushed to OCI with the same major.minor as the package.
Confirm the chart metadata commit lands on main (Chart.yaml, values.yaml).
Confirm Compose/Railway still build the root Dockerfile, and the controller starts with node dist/controller-main.js.
Manual rebuild path: gh workflow run release-chart.yml --ref main.

Note

High Risk
Changes production release automation, how prod images are built, and CI pushes chart metadata directly to protected main; misconfiguration can break publishes or ship mismatched chart/image versions.

Overview
Decouples npm/PyPI publishing from container and Helm releases, and replaces the npm-install production image with a single from-source root Dockerfile (former Dockerfile.dev is removed).

After pack and Windows smoke, release-packages.yml (renamed from release.yml) now runs npm, PyPI, and a new chart/v<version> tag job in parallel (App token so the tag can trigger downstream workflows). It no longer waits on npm or calls the deleted build-and-prepare-chart-release.yml / release-chart/trueforge bot PR flow.

release-chart.yml is driven by chart/v* tags (or manual dispatch): resolve versions from packages/trueforge/package.json, build and push {version}-{shortSha} from the workspace Dockerfile, lint/package/push the Helm chart (treats registry “already exists” as success), then commit Chart.yaml and values.yaml to main with controller command node dist/controller-main.js.

scripts/resolve-chart-version.sh now pins chart major.minor to the app package so chart, package, and image tag prefix stay aligned. Compose, Railway IaC, dev image workflow, and docs are updated to build the root Dockerfile.

Ops note: npm and PyPI trusted publishers must be updated to workflow filename release-packages.yml before the next publish.

^{Reviewed by Cursor Bugbot for commit 755405b. Bugbot is set up for automated code reviews on this repo. Configure here.}

Remove the npm-install Dockerfile so Docker and Helm no longer wait for npm, and publish the chart in the same image workflow instead of a bot PR. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>

changeset-bot · 2026-09-23T11:41:20Z

⚠️ No Changeset found

Latest commit: 755405b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

cursor

Stale Bugbot comment from a previous run.

Rename the package workflow to release-packages.yml, drop the Helm tag-retry job, and start the chart workflow on main with its own Changesets select-mode. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>

The package workflow now pushes chart/v<version> in parallel with npm and PyPI, and release-chart.yml builds the image and publishes the chart from that tag. Drops the chart-release bot PR and merge gate. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>

cursor

Stale Bugbot comment from a previous run.

Replay chart metadata onto origin/main with the App token, treat an already-published OCI version as success, and leave the npm-install controller command in values.yaml until that publish rewrites it. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>

cursor

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 755405b. Configure here.}

cursor · 2026-09-23T14:19:02Z

+            exit 1
+          fi
+          SHORT_SHA="${GITHUB_SHA:0:7}"
+          IMAGE_TAG="${APP_VERSION}-${SHORT_SHA}"


Image tag uses annotated tag SHA

Medium Severity

The automated chart path creates an annotated chart/v* tag, then tags the image with the first seven characters of GITHUB_SHA. On an annotated-tag push that value is the tag object, not the commit, so {version}-{shortSha} no longer matches the build commit. A later dispatch rebuild of the same commit gets a different image tag, and the reusable build checkout may also receive that tag-object SHA.

Additional Locations (1)

.github/workflows/release-packages.yml#L306-L308

^{Reviewed by Cursor Bugbot for commit 755405b. Configure here.}

cursor · 2026-09-23T14:19:02Z

      drainingSeconds: 35,
    },
    env: {
-      // From-source image; root Dockerfile is the published npm recipe and needs APP_VERSION.


Railway still points at deleted Dockerfile

High Severity

Existing Railway services still have RAILWAY_DOCKERFILE_PATH=Dockerfile.dev from the previous IaC apply. This change deletes Dockerfile.dev and only drops that env from railway.ts, which does not update already-applied services. The next git-triggered rebuild looks for a file that is gone and fails until someone re-applies IaC.

Additional Locations (1)

Dockerfile#L1-L12

^{Reviewed by Cursor Bugbot for commit 755405b. Configure here.}

chiragjn · 2026-09-24T04:25:14Z

+  # dist/controller-main.js when it publishes a from-source image.
  command:
    - node
    - node_modules/@truefoundry/trueforge/dist/controller-main.js


Is this still correct?

from what I understand, it should be

Suggested change

- node_modules/@truefoundry/trueforge/dist/controller-main.js

- dist/controller-main.js

chiragjn · 2026-09-24T04:25:49Z

should we just keep the old contents of this as Dockerfile.npm ?

chiragjn · 2026-09-24T04:27:41Z

where is this workflow used? we probably don't have use case for it

we can keep it, but rename to build-image.yml

chiragjn · 2026-09-24T04:28:23Z

let's not change the filename here please, all trusted publishing will break
we can do it as a separate activity

chiragjn · 2026-09-24T04:42:55Z

  push:
    tags:
-      - 'charts/trueforge@*'
+      - 'chart/v[0-9]*'


keep it same as before please?
'charts/trueforge@*'

It can be changed as part of new release process PRs

chiragjn · 2026-09-24T04:54:40Z

+            yq -i '.version = strenv(VERSION)' charts/trueforge/Chart.yaml
+            yq -i '.appVersion = strenv(APP_VERSION)' charts/trueforge/Chart.yaml
+            yq -i '.image.tag = strenv(IMAGE_TAG)' charts/trueforge/values.yaml
+            yq -i '.controller.command = ["node", "dist/controller-main.js"]' charts/trueforge/values.yaml


this doesn't need to be here, it can be just part of values directly?

chiragjn · 2026-09-24T05:01:38Z

-          helm push "$PACKAGE" "oci://$HELM_CHART_REPOSITORY"
+          set +e
+          PUSH_OUT=$(helm push "$PACKAGE" "oci://$HELM_CHART_REPOSITORY" 2>&1)
+          PUSH_RC=$?


what does RC in PUSH_RC mean here? 😅

Split package publish from from-source image and Helm release.

36e95e5

Remove the npm-install Dockerfile so Docker and Helm no longer wait for npm, and publish the chart in the same image workflow instead of a bot PR. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>

ramantehlan requested review from bhaveshpatel640, chiragjn, debajyoti-truefoundry, heerambavi1998, sr07asthana and thesujai as code owners September 23, 2026 11:41

cursor Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread charts/trueforge/values.yaml Outdated

Comment thread .github/workflows/release-image-chart.yml Outdated

ramantehlan added 2 commits September 23, 2026 17:54

Keep package publish and Helm release as separate workflows.

0e29845

Rename the package workflow to release-packages.yml, drop the Helm tag-retry job, and start the chart workflow on main with its own Changesets select-mode. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>

cursor Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread .github/workflows/release-chart.yml Outdated

ramantehlan added 2 commits September 23, 2026 19:35

Merge branch 'main' into sd/docker-updates

c1091a8

cursor Bot reviewed Sep 23, 2026

View reviewed changes

chiragjn reviewed Sep 24, 2026

View reviewed changes

Comment thread Dockerfile

chiragjn Sep 24, 2026

Copy link
Copy Markdown

Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we just keep the old contents of this as Dockerfile.npm ?

chiragjn reviewed Sep 24, 2026

View reviewed changes

	- node_modules/@truefoundry/trueforge/dist/controller-main.js
	- dist/controller-main.js

Conversation

ramantehlan commented Sep 23, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Release flows

1. Version or publish packages - release-packages.yml

2. Build server image and publish Helm chart - release-chart.yml

How they connect

Test plan

Uh oh!

changeset-bot Bot commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Sep 23, 2026

Choose a reason for hiding this comment

Image tag uses annotated tag SHA

Uh oh!

cursor Bot Sep 23, 2026

Choose a reason for hiding this comment

Railway still points at deleted Dockerfile

Uh oh!

chiragjn Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

chiragjn Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

chiragjn Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

chiragjn Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

chiragjn Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

chiragjn Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

chiragjn Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ramantehlan commented Sep 23, 2026 •

edited by cursor Bot

Loading

1. Version or publish packages - `release-packages.yml`

2. Build server image and publish Helm chart - `release-chart.yml`

changeset-bot Bot commented Sep 23, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading