Skip to content

fix(api/dashboard): interpolate Compose vars from the configured deploy env - #407

Merged
Hydralerne merged 3 commits into
oblien:mainfrom
Rish-it:fix/issue-383-compose-scan-env
Aug 3, 2026
Merged

fix(api/dashboard): interpolate Compose vars from the configured deploy env#407
Hydralerne merged 3 commits into
oblien:mainfrom
Rish-it:fix/issue-383-compose-scan-env

Conversation

@Rish-it

@Rish-it Rish-it commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

A Compose file that declares a required variable (${VAR:?message}) fails the wizard scan even when the operator has already entered that variable in the Openship deploy configuration. The scan interpolates against the repo .env alone, so the configured env never reaches the parser. Thread that env through to parseComposeFile, which already accepts it.

Motivation

toProjectInfo is the single place the wizard parses Compose (prepare.service.ts:780 on main):

const parsed = parseComposeFile(composeContent, { envFileContent: composeEnvContent });

composeEnvContent is a .env read from beside the Compose file. It is the only interpolation source. Source and ResolveOptions carry no env at all, so there is no way for a caller to supply one — the values the operator typed into the wizard are simply not in scope at the point the file is parsed.

Compose treats ${VAR:?message} as fatal when VAR is unset, so the parse throws and the wizard reports the file as broken. Reproduced against 0f59f94f with the reporter's layout — Compose pinned outside the repo root (the composePath feature from #330), one required variable, supplied through the deploy configuration:

supplied env main this branch
POSTGRES_PASSWORD=s3cret Error: Could not parse the Docker Compose file at "deploy/docker-compose": Set POSTGRES_PASSWORD in .env services[db].environment.POSTGRES_PASSWORD === "s3cret"
nothing supplied same error same error (unchanged)

The second row is the behaviour worth keeping: a genuinely unset required variable must still be reported. #339 established that swallowing a parse failure returns a services project with zero services and no reason why, and the existing reports missing required Compose variables test pins that. This change only suppresses the error when the operator actually provided the value.

The parser side already supports this. ComposeParseOptions.env is documented as "Explicit interpolation values. Overrides values loaded from envFileContent" (compose-parser.ts:69) and buildInterpolationEnv layers it over the .env map (:424). Nothing new was needed there — only the plumbing to reach it.

Related issue

Closes #383

Changes

apps/api

  • modules/deployments/prepare.service.tsResolveOptions gains env, mirrored onto both Source variants; forwarded through resolveFromReader into toProjectInfo and on to parseComposeFile. ResolveOptions already reaches both resolveFromGitHub and resolveFromLocal, so neither resolver needed touching.
  • modules/deployments/deployment.schema.tsPrepareDeployBody.env, a Type.Record(Type.String(), Type.String()), matching how project.schema.ts and service.schema.ts already declare env maps. The route is validated, so the shape is checked at the trust boundary rather than in the handler.
  • modules/deployments/deployment.controller.ts — pass the body's env into both the github and local Source. Interpolation-only: not persisted here, and the response already masks every service env through maskScanService (Service environment secrets returned in plaintext in scan/deployment/service API responses #336), so a supplied value cannot be echoed back unmasked.
  • test/modules/deployments/prepare.service.test.ts — one regression test on the existing tmpdir harness, reproducing the reporter's nested deploy/docker-compose layout.

apps/dashboard

  • context/deployment/useDeploymentConfig.ts — new scanEnv helper folds config.envVars into a record; rescanWithComposePath carries it through both initialize paths. Empty maps are dropped so a blank value never reaches the API, matching how the neighbouring scanComposePath handles an unset pin.
  • lib/api/deploy.tsenv on PrepareProjectSource.

rescanWithComposePath is the path that matters here: it re-reads the source precisely because projectType, the service list and each service's env can only come from the Compose file, so it is the one flow where the operator has already entered env and then triggers a fresh parse.

Verification

RED first, on the branch with the fix reverted — the new test reproduces the report verbatim:

 × interpolates required Compose variables from the configured deploy env
   Error: Could not parse the Docker Compose file at "deploy/docker-compose": Set POSTGRES_PASSWORD in .env
    789|       throw new Error(`Could not parse the Docker Compose file${where}…

GREEN, and the pre-existing missing-variable test still passes beside it:

 ✓ reports missing required Compose variables instead of returning no services
 ✓ interpolates required Compose variables from the configured deploy env
 Tests  16 passed (16)

Full API suite, two separate clean runs:

$ bun run --cwd apps/api test
 Test Files  174 passed | 1 skipped (175)
      Tests  1861 passed | 4 skipped (1865)
   Duration  41.39s

$ bun run --cwd apps/api test
 Test Files  174 passed | 1 skipped (175)
      Tests  1861 passed | 4 skipped (1865)
   Duration  45.86s

Dashboard suite and typecheck:

$ bun run --cwd apps/dashboard test
 Test Files  19 passed (19)
      Tests  236 passed (236)

$ turbo run lint --filter=@repo/api     # tsc --noEmit
 Tasks:    4 successful, 4 total

$ npx tsc --noEmit -p apps/dashboard/tsconfig.json
 (clean)

Re-run against committed HEAD after the three commits: 16/16 on the focused file, old failure gone.

Two notes on the checklist rather than a silent tick:

  • bun format is not run. All five touched files already fail prettier --check on main — verified by checking out main and running it there — so formatting them would reformat lines this PR has no business touching.
  • turbo run lint --filter=@repo/dashboard fails on main as well as here, with Invalid project directory provided, no such directory: apps/dashboard/lint. The next lint script is incompatible with the installed Next; unrelated to this change. Used tsc --noEmit on the dashboard instead, which is clean.

Questions for a maintainer

@Hydralerne — a few calls I would rather you make than assume:

  1. Scope of the endpoint change. The template asks for prior agreement on new endpoints and schema changes. This adds an optional field to an existing route's body rather than a new endpoint, and I read it as within "bug fix" — but it is a request-shape change, so say the word if you would rather it went through an issue first.

  2. Three sibling callers left unwired, deliberately. resolveProjectInfo has three other callers with the identical gap, and I did not touch them because each needs a decision about where project env is read from:

    • reconcileComposeDrift (build.service.ts:470) — the interesting one. Its parse failure is caught into a console.warn, so on a Compose file with required variables, drift reconciliation silently stops tracking upstream changes and nothing surfaces. It has the project record in hand, so wiring it is a question of which env you consider authoritative there.
    • detectStack (github.controller.ts:834) — a GET, so env would have to arrive as a query parameter or the route would need to change shape.
    • scanLocal (project.controller.ts:923).

    Happy to do any or all in this PR or a follow-up — tell me which, and where env should come from for the drift path.

  3. No dashboard test. apps/dashboard has no jsdom or Testing Library; its render tests use renderToStaticMarkup, which runs no effects, so a hook like rescanWithComposePath cannot be exercised. I did not add a test framework to cover it. If you would rather that half were covered, that is a jsdom addition and a bigger conversation than this fix.

The API half is genuinely covered — real resolver, real parser, real temp filesystem.

Checklist

  • One change per PR — one bug, or one agreed feature, with nothing unrelated bundled in
  • The diff is scoped — no reformatting or lint fixes on lines I wasn't otherwise changing
  • A test fails without this change and passes with it (or I explained above why there isn't one)
  • bun run test, bun run --cwd <workspace> lint, and bun format all pass locally — see the two notes under Verification: bun format deliberately not run, and dashboard lint is broken on main independently of this change
  • I understand every line of this diff and can explain it in review

Rish-it added 3 commits August 3, 2026 14:21
A compose file declaring a required variable (`${VAR:?message}`) was
scanned against the repo `.env` alone. Env the user had configured in
Openship never reached the parser, so the scan reported the file as
unparseable even though the deploy would resolve the same variable:

    Could not parse the Docker Compose file at "deploy/docker-compose":
    Set POSTGRES_PASSWORD in .env

`parseComposeFile` already accepts explicit interpolation values that
override the ones loaded from `envFileContent`. Thread the caller's env
to it: `Source` -> `ResolveOptions` -> `resolveFromReader` ->
`toProjectInfo`. `ResolveOptions` already reaches both the GitHub and
local resolvers, so neither needed changing.

Omitting the env keeps the existing behaviour — a genuinely unset
required variable is still reported rather than silently dropped.
Expose the resolver's new interpolation env on the scan route so the
wizard can send what the user already entered. Declared on
`PrepareDeployBody` as a string->string record, so the shape is
validated at the trust boundary like every other env map on the API.

The value is interpolation-only: it is not persisted here, and the
response already masks every service env via `maskScanService`, so a
supplied secret cannot be echoed back unmasked.
`rescanWithComposePath` re-reads the source because projectType, the
service list and each service's env can only come from the compose
file. On a file with required variables that re-scan failed, since the
env the user had just entered was not part of the request.

Carry `config.envVars` through both initialize paths into the prepare
body. Empty maps are dropped so a blank value never reaches the API,
matching how `scanComposePath` handles an unset pin.
Copilot AI review requested due to automatic review settings August 3, 2026 08:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Compose wizard scans failing on required ${VAR:?message} substitutions by interpolating Docker Compose using the deployment’s configured environment (in addition to the repo-adjacent .env), so scans succeed when the operator has already provided the needed variables.

Changes:

  • Adds an optional env map to the prepare request schema/types and threads it through API resolution into parseComposeFile.
  • Updates the dashboard prepare call path (rescanWithComposePath) to include already-configured env vars for interpolation.
  • Adds an API regression test covering a pinned Compose subpath with a required variable.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
apps/dashboard/src/lib/api/deploy.ts Extends PrepareProjectSource to allow sending an optional env map for compose interpolation.
apps/dashboard/src/context/deployment/useDeploymentConfig.ts Builds and forwards env during rescan flows so required vars don’t break Compose parsing.
apps/api/test/modules/deployments/prepare.service.test.ts Adds a regression test proving required vars interpolate from configured env with a pinned compose path.
apps/api/src/modules/deployments/prepare.service.ts Plumbs env through resolution and passes it into parseComposeFile options.
apps/api/src/modules/deployments/deployment.schema.ts Adds optional env to PrepareDeployBody at the validation boundary.
apps/api/src/modules/deployments/deployment.controller.ts Forwards optional env into the prepare source while keeping masked output behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Hydralerne
Hydralerne merged commit cca2fbe into oblien:main Aug 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: docker-compose scan not including env

3 participants