Skip to content

Add an opt-in SwiftPM job limit to coordinated builds and tests - #523

Open
morluto wants to merge 5 commits into
repoprompt:mainfrom
morluto:agent/476-swift-jobs
Open

Add an opt-in SwiftPM job limit to coordinated builds and tests#523
morluto wants to merge 5 commits into
repoprompt:mainfrom
morluto:agent/476-swift-jobs

Conversation

@morluto

@morluto morluto commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Why

Conductor serializes heavy jobs across worktrees, but one admitted Swift build can still use many compiler workers. Global heavy slots and SwiftPM worker limits solve different problems: one bounds concurrent conductor jobs; the other bounds parallelism inside one job.

This PR adds the second control without changing default behavior.

Closes #476.

What changes

Adds:

REPOPROMPT_SWIFT_JOBS=<positive integer>

When set, coordinated SwiftPM build, test, provider-test, packaging, run, and release-product paths receive --jobs <N>.

Contract

  • Unset: existing commands remain unchanged.
  • Set: positive decimal integer becomes the SwiftPM worker limit.
  • Invalid values are rejected only by SwiftPM-invoking operations.
  • Non-Swift operations remain unaffected.
  • Request fingerprints include the value, preventing request-key reuse across different limits.
  • Request environment is authoritative; daemon startup environment cannot leak a stale limit into jobs.

This is developer workload control, not application state, so it remains an environment contract at the coordinated command boundary.

Review order

  1. parsing and request-environment handling
  2. SwiftPM argv construction
  3. packaging propagation
  4. request-key and non-Swift regression tests
  5. developer documentation

Verification

  • conductor self-tests
  • environment-authority tests
  • request-key tests
  • shell/Python parsing parity tests
  • non-Swift operation regression test
  • guardrails and contribution preflights

morluto added 3 commits July 12, 2026 02:31
- `configured_swift_jobs` validates and parses the env var; `append_swift_jobs_arg`
  appends `--jobs N` only to coordinated SwiftPM invocations.
- Request env is authoritative: when an env dict is provided we do not fall back to
  the daemon's `os.environ`, preventing cross-job leakage.
- Integer validation matches the shell scripts exactly: positive decimal integer with
  no leading zeros and no sign (`^[1-9][0-9]*$`).
- Wired into `swift-build`, `test`, `provider-test`, `swift_build_all`, and the
  `package_app.sh` / `build_swiftpm_release_products.sh` packaging paths.
- Added `REPOPROMPT_SWIFT_JOBS` to `CONDUCTOR_ENV_KEYS` and echoed the effective limit
  in daemon logs and `run_operation_command` output.
- Added focused tests for env authority, fingerprint reuse, unaffected non-Swift ops,
  packaging-script regex parity, and shell/Python validation consistency.
- Documented the contract in `AGENTS.md`.

Closes repoprompt#476
Only parse/validate the Swift jobs limit in DaemonState._run_job for
swift-build, test, and provider-test operations. Packaging/build scripts and
non-Swift lanes (sleep, diagnostics, etc.) must not be blocked by an invalid
REPOPROMPT_SWIFT_JOBS value.

Add daemon-level regression tests proving that invalid values do not affect
sleep while swift-build product=all rejects the job in the daemon runner.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@baron

baron commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Current-main refresh on exact head a1d0fe066d927609722d37c2722a9f38ea3455fe: the opt-in job-limit goal remains relevant, but this branch conflicts across AGENTS.md, conductor, packaging, and tests. Please port the narrow limit onto current heavy-admission/conductor semantics and rerun fresh exact-head CI before re-review.

morluto commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

Audit disposition — rebase and validate the real SwiftPM commands (2026-08-14)

The request-environment authority, request-key fingerprinting, and propagation through coordinated build/test paths are coherent. The remaining gap is executable compatibility: argv-construction unit tests do not prove every generated form is accepted by the pinned SwiftPM.

Please run the exact commands, especially swift test list --jobs N, provider-test list/filter forms, and both packaging/release scripts. Bound the accepted numeric length/value so pathological input cannot fail during integer conversion. Subject to current-base CI and those live command checks, I do not see an architectural blocker.

morluto commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

Deep-review assessment — 2026-08-14

Disposition: code-ready after real SwiftPM invocation checks. Request-environment authority, request-key separation, shell/Python validation parity, and propagation through packaging are coherent. The remaining gap is that the tests mainly inspect generated argv; they do not establish that every emitted command shape is accepted by the active SwiftPM version.

Please execute each path at least once on the final head, especially swift test list --jobs N and provider-test/list option placement, and add a reasonable digit/maximum bound before converting the environment value to an integer. With those current-head checks, I did not find an architectural blocker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow conductor Swift jobs to cap build parallelism

2 participants