diff --git a/changelog/2026-11-06-local-build-tool-hints.md b/changelog/2026-11-06-local-build-tool-hints.md new file mode 100644 index 000000000..375e58be5 --- /dev/null +++ b/changelog/2026-11-06-local-build-tool-hints.md @@ -0,0 +1 @@ +Development: on a Mac with Docker Desktop, the isolated package builds now say to pick an output folder under your home folder when Docker can't mount one under `/private/tmp`. A local `npm run sdk:check` without `dsp_asm` now skips the Octatrack Analog BD package with a note instead of failing on a missing file. diff --git a/docs/DEVELOPER_WORKFLOW.md b/docs/DEVELOPER_WORKFLOW.md index e5502486f..06981e2bd 100644 --- a/docs/DEVELOPER_WORKFLOW.md +++ b/docs/DEVELOPER_WORKFLOW.md @@ -53,7 +53,7 @@ These stumbling blocks have each cost a contributor and the owner a repair round - **The first failure hides the next.** `npm run check` runs every catalogue, licence and changelog check of a stage and reports them together, but stops before the tests and build when one fails, so a fix can still uncover another. Run `pr:ready` until it is Ready instead of pushing after each fix. Between fixes, `npm run check -- --base origin/main --rerun-failed` skips what passed in the last run; it is a shortcut, not a pass. - **Two PRs, one changelog line.** Application changelog entries are one file each, `changelog/-.md`; module notes are one file per module in `src/community/module-changelogs/`. Never add entries to `CHANGELOG.md`: the check refuses them. - **Merged before green.** The `main` ruleset requires the `module-pr-ok` check, which passes only when every other PR check passed or was skipped by scope. The owner approves a first-time contributor's run and turns on auto-merge, and GitHub merges once it is green; a red or unfinished PR does not merge. -- **Compiled packages.** The `octatrack-source` and `elemod-source` jobs rebuild packages in the pinned Docker toolchain and require the committed ones to match. Reproduce them before you push: `bash scripts/build-modules-isolated.sh ` for the Octatrack, `bash scripts/build-elemod-isolated.sh` and `node scripts/verify-elemod-release-exports.mjs ` for the Digitakt and Digitone, from a clean, committed tree. `sdk:check` rebuilds and compares the published Octatrack packages (`scripts/octatrack-module-packages.py --check`, writes nothing) when `m68k-elf` binutils and the Elekloader checkout are both present; then it needs `dsp_asm`: `ELEKLOADER_DSP_ASM`, PATH, or the octabam checkout `npm run upstream:tools` keeps after `make setup` there (`sdk/machines/octatrack/module-adapters/base_bound.py`). +- **Compiled packages.** The `octatrack-source` and `elemod-source` jobs rebuild packages in the pinned Docker toolchain and require the committed ones to match. Reproduce them before you push: `bash scripts/build-modules-isolated.sh ` for the Octatrack, `bash scripts/build-elemod-isolated.sh` and `node scripts/verify-elemod-release-exports.mjs ` for the Digitakt and Digitone, from a clean, committed tree. `sdk:check` rebuilds and compares the published Octatrack packages (`scripts/octatrack-module-packages.py --check`, writes nothing) when `m68k-elf` binutils and the Elekloader checkout are both present; a package that carries a DSP engine (Analog BD) also needs `dsp_asm` (`ELEKLOADER_DSP_ASM`, PATH, or the octabam checkout `npm run upstream:tools` keeps after `make setup` there) and is skipped with a note without it. - **Checking a draft.** `npm run module:doctor -- --draft ` checks `sdk/drafts/` in place: it copies the folder under `sdk/octabam/modules/`, where every gate and `build.py --octabam` look (a symlink there is refused), and removes the copy when the run ends. The listing lines (catalog entry, module rules, packages, USB package, native comparison, pair table, package fingerprint) stay red for a draft; the gates, the USB package line aside, should be green. - **A check edited to pass.** A module pull request that changes a check which judges modules (the compatibility and pair gates, the module rules, the doctor, their tests, the exception file, a workflow) is sent back, and `pr:ready` refuses it. A red check names the fix; a wrong check is its own `Fix:` pull request, and the module waits for it. - **Archived source snapshots.** Before browser-builder scripts, run `node scripts/machine-modules.mjs` and `node scripts/module-changelogs.mjs --write`: their ignored catalogue/changelog JSON is absent from `git archive`. diff --git a/scripts/build-elemod-isolated.sh b/scripts/build-elemod-isolated.sh index dde6651af..e03fe4693 100644 --- a/scripts/build-elemod-isolated.sh +++ b/scripts/build-elemod-isolated.sh @@ -17,4 +17,9 @@ staging_dir=$(mktemp -d "$(dirname "$output_dir")/.modwerk-source.XXXXXX") trap 'rm -rf "$staging_dir"' EXIT # Archive tracked source only. Ignored local firmware, history and secrets never mount. git -C "$source_dir" archive "$source_commit" | tar -x -C "$staging_dir" -docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges --read-only --user "$(id -u):$(id -g)" --cpus 2 --memory 4g --pids-limit 128 --tmpfs /tmp:rw,nosuid,nodev,noexec,size=256m --tmpfs /test:rw,nosuid,nodev,exec,size=16m,mode=1777 --env HOME=/tmp --env LC_ALL=C --env TZ=UTC --env SOURCE_DATE_EPOCH=0 --mount "type=bind,source=$staging_dir,target=/source,readonly" --mount "type=bind,source=$output_dir,target=/output" "$image_id" node scripts/build-elemod-packages.mjs --output /output --source-commit "$source_commit" +docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges --read-only --user "$(id -u):$(id -g)" --cpus 2 --memory 4g --pids-limit 128 --tmpfs /tmp:rw,nosuid,nodev,noexec,size=256m --tmpfs /test:rw,nosuid,nodev,exec,size=16m,mode=1777 --env HOME=/tmp --env LC_ALL=C --env TZ=UTC --env SOURCE_DATE_EPOCH=0 --mount "type=bind,source=$staging_dir,target=/source,readonly" --mount "type=bind,source=$output_dir,target=/output" "$image_id" node scripts/build-elemod-packages.mjs --output /output --source-commit "$source_commit" || { + status=$? + # Docker Desktop on macOS can refuse a bind source under /private/tmp ("bind source path does not exist"). + case "$output_dir" in /private/*) echo "Docker may not share $(dirname "$output_dir"): choose an output folder under \$HOME." >&2;; esac + exit "$status" +} diff --git a/scripts/build-modules-isolated.sh b/scripts/build-modules-isolated.sh index e479db58a..3b7496f54 100644 --- a/scripts/build-modules-isolated.sh +++ b/scripts/build-modules-isolated.sh @@ -25,4 +25,9 @@ git -C "$source_dir" cat-file commit "$source_commit" | git -C "$staging_dir" ha git -C "$source_dir" rev-parse "$source_commit^{tree}" | git -C "$source_dir" pack-objects --revs --stdout | git -C "$staging_dir" unpack-objects -q git -C "$staging_dir" update-ref HEAD "$source_commit" git -C "$staging_dir" read-tree "$source_commit" -docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges --read-only --user "$(id -u):$(id -g)" --cpus 2 --memory 4g --pids-limit 128 --tmpfs /tmp:rw,nosuid,nodev,size=256m --env HOME=/tmp --mount "type=bind,source=$staging_dir,target=/source,readonly" --mount "type=bind,source=$output_dir,target=/output" "$image_id" python3 -B scripts/build-module-packages.py --vendor /opt/toolchain/vendor --output /output/packages --source-commit "$source_commit" +docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges --read-only --user "$(id -u):$(id -g)" --cpus 2 --memory 4g --pids-limit 128 --tmpfs /tmp:rw,nosuid,nodev,size=256m --env HOME=/tmp --mount "type=bind,source=$staging_dir,target=/source,readonly" --mount "type=bind,source=$output_dir,target=/output" "$image_id" python3 -B scripts/build-module-packages.py --vendor /opt/toolchain/vendor --output /output/packages --source-commit "$source_commit" || { + status=$? + # Docker Desktop on macOS can refuse a bind source under /private/tmp ("bind source path does not exist"). + case "$output_dir" in /private/*) echo "Docker may not share $(dirname "$output_dir"): choose an output folder under \$HOME." >&2;; esac + exit "$status" +} diff --git a/scripts/octatrack-module-packages.py b/scripts/octatrack-module-packages.py index f70cc190e..d1e749b49 100644 --- a/scripts/octatrack-module-packages.py +++ b/scripts/octatrack-module-packages.py @@ -182,6 +182,24 @@ def c_build(entry, out, *args, **kwargs): if sourced and tools: builder.build = c_build builder.tools = lambda entry, upstream: bool(shutil.which('m68k-elf-gcc')) if entry.get('sources') else tools(entry, upstream) + # A package that carries an engine (hook_engines.PACKAGES, built through base_bound) also needs the native DSP + # assembler, found as base_bound finds it. Without one, --check skips it as it skips the rest, and says why. + gated = getattr(builder, 'tools', None) + def engine_tools(entry, upstream): + if not entry.get('base'): + return gated(entry, upstream) + adapters = str(APP / 'sdk/machines/octatrack/module-adapters') + if adapters not in sys.path: + sys.path.append(adapters) + import base_bound + import hook_engines + if entry['id'] in hook_engines.PACKAGES and not ( + os.environ.get('ELEKLOADER_DSP_ASM') or shutil.which('dsp_asm') or base_bound.DSP_ASM.is_file()): + print('%-18s needs dsp_asm: ELEKLOADER_DSP_ASM, PATH, or `npm run upstream:tools` with octabam built' % entry['id']) + return False + return gated(entry, upstream) + if gated: + builder.tools = engine_tools # An SDK-owned recipe preserves an ordinary CF runtime's exact tested # image while shipping guarded stock copies. This host layer keeps the # base's pinned builder and existing standalone declarations unchanged;