Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/2026-11-06-local-build-tool-hints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Development: on a Mac with Docker Desktop, the isolated package builds now say to pick an output folder under your home folder when Docker can't mount one under `/private/tmp`. A local `npm run sdk:check` without `dsp_asm` now skips the Octatrack Analog BD package with a note instead of failing on a missing file.
2 changes: 1 addition & 1 deletion docs/DEVELOPER_WORKFLOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ These stumbling blocks have each cost a contributor and the owner a repair round
- **The first failure hides the next.** `npm run check` runs every catalogue, licence and changelog check of a stage and reports them together, but stops before the tests and build when one fails, so a fix can still uncover another. Run `pr:ready` until it is Ready instead of pushing after each fix. Between fixes, `npm run check -- --base origin/main --rerun-failed` skips what passed in the last run; it is a shortcut, not a pass.
- **Two PRs, one changelog line.** Application changelog entries are one file each, `changelog/<YYYY-MM-DD>-<topic>.md`; module notes are one file per module in `src/community/module-changelogs/`. Never add entries to `CHANGELOG.md`: the check refuses them.
- **Merged before green.** The `main` ruleset requires the `module-pr-ok` check, which passes only when every other PR check passed or was skipped by scope. The owner approves a first-time contributor's run and turns on auto-merge, and GitHub merges once it is green; a red or unfinished PR does not merge.
- **Compiled packages.** The `octatrack-source` and `elemod-source` jobs rebuild packages in the pinned Docker toolchain and require the committed ones to match. Reproduce them before you push: `bash scripts/build-modules-isolated.sh <clean checkout> <new dir> <image id>` for the Octatrack, `bash scripts/build-elemod-isolated.sh` and `node scripts/verify-elemod-release-exports.mjs <dir>` for the Digitakt and Digitone, from a clean, committed tree. `sdk:check` rebuilds and compares the published Octatrack packages (`scripts/octatrack-module-packages.py --check`, writes nothing) when `m68k-elf` binutils and the Elekloader checkout are both present; then it needs `dsp_asm`: `ELEKLOADER_DSP_ASM`, PATH, or the octabam checkout `npm run upstream:tools` keeps after `make setup` there (`sdk/machines/octatrack/module-adapters/base_bound.py`).
- **Compiled packages.** The `octatrack-source` and `elemod-source` jobs rebuild packages in the pinned Docker toolchain and require the committed ones to match. Reproduce them before you push: `bash scripts/build-modules-isolated.sh <clean checkout> <new dir> <image id>` for the Octatrack, `bash scripts/build-elemod-isolated.sh` and `node scripts/verify-elemod-release-exports.mjs <dir>` for the Digitakt and Digitone, from a clean, committed tree. `sdk:check` rebuilds and compares the published Octatrack packages (`scripts/octatrack-module-packages.py --check`, writes nothing) when `m68k-elf` binutils and the Elekloader checkout are both present; a package that carries a DSP engine (Analog BD) also needs `dsp_asm` (`ELEKLOADER_DSP_ASM`, PATH, or the octabam checkout `npm run upstream:tools` keeps after `make setup` there) and is skipped with a note without it.
- **Checking a draft.** `npm run module:doctor -- --draft <id>` checks `sdk/drafts/<id>` in place: it copies the folder under `sdk/octabam/modules/`, where every gate and `build.py --octabam` look (a symlink there is refused), and removes the copy when the run ends. The listing lines (catalog entry, module rules, packages, USB package, native comparison, pair table, package fingerprint) stay red for a draft; the gates, the USB package line aside, should be green.
- **A check edited to pass.** A module pull request that changes a check which judges modules (the compatibility and pair gates, the module rules, the doctor, their tests, the exception file, a workflow) is sent back, and `pr:ready` refuses it. A red check names the fix; a wrong check is its own `Fix:` pull request, and the module waits for it.
- **Archived source snapshots.** Before browser-builder scripts, run `node scripts/machine-modules.mjs` and `node scripts/module-changelogs.mjs --write`: their ignored catalogue/changelog JSON is absent from `git archive`.
Expand Down
7 changes: 6 additions & 1 deletion scripts/build-elemod-isolated.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,9 @@ staging_dir=$(mktemp -d "$(dirname "$output_dir")/.modwerk-source.XXXXXX")
trap 'rm -rf "$staging_dir"' EXIT
# Archive tracked source only. Ignored local firmware, history and secrets never mount.
git -C "$source_dir" archive "$source_commit" | tar -x -C "$staging_dir"
docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges --read-only --user "$(id -u):$(id -g)" --cpus 2 --memory 4g --pids-limit 128 --tmpfs /tmp:rw,nosuid,nodev,noexec,size=256m --tmpfs /test:rw,nosuid,nodev,exec,size=16m,mode=1777 --env HOME=/tmp --env LC_ALL=C --env TZ=UTC --env SOURCE_DATE_EPOCH=0 --mount "type=bind,source=$staging_dir,target=/source,readonly" --mount "type=bind,source=$output_dir,target=/output" "$image_id" node scripts/build-elemod-packages.mjs --output /output --source-commit "$source_commit"
docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges --read-only --user "$(id -u):$(id -g)" --cpus 2 --memory 4g --pids-limit 128 --tmpfs /tmp:rw,nosuid,nodev,noexec,size=256m --tmpfs /test:rw,nosuid,nodev,exec,size=16m,mode=1777 --env HOME=/tmp --env LC_ALL=C --env TZ=UTC --env SOURCE_DATE_EPOCH=0 --mount "type=bind,source=$staging_dir,target=/source,readonly" --mount "type=bind,source=$output_dir,target=/output" "$image_id" node scripts/build-elemod-packages.mjs --output /output --source-commit "$source_commit" || {
status=$?
# Docker Desktop on macOS can refuse a bind source under /private/tmp ("bind source path does not exist").
case "$output_dir" in /private/*) echo "Docker may not share $(dirname "$output_dir"): choose an output folder under \$HOME." >&2;; esac
exit "$status"
}
7 changes: 6 additions & 1 deletion scripts/build-modules-isolated.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,9 @@ git -C "$source_dir" cat-file commit "$source_commit" | git -C "$staging_dir" ha
git -C "$source_dir" rev-parse "$source_commit^{tree}" | git -C "$source_dir" pack-objects --revs --stdout | git -C "$staging_dir" unpack-objects -q
git -C "$staging_dir" update-ref HEAD "$source_commit"
git -C "$staging_dir" read-tree "$source_commit"
docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges --read-only --user "$(id -u):$(id -g)" --cpus 2 --memory 4g --pids-limit 128 --tmpfs /tmp:rw,nosuid,nodev,size=256m --env HOME=/tmp --mount "type=bind,source=$staging_dir,target=/source,readonly" --mount "type=bind,source=$output_dir,target=/output" "$image_id" python3 -B scripts/build-module-packages.py --vendor /opt/toolchain/vendor --output /output/packages --source-commit "$source_commit"
docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges --read-only --user "$(id -u):$(id -g)" --cpus 2 --memory 4g --pids-limit 128 --tmpfs /tmp:rw,nosuid,nodev,size=256m --env HOME=/tmp --mount "type=bind,source=$staging_dir,target=/source,readonly" --mount "type=bind,source=$output_dir,target=/output" "$image_id" python3 -B scripts/build-module-packages.py --vendor /opt/toolchain/vendor --output /output/packages --source-commit "$source_commit" || {
status=$?
# Docker Desktop on macOS can refuse a bind source under /private/tmp ("bind source path does not exist").
case "$output_dir" in /private/*) echo "Docker may not share $(dirname "$output_dir"): choose an output folder under \$HOME." >&2;; esac
exit "$status"
}
18 changes: 18 additions & 0 deletions scripts/octatrack-module-packages.py
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,24 @@ def c_build(entry, out, *args, **kwargs):
if sourced and tools:
builder.build = c_build
builder.tools = lambda entry, upstream: bool(shutil.which('m68k-elf-gcc')) if entry.get('sources') else tools(entry, upstream)
# A package that carries an engine (hook_engines.PACKAGES, built through base_bound) also needs the native DSP
# assembler, found as base_bound finds it. Without one, --check skips it as it skips the rest, and says why.
gated = getattr(builder, 'tools', None)
def engine_tools(entry, upstream):
if not entry.get('base'):
return gated(entry, upstream)
adapters = str(APP / 'sdk/machines/octatrack/module-adapters')
if adapters not in sys.path:
sys.path.append(adapters)
import base_bound
import hook_engines
if entry['id'] in hook_engines.PACKAGES and not (
os.environ.get('ELEKLOADER_DSP_ASM') or shutil.which('dsp_asm') or base_bound.DSP_ASM.is_file()):
print('%-18s needs dsp_asm: ELEKLOADER_DSP_ASM, PATH, or `npm run upstream:tools` with octabam built' % entry['id'])
return False
return gated(entry, upstream)
if gated:
builder.tools = engine_tools
# An SDK-owned recipe preserves an ordinary CF runtime's exact tested
# image while shipping guarded stock copies. This host layer keeps the
# base's pinned builder and existing standalone declarations unchanged;
Expand Down
Loading