Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ All notable changes to LLooM will be documented in this file. The format follows

### Added

- Atlas SparkQwen RC16 two-Spark recipe (`linux-nvidia-dgx-spark-2x-qwen38-flash-next-atlas`, profile `8x262k`) with a fail-closed pin gate and a digest-only image installer, and explicit per-node `${fabricHca}` settings for distributed recipes. Both Atlas recipes pin the published images with exact rail selection; missing settings and ambiguous discovery fail clearly.

- LLooM Hear CPU audio analysis, with structured estimates, optional dashboard images, and opt-in upstream interpretation. File and URL inputs require operator configuration.

- Four standalone NVIDIA Spark audio recipes (Qwen3-TTS 1.7B CustomVoice, Base voice cloning and VoiceDesign, and Whisper large-v3-turbo). Each runs one pinned, integrity-checked checkpoint in its own container from a locally built `spark-audio` image that replaces a host-local prebuilt image.
Expand Down
25 changes: 25 additions & 0 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,3 +186,28 @@ The model `nvidia/GLM-5.3-Flash-NVFP4` (MIT per SparkGLM's licensing notes)
and the DFlash2 drafter `incoai/GLM-5.3-Flash-DFlash2` (**CC BY-NC-ND 4.0,
non-commercial**) are downloaded from their publishers under their own terms.
See the [backend README](backends/atlas-sparkglm/README.md#credits-and-licenses).

## Atlas SparkQwen engine image (not included)

The optional Atlas SparkQwen lane in `backends/atlas-sparkqwen` and
`recipes/linux-nvidia-dgx-spark-2x-qwen38-flash-next-atlas.json` pulls the
container image `ghcr.io/enntity/atlas-sparkqwen` built from
[Enntity/sparkqwen](https://github.com/Enntity/sparkqwen) at
`85325e5104b2a6864e0cf31ec470a8c3327d4de4` (install tree `656234c336af`).
LLooM starts that image as a separate program; no engine source or binary is
included in the LLooM npm package, and LLooM's files for the lane remain MIT.

- Engine: [Atlas](https://github.com/Atlas-Inf/atlas) (Atlas-Inf) with
SparkGLM's upstream series and SparkQwen's Qwen3.8-Flash-Next work on the
[Enntity/atlas](https://github.com/Enntity/atlas) fork, at
[`1c43247d93ca050f02cb67b0603e88edfe3d92a5`](https://github.com/Enntity/atlas/tree/1c43247d93ca050f02cb67b0603e88edfe3d92a5)
(recorded in the image's `/opt/atlas/source-manifest.json`)
- License: AGPL-3.0-only. The corresponding source is Enntity/sparkqwen at the
revision above plus that Enntity/atlas commit; SparkQwen's `NOTICE` and
`docs/LICENSING.md` list what the image build fetches. The image ships
CUTLASS's license and the Rust crate notices under `/opt/atlas/notices/`;
the engine's license is at `/LICENSE`.

The model `nvidia/Qwen3.8-Flash-Next-NVFP4` (NVIDIA Open Model License per its
model card) is downloaded from its publisher under its own terms. See the
[backend README](backends/atlas-sparkqwen/README.md#credits-and-licenses).
6 changes: 3 additions & 3 deletions backends/atlas-sparkglm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
LLooM-managed two-node Atlas SparkGLM candidate for a directly connected pair
of NVIDIA DGX Spark systems. This directory is **MIT orchestration only**: no
Atlas engine source is committed here. The engine image is built from immutable
revision `3c7d67146929f350ed75fcdfd5e5349db843b483` of `Enntity/sparkglm` by
revision `7a67d9bce4b5f408ad67f2a958ef1a8787136fdd` of `Enntity/sparkglm` by
that repository's `install/build.sh`. The engine inside the image is
AGPL-3.0-only; see [Credits and licenses](#credits-and-licenses). Live hardware
and full serving qualification remain pending.
Expand All @@ -26,7 +26,7 @@ and full serving qualification remain pending.
install-tree label is a contents check, not an artifact identity.
- `install.sh` — prepares the image. The normal path reuses an already
verified local image, otherwise pulls the immutable manifest digest
`ghcr.io/enntity/atlas-sparkglm@sha256:e4d313b5e32c47545eab0059eea65a38e65d03fc8e0d5b8c2cfb497c39f0372a`
`ghcr.io/enntity/atlas-sparkglm@sha256:78420594991e6e7e3914f84a6300ac1cf2f4e18b68e331f8ddc74fd7db905d3c`
from GHCR. It never pulls the mutable tag, and a failed pinned digest pull is
a hard failure with no silent source build. The explicit `--source-build`
option instead clones `Enntity/sparkglm` at the pinned revision, checks that
Expand Down Expand Up @@ -235,7 +235,7 @@ is not part of LLooM:
`ghcr.io/enntity/atlas-sparkglm`, alongside LLooM (an aggregate); no engine
source or binary is included in LLooM.
- **Corresponding source.** [Enntity/sparkglm](https://github.com/Enntity/sparkglm)
at the pinned revision (`3c7d6714`) plus the Enntity/atlas commit recorded
at the pinned revision (`7a67d9bc`) plus the Enntity/atlas commit recorded
in the image's `/opt/atlas/source-manifest.json`
([`dfb9c26a`](https://github.com/Enntity/atlas/tree/dfb9c26a95c96faa9f94c44e23343b0f2b4f9db6)).
SparkGLM's `NOTICE` and `docs/LICENSING.md` list everything else it fetches.
Expand Down
8 changes: 4 additions & 4 deletions backends/atlas-sparkglm/pins.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@
"note": "Single pin manifest for the Atlas SparkGLM candidate. Portable source, image, model and converter identities are final for this candidate; live hardware and serving qualification remains pending. The normal install path addresses image.reference, the immutable GHCR manifest digest. The tag is retained as source-build metadata (it derives from the git tree of install/ at the pinned revision) and as the local tag produced by an explicit --source-build; the install-tree label is a contents check. A tag or label alone is never the artifact identity.",
"source": {
"repo": "Enntity/sparkglm",
"revision": "3c7d67146929f350ed75fcdfd5e5349db843b483",
"installTree": "246a3a948ab98d19e085bfd39797e554f0ee9261",
"revision": "7a67d9bce4b5f408ad67f2a958ef1a8787136fdd",
"installTree": "f95a9af14236a781bb13f57dcc9ea0342f339905",
"cloneUrl": "https://github.com/Enntity/sparkglm.git",
"buildScript": "install/build.sh"
},
"image": {
"reference": "ghcr.io/enntity/atlas-sparkglm@sha256:e4d313b5e32c47545eab0059eea65a38e65d03fc8e0d5b8c2cfb497c39f0372a",
"tag": "ghcr.io/enntity/atlas-sparkglm:246a3a948ab9",
"reference": "ghcr.io/enntity/atlas-sparkglm@sha256:78420594991e6e7e3914f84a6300ac1cf2f4e18b68e331f8ddc74fd7db905d3c",
"tag": "ghcr.io/enntity/atlas-sparkglm:f95a9af14236",
"label": "io.enntity.sparkglm.install-tree",
"entrypoint": "/opt/atlas/serve.py",
"profilePath": "/opt/atlas/profiles/4x1m.json",
Expand Down
132 changes: 132 additions & 0 deletions backends/atlas-sparkqwen/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# Atlas SparkQwen under LLooM

LLooM-managed two-node Atlas SparkQwen RC16 lane for a directly connected pair
of NVIDIA DGX Spark systems, serving `nvidia/Qwen3.8-Flash-Next-NVFP4` as
published (no overlay conversion, no separate drafter: speculation uses the
checkpoint's own MTP head). This directory is **MIT orchestration only**: no
Atlas engine source is committed here. The engine image is built from
`Enntity/sparkqwen` revision `8b28467324893583838871b6a6b6a28d3059e64f`
(install tree `4ccd961ecf09`) by that repository's `install/build.sh` and
published as `ghcr.io/enntity/atlas-sparkqwen:4ccd961ecf09`. The engine inside
the image is AGPL-3.0-only; see [Credits and licenses](#credits-and-licenses).
SparkQwen qualified RC16 on its own launcher; no live run through LLooM is
recorded yet.

## Image digest

`pins.json` addresses `ghcr.io/enntity/atlas-sparkqwen@sha256:9408884dc4bf2c84b7dd097d62effc73f58fffc25a1d67db73d0b4e69516a7c0`, the GHCR
manifest digest of tag `4ccd961ecf09` (recorded 2026-10-13 with
`docker buildx imagetools inspect`). Both recipe members use the same
reference as `bootstrap.image`.

## Layout

- `pins.json` — the single pin manifest: SparkQwen source revision and its
`install/` git tree, the engine commit, the immutable image reference with
the tag kept as metadata, the install-tree label, and the checkpoint revision.
- `verify-pins.mjs` — fail-closed pin gate. Exits non-zero while the manifest
is not `final`, any identity is missing or a DRAFT/TODO placeholder, the
revision, install tree or engine commit is not a 40-character git object id,
the entrypoint or profile is not a plain path under `/opt/atlas`, the tag is not
`ghcr.io/enntity/atlas-sparkqwen:` plus the first 12 characters of the install
tree, `image.reference` is not `ghcr.io/enntity/atlas-sparkqwen@sha256:` plus
64 lowercase hex characters, an image ID is pinned, or the model leaves
`nvidia/Qwen3.8-Flash-Next-NVFP4@fc694b54fb0174e0913e6adf86691ef85a4ead47`.
- `install.sh` — prepares the image: reuses an already verified local image,
otherwise pulls the immutable digest. It never pulls the mutable tag and
never builds; a failed pull or check is a hard failure. The image must be
arm64 with `io.enntity.sparkqwen.install-tree` equal to the pinned tree and
must ship `/opt/atlas/serve.py`, `/opt/atlas/profiles/8x262k.json` and the
pinned engine commit in `/opt/atlas/source-manifest.json`. `--check-only`
verifies without pulling. For a local build, run `install/build.sh` from
Enntity/sparkqwen at the pinned revision; that produces the tag, not the
digest this recipe addresses.

## Installation

```sh
lloom setup --recipe linux-nvidia-dgx-spark-2x-qwen38-flash-next-atlas --additive --apply --yes
lloom runtime-start qwen38-flash-next-atlas-cluster
```

The recipe is additive: it sets no default model and adds no aliases. Setup
steps, in order: `check-docker`, `verify-atlas-pins`, `download-atlas-model`
(the pinned checkpoint, about 124 GiB, into
`${modelRoot}/nvidia--Qwen3.8-Flash-Next-NVFP4`), `pull-atlas-image`
(`install.sh`), and `prepare-atlas-cuda-cache` (creates
`${installRoot}/atlas-sparkqwen-cuda-cache`, mounted at `/atlas-cuda-cache` so
compiled CUDA kernels survive container recreation). Run setup on both nodes.

## Container contract

The image ships its own entrypoint and profiles; LLooM supplies environment and
mounts only, mirroring SparkQwen's reviewed `install/start-node.sh`: host
network and IPC, 32 GiB shm, `--memory 114g`, all GPUs, `/dev/infiniband`,
`IPC_LOCK` and `SYS_NICE`, unlimited memlock, `no-new-privileges`, a 60 s stop
timeout, the checkpoint mounted read-only at the same absolute path, and the
CUDA JIT cache.

| Variable | Source | Meaning |
| ------------------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `NODE_RANK` | `${nodeRank}` | `0` on the leader, `1` on the worker |
| `MASTER_ADDR` | `${leaderAddress}` | the leader's direct-fabric address |
| `MASTER_PORT` | `29510` | distributed rendezvous port |
| `FABRIC_INTERFACE` | `${fabricInterface}` | the node's direct-fabric NIC, also `NCCL_SOCKET_IFNAME`/`GLOO_SOCKET_IFNAME` |
| `FABRIC_HCA` | `${fabricHca}` | the node's RoCE device (below); `serve.py` adds the device behind the port's other PCIe half and sets `NCCL_IB_HCA` and `ATLAS_RDMA_RAILS` from both |
| `MODEL_PATH` | `${modelRoot}/nvidia--Qwen3.8-Flash-Next-NVFP4` | the checkpoint |
| `SERVED_MODEL_NAME` | `qwen3.8-flash-next-atlas` | gateway model ID |
| `SPARKQWEN_PROFILE` | `8x262k` | `/opt/atlas/profiles/8x262k.json`: up to eight requests of up to 262,144 tokens from one shared BF16 KV pool (about 3.19M tokens at the profile's GPU memory utilization of 0.88); the image also ships `8x32k` and `4x262k` |
| `SPARKQWEN_FP8_GDN` | `0` | the lossy FP8 Gated DeltaNet opt-in stays off |

The profile sets every `ATLAS_*` and `NCCL_*` tuning value; `serve.py` drops
ambient `ATLAS_*` variables so the profile wins. To give the KV pool more of a
dedicated pair's memory, add `-e SPARKQWEN_GPU_MEMORY_UTILIZATION=0.80-0.95` to
both members in a local recipe override.

`${fabricHca}` and `${fabricInterface}` are materialized per node by LLooM from
the node's own config. Because recipes that consume `${fabricHca}` require exact,
node-local device names, LLooM expects `cluster.nodes.<id>.fabricHca` (a
comma-separated list of exact local RDMA device names, without spaces or `:port`
suffixes) and `cluster.nodes.<id>.fabricInterface` on every participating node;
it never guesses an HCA from the NIC name, since PCI naming and cabling differ by
host. A pair cabled on the second port therefore needs
`fabricHca: "rocep1s0f1"` and `fabricInterface: "enp1s0f1np1"` on both nodes.
Missing or malformed overrides stop recipe materialization with the node and
field named in the error.

Ports are fixed by the image's `serve.py` and loopback-bound: rank 0 serves
`127.0.0.1:8893` on the leader and rank 1 binds `127.0.0.1:8894` on the worker.
The leader's port is what LLooM health-checks and routes to; the worker is
readiness-checked through its container state. SparkGLM's lane uses the same
ports, so the two lanes cannot run on the same pair at once (neither fits
beside the other anyway).

## Distributed startup

The worker starts first (`order` 10, rank 1, `healthStrategy: "container"`, no
warmup); the leader starts second (`order` 20, rank 0), health-checks
`/health`, runs a POST warmup and owns routing. The first start compiles CUDA
kernels into the cache; allow several minutes.

## Credits and licenses

LLooM's files in this directory and the recipe are MIT. The engine they start
is not part of LLooM:

- **Engine.** [Atlas](https://github.com/Atlas-Inf/atlas) (Atlas-Inf) with
SparkGLM's upstream series and SparkQwen's Qwen3.8-Flash-Next work on the
[Enntity/atlas](https://github.com/Enntity/atlas) fork, branch
`sparkqwen/atlas-20261010-rc16`, licensed **AGPL-3.0-only**. It runs from a
separate container image, `ghcr.io/enntity/atlas-sparkqwen`, alongside LLooM
(an aggregate); no engine source or binary is included in LLooM.
- **Corresponding source.** [Enntity/sparkqwen](https://github.com/Enntity/sparkqwen)
at the pinned revision plus the Enntity/atlas commit recorded in the image's
`/opt/atlas/source-manifest.json`
([`1c43247d`](https://github.com/Enntity/atlas/tree/1c43247d93ca050f02cb67b0603e88edfe3d92a5)).
SparkQwen's `NOTICE` and `docs/LICENSING.md` list everything else the build
fetches; the image ships CUTLASS's license and the Rust crate notices under
`/opt/atlas/notices/` and the engine's license at `/LICENSE`.
- **Model.** Weights are downloaded from their publisher, not distributed by
LLooM. `nvidia/Qwen3.8-Flash-Next-NVFP4` is under the **NVIDIA Open Model
License** per its model card (SparkQwen `docs/LICENSING.md`); read the model
card before use and before redistributing anything derived from it.
Loading
Loading