Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,14 +158,14 @@ The optional Atlas SparkGLM lane in `backends/atlas-sparkglm` and
`recipes/linux-nvidia-dgx-spark-2x-glm53-atlas.json` pulls or builds the
container image `ghcr.io/enntity/atlas-sparkglm` from
[Enntity/sparkglm](https://github.com/Enntity/sparkglm) at
`5063c13ea6f196046b76ce6e5c8b4d4200cccc9a`. LLooM starts that image as a
`4dee61dcb522d6f3d2246c920ce1a52d87548c71`. LLooM starts that image as a
separate program; no engine source or binary is included in the LLooM npm
package, and LLooM's files for the lane remain MIT.

- Engine: [Atlas](https://github.com/Atlas-Inf/atlas) (Atlas-Inf) with
SparkGLM's GLM-5.3 Flash work on the
[Enntity/atlas](https://github.com/Enntity/atlas) fork, at
[`6f27e8fa0afe65180a169e54361a77565787ae27`](https://github.com/Enntity/atlas/tree/6f27e8fa0afe65180a169e54361a77565787ae27)
[`289cefbccdd6c13117228863c7237da17e47a92a`](https://github.com/Enntity/atlas/tree/289cefbccdd6c13117228863c7237da17e47a92a)
(recorded in the image's `/opt/atlas/source-manifest.json`)
- License: AGPL-3.0-only. The corresponding source is Enntity/sparkglm at the
revision above plus that Enntity/atlas commit; SparkGLM's `NOTICE` and
Expand Down
50 changes: 23 additions & 27 deletions backends/atlas-sparkglm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
LLooM-managed two-node Atlas SparkGLM candidate for a directly connected pair
of NVIDIA DGX Spark systems. This directory is **MIT orchestration only**: no
Atlas engine source is committed here. The engine image is built from immutable
revision `5063c13ea6f196046b76ce6e5c8b4d4200cccc9a` of `Enntity/sparkglm` by
revision `4dee61dcb522d6f3d2246c920ce1a52d87548c71` of `Enntity/sparkglm` by
that repository's `install/build.sh`. The engine inside the image is
AGPL-3.0-only; see [Credits and licenses](#credits-and-licenses). Live hardware
and full serving qualification remain pending. This revision includes the
issue 172 fix that restores full blocking-refusal content and the existing
per-node fabric support.
and full serving qualification remain pending. This revision adds image prompts
that use the prefix cache (content-keyed vision KV, per-sequence encoding) and
the display carveout from the driver's scanout heap (no CAP_SYS_ADMIN).

## Layout

Expand All @@ -28,7 +28,7 @@ per-node fabric support.
install-tree label is a contents check, not an artifact identity.
- `install.sh` — prepares the image. The normal path reuses an already
verified local image, otherwise pulls the immutable manifest digest
`ghcr.io/enntity/atlas-sparkglm@sha256:5ec57d5522a95463b600a17471ce60712b315566e04c0a3bb6af1d84a1de0a4a`
`ghcr.io/enntity/atlas-sparkglm@sha256:94b837ef3ea7b3f54c158ce568e611405b1bd167edd56c734d8e90d2d6291cb6`
from GHCR. It never pulls the mutable tag, and a failed pinned digest pull is
a hard failure with no silent source build. The explicit `--source-build`
option instead clones `Enntity/sparkglm` at the pinned revision, checks that
Expand Down Expand Up @@ -124,7 +124,7 @@ Inside the image:
share one physical FP8-latent KV pool, split between the nodes by `SPARKGLM_KV_SHARD=1` (about 2.38M tokens at the recipe's 0.91 GPU memory utilization with the display carveout and the disk prefix cache, up to 1,048,576 per request with `SPARKGLM_PROFILE=4x1m`; it also holds the prefix cache, with 16 recurrent-state snapshot slots); it does not reserve
four full windows. `disable-tool-grammar` is deliberately **not** set, so
structured output and tool grammar stay functional. The engine is built from
Enntity/atlas `fix/sparkglm-rc4.2-refusal-content` @ `6f27e8fa` (recorded in the
Enntity/atlas `sparkglm/atlas-20261014-vpc` @ `289cefbc` (recorded in the
image's `/opt/atlas/source-manifest.json`); measured results are in
Enntity/sparkglm `results/2026-09-30-decode-step/` (this engine) and
`results/2026-09-30-nvme-tier/` (this image).
Expand Down Expand Up @@ -186,25 +186,21 @@ cache on disk".
## Display memory as KV cache (on)

The recipe sets `SPARKGLM_DISPLAY_CARVEOUT=1` on both members. Each DGX Spark's
firmware reserves 2 GiB for the display (`DISPLAY_FRM`), which Linux never sees
and the GB10 driver never allocates from; the engine borrows it for whole
per-layer latent KV pools, so the pool grows about 19% (about 1.13M to 1.38M
tokens at 0.91) with system memory unchanged. The GPU does not cache this
memory in L2, so the sparse-index buffers stay in ordinary memory, where cold
prefill keeps its speed (Enntity/sparkglm#30). To make that possible the recipe adds, on
both members:

- `--cap-add SYS_ADMIN`: exporting the carveout needs it. The image starts the
engine through `spark display-carveout`, which exports the memory, takes a
host lock, drops `CAP_SYS_ADMIN` from every capability set (bounding set
included) and only then starts the server; `no-new-privileges` stays on.
- a bind mount of `${installRoot}/atlas-carveout-lock` (created by the
prepare step, so it survives reboots) at `/run/lock/sparkglm`: one process per
host may hold the carveout.

The engine uses the carveout only with validated NVIDIA drivers (580.173.02 and
580.178.04) and serves without it, logging why, when it cannot export it. To roll
back, set `SPARKGLM_DISPLAY_CARVEOUT=0` (the capability and mount are then
firmware reserves 2 GiB for the display (`DISPLAY_FRM`), which Linux never sees;
the NVIDIA driver manages it as the display's memory pool, and a headless Spark's
display uses only a few MiB of it. The image starts the engine through
`spark display-carveout`, which asks the driver for the rest (about 2042 MiB),
GPU-cacheable, and lends it to whole per-layer latent KV pools, so the pool grows
with system memory unchanged. Because the driver hands it out, it never overlaps
the display's own memory and needs no extra container capability. The recipe
adds, on both members, a bind mount of `${installRoot}/atlas-carveout-lock`
(created by the prepare step, so it survives reboots) at `/run/lock/sparkglm`:
one process per host holds the carveout.

Any NVIDIA open kernel module from 580.76.05 on lends it; the engine checks what
the driver gives rather than its version, and serves without the carveout,
logging why, when it cannot get it. To roll back, set
`SPARKGLM_DISPLAY_CARVEOUT=0` (the mount is then
unused). It is SparkGLM-only (the Enntity/atlas fork's SparkGLM layer); details
and credits are in Enntity/sparkglm's README under "Display memory as KV cache".

Expand Down Expand Up @@ -237,9 +233,9 @@ is not part of LLooM:
`ghcr.io/enntity/atlas-sparkglm`, alongside LLooM (an aggregate); no engine
source or binary is included in LLooM.
- **Corresponding source.** [Enntity/sparkglm](https://github.com/Enntity/sparkglm)
at the pinned revision (`5063c13e`) plus the Enntity/atlas commit recorded
at the pinned revision (`4dee61dc`) plus the Enntity/atlas commit recorded
in the image's `/opt/atlas/source-manifest.json`
([`6f27e8fa`](https://github.com/Enntity/atlas/tree/6f27e8fa0afe65180a169e54361a77565787ae27)).
([`289cefbc`](https://github.com/Enntity/atlas/tree/289cefbccdd6c13117228863c7237da17e47a92a)).
SparkGLM's `NOTICE` and `docs/LICENSING.md` list everything else it fetches.
- **Third-party notices in the image.** FlashKDA (MoonshotAI, MIT), FlashInfer
including NVIDIA's sparse-MLA prefill source (Apache-2.0) and CUTLASS
Expand Down
8 changes: 4 additions & 4 deletions backends/atlas-sparkglm/pins.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@
"note": "Single pin manifest for the Atlas SparkGLM candidate. Portable source, image, model and converter identities are final for this candidate; live hardware and serving qualification remains pending. The normal install path addresses image.reference, the immutable GHCR manifest digest. The tag is retained as source-build metadata (it derives from the git tree of install/ at the pinned revision) and as the local tag produced by an explicit --source-build; the install-tree label is a contents check. A tag or label alone is never the artifact identity.",
"source": {
"repo": "Enntity/sparkglm",
"revision": "5063c13ea6f196046b76ce6e5c8b4d4200cccc9a",
"installTree": "ed7548dfe442949283b54c7160605429046cd46f",
"revision": "4dee61dcb522d6f3d2246c920ce1a52d87548c71",
"installTree": "2b2970c2f05d346fa934da8df8f7d3ba551e07c3",
"cloneUrl": "https://github.com/Enntity/sparkglm.git",
"buildScript": "install/build.sh"
},
"image": {
"reference": "ghcr.io/enntity/atlas-sparkglm@sha256:5ec57d5522a95463b600a17471ce60712b315566e04c0a3bb6af1d84a1de0a4a",
"tag": "ghcr.io/enntity/atlas-sparkglm:ed7548dfe442",
"reference": "ghcr.io/enntity/atlas-sparkglm@sha256:94b837ef3ea7b3f54c158ce568e611405b1bd167edd56c734d8e90d2d6291cb6",
"tag": "ghcr.io/enntity/atlas-sparkglm:2b2970c2f05d",
"label": "io.enntity.sparkglm.install-tree",
"entrypoint": "/opt/atlas/serve.py",
"profilePath": "/opt/atlas/profiles/4x1m.json",
Expand Down
8 changes: 4 additions & 4 deletions docs/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -374,11 +374,11 @@ the git tree of its `install/` directory, the immutable image manifest
reference (`image.reference`) with the tag kept as source-build metadata and
the install-tree contents label, the `nvidia/GLM-5.3-Flash-NVFP4` and drafter
revisions, and the conversion marker contract. The current source pin is product revision
`5063c13ea6f196046b76ce6e5c8b4d4200cccc9a` with install tree
`ed7548dfe442949283b54c7160605429046cd46f`, so the normal install path
`4dee61dcb522d6f3d2246c920ce1a52d87548c71` with install tree
`2b2970c2f05d346fa934da8df8f7d3ba551e07c3`, so the normal install path
addresses the immutable digest
`ghcr.io/enntity/atlas-sparkglm@sha256:5ec57d5522a95463b600a17471ce60712b315566e04c0a3bb6af1d84a1de0a4a`.
The tag `ghcr.io/enntity/atlas-sparkglm:ed7548dfe442` is source-build metadata
`ghcr.io/enntity/atlas-sparkglm@sha256:94b837ef3ea7b3f54c158ce568e611405b1bd167edd56c734d8e90d2d6291cb6`.
The tag `ghcr.io/enntity/atlas-sparkglm:2b2970c2f05d` is source-build metadata
(first 12 characters of the install tree) and is never pulled; a tag alone is
not an immutable artifact identity. A later product revision is adopted by
changing the revision, the install tree, the derived tag and the pinned digest.
Expand Down
12 changes: 4 additions & 8 deletions recipes/linux-nvidia-dgx-spark-2x-glm53-atlas.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"provenance": {
"createdBy": "lloom",
"generatedBy": "manual-curated",
"source": "Atlas SparkGLM candidate at product revision 5063c13ea6f196046b76ce6e5c8b4d4200cccc9a (engine Enntity/atlas fix/sparkglm-rc4.2-refusal-content @ 6f27e8fa, AGPL-3.0-only, recorded in the image's /opt/atlas/source-manifest.json; corresponding source is Enntity/sparkglm at that revision plus that Enntity/atlas commit). The normal install path pulls the immutable GHCR manifest digest ghcr.io/enntity/atlas-sparkglm@sha256:5ec57d5522a95463b600a17471ce60712b315566e04c0a3bb6af1d84a1de0a4a; the tag ghcr.io/enntity/atlas-sparkglm:ed7548dfe442 is source-build metadata and is never pulled. Each host verifies the pulled image's io.enntity.sparkglm.install-tree contents label equals install tree ed7548dfe442949283b54c7160605429046cd46f and the image is arm64; the label is a contents check, not an artifact identity. An explicit --source-build builds from Enntity/sparkglm install/ at that revision and produces a LOCAL tag for an explicitly overridden local recipe only, and does not satisfy the normal immutable recipe. Model nvidia/GLM-5.3-Flash-NVFP4 revision 423acf37583782c51c142d145aef733d72943d93 requires a once-per-node overlay conversion gated on conversion.complete.json plus verification; the DFlash2 drafter incoai/GLM-5.3-Flash-DFlash2 revision 7d74cdd881ed7e32c31175984a67823127b66cfe (CC BY-NC-ND 4.0, non-commercial) is served from its checkpoint. The engine argument vector ships inside the image as /opt/atlas/profiles/4x1m.json, selected with SPARKGLM_PROFILE=4x1m, not in this recipe. The profile carries 1048576-token contexts, four sequences sharing one physical KV pool (GLM latent in FP8 with per-128 scales, split between the two nodes by SPARKGLM_KV_SHARD=1, qualified at about 2.38M tokens at the recipe's 0.91 GPU memory utilization with the display carveout, which overrides the profile default of 0.88, and shared with the disk prefix cache; the 48 GiB disk prefix-cache tier is enabled beside the shard, split into 24 GiB KV and 24 GiB SSM tiers per node), FP32 SSM state with KDA record rollback, and DFlash2 (gamma 8) on the head rank; disable-tool-grammar is deliberately not set so structured output and tools stay functional."
"source": "Atlas SparkGLM candidate at product revision 4dee61dcb522d6f3d2246c920ce1a52d87548c71 (engine Enntity/atlas sparkglm/atlas-20261014-vpc @ 289cefbc, AGPL-3.0-only, recorded in the image's /opt/atlas/source-manifest.json; corresponding source is Enntity/sparkglm at that revision plus that Enntity/atlas commit). The normal install path pulls the immutable GHCR manifest digest ghcr.io/enntity/atlas-sparkglm@sha256:94b837ef3ea7b3f54c158ce568e611405b1bd167edd56c734d8e90d2d6291cb6; the tag ghcr.io/enntity/atlas-sparkglm:2b2970c2f05d is source-build metadata and is never pulled. Each host verifies the pulled image's io.enntity.sparkglm.install-tree contents label equals install tree 2b2970c2f05d346fa934da8df8f7d3ba551e07c3 and the image is arm64; the label is a contents check, not an artifact identity. An explicit --source-build builds from Enntity/sparkglm install/ at that revision and produces a LOCAL tag for an explicitly overridden local recipe only, and does not satisfy the normal immutable recipe. Model nvidia/GLM-5.3-Flash-NVFP4 revision 423acf37583782c51c142d145aef733d72943d93 requires a once-per-node overlay conversion gated on conversion.complete.json plus verification; the DFlash2 drafter incoai/GLM-5.3-Flash-DFlash2 revision 7d74cdd881ed7e32c31175984a67823127b66cfe (CC BY-NC-ND 4.0, non-commercial) is served from its checkpoint. The engine argument vector ships inside the image as /opt/atlas/profiles/4x1m.json, selected with SPARKGLM_PROFILE=4x1m, not in this recipe. The profile carries 1048576-token contexts, four sequences sharing one physical KV pool (GLM latent in FP8 with per-128 scales, split between the two nodes by SPARKGLM_KV_SHARD=1, qualified at about 2.38M tokens at the recipe's 0.91 GPU memory utilization with the display carveout, which overrides the profile default of 0.88, and shared with the disk prefix cache; the 48 GiB disk prefix-cache tier is enabled beside the shard, split into 24 GiB KV and 24 GiB SSM tiers per node), FP32 SSM state with KDA record rollback, and DFlash2 (gamma 8) on the head rank; disable-tool-grammar is deliberately not set so structured output and tools stay functional."
},
"links": [
{
Expand All @@ -23,7 +23,7 @@
},
{
"rel": "engine",
"href": "https://github.com/Enntity/atlas/tree/6f27e8fa0afe65180a169e54361a77565787ae27"
"href": "https://github.com/Enntity/atlas/tree/289cefbccdd6c13117228863c7237da17e47a92a"
},
{
"rel": "engine-upstream",
Expand Down Expand Up @@ -284,7 +284,7 @@
"warmup": false,
"bootstrap": {
"adapter": "docker",
"image": "ghcr.io/enntity/atlas-sparkglm@sha256:5ec57d5522a95463b600a17471ce60712b315566e04c0a3bb6af1d84a1de0a4a",
"image": "ghcr.io/enntity/atlas-sparkglm@sha256:94b837ef3ea7b3f54c158ce568e611405b1bd167edd56c734d8e90d2d6291cb6",
"pull": false,
"pullTimeoutMs": 3600000,
"createArgs": [
Expand All @@ -306,8 +306,6 @@
"IPC_LOCK",
"--cap-add",
"SYS_NICE",
"--cap-add",
"SYS_ADMIN",
"--ulimit",
"memlock=-1:-1",
"--security-opt",
Expand Down Expand Up @@ -443,7 +441,7 @@
},
"bootstrap": {
"adapter": "docker",
"image": "ghcr.io/enntity/atlas-sparkglm@sha256:5ec57d5522a95463b600a17471ce60712b315566e04c0a3bb6af1d84a1de0a4a",
"image": "ghcr.io/enntity/atlas-sparkglm@sha256:94b837ef3ea7b3f54c158ce568e611405b1bd167edd56c734d8e90d2d6291cb6",
"pull": false,
"pullTimeoutMs": 3600000,
"createArgs": [
Expand All @@ -465,8 +463,6 @@
"IPC_LOCK",
"--cap-add",
"SYS_NICE",
"--cap-add",
"SYS_ADMIN",
"--ulimit",
"memlock=-1:-1",
"--security-opt",
Expand Down
17 changes: 9 additions & 8 deletions test/atlas-sparkglm-recipe.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ for (const member of members) {
assert.deepEqual(bootstrap.command ?? [], [], 'use the image entrypoint exactly once');

const rendered = bootstrap.createArgs.join(' ');
// The carveout comes from the driver's scanout heap; no capability is granted for it.
assert(!rendered.includes('SYS_ADMIN'), 'the carveout needs no CAP_SYS_ADMIN');
for (const expected of [
'--restart no',
'--network host',
Expand All @@ -154,8 +156,7 @@ for (const member of members) {
'/dev/infiniband:/dev/infiniband',
'--cap-add IPC_LOCK',
'--cap-add SYS_NICE',
// The GB10 display carveout: the launcher drops SYS_ADMIN before the server starts.
'--cap-add SYS_ADMIN',
// The GB10 display carveout's host lock.
'type=bind,src=${installRoot}/atlas-carveout-lock,dst=/run/lock/sparkglm',
'SPARKGLM_DISPLAY_CARVEOUT=1',
'--ulimit memlock=-1:-1',
Expand Down Expand Up @@ -528,26 +529,26 @@ assert.equal(pins.model.revision, '423acf37583782c51c142d145aef733d72943d93');
assert.equal(pins.model.repo, 'nvidia/GLM-5.3-Flash-NVFP4');
assert.equal(pins.source.repo, 'Enntity/sparkglm');
assert.equal(pins.status, 'final');
assert.equal(pins.source.revision, '5063c13ea6f196046b76ce6e5c8b4d4200cccc9a');
assert.equal(pins.source.installTree, 'ed7548dfe442949283b54c7160605429046cd46f');
assert.equal(pins.source.revision, '4dee61dcb522d6f3d2246c920ce1a52d87548c71');
assert.equal(pins.source.installTree, '2b2970c2f05d346fa934da8df8f7d3ba551e07c3');
assert.equal(pins.source.buildScript, 'install/build.sh');
// The normal install path addresses the immutable registry digest, not the
// mutable tag. The tag stays as source-build metadata only.
assert.equal(
pins.image.reference,
'ghcr.io/enntity/atlas-sparkglm@sha256:5ec57d5522a95463b600a17471ce60712b315566e04c0a3bb6af1d84a1de0a4a'
'ghcr.io/enntity/atlas-sparkglm@sha256:94b837ef3ea7b3f54c158ce568e611405b1bd167edd56c734d8e90d2d6291cb6'
);
assert.match(pins.image.reference, IMAGE_REFERENCE_PATTERN);
assert.equal(pins.image.reference.split('@')[0], IMAGE_REPOSITORY);
assert.equal(pins.image.tag, 'ghcr.io/enntity/atlas-sparkglm:ed7548dfe442');
assert.equal(pins.image.tag, 'ghcr.io/enntity/atlas-sparkglm:2b2970c2f05d');
assert.equal(pins.image.label, 'io.enntity.sparkglm.install-tree');
assert.equal(pins.image.architecture, 'arm64');
assert.equal(pins.image.entrypoint, '/opt/atlas/serve.py');
// The image identity is portable; no host-local receipt or derived-tag prefix.
assert(!Object.hasOwn(pins, 'installer'));
assert(!Object.hasOwn(pins.image, 'tagPrefix'));
assert.notEqual(pins.image.reference, pins.image.tag, 'the digest must not be the mutable tag');
assert(!pins.image.reference.includes(':ed7548dfe442'), 'the tag must never appear as the pulled reference');
assert(!pins.image.reference.includes(':2b2970c2f05d'), 'the tag must never appear as the pulled reference');
assert(pins.overlay.marker.includes('conversion.complete.json'));
assert.deepEqual(verifyPins(pins), []);

Expand Down Expand Up @@ -839,7 +840,7 @@ assert.match(missingImageContract.stderr, /missing the Atlas entrypoint\/profile
for (const identity of [`arm64 ${'f'.repeat(40)}`, `amd64 ${pins.source.installTree}`, 'arm64 <no value>']) {
const mismatched = runInstaller({ ...fakeEnv, ATLAS_TEST_IDENTITY: identity }, ['--check-only']);
assert.notEqual(mismatched.status, 0, `--check-only must reject image identity ${identity}`);
assert.match(mismatched.stderr, /expected 'arm64 ed7548dfe442949283b54c7160605429046cd46f'/);
assert.match(mismatched.stderr, /expected 'arm64 2b2970c2f05d346fa934da8df8f7d3ba551e07c3'/);
}
const noImageDir = fs.mkdtempSync(path.join(converterRoot, 'no-image-'));
const checkedMissingImage = runInstaller({ ...fakeEnv, ATLAS_TEST_IMAGE_DIR: noImageDir }, ['--check-only']);
Expand Down
Loading