add persistence controls for lower SSD writes#555
Open
Skater1808 wants to merge 10 commits into
Open
Conversation
… auto-size) Add c/tests/test_cuda_env.py with 9 integration tests that exercise the glm binary directly to verify: - COLI_CUDA=0 forced-CPU mode (suppression + COLI_GPU guard) - COLI_CUDA=1 hard-fail on CPU-only / unavailable GPU - COLI_CUDA unset auto-detect (fallback + enable paths) - CUDA_EXPERT_GB=0 suppresses auto-size messages Tests skip gracefully when: - glm binary absent - CPU-only build (GPU-dependent tests) - No GPU available (GPU-required tests) Adapted from PR JustVugg#75 by nalepy (now closed — feature landed via independent path). Guard messages adjusted to match current dev wording (glm.c L6424, L6436). CI-safe: all tests skip on ubuntu CPU-only runner.
GPU detection (resource_plan.py):
discover_gpus() parses nvidia-smi's memory.total/memory.free columns with
a bare int(), but unified-memory chips (no discrete VRAM pool) report
those fields as the literal string "[N/A]" instead of a number. The
resulting ValueError is silently swallowed, so the GPU never makes it
into the device list and coli doctor reports "no NVIDIA GPU detected"
even when nvidia-smi plainly sees the card. Fall back to /proc/meminfo
system-RAM figures when the memory fields aren't numeric -- a reasonable
stand-in given unified memory is the VRAM pool on this class of hardware.
Verified on an NVIDIA GB10 (Grace Blackwell, sm_121): coli doctor flips
from skip to pass ("CUDA engine and devices are available"), and live
nvidia-smi dmon monitoring during generation confirms real SM utilization
(bursts to 20-45%) that was flat 0% before the fix.
HF_TOKEN auth (tools/convert_fp8_to_int4.py):
The custom Range-resume downloader never reads HF_TOKEN or sends an
Authorization header, even though the tool prints huggingface_hub's
"unauthenticated requests" warning (from a separate HfApi call) -- so
setting HF_TOKEN had no effect on the actual file transfer. Add the
Bearer header to both the multi-stream and single-stream download paths
when HF_TOKEN is set in the environment.
…re HF_TOKEN into downloader Fixes GPU sizing on unified-memory chips (e.g. NVIDIA GB10) in the resource planner and threads HF_TOKEN into the FP8→int4 downloader for gated repos. Python-only (resource_plan.py + convert tool), no engine change. Verified: clean build, token-exact unchanged. Thanks @ideaclaw.
…ert budget auto-size) Adds test_cuda_env.py — integration tests for COLI_CUDA modes and expert-budget auto-sizing. Test-only, additive. Verified: clean build, token-exact unchanged. Thanks @dqiaok.
Release v1.1.0 — AMD backend, CUDA fmt=4 correctness, tool-calling root cause, security hardening, +4.7x CPU, dual-SSD
Release v1.1.0, take two: fix the tag-build workflow (JustVugg#512) + COLI_TEMP / ROCm collision (JustVugg#513)
Release v1.1.0 — the tag build, fixed for real
docs: Get started that actually starts with getting the program (+ COLI_TEMP, name, workflow fixes already in dev)
Release v1.1.1 — Defender false positive was our bug: 107 KB of zeros in .data
Owner
|
Thanks for this — persistence controls for lower SSD writes is a useful feature. One process note before review: this PR targets |
Author
|
Thanks, I've already changed it. If you have any further questions, I'm here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Describe the problem and the smallest change that solves it.
Validation
make -C c checkmake -C c cuda-test(if applicable)Compatibility