DRANZER implements the model, training loop, backpropagation, optimizer, tokenizer, and inference runtime without an ML framework or autodiff library. The result is a compact codebase for learning how transformers work all the way down to memory layout, numerical gradients, and GPU kernel dispatch.
The default build runs anywhere with a C compiler, and stays portable while still using the vector
instructions of whatever CPU it lands on: the AVX2, AVX-512, and NEON matmul kernels are selected at
runtime from CPUID rather than baked in with -march, so one binary uses the widest it finds and
falls back to portable C on a machine with none. On NVIDIA systems, forward and backward matrix
multiplication can also run through hand-written PTX loaded directly by the CUDA driver—no CUDA
toolkit required.
- Multi-layer, causal, multi-head self-attention with residual connections and layer normalization
- Full hand-written backpropagation, checked against numerical gradients
- AdamW/SGD, true minibatch gradient averaging, accumulation, dropout, clipping, and LR schedules
- Frozen tokenization with BOS/EOS, held-out perplexity, and exact resumable training checkpoints
- Single-file model bundles and ring-KV-cached greedy/top-k/top-p decoding
- Incremental generation callbacks, streamed CLI output, stop sequences, and repetition controls
- Portable CPU execution, runtime-dispatched AVX2/AVX-512/NEON matmul kernels, and optional OpenMP
- Optional NVIDIA GPU offload for forward and backward matmuls, with persistent buffers and a validated weight cache
- Built-in tests, benchmarks, hardware probing, serialization, and GitHub Actions CI
DRANZER's release-tested configuration is Ubuntu 24.04 on x86-64. You need Clang or GCC and GNU Make; see Supported platforms for other CPU, OS, OpenMP, mmap, and CUDA support levels.
git clone https://github.com/Scynth-Labs/DRANZER.git
cd DRANZER/src
make
# Train on the included sample
./app.out train --input ../test.txt --epochs 3
# Continue the most recent checkpoint exactly
./app.out train --resume latest --checkpoint-dir checkpoints
# Use the saved model
./app.out eval --model dranzer.pth --input ../test.txt
./app.out infer --prompt "hello"
./app.out generate --prompt "hello" --length 20 --sampling topp --top-p 0.9 --seed 42Run ./app.out --help for the complete CLI.
tokens → BPE → token + positional embeddings
→ N × [causal multi-head attention → residual + layer norm
→ feed-forward network → residual + layer norm]
→ output projection → next-token logits
Training follows the same path in reverse through every transformer layer and trainable parameter. Parameters and gradients live in contiguous buffers, which keeps optimization and serialization simple and makes the low-level data flow easy to inspect.
From the repository root:
make -C src test # correctness suite
make -C src clean all CC=gcc OMP=1 # OpenMP build
make -C src clean test CC=clang ASAN=1 # memory-safety checks
make -C src bench && ./src/bench.out # model benchmarks
make -C src bench-bundle-load # copied vs mmap bundle startup/RSS
make -C src public-api-check # static/shared embedding artifacts
make -C src release-repro-check # clean GCC/Clang reproducibility gate
make -C src release-package-check # reproducible source/SDK archives
make -C src reference-model # pinned corpus, model, and expected metrics
make -C src seed-floor-tool # analyze adaptive cross-seed quality spread
make -C src seed-floor # run the pre-registered adaptive seed sweep
make -C src profile CC=gcc # frame-pointer build for perf
make -C src gpu-probe && ./src/gpu_probe.out
make -C src gpu-latency && ./src/gpu_latency.out # GPU per-call costGPU tests compile on every machine and self-skip when CUDA hardware is unavailable.
| Guide | What it covers |
|---|---|
| Usage and CLI | Installation, build variants, commands, flags, and troubleshooting |
| Architecture | Model flow, memory layout, modules, and repository structure |
| Model bundle | Portable artifact layout, validation, and legacy compatibility |
| Special tokens | Stable IDs, sequence boundaries, EOS stopping, and legacy mode |
| Generation runtime | Streaming callbacks, stop sequences, sampling controls, and result semantics |
| CPU matmul kernels | Portable and SIMD kernels, runtime dispatch, reproducibility, and the measurement workflow |
| Weight quantization | INT8/INT4 accuracy cost measured in weight, logit, and cross-entropy space, and what the seed count does to it |
| CPU threading | When an OpenMP region is worth entering, the measured cutoff, and why a persistent worker pool was rejected |
| GPU backend | PTX execution, capability probing, caching, limitations, and measurements |
| Development | Tests, CI/nightly jobs, sanitizers, benchmarks, and contribution workflow |
| Public C API | Opaque model/tokenizer/cache/generation handles and ownership rules |
| Supported platforms | Release-tested OS/toolchains, CPU dispatch, mmap, OpenMP, and optional CUDA boundaries |
| Migration notes | Moving legacy/internal integrations to the supported API and bundle contracts |
| Results | Every measured finding, with its method, its uncertainty, and what reproduces it |
| Reproducibility | What a seed guarantees, per axis, and how each cell was established |
| Corpus manifests | The rule that a reported result may not name a corpus a reader cannot obtain |
| Small reference model | Pinned public corpus, training recipe, package checks, and metric replay |
| Seed-variance floor | Pre-registered adaptive seed sweep and comparison-floor contract |
| Design checklist | Prioritized maturity roadmap and acceptance gates |
| Research checklist | What would stop a reader believing or reproducing a result from this codebase, and the defects that already threaten one |
DRANZER is an educational and systems-research implementation, not a production LLM runtime. Generation can continue beyond the model context window by evicting the oldest per-layer KV rows; quality is still limited by that fixed retained context and by positions beyond those seen during training. GPU offload is NVIDIA-only and covers matrix multiplications: all of them in the forward pass, and the two backward matmuls above a measured shape threshold. The optimizer step, attention scores, softmax, and layer normalization stay on the CPU, and activations round-trip to host memory between operations, so a training step is not GPU-resident.
MIT License. Built for learning, experimentation, and research.
When more than one coding agent works this repository at the same time, coordinate through gator-tools, vendored here as a submodule:
node gator-tools/skills/multi-agent-coordination/scripts/coord.mjsRun git submodule update --init if that directory is empty. Coordination state
lives in this repository's .git/, never in the submodule, and nothing the
project needs at runtime depends on it — a clone without submodules still works.