randlab is a Python wrapper for centralizing RNG evaluation with a simple CLI and structured outputs. It orchestrates native test engines installed under a managed dependency directory:
- NIST SP 800-22 Statistical Test Suite via
arcetri/sts3.2.7 - NIST SP 800-90B Entropy Assessment from
usnistgov/SP800-90B_EntropyAssessmentv1.1.8 - PractRand 0.96 from SourceForge (auto-downloaded on install)
- ENT from
Fourmilab/ent_random_sequence_tester@ 388d2cc - gjrand
testunif/mcp4.3.0 from SourceForge - TestU01-2009 finite-file batteries from
umontreal-simul/TestU01-2009@ 57e98bf - Borel normality from the Abbott-Calude-Dinneen-Huang/Kavulich code archive
- Dieharder 3.31.1 as an optional stream-oriented battery (
dieharder, installed viasudo apt install dieharder) - BSI AIS 31 V1 through
perlab-uc3m/ais31_headlessv0.1.0 - GM/T 0005-2021 (Chinese SCA standard) via
Trisia/randomnessv1.5.3rddetector
Plots are intentionally out of scope for this package for now.
# 1. Python package
python -m venv .venv && source .venv/bin/activate
pip install -e .[dev]
# 2. Native dependencies (one command; downloads and builds everything)
randlab deps install --install-system
# 3. Run all suites against a binary file (one command)
randlab run --input sample.bin --format raw --profile paper --suite all --out results/run-001See Install and CLI below for per-step details.
From this directory:
python -m venv .venv
source .venv/bin/activate
python -m pip install -U pip
python -m pip install -e .[dev]List known native dependency sources:
randlab deps listInstall all managed native dependencies into .randlab/tools:
randlab deps install --install-systemThis is the recommended one-command setup on Debian/Ubuntu. With --install-system, the tool installs
the OS packages needed by the native engines, then downloads/clones every suite dependency from the
sources listed by randlab deps list. Source archives and cloned repositories are cached under
.randlab/tools/_sources; runnable binaries and state are installed under .randlab/tools. The wrapper
does not require a pre-existing sibling dependencies/ directory.
The default dependency set covers STS from arcetri/sts, NIST SP 800-90B, PractRand 0.96, ENT,
gjrand, TestU01, Borel normality, Dieharder, BSI AIS 31 V1, and GM/T 0005-2021. Without
--install-system, install system packages yourself first: git, make, gcc, g++,
libfftw3-dev, libbz2-dev, libdivsufsort-dev, libjsoncpp-dev, libssl-dev, libmpfr-dev,
libgmp-dev, default-jdk, golang-go, and dieharder.
You can also install the same default dependency set without sudo if those system packages are already available:
randlab deps installInstall one dependency independently with --dependency:
randlab deps install --dependency sp800-90b
randlab deps install --dependency gmt-randomnessThe Borel and AIS31 archives are downloaded automatically by the default installer. If you need a pinned local copy for reproducibility, pass it explicitly:
randlab deps install --dependency borel-archive --borel-archive ~/Downloads/programs.tgz
randlab deps install --dependency ais31-headless --dependency ais31-archive \
--ais31-archive ~/Downloads/AIS_31_testsuit_zip.zipcode.zip from the Zenodo record is also acceptable for Borel if it contains the Borel C++ source. The
tool extracts the archive under .randlab/tools/borel/src, finds a Borel C++ file, and compiles
.randlab/tools/borel/borel_test.
The legacy convenience flags are still accepted:
randlab deps install --with-borel --borel-archive ~/Downloads/programs.tgz
randlab deps install --with-ais31 --ais31-archive ~/Downloads/AIS_31_testsuit_zip.zip
randlab deps install --with-gmtFor migration only, adapters still know how to find the old parent-repository binaries when --tools-root .. is passed:
randlab check --tools-root ..Check whether tools are available:
randlab checkShow known profiles, input formats, and recommended data sizes:
randlab planRun tests against an existing raw binary file:
randlab run --input sample.bin --format raw --suite nist-sts --suite borel --suite ais31-p1-t1-t5 --suite gmt-sts --out results/run-001Run tests against an ASCII bitstream file containing 0 and 1 characters:
randlab run --input sample.bits --format bits --suite entropy-iid --out results/run-002Every run writes:
summary.json: compact run summary for humans and dashboards: overall status, suite statuses, failed metrics, warnings, and artifactsmanifest.json: full structured run record: inputs, profile, commands, statuses, timings, parsed metrics, warnings, and artifactssummary.csv: one row per parsed metric or subtestraw/: stdout/stderr and native result files copied or referenced where possible
The wrapper intentionally keeps the public interface narrow:
raw: bytes exactly as emitted by the RNG.bits: ASCII text containing0and1; whitespace is ignored and the file is normalized to raw bytes for suites that require binary input.
The native tools support more details internally, but these two formats keep the top-level CLI predictable.
For bits input, whitespace is ignored and remaining 0/1 characters are packed into bytes before
running suites that require binary input. If the bit count is not a multiple of 8, the final byte is
zero-padded on the right.
Use randlab plan --profile paper for the exact minimums enforced by the selected profile. The table
below captures the practical input contract for each suite currently wired into the wrapper.
| Suite | Native input used by wrapper | Minimum for a standard/paper run | Notes |
|---|---|---|---|
nist-sts |
raw or bits file passed to arcetri/sts |
1,250,000,000 bytes | Runs 10,000 streams of 1,000,000 bits (-i 10000 -S 1000000). |
entropy-iid |
normalized raw bytes | 1,000,000 bytes | SP 800-90B IID sequential dataset. Repeat independent 1 MB samples at the experiment level when you need replicate evidence. Inputs larger than 1,000,000 bytes are automatically truncated. |
entropy-non-iid |
normalized raw bytes | 1,000,000 bytes | SP 800-90B non-IID sequential dataset. Inputs larger than 1,000,000 bytes are automatically truncated. |
entropy-restart |
normalized raw bytes | 1,000,000 bytes | Interpreted as the 1000 x 1000 restart matrix with 8-bit symbols. Inputs larger than 1,000,000 bytes are automatically truncated. |
practrand |
raw bytes piped to RNG_test stdin |
268,435,456 bytes (256 MB) | PractRand 0.96 consumes exactly tlmax bytes from stdin and reports at 32 MB, 64 MB, 128 MB, 256 MB checkpoints. Detects biases with far less data than NIST SP 800-22. |
ent |
normalized raw bytes passed to ent |
1,000,000 bytes | Descriptive byte-level sanity check: entropy, chi-square, mean, Monte Carlo Pi, and serial correlation. ENT accepts any non-empty file; 1 MB avoids tiny-sample reports. |
gjrand-mcp |
raw bytes opened as seekable stdin for testunif/mcp |
1,073,741,824 bytes (1 GiB) | gjrand uniform-bit suite. quick uses 10 MB, nist-minimum uses 100 MB, and paper uses 1 GiB. |
testu01-rabbit |
normalized raw file passed to TestU01 RabbitFile |
134,217,728 bytes | Binary bitstream battery. quick uses 2^20 bits, nist-minimum uses 2^25 bits, and paper uses 2^30 bits. |
testu01-alphabit |
normalized raw file passed to TestU01 AlphabitFile |
134,217,728 bytes | Hardware-oriented binary bitstream battery over all 32 bits in each word. Same profile sizes as Rabbit. |
testu01-block-alphabit |
normalized raw file passed to TestU01 BlockAlphabitFile |
134,217,728 bytes | Repeats Alphabit after TestU01 bit-block reordering for block widths 1, 2, 4, 8, 16, and 32. |
borel |
normalized raw bytes | 8,388,608 bytes | Large single-sample Borel normality run. The Borel code does not define a formal standard minimum. |
ais31-p1-t0 |
normalized raw bytes | 393,216 bytes | AIS 31 V1 P1:T0, 8-bit symbol mode. |
ais31-p1-t1-t5 |
normalized raw bytes | 652,500 bytes | AIS 31 V1 P1:T1-T5, 8-bit symbol mode. |
ais31-p2 |
normalized raw bytes | 900,000 bytes | AIS 31 V1 P2, 8-bit symbol mode. |
gmt-sts |
wrapper-created directory of .bin samples |
125,000,000 bytes | Uses 1,000 samples of 125,000 bytes (1,000,000 bits). Smaller quick runs are smoke tests only. |
dieharder |
normalized raw bytes piped to dieharder -a -g 200 |
2,500,000,000 bytes in paper profile | Optional comparison/stress battery. Dieharder has no suite-wide fixed byte minimum; this is a conservative finite stdin budget for -a. |
The profiles are sized from each suite's own contract where one exists, and from explicit operating assumptions where the upstream tool does not define a fixed minimum.
quick is a smoke profile. It is meant to prove that dependencies, input conversion, parsers, and
output writing work. It deliberately uses smaller data where the upstream suite supports a short run,
or where the wrapper can make a non-normative adapter check useful.
paper is the default reporting profile. It uses standard or conservative sizes suitable for stable
comparisons, while still staying within what the native tools can run in a single CLI invocation:
- NIST SP 800-22: 10,000 bitstreams of 1,000,000 bits, i.e. 1,250,000,000 bytes total.
- NIST SP 800-90B IID and non-IID: at least one 1,000,000-byte sequential dataset per run. Inputs larger than 1,000,000 bytes are automatically truncated to prevent native tool execution failures or memory depletion. Use replicate files at the experiment level when you need replicate evidence.
- NIST SP 800-90B restart: 1000 x 1000 symbols, represented here as 1,000,000 bytes. Inputs larger than 1,000,000 bytes are automatically truncated.
- PractRand: 256 MB (
tlmax); evaluates at 32 MB, 64 MB, 128 MB, 256 MB checkpoints. - ENT: at least 1,000,000 bytes; descriptive byte-level metrics only.
- gjrand
testunif/mcp: 1 GiB for the standard run; 10 MB for quick smoke tests. - TestU01 finite-file batteries: 2^30 bits (134,217,728 bytes) for the large documented file runs; 2^20 bits for quick smoke tests.
- Borel normality: one 2^23-byte large sample.
- GM/T 0005-2021: 1,000 samples of 1,000,000 bits (125,000 bytes each), i.e. 125,000,000 bytes total.
- Dieharder: optional comparison battery over stdin with
-g 200. The paper profile uses a 2.5 GB finite stdin budget because Dieharder does not define one fixed-abyte requirement.
GM/T 0005-2021 is the randomness testing specification published by China's State Cryptography Administration (SCA / OSCCA). It is the normative standard for evaluating the statistical quality of RNGs used in Chinese commercial cryptographic products (the SM-family of algorithms: SM2, SM3, SM4, etc.).
| # | Test | Also in NIST SP 800-22? |
|---|---|---|
| 1 | Monobit Frequency | Yes |
| 2 | Frequency Within Block | Yes |
| 3 | Poker | GM/T only |
| 4 | Overlapping Template Matching | Yes (different params) |
| 5 | Runs | Yes |
| 6 | Runs Distribution | GM/T only |
| 7 | Longest Run of Ones/Zeros in a Block | Yes (different params) |
| 8 | Binary Derivative | GM/T only |
| 9 | Autocorrelation | GM/T only |
| 10 | Matrix Rank | Yes |
| 11 | Cumulative Sums | Yes |
| 12 | Approximate Entropy | Yes |
| 13 | Linear Complexity | Yes |
| 14 | Maurer's Universal Statistical | Yes |
| 15 | Discrete Fourier Transform | Yes (different normalization) |
The standard evaluates a batch of samples, not a single sequence. For each test item:
- Each individual sample passes if its p-value ≥ 0.01 and q-value ≥ 0.0001.
- The test item passes the battery if the proportion of passing samples is at least 98.1 % (≥ 981 out of 1,000).
This two-level evaluation (per-sample significance + batch proportion threshold) is more conservative than NIST SP 800-22's single uniformity check.
The standard defines three supported sample sizes:
| Mode | Bits per sample | Bytes per sample | Recommended samples | Min input file |
|---|---|---|---|---|
| Small | 20,000 | 2,500 | 1,000 | 2,500,000 bytes (2.5 MB) |
| Standard | 1,000,000 | 125,000 | 1,000 | 125,000,000 bytes (125 MB / 119 MiB) |
| Large | 100,000,000 | 12,500,000 | 1,000 | 12,500,000,000 bytes (12.5 GB / 11.6 GiB) |
This wrapper always uses the standard 1,000,000-bit mode. The input file is split into
125,000-byte chunks automatically. Inputs below one full sample are rejected; trailing bytes below
one full sample are ignored and reported in raw/gmt-sts/input_summary.json, manifest.json,
summary.json, and the warning stream.
The paper and nist-minimum profiles require at least 125,000,000 bytes (1,000 samples);
the quick profile allows as few as 10 samples (1,250,000 bytes), but results at that scale
are not statistically robust and should be treated as a smoke test only.
# Standard evaluation (≥ 125 MB / 119 MiB input, 1 000 samples)
randlab run --input sample.bin --format raw --suite gmt-sts --profile paper --out results/gmt-run-001
# Quick smoke test (≥ 1.25 MB input, ≥ 10 samples)
randlab run --input sample.bin --format raw --suite gmt-sts --profile quick --out results/gmt-smokeThe adapter writes three JSON files into the run directory:
raw/gmt-sts/input_summary.json: sample mode, bytes per sample, samples prepared, consumed bytes, and discarded trailing bytes.raw/gmt-sts/report.json: per-sample p-value and q-value for every test item.raw/gmt-sts/analysis.json: per-test pass count, pass rate, threshold, and overall pass/fail verdict. This becomes the structured metrics insummary.csvandmanifest.json.
Each row in summary.csv for gmt-sts represents one test item. The value column is the
proportion of samples that passed (0.0 to 1.0). The passed column is True when that proportion
reaches the 98.1 % threshold. Smoke runs with fewer than 1,000 samples report this threshold in the
metric details, but do not use it as a normative suite verdict.
The top-level summary.json keeps the most relevant results together: overall status, per-suite
status, metric counts, failed metrics, warnings, errors, and artifact paths. Use manifest.json
when you need the full command and metric detail for reproducibility.
Dieharder should be included, but not treated as the same kind of evidence as NIST SP 800-22, SP 800-90B, or Borel normality when we only have finite inputs. Its own documentation emphasizes testing generators or effectively unbounded streams. File input is supported, but if a test needs more random values than the file contains, Dieharder can cycle through the file again, which reduces the effective sample space and can make p-value histograms misleading.
The wrapper reports dieharder as a stress/comparison battery. It feeds bytes through stdin using -g 200, which avoids Dieharder's file rewind/cycling path. The quick profile still runs Dieharder's -a all-test mode, but with a reduced sample multiplier (-m 0.01) so smoke runs complete in a reasonable time. Larger profiles use Dieharder's default -a settings.
On Debian/Ubuntu, install from the system package manager:
sudo apt install dieharderThe wrapper is tested against version 3.31.1 (Debian/Ubuntu package: 3.31.1.2-1build1). Source tarballs and binaries for other platforms are available at the Dieharder page.
There is no single exact byte requirement for a full dieharder -a run that is independent of the input stream. Some subtests consume a fixed number of 32-bit random values for a given -p/-t configuration. Other subtests are data-dependent: for example, diehard_squeeze stops after a random process reaches a condition, and diehard_craps rolls until each simulated game terminates. For these tests, the exact amount of input can be known after a run, but not as a fixed pre-run constant for every possible sample.
For finite stdin runs, use 2,500,000,000 bytes (2.5 GB / about 2.33 GiB) as the paper-profile budget. This is not a mathematical guarantee for every possible dieharder -a execution, but it is a conservative operating default for a stress battery whose subtests do not share one fixed byte requirement. If a result matters, verify actual consumption with the measurement script or use a real generator stream.
Dieharder tracks how many input values a test consumed. An instrumented build can print the per-test counter and rewind count. Each completed test writes a line like this to stderr:
# randlab_dh_count: test=sts_monobit uint32=6000 bytes=24000 cumulative_uint32=6000 cumulative_bytes=24000 rewinds=0
Interpretation:
rewinds = 0: that completed test did not cycle file input.rewinds > 0: Dieharder reused finite file input; treat that test result as suspect.- no
randlab_dh_countline and stderr says the fileis too small: Dieharder rejected the file before producing a counted test result.
Build an instrumented Dieharder binary from a local source copy:
cd ../dieharder-3.31.1
./configure CFLAGS='-g -O2 -fcommon'
makeMeasure consumption without any possibility of cycling by feeding Dieharder through a FIFO backed by /dev/urandom:
cd ../rng-eval-dev
tools/measure_dieharder_consumption.py \
--dieharder-bin ../dieharder-3.31.1/dieharder/dieharder \
--all \
--runs 3Check whether a specific finite raw input file is large enough for the selected run:
cd ../rng-eval-dev
tools/measure_dieharder_consumption.py \
--dieharder-bin ../dieharder-3.31.1/dieharder/dieharder \
--input-file sample.bin \
--allThe JSON output includes cycled, max_rewinds, counted, and input_too_small for each run, plus per-test byte counters when Dieharder emits them. A short validation run showed fixed and data-dependent behavior clearly: sts_monobit consumed exactly 24,000 bytes in repeated runs with -p 3 -t 1000, while diehard_squeeze and diehard_craps consumed different byte counts across repeated /dev/urandom runs with the same parameters.
The nist-minimum profile records minimum upstream requirements where the bundled docs state them, especially SP 800-90B's 1,000,000 sequential samples and 1000 x 1000 restart matrix.
See docs/design.md for the design and naming notes.
TestU01-2009 is the official current 32-bit
TestU01 release from Pierre L'Ecuyer's group. TestU01 is primarily a C library for testing
programmed generators, but it also ships finite binary-file batteries that fit this wrapper well:
RabbitFile, AlphabitFile, and BlockAlphabitFile.
TestU01 is part of the default install set:
randlab deps installThe installer reuses the managed source cache when present, or clones the official GitHub repository
there. It runs configure and make, then builds
.randlab/tools/testu01/testu01_file_runner, a small C program that links statically against the
built TestU01 libraries and calls the official file-battery functions without modifying upstream
source.
The TestU01 file batteries take a binary file and a number of bits to test. For each test, TestU01 rewinds the file and starts from the beginning; if the requested bit count exceeds the file size, TestU01 uses the available bits. The wrapper selects a fixed bit count from the active profile:
| Profile | Bits tested | Minimum input |
|---|---|---|
quick |
2^20 bits | 131,072 bytes |
nist-minimum |
2^25 bits | 4,194,304 bytes |
paper |
2^30 bits | 134,217,728 bytes |
Rabbit is the broad binary bitstream battery. Alphabit is aimed especially at hardware random
bit generators and tests overlapping bit patterns. BlockAlphabit repeats Alphabit after bit-block
reordering and is therefore the slowest of the three.
The adapter suppresses TestU01's detailed per-test output and parses the final summary report. It
records the battery name, the number of bits tested, the number of statistics, and each p-value that
TestU01 reports outside its suspect interval [0.001, 0.9990]. A suite is marked failed when any
suspect p-value appears and passed when the summary says all tests passed.
# Quick smoke test (2^20 bits / 128 KiB input)
randlab run --input sample.bin --format raw --suite testu01-rabbit --profile quick --out results/testu01-smoke
# Larger binary-file batteries
randlab run --input sample.bin --format raw --profile paper \
--suite testu01-rabbit --suite testu01-alphabit --suite testu01-block-alphabit \
--out results/testu01-001
# Run every registered suite in one command; missing optional tools are skipped.
randlab run --input sample.bin --format raw --suite all --profile paper --out results/full-001gjrand includes a uniform-bit statistical suite named
testunif/mcp. The 3.4.4 archive is kept under the shared top-level dependencies/ directory
because it was the requested SourceForge package, but its README explicitly says the test suites
are omitted from that retro release. The wrapper therefore builds the testunif/mcp suite from
gjrand 4.3.0, the latest archive that includes those programs.
gjrand is part of the default install set:
randlab deps installThe installer downloads gjrand.4.3.0.tar.bz2 from SourceForge if needed, extracts it into the
managed source cache, builds the gjrand library and testunif binaries using the package's POSIX
shell compile scripts, and installs the runnable suite under
.randlab/tools/gjrand/testunif/.
mcp reads raw binary bytes from stdin and rewinds stdin between subtests, so the wrapper opens the
input file directly as a seekable stdin handle. The profile selects the mcp size argument:
| Profile | mcp size | Minimum input |
|---|---|---|
quick |
10M (--tiny) |
10,485,760 bytes |
nist-minimum |
100M (--small) |
104,857,600 bytes |
paper |
1G (--standard) |
1,073,741,824 bytes |
Inputs below the selected profile minimum are still passed through the common warning path, but
mcp itself reports trouble if a subtest cannot read the bytes it expects.
mcp runs 13 uniform-bit subtests and prints one-sided P-values where smaller numbers are worse.
The adapter records each subtest P-value plus overall_p_value from the final summary line:
Overall summary one sided P-value (smaller numbers bad)
P = 0.948 : ok
The suite is marked failed when the overall P-value is at or below 1e-6, warning-only when it is
at or below 0.1, and passed otherwise. Individual subtest P-values at or below 0.1 are emitted
as warnings but do not fail the suite unless the overall P-value crosses the failure threshold.
# Quick smoke test (10 MB input)
randlab run --input sample.bin --format raw --suite gjrand-mcp --profile quick --out results/gjrand-smoke
# Paper profile (1 GiB input)
randlab run --input sample.bin --format raw --suite gjrand-mcp --profile paper --out results/gjrand-001PractRand is a C++ statistical test battery that reads raw bytes
from stdin and evaluates them at increasing data sizes up to a configurable ceiling (tlmax).
Unlike NIST SP 800-22, it does not require pre-splitting into bitstreams and can detect weak
generators with substantially less data.
PractRand is part of the default install set. No extra flag is needed:
randlab deps install # downloads, extracts, and compiles PractRand automaticallyOn first run, the installer:
- Downloads
PractRand_0.96.zipfrom SourceForge into the managed source cache. - Extracts the source tree in that cache.
- Compiles the static library and the
RNG_testbinary usingg++. - Places the binary at
.randlab/tools/practrand/RNG_test.
Subsequent calls skip the download and extraction if the source is already present.
Prerequisite: g++ on PATH (sudo apt-get install g++ or randlab deps install --install-system).
The wrapper feeds raw bytes to RNG_test stdin via a stdin pipe and passes -tlmax to cap the
total bytes consumed. The profile determines tlmax:
| Profile | tlmax |
Checkpoints reported |
|---|---|---|
quick |
1 MB (1,048,576 bytes) | 256 KB, 512 KB, 1 MB |
nist-minimum |
64 MB (67,108,864 bytes) | 8 MB, 16 MB, 32 MB, 64 MB |
paper |
256 MB (268,435,456 bytes) | 32 MB, 64 MB, 128 MB, 256 MB |
-tlmin is set to max(256 KB, tlmax / 8) to ensure at least two intermediate checkpoints.
The input file must contain at least tlmax bytes; RNG_test reads exactly that many before
producing its final report.
PractRand prints a checkpoint line and anomaly lines (if any) after each doubling of consumed data:
length= 256 megabytes (2^28 bytes), time= 4.1 seconds
no anomalies in 1520 test result(s)
The adapter parses every anomaly line into a Metric:
| Field | Content |
|---|---|
name |
PractRand test ID, e.g. !BirthdaySpacings, BCFN(2+2,13-1,T) |
value |
reported p-value |
passed |
True for mildly suspicious/suspicious; False for FAIL |
details |
{"assessment": "...", "checkpoint": "256 megabytes"} |
A total_tests_at_final_checkpoint summary metric is always appended. The overall suite status
is failed if any anomaly at level FAIL is found; suspicious and very suspicious anomalies
produce warnings but do not fail the suite.
# Paper profile (256 MB input required)
randlab run --input sample.bin --format raw --suite practrand --profile paper --out results/practrand-001
# Quick smoke test (1 MB input)
randlab run --input sample.bin --format raw --suite practrand --profile quick --out results/practrand-smoke
# Combined run alongside other suites
randlab run --input sample.bin --format raw --profile paper \
--suite practrand --suite nist-sts --suite entropy-iid \
--out results/combined-001