Signet Forge maintains rigorous quality assurance across every layer of the library, validated against published cryptographic standards, regulatory frameworks, and independent third-party test vector suites.
| Layer | Coverage |
|---|---|
| Unit Tests | 830 tests across all features (C++), 35 Python tests, 10 Rust tests |
| Security Hardening | 10 dedicated audit passes — 458 vulnerabilities identified and fixed, zero open findings |
| Compliance Gaps | 92 of 92 enterprise compliance gaps resolved across 12 passes (FIPS, EU AI Act, MiFID II, GDPR, DORA, PME) |
| Sanitizer CI | AddressSanitizer, ThreadSanitizer, UndefinedBehaviorSanitizer on every push |
| SAST | CodeQL with security-extended query suite on every push |
| Fuzz Testing | 11 libFuzzer harnesses (reader, Thrift, WAL, RLE, Delta, Arrow, AES-GCM, PME, key metadata, HKDF, X25519) |
| Code Coverage | Clang source-based coverage reported to Codecov |
| Mutation Testing | Mull 0.24.0 mutation analysis on crypto module |
| Property-Based Testing | 7 C++ tests (Catch2 GENERATE) + 9 Python tests (Hypothesis) |
| Resilience Testing | 13 fault injection tests: file corruption, truncation, garbage input, WAL CRC corruption |
| Concurrency Stress | 32-thread MPMC ring + EventBus stress tests (16P×16C, 100K items) |
| API Stability | 19 regression tests locking enum values, struct sizes, error codes |
| Cross-Platform | Linux (GCC 13, Clang 18), macOS (Apple Clang 17), Windows (MSVC 2022) |
| Cross-Language | Independent test suites for C++, Python, Rust, and WASM bindings |
| Enterprise Benchmarks | 104 benchmark cases (45 core + 59 enterprise) with regression gate (100% threshold, build fails on regression) |
| Fuzz Corpus Persistence | GitHub Actions cache preserves libFuzzer corpus across runs — each session builds on prior discoveries |
| Secrets Scanning | gitleaks 8.24.3 on full repository history |
| License Compliance | SPDX header verification + GPL/AGPL contamination scan |
| SBOM | CycloneDX + SPDX JSON via anchore/syft (US EO 14028 / EU CRA) |
| Pass | Vulnerabilities Fixed | Scope |
|---|---|---|
| #1 | 6 | Core parsers, encoders, Python bindings |
| #2 | 6 | WAL, feature store, Thrift parser, Arrow interop |
| #3 | 23 | Crypto (CSPRNG, key zeroing, counter overflow), all encoders, AI tier, compliance |
| #4 | 29 | All language bindings (C FFI, Rust, WASM, Python), integer overflow, decompression bombs |
| #5 | 53 | Full-scale audit: constant-time GHASH, GCM hardening, BYTE_ARRAY bounds, typed statistics, EU AI Act cross-chain, MiFID II precision, training metadata |
| Follow-up | 11 | Static audit cross-reference: page CRC-32, mmap parity, Float16 UB, feature flush, WAL fsync, compliance errors |
| #6 | 91 | Comprehensive end-to-end: crypto side-channels, GCM IV handling, X25519 MSVC fix, encoding overflow guards, decompression bombs, mmap safety, data races, EU AI Act training provenance |
| #7 | 126 | Full-scale audit: 5 CRITICAL (key gates, CSV injection, CSPRNG IDs, enum validation), 22 HIGH (CTR overflow, tensor overflow, type-aware column index, hash chains, CRC verification), 50 MEDIUM, 33 LOW |
| #8 | 21 | Delta completeness audit: OS command injection (CWE-78), Python use-after-free (CWE-416), ColumnIndex type confusion (CWE-843), FeatureReader race condition (CWE-362), CRNGT partial-block (FIPS 140-3) |
| Thrift Correctness | 5 correctness issues | Audit-driven: zigzag_encode_i32 signed-shift UB (CWE-190), field-type validation absent (CWE-843), required-field silently defaulting (CWE-20), void deserialize error model, I8 wire type missing; 47 new tests (826 total) |
| Pentest (External) | 5 | External Strix.ai pentest: Rust UAF via schema() lifetime escape (CWE-416, HIGH), DELTA/BSS silent data corruption on count mismatch (CWE-130, MEDIUM), mmap SIGBUS on concurrent file truncation (CWE-367, MEDIUM), evaluation license counter reset via env-var path override (CWE-284, LOW); SchemaRef<'a> lifetime wrapper, count validation guards, owned-memory copy-before-parse, hardened usage_state_path(); 4 new tests (830 total) |
| Total | 468 | Entire codebase, all language interfaces, all compliance reporters — zero open findings |
Every cryptographic primitive is verified against authoritative published test vectors and third-party edge-case suites. This ensures correctness is not merely asserted but independently reproducible from public standards documents.
| Component | Standard | Verification |
|---|---|---|
| AES-256-GCM | NIST SP 800-38D | 18 NIST test vectors including Test Case 15 (256-bit key, 64-byte plaintext, no AAD — validates CTR mode ciphertext and 128-bit authentication tag independently). Counter overflow guard per §5.2.1. S-box cache-timing mitigation via full table prefetch. Constant-time gf_mul in MixColumns. |
| AES-256-GCM (Wycheproof) | Google Wycheproof | 4 edge-case vectors from the AES-256-GCM test group: empty plaintext with non-empty AAD (tcId 92), standard 16-byte message with ciphertext verification (tcId 97), modified tag rejection (16 byte-by-byte flips + all-zero + all-ones tags), and tampered ciphertext rejection (16 CT byte flips). These catch implementation bugs that NIST vectors alone miss — tag truncation acceptance, ciphertext malleability, and AAD binding failures. |
| AES-256-CTR | NIST SP 800-38A | Published CTR mode test vectors. Counter overflow guard (64 GiB limit matching GCM). |
| AES-256-only | NIST SP 800-131A Rev.2 | Intentional design decision: AES-128 and AES-192 excluded. AES-256 retains 128-bit security under Grover's algorithm (post-quantum threat model). Single key size eliminates key-length confusion bugs (CWE-326). Documented in crypto/aes_core.hpp. |
| IV Generation | Platform CSPRNG | arc4random_buf (macOS), getrandom (Linux), BCryptGenRandom (Windows). Uniqueness verified: consecutive generate_iv() calls produce distinct 12-byte nonces with overwhelming probability (2^-96 collision). EINTR retry on Linux getrandom. IV uniqueness is critical for GCM security — IV reuse enables key recovery via the "forbidden attack" (Joux 2006). |
| Kyber-768 KEM | NIST FIPS 203 | liboqs 0.15.0, runtime is_real_pq_crypto() query. ML-KEM-768 provides IND-CCA2 security at NIST Level 3. |
| Dilithium-3 | NIST FIPS 204 | liboqs 0.15.0. ML-DSA-65 provides EUF-CMA security at NIST Level 3. |
| X25519 | RFC 7748 | Montgomery ladder with scalar clamping per §5. MSVC 10-limb fe_sub corrected. Published test vectors. |
| X25519 (Wycheproof) | Google Wycheproof | 7 edge-case vectors: valid key exchange (tcId 1), RFC 7748 §6.1 test vector (tcId 100), RFC 8037 Appendix A.6 with manual scalar clamping (tcId 102), 5 low-order point rejection tests (all-zero shared secret detection), 3 non-canonical u-coordinates (must compute correctly per RFC 7748 §5), and 2 twist points (must produce valid shared secrets). These vectors catch common X25519 implementation flaws: missing scalar clamping, failure to reject small-subgroup points, and incorrect modular reduction. |
| Key material | Best practice | Volatile-pointer zeroing in destructors. SecureKeyBuffer with mlock/munlock prevents key material from being paged to swap. |
| Hybrid KEM | NIST SP 800-227 (Final, Sep 2025) | SHA-256 key combining with domain separation label "signet-forge-hybrid-kem-v1". Combines X25519 classical security with Kyber-768 post-quantum resistance — if either primitive remains secure, the combined key is secure. |
The MiFID II and EU AI Act compliance reporters undergo the same verification as every other component. Each test validates not just output format but regulatory substance — ensuring the library produces reports that meet the specific requirements auditors and regulators expect.
- 101 dedicated compliance tests verifying report structure, field content, chain integrity, and regulatory framework types (GDPR, DORA, EU AI Act, MiFID II)
- Field injection prevention: MAX_FIELD_LENGTH = 4096 truncation on all user-supplied strings prevents JSON/CSV injection in generated regulatory reports
- Timestamp correctness: 64-bit
time_tenforcement ensures timestamp accuracy beyond the 2038 Unix epoch rollover — critical for MiFID II trade reporting and EU AI Act log retention - Enterprise benchmark validation: Phases CR1-CR6 generate reports from real tick data schemas, verifying the full pipeline from raw market data through hash-chained decision logs to regulatory output
- Regulatory article traceability: Each reporter cites the specific articles it implements in source comments (e.g., MiFID II RTS 24 Annex I, EU AI Act Art. 12)
- Report ID uniqueness: CSPRNG random hex suffix on auto-generated MiFID II report IDs satisfies the uniqueness requirement per RTS 24 Annex I field 1 — predictable IDs would allow report spoofing
- Statistical consistency: 3-sigma anomaly detection in EU AI Act Article 12 reports uses mean + 3σ threshold consistently across per-record and aggregate modes — auditors can reproduce the classification
- Training provenance integrity:
InferenceRecordtraining provenance fields (training_dataset_id,training_dataset_size,training_data_characteristics) are included in the SHA-256 hash chain, ensuring training data characteristics cannot be tampered with after the fact — required by EU AI Act Art. 13(3)(b)(ii) - Regulatory coverage: GDPR Articles 5(1)(e), 15, 17, 30, 35 | DORA Articles 5-15, 24-30 | EU AI Act Articles 9-13, 15, 17, 61-62 | MiFID II RTS 24/RTS 6
Every push runs the full test suite under three independent sanitizers, catching vulnerability classes that conventional testing cannot reach:
| Sanitizer | What It Detects | Standard |
|---|---|---|
| AddressSanitizer + LeakSanitizer | Heap/stack buffer overflows (CWE-122), use-after-free (CWE-416), memory leaks (CWE-401) | NIST SP 800-53 SI-16 |
| ThreadSanitizer | Data races (CWE-362), lock-order inversions | CERT C CON00-C |
| UndefinedBehaviorSanitizer | Signed overflow (CWE-190), null dereference, misaligned access | MISRA C++:2023 |
All sanitizer jobs enable the commercial tier (SIGNET_ENABLE_COMMERCIAL=ON) to ensure
cryptographic code paths receive full runtime verification.
libFuzzer harnesses exercise every parser and cryptographic primitive with randomized inputs, instrumented with AddressSanitizer:
| Harness | Target | Coverage |
|---|---|---|
fuzz_parquet_reader |
Parquet file parsing | Magic bytes, Thrift footer, page decoding |
fuzz_thrift_decoder |
Compact Thrift format | All field types, varint, MAP, nesting; new types: LogicalTypeUnion, encryption, bloom filter header, ordering structs |
fuzz_wal_reader |
WAL entry parsing | CRC-32 validation, 64-bit seek bounds |
fuzz_rle_decoder |
RLE bit-packed hybrid | bit_width 0–64, iterator + batch API |
fuzz_delta_decoder |
Delta binary packed | Varint header, zigzag, block decoding |
fuzz_arrow_import |
Arrow C Data Interface | ArrowArray/ArrowSchema import validation |
fuzz_aes_gcm |
AES-256-GCM | Encrypt→decrypt roundtrip, tag verification |
fuzz_pme |
PME orchestrator | 4-part AAD ordinals, column key cache |
fuzz_key_metadata |
Key metadata TLV | Serialization/deserialization, overflow |
fuzz_hkdf |
HKDF RFC 5869 | Extract/Expand, variable-length IKM |
fuzz_x25519 |
X25519 RFC 7748 | Montgomery ladder, scalar clamping, low-order rejection |
Fuzz testing satisfies NIST SP 800-53 SA-11(8) "Dynamic Code Analysis" and DORA Art. 25 "Threat-Led Penetration Testing" requirements.
Mull 0.24.0 mutation analysis verifies that
the test suite detects real code mutations — not just that tests pass, but that they
would fail if logic were broken. The project includes a mull.yml
configuration that restricts mutation to Signet Forge source (include/signet/.*) and
excludes third-party dependencies (_deps/.*).
Running mutation testing locally (Linux, Clang 18 required):
# 1. Install Mull 0.24.0 (Ubuntu 24.04)
curl -sSfL https://github.com/mull-project/mull/releases/download/0.24.0/Mull-18-0.24.0-LLVM-18.1-ubuntu-24.04.deb -o mull.deb
sudo dpkg -i mull.deb
# 2. Create a compiler wrapper that applies the Mull plugin only to project source,
# skipping third-party deps (Catch2, etc.) which can crash the LLVM pass.
cat > /tmp/clang++-18-mull <<'WRAPPER'
#!/bin/bash
for arg in "$@"; do
case "$arg" in
*_deps/*|*catch2*|*Catch2*)
exec /usr/bin/clang++-18 "$@"
;;
esac
done
exec /usr/bin/clang++-18 -fpass-plugin=/usr/lib/mull-ir-frontend-18 -g -grecord-command-line "$@"
WRAPPER
chmod +x /tmp/clang++-18-mull
# 3. Configure with the Mull wrapper
cmake -S . -B build-mull -G Ninja \
-DCMAKE_C_COMPILER=clang-18 \
-DCMAKE_CXX_COMPILER=/tmp/clang++-18-mull \
-DCMAKE_BUILD_TYPE=Debug \
-DSIGNET_BUILD_TESTS=ON
# 4. Build
cmake --build build-mull --target signet_tests
# 5. Run mutation analysis on the crypto module
mull-runner-18 build-mull/signet_tests \
--reporters=Elements \
--report-name=mutation-report \
--include-path="include/signet/crypto/*" \
-- "[encryption]"The mull.yml file in the project root defines 19 mutation operators covering
arithmetic (+→-, *→/), comparison (==→!=, <→>=), logical (&&→||),
and constant replacement mutations. CI runs mutation testing automatically on every
push to main via the mutation-baseline job.
Why a compiler wrapper? Mull instruments code via an LLVM pass plugin applied at
compile time. Without the wrapper, the plugin processes third-party code (Catch2) and
can crash on unsupported LLVM IR patterns. The wrapper detects _deps/, catch2, or
Catch2 in compiler arguments and bypasses the plugin for those files.
Mutation testing satisfies NIST SP 800-53 SA-11(9) "Developer Testing and Evaluation: Mutation Testing" requirements.
- 7 C++ property-based tests validate universal invariants (
decode(encode(x)) == xfor all inputs) using randomized generation with deterministic seeding — catching edge cases that hand-picked test vectors miss. - 9 Python property-based tests using Hypothesis with automatic shrinking: int64/double/string roundtrip identity, length preservation, constant-value arrays, empty strings, and multi-column roundtrips. Run with
pip install hypothesis && pytest python/tests/test_property_based.py. - 19 API stability tests lock enum values, struct sizes, and error codes to prevent accidental ABI breakage in downstream systems.
- 100+ hardening tests verify safe rejection of malformed, adversarial, and boundary-condition inputs across all 10 security audit passes.
13 fault injection tests ([resilience] tag) verify graceful handling of adversarial
I/O conditions — no crashes, no undefined behavior, no silent data corruption:
| Test | What It Injects | Validates |
|---|---|---|
| Corrupted magic bytes | Overwrites PAR1 header/footer | Reader rejects with error, not crash |
| Truncated file (header only) | Resize to 8 bytes | Reader returns error |
| Truncated file (mid-footer) | Resize to 50% of file | Reader returns error |
| Zero-byte file | Empty file | Reader returns error |
| Corrupted Thrift footer | Overwrites footer metadata | No segfault on malformed Thrift |
| Random garbage content | 4KB of random data | Reader rejects non-Parquet input |
| Corrupted page data | Overwrites column data pages | No crash; error or silent (data layer) |
| Nonexistent directory | Write to /nonexistent/path/ |
Writer returns IO error |
| Nonexistent file | Read from missing path | Reader returns IO error |
| Truncated WAL | Resize WAL mid-entry | Reader recovers partial entries |
| Corrupted WAL CRC | Overwrites CRC bytes | Reader stops at corruption, no crash |
Resilience testing satisfies NIST SP 800-53 SI-10 "Information Input Validation" and DORA Art. 9 "ICT Risk Management" requirements.
High-thread-count stress tests verify lock-free data structures under contention levels exceeding typical production deployments:
| Test | Threads | Items | Ring Size | Validates |
|---|---|---|---|---|
| MpmcRing 16P×16C | 32 | 100K | 4096 | Checksum integrity under high contention |
| EventBus 16P×16C | 32 | 50K | 8192 | Full publish/pop path with ColumnBatch allocation |
| MpmcRing sustained 32-thread | 32 | 100K | 8192 | Throughput + completion within 10s deadline |
These tests exercise the Vyukov MPMC bounded queue under 4× the concurrency of the standard test suite (4P×4C → 16P×16C), catching ABA problems, false sharing, and memory ordering bugs that only manifest under heavy contention.
Run: ctest -R stress
16 GitHub Actions jobs run on every push:
| Job | Purpose |
|---|---|
| build-test | Linux (Ubuntu 24.04) + macOS (14) matrix |
| asan | AddressSanitizer + UBSan (commercial tier) |
| tsan | ThreadSanitizer (commercial tier) |
| ubsan | UndefinedBehaviorSanitizer (strict, halt-on-error) |
| windows | MSVC 2022 + /analyze static analysis |
| server-codecs | ZSTD + LZ4 + Gzip codec validation |
| commercial-full | Full test suite with commercial tier |
| post-quantum | Kyber-768 + Dilithium-3 via liboqs |
| fuzz | 11 libFuzzer harnesses, 60 seconds each |
| coverage | Clang source-based coverage to Codecov |
| benchmarks | Performance regression detection (200% threshold) |
| secrets-scan | gitleaks credential detection |
| python-sast | bandit Python security scanning |
| license-check | SPDX + GPL contamination verification |
| mutation-baseline | Mull mutation testing (crypto module) |
| CodeQL | CodeQL SAST with security-extended queries |
The following standards are directly implemented and tested against in Signet Forge:
| Standard | Use in Signet Forge |
|---|---|
| NIST SP 800-38D — GCM | AES-256-GCM footer encryption, 18 test vectors, counter overflow guard |
| NIST SP 800-38A — CTR | AES-256-CTR column encryption, published test vectors |
| NIST SP 800-131A Rev.2 | AES-256-only design decision (§4 long-term security recommendation) |
| NIST SP 800-227 — KEM | Hybrid KEM key combining with domain separation |
| NIST FIPS 203 — ML-KEM | Kyber-768 key encapsulation (NIST Level 3) |
| NIST FIPS 204 — ML-DSA | Dilithium-3 digital signatures (NIST Level 3) |
| RFC 7748 | X25519 key agreement, scalar clamping §5, test vectors §6.1 |
| RFC 5869 | HKDF key derivation for PME |
| Google Wycheproof | AES-256-GCM and X25519 edge-case test vectors |
| EU AI Act (Regulation 2024/1689) | Articles 9, 10, 11, 12, 13, 15, 17, 19, 61, 62 |
| MiFID II RTS 24 | Annex I report fields, report ID uniqueness, order lifecycle |
| MiFID II RTS 6 | Pre-trade risk checks (Art. 17) |
| DORA (Regulation 2022/2554) | ICT risk management, incident reporting, resilience testing |
| GDPR (Regulation 2016/679) | Right to erasure, DSAR, ROPA, DPIA, data retention |
| NIST SP 800-53 Rev. 5 | SA-11 developer testing, SI-10 input validation, SI-16 memory protection |
| NIST SP 800-218 (SSDF) | Secure Software Development Framework — SBOM, testing, code review |
| US Executive Order 14028 | §4(e): Software Bill of Materials (SBOM) requirements |
| EU Cyber Resilience Act | Art. 13, Annex I: SBOM, secure development, attack surface reduction |
| ISO/IEC 27001:2022 | A.8.25: Secure development lifecycle testing requirements |
| SOC 2 Type II (AICPA) | CC7.1: System operations monitoring and vulnerability management |
| CWE Top 25 (2024) | Hardening tests directly address 12 of the Top 25 weakness categories |
Signet Forge is authored and maintained by Johnson Ogundeji. All architectural decisions, security hardening, compliance mappings, and code were designed, reviewed, and accepted by the author. Commercial support is available through SIGNETSTACK (johnson@signetstack.io).
Do not open public GitHub issues for security vulnerabilities. Report them responsibly via email: johnson@signetstack.io. See SECURITY.md for our disclosure policy and response timeline.