Entropy Visualized: A Randomness Explorer

"Entropy measures uncertainty — the higher it is, the less predictable the world becomes."

An interactive Jupyter notebook that explores, visualizes, and analyzes different sources of randomness through the lens of information theory and cryptography.

Table of Contents

• Overview
• Features
• Installation
• Usage
• What You'll Learn
• Visualizations
• Statistical Tests Implemented
• Algorithms Compared
• Key Findings
• Requirements
• Project Structure
• Documentation
• Contributing
• License
• Author

Overview

Randomness lies at the heart of everything — from cryptography and AI to physics and chaos theory. This project is a visual exploration of how computers generate, measure, and perceive randomness.

Through comprehensive visualizations and rigorous statistical testing, this notebook demonstrates:

• How different Random Number Generators (RNGs) work
• The critical difference between pseudo-random and cryptographically secure randomness
• How to measure and test randomness quality
• Real-world applications including Monte Carlo simulations

Features

Core Experiments

• Shannon Entropy Calculation - Quantify uncertainty in different distributions
• PRNG vs CSPRNG Comparison - Understand security implications
• SHA256 Hash Distribution - Analyze cryptographic hash uniformity
• Blockchain VRF Simulation - Verifiable random functions
• Autocorrelation Analysis - Detect sequential dependencies
• 3D Randomness Visualization - Visual pattern detection

Advanced Analysis

• Multiple PRNG Implementations - LCG, Xorshift, Mersenne Twister
• Chi-Square Goodness-of-Fit Test - Statistical uniformity testing
• Kolmogorov-Smirnov Test - Distribution comparison
• Runs Test - Detect non-random patterns
• Serial Correlation Test - Measure consecutive value correlation
• Gap Test - Analyze distances between events
• Poker Test - Pattern frequency analysis
• Performance Benchmarking - Speed vs security trade-offs

Real-World Applications

• Monte Carlo Simulation - Estimate π using random sampling
• Convergence Analysis - Watch estimates improve with sample size
• Comprehensive Dashboard - All test results at a glance

Installation

Prerequisites

• Python 3.8 or higher
• pip package manager

Clone the Repository

git clone https://github.com/ReservedSnow673/entropy-visualized.git
cd entropy-visualized

Install Dependencies

pip install -r requirements.txt

Or install manually:

pip install numpy matplotlib seaborn scipy jupyter

Usage

Launch Jupyter Notebook

jupyter notebook entropy_visualized.ipynb

Run All Cells

Once the notebook opens in your browser:

1. Click Kernel → Restart & Run All
2. Or run cells individually using Shift + Enter

Explore Interactively

• Modify parameters (sample sizes, seeds, algorithms)
• Experiment with different RNG implementations
• Add your own statistical tests
• Generate custom visualizations

What You'll Learn

1. Entropy Fundamentals

• Shannon entropy formula and interpretation
• Relationship between entropy and predictability
• Maximum entropy for different distributions

2. Random Number Generation

Type	Security	Speed	Best For
PRNG	Low	Fast	Simulations, games
CSPRNG	High	Slower	Cryptography, security
Hash-based	High	Slowest	Deterministic randomness

3. Statistical Testing

• How to verify randomness quality
• Industry-standard tests (NIST, Diehard)
• Interpretation of p-values and test statistics

4. Security Implications

• Why random module is dangerous for security
• When to use secrets module
• Real-world security vulnerabilities

Visualizations

The notebook includes 30+ visualizations:

• Distribution Histograms - Compare uniformity across RNGs
• Entropy Bar Charts - Visual entropy comparison
• 3D Scatter Plots - Detect patterns in consecutive values
• Autocorrelation Plots - Measure sequential independence
• QQ Plots - Test statistical distributions
• Heatmaps - 2D randomness pattern detection
• Time Series - Entropy evolution over time
• Convergence Plots - Monte Carlo simulation accuracy
• Test Dashboards - Comprehensive results summary

Statistical Tests Implemented

Distribution Tests

• Chi-Square Test - Uniformity of distribution
• Kolmogorov-Smirnov Test - Compare with theoretical distribution
• Anderson-Darling Test - Distribution fit

Independence Tests

• Autocorrelation Analysis - Sequential correlation at different lags
• Runs Test - Detect clustering or alternating patterns
• Serial Correlation Test - Lag-1 correlation
• Gap Test - Distances between events

Pattern Tests

• Poker Test - Frequency of value patterns
• Visual Inspection - 2D and 3D plots

Entropy Metrics

• Shannon Entropy - Information-theoretic measure
• Entropy Efficiency - Percentage of maximum entropy

Algorithms Compared

Implemented from Scratch

1. Linear Congruential Generator (LCG)

   • Classic PRNG algorithm
   • Fast but has known correlations
   • Educational value

2. Xorshift

   • Modern, fast PRNG
   • Good statistical properties
   • Simple implementation

Built-in Algorithms

3. Mersenne Twister

   • Python's default (random module)
   • Industry standard for simulations
   • Excellent statistical properties

4. CSPRNG (Cryptographically Secure)

   • Python's secrets module
   • Uses OS entropy pool
   • Unpredictable and secure

5. SHA256 Hash-based

   • Deterministic randomness
   • Cryptographically secure
   • Blockchain applications

6. VRF (Verifiable Random Function)

   • Simulated blockchain randomness
   • Verifiable and unpredictable

Key Findings

Statistical Quality

All tested algorithms pass rigorous statistical tests:

• Chi-square p-values > 0.05
• Autocorrelation near zero
• Uniform distribution achieved
• No detectable patterns

Performance Trade-offs

Algorithm	Throughput	Security	Use Case
LCG	~10M/sec	Low	Education
Xorshift	~8M/sec	Medium	Games
Mersenne	~5M/sec	Medium	Science
CSPRNG	~0.5M/sec	High	Security
SHA256	~0.1M/sec	High	Blockchain

Critical Security Finding

NEVER use random module for security applications!

Examples of security-critical applications:

• Password generation
• Cryptographic keys
• Security tokens
• Session IDs
• Authentication nonces

Always use secrets module for security.

Requirements

numpy>=1.20.0
matplotlib>=3.3.0
seaborn>=0.11.0
scipy>=1.6.0
jupyter>=1.0.0

See requirements.txt for exact versions.

Documentation

• Setup Guide - Detailed installation instructions
• FAQ - Frequently asked questions
• Publishing Guide - How to publish this project
• Citation - How to cite in academic work
• Contributing - How to contribute
• Changelog - Version history
• Security - Security policy

Contributing

Contributions are welcome! Here are some ideas:

Potential Enhancements

• Add more PRNG algorithms (PCG, Philox)
• Implement NIST SP 800-22 test suite
• Add quantum random number generator integration
• Real blockchain VRF data (Chainlink, Ethereum)
• GPU-accelerated Monte Carlo simulations
• Interactive widgets with ipywidgets
• Export results to PDF/HTML
• Comparative performance across hardware

How to Contribute

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

See CONTRIBUTING.md for detailed guidelines.

Educational Use

This notebook is ideal for:

• Computer Science courses - Randomness, algorithms, cryptography
• Statistics courses - Hypothesis testing, distributions
• Security training - Understanding cryptographic randomness
• Self-learning - Interactive exploration of entropy

Citation

If you use this notebook in your research or teaching, please cite:

  title = {Entropy Visualized: A Randomness Explorer},
  year = {2025},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/ReservedSnow673/entropy-visualized}},
  url = {https://github.com/ReservedSnow673/entropy-visualized}
}

References

Books

• "The Art of Computer Programming Vol 2" - Donald Knuth
• "Cryptography Engineering" - Ferguson, Schneier, Kohno
• "Information Theory, Inference, and Learning Algorithms" - David MacKay

Papers & Standards

• NIST SP 800-90B - Recommendation for the Entropy Sources Used for Random Bit Generation
• RFC 4086 - Randomness Requirements for Security
• NIST SP 800-22 - A Statistical Test Suite for Random and Pseudorandom Number Generators

Online Resources

• Random.org - True random numbers from atmospheric noise
• NIST Randomness Beacon - Public randomness
• Chainlink VRF - Blockchain verifiable randomness

License

This project is licensed under the MIT License - see the LICENSE file for details.

Author

Author: Tanvir Singh Sandhu (ReservedSnow)

• GitHub: @ReservedSnow673
• LinkedIn: Tanvir Sandhu
• Twitter: @ReservedSnow

Acknowledgments

• Python community for excellent libraries
• NIST for randomness testing standards
• Donald Knuth for foundational work on PRNGs
• Open source community for inspiration

---

"The most random thing you can do is understand randomness."

Star this repository if you found it helpful!

Share with others learning about cryptography and randomness!

Report issues or suggest improvements!