"Backpropagation? Never heard of her."
A neural network that learns without gradients, without optimizers, and without any of the math your professor warned you about. Just pure, chaotic, beautiful Darwinian survival of the fittest — applied to a problem so simple your calculator could solve it, yet historically humbled an entire generation of single-layer perceptrons.
This project trains a feedforward neural network to solve the XOR problem using a Genetic Algorithm (GA) — no backpropagation, no Adam optimizer, no learning rate scheduling drama. Just 80 individuals fighting for their lives every generation until one of them figures out that 1 XOR 1 = 0.
The network architecture is a modest 2 → 4 → 1 MLP with ReLU hidden activation and Sigmoid output. The GA evolves the entire weight space (all 17 parameters) as a flat chromosome. Nature finds a way.
Input (2) → Dense(4) + ReLU → Dense(1) + Sigmoid → Output
| Component | Detail |
|---|---|
| Hidden Layer | 4 neurons, ReLU activation |
| Output Layer | 1 neuron, Sigmoid activation |
| Total Parameters | 17 (weights + biases, flattened) |
| Chromosome Length | 17 genes ∈ [-1, 1] (initialized) |
Because sometimes the best way to find a solution is to just... throw 80 random guesses at the wall and let them reproduce.
| Hyperparameter | Value |
|---|---|
| Population Size | 80 individuals |
| Generations | Up to 600 |
| Selection | Tournament (size = 3) |
| Crossover | Uniform (50% gene swap) |
| Mutation Rate | 12% per gene |
| Mutation Strength | Gaussian noise σ = 0.15 |
| Elitism | Top 2 survive unconditionally |
| Gene Clipping | [-5, 5] |
fitness = 1 / (1 + MSE) + 1.0 (bonus if all 4 predictions correct)
Max achievable fitness: ~2.0. The algorithm terminates early at ≥ 1.99 — because at that point, the network has achieved enlightenment.
The targets are swappable. Uncomment any gate in main.py and watch evolution pivot its entire worldview:
- ✅ XOR — the default villain
- AND, OR, NAND, NOR — warm-up exercises for the population
pip install numpy
python main.pyExpected output somewhere between generation 0 and 600:
=== XOR SOLVED! ===
If it doesn't solve it, just run it again. That's the beauty of stochastic optimization — blame entropy, not the code.
This is a proof-of-concept, but the architecture is a launchpad:
- Neuroevolution of Augmenting Topologies (NEAT) — evolve not just weights but the network structure itself. Why settle for
2→4→1when evolution can invent something weirder? - Multi-objective fitness — optimize for accuracy and network sparsity simultaneously using Pareto-front selection (NSGA-II).
- Coevolution — pit two populations against each other. One learns XOR, the other tries to fool it. Adversarial evolution before GANs made it cool.
- Continuous control tasks — swap XOR for OpenAI Gym environments. Same GA, now your chromosome controls a robot that's trying not to fall over.
- Distributed evolution — parallelize population evaluation across CPU cores or AWS Lambda for massive population sizes without the wait.
- Hybrid GA + Gradient — use GA to find a good weight initialization, then fine-tune with gradient descent. Best of both worlds; none of the commitment.
Because this is more fun. Also:
- No differentiability required — works on any black-box function
- Naturally parallelizable across the population
- Immune to vanishing/exploding gradients (the population just... dies instead, which is philosophically cleaner)
- Finds global optima more reliably on non-convex loss landscapes
numpy— the only dependency, as God intended
Built from scratch. No PyTorch. No TensorFlow. No regrets.