- perceptron.py: basic adder using perceptrons (heaviside step function) (25/01/2025)
- perceptron_pure_matrix.py: same as above, but using a larger matrix to include the bias translation in the multiplication (25/01/2025)
- backprop.py: backpropagation algorithm with stochastic gradient descent (25/01/2025)
- backprop_matrix.py: same as above, but allowing propagation of multiple inputs at once, also cross-entropy cost function (26/01/2026)
- backprop_batch.py: same as above, but using mini batches for gradient descent (26/01/2025)
- backprop_regularized.py: same as above, but with l2 regularization, also reducing starting weights based on input layer size (26/01/2025)
- backprop_momentum.py: same as above, but sutskever nesterov momentum-based (26/01/2025)
- backprop_activations.py: same as above, but with more activation functions and per-layer activation functions (26/01/2025)
- backprop_initializations.py: same as above, but with per-layer weight intialization depending on activation (30/05/2026)
- mnist.py: testing the neural network with the MNIST dataset
sutskever-nesterov acceleration momentum-based mini-batch stochastic gradient descent feed-forward neural network with l2 regularization and weight initialization