synapticCNN

This is the implement of synapticCNN based on Pytorch framework.

Installation

All experiments were conducted on the NVIDIA Geforce RTX3090, in a system environment of CentOS 7.9, with Pytorch version 1.8.0, and Python 3.8.0.

pip install -r requirements.txt

NN pre-training

The train.py script is used to train the models. You can specify the optimizer, dataset, number of epochs, learning rate, batch size, model name and some training recipes. The detailed parameter setting can be found in the train.py file, or by running

python train.py --help

The following are examples of how to train the models:

• train Basic Conv-Fc CNNs: We trained these types of models using the MNIST dataset, offering three models with different depths: cnn3, cnn5, cnn7.

  python train.py -o sgdm --model cnn3

• train VGG models: We trained these types of models using the CIFAR10 dataset, offering five models with different depths: VGG6, VGG8, VGG11, VGG13, VGG16.

  python train.py -o sgdm -d cifar -e 200 --lr 1e-2 -b 128 --eval-bsz 256 --model vgg11

• train ResNet models: We trained these types of models using the CIFAR100 dataset, offering four models with different depths: resnet18, resnet34, resnet50.

  python train.py -o sgdm -d cifar100 -e 200 --lr 1e-1 -b 128 --eval-bsz 256 --model res18

Here, we provide our pre-trained model weights for download:

Model Name	Dataset	ckpt	Acc@1
BasicCNN3	MNIST	ckpt	96.17
BasicCNN5	MNIST	ckpt	95.84
BasicCNN7	MNIST	ckpt	94.97
Vgg6	CIFAR10	ckpt	84.46
Vgg8	CIFAR10	ckpt	87.00
Vgg11	CIFAR10	ckpt	87.36
Vgg13	CIFAR10	ckpt	87.48
Vgg16	CIFAR10	ckpt	88.12
ResNet18	CIFAR100	ckpt	71.31
ResNet34	CIFAR100	ckpt	71.94
ResNet50	CIFAR100	ckpt	72.51

NN Testing

After models pre-trained on corresponding datasets, the performance of the models can be evaluated by running test.py.

Here is an example:

python test.py --ckpt ./ckpt/mnist/cnn3.pkl \
               -d mnist \
               -- model cnn3

NOTE: The --ckpt parameter specifies the path to the pre-trained model weights, you may replace it with your own local path. The --model parameter specifies the model name, and the -d parameter specifies the dataset name. The detailed parameter setting can be found in the test.py file, or by running

python test.py --help

NN Quantization/Mapping

The provided codebook is linearly scaled, and the scaling factor and bias factor are obtained using a simulated annealing algorithm for search optimization. Then, the trained model weights are mapped.

By using the quantize_test.py script, you can map the model weights and save the model weights. The detailed parameter setting can be found by running

python quantize_test.py --help

Here is an example:

python quantize_test.py -d mnit \
                        --model cnn3 \
                        --codebook ./data/codebook.xlsx --ckpt ./ckpt/mnist/cnn3.pkl

You may evaluate the performance of the mapped model by re-running test.py with new weights in the above step. We also provide the mapped model weights for download:

Model Name	Dataset	ckpt	Acc@1
BasicCNN3	MNIST	ckpt	95.17
BasicCNN5	MNIST	ckpt	93.76
BasicCNN7	MNIST	ckpt	94.17
Vgg6	CIFAR10	ckpt	82.69
Vgg8	CIFAR10	ckpt	85.93
Vgg11	CIFAR10	ckpt	86.60
Vgg13	CIFAR10	ckpt	86.30
Vgg16	CIFAR10	ckpt	86.57
ResNet18	CIFAR100	ckpt	68.20
ResNet34	CIFAR100	ckpt	69.02
ResNet50	CIFAR100	ckpt	69.47