Using the example with --device cpu leads to crash:
python inference/inference_sim.py -a resnet50 -b 512 -pcq_w -pcq_a -sh --qtype int4 -qw int4 --device cpu
The bug comes from calling torch.cuda.clear().
Likewise when the program finishes.
cuda device should not be called in cpu mode.
Using the example with --device cpu leads to crash:
python inference/inference_sim.py -a resnet50 -b 512 -pcq_w -pcq_a -sh --qtype int4 -qw int4 --device cpu
The bug comes from calling torch.cuda.clear().
Likewise when the program finishes.
cuda device should not be called in cpu mode.