Effl-SlimLLM

Install

python -m pip install -e ".[dev]"

Prune

python -m slimllm.cli \
  --model facebook/opt-125m \
  --output-dir outputs/opt125m_pruned \
  --pruning-ratio 0.2 \
  --calibration-samples 16 \
  --max-seq-len 128 \
  --capture-tokens 1024 \
  --dtype float16 \
  --device cuda \
  --gqa-policy mha_only

Evaluate Perplexity

python -m slimllm.evaluate \
  --model outputs/opt125m_pruned \
  --pruned \
  --dataset wikitext \
  --subset wikitext-2-raw-v1 \
  --split test \
  --samples 128 \
  --max-seq-len 128 \
  --dtype float16 \
  --device cuda

Package

• slimllm/cli.py: pruning CLI
• slimllm/pruner.py: pruning pipeline
• slimllm/importance.py: head, FFN, and layer scoring
• slimllm/prune_ops.py: physical tensor pruning
• slimllm/recovery.py: regression recovery
• slimllm/evaluate.py: perplexity evaluation