Enhancing Few-Shot Text Classification with Parameter-Efficient Tuning of Large Language Models

Few-shot text classification using BERT + IA³ adapters, augmented with knowledge-graph triples and TransE embeddings.

Method

A knowledge-aware multi-task framework combining few-shot classification with entity/event extraction:

• Backbone: bert-base-uncased with IA³ adapters for parameter-efficient fine-tuning
• Knowledge integration: spaCy triple extraction (subject–relation–object), aligned with WordNet and ConceptNet, encoded via TransE embeddings
• Sequence modeling: BiLSTM over BERT representations → softmax decoder (token-level)
• Loss: L_total = α · L_cls + λ · L_triple (α=1.0, λ=0.7)
• Few-shot setting: 400 samples per class (1600 total) from AG News (4 classes)

Results

Metric	Score
Accuracy	97.97%
Precision	98.00%
Recall	97.95%
F1	97.96%

Dataset

AG News — 4 categories: World, Sports, Business, Technology.
Balanced subset: 400 samples/class → dataset/agnews.csv.

Project Structure

├── main.py              # Training entry point
├── implementation.py    # Model definition, data loading, loss functions
├── plot.py              # Evaluation plots (confusion matrix, ROC, PR curve)
├── dataset/
│   └── agnews.csv
├── result/              # Output figures (generated at runtime)
└── requirements.txt

Quick Start

pip install -r requirements.txt
python -m spacy download en_core_web_sm

python main.py

Figures saved to result/: confusion matrix, accuracy/loss curves, PR curve, ROC curve, FPR/FNR bar chart.