EduTutor 🎓

Fine-tuning the Phi-2 open-weight model using LoRA (PEFT) on a curated educational Q&A dataset to improve domain-specific answer accuracy for student queries.

What This Does

• Fine-tunes Phi-2 using LoRA (Low-Rank Adaptation) via the HuggingFace PEFT library
• Trains on curriculum-aligned question-answer pairs
• Writes a custom training loop with gradient accumulation, LR scheduling, and checkpointing
• Runs an automated evaluation pipeline comparing fine-tuned vs base model using ROUGE scores
• Tracks all experiments (loss, ROUGE, hyperparameters) using Weights & Biases

Project Structure

edututor/
├── train.py          # LoRA fine-tuning script
├── evaluate.py       # Evaluation pipeline (ROUGE benchmark)
├── requirements.txt  # Dependencies
└── README.md

Setup

git clone https://github.com/Lokesh-9550/edututor.git
cd edututor
pip install -r requirements.txt

Usage

Fine-tune

python train.py

To use your own dataset, provide a CSV with question and answer columns and update csv_path in load_data().

Evaluate

python evaluate.py

Compares base Phi-2 vs LoRA fine-tuned model on ROUGE-1, ROUGE-2, and ROUGE-L.

Key Technical Choices

Choice	Reason
LoRA rank = 16	Balance between trainable params and quality
Target modules: q_proj, v_proj	Standard for Phi-2 attention fine-tuning
Gradient accumulation = 4	Simulates larger batch on limited GPU memory
fp16 training	Reduces memory footprint on CUDA

Results

Metric	Base Phi-2	EduTutor (LoRA)
ROUGE-1	~0.28	~0.41
ROUGE-L	~0.24	~0.38

Results on sample eval set. Fine-tune on larger curated dataset for better performance.

Tech Stack

Python · PyTorch · HuggingFace Transformers · PEFT (LoRA) · Datasets · Weights & Biases