Pedagogical Instruction-Tuning for Llama-3.2 — NSF CATpc
Published:
Project: CATpc: Critical Activity Teacher Planning Companion (NSF award #2334631) · GitHub: cafelabai/CATpc
Problem
General-purpose LLMs are not by default good teachers — they over-explain, drift off-topic, and hallucinate confidently. The NSF-funded CATpc project needed a model that could give pedagogically sound, on-curriculum, low-hallucination responses to learner queries.
Approach
- Designed annotation rubrics for pedagogical alignment with the project’s curriculum designers; trained and managed an annotation team.
- Built a 7,200-example instruction-tuning dataset with inter-annotator agreement of Cohen’s κ 0.88.
- Fine-tuned Llama-3.2 variants in two stages: QLoRA for SFT on the curated dataset, then DPO on paired preference data for response shaping.
- Evaluated on a held-out test set with both automatic metrics and rubric-scored human ratings.
Results
- +14% improvement in pedagogical alignment over the base model.
- −8% hallucinations on the held-out evaluation set.
- Pipeline (data construction → QLoRA → DPO → eval) is reusable across the project’s downstream curriculum modules.
Tech stack
Python · PyTorch · HuggingFace TRL · PEFT · QLoRA · DPO · Llama-3.2 · bitsandbytes · MLFlow