This repository contains the full experimental framework, datasets, and result analysis conducted as part of the Bachelor's Engineering Thesis: "Learning module with conversational educational system".
- University: Warsaw University of Technology (Politechnika Warszawska)
- Faculty: Faculty of Mathematics and Information Science (MiNI)
- Supervisor: dr inż. Anna Wróblewska
- Authors: Anna Ostrowska, Gabriela Majstrak, Jan Opala
The core objective of this module was to perform hyperparameter optimization for the RAG (Retrieval-Augmented Generation) pipeline. We utilized the RAGAS framework and the LLM-as-a-Judge paradigm to evaluate system performance across multiple dimensions.
- Faithfulness: Measuring resistance to hallucinations.
- Answer Relevancy: Assessing the alignment between user queries and AI responses.
- Context Recall & Precision: Evaluating the efficiency of the retrieval mechanism.
This repository is a supplementary component of the diploma process at Warsaw University of Technology.