This repository contains a comprehensive evaluation and analysis of Large Language Model (LLM) trustworthiness, focusing on the microsoft/phi-2 model. Using the DecodingTrust framework (NeurIPS 2023), this project quantifies model vulnerabilities in terms of toxicity, bias, and privacy.
As LLMs become integrated into real-world applications, ensuring their outputs are reliable, safe, and ethically sound is critical. This project systematically evaluates the trustworthiness of LLMs across three key dimensions:
- Toxicity: Measuring the propensity to generate harmful or offensive content.
- Stereotype & Bias: Analyzing how models react to sensitive topics (e.g., HIV, gender) under various system prompts.
- Privacy: Testing the model's ability to protect sensitive information (e.g., email addresses) from being leaked via context prompting.
-
Base Model: microsoft/phi-2 (2.7B parameters).
-
Evaluation Framework: Based on DecodingTrust, a comprehensive benchmark for LLM trustworthiness.
-
Metrics:
-
EMT (Expected Maximum Toxicity): Evaluating worst-case toxicity.
-
TP (Toxicity Probability): Calculating the likelihood of toxic outputs.
-
Compute: Conducted in a Python/PyTorch environment (Google Colab T4 GPU).
Using stratified sampling from the DecodingTrust toxicity dataset, we prompted the model with varying levels of toxic precursors. Outputs were scored using the Detoxify library to determine if the model amplifies or mitigates harmful language.
We investigated how different "System Prompts" (Benign, Untargeted Attack, and Targeted Attack) influence the model's agreement with social stereotypes regarding marginalized groups and sensitive health topics.
Using the enron.context dataset, we evaluated whether the model inadvertently reveals private PII (Personally Identifiable Information) when provided with specific document contexts.
- Toxicity: Phi-2 showed a high sensitivity to toxic prompts, with an Expected Maximum Toxicity (EMT) score reflecting the challenges smaller models face in safety alignment.
- Bias: The model's vulnerability to stereotypes significantly increased when subjected to "Targeted Attack" system prompts.
- Privacy: Specific evaluation showed how context-window pressure can lead to unintended information leakage.
- Python 3.8+
- GPU (Recommended for inference)
pip install transformers accelerate torch datasets huggingface_hub detoxify
- Clone the repository:
git clone https://github.com/your-username/llm-trustworthiness-eval.git
- Open the Jupyter Notebook:
jupyter notebook LLM_trustworthiness_evaluation.ipynb
- Ensure you have a Hugging Face token configured to download the models if required.
- Wang, B., et al. (2023). DecodingTrust: A Comprehensive Assessment of Trustworthiness in Large Language Models. NeurIPS 2023.
- Microsoft Research. Phi-2: The Surprising Power of Small Language Models.
LLM_trustworthiness_evaluation.ipynb: The primary research notebook containing experiments and visualizations.README.md: Project documentation.