This repository contains code and data for analyzing multilingual capabilities and safety vulnerabilities of large language models (LLMs), as presented in our paper "Towards Safe Multilingual Frontier AI".
The paper has been accepted for NeurIPS 2024 RegML and SoLaR Accepted (spotlight presentation) workshops. RegML CameraReady version available at https://arxiv.org/abs/2409.13708
This research investigates how language resourcing levels relate to LLM vulnerabilities to multilingual jailbreaks. We test five frontier AI models across the 24 official languages of the European Union, analyzing both capability gaps and safety vulnerabilities.
- Analysis of multilingual jailbreak vulnerabilities across EU languages
- Testing framework for evaluating LLM responses in multiple languages
- Statistical analysis of the relationship between language resourcing and model safety
- Comprehensive evaluation of five frontier AI models
If you use this code or data in your research, please cite our paper:
@article{kanepajs2024towards,
title={Towards Safe Multilingual Frontier AI},
author={Kanepajs, Arturs and Ivanov, Vladimir and Moulange, Richard},
year={2024}
}- Arturs Kanepajs - hello@kanepajs.eu
- Vladimir Ivanov - volodimir1024@gmail.com
- Richard Moulange - rjm246@cam.ac.uk
The prompts and responses in this repository may contain examples of harmful content used for research purposes. This content is stored securely and is not intended to be used for training or fine-tuning language models.