v0.8.0

What's Changed

Targets:

• HTTPTarget now supports rate limiting
• Some users encountered errors in Azure OpenAI when hitting content filter errors using error code 500. PyRIT now catches content filter responses with both error codes 400 (as before) and 500 (new) and returns a clean response record.

Datasets:

• fetch_babelscape_alert_dataset had a bug causing it to be limited to a single category even when users specified both. This is now fixed!
• added fetch_red_team_social_bias_dataset
• added fetch_darkbench_dataset
• added fetch_mlcommons_ailuminate_demo_dataset

Converters:

• added UnicodeReplacementConverter
• added sneaky_bits option to AsciiSmugglerConverter in the encoding_mode argument. The unicode_tags argument is now removed and replaced by more options in encoding_mode (i.e., unicode_tags, unicode_tags_control, and sneaky_bits).

Scanner: A basic version was introduced in v0.7.0 that supported only sending single-turn prompts. v0.8.0 expands on this with support for most multi-turn orchestrators (incl. adversarial chat targets and scorers) and memory. This feature is still considered experimental and may change considerably in the following versions.

Other:

• support for Python 3.13 in addition to 3.10-3.12.
• For single-piece responses, we now have a convenient get_value() method.
• PyRIT used to print warnings that torch isn't installed (unless the corresponding extra was installed). This was caused by transformers and is now turned off as it doesn't serve any purpose.
• In previous versions, PyRIT started supporting .env.local as an override to the .env file for endpoint secrets. However, when using this outside of the normal repository structure (e.g., when running PyRIT without cloning this repo) the code failed to discover .env.local in the current working directory. This is now fixed.

Full list of changes

• [DevContainer] Provide a uniform development environment by @bashirpartovi in https://github.com/Azure/PyRIT/pull/787
• FEAT: Add Rate Limit Support for HTTP Target by @nina-msft in https://github.com/Azure/PyRIT/pull/786
• DOC Updating contribution docs by @bashirpartovi in https://github.com/Azure/PyRIT/pull/788
• MAINT support python 3.13 by @AdrGav941 in https://github.com/Azure/PyRIT/pull/779
• FIX: fixed dev container permission issue by @bashirpartovi in https://github.com/Azure/PyRIT/pull/789
• FEAT: simplify extraction of converted values from responses by @paulinek13 in https://github.com/Azure/PyRIT/pull/783
• MAINT: improve organization of dataset fetch functions (refactoring) by @paulinek13 in https://github.com/Azure/PyRIT/pull/785
• FEAT: Added cross-platform compatibility and needed language support for toml and docker by @bashirpartovi in https://github.com/Azure/PyRIT/pull/797
• MAINT: Update release version to 0.7.1.dev0 by @jsong468 in https://github.com/Azure/PyRIT/pull/800
• FIX: prevent data overwrite in fetch_babelscape_alert_dataset by @paulinek13 in https://github.com/Azure/PyRIT/pull/799
• DOC contributor guide flowchart, small text updates, and add Roakey to README by @romanlutz in https://github.com/Azure/PyRIT/pull/798
• DOC: clarify OpenAITarget targets httpx_client_kwargs timeout settings by @clod81 in https://github.com/Azure/PyRIT/pull/801
• FIX: Add exception on response parsing when call to Openrouter.ai by @hagsmand in https://github.com/Azure/PyRIT/pull/796
• FIX make sure conversation IDs are not sent out as UUIDs to the database by @ayeganov in https://github.com/Azure/PyRIT/pull/723
• FEAT support adversarial_chat and scoring in scanner to enable automated multi-turn-orchestrators by @romanlutz in https://github.com/Azure/PyRIT/pull/706
• FIX move misplaced test file to tests/unit/converter by @romanlutz in https://github.com/Azure/PyRIT/pull/794
• FEAT: Added Red Team Social Bias dataset by @MoolmanM in https://github.com/Azure/PyRIT/pull/714
• DOC improve API reference for auth, cli, common, chat_message_normalizer by @romanlutz in https://github.com/Azure/PyRIT/pull/793
• FEAT: UnicodeReplacementConverter by @nina-msft in https://github.com/Azure/PyRIT/pull/803
• FIX: Updating pre-commit to fix build issues by @rlundeen2 in https://github.com/Azure/PyRIT/pull/810
• MAINT: Making test_connect more resilient by @rlundeen2 in https://github.com/Azure/PyRIT/pull/806
• [FIX] fix bad domain by @mgstate in https://github.com/Azure/PyRIT/pull/815
• [FIX] Integration test fixes: add hugging face token in notebook and fix test_fetch_datasets by @jsong468 in https://github.com/Azure/PyRIT/pull/819
• FEAT: Added memory config to scanner by @bashirpartovi in https://github.com/Azure/PyRIT/pull/808
• FEAT: add DarkBench dataset by @paulinek13 in https://github.com/Azure/PyRIT/pull/821
• MAINT: improving build/test time by @bashirpartovi in https://github.com/Azure/PyRIT/pull/820
• FIX handle Azure OpenAI content_filter errors with HTTP status code 500 by @romanlutz in https://github.com/Azure/PyRIT/pull/825
• FIX turn off transformers warning by @romanlutz in https://github.com/Azure/PyRIT/pull/829
• TEST: Adding integration test for content filters by @rlundeen2 in https://github.com/Azure/PyRIT/pull/830
• MAINT: Separating integration test local .env by @rlundeen2 in https://github.com/Azure/PyRIT/pull/817
• FEAT: add MLCommons AILuminate v1.0 DEMO Prompt Set by @paulinek13 in https://github.com/Azure/PyRIT/pull/828
• FIX find .env.local in current working directory by @romanlutz in https://github.com/Azure/PyRIT/pull/832
• BREAKING FEAT: Sneaky Bits - Advanced Data Smuggling Techniques by @KutalVolkan in https://github.com/Azure/PyRIT/pull/827
• FEAT add ps-fuzz prompts by @ryanjieh in https://github.com/Azure/PyRIT/pull/823

New Contributors

• @bashirpartovi made their first contribution in https://github.com/Azure/PyRIT/pull/787
• @clod81 made their first contribution in https://github.com/Azure/PyRIT/pull/801
• @hagsmand made their first contribution in https://github.com/Azure/PyRIT/pull/796
• @MoolmanM made their first contribution in https://github.com/Azure/PyRIT/pull/714
• @mgstate made their first contribution in https://github.com/Azure/PyRIT/pull/815
• @ryanjieh made their first contribution in https://github.com/Azure/PyRIT/pull/823

Full Changelog: Azure/PyRIT@v0.7.0...v0.8.0