Repository navigation
Expand file tree
/
Copy pathllms.txt
More file actions
42 lines (33 loc) · 2.13 KB
/
Copy pathllms.txt
File metadata and controls
42 lines (33 loc) · 2.13 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# OpenART
> OpenART (Open Agent Red Teaming) is an executable benchmark and Docker-native
> runtime for evaluating the safety of tool-using LLM agents in long-horizon,
> evolving environments: persistent environment state is mutated mid-run by an
> attacker while a fixed hidden safety contract is checked over full traces.
> Paper: arXiv:2608.00677. Relevant to agent safety, LLM agent red teaming,
> prompt injection, tool-use evaluation, and long-horizon agent benchmarks.
## Overview
- [README](https://github.com/AI45Lab/OpenART): Research summary, positioning
table, install, smoke test, model-backed evaluation, task generation.
- [Paper (arXiv:2608.00677)](https://arxiv.org/abs/2608.00677): Full results —
10K+ validated tasks, 50 domains, 500K+ tools, 15 agents × 5 models,
EMHA attacker, 85.0% pooled ASR.
- [CITATION.cff](https://github.com/AI45Lab/OpenART/blob/main/CITATION.cff):
Canonical bibliographic metadata for the repository and paper.
## Data
- [openart-planner-tasks](https://huggingface.co/datasets/dongdongunique/openart-planner-tasks):
6,597 verified planner-generated tasks in 34 shards, with checksums and
release validation.
- [openart-tools](https://huggingface.co/datasets/dongdongunique/openart-tools):
63,697 materialized tools selected from the 500K+ capability corpus.
## Agent navigation
- [AGENTS.md](https://github.com/AI45Lab/OpenART/blob/main/AGENTS.md): Entry
point for coding agents working inside this repository.
- [Operator skill](https://github.com/AI45Lab/OpenART/blob/main/OpenART/skills/openart/SKILL.md):
Single routing guide for planning, running, extending, and debugging OpenART.
## Docs
- [Documentation index](https://github.com/AI45Lab/OpenART/blob/main/OpenART/docs/README.md):
Design, implementation, usage, compatibility, and debugging guides.
- [Quickstart](https://github.com/AI45Lab/OpenART/blob/main/OpenART/docs/01_quickstart.md):
Install, build images, local smoke test, first evaluation run.
- [Planner guide](https://github.com/AI45Lab/OpenART/blob/main/OpenART/docs/12_planner_design_implementation_usage.md):
Generating validated long-horizon task bundles from scenario seeds.