Prompt Injection — Attacks, Defenses, Datasets and Benchmarks

An overview of research in prompt injection security for LLMs and agents

This repo collects papers, tools, datasets, and benchmarks on prompt injection — attacks that smuggle adversarial instructions into untrusted data to hijack LLM-integrated applications and agents.

【 Attacks → Defenses (Prevention / Detection / Localization) → Benchmarks.】

Quick Links

• Background & Surveys
• Attacks
  • Direct Injection
  • Indirect Injection
  • Automated & Optimization-based Attacks
  • Agent & Tool Exploitation
  • Web & Multimodal Injection
• Defenses
  • Prevention
  • Detection
  • Localization
  • Architectural & System-level Defenses
• Datasets & Benchmarks
  • Attack & Defense Benchmarks
  • Agent Security Benchmarks
    • General Agent & Prompt Injection
    • Tool, Memory & Skill Security
    • Web Agent Security
• Industry Reports & Standards

Background & Surveys

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
Formalizing and Benchmarking Prompt Injection Attacks and Defenses Yupei Liu, Yuqi Jia, Ruiqi Geng, Jinyuan Jia, Neil Zhenqiang Gong	Penn State, Duke University	2024	USENIX Security	建立形式化威胁模型，区分目标指令/数据与注入指令/数据；提出 Open-Prompt-Injection 基准。	Paper Project
The Landscape of Prompt Injection Threats in LLM Agents (AgentPI) Peiran Wang, Xinfeng Li, Chong Xiang, Jinghuai Zhang, Ying Li, Lixia Zhang, Xiaofeng Wang, Yuan Tian	UCLA, NTU, NVIDIA	2026	arXiv	提出 AgentPI 综述框架：包含 5 类上下文相关任务，研究超越静态用户提示的上下文感知攻击。	Paper

Attacks

Direct Injection

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
Ignore Previous Prompt: Attack Techniques for Language Models — Fábio Perez, Ian Ribeiro	Agency Enterprise	2022	NeurIPS ML Safety Workshop	提出 PromptInject，并研究两类基础直接注入目标：目标劫持与系统提示泄露。	Paper Code
Prompt Injection Attack against LLM-Integrated Applications — Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, Yang Liu	NTU Singapore–led multi-institution team	2023	arXiv	提出受 Web 注入攻击启发的黑盒注入框架 HouYi；评估 36 个真实应用，并成功攻击其中 31 个。	Paper
PLeak: Prompt Leaking Attacks against Large Language Model Applications — Bo Hui, Haolin Yuan, Neil Gong, Philippe Burlina, Yinzhi Cao	Johns Hopkins University, Duke University, Johns Hopkins APL	2024	ACM CCS	将提示提取建模为闭源环境下的对抗查询优化，并逐步重建隐藏的系统提示。	Paper Code

Indirect Injection

The important distinction in this section is that the attacker does not need to control the user’s prompt. Instead, the attacker controls a document, web page, message, tool response, or other data source that the application later places into the model context.

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, Mario Fritz	CISPA Helmholtz Center, Saarland University, Sequire Technology	2023	ACM AISec	奠基性的间接提示注入研究，说明嵌入检索内容中的恶意指令可导致数据窃取、API 滥用、持久操控与蠕虫式传播。	Paper ACM
Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models — Jingwei Yi, Yueqi Xie, Bin Zhu, Emre Kiciman, Guangzhong Sun, Xing Xie, Fangzhao Wu	Microsoft Research, University of Science and Technology of China	2025	KDD	提出 BIPIA，并指出间接注入的两个核心成因：模型无法区分信息性上下文与可执行指令，也无法识别外部内容不应控制模型。	Paper Code
Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents — Qiusi Zhan, Zhixiang Liang, Zifan Ying, Daniel Kang	University of Illinois Urbana-Champaign	2025	Findings of NAACL	在自适应威胁模型下评估主流间接注入防御，表明对静态提示看似稳健的防御，在攻击针对防御优化后可能失效。	Paper
Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems — Hongyan Chang, Ergute Bao, Xinjian Luo, Ting Yu	Mohamed bin Zayed University of Artificial Intelligence	2026	USENIX Security	将恶意内容拆分为保证进入检索结果的 trigger fragment 与执行攻击目标的 attack fragment，并通过黑盒优化解决真实语料库中的检索障碍。	Paper
ObliInjection: Order-Oblivious Prompt Injection Attack to LLM Agents with Multi-Source Data — Reachal Wang, Yuqi Jia, Neil Zhenqiang Gong	Duke University	2026	NDSS	面向多来源且顺序未知的输入，设计在攻击者无法预知恶意内容位置和排列顺序时仍保持有效的间接注入。	Paper Github

Automated & Optimization-based Attacks

These papers mark the transition from manually authored strings such as “ignore previous instructions” to attacks generated by gradient search, fuzzing, or reinforcement learning. This distinction matters for defense evaluation: a guardrail tested only against known templates can substantially overestimate its robustness against an adaptive optimizer.

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
Automatic and Universal Prompt Injection Attacks against Large Language Models — Xiaogeng Liu, Zhiyuan Yu, Yizhe Zhang, Ning Zhang, Chaowei Xiao	University of Wisconsin–Madison and collaborators	2024	arXiv	定义统一攻击目标，并使用基于梯度的优化生成可跨样本迁移、对防御仍有效的通用注入数据。	Paper Code
Neural Exec: Learning and Learning from Execution Triggers for Prompt Injection Attacks — Dario Pasquini, Martin Strohmeier, Carmela Troncoso	EPFL, armasuisse S+T	2024	ACM AISec	将执行触发器生成视为可微搜索问题；优化后的触发器可承受转换及 RAG 管线等多阶段预处理。	Paper
PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs — Jiahao Yu, Yangguang Shao, Hanwen Miao, Junzheng Shi, Xinyu Xing	Northwestern University–led multi-institution team	2024	arXiv	将覆盖率引导模糊测试用于选择、变异和优化注入提示，构建自动化稳健性测试框架，而非单一固定攻击。	Paper Code
Learning to Inject: Automated Prompt Injection via Reinforcement Learning — Xin Chen, Jie Zhang, Florian Tramèr	ETH Zurich	2026	arXiv	提出 AutoInject：使用黑盒强化学习生成通用、可迁移的对抗后缀，并联合优化攻击成功率与正常任务效用。	Paper
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses — Chenlong Yin, Runpeng Geng, Yanting Wang, Jinyuan Jia	Penn State	2026	COLM	训练攻击 LLM 自适应生成注入提示；引入自适应熵正则化与动态优势加权，在强防御造成稀疏奖励时维持探索并强化少量成功样本。	Paper Github
The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections — Milad Nasr, Nicholas Carlini, Chawin Sitawarin, Sander V. Schulhoff, Jamie Hayes, Michael Ilie, et al.	Anthropic, Google DeepMind, HackAPrompt, Northeastern, OpenAI, ETH Zurich, et al.	2026	USENIX Security	系统扩展梯度下降、强化学习、随机搜索和人工辅助搜索，并针对不同防御机制自适应优化攻击，揭示静态攻击集会高估防御鲁棒性。	Paper

Agent & Tool Exploitation

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
AgentPoison: Red-Teaming LLM Agents via Poisoning Memory or Knowledge Bases — Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, Bo Li	UIUC, UC Berkeley, University of Wisconsin–Madison	2024	NeurIPS	通过向长期记忆或知识库插入优化的触发关联样本，为 RAG 智能体植入后门；无需重训练模型或直接修改模型参数。	Paper Code
From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection — Rupeng Zhang, Haowei Wang, Junjie Wang, Mingyang Li, Yuekai Huang, Dandan Wang, Qing Wang	Institute of Software, Chinese Academy of Sciences; University of Chinese Academy of Sciences	2025	NAACL	提出 ToolCommander：向工具平台植入对抗工具并操控检索与调度，从而实现隐私窃取、拒绝服务或未授权工具调用。	Paper Code
Prompt Injection Attack to Tool Selection in LLM Agents — Jiawen Shi, Zenghui Yuan, Guiyao Tie, Pan Zhou, Neil Zhenqiang Gong, Lichao Sun	HUST, Duke University, Lehigh University	2026	NDSS	提出 ToolHijacker：向工具库注入经两阶段优化的恶意工具描述，使智能体在目标任务中检索并选择攻击者控制的工具。	Paper arXiv
Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems — Donghyun Lee, Mo Tiwari	University researchers	2025	arXiv	展示可在通信智能体之间传播的自复制指令，形成类似恶意软件的多智能体传播，并提出面向来源追踪的 LLM 标签缓解方案。	Paper
Conjunctive Prompt Attacks in Multi-Agent LLM Systems — Nokimul Hasan Arif, Qian Lou, Mengxin Zheng	University of Central Florida	2026	ACL	将攻击拆分为用户查询中的触发键和受控远程智能体中的隐藏对抗模板；两部分单独看似正常，仅在路由组合后激活恶意行为。	Paper Github
PARASITE: Conditional System Prompt Poisoning to Hijack LLMs — Viet Pham, Thai Le	Indiana University Bloomington	2026	ACL	面向第三方系统提示市场的供应链攻击：植入条件式休眠触发器，仅在目标查询出现时操纵模型响应。	Paper Github

Web & Multimodal Injection

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs — Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi, Vitaly Shmatikov	Cornell Tech, Ben-Gurion University	2023	arXiv	展示如何将攻击者指令编码进多模态模型处理的图像或音频输入，将间接注入扩展到可见自然语言文本之外。	Paper
WebInject: Prompt Injection Attack to Web Agents — Xilong Wang, Julian Bloch, Zedian Shao, Yushi Hu, Shuyan Zhou, Neil Zhenqiang Gong	Duke University and collaborators	2025	EMNLP	优化渲染网页中的细微修改，使视觉语言 Web 智能体遵循攻击指令，同时页面对用户仍保持视觉合理性。	Paper
Manipulating Multimodal Agents via Cross-Modal Prompt Injection — multiple authors	Multi-institution	2025	ACM Multimedia	提出 CrossInject：将恶意控制信息分布到不同模态，使任何单一文本或图像通道都不一定包含完整攻击。	ACM
The Dangers of Indirect Prompt Injection Attacks on Web Agents — Stephen Johnson et al.	Multi-institution	2025	EMNLP System Demonstrations	展示嵌入 HTML 与无障碍树表示的对抗内容，包括可跨页面和任务影响 Web 智能体操作的可迁移触发器。	Paper
When Efficiency Becomes a Vulnerability: Computational Cost Attacks on WebAgents — Liang-Bo Ning, Yuchen Zhu, Heqing Huang, Xin Wang, Yi Chang, Qing Li, Wenqi Fan	Hong Kong Polytechnic University-led multi-institution team	2026	ACL	提出 CostBomb：向网页植入提示，在不妨碍正常任务完成的同时诱导 Web 智能体生成过长推理轨迹并大量消耗计算资源。	Paper
Physical Prompt Injection Attacks on Large Vision-Language Models — Chen Ling, Kai Hu, Hangcheng Liu, Xingshuo Han, Tianwei Zhang, Changhai Ou	Wuhan University, NTU, NUAA, et al.	2026	arXiv	提出 PPIA：将恶意排版指令置于真实物体或环境中，并结合离线提示选择与基于时空注意力的物理放置策略。	Paper Github

Defenses

Prevention

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
StruQ: Defending Against Prompt Injection with Structured Queries — Sizhe Chen, Julien Piet, Chawin Sitawarin, David Wagner	UC Berkeley	2025	USENIX Security	将受信任指令与不受信任数据分离到结构化通道，并训练模型只服从特权通道中的指令。	Paper USENIX Code
Jatmo: Prompt Injection Defense by Task-Specific Finetuning — Julien Piet et al.	UC Berkeley, KFUPM	2024	ESORICS	将通用指令遵循模型转化为任务专用模型，使其执行目标操作，但更难遵循嵌入数据中的任意指令。	Paper Code
Spotlighting: Defending Against Prompt Injection Attacks — Keegan Hines et al.	Microsoft	2024	CAMLIS	通过分隔、数据标记和编码等来源转换，使模型能够区分外部内容与受信任的应用指令。	Paper
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions — Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, Alex Beutel	OpenAI	2024	arXiv	训练模型优先遵循系统消息等高权限指令，而非与之冲突的低权限用户或第三方内容。	Paper
SecAlign: Defending Against Prompt Injection with Preference Optimization — Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar, Kamalika Chaudhuri, David Wagner, Chuan Guo	Meta FAIR, UC Berkeley	2025	ACM CCS	构造安全与不安全响应偏好对，并使用偏好优化训练模型优先完成合法任务，而非注入任务。（DPO）	Paper Code
Meta SecAlign: A Secure Foundation LLM Against Prompt Injection Attacks — Sizhe Chen, Arman Zharmagambetov, David Wagner, Chuan Guo	Meta FAIR, UC Berkeley	2025	arXiv	发布内置 SecAlign 式防护的开放权重 8B 与 70B 基础模型，并评估其向工具调用和网页导航任务的迁移能力。	Paper Code 70B Model 8B Model
Defense Against Prompt Injection Attack by Leveraging Attack Techniques — Yulin Chen, Haoran Li, Zihao Zheng, Dekai Wu, Yangqiu Song, Bryan Hooi	HKUST, National University of Singapore	2025	ACL	在构建防御时使用对抗攻击生成技术，使模型接触逐步增强的注入样本，而非固定的人工样本集。	Paper ACL
MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents — Kaijie Zhu, Xianjun Yang, Jindong Wang, Wenbo Guo, William Yang Wang	UC Santa Barbara, Microsoft Research and collaborators	2025	ICML	遮蔽用户任务后重新执行智能体轨迹；若原始执行与遮蔽执行产生相似工具操作，则说明该操作更可能由注入内容而非用户意图驱动。	Paper Code

Detection

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks — Yupei Liu, Yuqi Jia, Jinyuan Jia, Dawn Song, Neil Zhenqiang Gong	Penn State, Duke University, UC Berkeley	2025	arXiv	将检测器训练建模为极小极大优化：攻击者自适应搜索绕过检测的输入，检测器则从这些攻击中持续学习。	Paper
InjecGuard: Benchmarking and Mitigating Over-Defense in Prompt Injection Guardrail Models — Hao Li, Xiaogeng Liu	University of Wisconsin–Madison	2024	arXiv	提出良性数据集 NotInject，用于衡量可疑触发词造成的误报，并训练旨在缓解过度防御的开放式防护模型。	Paper Code
PromptShield: Deployable Detection for Prompt Injection Attacks — Dennis Jacob, Hend Alzahrani, Zhanhao Hu, Basel Alomair, David Wagner	UC Berkeley and collaborators	2025	arXiv	定义面向部署的检测器要求，并发布基准与训练后的提示注入检测器，重点关注数据构建和实际误报率。	Paper
Attention Tracker: Detecting Prompt Injection Attacks in LLMs — Kuan-Hao Hung et al.	Multi-institution	2025	Findings of NAACL	无需训练，利用注意力行为识别生成结果是否受到输入上下文中可疑片段的不成比例控制。	Paper
InstructDetector: Detecting Instruction Injection Attacks through Model Internals — Tianhao Wen et al.	Multi-institution	2025	Findings of EMNLP	使用隐藏状态和梯度衍生特征，检测不受信任内容是在发挥指令作用，还是仅在提供信息。	Paper
AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations — Yu He, Haozhe Zhu, Yiming Li, Shuo Shao, Hongwei Yao, Zhihao Liu, Zhan Qin	Zhejiang University and collaborators	2026	arXiv	对拟议工具调用进行反事实影子重放；当削弱不受信任观察的控制影响后某操作消失时，阻止该操作。	Paper
AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruption — Yanting Wang, Runpeng Geng, Ying Chen, Jinyuan Jia	Penn State	2026	IEEE S&P	使用基于注意力的上下文回溯识别导致响应的来源片段，为长上下文提示注入提供“先归因、后检测”流程。	Paper

Localization

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
PromptLocate: Localizing Prompt Injection Attacks — Yuqi Jia, Yupei Liu, Zedian Shao, Jinyuan Jia, Neil Gong	Duke University, Penn State	2025	arXiv	首个专门的提示注入定位方法：将受污染输入拆分为语义片段，并分别识别含有注入指令与注入数据的片段。	Paper Code
WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agents — Xilong Wang, Yinuo Liu, Zhun Wang, Dawn Song, Neil Gong	Duke University, UC Berkeley	2026	arXiv	提取可能受污染的网页片段，评估各片段与周围页面的语义一致性，同时返回检测结论与攻击位置。	Paper Code
AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruption — Yanting Wang, Runpeng Geng, Ying Chen, Jinyuan Jia	Penn State	2026	IEEE S&P	定位对模型输出贡献最大的上下文片段，实现对长文档和 RAG 上下文中注入指令的取证回溯。	Paper

Architectural & System-level Defenses

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
Defeating Prompt Injections by Design — Edoardo Debenedetti et al.	Google DeepMind, ETH Zurich	2025	arXiv	提出 CaMeL：分离控制流与不受信任的数据流，并使用基于能力的安全机制和显式数据依赖约束智能体操作。	Paper Code
IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems — Yuhao Wu, Franziska Roesner, Tadayoshi Kohno, Ning Zhang, Umar Iqbal	University of Washington, Virginia Tech and collaborators	2025	NDSS	将第三方工具和扩展隔离到独立执行上下文，并调解其通信，以减少跨工具数据泄露和混淆代理攻击。	Paper NDSS
Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents — Feiran Jia et al.	Penn State, Princeton University, CSU Long Beach	2025	ACL	维护授权任务的显式表示，并在执行前检查拟议的中间操作是否仍与该任务保持一致。	Paper ACL
Progent: Programmable Privilege Control for LLM Agents — Tianneng Shi et al.	UC Berkeley and collaborators	2025	arXiv	为工具调用及参数定义可编程策略，在 LLM 拟议操作与外部环境之间实施确定性的权限控制。	Paper
TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems — Kai Wang, Biaojie Zeng, Zeming Wei, Chang Jin, Hefeng Zhou, Xiangtian Li, Chao Yang, Jingjing Qu, Xingcheng Xu, Xia Hu	Shanghai AI Laboratory and collaborators	2026	arXiv	提供基于 OWASP 的三层分类体系，并结合多智能体系统抽象、特定风险评估模块、结构化轨迹分析与运行时监控智能体。	Paper

Datasets & Benchmarks

Attack & Defense Benchmarks

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs Through a Global Prompt Hacking Competition — Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Kost, Christopher Carnahan, Jordan Boyd-Graber	University of Maryland and multi-institution community	2023	EMNLP	来自全球竞赛的数据集，包含针对 3 个 LLM 的 60 多万个对抗提示，并提供人工提示攻击技术的分类体系。	Paper Dataset
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game — Sam Toyer, Olivia Watkins, Ethan Adrian Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Stuart Russell, Jacob Steinhardt	UC Berkeley and collaborators	2024	ICLR	收集在线游戏中人类生成的攻击与防御，利用自然演化的攻击策略支持提示提取和指令劫持基准评测。	Paper Website Code
Formalizing and Benchmarking Prompt Injection Attacks and Defenses — Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, Neil Zhenqiang Gong	Penn State, Duke University	2024	USENIX Security	Open-Prompt-Injection 基准实现形式化威胁模型，覆盖多种应用任务、攻击、防御与目标模型。	Paper Code
Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models — Jingwei Yi, Yueqi Xie, Bin Zhu, Emre Kiciman, Guangzhong Sun, Xing Xie, Fangzhao Wu	Microsoft Research, University of Science and Technology of China	2025	KDD	BIPIA 基准覆盖外部信息增强应用中的间接注入，包含攻击模板、应用任务、黑盒防御和白盒防御训练。	Paper Code
InjecGuard / NotInject: Benchmarking and Mitigating Over-Defense in Prompt Injection Guardrail Models — Hao Li, Xiaogeng Liu	University of Wisconsin–Madison	2024	arXiv	NotInject 包含 339 个富含注入相关触发词的良性样本，用于衡量检测器是否因词汇偏差拒绝无害指令。	Paper Code
PromptShield: Deployable Detection for Prompt Injection Attacks — Dennis Jacob, Hend Alzahrani, Zhanhao Hu, Basel Alomair, David Wagner	UC Berkeley and collaborators	2025	arXiv	面向实际检测器部署设计的基准，覆盖安全性能、良性输入误报、数据构成、模型规模与分类阈值。	Paper
CAPTURE: Context-Aware Prompt Injection Testing and Unified Robustness Evaluation — Gauri Kholkar, Ratinder Ahuja	Pure Storage	2025	LLMSEC Workshop at ACL	在多个应用领域评估上下文感知攻击与过度防御，而非将所有疑似指令的短语一律视为恶意。	Paper
PIArena: A Platform for Prompt Injection Evaluation — Runpeng Geng, Chenlong Yin, Yanting Wang, Ying Chen, Jinyuan Jia	Penn State	2026	arXiv	统一且可扩展的平台，整合攻击、防御、模型和基准，并包含基于自适应策略的攻击以评估防御鲁棒性。	Paper Code

Agent Security Benchmarks

General Agent & Prompt Injection

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, Florian Tramèr	ETH Zurich, Invariant Labs	2024	NeurIPS D&B	可扩展沙箱，覆盖邮件、银行和旅行场景中的 97 个真实任务与 629 个安全测试用例。	Paper Github
ARGUS / AgentLure: Context-Aware Prompt Injection Benchmark Shihao Weng, Yang Feng, Jinrui Zhang, Xiaofei Xie, Jiongchi Yu, Jia Liu	Nanjing University, SMU	2026	arXiv	AgentLure 基准包含 320 个样本、8 类攻击向量和 4 个领域，用于评估上下文相关的智能体间接提示注入。	Paper Code
AgentCanary: A Security Evaluation Framework for Autonomous AI Agents in Real Executable Environments Peiyang Li, Songping Wang, Yi Huang, Yanhua Shi, Chenhao Zhang, Qi Li, et al.	Ant Group, Tsinghua, Nanjing, PKU	2026	arXiv	提出“入口×影响”分类体系，在真实工具沙箱中从安全性、感知能力与效用三个维度进行轨迹评分。	Paper Github
AgentPI — Peiran Wang, Xinfeng Li, Chong Xiang, Jinghuai Zhang, Ying Li, Lixia Zhang, Xiaofeng Wang, Yuan Tian	UCLA, Nanyang Technological University, NVIDIA	2026	arXiv	基于 AgentPI 分类体系的上下文相关基准；评估攻击成功是否依赖智能体当前任务、外部上下文与工作流状态，而非仅依赖静态提示。	Paper
AgentDyn: A Dynamic Benchmark for Prompt Injection Attacks and Defenses in LLM Agents — H. Li, R. Wen, S. Shi, N. Zhang, C. Xiao	Multi-institution	2026	arXiv	生成动态变化的智能体环境与攻击配置，以减少对固定基准实例和静态工具输出的过拟合。	Paper
WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks — multiple authors	Multi-institution	2025	arXiv	评估嵌入真实网页的间接注入，同时衡量合法浏览任务完成情况与攻击者目标执行情况。	Paper

Tool, Memory & Skill Security

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated LLM Agents Qiusi Zhan, Zhixiang Liang, Zifan Ying, Daniel Kang	UIUC	2024	ACL Findings	包含 1,054 个间接提示注入测试用例，覆盖 17 个用户工具、62 个攻击者工具，以及直接危害与数据窃取场景。	Paper Github
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, Yongfeng Zhang	Zhejiang University, Rutgers	2025	ICLR	覆盖 10 个场景、400 多个工具和 27 类攻击/防御，包括直接注入、间接注入、记忆投毒与思维链后门。	Paper Github
MCP-SafetyBench: A Benchmark for Safety Evaluation of LLMs with Real-World MCP Servers Xuanjun Zong, Zhiqi Shen, Lei Wang, Yunshi Lan, Chao Yang	ECNU, Salesforce AI, SMU, Shanghai AI Lab	2025	arXiv	包含 245 个多轮案例，覆盖 5 个真实领域及服务器、主机、用户三层的 20 类 MCP 攻击。	Paper Github
MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers Zhiqiang Wang, Yichao Gao, Yanting Wang, Suyuan Liu, Haifeng Sun, Haoran Cheng, Guanquan Shi, Haohua Du, Xiangyang Li	USTC, Beihang	2026	AAAI	从 45 个在线 MCP 服务器的 353 个工具构建 1,348 个投毒案例；元数据级工具投毒的攻击成功率最高达 72.8%。	Paper Github
Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks David Schmotz, Luca Beurer-Kellner, Sahar Abdelnabi, Maksym Andriushchenko	ETH Zurich, CISPA, Invariant Labs	2026	arXiv	覆盖 23 个技能的 202 个“注入—任务”对，对比明显式与上下文式技能文件提示注入。	Paper Website
ClawSafety: "Safe" LLMs, Unsafe Agents Bowen Wei, Yunbei Zhang, Jinhao Pan, Kai Mei, Xiao Wang, Jihun Hamm, Ziwei Zhu, Yingqiang Ge	GMU, Tulane, Rutgers, ORNL	2026	arXiv	包含 120 个高权限工作区案例，覆盖个人智能体中的技能、邮件和网页注入向量。	Paper Github
SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents Yuchuan Tian, Mengyu Zheng, Haocheng Mei, Ye Yuan, Chao Xu, Xinghao Chen, Hanting Chen, Yu Wang	PKU, BJTU, SUIBE, Huawei, Tsinghua	2026	arXiv	包含 600 个分阶段任务，覆盖间接注入、工具返回注入、记忆投毒/提取，并区分 3 类危害终点。	Paper Dataset
From Untrusted Input to Trusted Memory (MPBench) Pritam Dash, Tongyu Ge, Aditi Jain, Tanmay Shah, Zhiwei Shang	University of British Columbia	2026	ICML	MPBench 覆盖 4 种写入通道中的 6 类记忆投毒，并采用写入/检索两阶段评估。	Paper

Web Agent Security

Title & Authors	Affiliation	Year	Conference	Brief	Links	Overview
WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents Yinuo Liu, Ruohan Xu, Xilong Wang, Yuqi Jia, Neil Zhenqiang Gong	Duke University	2025	arXiv	首个面向 Web 智能体提示注入的系统性文本+图像检测基准。	Paper Dataset
SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents Zonghao Ying, Yangguang Shao, Jianle Gan, Gan Xu, Wenxin Zhang, Quanchen Zou, Junzheng Shi, Zhenfei Yin, Mingchuan Zhang, Aishan Liu, Xianglong Liu	Beihang, CAS, U. Sydney	2026	ACL Findings	包含 6 个真实 Web 环境、2,970 条对抗轨迹，以及 6 类用户/环境攻击向量。	Paper
GUIGuard-Bench: Toward a General Evaluation for Privacy-Preserving GUI Agents Yanxi Wang, Zhiling Zhang, Wenbo Zhou, Weiming Zhang, Jie Zhang, Qiannan Zhu, Yu Shi, Shuxin Zheng, Jiyan He	BNU, Zhongguancun Academy, USTC, A*STAR	2026	arXiv	包含 241 条轨迹和 4,080 张截图，为 GUI 智能体工作流提供区域级隐私标注。	Paper Project

Industry Reports & Standards

Title & Authors	Affiliation	Year	Conference	Introduction	Links
OWASP Top 10 for LLM Applications (2025) OWASP	OWASP Foundation	2025	OWASP	Industry standard threat model; LLM01 covers prompt injection	Website
Google SAIF: Secure AI Framework Google	Google	2024	Google	Framework for securing AI systems including agent prompt injection	Website

Acknowledgements

Inspired by Neil Gong's talk Securing LLM Agents Against Prompt Injection Attacks, this list follows the taxonomy: Attacks → Defenses (Prevention / Detection / Localization) → Benchmarks.

And Codex is used for searching newest works and formating the file.