LLMail-Inject: Adaptive Prompt Injection Challenge is a large language model security evaluation dataset released by Microsoft in 2025, designed to study and evaluate the defensive capabilities of LLM-integrated email clients when facing adaptive prompt injection attacks.

The dataset collects attack submissions from the LLMail-Inject adaptive prompt injection challenge, covering tasks such as email summarization and data exfiltration, and constructs multiple challenge levels by combining different retrieval environments, LLM models, and defense mechanisms. The data mainly includes the subject and body of attack emails, as well as execution feedback such as retrieval, detection, and tool invocation, which can be used for prompt injection attack research, LLM security evaluation, and defense mechanism analysis.

Dataset Composition

The dataset mainly contains attack submissions from two phases:

• Phase1: Phase 1 challenge data, stored in data/raw_submissions_phase1.jsonl.
• Phase2: Phase 2 challenge data, stored in data/raw_submissions_phase2.jsonl.

Each submission mainly contains the following information:

• Subject: The subject of the attack email.
• Body: The body of the attack email.
• Level: Challenge level, determined by the combination of specific scenario, defense mechanism, and LLM model.
• Retrieval status: Retrieval status.
• Detection status: Prompt injection detection status.
• Tool invocation status: Tool invocation status.

Citation

@article{abdelnabi2025,
  title     = {LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge},
  author    = {Sahar Abdelnabi and Aideen Fay and Ahmed Salem and Egor Zverev and Chi-Huang Liu and Chun-Chih Kuo and Jannis Weigend and Danyael Manlangit and Alex Apostolov and Haris Umair and João Donato and Masayuki Kawakita and Athar Mahboob and Tran Huu Bach and Tsun-Han Chiang and Myeongjin Cho and Hajin Choi and Byeonghyeon Kim and Hyeonjin Lee and Benjamin Pannell and Conor Mac Amhlaoibh and Mark Russinovich and Andrew Paverd and Giovanni Cherubin},
  year      = {2025},
  journal   = {Under submission},
  note      = {Challenge dataset and results from the LLMail-Inject Adaptive Prompt Injection Challenge. Available at \url{https://github.com/microsoft/llmail-inject-challenge-analysis}},
}