promptfoo--promptfoo/site/docs/red-team/owasp-agentic-ai.md

sidebar_position, description

sidebar_position	description
22	Red team agentic AI applications against OWASP Top 10 for Agentic Applications to protect autonomous AI systems from goal hijacking, tool misuse, and multi-agent attacks

OWASP Top 10 for Agentic Applications

The OWASP Top 10 for Agentic Applications was announced during Black Hat Europe 2025 and the OWASP Agentic Security Summit. It represents the most critical security risks for AI agent applications.

Unlike traditional LLM applications, agentic systems introduce unique security challenges due to their:

• Autonomous decision-making: Agents independently determine steps to achieve goals
• Persistent memory: Both short-term and long-term memory across sessions
• Tool and API access: Direct interaction with external systems
• Multi-agent coordination: Complex inter-agent communication and delegation

The Top 10 Risks

ID	Risk Name	Description
ASI01	Agent Goal Hijack	Attackers alter agent objectives through malicious content
ASI02	Tool Misuse and Exploitation	Agents use legitimate tools in unsafe ways
ASI03	Identity and Privilege Abuse	Agents inherit or escalate high-privilege credentials
ASI04	Agentic Supply Chain Vulnerabilities	Compromised tools, plugins, or external components
ASI05	Unexpected Code Execution	Agents generate or run code/commands unsafely
ASI06	Memory and Context Poisoning	Attackers poison agent memory systems and RAG databases
ASI07	Insecure Inter-Agent Communication	Multi-agent systems face spoofing and tampering
ASI08	Cascading Failures	Small errors propagate across planning and execution
ASI09	Human Agent Trust Exploitation	Users over-trust agent recommendations
ASI10	Rogue Agents	Compromised agents act harmfully while appearing legitimate

Scanning for OWASP Agentic Risks

To test against all 10 risks:

redteam:
  plugins:
    - owasp:agentic
  strategies:
    - jailbreak
    - jailbreak-templates
    - crescendo

Or target specific risks:

redteam:
  plugins:
    - owasp:agentic:asi01 # Agent Goal Hijack
    - owasp:agentic:asi02 # Tool Misuse and Exploitation
    - owasp:agentic:asi05 # Unexpected Code Execution

To set up the scan through the Promptfoo UI, select the OWASP Agentic preset on the Plugins page.

ASI01: Agent Goal Hijack

Agent Goal Hijack occurs when an attacker alters an agent's objectives or decision path through malicious content, exploiting the agent's planning and reasoning capabilities.

Attack Scenarios

• Gradual plan injection through subtle sub-goals that modify planning frameworks
• Direct instruction injection to ignore original objectives and execute unauthorized tool chains
• Reflection loop traps that trigger infinite self-analysis cycles

Testing Strategy

redteam:
  plugins:
    - hijacking
    - system-prompt-override
    - indirect-prompt-injection
    - intent
  strategies:
    - jailbreak
    - jailbreak-templates
    - jailbreak:composite

ASI02: Tool Misuse and Exploitation

Tool Misuse occurs when agents use legitimate tools in unsafe ways, whether through parameter pollution, tool chain manipulation, or automated abuse of granted permissions.

Attack Scenarios

• Parameter pollution: Manipulating function call parameters beyond intended scope
• Tool chain manipulation: Exploiting sequential tool calls to achieve unauthorized outcomes
• Automated abuse: Using authorized tools to perform actions at harmful scale

Testing Strategy

redteam:
  plugins:
    - excessive-agency
    - mcp
    - tool-discovery
  strategies:
    - jailbreak
    - jailbreak-templates

ASI03: Identity and Privilege Abuse

Agents inherit user/system identities with high-privilege credentials, creating opportunities for privilege escalation and unauthorized access across systems.

Attack Scenarios

• Dynamic permission escalation through temporary administrative privilege manipulation
• Cross-system exploitation due to inadequate scope enforcement
• Shadow agent deployment that inherits legitimate credentials

Testing Strategy

redteam:
  plugins:
    - rbac
    - bfla
    - bola
    - imitation
  strategies:
    - jailbreak
    - jailbreak-templates

ASI04: Agentic Supply Chain Vulnerabilities

Compromised tools, plugins, prompt templates, and external servers introduce vulnerabilities that agents may unknowingly leverage.

Attack Scenarios

• Malicious tool packages that execute hidden functionality
• Compromised prompt templates that inject adversarial instructions
• External API dependencies that return poisoned data

Testing Strategy

redteam:
  plugins:
    - indirect-prompt-injection
    - mcp
  strategies:
    - jailbreak-templates

ASI05: Unexpected Code Execution

Agents generate or run code/commands unsafely, creating opportunities for remote code execution, sandbox escapes, and data exfiltration.

Attack Scenarios

• DevOps agent compromise through generated scripts with hidden commands
• Workflow engine exploitation with AI-generated scripts containing backdoors
• Linguistic vulnerability exploitation to craft data exfiltration commands

Testing Strategy

redteam:
  plugins:
    - shell-injection
    - sql-injection
    - harmful:cybercrime:malicious-code
    - ssrf
  strategies:
    - jailbreak
    - jailbreak-templates

ASI06: Memory and Context Poisoning

Attackers poison agent memory systems, embeddings, and RAG databases to corrupt stored information and manipulate decision-making across sessions.

Attack Scenarios

• Gradual memory poisoning through repeated interactions
• Exploiting memory limits to prevent recognition of privilege escalation
• Corrupting shared memory in multi-agent systems

Testing Strategy

redteam:
  plugins:
    - agentic:memory-poisoning
    - cross-session-leak
    - indirect-prompt-injection
  strategies:
    - jailbreak
    - crescendo

ASI07: Insecure Inter-Agent Communication

Multi-agent systems face spoofed identities, replayed messages, and tampering in communication channels between agents.

Attack Scenarios

• Injecting misleading information to influence collaborative decision-making
• Forging false consensus messages to exploit authentication weaknesses
• Strategically planting false data that cascades through agent networks

Testing Strategy

redteam:
  plugins:
    - indirect-prompt-injection
    - hijacking
    - imitation
  strategies:
    - jailbreak-templates

ASI08: Cascading Failures

Small errors in one agent propagate across planning, execution, and memory, amplifying through interconnected systems.

Attack Scenarios

• Injecting false product details that accumulate in long-term memory
• Introducing hallucinated API endpoints that cause data leaks
• Implanting false information that progressively worsens through self-reinforcement

Testing Strategy

redteam:
  plugins:
    - hallucination
    - harmful:misinformation-disinformation
    - divergent-repetition
  strategies:
    - jailbreak
    - jailbreak-templates

ASI09: Human Agent Trust Exploitation

Users over-trust agent recommendations or explanations, enabling social engineering and covert harmful actions.

Attack Scenarios

• AI-powered invoice fraud replacing legitimate vendor details
• AI-driven phishing with deceptive messages containing malicious links
• Misinformation campaigns through trusted agent interfaces

Testing Strategy

redteam:
  plugins:
    - overreliance
    - imitation
    - harmful:misinformation-disinformation
  strategies:
    - crescendo

ASI10: Rogue Agents

Compromised or misaligned agents act harmfully while appearing legitimate, exploiting trust mechanisms in multi-agent workflows.

Attack Scenarios

• Malicious workflow injection impersonating approval agents
• Orchestration hijacking to route fraudulent transactions
• Coordinated agent flooding to overwhelm computing resources

Testing Strategy

redteam:
  plugins:
    - excessive-agency
    - hijacking
    - rbac
    - goal-misalignment
  strategies:
    - jailbreak
    - crescendo

Relationship to OWASP LLM Top 10

The OWASP Top 10 for Agentic Applications extends and complements the OWASP LLM Top 10:

Agentic Risk	Related LLM Top 10
ASI01: Agent Goal Hijack	LLM01: Prompt Injection
ASI02: Tool Misuse and Exploitation	LLM06: Excessive Agency
ASI03: Identity and Privilege Abuse	LLM06: Excessive Agency
ASI05: Unexpected Code Execution	LLM01, LLM05: Improper Output Handling
ASI06: Memory and Context Poisoning	LLM04: Data and Model Poisoning
ASI08: Cascading Failures	LLM09: Misinformation

Test both frameworks together for comprehensive coverage:

redteam:
  plugins:
    - owasp:agentic
    - owasp:llm
  strategies:
    - jailbreak
    - jailbreak-templates
    - crescendo

What's Next

To learn more about red teaming agents, see:

• How to Red Team LLM Agents
• Introduction to LLM Red Teaming
• Red Team Configuration

Additional Resources

• OWASP Top 10 for Agentic Applications
• OWASP GenAI Security Project
• OWASP LLM Top 10
• OWASP API Security Top 10