Files
promptfoo--promptfoo/site/docs/providers/a2a.md
T
wehub-resource-sync 0d3cb498a3
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:24:08 +08:00

441 lines
14 KiB
Markdown

---
sidebar_label: A2A
title: A2A Provider
description: Use Agent2Agent (A2A) HTTP+JSON agents as providers in promptfoo for evals and red teams
---
# A2A Provider
The `a2a` provider allows you to use agents that implement the
[Agent2Agent protocol](https://a2a-protocol.org/latest/specification/) directly as providers in
promptfoo. This is useful for testing agentic applications that expose an A2A interface for
message-based interaction, streaming responses, and asynchronous task execution.
Promptfoo sends each test prompt as an A2A message, waits for the agent to respond or complete a
task, and extracts the final text from the A2A response. When an Agent Card is available, promptfoo
can also use it to discover the agent endpoint and add advertised skills to red team generation
context.
## Setup
To use the A2A provider, you need an A2A server that supports the HTTP+JSON REST binding. The
current provider supports:
- `POST /message:send`
- `POST /message:stream`
- `GET /tasks/{id}` polling
- Agent Card discovery from `/.well-known/agent-card.json`
JSON-RPC, gRPC, and push-notification webhooks are not supported in this provider yet.
## Basic Configuration
The simplest configuration points promptfoo at the base URL for the A2A HTTP+JSON interface:
```yaml title="promptfooconfig.yaml"
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
auth:
type: bearer
token: '{{ env.A2A_API_KEY }}'
```
The `a2a:<url>` shorthand sets `config.url`. Promptfoo appends the operation paths, such as
`/message:send`, `/message:stream`, and `/tasks/{id}`, to this base URL.
## Agent Card Discovery
If your agent publishes an Agent Card, you can configure `agentCardUrl` instead of hardcoding the
A2A endpoint:
```yaml title="promptfooconfig.yaml"
providers:
- id: a2a
config:
agentCardUrl: https://agent.example.com/.well-known/agent-card.json
auth:
type: bearer
token: '{{ env.A2A_API_KEY }}'
mode: auto
```
When an Agent Card is configured, promptfoo selects the first supported interface with
`protocolBinding: HTTP+JSON` and uses its URL, tenant, protocol version, and streaming capability
unless you explicitly override them in `config`.
During red team generation, promptfoo also extracts useful Agent Card metadata such as the agent
name, description, capabilities, and skills. This gives the attack generator more target-specific
context, similar to how the MCP provider uses discovered tools.
## Configuration Options
| Option | Type | Default | Description |
| -------------------- | ---------------------------- | -------- | --------------------------------------------------------------------------- |
| `url` | string | - | Base URL for the A2A HTTP+JSON interface |
| `agentCardUrl` | string | - | URL of the Agent Card used for endpoint and capability discovery |
| `auth` | object | - | Bearer, basic, API key, or OAuth authentication configuration |
| `headers` | `Record<string, string>` | `{}` | Headers sent to the Agent Card endpoint and A2A operation requests |
| `mode` | `auto` \| `send` \| `stream` | `auto` | Whether to call `message:send`, `message:stream`, or choose automatically |
| `tenant` | string | - | Tenant override. Defaults to the selected Agent Card interface tenant |
| `protocolVersion` | string | `1.0` | Value for the `A2A-Version` header |
| `polling.enabled` | boolean | `true` | Poll non-terminal tasks returned by `message:send` |
| `polling.intervalMs` | number | `1000` | Delay between `GET /tasks/{id}` polls |
| `polling.timeoutMs` | number | `300000` | Maximum time to wait for task completion |
| `message` | object | - | Custom A2A message template |
| `configuration` | object | - | A2A message configuration sent with each request |
| `transformResponse` | string \| Function | - | JavaScript transform for reshaping the final provider response |
| `timeoutMs` | number | - | Per-request HTTP timeout. Defaults to promptfoo's provider request timeout. |
## Authentication
Use `auth` for common authentication schemes. Promptfoo applies it to both Agent Card discovery
requests and A2A operation requests. Values support Nunjucks variables, so use the `env` global for
environment variables, such as `{{ env.A2A_API_KEY }}`.
### Bearer Token
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
auth:
type: bearer
token: '{{ env.A2A_API_KEY }}'
```
### Basic Auth
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
auth:
type: basic
username: '{{ env.A2A_USERNAME }}'
password: '{{ env.A2A_PASSWORD }}'
```
### API Key
API keys default to the `X-API-Key` header. Set `placement: query` when the server expects a query
parameter instead.
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
auth:
type: api_key
keyName: X-API-Key
value: '{{ env.A2A_API_KEY }}'
```
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
auth:
type: api_key
placement: query
keyName: api_key
value: '{{ env.A2A_API_KEY }}'
```
### OAuth 2.0
OAuth supports the client credentials and password grants. If `tokenUrl` is omitted, promptfoo tries
OAuth authorization-server metadata discovery from the A2A server URL.
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
auth:
type: oauth
grantType: client_credentials
tokenUrl: https://auth.example.com/oauth/token
clientId: '{{ env.A2A_CLIENT_ID }}'
clientSecret: '{{ env.A2A_CLIENT_SECRET }}'
scopes:
- a2a.send
```
You can still use `headers` for custom headers that are not authentication-specific. If both
`headers.Authorization` and `auth` produce an `Authorization` header, the `auth` header wins.
## Request Modes
### Auto Mode
`mode: auto` chooses streaming when the Agent Card advertises streaming support. Otherwise it uses
`message:send`.
```yaml
providers:
- id: a2a
config:
agentCardUrl: https://agent.example.com/.well-known/agent-card.json
mode: auto
```
If you configure only `url` and no Agent Card, `auto` uses `message:send` because promptfoo has no
capability metadata to indicate that streaming is supported.
### Send and Poll
Use `mode: send` to call `POST /message:send`. If the response returns a non-terminal task,
promptfoo polls `GET /tasks/{id}` until the task completes, fails, is canceled, is rejected, or
requires more input/authentication.
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
mode: send
polling:
enabled: true
intervalMs: 1000
timeoutMs: 300000
```
### Streaming
Use `mode: stream` to call `POST /message:stream` and consume Server-Sent Events (SSE). Promptfoo
returns one final `ProviderResponse` when the stream closes or a terminal task state is reached.
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
mode: stream
```
The provider supports stream events containing `message`, `task`, `statusUpdate`, and
`artifactUpdate` payloads.
## Custom Messages
By default, promptfoo sends a `ROLE_USER` message with a single text part containing `{{prompt}}`.
You can provide a custom message template:
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
message:
role: ROLE_USER
parts:
- text: '{{prompt}}'
configuration:
returnImmediately: false
```
Promptfoo renders Nunjucks variables in `message`, `auth`, `headers`, `agentCardUrl`, `url`, and
`configuration`. It also adds a stable `messageId` and uses `sessionId` as the A2A `contextId` when
available.
## Response Transforms
Use `transformResponse` when the A2A response needs to be reshaped before promptfoo evaluates it.
This is useful when your agent returns multiple artifacts, structured data, or metadata that should
be promoted into the final provider response.
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
transformResponse: |
{
output: text,
metadata: { taskId: context.task?.id, mode: context.mode }
}
```
The transform receives three arguments, matching the HTTP provider convention:
- `json`: The normalized A2A result `{ message, task, events, raw }`
- `text`: Promptfoo's default extracted output
- `context`: A2A metadata `{ message, task, events, raw, mode }`
You can provide the transform as a JavaScript expression, a function, or a file reference:
```yaml
transformResponse: 'file://path/to/parser.js'
```
```javascript
module.exports = (json, text, context) => ({
output: json.task?.artifacts?.[0]?.parts?.[0]?.text ?? text,
metadata: { mode: context.mode },
});
```
Return a primitive value to set `output`, or return a full `ProviderResponse` object when you need
fields such as `metadata`, `guardrails`, or `sessionId`. Function and file-based transforms may be
async; promptfoo awaits them before evaluating the response.
For inline JavaScript expressions, `result` is also available as an alias for `json`.
## Output Extraction
If you do not provide `transformResponse`, promptfoo extracts output in this order:
1. Direct A2A `message` text parts
2. Completed task artifact text parts
3. Task status message text
4. Task history text
5. Raw JSON fallback
Text parts are joined with newlines. Structured data remains available through `raw`,
`metadata.a2a`, and `transformResponse`.
### Direct message response
If the agent returns a direct message:
```json
{
"message": {
"role": "ROLE_AGENT",
"parts": [{ "text": "I can help book that flight." }]
}
}
```
Promptfoo output is:
```text
I can help book that flight.
```
### Completed task artifact
If `message:send` returns a task and polling later returns a completed task with artifacts:
```json
{
"id": "task-123",
"status": {
"state": "TASK_STATE_COMPLETED",
"message": {
"role": "ROLE_AGENT",
"parts": [{ "text": "Completed" }]
}
},
"artifacts": [
{
"artifactId": "final-answer",
"parts": [{ "text": "The best itinerary is SFO to JFK at 9:00 AM." }]
}
]
}
```
Promptfoo output is the artifact text, not the lifecycle status message:
```text
The best itinerary is SFO to JFK at 9:00 AM.
```
### Status message fallback
If there is no direct message and no artifact, promptfoo falls back to task status text:
```json
{
"id": "task-123",
"status": {
"state": "TASK_STATE_COMPLETED",
"message": {
"role": "ROLE_AGENT",
"parts": [{ "text": "Completed with no artifact." }]
}
}
}
```
Promptfoo output is:
```text
Completed with no artifact.
```
### Structured output
For agents that return useful non-text fields, use `transformResponse` to shape the output:
```yaml
providers:
- id: a2a:https://agent.example.com/a2a/v1
config:
transformResponse: |
{
output: text,
metadata: {
taskId: json.task?.id,
state: json.task?.status?.state,
eventCount: json.events?.length ?? 0
}
}
```
## Red Team Testing with A2A
A2A targets work with normal promptfoo red team configuration. When `agentCardUrl` is configured,
the provider can add Agent Card skills and capabilities to the generated attack context, helping
promptfoo produce probes that are specific to the target agent.
```yaml title="promptfooconfig.yaml"
description: A2A travel agent red team
providers:
- id: a2a
config:
agentCardUrl: https://travel-agent.example.com/.well-known/agent-card.json
auth:
type: bearer
token: '{{ env.A2A_API_KEY }}'
redteam:
purpose: |
The system is a travel booking agent that helps users search for flights,
compare itineraries, and manage reservations.
plugins:
- pii
- bola
- bfla
- excessive-agency
strategies:
- basic
```
## Error Handling
The A2A provider returns provider errors for common failure cases:
- Agent Card or operation requests return non-2xx HTTP responses
- Responses are not valid JSON
- Streaming responses contain malformed SSE frames
- Tasks reach `TASK_STATE_FAILED`, `TASK_STATE_CANCELED`, or `TASK_STATE_REJECTED`
- Tasks require additional input or authentication
- Polling exceeds `polling.timeoutMs`
## Limitations
- Only the A2A HTTP+JSON REST binding is supported
- JSON-RPC and gRPC bindings are not supported yet
- Push-notification webhooks are not supported because promptfoo evals require a synchronous result
- Streaming is consumed into a single final `ProviderResponse`; token-by-token UI streaming is not exposed
- Agent Card discovery currently selects the first `HTTP+JSON` supported interface
## See Also
- [A2A specification](https://a2a-protocol.org/latest/specification/)
- [A2A definitions](https://a2a-protocol.org/latest/definitions/)
- [MCP Provider](./mcp.md)
- [HTTP Provider](./http.md)
- [Red Team Testing Guide](../red-team/index.md)