promptfoo--promptfoo/site/docs/providers/aws-bedrock.md

---
title: AWS Bedrock
sidebar_label: AWS Bedrock
sidebar_position: 3
description: Configure Amazon Bedrock for LLM evals with Claude, Llama, Nova, and Mistral models using AWS-managed infrastructure
---

# Bedrock

The `bedrock` provider lets you use Amazon Bedrock in your evals. It supports Bedrock model IDs directly, including regional IDs and inference profile IDs. Because AWS changes the Bedrock catalog over time, use the [AWS supported models documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html), [model IDs documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns), or `aws bedrock list-foundation-models` as the source of truth for current model IDs and regional availability.

## Setup

1. **Model Access**: Access rules vary by provider and can change over time.
   - Check the [AWS supported models documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html) for the current access path and regional availability of the model you want to use
   - **Anthropic models**: May require one-time use case submission through the model catalog
   - **AWS Marketplace models**: Some third-party models require IAM permissions with `aws-marketplace:Subscribe`
   - **Access control**: Organizations maintain control through IAM policies and Service Control Policies (SCPs)

2. Install the `@aws-sdk/client-bedrock-runtime` package:

   ```sh
   npm install @aws-sdk/client-bedrock-runtime
   ```

3. The AWS SDK will automatically pull credentials from the following locations:
   - IAM roles on EC2
   - `~/.aws/credentials`
   - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables

   See [setting node.js credentials (AWS)](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html) for more details.

4. Edit your configuration file to point to the AWS Bedrock provider. Here's an example:

   ```yaml
   providers:
     - id: bedrock:us.anthropic.claude-sonnet-4-6
   ```

   Note that the provider is `bedrock:` followed by the [ARN/model id](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns) of the model.

5. Additional config parameters are passed like so:

   ```yaml
   providers:
     - id: bedrock:us.anthropic.claude-sonnet-4-6
       config:
         accessKeyId: YOUR_ACCESS_KEY_ID
         secretAccessKey: YOUR_SECRET_ACCESS_KEY
         region: 'us-west-2'
         max_tokens: 256
         temperature: 0.7
   ```

## Application Inference Profiles

AWS Bedrock supports Application Inference Profiles, which allow you to use a single ARN to access multiple foundation models across different regions. This helps optimize costs and availability while maintaining consistent performance.

### Using Inference Profiles

When using an inference profile ARN, you must specify the `inferenceModelType` in your configuration to indicate which model family the profile is configured for:

```yaml
providers:
  - id: bedrock:arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-profile
    config:
      inferenceModelType: 'claude' # Required for inference profiles
      region: 'us-east-1'
      max_tokens: 256
      temperature: 0.7
```

### Supported Model Types

The `inferenceModelType` config option supports the following values:

- `claude` - For Anthropic Claude models
- `nova` - For Amazon Nova models (v1)
- `nova2` - For Amazon Nova 2 models (with reasoning support)
- `llama` - For Meta Llama models (defaults to Llama 4)
- `llama2` - For Meta Llama 2 models
- `llama3` - For Meta Llama 3 models
- `llama3.1` or `llama3_1` - For Meta Llama 3.1 models
- `llama3.2` or `llama3_2` - For Meta Llama 3.2 models
- `llama3.3` or `llama3_3` - For Meta Llama 3.3 models
- `llama4` - For Meta Llama 4 models
- `mistral` - For Mistral models
- `cohere` - For Cohere models
- `ai21` - For AI21 models
- `titan` - For Amazon Titan models
- `deepseek` - For DeepSeek models
- `openai` - For OpenAI open-weight (gpt-oss) models
- `qwen` - For Alibaba Qwen models
- `zai` - For Z.AI GLM models
- `minimax` - For MiniMax models
- `moonshot` - For Moonshot Kimi models
- `nvidia` - For NVIDIA Nemotron models
- `writer` - For Writer Palmyra models
- `gemma` - For Google Gemma models

### Example: Multi-Region Inference Profile

```yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
providers:
  # Claude Opus 4.7 via global inference profile
  - id: bedrock:arn:aws:bedrock:us-east-2::inference-profile/global.anthropic.claude-opus-4-7
    config:
      inferenceModelType: 'claude'
      region: 'us-east-2'
      max_tokens: 1024
      temperature: 0.7

  # Using an inference profile that routes to Claude models
  - id: bedrock:arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/claude-profile
    config:
      inferenceModelType: 'claude'
      max_tokens: 1024
      temperature: 0.7
      anthropic_version: 'bedrock-2023-05-31'

  # Using an inference profile for Llama models
  - id: bedrock:arn:aws:bedrock:us-west-2:123456789012:application-inference-profile/llama-profile
    config:
      inferenceModelType: 'llama3.3'
      max_gen_len: 1024
      temperature: 0.7

  # Using an inference profile for Nova models
  - id: bedrock:arn:aws:bedrock:eu-west-1:123456789012:application-inference-profile/nova-profile
    config:
      inferenceModelType: 'nova'
      interfaceConfig:
        max_new_tokens: 1024
        temperature: 0.7
```

:::tip

Application Inference Profiles provide several benefits:

- **Automatic failover**: If one region is unavailable, requests automatically route to another region
- **Cost optimization**: Routes to the most cost-effective available model
- **Simplified management**: Use a single ARN instead of managing multiple model IDs

When using inference profiles, ensure the `inferenceModelType` matches the model family your profile is configured for, as the configuration parameters differ between model types.

:::

## Converse API

The Converse API provides a unified interface across supported Bedrock models with
native support for extended thinking (reasoning), tool calling, and guardrails. Use
the `bedrock:converse:` prefix to access this API.

### Basic Usage

```yaml
providers:
  - id: bedrock:converse:anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      region: us-east-1
      maxTokens: 4096
      temperature: 0.7
```

### Extended Thinking

Enable Claude's extended thinking capabilities for complex reasoning tasks:

```yaml
providers:
  - id: bedrock:converse:us.anthropic.claude-sonnet-4-5-20250929-v1:0
    config:
      region: us-west-2
      maxTokens: 20000
      thinking:
        type: enabled
        budget_tokens: 16000
      showThinking: true # Include thinking content in output
```

The `thinking` configuration controls Claude's reasoning behavior:

- `type: enabled` - Activates extended thinking
- `budget_tokens` - Maximum tokens allocated for thinking (minimum 1024)
- For Claude Opus 4.7 and 4.8, promptfoo converts `type: enabled` to adaptive thinking because manual thinking is not accepted by those models.

Use `showThinking: true` to include the model's reasoning process in the output, or `false` to only show the final response.

:::warning
Do not set `temperature`, `topP`, or `topK` when using extended thinking. These sampling parameters are incompatible with reasoning mode.
:::

### Configuration Options

| Option                | Description                                               |
| --------------------- | --------------------------------------------------------- |
| `maxTokens`           | Maximum output tokens                                     |
| `temperature`         | Sampling temperature (0-1)                                |
| `topP`                | Nucleus sampling parameter                                |
| `stopSequences`       | Array of stop sequences                                   |
| `thinking`            | Extended thinking configuration (Claude models)           |
| `reasoningConfig`     | Reasoning configuration (Amazon Nova 2 models)            |
| `showThinking`        | Include thinking in output (default: true)                |
| `performanceConfig`   | Performance settings (`latency: optimized`)               |
| `serviceTier`         | Service tier object (`type: priority \| default \| flex`) |
| `guardrailIdentifier` | Guardrail ID for content filtering                        |
| `guardrailVersion`    | Guardrail version (default: DRAFT)                        |

### Performance Configuration

Optimize for latency or cost:

```yaml
providers:
  - id: bedrock:converse:anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      performanceConfig:
        latency: optimized # or 'standard'
      serviceTier:
        type: priority # or 'default', 'flex'
```

### Supported Models

The Converse API works with Bedrock models that support the `Converse` operation.
Because AWS changes that compatibility matrix over time, use the
[AWS Converse supported models documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html)
as the source of truth for current support.

### Model Context Protocol (MCP) Servers

The Converse provider can attach [Model Context Protocol](https://modelcontextprotocol.io)
servers and surface their tools to the model alongside any `tools` you configure
manually. MCP tool definitions are discovered at provider startup, converted to
Bedrock `toolSpec` entries, and sent on every request.

```yaml
providers:
  - id: bedrock:converse:us.anthropic.claude-sonnet-4-6
    config:
      region: us-east-1
      maxTokens: 1024
      mcp:
        enabled: true
        servers:
          # Remote MCP server (Streamable HTTP)
          - name: deepwiki
            url: https://mcp.deepwiki.com/mcp
          # Or a local stdio MCP server
          # - name: filesystem
          #   command: npx
          #   args: ['-y', '@modelcontextprotocol/server-filesystem', '/tmp']
        # Optional: only expose specific tools
        tools:
          - ask_question
      toolChoice: auto
```

**Single-turn execution.** When the model returns a `tool_use` block, the provider
executes the requested MCP tool and returns the **raw tool result** as the final
output. The result is not fed back to the model for a follow-up turn — there is no
agent loop. Write your assertions against the tool output text directly, or wrap
the provider in an agent harness if you need a synthesized natural-language answer.

**Tool name collisions.** If an entry under `config.tools` has the same `name` as
an MCP-discovered tool, the MCP version wins and the duplicate is dropped with a
warning. Bedrock rejects duplicate tool names with `ValidationException`, so
deduping is required.

**Lifecycle.** Stdio MCP servers spawn a child process; the provider registers
itself with the evaluator's shutdown hook so transports are released when the
eval finishes. If MCP initialization fails (bad URL, missing binary, handshake
failure), the failure is surfaced as a `ProviderResponse.error` on the first
`callApi` rather than crashing the eval. MCP errors during a tool call are
likewise propagated to `error` so failed runs do not pass silently.

**Disabling tools.** Setting `toolChoice: none` (or `tool_choice: none`) skips
the entire tool path: no MCP definitions are sent in the request and no MCP
tools are invoked even if the model returns a stale `tool_use` block.

## Authentication

Amazon Bedrock supports multiple authentication methods, including API key
authentication for simplified access. Credentials are resolved in this priority order:

### Credential Resolution Order

Credentials are resolved in the following priority order:

1. **Explicit credentials in config** (`accessKeyId`, `secretAccessKey`)
2. **Bedrock API Key authentication** (`apiKey`)
3. **SSO profile authentication** (`profile`)
4. **AWS default credential chain** (environment variables, `~/.aws/credentials`)

The first available credential method is used automatically.

### Authentication Options

#### 1. Explicit credentials (highest priority)

Specify AWS access keys directly in your configuration. **For security, use environment variables instead of hardcoding credentials:**

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      accessKeyId: '{{env.AWS_ACCESS_KEY_ID}}'
      secretAccessKey: '{{env.AWS_SECRET_ACCESS_KEY}}'
      sessionToken: '{{env.AWS_SESSION_TOKEN}}' # Optional, for temporary credentials
      region: 'us-east-1' # Optional, defaults to us-east-1
```

**Environment variables:**

```bash
export AWS_ACCESS_KEY_ID="your_access_key_id"
export AWS_SECRET_ACCESS_KEY="your_secret_access_key"
export AWS_SESSION_TOKEN="your_session_token"  # Optional
```

:::warning Security Best Practice

**Do not commit credentials to version control.** Use environment variables or a dedicated secrets management system to handle sensitive keys.

:::

This method overrides all other credential sources, including EC2 instance roles and SSO profiles.

#### 2. API Key authentication

Amazon Bedrock API keys provide simplified authentication without managing AWS IAM credentials.

**Using environment variables:**

Set the `AWS_BEARER_TOKEN_BEDROCK` environment variable:

```bash
export AWS_BEARER_TOKEN_BEDROCK="your-api-key-here"
```

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      region: 'us-east-1' # Optional, defaults to us-east-1
```

**Using config file:**

Specify the API key directly in your configuration:

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      apiKey: 'your-api-key-here'
      region: 'us-east-1' # Optional, defaults to us-east-1
```

:::note

API keys are limited to Amazon Bedrock and Amazon Bedrock Runtime actions. They cannot be used with:

- InvokeModelWithBidirectionalStream operations
- Agents for Amazon Bedrock API operations
- Data Automation for Amazon Bedrock API operations

For these advanced features, use traditional AWS IAM credentials instead.

:::

#### 3. SSO profile authentication

Use a named profile from your AWS configuration for AWS SSO setups or managing multiple AWS accounts:

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      profile: 'YOUR_SSO_PROFILE'
      region: 'us-east-1' # Optional, defaults to us-east-1
```

**Prerequisites for SSO profiles:**

1. **Install AWS CLI v2**: Ensure AWS CLI v2 is installed and on your PATH.

2. **Configure AWS SSO**: Set up AWS SSO using the AWS CLI:

   ```bash
   aws configure sso
   ```

3. **Profile configuration**: Your `~/.aws/config` should contain the profile:

   ```ini
   [profile YOUR_SSO_PROFILE]
   sso_start_url = https://your-sso-portal.awsapps.com/start
   sso_region = us-east-1
   sso_account_id = 123456789012
   sso_role_name = YourRoleName
   region = us-east-1
   ```

4. **Active SSO session**: Ensure you have an active SSO session:
   ```bash
   aws sso login --profile YOUR_SSO_PROFILE
   ```

**Use SSO profiles when:**

- Managing multi-account AWS environments
- Working in organizations with centralized AWS SSO
- Your team needs different role-based permissions
- You need to switch between different AWS contexts

#### 4. Default credentials (lowest priority)

Use the AWS SDK's standard credential chain:

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      region: 'us-east-1' # Only region specified
```

**The AWS SDK checks these sources in order:**

1. **Environment variables**: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`
2. **Shared credentials file**: `~/.aws/credentials` (from `aws configure`)
3. **AWS IAM roles**: EC2 instance profiles, ECS task roles, Lambda execution roles
4. **Shared AWS CLI credentials**: Including cached SSO credentials

**Use default credentials when:**

- Running on AWS infrastructure (EC2, ECS, Lambda) with IAM roles
- Developing locally with AWS CLI configured (`aws configure`)
- Working in CI/CD environments with IAM roles or environment variables

**Quick setup for local development:**

```bash
# Option 1: Using AWS CLI
aws configure

# Option 2: Using environment variables
export AWS_ACCESS_KEY_ID="your_access_key"
export AWS_SECRET_ACCESS_KEY="your_secret_key"
export AWS_DEFAULT_REGION="us-east-1"
```

## Example

See [GitHub](https://github.com/promptfoo/promptfoo/tree/main/examples/amazon-bedrock) for full examples of Claude, Nova, AI21, Llama 3.3, Grok, Mantle Chat Completions, and OpenAI-compatible Bedrock model usage.

```yaml title="promptfooconfig.yaml"
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
prompts:
  - 'Write a tweet about {{topic}}'

providers:
  # Using inference profiles (requires inferenceModelType)
  - id: bedrock:arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-claude-profile
    config:
      inferenceModelType: 'claude'
      region: 'us-east-1'
      temperature: 0.7
      max_tokens: 256

  # Using regular model IDs
  - id: bedrock:meta.llama3-1-405b-instruct-v1:0
    config:
      region: 'us-east-1'
      temperature: 0.7
      max_tokens: 256
  - id: bedrock:us.meta.llama3-3-70b-instruct-v1:0
    config:
      max_gen_len: 256
  - id: bedrock:amazon.nova-lite-v1:0
    config:
      region: 'us-east-1'
      interfaceConfig:
        temperature: 0.7
        max_new_tokens: 256
  - id: bedrock:us.amazon.nova-premier-v1:0
    config:
      region: 'us-east-1'
      interfaceConfig:
        temperature: 0.7
        max_new_tokens: 256
  - id: bedrock:us.anthropic.claude-sonnet-4-5-20250929-v1:0
    config:
      region: 'us-east-1'
      temperature: 0.7
      max_tokens: 256
  - id: bedrock:us.anthropic.claude-opus-4-1-20250805-v1:0
    config:
      region: 'us-east-1'
      temperature: 0.7
      max_tokens: 256
  - id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      region: 'us-east-1'
      temperature: 0.7
      max_tokens: 256
  - id: bedrock:us.anthropic.claude-3-5-haiku-20241022-v1:0
    config:
      region: 'us-east-1'
      temperature: 0.7
      max_tokens: 256
  - id: bedrock:us.anthropic.claude-opus-4-8
    config:
      region: 'us-east-1'
      temperature: 0.7
      max_tokens: 256
  - id: bedrock:openai.gpt-5.5 # frontier: Responses API, needs a Bedrock API key
    config:
      region: 'us-east-2'
      apiKey: '{{env.AWS_BEARER_TOKEN_BEDROCK}}'
      reasoning_effort: 'medium'
      max_output_tokens: 256
  - id: bedrock:openai.gpt-oss-120b-1:0
    config:
      region: 'us-west-2'
      temperature: 0.7
      max_completion_tokens: 256
      reasoning_effort: 'medium'
  - id: bedrock:openai.gpt-oss-20b-1:0
    config:
      region: 'us-west-2'
      temperature: 0.7
      max_completion_tokens: 256
      reasoning_effort: 'low'
  - id: bedrock:qwen.qwen3-coder-480b-a35b-v1:0
    config:
      region: 'us-west-2'
      temperature: 0.7
      max_tokens: 256
      showThinking: true
  - id: bedrock:qwen.qwen3-32b-v1:0
    config:
      region: 'us-east-1'
      temperature: 0.7
      max_tokens: 256

tests:
  - vars:
      topic: Our eco-friendly packaging
  - vars:
      topic: A sneak peek at our secret menu item
  - vars:
      topic: Behind-the-scenes at our latest photoshoot
```

## Model-specific Configuration

Different models may support different configuration options. Here are some model-specific parameters:

### General Configuration Options

- `inferenceModelType`: (Required for inference profiles) Specifies the model family when using application inference profiles. See [Supported Model Types](#supported-model-types) for the full list of values.

### Amazon Nova Models

Amazon Nova models (e.g., `amazon.nova-lite-v1:0`, `amazon.nova-pro-v1:0`, `amazon.nova-micro-v1:0`, `amazon.nova-premier-v1:0`) support advanced features like tool use and structured outputs. You can configure them with the following options:

```yaml
providers:
  - id: bedrock:amazon.nova-lite-v1:0
    config:
      interfaceConfig:
        max_new_tokens: 256 # Maximum number of tokens to generate
        temperature: 0.7 # Controls randomness (0.0 to 1.0)
        top_p: 0.9 # Nucleus sampling parameter
        top_k: 50 # Top-k sampling parameter
        stopSequences: ['END'] # Optional stop sequences
      toolConfig: # Optional tool configuration
        tools:
          - toolSpec:
              name: 'calculator'
              description: 'A basic calculator for arithmetic operations'
              inputSchema:
                json:
                  type: 'object'
                  properties:
                    expression:
                      description: 'The arithmetic expression to evaluate'
                      type: 'string'
                  required: ['expression']
        toolChoice: # Optional tool selection
          tool:
            name: 'calculator'
```

:::note

Nova models use a slightly different configuration structure compared to other Bedrock models, with separate `interfaceConfig` and `toolConfig` sections.

:::

### Amazon Nova 2 Models (Reasoning)

Amazon Nova 2 models introduce extended thinking capabilities with configurable reasoning levels. Nova 2 Lite (`amazon.nova-2-lite-v1:0`) supports step-by-step reasoning and task decomposition with a 1 million token context window.

```yaml
providers:
  # Use cross-region model ID (us.) for on-demand access
  - id: bedrock:us.amazon.nova-2-lite-v1:0
    config:
      interfaceConfig:
        max_new_tokens: 4096
      reasoningConfig:
        type: enabled # Enable extended thinking
        maxReasoningEffort: medium # low, medium, or high
```

**Reasoning Configuration:**

- `type`: Set to `enabled` to activate extended thinking, or `disabled` for fast responses (default)
- `maxReasoningEffort`: Controls thinking depth - `low`, `medium`, or `high`

When extended thinking is enabled, the model's reasoning process is captured in the response output with `<thinking>` tags, similar to other reasoning models.

:::warning

When using `reasoningConfig` with `type: enabled`:

- **For all reasoning modes**: Do not set `temperature`, `top_p`, or `top_k` - these are incompatible with reasoning mode
- **For `maxReasoningEffort: high`**: Also do not set `max_new_tokens` - the model manages output length automatically

:::

**Regional Model IDs:**

Nova 2 models require cross-region inference profiles for on-demand access:

- `us.amazon.nova-2-lite-v1:0` - US region (recommended)
- `eu.amazon.nova-2-lite-v1:0` - EU region
- `apac.amazon.nova-2-lite-v1:0` - Asia Pacific region
- `global.amazon.nova-2-lite-v1:0` - Global cross-region inference

**Using Nova 2 with Converse API:**

Nova 2 reasoning is also supported via the Converse API, which provides a unified interface across Bedrock models:

```yaml
providers:
  - id: bedrock:converse:us.amazon.nova-2-lite-v1:0
    config:
      maxTokens: 4096
      reasoningConfig:
        type: enabled
        maxReasoningEffort: medium
```

The same parameter constraints apply when using the Converse API.

### Amazon Nova Sonic Model

The Amazon Nova Sonic model (`amazon.nova-sonic-v1:0`) is a multimodal model that supports audio input and text/audio output with tool-using capabilities. It has a different configuration structure compared to other Nova models:

```yaml
providers:
  - id: bedrock:amazon.nova-sonic-v1:0
    config:
      inferenceConfiguration:
        maxTokens: 1024 # Maximum number of tokens to generate
        temperature: 0.7 # Controls randomness (0.0 to 1.0)
        topP: 0.95 # Nucleus sampling parameter
      textOutputConfiguration:
        mediaType: text/plain
      toolConfiguration: # Optional tool configuration
        tools:
          - toolSpec:
              name: 'getDateTool'
              description: 'Get information about the current date'
              inputSchema:
                json: '{"$schema":"http://json-schema.org/draft-07/schema#","type":"object","properties":{},"required":[]}'
      toolUseOutputConfiguration:
        mediaType: application/json
      # Optional audio output configuration
      audioOutputConfiguration:
        mediaType: audio/lpcm
        sampleRateHertz: 24000
        sampleSizeBits: 16
        channelCount: 1
        voiceId: matthew
        encoding: base64
        audioType: SPEECH
```

Note: Nova Sonic has advanced multimodal capabilities including audio input/output, but audio input requires base64 encoded data which may be better handled through the API directly rather than in the configuration file.

### Amazon Nova Reel (Video Generation)

Amazon Nova Reel (`amazon.nova-reel-v1:1`) generates studio-quality videos from text prompts. Videos are generated in 6-second increments up to 2 minutes.

:::note Prerequisites

Nova Reel requires an Amazon S3 bucket for video output. Your AWS credentials must have:

- `bedrock:InvokeModel` and `bedrock:StartAsyncInvoke` permissions
- `s3:PutObject` permission on the output bucket
- `s3:GetObject` permission for downloading generated videos

:::

Check the [AWS supported models documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html)
for current Nova Reel regional availability.

#### Basic Configuration

```yaml
providers:
  - id: bedrock:video:amazon.nova-reel-v1:1
    config:
      region: us-east-1
      s3OutputUri: s3://my-bucket/videos # Required
      durationSeconds: 6 # Default: 6
      seed: 42 # Optional: for reproducibility
```

#### Task Types

Nova Reel supports three task types:

**TEXT_VIDEO (default)** - Generate a 6-second video from a text prompt:

```yaml
providers:
  - id: bedrock:video:amazon.nova-reel-v1:1
    config:
      s3OutputUri: s3://my-bucket/videos
      taskType: TEXT_VIDEO
      durationSeconds: 6
```

**MULTI_SHOT_AUTOMATED** - Generate longer videos (12-120 seconds) from a single prompt:

```yaml
providers:
  - id: bedrock:video:amazon.nova-reel-v1:1
    config:
      s3OutputUri: s3://my-bucket/videos
      taskType: MULTI_SHOT_AUTOMATED
      durationSeconds: 18 # Must be multiple of 6
```

**MULTI_SHOT_MANUAL** - Define individual shots with separate prompts:

```yaml
providers:
  - id: bedrock:video:amazon.nova-reel-v1:1
    config:
      s3OutputUri: s3://my-bucket/videos
      taskType: MULTI_SHOT_MANUAL
      durationSeconds: 12
      shots:
        - text: 'Drone footage of a forest from high altitude'
        - text: 'Camera arcs around vehicles in a forest'
```

#### Image-to-Video Generation

Use an image as the starting frame (must be 1280x720):

```yaml
providers:
  - id: bedrock:video:amazon.nova-reel-v1:1
    config:
      s3OutputUri: s3://my-bucket/videos
      image: file://path/to/image.png
```

#### Configuration Options

| Option            | Description                                                  | Default    |
| ----------------- | ------------------------------------------------------------ | ---------- |
| `s3OutputUri`     | S3 bucket URI for output (required)                          | -          |
| `taskType`        | `TEXT_VIDEO`, `MULTI_SHOT_AUTOMATED`, or `MULTI_SHOT_MANUAL` | TEXT_VIDEO |
| `durationSeconds` | Video duration (6, or 12-120 in multiples of 6)              | 6          |
| `seed`            | Random seed (0-2,147,483,646)                                | -          |
| `image`           | Starting frame image (file:// path or base64)                | -          |
| `shots`           | Shot definitions for MULTI_SHOT_MANUAL                       | -          |
| `pollIntervalMs`  | Polling interval in ms                                       | 10000      |
| `maxPollTimeMs`   | Maximum polling time in ms                                   | 900000     |
| `downloadFromS3`  | Download video to local blob storage                         | true       |

Generated videos are 1280x720 resolution at 24 FPS in MP4 format.

:::warning Generation Time
Video generation is asynchronous and takes approximately:

- 6-second video: ~90 seconds
- 2-minute video: ~14-17 minutes

The provider polls for completion automatically.
:::

### AI21 Models

For AI21 models (e.g., `ai21.jamba-1-5-mini-v1:0`, `ai21.jamba-1-5-large-v1:0`), you can use the following configuration options:

```yaml
config:
  max_tokens: 256
  temperature: 0.7
  top_p: 0.9
  frequency_penalty: 0.5
  presence_penalty: 0.3
```

### Claude Models

For Claude models (e.g., `anthropic.claude-fable-5`, `anthropic.claude-sonnet-5`, `anthropic.claude-sonnet-4-6`, `anthropic.claude-sonnet-4-5-20250929-v1:0`, `anthropic.claude-haiku-4-5-20251001-v1:0`, `anthropic.claude-sonnet-4-20250514-v1:0`, `anthropic.us.claude-3-5-sonnet-20241022-v2:0`), you can use the following configuration options:

**Note**: Claude Opus 4.8 (`anthropic.claude-opus-4-8`) and Claude Opus 4.7 (`anthropic.claude-opus-4-7`) are available via cross-region inference profiles (`us.`, `eu.`, `jp.`, `global.`) and, in select regions, through the base foundation model ID. Claude Opus 4.6 (`anthropic.claude-opus-4-6-v1`) and Claude Opus 4.5 (`anthropic.claude-opus-4-5-20251101-v1:0`) require an inference profile ARN and cannot be used as a direct model ID. See the [Application Inference Profiles](#application-inference-profiles) section for setup. promptfoo automatically omits unsupported sampling parameters (`temperature`, `topP`, and `topK` — including raw `top_k` in `additionalModelRequestFields`) and converts configured manual thinking to adaptive thinking for Opus 4.7, Opus 4.8, and Sonnet 5.

**Note**: Claude Sonnet 5 (`anthropic.claude-sonnet-5`) is available through the base foundation model ID and the `us.`/`eu.`/`global.` cross-region inference profiles (e.g. `bedrock:global.anthropic.claude-sonnet-5`); use the `global.` profile for dynamic routing. Cost is reported on both the default `bedrock:` (InvokeModel) and `bedrock:converse:` paths — the `global.` endpoint bills at the standard $3/$15 rate and regional/geo profiles (`us.`/`eu.`) add the 10% Claude 4.5+ regional premium.

#### Claude Fable and Mythos models

[Claude Fable 5](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-fable-5.html)
supports Bedrock Runtime and Converse. Use the `global.anthropic.claude-fable-5`
inference profile — on-demand invocation of the base `anthropic.claude-fable-5` ID
returns a `ValidationException`, and the `us.`/`eu.` geo profiles listed on the
model card may not be provisioned in every region. Fable 5 also supports
Bedrock's Anthropic-compatible Messages endpoint through the explicit
`bedrock:messages:anthropic.claude-fable-5` provider ID in `us-east-1` and
`eu-north-1` (this route may additionally require account enablement from AWS).

[Claude Mythos 5](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-mythos-5.html)
is available only through the Anthropic-compatible Messages endpoint in `us-east-1`.
Promptfoo routes the bare `bedrock:anthropic.claude-mythos-5` ID to that endpoint.
Set a Bedrock API key in `AWS_BEARER_TOKEN_BEDROCK` or `config.apiKey`:

```yaml
providers:
  - id: bedrock:anthropic.claude-mythos-5
    config:
      region: us-east-1
      apiKey: '{{env.AWS_BEARER_TOKEN_BEDROCK}}'
```

AWS requires provider data sharing to be enabled for Fable 5 and Mythos 5 — without
it every request fails with `data retention mode 'default' is not available for this
model`. Opt in per region via the Data Retention API:

```bash
aws bedrock put-account-data-retention --mode provider_data_share --region us-east-1
```

Both models use always-on adaptive thinking, so promptfoo omits sampling controls,
converts manual thinking budgets (`thinking: { type: 'enabled', budget_tokens: N }`)
to adaptive thinking, and omits `thinking: { type: 'disabled' }`. Regional and geo
endpoints cost 10% more than the global endpoint; Promptfoo applies that premium when
calculating costs.

```yaml
config:
  max_tokens: 256
  temperature: 0.7
  anthropic_version: 'bedrock-2023-05-31'
  tools: [...] # Optional: Specify available tools
  tool_choice: { ... } # Optional: Specify tool choice
  thinking: { ... } # Optional: Enable Claude's extended thinking capability
  showThinking: true # Optional: Control whether thinking content is included in output
```

When using Claude's extended thinking capability, you can configure it like this:

```yaml
config:
  max_tokens: 20000
  thinking:
    type: 'enabled'
    budget_tokens: 16000 # Must be ≥1024 and less than max_tokens
  showThinking: true # Whether to include thinking content in the output (default: true)
```

:::tip

The `showThinking` parameter controls whether thinking content is included in the response output:

- When set to `true` (default), thinking content will be included in the output
- When set to `false`, thinking content will be excluded from the output

This is useful when you want to use thinking for better reasoning but don't want to expose the thinking process to end users.

:::

### Titan Models

:::warning Retired

Amazon **Titan text** models (`amazon.titan-text-express/lite/premier`) have been retired on
Bedrock and are no longer available in any Region. Use [Amazon Nova](#amazon-nova-models)
instead. Titan **embeddings** models remain available (see [Embeddings](#embeddings)).

:::

For the (legacy) Titan text models, you can use the following configuration options:

```yaml
config:
  maxTokenCount: 256
  temperature: 0.7
  topP: 0.9
  stopSequences: ['END']
```

### Llama

For Llama models (e.g., `meta.llama3-1-70b-instruct-v1:0`, `meta.llama3-2-90b-instruct-v1:0`, `meta.llama3-3-70b-instruct-v1:0`, `meta.llama4-scout-17b-instruct-v1:0`, `meta.llama4-maverick-17b-instruct-v1:0`), you can use the following configuration options:

```yaml
config:
  max_gen_len: 256
  temperature: 0.7
  top_p: 0.9
```

#### Llama 3.2 Vision

Llama 3.2 Vision models (`us.meta.llama3-2-11b-instruct-v1:0`, `us.meta.llama3-2-90b-instruct-v1:0`) support image inputs. You can use them with either the legacy InvokeModel API or the Converse API:

**Using InvokeModel API (legacy):**

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:us.meta.llama3-2-11b-instruct-v1:0
    config:
      region: us-east-1
      max_gen_len: 256

prompts:
  - file://llama_vision_prompt.json

tests:
  - vars:
      image: file://path/to/image.jpg
```

```json title="llama_vision_prompt.json"
[
  {
    "role": "user",
    "content": [
      {
        "type": "image",
        "source": {
          "type": "base64",
          "media_type": "image/jpeg",
          "data": "{{image}}"
        }
      },
      {
        "type": "text",
        "text": "What is in this image?"
      }
    ]
  }
]
```

**Using Converse API:**

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:converse:us.meta.llama3-2-11b-instruct-v1:0
    config:
      region: us-east-1
      maxTokens: 256
```

The Converse API uses the same prompt format shown above for [Nova Vision](#nova-vision-capabilities).

### Cohere Models

For Cohere models (e.g., `cohere.command-r-v1:0`), you can use the following configuration options:

```yaml
config:
  max_tokens: 256
  temperature: 0.7
  p: 0.9
  k: 0
  stop_sequences: ['END']
```

### Mistral Models

Legacy Mistral text-completion models such as `mistral.mistral-7b-instruct-v0:2` support:

```yaml
config:
  max_tokens: 256
  temperature: 0.7
  top_p: 0.9
  top_k: 50
```

Mistral chat-completion models such as `mistral.mistral-large-2407-v1:0`,
`mistral.devstral-2-123b`, `mistral.mistral-large-3-675b-instruct`, and
`mistral.pixtral-large-2502-v1:0` use `messages` requests and support the same
options except `top_k`.

### DeepSeek Models

For DeepSeek models, you can use the following configuration options:

```yaml
config:
  # Deepseek params
  max_tokens: 256
  temperature: 0.7
  top_p: 0.9

  # Promptfoo control params
  showThinking: true # Optional: Control whether thinking content is included in output
```

`deepseek.r1-v1:0` supports extended thinking output. The `showThinking` parameter controls whether R1 thinking content is included in the response output:

- When set to `true` (default), thinking content will be included in the output
- When set to `false`, thinking content will be excluded from the output

`deepseek.v3-v1:0` and `deepseek.v3.2` use chat-completion style `messages`
requests and return the final assistant message directly.

### OpenAI Models

Amazon Bedrock hosts two families of OpenAI models, and they are served by **different
APIs**. promptfoo routes each `bedrock:openai.*` id to the correct one automatically.

#### Frontier models (GPT-5.x)

- **`openai.gpt-5.5`**: Flagship frontier model (US East / Ohio `us-east-2`)
- **`openai.gpt-5.4`**: Frontier model (US East / Ohio `us-east-2` and US West / Oregon `us-west-2`)

The frontier models are served only through Bedrock's **OpenAI-compatible Responses API**
on the regional "mantle" endpoint (`https://bedrock-mantle.<region>.api.aws/openai/v1`) —
not the native `InvokeModel` API. promptfoo routes `bedrock:openai.gpt-5.5` to its OpenAI
Responses provider pointed at that endpoint, so the output is **identical to the
[`openai:responses:gpt-5.5`](/docs/providers/openai/) provider** (the clean final answer;
chain-of-thought is hidden).

Authentication uses an **Amazon Bedrock API key**, not the AWS SDK credential chain. Set
`AWS_BEARER_TOKEN_BEDROCK` (or `config.apiKey`):

```yaml
providers:
  - id: bedrock:openai.gpt-5.5
    config:
      region: us-east-2 # gpt-5.5: us-east-2; gpt-5.4: us-east-2 or us-west-2
      apiKey: '{{env.AWS_BEARER_TOKEN_BEDROCK}}' # or just export AWS_BEARER_TOKEN_BEDROCK
      reasoning_effort: low # none | low | medium | high | xhigh
      max_output_tokens: 2048
```

Prefer the `bedrock:openai.gpt-5.5` form above. It wraps the OpenAI Responses provider,
points it at the mantle endpoint, and normalizes the `openai.`-prefixed id for GPT-5
capability detection (reasoning effort, verbosity) and billing. Using
`openai:responses:openai.gpt-5.5` directly is **not** equivalent — the base provider does
not recognize the `openai.` prefix as a GPT-5 model, so reasoning/verbosity controls would
be dropped.

#### Open-weight models (GPT OSS)

- **`openai.gpt-oss-120b-1:0`**: 120 billion parameter general-purpose model
- **`openai.gpt-oss-20b-1:0`**: 20 billion parameter general-purpose model
- **`openai.gpt-oss-safeguard-120b`**: 120 billion parameter safety model
- **`openai.gpt-oss-safeguard-20b`**: 20 billion parameter safety model

The open-weight models are served through Bedrock's native `InvokeModel` API and use the
standard AWS SDK credential chain, with OpenAI-style request parameters:

```yaml
providers:
  - id: bedrock:openai.gpt-oss-120b-1:0
    config:
      region: us-west-2
      max_completion_tokens: 1024 # OpenAI-style parameter (not max_tokens)
      temperature: 0.7
      top_p: 0.9
      frequency_penalty: 0.1
      presence_penalty: 0.1
      stop: ['END', 'STOP']
      reasoning_effort: medium # low | medium | high
      showThinking: false # strip the <reasoning> block from output (see below)
```

#### Reasoning Effort

Both families accept the native `reasoning_effort` request parameter, which promptfoo
forwards as-is so the API validates it for the specific model:

- **GPT OSS** (`openai.gpt-oss-*`): `low`, `medium`, `high`
- **Frontier** (`openai.gpt-5.x`): `none`, `low`, `medium`, `high`, `xhigh`

Note that `minimal` is **not** a valid value for these Bedrock models (the API rejects it).
Higher effort produces more thorough reasoning at the cost of latency and output tokens.

#### Reasoning Output and `showThinking` (GPT OSS only)

When invoked through `InvokeModel`, the open-weight models prepend their chain-of-thought
wrapped in `<reasoning>...</reasoning>` before the final answer. This differs from OpenAI's
first-party API, which hides chain-of-thought.

By default promptfoo returns this output **verbatim**, so the reasoning stays visible to your
assertions and red-team graders — an eval framework should not hide model-returned content by
default. Use `showThinking` to transform it:

- **`showThinking: false`** — strip the reasoning block so `output` is the clean final answer,
  matching the [`openai:` providers](/docs/providers/openai/) (which hide chain-of-thought).
- **`showThinking: true`** — surface the reasoning in the `Thinking: <reasoning>\n\n<answer>`
  format the OpenAI chat provider uses.

The frontier models return clean output already, so this option does not apply to them.

:::note Codex on Bedrock

OpenAI's [Codex](https://developers.openai.com/codex/) coding agent uses these same
frontier model IDs (`openai.gpt-5.5`, `openai.gpt-5.4`). To run the full coding agent
against Bedrock, use `openai:codex-sdk` with `model_provider: amazon-bedrock` — see
[Run on Amazon Bedrock](/docs/providers/openai-codex-sdk/#option-3-run-on-amazon-bedrock)
in the Codex SDK docs. For direct (non-agentic) inference, use `bedrock:openai.gpt-5.5`
as shown above.

:::

### xAI Grok Models

xAI's **Grok 4.3** (`xai.grok-4.3`) runs on the same Bedrock **Mantle** engine as the OpenAI
frontier models and is served through the **OpenAI-compatible Responses API** on the regional
mantle endpoint (`https://bedrock-mantle.<region>.api.aws/openai/v1`) — not `InvokeModel` or
`Converse`. It is offered in **`us-west-2`** (check the Bedrock model card for current regional
availability) and authenticates with an **Amazon Bedrock API key** (set
`AWS_BEARER_TOKEN_BEDROCK`, or `config.apiKey`).

```yaml
providers:
  - id: bedrock:xai.grok-4.3
    config:
      region: us-west-2 # Grok 4.3 is only available in us-west-2
      apiKey: '{{env.AWS_BEARER_TOKEN_BEDROCK}}' # or just export AWS_BEARER_TOKEN_BEDROCK
      reasoning_effort: low # Grok is reasoning-first: none | low | medium | high
      max_output_tokens: 4096
```

:::note

- Grok 4.3 is **reasoning-first**: reasoning is always active and the effort is configurable
  (`none` | `low` | `medium` | `high`). promptfoo forwards `reasoning_effort` (or
  `reasoning: { effort }`) and surfaces reasoning token counts in `tokenUsage`.
- Grok accepts an explicit `temperature`. When you omit it, promptfoo does not inject the OpenAI
  provider default, so Bedrock uses Grok's model default instead.
- **Cost is not reported** for Grok (`cost` is left undefined). The Responses
  billing tables are keyed on OpenAI model names; refer to the
  [Amazon Bedrock pricing page](https://aws.amazon.com/bedrock/pricing/) for Grok rates.

:::

### Mantle Chat Completions (`bedrock:mantle:`) {#mantle-chat-completions}

The Bedrock **Mantle** engine also exposes an OpenAI-compatible **Chat Completions** API. Most
mantle chat models use `https://bedrock-mantle.<region>.api.aws/v1/chat/completions`; xAI and
Gemma 4 chat models use the `/openai/v1/chat/completions` variant. Use the
**`bedrock:mantle:<id>`** prefix to talk to it. This is the only way to reach mantle-served chat models that the native
`InvokeModel`/`Converse` APIs don't serve — so they don't appear in
`aws bedrock list-foundation-models` — for example `zai.glm-4.6`, `deepseek.v3.1`,
`google.gemma-4-*`, and the mantle-namespaced Qwen `*-instruct` IDs.

Like the other mantle paths, it authenticates with an **Amazon Bedrock API key**
(`AWS_BEARER_TOKEN_BEDROCK`, or `config.apiKey`):

```yaml
providers:
  - id: bedrock:mantle:zai.glm-4.6
    config:
      region: us-west-2
      apiKey: '{{env.AWS_BEARER_TOKEN_BEDROCK}}' # or just export AWS_BEARER_TOKEN_BEDROCK
      max_tokens: 1024
```

:::note

- **The mantle catalog is regional.** List the models available in a Region with
  `GET https://bedrock-mantle.<region>.api.aws/v1/models`, and set `region` accordingly —
  the default is `us-east-1`.
- Use the bare `bedrock:openai.gpt-5.5` / `bedrock:xai.grok-4.3` forms (above) for the OpenAI
  frontier and Grok models: those go through the **Responses API** and surface reasoning
  tokens. `bedrock:mantle:` is the **Chat Completions** path for mantle chat models such as
  `zai.glm-4.6`, `deepseek.v3.1`, `google.gemma-4-*`, and supported xAI chat ids.
- Models that the native APIs do serve (Claude, Nova, Llama, Qwen, the
  [OpenAI-compatible families](#openai-compatible-models) above, etc.) are usually better
  reached via `bedrock:<id>` or `bedrock:converse:<id>`.

:::

### Qwen Models

Qwen model IDs include `qwen.qwen3-coder-next`, `qwen.qwen3-next-80b-a3b`,
`qwen.qwen3-vl-235b-a22b`, `qwen.qwen3-coder-480b-a35b-v1:0`,
`qwen.qwen3-coder-30b-a3b-v1:0`, `qwen.qwen3-235b-a22b-2507-v1:0`, and
`qwen.qwen3-32b-v1:0`. Qwen models support advanced features including hybrid
thinking modes, tool calling, and extended context understanding.

**Regional Availability**: Check the [AWS Bedrock console](https://console.aws.amazon.com/bedrock/home) or use `aws bedrock list-foundation-models` to verify which Qwen models are available in your target region, as availability varies by model and region.

You can configure them with the following options:

```yaml
config:
  max_tokens: 2048 # Maximum number of tokens to generate
  temperature: 0.7 # Controls randomness (0.0 to 1.0)
  top_p: 0.9 # Nucleus sampling parameter
  frequency_penalty: 0.1 # Reduces repetition of frequent tokens
  presence_penalty: 0.1 # Reduces repetition of any tokens
  stop: ['END', 'STOP'] # Stop sequences
  showThinking: true # Control whether thinking content is included in output
  tools: [...] # Tool calling configuration (optional)
  tool_choice: 'auto' # Tool selection strategy (optional)
```

#### Hybrid Thinking Modes

Qwen models support hybrid thinking modes where the model can apply step-by-step reasoning before delivering the final answer. The `showThinking` parameter controls whether thinking content is included in the response output:

- When set to `true` (default), thinking content will be included in the output
- When set to `false`, thinking content will be excluded from the output

This allows you to access the model's reasoning process during generation while having the option to present only the final response to end users.

#### Tool Calling

Qwen models support tool calling with OpenAI-compatible function definitions.

```yaml
config:
  tools:
    - type: function
      function:
        name: calculate
        description: Perform arithmetic calculations
        parameters:
          type: object
          properties:
            expression:
              type: string
              description: The mathematical expression to evaluate
          required: ['expression']
  tool_choice: auto # 'auto', 'none', or specific function name
```

#### Model Variants

- **Qwen3-Coder-480B-A35B**: Mixture-of-experts model optimized for coding and agentic tasks with 480B total parameters and 35B active parameters
- **Qwen3-Coder-30B-A3B**: Smaller MoE model with 30B total parameters and 3B active parameters, optimized for coding tasks
- **Qwen3-Coder-Next**: Coding model exposed through Bedrock
- **Qwen3-Next-80B-A3B**: General-purpose MoE model
- **Qwen3-VL-235B-A22B**: Vision-language model that also accepts text prompts
- **Qwen3-235B-A22B**: General-purpose MoE model with 235B total parameters and 22B active parameters for reasoning and coding
- **Qwen3-32B**: Dense model with 32B parameters for consistent performance in resource-constrained environments

#### Usage Example

```yaml
providers:
  - id: bedrock:qwen.qwen3-coder-480b-a35b-v1:0
    config:
      region: us-west-2
      max_tokens: 2048
      temperature: 0.7
      top_p: 0.9
      showThinking: true
      tools:
        - type: function
          function:
            name: code_analyzer
            description: Analyze code for potential issues
            parameters:
              type: object
              properties:
                code:
                  type: string
                  description: The code to analyze
              required: ['code']
      tool_choice: auto
```

### OpenAI-compatible Models (GLM, MiniMax, Kimi, Nemotron, Gemma, Palmyra) {#openai-compatible-models}

Several Bedrock families speak the OpenAI Chat Completions schema over `InvokeModel`
(`{ messages, max_tokens, ... }` → `{ choices: [{ message: { content } }] }`), so they share
one handler and the same configuration options. They also work through the [Converse API](#converse-api)
(`bedrock:converse:<id>`).

| Family          | Example model IDs                                                                                                         |
| --------------- | ------------------------------------------------------------------------------------------------------------------------- |
| Z.AI GLM        | `zai.glm-5`, `zai.glm-4.7`, `zai.glm-4.7-flash`                                                                           |
| MiniMax         | `minimax.minimax-m2`, `minimax.minimax-m2.1`, `minimax.minimax-m2.5`                                                      |
| Moonshot Kimi   | `moonshotai.kimi-k2.5`, `moonshot.kimi-k2-thinking`                                                                       |
| NVIDIA Nemotron | `nvidia.nemotron-nano-9b-v2`, `nvidia.nemotron-nano-12b-v2`, `nvidia.nemotron-nano-3-30b`, `nvidia.nemotron-super-3-120b` |
| Google Gemma 3  | `google.gemma-3-4b-it`, `google.gemma-3-12b-it`, `google.gemma-3-27b-it`                                                  |
| Writer Palmyra  | `us.writer.palmyra-x5-v1:0`, `us.writer.palmyra-x4-v1:0`, `writer.palmyra-vision-7b`                                      |

```yaml
providers:
  - id: bedrock:zai.glm-5
    config:
      region: us-east-1
      max_tokens: 1024 # Maximum number of tokens to generate
      temperature: 0.7 # Optional — omit to use the model's own default
      top_p: 0.9 # Optional nucleus sampling
      stop: ['END'] # Optional stop sequences
      reasoning_effort: high # Optional, reasoning models only ('low' | 'medium' | 'high')
      showThinking: false # Strip <think>/<reasoning> blocks from the output (default: keep)
      tools: [...] # Optional OpenAI-format tool definitions
      tool_choice: 'auto' # Optional tool selection strategy
```

:::note

- **Writer Palmyra** is served for on-demand throughput only through its `us.` inference
  profile (`bedrock:us.writer.palmyra-x5-v1:0`); the bare `writer.palmyra-x*` IDs reject
  on-demand `InvokeModel`.
- **Reasoning models** (MiniMax M2, Kimi K2 Thinking) emit a `<think>` or `<reasoning>`
  block. By default it is returned verbatim; set `showThinking: false` to return only the
  final answer. Give reasoning models a larger `max_tokens` budget so the answer is not
  truncated by the reasoning.
- **NVIDIA Nemotron** reasons in-line without tags, so `showThinking` cannot strip it.
  Disable its reasoning with NVIDIA's `/no_think` system directive instead (add a
  `system` message of `/no_think` to your prompt) for a direct answer.
- This handler does not force a `temperature`/`top_p` default, so each model uses its
  provider-recommended sampling unless you set them explicitly.

:::

**Regional Availability**: Check the [AWS Bedrock console](https://console.aws.amazon.com/bedrock/home)
or AWS model cards to confirm which of these models are enabled in your target region. Use
`aws bedrock list-foundation-models` for direct foundation model IDs and
`aws bedrock list-inference-profiles` for inference profiles such as Writer Palmyra's `us.`
route — availability varies by model and region. TwelveLabs Pegasus
(`twelvelabs.pegasus-1-2-v1:0`, video understanding) is also available through the Converse
API, and TwelveLabs Marengo (`twelvelabs.marengo-embed-*`) is an [embeddings](#embeddings) model.

## Model-graded tests

You can use Bedrock models to grade outputs. By default, model-graded tests use an OpenAI grader and require the `OPENAI_API_KEY` environment variable to be set. However, when using AWS Bedrock, you have the option of overriding the grader for [model-graded assertions](/docs/configuration/expected-outputs/model-graded/) to point to AWS Bedrock or other providers.

You can use either regular model IDs or application inference profiles for grading:

:::warning

Because of how model-graded evals are implemented, **the LLM grading models must support chat-formatted prompts** (except for embedding or classification models).

:::

To set this for all your test cases, add the [`defaultTest`](/docs/configuration/guide/#default-test-cases) property to your config:

```yaml title="promptfooconfig.yaml"
defaultTest:
  options:
    provider:
      # Using a regular model ID
      id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
      config:
        temperature: 0
        # Other provider config options

      # Or using an inference profile
      # id: bedrock:arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/grading-profile
      # config:
      #   inferenceModelType: 'claude'
      #   temperature: 0
```

You can also do this for individual assertions:

```yaml
# ...
assert:
  - type: llm-rubric
    value: Do not mention that you are an AI or chat assistant
    provider:
      text:
        id: provider:chat:modelname
        config:
          region: us-east-1
          temperature: 0
          # Other provider config options...
```

Or for individual tests:

```yaml
# ...
tests:
  - vars:
      # ...
    options:
      provider:
        id: provider:chat:modelname
        config:
          temperature: 0
          # Other provider config options
    assert:
      - type: llm-rubric
        value: Do not mention that you are an AI or chat assistant
```

## Multimodal Capabilities

Several Bedrock models support multimodal inputs including images and text:

- **Amazon Nova** - Supports images and videos
- **Llama 3.2 Vision** - Supports images (11B and 90B variants)
- **Claude 3+** - Supports images (via Converse API)
- **Pixtral Large** - Supports images (via Converse API)

To use these capabilities, structure your prompts to include both image data and text content.

### Nova Vision Capabilities

Amazon Nova supports comprehensive vision understanding for both images and videos:

- **Images**: Supports PNG, JPG, JPEG, GIF, WebP formats via Base-64 encoding. Multiple images allowed per payload (up to 25MB total).
- **Videos**: Supports various formats (MP4, MKV, MOV, WEBM, etc.) via Base-64 (less than 25MB) or Amazon S3 URI (up to 1GB).

Here's an example configuration for running multimodal evaluations:

```yaml title="promptfooconfig.yaml"
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: 'Bedrock Nova Eval with Images'

prompts:
  - file://nova_multimodal_prompt.json

providers:
  - id: bedrock:amazon.nova-pro-v1:0
    config:
      region: 'us-east-1'
      inferenceConfig:
        temperature: 0.7
        max_new_tokens: 256

tests:
  - vars:
      image: file://path/to/image.jpg
```

The prompt file (`nova_multimodal_prompt.json`) should be structured to include both image and text content. This format will depend on the specific model you're using:

```json title="nova_multimodal_prompt.json"
[
  {
    "role": "user",
    "content": [
      {
        "image": {
          "format": "jpg",
          "source": { "bytes": "{{image}}" }
        }
      },
      {
        "text": "What is this a picture of?"
      }
    ]
  }
]
```

See [GitHub](https://github.com/promptfoo/promptfoo/blob/main/examples/amazon-bedrock/models/promptfooconfig.nova.multimodal.yaml) for a runnable example.

When loading image files as variables, promptfoo automatically converts them to the appropriate format for the model. The supported image formats include:

- jpg/jpeg
- png
- gif
- bmp
- webp
- svg

## Embeddings

To override the embeddings provider for all assertions that require embeddings (such as similarity), use `defaultTest`:

```yaml
defaultTest:
  options:
    provider:
      embedding:
        id: bedrock:embeddings:amazon.titan-embed-text-v2:0
        config:
          region: us-east-1
```

## Guardrails

To use guardrails, set the `guardrailIdentifier` and `guardrailVersion` in the provider config.

For example:

```yaml
providers:
  - id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      guardrailIdentifier: 'test-guardrail'
      guardrailVersion: 1 # The version number for the guardrail. The value can also be DRAFT.
```

## Environment Variables

The following environment variables can be used to configure the Bedrock provider:

**Authentication:**

- `AWS_BEARER_TOKEN_BEDROCK`: Bedrock API key for simplified authentication

**Configuration:**

- `AWS_BEDROCK_REGION`: Default region for Bedrock API calls
- `AWS_BEDROCK_MAX_TOKENS`: Default maximum number of tokens to generate
- `AWS_BEDROCK_TEMPERATURE`: Default temperature for generation
- `AWS_BEDROCK_TOP_P`: Default top_p value for generation
- `AWS_BEDROCK_FREQUENCY_PENALTY`: Default frequency penalty (for supported models)
- `AWS_BEDROCK_PRESENCE_PENALTY`: Default presence penalty (for supported models)
- `AWS_BEDROCK_STOP`: Default stop sequences (as a JSON string)
- `AWS_BEDROCK_MAX_RETRIES`: Number of retry attempts for failed API calls (default: 10)

Model-specific environment variables:

- `MISTRAL_MAX_TOKENS`, `MISTRAL_TEMPERATURE`, `MISTRAL_TOP_P`, `MISTRAL_TOP_K`: For Mistral models
- `COHERE_TEMPERATURE`, `COHERE_P`, `COHERE_K`, `COHERE_MAX_TOKENS`: For Cohere models

These environment variables can be overridden by the configuration specified in the YAML file.

## Troubleshooting

### Authentication Issues

#### "Unable to locate credentials" Error

```text
Error: Unable to locate credentials. You can configure credentials by running "aws configure".
```

**Solutions:**

1. **Check credential priority**: Ensure credentials are available in the expected priority order
2. **Verify AWS CLI setup**: Run `aws configure list` to see active credentials
3. **SSO session expired**: Run `aws sso login --profile YOUR_PROFILE`
4. **Environment variables**: Verify `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are set

#### "AccessDenied" or "UnauthorizedOperation" Errors

**Solutions:**

1. **Check IAM permissions**: Ensure your credentials have `bedrock:InvokeModel` permission
2. **Model access**: Enable model access in the AWS Bedrock console
3. **Region mismatch**: Verify the region in your config matches where you enabled model access

#### SSO-Specific Issues

**"SSO session has expired":**

```bash
aws sso login --profile YOUR_PROFILE
```

**"Profile not found":**

- Check `~/.aws/config` contains the profile
- Verify profile name matches exactly (case-sensitive)

#### Debugging Authentication

Enable debug logging to see which credentials are being used:

```bash
export AWS_SDK_JS_LOG=1
npx promptfoo eval
```

This will show detailed AWS SDK logs including credential resolution.

### Model Configuration Issues

#### Inference profile requires inferenceModelType

If you see this error when using an inference profile ARN:

```text
Error: Inference profile requires inferenceModelType to be specified in config. Options: claude, nova, nova2, llama (defaults to v4), llama2, llama3, llama3.1, llama3.2, llama3.3, llama4, mistral, cohere, ai21, titan, deepseek, openai, qwen, zai, minimax, moonshot, nvidia, writer, gemma
```

This means you're using an application inference profile ARN but haven't specified which model family it's configured for. Add the `inferenceModelType` to your configuration:

```yaml
providers:
  # Incorrect - missing inferenceModelType
  - id: bedrock:arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-profile

  # Correct - includes inferenceModelType
  - id: bedrock:arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-profile
    config:
      inferenceModelType: 'claude' # Specify the model family
```

#### ValidationException: On-demand throughput isn't supported

If you see this error:

```text
ValidationException: Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn't supported. Retry your request with the ID or ARN of an inference profile that contains this model.
```

This usually means you need to use the region-specific model ID. Update your provider configuration to include the regional prefix:

```yaml
providers:
  # Instead of this:
  - id: bedrock:anthropic.claude-sonnet-4-5-20250929-v1:0
  # Use this:
  - id: bedrock:us.anthropic.claude-sonnet-4-5-20250929-v1:0 # US region
  # or
  - id: bedrock:eu.anthropic.claude-sonnet-4-5-20250929-v1:0 # EU region
  # or
  - id: bedrock:apac.anthropic.claude-sonnet-4-5-20250929-v1:0 # APAC region
```

Make sure to:

1. Choose the correct regional prefix (`us.`, `eu.`, or `apac.`) based on your AWS region
2. Configure the corresponding region in your provider config
3. Ensure you have model access enabled in your AWS Bedrock console for that region

### AccessDeniedException: You don't have access to the model with the specified model ID

If you see this error, the cause depends on which model provider you're using:

**For models without provider-specific access steps**:

- Check the [AWS supported models documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html)
  for the current access flow
- Verify your IAM permissions include `bedrock:InvokeModel`
- Check your region configuration matches the model's region

**For Anthropic models (Claude)**:

- First-time use may require submitting use case details in the Bedrock console
- Check the AWS documentation for the current access flow

**For AWS Marketplace models**:

- Ensure your IAM permissions include `aws-marketplace:Subscribe`
- Subscribe to the model through AWS Marketplace

## Knowledge Base

AWS Bedrock Knowledge Bases provide Retrieval Augmented Generation (RAG) functionality, allowing you to query a knowledge base with natural language and get responses based on your data.

### Prerequisites

To use the Knowledge Base provider, you need:

1. An existing Knowledge Base created in AWS Bedrock
2. Install the `@aws-sdk/client-bedrock-agent-runtime` package:

   ```sh
   npm install @aws-sdk/client-bedrock-agent-runtime
   ```

### Configuration

Configure the Knowledge Base provider by specifying `kb` in your provider ID. Note that the model ID needs to include the regional prefix (`us.`, `eu.`, or `apac.`):

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:kb:us.anthropic.claude-3-7-sonnet-20250219-v1:0
    config:
      region: 'us-east-2'
      knowledgeBaseId: 'YOUR_KNOWLEDGE_BASE_ID'
      temperature: 0.0
      max_tokens: 1000
      numberOfResults: 5 # Optional: number of chunks to retrieve (AWS default when not specified)
```

The provider ID follows this pattern: `bedrock:kb:[REGIONAL_MODEL_ID]`

For example:

- `bedrock:kb:us.anthropic.claude-3-5-sonnet-20241022-v2:0` (US region)
- `bedrock:kb:eu.anthropic.claude-3-5-sonnet-20241022-v2:0` (EU region)

Configuration options include:

- `knowledgeBaseId` (required): The ID of your AWS Bedrock Knowledge Base
- `region`: AWS region where your Knowledge Base is deployed (e.g., 'us-east-1', 'us-east-2', 'eu-west-1')
- `temperature`: Controls randomness in response generation (default: 0.0)
- `max_tokens`: Maximum number of tokens in the generated response
- `numberOfResults`: Number of chunks to retrieve from the knowledge base (optional, uses AWS default when not specified)
- `accessKeyId`, `secretAccessKey`, `sessionToken`: AWS credentials (if not using environment variables or IAM roles)
- `profile`: AWS profile name for SSO authentication

### Knowledge Base Example

Here's a complete example to test your Knowledge Base with a few questions:

```yaml title="promptfooconfig.yaml"
prompts:
  - 'What is the capital of France?'
  - 'Tell me about quantum computing.'

providers:
  - id: bedrock:kb:us.anthropic.claude-3-7-sonnet-20250219-v1:0
    config:
      region: 'us-east-2'
      knowledgeBaseId: 'YOUR_KNOWLEDGE_BASE_ID'
      temperature: 0.0
      max_tokens: 1000
      numberOfResults: 10

  # Regular Claude model for comparison
  - id: bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0
    config:
      region: 'us-east-2'
      temperature: 0.0
      max_tokens: 1000

tests:
  - description: 'Basic factual questions from the knowledge base'
```

### Citations

The Knowledge Base provider returns both the generated response and citations from the source documents. These citations are included in the eval results and can be used to verify the accuracy of the responses.

:::info

When viewing eval results in the UI, citations appear in a separate section within the details view of each response. You can click on the source links to visit the original documents or copy citation content for reference.

:::

### Response Format

When using the Knowledge Base provider, the response will include:

1. **output**: The text response generated by the model based on your query
2. **metadata.citations**: An array of citations that includes:
   - `retrievedReferences`: References to source documents that informed the response
   - `generatedResponsePart`: Parts of the response that correspond to specific citations

### Context Evaluation with contextTransform

The Knowledge Base provider supports extracting context from citations for evaluation using the `contextTransform` feature:

```yaml title="promptfooconfig.yaml"
tests:
  - vars:
      query: 'What is promptfoo?'
    assert:
      # Extract context from all citations
      - type: context-faithfulness
        contextTransform: |
          if (!metadata?.citations) return '';
          return metadata.citations
            .flatMap(citation => citation.retrievedReferences || [])
            .map(ref => ref.content?.text || '')
            .filter(text => text.length > 0)
            .join('\n\n');
        threshold: 0.7

      # Extract context from first citation only
      - type: context-relevance
        contextTransform: 'metadata?.citations?.[0]?.retrievedReferences?.[0]?.content?.text || ""'
        threshold: 0.6
```

This approach allows you to:

- **Evaluate real retrieval**: Test against the actual context retrieved by your Knowledge Base
- **Measure faithfulness**: Verify responses don't hallucinate beyond the retrieved content
- **Assess relevance**: Check if retrieved context is relevant to the query
- **Validate recall**: Ensure important information appears in retrieved context

See the [Knowledge Base contextTransform example](https://github.com/promptfoo/promptfoo/tree/main/examples/amazon-bedrock) for complete configuration examples.

## Bedrock Agents

Amazon Bedrock Agents uses the reasoning of foundation models (FMs), APIs, and data to break down user requests, gathers relevant information, and efficiently completes tasks—freeing teams to focus on high-value work. For detailed information on testing and evaluating deployed agents, see the [AWS Bedrock Agents Provider](./bedrock-agents.md) documentation.

Quick example:

```yaml
providers:
  - id: bedrock-agent:YOUR_AGENT_ID
    config:
      agentAliasId: PROD_ALIAS
      region: us-east-1
      enableTrace: true
```

## Video Generation

AWS Bedrock supports video generation through asynchronous invoke APIs. Videos are generated in the cloud and output to an S3 bucket that you specify.

### Luma Ray 2

Generate videos using Luma Ray 2, which produces high-quality videos from text prompts or images.

**Provider ID:** `bedrock:video:luma.ray-v2:0`

Check the [AWS supported models documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html)
for current Luma Ray regional availability.

#### Basic Configuration

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:video:luma.ray-v2:0
    config:
      region: us-west-2
      s3OutputUri: s3://my-bucket/luma-outputs/
```

#### Configuration Options

| Option           | Type    | Default | Description                                |
| ---------------- | ------- | ------- | ------------------------------------------ |
| `s3OutputUri`    | string  | -       | **Required.** S3 bucket for video output   |
| `duration`       | string  | "5s"    | Video duration: "5s" or "9s"               |
| `resolution`     | string  | "720p"  | Output resolution: "540p" or "720p"        |
| `aspectRatio`    | string  | "16:9"  | Aspect ratio (see supported ratios below)  |
| `loop`           | boolean | false   | Whether video should seamlessly loop       |
| `startImage`     | string  | -       | Start frame image (file:// path or base64) |
| `endImage`       | string  | -       | End frame image (file:// path or base64)   |
| `pollIntervalMs` | number  | 10000   | Polling interval in milliseconds           |
| `maxPollTimeMs`  | number  | 600000  | Maximum wait time (10 min default)         |
| `downloadFromS3` | boolean | true    | Download video from S3 after generation    |

#### Supported Aspect Ratios

- `1:1` - Square
- `16:9` - Widescreen (default)
- `9:16` - Vertical/Portrait
- `4:3` - Standard
- `3:4` - Portrait standard
- `21:9` - Ultrawide
- `9:21` - Ultra-tall

#### Text-to-Video Example

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:video:luma.ray-v2:0
    config:
      region: us-west-2
      s3OutputUri: s3://my-bucket/videos/
      duration: '5s'
      resolution: '720p'
      aspectRatio: '16:9'

prompts:
  - 'A majestic eagle soaring through clouds at golden hour'

tests:
  - vars: {}
```

#### Image-to-Video Example

Animate images by providing start and/or end frames:

```yaml title="promptfooconfig.yaml"
providers:
  - id: bedrock:video:luma.ray-v2:0
    config:
      region: us-west-2
      s3OutputUri: s3://my-bucket/videos/
      startImage: file://./start-frame.jpg
      endImage: file://./end-frame.jpg
      duration: '5s'

prompts:
  - 'Smooth transition with camera movement'

tests:
  - vars: {}
```

#### Processing Time

- **5-second videos:** 2-5 minutes
- **9-second videos:** 4-8 minutes

#### Required Permissions

Your AWS credentials need these IAM permissions:

```json
{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:GetAsyncInvoke", "bedrock:StartAsyncInvoke"],
      "Resource": "arn:aws:bedrock:*:*:model/luma.ray-v2:0"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject"],
      "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}
```

## See Also

- [Amazon SageMaker Provider](./sagemaker.md) - For custom-deployed or fine-tuned models on AWS
- [RAG Evaluation Guide](../guides/evaluate-rag.md) - Complete guide to evaluating RAG systems with context-based assertions
- [Context-based Assertions](../configuration/expected-outputs/model-graded/index.md) - Documentation on context-faithfulness, context-relevance, and context-recall
- [Configuration Reference](../configuration/reference.md) - Complete configuration options including contextTransform
- [Command Line Interface](../usage/command-line.md) - How to use promptfoo from the command line
- [Provider Options](../providers/index.md) - Overview of all supported providers
- [Amazon Bedrock Examples](https://github.com/promptfoo/promptfoo/tree/main/examples/amazon-bedrock) - Runnable examples of Bedrock integration, including Knowledge Base and contextTransform examples