promptfoo--promptfoo/site/docs/providers/ai21.md

---
sidebar_label: AI21 Labs
description: "Deploy AI21 Labs' Jamba models for enterprise text generation with task-specific optimization and control"
---

# AI21 Labs

The [AI21 Labs API](https://docs.ai21.com/reference/chat-completion) offers access to AI21 models such as `jamba-mini` and `jamba-large`.

## API Key

To use AI21 Labs, you need to set the `AI21_API_KEY` environment variable, or specify the `apiKey` in the provider configuration.

Example of setting the environment variable:

```sh
export AI21_API_KEY=your_api_key_here
```

## Model Selection

You can specify which AI21 model to use in your configuration. The current public aliases are:

1. `jamba-mini`
2. `jamba-large`

The provider also recognizes the versioned IDs `jamba-mini-2`, `jamba-mini-2-2026-01`,
`jamba-large-1.7`, and `jamba-large-1.7-2025-07`.

Here's an example config that compares AI21 models:

```yaml
providers:
  - ai21:jamba-mini
  - ai21:jamba-large
```

## Options

The AI21 provider supports several options to customize the behavior of the model. These include:

- `temperature`: Controls the randomness of the output.
- `top_p`: Controls nucleus sampling, affecting the randomness of the output.
- `max_tokens`: The maximum length of the generated text.
- `response_format`: Set to `{ type: 'json_object' }` for JSON output or `{ type: 'text' }` for text output.
- `apiKeyEnvar`: An environment variable that contains the API key.
- `apiBaseUrl`: The base URL of the AI21 API.

## Example Configuration

Here's an example configuration for the AI21 provider:

```yaml
providers:
  - ai21:jamba-mini
    config:
      apiKey: your_api_key_here
      temperature: 0.1
      top_p: 1
      max_tokens: 1024
      response_format: { type: 'json_object' }
```

This configuration uses the `jamba-mini` model with a temperature of 0.1, top-p sampling
with a value of 1, a maximum output length of 1024 tokens, and JSON-formatted output.

## Cost

The cost of using AI21 models depends on the model and the number of input and output tokens. Here are the costs for the available models:

- `jamba-mini`: $0.2 per 1M input tokens, $0.4 per 1M output tokens
- `jamba-large`: $2 per 1M input tokens, $8 per 1M output tokens

You can override promptfoo's built-in pricing with `inputCost` and `outputCost` in the
provider configuration. The legacy `cost` option still works as a shared fallback when you
want the same rate for both directions.

## Supported environment variables

These AI21-related environment variables are supported:

| Variable            | Description                                                        |
| ------------------- | ------------------------------------------------------------------ |
| `AI21_API_BASE_URL` | The base URL (protocol + hostname + port) to use for the AI21 API. |
| `AI21_API_KEY`      | AI21 API key.                                                      |