promptfoo--promptfoo/site/docs/providers/cloudflare-gateway.md

---
sidebar_label: Cloudflare AI Gateway
sidebar_position: 47
description: Route AI requests through Cloudflare AI Gateway for caching, rate limiting, and analytics.
---

# Cloudflare AI Gateway

[Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/) is a proxy service that routes requests to AI providers through Cloudflare's infrastructure. It provides:

- **Caching** - Reduce costs by caching identical requests
- **Rate limiting** - Control request rates to avoid quota issues
- **Analytics** - Track usage and costs across providers
- **Logging** - Monitor requests and responses
- **Fallback** - Configure fallback providers for reliability

The `cloudflare-gateway` provider lets you route your promptfoo evaluations through Cloudflare AI Gateway to any supported AI provider.

## Provider Format

```
cloudflare-gateway:{provider}:{model}
```

**Examples:**

- `cloudflare-gateway:openai:gpt-5.2`
- `cloudflare-gateway:anthropic:claude-sonnet-4-5-20250929`
- `cloudflare-gateway:groq:llama-3.3-70b-versatile`

## Required Configuration

Set your Cloudflare account ID and gateway ID:

```sh
export CLOUDFLARE_ACCOUNT_ID=your_account_id_here
export CLOUDFLARE_GATEWAY_ID=your_gateway_id_here
```

### Provider API Keys

You need API keys for the providers you're routing through:

```sh
# For OpenAI
export OPENAI_API_KEY=your_openai_key

# For Anthropic
export ANTHROPIC_API_KEY=your_anthropic_key

# For Groq
export GROQ_API_KEY=your_groq_key
```

### Using BYOK (Bring Your Own Keys)

If you've configured [BYOK in Cloudflare](https://developers.cloudflare.com/ai-gateway/configuration/byok/), you can omit provider API keys entirely. Cloudflare will use the keys stored in your gateway configuration.

```yaml
providers:
  # No OPENAI_API_KEY needed - Cloudflare uses stored key
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      cfAigToken: '{{env.CF_AIG_TOKEN}}'
```

:::note
BYOK works best with OpenAI-compatible providers. Anthropic requires an API key because the SDK mandates it.
:::

### Authenticated Gateways

If your gateway has [Authenticated Gateway](https://developers.cloudflare.com/ai-gateway/configuration/authenticated-gateway/) enabled, you must provide the `cfAigToken`:

```sh
export CF_AIG_TOKEN=your_gateway_token_here
```

## Basic Usage

```yaml title="promptfooconfig.yaml"
prompts:
  - 'Answer this question: {{question}}'

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      temperature: 0.7

tests:
  - vars:
      question: What is the capital of France?
```

## Supported Providers

Cloudflare AI Gateway supports routing to these providers:

| Provider         | Gateway Name       | API Key Environment Variable |
| ---------------- | ------------------ | ---------------------------- |
| OpenAI           | `openai`           | `OPENAI_API_KEY`             |
| Anthropic        | `anthropic`        | `ANTHROPIC_API_KEY`          |
| Groq             | `groq`             | `GROQ_API_KEY`               |
| Perplexity       | `perplexity-ai`    | `PERPLEXITY_API_KEY`         |
| Google AI Studio | `google-ai-studio` | `GOOGLE_API_KEY`             |
| Mistral          | `mistral`          | `MISTRAL_API_KEY`            |
| Cohere           | `cohere`           | `COHERE_API_KEY`             |
| Azure OpenAI     | `azure-openai`     | `AZURE_OPENAI_API_KEY`       |
| Workers AI       | `workers-ai`       | `CLOUDFLARE_API_KEY`         |
| Hugging Face     | `huggingface`      | `HUGGINGFACE_API_KEY`        |
| Replicate        | `replicate`        | `REPLICATE_API_KEY`          |
| Grok (xAI)       | `grok`             | `XAI_API_KEY`                |

:::note
AWS Bedrock is not supported through Cloudflare AI Gateway because it requires AWS request signing, which is incompatible with the gateway proxy approach.
:::

## Configuration Options

### Gateway Configuration

| Option            | Type   | Description                                                                   |
| ----------------- | ------ | ----------------------------------------------------------------------------- |
| `accountId`       | string | Cloudflare account ID                                                         |
| `accountIdEnvar`  | string | Custom environment variable for account ID (default: `CLOUDFLARE_ACCOUNT_ID`) |
| `gatewayId`       | string | AI Gateway ID                                                                 |
| `gatewayIdEnvar`  | string | Custom environment variable for gateway ID (default: `CLOUDFLARE_GATEWAY_ID`) |
| `cfAigToken`      | string | Optional gateway authentication token                                         |
| `cfAigTokenEnvar` | string | Custom environment variable for gateway token (default: `CF_AIG_TOKEN`)       |

### Azure OpenAI Configuration

Azure OpenAI requires additional configuration:

| Option           | Type   | Description                                       |
| ---------------- | ------ | ------------------------------------------------- |
| `resourceName`   | string | Azure OpenAI resource name (required)             |
| `deploymentName` | string | Azure OpenAI deployment name (required)           |
| `apiVersion`     | string | Azure API version (default: `2024-12-01-preview`) |

```yaml
providers:
  - id: cloudflare-gateway:azure-openai:gpt-4
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      resourceName: my-azure-resource
      deploymentName: my-gpt4-deployment
      apiVersion: 2024-12-01-preview
```

### Workers AI Configuration

Workers AI routes requests to Cloudflare's edge-deployed models. The model name is included in the URL path:

```yaml
providers:
  - id: cloudflare-gateway:workers-ai:@cf/meta/llama-3.1-8b-instruct
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
```

### Provider-Specific Options

All options from the underlying provider are supported. For example, when using `cloudflare-gateway:openai:gpt-5.2`, you can use any [OpenAI provider options](/docs/providers/openai).

```yaml
providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      temperature: 0.8
      max_tokens: 1000
      top_p: 0.9
```

## Examples

### Multiple Providers

Compare responses from different providers, all routed through your Cloudflare gateway:

```yaml title="promptfooconfig.yaml"
prompts:
  - 'Explain {{topic}} in simple terms.'

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

  - id: cloudflare-gateway:anthropic:claude-sonnet-4-5-20250929
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

  - id: cloudflare-gateway:groq:llama-3.3-70b-versatile
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

tests:
  - vars:
      topic: quantum computing
```

### Authenticated Gateway

If your AI Gateway requires authentication:

```yaml
providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      cfAigToken: '{{env.CF_AIG_TOKEN}}'
```

### Custom Environment Variables

Use custom environment variable names for different projects or environments:

```yaml
providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountIdEnvar: MY_CF_ACCOUNT
      gatewayIdEnvar: MY_CF_GATEWAY
      apiKeyEnvar: MY_OPENAI_KEY
```

## Gateway URL Structure

The provider constructs the gateway URL in this format:

```
https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}
```

For example, with `accountId: abc123` and `gatewayId: my-gateway`, requests to OpenAI would be routed through:

```
https://gateway.ai.cloudflare.com/v1/abc123/my-gateway/openai
```

## Benefits of Using AI Gateway

### Cost Reduction Through Caching

AI Gateway can cache identical requests, reducing costs when you run the same prompts multiple times (common during development and testing).

### Unified Analytics

View usage across all your AI providers in a single Cloudflare dashboard, making it easier to track costs and usage patterns.

### Rate Limit Protection

AI Gateway can help manage rate limits by queuing requests, preventing your evaluations from failing due to provider rate limits.

### Logging and Debugging

All requests and responses are logged in Cloudflare, making it easier to debug issues and audit AI usage.

## See Also

- [Cloudflare AI Gateway Documentation](https://developers.cloudflare.com/ai-gateway/)
- [Cloudflare Workers AI Provider](/docs/providers/cloudflare-ai) - For running models directly on Cloudflare's edge
- [OpenAI Provider](/docs/providers/openai)
- [Anthropic Provider](/docs/providers/anthropic)