chore: import upstream snapshot with attribution
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:24:08 +08:00
commit 0d3cb498a3
5438 changed files with 1316560 additions and 0 deletions
+57
View File
@@ -0,0 +1,57 @@
# azure/comparison (Azure Model Comparison)
This example demonstrates how to compare models from different providers on Azure AI Foundry, including OpenAI, Anthropic Claude, Meta Llama, and Mistral.
You can run this example with:
```bash
npx promptfoo@latest init --example azure/comparison
cd azure/comparison
```
## Setup
1. Deploy models from different providers in Azure AI Foundry
2. Set your environment variables:
```bash
export AZURE_API_KEY=your-api-key
# Set apiHost in promptfooconfig.yaml for each provider's deployment
```
## Models Compared
| Provider | Model | Label |
| --------- | ---------------------------------------- | ------------- |
| OpenAI | `gpt-5.1` | gpt-5.1 |
| Anthropic | `claude-sonnet-4-6` | claude-sonnet |
| Meta | `Llama-4-Maverick-17B-128E-Instruct-FP8` | llama-4 |
| Mistral | `Mistral-Large-2411` | mistral-large |
## Running the Example
```bash
npx promptfoo@latest eval
npx promptfoo@latest view
```
## Customization
Modify `promptfooconfig.yaml` to:
- Add or remove models
- Change test questions
- Adjust evaluation criteria
- Compare cost vs performance
## Use Cases
- Benchmark different models on your specific tasks
- Evaluate cost-effectiveness across providers
- Find the best model for your use case
- A/B test model updates
## Documentation
- [Azure Provider Documentation](https://promptfoo.dev/docs/providers/azure/)
- [Azure AI Foundry](https://azure.microsoft.com/en-us/products/ai-services/ai-foundry/)
@@ -0,0 +1,77 @@
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: Azure model comparison
prompts:
- |
You are a helpful assistant. Answer the following question concisely and accurately.
Question: {{question}}
providers:
# OpenAI GPT-5.1 (Latest flagship)
- id: azure:chat:gpt-5.1
label: gpt-5.1
config:
apiHost: 'your-resource.openai.azure.com'
max_tokens: 1024
temperature: 0.7
# Anthropic Claude Sonnet 4.6
- id: azure:chat:claude-sonnet-4-6
label: claude-sonnet
config:
apiHost: 'your-deployment.services.ai.azure.com'
apiVersion: '2025-04-01-preview'
max_tokens: 1024
temperature: 0.7
# Meta Llama 4 Maverick
- id: azure:chat:Llama-4-Maverick-17B-128E-Instruct-FP8
label: llama-4
config:
apiHost: 'your-deployment.services.ai.azure.com'
apiVersion: '2025-04-01-preview'
max_tokens: 1024
temperature: 0.7
# Mistral Large
- id: azure:chat:Mistral-Large-2411
label: mistral-large
config:
apiHost: 'your-deployment.services.ai.azure.com'
apiVersion: '2025-04-01-preview'
max_tokens: 1024
temperature: 0.7
tests:
- vars:
question: What are the main differences between machine learning and deep learning?
assert:
- type: llm-rubric
value: Accurately explains the relationship and differences
- type: contains-any
value: ['neural network', 'subset', 'feature']
- vars:
question: Explain the concept of recursion in programming with a simple example.
assert:
- type: llm-rubric
value: Provides a clear explanation with a working example
- type: contains-any
value: ['base case', 'function', 'itself']
- vars:
question: What is the difference between REST and GraphQL APIs?
assert:
- type: llm-rubric
value: Accurately compares both API paradigms
- type: contains-any
value: ['endpoint', 'query', 'schema']
- vars:
question: How does HTTPS encryption work?
assert:
- type: llm-rubric
value: Explains the encryption process accurately
- type: contains-any
value: ['TLS', 'SSL', 'certificate', 'handshake']