chore: import upstream snapshot with attribution
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:24:08 +08:00
commit 0d3cb498a3
5438 changed files with 1316560 additions and 0 deletions
+109
View File
@@ -0,0 +1,109 @@
# provider-cerebras (Cerebras Example (High-Performance LLM Inference))
This example demonstrates how to use the Cerebras provider with promptfoo to evaluate Cerebras Inference API models, which offer high-performance inference for Llama and other LLM models.
You can run this example with:
```bash
npx promptfoo@latest init --example provider-cerebras
cd provider-cerebras
```
## Prerequisites
### API Key Setup
1. Sign up for an account at [Cerebras AI](https://console.cerebras.ai/)
2. Navigate to your account settings to generate an API key
3. Set your Cerebras API key as an environment variable:
```bash
export CEREBRAS_API_KEY="your-api-key-here"
```
Alternatively, you can add it to your `.env` file:
```text
CEREBRAS_API_KEY=your-api-key-here
```
## Example Configurations
This repository contains three example configurations demonstrating different Cerebras features:
### 1. Basic Model Evaluation (`promptfooconfig.yaml`)
This configuration evaluates two Cerebras models on their ability to explain complex concepts in simple terms.
```bash
promptfoo eval
```
**Expected output:** You'll see a comparison of how each model explains concepts from different domains, with metrics on clarity, accuracy, and response time.
### 2. Structured Outputs (`promptfooconfig-structured.yaml`)
The structured output example demonstrates Cerebras's JSON schema enforcement capabilities, ensuring the model returns consistent, structured recipe data with proper types and required fields.
```bash
promptfoo eval -c promptfooconfig-structured.yaml
```
**Expected output:** You'll receive structured JSON outputs for different recipes, with consistent fields like cuisine type, difficulty level, ingredients, and cooking instructions - all following the defined schema.
Example output:
```json
{
"name": "Traditional Pasta Carbonara",
"cuisine": "Italian",
"difficulty": "medium",
"prepTime": 15,
"cookTime": 20,
"ingredients": [
{ "name": "spaghetti", "amount": "400g" },
{ "name": "pancetta", "amount": "150g" },
{ "name": "eggs", "amount": "3 large" },
{ "name": "parmesan cheese", "amount": "50g" }
],
"instructions": [
"Bring a large pot of salted water to boil",
"Cook spaghetti according to package instructions",
"In a separate pan, cook pancetta until crispy",
"In a bowl, whisk eggs and grated parmesan cheese",
"Drain pasta, reserving some pasta water",
"Toss hot pasta with pancetta, then quickly mix in egg mixture",
"Add pasta water as needed to create a silky sauce"
]
}
```
### 3. Tool Use (`promptfooconfig-tools.yaml`)
The tool use example demonstrates Cerebras's function calling capabilities with a calculator tool that the model can use to solve math problems.
```bash
promptfoo eval -c promptfooconfig-tools.yaml
```
**Expected output:** The model will use the calculator tool to solve math problems and provide step-by-step explanations of the solution process. For example, when given "15 × 7", it will calculate 105 and explain multiplication concepts.
## Model Capabilities
Cerebras supports several powerful models:
- `llama-4-scout-17b-16e-instruct` - Llama 4 Scout 17B model with 16 expert MoE (featured in examples)
- `llama3.1-8b` - Llama 3.1 8B model
- `llama-3.3-70b` - Llama 3.3 70B model
- `deepSeek-r1-distill-llama-70B` (private preview)
## Pricing & Usage
Cerebras Inference API offers competitive pricing compared to other inference services. Check the [official pricing page](https://docs.cerebras.ai) for the most current rates. Usage is billed based on input and output tokens.
## Learn More
- [Cerebras Provider Documentation](https://promptfoo.dev/docs/providers/cerebras)
- [Cerebras API Reference](https://docs.cerebras.ai/)
- [Cerebras Structured Outputs Guide](https://docs.cerebras.ai/capabilities/structured-outputs/)
- [Cerebras Tool Use Guide](https://docs.cerebras.ai/capabilities/tool-use/)
@@ -0,0 +1,68 @@
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: Schema-enforced JSON recipe generation with Llama 4
prompts:
- file://structured_prompts.txt
providers:
- id: cerebras:llama-4-scout-17b-16e-instruct
config:
temperature: 0.7
max_completion_tokens: 1024
response_format:
type: 'json_schema'
json_schema:
name: 'recipe_schema'
strict: true
schema:
type: 'object'
properties:
name: { 'type': 'string' }
cuisine: { 'type': 'string' }
difficulty:
type: 'string'
enum: ['easy', 'medium', 'hard']
prepTime: { 'type': 'integer' }
cookTime: { 'type': 'integer' }
ingredients:
type: 'array'
items:
type: 'object'
properties:
name: { 'type': 'string' }
amount: { 'type': 'string' }
required: ['name', 'amount']
instructions:
type: 'array'
items: { 'type': 'string' }
required: ['name', 'cuisine', 'difficulty', 'ingredients', 'instructions']
additionalProperties: false
tests:
- vars:
cuisine: Italian
dish: Cacio e Pepe
assert:
- type: json-schema-valid
value: |
{
"type": "object",
"required": ["name", "cuisine", "difficulty", "ingredients", "instructions"]
}
- vars:
cuisine: Japanese
dish: Okonomiyaki
assert:
- type: contains-json-property
value: ['ingredients', 'instructions']
- type: javascript
value: |
return output.ingredients.length >= 5 ? 'pass' : 'fail: needs at least 5 ingredients'
- vars:
cuisine: Mexican
dish: Chiles en Nogada
assert:
- type: contains-json-property
value: ['difficulty']
- type: javascript
value: |
return output.instructions.length >= 3 ? 'pass' : 'fail: needs detailed instructions'
outputs:
- type: json
@@ -0,0 +1,49 @@
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: Function calling with Llama 4 for mathematical problem solving
prompts:
- file://tool_prompts.txt
providers:
- id: cerebras:llama-4-scout-17b-16e-instruct
config:
temperature: 0.7
max_completion_tokens: 1024
tools:
- type: 'function'
function:
name: 'calculate'
description: 'A calculator that can perform basic arithmetic operations'
parameters:
type: 'object'
properties:
expression:
type: 'string'
description: 'The mathematical expression to evaluate'
required: ['expression']
strict: true
tests:
- vars:
problem: '15 × 7'
explanation: 'the product of fifteen and seven'
assert:
- type: contains
value: '105'
- type: contains-any
value: ['multiply', 'multiplication', 'multiplying', 'product']
- vars:
problem: '(42 × 3) ÷ 6'
explanation: 'forty-two times three, divided by six'
assert:
- type: contains
value: '21'
- type: contains-any
value: ['order of operations', 'PEMDAS', 'parentheses', 'brackets']
- vars:
problem: '√(144) + 25²'
explanation: 'the square root of one hundred forty-four plus twenty-five squared'
assert:
- type: contains
value: '637'
- type: contains-any
value: ['square root', 'exponent', 'power', 'squared']
outputs:
- type: text
@@ -0,0 +1,42 @@
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: Evaluating Llama models for educational concept explanations
prompts:
- file://prompts.txt
providers:
- id: cerebras:llama-4-scout-17b-16e-instruct
config:
temperature: 0.7
max_completion_tokens: 1024
- id: cerebras:llama-3.3-70b
config:
temperature: 0.5
max_completion_tokens: 1024
tests:
- vars:
topic: quantum computing
concept: quantum entanglement
assert:
- type: contains-any
value: ['particles', 'correlation', 'distance', 'spooky action', 'Einstein']
- vars:
topic: artificial intelligence
concept: transformer architecture
assert:
- type: contains-any
value:
['attention mechanism', 'self-attention', 'parallel processing', 'encoder', 'decoder']
- vars:
topic: astrophysics
concept: black hole information paradox
assert:
- type: contains-any
value:
[
'Hawking radiation',
'event horizon',
'quantum mechanics',
'information loss',
'firewall',
]
outputs:
- type: csv
+5
View File
@@ -0,0 +1,5 @@
You are an expert in {{topic}} with exceptional teaching skills. Your task is to explain the concept of {{concept}} in simple terms that a high school student could understand.
Keep your explanation under 300 words and include 2-3 real-world examples or analogies that make this concept intuitive and memorable. Avoid overly technical jargon, and when you must use specialized terms, define them clearly.
Begin with a captivating introduction that sparks curiosity, follow with the main explanation using everyday language, and conclude with why understanding this concept matters in both academic and practical contexts.
@@ -0,0 +1,7 @@
You are a professional chef with expertise in various cuisines.
Create a {{cuisine}} recipe for {{dish}}.
Include a name for the recipe, the cuisine type, difficulty level (easy, medium, or hard), preparation time in minutes, cooking time in minutes, a list of ingredients with amounts, and step-by-step cooking instructions.
Your response should be structured as valid JSON.
@@ -0,0 +1,7 @@
You are a helpful math tutor with access to a calculator tool.
Calculate {{problem}} ({{explanation}}).
First, use your calculator tool to solve the problem. Then, briefly explain how to solve this type of problem step by step, as if you were teaching a student.
Make sure to show your final answer and check if it's correct.