wehub/promptfoo--promptfoo
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
CI / Shell Format Check (push) Has been cancelled
Details
CI / Check Ruby (3.4) (push) Has been cancelled
Details
CI / CI Config (push) Has been cancelled
Details
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
Details
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
Details
CI / Style Check (push) Has been cancelled
Details
CI / Generate Assets (push) Has been cancelled
Details
CI / Check Python (3.14) (push) Has been cancelled
Details
CI / Check Python (3.9) (push) Has been cancelled
Details
CI / Build Docs (push) Has been cancelled
Details
CI / Code Scan Action (push) Has been cancelled
Details
CI / Site tests (push) Has been cancelled
Details
CI / webui tests (push) Has been cancelled
Details
CI / Run Integration Tests (push) Has been cancelled
Details
CI / Run Smoke Tests (push) Has been cancelled
Details
CI / Go Tests (push) Has been cancelled
Details
CI / Share Test (push) Has been cancelled
Details
CI / Redteam (Production API) (push) Has been cancelled
Details
CI / Redteam (Staging API) (push) Has been cancelled
Details
CI / GitHub Actions Lint (push) Has been cancelled
Details
CI / Check Ruby (3.0) (push) Has been cancelled
Details
release-please / release-please (push) Has been cancelled
Details
release-please / build (push) Has been cancelled
Details
release-please / publish-npm (push) Has been cancelled
Details
release-please / publish-npm-backfill (push) Has been cancelled
Details
release-please / docker (push) Has been cancelled
Details
release-please / publish-code-scan-action (push) Has been cancelled
Details
release-please / attest-code-scan-action (push) Has been cancelled
Details
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Details
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Details
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Details
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Details
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Details
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Details
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:24:08 +08:00
commit 0d3cb498a3
5438 changed files with 1316560 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/simple-cli/README.md
+22
View File
@@ -0,0 +1,22 @@
# simple-cli (Simple Cli)
You can run this example with:
```bash
npx promptfoo@latest init --example simple-cli
cd simple-cli
```
## Usage
This example is pre-configured in `promptfooconfig.yaml`. That means you can just run:
```bash
promptfoo eval
```
To override prompts, providers, output, etc. you can run:
```bash
promptfoo eval --prompts prompts.txt --providers openai:chat --output output.json
```
examples/simple-cli/output.csv
+37
View File
@@ -0,0 +1,37 @@
body,language,[openai:chat:gpt-5.4-mini] Rephrase this in {{language}}: {{body}},[openai:chat:gpt-5.4-mini] Translate this to conversational {{language}}: {{body}}
Hello world,French,"[PASS] (1.00)
Bonjour le monde.
Pass Reason: No assertions","[PASS] (1.00)
Salut tout le monde !
Pass Reason: No assertions"
I'm hungry,French,"[PASS] (1.00)
J'ai faim.
Pass Reason: No assertions","[PASS] (1.00)
J'ai faim.
Pass Reason: No assertions"
Hello world,Pirate,"[PASS] (1.00)
Ahoy, me hearties!
Pass Reason: No assertions","[PASS] (1.00)
Ahoy, me hearty! Greetings to ye, world!
Pass Reason: No assertions"
I'm hungry,Pirate,"[PASS] (1.00)
Arrr, me belly be grumblin'!
Pass Reason: No assertions","[PASS] (1.00)
Arrr, me belly be rumblin'! I be feelin' mighty hungry, matey!
Pass Reason: No assertions"
1 body language [openai:chat:gpt-5.4-mini] Rephrase this in {{language}}: {{body}} [openai:chat:gpt-5.4-mini] Translate this to conversational {{language}}: {{body}}
2 Hello world French [PASS] (1.00) Bonjour le monde. Pass Reason: No assertions [PASS] (1.00) Salut tout le monde ! Pass Reason: No assertions
3 I'm hungry French [PASS] (1.00) J'ai faim. Pass Reason: No assertions [PASS] (1.00) J'ai faim. Pass Reason: No assertions
4 Hello world Pirate [PASS] (1.00) Ahoy, me hearties! Pass Reason: No assertions [PASS] (1.00) Ahoy, me hearty! Greetings to ye, world! Pass Reason: No assertions
5 I'm hungry Pirate [PASS] (1.00) Arrr, me belly be grumblin'! Pass Reason: No assertions [PASS] (1.00) Arrr, me belly be rumblin'! I be feelin' mighty hungry, matey! Pass Reason: No assertions
examples/simple-cli/output.html
+161
View File
@@ -0,0 +1,161 @@
<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width" />
<title>Simple test</title>
<style>
body {
font-family:
-apple-system,
BlinkMacSystemFont,
Segoe UI,
Roboto,
Helvetica,
Arial,
sans-serif;
}
table,
th,
td {
border: 1px solid black;
border-collapse: collapse;
text-align: left;
word-break: break-all;
}
th,
td {
padding: 5px;
min-width: 200px;
white-space: pre-line;
vertical-align: top;
}
tr > td[data-content^='[PASS]'] {
color: green;
}
tr > td[data-content^='[FAIL]'] {
color: #ad0000;
}
</style>
</head>
<body>
<table>
<thead>
<th>body</th>
<th>language</th>
<th>[openai:chat:gpt-5.4-mini] Rephrase this in {{language}}: {{body}}</th>
<th>[openai:chat:gpt-5.4-mini] Translate this to conversational {{language}}: {{body}}</th>
</thead>
<tbody>
<tr>
<td data-content="Hello world">Hello world</td>
<td data-content="French">French</td>
<td
data-content="[PASS] (1.00)
Bonjour le monde.
Pass Reason: No assertions"
>
[PASS] (1.00) Bonjour le monde. Pass Reason: No assertions
</td>
<td
data-content="[PASS] (1.00)
Salut tout le monde !
Pass Reason: No assertions"
>
[PASS] (1.00) Salut tout le monde ! Pass Reason: No assertions
</td>
</tr>
<tr>
<td data-content="I&#39;m hungry">I'm hungry</td>
<td data-content="French">French</td>
<td
data-content="[PASS] (1.00)
J&#39;ai faim.
Pass Reason: No assertions"
>
[PASS] (1.00) J'ai faim. Pass Reason: No assertions
</td>
<td
data-content="[PASS] (1.00)
J&#39;ai faim.
Pass Reason: No assertions"
>
[PASS] (1.00) J'ai faim. Pass Reason: No assertions
</td>
</tr>
<tr>
<td data-content="Hello world">Hello world</td>
<td data-content="Pirate">Pirate</td>
<td
data-content="[PASS] (1.00)
Ahoy, me hearties!
Pass Reason: No assertions"
>
[PASS] (1.00) Ahoy, me hearties! Pass Reason: No assertions
</td>
<td
data-content="[PASS] (1.00)
Ahoy, me hearty! Greetings to ye, world!
Pass Reason: No assertions"
>
[PASS] (1.00) Ahoy, me hearty! Greetings to ye, world! Pass Reason: No assertions
</td>
</tr>
<tr>
<td data-content="I&#39;m hungry">I'm hungry</td>
<td data-content="Pirate">Pirate</td>
<td
data-content="[PASS] (1.00)
Arrr, me belly be grumblin&#39;!
Pass Reason: No assertions"
>
[PASS] (1.00) Arrr, me belly be grumblin'! Pass Reason: No assertions
</td>
<td
data-content="[PASS] (1.00)
Arrr, me belly be rumblin&#39;! I be feelin&#39; mighty hungry, matey!
Pass Reason: No assertions"
>
[PASS] (1.00) Arrr, me belly be rumblin'! I be feelin' mighty hungry, matey! Pass
Reason: No assertions
</td>
</tr>
</tbody>
</table>
</body>
</html>
examples/simple-cli/output.json
+529
View File
@@ -0,0 +1,529 @@
{
"evalId": "eval-d8q-2024-10-04T17:33:11",
"results": {
"version": 3,
"timestamp": "2024-10-04T17:33:11.912Z",
"prompts": [
{
"raw": "Rephrase this in {{language}}: {{body}}",
"label": "Rephrase this in {{language}}: {{body}}",
"id": "dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207",
"provider": "openai:chat:gpt-5.4-mini",
"metrics": {
"score": 4,
"testPassCount": 4,
"testFailCount": 0,
"assertPassCount": 0,
"assertFailCount": 0,
"totalLatencyMs": 29,
"tokenUsage": {
"total": 87,
"prompt": 0,
"completion": 0,
"cached": 87
},
"namedScores": {},
"namedScoresCount": {},
"cost": 0.000025199999999999996
}
},
{
"raw": "Translate this to conversational {{language}}: {{body}}",
"label": "Translate this to conversational {{language}}: {{body}}",
"id": "68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c",
"provider": "openai:chat:gpt-5.4-mini",
"metrics": {
"score": 4,
"testPassCount": 4,
"testFailCount": 0,
"assertPassCount": 0,
"assertFailCount": 0,
"totalLatencyMs": 36,
"tokenUsage": {
"total": 103,
"prompt": 0,
"completion": 0,
"cached": 103
},
"namedScores": {},
"namedScoresCount": {},
"cost": 0.0000348
}
}
],
"results": [
{
"cost": 0.0000046499999999999995,
"gradingResult": {
"pass": true,
"score": 1,
"reason": "No assertions",
"tokensUsed": {
"total": 0,
"prompt": 0,
"completion": 0,
"cached": 0
},
"assertion": null
},
"id": "3d8d47d5-c198-42e5-9387-e1677705048e",
"latencyMs": 3,
"namedScores": {},
"prompt": {
"raw": "Rephrase this in French: I'm hungry",
"label": "Rephrase this in {{language}}: {{body}}"
},
"promptId": "dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207",
"promptIdx": 0,
"provider": {
"id": "openai:chat:gpt-5.4-mini",
"label": ""
},
"response": {
"output": "J'ai faim.",
"tokenUsage": {
"cached": 19,
"total": 19
},
"cached": true,
"cost": 0.0000046499999999999995
},
"score": 1,
"success": true,
"testCase": {
"vars": {
"language": "French",
"body": "I'm hungry"
},
"assert": [],
"options": {},
"metadata": {}
},
"testIdx": 1,
"vars": {
"language": "French",
"body": "I'm hungry"
}
},
{
"cost": 0.0000046499999999999995,
"gradingResult": {
"pass": true,
"score": 1,
"reason": "No assertions",
"tokensUsed": {
"total": 0,
"prompt": 0,
"completion": 0,
"cached": 0
},
"assertion": null
},
"id": "94414e60-888a-4f72-b50c-65916dbb9cab",
"latencyMs": 9,
"namedScores": {},
"prompt": {
"raw": "Translate this to conversational French: I'm hungry",
"label": "Translate this to conversational {{language}}: {{body}}"
},
"promptId": "68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c",
"promptIdx": 1,
"provider": {
"id": "openai:chat:gpt-5.4-mini",
"label": ""
},
"response": {
"output": "J'ai faim.",
"tokenUsage": {
"cached": 19,
"total": 19
},
"cached": true,
"cost": 0.0000046499999999999995
},
"score": 1,
"success": true,
"testCase": {
"vars": {
"language": "French",
"body": "I'm hungry"
},
"assert": [],
"options": {},
"metadata": {}
},
"testIdx": 1,
"vars": {
"language": "French",
"body": "I'm hungry"
}
},
{
"cost": 0.0000046499999999999995,
"gradingResult": {
"pass": true,
"score": 1,
"reason": "No assertions",
"tokensUsed": {
"total": 0,
"prompt": 0,
"completion": 0,
"cached": 0
},
"assertion": null
},
"id": "f443fbde-6fe0-4918-8374-2a1806de375e",
"latencyMs": 14,
"namedScores": {},
"prompt": {
"raw": "Rephrase this in French: Hello world",
"label": "Rephrase this in {{language}}: {{body}}"
},
"promptId": "dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207",
"promptIdx": 0,
"provider": {
"id": "openai:chat:gpt-5.4-mini",
"label": ""
},
"response": {
"output": "Bonjour le monde.",
"tokenUsage": {
"cached": 19,
"total": 19
},
"cached": true,
"cost": 0.0000046499999999999995
},
"score": 1,
"success": true,
"testCase": {
"vars": {
"language": "French",
"body": "Hello world"
},
"assert": [],
"options": {},
"metadata": {}
},
"testIdx": 0,
"vars": {
"language": "French",
"body": "Hello world"
}
},
{
"cost": 0.0000052500000000000006,
"gradingResult": {
"pass": true,
"score": 1,
"reason": "No assertions",
"tokensUsed": {
"total": 0,
"prompt": 0,
"completion": 0,
"cached": 0
},
"assertion": null
},
"id": "0492ff53-adeb-4c31-b42b-eff770265c4f",
"latencyMs": 15,
"namedScores": {},
"prompt": {
"raw": "Translate this to conversational French: Hello world",
"label": "Translate this to conversational {{language}}: {{body}}"
},
"promptId": "68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c",
"promptIdx": 1,
"provider": {
"id": "openai:chat:gpt-5.4-mini",
"label": ""
},
"response": {
"output": "Salut tout le monde !",
"tokenUsage": {
"cached": 20,
"total": 20
},
"cached": true,
"cost": 0.0000052500000000000006
},
"score": 1,
"success": true,
"testCase": {
"vars": {
"language": "French",
"body": "Hello world"
},
"assert": [],
"options": {},
"metadata": {}
},
"testIdx": 0,
"vars": {
"language": "French",
"body": "Hello world"
}
},
{
"cost": 0.000006449999999999999,
"gradingResult": {
"pass": true,
"score": 1,
"reason": "No assertions",
"tokensUsed": {
"total": 0,
"prompt": 0,
"completion": 0,
"cached": 0
},
"assertion": null
},
"id": "657a9223-a33d-416d-9a80-a2770fe88d5c",
"latencyMs": 8,
"namedScores": {},
"prompt": {
"raw": "Rephrase this in Pirate: Hello world",
"label": "Rephrase this in {{language}}: {{body}}"
},
"promptId": "dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207",
"promptIdx": 0,
"provider": {
"id": "openai:chat:gpt-5.4-mini",
"label": ""
},
"response": {
"output": "Ahoy, me hearties!",
"tokenUsage": {
"cached": 22,
"total": 22
},
"cached": true,
"cost": 0.000006449999999999999
},
"score": 1,
"success": true,
"testCase": {
"vars": {
"language": "Pirate",
"body": "Hello world"
},
"assert": [],
"options": {},
"metadata": {}
},
"testIdx": 2,
"vars": {
"language": "Pirate",
"body": "Hello world"
}
},
{
"cost": 0.00000945,
"gradingResult": {
"pass": true,
"score": 1,
"reason": "No assertions",
"tokensUsed": {
"total": 0,
"prompt": 0,
"completion": 0,
"cached": 0
},
"assertion": null
},
"id": "a5b11097-af01-4869-aec7-b3ba428aa926",
"latencyMs": 4,
"namedScores": {},
"prompt": {
"raw": "Rephrase this in Pirate: I'm hungry",
"label": "Rephrase this in {{language}}: {{body}}"
},
"promptId": "dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207",
"promptIdx": 0,
"provider": {
"id": "openai:chat:gpt-5.4-mini",
"label": ""
},
"response": {
"output": "Arrr, me belly be grumblin'!",
"tokenUsage": {
"cached": 27,
"total": 27
},
"cached": true,
"cost": 0.00000945
},
"score": 1,
"success": true,
"testCase": {
"vars": {
"language": "Pirate",
"body": "I'm hungry"
},
"assert": [],
"options": {},
"metadata": {}
},
"testIdx": 3,
"vars": {
"language": "Pirate",
"body": "I'm hungry"
}
},
{
"cost": 0.00000945,
"gradingResult": {
"pass": true,
"score": 1,
"reason": "No assertions",
"tokensUsed": {
"total": 0,
"prompt": 0,
"completion": 0,
"cached": 0
},
"assertion": null
},
"id": "5cd8e725-9bf1-4c40-9337-655b8d9d41f8",
"latencyMs": 7,
"namedScores": {},
"prompt": {
"raw": "Translate this to conversational Pirate: Hello world",
"label": "Translate this to conversational {{language}}: {{body}}"
},
"promptId": "68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c",
"promptIdx": 1,
"provider": {
"id": "openai:chat:gpt-5.4-mini",
"label": ""
},
"response": {
"output": "Ahoy, me hearty! Greetings to ye, world!",
"tokenUsage": {
"cached": 27,
"total": 27
},
"cached": true,
"cost": 0.00000945
},
"score": 1,
"success": true,
"testCase": {
"vars": {
"language": "Pirate",
"body": "Hello world"
},
"assert": [],
"options": {},
"metadata": {}
},
"testIdx": 2,
"vars": {
"language": "Pirate",
"body": "Hello world"
}
},
{
"cost": 0.00001545,
"gradingResult": {
"pass": true,
"score": 1,
"reason": "No assertions",
"tokensUsed": {
"total": 0,
"prompt": 0,
"completion": 0,
"cached": 0
},
"assertion": null
},
"id": "df2bb14b-8770-4237-bff0-19d0257703ee",
"latencyMs": 5,
"namedScores": {},
"prompt": {
"raw": "Translate this to conversational Pirate: I'm hungry",
"label": "Translate this to conversational {{language}}: {{body}}"
},
"promptId": "68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c",
"promptIdx": 1,
"provider": {
"id": "openai:chat:gpt-5.4-mini",
"label": ""
},
"response": {
"output": "Arrr, me belly be rumblin'! I be feelin' mighty hungry, matey!",
"tokenUsage": {
"cached": 37,
"total": 37
},
"cached": true,
"cost": 0.00001545
},
"score": 1,
"success": true,
"testCase": {
"vars": {
"language": "Pirate",
"body": "I'm hungry"
},
"assert": [],
"options": {},
"metadata": {}
},
"testIdx": 3,
"vars": {
"language": "Pirate",
"body": "I'm hungry"
}
}
],
"stats": {
"successes": 8,
"failures": 0,
"tokenUsage": {
"cached": 190,
"completion": 0,
"prompt": 0,
"total": 190
}
}
},
"config": {
"description": "Simple test",
"prompts": [
"Rephrase this in {{language}}: {{body}}",
"Translate this to conversational {{language}}: {{body}}"
],
"providers": ["openai:chat:gpt-5.4-mini"],
"tests": [
{
"vars": {
"language": "French",
"body": "Hello world"
}
},
{
"vars": {
"language": "French",
"body": "I'm hungry"
}
},
{
"vars": {
"language": "Pirate",
"body": "Hello world"
}
},
{
"vars": {
"language": "Pirate",
"body": "I'm hungry"
}
}
],
"sharing": true,
"outputPath": ["output.yaml", "output.json"],
"extensions": []
},
"shareableUrl": null
}
examples/simple-cli/output.yaml
+398
View File
@@ -0,0 +1,398 @@
evalId: eval-d8q-2024-10-04T17:33:11
results:
version: 3
timestamp: '2024-10-04T17:33:11.912Z'
prompts:
- raw: 'Rephrase this in {{language}}: {{body}}'
label: 'Rephrase this in {{language}}: {{body}}'
id: dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207
provider: openai:chat:gpt-5.4-mini
metrics:
score: 4
testPassCount: 4
testFailCount: 0
assertPassCount: 0
assertFailCount: 0
totalLatencyMs: 29
tokenUsage:
total: 87
prompt: 0
completion: 0
cached: 87
namedScores: {}
namedScoresCount: {}
cost: 0.000025199999999999996
- raw: 'Translate this to conversational {{language}}: {{body}}'
label: 'Translate this to conversational {{language}}: {{body}}'
id: 68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c
provider: openai:chat:gpt-5.4-mini
metrics:
score: 4
testPassCount: 4
testFailCount: 0
assertPassCount: 0
assertFailCount: 0
totalLatencyMs: 36
tokenUsage:
total: 103
prompt: 0
completion: 0
cached: 103
namedScores: {}
namedScoresCount: {}
cost: 0.0000348
results:
- cost: 0.0000046499999999999995
gradingResult:
pass: true
score: 1
reason: No assertions
tokensUsed:
total: 0
prompt: 0
completion: 0
cached: 0
assertion: null
id: 3d8d47d5-c198-42e5-9387-e1677705048e
latencyMs: 3
namedScores: {}
prompt:
raw: "Rephrase this in French: I'm hungry"
label: 'Rephrase this in {{language}}: {{body}}'
promptId: dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207
promptIdx: 0
provider:
id: openai:chat:gpt-5.4-mini
label: ''
response:
output: J'ai faim.
tokenUsage:
cached: 19
total: 19
cached: true
cost: 0.0000046499999999999995
score: 1
success: true
testCase:
vars: &ref_0
language: French
body: I'm hungry
assert: []
options: {}
metadata: {}
testIdx: 1
vars: *ref_0
- cost: 0.0000046499999999999995
gradingResult:
pass: true
score: 1
reason: No assertions
tokensUsed:
total: 0
prompt: 0
completion: 0
cached: 0
assertion: null
id: 94414e60-888a-4f72-b50c-65916dbb9cab
latencyMs: 9
namedScores: {}
prompt:
raw: "Translate this to conversational French: I'm hungry"
label: 'Translate this to conversational {{language}}: {{body}}'
promptId: 68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c
promptIdx: 1
provider:
id: openai:chat:gpt-5.4-mini
label: ''
response:
output: J'ai faim.
tokenUsage:
cached: 19
total: 19
cached: true
cost: 0.0000046499999999999995
score: 1
success: true
testCase:
vars: &ref_1
language: French
body: I'm hungry
assert: []
options: {}
metadata: {}
testIdx: 1
vars: *ref_1
- cost: 0.0000046499999999999995
gradingResult:
pass: true
score: 1
reason: No assertions
tokensUsed:
total: 0
prompt: 0
completion: 0
cached: 0
assertion: null
id: f443fbde-6fe0-4918-8374-2a1806de375e
latencyMs: 14
namedScores: {}
prompt:
raw: 'Rephrase this in French: Hello world'
label: 'Rephrase this in {{language}}: {{body}}'
promptId: dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207
promptIdx: 0
provider:
id: openai:chat:gpt-5.4-mini
label: ''
response:
output: Bonjour le monde.
tokenUsage:
cached: 19
total: 19
cached: true
cost: 0.0000046499999999999995
score: 1
success: true
testCase:
vars: &ref_2
language: French
body: Hello world
assert: []
options: {}
metadata: {}
testIdx: 0
vars: *ref_2
- cost: 0.0000052500000000000006
gradingResult:
pass: true
score: 1
reason: No assertions
tokensUsed:
total: 0
prompt: 0
completion: 0
cached: 0
assertion: null
id: 0492ff53-adeb-4c31-b42b-eff770265c4f
latencyMs: 15
namedScores: {}
prompt:
raw: 'Translate this to conversational French: Hello world'
label: 'Translate this to conversational {{language}}: {{body}}'
promptId: 68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c
promptIdx: 1
provider:
id: openai:chat:gpt-5.4-mini
label: ''
response:
output: Salut tout le monde !
tokenUsage:
cached: 20
total: 20
cached: true
cost: 0.0000052500000000000006
score: 1
success: true
testCase:
vars: &ref_3
language: French
body: Hello world
assert: []
options: {}
metadata: {}
testIdx: 0
vars: *ref_3
- cost: 0.000006449999999999999
gradingResult:
pass: true
score: 1
reason: No assertions
tokensUsed:
total: 0
prompt: 0
completion: 0
cached: 0
assertion: null
id: 657a9223-a33d-416d-9a80-a2770fe88d5c
latencyMs: 8
namedScores: {}
prompt:
raw: 'Rephrase this in Pirate: Hello world'
label: 'Rephrase this in {{language}}: {{body}}'
promptId: dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207
promptIdx: 0
provider:
id: openai:chat:gpt-5.4-mini
label: ''
response:
output: Ahoy, me hearties!
tokenUsage:
cached: 22
total: 22
cached: true
cost: 0.000006449999999999999
score: 1
success: true
testCase:
vars: &ref_4
language: Pirate
body: Hello world
assert: []
options: {}
metadata: {}
testIdx: 2
vars: *ref_4
- cost: 0.00000945
gradingResult:
pass: true
score: 1
reason: No assertions
tokensUsed:
total: 0
prompt: 0
completion: 0
cached: 0
assertion: null
id: a5b11097-af01-4869-aec7-b3ba428aa926
latencyMs: 4
namedScores: {}
prompt:
raw: "Rephrase this in Pirate: I'm hungry"
label: 'Rephrase this in {{language}}: {{body}}'
promptId: dd3415036994f2d82d0ba70cc5bb2caed82da6ccbdaca8a2174cf3992141c207
promptIdx: 0
provider:
id: openai:chat:gpt-5.4-mini
label: ''
response:
output: Arrr, me belly be grumblin'!
tokenUsage:
cached: 27
total: 27
cached: true
cost: 0.00000945
score: 1
success: true
testCase:
vars: &ref_5
language: Pirate
body: I'm hungry
assert: []
options: {}
metadata: {}
testIdx: 3
vars: *ref_5
- cost: 0.00000945
gradingResult:
pass: true
score: 1
reason: No assertions
tokensUsed:
total: 0
prompt: 0
completion: 0
cached: 0
assertion: null
id: 5cd8e725-9bf1-4c40-9337-655b8d9d41f8
latencyMs: 7
namedScores: {}
prompt:
raw: 'Translate this to conversational Pirate: Hello world'
label: 'Translate this to conversational {{language}}: {{body}}'
promptId: 68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c
promptIdx: 1
provider:
id: openai:chat:gpt-5.4-mini
label: ''
response:
output: Ahoy, me hearty! Greetings to ye, world!
tokenUsage:
cached: 27
total: 27
cached: true
cost: 0.00000945
score: 1
success: true
testCase:
vars: &ref_6
language: Pirate
body: Hello world
assert: []
options: {}
metadata: {}
testIdx: 2
vars: *ref_6
- cost: 0.00001545
gradingResult:
pass: true
score: 1
reason: No assertions
tokensUsed:
total: 0
prompt: 0
completion: 0
cached: 0
assertion: null
id: df2bb14b-8770-4237-bff0-19d0257703ee
latencyMs: 5
namedScores: {}
prompt:
raw: "Translate this to conversational Pirate: I'm hungry"
label: 'Translate this to conversational {{language}}: {{body}}'
promptId: 68f67e9473a10ba8e5294074e90b8cc9daa9aed6256e89391e4ad7adbffba52c
promptIdx: 1
provider:
id: openai:chat:gpt-5.4-mini
label: ''
response:
output: Arrr, me belly be rumblin'! I be feelin' mighty hungry, matey!
tokenUsage:
cached: 37
total: 37
cached: true
cost: 0.00001545
score: 1
success: true
testCase:
vars: &ref_7
language: Pirate
body: I'm hungry
assert: []
options: {}
metadata: {}
testIdx: 3
vars: *ref_7
stats:
successes: 8
failures: 0
tokenUsage:
cached: 190
completion: 0
prompt: 0
total: 190
config:
description: Simple test
prompts:
- 'Rephrase this in {{language}}: {{body}}'
- 'Translate this to conversational {{language}}: {{body}}'
providers:
- openai:chat:gpt-5.4-mini
tests:
- vars:
language: French
body: Hello world
- vars:
language: French
body: I'm hungry
- vars:
language: Pirate
body: Hello world
- vars:
language: Pirate
body: I'm hungry
sharing: true
outputPath:
- output.yaml
- output.json
extensions: []
shareableUrl: null
examples/simple-cli/promptfooconfig.yaml
+23
View File
@@ -0,0 +1,23 @@
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: 'Simple test'
prompts:
- 'Rephrase this in {{language}}: {{body}}'
- 'Translate this to conversational {{language}}: {{body}}'
providers:
- openai:chat:gpt-5.4-mini
tests:
- vars:
language: French
body: Hello world
- vars:
language: French
body: I'm hungry
- vars:
language: Pirate
body: Hello world
- vars:
language: Pirate
body: I'm hungry