chore: import upstream snapshot with attribution

2026-07-13 13:24:08 +08:00
commit 0d3cb498a3
5438 changed files with 1316560 additions and 0 deletions
@@ -0,0 +1,18 @@
+# openai-eval-factuality (Openai Eval Factuality)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example openai-eval-factuality
+cd openai-eval-factuality
+```
+
+## Usage
+
+This example uses the factuality and closed QA evaluation method from OpenAI.
+
+It is pre-configured in `promptfooconfig.yaml`. That means you can just run:
+
+```bash
+promptfoo eval
+```
@@ -0,0 +1,20 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: OpenAI factuality evaluation with ground truth comparison
+
+prompts:
+  - file://prompts/name_capitals_concise.txt
+  - file://prompts/name_capitals_verbose.txt
+providers:
+  - openai:chat:gpt-5.4-mini
+tests:
+  - vars:
+      location: Sacramento
+
+    assert:
+      # Ensure the answer agrees with the provided facts
+      - type: factuality
+        value: The capital of California is Sacramento
+
+      # Ensure the answer meets the criteria
+      - type: model-graded-closedqa
+        value: Does not include any extra information or facts about Sacramento
@@ -0,0 +1 @@
+Output the capital of {{location}}
@@ -0,0 +1 @@
+Tell me about the capital of {{location}}