chore: import upstream snapshot with attribution
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:52 +08:00
commit e768098d0e
4004 changed files with 2804145 additions and 0 deletions
+25
View File
@@ -0,0 +1,25 @@
# Basic Eval
Basic evaluator prompt for QA scenario
## Prerequisites
Install `promptflow-devkit`:
```bash
pip install promptflow-devkit
```
## Run prompty
- Prepare your Azure OpenAI resource follow this [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal) and get your `api_key` if you don't have one.
- Setup environment variables
Ensure you have put your azure OpenAI endpoint key in [.env](../.env) file. You can create one refer to this [example file](../.env.example).
```bash
cat ../.env
```
- Test flow
```bash
pf flow test --flow eval.prompty --env --inputs sample.json
```
+48
View File
@@ -0,0 +1,48 @@
---
name: basic evaluate
description: basic evaluator for QA scenario
model:
api: chat
configuration:
type: azure_openai
azure_deployment: gpt-4o
api_key: ${env:AZURE_OPENAI_API_KEY}
azure_endpoint: ${env:AZURE_OPENAI_ENDPOINT}
parameters:
temperature: 0.2
max_tokens: 200
top_p: 1.0
response_format:
type: json_object
inputs:
question:
type: string
answer:
type: string
ground_truth:
type: string
outputs:
score:
type: string
explanation:
type: string
---
system:
You are an AI assistant.
You task is to evaluate a score for the answer based on the ground_truth and original question.
This score value should always be an integer between 1 and 5. So the score produced should be 1 or 2 or 3 or 4 or 5.
The output should be valid JSON.
**Example**
question: "What is the capital of France?"
answer: "Paris"
ground_truth: "Paris"
output:
{"score": "5", "explanation":"paris is the capital of France"}
user:
question: {{question}}
answer: {{answer}}
statement: {{statement}}
output:
+5
View File
@@ -0,0 +1,5 @@
{
"question": "what's the capital of China?",
"answer": "Shanghai",
"ground_truth": "Beijing"
}