wehub-resource-sync
e768098d0e
Flake8 Lint / flake8 (push) Waiting to run
Spell check CI / Spell_Check (push) Waiting to run
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
48 lines
1.0 KiB
Plaintext
48 lines
1.0 KiB
Plaintext
---
|
|
name: basic evaluate
|
|
description: basic evaluator for QA scenario
|
|
model:
|
|
api: chat
|
|
configuration:
|
|
type: azure_openai
|
|
azure_deployment: gpt-4o
|
|
api_key: ${env:AZURE_OPENAI_API_KEY}
|
|
azure_endpoint: ${env:AZURE_OPENAI_ENDPOINT}
|
|
parameters:
|
|
temperature: 0.2
|
|
max_tokens: 200
|
|
top_p: 1.0
|
|
response_format:
|
|
type: json_object
|
|
|
|
inputs:
|
|
question:
|
|
type: string
|
|
answer:
|
|
type: string
|
|
ground_truth:
|
|
type: string
|
|
outputs:
|
|
score:
|
|
type: string
|
|
explanation:
|
|
type: string
|
|
---
|
|
system:
|
|
You are an AI assistant.
|
|
You task is to evaluate a score for the answer based on the ground_truth and original question.
|
|
This score value should always be an integer between 1 and 5. So the score produced should be 1 or 2 or 3 or 4 or 5.
|
|
The output should be valid JSON.
|
|
|
|
**Example**
|
|
question: "What is the capital of France?"
|
|
answer: "Paris"
|
|
ground_truth: "Paris"
|
|
output:
|
|
{"score": "5", "explanation":"paris is the capital of France"}
|
|
|
|
user:
|
|
question: {{question}}
|
|
answer: {{answer}}
|
|
statement: {{statement}}
|
|
output: |