wehub-resource-sync
e768098d0e
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
43 lines
1006 B
Plaintext
43 lines
1006 B
Plaintext
|
|
---
|
|
name: Evaluate based on a checklist
|
|
description: Evaluate the quality of code snippet.
|
|
model:
|
|
api: chat
|
|
configuration:
|
|
type: azure_openai
|
|
azure_deployment: gpt-4o
|
|
parameters:
|
|
max_tokens: 256
|
|
temperature: 0.7
|
|
|
|
inputs:
|
|
examples:
|
|
type: list
|
|
answer:
|
|
type: string
|
|
statement:
|
|
type: string
|
|
sample: ${file:sample.json}
|
|
---
|
|
|
|
# system:
|
|
You are an AI assistant.
|
|
You task is to evaluate a score based on how the statement applies for the answer.
|
|
Only accepts valid JSON format response without extra prefix or postfix.
|
|
|
|
# user:
|
|
This score value should always be an integer between 1 and 5. So the score produced should be 1 or 2 or 3 or 4 or 5.
|
|
|
|
Here are a few examples:
|
|
{% for ex in examples %}
|
|
answer: {{ex.answer}}
|
|
statement: {{ex.statement}}
|
|
OUTPUT:
|
|
{"score": "{{ex.score}}", "explanation":"{{ex.explanation}}"}
|
|
{% endfor %}
|
|
|
|
For a given answer, valuate the answer based on how the statement applies for the answer:
|
|
answer: {{answer}}
|
|
statement: {{statement}}
|
|
OUTPUT: |