wehub-resource-sync
e768098d0e
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
32 lines
1.1 KiB
Django/Jinja
32 lines
1.1 KiB
Django/Jinja
You will be given one summary written for a source document.
|
|
|
|
Your task is to rate the summary on one metric.
|
|
|
|
Please make sure you read and understand these instructions carefully. Please keep this document open while reviewing, and refer to it as needed.
|
|
|
|
Evaluation Criteria:
|
|
|
|
Coherence (1-5) - the collective quality of all sentences. We align this dimension with the DUC quality question of structure and coherence whereby "the summary should be well-structured and well-organized. The summary should not just be a heap of related information, but should build from sentence to a coherent body of information about a topic."
|
|
|
|
Evaluation Steps:
|
|
|
|
1. Read the source document carefully and identify the main topic and key points.
|
|
2. Read the summary and compare it to the source document. Check if the summary covers the main topic and key points of the source document, and if it presents them in a clear and logical order.
|
|
3. Assign a score for coherence on a scale of 1 to 5, where 1 is the lowest and 5 is the highest based on the Evaluation Criteria.
|
|
|
|
|
|
Example:
|
|
|
|
|
|
Source Document:
|
|
|
|
{{Document}}
|
|
|
|
Summary:
|
|
|
|
{{Summary}}
|
|
|
|
|
|
Evaluation Form (scores ONLY):
|
|
|
|
- Coherence: |