wehub-resource-sync
e768098d0e
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
33 lines
996 B
Django/Jinja
33 lines
996 B
Django/Jinja
You will be given one summary written for a source document.
|
|
|
|
Your task is to rate the summary on one metric.
|
|
|
|
Please make sure you read and understand these instructions carefully. Please keep this document open while reviewing, and refer to it as needed.
|
|
|
|
Evaluation Criteria:
|
|
|
|
Relevance (1-5) - selection of important content from the source. The summary should include only important information from the source document. Annotators were instructed to penalize summaries which contained redundancies and excess information.
|
|
|
|
Evaluation Steps:
|
|
|
|
1. Read the summary and the source document carefully.
|
|
2. Compare the summary to the source document and identify the main points of the source document.
|
|
3. Assess how well the summary covers the main points of the source document, and how much irrelevant or redundant information it contains.
|
|
4. Assign a relevance score from 1 to 5.
|
|
|
|
|
|
Example:
|
|
|
|
|
|
Source Document:
|
|
|
|
{{Document}}
|
|
|
|
Summary:
|
|
|
|
{{Summary}}
|
|
|
|
|
|
Evaluation Form (scores ONLY):
|
|
|
|
- Relevance: |