chore: import upstream snapshot with attribution

examples/flows/evaluation/eval-basic/README.md

@@ -0,0 +1,43 @@
+# Basic Eval
+This example shows how to create a basic evaluation flow. 
+
+Tools used in this flow：
+- `python` tool
+
+## Prerequisites
+
+Install promptflow sdk and other dependencies in this folder:
+```bash
+pip install -r requirements.txt
+```
+
+## What you will learn
+
+In this flow, you will learn
+- how to compose a point based evaluation flow, where you can calculate point-wise metrics.
+- the way to log metrics. use `from promptflow.core import log_metric`
+    - see file [aggregate](aggregate.py).
+
+### 1. Test flow with single line data
+
+Testing flow/node:
+```bash
+# test with default input value in flow.dag.yaml
+pf flow test --flow .
+
+# test with flow inputs
+pf flow test --flow . --inputs groundtruth=ABC prediction=ABC
+
+# test node with inputs
+pf flow test --flow . --node line_process --inputs groundtruth=ABC prediction=ABC
+```
+
+### 2. create flow run with multi line data
+There are two ways to evaluate an classification flow.
+
+```bash
+pf run create --flow . --data ./data.jsonl --column-mapping groundtruth='${data.groundtruth}' prediction='${data.prediction}' --stream
+```
+
+You can also skip providing `column-mapping` if provided data has same column name as the flow.
+Reference [here](https://aka.ms/pf/column-mapping) for default behavior when `column-mapping` not provided in CLI.

examples/flows/evaluation/eval-basic/aggregate.py

@@ -0,0 +1,24 @@
+from typing import List
+
+from promptflow.core import tool
+
+
+@tool
+def aggregate(processed_results: List[str]):
+    """
+    This tool aggregates the processed result of all lines to the variant level and log metric for each variant.
+
+    :param processed_results: List of the output of line_process node.
+    """
+
+    # Add your aggregation logic here
+    # aggregated_results should be a dictionary with the metric name as the key and the metric value as the value.
+    results_num = len(processed_results)
+    print(results_num)
+    print(processed_results)
+
+    # Log metric for each variant
+    from promptflow.core import log_metric
+    log_metric(key="results_num", value=results_num)
+
+    return results_num

examples/flows/evaluation/eval-basic/data.jsonl

@@ -0,0 +1 @@
+{"groundtruth": "Tomorrow's weather will be sunny.","prediction": "The weather will be sunny tomorrow."}

examples/flows/evaluation/eval-basic/flow.dag.yaml

@@ -0,0 +1,31 @@
+$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Flow.schema.json
+inputs:
+  groundtruth:
+    type: string
+    default: groundtruth
+  prediction:
+    type: string
+    default: prediction
+outputs:
+  results:
+    type: string
+    reference: ${line_process.output}
+nodes:
+- name: line_process
+  type: python
+  source:
+    type: code
+    path: line_process.py
+  inputs:
+    groundtruth: ${inputs.groundtruth}
+    prediction: ${inputs.prediction}
+- name: aggregate
+  type: python
+  source:
+    type: code
+    path: aggregate.py
+  inputs:
+    processed_results: ${line_process.output}
+  aggregation: true
+environment:
+    python_requirements_txt: requirements.txt

examples/flows/evaluation/eval-basic/line_process.py

@@ -0,0 +1,14 @@
+from promptflow.core import tool
+
+
+@tool
+def line_process(groundtruth: str, prediction: str):
+    """
+    This tool processes the prediction of a single line and returns the processed result.
+
+    :param groundtruth: the groundtruth of a single line.
+    :param prediction: the prediction of a single line.
+    """
+
+    # Add your line processing logic here
+    return "Correct" if groundtruth.lower() == prediction.lower() else "Incorrect"

examples/flows/evaluation/eval-basic/requirements.txt

@@ -0,0 +1,2 @@
+promptflow
+promptflow-tools