chore: import upstream snapshot with attribution

examples/flows/evaluation/eval-classification-accuracy/README.md

@@ -0,0 +1,50 @@
+# Classification Accuracy Evaluation
+
+This is a flow illustrating how to evaluate the performance of a classification system. It involves comparing each prediction to the groundtruth and assigns a "Correct" or "Incorrect" grade, and aggregating the results to produce metrics such as accuracy, which reflects how good the system is at classifying the data.
+
+Tools used in this flow：
+- `python` tool
+
+## What you will learn
+
+In this flow, you will learn
+- how to compose a point based evaluation flow, where you can calculate point-wise metrics.
+- the way to log metrics. use `from promptflow.core import log_metric`
+    - see file [calculate_accuracy.py](calculate_accuracy.py)
+
+### 0. Setup connection
+
+Prepare your Azure OpenAI resource follow this [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal) and get your `api_key` if you don't have one.
+
+```bash
+# Override keys with --set to avoid yaml file changes
+pf connection create --file ../../../connections/azure_openai.yml --set api_key=<your_api_key> api_base=<your_api_base>
+```
+
+### 1. Test flow/node
+
+```bash
+# test with default input value in flow.dag.yaml
+pf flow test --flow .
+
+# test with flow inputs
+pf flow test --flow . --inputs groundtruth=APP prediction=APP
+
+# test node with inputs
+pf flow test --flow . --node grade --inputs groundtruth=groundtruth prediction=prediction
+```
+
+### 2. create flow run with multi line data
+There are two ways to evaluate an classification flow.
+
+```bash
+pf run create --flow . --data ./data.jsonl --column-mapping groundtruth='${data.groundtruth}' prediction='${data.prediction}' --stream
+```
+
+You can also skip providing `column-mapping` if provided data has same column name as the flow.
+Reference [here](https://aka.ms/pf/column-mapping) for default behavior when `column-mapping` not provided in CLI.
+
+### 3. create run against other flow run
+
+Learn more in [web-classification](../../standard/web-classification/README.md)
+

examples/flows/evaluation/eval-classification-accuracy/calculate_accuracy.py

@@ -0,0 +1,17 @@
+from typing import List
+
+from promptflow.core import log_metric, tool
+
+
+@tool
+def calculate_accuracy(grades: List[str]):
+    result = []
+    for index in range(len(grades)):
+        grade = grades[index]
+        result.append(grade)
+
+    # calculate accuracy for each variant
+    accuracy = round((result.count("Correct") / len(result)), 2)
+    log_metric("accuracy", accuracy)
+
+    return result

examples/flows/evaluation/eval-classification-accuracy/data.jsonl

@@ -0,0 +1,3 @@
+{"groundtruth": "App","prediction": "App"}
+{"groundtruth": "Channel","prediction": "Channel"}
+{"groundtruth": "Academic","prediction": "Academic"}

examples/flows/evaluation/eval-classification-accuracy/flow.dag.yaml

@@ -0,0 +1,35 @@
+$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Flow.schema.json
+inputs:
+  groundtruth:
+    type: string
+    description: Please specify the groundtruth column, which contains the true label
+      to the outputs that your flow produces.
+    default: APP
+  prediction:
+    type: string
+    description: Please specify the prediction column, which contains the predicted
+      outputs that your flow produces.
+    default: APP
+outputs:
+  grade:
+    type: string
+    reference: ${grade.output}
+nodes:
+- name: grade
+  type: python
+  source:
+    type: code
+    path: grade.py
+  inputs:
+    groundtruth: ${inputs.groundtruth}
+    prediction: ${inputs.prediction}
+- name: calculate_accuracy
+  type: python
+  source:
+    type: code
+    path: calculate_accuracy.py
+  inputs:
+    grades: ${grade.output}
+  aggregation: true
+environment:
+    python_requirements_txt: requirements.txt

examples/flows/evaluation/eval-classification-accuracy/grade.py

@@ -0,0 +1,6 @@
+from promptflow.core import tool
+
+
+@tool
+def grade(groundtruth: str, prediction: str):
+    return "Correct" if groundtruth.lower() == prediction.lower() else "Incorrect"

examples/flows/evaluation/eval-classification-accuracy/requirements.txt

@@ -0,0 +1,2 @@
+promptflow
+promptflow-tools