chore: import upstream snapshot with attribution

examples/flows/standard/web-classification/README.md

@@ -0,0 +1,129 @@
+# Web Classification
+
+This is a flow demonstrating multi-class classification with LLM. Given an url, it will classify the url into one web category with just a few shots, simple summarization and classification prompts.
+
+## Tools used in this flow
+- LLM Tool
+- Python Tool
+
+## What you will learn
+
+In this flow, you will learn
+- how to compose a classification flow with LLM.
+- how to feed few shots to LLM classifier.
+
+## Prerequisites
+
+Install promptflow sdk and other dependencies:
+```bash
+pip install -r requirements.txt
+```
+
+## Getting Started
+
+### 1. Setup connection
+
+If you are using Azure OpenAI, prepare your resource follow this [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal) and get your `api_key` if you don't have one.
+
+```bash
+# Override keys with --set to avoid yaml file changes
+pf connection create --file ../../../connections/azure_openai.yml --set api_key=<your_api_key> api_base=<your_api_base> --name open_ai_connection
+```
+
+If you using OpenAI, sign up account [OpenAI website](https://openai.com/), login and [find personal API key](https://platform.openai.com/account/api-keys).
+
+```shell
+pf connection create --file ../../../connections/openai.yml --set api_key=<your_api_key>
+```
+
+### 2. Configure the flow with your connection
+`flow.dag.yaml` is already configured with connection named `open_ai_connection`.
+
+### 3. Test flow with single line data
+
+```bash
+# test with default input value in flow.dag.yaml
+pf flow test --flow .
+# test with user specified inputs
+pf flow test --flow . --inputs url='https://www.youtube.com/watch?v=kYqRtjDBci8'
+```
+
+### 4. Run with multi-line data
+
+```bash
+# create run using command line args
+pf run create --flow . --data ./data.jsonl --column-mapping url='${data.url}' --stream
+
+# (Optional) create a random run name
+run_name="web_classification_"$(openssl rand -hex 12)
+# create run using yaml file, run_name will be used in following contents, --name is optional
+pf run create --file run.yml --stream --name $run_name
+```
+
+You can also skip providing `column-mapping` if provided data has same column name as the flow.
+Reference [here](https://aka.ms/pf/column-mapping) for default behavior when `column-mapping` not provided in CLI.
+
+```bash
+# list run
+pf run list
+# show run
+pf run show --name $run_name
+# show run outputs
+pf run show-details --name $run_name
+```
+
+### 5. Run with classification evaluation flow
+
+create `evaluation` run:
+```bash
+# (Optional) save previous run name into variable, and create a new random run name for further use
+prev_run_name=$run_name
+run_name="classification_accuracy_"$(openssl rand -hex 12)
+# create run using command line args
+pf run create --flow ../../evaluation/eval-classification-accuracy --data ./data.jsonl --column-mapping groundtruth='${data.answer}' prediction='${run.outputs.category}' --run $prev_run_name --stream
+# create run using yaml file, --name is optional
+pf run create --file run_evaluation.yml --run $prev_run_name --stream --name $run_name
+```
+
+```bash
+pf run show-details --name $run_name
+pf run show-metrics --name $run_name
+pf run visualize --name $run_name
+```
+
+
+### 6. Submit run to cloud
+```bash
+# set default workspace
+az account set -s <your_subscription_id>
+az configure --defaults group=<your_resource_group_name> workspace=<your_workspace_name>
+
+# create run
+pfazure run create --flow . --data ./data.jsonl --column-mapping url='${data.url}' --stream
+
+# (Optional) create a new random run name for further use
+run_name="web_classification_"$(openssl rand -hex 12)
+
+# create run using yaml file, --name is optional
+pfazure run create --file run.yml --name $run_name
+
+
+pfazure run stream --name $run_name
+pfazure run show-details --name $run_name
+pfazure run show-metrics --name $run_name
+
+
+# (Optional) save previous run name into variable, and create a new random run name for further use
+prev_run_name=$run_name
+run_name="classification_accuracy_"$(openssl rand -hex 12)
+
+# create evaluation run, --name is optional
+pfazure run create --flow ../../evaluation/eval-classification-accuracy --data ./data.jsonl --column-mapping groundtruth='${data.answer}' prediction='${run.outputs.category}' --run $prev_run_name
+pfazure run create --file run_evaluation.yml --run $prev_run_name --stream --name $run_name
+
+pfazure run stream --name $run_name
+pfazure run show --name $run_name
+pfazure run show-details --name $run_name
+pfazure run show-metrics --name $run_name
+pfazure run visualize --name $run_name
+```

examples/flows/standard/web-classification/classify_with_llm.jinja2

@@ -0,0 +1,21 @@
+# system:
+Your task is to classify a given url into one of the following categories:
+Movie, App, Academic, Channel, Profile, PDF or None based on the text content information.
+The classification will be based on the url, the webpage text content summary, or both.
+
+# user:
+The selection range of the value of "category" must be within "Movie", "App", "Academic", "Channel", "Profile", "PDF" and "None".
+The selection range of the value of "evidence" must be within "Url", "Text content", and "Both".
+Here are a few examples:
+{% for ex in examples %}
+URL: {{ex.url}}
+Text content: {{ex.text_content}}
+OUTPUT:
+{"category": "{{ex.category}}", "evidence": "{{ex.evidence}}"}
+
+{% endfor %}
+
+For a given URL and text content, classify the url to complete the category and indicate evidence:
+URL: {{url}}
+Text content: {{text_content}}.
+OUTPUT:

examples/flows/standard/web-classification/convert_to_dict.py

@@ -0,0 +1,12 @@
+import json
+
+from promptflow.core import tool
+
+
+@tool
+def convert_to_dict(input_str: str):
+    try:
+        return json.loads(input_str)
+    except Exception as e:
+        print("The input is not valid, error: {}".format(e))
+        return {"category": "None", "evidence": "None"}

examples/flows/standard/web-classification/data.jsonl

@@ -0,0 +1,3 @@
+{"url": "https://www.youtube.com/watch?v=kYqRtjDBci8", "answer": "Channel", "evidence": "Both"}
+{"url": "https://arxiv.org/abs/2307.04767", "answer": "Academic", "evidence": "Both"}
+{"url": "https://play.google.com/store/apps/details?id=com.twitter.android", "answer": "App", "evidence": "Both"}

examples/flows/standard/web-classification/fetch_text_content_from_url.py

@@ -0,0 +1,30 @@
+import bs4
+import requests
+
+from promptflow.core import tool
+
+
+@tool
+def fetch_text_content_from_url(url: str):
+    # Send a request to the URL
+    try:
+        headers = {
+            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) "
+            "Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.35"
+        }
+        response = requests.get(url, headers=headers)
+        if response.status_code == 200:
+            # Parse the HTML content using BeautifulSoup
+            soup = bs4.BeautifulSoup(response.text, "html.parser")
+            soup.prettify()
+            return soup.get_text()[:2000]
+        else:
+            msg = (
+                f"Get url failed with status code {response.status_code}.\nURL: {url}\nResponse: "
+                f"{response.text[:100]}"
+            )
+            print(msg)
+            return "No available content"
+    except Exception as e:
+        print("Get url failed with error: {}".format(e))
+        return "No available content"

examples/flows/standard/web-classification/flow.dag.yaml

@@ -0,0 +1,84 @@
+$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Flow.schema.json
+environment:
+  python_requirements_txt: requirements.txt
+inputs:
+  url:
+    type: string
+    default: https://play.google.com/store/apps/details?id=com.twitter.android
+outputs:
+  category:
+    type: string
+    reference: ${convert_to_dict.output.category}
+  evidence:
+    type: string
+    reference: ${convert_to_dict.output.evidence}
+nodes:
+- name: fetch_text_content_from_url
+  type: python
+  source:
+    type: code
+    path: fetch_text_content_from_url.py
+  inputs:
+    url: ${inputs.url}
+- name: summarize_text_content
+  use_variants: true
+- name: prepare_examples
+  type: python
+  source:
+    type: code
+    path: prepare_examples.py
+  inputs: {}
+- name: classify_with_llm
+  type: llm
+  source:
+    type: code
+    path: classify_with_llm.jinja2
+  inputs:
+    deployment_name: gpt-35-turbo
+    model: gpt-3.5-turbo
+    max_tokens: 128
+    temperature: 0.2
+    url: ${inputs.url}
+    text_content: ${summarize_text_content.output}
+    examples: ${prepare_examples.output}
+  connection: open_ai_connection
+  api: chat
+- name: convert_to_dict
+  type: python
+  source:
+    type: code
+    path: convert_to_dict.py
+  inputs:
+    input_str: ${classify_with_llm.output}
+node_variants:
+  summarize_text_content:
+    default_variant_id: variant_0
+    variants:
+      variant_0:
+        node:
+          type: llm
+          source:
+            type: code
+            path: summarize_text_content.jinja2
+          inputs:
+            deployment_name: gpt-35-turbo
+            model: gpt-3.5-turbo
+            max_tokens: 128
+            temperature: 0.2
+            text: ${fetch_text_content_from_url.output}
+          connection: open_ai_connection
+          api: chat
+      variant_1:
+        node:
+          type: llm
+          source:
+            type: code
+            path: summarize_text_content__variant_1.jinja2
+          inputs:
+            deployment_name: gpt-35-turbo
+            model: gpt-3.5-turbo
+            max_tokens: 256
+            temperature: 0.3
+            text: ${fetch_text_content_from_url.output}
+          connection: open_ai_connection
+          api: chat

examples/flows/standard/web-classification/prepare_examples.py

@@ -0,0 +1,44 @@
+from promptflow.core import tool
+
+
+@tool
+def prepare_examples():
+    return [
+        {
+            "url": "https://play.google.com/store/apps/details?id=com.spotify.music",
+            "text_content": "Spotify is a free music and podcast streaming app with millions of songs, albums, and "
+            "original podcasts. It also offers audiobooks, so users can enjoy thousands of stories. "
+            "It has a variety of features such as creating and sharing music playlists, discovering "
+            "new music, and listening to popular and exclusive podcasts. It also has a Premium "
+            "subscription option which allows users to download and listen offline, and access "
+            "ad-free music. It is available on all devices and has a variety of genres and artists "
+            "to choose from.",
+            "category": "App",
+            "evidence": "Both",
+        },
+        {
+            "url": "https://www.youtube.com/channel/UC_x5XG1OV2P6uZZ5FSM9Ttw",
+            "text_content": "NFL Sunday Ticket is a service offered by Google LLC that allows users to watch NFL "
+            "games on YouTube. It is available in 2023 and is subject to the terms and privacy policy "
+            "of Google LLC. It is also subject to YouTube's terms of use and any applicable laws.",
+            "category": "Channel",
+            "evidence": "URL",
+        },
+        {
+            "url": "https://arxiv.org/abs/2303.04671",
+            "text_content": "Visual ChatGPT is a system that enables users to interact with ChatGPT by sending and "
+            "receiving not only languages but also images, providing complex visual questions or "
+            "visual editing instructions, and providing feedback and asking for corrected results. "
+            "It incorporates different Visual Foundation Models and is publicly available. Experiments "
+            "show that Visual ChatGPT opens the door to investigating the visual roles of ChatGPT with "
+            "the help of Visual Foundation Models.",
+            "category": "Academic",
+            "evidence": "Text content",
+        },
+        {
+            "url": "https://ab.politiaromana.ro/",
+            "text_content": "There is no content available for this text.",
+            "category": "None",
+            "evidence": "None",
+        },
+    ]

examples/flows/standard/web-classification/requirements.txt

@@ -0,0 +1,3 @@
+promptflow[azure]>=1.7.0
+promptflow-tools
+bs4

examples/flows/standard/web-classification/run.yml

@@ -0,0 +1,6 @@
+$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Run.schema.json
+flow: .
+data: data.jsonl
+variant: ${summarize_text_content.variant_1}
+column_mapping:
+   url: ${data.url}

examples/flows/standard/web-classification/run_evaluation.yml

@@ -0,0 +1,7 @@
+$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Run.schema.json
+flow: ../../evaluation/eval-classification-accuracy
+data: data.jsonl
+run: web_classification_variant_1_20230724_173442_973403 # replace with your run name
+column_mapping:
+  groundtruth: ${data.answer}
+  prediction: ${run.outputs.category}

examples/flows/standard/web-classification/summarize_text_content.jinja2

@@ -0,0 +1,7 @@
+# system:
+Please summarize the following text in one paragraph. 100 words.
+Do not add any information that is not in the text.
+
+# user:
+Text: {{text}}
+Summary:

examples/flows/standard/web-classification/summarize_text_content__variant_1.jinja2

@@ -0,0 +1,7 @@
+# system:
+Please summarize some keywords of this paragraph and have some details of each keywords.
+Do not add any information that is not in the text.
+
+# user:
+Text: {{text}}
+Summary: