microsoft--promptflow/examples/flex-flows/basic/flex-flow-quickstart-azure.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting started with flex flow in Azure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Learning Objectives** - Upon completing this tutorial, you should be able to:\n",
    "\n",
    "- Write an LLM application using a notebook and visualize the trace of your application.\n",
    "- Convert the application into a flow and batch-run it against multiple lines of data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. Install dependent packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install -r ./requirements-azure.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Connection to workspace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure credential\n",
    "\n",
    "We are using `DefaultAzureCredential` to access the workspace. \n",
    "`DefaultAzureCredential` should be capable of handling most Azure SDK authentication scenarios. \n",
    "\n",
    "Reference for other credentials if this does not work for you: [configure credential example](https://github.com/microsoft/promptflow/blob/main/examples/configuration.ipynb), [azure-identity reference doc](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity?view=azure-python)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential\n",
    "\n",
    "try:\n",
    "    credential = DefaultAzureCredential()\n",
    "    # Check if given credential can get token successfully.\n",
    "    credential.get_token(\"https://management.azure.com/.default\")\n",
    "except Exception as ex:\n",
    "    # Fall back to InteractiveBrowserCredential in case DefaultAzureCredential does not work\n",
    "    credential = InteractiveBrowserCredential()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Connect to the workspace\n",
    "\n",
    "We use a config file to connect to a workspace. The Azure ML workspace should be configured with a computer cluster. [Check this notebook for how to configure a workspace](https://github.com/microsoft/promptflow/blob/main/examples/configuration.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from promptflow.azure import PFClient\n",
    "\n",
    "# Connect to the workspace\n",
    "pf = PFClient.from_config(credential=credential)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create necessary connections\n",
    "A connection helps securely store and manage secret keys or other sensitive credentials required for interacting with the LLM and other external tools, for example Azure Content Safety.\n",
    "\n",
    "In this notebook, we will use the `basic` & `eval-code-quality` flex flow, which uses the connection `open_ai_connection`.  We need to set up the connection if we haven't added it before.\n",
    "\n",
    "To prepare your Azure OpenAI resource, follow these [instructions](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal) and get your `api_key` if you don't have one.\n",
    "\n",
    "Go to [workspace portal](https://ml.azure.com/), click `Prompt flow` -> `Connections` -> `Create`, then follow the instruction to create your own connections. \n",
    "Learn more on [connections](https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/concept-connections?view=azureml-api-2)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Batch run the function as a flow with multi-line data.\n",
    "\n",
    "Create a `flow.flex.yaml` file to define a flow whose entry points to the python function we defined.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show the flow.flex.yaml content\n",
    "with open(\"flow.flex.yaml\") as fin:\n",
    "    print(fin.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batch run with a data file (with multiple lines of test data)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flow = \".\"  # Path to the flow directory\n",
    "data = \"./data.jsonl\"  # Path to the data file\n",
    "\n",
    "# Create a run with the flow and data\n",
    "base_run = pf.run(\n",
    "    flow=flow,\n",
    "    data=data,\n",
    "    column_mapping={\n",
    "        \"text\": \"${data.text}\",\n",
    "    },\n",
    "    environment_variables={\n",
    "        \"AZURE_OPENAI_API_KEY\": \"${open_ai_connection.api_key}\",\n",
    "        \"AZURE_OPENAI_ENDPOINT\": \"${open_ai_connection.api_base}\",\n",
    "    },\n",
    "    stream=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "details = pf.get_details(base_run)\n",
    "details.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Evaluate your flow\n",
    "Then you can use an evaluation method to evaluate your flow. The evaluation methods are also flows which usually use an LLM to verify the produced output matches the expected output. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup model configuration with connection\n",
    "\n",
    "When using Promptflow in Azure, create a model configuration object with connection name. \n",
    "The model config will connect to the cloud-hosted Promptflow instance while running the flow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from promptflow.core import AzureOpenAIModelConfiguration\n",
    "\n",
    "model_config = AzureOpenAIModelConfiguration(\n",
    "    connection=\"open_ai_connection\",\n",
    "    azure_deployment=\"gpt-4o\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate the previous batch run\n",
    "The **base_run** is the batch run we completed in step 2 above, for web-classification flow with \"data.jsonl\" as input.  The evaluation takes the outputs of that **base_run**, and uses an LLM to compare them to your desired outputs, and then visualizes the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "eval_flow = \"../eval-code-quality/flow.flex.yaml\"\n",
    "\n",
    "eval_run = pf.run(\n",
    "    flow=eval_flow,\n",
    "    init={\"model_config\": model_config},\n",
    "    data=\"./data.jsonl\",  # path to the data file\n",
    "    run=base_run,  # specify the base_run as the run you want to evaluate\n",
    "    column_mapping={\n",
    "        \"code\": \"${run.outputs.output}\",\n",
    "    },\n",
    "    stream=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "details = pf.get_details(eval_run)\n",
    "details.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "metrics = pf.get_metrics(eval_run)\n",
    "print(json.dumps(metrics, indent=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pf.visualize([base_run, eval_run])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "You've successfully run your first flex flow and evaluated it. That's great!\n",
    "\n",
    "You can check out more examples:\n",
    "- [Basic Chat](https://github.com/microsoft/promptflow/tree/main/examples/flex-flows/chat-basic): demonstrates how to create a chatbot that can remember previous interactions and use the conversation history to generate the next message."
   ]
  }
 ],
 "metadata": {
  "build_doc": {
   "author": [
    "D-W-@github.com",
    "wangchao1230@github.com"
   ],
   "category": "azure",
   "section": "Flow",
   "weight": 10
  },
  "description": "A quickstart tutorial to run a flex flow and evaluate it in Azure.",
  "kernelspec": {
   "display_name": "prompt_flow",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  },
  "resources": "examples/requirements-azure.txt, examples/flex-flows/basic, examples/flex-flows/eval-code-quality"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}