microsoft--promptflow/examples/flex-flows/chat-basic/chat-with-class-based-flow-azure.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chat with class based flex flow in Azure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Learning Objectives** - Upon completing this tutorial, you should be able to:\n",
    "\n",
    "- Submit batch run with a flow defined with python class and evaluate it in azure.\n",
    "\n",
    "## 0. Install dependent packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install -r ./requirements-azure.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Connection to workspace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure credential\n",
    "\n",
    "We are using `DefaultAzureCredential` to get access to workspace. \n",
    "`DefaultAzureCredential` should be capable of handling most Azure SDK authentication scenarios. \n",
    "\n",
    "Reference for more available credentials if it does not work for you: [configure credential example](https://github.com/microsoft/promptflow/blob/main/examples/configuration.ipynb), [azure-identity reference doc](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity?view=azure-python)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential\n",
    "\n",
    "try:\n",
    "    credential = DefaultAzureCredential()\n",
    "    # Check if given credential can get token successfully.\n",
    "    credential.get_token(\"https://management.azure.com/.default\")\n",
    "except Exception as ex:\n",
    "    # Fall back to InteractiveBrowserCredential in case DefaultAzureCredential not work\n",
    "    credential = InteractiveBrowserCredential()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get a handle to the workspace\n",
    "\n",
    "We use config file to connect to a workspace. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from promptflow.azure import PFClient\n",
    "\n",
    "# Get a handle to workspace\n",
    "pf = PFClient.from_config(credential=credential)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create necessary connections\n",
    "Connection helps securely store and manage secret keys or other sensitive credentials required for interacting with LLM and other external tools for example Azure Content Safety.\n",
    "\n",
    "In this notebook, we will use flow `basic` flex flow which uses connection `open_ai_connection` inside, we need to set up the connection if we haven't added it before.\n",
    "\n",
    "Prepare your Azure OpenAI resource follow this [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal) and get your `api_key` if you don't have one.\n",
    "\n",
    "Please go to [workspace portal](https://ml.azure.com/), click `Prompt flow` -> `Connections` -> `Create`, then follow the instruction to create your own connections. \n",
    "Learn more on [connections](https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/concept-connections?view=azureml-api-2)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Batch run the function as flow with multi-line data\n",
    "\n",
    "Create a `flow.flex.yaml` file to define a flow which entry pointing to the python function we defined.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# show the flow.flex.yaml content\n",
    "with open(\"flow.flex.yaml\") as fin:\n",
    "    print(fin.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from promptflow.core import AzureOpenAIModelConfiguration\n",
    "\n",
    "# create the model config to be used in below flow calls\n",
    "config = AzureOpenAIModelConfiguration(\n",
    "    connection=\"open_ai_connection\", azure_deployment=\"gpt-4o\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batch run with a data file (with multiple lines of test data)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flow = \".\"  # path to the flow directory\n",
    "data = \"./data.jsonl\"  # path to the data file\n",
    "\n",
    "# create run with the flow and data\n",
    "base_run = pf.run(\n",
    "    flow=flow,\n",
    "    init={\n",
    "        \"model_config\": config,\n",
    "    },\n",
    "    data=data,\n",
    "    column_mapping={\n",
    "        \"question\": \"${data.question}\",\n",
    "        \"chat_history\": \"${data.chat_history}\",\n",
    "    },\n",
    "    stream=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "details = pf.get_details(base_run)\n",
    "details.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Evaluate your flow\n",
    "Then you can use an evaluation method to evaluate your flow. The evaluation methods are also flows which usually using LLM assert the produced output matches certain expectation. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run evaluation on the previous batch run\n",
    "The **base_run** is the batch run we completed in step 2 above, for web-classification flow with \"data.jsonl\" as input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "eval_flow = \"../eval-checklist/flow.flex.yaml\"\n",
    "config = AzureOpenAIModelConfiguration(\n",
    "    connection=\"open_ai_connection\", azure_deployment=\"gpt-4o\"\n",
    ")\n",
    "eval_run = pf.run(\n",
    "    flow=eval_flow,\n",
    "    init={\n",
    "        \"model_config\": config,\n",
    "    },\n",
    "    data=\"./data.jsonl\",  # path to the data file\n",
    "    run=base_run,  # specify base_run as the run you want to evaluate\n",
    "    column_mapping={\n",
    "        \"answer\": \"${run.outputs.output}\",\n",
    "        \"statements\": \"${data.statements}\",\n",
    "    },\n",
    "    stream=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "details = pf.get_details(eval_run)\n",
    "details.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "metrics = pf.get_metrics(eval_run)\n",
    "print(json.dumps(metrics, indent=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pf.visualize([base_run, eval_run])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "By now you've successfully run your chat flow and did evaluation on it. That's great!\n",
    "\n",
    "You can check out more examples:\n",
    "- [Stream Chat](https://github.com/microsoft/promptflow/tree/main/examples/flex-flows/chat-stream): demonstrates how to create a chatbot that can remember previous interactions and use the conversation history to generate next message."
   ]
  }
 ],
 "metadata": {
  "build_doc": {
   "author": [
    "D-W-@github.com",
    "wangchao1230@github.com"
   ],
   "category": "azure",
   "section": "Flow",
   "weight": 11
  },
  "description": "A quickstart tutorial to run a class based flex flow and evaluate it in azure.",
  "kernelspec": {
   "display_name": "prompt_flow",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  },
  "resources": "examples/requirements-azure.txt, examples/flex-flows/basic, examples/flex-flows/eval-checklist"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}