chore: import upstream snapshot with attribution

examples/tutorials/tracing/llm/.env.example

@@ -0,0 +1,2 @@
+AZURE_OPENAI_API_KEY=<your_AOAI_key>
+AZURE_OPENAI_ENDPOINT=<your_AOAI_endpoint>

examples/tutorials/tracing/llm/requirements.txt

@@ -0,0 +1,3 @@
+promptflow
+openai>=1.0.0
+python-dotenv

examples/tutorials/tracing/llm/trace-llm.ipynb

@@ -0,0 +1,201 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Tracing with LLM application"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Tracing is a powerful tool for understanding the behavior of your LLM application, prompt flow tracing capability supports instrumentation for such scenario.\n",
+    "\n",
+    "This notebook will demonstrate how to use prompt flow to instrument and understand your LLM application.\n",
+    "\n",
+    "**Learning Objective** - Upon completion of this notebook, you will be able to:\n",
+    "\n",
+    "- Trace LLM application and visualize with prompt flow."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Requirements\n",
+    "\n",
+    "To run this notebook example, please install required dependencies."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture --no-stderr\n",
+    "%pip install -r ./requirements.txt"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Please configure your API key using an `.env` file, we have provided an example `.env.example` for reference."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# load api key and endpoint from .env to environ\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create your LLM application\n",
+    "\n",
+    "This notebook example will build a LLM application with Azure OpenAI service."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from openai import AzureOpenAI\n",
+    "\n",
+    "# in this notebook example, we will use model \"gpt-35-turbo-16k\"\n",
+    "deployment_name = \"gpt-35-turbo-16k\"\n",
+    "\n",
+    "client = AzureOpenAI(\n",
+    "    azure_deployment=deployment_name,\n",
+    "    api_version=\"2024-02-01\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# prepare one classic question for LLM\n",
+    "conversation = [\n",
+    "    {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
+    "    {\"role\": \"user\", \"content\": \"What is the meaning of life?\"},\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "response = client.chat.completions.create(\n",
+    "    messages=conversation,\n",
+    "    model=deployment_name,\n",
+    ")\n",
+    "print(response.choices[0].message.content)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Start trace using `promptflow.tracing.start_trace` to leverage prompt flow tracing capability; this will print a link to trace UI, where you can visualize the trace."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from promptflow.tracing import start_trace\n",
+    "\n",
+    "# start a trace session, and print a url for user to check trace\n",
+    "start_trace(collection=\"trace-llm\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Run the LLM application again, and you should be able to see new trace logged in the trace UI, and it is clickable to see more details.\n",
+    "\n",
+    "![trace-detail](./media/trace-llm/trace-detail.png)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "response = client.chat.completions.create(\n",
+    "    messages=conversation,\n",
+    "    model=deployment_name,\n",
+    ")\n",
+    "print(response.choices[0].message.content)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Next Steps\n",
+    "\n",
+    "By now you have successfully tracing your LLM application with prompt flow.\n",
+    "\n",
+    "You can check out more examples:\n",
+    "\n",
+    "- [Trace LangChain](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/tracing/langchain/trace-langchain.ipynb): tracing `LangChain` and visualize leveraging prompt flow.\n",
+    "- [Trace AutoGen](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/tracing/autogen-groupchat/trace-autogen-groupchat.ipynb): tracing `AutoGen` and visualize leveraging prompt flow.\n",
+    "- [Trace your flow](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/basic/flex-flow-quickstart.ipynb): using promptflow @trace to structurally tracing your app and do evaluation on it with batch run.\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "build_doc": {
+   "author": [
+    "zhengfeiwang@github.com"
+   ],
+   "category": "local",
+   "section": "Tracing",
+   "weight": 10
+  },
+  "description": "Tracing LLM application",
+  "kernelspec": {
+   "display_name": "pf-dev",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.8"
+  },
+  "resources": ""
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}