microsoft--promptflow/examples/tutorials/tracing/langchain/trace-langchain.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tracing with LangChain apps"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The tracing capability provided by Prompt flow is built on top of [OpenTelemetry](https://opentelemetry.io/) that gives you complete observability over your LLM applications. \n",
    "And there is already a rich set of OpenTelemetry [instrumentation packages](https://opentelemetry.io/ecosystem/registry/?language=python&component=instrumentation) available in OpenTelemetry Eco System. \n",
    "\n",
    "In this example we will demo how to use [opentelemetry-instrumentation-langchain](https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-langchain) package provided by [Traceloop](https://www.traceloop.com/) to instrument [LangChain](https://python.langchain.com/docs/tutorials/) apps.\n",
    "\n",
    "\n",
    "**Learning Objectives** - Upon completing this tutorial, you should be able to:\n",
    "\n",
    "- Trace `LangChain` applications and visualize the trace of your application in prompt flow.\n",
    "\n",
    "## Requirements\n",
    "\n",
    "To run this notebook example, please install required dependencies:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install -r ./requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start tracing LangChain using promptflow\n",
    "\n",
    "Start trace using `promptflow.start_trace`, click the printed url to view the trace ui."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from promptflow.tracing import start_trace\n",
    "\n",
    "# start a trace session, and print a url for user to check trace\n",
    "start_trace()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, `opentelemetry-instrumentation-langchain` instrumentation logs prompts, completions, and embeddings to span attributes. This gives you a clear visibility into how your LLM application is working, and can make it easy to debug and evaluate the quality of the outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# enable langchain instrumentation\n",
    "from opentelemetry.instrumentation.langchain import LangchainInstrumentor\n",
    "\n",
    "instrumentor = LangchainInstrumentor()\n",
    "if not instrumentor.is_instrumented_by_opentelemetry:\n",
    "    instrumentor.instrument()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run a simple LangChain\n",
    "\n",
    "Below is an example targeting an AzureOpenAI resource. Please configure you `API_KEY` using an `.env` file, see `../.env.example`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "from langchain.chat_models import AzureChatOpenAI\n",
    "from langchain.prompts.chat import ChatPromptTemplate\n",
    "from langchain.chains import LLMChain\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "if \"AZURE_OPENAI_API_KEY\" not in os.environ:\n",
    "    # load environment variables from .env file\n",
    "    load_dotenv()\n",
    "\n",
    "llm = AzureChatOpenAI(\n",
    "    deployment_name=os.environ[\"CHAT_DEPLOYMENT_NAME\"],\n",
    "    openai_api_key=os.environ[\"AZURE_OPENAI_API_KEY\"],\n",
    "    azure_endpoint=os.environ[\"AZURE_OPENAI_ENDPOINT\"],\n",
    "    openai_api_type=\"azure\",\n",
    "    openai_api_version=\"2023-07-01-preview\",\n",
    "    temperature=0,\n",
    ")\n",
    "\n",
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", \"You are world class technical documentation writer.\"),\n",
    "        (\"user\", \"{input}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "chain = LLMChain(llm=llm, prompt=prompt, output_key=\"metrics\")\n",
    "chain({\"input\": \"What is ChatGPT?\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should be able to see traces of the chain in promptflow UI now. Check the cell with `start_trace` on the trace UI url."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "By now you've successfully tracing LLM calls in your app using prompt flow.\n",
    "\n",
    "You can check out more examples:\n",
    "- [Trace your flow](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/basic/flex-flow-quickstart.ipynb): using promptflow @trace to structurally tracing your app and do evaluation on it with batch run."
   ]
  }
 ],
 "metadata": {
  "build_doc": {
   "author": [
    "zhengfeiwang@github.com",
    "wangchao1230@github.com"
   ],
   "category": "local",
   "section": "Tracing",
   "weight": 30
  },
  "description": "Tracing LLM calls in langchain application",
  "kernelspec": {
   "display_name": "prompt_flow",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.17"
  },
  "resources": ""
 },
 "nbformat": 4,
 "nbformat_minor": 2
}