run-llama--llama_index/docs/examples/observability/Langfuse-Instrumentation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/observability/Langfuse-Instrumentation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "\n",
    "# Cookbook LlamaIndex Integration (Instrumentation Module)\n",
    "\n",
    "This is a simple cookbook that demonstrates how to use the [LlamaIndex Langfuse integration](https://langfuse.com/docs/integrations/llama-index/get-started) using the [instrumentation module](https://docs.llamaindex.ai/en/stable/module_guides/observability/instrumentation/) by LlamaIndex (available in llama-index v0.10.20 and later)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"position: relative; padding-top: 69.85769728331177%;\">\n",
    "  <iframe\n",
    "    src=\"https://customer-xnej9vqjtgxpafyk.cloudflarestream.com/221d302474945951062d06767c475c0a/iframe?muted=true&loop=true&autoplay=true&poster=https%3A%2F%2Fcustomer-xnej9vqjtgxpafyk.cloudflarestream.com%2F221d302474945951062d06767c475c0a%2Fthumbnails%2Fthumbnail.jpg%3Ftime%3D%26height%3D600\"\n",
    "    loading=\"lazy\"\n",
    "    style=\"border: white; position: absolute; top: 0; left: 0; height: 100%; width: 100%; border-radius: 10px;\"\n",
    "    allow=\"accelerometer; gyroscope; autoplay; encrypted-media; picture-in-picture;\"\n",
    "    allowfullscreen=\"true\"\n",
    "  ></iframe>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Make sure you have both `llama-index` and `langfuse` installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install langfuse llama_index --upgrade"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize the integration. Get your API keys from the Langfuse project settings. This example uses OpenAI for embeddings and chat completions. You can also use any other model supported by LlamaIndex."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Get keys for your project from the project settings page: https://cloud.langfuse.com\n",
    "os.environ[\"LANGFUSE_PUBLIC_KEY\"] = \"\"\n",
    "os.environ[\"LANGFUSE_SECRET_KEY\"] = \"\"\n",
    "os.environ[\"LANGFUSE_HOST\"] = \"https://cloud.langfuse.com\"  # 🇪🇺 EU region\n",
    "# os.environ[\"LANGFUSE_HOST\"] = \"https://us.cloud.langfuse.com\" # 🇺🇸 US region\n",
    "\n",
    "# Your openai key\n",
    "os.environ[\"OPENAI_API_KEY\"] = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Register the Langfuse `LlamaIndexInstrumentor`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langfuse.llama_index import LlamaIndexInstrumentor\n",
    "\n",
    "instrumentor = LlamaIndexInstrumentor()\n",
    "instrumentor.start()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example context, thx ChatGPT\n",
    "from llama_index.core import Document\n",
    "\n",
    "doc1 = Document(\n",
    "    text=\"\"\"\n",
    "Maxwell \"Max\" Silverstein, a lauded movie director, screenwriter, and producer, was born on October 25, 1978, in Boston, Massachusetts. A film enthusiast from a young age, his journey began with home movies shot on a Super 8 camera. His passion led him to the University of Southern California (USC), majoring in Film Production. Eventually, he started his career as an assistant director at Paramount Pictures. Silverstein's directorial debut, “Doors Unseen,” a psychological thriller, earned him recognition at the Sundance Film Festival and marked the beginning of a successful directing career.\n",
    "\"\"\"\n",
    ")\n",
    "doc2 = Document(\n",
    "    text=\"\"\"\n",
    "Throughout his career, Silverstein has been celebrated for his diverse range of filmography and unique narrative technique. He masterfully blends suspense, human emotion, and subtle humor in his storylines. Among his notable works are \"Fleeting Echoes,\" \"Halcyon Dusk,\" and the Academy Award-winning sci-fi epic, \"Event Horizon's Brink.\" His contribution to cinema revolves around examining human nature, the complexity of relationships, and probing reality and perception. Off-camera, he is a dedicated philanthropist living in Los Angeles with his wife and two children.\n",
    "\"\"\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example index construction + LLM query\n",
    "from llama_index.core import VectorStoreIndex\n",
    "\n",
    "index = VectorStoreIndex.from_documents([doc1, doc2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "He made home movies using a Super 8 camera.\n"
     ]
    }
   ],
   "source": [
    "# Query\n",
    "response = index.as_query_engine().query(\"What did he do growing up?\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example trace: https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/d933c7cc-20bf-4db3-810d-bab1c8d9a2a1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "He made home movies using a Super 8 camera growing up.\n"
     ]
    }
   ],
   "source": [
    "# Chat\n",
    "response = index.as_chat_engine().chat(\"What did he do growing up?\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example trace: https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/4e285b8f-9789-4cf0-a8b4-45473ac420f1\n",
    "\n",
    "![LlamaIndex Chat Engine Trace in Langfuse (via instrumentation module)](https://langfuse.com/images/cookbook/integration_llama-index_instrumentation_chatengine_trace.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom trace properties"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use the `instrumentor.observe` context manager to manage trace IDs, set custom trace properties, and access the trace client for later scoring."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with instrumentor.observe(user_id=\"my-user\", session_id=\"my-session\") as trace:\n",
    "    response = index.as_query_engine().query(\"What did he do growing up?\")\n",
    "\n",
    "# Use the trace client yielded by the context manager for e.g. scoring:\n",
    "trace.score(name=\"my-score\", value=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example trace: https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/6f554d6b-a2bc-4fba-904f-aa54de2897ca"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}