comet-ml--opik/apps/opik-documentation/documentation/docs/cookbook/instructor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Opik with Instructor\n",
    "\n",
    "[Instructor](https://github.com/instructor-ai/instructor) is a Python library for working with structured outputs for LLMs built on top of Pydantic. It provides a simple way to manage schema validations, retries and streaming responses."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating an account on Comet.com\n",
    "\n",
    "[Comet](https://www.comet.com/site?from=llm&utm_source=opik&utm_medium=colab&utm_content=haystack&utm_campaign=opik) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm&utm_source=opik&utm_medium=colab&utm_content=haystack&utm_campaign=opik) and grab your API Key.\n",
    "\n",
    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/overview/?from=llm&utm_source=opik&utm_medium=colab&utm_content=haystack&utm_campaign=opik) for more information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet opik instructor anthropic google-generativeai google-genai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import getpass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Opik Config\n",
    "\n",
    "Configure your development environment (If you click the key icon on the left side, you can set API keys that are reusable across notebooks.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import opik\n",
    "\n",
    "opik.configure(use_local=False)\n",
    "\n",
    "os.environ[\"OPIK_PROJECT_NAME\"] = \"opik-cookbook-instructor\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this demo we are going to use an OpenAI, Anthropic and Gemini, so we will need to configure our API keys:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")\n",
    "\n",
    "if \"ANTHROPIC_API_KEY\" not in os.environ:\n",
    "    os.environ[\"ANTHROPIC_API_KEY\"] = getpass.getpass(\"Enter your Anthropic API key: \")\n",
    "\n",
    "if \"GOOGLE_API_KEY\" not in os.environ:\n",
    "    os.environ[\"GOOGLE_API_KEY\"] = getpass.getpass(\"Enter your Google API key: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Opik with Instructor library\n",
    "\n",
    "In order to log traces from Instructor into Opik, we are going to patch the `instructor` library. This will log each LLM call to the Opik platform.\n",
    "\n",
    "For all the integrations, we will first add tracking to the LLM client and then pass it to the Instructor library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from opik.integrations.openai import track_openai\n",
    "import instructor\n",
    "from pydantic import BaseModel\n",
    "from openai import OpenAI\n",
    "\n",
    "\n",
    "# We will first create the OpenAI client and add the `track_openai`\n",
    "# method to log data to Opik\n",
    "openai_client = track_openai(OpenAI())\n",
    "\n",
    "# Patch the OpenAI client for Instructor\n",
    "client = instructor.from_openai(openai_client)\n",
    "\n",
    "\n",
    "# Define your desired output structure\n",
    "class UserInfo(BaseModel):\n",
    "    name: str\n",
    "    age: int\n",
    "\n",
    "\n",
    "user_info = client.chat.completions.create(\n",
    "    model=\"gpt-4o-mini\",\n",
    "    response_model=UserInfo,\n",
    "    messages=[{\"role\": \"user\", \"content\": \"John Doe is 30 years old.\"}],\n",
    ")\n",
    "\n",
    "print(user_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thanks to the `track_openai` method, all the calls made to OpenAI will be logged to the Opik platform. This approach also works well if you are also using the `opik.track` decorator as it will automatically log the LLM call made with Instructor to the relevant trace.\n",
    "\n",
    "![Trace screenshot](https://raw.githubusercontent.com/comet-ml/opik/main/apps/opik-documentation/documentation/fern/img/cookbook/instructor_cookbook.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Integrating with other LLM providers\n",
    "\n",
    "The instructor library supports many LLM providers beyond OpenAI, including: Anthropic, AWS Bedrock, Gemini, etc. Opik supports the majority of these providers aswell.\n",
    "\n",
    "Here are two additional code snippets needed for the integration."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Anthropic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from opik.integrations.anthropic import track_anthropic\n",
    "import instructor\n",
    "from anthropic import Anthropic\n",
    "\n",
    "# Add Opik tracking\n",
    "anthropic_client = track_anthropic(Anthropic())\n",
    "\n",
    "# Patch the Anthropic client for Instructor\n",
    "client = instructor.from_anthropic(\n",
    "    anthropic_client, mode=instructor.Mode.ANTHROPIC_JSON\n",
    ")\n",
    "\n",
    "user_info = client.chat.completions.create(\n",
    "    model=\"claude-3-5-sonnet-20241022\",\n",
    "    response_model=UserInfo,\n",
    "    messages=[{\"role\": \"user\", \"content\": \"John Doe is 30 years old.\"}],\n",
    "    max_tokens=1000,\n",
    ")\n",
    "\n",
    "print(user_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gemini"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from opik.integrations.genai import track_genai\n",
    "import instructor\n",
    "from google import genai\n",
    "\n",
    "# Add Opik tracking\n",
    "gemini_client = track_genai(genai.Client())\n",
    "\n",
    "# Patch the GenAI client for Instructor\n",
    "client = instructor.from_genai(\n",
    "    gemini_client, mode=instructor.Mode.GENAI_STRUCTURED_OUTPUTS\n",
    ")\n",
    "\n",
    "user_info = client.chat.completions.create(\n",
    "    model=\"gemini-2.0-flash-001\",\n",
    "    response_model=UserInfo,\n",
    "    messages=[{\"role\": \"user\", \"content\": \"John Doe is 30 years old.\"}],\n",
    ")\n",
    "\n",
    "print(user_info)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py312_llm_eval",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}