comet-ml--opik/apps/opik-documentation/documentation/docs/cookbook/agentspec.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Opik with Agent Spec\n",
    "\n",
    "[Agent Spec](https://oracle.github.io/agent-spec/development/agentspec/index.html) is a portable configuration language for defining agentic systems (agents, tools, and structured workflows).\n",
    "\n",
    "In this notebook, we will build a simple Agent Spec agent and use Opik's `AgentSpecInstrumentor` to capture a trace of the agent's tool and LLM execution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating an account on Comet.com\n",
    "\n",
    "[Comet](https://www.comet.com/site?from=llm&utm_source=opik&utm_medium=colab&utm_content=agentspec&utm_campaign=opik) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm&utm_source=opik&utm_medium=colab&utm_content=agentspec&utm_campaign=opik) and grab your API Key.\n",
    "\n",
    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/overview/?from=llm&utm_source=opik&utm_medium=colab&utm_content=agentspec&utm_campaign=opik) for more information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "%pip install --upgrade opik \"pyagentspec[langgraph]\" opentelemetry-sdk opentelemetry-instrumentation"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import opik\n",
    "\n",
    "opik.configure(use_local=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing our environment\n",
    "\n",
    "This demo uses OpenAI as the LLM provider. Set your OpenAI API key as an environment variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import getpass\n",
    "\n",
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define an Agent Spec agent\n",
    "\n",
    "We'll define a small calculator agent with a couple of tools:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyagentspec.agent import Agent\n",
    "from pyagentspec.llms import OpenAiConfig\n",
    "from pyagentspec.property import FloatProperty\n",
    "from pyagentspec.tools import ServerTool\n",
    "\n",
    "\n",
    "def build_agentspec_agent() -> Agent:\n",
    "    tools = [\n",
    "        ServerTool(\n",
    "            name=\"sum\",\n",
    "            description=\"Sum two numbers\",\n",
    "            inputs=[FloatProperty(title=\"a\"), FloatProperty(title=\"b\")],\n",
    "            outputs=[FloatProperty(title=\"result\")],\n",
    "        ),\n",
    "        ServerTool(\n",
    "            name=\"subtract\",\n",
    "            description=\"Subtract two numbers\",\n",
    "            inputs=[FloatProperty(title=\"a\"), FloatProperty(title=\"b\")],\n",
    "            outputs=[FloatProperty(title=\"result\")],\n",
    "        ),\n",
    "    ]\n",
    "\n",
    "    return Agent(\n",
    "        name=\"calculator_agent\",\n",
    "        description=\"An agent that provides assistance with tool use.\",\n",
    "        llm_config=OpenAiConfig(name=\"openai-gpt-5-mini\", model_id=\"gpt-5-mini\"),\n",
    "        system_prompt=(\n",
    "            \"You are a helpful calculator agent.\\n\"\n",
    "            \"Your duty is to compute the result of the given operation using tools, \"\n",
    "            \"and to output the result.\\n\"\n",
    "            \"It's important that you reply with the result only.\\n\"\n",
    "        ),\n",
    "        tools=tools,\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the agent with Opik tracing enabled\n",
    "\n",
    "Wrap the agent execution in `AgentSpecInstrumentor().instrument_context(...)` to capture traces in Opik.\n",
    "\n",
    "> Agent traces can include prompts, tool inputs/outputs, and messages. If you need to avoid logging sensitive information, set `mask_sensitive_information=True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from opik.integrations.agentspec import AgentSpecInstrumentor\n",
    "from pyagentspec.adapters.langgraph import AgentSpecLoader\n",
    "\n",
    "agent = build_agentspec_agent()\n",
    "\n",
    "tool_registry = {\n",
    "    \"sum\": lambda a, b: a + b,\n",
    "    \"subtract\": lambda a, b: a - b,\n",
    "}\n",
    "\n",
    "langgraph_agent = AgentSpecLoader(tool_registry=tool_registry).load_component(agent)\n",
    "\n",
    "with AgentSpecInstrumentor().instrument_context(\n",
    "    project_name=\"agentspec-demo\",\n",
    "    mask_sensitive_information=False,\n",
    "):\n",
    "    messages = []\n",
    "\n",
    "    messages.append({\"role\": \"user\", \"content\": \"Compute 13.5 + 2.25 using the sum tool.\"})\n",
    "    response = langgraph_agent.invoke(\n",
    "        input={\"messages\": messages},\n",
    "        config={\"configurable\": {\"thread_id\": \"1\"}},\n",
    "    )\n",
    "    agent_answer = response[\"messages\"][-1].content.strip()\n",
    "    print(\"AGENT >>>\", agent_answer)\n",
    "    messages.append({\"role\": \"assistant\", \"content\": agent_answer})\n",
    "\n",
    "    messages.append({\"role\": \"user\", \"content\": \"Now compute 10 - 3.5 using the subtract tool.\"})\n",
    "    response = langgraph_agent.invoke(\n",
    "        input={\"messages\": messages},\n",
    "        config={\"configurable\": {\"thread_id\": \"1\"}},\n",
    "    )\n",
    "    agent_answer = response[\"messages\"][-1].content.strip()\n",
    "    print(\"AGENT >>>\", agent_answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After running the cell above, open Opik and navigate to the `agentspec-demo` project to inspect the trace tree and debug tool usage and LLM generations."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py312_llm_eval",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}