microsoft--promptflow/examples/tutorials/tracing/custom-otlp-collector/otlp-trace-collector.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tracing with Custom OpenTelemetry Collector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In certain scenario you might want to user your own [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) and keep your dependency mimimal.\n",
    "\n",
    "In such case you can avoid the dependency of [promptflow-devkit](https://pypi.org/project/promptflow-devkit/) which provides the default collector from promptflow, and only depdent on [promptflow-tracing](https://pypi.org/project/promptflow-tracing), \n",
    "\n",
    "\n",
    "**Learning Objectives** - Upon completing this tutorial, you should be able to:\n",
    "\n",
    "- Trace LLM (OpenAI) Calls using Custom OpenTelemetry Collector.\n",
    "\n",
    "## 0. Install dependent packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install -r ./requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Set up an OpenTelemetry collector\n",
    "\n",
    "Implement a simple collector that print the traces to stdout."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import threading\n",
    "from http.server import BaseHTTPRequestHandler, HTTPServer\n",
    "\n",
    "from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import (\n",
    "    ExportTraceServiceRequest,\n",
    ")\n",
    "\n",
    "\n",
    "class OTLPCollector(BaseHTTPRequestHandler):\n",
    "    def do_POST(self):\n",
    "        content_length = int(self.headers[\"Content-Length\"])\n",
    "        post_data = self.rfile.read(content_length)\n",
    "\n",
    "        traces_request = ExportTraceServiceRequest()\n",
    "        traces_request.ParseFromString(post_data)\n",
    "\n",
    "        print(\"Received a POST request with data:\")\n",
    "        print(traces_request)\n",
    "\n",
    "        self.send_response(200, \"Traces received\")\n",
    "        self.end_headers()\n",
    "        self.wfile.write(b\"Data received and printed to stdout.\\n\")\n",
    "\n",
    "\n",
    "def run_server(port: int):\n",
    "    server_address = (\"\", port)\n",
    "    httpd = HTTPServer(server_address, OTLPCollector)\n",
    "    httpd.serve_forever()\n",
    "\n",
    "\n",
    "def start_server(port: int):\n",
    "    server_thread = threading.Thread(target=run_server, args=(port,))\n",
    "    server_thread.daemon = True\n",
    "    server_thread.start()\n",
    "    print(f\"Server started on port {port}. Access http://localhost:{port}/\")\n",
    "    return server_thread"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# invoke the collector service, serving on OTLP port\n",
    "start_server(port=4318)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Trace your application with tracing\n",
    "Assume we already have a Python function that calls OpenAI API\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llm import my_llm_tool\n",
    "\n",
    "deployment_name = \"gpt-35-turbo-16k\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Call `start_trace()`, and configure the OTLP exporter to above collector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from promptflow.tracing import start_trace\n",
    "\n",
    "start_trace()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from opentelemetry import trace\n",
    "from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter\n",
    "from opentelemetry.sdk.trace.export import BatchSpanProcessor\n",
    "\n",
    "tracer_provider = trace.get_tracer_provider()\n",
    "otlp_span_exporter = OTLPSpanExporter()\n",
    "tracer_provider.add_span_processor(BatchSpanProcessor(otlp_span_exporter))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize traces in the stdout."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = my_llm_tool(\n",
    "    prompt=\"Write a simple Hello, world! python program that displays the greeting message. Output code only.\",\n",
    "    deployment_name=deployment_name,\n",
    ")\n",
    "result\n",
    "# view the traces under this cell"
   ]
  }
 ],
 "metadata": {
  "build_doc": {
   "author": [
    "zhengfeiwang@github.com",
    "wangchao1230@github.com"
   ],
   "category": "local",
   "section": "Tracing",
   "weight": 40
  },
  "description": "A tutorial on how to levarage custom OTLP collector.",
  "kernelspec": {
   "display_name": "tracing-rel",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.17"
  },
  "resources": ""
 },
 "nbformat": 4,
 "nbformat_minor": 2
}