run-llama--llama_index/docs/examples/llm/perplexity.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Perplexity\n",
    "\n",
    "Perplexity's Sonar API offers a solution that combines real-time, grounded web search with advanced reasoning and deep research capabilities. \n",
    "\n",
    "When to use:\n",
    "\n",
    "- When your application requires timely, relevant data directly from the web, such as dynamic content updates or current event tracking.\n",
    "- For products that need to support complex user queries with integrated reasoning and deep research, like digital assistants or advanced search engines.\n",
    "\n",
    "Before we get started, make sure you install `llama_index`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install llama-index-llms-perplexity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install llama-index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initial Setup\n",
    "\n",
    "As of April 12th, 2025 - the following models are supported with the Perplexity LLM class in LLaMa Index:\n",
    "\n",
    "| Model                | Context Length | Model Type      |\n",
    "|----------------------|----------------|-----------------|\n",
    "| `sonar-deep-research`  | 128k           | Chat Completion |\n",
    "| `sonar-reasoning-pro`  | 128k           | Chat Completion |\n",
    "| `sonar-reasoning`      | 128k           | Chat Completion |\n",
    "| `sonar-pro`            | 200k           | Chat Completion |\n",
    "| `sonar`                | 128k           | Chat Completion |\n",
    "| `r1-1776`              | 128k           | Chat Completion |\n",
    "\n",
    "- `sonar-pro` has a max output token limit of 8k.\n",
    "- The reasoning models output Chain of Thought responses.\n",
    "- `r1-1776` is an offline chat model that does not use the Perplexity search subsystem.\n",
    "\n",
    "\n",
    "\n",
    "You can find the latest supported models [here](https://docs.perplexity.ai/docs/model-cards) \\\n",
    "Rate limits are found [here](https://docs.perplexity.ai/docs/rate-limits) \\\n",
    "Pricing can be found [here](https://docs.perplexity.ai/guides/pricing). \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "if \"PPLX_API_KEY\" not in os.environ:\n",
    "    os.environ[\"PPLX_API_KEY\"] = getpass.getpass(\n",
    "        \"Enter your Perplexity API key: \"\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.llms.perplexity import Perplexity\n",
    "\n",
    "PPLX_API_KEY = __import__(\"os\").environ.get(\"PPLX_API_KEY\")\n",
    "\n",
    "llm = Perplexity(api_key=PPLX_API_KEY, model=\"sonar-pro\", temperature=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ChatMessage(role=<MessageRole.SYSTEM: 'system'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='Be precise and concise.')]), ChatMessage(role=<MessageRole.USER: 'user'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='Tell me the latest news about the US Stock Market.')])]\n"
     ]
    }
   ],
   "source": [
    "# Import the ChatMessage class from the llama_index library.\n",
    "from llama_index.core.llms import ChatMessage\n",
    "\n",
    "# Create a list of dictionaries where each dictionary represents a chat message.\n",
    "# Each dictionary contains a 'role' key (e.g., system or user) and a 'content' key with the corresponding message.\n",
    "messages_dict = [\n",
    "    {\"role\": \"system\", \"content\": \"Be precise and concise.\"},\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"Tell me the latest news about the US Stock Market.\",\n",
    "    },\n",
    "]\n",
    "\n",
    "# Convert each dictionary in the list to a ChatMessage object using unpacking (**msg) in a list comprehension.\n",
    "messages = [ChatMessage(**msg) for msg in messages_dict]\n",
    "\n",
    "# Print the list of ChatMessage objects to verify the conversion.\n",
    "print(messages)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "assistant: The latest update on the U.S. stock market indicates a strong performance recently. A significant 10% rally occurred on Wednesday, which contributed substantially to market gains. Additionally, the market closed strongly on Friday, with a 2% increase, ending near the intraday high. This reflects robust momentum, particularly in mega and large-cap growth stocks[1].\n"
     ]
    }
   ],
   "source": [
    "response = llm.chat(messages)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Async Chat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For asynchronous conversation processing, use the `achat` method to send messages and await the response:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "assistant: The U.S. stock market has recently experienced significant gains. A major rally on Wednesday resulted in a 10% surge, contributing substantially to the market's overall upside. Additionally, the market closed strongly on Friday, with a 2% increase, ending near the intraday high. This performance highlights robust momentum, particularly in mega-cap and large-cap growth stocks[1].\n"
     ]
    }
   ],
   "source": [
    "# Asynchronously send the list of chat messages to the LLM using the 'achat' method.\n",
    "# This method returns a ChatResponse object containing the model's answer.\n",
    "response = await llm.achat(messages)\n",
    "\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stream Chat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For cases where you want to receive a response token by token in real time, use the `stream_chat` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The latest news about the U.S. stock market indicates a strong performance recently. The New York Stock Exchange (NYSE) experienced a significant rally, with a 10% surge on Wednesday, followed by a 2% gain on Friday. This upward momentum brought the market near its intraday high, driven by strength in mega-cap and large-cap growth stocks[1]."
     ]
    }
   ],
   "source": [
    "# Call the stream_chat method on the LLM instance, which returns a generator or iterable\n",
    "# for streaming the chat response one delta (token or chunk) at a time.\n",
    "response = llm.stream_chat(messages)\n",
    "\n",
    "# Iterate over each streaming response chunk.\n",
    "for r in response:\n",
    "    # Print the delta (the new chunk of generated text) without adding a newline.\n",
    "    print(r.delta, end=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Async Stream Chat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, for asynchronous streaming, the `astream_chat` method provides a way to process response deltas asynchronously:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The latest updates on the U.S. stock market indicate significant positive momentum. The New York Stock Exchange (NYSE) experienced a strong rally, with a notable 10% surge on Wednesday. This was followed by a 2% gain on Friday, closing near the intraday high. The market's performance has been driven by mega and large-cap growth stocks, contributing to the overall upside[1]."
     ]
    }
   ],
   "source": [
    "# Asynchronously call the astream_chat method on the LLM instance,\n",
    "# which returns an asynchronous generator that yields response chunks.\n",
    "resp = await llm.astream_chat(messages)\n",
    "\n",
    "# Asynchronously iterate over each response chunk from the generator.\n",
    "# For each chunk (delta), print the chunk's text content.\n",
    "async for delta in resp:\n",
    "    print(delta.delta, end=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tool calling \n",
    "\n",
    "Perplexity models can easily be wrapped into a llamaindex tool so that it can be called as part of your data processing or conversational workflows. This tool uses real-time generative search powered by Perplexity, and it’s configured with the updated default model (\"sonar-pro\") and the enable_search_classifier parameter enabled.\n",
    "\n",
    "Below is an example of how to define and register the tool:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core.tools import FunctionTool\n",
    "from llama_index.llms.perplexity import Perplexity\n",
    "from llama_index.core.llms import ChatMessage\n",
    "\n",
    "\n",
    "def query_perplexity(query: str) -> str:\n",
    "    \"\"\"\n",
    "    Queries the Perplexity API via the LlamaIndex integration.\n",
    "\n",
    "    This function instantiates a Perplexity LLM with updated default settings\n",
    "    (using model \"sonar-pro\" and enabling search classifier so that the API can\n",
    "    intelligently decide if a search is needed), wraps the query into a ChatMessage,\n",
    "    and returns the generated response content.\n",
    "    \"\"\"\n",
    "    pplx_api_key = (\n",
    "        \"your-perplexity-api-key\"  # Replace with your actual API key\n",
    "    )\n",
    "\n",
    "    llm = Perplexity(\n",
    "        api_key=pplx_api_key,\n",
    "        model=\"sonar-pro\",\n",
    "        temperature=0.7,\n",
    "        enable_search_classifier=True,  # This will determine if the search component is necessary in this particular context\n",
    "    )\n",
    "\n",
    "    messages = [ChatMessage(role=\"user\", content=query)]\n",
    "    response = llm.chat(messages)\n",
    "    return response.message.content\n",
    "\n",
    "\n",
    "# Create the tool from the query_perplexity function\n",
    "query_perplexity_tool = FunctionTool.from_defaults(fn=query_perplexity)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "quotient",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}