run-llama--llama_index/docs/examples/llm/llama_api.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7184eab5",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/llm/llama_api.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e86f9832-5d11-49cc-ab53-f32a1c0913d3",
   "metadata": {},
   "source": [
    "# Llama API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f08e59f-5e26-43c0-bfc9-1b159c5113dc",
   "metadata": {},
   "source": [
    "[Llama API](https://www.llama-api.com/) is a hosted API for Llama 2 with function calling support."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5e3bd05-5ddd-42f3-ae4f-0fc216832873",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2ec5308-a3f6-477e-a3bb-782417fc0e31",
   "metadata": {},
   "source": [
    "To start, go to https://www.llama-api.com/ to obtain an API key"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eee9ede5",
   "metadata": {},
   "source": [
    "If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1cf0fc2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install llama-index-program-openai\n",
    "%pip install llama-index-llms-llama-api"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "128d1fcd",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install llama-index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8ff26e2-9f60-4af4-a5c0-ec8dc8810cc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.llms.llama_api import LlamaAPI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73757c97-a5e1-4397-80a9-988974bcc377",
   "metadata": {},
   "outputs": [],
   "source": [
    "api_key = \"LL-your-key\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80ecc0c9-5a9b-4e4a-9577-4398247e193d",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = LlamaAPI(api_key=api_key)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96604cfd-ad2a-4be9-bc9c-5c3cbb90045b",
   "metadata": {},
   "source": [
    "## Basic Usage"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da886666-80ff-43ed-beb7-2f37b86d3af3",
   "metadata": {},
   "source": [
    "#### Call `complete` with a prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e99f2eb-5d7e-42e5-82f1-f766e477697b",
   "metadata": {},
   "outputs": [],
   "source": [
    "resp = llm.complete(\"Paul Graham is \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f57fcccf-393b-465f-8aca-1d05a79448db",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Paul Graham is a well-known computer scientist and entrepreneur, best known for his work as a co-founder of Viaweb and later Y Combinator, a successful startup accelerator. He is also a prominent essayist and has written extensively on topics such as entrepreneurship, software development, and the tech industry.\n"
     ]
    }
   ],
   "source": [
    "print(resp)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "847f5466-9476-40e6-98fd-4ef7800d4c35",
   "metadata": {},
   "source": [
    "#### Call `chat` with a list of messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82857c48-c0cd-4d39-83ce-1e6c63a951f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core.llms import ChatMessage\n",
    "\n",
    "messages = [\n",
    "    ChatMessage(\n",
    "        role=\"system\", content=\"You are a pirate with a colorful personality\"\n",
    "    ),\n",
    "    ChatMessage(role=\"user\", content=\"What is your name\"),\n",
    "]\n",
    "resp = llm.chat(messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a4797b4-a064-456c-9d6f-dfa7fcce9a76",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "assistant: Arrrr, me hearty! Me name be Captain Blackbeak, the scurviest dog on the seven seas! Yer lookin' fer a swashbucklin' adventure, eh? Well, hoist the sails and set course fer the high seas, matey! I be here to help ye find yer treasure and battle any scurvy dogs who dare cross our path! So, what be yer first question, landlubber?\n"
     ]
    }
   ],
   "source": [
    "print(resp)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78d4e547-b969-4535-b182-dbab0cb72b03",
   "metadata": {},
   "source": [
    "## Function Calling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "019f4155-cbfb-4d32-b2e1-9b4f74e612aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "from llama_index.core.llms.openai_utils import to_openai_function\n",
    "\n",
    "\n",
    "class Song(BaseModel):\n",
    "    \"\"\"A song with name and artist\"\"\"\n",
    "\n",
    "    name: str\n",
    "    artist: str\n",
    "\n",
    "\n",
    "song_fn = to_openai_function(Song)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cacfc181-5e2e-48ac-a891-24c51be4d729",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'name': 'Song', 'arguments': {'name': 'Happy', 'artist': 'Pharrell Williams'}}\n"
     ]
    }
   ],
   "source": [
    "llm = LlamaAPI(api_key=api_key)\n",
    "response = llm.complete(\"Generate a song\", functions=[song_fn])\n",
    "function_call = response.additional_kwargs[\"function_call\"]\n",
    "print(function_call)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8947a026-a841-4969-891b-8cfb699e157f",
   "metadata": {},
   "source": [
    "## Structured Data Extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52b4c825-15ec-468e-b58e-c38cfb2a72f8",
   "metadata": {},
   "source": [
    "This is a simple example of parsing an output into an `Album` schema, which can contain multiple songs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a4736a0-ec0b-4e70-87df-d64e09eeb606",
   "metadata": {},
   "source": [
    "Define output schema"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdc0e374-b19e-44d4-88c4-733b32509b2d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "from typing import List\n",
    "\n",
    "\n",
    "class Song(BaseModel):\n",
    "    \"\"\"Data model for a song.\"\"\"\n",
    "\n",
    "    title: str\n",
    "    length_mins: int\n",
    "\n",
    "\n",
    "class Album(BaseModel):\n",
    "    \"\"\"Data model for an album.\"\"\"\n",
    "\n",
    "    name: str\n",
    "    artist: str\n",
    "    songs: List[Song]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d236876-aa1c-484e-9f18-fcf3a23029f8",
   "metadata": {},
   "source": [
    "Define pydantic program (llama API is OpenAI-compatible)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44527533-8fee-4bc5-b40c-d62042adc7fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.program.openai import OpenAIPydanticProgram\n",
    "\n",
    "prompt_template_str = \"\"\"\\\n",
    "Extract album and songs from the text provided.\n",
    "For each song, make sure to specify the title and the length_mins.\n",
    "{text}\n",
    "\"\"\"\n",
    "\n",
    "llm = LlamaAPI(api_key=api_key, temperature=0.0)\n",
    "\n",
    "program = OpenAIPydanticProgram.from_defaults(\n",
    "    output_cls=Album,\n",
    "    llm=llm,\n",
    "    prompt_template_str=prompt_template_str,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "098ed2cd-796f-4b61-bf21-dbd81cca2cd4",
   "metadata": {},
   "source": [
    "Run program to get structured output.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c25cc6b4-026f-491c-8f46-34836107ba31",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Function call: Album with args: {'name': 'Echoes of Eternity', 'artist': 'Seraphina Rivers', 'songs': [{'title': 'Stardust Serenade', 'length_mins': 6}, {'title': 'Eclipse of the Soul', 'length_mins': 8}, {'title': 'Infinity Embrace', 'length_mins': 10}]}\n"
     ]
    }
   ],
   "source": [
    "output = program(\n",
    "    text=\"\"\"\n",
    "\"Echoes of Eternity\" is a compelling and thought-provoking album, skillfully crafted by the renowned artist, Seraphina Rivers. \\\n",
    "This captivating musical collection takes listeners on an introspective journey, delving into the depths of the human experience \\\n",
    "and the vastness of the universe. With her mesmerizing vocals and poignant songwriting, Seraphina Rivers infuses each track with \\\n",
    "raw emotion and a sense of cosmic wonder. The album features several standout songs, including the hauntingly beautiful \"Stardust \\\n",
    "Serenade,\" a celestial ballad that lasts for six minutes, carrying listeners through a celestial dreamscape. \"Eclipse of the Soul\" \\\n",
    "captivates with its enchanting melodies and spans over eight minutes, inviting introspection and contemplation. Another gem, \"Infinity \\\n",
    "Embrace,\" unfolds like a cosmic odyssey, lasting nearly ten minutes, drawing listeners deeper into its ethereal atmosphere. \"Echoes of Eternity\" \\\n",
    "is a masterful testament to Seraphina Rivers' artistic prowess, leaving an enduring impact on all who embark on this musical voyage through \\\n",
    "time and space.\n",
    "\"\"\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5204a175-735a-4588-b8f8-b7b8d7c143a9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Album(name='Echoes of Eternity', artist='Seraphina Rivers', songs=[Song(title='Stardust Serenade', length_mins=6), Song(title='Eclipse of the Soul', length_mins=8), Song(title='Infinity Embrace', length_mins=10)])"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}