run-llama--llama_index/docs/examples/llm/deepinfra.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3e5adfb5c737033",
   "metadata": {},
   "outputs": [],
   "source": [
    "<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/llm/deepinfra.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "introduction",
   "metadata": {},
   "source": [
    "# DeepInfra\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "installation",
   "metadata": {},
   "source": [
    "## Installation\n",
    "\n",
    "First, install the necessary package:\n",
    "\n",
    "```bash\n",
    "%pip install llama-index-llms-deepinfra\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "installation-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install llama-index-llms-deepinfra"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "initialization",
   "metadata": {},
   "source": [
    "## Initialization\n",
    "\n",
    "Set up the `DeepInfraLLM` class with your API key and desired parameters:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "initialization-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.llms.deepinfra import DeepInfraLLM\n",
    "import asyncio\n",
    "\n",
    "llm = DeepInfraLLM(\n",
    "    model=\"mistralai/Mixtral-8x22B-Instruct-v0.1\",  # Default model name\n",
    "    api_key=\"your-deepinfra-api-key\",  # Replace with your DeepInfra API key\n",
    "    temperature=0.5,\n",
    "    max_tokens=50,\n",
    "    additional_kwargs={\"top_p\": 0.9},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sync-complete",
   "metadata": {},
   "source": [
    "## Synchronous Complete\n",
    "\n",
    "Generate a text completion synchronously using the `complete` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sync-complete-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = llm.complete(\"Hello World!\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sync-stream-complete",
   "metadata": {},
   "source": [
    "## Synchronous Stream Complete\n",
    "\n",
    "Generate a streaming text completion synchronously using the `stream_complete` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sync-stream-complete-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "content = \"\"\n",
    "for completion in llm.stream_complete(\"Once upon a time\"):\n",
    "    content += completion.delta\n",
    "    print(completion.delta, end=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sync-chat",
   "metadata": {},
   "source": [
    "## Synchronous Chat\n",
    "\n",
    "Generate a chat response synchronously using the `chat` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sync-chat-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core.base.llms.types import ChatMessage\n",
    "\n",
    "messages = [\n",
    "    ChatMessage(role=\"user\", content=\"Tell me a joke.\"),\n",
    "]\n",
    "chat_response = llm.chat(messages)\n",
    "print(chat_response.message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sync-stream-chat",
   "metadata": {},
   "source": [
    "## Synchronous Stream Chat\n",
    "\n",
    "Generate a streaming chat response synchronously using the `stream_chat` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sync-stream-chat-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = [\n",
    "    ChatMessage(role=\"system\", content=\"You are a helpful assistant.\"),\n",
    "    ChatMessage(role=\"user\", content=\"Tell me a story.\"),\n",
    "]\n",
    "content = \"\"\n",
    "for chat_response in llm.stream_chat(messages):\n",
    "    content += chat_response.message.delta\n",
    "    print(chat_response.message.delta, end=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "async-complete",
   "metadata": {},
   "source": [
    "## Asynchronous Complete\n",
    "\n",
    "Generate a text completion asynchronously using the `acomplete` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "async-complete-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "async def async_complete():\n",
    "    response = await llm.acomplete(\"Hello Async World!\")\n",
    "    print(response.text)\n",
    "\n",
    "\n",
    "asyncio.run(async_complete())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "async-stream-complete",
   "metadata": {},
   "source": [
    "## Asynchronous Stream Complete\n",
    "\n",
    "Generate a streaming text completion asynchronously using the `astream_complete` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "async-stream-complete-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "async def async_stream_complete():\n",
    "    content = \"\"\n",
    "    response = await llm.astream_complete(\"Once upon an async time\")\n",
    "    async for completion in response:\n",
    "        content += completion.delta\n",
    "        print(completion.delta, end=\"\")\n",
    "\n",
    "\n",
    "asyncio.run(async_stream_complete())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "async-chat",
   "metadata": {},
   "source": [
    "## Asynchronous Chat\n",
    "\n",
    "Generate a chat response asynchronously using the `achat` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "async-chat-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "async def async_chat():\n",
    "    messages = [\n",
    "        ChatMessage(role=\"user\", content=\"Tell me an async joke.\"),\n",
    "    ]\n",
    "    chat_response = await llm.achat(messages)\n",
    "    print(chat_response.message.content)\n",
    "\n",
    "\n",
    "asyncio.run(async_chat())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "async-stream-chat",
   "metadata": {},
   "source": [
    "## Asynchronous Stream Chat\n",
    "\n",
    "Generate a streaming chat response asynchronously using the `astream_chat` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "async-stream-chat-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "async def async_stream_chat():\n",
    "    messages = [\n",
    "        ChatMessage(role=\"system\", content=\"You are a helpful assistant.\"),\n",
    "        ChatMessage(role=\"user\", content=\"Tell me an async story.\"),\n",
    "    ]\n",
    "    content = \"\"\n",
    "    response = await llm.astream_chat(messages)\n",
    "    async for chat_response in response:\n",
    "        content += chat_response.message.delta\n",
    "        print(chat_response.message.delta, end=\"\")\n",
    "\n",
    "\n",
    "asyncio.run(async_stream_chat())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34ddb88cd45521a9",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "For any questions or feedback, please contact us at [feedback@deepinfra.com](mailto:feedback@deepinfra.com).\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}