run-llama--llama_index/docs/examples/llm/mymagic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MyMagic AI LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "This notebook demonstrates how to use MyMagicAI for batch inference on massive data stored in cloud buckets. The only enpoints implemented are `complete` and `acomplete` which can work on many use cases including Completion, Summariation and Extraction.\n",
    "To use this notebook, you need an API key (Personal Access Token) from MyMagicAI and data stored in cloud buckets.\n",
    "Sign up by clicking Get Started at [MyMagicAI's website](https://mymagic.ai/) to get your API key.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "To set up your bucket and grant MyMagic API a secure access to your cloud storage, please visit [MyMagic docs](https://docs.mymagic.ai/) for reference.\n",
    "If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install llama-index-llms-mymagic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install llama-index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.llms.mymagic import MyMagicAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = MyMagicAI(\n",
    "    api_key=\"your-api-key\",\n",
    "    storage_provider=\"s3\",  # s3, gcs\n",
    "    bucket_name=\"your-bucket-name\",\n",
    "    session=\"your-session-name\",  # files should be located in this folder on which batch inference will be run\n",
    "    role_arn=\"your-role-arn\",\n",
    "    system_prompt=\"your-system-prompt\",\n",
    "    region=\"your-bucket-region\",\n",
    "    return_output=False,  # Whether you want MyMagic API to return the output json\n",
    "    input_json_file=None,  # name of the input file (stored on the bucket)\n",
    "    list_inputs=None,  # Option to provide inputs as a list in case of small batch\n",
    "    structured_output=None,  # json schema of the output\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: if return_output is set True above, max_tokens should be set to at least 100 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "resp = llm.complete(\n",
    "    question=\"your-question\",\n",
    "    model=\"chhoose-model\",  # currently we support mistral7b, llama7b, mixtral8x7b, codellama70b, llama70b, more to come...\n",
    "    max_tokens=5,  # number of tokens to generate, default is 10\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The response indicated that the final output is stored in your bucket or raises an exception if the job failed\n",
    "print(resp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Asynchronous Requests by using `acomplete` endpoint\n",
    "For asynchronous operations, use the following approach."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import asyncio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "async def main():\n",
    "    response = await llm.acomplete(\n",
    "        question=\"your-question\",\n",
    "        model=\"choose-model\",  # supported models constantly updated and are listed at docs.mymagic.ai\n",
    "        max_tokens=5,  # number of tokens to generate, default is 10\n",
    "    )\n",
    "\n",
    "    print(\"Async completion response:\", response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "await main()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llama-index-dKXgjzWQ-py3.11",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}