run-llama--llama_index/docs/examples/usecases/fastapi_rag_ollama.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FastAPI + LlamaIndex RAG Example (Ollama)\n",
    "\n",
    "This example demonstrates how to build a simple\n",
    "Retrieval-Augmented Generation (RAG) API using\n",
    "LlamaIndex and FastAPI, powered by a **local LLM via Ollama**.\n",
    "\n",
    "## Features\n",
    "\n",
    "- Uses `llama3` for text generation and `nomic-embed-text` for embeddings via Ollama\n",
    "- Local, free LLM (no API keys required)\n",
    "- Document ingestion and indexing\n",
    "- Simple query API endpoint\n",
    "- Clean, production-style structure\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "- Python 3.9+\n",
    "- Ollama installed locally\n",
    "\n",
    "## Setup\n",
    "\n",
    "Pull a model using Ollama:\n",
    "\n",
    "```bash\n",
    "ollama pull llama3\n",
    "```\n",
    "\n",
    "Install dependencies:\n",
    "\n",
    "```bash\n",
    "pip install -r requirements.txt\n",
    "```\n",
    "\n",
    "## Run\n",
    "\n",
    "```bash\n",
    "uvicorn app:app --reload\n",
    "```\n",
    "\n",
    "Alternatively, you can test the API using FastAPI's built-in Swagger UI:\n",
    "\n",
    "Open your browser and visit:\n",
    "http://localhost:8000/docs\n",
    "\n",
    "## Example Request\n",
    "\n",
    "```bash\n",
    "curl -X POST \"http://localhost:8000/query\" \\\n",
    "-H \"Content-Type: application/json\" \\\n",
    "-d '{\"query\": \"What is this example about?\"}'\n",
    "```\n",
    "\n",
    "## Notes\n",
    "\n",
    "- This example uses a local LLM by default.\n",
    "- It is intended as a minimal, beginner-friendly\n",
    "  demonstration of integrating LlamaIndex with FastAPI.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install fastapi uvicorn llama-index-llms-ollama llama-index-embeddings-ollama ollama"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastapi import FastAPI\n",
    "from pydantic import BaseModel\n",
    "\n",
    "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n",
    "from llama_index.core.settings import Settings\n",
    "from llama_index.llms.ollama import Ollama\n",
    "from llama_index.embeddings.ollama import OllamaEmbedding\n",
    "\n",
    "app = FastAPI(title=\"LlamaIndex FastAPI RAG (Ollama)\")\n",
    "\n",
    "# Configure local LLM and embedding model via Ollama\n",
    "Settings.llm = Ollama(model=\"llama3\")\n",
    "Settings.embed_model = OllamaEmbedding(model_name=\"nomic-embed-text\")\n",
    "\n",
    "# Load documents and build index at startup\n",
    "documents = SimpleDirectoryReader(\"data\").load_data()\n",
    "index = VectorStoreIndex.from_documents(documents)\n",
    "query_engine = index.as_query_engine()\n",
    "\n",
    "\n",
    "class QueryRequest(BaseModel):\n",
    "    query: str\n",
    "\n",
    "\n",
    "@app.post(\"/query\")\n",
    "def query_documents(request: QueryRequest):\n",
    "    # Query indexed documents using a local LLM via Ollama.\n",
    "    response = query_engine.query(request.query)\n",
    "    return {\"response\": str(response)}"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}