run-llama--llama_index/docs/examples/data_connectors/deplot/DeplotReader.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9693dcaf-edc6-46e6-9371-f2571a4a1b3c",
   "metadata": {},
   "source": [
    "# Deplot Reader Demo\n",
    "\n",
    "In this notebook we showcase the capabilities of our ImageTabularChartReader, which is powered by the DePlot model https://arxiv.org/abs/2212.10505."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f25d9c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install llama-index-readers-file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17bc9aca-456b-4e0a-95cc-bcaa70ffb833",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.readers.file import ImageTabularChartReader\n",
    "from llama_index.core import SummaryIndex\n",
    "from llama_index.core.response.notebook_utils import display_response\n",
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08c00e87-81d0-462e-9b35-4149c9492983",
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = ImageTabularChartReader(keep_image=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a329200-bf9b-43d4-a5a2-8255cd3b4db3",
   "metadata": {},
   "source": [
    "## Load Protected Waters Chart\n",
    "\n",
    "This chart shows the percentage of marine territorial waters that are protected for each country."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb74c5f3-439e-440e-8955-eac5dbdb2a9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "documents = loader.load_data(file=Path(\"./marine_chart.png\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73b26b5a-5d95-4e8d-8ebd-565085df7be2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Figure or chart with tabular data: Country | Share of marine territorial waters that are protected, 2016 <0x0A> Greenland | 4.52 <0x0A> Mauritania | 4.15 <0x0A> Indonesia | 2.88 <0x0A> Ireland | 2.33\n"
     ]
    }
   ],
   "source": [
    "print(documents[0].text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "225a1330-addf-4465-b368-23e62afe4d65",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIConnectionError: Error communicating with OpenAI: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')).\n"
     ]
    }
   ],
   "source": [
    "summary_index = SummaryIndex.from_documents(documents)\n",
    "response = summary_index.as_query_engine().query(\n",
    "    \"What is the difference between the shares of Greenland and the share of\"\n",
    "    \" Mauritania?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed97789d-8907-40ed-86f8-309c2634a090",
   "metadata": {},
   "outputs": [],
   "source": [
    "display_response(response, show_source=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81a66b05-f1f9-4a6f-80ce-b661688a6f21",
   "metadata": {},
   "source": [
    "## Load Pew Research Chart\n",
    "\n",
    "Here we load in a Pew Research chart showing international views of the US/Biden.\n",
    "\n",
    "Source: https://www.pewresearch.org/global/2023/06/27/international-views-of-biden-and-u-s-largely-positive/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64835171-5e0d-4e86-a7d3-63e5710a18d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "documents = loader.load_data(file=Path(\"./pew1.png\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9cc2b5b3-e413-45a9-8980-699701371251",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Figure or chart with tabular data: Entity | Values <0x0A> Does not | 50.0 <0x0A> % who say the U.S take into account the interests of countries like theirs | 49.0 <0x0A> Does not | 38.0 <0x0A> % who say the U.S contribute to peace and stability around the world | 61.0 <0x0A> Does not | 15.0 <0x0A> % who say the U.S interfere in the affairs of other countries | 15.0 <0x0A>% who have confidence | 54.0 <0x0A> Views of President Biden | 30.0 <0x0A> Favorable | 59.0 <0x0A> Views of the U.S. | 9.0\n"
     ]
    }
   ],
   "source": [
    "print(documents[0].text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71574aae-df92-49a5-b251-e3bab4b8de3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "summary_index = SummaryIndex.from_documents(documents)\n",
    "response = summary_index.as_query_engine().query(\n",
    "    \"What percentage says that the US contributes to peace and stability?\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a5a5afd-af6c-4781-9ea5-82d206797cbb",
   "metadata": {},
   "outputs": [],
   "source": [
    "display_response(response, show_source=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llama_index_v2",
   "language": "python",
   "name": "llama_index_v2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}