wehub-resource-sync
a0c8464e58
Build Package / build (ubuntu-latest) (push) Failing after 1s
CodeQL / Analyze (python) (push) Failing after 1s
Core Typecheck / core-typecheck (push) Failing after 1s
Linting / lint (push) Failing after 1s
llama-dev tests / test-llama-dev (push) Failing after 1s
Publish Sub-Package to PyPI if Needed / publish_subpackage_if_needed (push) Has been skipped
Sync Docs to Developer Hub / sync-docs (push) Failing after 0s
Build Package / build (windows-latest) (push) Has been cancelled
454 lines
19 KiB
Plaintext
454 lines
19 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/query_engine/RetrieverRouterQueryEngine.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Retriever Router Query Engine\n",
|
|
"In this tutorial, we define a router query engine based on a retriever. The retriever will select a set of nodes, and we will in turn select the right QueryEngine.\n",
|
|
"\n",
|
|
"We use our new `ToolRetrieverRouterQueryEngine` class for this! "
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Setup"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install llama-index"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# NOTE: This is ONLY necessary in jupyter notebook.\n",
|
|
"# Details: Jupyter runs an event-loop behind the scenes.\n",
|
|
"# This results in nested event-loops when we start an event-loop to make async queries.\n",
|
|
"# This is normally not allowed, we use nest_asyncio to allow it for convenience.\n",
|
|
"import nest_asyncio\n",
|
|
"\n",
|
|
"nest_asyncio.apply()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"INFO:numexpr.utils:Note: NumExpr detected 12 cores but \"NUMEXPR_MAX_THREADS\" not set, so enforcing safe limit of 8.\n",
|
|
"Note: NumExpr detected 12 cores but \"NUMEXPR_MAX_THREADS\" not set, so enforcing safe limit of 8.\n",
|
|
"INFO:numexpr.utils:NumExpr defaulting to 8 threads.\n",
|
|
"NumExpr defaulting to 8 threads.\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/jerryliu/Programming/gpt_index/.venv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
|
|
" from .autonotebook import tqdm as notebook_tqdm\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import logging\n",
|
|
"import sys\n",
|
|
"\n",
|
|
"logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n",
|
|
"logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n",
|
|
"\n",
|
|
"from llama_index.core import (\n",
|
|
" VectorStoreIndex,\n",
|
|
" SimpleDirectoryReader,\n",
|
|
" StorageContext,\n",
|
|
")\n",
|
|
"from llama_index.core import SummaryIndex"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Download Data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!mkdir -p 'data/paul_graham/'\n",
|
|
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Load Data\n",
|
|
"\n",
|
|
"We first show how to convert a Document into a set of Nodes, and insert into a DocumentStore."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# load documents\n",
|
|
"documents = SimpleDirectoryReader(\"./data/paul_graham\").load_data()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from llama_index.core import Settings\n",
|
|
"\n",
|
|
"# initialize settings (set chunk size)\n",
|
|
"Settings.chunk_size = 1024\n",
|
|
"nodes = Settings.node_parser.get_nodes_from_documents(documents)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# initialize storage context (by default it's in-memory)\n",
|
|
"storage_context = StorageContext.from_defaults()\n",
|
|
"storage_context.docstore.add_documents(nodes)"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Define Summary Index and Vector Index over Same Data "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
|
|
"> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 0 tokens\n",
|
|
"> [build_index_from_nodes] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
|
|
"> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 17038 tokens\n",
|
|
"> [build_index_from_nodes] Total embedding token usage: 17038 tokens\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"summary_index = SummaryIndex(nodes, storage_context=storage_context)\n",
|
|
"vector_index = VectorStoreIndex(nodes, storage_context=storage_context)"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Define Query Engine and Tool for these Indices\n",
|
|
"\n",
|
|
"We define a Query Engine for each Index. We then wrap these with our `QueryEngineTool`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from llama_index.core.tools import QueryEngineTool\n",
|
|
"\n",
|
|
"list_query_engine = summary_index.as_query_engine(\n",
|
|
" response_mode=\"tree_summarize\", use_async=True\n",
|
|
")\n",
|
|
"vector_query_engine = vector_index.as_query_engine(\n",
|
|
" response_mode=\"tree_summarize\", use_async=True\n",
|
|
")\n",
|
|
"\n",
|
|
"list_tool = QueryEngineTool.from_defaults(\n",
|
|
" query_engine=list_query_engine,\n",
|
|
" description=\"Useful for questions asking for a biography of the author.\",\n",
|
|
")\n",
|
|
"vector_tool = QueryEngineTool.from_defaults(\n",
|
|
" query_engine=vector_query_engine,\n",
|
|
" description=(\n",
|
|
" \"Useful for retrieving specific snippets from the author's life, like\"\n",
|
|
" \" his time in college, his time in YC, or more.\"\n",
|
|
" ),\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Define Retrieval-Augmented Router Query Engine\n",
|
|
"\n",
|
|
"We define a router query engine that's augmented with a retrieval mechanism, to help deal with the case when the set of choices is too large. \n",
|
|
"\n",
|
|
"To do this, we first define an `ObjectIndex` over the set of query engine tools. The `ObjectIndex` is defined an underlying index data structure (e.g. a vector index, keyword index), and can serialize QueryEngineTool objects to/from our indices.\n",
|
|
"\n",
|
|
"We then use our `ToolRetrieverRouterQueryEngine` class, and pass in an `ObjectRetriever` over `QueryEngineTool` objects.\n",
|
|
"The `ObjectRetriever` corresponds to our `ObjectIndex`. \n",
|
|
"\n",
|
|
"This retriever can then dyamically retrieve the relevant query engines during query-time. This allows us to pass in an arbitrary number of query engine tools without worrying about prompt limitations. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
|
|
"> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 59 tokens\n",
|
|
"> [build_index_from_nodes] Total embedding token usage: 59 tokens\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from llama_index.core import VectorStoreIndex\n",
|
|
"from llama_index.core.objects import ObjectIndex\n",
|
|
"\n",
|
|
"obj_index = ObjectIndex.from_objects(\n",
|
|
" [list_tool, vector_tool],\n",
|
|
" index_cls=VectorStoreIndex,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from llama_index.core.query_engine import ToolRetrieverRouterQueryEngine\n",
|
|
"\n",
|
|
"query_engine = ToolRetrieverRouterQueryEngine(obj_index.as_retriever())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 10 tokens\n",
|
|
"> [retrieve] Total embedding token usage: 10 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 0 tokens\n",
|
|
"> [retrieve] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 2111 tokens\n",
|
|
"> [get_response] Total LLM token usage: 2111 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 0 tokens\n",
|
|
"> [retrieve] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 2148 tokens\n",
|
|
"> [get_response] Total LLM token usage: 2148 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.query_engine.router_query_engine:Combining responses from multiple query engines.\n",
|
|
"Combining responses from multiple query engines.\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1063 tokens\n",
|
|
"> [get_response] Total LLM token usage: 1063 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"> [get_response] Total embedding token usage: 0 tokens\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"response = query_engine.query(\"What is a biography of the author's life?\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"The author is a creative person who has had a varied and interesting life. They grew up in the US and went to college, but then decided to take a break and pursue their passion for art. They applied to two art schools, RISD in the US and the Accademia di Belli Arti in Florence, and were accepted to both. They chose to go to Florence, where they took the entrance exam and passed. They then spent a year living in Florence, studying art at the Accademia and painting still lives in their bedroom. After their year in Florence, the author returned to the US and completed their BFA program at RISD. They then went on to pursue a PhD in computer science at MIT, where they wrote a dissertation on the evolution of computers. During their time at MIT, they also did consulting work and wrote essays on topics they had been thinking about. After completing their PhD, the author started a software company, Viaweb, which was eventually acquired by Yahoo. They then went on to write essays and articles about their experiences in the tech industry. They also wrote an essay about how to choose what to work on, which was based on their own experience. The author then moved back to Florence, where they found a rent-stabilized apartment and continued to pursue their interest in art. They wrote about their experiences in the art world, and experienced the reactions of readers to their essays. The author is now a successful writer and continues to write essays and articles about topics they are passionate about. \n",
|
|
"\n",
|
|
"In summary, the author's life has been a journey of exploration and creativity. They have experienced a wide range of different things in their life, from art school to computer science to the tech industry, and have used their experiences to inform their writing. They have pursued their passion for art, and have used their knowledge and experience to create meaningful work.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(str(response))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"\"\\nThe author is a creative person who has had a varied and interesting life. They grew up in the US and went to college, but then decided to take a break and pursue their passion for art. They applied to two art schools, RISD in the US and the Accademia di Belli Arti in Florence, and were accepted to both. They chose to go to Florence, where they took the entrance exam and passed. They then spent a year living in Florence, studying art at the Accademia and painting still lives in their bedroom. After their year in Florence, the author returned to the US and completed their BFA program at RISD. They then went on to pursue a PhD in computer science at MIT, where they wrote a dissertation on the evolution of computers. During their time at MIT, they also did consulting work and wrote essays on topics they had been thinking about. After completing their PhD, the author started a software company, Viaweb, which was eventually acquired by Yahoo. They then went on to write essays and articles about their experiences in the tech industry. They also wrote an essay about how to choose what to work on, which was based on their own experience. The author then moved back to Florence, where they found a rent-stabilized apartment and continued to pursue their interest in art. They wrote about their experiences in the art world, and experienced the reactions of readers to their essays. The author is now a successful writer and continues to write essays and articles about topics they are passionate about. \\n\\nIn summary, the author's life has been a journey of exploration and creativity. They have experienced a wide range of different things in their life, from art school to computer science to the tech industry, and have used their experiences to inform their writing. They have pursued their passion for art, and have used their knowledge and experience to create meaningful work.\""
|
|
]
|
|
},
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"response"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 11 tokens\n",
|
|
"> [retrieve] Total embedding token usage: 11 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 0 tokens\n",
|
|
"> [retrieve] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1947 tokens\n",
|
|
"> [get_response] Total LLM token usage: 1947 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"> [retrieve] Total LLM token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 0 tokens\n",
|
|
"> [retrieve] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1947 tokens\n",
|
|
"> [get_response] Total LLM token usage: 1947 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"INFO:llama_index.query_engine.router_query_engine:Combining responses from multiple query engines.\n",
|
|
"Combining responses from multiple query engines.\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 316 tokens\n",
|
|
"> [get_response] Total LLM token usage: 316 tokens\n",
|
|
"INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens\n",
|
|
"> [get_response] Total embedding token usage: 0 tokens\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"response = query_engine.query(\n",
|
|
" \"What did Paul Graham do during his time in college?\"\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"Paul Graham studied philosophy in college, but he did not pursue AI. He continued to work on programming outside of school, writing simple games, a program to predict how high his model rockets would fly, and a word processor. He eventually convinced his father to buy him a TRS-80 computer, which he used to further his programming skills.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(str(response))"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "llama_index_v2",
|
|
"language": "python",
|
|
"name": "llama_index_v2"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|