chore: import upstream snapshot with attribution

docs/examples/node_postprocessor/TimeWeightedPostprocessorDemo.ipynb

@@ -0,0 +1,351 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "b1c1ebaa-50de-4851-a720-acbb977551ea",
+   "metadata": {},
+   "source": [
+    "# Time-Weighted Rerank\n",
+    "\n",
+    "Showcase capabilities of time-weighted node postprocessor"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "92d06b38-2103-4a40-93c3-60e0708a1124",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/home/loganm/miniconda3/envs/llama-index/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
+      "  from .autonotebook import tqdm as notebook_tqdm\n"
+     ]
+    }
+   ],
+   "source": [
+    "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n",
+    "from llama_index.core.postprocessor import TimeWeightedPostprocessor\n",
+    "from llama_index.core.node_parser import SentenceSplitter\n",
+    "from llama_index.core.storage.docstore import SimpleDocumentStore\n",
+    "from llama_index.core.response.notebook_utils import display_response\n",
+    "from datetime import datetime, timedelta"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "67020156-2975-4bbb-8e98-afc55abb3d72",
+   "metadata": {},
+   "source": [
+    "### Parse Documents into Nodes, add to Docstore\n",
+    "\n",
+    "In this example, there are 3 different versions of PG's essay. They are largely identical **except** \n",
+    "for one specific section, which details the amount of funding they raised for Viaweb. \n",
+    "\n",
+    "V1: 50k, V2: 30k, V3: 10K\n",
+    "\n",
+    "V1: -1 day, V2: -2 days, V3: -3 days\n",
+    "\n",
+    "The idea is to encourage index to fetch the most recent info (which is V3)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "caddd84e-9827-40a4-9520-dba6405fd1fd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# load documents\n",
+    "from llama_index.core import StorageContext\n",
+    "\n",
+    "\n",
+    "now = datetime.now()\n",
+    "key = \"__last_accessed__\"\n",
+    "\n",
+    "\n",
+    "doc1 = SimpleDirectoryReader(\n",
+    "    input_files=[\"./test_versioned_data/paul_graham_essay_v1.txt\"]\n",
+    ").load_data()[0]\n",
+    "\n",
+    "\n",
+    "doc2 = SimpleDirectoryReader(\n",
+    "    input_files=[\"./test_versioned_data/paul_graham_essay_v2.txt\"]\n",
+    ").load_data()[0]\n",
+    "\n",
+    "doc3 = SimpleDirectoryReader(\n",
+    "    input_files=[\"./test_versioned_data/paul_graham_essay_v3.txt\"]\n",
+    ").load_data()[0]\n",
+    "\n",
+    "\n",
+    "# define settings\n",
+    "from llama_index.core import Settings\n",
+    "\n",
+    "Settings.text_splitter = SentenceSplitter(chunk_size=512)\n",
+    "\n",
+    "# use node parser from settings to parse docs into nodes\n",
+    "nodes1 = Settings.text_splitter.get_nodes_from_documents([doc1])\n",
+    "nodes2 = Settings.text_splitter.get_nodes_from_documents([doc2])\n",
+    "nodes3 = Settings.text_splitter.get_nodes_from_documents([doc3])\n",
+    "\n",
+    "\n",
+    "# fetch the modified chunk from each document, set metadata\n",
+    "# also exclude the date from being read by the LLM\n",
+    "nodes1[14].metadata[key] = (now - timedelta(hours=3)).timestamp()\n",
+    "nodes1[14].excluded_llm_metadata_keys = [key]\n",
+    "\n",
+    "nodes2[14].metadata[key] = (now - timedelta(hours=2)).timestamp()\n",
+    "nodes2[14].excluded_llm_metadata_keys = [key]\n",
+    "\n",
+    "nodes3[14].metadata[key] = (now - timedelta(hours=1)).timestamp()\n",
+    "nodes2[14].excluded_llm_metadata_keys = [key]\n",
+    "\n",
+    "\n",
+    "# add to docstore\n",
+    "docstore = SimpleDocumentStore()\n",
+    "nodes = [nodes1[14], nodes2[14], nodes3[14]]\n",
+    "docstore.add_documents(nodes)\n",
+    "\n",
+    "storage_context = StorageContext.from_defaults(docstore=docstore)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e5a25b95-de5e-4e56-a846-51e9c6eba181",
+   "metadata": {},
+   "source": [
+    "### Build Index"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5f7f68d6-2389-4f6c-bc4e-8612a1a53fb8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# build index\n",
+    "index = VectorStoreIndex(nodes, storage_context=storage_context)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "86c5e8aa-18d8-4229-b7b2-a1c97c11a09a",
+   "metadata": {},
+   "source": [
+    "### Define Recency Postprocessors"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ba5e10c9-5a7e-4ea8-a74d-0e0f74b5cd1b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "node_postprocessor = TimeWeightedPostprocessor(\n",
+    "    time_decay=0.5, time_access_refresh=False, top_k=1\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "efcfffe4-a8aa-486d-b46d-f73f985dffca",
+   "metadata": {},
+   "source": [
+    "### Query Index"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "78d6c3db-61e6-4d9a-a84d-d7be846b4112",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# naive query\n",
+    "query_engine = index.as_query_engine(\n",
+    "    similarity_top_k=3,\n",
+    ")\n",
+    "response = query_engine.query(\n",
+    "    \"How much did the author raise in seed funding from Idelle's husband\"\n",
+    "    \" (Julian) for Viaweb?\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "84a608cd-fd70-40ba-a2f5-1414148db7de",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/markdown": [
+       "**`Final Response:`** $50,000"
+      ],
+      "text/plain": [
+       "<IPython.core.display.Markdown object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "display_response(response)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1d672c52-c0ac-4e5f-9175-855e66eb97ba",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# query using time weighted node postprocessor\n",
+    "\n",
+    "query_engine = index.as_query_engine(\n",
+    "    similarity_top_k=3, node_postprocessors=[node_postprocessor]\n",
+    ")\n",
+    "response = query_engine.query(\n",
+    "    \"How much did the author raise in seed funding from Idelle's husband\"\n",
+    "    \" (Julian) for Viaweb?\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cd4b971e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/markdown": [
+       "**`Final Response:`** The author raised $10,000 in seed funding from Idelle's husband (Julian) for Viaweb."
+      ],
+      "text/plain": [
+       "<IPython.core.display.Markdown object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "display_response(response)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dd00cc97-4de7-4c61-9c0c-3f9ee3598528",
+   "metadata": {},
+   "source": [
+    "### Query Index (Lower-Level Usage)\n",
+    "\n",
+    "In this example we first get the full set of nodes from a query call, and then send to node postprocessor, and then\n",
+    "finally synthesize response through a summary index."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "350b039e-d45d-4b6b-957a-4b14d8816cbd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_index.core import SummaryIndex"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "234f909f-6faa-43e6-96f8-0966699c9552",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query_str = (\n",
+    "    \"How much did the author raise in seed funding from Idelle's husband\"\n",
+    "    \" (Julian) for Viaweb?\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "20afbf6b-9473-446e-b522-b90fef2e3bf0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query_engine = index.as_query_engine(\n",
+    "    similarity_top_k=3, response_mode=\"no_text\"\n",
+    ")\n",
+    "init_response = query_engine.query(\n",
+    "    query_str,\n",
+    ")\n",
+    "resp_nodes = [n for n in init_response.source_nodes]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cdc03574-a806-4255-953c-6f82fc3f202f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# get the post-processed nodes -- which should be the top-1 sorted by date\n",
+    "new_resp_nodes = node_postprocessor.postprocess_nodes(resp_nodes)\n",
+    "\n",
+    "summary_index = SummaryIndex([n.node for n in new_resp_nodes])\n",
+    "query_engine = summary_index.as_query_engine()\n",
+    "response = query_engine.query(query_str)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9d4de372-653f-4d57-9e0f-17a90f39b874",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/markdown": [
+       "**`Final Response:`** The author raised $10,000 in seed funding from Idelle's husband (Julian) for Viaweb."
+      ],
+      "text/plain": [
+       "<IPython.core.display.Markdown object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "display_response(response)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llama-index",
+   "language": "python",
+   "name": "llama-index"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}