wehub-resource-sync
1305 lines
48 KiB
Plaintext
1305 lines
48 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "CbAXAsjdHh_n"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Copyright 2024 Google LLC\n",
|
|
"#\n",
|
|
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
|
|
"# you may not use this file except in compliance with the License.\n",
|
|
"# You may obtain a copy of the License at\n",
|
|
"#\n",
|
|
"# https://www.apache.org/licenses/LICENSE-2.0\n",
|
|
"#\n",
|
|
"# Unless required by applicable law or agreed to in writing, software\n",
|
|
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
|
|
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
|
|
"# See the License for the specific language governing permissions and\n",
|
|
"# limitations under the License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "HJrhVUWzZWhI"
|
|
},
|
|
"source": [
|
|
"# Chain of Thought & ReAct"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "jx8l8JyMHwWn"
|
|
},
|
|
"source": [
|
|
"<table align=\"left\">\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fprompts%2Fexamples%2Fchain_of_thought_react.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
|
|
" </a>\n",
|
|
" </td> \n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/prompts/examples/chain_of_thought_react.ipynb\">\n",
|
|
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Workbench\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
" <td style=\"text-align: center\">\n",
|
|
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb\">\n",
|
|
" <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
|
|
" </a>\n",
|
|
" </td>\n",
|
|
"</table>\n",
|
|
"\n",
|
|
"<div style=\"clear: both;\"></div>\n",
|
|
"\n",
|
|
"<b>Share to:</b>\n",
|
|
"\n",
|
|
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
|
|
"</a>\n",
|
|
"\n",
|
|
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/prompts/examples/chain_of_thought_react.ipynb\" target=\"_blank\">\n",
|
|
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
|
|
"</a> "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "qBQpaP2kZEjn"
|
|
},
|
|
"source": [
|
|
"| Authors |\n",
|
|
"| --- |\n",
|
|
"| [Chris Hanna](https://github.com/ChrisHanna) |\n",
|
|
"| [Rupjit Chakraborty](https://github.com/lazyprgmr) |\n",
|
|
"| [Deepak Moonat](https://github.com/dmoonat) |"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "swNSCZctIoQ_"
|
|
},
|
|
"source": [
|
|
"# Overview"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "87kXlLlBItWx"
|
|
},
|
|
"source": [
|
|
"This notebook demonstrates advanced prompting techniques such as chain of thought reasoning, and building [ReAct](https://arxiv.org/abs/2210.03629.pdf) agents using [LangChain](https://www.langchain.com/) and Vertex AI. You will start with exploring chain of thought, and how it can be used to improve the performance of language models. Then you will learn how to build ReAct agents using LangChain.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "u99ipT8HJoqE"
|
|
},
|
|
"source": [
|
|
"## Getting Started"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "kCaWRKbUotUL"
|
|
},
|
|
"source": [
|
|
"### Install Vertex AI SDK and other required packages\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "N7VCLWGiKR6a"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install -q --user --upgrade langchain==0.3.0 \\\n",
|
|
" google-cloud-aiplatform==1.67.1 \\\n",
|
|
" prettyprinter==0.18.0 \\\n",
|
|
" wikipedia==1.4.0 \\\n",
|
|
" google-cloud-bigquery==3.25.0 \\\n",
|
|
" langchain-core==0.3.2 \\\n",
|
|
" langchain-google-vertexai==2.0.1 \\\n",
|
|
" langchain-experimental==0.3.0 \\\n",
|
|
" bigframes==1.18.0"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "I5VhU_S5ZWhK"
|
|
},
|
|
"source": [
|
|
"### Authenticate your notebook environment (Colab only)\n",
|
|
"\n",
|
|
"If you are running this notebook on Google Colab, run the cell below to authenticate your environment.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "cil9JtGHKEXv"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import sys\n",
|
|
"\n",
|
|
"if \"google.colab\" in sys.modules:\n",
|
|
" from google.colab import auth\n",
|
|
"\n",
|
|
" auth.authenticate_user()\n",
|
|
" print(\"Authenticated\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "_fiL-DovoB8C"
|
|
},
|
|
"source": [
|
|
"### Set Google Cloud project information and initialize Vertex AI SDK\n",
|
|
"\n",
|
|
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
|
|
"\n",
|
|
"Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment).\"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "ZicXqcrFLJdp"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"PROJECT_ID = \"\" # @param {type:\"string\"}\n",
|
|
"LOCATION = \"\" # @param {type:\"string\"}\n",
|
|
"MODEL_NAME = \"gemini-2.0-flash\" # @param {type:\"string\"}\n",
|
|
"\n",
|
|
"import vertexai\n",
|
|
"\n",
|
|
"vertexai.init(project=PROJECT_ID, location=LOCATION)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "Zc8bgArrZWhK"
|
|
},
|
|
"source": [
|
|
"## Import packages"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "XyRhFU9yKivk"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from IPython.display import Markdown, display\n",
|
|
"from langchain_google_vertexai import VertexAI"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "2fLP4777LS8I"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"llm = VertexAI(model_name=MODEL_NAME, max_output_tokens=1000)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "sM4PPhw5MZv2"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Test to confirm if packages are installed and working correctly.\n",
|
|
"response = llm.invoke(\n",
|
|
" \"Improve this description : In this notebook we'll explore advanced prompting techniques, and building ReAct agents using LangChain and Vertex AI \"\n",
|
|
").replace(\"*\", \"\")\n",
|
|
"print(response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "-jdYIk8g0rax"
|
|
},
|
|
"source": [
|
|
"# Chain of Thought - Introduction\n",
|
|
"\n",
|
|
"---\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "SxdW0tg75zHD"
|
|
},
|
|
"source": [
|
|
"The technique introduced in this [paper](https://arxiv.org/pdf/2201.11903.pdf) is a novel approach to enhance the reasoning capabilities of Large Language Models (LLMs), especially in multi-step reasoning tasks.\n",
|
|
"\n",
|
|
"In contrast to the standard prompting, where models are asked to directly produce the final answer, 'Chain of Thought Prompting' encourages LLMs to generate intermediate reasoning steps before providing the final answer to a problem. The advantage of this technique lies in its ability to break down complex problems into manageable, intermediate steps. By doing this, the model-generated 'chain of thought' can mimic an intuitive human thought process when working through multi-step problems."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "8P5dNoyRZyR_"
|
|
},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "Q5AiwI-rU1Zs"
|
|
},
|
|
"source": [
|
|
"# Chain of Thought - Use Cases"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "F9gj5uzNUYun"
|
|
},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "Lie-U8h3hRY2"
|
|
},
|
|
"source": [
|
|
"# Chain of Thought - Sample"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "aQ0knAMr03xU"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"question = \"\"\"\n",
|
|
" Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.\n",
|
|
" Each can has 3 tennis balls. How many tennis balls does he have now?\n",
|
|
" A: The answer is 11.\n",
|
|
" Q: The cafeteria had 33 apples.\n",
|
|
" If they used 20 to make lunch and bought 6 more, how many apples do they have?\n",
|
|
" A:\n",
|
|
"\"\"\"\n",
|
|
"print(llm.invoke(question))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "7c3CRFFi6Y5C"
|
|
},
|
|
"source": [
|
|
"Rewriting the prompt to include a chain of thought shows the LLM how to decompose the question into multiple simple steps of reasoning.\n",
|
|
"\n",
|
|
"The model response then follows a similar chain of thought, increasing the likelihood of a correct answer."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "1H018t0r3_sT"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"question = \"\"\"\n",
|
|
" Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.\n",
|
|
" Each can has 3 tennis balls. How many tennis balls does he have now?\n",
|
|
" A: Roger started with 5 balls. 2 cans of 3 tennis balls\n",
|
|
" each is 6 tennis balls. 5 + 6 = 11. The answer is 11.\n",
|
|
" Q: The cafeteria had 23 apples.\n",
|
|
" If they used 20 to make lunch and bought 6 more, how many apples do they have?\n",
|
|
" A:\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"print(llm.invoke(question))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "5bs54tPg6jIh"
|
|
},
|
|
"source": [
|
|
"Notice the chain of thought includes both text describing the steps to follow and intermediate outputs/conclusions from each reasoning step."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "9csdQAwtaver"
|
|
},
|
|
"source": [
|
|
"# Chain of Thought(CoT) - Zero Shot"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "cov76yM2bqku"
|
|
},
|
|
"source": [
|
|
"Zero-shot CoT prompting is a technique that allows large language models (LLMs) to generate more accurate answers to questions. It does this by appending the words \"Let's think step by step.\" to the end of a question. This simple prompt helps the LLM to generate a chain of thought that answers the question, from which the LLM can then extract a more accurate answer."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "3PCXqkxMbEEA"
|
|
},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "_MX4M2Cxbs0O"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"question = \"\"\"\n",
|
|
" Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.\n",
|
|
" Each can has 3 tennis balls. How many tennis balls does he have now?\n",
|
|
" A: The answer is 11.\n",
|
|
" Q: The cafeteria had 23 apples.\n",
|
|
" If they used 20 to make lunch and bought 6 more, how many apples do they have?\n",
|
|
" A:\n",
|
|
"\"\"\"\n",
|
|
"print(llm.invoke(question))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "DcmzznHGcGKG"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"question = \"\"\"\n",
|
|
"Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.\n",
|
|
"Each can has 3 tennis balls. How many tennis balls does he have now?\n",
|
|
"A: The answer is 11.\n",
|
|
"\n",
|
|
"Q: The cafeteria had 23 apples.\n",
|
|
"If they used 20 to make lunch and bought 6 more, how many apples do they have?\n",
|
|
"A: Let's think step by step.\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"print(llm.invoke(question))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "vCNXgrMYIFmo"
|
|
},
|
|
"source": [
|
|
"With just adding `Let's think step by step.`, LLM is now able to provide answer with proper reasoning."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "Al9LmlUzcH5k"
|
|
},
|
|
"source": [
|
|
"# Chain of Thought - Self Consistency"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "OQcRB3fdZWhl"
|
|
},
|
|
"source": [
|
|
"An improvement upon CoT prompting is by doing CoT with self-consistency, whereby you generate multiple candidate answers through CoT with the same input.\n",
|
|
"\n",
|
|
"As we are making multiple LLM calls, the cost incurred will be higher than the Chain of Thought(CoT) approach."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "y7z1XZRpr7Oa"
|
|
},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "5jL5-abacTk2"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from operator import itemgetter\n",
|
|
"\n",
|
|
"from langchain.prompts import PromptTemplate\n",
|
|
"from langchain.schema import StrOutputParser\n",
|
|
"from langchain.schema.runnable import RunnablePassthrough\n",
|
|
"\n",
|
|
"question = \"\"\"The cafeteria had 23 apples.\n",
|
|
"If they used 20 to make lunch and bought 6 more, how many apples do they have?\"\"\"\n",
|
|
"\n",
|
|
"context = \"\"\"Answer questions showing the full math and reasoning.\n",
|
|
"Follow the pattern in the example.\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"one_shot_exemplar = \"\"\"Example Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.\n",
|
|
"Each can has 3 tennis balls. How many tennis balls does he have now?\n",
|
|
"A: Roger started with 5 balls. 2 cans of 3 tennis balls\n",
|
|
"each is 6 tennis balls. 5 + 6 = 11.\n",
|
|
"The answer is 11.\n",
|
|
"\n",
|
|
"Q: \"\"\"\n",
|
|
"\n",
|
|
"\n",
|
|
"planner = (\n",
|
|
" PromptTemplate.from_template(context + one_shot_exemplar + \" {input}\")\n",
|
|
" | VertexAI()\n",
|
|
" | StrOutputParser()\n",
|
|
" | {\"base_response\": RunnablePassthrough()}\n",
|
|
")\n",
|
|
"\n",
|
|
"answer_1 = (\n",
|
|
" PromptTemplate.from_template(\"{base_response} A:\")\n",
|
|
" | VertexAI(temperature=0, max_output_tokens=400)\n",
|
|
" | StrOutputParser()\n",
|
|
")\n",
|
|
"\n",
|
|
"answer_2 = (\n",
|
|
" PromptTemplate.from_template(\"{base_response} A:\")\n",
|
|
" | VertexAI(temperature=0.3, max_output_tokens=400)\n",
|
|
" | StrOutputParser()\n",
|
|
")\n",
|
|
"\n",
|
|
"answer_3 = (\n",
|
|
" PromptTemplate.from_template(\"{base_response} A:\")\n",
|
|
" | VertexAI(temperature=0.5, max_output_tokens=400)\n",
|
|
" | StrOutputParser()\n",
|
|
")\n",
|
|
"\n",
|
|
"\n",
|
|
"final_responder = (\n",
|
|
" PromptTemplate.from_template(\n",
|
|
" \"Output all the final results in this markdown format: Result 1: {results_1} \\n Result 2:{results_2} \\n Result 3: {results_3}\"\n",
|
|
" )\n",
|
|
" | VertexAI(max_output_tokens=1024)\n",
|
|
" | StrOutputParser()\n",
|
|
")\n",
|
|
"\n",
|
|
"chain = (\n",
|
|
" planner\n",
|
|
" | {\n",
|
|
" \"results_1\": answer_1,\n",
|
|
" \"results_2\": answer_2,\n",
|
|
" \"results_3\": answer_3,\n",
|
|
" \"original_response\": itemgetter(\"base_response\"),\n",
|
|
" }\n",
|
|
" | final_responder\n",
|
|
")\n",
|
|
"\n",
|
|
"\n",
|
|
"answers = chain.invoke({\"input\": question})\n",
|
|
"display(Markdown(answers))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "BX4rKnEHZWho"
|
|
},
|
|
"source": [
|
|
"As seen in the output above, three answers were generated, the next step is to extract the most popular answer, in this case it's `9`.\n",
|
|
"\n",
|
|
"Be mindful of the number of LLM calls(iterations) to get the popular answer as the cost incurred will be higher. Also, could add mechanisms to stop the LLM calls after getting same answers in a series of call."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "Kvyz7_0_X755"
|
|
},
|
|
"source": [
|
|
"# Chain of Thought - JSON Data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "3Dlaj-4IZWhp"
|
|
},
|
|
"source": [
|
|
"Sample from [Advanced Prompt Engineering](https://github.com/GoogleCloudPlatform/applied-ai-engineering-samples/tree/main/genai-on-vertex-ai/advanced_prompting_training) by [Michael Sherman](https://github.com/michaelwsherman)\n",
|
|
"\n",
|
|
"Refer to above link for CoT best practices"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "-dnD07DWYJWU"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"context = \"\"\"Given a JSON entry of a data source, output a JSON with the following fields and explain the reasoning:\n",
|
|
"pii: True/False, the dataset contains Personally Identifiable Information.\n",
|
|
"age: How many years since the dataset was last modified.\n",
|
|
"keywords: New keywords to index this dataset under, beyond the current set of keywords.\n",
|
|
"The last text output should be the JSON.\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"\n",
|
|
"question = \"\"\"\n",
|
|
"{\n",
|
|
" \"@type\" : \"dcat:Dataset\",\n",
|
|
" \"description\" : \"<p>The MDS 3.0 Frequency Report summarizes information for active residents currently in nursing homes. The source of these counts is the residents MDS assessment record. The MDS assessment information for each active nursing home resident is consolidated to create a profile of the most recent standard information for the resident.</p>\\n\",\n",
|
|
" \"title\" : \"MDS 3.0 Frequency Report\",\n",
|
|
" \"accessLevel\" : \"public\",\n",
|
|
" \"identifier\" : \"465\",\n",
|
|
" \"license\" : \"http://opendefinition.org/licenses/odc-odbl/\",\n",
|
|
" \"modified\" : \"2016-04-05\",\n",
|
|
" \"temporal\" : \"2012-01-01T00:00:00-05:00/2015-12-31T00:00:00-05:00\",\n",
|
|
" \"contactPoint\" : {\n",
|
|
" \"@type\" : \"vcard:Contact\",\n",
|
|
" \"fn\" : \"Health Data Initiative\",\n",
|
|
" \"hasEmail\" : \"mailto:HealthData@hhs.gov\"\n",
|
|
" },\n",
|
|
" \"bureauCode\" : [ \"009:38\" ],\n",
|
|
" \"keyword\" : [ \"Activities of Daily Living (ADL)\" ],\n",
|
|
" \"language\" : [ \"en\" ],\n",
|
|
" \"programCode\" : [ \"009:000\" ],\n",
|
|
" \"publisher\" : {\n",
|
|
" \"@type\" : \"org:Organization\",\n",
|
|
" \"name\" : \"Centers for Medicare & Medicaid Services\",\n",
|
|
" \"subOrganizationOf\" : {\n",
|
|
" \"@type\" : \"org:Organization\",\n",
|
|
" \"name\" : \"Department of Health & Human Services\"\n",
|
|
" }\n",
|
|
" }\n",
|
|
" }\n",
|
|
"\n",
|
|
"\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"llm_prompt = f\"{context}\\nJSON:{question}\\nAnswer:\"\n",
|
|
"\n",
|
|
"display(Markdown(llm.invoke(llm_prompt)))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "QUH1T1yDY-fj"
|
|
},
|
|
"source": [
|
|
"As seen in the output above, the JSON formatting is correct, but age is incorrect, and no keywords were extracted.\n",
|
|
"\n",
|
|
"To improve the response quality, you can add one exemplar, which should lead to a correct response:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "1dzIStz0ZAFG"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"one_shot_example = \"\"\"\n",
|
|
"JSON:\n",
|
|
"{\n",
|
|
"\n",
|
|
" \"@type\" : \"dcat:Dataset\",\n",
|
|
" \"description\" : \"The primary purpose of this system of records is to properly pay medical insurance benefits to or on behalf of entitled beneficiaries.\",\n",
|
|
" \"title\" : \"Medicare Multi-Carrier Claims System\",\n",
|
|
" \"accessLevel\" : \"restricted public\",\n",
|
|
" \"dataQuality\" : true,\n",
|
|
" \"identifier\" : \"b6ffafab-1cfd-42dd-b8cb-7a554efaefa7\",\n",
|
|
" \"landingPage\" : \"http://www.cms.gov/Research-Statistics-Data-and-Systems/Computer-Data-and-Systems/Privacy/Systems-of-Records-Items/09-70-0501-MCS.html\",\n",
|
|
" \"license\" : \"http://www.usa.gov/publicdomain/label/1.0/\",\n",
|
|
" \"modified\" : \"2014-09-30\",\n",
|
|
" \"rights\" : \"Contains personally identifiable information and is subject to the Privacy Act of 1974, as amended at 5 United States Code (U.S.C.) 552a. Requests should be directed to the appropriate System Manager, identified in the System of Records notice.\",\n",
|
|
" \"primaryITInvestmentUII\" : \"009-000004256, 009-000004254\",\n",
|
|
" \"systemOfRecords\" : \"09-70-0501\",\n",
|
|
"\n",
|
|
" \"contactPoint\" : {\n",
|
|
" \"@type\" : \"vcard:Contact\",\n",
|
|
" \"fn\" : \"Health Data Initiative\",\n",
|
|
" \"hasEmail\" : \"mailto:Healthdata@hhs.gov\"\n",
|
|
" },\n",
|
|
" \"bureauCode\" : [ \"009:38\" ],\n",
|
|
" \"keyword\" : [ \"medicare\", \"part b\", \"claims\" ],\n",
|
|
" \"programCode\" : [ \"009:078\" ],\n",
|
|
" \"theme\" : [ \"Medicare\" ],\n",
|
|
" \"publisher\" : {\n",
|
|
" \"@type\" : \"org:Organization\",\n",
|
|
" \"name\" : \"Centers for Medicare & Medicaid Services\",\n",
|
|
" \"subOrganizationOf\" : {\n",
|
|
" \"@type\" : \"org:Organization\",\n",
|
|
" \"name\" : \"Department of Health & Human Services\"\n",
|
|
" }\n",
|
|
" }\n",
|
|
" }\n",
|
|
"\n",
|
|
"Answer: The 'rights' tag says 'Contains personally identifiable information' so pii is True.\n",
|
|
"The 'modified' tag is '2014-09-30'. The current year is 2023, 2023 minus 2014 is 9, so the age is 9.\n",
|
|
"To determine keywords I will look at all the fields that describe the dataset.\n",
|
|
"Then I will take the most salient and distinctive aspects of the fields and make those keywords.\n",
|
|
"Looking at all the fields, the ones that describe the dataset are \"description\" and \"title\".\n",
|
|
"The \"title\" field is \"Medicare Multi-Carrier Claims System\".\n",
|
|
"Good keywords from the \"title\" field are \"medicare\" and \"claims\".\n",
|
|
"The \"description\" field is \"\"The primary purpose of this system of records is to properly pay medical insurance benefits to or on behalf of entitled beneficiaries.\"\n",
|
|
"Good keywords from the \"description\" field are \"medical insurance benefits\".\n",
|
|
"Good proposed keywords from both fields are \"medicare\", \"claims\", and \"medical insurance benefits\".\n",
|
|
"Next inspect the \"keyword\" field to make sure the proposed keywords are not already included.\n",
|
|
"The \"keyword\" field contains the keywords \"medicare\", \"part b\", and \"claims\".\n",
|
|
"From our proposed keywords, \"medicare\" should not be output since it is already in the \"keyword\" field.\n",
|
|
"That leaves \"claims\" and \"medical insurance benefits\" as proposed keywords.\n",
|
|
"\n",
|
|
"Output JSON:\n",
|
|
"{\n",
|
|
" \"pii\" : true,\n",
|
|
" \"age\" : 9,\n",
|
|
" \"keywords\" : [\"claims\", \"medical insurance benefits\"]\n",
|
|
"}\n",
|
|
"\"\"\"\n",
|
|
"\n",
|
|
"# Prepending the one shot exemplar before the question we want answered.\n",
|
|
"llm_prompt = f\"{context}{one_shot_example}\\nJSON:{question}\\nAnswer:\"\n",
|
|
"\n",
|
|
"display(Markdown(llm.invoke(llm_prompt)))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "iarBTZZAGkkg"
|
|
},
|
|
"source": [
|
|
"# ReAct - Introduction\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "6Bisk_69yBgh"
|
|
},
|
|
"source": [
|
|
"[ReAct](https://arxiv.org/abs/2210.03629.pdf) (short for Reasoning & Acting) combines chain of thought and tool usage together to reason through complex tasks by interacting with external systems. ReAct is particularly useful if you want the LLM or an LLM-based chatbot to reason and take action on external systems through extensions.\n",
|
|
"\n",
|
|
"\n",
|
|
"Refer to following [link](https://github.com/GoogleCloudPlatform/applied-ai-engineering-samples/tree/main/genai-on-vertex-ai/advanced_prompting_training) for ReAct best practices"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "kYFOwyCLZWhp"
|
|
},
|
|
"source": [
|
|
"For example, LLMs do not know today's date:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "-0qVZzsdZWhp"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"llm.invoke(\"What is today's date?\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "H6JyPSQbZWhp"
|
|
},
|
|
"source": [
|
|
"But you can easily create a Python function to fetch today's date:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "hMqbWTaaZWhp"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def get_current_date():\n",
|
|
" \"\"\"\n",
|
|
" Gets the current date (today), in the format YYYY-MM-DD\n",
|
|
" \"\"\"\n",
|
|
"\n",
|
|
" from datetime import datetime\n",
|
|
"\n",
|
|
" todays_date = datetime.today().strftime(\"%Y-%m-%d\")\n",
|
|
"\n",
|
|
" return todays_date"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "7gKM42BbZWhp"
|
|
},
|
|
"source": [
|
|
"To enable the LLM to use this function, you can use tools with a ReAct agent:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "oVkkprEeZWhp"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain.agents.agent_types import AgentType\n",
|
|
"from langchain.agents.initialize import initialize_agent\n",
|
|
"from langchain.llms import VertexAI\n",
|
|
"from langchain.tools import StructuredTool, WikipediaQueryRun\n",
|
|
"from langchain.utilities import WikipediaAPIWrapper\n",
|
|
"import vertexai\n",
|
|
"\n",
|
|
"t_get_current_date = StructuredTool.from_function(get_current_date)\n",
|
|
"\n",
|
|
"tools = [\n",
|
|
" t_get_current_date,\n",
|
|
"]\n",
|
|
"\n",
|
|
"agent = initialize_agent(\n",
|
|
" tools,\n",
|
|
" llm,\n",
|
|
" agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
|
|
" verbose=True,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "XFu9ksGEZWhp"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"agent.invoke(\"What's today's date?\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "JZfQZurHGsmp"
|
|
},
|
|
"source": [
|
|
"# ReAct - Wikipedia"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "INg_H3T6ZWhq"
|
|
},
|
|
"source": [
|
|
"In the example below, you can enable the LLM to call `Wikipedia` tool:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "STyRXz5jOicV"
|
|
},
|
|
"source": [
|
|
"The code snippet demonstrates how to use the tools to interact with external systems, such as Wikipedia, from within a language model (LLM).\n",
|
|
"\n",
|
|
"1. **Import Necessary Modules:**\n",
|
|
" - Import the `WikipediaAPIWrapper` class, which provides an interface to interact with the Wikipedia API.\n",
|
|
"\n",
|
|
"2. **Create a Wikipedia Tool:**\n",
|
|
" - Create an instance of the `WikipediaAPIWrapper` class, passing in the `llm` object as a parameter.\n",
|
|
" - This tool allows the LLM to query Wikipedia and retrieve relevant information.\n",
|
|
"\n",
|
|
"3. **Initialize ReAct Agent:**\n",
|
|
" - Load tools, passing in the list of tools (in this case, only the Wikipedia tool).\n",
|
|
" - Initialize an agent using the `initialize_agent` function, passing in the tools, llm object and the agent type.\n",
|
|
"\n",
|
|
"4. **Agent Invocation:**\n",
|
|
" - Call the `invoke` method on the agent, passing in a query string.\n",
|
|
" - The agent will use the Wikipedia tool to retrieve information from Wikipedia and generate a response based on the query.\n",
|
|
"\n",
|
|
"5. **Output:**\n",
|
|
" - The output of the `invoke` method will be a response generated by the LLM, incorporating information retrieved from Wikipedia."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "4jIjwZ-ZSCDQ"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain.agents import load_tools\n",
|
|
"\n",
|
|
"_ = WikipediaQueryRun(api_wrapper=WikipediaAPIWrapper())\n",
|
|
"\n",
|
|
"tools = load_tools([\"wikipedia\"], llm=llm)\n",
|
|
"\n",
|
|
"agent = initialize_agent(\n",
|
|
" tools,\n",
|
|
" llm,\n",
|
|
" agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
|
|
" verbose=True,\n",
|
|
")\n",
|
|
"\n",
|
|
"# Let's search some information generally available on wikipedia.\n",
|
|
"agent.invoke(\n",
|
|
" \"Give me a short summary of a famous monument along with the name of the wikipedia page referenced\"\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "VJ6Rc5c6ws6O"
|
|
},
|
|
"source": [
|
|
"# ReAct - BigQuery"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "gs80xhVxsute"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from google.cloud import bigquery\n",
|
|
"from langchain.agents.agent import Agent, AgentOutputParser\n",
|
|
"from langchain.agents.react.output_parser import ReActOutputParser\n",
|
|
"from langchain.llms import VertexAI\n",
|
|
"from langchain.output_parsers import CommaSeparatedListOutputParser\n",
|
|
"from langchain.prompts.base import BasePromptTemplate\n",
|
|
"from langchain.prompts.prompt import PromptTemplate\n",
|
|
"from langchain.tools.base import BaseTool\n",
|
|
"from langchain_experimental.tools.python.tool import PythonREPLTool\n",
|
|
"\n",
|
|
"bq = bigquery.Client(project=PROJECT_ID)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "ydlfHJhPsxuz"
|
|
},
|
|
"source": [
|
|
"Define custom tools"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "hKsCzC37P-YD"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"QUERY = \"\"\"\n",
|
|
"SELECT * FROM `bigquery-public-data.hacker_news.full` \n",
|
|
"WHERE CONCAT(title, url, text) IS NOT NULL\n",
|
|
"LIMIT 10\n",
|
|
"\"\"\"\n",
|
|
"df = bq.query(QUERY).to_dataframe()\n",
|
|
"df"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "dZVxkywtw01P"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def get_comment_by_id(id: str) -> str:\n",
|
|
" \"\"\"\n",
|
|
" Get a pandas dataframe of comment by id.\n",
|
|
" \"\"\"\n",
|
|
" QUERY = \"SELECT text FROM bigquery-public-data.hacker_news.full WHERE ID = {id} LIMIT 1\".format(\n",
|
|
" id=id\n",
|
|
" )\n",
|
|
" df = bq.query(QUERY).to_dataframe()\n",
|
|
" return df[\"text\"].values.tolist()[0]\n",
|
|
"\n",
|
|
"\n",
|
|
"def get_comment_by_user(user):\n",
|
|
" \"\"\"\n",
|
|
" Get a pandas dataframe of comments by user.\n",
|
|
" \"\"\"\n",
|
|
" QUERY = \"SELECT text FROM bigquery-public-data.hacker_news.full WHERE `BY` = {user} LIMIT 10\".format(\n",
|
|
" user=user\n",
|
|
" )\n",
|
|
" df = bq.query(QUERY).to_dataframe()\n",
|
|
" return df[\"text\"].values.tolist()\n",
|
|
"\n",
|
|
"\n",
|
|
"def generate_response_for_comment(comment):\n",
|
|
" \"\"\"\n",
|
|
" Get a LLM response for the user comment.\n",
|
|
" \"\"\"\n",
|
|
" question = \"\"\"Create a 1 sentence friendly response to the following comment: {comment}\"\"\".format(\n",
|
|
" comment=comment\n",
|
|
" )\n",
|
|
" response = llm.predict(question)\n",
|
|
"\n",
|
|
" return response\n",
|
|
"\n",
|
|
"\n",
|
|
"def generate_sentiment_for_comment(comment):\n",
|
|
" \"\"\"\n",
|
|
" Get a sentiment from LLM for the user comment.\n",
|
|
" \"\"\"\n",
|
|
" question = \"\"\"What is the sentiment of the comment (Negative, Positive, Neutral): {comment}\"\"\".format(\n",
|
|
" comment=comment\n",
|
|
" )\n",
|
|
" response = llm.predict(question)\n",
|
|
"\n",
|
|
" return response\n",
|
|
"\n",
|
|
"\n",
|
|
"def generate_category_for_comment(comment):\n",
|
|
" \"\"\"\n",
|
|
" Get a category generated by LLM for the user comment.\n",
|
|
" \"\"\"\n",
|
|
" question = \"\"\"Put the comment into one of these categories (Technology, Politics, Products, News): {comment}\"\"\".format(\n",
|
|
" comment=comment\n",
|
|
" )\n",
|
|
" response = llm.predict(question)\n",
|
|
"\n",
|
|
" return response"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "8nz3XGx8vTFo"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain.agents import Tool\n",
|
|
"\n",
|
|
"# Create a pool of tools which will be utilized by the LLM to generate response.\n",
|
|
"\n",
|
|
"tools = [\n",
|
|
" Tool(\n",
|
|
" name=\"GetCommentsById\",\n",
|
|
" func=get_comment_by_id,\n",
|
|
" description=\"Get a pandas dataframe of comment by id.\",\n",
|
|
" ),\n",
|
|
" Tool(\n",
|
|
" name=\"GetCommentsByUser\",\n",
|
|
" func=get_comment_by_user,\n",
|
|
" description=\"Get a pandas dataframe of comments by user.\",\n",
|
|
" ),\n",
|
|
" Tool(\n",
|
|
" name=\"GenerateCommentResponse\",\n",
|
|
" func=generate_response_for_comment,\n",
|
|
" description=\"Get an AI response for the user comment.\",\n",
|
|
" ),\n",
|
|
" Tool(\n",
|
|
" name=\"GenerateCommentSentiment\",\n",
|
|
" func=generate_sentiment_for_comment,\n",
|
|
" description=\"Get an AI sentiment for the user comment.\",\n",
|
|
" ),\n",
|
|
" Tool(\n",
|
|
" name=\"GenerateCategorySentiment\",\n",
|
|
" func=generate_category_for_comment,\n",
|
|
" description=\"Get an AI category for the user comment.\",\n",
|
|
" ),\n",
|
|
" PythonREPLTool(),\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "OhWUNUd1tEFt"
|
|
},
|
|
"source": [
|
|
"Setup Prompt and Examples"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "9ndaMHTUs6kE"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"EXAMPLES = [\n",
|
|
" \"\"\"Question: Write a response to the following Comment 1234 ?\n",
|
|
" Thought: I need to get comment 1234 using GetCommentsById.\n",
|
|
" Action: GetCommentsById[1234]\n",
|
|
" Observation: \"Comment Text\"\n",
|
|
" Thought: I need to generate a response to the comment.\n",
|
|
" Action: GenerateCommentResponse[\"Comment Text\"]\n",
|
|
" Observation: LLM Generated response\n",
|
|
" Thought: So the answer is \"LLM Generated response\".\n",
|
|
" Action: Finish[\"LLM Generated response\"]\n",
|
|
" Question: Write a response to all the comments by user xx234 ?\n",
|
|
" Thought: I need to get all the comments by xx234 using GetCommentsByUser.\n",
|
|
" Action: GetCommentsByUser['xx234']\n",
|
|
" Observation: \"Comment Text\"\n",
|
|
" Thought: I need to generate a response to each comment.\n",
|
|
" Action: GenerateCommentResponse[\"Comment Text 1\"]\n",
|
|
" Observation: \"LLM Generated response 1\"\n",
|
|
" Thought: I need to generate a response to each comment.\n",
|
|
" Action: GenerateCommentResponse[\"Comment Text 2\"]\n",
|
|
" Observation: \"LLM Generated response 2\"\n",
|
|
" Thought: I need to generate a response to each comment.\n",
|
|
" Action: GenerateCommentResponse[\"Comment Text 3\"]\n",
|
|
" Observation: \"LLM Generated response 3\"\n",
|
|
" Thought: I Generated responses for all the comments.\n",
|
|
" Action: Finish[\"Done\"],\n",
|
|
" Question: Sentiment for all the comments by user xx234 ?\n",
|
|
" Thought: I need to get all the comments by xx234 using GetCommentsByUser.\n",
|
|
" Action: GetCommentsByUser['xx234']\n",
|
|
" Observation: \"Comment Text\"\n",
|
|
" Thought: I need to determine sentiment of each comment.\n",
|
|
" Action: GenerateCommentSentiment[\"Comment Text 1\"]\n",
|
|
" Observation: \"LLM Generated Sentiment 1\"\n",
|
|
" Thought: I need to determine sentiment of each comment.\n",
|
|
" Action: GenerateCommentSentiment[\"Comment Text 2\"]\n",
|
|
" Observation: \"LLM Generated Sentiment 2\"\n",
|
|
" Thought: I need to generate a response to each comment.\n",
|
|
" Action: GenerateCommentSentiment[\"Comment Text 3\"]\n",
|
|
" Observation: \"LLM Generated Sentiment 3\"\n",
|
|
" Thought: I determined sentiment for all the comments.\n",
|
|
" Action: Finish[\"Done\"],\n",
|
|
" Question: Category for all the comments by user xx234 ?\n",
|
|
" Thought: I need to get all the comments by xx234 using GetCommentsByUser.\n",
|
|
" Action: GetCommentsByUser['xx234']\n",
|
|
" Observation: \"Comment Text\"\n",
|
|
" Thought: I need to determine the category of each comment.\n",
|
|
" Action: GenerateCategorySentiment[\"Comment Text 1\"]\n",
|
|
" Observation: \"LLM Generated Category 1\"\n",
|
|
" Thought: I need to determine category of each comment.\n",
|
|
" Action: GenerateCategorySentiment[\"Comment Text 2\"]\n",
|
|
" Observation: \"LLM Generated Category 2\"\n",
|
|
" Thought: I need to generate a category to each comment.\n",
|
|
" Action: GenerateCategorySentiment[\"Comment Text 3\"]\n",
|
|
" Observation: \"LLM Generated Category 3\"\n",
|
|
" Thought: I determined Category for all the comments.\n",
|
|
" Action: Finish[\"Done\"]\n",
|
|
"\"\"\"\n",
|
|
"]\n",
|
|
"\n",
|
|
"SUFFIX = \"\"\"\n",
|
|
"In each action, you cannot use the nested functions, such as GenerateCommentResponse[GetCommentsByUser[\"A\"], GetCommentsById[\"B\"]].\n",
|
|
"Instead, you should parse into 3 actions - GetCommentsById['A'], GetCommentsByUser['B'], and GenerateCommentResponse(\"Comment\").\n",
|
|
"\n",
|
|
"Let's start.\n",
|
|
"\n",
|
|
"Question: {input}\n",
|
|
"{agent_scratchpad} \"\"\"\n",
|
|
"\n",
|
|
"output_parser = CommaSeparatedListOutputParser()\n",
|
|
"\n",
|
|
"format_instructions = output_parser.get_format_instructions()\n",
|
|
"\n",
|
|
"TEST_PROMPT = PromptTemplate.from_examples(\n",
|
|
" examples=EXAMPLES,\n",
|
|
" suffix=SUFFIX,\n",
|
|
" input_variables=[\"input\", \"agent_scratchpad\"],\n",
|
|
")\n",
|
|
"\n",
|
|
"\n",
|
|
"class ReActTestAgent(Agent):\n",
|
|
" \"\"\"\n",
|
|
" Class to test the ReAct agent.\n",
|
|
" \"\"\"\n",
|
|
"\n",
|
|
" @classmethod\n",
|
|
" def _get_default_output_parser(cls, **kwargs: Any) -> AgentOutputParser:\n",
|
|
" \"\"\"\n",
|
|
" Get the LangChain default output parser.\n",
|
|
" \"\"\"\n",
|
|
" return ReActOutputParser()\n",
|
|
"\n",
|
|
" @classmethod\n",
|
|
" def create_prompt(cls, tools: Sequence[BaseTool]) -> BasePromptTemplate:\n",
|
|
" \"\"\"\n",
|
|
" Create the prompt.\n",
|
|
" \"\"\"\n",
|
|
" return TEST_PROMPT\n",
|
|
"\n",
|
|
" @classmethod\n",
|
|
" def _validate_tools(cls, tools: Sequence[BaseTool]) -> None:\n",
|
|
" \"\"\"\n",
|
|
" Method to validate the tools.\n",
|
|
" \"\"\"\n",
|
|
" if len(tools) != 6:\n",
|
|
" raise ValueError(\"The number of tools is invalid.\")\n",
|
|
" tool_names = {tool.name for tool in tools}\n",
|
|
" if tool_names != {\n",
|
|
" \"GetCommentsById\",\n",
|
|
" \"GetCommentsByUser\",\n",
|
|
" \"GenerateCommentResponse\",\n",
|
|
" \"GenerateCommentSentiment\",\n",
|
|
" \"GenerateCategorySentiment\",\n",
|
|
" \"Python_REPL\",\n",
|
|
" }:\n",
|
|
" raise ValueError(\"The name of tools is invalid.\")\n",
|
|
"\n",
|
|
" @property\n",
|
|
" def _agent_type(self) -> str:\n",
|
|
" return \"react-test\"\n",
|
|
"\n",
|
|
" @property\n",
|
|
" def finish_tool_name(self) -> str:\n",
|
|
" return \"Final Answer: \"\n",
|
|
"\n",
|
|
" @property\n",
|
|
" def observation_prefix(self) -> str:\n",
|
|
" return \"Observation: \"\n",
|
|
"\n",
|
|
" @property\n",
|
|
" def llm_prefix(self) -> str:\n",
|
|
" return \"Thought: \""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "yqgDvSyGOBHe"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain.agents.agent import AgentExecutor\n",
|
|
"\n",
|
|
"agent = ReActTestAgent.from_llm_and_tools(llm, tools, verbose=True)\n",
|
|
"\n",
|
|
"agent_executor = AgentExecutor.from_agent_and_tools(\n",
|
|
" agent=agent, tools=tools, verbose=True\n",
|
|
")\n",
|
|
"agent_executor.handle_parsing_errors = True"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "uXyfsvpby6_U"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"input = \"Category for the comment 919518\"\n",
|
|
"agent_executor.invoke(input)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "D1KkR7fzzEV7"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"input = \"Response for all comments by spicywebdesign.\"\n",
|
|
"agent_executor.invoke(input)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "LqybYiBfy0zP"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"input = \"Get comments by coleifer\"\n",
|
|
"agent_executor.invoke(input)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"name": "chain_of_thought_react.ipynb",
|
|
"toc_visible": true
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"name": "python3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
}
|