wehub-resource-sync
403 lines
537 KiB
Plaintext
Vendored
403 lines
537 KiB
Plaintext
Vendored
{
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0,
|
|
"metadata": {
|
|
"colab": {
|
|
"provenance": []
|
|
},
|
|
"kernelspec": {
|
|
"name": "python3",
|
|
"display_name": "Python 3"
|
|
},
|
|
"language_info": {
|
|
"name": "python"
|
|
}
|
|
},
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"[](https://colab.research.google.com/drive/1lg7IjKrMKyCkrqq4xffviw7DaOac-I3y?usp=sharing)"
|
|
],
|
|
"metadata": {
|
|
"id": "LtPu3yuiEwRe"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"# A Simple implementation of the ReAct Agents with Porktey\n",
|
|
"This notebook has been forked from Simon Willison's implementation of simple Python implementation of the ReAct pattern for LLMs- https://til.simonwillison.net/llms/python-react-pattern.\n",
|
|
"\n",
|
|
"ReACT (Reasoning and Action) agent is a framework designed to integrate the reasoning capabilities of large language models (LLMs) with the ability to take actionable steps, creating a more systematic system that can understand and process information, evaluate decisions, take appropriate actions, communicate responses, and track actions.\n",
|
|
"\n",
|
|
"\n",
|
|
"[Portkey](https://portkey.ai) helps you grain granular insights into your Agents workflow\n",
|
|
"Key Features. By routing your agent's requests through Portkey, you gain access to the following production-grade features:\n",
|
|
"\n",
|
|
"1. **Interoperability**\n",
|
|
"- Easily switch between 200+ LLMs\n",
|
|
"- Call various LLMs (Anthropic, Gemini, Mistral, Azure OpenAI, Google Vertex AI, AWS Bedrock, and more)\n",
|
|
"- Choose the best LLM for your ReAct based agent\n",
|
|
"\n",
|
|
"2. **Observability**\n",
|
|
"- Get granular insights into cost, tokens used, and latency using Portkey for understanding ReAct agents' performance\n",
|
|
"- Helps you identify bottlenecks and optimize resource usage\n",
|
|
"\n",
|
|
"\n",
|
|
"3. **Reliability**\n",
|
|
"- Long ReAct pipelines with multiple API calls can fail at any stage, disrupting the entire process.\n",
|
|
"- Portkey offers built-in fallbacks between different LLMs or providers\n",
|
|
"- Implements load-balancing across multiple instances or API keys\n",
|
|
"- Provides automatic retries and request timeouts\n",
|
|
"- Makes ReAct more reliable and resilient\n",
|
|
"\n",
|
|
"\n",
|
|
"4. **Caching** (simple & semantic)\n",
|
|
"- Store frequently accessed data to reduce expensive API calls\n",
|
|
"- Portkey offers Simple and Semantics caching for your ReAct agents.\n",
|
|
"\n",
|
|
"5. **Logs**\n",
|
|
"- ReAct run is complex. Logs are essential for diagnosing issues, understanding LLMs behavior, and improving performance.\n",
|
|
"- Portkey provides detailed records of actions and tool use\n",
|
|
"- Crucial for debugging and managing complex ReAct runs\n",
|
|
"\n",
|
|
"6. **Prompt Management**\n",
|
|
"- Manage ReAct Prompts in Portkey's Prompt Library\n",
|
|
"- Try new versions of Prompts without changing code\n",
|
|
"- Utilize Portkey's inbuilt Prompt Management Suite\n",
|
|
"\n",
|
|
"\n",
|
|
"For more information on how to use these features please refer to the [Portkey documentation](https://docs.portkey.ai/docs)."
|
|
],
|
|
"metadata": {
|
|
"id": "_NMXSDOYB-hi"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Installing Dependencies"
|
|
],
|
|
"metadata": {
|
|
"id": "-Xu2ANcSDezG"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"!pip install portkey-ai openai"
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"collapsed": true,
|
|
"id": "CNV8tqDc9Ggc",
|
|
"outputId": "3e1354b7-970b-43ba-c921-2b5d6cb99923"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": [
|
|
{
|
|
"output_type": "stream",
|
|
"name": "stdout",
|
|
"text": [
|
|
"Collecting portkey-ai\n",
|
|
" Downloading portkey_ai-1.7.1-py3-none-any.whl.metadata (7.3 kB)\n",
|
|
"Collecting openai\n",
|
|
" Downloading openai-1.37.1-py3-none-any.whl.metadata (22 kB)\n",
|
|
"Collecting httpx (from portkey-ai)\n",
|
|
" Downloading httpx-0.27.0-py3-none-any.whl.metadata (7.2 kB)\n",
|
|
"Collecting mypy<2.0,>=0.991 (from portkey-ai)\n",
|
|
" Downloading mypy-1.11.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl.metadata (1.9 kB)\n",
|
|
"Requirement already satisfied: typing-extensions<5.0,>=4.7.1 in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (4.12.2)\n",
|
|
"Requirement already satisfied: pydantic>=1.10.12 in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (2.8.2)\n",
|
|
"Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (3.7.1)\n",
|
|
"Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from portkey-ai) (1.7.0)\n",
|
|
"Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (1.3.1)\n",
|
|
"Collecting cached-property (from portkey-ai)\n",
|
|
" Downloading cached_property-1.5.2-py2.py3-none-any.whl.metadata (11 kB)\n",
|
|
"Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from portkey-ai) (4.66.4)\n",
|
|
"Collecting types-requests (from portkey-ai)\n",
|
|
" Downloading types_requests-2.32.0.20240712-py3-none-any.whl.metadata (1.9 kB)\n",
|
|
"Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->portkey-ai) (3.7)\n",
|
|
"Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->portkey-ai) (1.2.2)\n",
|
|
"Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->portkey-ai) (2024.7.4)\n",
|
|
"Collecting httpcore==1.* (from httpx->portkey-ai)\n",
|
|
" Downloading httpcore-1.0.5-py3-none-any.whl.metadata (20 kB)\n",
|
|
"Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx->portkey-ai)\n",
|
|
" Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)\n",
|
|
"Collecting mypy-extensions>=1.0.0 (from mypy<2.0,>=0.991->portkey-ai)\n",
|
|
" Downloading mypy_extensions-1.0.0-py3-none-any.whl.metadata (1.1 kB)\n",
|
|
"Requirement already satisfied: tomli>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from mypy<2.0,>=0.991->portkey-ai) (2.0.1)\n",
|
|
"Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10.12->portkey-ai) (0.7.0)\n",
|
|
"Requirement already satisfied: pydantic-core==2.20.1 in /usr/local/lib/python3.10/dist-packages (from pydantic>=1.10.12->portkey-ai) (2.20.1)\n",
|
|
"Requirement already satisfied: urllib3>=2 in /usr/local/lib/python3.10/dist-packages (from types-requests->portkey-ai) (2.0.7)\n",
|
|
"Downloading portkey_ai-1.7.1-py3-none-any.whl (405 kB)\n",
|
|
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m406.0/406.0 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
|
|
"\u001b[?25hDownloading openai-1.37.1-py3-none-any.whl (337 kB)\n",
|
|
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m337.0/337.0 kB\u001b[0m \u001b[31m20.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
|
|
"\u001b[?25hDownloading httpx-0.27.0-py3-none-any.whl (75 kB)\n",
|
|
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
|
|
"\u001b[?25hDownloading httpcore-1.0.5-py3-none-any.whl (77 kB)\n",
|
|
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m4.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
|
|
"\u001b[?25hDownloading mypy-1.11.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (12.5 MB)\n",
|
|
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.5/12.5 MB\u001b[0m \u001b[31m65.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
|
|
"\u001b[?25hDownloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)\n",
|
|
"Downloading types_requests-2.32.0.20240712-py3-none-any.whl (15 kB)\n",
|
|
"Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
|
|
"Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
|
|
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
|
|
"\u001b[?25hInstalling collected packages: cached-property, types-requests, mypy-extensions, h11, mypy, httpcore, httpx, portkey-ai, openai\n",
|
|
"Successfully installed cached-property-1.5.2 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 mypy-1.11.1 mypy-extensions-1.0.0 openai-1.37.1 portkey-ai-1.7.1 types-requests-2.32.0.20240712\n"
|
|
]
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Importing Libraries"
|
|
],
|
|
"metadata": {
|
|
"id": "rDScTDRhDjAM"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"import re\n",
|
|
"import httpx\n",
|
|
"\n",
|
|
"from openai import OpenAI\n",
|
|
"from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders\n"
|
|
],
|
|
"metadata": {
|
|
"id": "tTtPkKnP7vR5"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Creating A Basic ReAct agent with Portkey"
|
|
],
|
|
"metadata": {
|
|
"id": "VqStK8BaDxO8"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"client = OpenAI(\n",
|
|
" api_key=\"OPENAI_API_KEY\", #Enter your OpenAI API Key\n",
|
|
" base_url=PORTKEY_GATEWAY_URL,\n",
|
|
" default_headers=createHeaders(\n",
|
|
" provider=\"openai\",\n",
|
|
" api_key=\"PORTKEY_API_KEY\", #Enter your Portkey API Key\n",
|
|
" trace_id=\"react-agent\" #Trace ID for agent\n",
|
|
" )\n",
|
|
")\n",
|
|
"\n",
|
|
"\n",
|
|
"class ChatBot:\n",
|
|
" def __init__(self, system=\"\"):\n",
|
|
" self.system = system\n",
|
|
" self.messages = []\n",
|
|
" if self.system:\n",
|
|
" self.messages.append({\"role\": \"system\", \"content\": system})\n",
|
|
"\n",
|
|
" def __call__(self, message):\n",
|
|
" self.messages.append({\"role\": \"user\", \"content\": message})\n",
|
|
" result = self.execute()\n",
|
|
" self.messages.append({\"role\": \"assistant\", \"content\": result})\n",
|
|
" return result\n",
|
|
"\n",
|
|
" def execute(self):\n",
|
|
" completion = client.chat.completions.create(model=\"gpt-3.5-turbo\", messages=self.messages)\n",
|
|
"\n",
|
|
" return completion.choices[0].message.content\n",
|
|
"\n",
|
|
"prompt = \"\"\"\n",
|
|
"You run in a loop of Thought, Action, PAUSE, Observation.\n",
|
|
"At the end of the loop you output an Answer\n",
|
|
"Use Thought to describe your thoughts about the question you have been asked.\n",
|
|
"Use Action to run one of the actions available to you - then return PAUSE.\n",
|
|
"Observation will be the result of running those actions.\n",
|
|
"\n",
|
|
"Your available actions are:\n",
|
|
"\n",
|
|
"calculate:\n",
|
|
"e.g. calculate: 4 * 7 / 3\n",
|
|
"Runs a calculation and returns the number - uses Python so be sure to use floating point syntax if necessary\n",
|
|
"\n",
|
|
"wikipedia:\n",
|
|
"e.g. wikipedia: Django\n",
|
|
"Returns a summary from searching Wikipedia\n",
|
|
"\n",
|
|
"simon_blog_search:\n",
|
|
"e.g. simon_blog_search: Django\n",
|
|
"Search Simon's blog for that term\n",
|
|
"\n",
|
|
"Always look things up on Wikipedia if you have the opportunity to do so.\n",
|
|
"\n",
|
|
"Example session:\n",
|
|
"\n",
|
|
"Question: What is the capital of France?\n",
|
|
"Thought: I should look up France on Wikipedia\n",
|
|
"Action: wikipedia: France\n",
|
|
"PAUSE\n",
|
|
"\n",
|
|
"You will be called again with this:\n",
|
|
"\n",
|
|
"Observation: France is a country. The capital is Paris.\n",
|
|
"\n",
|
|
"You then output:\n",
|
|
"\n",
|
|
"Answer: The capital of France is Paris\n",
|
|
"\"\"\".strip()\n",
|
|
"\n",
|
|
"\n",
|
|
"action_re = re.compile('^Action: (\\w+): (.*)$')\n",
|
|
"\n",
|
|
"def query(question, max_turns=5):\n",
|
|
" i = 0\n",
|
|
" bot = ChatBot(prompt)\n",
|
|
" next_prompt = question\n",
|
|
" while i < max_turns:\n",
|
|
" i += 1\n",
|
|
" result = bot(next_prompt)\n",
|
|
" print(result)\n",
|
|
" actions = [action_re.match(a) for a in result.split('\\n') if action_re.match(a)]\n",
|
|
" if actions:\n",
|
|
" # There is an action to run\n",
|
|
" action, action_input = actions[0].groups()\n",
|
|
" if action not in known_actions:\n",
|
|
" raise Exception(\"Unknown action: {}: {}\".format(action, action_input))\n",
|
|
" print(\" -- running {} {}\".format(action, action_input))\n",
|
|
" observation = known_actions[action](action_input)\n",
|
|
" print(\"Observation:\", observation)\n",
|
|
" next_prompt = \"Observation: {}\".format(observation)\n",
|
|
" else:\n",
|
|
" return\n",
|
|
"\n",
|
|
"\n",
|
|
"def wikipedia(q):\n",
|
|
" return httpx.get(\"https://en.wikipedia.org/w/api.php\", params={\n",
|
|
" \"action\": \"query\",\n",
|
|
" \"list\": \"search\",\n",
|
|
" \"srsearch\": q,\n",
|
|
" \"format\": \"json\"\n",
|
|
" }).json()[\"query\"][\"search\"][0][\"snippet\"]\n",
|
|
"\n",
|
|
"\n",
|
|
"def simon_blog_search(q):\n",
|
|
" results = httpx.get(\"https://datasette.simonwillison.net/simonwillisonblog.json\", params={\n",
|
|
" \"sql\": \"\"\"\n",
|
|
" select\n",
|
|
" blog_entry.title || ': ' || substr(html_strip_tags(blog_entry.body), 0, 1000) as text,\n",
|
|
" blog_entry.created\n",
|
|
" from\n",
|
|
" blog_entry join blog_entry_fts on blog_entry.rowid = blog_entry_fts.rowid\n",
|
|
" where\n",
|
|
" blog_entry_fts match escape_fts(:q)\n",
|
|
" order by\n",
|
|
" blog_entry_fts.rank\n",
|
|
" limit\n",
|
|
" 1\"\"\".strip(),\n",
|
|
" \"_shape\": \"array\",\n",
|
|
" \"q\": q,\n",
|
|
" }).json()\n",
|
|
" return results[0][\"text\"]\n",
|
|
"\n",
|
|
"def calculate(what):\n",
|
|
" return eval(what)\n",
|
|
"\n",
|
|
"known_actions = {\n",
|
|
" \"wikipedia\": wikipedia,\n",
|
|
" \"calculate\": calculate,\n",
|
|
" \"simon_blog_search\": simon_blog_search\n",
|
|
"}"
|
|
],
|
|
"metadata": {
|
|
"id": "6M-Cxvz5DtEv"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Asking the Query and Output"
|
|
],
|
|
"metadata": {
|
|
"id": "kvw-VdoQEPNI"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "G4W2z1m07ufR",
|
|
"outputId": "97fda9e5-fe12-4bc5-e949-71a023dc2c72"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"output_type": "stream",
|
|
"name": "stdout",
|
|
"text": [
|
|
"Thought: I should look up India on Wikipedia to find information about its neighboring countries.\n",
|
|
"Action: wikipedia: India\n",
|
|
"PAUSE\n",
|
|
" -- running wikipedia India\n",
|
|
"Observation: <span class=\"searchmatch\">India</span>, officially the Republic of <span class=\"searchmatch\">India</span> (ISO: Bhārat Gaṇarājya), is a country in South Asia. It is the seventh-largest country by area; the most populous\n",
|
|
"Answer: India shares its borders with several countries in South Asia.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"query(\"What does India share borders with?\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"## Observability with Portkey\n"
|
|
],
|
|
"metadata": {
|
|
"id": "_3NOpSe1FL35"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [],
|
|
"metadata": {
|
|
"id": "JVQm-UoVlOF3"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"\n",
|
|
"By routing requests through Portkey you can track a number of metrics like - tokens used, latency, cost, etc.\n",
|
|
"\n",
|
|
"Here's a screenshot of the dashboard you get with Portkey!\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"## **Logs**\n",
|
|
""
|
|
],
|
|
"metadata": {
|
|
"id": "wNnQkUyrE_wd"
|
|
}
|
|
}
|
|
]
|
|
}
|