wehub-resource-sync
558 lines
32 KiB
Plaintext
558 lines
32 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Natural Language Queries for SQL using SLIM (Full End-to-End Example)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Introduction"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"You've might have heard of SQL. It's a widely used programming language for storing and processing information in relational databases - simply put, relational databases store data in tables, where each row stores an entity and each column stores an attribute for that entity.\n",
|
|
"\n",
|
|
"Let's say we have a table called customers in a relational database. If I wanted to access the names of all customers (customer_names) that have an annual_spend of at least $1000, I would have to formulate an SQL query like this:\n",
|
|
"```SQL\n",
|
|
"SELECT customer_names FROM customers WHERE annual_spend >= 1000\n",
|
|
"```\n",
|
|
"I would then run this query against the database to access my results.\n",
|
|
"\n",
|
|
"But what if AI 🤖 could do all this for us?\n",
|
|
"\n",
|
|
"LLMWare allows us to do just that, making use of small language models. These are models of a smaller scale, with fewer and less precise parameters that don't require much computational power to run. Their main advantage is that they run locally on a CPU, without an internet connection or a GPU. This enables use to make our queries entirely in natural language and still get accurate results!\n",
|
|
"\n",
|
|
"Let's look at an example of how to do this from start to finish."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### For Google Colab users"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"If you are using Colab for free, we highly recommend you activate the T4 GPU hardware accelerator. Our models are designed to run with at least 16GB of RAM, activating T4 will grant the notebook 16GB of GDDR6 RAM as apposed to the ~13GB Colab gives automatically.\n",
|
|
"\n",
|
|
"To activate T4:\n",
|
|
"1. click on the \"Runtime\" tab\n",
|
|
"2. click on \"Change runtime type\"\n",
|
|
"3. select T4 GPU under Hardware Accelerator\n",
|
|
"\n",
|
|
"NOTE: there is a weekly usage limit on using T4 for free"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Installing and importing dependencies"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Defaulting to user installation because normal site-packages is not writeable\n",
|
|
"Requirement already satisfied: llmware in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (0.2.14)\n",
|
|
"Requirement already satisfied: boto3==1.24.53 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (1.24.53)\n",
|
|
"Requirement already satisfied: huggingface-hub>=0.19.4 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.19.4)\n",
|
|
"Requirement already satisfied: numpy>=1.23.2 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (1.26.0)\n",
|
|
"Requirement already satisfied: openai>=1.0.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (1.13.3)\n",
|
|
"Requirement already satisfied: pymongo>=4.7.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (4.7.2)\n",
|
|
"Requirement already satisfied: tokenizers>=0.15.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.15.2)\n",
|
|
"Requirement already satisfied: torch>=1.13.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (2.1.0+cu121)\n",
|
|
"Requirement already satisfied: transformers>=4.36.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (4.38.2)\n",
|
|
"Requirement already satisfied: Wikipedia-API==0.6.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.6.0)\n",
|
|
"Requirement already satisfied: psycopg-binary==3.1.17 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (3.1.17)\n",
|
|
"Requirement already satisfied: psycopg==3.1.17 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (3.1.17)\n",
|
|
"Requirement already satisfied: pgvector==0.2.4 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.2.4)\n",
|
|
"Requirement already satisfied: colorama==0.4.6 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.4.6)\n",
|
|
"Requirement already satisfied: einops==0.7.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.7.0)\n",
|
|
"Requirement already satisfied: librosa>=0.10.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.10.2.post1)\n",
|
|
"Requirement already satisfied: botocore<1.28.0,>=1.27.53 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from boto3==1.24.53->llmware) (1.27.96)\n",
|
|
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from boto3==1.24.53->llmware) (1.0.1)\n",
|
|
"Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from boto3==1.24.53->llmware) (0.6.2)\n",
|
|
"Requirement already satisfied: typing-extensions>=4.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from psycopg==3.1.17->llmware) (4.8.0)\n",
|
|
"Requirement already satisfied: tzdata in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from psycopg==3.1.17->llmware) (2023.3)\n",
|
|
"Requirement already satisfied: requests in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from Wikipedia-API==0.6.0->llmware) (2.31.0)\n",
|
|
"Requirement already satisfied: filelock in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (3.13.1)\n",
|
|
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (2023.10.0)\n",
|
|
"Requirement already satisfied: tqdm>=4.42.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (4.66.1)\n",
|
|
"Requirement already satisfied: pyyaml>=5.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (6.0.1)\n",
|
|
"Requirement already satisfied: packaging>=20.9 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (23.1)\n",
|
|
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (3.0.1)\n",
|
|
"Requirement already satisfied: scipy>=1.2.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.11.3)\n",
|
|
"Requirement already satisfied: scikit-learn>=0.20.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.3.1)\n",
|
|
"Requirement already satisfied: joblib>=0.14 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.3.2)\n",
|
|
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (5.1.1)\n",
|
|
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (0.59.1)\n",
|
|
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (0.12.1)\n",
|
|
"Requirement already satisfied: pooch>=1.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.8.1)\n",
|
|
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (0.3.7)\n",
|
|
"Requirement already satisfied: lazy-loader>=0.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (0.4)\n",
|
|
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.0.8)\n",
|
|
"Requirement already satisfied: anyio<5,>=3.5.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from openai>=1.0.0->llmware) (4.0.0)\n",
|
|
"Requirement already satisfied: distro<2,>=1.7.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from openai>=1.0.0->llmware) (1.9.0)\n",
|
|
"Requirement already satisfied: httpx<1,>=0.23.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from openai>=1.0.0->llmware) (0.23.3)\n",
|
|
"Requirement already satisfied: pydantic<3,>=1.9.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from openai>=1.0.0->llmware) (2.6.4)\n",
|
|
"Requirement already satisfied: sniffio in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from openai>=1.0.0->llmware) (1.3.0)\n",
|
|
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from pymongo>=4.7.0->llmware) (2.6.1)\n",
|
|
"Requirement already satisfied: sympy in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from torch>=1.13.1->llmware) (1.12)\n",
|
|
"Requirement already satisfied: networkx in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from torch>=1.13.1->llmware) (3.2.1)\n",
|
|
"Requirement already satisfied: jinja2 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from torch>=1.13.1->llmware) (3.1.2)\n",
|
|
"Requirement already satisfied: regex!=2019.12.17 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from transformers>=4.36.0->llmware) (2023.12.25)\n",
|
|
"Requirement already satisfied: safetensors>=0.4.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from transformers>=4.36.0->llmware) (0.4.2)\n",
|
|
"Requirement already satisfied: idna>=2.8 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from anyio<5,>=3.5.0->openai>=1.0.0->llmware) (2.10)\n",
|
|
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from botocore<1.28.0,>=1.27.53->boto3==1.24.53->llmware) (2.8.2)\n",
|
|
"Requirement already satisfied: urllib3<1.27,>=1.25.4 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from botocore<1.28.0,>=1.27.53->boto3==1.24.53->llmware) (1.26.18)\n",
|
|
"Requirement already satisfied: certifi in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from httpx<1,>=0.23.0->openai>=1.0.0->llmware) (2023.7.22)\n",
|
|
"Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from httpx<1,>=0.23.0->openai>=1.0.0->llmware) (0.16.3)\n",
|
|
"Requirement already satisfied: rfc3986<2,>=1.3 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from rfc3986[idna2008]<2,>=1.3->httpx<1,>=0.23.0->openai>=1.0.0->llmware) (1.5.0)\n",
|
|
"Requirement already satisfied: llvmlite<0.43,>=0.42.0dev0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from numba>=0.51.0->librosa>=0.10.0->llmware) (0.42.0)\n",
|
|
"Requirement already satisfied: platformdirs>=2.5.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from pooch>=1.1->librosa>=0.10.0->llmware) (3.10.0)\n",
|
|
"Requirement already satisfied: annotated-types>=0.4.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from pydantic<3,>=1.9.0->openai>=1.0.0->llmware) (0.6.0)\n",
|
|
"Requirement already satisfied: pydantic-core==2.16.3 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from pydantic<3,>=1.9.0->openai>=1.0.0->llmware) (2.16.3)\n",
|
|
"Requirement already satisfied: charset-normalizer<4,>=2 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from requests->Wikipedia-API==0.6.0->llmware) (3.2.0)\n",
|
|
"Requirement already satisfied: threadpoolctl>=2.0.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from scikit-learn>=0.20.0->librosa>=0.10.0->llmware) (3.2.0)\n",
|
|
"Requirement already satisfied: cffi>=1.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from soundfile>=0.12.1->librosa>=0.10.0->llmware) (1.15.1)\n",
|
|
"Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from jinja2->torch>=1.13.1->llmware) (2.1.3)\n",
|
|
"Requirement already satisfied: mpmath>=0.19 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from sympy->torch>=1.13.1->llmware) (1.3.0)\n",
|
|
"Requirement already satisfied: pycparser in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from cffi>=1.0->soundfile>=0.12.1->librosa>=0.10.0->llmware) (2.21)\n",
|
|
"Requirement already satisfied: h11<0.15,>=0.13 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from httpcore<0.17.0,>=0.15.0->httpx<1,>=0.23.0->openai>=1.0.0->llmware) (0.14.0)\n",
|
|
"Requirement already satisfied: six>=1.5 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.28.0,>=1.27.53->boto3==1.24.53->llmware) (1.16.0)\n",
|
|
"Note: you may need to restart the kernel to use updated packages.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%pip install llmware"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"\n",
|
|
"from llmware.agents import SQLTables, LLMfx\n",
|
|
"from llmware.models import ModelCatalog\n",
|
|
"from llmware.configs import LLMWareConfig"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Worker function"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The process to generate an output for the given input query is as follows:\n",
|
|
"1. If `create_new_table` is set to `True`, then:\n",
|
|
" - Check to see if the SLIM SQL tool is already downloaded, if not, download it using the `ModelCatalog` class\n",
|
|
" - Load in the sample CSV file `customer_table.csv`\n",
|
|
" - Create an SQL table from the CSV file using the `SQLTables` class\n",
|
|
"2. Create an agent, an instance of the `LLMfx` class, and load in the SQL tool\n",
|
|
"3. For each query in the `query_list`, call the `query_db` function on the agent, which:\n",
|
|
" - looks up the table schema in the db using the table_name\n",
|
|
" - packages the text-2-sql query prompt\n",
|
|
" - executes sql method to convert the prompt into a sql query\n",
|
|
" - attempts to execute the sql query on the db\n",
|
|
" - returns the db results as 'research' output\n",
|
|
"4. Print the result from the `agent`'s `research_list`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def sql_e2e_test_script(table_name=\"customers1\",create_new_table=False):\n",
|
|
"\n",
|
|
" \"\"\" This is the end-to-end execution script. \"\"\"\n",
|
|
"\n",
|
|
" # create table if needed to set up\n",
|
|
" if create_new_table:\n",
|
|
"\n",
|
|
" # looks to pull sample csv 'customer_table.csv' from slim-sql-tool model package files\n",
|
|
" sql_tool_repo_path = os.path.join(LLMWareConfig().get_model_repo_path(), \"slim-sql-tool\")\n",
|
|
"\n",
|
|
" if not os.path.exists(sql_tool_repo_path):\n",
|
|
" ModelCatalog().load_model(\"llmware/slim-sql-tool\")\n",
|
|
"\n",
|
|
" files = os.listdir(sql_tool_repo_path)\n",
|
|
" csv_file = \"customer_table.csv\"\n",
|
|
"\n",
|
|
" if csv_file in files:\n",
|
|
"\n",
|
|
" # to create a testing table from a csv\n",
|
|
" sql_db = SQLTables(experimental=True)\n",
|
|
" sql_db.create_new_table_from_csv(sql_tool_repo_path, csv_file, table_name=table_name)\n",
|
|
" # end - creating table\n",
|
|
"\n",
|
|
" print(\"update: successfully created new db table\")\n",
|
|
" else:\n",
|
|
" print(\"something has gone wrong - could not find customer_table.csv inside the slim-sql-tool file package\")\n",
|
|
"\n",
|
|
" # query starts here\n",
|
|
" agent = LLMfx()\n",
|
|
" agent.load_tool(\"sql\", sample=False, get_logits=True, temperature=0.0)\n",
|
|
"\n",
|
|
" # Pass direct queries to the DB\n",
|
|
"\n",
|
|
" query_list = [\"Which customers have vip customer status of yes?\",\n",
|
|
" \"What is the highest annual spend of any customer?\",\n",
|
|
" \"Which customer has account number 1234953\",\n",
|
|
" \"Which customer has the lowest annual spend?\",\n",
|
|
" \"Is Susan Soinsin a vip customer?\"]\n",
|
|
"\n",
|
|
" for i, query in enumerate(query_list):\n",
|
|
"\n",
|
|
" # query_db method is doing all of the work\n",
|
|
" # -- looks up the table schema in the db using the table_name\n",
|
|
" # -- packages the text-2-sql query prompt\n",
|
|
" # -- executes sql method to convert the prompt into a sql query\n",
|
|
" # -- attempts to execute the sql query on the db\n",
|
|
" # -- returns the db results as 'research' output\n",
|
|
"\n",
|
|
" response = agent.query_db(query, table=table_name)\n",
|
|
"\n",
|
|
" for x in range(0,len(agent.research_list)):\n",
|
|
" print(\"research: \", x, agent.research_list[x])\n",
|
|
"\n",
|
|
" return 0"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Helper functions"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"If you want to delete a table in the experimental database, you can use the `delete_table()` function."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def delete_table(table_name):\n",
|
|
"\n",
|
|
" \"\"\" Start fresh in testing - delete table in experimental local SQLite DB \"\"\"\n",
|
|
"\n",
|
|
" sql_db = SQLTables(experimental=True)\n",
|
|
" sql_db.delete_table(table_name, confirm_delete=True)\n",
|
|
"\n",
|
|
" return True"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"If you want to delete the entire database, you use the `delete_db()` function."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def delete_db():\n",
|
|
"\n",
|
|
" \"\"\" Start fresh in testing - deletes SQLite DB and starts over. \"\"\"\n",
|
|
"\n",
|
|
" sql_db = SQLTables(experimental=True)\n",
|
|
" sql_db.delete_experimental_db(confirm_delete=True)\n",
|
|
"\n",
|
|
" return True"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Main block"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We can now run our code by calling the `sql_e2e_test_script` function."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "2d9d5f328d9b4b0f8bac965385d61d89",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Fetching 6 files: 0%| | 0/6 [00:00<?, ?it/s]"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "3d9518d0bdb742469d7efb5f2a589db8",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"library_table.csv: 0%| | 0.00/317 [00:00<?, ?B/s]"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "1b12a5ddf963431eb01893edb1bd2071",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
".gitattributes: 0%| | 0.00/1.57k [00:00<?, ?B/s]"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "3de118e4004e4d8fbde1b8114b5e80f4",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"README.md: 0%| | 0.00/1.98k [00:00<?, ?B/s]"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "bb89b15e1d9f45e7a927858b2906354e",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"config.json: 0%| | 0.00/3.96k [00:00<?, ?B/s]"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "e5c4e076065b4a01a73e0168ad6bed71",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"customer_table.csv: 0%| | 0.00/929 [00:00<?, ?B/s]"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "76d4ca969d11404fa50bb1822fe07e90",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"slim-sql.gguf: 0%| | 0.00/669M [00:00<?, ?B/s]"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"update: successfully created new db table\n",
|
|
"update: Launching LLMfx process\n",
|
|
"step - \t1 - \tcreating object - ready to start processing.\n",
|
|
"step - \t2 - \tloading tool - sql\n",
|
|
"step - \t3 - \tloading new processing text - 1 new entries\n",
|
|
"step - \t4 - \texecuting function call - deploying - text-to-sql\n",
|
|
"\t\t\t\t -- query - Which customers have vip customer status of yes?\n",
|
|
"\t\t\t\t -- table_schema - CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )\n",
|
|
"step - \t5 - \texecuting function call - getting response - sql\n",
|
|
"\t\t\t\t -- llm_response - SELECT customer_name FROM customer1 WHERE vip_customer = 'yes'\n",
|
|
"\t\t\t\t -- output type - text\n",
|
|
"\t\t\t\t -- usage - {'input': 57, 'output': 16, 'total': 73, 'metric': 'tokens', 'processing_time': 0.877871036529541}\n",
|
|
"step - \t6 - \texecuting research call - executing query on db\n",
|
|
"\t\t\t\t -- db - C:\\Users\\prash\\llmware_data\\accounts\\sqlite_experimental.db\n",
|
|
"\t\t\t\t -- sql_query - SELECT customer_name FROM customer1 WHERE vip_customer = 'yes'\n",
|
|
"step - \t7 - \texecuting research - getting response - sql\n",
|
|
"\t\t\t\t -- result - [('Martha Williams',), ('John Jones',), ('Arvind Arora',), ('Vinod Aggarwal',), ('Allison Winters',), ('Olivia Smith',), ('Alan Wang',), ('Wilmer Rodriguez',), ('Eliza Listron',), ('Douglas Hudson',)]\n",
|
|
"step - \t8 - \tloading new processing text - 1 new entries\n",
|
|
"step - \t9 - \texecuting function call - deploying - text-to-sql\n",
|
|
"\t\t\t\t -- query - What is the highest annual spend of any customer?\n",
|
|
"\t\t\t\t -- table_schema - CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )\n",
|
|
"step - \t10 - \texecuting function call - getting response - sql\n",
|
|
"\t\t\t\t -- llm_response - SELECT MAX(annual_spend) FROM customer1\n",
|
|
"\t\t\t\t -- output type - text\n",
|
|
"\t\t\t\t -- usage - {'input': 56, 'output': 12, 'total': 68, 'metric': 'tokens', 'processing_time': 0.4298675060272217}\n",
|
|
"step - \t11 - \texecuting research call - executing query on db\n",
|
|
"\t\t\t\t -- db - C:\\Users\\prash\\llmware_data\\accounts\\sqlite_experimental.db\n",
|
|
"\t\t\t\t -- sql_query - SELECT MAX(annual_spend) FROM customer1\n",
|
|
"step - \t12 - \texecuting research - getting response - sql\n",
|
|
"\t\t\t\t -- result - [(93540,)]\n",
|
|
"step - \t13 - \tloading new processing text - 1 new entries\n",
|
|
"step - \t14 - \texecuting function call - deploying - text-to-sql\n",
|
|
"\t\t\t\t -- query - Which customer has account number 1234953\n",
|
|
"\t\t\t\t -- table_schema - CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )\n",
|
|
"step - \t15 - \texecuting function call - getting response - sql\n",
|
|
"\t\t\t\t -- llm_response - SELECT customer_name FROM customer1 WHERE account_number = 1234953\n",
|
|
"\t\t\t\t -- output type - text\n",
|
|
"\t\t\t\t -- usage - {'input': 60, 'output': 20, 'total': 80, 'metric': 'tokens', 'processing_time': 0.6454498767852783}\n",
|
|
"step - \t16 - \texecuting research call - executing query on db\n",
|
|
"\t\t\t\t -- db - C:\\Users\\prash\\llmware_data\\accounts\\sqlite_experimental.db\n",
|
|
"\t\t\t\t -- sql_query - SELECT customer_name FROM customer1 WHERE account_number = 1234953\n",
|
|
"step - \t17 - \texecuting research - getting response - sql\n",
|
|
"\t\t\t\t -- result - [('Melinda Lyons',)]\n",
|
|
"step - \t18 - \tloading new processing text - 1 new entries\n",
|
|
"step - \t19 - \texecuting function call - deploying - text-to-sql\n",
|
|
"\t\t\t\t -- query - Which customer has the lowest annual spend?\n",
|
|
"\t\t\t\t -- table_schema - CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )\n",
|
|
"step - \t20 - \texecuting function call - getting response - sql\n",
|
|
"\t\t\t\t -- llm_response - SELECT customer_name FROM customer1 ORDER BY annual_spend LIMIT 1\n",
|
|
"\t\t\t\t -- output type - text\n",
|
|
"\t\t\t\t -- usage - {'input': 55, 'output': 16, 'total': 71, 'metric': 'tokens', 'processing_time': 0.5111117362976074}\n",
|
|
"step - \t21 - \texecuting research call - executing query on db\n",
|
|
"\t\t\t\t -- db - C:\\Users\\prash\\llmware_data\\accounts\\sqlite_experimental.db\n",
|
|
"\t\t\t\t -- sql_query - SELECT customer_name FROM customer1 ORDER BY annual_spend LIMIT 1\n",
|
|
"step - \t22 - \texecuting research - getting response - sql\n",
|
|
"\t\t\t\t -- result - [('Michael Rogers',)]\n",
|
|
"step - \t23 - \tloading new processing text - 1 new entries\n",
|
|
"step - \t24 - \texecuting function call - deploying - text-to-sql\n",
|
|
"\t\t\t\t -- query - Is Susan Soinsin a vip customer?\n",
|
|
"\t\t\t\t -- table_schema - CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\u001b[37mWARNING: update: delete sqlite experimental db - table - customer1 \u001b[39m\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"step - \t25 - \texecuting function call - getting response - sql\n",
|
|
"\t\t\t\t -- llm_response - SELECT vip_customer FROM customer1 WHERE customer_name = 'Susan Soinsin'\n",
|
|
"\t\t\t\t -- output type - text\n",
|
|
"\t\t\t\t -- usage - {'input': 56, 'output': 21, 'total': 77, 'metric': 'tokens', 'processing_time': 0.6235740184783936}\n",
|
|
"step - \t26 - \texecuting research call - executing query on db\n",
|
|
"\t\t\t\t -- db - C:\\Users\\prash\\llmware_data\\accounts\\sqlite_experimental.db\n",
|
|
"\t\t\t\t -- sql_query - SELECT vip_customer FROM customer1 WHERE customer_name = 'Susan Soinsin'\n",
|
|
"step - \t27 - \texecuting research - getting response - sql\n",
|
|
"\t\t\t\t -- result - [('no',)]\n",
|
|
"research: 0 {'step': 6, 'tool': 'sql', 'db_response': [('Martha Williams',), ('John Jones',), ('Arvind Arora',), ('Vinod Aggarwal',), ('Allison Winters',), ('Olivia Smith',), ('Alan Wang',), ('Wilmer Rodriguez',), ('Eliza Listron',), ('Douglas Hudson',)], 'sql_query': \"SELECT customer_name FROM customer1 WHERE vip_customer = 'yes'\", 'query': 'Which customers have vip customer status of yes?', 'db': 'C:\\\\Users\\\\prash\\\\llmware_data\\\\accounts\\\\sqlite_experimental.db', 'work_item': 'CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )'}\n",
|
|
"research: 1 {'step': 11, 'tool': 'sql', 'db_response': [(93540,)], 'sql_query': 'SELECT MAX(annual_spend) FROM customer1', 'query': 'What is the highest annual spend of any customer?', 'db': 'C:\\\\Users\\\\prash\\\\llmware_data\\\\accounts\\\\sqlite_experimental.db', 'work_item': 'CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )'}\n",
|
|
"research: 2 {'step': 16, 'tool': 'sql', 'db_response': [('Melinda Lyons',)], 'sql_query': 'SELECT customer_name FROM customer1 WHERE account_number = 1234953', 'query': 'Which customer has account number 1234953', 'db': 'C:\\\\Users\\\\prash\\\\llmware_data\\\\accounts\\\\sqlite_experimental.db', 'work_item': 'CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )'}\n",
|
|
"research: 3 {'step': 21, 'tool': 'sql', 'db_response': [('Michael Rogers',)], 'sql_query': 'SELECT customer_name FROM customer1 ORDER BY annual_spend LIMIT 1', 'query': 'Which customer has the lowest annual spend?', 'db': 'C:\\\\Users\\\\prash\\\\llmware_data\\\\accounts\\\\sqlite_experimental.db', 'work_item': 'CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )'}\n",
|
|
"research: 4 {'step': 26, 'tool': 'sql', 'db_response': [('no',)], 'sql_query': \"SELECT vip_customer FROM customer1 WHERE customer_name = 'Susan Soinsin'\", 'query': 'Is Susan Soinsin a vip customer?', 'db': 'C:\\\\Users\\\\prash\\\\llmware_data\\\\accounts\\\\sqlite_experimental.db', 'work_item': 'CREATE TABLE customer1 (customer_name text, account_number integer, customer_level text, vip_customer text, annual_spend integer, user_name text )'}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"if __name__ == \"__main__\":\n",
|
|
"\n",
|
|
" ModelCatalog().get_llm_toolkit(tool_list=[\"sql\"])\n",
|
|
"\n",
|
|
" # run an end-to-end test\n",
|
|
" sql_e2e_test_script(table_name=\"customer1\",create_new_table=True)\n",
|
|
"\n",
|
|
" # third - delete and start fresh for further testing\n",
|
|
" delete_table(\"customer1\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We now have the output of our program. The output for each natural language question is a dictionary containing a lot of detailed information about the steps carried out by the agent, but here are some of the more interesting parts of it:\n",
|
|
"- `db_response` gives us what we want, the answer to the question we ask\n",
|
|
"- `sql_query` shows us the SQL query that was generated from our natural language question using the SLIM SQL tool."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.0"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|