wehub-resource-sync
475 lines
24 KiB
Plaintext
475 lines
24 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Local Ask and Answer Dueling Bots with Small Language Models"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Introduction"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"You've probably asked a question to a language model before and then had it give you an answer. After all, this is what we most commonly use language models for.\n",
|
||
"\n",
|
||
"But have you ever received a question from a language model? While not as common, this application of AI has diverse use cases in areas like education, where you might want a model to give you practice questions for a test, and in sales enablement, where you question your business's sales team about your products to improve their ability make sales.\n",
|
||
"\n",
|
||
"Now, what if we had a face off⚔️ between two different models: one that asked questions about a topic and another that answered them? All without human intervention?\n",
|
||
"\n",
|
||
"In this notebook, we're going to look at exactly that. We'll provide a sample passage about OpenAI's AI safety team as context to our models. We'll then let our models duel it out! One model will ask questions based on this passage, and another model will respond!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### For Google Colab users"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"If you are using Colab for free, we highly recommend you activate the T4 GPU hardware accelerator. Our models are designed to run with at least 16GB of RAM, activating T4 will grant the notebook 16GB of GDDR6 RAM as apposed to the ~13GB Colab gives automatically.\n",
|
||
"\n",
|
||
"To activate T4:\n",
|
||
"1. click on the \"Runtime\" tab\n",
|
||
"2. click on \"Change runtime type\"\n",
|
||
"3. select T4 GPU under Hardware Accelerator\n",
|
||
"\n",
|
||
"NOTE: there is a weekly usage limit on using T4 for free"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Streamlit example"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"We have an [interactive Streamlit program](https://github.com/llmware-ai/llmware/blob/main/examples/UI/dueling_chatbot.py) for this example in our repository. To run it, navigate to the directory the file is located in, and run `streamlit run dueling_chatbot.py` in a terminal."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Installing and importing dependencies"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Defaulting to user installation because normal site-packages is not writeable\n",
|
||
"Requirement already satisfied: llmware in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (0.3.3)\n",
|
||
"Requirement already satisfied: boto3>=1.24.53 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (1.24.53)\n",
|
||
"Requirement already satisfied: huggingface-hub>=0.19.4 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.19.4)\n",
|
||
"Requirement already satisfied: numpy>=1.23.2 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (1.26.0)\n",
|
||
"Requirement already satisfied: pymongo>=4.7.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (4.7.2)\n",
|
||
"Requirement already satisfied: tokenizers>=0.15.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.15.2)\n",
|
||
"Requirement already satisfied: psycopg-binary==3.1.17 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (3.1.17)\n",
|
||
"Requirement already satisfied: psycopg==3.1.17 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (3.1.17)\n",
|
||
"Requirement already satisfied: pgvector==0.2.4 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.2.4)\n",
|
||
"Requirement already satisfied: colorama==0.4.6 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.4.6)\n",
|
||
"Requirement already satisfied: librosa>=0.10.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from llmware) (0.10.2.post1)\n",
|
||
"Requirement already satisfied: typing-extensions>=4.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from psycopg==3.1.17->llmware) (4.8.0)\n",
|
||
"Requirement already satisfied: tzdata in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from psycopg==3.1.17->llmware) (2023.3)\n",
|
||
"Requirement already satisfied: botocore<1.28.0,>=1.27.53 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from boto3>=1.24.53->llmware) (1.27.96)\n",
|
||
"Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from boto3>=1.24.53->llmware) (1.0.1)\n",
|
||
"Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from boto3>=1.24.53->llmware) (0.6.2)\n",
|
||
"Requirement already satisfied: filelock in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (3.13.1)\n",
|
||
"Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (2023.10.0)\n",
|
||
"Requirement already satisfied: requests in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (2.31.0)\n",
|
||
"Requirement already satisfied: tqdm>=4.42.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (4.66.1)\n",
|
||
"Requirement already satisfied: pyyaml>=5.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (6.0.1)\n",
|
||
"Requirement already satisfied: packaging>=20.9 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from huggingface-hub>=0.19.4->llmware) (23.1)\n",
|
||
"Requirement already satisfied: audioread>=2.1.9 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (3.0.1)\n",
|
||
"Requirement already satisfied: scipy>=1.2.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.11.3)\n",
|
||
"Requirement already satisfied: scikit-learn>=0.20.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.3.1)\n",
|
||
"Requirement already satisfied: joblib>=0.14 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.3.2)\n",
|
||
"Requirement already satisfied: decorator>=4.3.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (5.1.1)\n",
|
||
"Requirement already satisfied: numba>=0.51.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (0.59.1)\n",
|
||
"Requirement already satisfied: soundfile>=0.12.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (0.12.1)\n",
|
||
"Requirement already satisfied: pooch>=1.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.8.1)\n",
|
||
"Requirement already satisfied: soxr>=0.3.2 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (0.3.7)\n",
|
||
"Requirement already satisfied: lazy-loader>=0.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (0.4)\n",
|
||
"Requirement already satisfied: msgpack>=1.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from librosa>=0.10.0->llmware) (1.0.8)\n",
|
||
"Requirement already satisfied: dnspython<3.0.0,>=1.16.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from pymongo>=4.7.0->llmware) (2.6.1)\n",
|
||
"Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from botocore<1.28.0,>=1.27.53->boto3>=1.24.53->llmware) (2.8.2)\n",
|
||
"Requirement already satisfied: urllib3<1.27,>=1.25.4 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from botocore<1.28.0,>=1.27.53->boto3>=1.24.53->llmware) (1.26.18)\n",
|
||
"Requirement already satisfied: llvmlite<0.43,>=0.42.0dev0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from numba>=0.51.0->librosa>=0.10.0->llmware) (0.42.0)\n",
|
||
"Requirement already satisfied: platformdirs>=2.5.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from pooch>=1.1->librosa>=0.10.0->llmware) (3.10.0)\n",
|
||
"Requirement already satisfied: charset-normalizer<4,>=2 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from requests->huggingface-hub>=0.19.4->llmware) (3.2.0)\n",
|
||
"Requirement already satisfied: idna<4,>=2.5 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from requests->huggingface-hub>=0.19.4->llmware) (2.10)\n",
|
||
"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from requests->huggingface-hub>=0.19.4->llmware) (2023.7.22)\n",
|
||
"Requirement already satisfied: threadpoolctl>=2.0.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from scikit-learn>=0.20.0->librosa>=0.10.0->llmware) (3.2.0)\n",
|
||
"Requirement already satisfied: cffi>=1.0 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from soundfile>=0.12.1->librosa>=0.10.0->llmware) (1.15.1)\n",
|
||
"Requirement already satisfied: pycparser in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from cffi>=1.0->soundfile>=0.12.1->librosa>=0.10.0->llmware) (2.21)\n",
|
||
"Requirement already satisfied: six>=1.5 in c:\\users\\prash\\appdata\\roaming\\python\\python311\\site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.28.0,>=1.27.53->boto3>=1.24.53->llmware) (1.16.0)\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\n",
|
||
"[notice] A new release of pip is available: 24.0 -> 24.1.2\n",
|
||
"[notice] To update, run: python.exe -m pip install --upgrade pip\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"!pip install llmware"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from llmware.models import ModelCatalog"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Generating questions"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"In this function, we will see how we can generate questions about the `source_passage` using the slim-q-gen-tiny-tool. The workflow for this function is as follows:\n",
|
||
"- Load in the the model using `ModelCatalog`\n",
|
||
"- Use `function_call()` to generate questions from the model `number_of_tries` times\n",
|
||
"- Add the generated question to the `questions` list only if it is unique and is not already in the list\n",
|
||
"- Output the `questions`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def hello_world_test(source_passage, q_model=\"slim-q-gen-tiny-tool\", number_of_tries=10, question_type=\"question\", temperature=0.5):\n",
|
||
"\n",
|
||
" \"\"\" Shows a basic example of generating questions from a text passage, running a number of inferences,\n",
|
||
" and then keeping only the unique questions generated.\n",
|
||
"\n",
|
||
" -- source_passage = text passage\n",
|
||
" -- number_of_tries = integer number of times to call the model to generate a question\n",
|
||
" -- question_type = \"question\" | \"boolean\" | \"multiple choice\"\n",
|
||
"\n",
|
||
" \"\"\"\n",
|
||
"\n",
|
||
" # recommend using temperature of 0.2 - 0.8 - for multiple choice, use lower end of the range\n",
|
||
" q_model = ModelCatalog().load_model(q_model, sample=True, temperature=temperature)\n",
|
||
"\n",
|
||
" questions = []\n",
|
||
"\n",
|
||
" for x in range(0, number_of_tries):\n",
|
||
"\n",
|
||
" response = q_model.function_call(source_passage, params=[question_type], get_logits=False)\n",
|
||
"\n",
|
||
" # expect response in the form of: \"llm_response\": {\"question\": [\"generated question?\"] }\n",
|
||
"\n",
|
||
" if response:\n",
|
||
" if \"llm_response\" in response:\n",
|
||
" if \"question\" in response[\"llm_response\"]:\n",
|
||
" new_q = response[\"llm_response\"][\"question\"]\n",
|
||
"\n",
|
||
" # keep only new questions\n",
|
||
" if new_q not in questions:\n",
|
||
" questions.append(new_q)\n",
|
||
"\n",
|
||
" print(f\"inference {x} - response: {response['llm_response']}\")\n",
|
||
"\n",
|
||
" print(f\"\\nDe-duped list of questions created\\n\")\n",
|
||
" for i, question in enumerate(questions):\n",
|
||
"\n",
|
||
" print(f\"new generated questions: {i} - {question}\")\n",
|
||
"\n",
|
||
" return questions"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Dueling AIs"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"This function will allow us to generate the questions using the same process as above, but then have a different model answer those questions. The process is as follows:\n",
|
||
"- Generate the `questions` list using the same steps as the previous function.\n",
|
||
"- Load in the answer model using `ModelCatalog`\n",
|
||
"- Answer each question in `questions` using the `inference()` function of the answer model."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def ask_and_answer_game(source_passage, q_model=\"slim-q-gen-tiny-tool\", number_of_tries=10, question_type=\"question\",\n",
|
||
" temperature=0.5):\n",
|
||
"\n",
|
||
" \"\"\" Shows a simple two model game of using q-gen model to generate a question, and then a second model\n",
|
||
" to answer the question generated. \"\"\"\n",
|
||
"\n",
|
||
" # this is the model that will generate the 'question'\n",
|
||
" q_model = ModelCatalog().load_model(q_model, sample=True, temperature=temperature)\n",
|
||
"\n",
|
||
" # this will be the model used to 'answer' the question\n",
|
||
" answer_model = ModelCatalog().load_model(\"bling-phi-3-gguf\")\n",
|
||
"\n",
|
||
" questions = []\n",
|
||
"\n",
|
||
" print(f\"\\nGenerating a set of questions automatically from the source passage.\\n\")\n",
|
||
"\n",
|
||
" for x in range(0,number_of_tries):\n",
|
||
"\n",
|
||
" response = q_model.function_call(source_passage, params=[question_type], get_logits=False)\n",
|
||
"\n",
|
||
" if response:\n",
|
||
" if \"llm_response\" in response:\n",
|
||
" if \"question\" in response[\"llm_response\"]:\n",
|
||
" new_q = response[\"llm_response\"][\"question\"]\n",
|
||
"\n",
|
||
" # only keep new questions\n",
|
||
" if new_q and new_q not in questions:\n",
|
||
" questions.append(new_q)\n",
|
||
"\n",
|
||
" print(f\"inference - {x} - response: {response['llm_response']}\")\n",
|
||
"\n",
|
||
" print(\"\\nAnswering the generated questions\\n\")\n",
|
||
" for i, question in enumerate(questions):\n",
|
||
"\n",
|
||
" print(f\"\\nquestion: {i} - {question}\")\n",
|
||
" if isinstance(question, list) and len(question) > 0:\n",
|
||
" response = answer_model.inference(question[0], add_context=source_passage)\n",
|
||
" print(f\"response: \", response[\"llm_response\"])\n",
|
||
"\n",
|
||
" return True"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Main block"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Here, we state our source text and call both functions above."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# test passage pulled from CNBC news story on Tuesday, May 28, 2024\n",
|
||
"test_passage = (\"OpenAI said Tuesday it has established a new committee to make recommendations to the \"\n",
|
||
" \"company’s board about safety and security, weeks after dissolving a team focused on AI safety. \"\n",
|
||
" \"In a blog post, OpenAI said the new committee would be led by CEO Sam Altman as well as \"\n",
|
||
" \"Bret Taylor, the company’s board chair, and board member Nicole Seligman. The announcement \"\n",
|
||
" \"follows the high-profile exit this month of an OpenAI executive focused on safety, \"\n",
|
||
" \"Jan Leike. Leike resigned from OpenAI leveling criticisms that the company had \"\n",
|
||
" \"under-invested in AI safety work and that tensions with OpenAI’s leadership had \"\n",
|
||
" \"reached a breaking point.\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"inference 0 - response: {'question': ['Who are the two members of the new safety and security committee?']}\n",
|
||
"inference 1 - response: {'question': ['What is the name of the new committee?']}\n",
|
||
"inference 2 - response: {'question': ['What is the name of the executive who resigned?']}\n",
|
||
"inference 3 - response: {'question': ['What is the name of the CEO of the company?']}\n",
|
||
"inference 4 - response: {'question': ['What is the name of the board chair?']}\n",
|
||
"inference 5 - response: {'question': ['Who is the chairman of OpenAI?']}\n",
|
||
"inference 6 - response: {'question': ['What is a list of the key points in the announcement?']}\n",
|
||
"inference 7 - response: {'question': ['What is a list of three key points?']}\n",
|
||
"inference 8 - response: {'question': ['What is the name of the executive who resigned?']}\n",
|
||
"inference 9 - response: {'question': ['What is the name of the executive who resigned from OpenAI?']}\n",
|
||
"\n",
|
||
"De-duped list of questions created\n",
|
||
"\n",
|
||
"new generated questions: 0 - ['Who are the two members of the new safety and security committee?']\n",
|
||
"new generated questions: 1 - ['What is the name of the new committee?']\n",
|
||
"new generated questions: 2 - ['What is the name of the executive who resigned?']\n",
|
||
"new generated questions: 3 - ['What is the name of the CEO of the company?']\n",
|
||
"new generated questions: 4 - ['What is the name of the board chair?']\n",
|
||
"new generated questions: 5 - ['Who is the chairman of OpenAI?']\n",
|
||
"new generated questions: 6 - ['What is a list of the key points in the announcement?']\n",
|
||
"new generated questions: 7 - ['What is a list of three key points?']\n",
|
||
"new generated questions: 8 - ['What is the name of the executive who resigned from OpenAI?']\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"[['Who are the two members of the new safety and security committee?'],\n",
|
||
" ['What is the name of the new committee?'],\n",
|
||
" ['What is the name of the executive who resigned?'],\n",
|
||
" ['What is the name of the CEO of the company?'],\n",
|
||
" ['What is the name of the board chair?'],\n",
|
||
" ['Who is the chairman of OpenAI?'],\n",
|
||
" ['What is a list of the key points in the announcement?'],\n",
|
||
" ['What is a list of three key points?'],\n",
|
||
" ['What is the name of the executive who resigned from OpenAI?']]"
|
||
]
|
||
},
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"hello_world_test(test_passage,q_model=\"slim-q-gen-tiny-tool\",number_of_tries=10, question_type=\"question\", temperature=0.5)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"We can see each question that was generated `number_of_tries` times, and then the final question list with the duplicates removed."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\n",
|
||
"Generating a set of questions automatically from the source passage.\n",
|
||
"\n",
|
||
"inference - 0 - response: {'question': ['What is the name of the executive who resigned?']}\n",
|
||
"inference - 1 - response: {'question': ['When was this announcement made?']}\n",
|
||
"inference - 2 - response: {'question': ['What is the name of one of the members of the new committee?']}\n",
|
||
"inference - 3 - response: {'question': ['What is one of the names of the people who will lead the new committee?']}\n",
|
||
"inference - 4 - response: {'question': ['What is the name of the person who will lead the new committee?']}\n",
|
||
"inference - 5 - response: {'question': ['Who is leading the new advisory group?']}\n",
|
||
"inference - 6 - response: {'question': ['What is the name of the executive who resigned?']}\n",
|
||
"inference - 7 - response: {'question': ['What is the name of the person who is leading the new committee?']}\n",
|
||
"inference - 8 - response: {'question': ['What is one role of Nicole Seligman?']}\n",
|
||
"inference - 9 - response: {'question': ['What is one of the names of one of the members of this new committee?']}\n",
|
||
"\n",
|
||
"Answering the generated questions\n",
|
||
"\n",
|
||
"\n",
|
||
"question: 0 - ['What is the name of the executive who resigned?']\n",
|
||
"response: Jan Leike\n",
|
||
"\n",
|
||
"question: 1 - ['When was this announcement made?']\n",
|
||
"response: Tuesday\n",
|
||
"\n",
|
||
"question: 2 - ['What is the name of one of the members of the new committee?']\n",
|
||
"response: Bret Taylor\n",
|
||
"\n",
|
||
"question: 3 - ['What is one of the names of the people who will lead the new committee?']\n",
|
||
"response: Sam Altman\n",
|
||
"\n",
|
||
"question: 4 - ['What is the name of the person who will lead the new committee?']\n",
|
||
"response: Sam Altman\n",
|
||
"\n",
|
||
"question: 5 - ['Who is leading the new advisory group?']\n",
|
||
"response: CEO Sam Altman\n",
|
||
"\n",
|
||
"question: 6 - ['What is the name of the person who is leading the new committee?']\n",
|
||
"response: Sam Altman\n",
|
||
"\n",
|
||
"question: 7 - ['What is one role of Nicole Seligman?']\n",
|
||
"response: Board Chair of OpenAI\n",
|
||
"\n",
|
||
"question: 8 - ['What is one of the names of one of the members of this new committee?']\n",
|
||
"response: Bret Taylor\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"True"
|
||
]
|
||
},
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"ask_and_answer_game(test_passage,q_model=\"slim-q-gen-phi-3-tool\", number_of_tries=10, question_type=\"question\", temperature=0.5)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Here, we can see each question generated, then the responses to each unique (non-duplicate) question."
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.11.0"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 2
|
||
}
|