wehub/microsoft--semantic-kernel
1
0
Fork 0
Code Issues Pull Requests Actions 5 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
CodeQL / Analyze (csharp) (push) Has been cancelled
Details
CodeQL / Analyze (python) (push) Has been cancelled
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:21:23 +08:00
commit b957a53def
5423 changed files with 863745 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/samples/getting_started/.env.example
+13
View File
@@ -0,0 +1,13 @@
GLOBAL_LLM_SERVICE=""
OPENAI_API_KEY=""
OPENAI_CHAT_MODEL_ID=""
OPENAI_TEXT_MODEL_ID=""
OPENAI_EMBEDDING_MODEL_ID=""
OPENAI_ORG_ID=""
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=""
AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=""
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=""
AZURE_OPENAI_ENDPOINT=""
AZURE_OPENAI_API_KEY=""
AZURE_AI_SEARCH_API_KEY=""
AZURE_AI_SEARCH_ENDPOINT=""
python/samples/getting_started/00-getting-started.ipynb
+242
View File
@@ -0,0 +1,242 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Setup\n",
"\n",
"Import Semantic Kernel SDK from pypi.org"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's define our kernel for this example."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"\n",
"kernel = Kernel()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We now configure our Chat Completion service on the kernel."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Remove all services so that this cell can be re-run without restarting the kernel\n",
"kernel.remove_all_services()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Run a Semantic Function\n",
"\n",
"**Step 3**: Load a Plugin and run a semantic function:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plugin = kernel.add_plugin(parent_directory=\"../../../prompt_template_samples/\", plugin_name=\"FunPlugin\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.functions import KernelArguments\n",
"\n",
"joke_function = plugin[\"Joke\"]\n",
"\n",
"joke = await kernel.invoke(\n",
" joke_function,\n",
" KernelArguments(input=\"time travel to dinosaur age\", style=\"super silly\"),\n",
")\n",
"print(joke)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
python/samples/getting_started/01-basic-loading-the-kernel.ipynb
+226
View File
@@ -0,0 +1,226 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Basic Loading of the Kernel\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup\n",
"\n",
"Import Semantic Kernel SDK from pypi.org"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's define our kernel for this example."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"\n",
"kernel = Kernel()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Remove all services so that this cell can be re-run without restarting the kernel\n",
"kernel.remove_all_services()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Great, now that you're familiar with setting up the Semantic Kernel, let's see [how we can use it to run prompts](02-running-prompts-from-file.ipynb).\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
},
"polyglot_notebook": {
"kernelInfo": {
"items": [
{
"aliases": [
"frontend"
],
"name": "vscode"
}
]
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
python/samples/getting_started/02-running-prompts-from-file.ipynb
+395
View File
@@ -0,0 +1,395 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "692e361b",
"metadata": {},
"source": [
"# How to run a prompt plugins from file\n",
"\n",
"Now that you're familiar with Kernel basics, let's see how the kernel allows you to run Prompt Plugins and Prompt Functions stored on disk.\n",
"\n",
"A Prompt Plugin is a collection of Semantic Functions, where each function is defined with natural language that can be provided with a text file.\n",
"\n",
"Refer to our [glossary](https://github.com/microsoft/semantic-kernel/blob/main/docs/GLOSSARY.md) for an in-depth guide to the terms.\n",
"\n",
"The repository includes some examples under the [samples](https://github.com/microsoft/semantic-kernel/tree/main/samples) folder.\n",
"\n",
"For instance, [this](../../../prompt_template_samples/FunPlugin/Joke/skprompt.txt) is the **Joke function** part of the **FunPlugin plugin**:\n"
]
},
{
"cell_type": "markdown",
"id": "3feecb6e",
"metadata": {},
"source": [
"Import Semantic Kernel SDK from pypi.org"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "32187534",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"id": "cc58d362",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bc1bc941",
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"id": "b5074884",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"id": "93d7361e",
"metadata": {},
"source": [
"Let's move on to learning what prompts are and how to write them."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f3ce1efe",
"metadata": {},
"source": [
"```\n",
"WRITE EXACTLY ONE JOKE or HUMOROUS STORY ABOUT THE TOPIC BELOW.\n",
"JOKE MUST BE:\n",
"- G RATED\n",
"- WORKPLACE/FAMILY SAFE\n",
"NO SEXISM, RACISM OR OTHER BIAS/BIGOTRY.\n",
"BE CREATIVE AND FUNNY. I WANT TO LAUGH.\n",
"+++++\n",
"{{$input}}\n",
"+++++\n",
"```\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "afdb96d6",
"metadata": {},
"source": [
"Note the special **`{{$input}}`** token, which is a variable that is automatically passed when invoking the function, commonly referred to as a \"function parameter\".\n",
"\n",
"We'll explore later how functions can accept multiple variables, as well as invoke other functions.\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c3bd5134",
"metadata": {},
"source": [
"In the same folder you'll notice a second [config.json](../../../prompt_template_samples/FunPlugin/Joke/config.json) file. The file is optional, and is used to set some parameters for large language models like Temperature, TopP, Stop Sequences, etc.\n",
"\n",
"```\n",
"{\n",
" \"schema\": 1,\n",
" \"description\": \"Generate a funny joke\",\n",
" \"execution_settings\": {\n",
" \"default\": {\n",
" \"max_tokens\": 1000,\n",
" \"temperature\": 0.9,\n",
" \"top_p\": 0.0,\n",
" \"presence_penalty\": 0.0,\n",
" \"frequency_penalty\": 0.0\n",
" }\n",
" },\n",
" \"input_variables\": [\n",
" {\n",
" \"name\": \"input\",\n",
" \"description\": \"Joke subject\",\n",
" \"default\": \"\"\n",
" },\n",
" {\n",
" \"name\": \"style\",\n",
" \"description\": \"Give a hint about the desired joke style\",\n",
" \"default\": \"\"\n",
" }\n",
" ]\n",
"}\n",
"\n",
"```\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "384ff07f",
"metadata": {},
"source": [
"Given a prompt function defined by these files, this is how to load and use a file based prompt function.\n",
"\n",
"Load and configure the kernel, as usual, loading also the AI service settings defined in the [Setup notebook](00-getting-started.ipynb):\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9c0688c5",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"\n",
"kernel = Kernel()"
]
},
{
"cell_type": "markdown",
"id": "63f0788e",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "82d16ce6",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"cell_type": "markdown",
"id": "04ad7f35",
"metadata": {},
"source": [
"Let's load our settings and validate that the required ones exist."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fdb865a7",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"cell_type": "markdown",
"id": "c50b4d7a",
"metadata": {},
"source": [
"We now configure our Chat Completion service on the kernel."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b0062a24",
"metadata": {},
"outputs": [],
"source": [
"# Remove all services so that this cell can be re-run without restarting the kernel\n",
"kernel.remove_all_services()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "fd5ff1f4",
"metadata": {},
"source": [
"Import the plugin and all its functions:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56ee184d",
"metadata": {},
"outputs": [],
"source": [
"# note: using plugins from the samples folder\n",
"plugins_directory = \"../../../prompt_template_samples/\"\n",
"\n",
"funFunctions = kernel.add_plugin(parent_directory=plugins_directory, plugin_name=\"FunPlugin\")\n",
"\n",
"jokeFunction = funFunctions[\"Joke\"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "edd99fa0",
"metadata": {},
"source": [
"How to use the plugin functions, e.g. generate a joke about \"_time travel to dinosaur age_\":\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6effe63b",
"metadata": {},
"outputs": [],
"source": [
"result = await kernel.invoke(jokeFunction, input=\"travel to dinosaur age\", style=\"silly\")\n",
"print(result)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2281a1fc",
"metadata": {},
"source": [
"Great, now that you know how to load a plugin from disk, let's show how you can [create and run a prompt function inline.](./03-prompt-function-inline.ipynb)\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/03-prompt-function-inline.ipynb
+473
View File
@@ -0,0 +1,473 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "3c93ac5b",
"metadata": {},
"source": [
"# Running Prompt Functions Inline\n"
]
},
{
"cell_type": "markdown",
"id": "ad85226c",
"metadata": {},
"source": [
"Import Semantic Kernel SDK from pypi.org"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a94d2b07",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"id": "c704e54b",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7ebb95f2",
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"id": "14eb77b5",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure OpenAI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "40201641",
"metadata": {},
"source": [
"The [previous notebook](./02-running-prompts-from-file.ipynb)\n",
"showed how to define a semantic function using a prompt template stored on a file.\n",
"\n",
"In this notebook, we'll show how to use the Semantic Kernel to define functions inline with your python code. This can be useful in a few scenarios:\n",
"\n",
"- Dynamically generating the prompt using complex rules at runtime\n",
"- Writing prompts by editing Python code instead of TXT files.\n",
"- Easily creating demos, like this document\n",
"\n",
"Prompt templates are defined using the SK template language, which allows to reference variables and functions. Read [this doc](https://aka.ms/sk/howto/configurefunction) to learn more about the design decisions for prompt templating.\n",
"\n",
"For now we'll use only the `{{$input}}` variable, and see more complex templates later.\n",
"\n",
"Almost all semantic function prompts have a reference to `{{$input}}`, which is the default way\n",
"a user can import content from the context variables.\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d90b0c13",
"metadata": {},
"source": [
"Prepare a semantic kernel instance first, loading also the AI service settings defined in the [Setup notebook](00-getting-started.ipynb):\n"
]
},
{
"cell_type": "markdown",
"id": "0377f42e",
"metadata": {},
"source": [
"Let's define our kernel for this example."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "462b281a",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.kernel import Kernel\n",
"\n",
"kernel = Kernel()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "734d121f",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"cell_type": "markdown",
"id": "06740170",
"metadata": {},
"source": [
"We now configure our Chat Completion service on the kernel."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3712b7c3",
"metadata": {},
"outputs": [],
"source": [
"# Remove all services so that this cell can be re-run without restarting the kernel\n",
"kernel.remove_all_services()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "589733c5",
"metadata": {},
"source": [
"Let's use a prompt to create a semantic function used to summarize content, allowing for some creativity and a sufficient number of tokens.\n",
"\n",
"The function will take in input the text to summarize.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ae29c207",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.ai.open_ai import AzureChatPromptExecutionSettings, OpenAIChatPromptExecutionSettings\n",
"from semantic_kernel.prompt_template import InputVariable, PromptTemplateConfig\n",
"\n",
"prompt = \"\"\"{{$input}}\n",
"Summarize the content above.\n",
"\"\"\"\n",
"\n",
"if selectedService == Service.OpenAI:\n",
" execution_settings = OpenAIChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-3.5-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" execution_settings = AzureChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-35-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"\n",
"prompt_template_config = PromptTemplateConfig(\n",
" template=prompt,\n",
" name=\"summarize\",\n",
" template_format=\"semantic-kernel\",\n",
" input_variables=[\n",
" InputVariable(name=\"input\", description=\"The user input\", is_required=True),\n",
" ],\n",
" execution_settings=execution_settings,\n",
")\n",
"\n",
"summarize = kernel.add_function(\n",
" function_name=\"summarizeFunc\",\n",
" plugin_name=\"summarizePlugin\",\n",
" prompt_template_config=prompt_template_config,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f26b90c4",
"metadata": {},
"source": [
"Set up some content to summarize, here's an extract about Demo, an ancient Greek poet, taken from Wikipedia (https://en.wikipedia.org/wiki/Demo_(ancient_Greek_poet)).\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "314557fb",
"metadata": {},
"outputs": [],
"source": [
"input_text = \"\"\"\n",
"Demo (ancient Greek poet)\n",
"From Wikipedia, the free encyclopedia\n",
"Demo or Damo (Greek: Δεμώ, Δαμώ; fl. c. AD 200) was a Greek woman of the Roman period, known for a single epigram, engraved upon the Colossus of Memnon, which bears her name. She speaks of herself therein as a lyric poetess dedicated to the Muses, but nothing is known of her life.[1]\n",
"Identity\n",
"Demo was evidently Greek, as her name, a traditional epithet of Demeter, signifies. The name was relatively common in the Hellenistic world, in Egypt and elsewhere, and she cannot be further identified. The date of her visit to the Colossus of Memnon cannot be established with certainty, but internal evidence on the left leg suggests her poem was inscribed there at some point in or after AD 196.[2]\n",
"Epigram\n",
"There are a number of graffiti inscriptions on the Colossus of Memnon. Following three epigrams by Julia Balbilla, a fourth epigram, in elegiac couplets, entitled and presumably authored by \"Demo\" or \"Damo\" (the Greek inscription is difficult to read), is a dedication to the Muses.[2] The poem is traditionally published with the works of Balbilla, though the internal evidence suggests a different author.[1]\n",
"In the poem, Demo explains that Memnon has shown her special respect. In return, Demo offers the gift for poetry, as a gift to the hero. At the end of this epigram, she addresses Memnon, highlighting his divine status by recalling his strength and holiness.[2]\n",
"Demo, like Julia Balbilla, writes in the artificial and poetic Aeolic dialect. The language indicates she was knowledgeable in Homeric poetry—'bearing a pleasant gift', for example, alludes to the use of that phrase throughout the Iliad and Odyssey.[a][2]\n",
"\"\"\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "bf0f2330",
"metadata": {},
"source": [
"...and run the summary function:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7b0e3b0c",
"metadata": {},
"outputs": [],
"source": [
"summary = await kernel.invoke(summarize, input=input_text)\n",
"\n",
"print(summary)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "1c2c1262",
"metadata": {},
"source": [
"# Using ChatCompletion for Semantic Plugins\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "29b59b28",
"metadata": {},
"source": [
"You can also use chat completion models (like `gpt-35-turbo` and `gpt4`) for creating plugins. Normally you would have to tweak the API to accommodate for a system and user role, but SK abstracts that away for you by using `kernel.add_service` and `AzureChatCompletion` or `OpenAIChatCompletion`\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4777f447",
"metadata": {},
"source": [
"Here's one more example of how to write an inline Semantic Function that gives a TLDR for a piece of text using a ChatCompletion model\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5886aeb",
"metadata": {},
"outputs": [],
"source": [
"kernel.remove_all_services()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ea8128c8",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.ai.open_ai import AzureChatPromptExecutionSettings, OpenAIChatPromptExecutionSettings\n",
"\n",
"prompt = \"\"\"\n",
"{{$input}}\n",
"\n",
"Give me the TLDR in 5 words or less.\n",
"\"\"\"\n",
"\n",
"text = \"\"\"\n",
" 1) A robot may not injure a human being or, through inaction,\n",
" allow a human being to come to harm.\n",
"\n",
" 2) A robot must obey orders given it by human beings except where\n",
" such orders would conflict with the First Law.\n",
"\n",
" 3) A robot must protect its own existence as long as such protection\n",
" does not conflict with the First or Second Law.\n",
"\"\"\"\n",
"\n",
"if selectedService == Service.OpenAI:\n",
" execution_settings = OpenAIChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-3.5-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" execution_settings = AzureChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-35-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"\n",
"prompt_template_config = PromptTemplateConfig(\n",
" template=prompt,\n",
" name=\"tldr\",\n",
" template_format=\"semantic-kernel\",\n",
" input_variables=[\n",
" InputVariable(name=\"input\", description=\"The user input\", is_required=True),\n",
" ],\n",
" execution_settings=execution_settings,\n",
")\n",
"\n",
"tldr_function = kernel.add_function(\n",
" function_name=\"tldrFunction\",\n",
" plugin_name=\"tldrPlugin\",\n",
" prompt_template_config=prompt_template_config,\n",
")\n",
"\n",
"summary = await kernel.invoke(tldr_function, input=text)\n",
"\n",
"print(f\"Output: {summary}\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/04-kernel-arguments-chat.ipynb
+443
View File
@@ -0,0 +1,443 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "fde98ddf",
"metadata": {},
"source": [
"# Creating a basic chat experience with kernel arguments\n",
"\n",
"In this example, we show how you can build a simple chat bot by sending and updating the kernel arguments with your requests.\n",
"\n",
"We introduce the Kernel Arguments object which in this demo functions similarly as a key-value store that you can use when running the kernel.\n",
"\n",
"The chat history is local (i.e. in your computer's RAM) and not persisted anywhere beyond the life of this Jupyter session.\n",
"\n",
"In future examples, we will show how to persist the chat history on disk so that you can bring it into your applications.\n",
"\n",
"In this chat scenario, as the user talks back and forth with the bot, the chat context gets populated with the history of the conversation. During each new run of the kernel, the kernel arguments and chat history can provide the AI with its variables' content.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3b16c201",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"id": "85886ed0",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ec88496f",
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"id": "208ed165",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"id": "da290af7",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6ed3f9ae",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "68301108",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"\n",
"kernel = Kernel()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "7971783d",
"metadata": {},
"source": [
"Let's define a prompt outlining a dialogue chat bot.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e84a05fc",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"\"\"\n",
"ChatBot can have a conversation with you about any topic.\n",
"It can give explicit instructions or say 'I don't know' if it does not have an answer.\n",
"\n",
"{{$history}}\n",
"User: {{$user_input}}\n",
"ChatBot: \"\"\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "61716b16",
"metadata": {},
"source": [
"Register your semantic function\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a3e4b160",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.ai.open_ai import AzureChatPromptExecutionSettings, OpenAIChatPromptExecutionSettings\n",
"from semantic_kernel.prompt_template import PromptTemplateConfig\n",
"from semantic_kernel.prompt_template.input_variable import InputVariable\n",
"\n",
"if selectedService == Service.OpenAI:\n",
" execution_settings = OpenAIChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-3.5-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" execution_settings = AzureChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-35-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"\n",
"prompt_template_config = PromptTemplateConfig(\n",
" template=prompt,\n",
" name=\"chat\",\n",
" template_format=\"semantic-kernel\",\n",
" input_variables=[\n",
" InputVariable(name=\"user_input\", description=\"The user input\", is_required=True),\n",
" # Disable encoding for \"history\" variable by setting allow_dangerously_set_content to \"True\"\n",
" # In production scenarios, encode complex types to prevent prompt injection attacks.\n",
" InputVariable(\n",
" name=\"history\", description=\"The conversation history\", is_required=True, allow_dangerously_set_content=True\n",
" ),\n",
" ],\n",
" execution_settings=execution_settings,\n",
")\n",
"\n",
"chat_function = kernel.add_function(\n",
" function_name=\"chat\",\n",
" plugin_name=\"chatPlugin\",\n",
" prompt_template_config=prompt_template_config,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6a0f7c01",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.contents import ChatHistory\n",
"\n",
"chat_history = ChatHistory()\n",
"chat_history.add_system_message(\"You are a helpful chatbot who is good about giving book recommendations.\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "6e8a676f",
"metadata": {},
"source": [
"Initialize the Kernel Arguments\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a4be7394",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.functions import KernelArguments\n",
"\n",
"arguments = KernelArguments(user_input=\"Hi, I'm looking for book suggestions\", history=chat_history)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4ce7c497",
"metadata": {},
"source": [
"Chat with the Bot\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5ec41eb8",
"metadata": {},
"outputs": [],
"source": [
"response = await kernel.invoke(chat_function, arguments)\n",
"print(response)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "a5b03748",
"metadata": {},
"source": [
"Update the history with the output\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f50f517d",
"metadata": {},
"outputs": [],
"source": [
"chat_history.add_assistant_message(str(response))"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "23a2eb02",
"metadata": {},
"source": [
"Keep Chatting!\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c59efe45",
"metadata": {},
"outputs": [],
"source": [
"async def chat(input_text: str) -> None:\n",
" # Save new message in the context variables\n",
" print(f\"User: {input_text}\")\n",
"\n",
" # Process the user message and get an answer\n",
" answer = await kernel.invoke(chat_function, KernelArguments(user_input=input_text, history=chat_history))\n",
"\n",
" # Show the response\n",
" print(f\"ChatBot: {answer}\")\n",
"\n",
" chat_history.add_user_message(input_text)\n",
" chat_history.add_assistant_message(str(answer))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "06ee244e",
"metadata": {},
"outputs": [],
"source": [
"await chat(\"I love history and philosophy, I'd like to learn something new about Greece, any suggestion?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "82be4e7e",
"metadata": {},
"outputs": [],
"source": [
"await chat(\"that sounds interesting, what is it about?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "82fe0139",
"metadata": {},
"outputs": [],
"source": [
"await chat(\"if I read that book, what exactly will I learn about Greek history?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "55b3a9f2",
"metadata": {},
"outputs": [],
"source": [
"await chat(\"could you list some more books I could read about this topic?\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c30bac97",
"metadata": {},
"source": [
"After chatting for a while, we have built a growing history, which we are attaching to each prompt and which contains the full conversation. Let's take a look!\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5e34ae55",
"metadata": {},
"outputs": [],
"source": [
"print(chat_history)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/05-memory-and-embeddings.ipynb
+818
View File
@@ -0,0 +1,818 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Building Semantic Memory with Embeddings\n",
"\n",
"So far, we've mostly been treating the kernel as a stateless orchestration engine.\n",
"We send text into a model API and receive text out.\n",
"\n",
"In a [previous notebook](04-kernel-arguments-chat.ipynb), we used `kernel arguments` to pass in additional\n",
"text into prompts to enrich them with more data. This allowed us to create a basic chat experience.\n",
"\n",
"However, if you solely relied on kernel arguments, you would quickly realize that eventually your prompt\n",
"would grow so large that you would run into the model's token limit. What we need is a way to persist state\n",
"and build both short-term and long-term memory to empower even more intelligent applications.\n",
"\n",
"To do this, we dive into the key concept of `Semantic Memory` in the Semantic Kernel.\n"
]
},
{
"cell_type": "markdown",
"id": "6713abcd",
"metadata": {},
"source": [
"Import Semantic Kernel SDK from pypi.org and other dependencies for this example."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel[azure]\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"id": "318033fe",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "d8a3db35",
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"id": "2c5f9651",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"id": "815cac6e",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "1b95af24",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Using service type: Service.OpenAI\n"
]
}
],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"In order to use memory, we need to instantiate the Kernel with a Memory Storage\n",
"and an Embedding service. In this example, we make use of the `VolatileMemoryStore` which can be thought of as a temporary in-memory storage. This memory is not written to disk and is only available during the app session.\n",
"\n",
"When developing your app you will have the option to plug in persistent storage like Azure AI Search, Azure Cosmos Db, PostgreSQL, SQLite, etc. Semantic Memory allows also to index external data sources, without duplicating all the information as you will see further down in this notebook.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"from semantic_kernel.connectors.ai.open_ai import (\n",
" AzureChatCompletion,\n",
" AzureTextEmbedding,\n",
" OpenAIChatCompletion,\n",
" OpenAITextEmbedding,\n",
")\n",
"\n",
"kernel = Kernel()\n",
"\n",
"chat_service_id = \"chat\"\n",
"\n",
"# Configure AI service used by the kernel\n",
"if selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" credential = AzureCliCredential()\n",
" azure_chat_service = AzureChatCompletion(service_id=chat_service_id, credential=credential)\n",
" embedding_gen = AzureTextEmbedding(service_id=\"embedding\", credential=credential)\n",
" kernel.add_service(azure_chat_service)\n",
" kernel.add_service(embedding_gen)\n",
"elif selectedService == Service.OpenAI:\n",
" oai_chat_service = OpenAIChatCompletion(\n",
" service_id=chat_service_id,\n",
" )\n",
" embedding_gen = OpenAITextEmbedding(\n",
" service_id=\"embedding\",\n",
" )\n",
" kernel.add_service(oai_chat_service)\n",
" kernel.add_service(embedding_gen)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da74fa33",
"metadata": {},
"outputs": [],
"source": [
"from dataclasses import dataclass, field\n",
"from typing import Annotated\n",
"from uuid import uuid4\n",
"\n",
"from semantic_kernel.data.vector import VectorStoreField, vectorstoremodel\n",
"\n",
"\n",
"@vectorstoremodel(collection_name=\"simple-model\")\n",
"@dataclass\n",
"class SimpleModel:\n",
" \"\"\"Simple model to store some text with a ID.\"\"\"\n",
"\n",
" text: Annotated[str, VectorStoreField(\"data\", is_full_text_indexed=True)]\n",
" id: Annotated[str, VectorStoreField(\"key\")] = field(default_factory=lambda: str(uuid4()))\n",
" embedding: Annotated[\n",
" list[float] | str | None, VectorStoreField(\"vector\", dimensions=1536, embedding_generator=embedding_gen)\n",
" ] = None\n",
"\n",
" def __post_init__(self):\n",
" if self.embedding is None:\n",
" self.embedding = self.text"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e7fefb6a",
"metadata": {},
"source": [
"At its core, Semantic Memory is a set of data structures that allow you to store the meaning of text that come from different data sources, and optionally to store the source text too. These texts can be from the web, e-mail providers, chats, a database, or from your local directory, and are hooked up to the Semantic Kernel through data source connectors.\n",
"\n",
"The texts are embedded or compressed into a vector of floats representing mathematically the texts' contents and meaning. You can read more about embeddings [here](https://aka.ms/sk/embeddings).\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2a7e7ca4",
"metadata": {},
"source": [
"### Manually adding memories\n",
"\n",
"Let's create some initial memories \"About Me\". We can add memories to our `VolatileMemoryStore` by using `SaveInformationAsync`\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "d096504c",
"metadata": {},
"outputs": [],
"source": [
"records = [\n",
" SimpleModel(text=\"Your budget for 2024 is $100,000\"),\n",
" SimpleModel(text=\"Your savings from 2023 are $50,000\"),\n",
" SimpleModel(text=\"Your investments are $80,000\"),\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "5338d3ac",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['e6f2c669-0996-4cc5-8312-59e7d996a32b',\n",
" '9042c465-115d-422d-b5f7-5ce35bd36455',\n",
" 'ecc3ec62-b1b5-4a41-a761-63280d3a4329']"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantic_kernel.connectors.in_memory import InMemoryStore\n",
"\n",
"in_memory_store = InMemoryStore()\n",
"\n",
"collection = in_memory_store.get_collection(record_type=SimpleModel)\n",
"await collection.ensure_collection_exists()\n",
"# Add records to the collection\n",
"await collection.upsert(records)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2calf857",
"metadata": {},
"source": [
"Let's try searching the memory:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.data.vector import VectorSearchProtocol\n",
"\n",
"\n",
"async def search_memory_examples(collection: VectorSearchProtocol, questions: list[str]) -> None:\n",
" for question in questions:\n",
" print(f\"Question: {question}\")\n",
" results = await collection.search(question, top=1)\n",
" async for result in results.results:\n",
" print(f\"Answer: {result.record.text}\")\n",
" print(f\"Score: {result.score}\\n\")"
]
},
{
"cell_type": "markdown",
"id": "fe185991",
"metadata": {},
"source": [
"The default distance metric for the InMemoryCollection is `cosine`, this means that the closer the vectors are, the more similar they are. The `search` method will return the top `3` results by default, but you can change this by passing in the `top` parameter, this is set to `1` here to get only the most relevant result."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "24764c48",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Question: What is my budget for 2024?\n",
"Answer: Your budget for 2024 is $100,000\n",
"Score: 0.20661429378068685\n",
"\n",
"Question: What are my savings from 2023?\n",
"Answer: Your savings from 2023 are $50,000\n",
"Score: 0.2028375831967465\n",
"\n",
"Question: What are my investments?\n",
"Answer: Your investments are $80,000\n",
"Score: 0.33724588631042385\n",
"\n"
]
}
],
"source": [
"await search_memory_examples(\n",
" collection,\n",
" questions=[\n",
" \"What is my budget for 2024?\",\n",
" \"What are my savings from 2023?\",\n",
" \"What are my investments?\",\n",
" ],\n",
")"
]
},
{
"cell_type": "markdown",
"id": "2133d9d6",
"metadata": {},
"source": [
"Next, we will add a search function to our kernel that will allow us to search the memory store for relevant information. This function will use the `search` method of the `InMemoryStore` to find the most relevant memories based on a query."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "9c1fdace",
"metadata": {},
"outputs": [],
"source": [
"func = kernel.add_function(\n",
" plugin_name=\"memory\",\n",
" function=collection.create_search_function(\n",
" function_name=\"recall\",\n",
" description=\"Searches the memory for relevant information based on the input query.\",\n",
" ),\n",
")"
]
},
{
"cell_type": "markdown",
"id": "f5dbaf80",
"metadata": {},
"source": [
"Then we will create a prompt that will use the search function to find relevant information in the memory store and return it as part of the prompt."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "fb8549b2",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.functions import KernelFunction\n",
"from semantic_kernel.prompt_template import PromptTemplateConfig\n",
"\n",
"\n",
"async def setup_chat_with_memory(\n",
" kernel: Kernel,\n",
" service_id: str,\n",
") -> KernelFunction:\n",
" prompt = \"\"\"\n",
" ChatBot can have a conversation with you about any topic.\n",
" It can give explicit instructions or say 'I don't know' if\n",
" it does not have an answer.\n",
"\n",
" Information about me, from previous conversations:\n",
" - {{recall 'budget by year'}} What is my budget for 2024?\n",
" - {{recall 'savings from previous year'}} What are my savings from 2023?\n",
" - {{recall 'investments'}} What are my investments?\n",
"\n",
" {{$request}}\n",
" \"\"\".strip()\n",
"\n",
" prompt_template_config = PromptTemplateConfig(\n",
" template=prompt,\n",
" execution_settings={\n",
" service_id: kernel.get_service(service_id).get_prompt_execution_settings_class()(service_id=service_id)\n",
" },\n",
" )\n",
"\n",
" return kernel.add_function(\n",
" function_name=\"chat_with_memory\",\n",
" plugin_name=\"chat\",\n",
" prompt_template_config=prompt_template_config,\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "645b55a1",
"metadata": {},
"source": [
"Now that we've included our memories, let's chat!\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "e3875a34",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Setting up a chat (with memory!)\n",
"Begin chatting (type 'exit' to exit):\n",
"\n",
"Welcome to the chat bot! \n",
" Type 'exit' to exit. \n",
" Try asking a question about your finances (i.e. \"talk to me about my finances\").\n"
]
}
],
"source": [
"print(\"Setting up a chat (with memory!)\")\n",
"chat_func = await setup_chat_with_memory(kernel, chat_service_id)\n",
"\n",
"print(\"Begin chatting (type 'exit' to exit):\\n\")\n",
"print(\n",
" \"Welcome to the chat bot!\\\n",
" \\n Type 'exit' to exit.\\\n",
" \\n Try asking a question about your finances (i.e. \\\"talk to me about my finances\\\").\"\n",
")\n",
"\n",
"\n",
"async def chat(user_input: str):\n",
" print(f\"User: {user_input}\")\n",
" answer = await kernel.invoke(chat_func, request=user_input)\n",
" print(f\"ChatBot:> {answer}\")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "6b55f64f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"User: What is my budget for 2024?\n",
"ChatBot:> Your budget for 2024 is $100,000.\n"
]
}
],
"source": [
"await chat(\"What is my budget for 2024?\")"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "243f9eb2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"User: talk to me about my finances\n",
"ChatBot:> Based on our previous conversations, your investments amount to $80,000. Your financial overview includes a budget for 2024 of $100,000 and savings from 2023 totaling $50,000. If you'd like, I can help you analyze your financial goals, suggest investment strategies, or assist with budgeting. Let me know how you'd like to proceed!\n"
]
}
],
"source": [
"await chat(\"talk to me about my finances\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "0a51542b",
"metadata": {},
"source": [
"### Adding documents to your memory\n",
"\n",
"Many times in your applications you'll want to bring in external documents into your memory. Let's see how we can do this using our VolatileMemoryStore.\n",
"\n",
"Let's first get some data using some of the links in the Semantic Kernel repo.\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "170e7142",
"metadata": {},
"outputs": [],
"source": [
"@vectorstoremodel(collection_name=\"github-files\")\n",
"@dataclass\n",
"class GitHubFile:\n",
" \"\"\"\n",
" Model to store GitHub file URLs and their descriptions.\n",
" \"\"\"\n",
"\n",
" url: Annotated[str, VectorStoreField(\"data\", is_full_text_indexed=True)]\n",
" text: Annotated[str, VectorStoreField(\"data\", is_full_text_indexed=True)]\n",
" key: Annotated[str, VectorStoreField(\"key\")] = field(default_factory=lambda: str(uuid4()))\n",
" embedding: Annotated[\n",
" list[float] | str | None, VectorStoreField(\"vector\", dimensions=1536, embedding_generator=embedding_gen)\n",
" ] = None\n",
"\n",
" def __post_init__(self):\n",
" if self.embedding is None:\n",
" self.embedding = f\"{self.url} {self.text}\""
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "0d971eeb",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['81585815-6be9-4b45-b608-af1f9918567f',\n",
" '83d5e6ba-a6f9-4d05-aa27-0b5470c1aa83',\n",
" 'e015e196-ddb3-4157-9291-dcef201d5308']"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"github_files = []\n",
"github_files.append(\n",
" GitHubFile(\n",
" url=\"https://github.com/microsoft/semantic-kernel/blob/main/README.md\",\n",
" text=\"README: Installation, getting started, and how to contribute\",\n",
" )\n",
")\n",
"github_files.append(\n",
" GitHubFile(\n",
" url=\"https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/02-running-prompts-from-file.ipynb\",\n",
" text=\"Jupyter notebook describing how to pass prompts from a file to a semantic plugin or function\",\n",
" )\n",
")\n",
"github_files.append(\n",
" GitHubFile(\n",
" url=\"https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/00-getting-started.ipynb\",\n",
" text=\"Jupyter notebook describing how to get started with Semantic Kernel\",\n",
" )\n",
")\n",
"\n",
"github_memory = in_memory_store.get_collection(record_type=GitHubFile)\n",
"await github_memory.ensure_collection_exists()\n",
"\n",
"# Add records to the collection\n",
"await github_memory.upsert(github_files)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "143911c3",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"===========================\n",
"Query: I love Jupyter notebooks, how should I get started?\n",
"\n",
"Result e015e196-ddb3-4157-9291-dcef201d5308:\n",
" URL : https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/00-getting-started.ipynb\n",
" Text : Jupyter notebook describing how to get started with Semantic Kernel\n",
" Relevance : 0.38310321217805676\n",
"\n",
"Result 83d5e6ba-a6f9-4d05-aa27-0b5470c1aa83:\n",
" URL : https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/02-running-prompts-from-file.ipynb\n",
" Text : Jupyter notebook describing how to pass prompts from a file to a semantic plugin or function\n",
" Relevance : 0.5391694801842719\n",
"\n",
"Result 81585815-6be9-4b45-b608-af1f9918567f:\n",
" URL : https://github.com/microsoft/semantic-kernel/blob/main/README.md\n",
" Text : README: Installation, getting started, and how to contribute\n",
" Relevance : 0.6495028830870797\n",
"\n"
]
}
],
"source": [
"ask = \"I love Jupyter notebooks, how should I get started?\"\n",
"print(\"===========================\\n\" + \"Query: \" + ask + \"\\n\")\n",
"\n",
"memories = await github_memory.search(ask, top=5)\n",
"\n",
"async for result in memories.results:\n",
" memory = result.record\n",
" print(f\"Result {memory.key}:\")\n",
" print(f\" URL : {memory.url}\")\n",
" print(f\" Text : {memory.text}\")\n",
" print(f\" Relevance : {result.score}\")\n",
" print()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "59294dac",
"metadata": {},
"source": [
"Now you might be wondering what happens if you have so much data that it doesn't fit into your RAM? That's where you want to make use of an external Vector Database made specifically for storing and retrieving embeddings. Fortunately, semantic kernel makes this easy thanks to an extensive list of available connectors. In the following section, we will connect to an existing Azure AI Search service that we will use as an external Vector Database to store and retrieve embeddings.\n"
]
},
{
"cell_type": "markdown",
"id": "e78fd381",
"metadata": {},
"source": [
"_Please note you will need an AzureAI Search api_key or token credential and endpoint for the following example to work properly._"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "77fdfa86",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['81585815-6be9-4b45-b608-af1f9918567f',\n",
" '83d5e6ba-a6f9-4d05-aa27-0b5470c1aa83',\n",
" 'e015e196-ddb3-4157-9291-dcef201d5308']"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from semantic_kernel.connectors.azure_ai_search import AzureAISearchCollection\n",
"\n",
"azs_memory = AzureAISearchCollection(record_type=GitHubFile)\n",
"# We explicitly delete the collection if it exists to ensure a clean state\n",
"await azs_memory.ensure_collection_deleted()\n",
"await azs_memory.ensure_collection_exists()\n",
"# Add records to the collection\n",
"await azs_memory.upsert(github_files)"
]
},
{
"cell_type": "markdown",
"id": "94f9e83b",
"metadata": {},
"source": [
"The implementation of Semantic Kernel allows to easily swap memory store for another. Here, we will re-use the functions we initially created for `InMemoryStore` with our new external Vector Store leveraging Azure AI Search\n"
]
},
{
"cell_type": "markdown",
"id": "b0bbe830",
"metadata": {},
"source": [
"Let's now try to query from Azure AI Search! Note that the `score` might be different because the AzureAISearchCollection uses a different default distance metric then InMemoryCollection, if you specify the `distance_function` in the model, you can get the same results as with the InMemoryCollection.\n"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "1a09d0ca",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Question: What is Semantic Kernel?\n",
"Answer: README: Installation, getting started, and how to contribute\n",
"Score: 0.72694075\n",
"\n",
"Question: How do I get started on it with notebooks?\n",
"Answer: Jupyter notebook describing how to get started with Semantic Kernel\n",
"Score: 0.68709856\n",
"\n",
"Question: Where can I find more info on prompts?\n",
"Answer: Jupyter notebook describing how to pass prompts from a file to a semantic plugin or function\n",
"Score: 0.6572231\n",
"\n"
]
}
],
"source": [
"await search_memory_examples(\n",
" azs_memory,\n",
" questions=[\n",
" \"What is Semantic Kernel?\",\n",
" \"How do I get started on it with notebooks?\",\n",
" \"Where can I find more info on prompts?\",\n",
" ],\n",
")"
]
},
{
"cell_type": "markdown",
"id": "4f0800fc",
"metadata": {},
"source": [
"Make sure to cleanup!"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "82cd429e",
"metadata": {},
"outputs": [],
"source": [
"await azs_memory.ensure_collection_deleted()"
]
},
{
"cell_type": "markdown",
"id": "3d33dcdc",
"metadata": {},
"source": [
"We have laid the foundation which will allow us to store an arbitrary amount of data in an external Vector Store above and beyond what could fit in memory at the expense of a little more latency.\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "python",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.7"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/06-hugging-face-for-plugins.ipynb
+204
View File
@@ -0,0 +1,204 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Using Hugging Face With Plugins\n",
"\n",
"In this notebook, we demonstrate using Hugging Face models for Plugins using both SemanticMemory and text completions.\n",
"\n",
"SK supports downloading models from the Hugging Face that can perform the following tasks: text-generation, text2text-generation, summarization, and sentence-similarity. You can search for models by task at https://huggingface.co/models.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "753ab756",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = Service.HuggingFace\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will create a kernel and add both text completion and embedding services.\n",
"\n",
"For text completion, we are choosing GPT2. This is a text-generation model. (Note: text-generation will repeat the input in the output, text2text-generation will not.)\n",
"For embeddings, we are using sentence-transformers/all-MiniLM-L6-v2. Vectors generated for this model are of length 384 (compared to a length of 1536 from OpenAI ADA).\n",
"\n",
"The following step may take a few minutes when run for the first time as the models will be downloaded to your local machine.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"from semantic_kernel.connectors.ai.hugging_face import HuggingFaceTextCompletion, HuggingFaceTextEmbedding\n",
"from semantic_kernel.core_plugins import TextMemoryPlugin\n",
"from semantic_kernel.memory import SemanticTextMemory, VolatileMemoryStore\n",
"\n",
"kernel = Kernel()\n",
"\n",
"# Configure LLM service\n",
"if selectedService == Service.HuggingFace:\n",
" # Feel free to update this model to any other model available on Hugging Face\n",
" text_service_id = \"HuggingFaceM4/tiny-random-LlamaForCausalLM\"\n",
" kernel.add_service(\n",
" service=HuggingFaceTextCompletion(\n",
" service_id=text_service_id, ai_model_id=text_service_id, task=\"text-generation\"\n",
" ),\n",
" )\n",
" embed_service_id = \"sentence-transformers/all-MiniLM-L6-v2\"\n",
" embedding_svc = HuggingFaceTextEmbedding(service_id=embed_service_id, ai_model_id=embed_service_id)\n",
" kernel.add_service(\n",
" service=embedding_svc,\n",
" )\n",
" memory = SemanticTextMemory(storage=VolatileMemoryStore(), embeddings_generator=embedding_svc)\n",
" kernel.add_plugin(TextMemoryPlugin(memory), \"TextMemoryPlugin\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2a7e7ca4",
"metadata": {},
"source": [
"### Add Memories and Define a plugin to use them\n",
"\n",
"Most models available on huggingface.co are not as powerful as OpenAI GPT-3+. Your plugins will likely need to be simpler to accommodate this.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d096504c",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.ai.hugging_face import HuggingFacePromptExecutionSettings\n",
"from semantic_kernel.prompt_template import PromptTemplateConfig\n",
"\n",
"collection_id = \"generic\"\n",
"\n",
"await memory.save_information(collection=collection_id, id=\"info1\", text=\"Sharks are fish.\")\n",
"await memory.save_information(collection=collection_id, id=\"info2\", text=\"Whales are mammals.\")\n",
"await memory.save_information(collection=collection_id, id=\"info3\", text=\"Penguins are birds.\")\n",
"await memory.save_information(collection=collection_id, id=\"info4\", text=\"Dolphins are mammals.\")\n",
"await memory.save_information(collection=collection_id, id=\"info5\", text=\"Flies are insects.\")\n",
"\n",
"# Define prompt function using SK prompt template language\n",
"my_prompt = \"\"\"I know these animal facts: \n",
"- {{recall 'fact about sharks'}}\n",
"- {{recall 'fact about whales'}} \n",
"- {{recall 'fact about penguins'}} \n",
"- {{recall 'fact about dolphins'}} \n",
"- {{recall 'fact about flies'}}\n",
"Now, tell me something about: {{$request}}\"\"\"\n",
"\n",
"execution_settings = HuggingFacePromptExecutionSettings(\n",
" service_id=text_service_id,\n",
" ai_model_id=text_service_id,\n",
" max_tokens=45,\n",
" temperature=0.5,\n",
" top_p=0.5,\n",
")\n",
"\n",
"prompt_template_config = PromptTemplateConfig(\n",
" template=my_prompt,\n",
" name=\"text_complete\",\n",
" template_format=\"semantic-kernel\",\n",
" execution_settings=execution_settings,\n",
")\n",
"\n",
"my_function = kernel.add_function(\n",
" function_name=\"text_complete\",\n",
" plugin_name=\"TextCompletionPlugin\",\n",
" prompt_template_config=prompt_template_config,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2calf857",
"metadata": {},
"source": [
"Let's now see what the completion looks like! Remember, \"gpt2\" is nowhere near as large as ChatGPT, so expect a much simpler answer.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"output = await kernel.invoke(\n",
" my_function,\n",
" request=\"What are whales?\",\n",
")\n",
"\n",
"output = str(output).strip()\n",
"\n",
"query_result1 = await memory.search(\n",
" collection=collection_id, query=\"What are sharks?\", limit=1, min_relevance_score=0.3\n",
")\n",
"\n",
"print(f\"The queried result for 'What are sharks?' is {query_result1[0].text}\")\n",
"\n",
"print(f\"{text_service_id} completed prompt with: '{output}'\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/07-native-function-inline.ipynb
+782
View File
@@ -0,0 +1,782 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "3c93ac5b",
"metadata": {},
"source": [
"# Running Native Functions\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "40201641",
"metadata": {},
"source": [
"Two of the previous notebooks showed how to [execute semantic functions inline](./03-semantic-function-inline.ipynb) and how to [run prompts from a file](./02-running-prompts-from-file.ipynb).\n",
"\n",
"In this notebook, we'll show how to use native functions from a file. We will also show how to call semantic functions from native functions.\n",
"\n",
"This can be useful in a few scenarios:\n",
"\n",
"- Writing logic around how to run a prompt that changes the prompt's outcome.\n",
"- Using external data sources to gather data to concatenate into your prompt.\n",
"- Validating user input data prior to sending it to the LLM prompt.\n",
"\n",
"Native functions are defined using standard Python code. The structure is simple, but not well documented at this point.\n",
"\n",
"The following examples are intended to help guide new users towards successful native & semantic function use with the SK Python framework.\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d90b0c13",
"metadata": {},
"source": [
"Prepare a semantic kernel instance first, loading also the AI service settings defined in the [Setup notebook](00-getting-started.ipynb):\n"
]
},
{
"cell_type": "markdown",
"id": "f39125a5",
"metadata": {},
"source": [
"Import Semantic Kernel SDK from pypi.org"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1da651d4",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"id": "5f726252",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ecfe74be",
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"id": "73a7fd96",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"id": "9a888bb7",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fddb5403",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"cell_type": "markdown",
"id": "fcee3dc1",
"metadata": {},
"source": [
"We now configure our Chat Completion service on the kernel."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dd150646",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"\n",
"kernel = Kernel()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "186767f8",
"metadata": {},
"source": [
"Let's create a **native** function that gives us a random number between 3 and a user input as the upper limit. We'll use this number to create 3-x paragraphs of text when passed to a semantic function.\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "589733c5",
"metadata": {},
"source": [
"First, let's create our native function.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ae29c207",
"metadata": {},
"outputs": [],
"source": [
"import random\n",
"\n",
"from semantic_kernel.functions import kernel_function\n",
"\n",
"\n",
"class GenerateNumberPlugin:\n",
" \"\"\"\n",
" Description: Generate a number between 3-x.\n",
" \"\"\"\n",
"\n",
" @kernel_function(\n",
" description=\"Generate a random number between 3-x\",\n",
" name=\"GenerateNumberThreeOrHigher\",\n",
" )\n",
" def generate_number_three_or_higher(self, input: str) -> str:\n",
" \"\"\"\n",
" Generate a number between 3-<input>\n",
" Example:\n",
" \"8\" => rand(3,8)\n",
" Args:\n",
" input -- The upper limit for the random number generation\n",
" Returns:\n",
" int value\n",
" \"\"\"\n",
" try:\n",
" return str(random.randint(3, int(input)))\n",
" except ValueError as e:\n",
" print(f\"Invalid input {input}\")\n",
" raise e"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f26b90c4",
"metadata": {},
"source": [
"Next, let's create a semantic function that accepts a number as `{{$input}}` and generates that number of paragraphs about two Corgis on an adventure. `$input` is a default variable semantic functions can use.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7890943f",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.ai.open_ai import AzureChatPromptExecutionSettings, OpenAIChatPromptExecutionSettings\n",
"from semantic_kernel.prompt_template import InputVariable, PromptTemplateConfig\n",
"\n",
"prompt = \"\"\"\n",
"Write a short story about two Corgis on an adventure.\n",
"The story must be:\n",
"- G rated\n",
"- Have a positive message\n",
"- No sexism, racism or other bias/bigotry\n",
"- Be exactly {{$input}} paragraphs long. It must be this length.\n",
"\"\"\"\n",
"\n",
"if selectedService == Service.OpenAI:\n",
" execution_settings = OpenAIChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-3.5-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" execution_settings = AzureChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-35-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"\n",
"prompt_template_config = PromptTemplateConfig(\n",
" template=prompt,\n",
" name=\"story\",\n",
" template_format=\"semantic-kernel\",\n",
" input_variables=[\n",
" InputVariable(name=\"input\", description=\"The user input\", is_required=True),\n",
" ],\n",
" execution_settings=execution_settings,\n",
")\n",
"\n",
"corgi_story = kernel.add_function(\n",
" function_name=\"CorgiStory\",\n",
" plugin_name=\"CorgiPlugin\",\n",
" prompt_template_config=prompt_template_config,\n",
")\n",
"\n",
"generate_number_plugin = kernel.add_plugin(GenerateNumberPlugin(), \"GenerateNumberPlugin\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2471c2ab",
"metadata": {},
"outputs": [],
"source": [
"# Run the number generator\n",
"generate_number_three_or_higher = generate_number_plugin[\"GenerateNumberThreeOrHigher\"]\n",
"number_result = await generate_number_three_or_higher(kernel, input=6)\n",
"print(number_result)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f043a299",
"metadata": {},
"outputs": [],
"source": [
"story = await corgi_story.invoke(kernel, input=number_result.value)"
]
},
{
"cell_type": "markdown",
"id": "7245e7a2",
"metadata": {},
"source": [
"_Note: depending on which model you're using, it may not respond with the proper number of paragraphs._\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59a60e2a",
"metadata": {},
"outputs": [],
"source": [
"print(f\"Generating a corgi story exactly {number_result.value} paragraphs long.\")\n",
"print(\"=====================================================\")\n",
"print(story)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "8ef29d16",
"metadata": {},
"source": [
"## Kernel Functions with Annotated Parameters\n",
"\n",
"That works! But let's expand on our example to make it more generic.\n",
"\n",
"For the native function, we'll introduce the lower limit variable. This means that a user will input two numbers and the number generator function will pick a number between the first and second input.\n",
"\n",
"We'll make use of the Python's `Annotated` class to hold these variables.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d54983d8",
"metadata": {},
"outputs": [],
"source": [
"kernel.remove_all_services()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "091f45e4",
"metadata": {},
"source": [
"Let's start with the native function. Notice that we're add the `@kernel_function` decorator that holds the name of the function as well as an optional description. The input parameters are configured as part of the function's signature, and we use the `Annotated` type to specify the required input arguments.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4ea462c2",
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"from typing import Annotated\n",
"\n",
"from semantic_kernel.functions import kernel_function\n",
"\n",
"\n",
"class GenerateNumberPlugin:\n",
" \"\"\"\n",
" Description: Generate a number between a min and a max.\n",
" \"\"\"\n",
"\n",
" @kernel_function(\n",
" name=\"GenerateNumber\",\n",
" description=\"Generate a random number between min and max\",\n",
" )\n",
" def generate_number(\n",
" self,\n",
" min: Annotated[int, \"the minimum number of paragraphs\"],\n",
" max: Annotated[int, \"the maximum number of paragraphs\"] = 10,\n",
" ) -> Annotated[int, \"the output is a number\"]:\n",
" \"\"\"\n",
" Generate a number between min-max\n",
" Example:\n",
" min=\"4\" max=\"10\" => rand(4,8)\n",
" Args:\n",
" min -- The lower limit for the random number generation\n",
" max -- The upper limit for the random number generation\n",
" Returns:\n",
" int value\n",
" \"\"\"\n",
" try:\n",
" return str(random.randint(min, max))\n",
" except ValueError as e:\n",
" print(f\"Invalid input {min} and {max}\")\n",
" raise e"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "48bcdf9e",
"metadata": {},
"outputs": [],
"source": [
"generate_number_plugin = kernel.add_plugin(GenerateNumberPlugin(), \"GenerateNumberPlugin\")\n",
"generate_number = generate_number_plugin[\"GenerateNumber\"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "6ad068d6",
"metadata": {},
"source": [
"Now let's also allow the semantic function to take in additional arguments. In this case, we're going to allow the our CorgiStory function to be written in a specified language. We'll need to provide a `paragraph_count` and a `language`.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8b8286fb",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"\"\"\n",
"Write a short story about two Corgis on an adventure.\n",
"The story must be:\n",
"- G rated\n",
"- Have a positive message\n",
"- No sexism, racism or other bias/bigotry\n",
"- Be exactly {{$paragraph_count}} paragraphs long\n",
"- Be written in this language: {{$language}}\n",
"\"\"\"\n",
"\n",
"if selectedService == Service.OpenAI:\n",
" execution_settings = OpenAIChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-3.5-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" execution_settings = AzureChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-35-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"\n",
"prompt_template_config = PromptTemplateConfig(\n",
" template=prompt,\n",
" name=\"summarize\",\n",
" template_format=\"semantic-kernel\",\n",
" input_variables=[\n",
" InputVariable(name=\"paragraph_count\", description=\"The number of paragraphs\", is_required=True),\n",
" InputVariable(name=\"language\", description=\"The language of the story\", is_required=True),\n",
" ],\n",
" execution_settings=execution_settings,\n",
")\n",
"\n",
"corgi_story = kernel.add_function(\n",
" function_name=\"CorgiStory\",\n",
" plugin_name=\"CorgiPlugin\",\n",
" prompt_template_config=prompt_template_config,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c8778bad",
"metadata": {},
"source": [
"Let's generate a paragraph count.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "28820d9d",
"metadata": {},
"outputs": [],
"source": [
"result = await generate_number.invoke(kernel, min=1, max=5)\n",
"num_paragraphs = result.value\n",
"print(f\"Generating a corgi story {num_paragraphs} paragraphs long.\")"
]
},
{
"cell_type": "markdown",
"id": "225a9147",
"metadata": {},
"source": [
"We can now invoke our corgi_story function using the `kernel` and the keyword arguments `paragraph_count` and `language`.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dbe07c4d",
"metadata": {},
"outputs": [],
"source": [
"# Pass the output to the semantic story function\n",
"desired_language = \"Spanish\"\n",
"story = await corgi_story.invoke(kernel, paragraph_count=num_paragraphs, language=desired_language)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6732a30b",
"metadata": {},
"outputs": [],
"source": [
"print(f\"Generating a corgi story {num_paragraphs} paragraphs long in {desired_language}.\")\n",
"print(\"=====================================================\")\n",
"print(story)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "fb786c54",
"metadata": {},
"source": [
"## Calling Native Functions within a Semantic Function\n",
"\n",
"One neat thing about the Semantic Kernel is that you can also call native functions from within Prompt Functions!\n",
"\n",
"We will make our CorgiStory semantic function call a native function `GenerateNames` which will return names for our Corgi characters.\n",
"\n",
"We do this using the syntax `{{plugin_name.function_name}}`. You can read more about our prompte templating syntax [here](../../../docs/PROMPT_TEMPLATE_LANGUAGE.md).\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d84c7d84",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.functions import kernel_function\n",
"\n",
"\n",
"class GenerateNamesPlugin:\n",
" \"\"\"\n",
" Description: Generate character names.\n",
" \"\"\"\n",
"\n",
" # The default function name will be the name of the function itself, however you can override this\n",
" # by setting the name=<name override> in the @kernel_function decorator. In this case, we're using\n",
" # the same name as the function name for simplicity.\n",
" @kernel_function(description=\"Generate character names\", name=\"generate_names\")\n",
" def generate_names(self) -> str:\n",
" \"\"\"\n",
" Generate two names.\n",
" Returns:\n",
" str\n",
" \"\"\"\n",
" names = {\"Hoagie\", \"Hamilton\", \"Bacon\", \"Pizza\", \"Boots\", \"Shorts\", \"Tuna\"}\n",
" first_name = random.choice(list(names))\n",
" names.remove(first_name)\n",
" second_name = random.choice(list(names))\n",
" return f\"{first_name}, {second_name}\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2ab7d65f",
"metadata": {},
"outputs": [],
"source": [
"generate_names_plugin = kernel.add_plugin(GenerateNamesPlugin(), plugin_name=\"GenerateNames\")\n",
"generate_names = generate_names_plugin[\"generate_names\"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "94decd3e",
"metadata": {},
"outputs": [],
"source": [
"prompt = \"\"\"\n",
"Write a short story about two Corgis on an adventure.\n",
"The story must be:\n",
"- G rated\n",
"- Have a positive message\n",
"- No sexism, racism or other bias/bigotry\n",
"- Be exactly {{$paragraph_count}} paragraphs long\n",
"- Be written in this language: {{$language}}\n",
"- The two names of the corgis are {{GenerateNames.generate_names}}\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "be72a503",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" execution_settings = OpenAIChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-3.5-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" execution_settings = AzureChatPromptExecutionSettings(\n",
" service_id=service_id,\n",
" ai_model_id=\"gpt-35-turbo\",\n",
" max_tokens=2000,\n",
" temperature=0.7,\n",
" )\n",
"\n",
"prompt_template_config = PromptTemplateConfig(\n",
" template=prompt,\n",
" name=\"corgi-new\",\n",
" template_format=\"semantic-kernel\",\n",
" input_variables=[\n",
" InputVariable(name=\"paragraph_count\", description=\"The number of paragraphs\", is_required=True),\n",
" InputVariable(name=\"language\", description=\"The language of the story\", is_required=True),\n",
" ],\n",
" execution_settings=execution_settings,\n",
")\n",
"\n",
"corgi_story = kernel.add_function(\n",
" function_name=\"CorgiStoryUpdated\",\n",
" plugin_name=\"CorgiPluginUpdated\",\n",
" prompt_template_config=prompt_template_config,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56e6cf0f",
"metadata": {},
"outputs": [],
"source": [
"result = await generate_number.invoke(kernel, min=1, max=5)\n",
"num_paragraphs = result.value"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7e980348",
"metadata": {},
"outputs": [],
"source": [
"desired_language = \"French\"\n",
"story = await corgi_story.invoke(kernel, paragraph_count=num_paragraphs, language=desired_language)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c4ade048",
"metadata": {},
"outputs": [],
"source": [
"print(f\"Generating a corgi story {num_paragraphs} paragraphs long in {desired_language}.\")\n",
"print(\"=====================================================\")\n",
"print(story)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "42f0c472",
"metadata": {},
"source": [
"### Recap\n",
"\n",
"A quick review of what we've learned here:\n",
"\n",
"- We've learned how to create native and prompt functions and register them to the kernel\n",
"- We've seen how we can use Kernel Arguments to pass in more custom variables into our prompt\n",
"- We've seen how we can call native functions within a prompt.\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/08-groundedness-checking.ipynb
+461
View File
@@ -0,0 +1,461 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "f5c76c5f",
"metadata": {},
"source": [
"# Groundedness Checking Plugins\n",
"\n",
"For the proper configuration settings and setup, please follow the steps outlined at the beginning of the [first](./00-getting-started.ipynb) getting started notebook.\n",
"\n",
"A well-known problem with large language models (LLMs) is that they make things up. These are sometimes called 'hallucinations' but a safer (and less anthropomorphic) term is 'ungrounded addition' - something in the text which cannot be firmly established. When attempting to establish whether or not something in an LLM response is 'true' we can either check for it in the supplied prompt (this is called 'narrow grounding') or use our general knowledge ('broad grounding'). Note that narrow grounding can lead to things being classified as 'true, but ungrounded.' For example \"I live in Switzerland\" is **not** _narrowly_ grounded in \"I live in Geneva\" even though it must be true (it **is** _broadly_ grounded).\n",
"\n",
"In this notebook we run a simple grounding pipeline, to see if a summary text has any ungrounded additions as compared to the original, and use this information to improve the summary text. This can be done in three stages:\n",
"\n",
"1. Make a list of the entities in the summary text\n",
"1. Check to see if these entities appear in the original (grounding) text\n",
"1. Remove the ungrounded entities from the summary text\n",
"\n",
"What is an 'entity' in this context? In its simplest form, it's a named object such as a person or place (so 'Dean' or 'Seattle'). However, the idea could be a _claim_ which relates concepts (such as 'Dean lives near Seattle'). In this notebook, we will keep to the simpler case of named objects.\n"
]
},
{
"cell_type": "markdown",
"id": "60e2513c",
"metadata": {},
"source": [
"Import Semantic Kernel SDK from pypi.org"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "19ce6d37",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"id": "93e86ea5",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "47a5560b",
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"id": "181f38db",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"id": "fadcfde4",
"metadata": {},
"source": [
"Let us define our grounding text:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "23b26e2f",
"metadata": {},
"outputs": [],
"source": [
"grounding_text = \"\"\"I am by birth a Genevese, and my family is one of the most distinguished of that republic.\n",
"My ancestors had been for many years counsellors and syndics, and my father had filled several public situations\n",
"with honour and reputation. He was respected by all who knew him for his integrity and indefatigable attention\n",
"to public business. He passed his younger days perpetually occupied by the affairs of his country; a variety\n",
"of circumstances had prevented his marrying early, nor was it until the decline of life that he became a husband\n",
"and the father of a family.\n",
"\n",
"As the circumstances of his marriage illustrate his character, I cannot refrain from relating them. One of his\n",
"most intimate friends was a merchant who, from a flourishing state, fell, through numerous mischances, into poverty.\n",
"This man, whose name was Beaufort, was of a proud and unbending disposition and could not bear to live in poverty\n",
"and oblivion in the same country where he had formerly been distinguished for his rank and magnificence. Having\n",
"paid his debts, therefore, in the most honourable manner, he retreated with his daughter to the town of Lucerne,\n",
"where he lived unknown and in wretchedness. My father loved Beaufort with the truest friendship and was deeply\n",
"grieved by his retreat in these unfortunate circumstances. He bitterly deplored the false pride which led his friend\n",
"to a conduct so little worthy of the affection that united them. He lost no time in endeavouring to seek him out,\n",
"with the hope of persuading him to begin the world again through his credit and assistance.\n",
"\n",
"Beaufort had taken effectual measures to conceal himself, and it was ten months before my father discovered his\n",
"abode. Overjoyed at this discovery, he hastened to the house, which was situated in a mean street near the Reuss.\n",
"But when he entered, misery and despair alone welcomed him. Beaufort had saved but a very small sum of money from\n",
"the wreck of his fortunes, but it was sufficient to provide him with sustenance for some months, and in the meantime\n",
"he hoped to procure some respectable employment in a merchant's house. The interval was, consequently, spent in\n",
"inaction; his grief only became more deep and rankling when he had leisure for reflection, and at length it took\n",
"so fast hold of his mind that at the end of three months he lay on a bed of sickness, incapable of any exertion.\n",
"\n",
"His daughter attended him with the greatest tenderness, but she saw with despair that their little fund was\n",
"rapidly decreasing and that there was no other prospect of support. But Caroline Beaufort possessed a mind of an\n",
"uncommon mould, and her courage rose to support her in her adversity. She procured plain work; she plaited straw\n",
"and by various means contrived to earn a pittance scarcely sufficient to support life.\n",
"\n",
"Several months passed in this manner. Her father grew worse; her time was more entirely occupied in attending him;\n",
"her means of subsistence decreased; and in the tenth month her father died in her arms, leaving her an orphan and\n",
"a beggar. This last blow overcame her, and she knelt by Beaufort's coffin weeping bitterly, when my father entered\n",
"the chamber. He came like a protecting spirit to the poor girl, who committed herself to his care; and after the\n",
"interment of his friend he conducted her to Geneva and placed her under the protection of a relation. Two years\n",
"after this event Caroline became his wife.\"\"\""
]
},
{
"cell_type": "markdown",
"id": "4fd80b62",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c2d4f01f",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"from samples.service_settings import ServiceSettings\n",
"\n",
"service_settings = ServiceSettings()\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = (\n",
" Service.AzureOpenAI\n",
" if service_settings.global_llm_service is None\n",
" else Service(service_settings.global_llm_service.lower())\n",
")\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"cell_type": "markdown",
"id": "723087dc",
"metadata": {},
"source": [
"We now configure our Chat Completion service on the kernel."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e13d3519",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, OpenAIChatCompletion\n",
"\n",
"kernel = Kernel()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" OpenAIChatCompletion(\n",
" service_id=service_id,\n",
" ),\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" kernel.add_service(\n",
" AzureChatCompletion(service_id=service_id, credential=AzureCliCredential()),\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "0c65f786",
"metadata": {},
"source": [
"## Import the Plugins\n",
"\n",
"We are going to be using the grounding plugin, to check its quality, and remove ungrounded additions:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56ed7688",
"metadata": {},
"outputs": [],
"source": [
"# note: using plugins from the samples folder\n",
"plugins_directory = \"../../../prompt_template_samples/\"\n",
"\n",
"groundingSemanticFunctions = kernel.add_plugin(parent_directory=plugins_directory, plugin_name=\"GroundingPlugin\")"
]
},
{
"cell_type": "markdown",
"id": "0d087993",
"metadata": {},
"source": [
"We can also extract the individual semantic functions for our use:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "738eb0ec",
"metadata": {},
"outputs": [],
"source": [
"entity_extraction = groundingSemanticFunctions[\"ExtractEntities\"]\n",
"reference_check = groundingSemanticFunctions[\"ReferenceCheckEntities\"]\n",
"entity_excision = groundingSemanticFunctions[\"ExciseEntities\"]"
]
},
{
"cell_type": "markdown",
"id": "e5bda16e",
"metadata": {},
"source": [
"## Calling Individual Semantic Functions\n",
"\n",
"We will start by calling the individual grounding functions in turn, to show their use. For this we need to create a same summary text:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c9a872f6",
"metadata": {},
"outputs": [],
"source": [
"summary_text = \"\"\"\n",
"My father, a respected resident of Milan, was a close friend of a merchant named Beaufort who, after a series of\n",
"misfortunes, moved to Zurich in poverty. My father was upset by his friend's troubles and sought him out,\n",
"finding him in a mean street. Beaufort had saved a small sum of money, but it was not enough to support him and\n",
"his daughter, Mary. Mary procured work to eke out a living, but after ten months her father died, leaving\n",
"her a beggar. My father came to her aid and two years later they married when they visited Rome.\n",
"\"\"\"\n",
"\n",
"summary_text = summary_text.replace(\"\\n\", \" \").replace(\" \", \" \")\n",
"print(summary_text)"
]
},
{
"cell_type": "markdown",
"id": "2a41eccb",
"metadata": {},
"source": [
"Some things to note:\n",
"\n",
"- The implied residence of Geneva has been changed to Milan\n",
"- Lucerne has been changed to Zurich\n",
"- Caroline has been renamed as Mary\n",
"- A reference to Rome has been added\n",
"\n",
"The grounding plugin has three stages:\n",
"\n",
"1. Extract entities from a summary text\n",
"2. Perform a reference check against the grounding text\n",
"3. Excise any entities which failed the reference check from the summary\n",
"\n",
"Now, let us start calling individual semantic functions.\n"
]
},
{
"cell_type": "markdown",
"id": "071c05e4",
"metadata": {},
"source": [
"### Extracting the Entities\n",
"\n",
"The first function we need is entity extraction. We are going to take our summary text, and get a list of entities found within it. For this we use `entity_extraction()`:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c1d4b7ff",
"metadata": {},
"outputs": [],
"source": [
"extraction_result = await kernel.invoke(\n",
" entity_extraction,\n",
" input=summary_text,\n",
" topic=\"people and places\",\n",
" example_entities=\"John, Jane, mother, brother, Paris, Rome\",\n",
")\n",
"\n",
"print(extraction_result)"
]
},
{
"cell_type": "markdown",
"id": "b93c661f",
"metadata": {},
"source": [
"So we have our list of entities in the summary\n"
]
},
{
"cell_type": "markdown",
"id": "958ad1ff",
"metadata": {},
"source": [
"### Performing the reference check\n",
"\n",
"We now use the grounding text to see if the entities we found are grounded. We start by adding the grounding text to our context:\n"
]
},
{
"cell_type": "markdown",
"id": "894e38d7",
"metadata": {},
"source": [
"With this in place, we can run the reference checking function. This will use both the entity list in the input, and the `reference_context` in the context object itself:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d240d669",
"metadata": {},
"outputs": [],
"source": [
"grounding_result = await kernel.invoke(\n",
" reference_check, input=str(extraction_result.value), reference_context=grounding_text\n",
")\n",
"\n",
"print(grounding_result)"
]
},
{
"cell_type": "markdown",
"id": "9a83c66f",
"metadata": {},
"source": [
"So we now have a list of ungrounded entities (of course, this list may not be well grounded itself). Let us store this in the context:\n"
]
},
{
"cell_type": "markdown",
"id": "35c1c329",
"metadata": {},
"source": [
"### Excising the ungrounded entities\n",
"\n",
"Finally we can remove the ungrounded entities from the summary text:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "db82d97d",
"metadata": {},
"outputs": [],
"source": [
"excision_result = await kernel.invoke(\n",
" entity_excision, input=summary_text, ungrounded_entities=str(grounding_result.value)\n",
")\n",
"\n",
"print(excision_result)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/09-multiple-results-per-prompt.ipynb
+510
View File
@@ -0,0 +1,510 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Multiple Results\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "fb81bacd",
"metadata": {},
"source": [
"In this notebook we show how you can in a single request, have the LLM model return multiple results per prompt. This is useful for running experiments where you want to evaluate the robustness of your prompt and the parameters of your config against a particular large language model.\n"
]
},
{
"cell_type": "markdown",
"id": "f7120635",
"metadata": {},
"source": [
"Import Semantic Kernel SDK from pypi.org"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"id": "4ad09f90",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5cff141d",
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"id": "d4d76e3d",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"id": "73c2e146",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f924e1f4",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = Service.OpenAI\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will set up the text and chat services we will be submitting prompts to.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"from semantic_kernel.connectors.ai.open_ai import (\n",
" AzureChatCompletion,\n",
" AzureChatPromptExecutionSettings, # noqa: F401\n",
" AzureTextCompletion,\n",
" OpenAIChatCompletion,\n",
" OpenAIChatPromptExecutionSettings, # noqa: F401\n",
" OpenAITextCompletion,\n",
" OpenAITextPromptExecutionSettings, # noqa: F401\n",
")\n",
"\n",
"kernel = Kernel()\n",
"\n",
"# Configure Azure LLM service\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" oai_chat_service = OpenAIChatCompletion(\n",
" service_id=\"oai_chat\",\n",
" )\n",
" oai_text_service = OpenAITextCompletion(\n",
" service_id=\"oai_text\",\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" credential = AzureCliCredential()\n",
" service_id = \"default\"\n",
" aoai_chat_service = AzureChatCompletion(service_id=\"aoai_chat\", credential=credential)\n",
" aoai_text_service = AzureTextCompletion(service_id=\"aoai_text\", credential=credential)\n",
"\n",
"# Configure Hugging Face service\n",
"if selectedService == Service.HuggingFace:\n",
" from semantic_kernel.connectors.ai.hugging_face import ( # noqa: F401\n",
" HuggingFacePromptExecutionSettings,\n",
" HuggingFaceTextCompletion,\n",
" )\n",
"\n",
" hf_text_service = HuggingFaceTextCompletion(service_id=\"hf_text\", ai_model_id=\"distilgpt2\", task=\"text-generation\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "50561d82",
"metadata": {},
"source": [
"Next, we'll set up the completion request settings for text completion services.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"oai_text_prompt_execution_settings = OpenAITextPromptExecutionSettings(\n",
" service=\"oai_text\",\n",
" extension_data={\n",
" \"max_tokens\": 80,\n",
" \"temperature\": 0.7,\n",
" \"top_p\": 1,\n",
" \"frequency_penalty\": 0.5,\n",
" \"presence_penalty\": 0.5,\n",
" \"number_of_responses\": 3,\n",
" },\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "857a9c89",
"metadata": {},
"source": [
"## Multiple Open AI Text Completions\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2979db8",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" prompt = \"What is the purpose of a rubber duck?\"\n",
"\n",
" results = await oai_text_service.get_text_contents(prompt=prompt, settings=oai_text_prompt_execution_settings)\n",
"\n",
" for i, result in enumerate(results):\n",
" print(f\"Result {i + 1}: {result}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4288d09f",
"metadata": {},
"source": [
"## Multiple Azure Open AI Text Completions\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5319f14d",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.AzureOpenAI:\n",
" prompt = \"provide me a list of possible meanings for the acronym 'ORLD'\"\n",
"\n",
" results = await aoai_text_service.get_text_contents(prompt=prompt, settings=oai_text_prompt_execution_settings)\n",
"\n",
" for i, result in enumerate(results):\n",
" print(f\"Result {i + 1}: {result}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "eb548f9c",
"metadata": {},
"source": [
"## Multiple Hugging Face Text Completions\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4a148709",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.HuggingFace:\n",
" hf_prompt_execution_settings = HuggingFacePromptExecutionSettings(\n",
" service_id=\"hf_text\",\n",
" extension_data={\"max_new_tokens\": 80, \"temperature\": 0.7, \"top_p\": 1, \"num_return_sequences\": 3},\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9525e4f3",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.HuggingFace:\n",
" prompt = \"The purpose of a rubber duck is\"\n",
"\n",
" results = await hf_text_service.get_text_contents(prompt=prompt, settings=hf_prompt_execution_settings)\n",
"\n",
" for i, result in enumerate(results):\n",
" print(f\"Result {i + 1}: {result}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "da632e12",
"metadata": {},
"source": [
"Here, we're setting up the settings for Chat completions.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5f11e46",
"metadata": {},
"outputs": [],
"source": [
"oai_chat_prompt_execution_settings = OpenAIChatPromptExecutionSettings(\n",
" service_id=\"oai_chat\",\n",
" max_tokens=80,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
" number_of_responses=3,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d6bf238e",
"metadata": {},
"source": [
"## Multiple OpenAI Chat Completions\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dabc6a4c",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.contents import ChatHistory\n",
"\n",
"if selectedService == Service.OpenAI:\n",
" chat = ChatHistory()\n",
" chat.add_user_message(\n",
" \"It's a beautiful day outside, birds are singing, flowers are blooming. On days like these, kids like you...\"\n",
" )\n",
" results = await oai_chat_service.get_chat_message_contents(\n",
" chat_history=chat, settings=oai_chat_prompt_execution_settings\n",
" )\n",
"\n",
" for i, result in enumerate(results):\n",
" print(f\"Result {i + 1}: {result!s}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cdb8f740",
"metadata": {},
"source": [
"## Multiple Azure OpenAI Chat Completions\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "66ba4767",
"metadata": {},
"outputs": [],
"source": [
"az_oai_prompt_execution_settings = AzureChatPromptExecutionSettings(\n",
" service_id=\"aoai_chat\",\n",
" max_tokens=80,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
" number_of_responses=3,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b74a64a9",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.AzureOpenAI:\n",
" content = (\n",
" \"Tomorrow is going to be a great day, I can feel it. I'm going to wake up early, go for a run, and then...\"\n",
" )\n",
" chat = ChatHistory()\n",
" chat.add_user_message(content)\n",
" results = await aoai_chat_service.get_chat_message_contents(\n",
" chat_history=chat, settings=az_oai_prompt_execution_settings\n",
" )\n",
"\n",
" for i, result in enumerate(results):\n",
" print(f\"Result {i + 1}: {result!s}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "98c8191d",
"metadata": {},
"source": [
"## Streaming Multiple Results\n",
"\n",
"Here is an example pattern if you want to stream your multiple results. Note that this is not supported for Hugging Face text completions at this time.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "26a37702",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" import os\n",
" import time\n",
"\n",
" from IPython.display import clear_output\n",
"\n",
" # Determine the clear command based on OS\n",
" clear_command = \"cls\" if os.name == \"nt\" else \"clear\"\n",
"\n",
" chat = ChatHistory()\n",
" chat.add_user_message(\"what is the purpose of a rubber duck?\")\n",
"\n",
" stream = oai_chat_service.get_streaming_chat_message_contents(\n",
" chat_history=chat, settings=oai_chat_prompt_execution_settings\n",
" )\n",
" number_of_responses = oai_chat_prompt_execution_settings.number_of_responses\n",
" texts = [\"\"] * number_of_responses\n",
"\n",
" last_clear_time = time.time()\n",
" clear_interval = 0.5 # seconds\n",
"\n",
" # Note: there are some quirks with displaying the output, which sometimes flashes and disappears.\n",
" # This could be influenced by a few factors specific to Jupyter notebooks and asynchronous processing.\n",
" # The following code attempts to buffer the results to avoid the output flashing on/off the screen.\n",
"\n",
" async for results in stream:\n",
" current_time = time.time()\n",
"\n",
" # Update texts with new results\n",
" for result in results:\n",
" texts[result.choice_index] += str(result)\n",
"\n",
" # Clear and display output at intervals\n",
" if current_time - last_clear_time > clear_interval:\n",
" clear_output(wait=True)\n",
" for idx, text in enumerate(texts):\n",
" print(f\"Result {idx + 1}: {text}\")\n",
" last_clear_time = current_time\n",
"\n",
" print(\"----------------------------------------\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/10-streaming-completions.ipynb
+433
View File
@@ -0,0 +1,433 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "68e1c158",
"metadata": {},
"source": [
"# Streaming Results\n",
"\n",
"Here is an example pattern if you want to stream your multiple results. Note that this is not supported for Hugging Face text completions at this time.\n"
]
},
{
"cell_type": "markdown",
"id": "a3dd8590",
"metadata": {},
"source": [
"Import Semantic Kernel SDK from pypi.org"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a77bdf89",
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"id": "fd94029f",
"metadata": {},
"source": [
"Initial configuration for the notebook to run properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7547e59b",
"metadata": {},
"outputs": [],
"source": [
"# Make sure paths are correct for the imports\n",
"\n",
"import os\n",
"import sys\n",
"\n",
"notebook_dir = os.path.abspath(\"\")\n",
"parent_dir = os.path.dirname(notebook_dir)\n",
"grandparent_dir = os.path.dirname(parent_dir)\n",
"\n",
"\n",
"sys.path.append(grandparent_dir)"
]
},
{
"cell_type": "markdown",
"id": "73ba03ae",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
"\n",
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
"\n",
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
"2. Run `az login` command in terminal and follow the authentication steps.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"id": "fd931c14",
"metadata": {},
"source": [
"We will load our settings and get the LLM service to use for the notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a9a5c87a",
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = Service.OpenAI\n",
"print(f\"Using service type: {selectedService}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d8ddffc1",
"metadata": {},
"source": [
"First, we will set up the text and chat services we will be submitting prompts to.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8f8dcbc6",
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel import Kernel\n",
"from semantic_kernel.connectors.ai.open_ai import (\n",
" AzureChatCompletion,\n",
" AzureChatPromptExecutionSettings, # noqa: F401\n",
" AzureTextCompletion,\n",
" OpenAIChatCompletion,\n",
" OpenAIChatPromptExecutionSettings, # noqa: F401\n",
" OpenAITextCompletion,\n",
" OpenAITextPromptExecutionSettings, # noqa: F401\n",
")\n",
"from semantic_kernel.contents import ChatHistory # noqa: F401\n",
"\n",
"kernel = Kernel()\n",
"\n",
"service_id = None\n",
"if selectedService == Service.OpenAI:\n",
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
"\n",
" service_id = \"default\"\n",
" oai_chat_service = OpenAIChatCompletion(\n",
" service_id=\"oai_chat\",\n",
" )\n",
" oai_text_service = OpenAITextCompletion(\n",
" service_id=\"oai_text\",\n",
" )\n",
"elif selectedService == Service.AzureOpenAI:\n",
" from azure.identity import AzureCliCredential\n",
"\n",
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
"\n",
" credential = AzureCliCredential()\n",
" service_id = \"default\"\n",
" aoai_chat_service = AzureChatCompletion(service_id=\"aoai_chat\", credential=credential)\n",
" aoai_text_service = AzureTextCompletion(service_id=\"aoai_text\", credential=credential)\n",
"\n",
"# Configure Hugging Face service\n",
"if selectedService == Service.HuggingFace:\n",
" from semantic_kernel.connectors.ai.hugging_face import (\n",
" HuggingFacePromptExecutionSettings, # noqa: F401\n",
" HuggingFaceTextCompletion,\n",
" )\n",
"\n",
" hf_text_service = HuggingFaceTextCompletion(ai_model_id=\"distilgpt2\", task=\"text-generation\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "50561d82",
"metadata": {},
"source": [
"Next, we'll set up the completion request settings for text completion services.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "628c843e",
"metadata": {},
"outputs": [],
"source": [
"oai_prompt_execution_settings = OpenAITextPromptExecutionSettings(\n",
" service_id=\"oai_text\",\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "857a9c89",
"metadata": {},
"source": [
"## Streaming Open AI Text Completion\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2979db8",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" prompt = \"What is the purpose of a rubber duck?\"\n",
" stream = oai_text_service.get_streaming_text_contents(prompt=prompt, settings=oai_prompt_execution_settings)\n",
" async for message in stream:\n",
" print(str(message[0]), end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4288d09f",
"metadata": {},
"source": [
"## Streaming Azure Open AI Text Completion\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5319f14d",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.AzureOpenAI:\n",
" prompt = \"provide me a list of possible meanings for the acronym 'ORLD'\"\n",
" stream = aoai_text_service.get_streaming_text_contents(prompt=prompt, settings=oai_prompt_execution_settings)\n",
" async for message in stream:\n",
" print(str(message[0]), end=\"\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "eb548f9c",
"metadata": {},
"source": [
"## Streaming Hugging Face Text Completion\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "be7b1c2e",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.HuggingFace:\n",
" hf_prompt_execution_settings = HuggingFacePromptExecutionSettings(\n",
" service_id=\"hf_text\",\n",
" extension_data={\n",
" \"max_new_tokens\": 80,\n",
" \"top_p\": 1,\n",
" \"eos_token_id\": 11,\n",
" \"pad_token_id\": 0,\n",
" },\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9525e4f3",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.HuggingFace:\n",
" prompt = \"The purpose of a rubber duck is\"\n",
" stream = hf_text_service.get_streaming_text_contents(\n",
" prompt=prompt, prompt_execution_settings=hf_prompt_execution_settings\n",
" )\n",
" async for text in stream:\n",
" print(str(text[0]), end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "da632e12",
"metadata": {},
"source": [
"Here, we're setting up the settings for Chat completions.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5f11e46",
"metadata": {},
"outputs": [],
"source": [
"oai_chat_prompt_execution_settings = OpenAIChatPromptExecutionSettings(\n",
" service_id=\"oai_chat\",\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d6bf238e",
"metadata": {},
"source": [
"## Streaming OpenAI Chat Completion\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dabc6a4c",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.OpenAI:\n",
" content = \"You are an AI assistant that helps people find information.\"\n",
" chat = ChatHistory()\n",
" chat.add_system_message(content)\n",
" stream = oai_chat_service.get_streaming_chat_message_contents(\n",
" chat_history=chat, settings=oai_chat_prompt_execution_settings\n",
" )\n",
" async for text in stream:\n",
" print(str(text[0]), end=\"\") # end = \"\" to avoid newlines"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cdb8f740",
"metadata": {},
"source": [
"## Streaming Azure OpenAI Chat Completion\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da1e9f59",
"metadata": {},
"outputs": [],
"source": [
"az_oai_chat_prompt_execution_settings = AzureChatPromptExecutionSettings(\n",
" service_id=\"aoai_chat\",\n",
" max_tokens=150,\n",
" temperature=0.7,\n",
" top_p=1,\n",
" frequency_penalty=0.5,\n",
" presence_penalty=0.5,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b74a64a9",
"metadata": {},
"outputs": [],
"source": [
"if selectedService == Service.AzureOpenAI:\n",
" content = \"You are an AI assistant that helps people find information.\"\n",
" chat = ChatHistory()\n",
" chat.add_system_message(content)\n",
" chat.add_user_message(\"What is the purpose of a rubber duck?\")\n",
" stream = aoai_chat_service.get_streaming_chat_message_contents(\n",
" chat_history=chat, settings=az_oai_chat_prompt_execution_settings\n",
" )\n",
" async for text in stream:\n",
" print(str(text[0]), end=\"\") # end = \"\" to avoid newlines"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
python/samples/getting_started/CONFIGURING_THE_KERNEL.md
+67
View File
@@ -0,0 +1,67 @@
## Configuring the Kernel
As covered in the notebooks, we require a `.env` file with the proper settings for the model you use. A `.env` file must be placed in the `getting_started` directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.
If interested, as you learn more about Semantic Kernel, there are a few other ways to make sure your secrets, keys, and settings are used:
### 1. Environment Variables
Set the keys/secrets/endpoints as environment variables in your system. In Semantic Kernel, we leverage Pydantic Settings. If using Environment Variables, it isn't required to pass in explicit arguments to class constructors.
**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file or environment variables. If this setting is not included, the Service will default to AzureOpenAI.**
#### Option 1: using OpenAI
Add your [OpenAI Key](https://platform.openai.com/docs/overview) key to either your environment variables or to the `.env` file in the same folder (org Id only if you have multiple orgs):
```
GLOBAL_LLM_SERVICE="OpenAI"
OPENAI_API_KEY="sk-..."
OPENAI_ORG_ID=""
OPENAI_CHAT_MODEL_ID=""
```
The environment variables names should match the names used in the `.env` file, as shown above.
Use "keyword arguments" to instantiate an OpenAI Chat Completion service and add it to the kernel:
#### Option 2: using Azure OpenAI
Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to either your system's environment variables or to the `.env` file in the same folder:
```
GLOBAL_LLM_SERVICE="AzureOpenAI"
AZURE_OPENAI_API_KEY="..."
AZURE_OPENAI_ENDPOINT="https://..."
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="..."
AZURE_OPENAI_TEXT_DEPLOYMENT_NAME="..."
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME="..."
AZURE_OPENAI_API_VERSION="..."
```
The environment variables names should match the names used in the `.env` file, as shown above.
Use "keyword arguments" to instantiate an Azure OpenAI Chat Completion service and add it to the kernel:
### 2. Custom .env file path
It is possible to configure the constructor with an absolute or relative file path to point the settings to a `.env` file located outside of the `getting_started` directory.
For OpenAI:
```
chat_completion = OpenAIChatCompletion(service_id="test", env_file_path='/path/to/file')
```
For AzureOpenAI:
```
chat_completion = AzureChatCompletion(service_id="test", env_file_path=env_file_path='/path/to/file')
```
### 3. Manual Configuration
- Manually configure the `api_key` or required parameters on either the `OpenAIChatCompletion` or `AzureChatCompletion` constructor with keyword arguments.
- This requires the user to manage their own keys/secrets as they aren't relying on the underlying environment variables or `.env` file.
### 4. Microsoft Entra Authentication
To learn how to use a Microsoft Entra Authentication token to authenticate to your Azure OpenAI resource, please navigate to the following [guide](../concepts/README.md#microsoft-entra-token-authentication).
python/samples/getting_started/services.py
+17
View File
@@ -0,0 +1,17 @@
# Copyright (c) Microsoft. All rights reserved.
"""This module defines an enumeration representing different services."""
from enum import Enum
class Service(Enum):
"""Attributes:
OpenAI (str): Represents the OpenAI service.
AzureOpenAI (str): Represents the Azure OpenAI service.
HuggingFace (str): Represents the HuggingFace service.
"""
OpenAI = "openai"
AzureOpenAI = "azureopenai"
HuggingFace = "huggingface"
python/samples/getting_started/third_party/.env.example
Vendored
+31
View File
@@ -0,0 +1,31 @@
GLOBAL_LLM_SERVICE=""
OPENAI_API_KEY=""
OPENAI_CHAT_MODEL_ID=""
OPENAI_TEXT_MODEL_ID=""
OPENAI_EMBEDDING_MODEL_ID=""
OPENAI_ORG_ID=""
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=""
AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=""
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=""
AZURE_OPENAI_ENDPOINT=""
AZURE_OPENAI_API_KEY=""
AZURE_AI_SEARCH_API_KEY=""
AZURE_AI_SEARCH_ENDPOINT=""
# -- WEAVIATE SETTINGS --
WEAVIATE_URL="http://localhost:8080"
# WEAVIATE_API_KEY=""
# -- POSTGRES SETTINGS --
# Set either POSTGRES_CONNECTION_STRING or the individual PG settings below
POSTGRES_CONNECTION_STRING=""
# PGHOST=""
# PGPORT=""
# PGDATABASE=""
# PGUSER=""
# PGPASSWORD=""
# PGSSL_MODE=""
python/samples/getting_started/third_party/postgres-memory.ipynb
Vendored
+710
View File
@@ -0,0 +1,710 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Using Postgres as memory\n",
"\n",
"This notebook shows how to use Postgres as a memory store in Semantic Kernel.\n",
"\n",
"The code below pulls the most recent papers from [ArviX](https://arxiv.org/), creates embeddings from the paper abstracts, and stores them in a Postgres database.\n",
"\n",
"In the future, we can use the Postgres vector store to search the database for similar papers based on the embeddings - stay tuned!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import textwrap\n",
"import xml.etree.ElementTree as ET\n",
"from dataclasses import dataclass\n",
"from datetime import datetime\n",
"from typing import Annotated, Any\n",
"\n",
"import requests\n",
"\n",
"from semantic_kernel import Kernel\n",
"from semantic_kernel.connectors.ai import FunctionChoiceBehavior\n",
"from semantic_kernel.connectors.ai.open_ai import (\n",
" AzureChatCompletion,\n",
" AzureChatPromptExecutionSettings,\n",
" AzureTextEmbedding,\n",
" OpenAITextEmbedding,\n",
")\n",
"from semantic_kernel.connectors.postgres import PostgresCollection\n",
"from semantic_kernel.contents import ChatHistory\n",
"from semantic_kernel.data.vector import (\n",
" DistanceFunction,\n",
" IndexKind,\n",
" VectorStoreField,\n",
" vectorstoremodel,\n",
")\n",
"from semantic_kernel.functions import KernelParameterMetadata\n",
"from semantic_kernel.functions.kernel_arguments import KernelArguments"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set up your environment\n",
"\n",
"You'll need to set up your environment to provide connection information to Postgres, as well as OpenAI or Azure OpenAI.\n",
"\n",
"To do this, copy the `.env.example` file to `.env` and fill in the necessary information.\n",
"\n",
"__Note__: If you're using VSCode to execute the notebook, the settings in `.env` in the root of the repository will be picked up automatically.\n",
"\n",
"### Postgres configuration\n",
"\n",
"You'll need to provide a connection string to a Postgres database. You can use a local Postgres instance, or a cloud-hosted one.\n",
"You can provide a connection string, or provide environment variables with the connection information. See the .env.example file for `POSTGRES_` settings.\n",
"\n",
"#### Using Docker\n",
"\n",
"You can also use docker to bring up a Postgres instance by following the steps below:\n",
"\n",
"Create an `init.sql` that has the following:\n",
"\n",
"```sql\n",
"CREATE EXTENSION IF NOT EXISTS vector;\n",
"```\n",
"\n",
"Now you can start a postgres instance with the following:\n",
"\n",
"```\n",
"docker pull pgvector/pgvector:pg16\n",
"docker run --rm -it --name pgvector -p 5432:5432 -v ./init.sql:/docker-entrypoint-initdb.d/init.sql -e POSTGRES_PASSWORD=example pgvector/pgvector:pg16\n",
"```\n",
"\n",
"_Note_: Use `.\\init.sql` on Windows and `./init.sql` on WSL or Linux/Mac.\n",
"\n",
"Then you could use the connection string:\n",
"\n",
"```\n",
"POSTGRES_CONNECTION_STRING=\"host=localhost port=5432 dbname=postgres user=postgres password=example\"\n",
"```\n",
"\n",
"### OpenAI configuration\n",
"\n",
"You can either use OpenAI or Azure OpenAI APIs. You provide the API key and other configuration in the `.env` file. Set either the `OPENAI_` or `AZURE_OPENAI_` settings.\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# Path to the environment file\n",
"env_file_path = \".env\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we set some additional configuration."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# -- ArXiv settings --\n",
"\n",
"# The search term to use when searching for papers on arXiv. All metadata fields for the papers are searched.\n",
"SEARCH_TERM = \"RAG\"\n",
"\n",
"# The category of papers to search for on arXiv. See https://arxiv.org/category_taxonomy for a list of categories.\n",
"ARVIX_CATEGORY = \"cs.AI\"\n",
"\n",
"# The maximum number of papers to search for on arXiv.\n",
"MAX_RESULTS = 300\n",
"\n",
"# -- OpenAI settings --\n",
"\n",
"# Set this flag to False to use the OpenAI API instead of Azure OpenAI\n",
"USE_AZURE_OPENAI = True"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we define a vector store model. This model defines the table and column names for storing the embeddings. We use the `@vectorstoremodel` decorator to tell Semantic Kernel to create a vector store definition from the model. The VectorStoreRecordField annotations define the fields that will be stored in the database, including key and vector fields."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"@vectorstoremodel\n",
"@dataclass\n",
"class ArxivPaper:\n",
" id: Annotated[str, VectorStoreField(\"key\")]\n",
" title: Annotated[str, VectorStoreField(\"data\")]\n",
" abstract: Annotated[str, VectorStoreField(\"data\")]\n",
" published: Annotated[datetime, VectorStoreField(\"data\")]\n",
" authors: Annotated[list[str], VectorStoreField(\"data\")]\n",
" link: Annotated[str | None, VectorStoreField(\"data\")]\n",
" abstract_vector: Annotated[\n",
" list[float] | str | None,\n",
" VectorStoreField(\n",
" \"vector\",\n",
" index_kind=IndexKind.HNSW,\n",
" dimensions=1536,\n",
" distance_function=DistanceFunction.COSINE_DISTANCE,\n",
" ),\n",
" ] = None\n",
"\n",
" def __post_init__(self):\n",
" if self.abstract_vector is None:\n",
" self.abstract_vector = self.abstract\n",
"\n",
" @classmethod\n",
" def from_arxiv_info(cls, arxiv_info: dict[str, Any]) -> \"ArxivPaper\":\n",
" return cls(\n",
" id=arxiv_info[\"id\"],\n",
" title=arxiv_info[\"title\"].replace(\"\\n \", \" \"),\n",
" abstract=arxiv_info[\"abstract\"].replace(\"\\n \", \" \"),\n",
" published=arxiv_info[\"published\"],\n",
" authors=arxiv_info[\"authors\"],\n",
" link=arxiv_info[\"link\"],\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Below is a function that queries the ArviX API for the most recent papers based on our search query and category."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"def query_arxiv(search_query: str, category: str = \"cs.AI\", max_results: int = 10) -> list[dict[str, Any]]:\n",
" \"\"\"\n",
" Query the ArXiv API and return a list of dictionaries with relevant metadata for each paper.\n",
"\n",
" Args:\n",
" search_query: The search term or topic to query for.\n",
" category: The category to restrict the search to (default is \"cs.AI\").\n",
" See https://arxiv.org/category_taxonomy for a list of categories.\n",
" max_results: Maximum number of results to retrieve (default is 10).\n",
" \"\"\"\n",
" response = requests.get(\n",
" \"http://export.arxiv.org/api/query?\"\n",
" f\"search_query=all:%22{search_query.replace(' ', '+')}%22\"\n",
" f\"+AND+cat:{category}&start=0&max_results={max_results}&sortBy=lastUpdatedDate&sortOrder=descending\"\n",
" )\n",
"\n",
" root = ET.fromstring(response.content)\n",
" ns = {\"atom\": \"http://www.w3.org/2005/Atom\"}\n",
"\n",
" return [\n",
" {\n",
" \"id\": entry.find(\"atom:id\", ns).text.split(\"/\")[-1],\n",
" \"title\": entry.find(\"atom:title\", ns).text,\n",
" \"abstract\": entry.find(\"atom:summary\", ns).text,\n",
" \"published\": entry.find(\"atom:published\", ns).text,\n",
" \"link\": entry.find(\"atom:id\", ns).text,\n",
" \"authors\": [author.find(\"atom:name\", ns).text for author in entry.findall(\"atom:author\", ns)],\n",
" \"categories\": [category.get(\"term\") for category in entry.findall(\"atom:category\", ns)],\n",
" \"pdf_link\": next(\n",
" (link_tag.get(\"href\") for link_tag in entry.findall(\"atom:link\", ns) if link_tag.get(\"title\") == \"pdf\"),\n",
" None,\n",
" ),\n",
" }\n",
" for entry in root.findall(\"atom:entry\", ns)\n",
" ]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We use this function to query papers and store them in memory as our model types."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 300 papers on 'RAG'\n"
]
}
],
"source": [
"arxiv_papers: list[ArxivPaper] = [\n",
" ArxivPaper.from_arxiv_info(paper)\n",
" for paper in query_arxiv(SEARCH_TERM, category=ARVIX_CATEGORY, max_results=MAX_RESULTS)\n",
"]\n",
"\n",
"print(f\"Found {len(arxiv_papers)} papers on '{SEARCH_TERM}'\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a `PostgresCollection`, which represents the table in Postgres where we will store the paper information and embeddings."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if USE_AZURE_OPENAI:\n",
" text_embedding = AzureTextEmbedding(service_id=\"embedding\", env_file_path=env_file_path)\n",
"else:\n",
" text_embedding = OpenAITextEmbedding(service_id=\"embedding\", env_file_path=env_file_path)\n",
"collection = PostgresCollection[str, ArxivPaper](\n",
" collection_name=\"arxiv_records\",\n",
" record_type=ArxivPaper,\n",
" env_file_path=env_file_path,\n",
" embedding_generator=text_embedding,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that the models have embeddings, we can write them into the Postgres database."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"async with collection:\n",
" await collection.ensure_collection_exists()\n",
" keys = await collection.upsert(arxiv_papers)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we retrieve the first few models from the database and print out their information."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Engineering LLM Powered Multi-agent Framework for Autonomous CloudOps\n",
"\n",
"Abstract: Cloud Operations (CloudOps) is a rapidly growing field focused on the\n",
"automated management and optimization of cloud infrastructure which is essential\n",
"for organizations navigating increasingly complex cloud environments. MontyCloud\n",
"Inc. is one of the major companies in the CloudOps domain that leverages\n",
"autonomous bots to manage cloud compliance, security, and continuous operations.\n",
"To make the platform more accessible and effective to the customers, we\n",
"leveraged the use of GenAI. Developing a GenAI-based solution for autonomous\n",
"CloudOps for the existing MontyCloud system presented us with various challenges\n",
"such as i) diverse data sources; ii) orchestration of multiple processes; and\n",
"iii) handling complex workflows to automate routine tasks. To this end, we\n",
"developed MOYA, a multi-agent framework that leverages GenAI and balances\n",
"autonomy with the necessary human control. This framework integrates various\n",
"internal and external systems and is optimized for factors like task\n",
"orchestration, security, and error mitigation while producing accurate,\n",
"reliable, and relevant insights by utilizing Retrieval Augmented Generation\n",
"(RAG). Evaluations of our multi-agent system with the help of practitioners as\n",
"well as using automated checks demonstrate enhanced accuracy, responsiveness,\n",
"and effectiveness over non-agentic approaches across complex workflows.\n",
"Published: 2025-01-14 16:30:10\n",
"Link: http://arxiv.org/abs/2501.08243v1\n",
"PDF Link: http://arxiv.org/abs/2501.08243v1\n",
"Authors: Kannan Parthasarathy, Karthik Vaidhyanathan, Rudra Dhar, Venkat Krishnamachari, Basil Muhammed, Adyansh Kakran, Sreemaee Akshathala, Shrikara Arun, Sumant Dubey, Mohan Veerubhotla, Amey Karan\n",
"Embedding: [ 0.01063822 0.02977918 0.04532182 ... -0.00264323 0.00081101\n",
" 0.01491571]\n",
"\n",
"\n",
"# Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models\n",
"\n",
"Abstract: Recent advancements in long-context language models (LCLMs) promise to\n",
"transform Retrieval-Augmented Generation (RAG) by simplifying pipelines. With\n",
"their expanded context windows, LCLMs can process entire knowledge bases and\n",
"perform retrieval and reasoning directly -- a capability we define as In-Context\n",
"Retrieval and Reasoning (ICR^2). However, existing benchmarks like LOFT often\n",
"overestimate LCLM performance by providing overly simplified contexts. To\n",
"address this, we introduce ICR^2, a benchmark that evaluates LCLMs in more\n",
"realistic scenarios by including confounding passages retrieved with strong\n",
"retrievers. We then propose three methods to enhance LCLM performance: (1)\n",
"retrieve-then-generate fine-tuning, (2) retrieval-attention-probing, which uses\n",
"attention heads to filter and de-noise long contexts during decoding, and (3)\n",
"joint retrieval head training alongside the generation head. Our evaluation of\n",
"five well-known LCLMs on LOFT and ICR^2 demonstrates significant gains with our\n",
"best approach applied to Mistral-7B: +17 and +15 points by Exact Match on LOFT,\n",
"and +13 and +2 points on ICR^2, compared to vanilla RAG and supervised fine-\n",
"tuning, respectively. It even outperforms GPT-4-Turbo on most tasks despite\n",
"being a much smaller model.\n",
"Published: 2025-01-14 16:38:33\n",
"Link: http://arxiv.org/abs/2501.08248v1\n",
"PDF Link: http://arxiv.org/abs/2501.08248v1\n",
"Authors: Yifu Qiu, Varun Embar, Yizhe Zhang, Navdeep Jaitly, Shay B. Cohen, Benjamin Han\n",
"Embedding: [-0.01305697 0.01166064 0.06267344 ... -0.01627254 0.00974741\n",
" -0.00573298]\n",
"\n",
"\n",
"# ADAM-1: AI and Bioinformatics for Alzheimer's Detection and Microbiome-Clinical Data Integrations\n",
"\n",
"Abstract: The Alzheimer's Disease Analysis Model Generation 1 (ADAM) is a multi-agent\n",
"large language model (LLM) framework designed to integrate and analyze multi-\n",
"modal data, including microbiome profiles, clinical datasets, and external\n",
"knowledge bases, to enhance the understanding and detection of Alzheimer's\n",
"disease (AD). By leveraging retrieval-augmented generation (RAG) techniques\n",
"along with its multi-agent architecture, ADAM-1 synthesizes insights from\n",
"diverse data sources and contextualizes findings using literature-driven\n",
"evidence. Comparative evaluation against XGBoost revealed similar mean F1 scores\n",
"but significantly reduced variance for ADAM-1, highlighting its robustness and\n",
"consistency, particularly in small laboratory datasets. While currently tailored\n",
"for binary classification tasks, future iterations aim to incorporate additional\n",
"data modalities, such as neuroimaging and biomarkers, to broaden the scalability\n",
"and applicability for Alzheimer's research and diagnostics.\n",
"Published: 2025-01-14 18:56:33\n",
"Link: http://arxiv.org/abs/2501.08324v1\n",
"PDF Link: http://arxiv.org/abs/2501.08324v1\n",
"Authors: Ziyuan Huang, Vishaldeep Kaur Sekhon, Ouyang Guo, Mark Newman, Roozbeh Sadeghian, Maria L. Vaida, Cynthia Jo, Doyle Ward, Vanni Bucci, John P. Haran\n",
"Embedding: [ 0.03896349 0.00422515 0.05525447 ... 0.03374933 -0.01468264\n",
" 0.01850895]\n",
"\n",
"\n"
]
}
],
"source": [
"async with collection:\n",
" results = await collection.get(keys[:3])\n",
" if results:\n",
" for result in results:\n",
" print(f\"# {result.title}\")\n",
" print()\n",
" wrapped_abstract = textwrap.fill(result.abstract, width=80)\n",
" print(f\"Abstract: {wrapped_abstract}\")\n",
" print(f\"Published: {result.published}\")\n",
" print(f\"Link: {result.link}\")\n",
" print(f\"PDF Link: {result.link}\")\n",
" print(f\"Authors: {', '.join(result.authors)}\")\n",
" print(f\"Embedding: {result.abstract_vector}\")\n",
" print()\n",
" print()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `VectorStoreTextSearch` object gives us the ability to retrieve semantically similar documents directly from a prompt.\n",
"Here we search for the top 5 ArXiV abstracts in our database similar to the query about chunking strategies in RAG applications:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 5 results for query.\n",
"Advanced ingestion process powered by LLM parsing for RAG system: 0.38676463602221456\n",
"StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization: 0.39733734194342085\n",
"UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis: 0.3981809737466562\n",
"R^2AG: Incorporating Retrieval Information into Retrieval Augmented Generation: 0.4134050114864055\n",
"Enhancing Retrieval-Augmented Generation: A Study of Best Practices: 0.4144733752075731\n"
]
}
],
"source": [
"query = \"What are good chunking strategies to use for unstructured text in Retrieval-Augmented Generation applications?\"\n",
"\n",
"async with collection:\n",
" search_results = await collection.search(query, top=5, include_total_count=True)\n",
" print(f\"Found {search_results.total_count} results for query.\")\n",
" async for search_result in search_results.results:\n",
" title = search_result.record.title\n",
" score = search_result.score\n",
" print(f\"{title}: {score}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can enable chat completion to utilize the text search by creating a kernel function for searching the database..."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"kernel = Kernel()\n",
"plugin = kernel.add_functions(\n",
" plugin_name=\"arxiv_plugin\",\n",
" functions=[\n",
" collection.create_search_function(\n",
" # The default parameters match the parameters of the VectorSearchOptions class.\n",
" description=\"Searches for ArXiv papers that are related to the query.\",\n",
" parameters=[\n",
" KernelParameterMetadata(\n",
" name=\"query\", description=\"What to search for.\", type=\"str\", is_required=True, type_object=str\n",
" ),\n",
" KernelParameterMetadata(\n",
" name=\"top\",\n",
" description=\"Number of results to return.\",\n",
" type=\"int\",\n",
" default_value=2,\n",
" type_object=int,\n",
" ),\n",
" ],\n",
" ),\n",
" ],\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"...and then setting up a chat completions service that uses `FunctionChoiceBehavior.Auto` to automatically call the search function when appropriate to the users query. We also create the chat function that will be invoked by the kernel."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"# Create the chat completion service. This requires an Azure OpenAI completions model deployment and configuration.\n",
"chat_completion = AzureChatCompletion(service_id=\"completions\")\n",
"kernel.add_service(chat_completion)\n",
"\n",
"# Now we create the chat function that will use the chat service.\n",
"chat_function = kernel.add_function(\n",
" prompt=\"{{$chat_history}}{{$user_input}}\",\n",
" plugin_name=\"ChatBot\",\n",
" function_name=\"Chat\",\n",
")\n",
"\n",
"# we set the function choice to Auto, so that the LLM can choose the correct function to call.\n",
"# and we exclude the ChatBot plugin, so that it does not call itself.\n",
"execution_settings = AzureChatPromptExecutionSettings(\n",
" function_choice_behavior=FunctionChoiceBehavior.Auto(filters={\"excluded_plugins\": [\"ChatBot\"]}),\n",
" service_id=\"chat\",\n",
" max_tokens=7000,\n",
" temperature=0.7,\n",
" top_p=0.8,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we create a chat history with a system message and some initial context:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"history = ChatHistory()\n",
"system_message = \"\"\"\n",
"You are a chat bot. Your name is Archie and\n",
"you have one goal: help people find answers\n",
"to technical questions by relying on the latest\n",
"research papers published on ArXiv.\n",
"You communicate effectively in the style of a helpful librarian. \n",
"You always make sure to include the\n",
"ArXiV paper references in your responses.\n",
"If you cannot find the answer in the papers,\n",
"you will let the user know, but also provide the papers\n",
"you did find to be most relevant. If the abstract of the \n",
"paper does not specifically reference the user's inquiry,\n",
"but you believe it might be relevant, you can still include it\n",
"BUT you must make sure to mention that the paper might not directly\n",
"address the user's inquiry. Make certain that the papers you link are\n",
"from a specific search result.\n",
"\"\"\"\n",
"history.add_system_message(system_message)\n",
"history.add_user_message(\"Hi there, who are you?\")\n",
"history.add_assistant_message(\n",
" \"I am Archie, the ArXiV chat bot. I'm here to help you find the latest research papers from ArXiv that relate to your inquiries.\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can now invoke the chat function via the Kernel to get chat completions:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"arguments = KernelArguments(\n",
" user_input=query,\n",
" chat_history=history,\n",
" settings=execution_settings,\n",
")\n",
"\n",
"result = await kernel.invoke(chat_function, arguments=arguments)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Printing the result shows that the chat completion service used our text search to locate relevant ArXiV papers based on the query:"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Archie:>\n",
"What an excellent and timely question! Chunking strategies for unstructured text are\n",
"critical for optimizing Retrieval-Augmented Generation (RAG) systems since they\n",
"significantly affect how effectively a RAG model can retrieve and generate contextually\n",
"relevant information. Let me consult the latest papers on this topic from ArXiv and\n",
"provide you with relevant insights.\n",
"---\n",
"Here are some recent papers that dive into chunking strategies or similar concepts for\n",
"retrieval-augmented frameworks:\n",
"1. **\"Post-training optimization of retrieval-augmented generation models\"**\n",
" *Authors*: Vibhor Agarwal et al.\n",
" *Abstract*: While the paper discusses optimization strategies for retrieval-augmented\n",
"generation models, there is a discussion on handling unstructured text that could apply to\n",
"chunking methodologies. Chunking isn't always explicitly mentioned as \"chunking\" but may\n",
"be referred to in contexts like splitting data for retrieval.\n",
" *ArXiv link*: [arXiv:2308.10701](https://arxiv.org/abs/2308.10701)\n",
" *Note*: This paper may not focus entirely on chunking strategies but might discuss\n",
"relevant downstream considerations. It could still provide a foundation for you to explore\n",
"how chunking integrates with retrievers.\n",
"2. **\"Beyond Text: Retrieval-Augmented Reranking for Open-Domain Tasks\"**\n",
" *Authors*: Younggyo Seo et al.\n",
" *Abstract*: Although primarily focused on retrieval augmentation for reranking, there\n",
"are reflections on how document structure impacts task performance. Chunking unstructured\n",
"text to improve retrievability for such tasks could indirectly relate to this work.\n",
" *ArXiv link*: [arXiv:2310.03714](https://arxiv.org/abs/2310.03714)\n",
"3. **\"ALMA: Alignment of Generative and Retrieval Models for Long Documents\"**\n",
" *Authors*: Yao Fu et al.\n",
" *Abstract excerpt*: \"Our approach is designed to handle retrieval and generation for\n",
"long documents by aligning the retrieval and generation models more effectively.\"\n",
"Strategies to divide and process long documents into smaller chunks for efficient\n",
"alignment are explicitly discussed. A focus on handling unstructured long-form content\n",
"makes this paper highly relevant.\n",
" *ArXiv link*: [arXiv:2308.05467](https://arxiv.org/abs/2308.05467)\n",
"4. **\"Enhancing Context-aware Question Generation with Multi-modal Knowledge\"**\n",
" *Authors*: Jialong Han et al.\n",
" *Abstract excerpt*: \"Proposed techniques focus on improving retrievals through better\n",
"division of available knowledge.\" It doesn’t focus solely on text chunking in the RAG\n",
"framework but might be interesting since contextual awareness often relates to\n",
"preprocessing unstructured input into structured chunks.\n",
" *ArXiv link*: [arXiv:2307.12345](https://arxiv.org/abs/2307.12345)\n",
"---\n",
"### Practical Approaches Discussed in Literature:\n",
"From my broad understanding of RAG systems and some of the details in these papers, here\n",
"are common chunking strategies discussed in the research community:\n",
"1. **Sliding Window Approach**: Divide the text into overlapping chunks of fixed lengths\n",
"(e.g., 512 tokens with an overlap of 128 tokens). This helps ensure no important context\n",
"is left behind when chunks are created.\n",
"\n",
"2. **Semantic Chunking**: Use sentence embeddings or clustering techniques (e.g., via Bi-\n",
"Encoders or Sentence Transformers) to ensure chunks align semantically rather than naively\n",
"by token count.\n",
"3. **Dynamic Partitioning**: Implement chunking based on higher-order structure in the\n",
"text, such as splitting at sentence boundaries, paragraph breaks, or logical sections.\n",
"4. **Content-aware Chunking**: Experiment with LLMs to pre-identify contextual relevance\n",
"of different parts of the text and chunk accordingly.\n",
"---\n",
"If you'd like, I can search more specifically on a sub-part of chunking strategies or\n",
"related RAG optimizations. Let me know!\n"
]
}
],
"source": [
"def wrap_text(text, width=90):\n",
" paragraphs = text.split(\"\\n\\n\") # Split the text into paragraphs\n",
" wrapped_paragraphs = [\n",
" \"\\n\".join(textwrap.fill(part, width=width) for paragraph in paragraphs for part in paragraph.split(\"\\n\"))\n",
" ] # Wrap each paragraph, split by newlines\n",
" return \"\\n\\n\".join(wrapped_paragraphs) # Join the wrapped paragraphs back together\n",
"\n",
"\n",
"print(f\"Archie:>\\n{wrap_text(str(result))}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.15"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
python/samples/getting_started/third_party/weaviate-persistent-memory.ipynb
Vendored
+538
View File
@@ -0,0 +1,538 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Introduction\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook shows how to replace the `VolatileMemoryStore` memory storage used in a [previous notebook](./06-memory-and-embeddings.ipynb) with a `WeaviateMemoryStore`.\n",
"\n",
"`WeaviateMemoryStore` is an example of a persistent (i.e. long-term) memory store backed by the Weaviate vector database.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Configuring the Kernel\n",
"\n",
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
"\n",
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
"\n",
"#### Option 1: using OpenAI\n",
"\n",
"Add your [OpenAI Key](https://platform.openai.com/docs/overview) key to your `.env` file (org Id only if you have multiple orgs):\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
"OPENAI_API_KEY=\"sk-...\"\n",
"OPENAI_ORG_ID=\"\"\n",
"OPENAI_CHAT_MODEL_ID=\"\"\n",
"OPENAI_TEXT_MODEL_ID=\"\"\n",
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"#### Option 2: using Azure OpenAI\n",
"\n",
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
"\n",
"```\n",
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
"AZURE_OPENAI_API_KEY=\"...\"\n",
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
"AZURE_OPENAI_API_VERSION=\"...\"\n",
"```\n",
"The names should match the names used in the `.env` file, as shown above.\n",
"\n",
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# About Weaviate\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Weaviate](https://weaviate.io/) is an open-source vector database designed to scale seamlessly into billions of data objects. This implementation supports hybrid search out-of-the-box (meaning it will perform better for keyword searches).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can run Weaviate in 5 ways:\n",
"\n",
"- **SaaS** – with [Weaviate Cloud Services (WCS)](https://weaviate.io/pricing).\n",
"\n",
" WCS is a fully managed service that takes care of hosting, scaling, and updating your Weaviate instance. You can try it out for free with a sandbox that lasts for 14 days.\n",
"\n",
" To set up a SaaS Weaviate instance with WCS:\n",
"\n",
" 1. Navigate to [Weaviate Cloud Console](https://console.weaviate.cloud/).\n",
" 2. Register or sign in to your WCS account.\n",
" 3. Create a new cluster with the following settings:\n",
" - `Subscription Tier` – Free sandbox for a free trial, or contact [hello@weaviate.io](mailto:hello@weaviate.io) for other options.\n",
" - `Cluster name` – a unique name for your cluster. The name will become part of the URL used to access this instance.\n",
" - `Enable Authentication?` – Enabled by default. This will generate a static API key that you can use to authenticate.\n",
" 4. Wait for a few minutes until your cluster is ready. You will see a green tick ✔️ when it's done. Copy your cluster URL.\n",
"\n",
"- **Hybrid SaaS**\n",
"\n",
" > If you need to keep your data on-premise for security or compliance reasons, Weaviate also offers a Hybrid SaaS option: Weaviate runs within your cloud instances, but the cluster is managed remotely by Weaviate. This gives you the benefits of a managed service without sending data to an external party.\n",
"\n",
" The Weaviate Hybrid SaaS is a custom solution. If you are interested in this option, please reach out to [hello@weaviate.io](mailto:hello@weaviate.io).\n",
"\n",
"- **Self-hosted** – with a Docker container\n",
"\n",
" To set up a Weaviate instance with Docker:\n",
"\n",
" 1. [Install Docker](https://docs.docker.com/engine/install/) on your local machine if it is not already installed.\n",
" 2. [Install the Docker Compose Plugin](https://docs.docker.com/compose/install/)\n",
" 3. Download a `docker-compose.yml` file with this `curl` command:\n",
"\n",
" ```\n",
" curl -o docker-compose.yml \"https://configuration.weaviate.io/v2/docker-compose/docker-compose.yml?modules=standalone&runtime=docker-compose&weaviate_version=v1.19.6\"\n",
" ```\n",
"\n",
" Alternatively, you can use Weaviate's docker compose [configuration tool](https://weaviate.io/developers/weaviate/installation/docker-compose) to generate your own `docker-compose.yml` file.\n",
"\n",
" 4. Run `docker compose up -d` to spin up a Weaviate instance.\n",
"\n",
" > To shut it down, run `docker compose down`.\n",
"\n",
"- **Self-hosted** – with a Kubernetes cluster\n",
"\n",
" To configure a self-hosted instance with Kubernetes, follow Weaviate's [documentation](https://weaviate.io/developers/weaviate/installation/kubernetes).|\n",
"\n",
"- **Embedded** - start a weaviate instance right from your application code using the client library\n",
"\n",
" This code snippet shows how to instantiate an embedded weaviate instance and upload a document:\n",
"\n",
" ```python\n",
" import weaviate\n",
" from weaviate.embedded import EmbeddedOptions\n",
"\n",
" client = weaviate.Client(\n",
" embedded_options=EmbeddedOptions()\n",
" )\n",
"\n",
" data_obj = {\n",
" \"name\": \"Chardonnay\",\n",
" \"description\": \"Goes with fish\"\n",
" }\n",
"\n",
" client.data_object.create(data_obj, \"Wine\")\n",
" ```\n",
"\n",
" Refer to the [documentation](https://weaviate.io/developers/weaviate/installation/embedded) for more details about this deployment method.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Setup\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Note: if using a virtual environment, do not run this cell\n",
"%pip install -U semantic-kernel[weaviate]\n",
"from semantic_kernel import __version__\n",
"\n",
"__version__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## OS-specific notes:\n",
"\n",
"- if you run into SSL errors when connecting to OpenAI on macOS, see this issue for a [potential solution](https://github.com/microsoft/semantic-kernel/issues/627#issuecomment-1580912248)\n",
"- on Windows, you may need to run Docker Desktop as administrator\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, we instantiate the Weaviate memory store. Uncomment ONE of the options below, depending on how you want to use Weaviate:\n",
"\n",
"- from a Docker instance\n",
"- from WCS\n",
"- directly from the client (embedded Weaviate), which works on Linux only at the moment\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.memory.weaviate import WeaviateMemoryStore\n",
"\n",
"# Note the Weaviate Config values need to be either configured as environment variables\n",
"# or in the .env file, as a back up. When creating the instance of the `weaviate_memory_store`\n",
"# pass in `env_file_path=<path_to_file>` to read the config values from the `.env` file, otherwise\n",
"# the values will be read from environment variables.\n",
"# Env variables or .env file config should look like:\n",
"# WEAVIATE_URL=\"http://localhost:8080\"\n",
"# WEAVIATE_API_KEY=\"\"\n",
"# WEAVIATE_USE_EMBED=True|False\n",
"\n",
"store = WeaviateMemoryStore()\n",
"store.client.schema.delete_all()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then, we register the memory store to the kernel:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from services import Service\n",
"\n",
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
"selectedService = Service.OpenAI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAITextEmbedding\n",
"from semantic_kernel.core_plugins.text_memory_plugin import TextMemoryPlugin\n",
"from semantic_kernel.kernel import Kernel\n",
"from semantic_kernel.memory.semantic_text_memory import SemanticTextMemory\n",
"from semantic_kernel.memory.volatile_memory_store import VolatileMemoryStore\n",
"\n",
"kernel = Kernel()\n",
"\n",
"chat_service_id = \"chat\"\n",
"if selectedService == Service.OpenAI:\n",
" oai_chat_service = OpenAIChatCompletion(\n",
" service_id=chat_service_id,\n",
" ai_model_id=\"gpt-3.5-turbo\",\n",
" )\n",
" embedding_gen = OpenAITextEmbedding(ai_model_id=\"text-embedding-ada-002\")\n",
" kernel.add_service(oai_chat_service)\n",
" kernel.add_service(embedding_gen)\n",
"\n",
"memory = SemanticTextMemory(storage=VolatileMemoryStore(), embeddings_generator=embedding_gen)\n",
"kernel.add_plugin(TextMemoryPlugin(memory), \"TextMemoryPlugin\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Manually adding memories\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's create some initial memories \"About Me\". We can add memories to our weaviate memory store by using `save_information`\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"collection_id = \"generic\"\n",
"\n",
"\n",
"async def populate_memory(memory: SemanticTextMemory) -> None:\n",
" # Add some documents to the semantic memory\n",
" await memory.save_information(collection=collection_id, id=\"info1\", text=\"Your budget for 2024 is $100,000\")\n",
" await memory.save_information(collection=collection_id, id=\"info2\", text=\"Your savings from 2023 are $50,000\")\n",
" await memory.save_information(collection=collection_id, id=\"info3\", text=\"Your investments are $80,000\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"await populate_memory(memory)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Searching is done through `search`:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"async def search_memory_examples(memory: SemanticTextMemory) -> None:\n",
" questions = [\"What is my budget for 2024?\", \"What are my savings from 2023?\", \"What are my investments?\"]\n",
"\n",
" for question in questions:\n",
" print(f\"Question: {question}\")\n",
" result = await memory.search(collection_id, question)\n",
" print(f\"Answer: {result[0].text}\\n\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"await search_memory_examples(memory)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here's how to use the weaviate memory store in a chat application:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from semantic_kernel.functions.kernel_function import KernelFunction\n",
"from semantic_kernel.prompt_template import PromptTemplateConfig\n",
"\n",
"\n",
"async def setup_chat_with_memory(\n",
" kernel: Kernel,\n",
" service_id: str,\n",
") -> KernelFunction:\n",
" prompt = \"\"\"\n",
" ChatBot can have a conversation with you about any topic.\n",
" It can give explicit instructions or say 'I don't know' if\n",
" it does not have an answer.\n",
"\n",
" Information about me, from previous conversations:\n",
" - {{recall 'budget by year'}} What is my budget for 2024?\n",
" - {{recall 'savings from previous year'}} What are my savings from 2023?\n",
" - {{recall 'investments'}} What are my investments?\n",
"\n",
" {{$request}}\n",
" \"\"\".strip()\n",
"\n",
" prompt_template_config = PromptTemplateConfig(\n",
" template=prompt,\n",
" execution_settings={\n",
" service_id: kernel.get_service(service_id).get_prompt_execution_settings_class()(service_id=service_id)\n",
" },\n",
" )\n",
"\n",
" return kernel.add_function(\n",
" function_name=\"chat_with_memory\",\n",
" plugin_name=\"TextMemoryPlugin\",\n",
" prompt_template_config=prompt_template_config,\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"async def chat(kernel: Kernel, chat_func: KernelFunction) -> bool:\n",
" try:\n",
" user_input = input(\"User:> \")\n",
" except KeyboardInterrupt:\n",
" print(\"\\n\\nExiting chat...\")\n",
" return False\n",
" except EOFError:\n",
" print(\"\\n\\nExiting chat...\")\n",
" return False\n",
"\n",
" if user_input == \"exit\":\n",
" print(\"\\n\\nExiting chat...\")\n",
" return False\n",
"\n",
" answer = await kernel.invoke(chat_func, request=user_input)\n",
"\n",
" print(f\"ChatBot:> {answer}\")\n",
" return True"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Populating memory...\")\n",
"await populate_memory(memory)\n",
"\n",
"print(\"Asking questions... (manually)\")\n",
"await search_memory_examples(memory)\n",
"\n",
"print(\"Setting up a chat (with memory!)\")\n",
"chat_func = await setup_chat_with_memory(kernel, chat_service_id)\n",
"\n",
"print(\"Begin chatting (type 'exit' to exit):\\n\")\n",
"print(\n",
" \"Welcome to the chat bot!\\\n",
" \\n Type 'exit' to exit.\\\n",
" \\n Try asking a question about your finances (i.e. \\\"talk to me about my finances\\\").\"\n",
")\n",
"chatting = True\n",
"while chatting:\n",
" chatting = await chat(kernel, chat_func)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Adding documents to your memory\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a dictionary to hold some files. The key is the hyperlink to the file and the value is the file's content:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"github_files = {}\n",
"github_files[\"https://github.com/microsoft/semantic-kernel/blob/main/README.md\"] = (\n",
" \"README: Installation, getting started, and how to contribute\"\n",
")\n",
"github_files[\n",
" \"https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/02-running-prompts-from-file.ipynb\"\n",
"] = \"Jupyter notebook describing how to pass prompts from a file to a semantic plugin or function\"\n",
"github_files[\"https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/00-getting-started.ipynb\"] = (\n",
" \"Jupyter notebook describing how to get started with the Semantic Kernel\"\n",
")\n",
"github_files[\"https://github.com/microsoft/semantic-kernel/tree/main/samples/plugins/ChatPlugin/ChatGPT\"] = (\n",
" \"Sample demonstrating how to create a chat plugin interfacing with ChatGPT\"\n",
")\n",
"github_files[\n",
" \"https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/Memory/Volatile/VolatileMemoryStore.cs\"\n",
"] = \"C# class that defines a volatile embedding store\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use `save_reference` to save the file:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"COLLECTION = \"SKGitHub\"\n",
"\n",
"print(\"Adding some GitHub file URLs and their descriptions to a volatile Semantic Memory.\")\n",
"for index, (entry, value) in enumerate(github_files.items()):\n",
" await memory.save_reference(\n",
" collection=COLLECTION,\n",
" description=value,\n",
" text=value,\n",
" external_id=entry,\n",
" external_source_name=\"GitHub\",\n",
" )\n",
" print(\" URL {} saved\".format(index))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use `search` to ask a question:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ask = \"I love Jupyter notebooks, how should I get started?\"\n",
"print(\"===========================\\n\" + \"Query: \" + ask + \"\\n\")\n",
"\n",
"memories = await memory.search(COLLECTION, ask, limit=5, min_relevance_score=0.77)\n",
"\n",
"for index, memory in enumerate(memories):\n",
" print(f\"Result {index}:\")\n",
" print(\" URL: : \" + memory.id)\n",
" print(\" Title : \" + memory.description)\n",
" print(\" Relevance: \" + str(memory.relevance))\n",
" print()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.13"
}
},
"nbformat": 4,
"nbformat_minor": 2
}