wehub-resource-sync
434 lines
13 KiB
Plaintext
434 lines
13 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "68e1c158",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Streaming Results\n",
|
|
"\n",
|
|
"Here is an example pattern if you want to stream your multiple results. Note that this is not supported for Hugging Face text completions at this time.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a3dd8590",
|
|
"metadata": {},
|
|
"source": [
|
|
"Import Semantic Kernel SDK from pypi.org"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a77bdf89",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Note: if using a virtual environment, do not run this cell\n",
|
|
"%pip install -U semantic-kernel\n",
|
|
"from semantic_kernel import __version__\n",
|
|
"\n",
|
|
"__version__"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fd94029f",
|
|
"metadata": {},
|
|
"source": [
|
|
"Initial configuration for the notebook to run properly."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "7547e59b",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Make sure paths are correct for the imports\n",
|
|
"\n",
|
|
"import os\n",
|
|
"import sys\n",
|
|
"\n",
|
|
"notebook_dir = os.path.abspath(\"\")\n",
|
|
"parent_dir = os.path.dirname(notebook_dir)\n",
|
|
"grandparent_dir = os.path.dirname(parent_dir)\n",
|
|
"\n",
|
|
"\n",
|
|
"sys.path.append(grandparent_dir)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "73ba03ae",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Configuring the Kernel\n",
|
|
"\n",
|
|
"Let's get started with the necessary configuration to run Semantic Kernel. For Notebooks, we require a `.env` file with the proper settings for the model you use. Create a new file named `.env` and place it in this directory. Copy the contents of the `.env.example` file from this directory and paste it into the `.env` file that you just created.\n",
|
|
"\n",
|
|
"**NOTE: Please make sure to include `GLOBAL_LLM_SERVICE` set to either OpenAI, AzureOpenAI, or HuggingFace in your .env file. If this setting is not included, the Service will default to AzureOpenAI.**\n",
|
|
"\n",
|
|
"#### Option 1: using OpenAI\n",
|
|
"\n",
|
|
"Add your [OpenAI Key](https://openai.com/product/) key to your `.env` file (org Id only if you have multiple orgs):\n",
|
|
"\n",
|
|
"```\n",
|
|
"GLOBAL_LLM_SERVICE=\"OpenAI\"\n",
|
|
"OPENAI_API_KEY=\"sk-...\"\n",
|
|
"OPENAI_ORG_ID=\"\"\n",
|
|
"OPENAI_CHAT_MODEL_ID=\"\"\n",
|
|
"OPENAI_TEXT_MODEL_ID=\"\"\n",
|
|
"OPENAI_EMBEDDING_MODEL_ID=\"\"\n",
|
|
"```\n",
|
|
"The names should match the names used in the `.env` file, as shown above.\n",
|
|
"\n",
|
|
"#### Option 2: using Azure OpenAI\n",
|
|
"\n",
|
|
"Add your [Azure Open AI Service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=programming-language-studio) settings to the `.env` file in the same folder:\n",
|
|
"\n",
|
|
"```\n",
|
|
"GLOBAL_LLM_SERVICE=\"AzureOpenAI\"\n",
|
|
"AZURE_OPENAI_API_KEY=\"...\"\n",
|
|
"AZURE_OPENAI_ENDPOINT=\"https://...\"\n",
|
|
"AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=\"...\"\n",
|
|
"AZURE_OPENAI_TEXT_DEPLOYMENT_NAME=\"...\"\n",
|
|
"AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=\"...\"\n",
|
|
"AZURE_OPENAI_API_VERSION=\"...\"\n",
|
|
"```\n",
|
|
"The names should match the names used in the `.env` file, as shown above.\n",
|
|
"\n",
|
|
"As alternative to `AZURE_OPENAI_API_KEY`, it's possible to authenticate using `credential` parameter, more information here: [Azure Identity](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme).\n",
|
|
"\n",
|
|
"In the following example, `AzureCliCredential` is used. To authenticate using Azure CLI:\n",
|
|
"\n",
|
|
"1. Install [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).\n",
|
|
"2. Run `az login` command in terminal and follow the authentication steps.\n",
|
|
"\n",
|
|
"For more advanced configuration, please follow the steps outlined in the [setup guide](./CONFIGURING_THE_KERNEL.md)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fd931c14",
|
|
"metadata": {},
|
|
"source": [
|
|
"We will load our settings and get the LLM service to use for the notebook."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a9a5c87a",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from services import Service\n",
|
|
"\n",
|
|
"# Select a service to use for this notebook (available services: OpenAI, AzureOpenAI, HuggingFace)\n",
|
|
"selectedService = Service.OpenAI\n",
|
|
"print(f\"Using service type: {selectedService}\")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "d8ddffc1",
|
|
"metadata": {},
|
|
"source": [
|
|
"First, we will set up the text and chat services we will be submitting prompts to.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "8f8dcbc6",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from semantic_kernel import Kernel\n",
|
|
"from semantic_kernel.connectors.ai.open_ai import (\n",
|
|
" AzureChatCompletion,\n",
|
|
" AzureChatPromptExecutionSettings, # noqa: F401\n",
|
|
" AzureTextCompletion,\n",
|
|
" OpenAIChatCompletion,\n",
|
|
" OpenAIChatPromptExecutionSettings, # noqa: F401\n",
|
|
" OpenAITextCompletion,\n",
|
|
" OpenAITextPromptExecutionSettings, # noqa: F401\n",
|
|
")\n",
|
|
"from semantic_kernel.contents import ChatHistory # noqa: F401\n",
|
|
"\n",
|
|
"kernel = Kernel()\n",
|
|
"\n",
|
|
"service_id = None\n",
|
|
"if selectedService == Service.OpenAI:\n",
|
|
" from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion\n",
|
|
"\n",
|
|
" service_id = \"default\"\n",
|
|
" oai_chat_service = OpenAIChatCompletion(\n",
|
|
" service_id=\"oai_chat\",\n",
|
|
" )\n",
|
|
" oai_text_service = OpenAITextCompletion(\n",
|
|
" service_id=\"oai_text\",\n",
|
|
" )\n",
|
|
"elif selectedService == Service.AzureOpenAI:\n",
|
|
" from azure.identity import AzureCliCredential\n",
|
|
"\n",
|
|
" from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
|
|
"\n",
|
|
" credential = AzureCliCredential()\n",
|
|
" service_id = \"default\"\n",
|
|
" aoai_chat_service = AzureChatCompletion(service_id=\"aoai_chat\", credential=credential)\n",
|
|
" aoai_text_service = AzureTextCompletion(service_id=\"aoai_text\", credential=credential)\n",
|
|
"\n",
|
|
"# Configure Hugging Face service\n",
|
|
"if selectedService == Service.HuggingFace:\n",
|
|
" from semantic_kernel.connectors.ai.hugging_face import (\n",
|
|
" HuggingFacePromptExecutionSettings, # noqa: F401\n",
|
|
" HuggingFaceTextCompletion,\n",
|
|
" )\n",
|
|
"\n",
|
|
" hf_text_service = HuggingFaceTextCompletion(ai_model_id=\"distilgpt2\", task=\"text-generation\")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "50561d82",
|
|
"metadata": {},
|
|
"source": [
|
|
"Next, we'll set up the completion request settings for text completion services.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "628c843e",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"oai_prompt_execution_settings = OpenAITextPromptExecutionSettings(\n",
|
|
" service_id=\"oai_text\",\n",
|
|
" max_tokens=150,\n",
|
|
" temperature=0.7,\n",
|
|
" top_p=1,\n",
|
|
" frequency_penalty=0.5,\n",
|
|
" presence_penalty=0.5,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "857a9c89",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Streaming Open AI Text Completion\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e2979db8",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if selectedService == Service.OpenAI:\n",
|
|
" prompt = \"What is the purpose of a rubber duck?\"\n",
|
|
" stream = oai_text_service.get_streaming_text_contents(prompt=prompt, settings=oai_prompt_execution_settings)\n",
|
|
" async for message in stream:\n",
|
|
" print(str(message[0]), end=\"\") # end = \"\" to avoid newlines"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "4288d09f",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Streaming Azure Open AI Text Completion\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "5319f14d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if selectedService == Service.AzureOpenAI:\n",
|
|
" prompt = \"provide me a list of possible meanings for the acronym 'ORLD'\"\n",
|
|
" stream = aoai_text_service.get_streaming_text_contents(prompt=prompt, settings=oai_prompt_execution_settings)\n",
|
|
" async for message in stream:\n",
|
|
" print(str(message[0]), end=\"\")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "eb548f9c",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Streaming Hugging Face Text Completion\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "be7b1c2e",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if selectedService == Service.HuggingFace:\n",
|
|
" hf_prompt_execution_settings = HuggingFacePromptExecutionSettings(\n",
|
|
" service_id=\"hf_text\",\n",
|
|
" extension_data={\n",
|
|
" \"max_new_tokens\": 80,\n",
|
|
" \"top_p\": 1,\n",
|
|
" \"eos_token_id\": 11,\n",
|
|
" \"pad_token_id\": 0,\n",
|
|
" },\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9525e4f3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if selectedService == Service.HuggingFace:\n",
|
|
" prompt = \"The purpose of a rubber duck is\"\n",
|
|
" stream = hf_text_service.get_streaming_text_contents(\n",
|
|
" prompt=prompt, prompt_execution_settings=hf_prompt_execution_settings\n",
|
|
" )\n",
|
|
" async for text in stream:\n",
|
|
" print(str(text[0]), end=\"\") # end = \"\" to avoid newlines"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "da632e12",
|
|
"metadata": {},
|
|
"source": [
|
|
"Here, we're setting up the settings for Chat completions.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e5f11e46",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"oai_chat_prompt_execution_settings = OpenAIChatPromptExecutionSettings(\n",
|
|
" service_id=\"oai_chat\",\n",
|
|
" max_tokens=150,\n",
|
|
" temperature=0.7,\n",
|
|
" top_p=1,\n",
|
|
" frequency_penalty=0.5,\n",
|
|
" presence_penalty=0.5,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "d6bf238e",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Streaming OpenAI Chat Completion\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "dabc6a4c",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if selectedService == Service.OpenAI:\n",
|
|
" content = \"You are an AI assistant that helps people find information.\"\n",
|
|
" chat = ChatHistory()\n",
|
|
" chat.add_system_message(content)\n",
|
|
" stream = oai_chat_service.get_streaming_chat_message_contents(\n",
|
|
" chat_history=chat, settings=oai_chat_prompt_execution_settings\n",
|
|
" )\n",
|
|
" async for text in stream:\n",
|
|
" print(str(text[0]), end=\"\") # end = \"\" to avoid newlines"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "cdb8f740",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Streaming Azure OpenAI Chat Completion\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "da1e9f59",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"az_oai_chat_prompt_execution_settings = AzureChatPromptExecutionSettings(\n",
|
|
" service_id=\"aoai_chat\",\n",
|
|
" max_tokens=150,\n",
|
|
" temperature=0.7,\n",
|
|
" top_p=1,\n",
|
|
" frequency_penalty=0.5,\n",
|
|
" presence_penalty=0.5,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b74a64a9",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"if selectedService == Service.AzureOpenAI:\n",
|
|
" content = \"You are an AI assistant that helps people find information.\"\n",
|
|
" chat = ChatHistory()\n",
|
|
" chat.add_system_message(content)\n",
|
|
" chat.add_user_message(\"What is the purpose of a rubber duck?\")\n",
|
|
" stream = aoai_chat_service.get_streaming_chat_message_contents(\n",
|
|
" chat_history=chat, settings=az_oai_chat_prompt_execution_settings\n",
|
|
" )\n",
|
|
" async for text in stream:\n",
|
|
" print(str(text[0]), end=\"\") # end = \"\" to avoid newlines"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.12.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|