# Semantic Kernel Concepts by Feature

## Table of Contents

### Agents - Creating and using [agents](../../semantic_kernel/agents/) in Semantic Kernel

#### [Azure AI Agent](../../semantic_kernel/agents/azure_ai/azure_ai_agent.py)

- [Azure AI Agent as Kernel Function](./agents/azure_ai_agent/azure_ai_agent_as_kernel_function.py)
- [Azure AI Agent with Auto Function Invocation Filter Streaming](./agents/azure_ai_agent/azure_ai_agent_auto_func_invocation_filter_streaming.py)
- [Azure AI Agent with Auto Function Invocation Filter](./agents/azure_ai_agent/azure_ai_agent_auto_func_invocation_filter.py)
- [Azure AI Agent with Azure AI Search](./agents/azure_ai_agent/azure_ai_agent_azure_ai_search.py)
- [Azure AI Agent with Bing Grounding Streaming with Message Callback](./agents/azure_ai_agent/azure_ai_agent_bing_grounding_streaming_with_message_callback.py)
- [Azure AI Agent with Bing Grounding](./agents/azure_ai_agent/azure_ai_agent_bing_grounding.py)
- [Azure AI Agent with Code Interpreter Streaming with Message Callback](./agents/azure_ai_agent/azure_ai_agent_code_interpreter_streaming_with_message_callback.py)
- [Azure AI Agent Declarative with Azure AI Search](./agents/azure_ai_agent/azure_ai_agent_declarative_azure_ai_search.py)
- [Azure AI Agent Declarative with Bing Grounding](./agents/azure_ai_agent/azure_ai_agent_declarative_bing_grounding.py)
- [Azure AI Agent Declarative with Code Interpreter](./agents/azure_ai_agent/azure_ai_agent_declarative_code_interpreter.py)
- [Azure AI Agent Declarative with File Search](./agents/azure_ai_agent/azure_ai_agent_declarative_file_search.py)
- [Azure AI Agent Declarative with Function Calling From File](./agents/azure_ai_agent/azure_ai_agent_declarative_function_calling_from_file.py)
- [Azure AI Agent Declarative with OpenAPI Interpreter](./agents/azure_ai_agent/azure_ai_agent_declarative_openapi.py)
- [Azure AI Agent Declarative with Existing Agent ID](./agents/azure_ai_agent/azure_ai_agent_declarative_with_existing_agent_id.py)
- [Azure AI Agent File Manipulation](./agents/azure_ai_agent/azure_ai_agent_file_manipulation.py)
- [Azure AI Agent MCP Streaming](./agents/azure_ai_agent/azure_ai_agent_mcp_streaming.py)
- [Azure AI Agent Prompt Templating](./agents/azure_ai_agent/azure_ai_agent_prompt_templating.py)
- [Azure AI Agent Message Callback Streaming](./agents/azure_ai_agent/azure_ai_agent_message_callback_streaming.py)
- [Azure AI Agent Message Callback](./agents/azure_ai_agent/azure_ai_agent_message_callback.py)
- [Azure AI Agent Retrieve Messages from Thread](./agents/azure_ai_agent/azure_ai_agent_retrieve_messages_from_thread.py)
- [Azure AI Agent Streaming](./agents/azure_ai_agent/azure_ai_agent_streaming.py)
- [Azure AI Agent Structured Outputs](./agents/azure_ai_agent/azure_ai_agent_structured_outputs.py)
- [Azure AI Agent Truncation Strategy](./agents/azure_ai_agent/azure_ai_agent_truncation_strategy.py)

#### [Bedrock Agent](../../semantic_kernel/agents/bedrock/bedrock_agent.py)

- [Bedrock Agent Simple Chat Streaming](./agents/bedrock_agent/bedrock_agent_simple_chat_streaming.py)
- [Bedrock Agent Simple Chat](./agents/bedrock_agent/bedrock_agent_simple_chat.py)
- [Bedrock Agent With Code Interpreter Streaming](./agents/bedrock_agent/bedrock_agent_with_code_interpreter_streaming.py)
- [Bedrock Agent With Code Interpreter](./agents/bedrock_agent/bedrock_agent_with_code_interpreter.py)
- [Bedrock Agent With Kernel Function Simple](./agents/bedrock_agent/bedrock_agent_with_kernel_function_simple.py)
- [Bedrock Agent With Kernel Function Streaming](./agents/bedrock_agent/bedrock_agent_with_kernel_function_streaming.py)
- [Bedrock Agent With Kernel Function](./agents/bedrock_agent/bedrock_agent_with_kernel_function.py)
- [Bedrock Agent Mixed Chat Agents Streaming](./agents/bedrock_agent/bedrock_mixed_chat_agents_streaming.py)
- [Bedrock Agent Mixed Chat Agents](./agents/bedrock_agent/bedrock_mixed_chat_agents.py)

#### [Chat Completion Agent](../../semantic_kernel/agents/chat_completion/chat_completion_agent.py)

- [Chat Completion Agent as Kernel Function](./agents/chat_completion_agent/chat_completion_agent_as_kernel_function.py)
- [Chat Completion Agent Function Termination](./agents/chat_completion_agent/chat_completion_agent_function_termination.py)
- [Chat Completion Agent Message Callback Streaming](./agents/chat_completion_agent/chat_completion_agent_message_callback_streaming.py)
- [Chat Completion Agent Message Callback](./agents/chat_completion_agent/chat_completion_agent_message_callback.py)
- [Chat Completion Agent Templating](./agents/chat_completion_agent/chat_completion_agent_prompt_templating.py)
- [Chat Completion Agent Streaming Token Usage](./agents/chat_completion_agent/chat_completion_agent_streaming_token_usage.py)
- [Chat Completion Agent Summary History Reducer Agent Chat](./agents/chat_completion_agent/chat_completion_agent_summary_history_reducer_agent_chat.py)
- [Chat Completion Agent Summary History Reducer Single Agent](./agents/chat_completion_agent/chat_completion_agent_summary_history_reducer_single_agent.py)
- [Chat Completion Agent Token Usage](./agents/chat_completion_agent/chat_completion_agent_token_usage.py)
- [Chat Completion Agent Truncate History Reducer Agent Chat](./agents/chat_completion_agent/chat_completion_agent_truncate_history_reducer_agent_chat.py)
- [Chat Completion Agent Truncate History Reducer Single Agent](./agents/chat_completion_agent/chat_completion_agent_truncate_history_reducer_single_agent.py)

#### [Mixed Agent Group Chat](../../semantic_kernel/agents/group_chat/agent_group_chat.py)

- [Mixed Chat Agents Plugins](./agents/mixed_chat/mixed_chat_agents_plugins.py)
- [Mixed Chat Agents](./agents/mixed_chat/mixed_chat_agents.py)
- [Mixed Chat Files](./agents/mixed_chat/mixed_chat_files.py)
- [Mixed Chat Images](./agents/mixed_chat/mixed_chat_images.py)
- [Mixed Chat Reset](./agents/mixed_chat/mixed_chat_reset.py)
- [Mixed Chat Streaming](./agents/mixed_chat/mixed_chat_streaming.py)

#### [OpenAI Assistant Agent](../../semantic_kernel/agents/open_ai/openai_assistant_agent.py)

- [Azure OpenAI Assistant Declarative Code Interpreter](./agents/openai_assistant/azure_openai_assistant_declarative_code_interpreter.py)
- [Azure OpenAI Assistant Declarative File Search](./agents/openai_assistant/azure_openai_assistant_declarative_file_search.py)
- [Azure OpenAI Assistant Declarative Function Calling From File](./agents/openai_assistant/azure_openai_assistant_declarative_function_calling_from_file.py)
- [Azure OpenAI Assistant Declarative Templating](./agents/openai_assistant/azure_openai_assistant_declarative_templating.py)
- [Azure OpenAI Assistant Declarative With Existing Agent ID](./agents/openai_assistant/azure_openai_assistant_declarative_with_existing_agent_id.py)
- [OpenAI Assistant Auto Function Invocation Filter Streaming](./agents/openai_assistant/openai_assistant_auto_func_invocation_filter_streaming.py)
- [OpenAI Assistant Auto Function Invocation Filter](./agents/openai_assistant/openai_assistant_auto_func_invocation_filter.py)
- [OpenAI Assistant Chart Maker Streaming](./agents/openai_assistant/openai_assistant_chart_maker_streaming.py)
- [OpenAI Assistant Chart Maker](./agents/openai_assistant/openai_assistant_chart_maker.py)
- [OpenAI Assistant Declarative Code Interpreter](./agents/openai_assistant/openai_assistant_declarative_code_interpreter.py)
- [OpenAI Assistant Declarative File Search](./agents/openai_assistant/openai_assistant_declarative_file_search.py)
- [OpenAI Assistant Declarative Function Calling From File](./agents/openai_assistant/openai_assistant_declarative_function_calling_from_file.py)
- [OpenAI Assistant Declarative Templating](./agents/openai_assistant/openai_assistant_declarative_templating.py)
- [OpenAI Assistant Declarative With Existing Agent ID](./agents/openai_assistant/openai_assistant_declarative_with_existing_agent_id.py)
- [OpenAI Assistant File Manipulation Streaming](./agents/openai_assistant/openai_assistant_file_manipulation_streaming.py)
- [OpenAI Assistant File Manipulation](./agents/openai_assistant/openai_assistant_file_manipulation.py)
- [OpenAI Assistant Retrieval](./agents/openai_assistant/openai_assistant_retrieval.py)
- [OpenAI Assistant Message Callback Streaming](./agents/openai_assistant/openai_assistant_message_callback_streaming.py)
- [OpenAI Assistant Message Callback](./agents/openai_assistant/openai_assistant_message_callback.py)
- [OpenAI Assistant Streaming](./agents/openai_assistant/openai_assistant_streaming.py)
- [OpenAI Assistant Structured Outputs](./agents/openai_assistant/openai_assistant_structured_outputs.py)
- [OpenAI Assistant Templating Streaming](./agents/openai_assistant/openai_assistant_templating_streaming.py)
- [OpenAI Assistant Vision Streaming](./agents/openai_assistant/openai_assistant_vision_streaming.py)

#### [OpenAI Responses Agent](../../semantic_kernel/agents/open_ai/openai_responses_agent.py)

- [Azure OpenAI Responses Agent Declarative File Search](./agents/openai_responses/azure_openai_responses_agent_declarative_file_search.py)
- [Azure OpenAI Responses Agent Declarative Function Calling From File](./agents/openai_responses/azure_openai_responses_agent_declarative_function_calling_from_file.py)
- [Azure OpenAI Responses Agent Declarative Templating](./agents/openai_responses/azure_openai_responses_agent_declarative_templating.py)
- [OpenAI Responses Agent Declarative File Search](./agents/openai_responses/openai_responses_agent_declarative_file_search.py)
- [OpenAI Responses Agent Declarative Function Calling From File](./agents/openai_responses/openai_responses_agent_declarative_function_calling_from_file.py)
- [OpenAI Responses Agent Declarative Web Search](./agents/openai_responses/openai_responses_agent_declarative_web_search.py)
- [OpenAI Responses Binary Content Upload](./agents/openai_responses/responses_agent_binary_content_upload.py)
- [OpenAI Responses Message Callback Streaming](./agents/openai_responses/responses_agent_message_callback_streaming.py)
- [OpenAI Responses Message Callback](./agents/openai_responses/responses_agent_message_callback.py)
- [OpenAI Responses File Search Streaming](./agents/openai_responses/responses_agent_file_search_streaming.py)
- [OpenAI Responses Plugins Streaming](./agents/openai_responses/responses_agent_plugins_streaming.py)
- [OpenAI Responses Reuse Existing Thread ID](./agents/openai_responses/responses_agent_reuse_existing_thread_id.py)
- [OpenAI Responses Web Search Streaming](./agents/openai_responses/responses_agent_web_search_streaming.py)

### Audio - Using services that support audio-to-text and text-to-audio conversion

- [Chat with Audio Input](./audio/01-chat_with_audio_input.py)
- [Chat with Audio Output](./audio/02-chat_with_audio_output.py)
- [Chat with Audio Input and Output](./audio/03-chat_with_audio_input_output.py)
- [Audio Player](./audio/audio_player.py)
- [Audio Recorder](./audio/audio_recorder.py)

### AutoFunctionCalling - Using `Auto Function Calling` to allow function call capable models to invoke Kernel Functions automatically

- [Azure Python Code Interpreter Function Calling](./auto_function_calling/azure_python_code_interpreter_function_calling.py)
- [Function Calling with Required Type](./auto_function_calling/function_calling_with_required_type.py)
- [Parallel Function Calling](./auto_function_calling/parallel_function_calling.py)
- [Chat Completion with Auto Function Calling Streaming](./auto_function_calling/chat_completion_with_auto_function_calling_streaming.py)
- [Functions Defined in JSON Prompt](./auto_function_calling/functions_defined_in_json_prompt.py)
- [Chat Completion with Manual Function Calling Streaming](./auto_function_calling/chat_completion_with_manual_function_calling_streaming.py)
- [Functions Defined in YAML Prompt](./auto_function_calling/functions_defined_in_yaml_prompt.py)
- [Chat Completion with Auto Function Calling](./auto_function_calling/chat_completion_with_auto_function_calling.py)
- [Chat Completion with Manual Function Calling](./auto_function_calling/chat_completion_with_manual_function_calling.py)
- [Nexus Raven](./auto_function_calling/nexus_raven.py)

### ChatCompletion - Using [`ChatCompletion`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/connectors/ai/chat_completion_client_base.py) messaging capable service with models

- [Simple Chatbot](./chat_completion/simple_chatbot.py)
- [Simple Chatbot Kernel Function](./chat_completion/simple_chatbot_kernel_function.py)
- [Simple Chatbot Logit Bias](./chat_completion/simple_chatbot_logit_bias.py)
- [Simple Chatbot Store Metadata](./chat_completion/simple_chatbot_store_metadata.py)
- [Simple Chatbot Streaming](./chat_completion/simple_chatbot_streaming.py)
- [Simple Chatbot with Image](./chat_completion/simple_chatbot_with_image.py)
- [Simple Chatbot with Summary History Reducer Keeping Function Content](./chat_completion/simple_chatbot_with_summary_history_reducer_keep_func_content.py)
- [Simple Chatbot with Summary History Reducer](./chat_completion/simple_chatbot_with_summary_history_reducer.py)
- [Simple Chatbot with Truncation History Reducer](./chat_completion/simple_chatbot_with_truncation_history_reducer.py)
- [Simple Chatbot with Summary History Reducer using Auto Reduce](./chat_completion/simple_chatbot_with_summary_history_reducer_autoreduce.py)
- [Simple Chatbot with Truncation History Reducer using Auto Reduce](./chat_completion/simple_chatbot_with_truncation_history_reducer_autoreduce.py)

### ChatHistory - Using and serializing the [`ChatHistory`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/contents/chat_history.py)

- [Serialize Chat History](./chat_history/serialize_chat_history.py)
- [Store Chat History in CosmosDB](./chat_history/store_chat_history_in_cosmosdb.py)

### Filtering - Creating and using Filters

- [Auto Function Invoke Filters](./filtering/auto_function_invoke_filters.py)
- [Function Invocation Filters](./filtering/function_invocation_filters.py)
- [Function Invocation Filters Stream](./filtering/function_invocation_filters_stream.py)
- [Prompt Filters](./filtering/prompt_filters.py)
- [Retry with Filters](./filtering/retry_with_filters.py)

### Functions - Invoking [`Method`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/functions/kernel_function_from_method.py) or [`Prompt`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/functions/kernel_function_from_prompt.py) functions with [`Kernel`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/kernel.py)

- [Kernel Arguments](./functions/kernel_arguments.py)

### Grounding - An example of how to perform LLM grounding

- [Grounded](./grounding/grounded.py)

### Local Models - Using the [`OpenAI connector`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion.py) and [`OnnxGenAI connector`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/connectors/ai/onnx/services/onnx_gen_ai_chat_completion.py) to talk to models hosted locally in Ollama, OnnxGenAI, and LM Studio

- [ONNX Chat Completion](./local_models/onnx_chat_completion.py)
- [LM Studio Text Embedding](./local_models/lm_studio_text_embedding.py)
- [LM Studio Chat Completion](./local_models/lm_studio_chat_completion.py)
- [ONNX Phi3 Vision Completion](./local_models/onnx_phi3_vision_completion.py)
- [Ollama Chat Completion](./local_models/ollama_chat_completion.py)
- [ONNX Text Completion](./local_models/onnx_text_completion.py)

### Logging - Showing how to set up logging

- [Setup Logging](./logging/setup_logging.py)

### Memory - Using [`Memory`](https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/?pivots=programming-language-python) AI concepts

- [Simple Memory](./memory/simple_memory.py)
- [Memory Data Models](./memory/data_models.py)
- [Memory with Pandas Dataframes](./memory/memory_with_pandas.py)
- [Complex memory](./memory/complex_memory.py)
- [Full sample with Azure AI Search including function calling](./memory/azure_ai_search_hotel_samples/README.md)

### Model-as-a-Service - Using models deployed as [`serverless APIs on Azure AI Studio`](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-serverless?tabs=azure-ai-studio) to benchmark model performance against open-source datasets

- [MMLU Model Evaluation](./model_as_a_service/mmlu_model_eval.py)

### On Your Data - Examples of using AzureOpenAI [`On Your Data`](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data?tabs=mongo-db)

- [Azure Chat GPT with Data API](./on_your_data/azure_chat_gpt_with_data_api.py)
- [Azure Chat GPT with Data API Function Calling](./on_your_data/azure_chat_gpt_with_data_api_function_calling.py)
- [Azure Chat GPT with Data API Vector Search](./on_your_data/azure_chat_gpt_with_data_api_vector_search.py)

### Plugins - Different ways of creating and using [`Plugins`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/functions/kernel_plugin.py)

- [Azure Key Vault Settings](./plugins/azure_key_vault_settings.py)
- [Azure Python Code Interpreter](./plugins/azure_python_code_interpreter.py)
- [OpenAI Function Calling with Custom Plugin](./plugins/openai_function_calling_with_custom_plugin.py)
- [Plugins from Directory](./plugins/plugins_from_dir.py)

### Processes - Examples of using the [`Process Framework`](../../semantic_kernel/processes/)

- [Cycles with Fan-In](./processes/cycles_with_fan_in.py)
- [Nested Process](./processes/nested_process.py)
- [Plan and Execute](./processes/plan_and_execute.py)

### PromptTemplates - Using [`Templates`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/prompt_template/prompt_template_base.py) with parametrization for `Prompt` rendering

- [Template Language](./prompt_templates/template_language.py)
- [Azure Chat GPT API Jinja2](./prompt_templates/azure_chat_gpt_api_jinja2.py)
- [Load YAML Prompt](./prompt_templates/load_yaml_prompt.py)
- [Azure Chat GPT API Handlebars](./prompt_templates/azure_chat_gpt_api_handlebars.py)
- [Configuring Prompts](./prompt_templates/configuring_prompts.py)

### RAG - Different ways of `RAG` (Retrieval-Augmented Generation)

- [RAG with Vector Collection](./rag/rag_with_vector_collection.py)
- [Self-Critique RAG](./rag/self_critique_rag.py)

### Reasoning - Using [`ChatCompletion`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/connectors/ai/chat_completion_client_base.py) to reason with OpenAI Reasoning

- [Simple Chatbot](./reasoning/simple_reasoning.py)
- [Simple Function Calling](./reasoning/simple_reasoning_function_calling.py)

### Search - Using [`Search`](https://github.com/microsoft/semantic-kernel/tree/main/python/semantic_kernel/connectors/search) services information

- [Bing Text Search as Plugin](./search/bing_text_search_as_plugin.py)
- [Brave Text Search as Plugin](./search/brave_text_search_as_plugin.py)
- [Google Text Search as Plugin](./search/google_text_search_as_plugin.py)

### Service Selector - Shows how to create and use a custom service selector class

- [Custom Service Selector](./service_selector/custom_service_selector.py)

### Setup - How to set up environment variables for Semantic Kernel

- [OpenAI Environment Setup](./setup/openai_env_setup.py)
- [Chat Completion Services](./setup/chat_completion_services.py)

### Structured Outputs - How to leverage OpenAI's json_schema [`Structured Outputs`](https://platform.openai.com/docs/guides/structured-outputs) functionality

- [JSON Structured Outputs](./structured_outputs/json_structured_outputs.py)
- [JSON Structured Outputs Function Calling](./structured_outputs/json_structured_outputs_function_calling.py)

### TextGeneration - Using [`TextGeneration`](https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/connectors/ai/text_completion_client_base.py) capable service with models

- [Text Completion Client](./local_models/onnx_text_completion.py)

# Configuring the Kernel

In Semantic Kernel for Python, we leverage Pydantic Settings to manage configurations for AI and Memory Connectors, among other components. Here’s a clear guide on how to configure your settings effectively:

## Steps for Configuration

1. **Reading Environment Variables:**
   - **Primary Source:** Pydantic first attempts to read the required settings from environment variables.

2. **Using a .env File:**
   - **Fallback Source:** If the required environment variables are not set, Pydantic will look for a `.env` file in the current working directory.
   - **Custom Path (Optional):** You can specify an alternative path for the `.env` file via `env_file_path`. This can be either a relative or an absolute path.

3. **Direct Constructor Input:**
   - As an alternative to environment variables and `.env` files, you can pass the required settings directly through the constructor of the AI Connector or Memory Connector.

## Azure Authentication

To authenticate to your Azure resources, you must provide one of the following authentication methods to successfully authenticate:

1. **AsyncTokenCredential** - provide one of the `AsyncTokenCredential` types (e.g. `AzureCliCredential`, `ManagedIdentityCredential`). More information here: [Credentials for asynchronous Azure SDK clients]("https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.aio?view=azure-python").
2. **Custom AsyncAzureOpenAI client** - Pass a pre-configured client instance.
3. **Access Token (`ad_token`)** - Provide a valid Microsoft Entra access token directly.
4. **Token Provider (`ad_token_provider`)** - Provide a callable that returns a valid access token.
5. **API Key** - Provide through an environment variable, a `.env` file, or the constructor.

To successfully retrieve and use the Entra Auth Token, you need the `Cognitive Services OpenAI Contributor` role assigned to your Azure OpenAI resource. By default, the `https://cognitiveservices.azure.com` token endpoint is used. You can override this endpoint by setting an environment variable `.env` variable as `AZURE_OPENAI_TOKEN_ENDPOINT` or by passing a new value to the `AzureChatCompletion` constructor as part of the `AzureOpenAISettings`.

## Best Practices

- **.env File Placement:** We highly recommend placing the `.env` file in the `semantic-kernel/python` root directory. This is a common practice when developing in the Semantic Kernel repository.

By following these guidelines, you can ensure that your settings for various components are configured correctly, enabling seamless functionality and integration of Semantic Kernel in your Python projects.