wehub/microsoft--semantic-kernel
1
0
Fork 0
Code Issues Pull Requests Actions 2 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
CodeQL / Analyze (csharp) (push) Has been cancelled
Details
CodeQL / Analyze (python) (push) Has been cancelled
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:21:23 +08:00
commit b957a53def
5423 changed files with 863745 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dotnet/docs/EXPERIMENTS.md
+87
View File
@@ -0,0 +1,87 @@
# Experiments
The following capabilities are marked experimental in the .NET SDK. Once the APIs for these features are stable, the experimental attribute will be removed. In the meantime, these features are subject to change.
You can use the following diagnostic IDs to ignore warnings or errors for a particular experimental feature. For example, to ignore warnings for the embedding services, add `SKEXP0001` to your list of ignored warnings in your .NET project file as well as the ID for the embedding service you want to use. For example:
```xml
<PropertyGroup>
<NoWarn>$(NoWarn);SKEXP0001,SKEXP0010</NoWarn>
</PropertyGroup>
```
## Experimental Feature Codes
| SKEXP​ | Experimental Features Category​​ |
|-------|--------------------------------|
| SKEXP0001 | Semantic Kernel core features |
| SKEXP0010 | OpenAI and Azure OpenAI services |
| SKEXP0020 | Memory connectors |
| SKEXP0040 | Function types |
| SKEXP0050 | Out-of-the-box plugins |
| SKEXP0060 | Planners |
| SKEXP0070 | AI connectors |
| SKEXP0080 | Processes |
| SKEXP0100 | Advanced Semantic Kernel features |
| SKEXP0110 | Semantic Kernel Agents |
| SKEXP0120 | Native-AOT |
| SKEXP0130 | AI Context Providers |
| MEVD9000 | Microsoft.Extensions.VectorData experimental user-facing APIs |
| MEVD9001 | Microsoft.Extensions.VectorData experimental connector-facing APIs |
## Experimental Features Tracking
| SKEXP​ | Features​​ |
|-------|----------|
| SKEXP0001 | Embedding services |
| SKEXP0001 | Image services |
| SKEXP0001 | Memory connectors |
| SKEXP0001 | Kernel filters |
| SKEXP0001 | Audio services |
| | | | | | | |
| SKEXP0010 | Azure OpenAI with your data service |
| SKEXP0010 | OpenAI embedding service |
| SKEXP0010 | OpenAI image service |
| SKEXP0010 | OpenAI parameters |
| SKEXP0010 | OpenAI chat history extension |
| SKEXP0010 | OpenAI file service |
| | | | | | | |
| SKEXP0020 | Azure AI Search memory connector |
| SKEXP0020 | Chroma memory connector |
| SKEXP0020 | DuckDB memory connector |
| SKEXP0020 | Kusto memory connector |
| SKEXP0020 | Milvus memory connector |
| SKEXP0020 | Qdrant memory connector |
| SKEXP0020 | Redis memory connector |
| SKEXP0020 | Sqlite memory connector |
| SKEXP0020 | Weaviate memory connector |
| SKEXP0020 | MongoDB memory connector |
| SKEXP0020 | Pinecone memory connector |
| SKEXP0020 | Postgres memory connector |
| | | | | | | |
| SKEXP0040 | GRPC functions |
| SKEXP0040 | Markdown functions |
| SKEXP0040 | OpenAPI functions |
| SKEXP0040 | OpenAPI function extensions - API Manifest |
| SKEXP0040 | OpenAPI function extensions - Copilot Agent Plugin |
| SKEXP0040 | Prompty Format support |
| | | | | | | |
| SKEXP0050 | Core plugins |
| SKEXP0050 | Document plugins |
| SKEXP0050 | Memory plugins |
| SKEXP0050 | Microsoft 365 plugins |
| SKEXP0050 | Web plugins |
| SKEXP0050 | Text chunker plugin |
| | | | | | | |
| SKEXP0060 | Handlebars planner |
| SKEXP0060 | OpenAI Stepwise planner |
| | | | | | | |
| SKEXP0080 | Process Framework |
| SKEXP0081 | Process Framework - Foundry Process
| | | | | | | |
| SKEXP0101 | Experiment with Assistants |
| SKEXP0101 | Experiment with Flow Orchestration |
| | | | | | | |
| SKEXP0110 | Agent Framework |
| | | | | | | |
| SKEXP0120 | Native-AOT |
dotnet/docs/MODELS.md
+137
View File
@@ -0,0 +1,137 @@
# Models
This document describes the planned models to be supported by Semantic Kernel along with their current status. If you are interested in contributing to the development of a model, please use the attached links to the GitHub issues and comment that you're wanting to help.
## Supported deployment types
In the core Semantic Kernel repo, we plan on supporting up to four deployment types of each model:
- Dedicated API endpoints (e.g., OpenAI's APIs, Mistral.AI, and Google Gemini)
- Azure AI deployments via the [model catalog](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/model-catalog)
- Local deployments via [Ollama](https://ollama.ai/)
- Hugging face deployment using the [Hugging Face inference API](https://huggingface.co/docs/api-inference/index)
To support these different deployment types, we will follow a similar pattern to the Azure OpenAI and OpenAI connectors. Each connector uses the same underlying model and abstractions, but the connector constructors may take different parameters. For example, the Azure OpenAI connector expects an Azure endpoint and key, whereas the OpenAI connector expects an OpenAI organization ID and API key.
If there is another deployment type you'd like to see supported, please open an issue. We'll either work with you to add support for it or help you create a custom repository and NuGet package for your use case.
## Planned models
The following models are currently prioritized for development. If you'd like to see a model added to this list, please open an issue. If you'd like to contribute to the development of a model, please comment on the issue that you're wanting to help.
Please note that not all of the model interfaces are defined yet. As part of contributing a new model, we'll work with you to define the interface and then implement it. As part of implementing the connector, you may also determine that the currently planned interface isn't the best fit for the model. If that's the case, we'll work with you to update the interface.
### OpenAI
| Priority | Model | Status | Interface | Deployment type | GitHub issue | Developer | Reviewer |
| -------- | ----------------------- | ----------- | ------------------------------ | --------------- | ------------ | ------------ | ----------- |
| P0 | GPT-3.5-turbo | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-3.5-turbo | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | GPT-4 | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-4 | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | GPT-4v | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-4v | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | text-embedding-ada-002 | Preview | `IEmbeddingGeneration` | OpenAI API | N/A | N/A | N/A |
| P0 | text-embedding-ada-002 | Preview | `IEmbeddingGeneration` | Azure AI | N/A | N/A | N/A |
| P0 | DALL·E 3 | Preview | `ITextToImage` | OpenAI API | N/A | N/A | N/A |
| P0 | DALL·E 3 | Preview | `ITextToImage` | Azure AI | N/A | N/A | N/A |
| P0 | Text-to-speech | Complete | `ITextToSpeech` | OpenAI API | TBD | dmytrostruk | TBD |
| P0 | Speech-to-text | Complete | `ISpeechRecognition` | OpenAI API | TBD | dmytrostruk | TBD |
| P1 | openai-whisper-large-v3 | Not started | `ISpeechRecognition` | Azure AI | TBD | TBD | TBD |
| P1 | openai-whisper-large-v3 | Not started | `ISpeechRecognition` | Hugging Face | TBD | TBD | TBD |
| P2 | Moderation | In Progress | `ITextClassification` | OpenAI API | #5062 | Krzysztof318 | MarkWallace |
| P2 | clip-vit-base-patch32 | Not started | `IZeroShotImageClassification` | Azure AI | TBD | TBD | TBD |
| P2 | clip-vit-base-patch32 | Not started | `IZeroShotImageClassification` | Hugging Face | TBD | TBD | TBD |
### Microsoft
| Priority | Model | Status | Interface | Deployment type | GitHub issue | Developer | Reviewer |
| -------- | ----------------- | ----------- | ---------------------- | --------------- | ------------ | --------- | -------- |
| P0 | microsoft-phi-1-5 | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P0 | microsoft-phi-1-5 | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P0 | microsoft-phi-2 | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P0 | microsoft-phi-2 | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P2 | resnet-50 | Not started | `IImageClassification` | Azure AI | TBD | TBD | TBD |
| P2 | resnet-50 | Not started | `IImageClassification` | Hugging Face | TBD | TBD | TBD |
### Google
| Priority | Model | Status | Interface | Deployment type | GitHub issue | Developer | Reviewer |
| -------- | ----------------- | ----------- | ---------------------- | --------------- | ------------ | ------------ | ------------ |
| P0 | gemini-pro | In Progress | `IChatCompletion` | Google API | TBD | Krzysztof318 | RogerBarreto |
| P0 | gemini-pro-vision | In Progress | `IChatCompletion` | Google API | TBD | Krzysztof318 | RogerBarreto |
| P0 | gemini-ultra | In Progress | `IChatCompletion` | Google API | TBD | Krzysztof318 | RogerBarreto |
| P0 | embedding-001 | In Progress | `IEmbeddingGeneration` | Google API | TBD | Krzysztof318 | RogerBarreto |
### Facebook
| Priority | Model | Status | Interface | Deployment type | GitHub issue | Developer | Reviewer |
| -------- | ------------------------- | ----------- | ----------------- | --------------- | ------------ | --------- | -------- |
| P0 | Llama-2-7b-chat | Not started | `IChatCompletion` | Azure AI | TBD | TBD | TBD |
| P0 | Llama-2-7b-chat | Not started | `IChatCompletion` | Hugging Face | TBD | TBD | TBD |
| P0 | Llama-2-13b-chat | Not started | `IChatCompletion` | Azure AI | TBD | TBD | TBD |
| P0 | Llama-2-13b-chat | Not started | `IChatCompletion` | Hugging Face | TBD | TBD | TBD |
| P0 | Llama-2-70b-chat | Not started | `IChatCompletion` | Azure AI | TBD | TBD | TBD |
| P0 | Llama-2-70b-chat | Not started | `IChatCompletion` | Hugging Face | TBD | TBD | TBD |
| P0 | CodeLlama-7b-Instruct-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P0 | CodeLlama-7b-Instruct-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P0 | CodeLlama-13b-Instruct-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P0 | CodeLlama-13b-Instruct-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P0 | CodeLlama-34b-Instruct-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P0 | CodeLlama-34b-Instruct-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P1 | Llama-2-7b | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P1 | Llama-2-7b | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P1 | Llama-2-7b | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P1 | Llama-2-13b | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P1 | Llama-2-13b | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P1 | Llama-2-13b | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P1 | Llama-2-70b | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P1 | Llama-2-70b | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P1 | Llama-2-70b | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P1 | CodeLlama-7b-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P1 | CodeLlama-7b-hf | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P1 | CodeLlama-7b-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P1 | CodeLlama-13b-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P1 | CodeLlama-13b-hf | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P1 | CodeLlama-13b-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P1 | CodeLlama-34b-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P1 | CodeLlama-34b-hf | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P1 | CodeLlama-34b-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P1 | CodeLlama-7b-Python-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P1 | CodeLlama-7b-Python-hf | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P2 | CodeLlama-7b-Python-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P2 | CodeLlama-13b-Python-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P2 | CodeLlama-13b-Python-hf | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P2 | CodeLlama-13b-Python-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
| P2 | CodeLlama-34b-Python-hf | Not started | `ITextGeneration` | Azure AI | TBD | TBD | TBD |
| P2 | CodeLlama-34b-Python-hf | Not started | `ITextGeneration` | Ollama | TBD | TBD | TBD |
| P2 | CodeLlama-34b-Python-hf | Not started | `ITextGeneration` | Hugging Face | TBD | TBD | TBD |
### Mistral
| Priority | Model | Status | Interface | Deployment type | GitHub issue | Developer | Reviewer |
| -------- | ----------------------- | ----------- | ----------------- | --------------- | ------------ | --------- | -------- |
| P2 | Mistral-7B-v0.2 | Not started | `IChatCompletion` | Mistral API | TBD | TBD | TBD |
| P2 | Mistral-7B-v0.2 | Not started | `IChatCompletion` | Ollama | TBD | TBD | TBD |
| P2 | Mistral-7B-v0.1 | Not started | `IChatCompletion` | Azure AI | TBD | TBD | TBD |
| P2 | Mistral-7B-v0.1 | Not started | `IChatCompletion` | Hugging Face | TBD | TBD | TBD |
| P2 | Mistral-7B-Instruct-v01 | Not started | `IChatCompletion` | Azure AI | TBD | TBD | TBD |
| P2 | Mistral-7B-Instruct-v01 | Not started | `IChatCompletion` | Hugging Face | TBD | TBD | TBD |
| P2 | Mixtral-8X7B-v0.1 | Not started | `IChatCompletion` | Mistral API | TBD | TBD | TBD |
| P2 | Mixtral-8X7B-v0.1 | Not started | `IChatCompletion` | Azure AI | TBD | TBD | TBD |
| P2 | Mixtral-8X7B-v0.1 | Not started | `IChatCompletion` | Hugging Face | TBD | TBD | TBD |
| P2 | mistral-medium | Not started | `IChatCompletion` | Mistral API | TBD | TBD | TBD |
| P2 | mistral-embed | Not started | `IChatCompletion` | Mistral API | TBD | TBD | TBD |
### Other
| Priority | Model | Status | Interface | Deployment type | GitHub issue | Developer | Reviewer |
| -------- | ------------------------------ | ----------- | -------------------- | --------------- | ------------ | --------- | -------- |
| P0 | wav2vec2-large-xlsr-53-english | Not started | `ISpeechRecognition` | Azure AI | TBD | TBD | TBD |
| P1 | wav2vec2-large-xlsr-53-english | Not started | `ISpeechRecognition` | Hugging Face | TBD | TBD | TBD |
| P2 | bert-base-uncased | Not started | `IFillMask` | Azure AI | TBD | TBD | TBD |
| P2 | bert-base-uncased | Not started | `IFillMask` | Hugging Face | TBD | TBD | TBD |
| P2 | roberta-large | Not started | `IFillMask` | Azure AI | TBD | TBD | TBD |
| P2 | roberta-large | Not started | `IFillMask` | Hugging Face | TBD | TBD | TBD |
| P1 | stable-diffusion-xl-base-1.0 | Not started | `ITextToImage` | Azure AI | TBD | TBD | TBD |
| P1 | stable-diffusion-xl-base-1.0 | Not started | `ITextToImage` | Hugging Face | TBD | TBD | TBD |
dotnet/docs/OPENAI-CONNECTOR-MIGRATION.md
+216
View File
@@ -0,0 +1,216 @@
# OpenAI Connector Migration Guide
This manual prepares you for the migration of your OpenAI Connector to the new OpenAI Connector. The new OpenAI Connector is a complete rewrite of the existing OpenAI Connector and is designed to be more efficient, reliable, and scalable. This manual will guide you through the migration process and help you understand the changes that have been made to the OpenAI Connector.
## 1. Package Setup when Using Azure
If you are working with Azure and or OpenAI public APIs, you will need to change the package from `Microsoft.SemanticKernel.Connectors.OpenAI` to `Microsoft.SemanticKernel.Connectors.AzureOpenAI`,
> [!IMPORTANT]
> The `Microsoft.SemanticKernel.Connectors.AzureOpenAI` package depends on the `Microsoft.SemanticKernel.Connectors.OpenAI` package so there's no need to add both to your project when using `OpenAI` related types.
```diff
- // Before
- using Microsoft.SemanticKernel.Connectors.OpenAI;
+ After
+ using Microsoft.SemanticKernel.Connectors.AzureOpenAI;
```
### 1.1 AzureOpenAIClient
When using Azure with OpenAI, before where you were using `OpenAIClient` you will need to update your code to use the new `AzureOpenAIClient` type.
### 1.2 Services
All services below now belong to the `Microsoft.SemanticKernel.Connectors.AzureOpenAI` namespace.
- `AzureOpenAIAudioToTextService`
- `AzureOpenAIChatCompletionService`
- `AzureOpenAITextEmbeddingGenerationService`
- `AzureOpenAITextToAudioService`
- `AzureOpenAITextToImageService`
## 2. Text Generation Deprecated
The latest `OpenAI` SDK does not support text generation modality, when migrating to their underlying SDK we had to drop the support and removed `TextGeneration` specific services but the existing `ChatCompletion` ones still supports (implements `ITextGenerationService`).
If you were using any of the `OpenAITextGenerationService` or `AzureOpenAITextGenerationService` you will need to update your code to target a chat completion model instead, using `OpenAIChatCompletionService` or `AzureOpenAIChatCompletionService` instead.
> [!NOTE]
> OpenAI and AzureOpenAI `ChatCompletion` services also implement the `ITextGenerationService` interface and that may not require any changes to your code if you were targeting the `ITextGenerationService` interface.
tags:
`OpenAITextGenerationService`,`AzureOpenAITextGenerationService`,
`AddOpenAITextGeneration`,`AddAzureOpenAITextGeneration`
## 3. ChatCompletion Multiple Choices Deprecated
The latest `OpenAI` SDK does not support multiple choices, when migrating to their underlying SDK we had to drop the support and removed `ResultsPerPrompt` also from the `OpenAIPromptExecutionSettings`.
tags: `ResultsPerPrompt`,`results_per_prompt`
## 4. OpenAI File Service Deprecation
The `OpenAIFileService` was deprecated in the latest version of the OpenAI Connector. We strongly recommend to update your code to use the new `OpenAIClient.GetFileClient()` for file management operations.
## 5. OpenAI ChatCompletion custom endpoint
The `OpenAIChatCompletionService` **experimental** constructor for custom endpoints will not attempt to auto-correct the endpoint and use it as is.
We have the two only specific cases where we attempted to auto-correct the endpoint.
1. If you provided `chat/completions` path before. Now those need to be removed as they are added automatically to the end of your original endpoint by `OpenAI SDK`.
```diff
- http://any-host-and-port/v1/chat/completions
+ http://any-host-and-port/v1
```
2. If you provided a custom endpoint without any path. We won't be adding the `v1/` as the first path. Now the `v1` path needs to provided as part of your endpoint.
```diff
- http://any-host-and-port/
+ http://any-host-and-port/v1
```
## 6. SemanticKernel MetaPackage
To be retro compatible with the new OpenAI and AzureOpenAI Connectors, our `Microsoft.SemanticKernel` meta package changed its dependency to use the new `Microsoft.SemanticKernel.Connectors.AzureOpenAI` package that depends on the `Microsoft.SemanticKernel.Connectors.OpenAI` package. This way if you are using the metapackage, no change is needed to get access to `Azure` related types.
## 7. Contents
### 7.1 OpenAIChatMessageContent
- The `Tools` property type has changed from `IReadOnlyList<ChatCompletionsToolCall>` to `IReadOnlyList<ChatToolCall>`.
- Inner content type has changed from `ChatCompletionsFunctionToolCall` to `ChatToolCall`.
- Metadata type `FunctionToolCalls` has changed from `IEnumerable<ChatCompletionsFunctionToolCall>` to `IEnumerable<ChatToolCall>`.
### 7.2 OpenAIStreamingChatMessageContent
- The `FinishReason` property type has changed from `CompletionsFinishReason` to `FinishReason`.
- The `ToolCallUpdate` property has been renamed to `ToolCallUpdates` and its type has changed from `StreamingToolCallUpdate?` to `IReadOnlyList<StreamingToolCallUpdate>?`.
- The `AuthorName` property is not initialized because it's not provided by the underlying library anymore.
## 7.3 Metrics for AzureOpenAI Connector
The meter `s_meter = new("Microsoft.SemanticKernel.Connectors.OpenAI");` and the relevant counters still have old names that contain "openai" in them, such as:
- `semantic_kernel.connectors.openai.tokens.prompt`
- `semantic_kernel.connectors.openai.tokens.completion`
- `semantic_kernel.connectors.openai.tokens.total`
## 8. Using Azure with your data (Data Sources)
With the new `AzureOpenAIClient`, you can now specify your datasource thru the options and that requires a small change in your code to the new type.
Before
```csharp
var promptExecutionSettings = new OpenAIPromptExecutionSettings
{
AzureChatExtensionsOptions = new AzureChatExtensionsOptions
{
Extensions = [ new AzureSearchChatExtensionConfiguration
{
SearchEndpoint = new Uri(TestConfiguration.AzureAISearch.Endpoint),
Authentication = new OnYourDataApiKeyAuthenticationOptions(TestConfiguration.AzureAISearch.ApiKey),
IndexName = TestConfiguration.AzureAISearch.IndexName
}]
};
};
```
After
```csharp
var promptExecutionSettings = new AzureOpenAIPromptExecutionSettings
{
AzureChatDataSource = new AzureSearchChatDataSource
{
Endpoint = new Uri(TestConfiguration.AzureAISearch.Endpoint),
Authentication = DataSourceAuthentication.FromApiKey(TestConfiguration.AzureAISearch.ApiKey),
IndexName = TestConfiguration.AzureAISearch.IndexName
}
};
```
## 9. Breaking glass scenarios
Breaking glass scenarios are scenarios where you may need to update your code to use the new OpenAI Connector. Below are some of the breaking changes that you may need to be aware of.
#### 9.1 KernelContent Metadata
Some of the keys in the content metadata dictionary have changed, you will need to update your code to when using the previous key names.
- `Created` -> `CreatedAt`
#### 9.2 Prompt Filter Results
The `PromptFilterResults` metadata type has changed from `IReadOnlyList<ContentFilterResultsForPrompt>` to `ContentFilterResultForPrompt`.
#### 9.3 Content Filter Results
The `ContentFilterResultsForPrompt` type has changed from `ContentFilterResultsForChoice` to `ContentFilterResultForResponse`.
#### 9.4 Finish Reason
The FinishReason metadata string value has changed from `stop` to `Stop`
#### 9.5 Tool Calls
The ToolCalls metadata string value has changed from `tool_calls` to `ToolCalls`
#### 9.6 LogProbs / Log Probability Info
The `LogProbabilityInfo` type has changed from `ChatChoiceLogProbabilityInfo` to `IReadOnlyList<ChatTokenLogProbabilityInfo>`.
#### 9.7 Finish Details, Index, and Enhancements
All of above have been removed.
#### 9.8 Token Usage
The Token usage naming convention from `OpenAI` changed from `Completion`, `Prompt` tokens to `Output` and `Input` respectively. You will need to update your code to use the new naming.
The type also changed from `CompletionsUsage` to `ChatTokenUsage`.
[Example of Token Usage Metadata Changes](https://github.com/microsoft/semantic-kernel/pull/7151/files#diff-a323107b9f8dc8559a83e50080c6e34551ddf6d9d770197a473f249589e8fb47)
```diff
- Before
- var usage = FunctionResult.Metadata?["Usage"] as CompletionsUsage;
- var completionTokesn = usage?.CompletionTokens ?? 0;
- var promptTokens = usage?.PromptTokens ?? 0;
+ After
+ var usage = FunctionResult.Metadata?["Usage"] as ChatTokenUsage;
+ var promptTokens = usage?.InputTokens ?? 0;
+ var completionTokens = completionTokens: usage?.OutputTokens ?? 0;
totalTokens: usage?.TotalTokens ?? 0;
```
#### 9.9 OpenAIClient
The `OpenAIClient` type previously was a Azure specific namespace type but now it is an `OpenAI` SDK namespace type, you will need to update your code to use the new `OpenAIClient` type.
When using Azure, you will need to update your code to use the new `AzureOpenAIClient` type.
#### 9.10 Pipeline Configuration
The new `OpenAI` SDK uses a different pipeline configuration, and has a dependency on `System.ClientModel` package. You will need to update your code to use the new `HttpClientPipelineTransport` transport configuration where before you were using `HttpClientTransport` from `Azure.Core.Pipeline`.
[Example of Pipeline Configuration](https://github.com/microsoft/semantic-kernel/pull/7151/files#diff-fab02d9a75bf43cb57f71dddc920c3f72882acf83fb125d8cad963a643d26eb3)
```diff
var clientOptions = new OpenAIClientOptions
{
- // Before: From Azure.Core.Pipeline
- Transport = new HttpClientTransport(httpClient),
+ // After: From OpenAI SDK -> System.ClientModel
+ Transport = new HttpClientPipelineTransport(httpClient),
};
```
dotnet/docs/TELEMETRY.md
+133
View File
@@ -0,0 +1,133 @@
# Telemetry
Telemetry in Semantic Kernel (SK) .NET implementation includes _logging_, _metering_ and _tracing_.
The code is instrumented using native .NET instrumentation tools, which means that it's possible to use different monitoring platforms (e.g. Application Insights, Aspire dashboard, Prometheus, Grafana etc.).
Code example using Application Insights can be found [here](../samples/Demos/TelemetryWithAppInsights/).
## Logging
The logging mechanism in this project relies on the `ILogger` interface from the `Microsoft.Extensions.Logging` namespace. Recent updates have introduced enhancements to the logger creation process. Instead of directly using the `ILogger` interface, instances of `ILogger` are now recommended to be created through an `ILoggerFactory` configured through a `ServiceCollection`.
By employing the `ILoggerFactory` approach, logger instances are generated with precise type information, facilitating more accurate logging and streamlined control over log filtering across various classes.
Log levels used in SK:
- Trace - this type of logs **should not be enabled in production environments**, since it may contain sensitive data. It can be useful in test environments for better observability. Logged information includes:
- Goal/Ask to create a plan
- Prompt (template and rendered version) for AI to create a plan
- Created plan with function arguments (arguments may contain sensitive data)
- Prompt (template and rendered version) for AI to execute a function
- Arguments to functions (arguments may contain sensitive data)
- Debug - contains more detailed messages without sensitive data. Can be enabled in production environments.
- Information (default) - log level that is enabled by default and provides information about general flow of the application. Contains following data:
- AI model used to create a plan
- Plan creation status (Success/Failed)
- Plan creation execution time (in seconds)
- Created plan without function arguments
- AI model used to execute a function
- Function execution status (Success/Failed)
- Function execution time (in seconds)
- Warning - includes information about unusual events that don't cause the application to fail.
- Error - used for logging exception details.
### Examples
Enable logging for Kernel instance:
```csharp
IKernelBuilder builder = Kernel.CreateBuilder();
// Assuming loggerFactory is already defined.
builder.Services.AddSingleton(loggerFactory);
...
var kernel = builder.Build();
```
All kernel functions and planners will be instrumented. It includes _logs_, _metering_ and _tracing_.
### Log Filtering Configuration
Log filtering configuration has been refined to strike a balance between visibility and relevance:
```csharp
using var loggerFactory = LoggerFactory.Create(builder =>
{
// Add OpenTelemetry as a logging provider
builder.AddOpenTelemetry(options =>
{
// Assuming connectionString is already defined.
options.AddAzureMonitorLogExporter(options => options.ConnectionString = connectionString);
// Format log messages. This is default to false.
options.IncludeFormattedMessage = true;
});
builder.AddFilter("Microsoft", LogLevel.Warning);
builder.AddFilter("Microsoft.SemanticKernel", LogLevel.Information);
}
```
> Read more at: https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/logs/customizing-the-sdk/README.md
## Metering
Metering is implemented with `Meter` class from `System.Diagnostics.Metrics` namespace.
Available meters:
- _Microsoft.SemanticKernel.Planning_ - contains all metrics related to planning. List of metrics:
- `semantic_kernel.planning.create_plan.duration` (Histogram) - execution time of plan creation (in seconds)
- `semantic_kernel.planning.invoke_plan.duration` (Histogram) - execution time of plan execution (in seconds)
- _Microsoft.SemanticKernel_ - captures metrics for `KernelFunction`. List of metrics:
- `semantic_kernel.function.invocation.duration` (Histogram) - function execution time (in seconds)
- `semantic_kernel.function.streaming.duration` (Histogram) - function streaming execution time (in seconds)
- `semantic_kernel.function.invocation.token_usage.prompt` (Histogram) - number of prompt token usage (only for `KernelFunctionFromPrompt`)
- `semantic_kernel.function.invocation.token_usage.completion` (Histogram) - number of completion token usage (only for `KernelFunctionFromPrompt`)
- _Microsoft.SemanticKernel.Connectors.OpenAI_ - captures metrics for OpenAI functionality. List of metrics:
- `semantic_kernel.connectors.openai.tokens.prompt` (Counter) - number of prompt tokens used.
- `semantic_kernel.connectors.openai.tokens.completion` (Counter) - number of completion tokens used.
- `semantic_kernel.connectors.openai.tokens.total` (Counter) - total number of tokens used.
Measurements will be associated with tags that will allow data to be categorized for analysis:
```csharp
TagList tags = new() { { "semantic_kernel.function.name", this.Name } };
s_invocationDuration.Record(duration.TotalSeconds, in tags);
```
### [Examples](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Demos/TelemetryWithAppInsights/Program.cs)
Depending on monitoring tool, there are different ways how to subscribe to available meters. Following example shows how to subscribe to available meters and export metrics to Application Insights using `OpenTelemetry.Sdk`:
```csharp
using var meterProvider = Sdk.CreateMeterProviderBuilder()
.AddMeter("Microsoft.SemanticKernel*")
.AddAzureMonitorMetricExporter(options => options.ConnectionString = connectionString)
.Build();
```
> Read more at: https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-enable?tabs=net
> Read more at: https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/metrics/customizing-the-sdk/README.md
## Tracing
Tracing is implemented with `Activity` class from `System.Diagnostics` namespace.
Available activity sources:
- _Microsoft.SemanticKernel.Planning_ - creates activities for all planners.
- _Microsoft.SemanticKernel_ - creates activities for `KernelFunction` as well as requests to models.
### Examples
Subscribe to available activity sources using `OpenTelemetry.Sdk`:
```csharp
using var traceProvider = Sdk.CreateTracerProviderBuilder()
.AddSource("Microsoft.SemanticKernel*")
.AddAzureMonitorTraceExporter(options => options.ConnectionString = connectionString)
.Build();
```
> Read more at: https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/trace/customizing-the-sdk/README.md