## Proposal ### IChatCompletion Before: ```csharp public interface IChatCompletion : IAIService { ChatHistory CreateNewChat(string? instructions = null); Task> GetChatCompletionsAsync(ChatHistory chat, ...); Task> GetChatCompletionsAsync(string prompt, ...); IAsyncEnumerable GetStreamingContentAsync(ChatHistory chatHistory, ...); } public static class ChatCompletionExtensions { public static async Task GenerateMessageAsync(ChatHistory chat, ...); } ``` After: ```csharp public interface IChatCompletion : IAIService { Task> GetChatContentsAsync(ChatHistory chat, ..> tags) IAsyncEnumerable GetStreamingChatContentsAsync(ChatHistory chatHistory, ...); } public static class ChatCompletionExtensions { // v Single vv Standardized Prompt (Parse tags) public static async Task GetChatContentAsync(string prompt, ...); // v Single public static async Task GetChatContentAsync(ChatHistory chatHistory, ...); public static IAsyncEnumerable GetStreamingChatContentsAsync(string prompt, ...); } ``` ### ITextCompletion Before: ```csharp public interface ITextCompletion : IAIService { Task> GetCompletionsAsync(string prompt, ...); IAsyncEnumerable GetStreamingContentAsync(string prompt, ...); } public static class TextCompletionExtensions { public static async Task CompleteAsync(string text, ...); public static IAsyncEnumerable GetStreamingContentAsync(string input, ...); } ``` After: ```csharp public interface ITextCompletion : IAIService { Task> GetTextContentsAsync(string prompt, ...); IAsyncEnumerable GetStreamingTextContentsAsync(string prompt, ...); } public static class TextCompletionExtensions { public static async Task GetTextContentAsync(string prompt, ...); } ``` ## Content Abstractions ### Model Comparisons #### Current Streaming Abstractions | Streaming (Current) | Specialized\* Streaming (Current) | | ------------------------------------------- | --------------------------------------------------------------- | | `StreamingChatContent` : `StreamingContent` | `OpenAIStreamingChatContent` | | `StreamingTextContent` : `StreamingContent` | `OpenAIStreamingTextContent`, `HuggingFaceStreamingTextContent` | #### Non-Streaming Abstractions (Before and After) | Non-Streaming (Before) | Non-Streaming (After) | Specialized\* Non-Streaming (After) | | ----------------------------- | ------------------------------ | --------------------------------------------- | | `IChatResult` : `IResultBase` | `ChatContent` : `ModelContent` | `OpenAIChatContent` | | `ITextResult` : `IResultBase` | `TextContent` : `ModelContent` | `OpenAITextContent`, `HuggingFaceTextContent` | | `ChatMessage` | `ChatContent` : `ModelContent` | `OpenAIChatContent` | _\*Specialized: Connector implementations that are specific to a single AI Service._ ### New Non-Streaming Abstractions: `ModelContent` was chosen to represent a `non-streaming content` top-most abstraction which can be specialized and contains all the information that the AI Service returned. (Metadata, Raw Content, etc.) ```csharp /// /// Base class for all AI non-streaming results /// public abstract class ModelContent { /// /// Raw content object reference. (Breaking glass). /// public object? InnerContent { get; } /// /// The metadata associated with the content. /// ⚠️ (Token Usage + More Backend API Metadata) info will be in this dictionary. Old IResult.ModelResult) ⚠️ /// public Dictionary? Metadata { get; } /// /// Initializes a new instance of the class. /// /// Raw content object reference /// Metadata associated with the content protected CompleteContent(object rawContent, Dictionary? metadata = null) { this.InnerContent = rawContent; this.Metadata = metadata; } } ``` ```csharp /// /// Chat content abstraction /// public class ChatContent : ModelContent { /// /// Role of the author of the message /// public AuthorRole Role { get; set; } /// /// Content of the message /// public string Content { get; protected set; } /// /// Creates a new instance of the class /// /// /// Dictionary for any additional metadata public ChatContent(ChatMessage chatMessage, Dictionary? metadata = null) : base(chatMessage, metadata) { this.Role = chatMessage.Role; this.Content = chatMessage.Content; } } ``` ```csharp /// /// Represents a text content result. /// public class TextContent : ModelContent { /// /// The text content. /// public string Text { get; set; } /// /// Initializes a new instance of the class. /// /// Text content /// Additional metadata public TextContent(string text, Dictionary? metadata = null) : base(text, metadata) { this.Text = text; } } ``` ### End-User Experience - No changes to the end-user experience when using `Function.InvokeAsync` or `Kernel.InvokeAsync` - Changes only when using Connector APIs directly #### Example 16 - Custom LLMS Before ```csharp await foreach (var message in textCompletion.GetStreamingContentAsync(prompt, executionSettings)) { Console.Write(message); } ``` After ```csharp await foreach (var message in textCompletion.GetStreamingTextContentAsync(prompt, executionSettings)) { Console.Write(message); } ``` #### Example 17 - ChatGPT Before ```csharp string reply = await chatGPT.GenerateMessageAsync(chatHistory); chatHistory.AddAssistantMessage(reply); ``` After ```csharp var reply = await chatGPT.GetChatContentAsync(chatHistory); chatHistory.AddMessage(reply); // OR chatHistory.AddAssistantMessage(reply.Content); ``` ### Clean-up All old interfaces and classes will be removed in favor of the new ones.