chore: import upstream snapshot with attribution
CodeQL / Analyze (csharp) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:25 +08:00
commit db620d33df
5151 changed files with 925932 additions and 0 deletions
+3
View File
@@ -0,0 +1,3 @@
chatkit-python
openai-chatkit-advanced-samples
chatkit-js
+27
View File
@@ -0,0 +1,27 @@
# ChatKit Package (agent-framework-chatkit)
Integration with OpenAI ChatKit (Python) for building chat UIs.
## Main Classes
- **`ThreadItemConverter`** - Converts between Agent Framework and ChatKit types
- **`stream_agent_response()`** - Stream agent responses to ChatKit
- **`simple_to_agent_input()`** - Convert simple input to agent input format
## Usage
```python
from agent_framework.chatkit import stream_agent_response, ThreadItemConverter
async for event in stream_agent_response(agent, messages):
# Handle ChatKit events
pass
```
## Import Path
```python
from agent_framework.chatkit import stream_agent_response
# or directly:
from agent_framework_chatkit import stream_agent_response
```
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) Microsoft Corporation.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+127
View File
@@ -0,0 +1,127 @@
# Agent Framework and ChatKit Integration
This package provides an integration layer between Microsoft Agent Framework
and [OpenAI ChatKit (Python)](https://github.com/openai/chatkit-python/).
Specifically, it mirrors the [Agent SDK integration](https://github.com/openai/chatkit-python/blob/main/docs/server.md#agents-sdk-integration), and provides the following helpers:
- `stream_agent_response`: A helper to convert a streamed `AgentResponseUpdate`
from a Microsoft Agent Framework agent that implements `SupportsAgentRun` to ChatKit events.
- `ThreadItemConverter`: A extendable helper class to convert ChatKit thread items to
`Message` objects that can be consumed by an Agent Framework agent.
- `simple_to_agent_input`: A helper function that uses the default implementation
of `ThreadItemConverter` to convert a ChatKit thread to a list of `Message`,
useful for getting started quickly.
## Installation
```bash
pip install agent-framework-chatkit --pre
```
This will install `agent-framework-core` and `openai-chatkit` as dependencies.
## Requirements and Limitations
### Frontend Requirements
The ChatKit integration requires the OpenAI ChatKit frontend library, which has the following requirements:
1. **Internet Connectivity Required**: The ChatKit UI is loaded from OpenAI's CDN (`cdn.platform.openai.com`). This library cannot be self-hosted or bundled locally.
2. **External Network Requests**: The ChatKit frontend makes requests to:
- `cdn.platform.openai.com` - UI library (required)
- `chatgpt.com/ces/v1/projects/oai/settings` - Configuration
- `api-js.mixpanel.com` - Telemetry (metadata only, not user messages)
3. **Domain Registration for Production**: Production deployments require registering your domain at [platform.openai.com](https://platform.openai.com/settings/organization/security/domain-allowlist) and configuring a domain key.
### Air-Gapped / Regulated Environments
**The ChatKit frontend is not suitable for air-gapped or highly-regulated environments** where outbound connections to OpenAI domains are restricted.
**What IS self-hostable:**
- The backend components (`chatkit-python`, `agent-framework-chatkit`) are fully open source and have no external dependencies
**What is NOT self-hostable:**
- The frontend UI (`chatkit.js`) requires connectivity to OpenAI's CDN
For environments with network restrictions, consider building a custom frontend that consumes the ChatKit server protocol, or using alternative UI libraries like `ai-sdk`.
See [openai/chatkit-js#57](https://github.com/openai/chatkit-js/issues/57) for tracking self-hosting feature requests.
## Example Usage
Here's a minimal example showing how to integrate Agent Framework with ChatKit:
```python
from collections.abc import AsyncIterator
from typing import Any
from azure.identity import AzureCliCredential
from fastapi import FastAPI, Request
from fastapi.responses import Response, StreamingResponse
from agent_framework import Agent
from agent_framework.openai import OpenAIChatCompletionClient
from agent_framework.chatkit import simple_to_agent_input, stream_agent_response
from chatkit.server import ChatKitServer
from chatkit.types import ThreadMetadata, UserMessageItem, ThreadStreamEvent
# You'll need to implement a Store - see the sample for a SQLiteStore implementation
from your_store import YourStore # type: ignore[import-not-found] # Replace with your Store implementation
# Define your agent with tools
agent = Agent(
client=OpenAIChatCompletionClient(credential=AzureCliCredential()),
instructions="You are a helpful assistant.",
tools=[], # Add your tools here
)
# Create a ChatKit server that uses your agent
class MyChatKitServer(ChatKitServer[dict[str, Any]]):
async def respond(
self,
thread: ThreadMetadata,
input_user_message: UserMessageItem | None,
context: dict[str, Any],
) -> AsyncIterator[ThreadStreamEvent]:
if input_user_message is None:
return
# Load full thread history to maintain conversation context
thread_items_page = await self.store.load_thread_items(
thread_id=thread.id,
after=None,
limit=1000,
order="asc",
context=context,
)
# Convert all ChatKit messages to Agent Framework format
agent_messages = await simple_to_agent_input(thread_items_page.data)
# Run the agent and stream responses
response_stream = agent.run(agent_messages, stream=True)
# Convert agent responses back to ChatKit events
async for event in stream_agent_response(response_stream, thread.id):
yield event
# Set up FastAPI endpoint
app = FastAPI()
chatkit_server = MyChatKitServer(YourStore()) # type: ignore[misc]
@app.post("/chatkit")
async def chatkit_endpoint(request: Request):
result = await chatkit_server.process(await request.body(), {"request": request})
if hasattr(result, '__aiter__'): # Streaming
return StreamingResponse(result, media_type="text/event-stream") # type: ignore[arg-type]
else: # Non-streaming
return Response(content=result.json, media_type="application/json") # type: ignore[union-attr]
```
For a complete end-to-end example with a full frontend, see the [weather agent sample](../../samples/05-end-to-end/chatkit-integration/README.md).
@@ -0,0 +1,25 @@
# Copyright (c) Microsoft. All rights reserved.
"""Agent Framework and ChatKit Integration.
This package provides an integration layer between Microsoft Agent Framework
and OpenAI ChatKit (Python). It mirrors the Agent SDK integration and provides
helpers to convert between Agent Framework and ChatKit types.
"""
import importlib.metadata
from ._converter import ThreadItemConverter, simple_to_agent_input
from ._streaming import stream_agent_response
try:
__version__ = importlib.metadata.version(__name__)
except importlib.metadata.PackageNotFoundError:
__version__ = "0.0.0" # Fallback for development mode
__all__ = [
"ThreadItemConverter",
"__version__",
"simple_to_agent_input",
"stream_agent_response",
]
@@ -0,0 +1,609 @@
# Copyright (c) Microsoft. All rights reserved.
"""Converter utilities for converting ChatKit thread items to Agent Framework messages."""
from __future__ import annotations
import logging
from collections.abc import Awaitable, Callable, Sequence
from agent_framework import (
Content,
Message,
)
from chatkit.types import (
AssistantMessageItem,
Attachment,
ClientToolCallItem,
EndOfTurnItem,
GeneratedImageItem,
HiddenContextItem,
ImageAttachment,
SDKHiddenContextItem,
StructuredInputItem,
TaskItem,
ThreadItem,
UserMessageItem,
UserMessageTagContent,
UserMessageTextContent,
WidgetItem,
WorkflowItem,
)
logger = logging.getLogger(__name__)
class ThreadItemConverter:
"""Helper class to convert ChatKit thread items to Agent Framework Message objects.
This class provides a base implementation for converting ChatKit thread items
to Agent Framework messages. It can be extended to handle attachments,
@-mentions, hidden context items, and custom thread item formats.
Args:
attachment_data_fetcher: Optional async function to fetch attachment binary data.
If provided, it should take an attachment ID and return the binary data as bytes.
If not provided, attachments will be converted to UriContent using available URLs.
"""
def __init__(
self,
attachment_data_fetcher: Callable[[str], Awaitable[bytes]] | None = None,
) -> None:
"""Initialize the converter.
Args:
attachment_data_fetcher: Optional async function to fetch attachment data by ID.
"""
self.attachment_data_fetcher = attachment_data_fetcher
async def user_message_to_input(
self, item: UserMessageItem, is_last_message: bool = True
) -> Message | list[Message] | None:
"""Convert a ChatKit UserMessageItem to Agent Framework Message(s).
This method is called internally by `to_agent_input()`. Override this method
to customize how user messages are converted.
Args:
item: The ChatKit user message item to convert.
is_last_message: Whether this is the last message in the thread (used for quoted_text handling).
Returns:
A Message, list of messages, or None to skip.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types and provides proper message ordering.
"""
# Extract text content from the user message
text_content = ""
if item.content:
for content_part in item.content:
if isinstance(content_part, UserMessageTextContent):
text_content += content_part.text
# Convert attachments to Content
data_contents: list[Content] = []
if item.attachments:
for attachment in item.attachments:
content = await self.attachment_to_message_content(attachment)
if content is not None:
data_contents.append(content)
# Create the message with text and attachments
if not text_content.strip() and not data_contents:
return None
# If only text and no attachments, use text parameter for simplicity
if text_content.strip() and not data_contents:
user_message = Message(role="user", contents=[text_content.strip()])
else:
# Build contents list with both text and attachments
contents: list[Content] = []
if text_content.strip():
contents.append(Content.from_text(text=text_content.strip()))
contents.extend(data_contents)
user_message = Message(role="user", contents=contents)
# Handle quoted text if this is the last message
messages = [user_message]
if item.quoted_text and is_last_message:
quoted_context = Message(
role="user",
contents=[f"The user is referring to this in particular:\n{item.quoted_text}"],
)
# Prepend quoted context before the main message
messages.insert(0, quoted_context)
return messages
async def attachment_to_message_content(self, attachment: Attachment) -> Content | None:
"""Convert a ChatKit attachment to Agent Framework content.
This method is called internally by `user_message_to_input()` to handle attachments.
Override this method to customize attachment handling for your storage backend.
The default implementation provides two strategies:
1. If an attachment_data_fetcher was provided, it fetches the binary data
and creates a DataContent object
2. Otherwise, for ImageAttachment with preview_url, it creates a UriContent object
For FileAttachment without a data fetcher, returns None (attachment is skipped).
Args:
attachment: The ChatKit attachment to convert (FileAttachment or ImageAttachment).
Returns:
DataContent if binary data is available, UriContent if only URL is available,
or None if the attachment cannot be converted.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types including attachments within user messages.
Examples:
.. code-block:: python
# With data fetcher
async def fetch_data(attachment_id: str) -> bytes:
return await my_storage.get_file(attachment_id)
converter = ThreadItemConverter(attachment_data_fetcher=fetch_data)
messages = await converter.to_agent_input(thread_items)
# Without data fetcher (uses URLs for images)
converter = ThreadItemConverter()
messages = await converter.to_agent_input(thread_items)
"""
# If we have a data fetcher, use it to get binary data
if self.attachment_data_fetcher is not None:
try:
data = await self.attachment_data_fetcher(attachment.id)
return Content.from_data(data=data, media_type=attachment.mime_type)
except Exception as e:
# If fetch fails, fall through to URL-based approach
logger.debug(f"Failed to fetch attachment data for {attachment.id}: {e}")
# For ImageAttachment, try to use preview_url
if isinstance(attachment, ImageAttachment) and attachment.preview_url:
return Content.from_uri(uri=str(attachment.preview_url), media_type=attachment.mime_type)
# For FileAttachment without data fetcher, skip the attachment
# Subclasses can override this method to provide custom handling
return None
def hidden_context_to_input(self, item: HiddenContextItem | SDKHiddenContextItem) -> Message | list[Message] | None:
"""Convert a ChatKit HiddenContextItem or SDKHiddenContextItem to Agent Framework Message(s).
This method is called internally by `to_agent_input()`. Override this method
to customize how hidden context is converted.
The default implementation wraps the hidden context in XML tags and returns
a system message. This allows the model to distinguish hidden context from
regular conversation.
Args:
item: The ChatKit hidden context item to convert.
Returns:
A Message with system role, a list of messages, or None to skip.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types and provides proper message ordering.
Examples:
.. code-block:: python
# Default behavior
converter = ThreadItemConverter()
hidden_item = HiddenContextItem(
id="ctx_1",
thread_id="thread_1",
created_at=datetime.now(),
content="User's email: user@example.com",
)
message = converter.hidden_context_to_input(hidden_item)
# Returns: Message(role=SYSTEM, contents=["<HIDDEN_CONTEXT>User's email: ...</HIDDEN_CONTEXT>"])
"""
return Message(role="system", contents=[f"<HIDDEN_CONTEXT>{item.content}</HIDDEN_CONTEXT>"])
def tag_to_message_content(self, tag: UserMessageTagContent) -> Content:
"""Convert a ChatKit tag (@-mention) to Agent Framework content.
This method is called internally by `user_message_to_input()` to handle tags.
Override this method to customize tag conversion for your application.
The default implementation extracts the tag's display name and wraps it in
XML tags to provide context to the model about the @-mention.
Args:
tag: The ChatKit tag content to convert.
Returns:
TextContent with the tag information.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types including tags within user messages.
Examples:
.. code-block:: python
# Default behavior
converter = ThreadItemConverter()
tag = UserMessageTagContent(
type="input_tag", id="tag_1", text="john", data={"name": "John Doe"}, interactive=False
)
content = converter.tag_to_message_content(tag)
# Returns: Content.from_text(text="<TAG>Name:John Doe</TAG>")
"""
name = getattr(tag.data, "name", tag.text if hasattr(tag, "text") else "unknown")
return Content.from_text(text=f"<TAG>Name:{name}</TAG>")
def task_to_input(self, item: TaskItem) -> Message | list[Message] | None:
"""Convert a ChatKit TaskItem to Agent Framework Message(s).
This method is called internally by `to_agent_input()`. Override this method
to customize how tasks are converted.
The default implementation converts custom tasks with title/content into
a user message explaining what task was displayed to the user.
Args:
item: The ChatKit task item to convert.
Returns:
A Message, a list of messages, or None to skip the task.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types and provides proper message ordering.
Examples:
.. code-block:: python
# Task with both title and content
from chatkit.types import Task
task_item = TaskItem(
id="task_1",
thread_id="thread_1",
created_at=datetime.now(),
task=Task(type="custom", title="Data Analysis", content="Analyzed sales data"),
)
message = converter.task_to_input(task_item)
# Returns message explaining the task was performed
"""
if item.task.type != "custom" or (not item.task.title and not item.task.content):
return None
title = item.task.title or ""
content = item.task.content or ""
task_text = f"{title}: {content}" if title and content else title or content
text = (
f"A message was displayed to the user that the following task was performed:\n<Task>\n{task_text}\n</Task>"
)
return Message(role="user", contents=[text])
def workflow_to_input(self, item: WorkflowItem) -> Message | list[Message] | None:
"""Convert a ChatKit WorkflowItem to Agent Framework Message(s).
This method is called internally by `to_agent_input()`. Override this method
to customize how workflows are converted.
The default implementation converts each custom task in the workflow into
a separate user message explaining what tasks were performed.
Args:
item: The ChatKit workflow item to convert.
Returns:
A list of ChatMessages (one per task), a single message, or None to skip.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types and provides proper message ordering.
Examples:
.. code-block:: python
# Workflow with multiple tasks
from chatkit.types import Workflow, Task
workflow_item = WorkflowItem(
id="wf_1",
thread_id="thread_1",
created_at=datetime.now(),
workflow=Workflow(
type="custom",
tasks=[
Task(type="custom", title="Step 1", content="Gathered data"),
Task(type="custom", title="Step 2", content="Analyzed results"),
],
),
)
messages = converter.workflow_to_input(workflow_item)
# Returns list of messages for each task
"""
messages: list[Message] = []
for task in item.workflow.tasks:
if task.type != "custom" or (not task.title and not task.content):
continue
title = task.title or ""
content = task.content or ""
task_text = f"{title}: {content}" if title and content else title or content
text = (
"A message was displayed to the user that the following task was performed:\n"
f"<Task>\n{task_text}\n</Task>"
)
messages.append(Message(role="user", contents=[text]))
return messages if messages else None
def widget_to_input(self, item: WidgetItem) -> Message | list[Message] | None:
"""Convert a ChatKit WidgetItem to Agent Framework Message(s).
This method is called internally by `to_agent_input()`. Override this method
to customize how widgets are converted.
The default implementation converts the widget to a JSON representation
and includes it in a user message, allowing the model to understand what
UI element was displayed to the user.
Args:
item: The ChatKit widget item to convert.
Returns:
A Message describing the widget, or None to skip.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types and provides proper message ordering.
Examples:
.. code-block:: python
# Widget item
from chatkit.widgets import Card, Text
widget_item = WidgetItem(
id="widget_1",
thread_id="thread_1",
created_at=datetime.now(),
widget=Card(children=[Text(value="Hello")]),
)
message = converter.widget_to_input(widget_item)
# Returns message with JSON representation of the widget
"""
try:
widget_json = item.widget.model_dump_json(exclude_unset=True, exclude_none=True)
text = f"The following graphical UI widget (id: {item.id}) was displayed to the user:{widget_json}"
return Message(role="user", contents=[text])
except Exception:
# If JSON serialization fails, skip the widget
return None
async def assistant_message_to_input(self, item: AssistantMessageItem) -> Message | list[Message] | None:
"""Convert a ChatKit AssistantMessageItem to Agent Framework Message(s).
The default implementation extracts text from all content parts and creates
an assistant message.
Args:
item: The ChatKit assistant message item to convert.
Returns:
A Message with assistant role, or None to skip.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types and provides proper message ordering.
"""
# Extract text from all content parts
text_parts = [content.text for content in item.content]
if not text_parts:
return None
return Message(role="assistant", contents=["".join(text_parts)])
async def client_tool_call_to_input(self, item: ClientToolCallItem) -> Message | list[Message] | None:
"""Convert a ChatKit ClientToolCallItem to Agent Framework Message(s).
The default implementation converts completed tool calls into function call
and result content.
Args:
item: The ChatKit client tool call item to convert.
Returns:
A list containing function call and result messages, or None for pending calls.
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types and provides proper message ordering.
"""
if item.status == "pending":
# Skip pending tool calls - they cannot be sent to the model
return None
import json
# Create function call message
function_call_msg = Message(
role="assistant",
contents=[
Content.from_function_call(
call_id=item.call_id,
name=item.name,
arguments=json.dumps(item.arguments),
)
],
)
# Create function result message
function_result_msg = Message(
role="tool",
contents=[
Content.from_function_result(
call_id=item.call_id,
result=json.dumps(item.output) if item.output is not None else "",
)
],
)
return [function_call_msg, function_result_msg]
async def end_of_turn_to_input(self, item: EndOfTurnItem) -> Message | list[Message] | None:
"""Convert a ChatKit EndOfTurnItem to Agent Framework Message(s).
The default implementation skips end-of-turn markers as they are only UI hints.
Args:
item: The ChatKit end-of-turn item to convert.
Returns:
None (end-of-turn items are not converted).
Note:
Instead of calling this method directly, use `to_agent_input()` which handles
all ThreadItem types and provides proper message ordering.
"""
# End-of-turn is only used for UI hints - skip it
return None
async def _thread_item_to_input_item(
self,
item: ThreadItem,
is_last_message: bool = True,
) -> list[Message]:
"""Internal method to convert a single ThreadItem to Message(s).
Args:
item: The thread item to convert.
is_last_message: Whether this is the last item in the thread.
Returns:
A list of Message objects (may be empty).
"""
match item:
case UserMessageItem():
out = await self.user_message_to_input(item, is_last_message) or []
return out if isinstance(out, list) else [out]
case AssistantMessageItem():
out = await self.assistant_message_to_input(item) or []
return out if isinstance(out, list) else [out]
case ClientToolCallItem():
out = await self.client_tool_call_to_input(item) or []
return out if isinstance(out, list) else [out]
case EndOfTurnItem():
out = await self.end_of_turn_to_input(item) or []
return out if isinstance(out, list) else [out]
case WidgetItem():
out = self.widget_to_input(item) or []
return out if isinstance(out, list) else [out]
case WorkflowItem():
out = self.workflow_to_input(item) or []
return out if isinstance(out, list) else [out]
case TaskItem():
out = self.task_to_input(item) or []
return out if isinstance(out, list) else [out]
case HiddenContextItem():
out = self.hidden_context_to_input(item) or []
return out if isinstance(out, list) else [out]
case SDKHiddenContextItem():
out = self.hidden_context_to_input(item) or []
return out if isinstance(out, list) else [out]
case GeneratedImageItem():
# TODO(evmattso): Implement generated image handling in a future PR
return []
case StructuredInputItem():
# TODO(evmattso): Implement structured input handling in a future PR
return []
case _:
# Unknown ThreadItem variant (e.g. types added in newer chatkit versions).
# Skip rather than fail so we remain forward-compatible with chatkit upgrades.
logger.debug("Skipping unsupported ThreadItem of type %s", type(item).__name__)
return []
async def to_agent_input(
self,
thread_items: Sequence[ThreadItem] | ThreadItem,
) -> list[Message]:
"""Convert ChatKit thread items to Agent Framework ChatMessages.
This is the main entry point for converting ChatKit thread items. It handles
all ThreadItem types (UserMessageItem, AssistantMessageItem, TaskItem, etc.)
and calls the appropriate conversion method for each.
Args:
thread_items: A single ThreadItem or a sequence of ThreadItems to convert.
Returns:
A list of Message objects that can be sent to an Agent Framework agent.
Examples:
.. code-block:: python
from agent_framework_chatkit import ThreadItemConverter
converter = ThreadItemConverter()
# Convert a single thread item
messages = await converter.to_agent_input(user_message_item)
# Convert multiple thread items
messages = await converter.to_agent_input([user_message_item, assistant_message_item, task_item])
# Use with agent
from agent_framework import Agent
agent = Agent(...)
response = await agent.run(messages)
"""
thread_items = list(thread_items) if isinstance(thread_items, Sequence) else [thread_items]
output: list[Message] = []
for item in thread_items:
output.extend(
await self._thread_item_to_input_item(
item,
is_last_message=item is thread_items[-1],
)
)
return output
# Default converter instance
_DEFAULT_CONVERTER = ThreadItemConverter()
async def simple_to_agent_input(thread_items: Sequence[ThreadItem] | ThreadItem) -> list[Message]:
"""Helper function that uses the default ThreadItemConverter.
This function provides a quick way to get started with ChatKit integration
without needing to create a custom ThreadItemConverter instance.
Args:
thread_items: A single ThreadItem or a sequence of ThreadItems to convert.
Returns:
A list of Message objects that can be sent to an Agent Framework agent.
Examples:
.. code-block:: python
from agent_framework_chatkit import simple_to_agent_input
# Convert a single item
messages = await simple_to_agent_input(user_message_item)
# Convert multiple items
messages = await simple_to_agent_input([user_message_item, assistant_message_item, task_item])
"""
return await _DEFAULT_CONVERTER.to_agent_input(thread_items)
@@ -0,0 +1,104 @@
# Copyright (c) Microsoft. All rights reserved.
"""Streaming utilities for converting Agent Framework responses to ChatKit events."""
import uuid
from collections.abc import AsyncIterable, AsyncIterator, Callable
from datetime import datetime
from agent_framework import AgentResponseUpdate
from chatkit.types import (
AssistantMessageContent,
AssistantMessageContentPartTextDelta,
AssistantMessageItem,
ThreadItemAddedEvent,
ThreadItemDoneEvent,
ThreadItemUpdated,
ThreadStreamEvent,
)
async def stream_agent_response(
response_stream: AsyncIterable[AgentResponseUpdate],
thread_id: str,
generate_id: Callable[[str], str] | None = None,
) -> AsyncIterator[ThreadStreamEvent]:
"""Convert a streamed AgentResponseUpdate from Agent Framework to ChatKit events.
This helper function takes a stream of AgentResponseUpdate objects from
a Microsoft Agent Framework agent and converts them to ChatKit ThreadStreamEvent
objects that can be consumed by the ChatKit UI.
The function supports real-time token-by-token streaming by emitting
ThreadItemUpdated events with AssistantMessageContentPartTextDelta for each
text chunk as it arrives from the agent.
Args:
response_stream: An async iterable of AgentResponseUpdate objects
from an Agent Framework agent.
thread_id: The ChatKit thread ID for the conversation.
generate_id: Optional function to generate IDs for ChatKit items.
If not provided, simple incremental IDs will be used.
Yields:
ThreadStreamEvent: ChatKit events representing the agent's response,
including incremental text deltas for streaming display.
"""
# Use provided ID generator or create default one
if generate_id is None:
def _default_id_generator(item_type: str) -> str:
return f"{item_type}_{uuid.uuid4().hex[:8]}"
message_id = _default_id_generator("msg")
else:
message_id = generate_id("msg")
# Track if we've started the message
message_started = False
accumulated_text = ""
content_index = 0
async for update in response_stream:
# Start the assistant message if not already started
if not message_started:
assistant_message = AssistantMessageItem(
id=message_id,
thread_id=thread_id,
type="assistant_message",
content=[],
created_at=datetime.now(),
)
yield ThreadItemAddedEvent(type="thread.item.added", item=assistant_message)
message_started = True
# Process the update content
if update.contents:
for content in update.contents:
# Handle text content - only TextContent has a text attribute
if content.type == "text" and content.text is not None:
# Yield incremental text delta for streaming display
yield ThreadItemUpdated(
type="thread.item.updated",
item_id=message_id,
update=AssistantMessageContentPartTextDelta(
content_index=content_index,
delta=content.text,
),
)
accumulated_text += content.text
# Finalize the message
if message_started:
final_message = AssistantMessageItem(
id=message_id,
thread_id=thread_id,
type="assistant_message",
content=[AssistantMessageContent(type="output_text", text=accumulated_text, annotations=[])]
if accumulated_text
else [],
created_at=datetime.now(),
)
yield ThreadItemDoneEvent(type="thread.item.done", item=final_message)
+99
View File
@@ -0,0 +1,99 @@
[project]
name = "agent-framework-chatkit"
description = "OpenAI ChatKit integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260528"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
urls.release_notes = "https://github.com/microsoft/agent-framework/releases?q=tag%3Apython-1&expanded=true"
urls.issues = "https://github.com/microsoft/agent-framework/issues"
classifiers = [
"License :: OSI Approved :: MIT License",
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.7.0,<2",
"openai-chatkit>=1.6.4,<2.0.0",
]
[tool.uv]
prerelease = "if-necessary-or-explicit"
environments = [
"sys_platform == 'darwin'",
"sys_platform == 'linux'",
"sys_platform == 'win32'"
]
[tool.uv-dynamic-versioning]
fallback-version = "0.0.0"
[tool.pytest.ini_options]
testpaths = 'tests'
addopts = "-ra -q -r fEX"
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
filterwarnings = []
timeout = 120
markers = [
"integration: marks tests as integration tests that require external services",
]
[tool.ruff]
extend = "../../pyproject.toml"
[tool.ruff.lint]
ignore = ["RUF029"]
[tool.coverage.run]
omit = [
"**/__init__.py"
]
[tool.pyright]
extends = "../../pyproject.toml"
include = ["agent_framework_chatkit"]
exclude = ['tests', 'chatkit-python', 'openai-chatkit-advanced-samples']
[tool.mypy]
plugins = ['pydantic.mypy']
strict = true
python_version = "3.10"
ignore_missing_imports = true
disallow_untyped_defs = true
no_implicit_optional = true
check_untyped_defs = true
warn_return_any = true
show_error_codes = true
warn_unused_ignores = false
disallow_incomplete_defs = true
disallow_untyped_decorators = true
[tool.bandit]
targets = ["agent_framework_chatkit"]
exclude_dirs = ["tests"]
[tool.poe]
executor.type = "uv"
include = "../../shared_tasks.toml"
[tool.poe.tasks.mypy]
help = "Run MyPy for this package."
cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_chatkit"
[tool.poe.tasks.test]
help = "Run the default unit test suite for this package."
cmd = 'pytest -m "not integration" --cov=agent_framework_chatkit --cov-report=term-missing:skip-covered tests'
[build-system]
requires = ["flit-core >= 3.11,<4.0"]
build-backend = "flit_core.buildapi"
@@ -0,0 +1,424 @@
# Copyright (c) Microsoft. All rights reserved.
"""Tests for ChatKit to Agent Framework converter utilities."""
from unittest.mock import Mock
import pytest
from agent_framework import Message
from chatkit.types import InferenceOptions, UserMessageTextContent
from pydantic import AnyUrl
from agent_framework_chatkit import ThreadItemConverter, simple_to_agent_input
class TestThreadItemConverter:
"""Tests for ThreadItemConverter class."""
@pytest.fixture
def converter(self):
"""Create a ThreadItemConverter instance for testing."""
return ThreadItemConverter()
async def test_to_agent_input_none(self, converter):
"""Test converting empty list returns empty list."""
result = await converter.to_agent_input([])
assert result == []
async def test_to_agent_input_with_text(self, converter):
"""Test converting user message with text content."""
from datetime import datetime
from chatkit.types import UserMessageItem
input_item = UserMessageItem(
id="msg_1",
thread_id="thread_1",
created_at=datetime.now(),
type="user_message",
content=[UserMessageTextContent(text="Hello, how can you help me?")],
attachments=[],
inference_options=InferenceOptions(),
)
result = await converter.to_agent_input(input_item)
assert len(result) == 1
assert isinstance(result[0], Message)
assert result[0].role == "user"
assert result[0].text == "Hello, how can you help me?"
async def test_to_agent_input_empty_text(self, converter):
"""Test converting user message with empty or whitespace-only text."""
from datetime import datetime
from chatkit.types import UserMessageItem
input_item = UserMessageItem(
id="msg_1",
thread_id="thread_1",
created_at=datetime.now(),
type="user_message",
content=[UserMessageTextContent(text=" ")],
attachments=[],
inference_options=InferenceOptions(),
)
result = await converter.to_agent_input(input_item)
assert result == []
async def test_to_agent_input_no_content(self, converter):
"""Test converting user message with no content."""
from datetime import datetime
from chatkit.types import UserMessageItem
input_item = UserMessageItem(
id="msg_1",
thread_id="thread_1",
created_at=datetime.now(),
type="user_message",
content=[],
attachments=[],
inference_options=InferenceOptions(),
)
result = await converter.to_agent_input(input_item)
assert result == []
async def test_to_agent_input_multiple_content_parts(self, converter):
"""Test converting user message with multiple text content parts."""
from datetime import datetime
from chatkit.types import UserMessageItem
input_item = UserMessageItem(
id="msg_1",
thread_id="thread_1",
created_at=datetime.now(),
type="user_message",
content=[
UserMessageTextContent(text="Hello "),
UserMessageTextContent(text="world!"),
],
attachments=[],
inference_options=InferenceOptions(),
)
result = await converter.to_agent_input(input_item)
assert len(result) == 1
assert result[0].text == "Hello world!"
def test_hidden_context_to_input(self, converter):
"""Test converting hidden context item to Message."""
hidden_item = Mock()
hidden_item.content = "This is hidden context information"
result = converter.hidden_context_to_input(hidden_item)
assert isinstance(result, Message)
assert result.role == "system"
assert result.text == "<HIDDEN_CONTEXT>This is hidden context information</HIDDEN_CONTEXT>"
def test_tag_to_message_content(self, converter):
"""Test converting tag to message content."""
from chatkit.types import UserMessageTagContent
tag = UserMessageTagContent(
type="input_tag",
id="tag_1",
text="john",
data={"name": "John Doe"},
interactive=False,
)
result = converter.tag_to_message_content(tag)
assert result.type == "text"
# Since data is a dict, getattr won't work, so it will fall back to text
assert result.text == "<TAG>Name:john</TAG>"
def test_tag_to_message_content_no_name(self, converter):
"""Test converting tag with no name to message content."""
from chatkit.types import UserMessageTagContent
tag = UserMessageTagContent(
type="input_tag",
id="tag_2",
text="jane",
data={},
interactive=False,
)
result = converter.tag_to_message_content(tag)
assert result.type == "text"
assert result.text == "<TAG>Name:jane</TAG>"
async def test_attachment_to_message_content_file_without_fetcher(self, converter):
"""Test that FileAttachment without data fetcher returns None."""
from chatkit.types import FileAttachment
attachment = FileAttachment(
id="file_123",
name="document.pdf",
mime_type="application/pdf",
type="file",
)
result = await converter.attachment_to_message_content(attachment)
assert result is None
async def test_attachment_to_message_content_image_with_preview_url(self, converter):
"""Test that ImageAttachment with preview_url creates UriContent."""
from chatkit.types import ImageAttachment
attachment = ImageAttachment(
id="img_123",
name="photo.jpg",
mime_type="image/jpeg",
type="image",
preview_url=AnyUrl("https://example.com/photo.jpg"),
)
result = await converter.attachment_to_message_content(attachment)
assert result.type == "uri"
assert result.uri == "https://example.com/photo.jpg"
assert result.media_type == "image/jpeg"
async def test_attachment_to_message_content_with_data_fetcher(self):
"""Test attachment conversion with data fetcher."""
from chatkit.types import FileAttachment
# Mock data fetcher
async def fetch_data(attachment_id: str) -> bytes:
return b"file content data"
converter = ThreadItemConverter(attachment_data_fetcher=fetch_data)
attachment = FileAttachment(
id="file_123",
name="document.pdf",
mime_type="application/pdf",
type="file",
)
result = await converter.attachment_to_message_content(attachment)
assert result is not None
assert result.type == "data"
assert result.media_type == "application/pdf"
async def test_to_agent_input_with_image_attachment(self):
"""Test converting user message with text and image attachment."""
from datetime import datetime
from chatkit.types import ImageAttachment, UserMessageItem
attachment = ImageAttachment(
id="img_123",
name="photo.jpg",
mime_type="image/jpeg",
type="image",
preview_url=AnyUrl("https://example.com/photo.jpg"),
)
input_item = UserMessageItem(
id="msg_1",
thread_id="thread_1",
created_at=datetime.now(),
type="user_message",
content=[UserMessageTextContent(text="Check out this photo!")],
attachments=[attachment],
inference_options=InferenceOptions(),
)
converter = ThreadItemConverter()
result = await converter.to_agent_input(input_item)
assert len(result) == 1
message = result[0]
assert message.role == "user"
assert len(message.contents) == 2
# First content should be text
assert message.contents[0].type == "text"
assert message.contents[0].text == "Check out this photo!"
# Second content should be UriContent for the image
assert message.contents[1].type == "uri"
assert message.contents[1].uri == "https://example.com/photo.jpg"
assert message.contents[1].media_type == "image/jpeg"
async def test_to_agent_input_with_file_attachment_and_fetcher(self):
"""Test converting user message with file attachment using data fetcher."""
from datetime import datetime
from chatkit.types import FileAttachment, UserMessageItem
attachment = FileAttachment(
id="file_123",
name="report.pdf",
mime_type="application/pdf",
type="file",
)
input_item = UserMessageItem(
id="msg_1",
thread_id="thread_1",
created_at=datetime.now(),
type="user_message",
content=[UserMessageTextContent(text="Here's the document")],
attachments=[attachment],
inference_options=InferenceOptions(),
)
# Create converter with data fetcher
async def fetch_data(attachment_id: str) -> bytes:
return b"PDF content data"
converter = ThreadItemConverter(attachment_data_fetcher=fetch_data)
result = await converter.to_agent_input(input_item)
assert len(result) == 1
message = result[0]
assert len(message.contents) == 2
# First content should be text
assert message.contents[0].type == "text"
# Second content should be DataContent for the file
assert message.contents[1].type == "data"
assert message.contents[1].media_type == "application/pdf"
def test_task_to_input(self, converter):
"""Test converting TaskItem to Message."""
from datetime import datetime
from chatkit.types import CustomTask, TaskItem
task_item = TaskItem(
id="task_1",
thread_id="thread_1",
created_at=datetime.now(),
type="task",
task=CustomTask(type="custom", title="Analysis", content="Analyzed the data"),
)
result = converter.task_to_input(task_item)
assert isinstance(result, Message)
assert result.role == "user"
assert "Analysis: Analyzed the data" in result.text
assert "<Task>" in result.text
def test_task_to_input_no_custom_task(self, converter):
"""Test that non-custom tasks return None."""
from datetime import datetime
from chatkit.types import TaskItem, ThoughtTask
task_item = TaskItem(
id="task_1",
thread_id="thread_1",
created_at=datetime.now(),
type="task",
task=ThoughtTask(type="thought", title="Think", content="Thinking..."),
)
result = converter.task_to_input(task_item)
assert result is None
def test_workflow_to_input(self, converter):
"""Test converting WorkflowItem to ChatMessages."""
from datetime import datetime
from chatkit.types import CustomTask, Workflow, WorkflowItem
workflow_item = WorkflowItem(
id="wf_1",
thread_id="thread_1",
created_at=datetime.now(),
type="workflow",
workflow=Workflow(
type="custom",
tasks=[
CustomTask(type="custom", title="Step 1", content="First step"),
CustomTask(type="custom", title="Step 2", content="Second step"),
],
),
)
result = converter.workflow_to_input(workflow_item)
assert isinstance(result, list)
assert len(result) == 2
assert all(isinstance(msg, Message) for msg in result)
assert "Step 1: First step" in result[0].text
assert "Step 2: Second step" in result[1].text
def test_workflow_to_input_empty(self, converter):
"""Test that workflows with no custom tasks return None."""
from datetime import datetime
from chatkit.types import Workflow, WorkflowItem
workflow_item = WorkflowItem(
id="wf_1",
thread_id="thread_1",
created_at=datetime.now(),
type="workflow",
workflow=Workflow(type="custom", tasks=[]),
)
result = converter.workflow_to_input(workflow_item)
assert result is None
def test_widget_to_input(self, converter):
"""Test converting WidgetItem to Message."""
from datetime import datetime
from chatkit.types import WidgetItem
from chatkit.widgets import Card, Text # ty: ignore[deprecated]
widget_item = WidgetItem(
id="widget_1",
thread_id="thread_1",
created_at=datetime.now(),
type="widget",
widget=Card(key="card1", children=[Text(value="Hello")]), # ty: ignore[deprecated]
)
result = converter.widget_to_input(widget_item)
assert isinstance(result, Message)
assert result.role == "user"
assert "widget_1" in result.text
assert "graphical UI widget" in result.text
class TestSimpleToAgentInput:
"""Tests for simple_to_agent_input helper function."""
async def test_simple_to_agent_input_empty_list(self):
"""Test simple conversion with empty list."""
result = await simple_to_agent_input([])
assert result == []
async def test_simple_to_agent_input_with_text(self):
"""Test simple conversion with text content."""
from datetime import datetime
from chatkit.types import UserMessageItem
input_item = UserMessageItem(
id="msg_1",
thread_id="thread_1",
created_at=datetime.now(),
type="user_message",
content=[UserMessageTextContent(text="Test message")],
attachments=[],
inference_options=InferenceOptions(),
)
result = await simple_to_agent_input(input_item)
assert len(result) == 1
assert isinstance(result[0], Message)
assert result[0].role == "user"
assert result[0].text == "Test message"
@@ -0,0 +1,156 @@
# Copyright (c) Microsoft. All rights reserved.
"""Tests for Agent Framework to ChatKit streaming utilities."""
from collections.abc import AsyncIterator
from unittest.mock import Mock
from agent_framework import AgentResponseUpdate, Content
from chatkit.types import (
AssistantMessageContent,
AssistantMessageContentPartTextDelta,
AssistantMessageItem,
ThreadItemAddedEvent,
ThreadItemDoneEvent,
ThreadItemUpdated,
)
from agent_framework_chatkit import stream_agent_response
class TestStreamAgentResponse:
"""Tests for stream_agent_response function."""
async def test_stream_empty_response(self):
"""Test streaming empty response."""
async def empty_stream() -> AsyncIterator[AgentResponseUpdate]:
return
yield # Make it a generator
events = []
async for event in stream_agent_response(empty_stream(), thread_id="test_thread"):
events.append(event)
assert len(events) == 0
async def test_stream_single_text_update(self):
"""Test streaming single text update."""
async def single_update_stream():
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Hello world")])
events = []
async for event in stream_agent_response(single_update_stream(), thread_id="test_thread"):
events.append(event)
# Should have: item_added, item_updated (delta), item_done
assert len(events) == 3
# Check event types
assert isinstance(events[0], ThreadItemAddedEvent)
assert isinstance(events[1], ThreadItemUpdated)
assert isinstance(events[2], ThreadItemDoneEvent)
# Check delta event
assert isinstance(events[1].update, AssistantMessageContentPartTextDelta)
assert events[1].update.delta == "Hello world"
# Check final message content
assert isinstance(events[2].item, AssistantMessageItem)
assert len(events[2].item.content) == 1
assert isinstance(events[2].item.content[0], AssistantMessageContent)
assert events[2].item.content[0].text == "Hello world"
async def test_stream_multiple_text_updates(self):
"""Test streaming multiple text updates."""
async def multiple_updates_stream():
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Hello ")])
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="world!")])
events = []
async for event in stream_agent_response(multiple_updates_stream(), thread_id="test_thread"):
events.append(event)
# Should have: item_added, item_updated (delta 1), item_updated (delta 2), item_done
assert len(events) == 4
# Check event types
assert isinstance(events[0], ThreadItemAddedEvent)
assert isinstance(events[1], ThreadItemUpdated)
assert isinstance(events[2], ThreadItemUpdated)
assert isinstance(events[3], ThreadItemDoneEvent)
# Check delta events
assert isinstance(events[1].update, AssistantMessageContentPartTextDelta)
assert isinstance(events[2].update, AssistantMessageContentPartTextDelta)
assert events[1].update.delta == "Hello "
assert events[2].update.delta == "world!"
# Check final accumulated text
final_message_event = events[-1]
assert isinstance(final_message_event, ThreadItemDoneEvent)
assert isinstance(final_message_event.item, AssistantMessageItem)
assert isinstance(final_message_event.item.content[0], AssistantMessageContent)
assert final_message_event.item.content[0].text == "Hello world!"
async def test_stream_with_custom_id_generator(self):
"""Test streaming with custom ID generator."""
def custom_id_generator(item_type: str) -> str:
return f"custom_{item_type}_123"
async def single_update_stream():
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Test")])
events = []
async for event in stream_agent_response(
single_update_stream(), thread_id="test_thread", generate_id=custom_id_generator
):
events.append(event)
# Check that custom IDs are used
message_added_event = events[0]
assert isinstance(message_added_event, ThreadItemAddedEvent)
assert message_added_event.item.id == "custom_msg_123"
async def test_stream_empty_content_updates(self):
"""Test streaming updates with empty content."""
async def empty_content_stream():
yield AgentResponseUpdate(role="assistant", contents=[])
yield AgentResponseUpdate(role="assistant", contents=None)
events = []
async for event in stream_agent_response(empty_content_stream(), thread_id="test_thread"):
events.append(event)
# Should have item_added and item_done
assert len(events) == 2
assert isinstance(events[0], ThreadItemAddedEvent)
assert isinstance(events[1], ThreadItemDoneEvent)
# Final message should have empty content
assert isinstance(events[1].item, AssistantMessageItem)
assert len(events[1].item.content) == 0
async def test_stream_non_text_content(self):
"""Test streaming updates with non-text content."""
# Mock a content object without text attribute
non_text_content = Mock(spec=Content)
non_text_content.type = "image"
# Don't set text attribute
non_text_content.text = None
async def non_text_stream():
yield AgentResponseUpdate(role="assistant", contents=[non_text_content])
events = []
async for event in stream_agent_response(non_text_stream(), thread_id="test_thread"):
events.append(event)
# Should have item_added and item_done, but no content since no text
assert len(events) == 2
assert isinstance(events[0], ThreadItemAddedEvent)
assert isinstance(events[1], ThreadItemDoneEvent)