chore: import upstream snapshot with attribution
CodeQL / Analyze (csharp) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
CodeQL / Analyze (csharp) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,3 @@
|
||||
chatkit-python
|
||||
openai-chatkit-advanced-samples
|
||||
chatkit-js
|
||||
@@ -0,0 +1,27 @@
|
||||
# ChatKit Package (agent-framework-chatkit)
|
||||
|
||||
Integration with OpenAI ChatKit (Python) for building chat UIs.
|
||||
|
||||
## Main Classes
|
||||
|
||||
- **`ThreadItemConverter`** - Converts between Agent Framework and ChatKit types
|
||||
- **`stream_agent_response()`** - Stream agent responses to ChatKit
|
||||
- **`simple_to_agent_input()`** - Convert simple input to agent input format
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from agent_framework.chatkit import stream_agent_response, ThreadItemConverter
|
||||
|
||||
async for event in stream_agent_response(agent, messages):
|
||||
# Handle ChatKit events
|
||||
pass
|
||||
```
|
||||
|
||||
## Import Path
|
||||
|
||||
```python
|
||||
from agent_framework.chatkit import stream_agent_response
|
||||
# or directly:
|
||||
from agent_framework_chatkit import stream_agent_response
|
||||
```
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) Microsoft Corporation.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,127 @@
|
||||
# Agent Framework and ChatKit Integration
|
||||
|
||||
This package provides an integration layer between Microsoft Agent Framework
|
||||
and [OpenAI ChatKit (Python)](https://github.com/openai/chatkit-python/).
|
||||
Specifically, it mirrors the [Agent SDK integration](https://github.com/openai/chatkit-python/blob/main/docs/server.md#agents-sdk-integration), and provides the following helpers:
|
||||
|
||||
- `stream_agent_response`: A helper to convert a streamed `AgentResponseUpdate`
|
||||
from a Microsoft Agent Framework agent that implements `SupportsAgentRun` to ChatKit events.
|
||||
- `ThreadItemConverter`: A extendable helper class to convert ChatKit thread items to
|
||||
`Message` objects that can be consumed by an Agent Framework agent.
|
||||
- `simple_to_agent_input`: A helper function that uses the default implementation
|
||||
of `ThreadItemConverter` to convert a ChatKit thread to a list of `Message`,
|
||||
useful for getting started quickly.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install agent-framework-chatkit --pre
|
||||
```
|
||||
|
||||
This will install `agent-framework-core` and `openai-chatkit` as dependencies.
|
||||
|
||||
## Requirements and Limitations
|
||||
|
||||
### Frontend Requirements
|
||||
|
||||
The ChatKit integration requires the OpenAI ChatKit frontend library, which has the following requirements:
|
||||
|
||||
1. **Internet Connectivity Required**: The ChatKit UI is loaded from OpenAI's CDN (`cdn.platform.openai.com`). This library cannot be self-hosted or bundled locally.
|
||||
|
||||
2. **External Network Requests**: The ChatKit frontend makes requests to:
|
||||
- `cdn.platform.openai.com` - UI library (required)
|
||||
- `chatgpt.com/ces/v1/projects/oai/settings` - Configuration
|
||||
- `api-js.mixpanel.com` - Telemetry (metadata only, not user messages)
|
||||
|
||||
3. **Domain Registration for Production**: Production deployments require registering your domain at [platform.openai.com](https://platform.openai.com/settings/organization/security/domain-allowlist) and configuring a domain key.
|
||||
|
||||
### Air-Gapped / Regulated Environments
|
||||
|
||||
**The ChatKit frontend is not suitable for air-gapped or highly-regulated environments** where outbound connections to OpenAI domains are restricted.
|
||||
|
||||
**What IS self-hostable:**
|
||||
|
||||
- The backend components (`chatkit-python`, `agent-framework-chatkit`) are fully open source and have no external dependencies
|
||||
|
||||
**What is NOT self-hostable:**
|
||||
|
||||
- The frontend UI (`chatkit.js`) requires connectivity to OpenAI's CDN
|
||||
|
||||
For environments with network restrictions, consider building a custom frontend that consumes the ChatKit server protocol, or using alternative UI libraries like `ai-sdk`.
|
||||
|
||||
See [openai/chatkit-js#57](https://github.com/openai/chatkit-js/issues/57) for tracking self-hosting feature requests.
|
||||
|
||||
## Example Usage
|
||||
|
||||
Here's a minimal example showing how to integrate Agent Framework with ChatKit:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncIterator
|
||||
from typing import Any
|
||||
|
||||
from azure.identity import AzureCliCredential
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.responses import Response, StreamingResponse
|
||||
|
||||
from agent_framework import Agent
|
||||
from agent_framework.openai import OpenAIChatCompletionClient
|
||||
from agent_framework.chatkit import simple_to_agent_input, stream_agent_response
|
||||
|
||||
from chatkit.server import ChatKitServer
|
||||
from chatkit.types import ThreadMetadata, UserMessageItem, ThreadStreamEvent
|
||||
|
||||
# You'll need to implement a Store - see the sample for a SQLiteStore implementation
|
||||
from your_store import YourStore # type: ignore[import-not-found] # Replace with your Store implementation
|
||||
|
||||
# Define your agent with tools
|
||||
agent = Agent(
|
||||
client=OpenAIChatCompletionClient(credential=AzureCliCredential()),
|
||||
instructions="You are a helpful assistant.",
|
||||
tools=[], # Add your tools here
|
||||
)
|
||||
|
||||
# Create a ChatKit server that uses your agent
|
||||
class MyChatKitServer(ChatKitServer[dict[str, Any]]):
|
||||
async def respond(
|
||||
self,
|
||||
thread: ThreadMetadata,
|
||||
input_user_message: UserMessageItem | None,
|
||||
context: dict[str, Any],
|
||||
) -> AsyncIterator[ThreadStreamEvent]:
|
||||
if input_user_message is None:
|
||||
return
|
||||
|
||||
# Load full thread history to maintain conversation context
|
||||
thread_items_page = await self.store.load_thread_items(
|
||||
thread_id=thread.id,
|
||||
after=None,
|
||||
limit=1000,
|
||||
order="asc",
|
||||
context=context,
|
||||
)
|
||||
|
||||
# Convert all ChatKit messages to Agent Framework format
|
||||
agent_messages = await simple_to_agent_input(thread_items_page.data)
|
||||
|
||||
# Run the agent and stream responses
|
||||
response_stream = agent.run(agent_messages, stream=True)
|
||||
|
||||
# Convert agent responses back to ChatKit events
|
||||
async for event in stream_agent_response(response_stream, thread.id):
|
||||
yield event
|
||||
|
||||
# Set up FastAPI endpoint
|
||||
app = FastAPI()
|
||||
chatkit_server = MyChatKitServer(YourStore()) # type: ignore[misc]
|
||||
|
||||
@app.post("/chatkit")
|
||||
async def chatkit_endpoint(request: Request):
|
||||
result = await chatkit_server.process(await request.body(), {"request": request})
|
||||
|
||||
if hasattr(result, '__aiter__'): # Streaming
|
||||
return StreamingResponse(result, media_type="text/event-stream") # type: ignore[arg-type]
|
||||
else: # Non-streaming
|
||||
return Response(content=result.json, media_type="application/json") # type: ignore[union-attr]
|
||||
```
|
||||
|
||||
For a complete end-to-end example with a full frontend, see the [weather agent sample](../../samples/05-end-to-end/chatkit-integration/README.md).
|
||||
@@ -0,0 +1,25 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Agent Framework and ChatKit Integration.
|
||||
|
||||
This package provides an integration layer between Microsoft Agent Framework
|
||||
and OpenAI ChatKit (Python). It mirrors the Agent SDK integration and provides
|
||||
helpers to convert between Agent Framework and ChatKit types.
|
||||
"""
|
||||
|
||||
import importlib.metadata
|
||||
|
||||
from ._converter import ThreadItemConverter, simple_to_agent_input
|
||||
from ._streaming import stream_agent_response
|
||||
|
||||
try:
|
||||
__version__ = importlib.metadata.version(__name__)
|
||||
except importlib.metadata.PackageNotFoundError:
|
||||
__version__ = "0.0.0" # Fallback for development mode
|
||||
|
||||
__all__ = [
|
||||
"ThreadItemConverter",
|
||||
"__version__",
|
||||
"simple_to_agent_input",
|
||||
"stream_agent_response",
|
||||
]
|
||||
@@ -0,0 +1,609 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Converter utilities for converting ChatKit thread items to Agent Framework messages."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Awaitable, Callable, Sequence
|
||||
|
||||
from agent_framework import (
|
||||
Content,
|
||||
Message,
|
||||
)
|
||||
from chatkit.types import (
|
||||
AssistantMessageItem,
|
||||
Attachment,
|
||||
ClientToolCallItem,
|
||||
EndOfTurnItem,
|
||||
GeneratedImageItem,
|
||||
HiddenContextItem,
|
||||
ImageAttachment,
|
||||
SDKHiddenContextItem,
|
||||
StructuredInputItem,
|
||||
TaskItem,
|
||||
ThreadItem,
|
||||
UserMessageItem,
|
||||
UserMessageTagContent,
|
||||
UserMessageTextContent,
|
||||
WidgetItem,
|
||||
WorkflowItem,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ThreadItemConverter:
|
||||
"""Helper class to convert ChatKit thread items to Agent Framework Message objects.
|
||||
|
||||
This class provides a base implementation for converting ChatKit thread items
|
||||
to Agent Framework messages. It can be extended to handle attachments,
|
||||
@-mentions, hidden context items, and custom thread item formats.
|
||||
|
||||
Args:
|
||||
attachment_data_fetcher: Optional async function to fetch attachment binary data.
|
||||
If provided, it should take an attachment ID and return the binary data as bytes.
|
||||
If not provided, attachments will be converted to UriContent using available URLs.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
attachment_data_fetcher: Callable[[str], Awaitable[bytes]] | None = None,
|
||||
) -> None:
|
||||
"""Initialize the converter.
|
||||
|
||||
Args:
|
||||
attachment_data_fetcher: Optional async function to fetch attachment data by ID.
|
||||
"""
|
||||
self.attachment_data_fetcher = attachment_data_fetcher
|
||||
|
||||
async def user_message_to_input(
|
||||
self, item: UserMessageItem, is_last_message: bool = True
|
||||
) -> Message | list[Message] | None:
|
||||
"""Convert a ChatKit UserMessageItem to Agent Framework Message(s).
|
||||
|
||||
This method is called internally by `to_agent_input()`. Override this method
|
||||
to customize how user messages are converted.
|
||||
|
||||
Args:
|
||||
item: The ChatKit user message item to convert.
|
||||
is_last_message: Whether this is the last message in the thread (used for quoted_text handling).
|
||||
|
||||
Returns:
|
||||
A Message, list of messages, or None to skip.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types and provides proper message ordering.
|
||||
"""
|
||||
# Extract text content from the user message
|
||||
text_content = ""
|
||||
if item.content:
|
||||
for content_part in item.content:
|
||||
if isinstance(content_part, UserMessageTextContent):
|
||||
text_content += content_part.text
|
||||
|
||||
# Convert attachments to Content
|
||||
data_contents: list[Content] = []
|
||||
if item.attachments:
|
||||
for attachment in item.attachments:
|
||||
content = await self.attachment_to_message_content(attachment)
|
||||
if content is not None:
|
||||
data_contents.append(content)
|
||||
|
||||
# Create the message with text and attachments
|
||||
if not text_content.strip() and not data_contents:
|
||||
return None
|
||||
|
||||
# If only text and no attachments, use text parameter for simplicity
|
||||
if text_content.strip() and not data_contents:
|
||||
user_message = Message(role="user", contents=[text_content.strip()])
|
||||
else:
|
||||
# Build contents list with both text and attachments
|
||||
contents: list[Content] = []
|
||||
if text_content.strip():
|
||||
contents.append(Content.from_text(text=text_content.strip()))
|
||||
contents.extend(data_contents)
|
||||
user_message = Message(role="user", contents=contents)
|
||||
|
||||
# Handle quoted text if this is the last message
|
||||
messages = [user_message]
|
||||
if item.quoted_text and is_last_message:
|
||||
quoted_context = Message(
|
||||
role="user",
|
||||
contents=[f"The user is referring to this in particular:\n{item.quoted_text}"],
|
||||
)
|
||||
# Prepend quoted context before the main message
|
||||
messages.insert(0, quoted_context)
|
||||
|
||||
return messages
|
||||
|
||||
async def attachment_to_message_content(self, attachment: Attachment) -> Content | None:
|
||||
"""Convert a ChatKit attachment to Agent Framework content.
|
||||
|
||||
This method is called internally by `user_message_to_input()` to handle attachments.
|
||||
Override this method to customize attachment handling for your storage backend.
|
||||
|
||||
The default implementation provides two strategies:
|
||||
1. If an attachment_data_fetcher was provided, it fetches the binary data
|
||||
and creates a DataContent object
|
||||
2. Otherwise, for ImageAttachment with preview_url, it creates a UriContent object
|
||||
|
||||
For FileAttachment without a data fetcher, returns None (attachment is skipped).
|
||||
|
||||
Args:
|
||||
attachment: The ChatKit attachment to convert (FileAttachment or ImageAttachment).
|
||||
|
||||
Returns:
|
||||
DataContent if binary data is available, UriContent if only URL is available,
|
||||
or None if the attachment cannot be converted.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types including attachments within user messages.
|
||||
|
||||
Examples:
|
||||
.. code-block:: python
|
||||
|
||||
# With data fetcher
|
||||
async def fetch_data(attachment_id: str) -> bytes:
|
||||
return await my_storage.get_file(attachment_id)
|
||||
|
||||
|
||||
converter = ThreadItemConverter(attachment_data_fetcher=fetch_data)
|
||||
messages = await converter.to_agent_input(thread_items)
|
||||
|
||||
# Without data fetcher (uses URLs for images)
|
||||
converter = ThreadItemConverter()
|
||||
messages = await converter.to_agent_input(thread_items)
|
||||
"""
|
||||
# If we have a data fetcher, use it to get binary data
|
||||
if self.attachment_data_fetcher is not None:
|
||||
try:
|
||||
data = await self.attachment_data_fetcher(attachment.id)
|
||||
return Content.from_data(data=data, media_type=attachment.mime_type)
|
||||
except Exception as e:
|
||||
# If fetch fails, fall through to URL-based approach
|
||||
logger.debug(f"Failed to fetch attachment data for {attachment.id}: {e}")
|
||||
|
||||
# For ImageAttachment, try to use preview_url
|
||||
if isinstance(attachment, ImageAttachment) and attachment.preview_url:
|
||||
return Content.from_uri(uri=str(attachment.preview_url), media_type=attachment.mime_type)
|
||||
|
||||
# For FileAttachment without data fetcher, skip the attachment
|
||||
# Subclasses can override this method to provide custom handling
|
||||
return None
|
||||
|
||||
def hidden_context_to_input(self, item: HiddenContextItem | SDKHiddenContextItem) -> Message | list[Message] | None:
|
||||
"""Convert a ChatKit HiddenContextItem or SDKHiddenContextItem to Agent Framework Message(s).
|
||||
|
||||
This method is called internally by `to_agent_input()`. Override this method
|
||||
to customize how hidden context is converted.
|
||||
|
||||
The default implementation wraps the hidden context in XML tags and returns
|
||||
a system message. This allows the model to distinguish hidden context from
|
||||
regular conversation.
|
||||
|
||||
Args:
|
||||
item: The ChatKit hidden context item to convert.
|
||||
|
||||
Returns:
|
||||
A Message with system role, a list of messages, or None to skip.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types and provides proper message ordering.
|
||||
|
||||
Examples:
|
||||
.. code-block:: python
|
||||
|
||||
# Default behavior
|
||||
converter = ThreadItemConverter()
|
||||
hidden_item = HiddenContextItem(
|
||||
id="ctx_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
content="User's email: user@example.com",
|
||||
)
|
||||
message = converter.hidden_context_to_input(hidden_item)
|
||||
# Returns: Message(role=SYSTEM, contents=["<HIDDEN_CONTEXT>User's email: ...</HIDDEN_CONTEXT>"])
|
||||
"""
|
||||
return Message(role="system", contents=[f"<HIDDEN_CONTEXT>{item.content}</HIDDEN_CONTEXT>"])
|
||||
|
||||
def tag_to_message_content(self, tag: UserMessageTagContent) -> Content:
|
||||
"""Convert a ChatKit tag (@-mention) to Agent Framework content.
|
||||
|
||||
This method is called internally by `user_message_to_input()` to handle tags.
|
||||
Override this method to customize tag conversion for your application.
|
||||
|
||||
The default implementation extracts the tag's display name and wraps it in
|
||||
XML tags to provide context to the model about the @-mention.
|
||||
|
||||
Args:
|
||||
tag: The ChatKit tag content to convert.
|
||||
|
||||
Returns:
|
||||
TextContent with the tag information.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types including tags within user messages.
|
||||
|
||||
Examples:
|
||||
.. code-block:: python
|
||||
|
||||
# Default behavior
|
||||
converter = ThreadItemConverter()
|
||||
tag = UserMessageTagContent(
|
||||
type="input_tag", id="tag_1", text="john", data={"name": "John Doe"}, interactive=False
|
||||
)
|
||||
content = converter.tag_to_message_content(tag)
|
||||
# Returns: Content.from_text(text="<TAG>Name:John Doe</TAG>")
|
||||
"""
|
||||
name = getattr(tag.data, "name", tag.text if hasattr(tag, "text") else "unknown")
|
||||
return Content.from_text(text=f"<TAG>Name:{name}</TAG>")
|
||||
|
||||
def task_to_input(self, item: TaskItem) -> Message | list[Message] | None:
|
||||
"""Convert a ChatKit TaskItem to Agent Framework Message(s).
|
||||
|
||||
This method is called internally by `to_agent_input()`. Override this method
|
||||
to customize how tasks are converted.
|
||||
|
||||
The default implementation converts custom tasks with title/content into
|
||||
a user message explaining what task was displayed to the user.
|
||||
|
||||
Args:
|
||||
item: The ChatKit task item to convert.
|
||||
|
||||
Returns:
|
||||
A Message, a list of messages, or None to skip the task.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types and provides proper message ordering.
|
||||
|
||||
Examples:
|
||||
.. code-block:: python
|
||||
|
||||
# Task with both title and content
|
||||
from chatkit.types import Task
|
||||
|
||||
task_item = TaskItem(
|
||||
id="task_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
task=Task(type="custom", title="Data Analysis", content="Analyzed sales data"),
|
||||
)
|
||||
message = converter.task_to_input(task_item)
|
||||
# Returns message explaining the task was performed
|
||||
"""
|
||||
if item.task.type != "custom" or (not item.task.title and not item.task.content):
|
||||
return None
|
||||
|
||||
title = item.task.title or ""
|
||||
content = item.task.content or ""
|
||||
task_text = f"{title}: {content}" if title and content else title or content
|
||||
text = (
|
||||
f"A message was displayed to the user that the following task was performed:\n<Task>\n{task_text}\n</Task>"
|
||||
)
|
||||
|
||||
return Message(role="user", contents=[text])
|
||||
|
||||
def workflow_to_input(self, item: WorkflowItem) -> Message | list[Message] | None:
|
||||
"""Convert a ChatKit WorkflowItem to Agent Framework Message(s).
|
||||
|
||||
This method is called internally by `to_agent_input()`. Override this method
|
||||
to customize how workflows are converted.
|
||||
|
||||
The default implementation converts each custom task in the workflow into
|
||||
a separate user message explaining what tasks were performed.
|
||||
|
||||
Args:
|
||||
item: The ChatKit workflow item to convert.
|
||||
|
||||
Returns:
|
||||
A list of ChatMessages (one per task), a single message, or None to skip.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types and provides proper message ordering.
|
||||
|
||||
Examples:
|
||||
.. code-block:: python
|
||||
|
||||
# Workflow with multiple tasks
|
||||
from chatkit.types import Workflow, Task
|
||||
|
||||
workflow_item = WorkflowItem(
|
||||
id="wf_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
workflow=Workflow(
|
||||
type="custom",
|
||||
tasks=[
|
||||
Task(type="custom", title="Step 1", content="Gathered data"),
|
||||
Task(type="custom", title="Step 2", content="Analyzed results"),
|
||||
],
|
||||
),
|
||||
)
|
||||
messages = converter.workflow_to_input(workflow_item)
|
||||
# Returns list of messages for each task
|
||||
"""
|
||||
messages: list[Message] = []
|
||||
for task in item.workflow.tasks:
|
||||
if task.type != "custom" or (not task.title and not task.content):
|
||||
continue
|
||||
|
||||
title = task.title or ""
|
||||
content = task.content or ""
|
||||
task_text = f"{title}: {content}" if title and content else title or content
|
||||
text = (
|
||||
"A message was displayed to the user that the following task was performed:\n"
|
||||
f"<Task>\n{task_text}\n</Task>"
|
||||
)
|
||||
|
||||
messages.append(Message(role="user", contents=[text]))
|
||||
|
||||
return messages if messages else None
|
||||
|
||||
def widget_to_input(self, item: WidgetItem) -> Message | list[Message] | None:
|
||||
"""Convert a ChatKit WidgetItem to Agent Framework Message(s).
|
||||
|
||||
This method is called internally by `to_agent_input()`. Override this method
|
||||
to customize how widgets are converted.
|
||||
|
||||
The default implementation converts the widget to a JSON representation
|
||||
and includes it in a user message, allowing the model to understand what
|
||||
UI element was displayed to the user.
|
||||
|
||||
Args:
|
||||
item: The ChatKit widget item to convert.
|
||||
|
||||
Returns:
|
||||
A Message describing the widget, or None to skip.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types and provides proper message ordering.
|
||||
|
||||
Examples:
|
||||
.. code-block:: python
|
||||
|
||||
# Widget item
|
||||
from chatkit.widgets import Card, Text
|
||||
|
||||
widget_item = WidgetItem(
|
||||
id="widget_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
widget=Card(children=[Text(value="Hello")]),
|
||||
)
|
||||
message = converter.widget_to_input(widget_item)
|
||||
# Returns message with JSON representation of the widget
|
||||
"""
|
||||
try:
|
||||
widget_json = item.widget.model_dump_json(exclude_unset=True, exclude_none=True)
|
||||
text = f"The following graphical UI widget (id: {item.id}) was displayed to the user:{widget_json}"
|
||||
return Message(role="user", contents=[text])
|
||||
except Exception:
|
||||
# If JSON serialization fails, skip the widget
|
||||
return None
|
||||
|
||||
async def assistant_message_to_input(self, item: AssistantMessageItem) -> Message | list[Message] | None:
|
||||
"""Convert a ChatKit AssistantMessageItem to Agent Framework Message(s).
|
||||
|
||||
The default implementation extracts text from all content parts and creates
|
||||
an assistant message.
|
||||
|
||||
Args:
|
||||
item: The ChatKit assistant message item to convert.
|
||||
|
||||
Returns:
|
||||
A Message with assistant role, or None to skip.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types and provides proper message ordering.
|
||||
"""
|
||||
# Extract text from all content parts
|
||||
text_parts = [content.text for content in item.content]
|
||||
if not text_parts:
|
||||
return None
|
||||
|
||||
return Message(role="assistant", contents=["".join(text_parts)])
|
||||
|
||||
async def client_tool_call_to_input(self, item: ClientToolCallItem) -> Message | list[Message] | None:
|
||||
"""Convert a ChatKit ClientToolCallItem to Agent Framework Message(s).
|
||||
|
||||
The default implementation converts completed tool calls into function call
|
||||
and result content.
|
||||
|
||||
Args:
|
||||
item: The ChatKit client tool call item to convert.
|
||||
|
||||
Returns:
|
||||
A list containing function call and result messages, or None for pending calls.
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types and provides proper message ordering.
|
||||
"""
|
||||
if item.status == "pending":
|
||||
# Skip pending tool calls - they cannot be sent to the model
|
||||
return None
|
||||
|
||||
import json
|
||||
|
||||
# Create function call message
|
||||
function_call_msg = Message(
|
||||
role="assistant",
|
||||
contents=[
|
||||
Content.from_function_call(
|
||||
call_id=item.call_id,
|
||||
name=item.name,
|
||||
arguments=json.dumps(item.arguments),
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
# Create function result message
|
||||
function_result_msg = Message(
|
||||
role="tool",
|
||||
contents=[
|
||||
Content.from_function_result(
|
||||
call_id=item.call_id,
|
||||
result=json.dumps(item.output) if item.output is not None else "",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
return [function_call_msg, function_result_msg]
|
||||
|
||||
async def end_of_turn_to_input(self, item: EndOfTurnItem) -> Message | list[Message] | None:
|
||||
"""Convert a ChatKit EndOfTurnItem to Agent Framework Message(s).
|
||||
|
||||
The default implementation skips end-of-turn markers as they are only UI hints.
|
||||
|
||||
Args:
|
||||
item: The ChatKit end-of-turn item to convert.
|
||||
|
||||
Returns:
|
||||
None (end-of-turn items are not converted).
|
||||
|
||||
Note:
|
||||
Instead of calling this method directly, use `to_agent_input()` which handles
|
||||
all ThreadItem types and provides proper message ordering.
|
||||
"""
|
||||
# End-of-turn is only used for UI hints - skip it
|
||||
return None
|
||||
|
||||
async def _thread_item_to_input_item(
|
||||
self,
|
||||
item: ThreadItem,
|
||||
is_last_message: bool = True,
|
||||
) -> list[Message]:
|
||||
"""Internal method to convert a single ThreadItem to Message(s).
|
||||
|
||||
Args:
|
||||
item: The thread item to convert.
|
||||
is_last_message: Whether this is the last item in the thread.
|
||||
|
||||
Returns:
|
||||
A list of Message objects (may be empty).
|
||||
"""
|
||||
match item:
|
||||
case UserMessageItem():
|
||||
out = await self.user_message_to_input(item, is_last_message) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case AssistantMessageItem():
|
||||
out = await self.assistant_message_to_input(item) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case ClientToolCallItem():
|
||||
out = await self.client_tool_call_to_input(item) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case EndOfTurnItem():
|
||||
out = await self.end_of_turn_to_input(item) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case WidgetItem():
|
||||
out = self.widget_to_input(item) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case WorkflowItem():
|
||||
out = self.workflow_to_input(item) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case TaskItem():
|
||||
out = self.task_to_input(item) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case HiddenContextItem():
|
||||
out = self.hidden_context_to_input(item) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case SDKHiddenContextItem():
|
||||
out = self.hidden_context_to_input(item) or []
|
||||
return out if isinstance(out, list) else [out]
|
||||
case GeneratedImageItem():
|
||||
# TODO(evmattso): Implement generated image handling in a future PR
|
||||
return []
|
||||
case StructuredInputItem():
|
||||
# TODO(evmattso): Implement structured input handling in a future PR
|
||||
return []
|
||||
case _:
|
||||
# Unknown ThreadItem variant (e.g. types added in newer chatkit versions).
|
||||
# Skip rather than fail so we remain forward-compatible with chatkit upgrades.
|
||||
logger.debug("Skipping unsupported ThreadItem of type %s", type(item).__name__)
|
||||
return []
|
||||
|
||||
async def to_agent_input(
|
||||
self,
|
||||
thread_items: Sequence[ThreadItem] | ThreadItem,
|
||||
) -> list[Message]:
|
||||
"""Convert ChatKit thread items to Agent Framework ChatMessages.
|
||||
|
||||
This is the main entry point for converting ChatKit thread items. It handles
|
||||
all ThreadItem types (UserMessageItem, AssistantMessageItem, TaskItem, etc.)
|
||||
and calls the appropriate conversion method for each.
|
||||
|
||||
Args:
|
||||
thread_items: A single ThreadItem or a sequence of ThreadItems to convert.
|
||||
|
||||
Returns:
|
||||
A list of Message objects that can be sent to an Agent Framework agent.
|
||||
|
||||
Examples:
|
||||
.. code-block:: python
|
||||
|
||||
from agent_framework_chatkit import ThreadItemConverter
|
||||
|
||||
converter = ThreadItemConverter()
|
||||
|
||||
# Convert a single thread item
|
||||
messages = await converter.to_agent_input(user_message_item)
|
||||
|
||||
# Convert multiple thread items
|
||||
messages = await converter.to_agent_input([user_message_item, assistant_message_item, task_item])
|
||||
|
||||
# Use with agent
|
||||
from agent_framework import Agent
|
||||
|
||||
agent = Agent(...)
|
||||
response = await agent.run(messages)
|
||||
"""
|
||||
thread_items = list(thread_items) if isinstance(thread_items, Sequence) else [thread_items]
|
||||
|
||||
output: list[Message] = []
|
||||
for item in thread_items:
|
||||
output.extend(
|
||||
await self._thread_item_to_input_item(
|
||||
item,
|
||||
is_last_message=item is thread_items[-1],
|
||||
)
|
||||
)
|
||||
return output
|
||||
|
||||
|
||||
# Default converter instance
|
||||
_DEFAULT_CONVERTER = ThreadItemConverter()
|
||||
|
||||
|
||||
async def simple_to_agent_input(thread_items: Sequence[ThreadItem] | ThreadItem) -> list[Message]:
|
||||
"""Helper function that uses the default ThreadItemConverter.
|
||||
|
||||
This function provides a quick way to get started with ChatKit integration
|
||||
without needing to create a custom ThreadItemConverter instance.
|
||||
|
||||
Args:
|
||||
thread_items: A single ThreadItem or a sequence of ThreadItems to convert.
|
||||
|
||||
Returns:
|
||||
A list of Message objects that can be sent to an Agent Framework agent.
|
||||
|
||||
Examples:
|
||||
.. code-block:: python
|
||||
|
||||
from agent_framework_chatkit import simple_to_agent_input
|
||||
|
||||
# Convert a single item
|
||||
messages = await simple_to_agent_input(user_message_item)
|
||||
|
||||
# Convert multiple items
|
||||
messages = await simple_to_agent_input([user_message_item, assistant_message_item, task_item])
|
||||
"""
|
||||
return await _DEFAULT_CONVERTER.to_agent_input(thread_items)
|
||||
@@ -0,0 +1,104 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Streaming utilities for converting Agent Framework responses to ChatKit events."""
|
||||
|
||||
import uuid
|
||||
from collections.abc import AsyncIterable, AsyncIterator, Callable
|
||||
from datetime import datetime
|
||||
|
||||
from agent_framework import AgentResponseUpdate
|
||||
from chatkit.types import (
|
||||
AssistantMessageContent,
|
||||
AssistantMessageContentPartTextDelta,
|
||||
AssistantMessageItem,
|
||||
ThreadItemAddedEvent,
|
||||
ThreadItemDoneEvent,
|
||||
ThreadItemUpdated,
|
||||
ThreadStreamEvent,
|
||||
)
|
||||
|
||||
|
||||
async def stream_agent_response(
|
||||
response_stream: AsyncIterable[AgentResponseUpdate],
|
||||
thread_id: str,
|
||||
generate_id: Callable[[str], str] | None = None,
|
||||
) -> AsyncIterator[ThreadStreamEvent]:
|
||||
"""Convert a streamed AgentResponseUpdate from Agent Framework to ChatKit events.
|
||||
|
||||
This helper function takes a stream of AgentResponseUpdate objects from
|
||||
a Microsoft Agent Framework agent and converts them to ChatKit ThreadStreamEvent
|
||||
objects that can be consumed by the ChatKit UI.
|
||||
|
||||
The function supports real-time token-by-token streaming by emitting
|
||||
ThreadItemUpdated events with AssistantMessageContentPartTextDelta for each
|
||||
text chunk as it arrives from the agent.
|
||||
|
||||
Args:
|
||||
response_stream: An async iterable of AgentResponseUpdate objects
|
||||
from an Agent Framework agent.
|
||||
thread_id: The ChatKit thread ID for the conversation.
|
||||
generate_id: Optional function to generate IDs for ChatKit items.
|
||||
If not provided, simple incremental IDs will be used.
|
||||
|
||||
Yields:
|
||||
ThreadStreamEvent: ChatKit events representing the agent's response,
|
||||
including incremental text deltas for streaming display.
|
||||
"""
|
||||
# Use provided ID generator or create default one
|
||||
if generate_id is None:
|
||||
|
||||
def _default_id_generator(item_type: str) -> str:
|
||||
return f"{item_type}_{uuid.uuid4().hex[:8]}"
|
||||
|
||||
message_id = _default_id_generator("msg")
|
||||
else:
|
||||
message_id = generate_id("msg")
|
||||
|
||||
# Track if we've started the message
|
||||
message_started = False
|
||||
accumulated_text = ""
|
||||
content_index = 0
|
||||
|
||||
async for update in response_stream:
|
||||
# Start the assistant message if not already started
|
||||
if not message_started:
|
||||
assistant_message = AssistantMessageItem(
|
||||
id=message_id,
|
||||
thread_id=thread_id,
|
||||
type="assistant_message",
|
||||
content=[],
|
||||
created_at=datetime.now(),
|
||||
)
|
||||
|
||||
yield ThreadItemAddedEvent(type="thread.item.added", item=assistant_message)
|
||||
message_started = True
|
||||
|
||||
# Process the update content
|
||||
if update.contents:
|
||||
for content in update.contents:
|
||||
# Handle text content - only TextContent has a text attribute
|
||||
if content.type == "text" and content.text is not None:
|
||||
# Yield incremental text delta for streaming display
|
||||
yield ThreadItemUpdated(
|
||||
type="thread.item.updated",
|
||||
item_id=message_id,
|
||||
update=AssistantMessageContentPartTextDelta(
|
||||
content_index=content_index,
|
||||
delta=content.text,
|
||||
),
|
||||
)
|
||||
accumulated_text += content.text
|
||||
|
||||
# Finalize the message
|
||||
if message_started:
|
||||
final_message = AssistantMessageItem(
|
||||
id=message_id,
|
||||
thread_id=thread_id,
|
||||
type="assistant_message",
|
||||
content=[AssistantMessageContent(type="output_text", text=accumulated_text, annotations=[])]
|
||||
if accumulated_text
|
||||
else [],
|
||||
created_at=datetime.now(),
|
||||
)
|
||||
|
||||
yield ThreadItemDoneEvent(type="thread.item.done", item=final_message)
|
||||
@@ -0,0 +1,99 @@
|
||||
[project]
|
||||
name = "agent-framework-chatkit"
|
||||
description = "OpenAI ChatKit integration for Microsoft Agent Framework."
|
||||
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
version = "1.0.0b260528"
|
||||
license-files = ["LICENSE"]
|
||||
urls.homepage = "https://aka.ms/agent-framework"
|
||||
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
|
||||
urls.release_notes = "https://github.com/microsoft/agent-framework/releases?q=tag%3Apython-1&expanded=true"
|
||||
urls.issues = "https://github.com/microsoft/agent-framework/issues"
|
||||
classifiers = [
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Development Status :: 4 - Beta",
|
||||
"Intended Audience :: Developers",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Typing :: Typed",
|
||||
]
|
||||
dependencies = [
|
||||
"agent-framework-core>=1.7.0,<2",
|
||||
"openai-chatkit>=1.6.4,<2.0.0",
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
prerelease = "if-necessary-or-explicit"
|
||||
environments = [
|
||||
"sys_platform == 'darwin'",
|
||||
"sys_platform == 'linux'",
|
||||
"sys_platform == 'win32'"
|
||||
]
|
||||
|
||||
[tool.uv-dynamic-versioning]
|
||||
fallback-version = "0.0.0"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = 'tests'
|
||||
addopts = "-ra -q -r fEX"
|
||||
asyncio_mode = "auto"
|
||||
asyncio_default_fixture_loop_scope = "function"
|
||||
filterwarnings = []
|
||||
timeout = 120
|
||||
markers = [
|
||||
"integration: marks tests as integration tests that require external services",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
extend = "../../pyproject.toml"
|
||||
|
||||
[tool.ruff.lint]
|
||||
ignore = ["RUF029"]
|
||||
|
||||
[tool.coverage.run]
|
||||
omit = [
|
||||
"**/__init__.py"
|
||||
]
|
||||
|
||||
[tool.pyright]
|
||||
extends = "../../pyproject.toml"
|
||||
include = ["agent_framework_chatkit"]
|
||||
exclude = ['tests', 'chatkit-python', 'openai-chatkit-advanced-samples']
|
||||
|
||||
[tool.mypy]
|
||||
plugins = ['pydantic.mypy']
|
||||
strict = true
|
||||
python_version = "3.10"
|
||||
ignore_missing_imports = true
|
||||
disallow_untyped_defs = true
|
||||
no_implicit_optional = true
|
||||
check_untyped_defs = true
|
||||
warn_return_any = true
|
||||
show_error_codes = true
|
||||
warn_unused_ignores = false
|
||||
disallow_incomplete_defs = true
|
||||
disallow_untyped_decorators = true
|
||||
|
||||
[tool.bandit]
|
||||
targets = ["agent_framework_chatkit"]
|
||||
exclude_dirs = ["tests"]
|
||||
|
||||
[tool.poe]
|
||||
executor.type = "uv"
|
||||
include = "../../shared_tasks.toml"
|
||||
|
||||
[tool.poe.tasks.mypy]
|
||||
help = "Run MyPy for this package."
|
||||
cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_chatkit"
|
||||
|
||||
[tool.poe.tasks.test]
|
||||
help = "Run the default unit test suite for this package."
|
||||
cmd = 'pytest -m "not integration" --cov=agent_framework_chatkit --cov-report=term-missing:skip-covered tests'
|
||||
|
||||
[build-system]
|
||||
requires = ["flit-core >= 3.11,<4.0"]
|
||||
build-backend = "flit_core.buildapi"
|
||||
@@ -0,0 +1,424 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for ChatKit to Agent Framework converter utilities."""
|
||||
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
from agent_framework import Message
|
||||
from chatkit.types import InferenceOptions, UserMessageTextContent
|
||||
from pydantic import AnyUrl
|
||||
|
||||
from agent_framework_chatkit import ThreadItemConverter, simple_to_agent_input
|
||||
|
||||
|
||||
class TestThreadItemConverter:
|
||||
"""Tests for ThreadItemConverter class."""
|
||||
|
||||
@pytest.fixture
|
||||
def converter(self):
|
||||
"""Create a ThreadItemConverter instance for testing."""
|
||||
return ThreadItemConverter()
|
||||
|
||||
async def test_to_agent_input_none(self, converter):
|
||||
"""Test converting empty list returns empty list."""
|
||||
result = await converter.to_agent_input([])
|
||||
assert result == []
|
||||
|
||||
async def test_to_agent_input_with_text(self, converter):
|
||||
"""Test converting user message with text content."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import UserMessageItem
|
||||
|
||||
input_item = UserMessageItem(
|
||||
id="msg_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="user_message",
|
||||
content=[UserMessageTextContent(text="Hello, how can you help me?")],
|
||||
attachments=[],
|
||||
inference_options=InferenceOptions(),
|
||||
)
|
||||
|
||||
result = await converter.to_agent_input(input_item)
|
||||
|
||||
assert len(result) == 1
|
||||
assert isinstance(result[0], Message)
|
||||
assert result[0].role == "user"
|
||||
assert result[0].text == "Hello, how can you help me?"
|
||||
|
||||
async def test_to_agent_input_empty_text(self, converter):
|
||||
"""Test converting user message with empty or whitespace-only text."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import UserMessageItem
|
||||
|
||||
input_item = UserMessageItem(
|
||||
id="msg_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="user_message",
|
||||
content=[UserMessageTextContent(text=" ")],
|
||||
attachments=[],
|
||||
inference_options=InferenceOptions(),
|
||||
)
|
||||
|
||||
result = await converter.to_agent_input(input_item)
|
||||
assert result == []
|
||||
|
||||
async def test_to_agent_input_no_content(self, converter):
|
||||
"""Test converting user message with no content."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import UserMessageItem
|
||||
|
||||
input_item = UserMessageItem(
|
||||
id="msg_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="user_message",
|
||||
content=[],
|
||||
attachments=[],
|
||||
inference_options=InferenceOptions(),
|
||||
)
|
||||
|
||||
result = await converter.to_agent_input(input_item)
|
||||
assert result == []
|
||||
|
||||
async def test_to_agent_input_multiple_content_parts(self, converter):
|
||||
"""Test converting user message with multiple text content parts."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import UserMessageItem
|
||||
|
||||
input_item = UserMessageItem(
|
||||
id="msg_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="user_message",
|
||||
content=[
|
||||
UserMessageTextContent(text="Hello "),
|
||||
UserMessageTextContent(text="world!"),
|
||||
],
|
||||
attachments=[],
|
||||
inference_options=InferenceOptions(),
|
||||
)
|
||||
|
||||
result = await converter.to_agent_input(input_item)
|
||||
|
||||
assert len(result) == 1
|
||||
assert result[0].text == "Hello world!"
|
||||
|
||||
def test_hidden_context_to_input(self, converter):
|
||||
"""Test converting hidden context item to Message."""
|
||||
hidden_item = Mock()
|
||||
hidden_item.content = "This is hidden context information"
|
||||
|
||||
result = converter.hidden_context_to_input(hidden_item)
|
||||
|
||||
assert isinstance(result, Message)
|
||||
assert result.role == "system"
|
||||
assert result.text == "<HIDDEN_CONTEXT>This is hidden context information</HIDDEN_CONTEXT>"
|
||||
|
||||
def test_tag_to_message_content(self, converter):
|
||||
"""Test converting tag to message content."""
|
||||
from chatkit.types import UserMessageTagContent
|
||||
|
||||
tag = UserMessageTagContent(
|
||||
type="input_tag",
|
||||
id="tag_1",
|
||||
text="john",
|
||||
data={"name": "John Doe"},
|
||||
interactive=False,
|
||||
)
|
||||
|
||||
result = converter.tag_to_message_content(tag)
|
||||
assert result.type == "text"
|
||||
# Since data is a dict, getattr won't work, so it will fall back to text
|
||||
assert result.text == "<TAG>Name:john</TAG>"
|
||||
|
||||
def test_tag_to_message_content_no_name(self, converter):
|
||||
"""Test converting tag with no name to message content."""
|
||||
from chatkit.types import UserMessageTagContent
|
||||
|
||||
tag = UserMessageTagContent(
|
||||
type="input_tag",
|
||||
id="tag_2",
|
||||
text="jane",
|
||||
data={},
|
||||
interactive=False,
|
||||
)
|
||||
|
||||
result = converter.tag_to_message_content(tag)
|
||||
assert result.type == "text"
|
||||
assert result.text == "<TAG>Name:jane</TAG>"
|
||||
|
||||
async def test_attachment_to_message_content_file_without_fetcher(self, converter):
|
||||
"""Test that FileAttachment without data fetcher returns None."""
|
||||
from chatkit.types import FileAttachment
|
||||
|
||||
attachment = FileAttachment(
|
||||
id="file_123",
|
||||
name="document.pdf",
|
||||
mime_type="application/pdf",
|
||||
type="file",
|
||||
)
|
||||
|
||||
result = await converter.attachment_to_message_content(attachment)
|
||||
assert result is None
|
||||
|
||||
async def test_attachment_to_message_content_image_with_preview_url(self, converter):
|
||||
"""Test that ImageAttachment with preview_url creates UriContent."""
|
||||
from chatkit.types import ImageAttachment
|
||||
|
||||
attachment = ImageAttachment(
|
||||
id="img_123",
|
||||
name="photo.jpg",
|
||||
mime_type="image/jpeg",
|
||||
type="image",
|
||||
preview_url=AnyUrl("https://example.com/photo.jpg"),
|
||||
)
|
||||
|
||||
result = await converter.attachment_to_message_content(attachment)
|
||||
assert result.type == "uri"
|
||||
assert result.uri == "https://example.com/photo.jpg"
|
||||
assert result.media_type == "image/jpeg"
|
||||
|
||||
async def test_attachment_to_message_content_with_data_fetcher(self):
|
||||
"""Test attachment conversion with data fetcher."""
|
||||
from chatkit.types import FileAttachment
|
||||
|
||||
# Mock data fetcher
|
||||
async def fetch_data(attachment_id: str) -> bytes:
|
||||
return b"file content data"
|
||||
|
||||
converter = ThreadItemConverter(attachment_data_fetcher=fetch_data)
|
||||
|
||||
attachment = FileAttachment(
|
||||
id="file_123",
|
||||
name="document.pdf",
|
||||
mime_type="application/pdf",
|
||||
type="file",
|
||||
)
|
||||
|
||||
result = await converter.attachment_to_message_content(attachment)
|
||||
assert result is not None
|
||||
assert result.type == "data"
|
||||
assert result.media_type == "application/pdf"
|
||||
|
||||
async def test_to_agent_input_with_image_attachment(self):
|
||||
"""Test converting user message with text and image attachment."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import ImageAttachment, UserMessageItem
|
||||
|
||||
attachment = ImageAttachment(
|
||||
id="img_123",
|
||||
name="photo.jpg",
|
||||
mime_type="image/jpeg",
|
||||
type="image",
|
||||
preview_url=AnyUrl("https://example.com/photo.jpg"),
|
||||
)
|
||||
|
||||
input_item = UserMessageItem(
|
||||
id="msg_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="user_message",
|
||||
content=[UserMessageTextContent(text="Check out this photo!")],
|
||||
attachments=[attachment],
|
||||
inference_options=InferenceOptions(),
|
||||
)
|
||||
|
||||
converter = ThreadItemConverter()
|
||||
result = await converter.to_agent_input(input_item)
|
||||
|
||||
assert len(result) == 1
|
||||
message = result[0]
|
||||
assert message.role == "user"
|
||||
assert len(message.contents) == 2
|
||||
|
||||
# First content should be text
|
||||
assert message.contents[0].type == "text"
|
||||
assert message.contents[0].text == "Check out this photo!"
|
||||
|
||||
# Second content should be UriContent for the image
|
||||
assert message.contents[1].type == "uri"
|
||||
assert message.contents[1].uri == "https://example.com/photo.jpg"
|
||||
assert message.contents[1].media_type == "image/jpeg"
|
||||
|
||||
async def test_to_agent_input_with_file_attachment_and_fetcher(self):
|
||||
"""Test converting user message with file attachment using data fetcher."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import FileAttachment, UserMessageItem
|
||||
|
||||
attachment = FileAttachment(
|
||||
id="file_123",
|
||||
name="report.pdf",
|
||||
mime_type="application/pdf",
|
||||
type="file",
|
||||
)
|
||||
|
||||
input_item = UserMessageItem(
|
||||
id="msg_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="user_message",
|
||||
content=[UserMessageTextContent(text="Here's the document")],
|
||||
attachments=[attachment],
|
||||
inference_options=InferenceOptions(),
|
||||
)
|
||||
|
||||
# Create converter with data fetcher
|
||||
async def fetch_data(attachment_id: str) -> bytes:
|
||||
return b"PDF content data"
|
||||
|
||||
converter = ThreadItemConverter(attachment_data_fetcher=fetch_data)
|
||||
result = await converter.to_agent_input(input_item)
|
||||
|
||||
assert len(result) == 1
|
||||
message = result[0]
|
||||
assert len(message.contents) == 2
|
||||
|
||||
# First content should be text
|
||||
assert message.contents[0].type == "text"
|
||||
|
||||
# Second content should be DataContent for the file
|
||||
assert message.contents[1].type == "data"
|
||||
assert message.contents[1].media_type == "application/pdf"
|
||||
|
||||
def test_task_to_input(self, converter):
|
||||
"""Test converting TaskItem to Message."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import CustomTask, TaskItem
|
||||
|
||||
task_item = TaskItem(
|
||||
id="task_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="task",
|
||||
task=CustomTask(type="custom", title="Analysis", content="Analyzed the data"),
|
||||
)
|
||||
|
||||
result = converter.task_to_input(task_item)
|
||||
assert isinstance(result, Message)
|
||||
assert result.role == "user"
|
||||
assert "Analysis: Analyzed the data" in result.text
|
||||
assert "<Task>" in result.text
|
||||
|
||||
def test_task_to_input_no_custom_task(self, converter):
|
||||
"""Test that non-custom tasks return None."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import TaskItem, ThoughtTask
|
||||
|
||||
task_item = TaskItem(
|
||||
id="task_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="task",
|
||||
task=ThoughtTask(type="thought", title="Think", content="Thinking..."),
|
||||
)
|
||||
|
||||
result = converter.task_to_input(task_item)
|
||||
assert result is None
|
||||
|
||||
def test_workflow_to_input(self, converter):
|
||||
"""Test converting WorkflowItem to ChatMessages."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import CustomTask, Workflow, WorkflowItem
|
||||
|
||||
workflow_item = WorkflowItem(
|
||||
id="wf_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="workflow",
|
||||
workflow=Workflow(
|
||||
type="custom",
|
||||
tasks=[
|
||||
CustomTask(type="custom", title="Step 1", content="First step"),
|
||||
CustomTask(type="custom", title="Step 2", content="Second step"),
|
||||
],
|
||||
),
|
||||
)
|
||||
|
||||
result = converter.workflow_to_input(workflow_item)
|
||||
assert isinstance(result, list)
|
||||
assert len(result) == 2
|
||||
assert all(isinstance(msg, Message) for msg in result)
|
||||
assert "Step 1: First step" in result[0].text
|
||||
assert "Step 2: Second step" in result[1].text
|
||||
|
||||
def test_workflow_to_input_empty(self, converter):
|
||||
"""Test that workflows with no custom tasks return None."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import Workflow, WorkflowItem
|
||||
|
||||
workflow_item = WorkflowItem(
|
||||
id="wf_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="workflow",
|
||||
workflow=Workflow(type="custom", tasks=[]),
|
||||
)
|
||||
|
||||
result = converter.workflow_to_input(workflow_item)
|
||||
assert result is None
|
||||
|
||||
def test_widget_to_input(self, converter):
|
||||
"""Test converting WidgetItem to Message."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import WidgetItem
|
||||
from chatkit.widgets import Card, Text # ty: ignore[deprecated]
|
||||
|
||||
widget_item = WidgetItem(
|
||||
id="widget_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="widget",
|
||||
widget=Card(key="card1", children=[Text(value="Hello")]), # ty: ignore[deprecated]
|
||||
)
|
||||
|
||||
result = converter.widget_to_input(widget_item)
|
||||
assert isinstance(result, Message)
|
||||
assert result.role == "user"
|
||||
assert "widget_1" in result.text
|
||||
assert "graphical UI widget" in result.text
|
||||
|
||||
|
||||
class TestSimpleToAgentInput:
|
||||
"""Tests for simple_to_agent_input helper function."""
|
||||
|
||||
async def test_simple_to_agent_input_empty_list(self):
|
||||
"""Test simple conversion with empty list."""
|
||||
result = await simple_to_agent_input([])
|
||||
assert result == []
|
||||
|
||||
async def test_simple_to_agent_input_with_text(self):
|
||||
"""Test simple conversion with text content."""
|
||||
from datetime import datetime
|
||||
|
||||
from chatkit.types import UserMessageItem
|
||||
|
||||
input_item = UserMessageItem(
|
||||
id="msg_1",
|
||||
thread_id="thread_1",
|
||||
created_at=datetime.now(),
|
||||
type="user_message",
|
||||
content=[UserMessageTextContent(text="Test message")],
|
||||
attachments=[],
|
||||
inference_options=InferenceOptions(),
|
||||
)
|
||||
|
||||
result = await simple_to_agent_input(input_item)
|
||||
|
||||
assert len(result) == 1
|
||||
assert isinstance(result[0], Message)
|
||||
assert result[0].role == "user"
|
||||
assert result[0].text == "Test message"
|
||||
@@ -0,0 +1,156 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for Agent Framework to ChatKit streaming utilities."""
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
from unittest.mock import Mock
|
||||
|
||||
from agent_framework import AgentResponseUpdate, Content
|
||||
from chatkit.types import (
|
||||
AssistantMessageContent,
|
||||
AssistantMessageContentPartTextDelta,
|
||||
AssistantMessageItem,
|
||||
ThreadItemAddedEvent,
|
||||
ThreadItemDoneEvent,
|
||||
ThreadItemUpdated,
|
||||
)
|
||||
|
||||
from agent_framework_chatkit import stream_agent_response
|
||||
|
||||
|
||||
class TestStreamAgentResponse:
|
||||
"""Tests for stream_agent_response function."""
|
||||
|
||||
async def test_stream_empty_response(self):
|
||||
"""Test streaming empty response."""
|
||||
|
||||
async def empty_stream() -> AsyncIterator[AgentResponseUpdate]:
|
||||
return
|
||||
yield # Make it a generator
|
||||
|
||||
events = []
|
||||
async for event in stream_agent_response(empty_stream(), thread_id="test_thread"):
|
||||
events.append(event)
|
||||
|
||||
assert len(events) == 0
|
||||
|
||||
async def test_stream_single_text_update(self):
|
||||
"""Test streaming single text update."""
|
||||
|
||||
async def single_update_stream():
|
||||
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Hello world")])
|
||||
|
||||
events = []
|
||||
async for event in stream_agent_response(single_update_stream(), thread_id="test_thread"):
|
||||
events.append(event)
|
||||
|
||||
# Should have: item_added, item_updated (delta), item_done
|
||||
assert len(events) == 3
|
||||
|
||||
# Check event types
|
||||
assert isinstance(events[0], ThreadItemAddedEvent)
|
||||
assert isinstance(events[1], ThreadItemUpdated)
|
||||
assert isinstance(events[2], ThreadItemDoneEvent)
|
||||
|
||||
# Check delta event
|
||||
assert isinstance(events[1].update, AssistantMessageContentPartTextDelta)
|
||||
assert events[1].update.delta == "Hello world"
|
||||
|
||||
# Check final message content
|
||||
assert isinstance(events[2].item, AssistantMessageItem)
|
||||
assert len(events[2].item.content) == 1
|
||||
assert isinstance(events[2].item.content[0], AssistantMessageContent)
|
||||
assert events[2].item.content[0].text == "Hello world"
|
||||
|
||||
async def test_stream_multiple_text_updates(self):
|
||||
"""Test streaming multiple text updates."""
|
||||
|
||||
async def multiple_updates_stream():
|
||||
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Hello ")])
|
||||
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="world!")])
|
||||
|
||||
events = []
|
||||
async for event in stream_agent_response(multiple_updates_stream(), thread_id="test_thread"):
|
||||
events.append(event)
|
||||
|
||||
# Should have: item_added, item_updated (delta 1), item_updated (delta 2), item_done
|
||||
assert len(events) == 4
|
||||
|
||||
# Check event types
|
||||
assert isinstance(events[0], ThreadItemAddedEvent)
|
||||
assert isinstance(events[1], ThreadItemUpdated)
|
||||
assert isinstance(events[2], ThreadItemUpdated)
|
||||
assert isinstance(events[3], ThreadItemDoneEvent)
|
||||
|
||||
# Check delta events
|
||||
assert isinstance(events[1].update, AssistantMessageContentPartTextDelta)
|
||||
assert isinstance(events[2].update, AssistantMessageContentPartTextDelta)
|
||||
assert events[1].update.delta == "Hello "
|
||||
assert events[2].update.delta == "world!"
|
||||
|
||||
# Check final accumulated text
|
||||
final_message_event = events[-1]
|
||||
assert isinstance(final_message_event, ThreadItemDoneEvent)
|
||||
assert isinstance(final_message_event.item, AssistantMessageItem)
|
||||
assert isinstance(final_message_event.item.content[0], AssistantMessageContent)
|
||||
assert final_message_event.item.content[0].text == "Hello world!"
|
||||
|
||||
async def test_stream_with_custom_id_generator(self):
|
||||
"""Test streaming with custom ID generator."""
|
||||
|
||||
def custom_id_generator(item_type: str) -> str:
|
||||
return f"custom_{item_type}_123"
|
||||
|
||||
async def single_update_stream():
|
||||
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Test")])
|
||||
|
||||
events = []
|
||||
async for event in stream_agent_response(
|
||||
single_update_stream(), thread_id="test_thread", generate_id=custom_id_generator
|
||||
):
|
||||
events.append(event)
|
||||
|
||||
# Check that custom IDs are used
|
||||
message_added_event = events[0]
|
||||
assert isinstance(message_added_event, ThreadItemAddedEvent)
|
||||
assert message_added_event.item.id == "custom_msg_123"
|
||||
|
||||
async def test_stream_empty_content_updates(self):
|
||||
"""Test streaming updates with empty content."""
|
||||
|
||||
async def empty_content_stream():
|
||||
yield AgentResponseUpdate(role="assistant", contents=[])
|
||||
yield AgentResponseUpdate(role="assistant", contents=None)
|
||||
|
||||
events = []
|
||||
async for event in stream_agent_response(empty_content_stream(), thread_id="test_thread"):
|
||||
events.append(event)
|
||||
|
||||
# Should have item_added and item_done
|
||||
assert len(events) == 2
|
||||
assert isinstance(events[0], ThreadItemAddedEvent)
|
||||
assert isinstance(events[1], ThreadItemDoneEvent)
|
||||
|
||||
# Final message should have empty content
|
||||
assert isinstance(events[1].item, AssistantMessageItem)
|
||||
assert len(events[1].item.content) == 0
|
||||
|
||||
async def test_stream_non_text_content(self):
|
||||
"""Test streaming updates with non-text content."""
|
||||
# Mock a content object without text attribute
|
||||
non_text_content = Mock(spec=Content)
|
||||
non_text_content.type = "image"
|
||||
# Don't set text attribute
|
||||
non_text_content.text = None
|
||||
|
||||
async def non_text_stream():
|
||||
yield AgentResponseUpdate(role="assistant", contents=[non_text_content])
|
||||
|
||||
events = []
|
||||
async for event in stream_agent_response(non_text_stream(), thread_id="test_thread"):
|
||||
events.append(event)
|
||||
|
||||
# Should have item_added and item_done, but no content since no text
|
||||
assert len(events) == 2
|
||||
assert isinstance(events[0], ThreadItemAddedEvent)
|
||||
assert isinstance(events[1], ThreadItemDoneEvent)
|
||||
Reference in New Issue
Block a user