chore: import upstream snapshot with attribution
CodeQL / Analyze (csharp) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:25 +08:00
commit db620d33df
5151 changed files with 925932 additions and 0 deletions
+25
View File
@@ -0,0 +1,25 @@
# Bedrock Package (agent-framework-bedrock)
Integration with AWS Bedrock for LLM inference.
## Main Classes
- **`BedrockChatClient`** - Chat client for AWS Bedrock models
- **`BedrockChatOptions`** - Options TypedDict for Bedrock-specific parameters
- **`BedrockGuardrailConfig`** - Configuration for Bedrock guardrails
- **`BedrockSettings`** - Pydantic settings for Bedrock configuration
## Usage
```python
from agent_framework.amazon import BedrockChatClient
client = BedrockChatClient(model="anthropic.claude-3-sonnet-20240229-v1:0")
response = await client.get_response("Hello")
```
## Import Path
```python
from agent_framework.amazon import BedrockChatClient
```
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) Microsoft Corporation.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
+19
View File
@@ -0,0 +1,19 @@
# Get Started with Microsoft Agent Framework Bedrock
Install the provider package:
```bash
pip install agent-framework-bedrock --pre
```
## Bedrock Integration
The Bedrock integration enables Microsoft Agent Framework applications to call Amazon Bedrock models with familiar chat abstractions, including tool/function calling when you attach tools through `ChatOptions`.
### Basic Usage Example
See the [Bedrock sample](../../samples/02-agents/providers/amazon/bedrock_chat_client.py) for a runnable end-to-end script that:
- Loads credentials from the `BEDROCK_*` environment variables
- Instantiates `BedrockChatClient`
- Sends a simple conversation turn and prints the response
@@ -0,0 +1,22 @@
# Copyright (c) Microsoft. All rights reserved.
import importlib.metadata
from ._chat_client import BedrockChatClient, BedrockChatOptions, BedrockGuardrailConfig, BedrockSettings
from ._embedding_client import BedrockEmbeddingClient, BedrockEmbeddingOptions, BedrockEmbeddingSettings
try:
__version__ = importlib.metadata.version(__name__)
except importlib.metadata.PackageNotFoundError:
__version__ = "0.0.0"
__all__ = [
"BedrockChatClient",
"BedrockChatOptions",
"BedrockEmbeddingClient",
"BedrockEmbeddingOptions",
"BedrockEmbeddingSettings",
"BedrockGuardrailConfig",
"BedrockSettings",
"__version__",
]
@@ -0,0 +1,895 @@
# Copyright (c) Microsoft. All rights reserved.
# type: ignore
# Because the Bedrock client does not have typing, we are ignoring type issues in this module.
from __future__ import annotations
import asyncio
import copy
import json
import logging
import sys
from collections import deque
from collections.abc import AsyncIterable, Awaitable, Mapping, MutableMapping, Sequence
from typing import Any, ClassVar, Generic, Literal, TypedDict
from uuid import uuid4
from agent_framework import (
BaseChatClient,
ChatAndFunctionMiddlewareTypes,
ChatMiddlewareLayer,
ChatOptions,
ChatResponse,
ChatResponseUpdate,
Content,
FinishReasonLiteral,
FunctionInvocationConfiguration,
FunctionInvocationLayer,
FunctionTool,
Message,
ResponseStream,
UsageDetails,
validate_tool_mode,
)
from agent_framework._settings import SecretString, load_settings
from agent_framework._telemetry import get_user_agent
from agent_framework.exceptions import ChatClientInvalidResponseException
from agent_framework.observability import ChatTelemetryLayer
from boto3.session import Session as Boto3Session
from botocore.client import BaseClient
from botocore.config import Config as BotoConfig
from botocore.exceptions import ClientError
from pydantic import BaseModel
if sys.version_info >= (3, 13):
from typing import TypeVar # pragma: no cover
else:
from typing_extensions import TypeVar # pragma: no cover
if sys.version_info >= (3, 12):
from typing import override # pragma: no cover
else:
from typing_extensions import override # pragma: no cover
if sys.version_info >= (3, 11):
from typing import TypedDict # pragma: no cover
else:
from typing_extensions import TypedDict # pragma: no cover
logger = logging.getLogger("agent_framework.bedrock")
__all__ = [
"BedrockChatClient",
"BedrockChatOptions",
"BedrockGuardrailConfig",
"BedrockSettings",
]
ResponseModelT = TypeVar("ResponseModelT", bound=BaseModel | None, default=None)
# region Bedrock Chat Options TypedDict
DEFAULT_REGION = "us-east-1"
DEFAULT_MAX_TOKENS = 1024
class BedrockGuardrailConfig(TypedDict, total=False):
"""Amazon Bedrock Guardrails configuration.
See: https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html
"""
guardrailIdentifier: str
"""The identifier of the guardrail to apply."""
guardrailVersion: str
"""The version of the guardrail to use."""
trace: Literal["enabled", "disabled"]
"""Whether to include guardrail trace information in the response."""
streamProcessingMode: Literal["sync", "async"]
"""How to process guardrails during streaming (sync blocks, async does not)."""
class BedrockChatOptions(ChatOptions[ResponseModelT], Generic[ResponseModelT], total=False):
"""Amazon Bedrock Converse API-specific chat options dict.
Extends base ChatOptions with Bedrock-specific parameters.
Bedrock uses a unified Converse API that works across multiple
foundation models (Claude, Titan, Llama, etc.).
See: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html
Keys:
# Inherited from ChatOptions (mapped to Bedrock):
model: The Bedrock model identifier,
translates to ``modelId`` in Bedrock API.
temperature: Sampling temperature,
translates to ``inferenceConfig.temperature``.
top_p: Nucleus sampling parameter,
translates to ``inferenceConfig.topP``.
max_tokens: Maximum number of tokens to generate,
translates to ``inferenceConfig.maxTokens``.
stop: Stop sequences,
translates to ``inferenceConfig.stopSequences``.
tools: List of tools available to the model,
translates to ``toolConfig.tools``.
tool_choice: How the model should use tools,
translates to ``toolConfig.toolChoice``.
response_format: Structured output format. Accepts a Pydantic BaseModel
subclass or an OpenAI-style dict schema
(``{"json_schema": {"name": ..., "schema": ...}}``).
When provided, the Converse API request includes
``outputConfig.textFormat`` with the schema serialized as a JSON
string. ``ChatResponse.value`` will be populated with the parsed
model instance. Only supported on models that support
``outputConfig.textFormat``. Unsupported models raise a ValueError.
# Options not supported in Bedrock Converse API:
seed: Not supported.
frequency_penalty: Not supported.
presence_penalty: Not supported.
allow_multiple_tool_calls: Not supported (models handle parallel calls automatically).
user: Not supported.
store: Not supported.
logit_bias: Not supported.
metadata: Not supported (use additional_properties for additionalModelRequestFields).
# Bedrock-specific options:
guardrailConfig: Guardrails configuration for content filtering.
performanceConfig: Performance optimization settings.
requestMetadata: Key-value metadata for the request.
promptVariables: Variables for prompt management (if using managed prompts).
"""
# Bedrock-specific options
guardrailConfig: BedrockGuardrailConfig
"""Guardrails configuration for content filtering and safety."""
performanceConfig: dict[str, Any]
"""Performance optimization settings (e.g., latency optimization).
See: https://docs.aws.amazon.com/bedrock/latest/userguide/inference-performance.html"""
requestMetadata: dict[str, str]
"""Key-value metadata for the request (max 2048 characters total)."""
promptVariables: dict[str, dict[str, str]]
"""Variables for prompt management when using managed prompts."""
# ChatOptions fields not supported in Bedrock
seed: None # type: ignore[misc]
"""Not supported in Bedrock Converse API."""
frequency_penalty: None # type: ignore[misc]
"""Not supported in Bedrock Converse API."""
presence_penalty: None # type: ignore[misc]
"""Not supported in Bedrock Converse API."""
allow_multiple_tool_calls: None # type: ignore[misc]
"""Not supported. Bedrock models handle parallel tool calls automatically."""
user: None # type: ignore[misc]
"""Not supported in Bedrock Converse API."""
store: None # type: ignore[misc]
"""Not supported in Bedrock Converse API."""
logit_bias: None # type: ignore[misc]
"""Not supported in Bedrock Converse API."""
BEDROCK_OPTION_TRANSLATIONS: dict[str, str] = {
"model": "modelId",
"max_tokens": "maxTokens",
"top_p": "topP",
"stop": "stopSequences",
}
"""Maps ChatOptions keys to Bedrock Converse API parameter names."""
BedrockChatOptionsT = TypeVar("BedrockChatOptionsT", bound=TypedDict, default="BedrockChatOptions", covariant=True) # type: ignore[valid-type]
# endregion
ROLE_MAP: dict[str, str] = {
"user": "user",
"assistant": "assistant",
"system": "user",
"tool": "user",
}
FINISH_REASON_MAP: dict[str, FinishReasonLiteral] = {
"end_turn": "stop",
"stop_sequence": "stop",
"max_tokens": "length",
"length": "length",
"content_filtered": "content_filter",
"tool_use": "tool_calls",
}
class BedrockSettings(TypedDict, total=False):
"""Bedrock configuration settings pulled from environment variables or .env files."""
region: str | None
chat_model: str | None
access_key: SecretString | None
secret_key: SecretString | None
session_token: SecretString | None
class BedrockChatClient(
FunctionInvocationLayer[BedrockChatOptionsT],
ChatMiddlewareLayer[BedrockChatOptionsT],
ChatTelemetryLayer[BedrockChatOptionsT],
BaseChatClient[BedrockChatOptionsT],
Generic[BedrockChatOptionsT],
):
"""Async chat client for Amazon Bedrock's Converse API with middleware, telemetry, and function invocation."""
OTEL_PROVIDER_NAME: ClassVar[str] = "aws.bedrock"
def __init__(
self,
*,
region: str | None = None,
model: str | None = None,
access_key: str | None = None,
secret_key: str | None = None,
session_token: str | None = None,
client: BaseClient | None = None,
boto3_session: Boto3Session | None = None,
additional_properties: dict[str, Any] | None = None,
middleware: Sequence[ChatAndFunctionMiddlewareTypes] | None = None,
function_invocation_configuration: FunctionInvocationConfiguration | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Create a Bedrock chat client and load AWS credentials.
Args:
region: Region to send Bedrock requests to; falls back to BEDROCK_REGION.
model: Default model identifier; falls back to BEDROCK_CHAT_MODEL.
access_key: Optional AWS access key for manual credential injection.
secret_key: Optional AWS secret key paired with ``access_key``.
session_token: Optional AWS session token for temporary credentials.
client: Preconfigured Bedrock runtime client; when omitted a boto3 session is created.
boto3_session: Custom boto3 session used to build the runtime client if provided.
additional_properties: Additional properties stored on the client instance.
middleware: Optional sequence of middlewares to include.
function_invocation_configuration: Optional function invocation configuration
env_file_path: Optional .env file path used by ``BedrockSettings`` to load defaults.
env_file_encoding: Encoding for the optional .env file.
Examples:
.. code-block:: python
from agent_framework.amazon import BedrockChatClient
# Basic usage with default credentials
client = BedrockChatClient(model="<model name>")
# Using custom ChatOptions with type safety:
from typing import TypedDict
from agent_framework_bedrock import BedrockChatOptions
class MyOptions(BedrockChatOptions, total=False):
my_custom_option: str
client = BedrockChatClient[MyOptions](model="<model name>")
response = await client.get_response("Hello", options={"my_custom_option": "value"})
"""
settings = load_settings(
BedrockSettings,
env_prefix="BEDROCK_",
region=region,
chat_model=model,
access_key=access_key,
secret_key=secret_key,
session_token=session_token,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
region = settings.get("region") or DEFAULT_REGION
chat_model = settings.get("chat_model")
if client:
self._bedrock_client = client
else:
session = boto3_session or self._create_session(settings)
self._bedrock_client = session.client(
"bedrock-runtime",
region_name=region,
config=BotoConfig(user_agent_extra=get_user_agent()),
)
super().__init__(
additional_properties=additional_properties,
middleware=middleware,
function_invocation_configuration=function_invocation_configuration,
)
self.model = chat_model
self.region = region
@staticmethod
def _create_session(settings: BedrockSettings) -> Boto3Session:
session_kwargs: dict[str, Any] = {"region_name": settings.get("region") or DEFAULT_REGION}
access_key = settings.get("access_key")
secret_key = settings.get("secret_key")
session_token = settings.get("session_token")
if access_key is not None and secret_key is not None:
session_kwargs["aws_access_key_id"] = access_key.get_secret_value()
session_kwargs["aws_secret_access_key"] = secret_key.get_secret_value()
if session_token is not None:
session_kwargs["aws_session_token"] = session_token.get_secret_value()
return Boto3Session(**session_kwargs)
def _invoke_converse(self, request: Mapping[str, Any]) -> dict[str, Any]:
try:
response = self._bedrock_client.converse(**request)
if not isinstance(response, Mapping):
raise ChatClientInvalidResponseException("Bedrock converse response must be a mapping.")
return response
except ClientError as e:
error_details = e.response.get("Error", {})
error_code = error_details.get("Code", "")
error_message = error_details.get("Message", "")
# "outputConfig" in error_message catches cases where Bedrock explicitly
# rejects the outputConfig field (unsupported model). Other ValidationExceptions
# (e.g. malformed schema shape, invalid property values) will not mention
# "outputConfig" and will bubble up as raw ClientError without being misdiagnosed.
if error_code == "ValidationException" and (
"outputconfig" in error_message.lower() or "outputconfig" in str(e).lower()
):
raise ValueError(
f"Model '{self.model}' does not support structured output via outputConfig.textFormat. "
"Check the model's Bedrock Converse outputConfig/textFormat support. "
f"AWS error Code: {error_code}. AWS error Message: {error_message}"
) from e
raise
@override
def _inner_get_response(
self,
*,
messages: Sequence[Message],
options: Mapping[str, Any],
stream: bool = False,
**kwargs: Any,
) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]:
request = self._prepare_options(messages, options, **kwargs)
if stream:
# Streaming mode - simulate streaming by yielding a single update
async def _stream() -> AsyncIterable[ChatResponseUpdate]:
response = await asyncio.to_thread(self._invoke_converse, request)
parsed_response = self._process_converse_response(response, options)
contents = list(parsed_response.messages[0].contents if parsed_response.messages else [])
if parsed_response.usage_details:
contents.append(Content.from_usage(usage_details=parsed_response.usage_details))
raw_finish_reason = (
parsed_response.finish_reason if isinstance(parsed_response.finish_reason, str) else None
)
finish_reason = self._map_finish_reason(raw_finish_reason)
yield ChatResponseUpdate(
response_id=parsed_response.response_id,
contents=contents,
model=parsed_response.model,
finish_reason=finish_reason,
raw_representation=parsed_response.raw_representation,
)
return self._build_response_stream(_stream(), response_format=options.get("response_format"))
# Non-streaming mode
async def _get_response() -> ChatResponse:
raw_response = await asyncio.to_thread(self._invoke_converse, request)
return self._process_converse_response(raw_response, options)
return _get_response()
def _prepare_options(
self,
messages: Sequence[Message],
options: Mapping[str, Any],
**kwargs: Any,
) -> dict[str, Any]:
model = options.get("model") or self.model
if not model:
raise ValueError(
"Bedrock model is required. Set via chat options or BEDROCK_CHAT_MODEL environment variable."
)
system_prompts, conversation = self._prepare_bedrock_messages(messages)
if not conversation:
raise ValueError("At least one non-system message is required for Bedrock requests.")
# Prepend instructions from options if they exist
if instructions := options.get("instructions"):
system_prompts = [{"text": instructions}, *system_prompts]
run_options: dict[str, Any] = {
"modelId": model,
"messages": conversation,
"inferenceConfig": {"maxTokens": options.get("max_tokens", DEFAULT_MAX_TOKENS)},
}
if system_prompts:
run_options["system"] = system_prompts
if (temperature := options.get("temperature")) is not None:
run_options["inferenceConfig"]["temperature"] = temperature
if (top_p := options.get("top_p")) is not None:
run_options["inferenceConfig"]["topP"] = top_p
if (stop := options.get("stop")) is not None:
run_options["inferenceConfig"]["stopSequences"] = stop
tool_config = self._prepare_tools(options.get("tools"))
if tool_mode := validate_tool_mode(options.get("tool_choice")):
if "allowed_tools" in tool_mode:
logger.warning("allowed_tools is not supported by Bedrock; the setting will be ignored")
match tool_mode.get("mode"):
case "none":
# Bedrock doesn't support toolChoice "none".
# Omit toolConfig entirely so the model won't attempt tool calls.
tool_config = None
case "auto":
if tool_config and "tools" in tool_config:
tool_config["toolChoice"] = {"auto": {}}
case "required":
if not (tool_config and "tools" in tool_config):
raise ValueError(
"tool_choice='required' requires at least one tool to be configured, "
"but no tools were provided."
)
if required_name := tool_mode.get("required_function_name"):
tool_config["toolChoice"] = {"tool": {"name": required_name}}
else:
tool_config["toolChoice"] = {"any": {}}
case _:
raise ValueError(f"Unsupported tool mode for Bedrock: {tool_mode.get('mode')}")
if tool_config:
run_options["toolConfig"] = tool_config
if output_config := self._prepare_output_config(options.get("response_format")):
run_options["outputConfig"] = output_config
return run_options
def _prepare_bedrock_messages(
self, messages: Sequence[Message]
) -> tuple[list[dict[str, str]], list[dict[str, Any]]]:
prompts: list[dict[str, str]] = []
conversation: list[dict[str, Any]] = []
pending_tool_use_ids: deque[str] = deque()
for message in messages:
if message.role == "system":
text_value = message.text
if text_value:
prompts.append({"text": text_value})
continue
content_blocks = self._convert_message_to_content_blocks(message)
if not content_blocks:
continue
role = ROLE_MAP.get(message.role, "user")
if role == "assistant":
pending_tool_use_ids = deque(
block["toolUse"]["toolUseId"]
for block in content_blocks
if isinstance(block, MutableMapping) and "toolUse" in block
)
elif message.role == "tool":
content_blocks = self._align_tool_results_with_pending(content_blocks, pending_tool_use_ids)
pending_tool_use_ids.clear()
if not content_blocks:
continue
else:
pending_tool_use_ids.clear()
conversation.append({"role": role, "content": content_blocks})
return prompts, conversation
def _align_tool_results_with_pending(
self, content_blocks: list[dict[str, Any]], pending_tool_use_ids: deque[str]
) -> list[dict[str, Any]]:
if not content_blocks:
return content_blocks
if not pending_tool_use_ids:
# No pending tool calls; drop toolResult blocks to avoid Bedrock validation errors
return [
block for block in content_blocks if not (isinstance(block, MutableMapping) and "toolResult" in block)
]
aligned_blocks: list[dict[str, Any]] = []
pending = deque(pending_tool_use_ids)
for block in content_blocks:
if not isinstance(block, MutableMapping):
aligned_blocks.append(block)
continue
tool_result = block.get("toolResult")
if not tool_result:
aligned_blocks.append(block)
continue
if not pending:
logger.debug("Dropping extra tool result block due to missing pending tool uses: %s", block)
continue
tool_use_id = tool_result.get("toolUseId")
if tool_use_id:
try:
pending.remove(tool_use_id)
except ValueError:
logger.debug("Tool result references unknown toolUseId '%s'. Dropping block.", tool_use_id)
continue
else:
tool_result["toolUseId"] = pending.popleft()
aligned_blocks.append(block)
return aligned_blocks
def _convert_message_to_content_blocks(self, message: Message) -> list[dict[str, Any]]:
blocks: list[dict[str, Any]] = []
for content in message.contents:
block = self._convert_content_to_bedrock_block(content)
if block is None:
logger.debug("Skipping unsupported content type for Bedrock: %s", type(content))
continue
blocks.append(block)
return blocks
def _convert_content_to_bedrock_block(self, content: Content) -> dict[str, Any] | None:
match content.type:
case "text":
return {"text": content.text}
case "function_call":
arguments = content.parse_arguments() or {}
return {
"toolUse": {
"toolUseId": content.call_id or self._generate_tool_call_id(),
"name": content.name,
"input": arguments,
}
}
case "function_result":
if content.items:
text_parts = [item.text or "" for item in content.items if item.type == "text"]
rich_items = [item for item in content.items if item.type in ("data", "uri")]
if rich_items:
logger.warning(
"Bedrock does not support rich content (images, audio) in tool results. "
"Rich content items will be omitted."
)
tool_result_text = "\n".join(text_parts) if text_parts else ""
tool_result_blocks = self._convert_tool_result_to_blocks(tool_result_text)
else:
tool_result_blocks = self._convert_tool_result_to_blocks(content.result)
tool_result_block = {
"toolResult": {
"toolUseId": content.call_id,
"content": tool_result_blocks,
"status": "error" if content.exception else "success",
}
}
if content.exception:
tool_result = tool_result_block["toolResult"]
existing_content = tool_result.get("content")
content_list: list[dict[str, Any]]
if isinstance(existing_content, list):
content_list = existing_content
else:
content_list = []
tool_result["content"] = content_list
content_list.append({"text": str(content.exception)})
return tool_result_block
case _:
# Bedrock does not support other content types at this time
pass
return None
def _convert_tool_result_to_blocks(self, result: Any) -> list[dict[str, Any]]:
if isinstance(result, str):
prepared_result = result
else:
parsed = FunctionTool.parse_result(result)
text_parts = [c.text or "" for c in parsed if c.type == "text"]
prepared_result = "\n".join(text_parts) if text_parts else str(result)
try:
parsed_result: object = json.loads(prepared_result)
except json.JSONDecodeError:
return [{"text": prepared_result}]
return self._convert_prepared_tool_result_to_blocks(parsed_result)
def _convert_prepared_tool_result_to_blocks(self, value: object) -> list[dict[str, Any]]:
if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
blocks: list[dict[str, Any]] = []
for item in value:
blocks.extend(self._convert_prepared_tool_result_to_blocks(item))
return blocks or [{"text": ""}]
return [self._normalize_tool_result_value(value)]
def _normalize_tool_result_value(self, value: object) -> dict[str, Any]:
if isinstance(value, dict):
return {"json": value}
if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
return {"json": [item for item in value]}
if isinstance(value, str):
return {"text": value}
if isinstance(value, (int, float, bool)) or value is None:
return {"json": value}
if isinstance(value, Content) and value.type == "text":
return {"text": value.text}
if hasattr(value, "to_dict"):
try:
return {"json": value.to_dict()} # type: ignore[call-arg]
except Exception: # pragma: no cover - defensive
return {"text": str(value)}
return {"text": str(value)}
def _prepare_tools(self, tools: list[FunctionTool | MutableMapping[str, Any]] | None) -> dict[str, Any] | None:
converted: list[dict[str, Any]] = []
if not tools:
return None
for tool in tools:
if isinstance(tool, MutableMapping):
converted.append(dict(tool))
continue
if isinstance(tool, FunctionTool):
converted.append({
"toolSpec": {
"name": tool.name,
"description": tool.description or "",
"inputSchema": {"json": tool.parameters()},
}
})
continue
logger.debug("Ignoring unsupported tool type for Bedrock: %s", type(tool))
return {"tools": converted} if converted else None
@staticmethod
def _generate_tool_call_id() -> str:
return f"tool-call-{uuid4().hex}"
def _process_converse_response(
self, response: dict[str, Any], options: Mapping[str, Any] | None = None
) -> ChatResponse:
"""Convert Bedrock Converse API response to ChatResponse."""
output = response.get("output") or {}
message = output.get("message") or {}
content_blocks = message.get("content") or []
contents = self._parse_message_contents(content_blocks)
chat_message = Message(role="assistant", contents=contents, raw_representation=message)
usage_source = response.get("usage") or output.get("usage")
usage_details = self._parse_usage(usage_source)
finish_reason = self._map_finish_reason(output.get("completionReason") or response.get("stopReason"))
response_id = response.get("responseId") or message.get("id")
model = response.get("modelId") or output.get("modelId") or self.model
return ChatResponse(
response_id=response_id,
messages=[chat_message],
usage_details=usage_details,
model=model,
finish_reason=finish_reason,
response_format=options.get("response_format") if options else None,
raw_representation=response,
)
def _parse_usage(self, usage: dict[str, Any] | None) -> UsageDetails | None:
if not usage:
return None
details: UsageDetails = {}
if (input_tokens := usage.get("inputTokens")) is not None:
details["input_token_count"] = input_tokens
if (output_tokens := usage.get("outputTokens")) is not None:
details["output_token_count"] = output_tokens
if (total_tokens := usage.get("totalTokens")) is not None:
details["total_token_count"] = total_tokens
# Bedrock Converse reports these when prompt caching is active.
if (cache_read := usage.get("cacheReadInputTokens")) is not None:
details["cache_read_input_token_count"] = cache_read
if (cache_write := usage.get("cacheWriteInputTokens")) is not None:
details["cache_creation_input_token_count"] = cache_write
return details or None
def _parse_message_contents(self, content_blocks: Sequence[dict[str, Any]]) -> list[Any]:
contents: list[Any] = []
for block in content_blocks:
if text_value := block.get("text"):
contents.append(Content.from_text(text=text_value, raw_representation=block))
continue
if (json_value := block.get("json")) is not None:
contents.append(
Content.from_text(text=json.dumps(json_value, ensure_ascii=False), raw_representation=block)
)
continue
tool_use_value = block.get("toolUse")
tool_use = (
tool_use_value
if isinstance(tool_use_value, dict)
else dict(tool_use_value)
if isinstance(tool_use_value, Mapping)
else None
)
if tool_use is not None:
tool_name_value = tool_use.get("name")
tool_name = tool_name_value if isinstance(tool_name_value, str) else None
if not tool_name:
raise ChatClientInvalidResponseException(
"Bedrock response missing required tool name in toolUse block."
)
tool_use_id = tool_use.get("toolUseId")
contents.append(
Content.from_function_call(
call_id=tool_use_id if isinstance(tool_use_id, str) else self._generate_tool_call_id(),
name=tool_name,
arguments=tool_use.get("input"),
raw_representation=block,
)
)
continue
tool_result_value = block.get("toolResult")
tool_result = (
tool_result_value
if isinstance(tool_result_value, dict)
else dict(tool_result_value)
if isinstance(tool_result_value, Mapping)
else None
)
if tool_result is not None:
status_value = tool_result.get("status")
status = (status_value if isinstance(status_value, str) else "success").lower()
exception = None
if status not in {"success", "ok"}:
exception = RuntimeError(f"Bedrock tool result status: {status}")
result_value = self._convert_bedrock_tool_result_to_value(tool_result.get("content"))
tool_use_id = tool_result.get("toolUseId")
contents.append(
Content.from_function_result(
call_id=tool_use_id if isinstance(tool_use_id, str) else self._generate_tool_call_id(),
result=result_value,
exception=str(exception) if exception else None,
raw_representation=block,
)
)
continue
logger.debug("Ignoring unsupported Bedrock content block: %s", block)
return contents
def _map_finish_reason(self, reason: str | None) -> FinishReasonLiteral | None:
if not reason:
return None
return FINISH_REASON_MAP.get(reason.lower())
def _prepare_output_config(self, response_format: Any | None) -> dict[str, Any] | None:
"""Convert response_format into the AWS Bedrock outputConfig wire format.
Args:
response_format: A Pydantic model class or a dict schema, or None.
Returns:
A dict for the Converse API ``outputConfig`` parameter, or None if
response_format is not set.
"""
if response_format is None:
return None
if isinstance(response_format, Mapping):
if "json_schema" in response_format:
# Shape A — OpenAI-style wrapper
json_schema_config = response_format["json_schema"]
schema_src = json_schema_config.get("schema", {})
name = json_schema_config.get("name", "output_schema")
elif "schema" in response_format:
# Shape B — inner shape directly {"name": ..., "schema": ...}
schema_src = response_format["schema"]
name = response_format.get("name", "output_schema")
else:
# Shape C — assume entire dict is the raw schema
logger.warning(
"response_format dict has no 'json_schema' or 'schema' key; "
"treating entire dict as raw JSON schema."
)
schema_src = dict(response_format)
name = "output_schema"
if isinstance(schema_src, str):
schema_src = json.loads(schema_src)
schema = copy.deepcopy(schema_src)
else:
if not isinstance(response_format, type) or not issubclass(response_format, BaseModel):
raise TypeError("response_format must be None, a dict JSON schema, or a Pydantic BaseModel subclass.")
# response_format is a Pydantic model class
schema = response_format.model_json_schema()
name = response_format.__name__
self._set_additional_properties_false(schema)
json_schema: dict[str, Any] = {
"name": name,
"schema": json.dumps(schema),
}
description = getattr(response_format, "__doc__", None) if not isinstance(response_format, Mapping) else None
if description and isinstance(description, str) and description.strip():
json_schema["description"] = description.strip()
return {
"textFormat": {
"type": "json_schema",
"structure": {"jsonSchema": json_schema},
}
}
def _set_additional_properties_false(self, schema: dict[str, Any]) -> None:
"""Recursively set additionalProperties: false on all object types in a JSON schema.
AWS requires strict schema enforcement. This mirrors the approach used by
AnthropicChatClient._prepare_response_format().
Args:
schema: The JSON schema dict to modify in-place.
"""
visited: set[int] = set()
def walk(node: Any) -> None:
if isinstance(node, dict):
node_id = id(node)
if node_id in visited:
return
visited.add(node_id)
if node.get("type") == "object" or ("properties" in node and "type" not in node):
existing = node.get("additionalProperties")
if existing is None or existing is True:
node["additionalProperties"] = False
for value in node.values():
if isinstance(value, (dict, list)):
walk(value)
elif isinstance(node, list):
node_id = id(node)
if node_id in visited:
return
visited.add(node_id)
for item in node:
if isinstance(item, (dict, list)):
walk(item)
walk(schema)
def service_url(self) -> str:
"""Returns the service URL for the Bedrock runtime in the configured AWS region.
Returns:
str: The Bedrock runtime service URL.
"""
return f"https://bedrock-runtime.{self.region}.amazonaws.com"
def _convert_bedrock_tool_result_to_value(self, content: object) -> object:
if not content:
return None
if isinstance(content, Sequence) and not isinstance(content, (str, bytes, bytearray)):
values: list[object] = []
for item in content:
item_dict = item if isinstance(item, dict) else dict(item) if isinstance(item, Mapping) else None
if item_dict is not None:
text_value = item_dict.get("text")
if isinstance(text_value, str):
values.append(text_value)
continue
if "json" in item_dict:
values.append(item_dict["json"])
continue
values.append(item)
return values[0] if len(values) == 1 else values
content_dict = content if isinstance(content, dict) else dict(content) if isinstance(content, Mapping) else None
if content_dict is not None:
text_value = content_dict.get("text")
if isinstance(text_value, str):
return text_value
if "json" in content_dict:
return content_dict["json"]
return content
@@ -0,0 +1,294 @@
# Copyright (c) Microsoft. All rights reserved.
# type: ignore
# Because the Bedrock client does not have typing, we are ignoring type issues in this module.
from __future__ import annotations
import asyncio
import json
import logging
import sys
from collections.abc import Sequence
from typing import Any, ClassVar, Generic, TypedDict
from agent_framework import (
BaseEmbeddingClient,
Embedding,
EmbeddingGenerationOptions,
GeneratedEmbeddings,
SecretString,
UsageDetails,
load_settings,
)
from agent_framework._telemetry import get_user_agent
from agent_framework.observability import EmbeddingTelemetryLayer
from boto3.session import Session as Boto3Session
from botocore.client import BaseClient
from botocore.config import Config as BotoConfig
if sys.version_info >= (3, 13):
from typing import TypeVar # pragma: no cover
else:
from typing_extensions import TypeVar # pragma: no cover
logger = logging.getLogger("agent_framework.bedrock")
DEFAULT_REGION = "us-east-1"
class BedrockEmbeddingSettings(TypedDict, total=False):
"""Bedrock embedding settings."""
region: str | None
embedding_model: str | None
access_key: SecretString | None
secret_key: SecretString | None
session_token: SecretString | None
class BedrockEmbeddingOptions(EmbeddingGenerationOptions, total=False):
"""Bedrock-specific embedding options.
Extends EmbeddingGenerationOptions with Bedrock-specific fields.
Examples:
.. code-block:: python
from agent_framework_bedrock import BedrockEmbeddingOptions
options: BedrockEmbeddingOptions = {
"model": "amazon.titan-embed-text-v2:0",
"dimensions": 1024,
"normalize": True,
}
"""
normalize: bool
BedrockEmbeddingOptionsT = TypeVar(
"BedrockEmbeddingOptionsT",
bound=TypedDict, # type: ignore[valid-type]
default="BedrockEmbeddingOptions",
covariant=True,
)
class RawBedrockEmbeddingClient(
BaseEmbeddingClient[str, list[float], BedrockEmbeddingOptionsT],
Generic[BedrockEmbeddingOptionsT],
):
"""Raw Bedrock embedding client without telemetry.
Keyword Args:
model: The Bedrock embedding model ID (e.g. "amazon.titan-embed-text-v2:0").
Can also be set via environment variable BEDROCK_EMBEDDING_MODEL.
region: AWS region. Will try to load from BEDROCK_REGION env var,
if not set, the regular Boto3 configuration/loading applies
(which may include other env vars, config files, or instance metadata).
access_key: AWS access key for manual credential injection.
secret_key: AWS secret key paired with access_key.
session_token: AWS session token for temporary credentials.
client: Preconfigured Bedrock runtime client.
boto3_session: Custom boto3 session used to build the runtime client.
env_file_path: Path to .env file for settings.
env_file_encoding: Encoding for .env file.
"""
def __init__(
self,
*,
region: str | None = None,
model: str | None = None,
access_key: str | None = None,
secret_key: str | None = None,
session_token: str | None = None,
client: BaseClient | None = None,
boto3_session: Boto3Session | None = None,
additional_properties: dict[str, Any] | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize a raw Bedrock embedding client."""
settings = load_settings(
BedrockEmbeddingSettings,
env_prefix="BEDROCK_",
required_fields=["embedding_model"],
region=region,
embedding_model=model,
access_key=access_key,
secret_key=secret_key,
session_token=session_token,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
resolved_region = settings.get("region") or DEFAULT_REGION
if client:
self._bedrock_client = client
else:
if not boto3_session:
session_kwargs: dict[str, Any] = {}
if region := settings.get("region"):
session_kwargs["region_name"] = region
if (access_key := settings.get("access_key")) and (secret_key := settings.get("secret_key")):
session_kwargs["aws_access_key_id"] = access_key.get_secret_value()
session_kwargs["aws_secret_access_key"] = secret_key.get_secret_value()
if session_token := settings.get("session_token"):
session_kwargs["aws_session_token"] = session_token.get_secret_value()
boto3_session = Boto3Session(**session_kwargs)
region_name = boto3_session.region_name
self._bedrock_client = boto3_session.client(
"bedrock-runtime",
region_name=region_name or resolved_region,
config=BotoConfig(user_agent_extra=get_user_agent()),
)
self.model: str = settings["embedding_model"] # type: ignore[assignment]
self.region = resolved_region
super().__init__(additional_properties=additional_properties)
def service_url(self) -> str:
"""Get the URL of the service."""
return str(self._bedrock_client.meta.endpoint_url)
async def get_embeddings(
self,
values: Sequence[str],
*,
options: BedrockEmbeddingOptionsT | None = None,
) -> GeneratedEmbeddings[list[float], BedrockEmbeddingOptionsT]:
"""Call the Bedrock invoke_model API for embeddings.
Uses the Amazon Titan Embeddings model format. Each value is embedded
individually since Titan's invoke_model API accepts one input at a time.
Args:
values: The text values to generate embeddings for.
options: Optional embedding generation options.
Returns:
Generated embeddings with usage metadata.
Raises:
ValueError: If model is not provided or values is empty.
"""
if not values:
return GeneratedEmbeddings([], options=options)
opts: dict[str, Any] = dict(options) if options else {}
model = opts.get("model") or self.model
if not model:
raise ValueError("model is required")
embedding_results = await asyncio.gather(
*(self._generate_embedding_for_text(opts, model, text) for text in values)
)
embeddings: list[Embedding[list[float]]] = []
total_input_tokens = 0
for embedding, input_tokens in embedding_results:
embeddings.append(embedding)
total_input_tokens += input_tokens
usage_dict: UsageDetails | None = None
if total_input_tokens > 0:
usage_dict = {"input_token_count": total_input_tokens}
return GeneratedEmbeddings(embeddings, options=options, usage=usage_dict)
async def _generate_embedding_for_text(
self,
opts: dict[str, Any],
model: str,
text: str,
) -> tuple[Embedding[list[float]], int]:
body: dict[str, Any] = {"inputText": text}
if dimensions := opts.get("dimensions"):
body["dimensions"] = dimensions
if (normalize := opts.get("normalize")) is not None:
body["normalize"] = normalize
response = await asyncio.to_thread(
self._bedrock_client.invoke_model,
modelId=model,
contentType="application/json",
accept="application/json",
body=json.dumps(body),
)
response_body = json.loads(response["body"].read())
embedding = Embedding(
vector=response_body["embedding"],
dimensions=len(response_body["embedding"]),
model=model,
)
input_tokens = int(response_body.get("inputTextTokenCount", 0))
return embedding, input_tokens
class BedrockEmbeddingClient(
EmbeddingTelemetryLayer[str, list[float], BedrockEmbeddingOptionsT],
RawBedrockEmbeddingClient[BedrockEmbeddingOptionsT],
Generic[BedrockEmbeddingOptionsT],
):
"""Bedrock embedding client with telemetry support.
Uses the Amazon Titan Embeddings model via Bedrock's invoke_model API.
Keyword Args:
model: The Bedrock embedding model ID (e.g. "amazon.titan-embed-text-v2:0").
Can also be set via environment variable BEDROCK_EMBEDDING_MODEL.
region: AWS region. Defaults to "us-east-1".
Can also be set via environment variable BEDROCK_REGION.
access_key: AWS access key for manual credential injection.
secret_key: AWS secret key paired with access_key.
session_token: AWS session token for temporary credentials.
client: Preconfigured Bedrock runtime client.
boto3_session: Custom boto3 session used to build the runtime client.
env_file_path: Path to .env file for settings.
env_file_encoding: Encoding for .env file.
Examples:
.. code-block:: python
from agent_framework_bedrock import BedrockEmbeddingClient
# Using default AWS credentials
client = BedrockEmbeddingClient(
model="amazon.titan-embed-text-v2:0",
)
# Generate embeddings
result = await client.get_embeddings(["Hello, world!"])
print(result[0].vector)
"""
OTEL_PROVIDER_NAME: ClassVar[str] = "aws.bedrock"
def __init__(
self,
*,
region: str | None = None,
model: str | None = None,
access_key: str | None = None,
secret_key: str | None = None,
session_token: str | None = None,
client: BaseClient | None = None,
boto3_session: Boto3Session | None = None,
otel_provider_name: str | None = None,
additional_properties: dict[str, Any] | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize a Bedrock embedding client."""
super().__init__(
region=region,
model=model,
access_key=access_key,
secret_key=secret_key,
session_token=session_token,
client=client,
boto3_session=boto3_session,
additional_properties=additional_properties,
otel_provider_name=otel_provider_name,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
+97
View File
@@ -0,0 +1,97 @@
[project]
name = "agent-framework-bedrock"
description = "Amazon Bedrock integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260709"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
urls.release_notes = "https://github.com/microsoft/agent-framework/releases?q=tag%3Apython-1&expanded=true"
urls.issues = "https://github.com/microsoft/agent-framework/issues"
classifiers = [
"License :: OSI Approved :: MIT License",
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.11.0,<2",
"boto3>=1.35.0,<2.0.0",
"botocore>=1.35.0,<2.0.0",
]
[tool.uv]
prerelease = "if-necessary-or-explicit"
environments = [
"sys_platform == 'darwin'",
"sys_platform == 'linux'",
"sys_platform == 'win32'"
]
[tool.uv-dynamic-versioning]
fallback-version = "0.0.0"
[tool.pytest.ini_options]
testpaths = 'tests'
addopts = "-ra -q -r fEX"
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
filterwarnings = []
markers = [
"integration: marks tests as integration tests that require external services",
]
timeout = 120
[tool.ruff]
extend = "../../pyproject.toml"
[tool.coverage.run]
omit = [
"**/__init__.py"
]
[tool.pyright]
extends = "../../pyproject.toml"
include = ["agent_framework_bedrock"]
[tool.mypy]
plugins = ['pydantic.mypy']
strict = true
python_version = "3.10"
ignore_missing_imports = true
disallow_untyped_defs = true
no_implicit_optional = true
check_untyped_defs = true
warn_return_any = true
show_error_codes = true
warn_unused_ignores = false
disallow_incomplete_defs = true
disallow_untyped_decorators = true
[tool.bandit]
targets = ["agent_framework_bedrock"]
exclude_dirs = ["tests"]
[tool.poe]
executor.type = "uv"
include = "../../shared_tasks.toml"
[tool.poe.tasks.mypy]
help = "Run MyPy for this package."
cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_bedrock"
[tool.poe.tasks.test]
help = "Run the default unit test suite for this package."
cmd = 'pytest -m "not integration" --cov=agent_framework_bedrock --cov-report=term-missing:skip-covered tests'
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
@@ -0,0 +1,169 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
import json
import os
from typing import Any
from unittest.mock import MagicMock
import pytest
from agent_framework import Embedding, GeneratedEmbeddings
from agent_framework_bedrock import BedrockEmbeddingClient, BedrockEmbeddingOptions
class _StubBedrockEmbeddingRuntime:
"""Stub for the Bedrock runtime client that handles invoke_model for embeddings."""
def __init__(self) -> None:
self.calls: list[dict[str, Any]] = []
self.meta = MagicMock(endpoint_url="https://bedrock-runtime.us-west-2.amazonaws.com")
def invoke_model(self, **kwargs: Any) -> dict[str, Any]:
self.calls.append(kwargs)
body = json.loads(kwargs.get("body", "{}"))
# Simulate Titan embedding response
dimensions = body.get("dimensions", 3)
return {
"body": MagicMock(
read=lambda: json.dumps({
"embedding": [0.1 * (i + 1) for i in range(dimensions)],
"inputTextTokenCount": 5,
}).encode()
),
}
async def test_bedrock_embedding_construction() -> None:
"""Test construction with explicit parameters."""
stub = _StubBedrockEmbeddingRuntime()
client = BedrockEmbeddingClient(
model="amazon.titan-embed-text-v2:0",
region="us-west-2",
client=stub, # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
assert client.model == "amazon.titan-embed-text-v2:0"
assert client.region == "us-west-2"
async def test_bedrock_embedding_construction_missing_model_raises(monkeypatch: pytest.MonkeyPatch) -> None:
"""Test that missing model raises an error."""
monkeypatch.delenv("BEDROCK_EMBEDDING_MODEL", raising=False)
from agent_framework.exceptions import SettingNotFoundError
with pytest.raises(SettingNotFoundError):
BedrockEmbeddingClient(region="us-west-2")
async def test_bedrock_embedding_get_embeddings() -> None:
"""Test generating embeddings via the Bedrock invoke_model API."""
stub = _StubBedrockEmbeddingRuntime()
client = BedrockEmbeddingClient(
model="amazon.titan-embed-text-v2:0",
region="us-west-2",
client=stub, # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
result = await client.get_embeddings(["hello", "world"])
assert isinstance(result, GeneratedEmbeddings)
assert len(result) == 2
assert len(result[0].vector) == 3
assert len(result[1].vector) == 3
assert result[0].model == "amazon.titan-embed-text-v2:0"
assert result.usage == {"input_token_count": 10}
# Two calls since Titan processes one input at a time
assert len(stub.calls) == 2
call_texts = {json.loads(call["body"])["inputText"] for call in stub.calls}
assert call_texts == {"hello", "world"}
async def test_bedrock_embedding_get_embeddings_empty_input() -> None:
"""Test generating embeddings with empty input."""
stub = _StubBedrockEmbeddingRuntime()
client = BedrockEmbeddingClient(
model="amazon.titan-embed-text-v2:0",
region="us-west-2",
client=stub, # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
result = await client.get_embeddings([])
assert isinstance(result, GeneratedEmbeddings)
assert len(result) == 0
assert len(stub.calls) == 0
async def test_bedrock_embedding_get_embeddings_with_options() -> None:
"""Test generating embeddings with custom options."""
stub = _StubBedrockEmbeddingRuntime()
client = BedrockEmbeddingClient(
model="amazon.titan-embed-text-v2:0",
region="us-west-2",
client=stub, # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
options: BedrockEmbeddingOptions = {
"dimensions": 5,
"normalize": True,
}
result = await client.get_embeddings(["hello"], options=options) # ty: ignore[invalid-argument-type]
assert len(result) == 1
assert len(result[0].vector) == 5
body = json.loads(stub.calls[0]["body"])
assert body["dimensions"] == 5
assert body["normalize"] is True
async def test_bedrock_embedding_get_embeddings_no_model_raises() -> None:
"""Test that missing model at call time raises ValueError."""
stub = _StubBedrockEmbeddingRuntime()
client = BedrockEmbeddingClient(
model="amazon.titan-embed-text-v2:0",
region="us-west-2",
client=stub, # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
client.model = None # type: ignore[assignment] # ty: ignore[invalid-assignment]
with pytest.raises(ValueError, match="model is required"):
await client.get_embeddings(["hello"])
async def test_bedrock_embedding_default_region() -> None:
"""Test that default region is us-east-1."""
stub = _StubBedrockEmbeddingRuntime()
client = BedrockEmbeddingClient(
model="amazon.titan-embed-text-v2:0",
client=stub, # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
assert client.region == "us-east-1"
# region: Integration Tests
skip_if_bedrock_embedding_integration_tests_disabled = pytest.mark.skipif(
os.getenv("BEDROCK_EMBEDDING_MODEL", "") in ("", "test-model")
or not (os.getenv("AWS_ACCESS_KEY_ID") or os.getenv("BEDROCK_ACCESS_KEY")),
reason="No real Bedrock embedding model or AWS credentials provided; skipping integration tests.",
)
@pytest.mark.flaky
@pytest.mark.integration
@skip_if_bedrock_embedding_integration_tests_disabled
async def test_bedrock_embedding_integration() -> None:
"""Integration test for Bedrock embedding client."""
client = BedrockEmbeddingClient()
result = await client.get_embeddings(["Hello, world!", "How are you?"])
assert isinstance(result, GeneratedEmbeddings)
assert len(result) == 2
for embedding in result:
assert isinstance(embedding, Embedding)
assert isinstance(embedding.vector, list)
assert len(embedding.vector) > 0
assert all(isinstance(v, float) for v in embedding.vector)
@@ -0,0 +1,236 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
import json
from typing import Any
import pytest
from agent_framework import Agent, Content, Message
from agent_framework_bedrock import BedrockChatClient
class _StubBedrockRuntime:
def __init__(self) -> None:
self.calls: list[dict[str, Any]] = []
def converse(self, **kwargs: Any) -> dict[str, Any]:
self.calls.append(kwargs)
return {
"modelId": kwargs["modelId"],
"responseId": "resp-123",
"usage": {"inputTokens": 10, "outputTokens": 5, "totalTokens": 15},
"output": {
"completionReason": "end_turn",
"message": {
"id": "msg-1",
"role": "assistant",
"content": [{"text": "Bedrock says hi"}],
},
},
}
def _make_client() -> BedrockChatClient:
"""Create a BedrockChatClient with a stub runtime for unit tests."""
return BedrockChatClient(
model="amazon.titan-text",
region="us-west-2",
client=_StubBedrockRuntime(), # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
def test_agent_accepts_bedrock_chat_client() -> None:
client = _make_client()
agent = Agent(client=client, instructions="test agent")
assert agent.client is client
async def test_get_response_invokes_bedrock_runtime() -> None:
stub = _StubBedrockRuntime()
client = BedrockChatClient(
model="amazon.titan-text",
region="us-west-2",
client=stub, # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
messages = [
Message(role="system", contents=[Content.from_text(text="You are concise.")]),
Message(role="user", contents=[Content.from_text(text="hello")]),
]
response = await client.get_response(messages=messages, options={"max_tokens": 32})
assert stub.calls, "Expected the runtime client to be called"
payload = stub.calls[0]
assert payload["modelId"] == "amazon.titan-text"
assert payload["messages"][0]["content"][0]["text"] == "hello"
assert response.messages[0].contents[0].text == "Bedrock says hi"
assert response.usage_details and response.usage_details["input_token_count"] == 10
def test_build_request_requires_non_system_messages() -> None:
client = BedrockChatClient(
model="amazon.titan-text",
region="us-west-2",
client=_StubBedrockRuntime(), # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
messages = [Message(role="system", contents=[Content.from_text(text="Only system text")])]
with pytest.raises(ValueError):
client._prepare_options(messages, {})
def test_prepare_options_tool_choice_none_omits_tool_config() -> None:
"""When tool_choice='none', toolConfig must be omitted entirely.
Bedrock's Converse API only accepts 'auto', 'any', or 'tool' as valid
toolChoice keys. Sending {"none": {}} causes a ParamValidationError.
The fix omits toolConfig so the model won't attempt tool calls.
Fixes #4529.
"""
client = _make_client()
messages = [Message(role="user", contents=[Content.from_text(text="hello")])]
# Even when tools are provided, tool_choice="none" should strip toolConfig
options: dict[str, Any] = {
"tool_choice": "none",
"tools": [
{"toolSpec": {"name": "get_weather", "description": "Get weather", "inputSchema": {"json": {}}}},
],
}
request = client._prepare_options(messages, options)
assert "toolConfig" not in request, (
f"toolConfig should be omitted when tool_choice='none', got: {request.get('toolConfig')}"
)
def test_prepare_options_tool_choice_auto_includes_tool_config() -> None:
"""When tool_choice='auto', toolConfig.toolChoice should be {'auto': {}}."""
client = _make_client()
messages = [Message(role="user", contents=[Content.from_text(text="hello")])]
options: dict[str, Any] = {
"tool_choice": "auto",
"tools": [
{"toolSpec": {"name": "get_weather", "description": "Get weather", "inputSchema": {"json": {}}}},
],
}
request = client._prepare_options(messages, options)
assert "toolConfig" in request
assert request["toolConfig"]["toolChoice"] == {"auto": {}}
def test_prepare_options_tool_choice_required_includes_any() -> None:
"""When tool_choice='required' (no specific function), toolChoice should be {'any': {}}."""
client = _make_client()
messages = [Message(role="user", contents=[Content.from_text(text="hello")])]
options: dict[str, Any] = {
"tool_choice": "required",
"tools": [
{"toolSpec": {"name": "get_weather", "description": "Get weather", "inputSchema": {"json": {}}}},
],
}
request = client._prepare_options(messages, options)
assert "toolConfig" in request
assert request["toolConfig"]["toolChoice"] == {"any": {}}
def test_prepare_options_tool_choice_auto_without_tools_omits_tool_config() -> None:
"""When tool_choice='auto' but no tools are provided, toolConfig must be omitted.
Without tools, setting toolChoice would cause a ParamValidationError from Bedrock.
"""
client = _make_client()
messages = [Message(role="user", contents=[Content.from_text(text="hello")])]
options: dict[str, Any] = {
"tool_choice": "auto",
}
request = client._prepare_options(messages, options)
assert "toolConfig" not in request, (
f"toolConfig should be omitted when no tools are provided, got: {request.get('toolConfig')}"
)
def test_prepare_options_tool_choice_required_without_tools_raises() -> None:
"""When tool_choice='required' but no tools are provided, a ValueError must be raised."""
client = _make_client()
messages = [Message(role="user", contents=[Content.from_text(text="hello")])]
options: dict[str, Any] = {
"tool_choice": "required",
}
with pytest.raises(ValueError, match="tool_choice='required' requires at least one tool"):
client._prepare_options(messages, options)
def test_process_converse_response_preserves_non_ascii_in_json_block() -> None:
"""Non-ASCII text in a Bedrock ``json`` content block must be preserved, not \\uXXXX-escaped.
The Converse API can return structured ``json`` content blocks. These are serialized to
text via ``json.dumps``; without ``ensure_ascii=False`` CJK characters and emoji are escaped
to ``\\uXXXX`` sequences and surface garbled to the user.
"""
client = _make_client()
json_payload = {"greeting": "你好世界", "emoji": "🎉"}
response: dict[str, Any] = {
"modelId": "amazon.titan-text",
"output": {
"completionReason": "end_turn",
"message": {
"role": "assistant",
"content": [{"json": json_payload}],
},
},
}
chat_response = client._process_converse_response(response)
text = chat_response.messages[0].text
assert "你好世界" in text
assert "🎉" in text
# Must not be escaped to Unicode code points.
assert "\\u" not in text
# Serialized text must remain valid JSON that round-trips to the original payload.
assert json.loads(text) == json_payload
def test_parse_usage_surfaces_cache_tokens() -> None:
"""Bedrock Converse reports cache token counts when prompt caching is used."""
client = _make_client()
details = client._parse_usage({
"inputTokens": 10,
"outputTokens": 5,
"totalTokens": 15,
"cacheReadInputTokens": 8,
"cacheWriteInputTokens": 3,
})
assert details is not None
assert details["input_token_count"] == 10
assert details["cache_read_input_token_count"] == 8
assert details["cache_creation_input_token_count"] == 3
def test_parse_usage_returns_none_when_no_recognized_keys() -> None:
"""A truthy usage payload with no recognized keys yields None, not an empty mapping."""
client = _make_client()
assert client._parse_usage({"unexpected": 1}) is None
assert client._parse_usage({}) is None
assert client._parse_usage(None) is None
@@ -0,0 +1,136 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from agent_framework import (
ChatOptions,
Content,
FunctionTool,
Message,
)
from agent_framework._settings import load_settings
from pydantic import BaseModel
from agent_framework_bedrock._chat_client import BedrockChatClient, BedrockSettings
class _WeatherArgs(BaseModel):
location: str
def _build_client() -> BedrockChatClient:
fake_runtime = MagicMock()
fake_runtime.converse.return_value = {}
return BedrockChatClient(model="test-model", client=fake_runtime)
def _dummy_weather(location: str) -> str: # pragma: no cover - helper
return f"Weather in {location}"
def test_settings_load_from_environment(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("BEDROCK_REGION", "us-west-2")
monkeypatch.setenv("BEDROCK_CHAT_MODEL", "anthropic.claude-v2")
settings = load_settings(BedrockSettings, env_prefix="BEDROCK_")
assert settings["region"] == "us-west-2"
assert settings["chat_model"] == "anthropic.claude-v2"
def test_build_request_includes_tool_config() -> None:
client = _build_client()
tool = FunctionTool(name="get_weather", description="desc", func=_dummy_weather, input_model=_WeatherArgs)
options = {
"tools": [tool],
"tool_choice": {"mode": "required", "required_function_name": "get_weather"},
}
messages = [Message(role="user", contents=[Content.from_text(text="hi")])]
request = client._prepare_options(messages, options)
assert request["toolConfig"]["tools"][0]["toolSpec"]["name"] == "get_weather"
assert request["toolConfig"]["toolChoice"] == {"tool": {"name": "get_weather"}}
def test_build_request_serializes_tool_history() -> None:
client = _build_client()
options: ChatOptions = {}
messages = [
Message(role="user", contents=[Content.from_text(text="how's weather?")]),
Message(
role="assistant",
contents=[
Content.from_function_call(call_id="call-1", name="get_weather", arguments='{"location": "SEA"}')
],
),
Message(
role="tool",
contents=[Content.from_function_result(call_id="call-1", result='{"answer": "72F"}')],
),
]
request = client._prepare_options(messages, options)
assistant_block = request["messages"][1]["content"][0]["toolUse"]
result_block = request["messages"][2]["content"][0]["toolResult"]
assert assistant_block["name"] == "get_weather"
assert assistant_block["input"] == {"location": "SEA"}
assert result_block["toolUseId"] == "call-1"
assert result_block["content"][0]["json"] == {"answer": "72F"}
def test_process_response_parses_tool_use_and_result() -> None:
client = _build_client()
response = {
"modelId": "model",
"output": {
"message": {
"id": "msg-1",
"content": [
{"toolUse": {"toolUseId": "call-1", "name": "get_weather", "input": {"location": "NYC"}}},
{"text": "Calling tool"},
],
},
"completionReason": "tool_use",
},
}
chat_response = client._process_converse_response(response)
contents = chat_response.messages[0].contents
assert contents[0].type == "function_call"
assert contents[0].name == "get_weather"
assert contents[1].type == "text"
assert chat_response.finish_reason == client._map_finish_reason("tool_use")
def test_process_response_parses_tool_result() -> None:
client = _build_client()
response = {
"modelId": "model",
"output": {
"message": {
"id": "msg-2",
"content": [
{
"toolResult": {
"toolUseId": "call-1",
"status": "success",
"content": [{"json": {"answer": 42}}],
}
}
],
},
"completionReason": "end_turn",
},
}
chat_response = client._process_converse_response(response)
contents = chat_response.messages[0].contents
assert contents[0].type == "function_result"
assert "answer" in str(contents[0].result)
assert contents[0].items is not None
@@ -0,0 +1,378 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
import copy
import json
from typing import Any
from unittest.mock import MagicMock, patch
import pytest
from agent_framework import Content, Message
from botocore.exceptions import ClientError
from pydantic import BaseModel
from agent_framework_bedrock import BedrockChatClient
# region Test models
class WeatherReport(BaseModel):
city: str
temperature: float
summary: str
class NestedAddress(BaseModel):
street: str
city: str
zip_code: str
class Person(BaseModel):
name: str
age: int
address: NestedAddress
# endregion
# region Helpers
class _StubBedrockRuntime:
"""Stub that records calls and returns a canned response."""
def __init__(self, response_text: str = "Bedrock says hi") -> None:
self.calls: list[dict[str, Any]] = []
self._response_text = response_text
def converse(self, **kwargs: Any) -> dict[str, Any]:
self.calls.append(kwargs)
return {
"modelId": kwargs["modelId"],
"responseId": "resp-structured",
"usage": {"inputTokens": 10, "outputTokens": 20, "totalTokens": 30},
"output": {
"completionReason": "end_turn",
"message": {
"id": "msg-structured",
"role": "assistant",
"content": [{"text": self._response_text}],
},
},
}
def _make_client(response_text: str = "Bedrock says hi") -> tuple[BedrockChatClient, _StubBedrockRuntime]:
stub = _StubBedrockRuntime(response_text)
client = BedrockChatClient(
model="us.anthropic.claude-haiku-4-5-v1:0",
region="us-east-1",
client=stub, # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
return client, stub
def _user_messages() -> list[Message]:
return [Message(role="user", contents=[Content.from_text(text="Give me a weather report")])]
# endregion
# region Tests
def test_prepare_output_config_correct_wire_shape() -> None:
"""_prepare_output_config(WeatherReport) must produce the correct
textFormat → structure → jsonSchema shape with type: 'json_schema'."""
client, _ = _make_client()
output_config = client._prepare_output_config(WeatherReport)
assert output_config is not None
text_format = output_config["textFormat"]
assert text_format["type"] == "json_schema"
assert "structure" in text_format
json_schema = text_format["structure"]["jsonSchema"]
assert json_schema["name"] == "WeatherReport"
assert "schema" in json_schema
def test_prepare_output_config_schema_is_json_string() -> None:
"""The schema value inside jsonSchema must be a JSON string, not a dict."""
client, _ = _make_client()
output_config = client._prepare_output_config(WeatherReport)
assert output_config is not None
schema_value = output_config["textFormat"]["structure"]["jsonSchema"]["schema"]
assert isinstance(schema_value, str), f"Expected str, got {type(schema_value)}"
# Verify it's valid JSON
parsed = json.loads(schema_value)
assert isinstance(parsed, dict)
assert parsed["type"] == "object"
def test_additional_properties_false_set_recursively() -> None:
"""additionalProperties: false must be set on all nested object types."""
client, _ = _make_client()
output_config = client._prepare_output_config(Person)
assert output_config is not None
schema_str = output_config["textFormat"]["structure"]["jsonSchema"]["schema"]
schema = json.loads(schema_str)
# Top-level object
assert schema.get("additionalProperties") is False
# Check $defs for NestedAddress
defs = schema.get("$defs", {})
assert "NestedAddress" in defs, "Expected NestedAddress to be present in $defs"
assert defs["NestedAddress"].get("additionalProperties") is False, (
"Expected additionalProperties=False on nested NestedAddress schema"
)
def test_no_output_config_when_response_format_none() -> None:
"""When response_format is None, no outputConfig key should appear in the request."""
client, stub = _make_client()
messages = _user_messages()
request = client._prepare_options(messages, {"max_tokens": 100})
assert "outputConfig" not in request, (
f"outputConfig should not be present when response_format is None, got: {request.get('outputConfig')}"
)
async def test_chat_response_value_populated() -> None:
"""After a mocked response with response_format, .value should be a populated Pydantic model."""
json_response = json.dumps({"city": "Seattle", "temperature": 72.5, "summary": "Sunny and warm"})
client, stub = _make_client(response_text=json_response)
messages = _user_messages()
response = await client.get_response(
messages=messages,
options={"max_tokens": 100, "response_format": WeatherReport},
)
assert response.text == json_response
assert response.value is not None
assert isinstance(response.value, WeatherReport)
assert response.value.city == "Seattle"
assert response.value.temperature == 72.5
assert response.value.summary == "Sunny and warm"
# Verify outputConfig was sent to the API
assert len(stub.calls) == 1
api_request = stub.calls[0]
assert "outputConfig" in api_request
assert api_request["outputConfig"]["textFormat"]["type"] == "json_schema"
def test_dict_schema_response_format() -> None:
"""_prepare_output_config should work when response_format is a dict, not just a Pydantic class."""
client, _ = _make_client()
dict_schema = {
"json_schema": {
"name": "weather_output",
"schema": {
"type": "object",
"properties": {
"city": {"type": "string"},
"temp": {"type": "number"},
},
},
}
}
output_config = client._prepare_output_config(dict_schema)
assert output_config is not None
json_schema = output_config["textFormat"]["structure"]["jsonSchema"]
assert json_schema["name"] == "weather_output"
schema_parsed = json.loads(json_schema["schema"])
assert schema_parsed["type"] == "object"
assert "city" in schema_parsed["properties"]
def test_prepare_output_config_none_returns_none() -> None:
"""_prepare_output_config(None) must return None."""
client, _ = _make_client()
result = client._prepare_output_config(None)
assert result is None
async def test_chat_response_value_populated_streaming() -> None:
"""In streaming mode, .value should also be populated on the final response."""
json_response = json.dumps({"city": "Portland", "temperature": 68.0, "summary": "Cloudy"})
client, stub = _make_client(response_text=json_response)
messages = _user_messages()
stream = client.get_response(
messages=messages,
stream=True,
options={"max_tokens": 100, "response_format": WeatherReport},
)
# Consume stream and get final response
async for _ in stream:
pass
response = await stream.get_final_response()
assert response.value is not None
assert isinstance(response.value, WeatherReport)
assert response.value.city == "Portland"
# Verify outputConfig was sent
assert len(stub.calls) == 1
assert "outputConfig" in stub.calls[0]
async def test_unsupported_model_validation_exception() -> None:
"""When a model doesn't support outputConfig, a clear error should be raised."""
class _FailingStubBedrockRuntime:
def converse(self, **kwargs: Any) -> dict[str, Any]:
# Simulate botocore ClientError for ValidationException
error_response = {"Error": {"Code": "ValidationException", "Message": "Invalid field outputConfig"}}
raise ClientError(error_response, "Converse")
client = BedrockChatClient(
model="us.anthropic.claude-v2",
region="us-east-1",
client=_FailingStubBedrockRuntime(), # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type] # pyright: ignore[reportArgumentType]
)
with pytest.raises(ValueError) as exc:
await client.get_response(
messages=_user_messages(),
options={"response_format": WeatherReport},
)
assert "does not support structured output via outputConfig.textFormat" in str(exc.value)
assert "Check the model's Bedrock Converse outputConfig/textFormat support." in str(exc.value)
def test_invalid_response_format_type_raises() -> None:
"""Non-dict, non-BaseModel response_format should raise TypeError."""
client, _ = _make_client()
with pytest.raises(TypeError, match="Pydantic BaseModel subclass"):
client._prepare_output_config("not_a_valid_format")
def test_mapping_response_format_accepted() -> None:
"""A non-dict Mapping response_format must be accepted and produce
correct outputConfig, not raise TypeError."""
from collections.abc import MutableMapping
class _WrappedMapping(MutableMapping):
def __init__(self, data):
self._data = dict(data)
def __getitem__(self, key):
return self._data[key]
def __setitem__(self, key, value):
self._data[key] = value
def __delitem__(self, key):
del self._data[key]
def __iter__(self):
return iter(self._data)
def __len__(self):
return len(self._data)
client, _ = _make_client()
mapping_format = _WrappedMapping({
"json_schema": {
"name": "test_output",
"schema": {
"type": "object",
"properties": {"result": {"type": "string"}},
},
}
})
output_config = client._prepare_output_config(mapping_format)
assert output_config is not None
json_schema = output_config["textFormat"]["structure"]["jsonSchema"]
assert json_schema["name"] == "test_output"
schema = json.loads(json_schema["schema"])
assert schema.get("additionalProperties") is False
def test_shape_b_dict_schema_wire_format() -> None:
"""Dict response_format in Shape B (inner shape directly) should
produce correct outputConfig."""
client, _ = _make_client()
response_format = {
"name": "weather_output",
"schema": {
"type": "object",
"properties": {
"city": {"type": "string"},
"temperature": {"type": "number"},
},
},
}
output_config = client._prepare_output_config(response_format)
assert output_config is not None
text_format = output_config["textFormat"]
assert text_format["type"] == "json_schema"
json_schema = text_format["structure"]["jsonSchema"]
assert json_schema["name"] == "weather_output"
schema = json.loads(json_schema["schema"])
assert schema.get("additionalProperties") is False
def test_dict_schema_not_mutated() -> None:
"""Caller's dict schema must not be mutated by _prepare_output_config."""
client, _ = _make_client()
original_schema = {
"json_schema": {
"name": "test",
"schema": {
"type": "object",
"properties": {"a": {"type": "string"}},
},
}
}
snapshot = copy.deepcopy(original_schema)
client._prepare_output_config(original_schema)
assert original_schema == snapshot, "Original dict schema was mutated"
async def test_non_outputconfig_validation_exception_propagates() -> None:
"""ValidationException unrelated to outputConfig must propagate
as raw ClientError, not be caught and reclassified."""
client, _ = _make_client()
error_response = {
"Error": {
"Code": "ValidationException",
"Message": "Invalid message format",
}
}
failing_client = MagicMock()
failing_client.converse.side_effect = ClientError(error_response, "Converse")
with patch.object(client, "_bedrock_client", failing_client), pytest.raises(ClientError):
await client.get_response(
messages=_user_messages(),
options={"max_tokens": 100},
)
# endregion