chore: import upstream snapshot with attribution
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
CodeQL / Analyze (csharp) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
CodeQL / Analyze (csharp) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,30 @@
|
||||
# Azure Cosmos DB Package (agent-framework-azure-cosmos)
|
||||
|
||||
Azure Cosmos DB history provider integration for Agent Framework.
|
||||
|
||||
## Main Classes
|
||||
|
||||
- **`CosmosHistoryProvider`** - Persistent conversation history storage backed by Azure Cosmos DB
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from agent_framework.azure import CosmosHistoryProvider
|
||||
|
||||
provider = CosmosHistoryProvider(
|
||||
endpoint="https://<account>.documents.azure.com:443/",
|
||||
credential="<key-or-token-credential>",
|
||||
database_name="agent-framework",
|
||||
container_name="chat-history",
|
||||
)
|
||||
```
|
||||
|
||||
Container name is configured on the provider. `session_id` is used as the partition key.
|
||||
|
||||
## Import Path
|
||||
|
||||
```python
|
||||
from agent_framework.azure import CosmosHistoryProvider
|
||||
# or directly:
|
||||
from agent_framework_azure_cosmos import CosmosHistoryProvider
|
||||
```
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) Microsoft Corporation.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE
|
||||
@@ -0,0 +1,126 @@
|
||||
# Get Started with Microsoft Agent Framework Azure Cosmos DB
|
||||
|
||||
Please install this package via pip:
|
||||
|
||||
```bash
|
||||
pip install agent-framework-azure-cosmos --pre
|
||||
```
|
||||
|
||||
## Azure Cosmos DB History Provider
|
||||
|
||||
The Azure Cosmos DB integration provides `CosmosHistoryProvider` for persistent conversation history storage.
|
||||
|
||||
### Basic Usage Example
|
||||
|
||||
```python
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from agent_framework_azure_cosmos import CosmosHistoryProvider
|
||||
|
||||
provider = CosmosHistoryProvider(
|
||||
endpoint="https://<account>.documents.azure.com:443/",
|
||||
credential=DefaultAzureCredential(),
|
||||
database_name="agent-framework",
|
||||
container_name="chat-history",
|
||||
)
|
||||
```
|
||||
|
||||
Credentials follow the same pattern used by other Azure connectors in the repository:
|
||||
|
||||
- Pass a credential object (for example `DefaultAzureCredential`)
|
||||
- Or pass a key string directly
|
||||
- Or set `AZURE_COSMOS_KEY` in the environment
|
||||
|
||||
Container naming behavior:
|
||||
|
||||
- Container name is configured on the provider (`container_name` or `AZURE_COSMOS_CONTAINER_NAME`)
|
||||
- `session_id` is used as the Cosmos partition key for reads/writes
|
||||
|
||||
See `samples/02-agents/conversations/cosmos_history_provider.py` for a runnable example.
|
||||
|
||||
## Cosmos DB Workflow Checkpoint Storage
|
||||
|
||||
`CosmosCheckpointStorage` implements the `CheckpointStorage` protocol, enabling
|
||||
durable workflow checkpointing backed by Azure Cosmos DB NoSQL. Workflows can be
|
||||
paused and resumed across process restarts by persisting checkpoint state in Cosmos DB.
|
||||
|
||||
### Basic Usage
|
||||
|
||||
#### Managed Identity / RBAC (recommended for production)
|
||||
|
||||
```python
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from agent_framework import WorkflowBuilder
|
||||
from agent_framework_azure_cosmos import CosmosCheckpointStorage
|
||||
|
||||
checkpoint_storage = CosmosCheckpointStorage(
|
||||
endpoint="https://<account>.documents.azure.com:443/",
|
||||
credential=DefaultAzureCredential(),
|
||||
database_name="agent-framework",
|
||||
container_name="workflow-checkpoints",
|
||||
)
|
||||
```
|
||||
|
||||
#### Account Key
|
||||
|
||||
```python
|
||||
from agent_framework_azure_cosmos import CosmosCheckpointStorage
|
||||
|
||||
checkpoint_storage = CosmosCheckpointStorage(
|
||||
endpoint="https://<account>.documents.azure.com:443/",
|
||||
credential="<your-account-key>",
|
||||
database_name="agent-framework",
|
||||
container_name="workflow-checkpoints",
|
||||
)
|
||||
```
|
||||
|
||||
#### Then use with a workflow
|
||||
|
||||
```python
|
||||
from agent_framework import WorkflowBuilder
|
||||
|
||||
# Build a workflow with checkpointing enabled
|
||||
workflow = WorkflowBuilder(
|
||||
start_executor=start,
|
||||
checkpoint_storage=checkpoint_storage,
|
||||
).build()
|
||||
|
||||
# Run the workflow — checkpoints are automatically saved after each superstep
|
||||
result = await workflow.run(message="input data")
|
||||
|
||||
# Resume from a checkpoint
|
||||
latest = await checkpoint_storage.get_latest(workflow_name=workflow.name)
|
||||
if latest:
|
||||
resumed = await workflow.run(checkpoint_id=latest.checkpoint_id)
|
||||
```
|
||||
|
||||
### Authentication Options
|
||||
|
||||
`CosmosCheckpointStorage` supports the same authentication modes as `CosmosHistoryProvider`:
|
||||
|
||||
- **Managed identity / RBAC** (recommended): Pass `DefaultAzureCredential()`,
|
||||
`ManagedIdentityCredential()`, or any Azure `TokenCredential`
|
||||
- **Account key**: Pass a key string via `credential` parameter
|
||||
- **Environment variables**: Set `AZURE_COSMOS_ENDPOINT`, `AZURE_COSMOS_DATABASE_NAME`,
|
||||
`AZURE_COSMOS_CONTAINER_NAME`, and `AZURE_COSMOS_KEY` (key not required when using
|
||||
Azure credentials)
|
||||
- **Pre-created client**: Pass an existing `CosmosClient` or `ContainerProxy`
|
||||
|
||||
### Database and Container Setup
|
||||
|
||||
The database and container are created automatically on first use (via
|
||||
`create_database_if_not_exists` and `create_container_if_not_exists`). The container
|
||||
uses `/workflow_name` as the partition key. You can also pre-create them in the Azure
|
||||
portal with this partition key configuration.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `AZURE_COSMOS_ENDPOINT` | Cosmos DB account endpoint |
|
||||
| `AZURE_COSMOS_DATABASE_NAME` | Database name |
|
||||
| `AZURE_COSMOS_CONTAINER_NAME` | Container name |
|
||||
| `AZURE_COSMOS_KEY` | Account key (optional if using Azure credentials) |
|
||||
|
||||
See `samples/03-workflows/checkpoint/cosmos_workflow_checkpointing.py` for a standalone example,
|
||||
or `samples/03-workflows/checkpoint/cosmos_workflow_checkpointing_foundry.py` for an end-to-end
|
||||
example with Azure AI Foundry agents.
|
||||
@@ -0,0 +1,17 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import importlib.metadata
|
||||
|
||||
from ._checkpoint_storage import CosmosCheckpointStorage
|
||||
from ._history_provider import CosmosHistoryProvider
|
||||
|
||||
try:
|
||||
__version__ = importlib.metadata.version(__name__)
|
||||
except importlib.metadata.PackageNotFoundError:
|
||||
__version__ = "0.0.0" # Fallback for development mode
|
||||
|
||||
__all__ = [
|
||||
"CosmosCheckpointStorage",
|
||||
"CosmosHistoryProvider",
|
||||
"__version__",
|
||||
]
|
||||
@@ -0,0 +1,456 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Azure Cosmos DB checkpoint storage for workflow checkpointing."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any, TypedDict
|
||||
|
||||
from agent_framework._settings import SecretString, load_settings
|
||||
from agent_framework._telemetry import get_user_agent
|
||||
from agent_framework._workflows._checkpoint import CheckpointID, WorkflowCheckpoint
|
||||
from agent_framework._workflows._checkpoint_encoding import decode_checkpoint_value, encode_checkpoint_value
|
||||
from agent_framework.exceptions import WorkflowCheckpointException
|
||||
from azure.core.credentials import TokenCredential
|
||||
from azure.core.credentials_async import AsyncTokenCredential
|
||||
from azure.cosmos import PartitionKey
|
||||
from azure.cosmos.aio import ContainerProxy, CosmosClient
|
||||
from azure.cosmos.exceptions import CosmosResourceNotFoundError
|
||||
|
||||
AzureCredentialTypes = TokenCredential | AsyncTokenCredential
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AzureCosmosCheckpointSettings(TypedDict, total=False):
|
||||
"""Settings for CosmosCheckpointStorage resolved from args and environment."""
|
||||
|
||||
endpoint: str | None
|
||||
database_name: str | None
|
||||
container_name: str | None
|
||||
key: SecretString | None
|
||||
|
||||
|
||||
class CosmosCheckpointStorage:
|
||||
"""Azure Cosmos DB-backed checkpoint storage for workflow checkpointing.
|
||||
|
||||
Implements the ``CheckpointStorage`` protocol using Azure Cosmos DB NoSQL
|
||||
as the persistent backend. Checkpoints are stored as JSON documents with
|
||||
``workflow_name`` as the partition key, enabling efficient per-workflow queries.
|
||||
|
||||
This storage uses the same hybrid JSON + pickle encoding as
|
||||
``FileCheckpointStorage``, allowing full Python object fidelity for
|
||||
complex workflow state while keeping the document structure human-readable.
|
||||
|
||||
Security warning: checkpoints use pickle for non-JSON-native values. Loading
|
||||
checkpoints from untrusted sources is unsafe and can execute arbitrary code
|
||||
during deserialization. The built-in deserialization restrictions reduce risk,
|
||||
but they do not make untrusted checkpoints safe to load. Extending
|
||||
``allowed_checkpoint_types`` may further increase risk and should only be done
|
||||
for trusted application types.
|
||||
|
||||
By default, checkpoint deserialization is restricted to a built-in set of safe
|
||||
Python types (primitives, datetime, uuid, ...) and all ``agent_framework``
|
||||
internal types. To allow additional application-specific types, pass them via
|
||||
the ``allowed_checkpoint_types`` parameter using ``"module:qualname"`` format.
|
||||
|
||||
Example:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from agent_framework_azure_cosmos import CosmosCheckpointStorage
|
||||
|
||||
storage = CosmosCheckpointStorage(
|
||||
endpoint="https://my-account.documents.azure.com:443/",
|
||||
credential=DefaultAzureCredential(),
|
||||
database_name="agent-db",
|
||||
container_name="checkpoints",
|
||||
allowed_checkpoint_types=[
|
||||
"my_app.models:MyState",
|
||||
],
|
||||
)
|
||||
|
||||
The database and container are created automatically on first use
|
||||
if they do not already exist. The container uses partition key
|
||||
``/workflow_name``.
|
||||
|
||||
Example using managed identity / RBAC:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from agent_framework_azure_cosmos import CosmosCheckpointStorage
|
||||
|
||||
storage = CosmosCheckpointStorage(
|
||||
endpoint="https://my-account.documents.azure.com:443/",
|
||||
credential=DefaultAzureCredential(),
|
||||
database_name="agent-db",
|
||||
container_name="checkpoints",
|
||||
)
|
||||
|
||||
Example using account key:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
storage = CosmosCheckpointStorage(
|
||||
endpoint="https://my-account.documents.azure.com:443/",
|
||||
credential="my-account-key",
|
||||
database_name="agent-db",
|
||||
container_name="checkpoints",
|
||||
)
|
||||
|
||||
Then use with a workflow builder:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
workflow = WorkflowBuilder(
|
||||
start_executor=start,
|
||||
checkpoint_storage=storage,
|
||||
).build()
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
endpoint: str | None = None,
|
||||
database_name: str | None = None,
|
||||
container_name: str | None = None,
|
||||
credential: str | AzureCredentialTypes | None = None,
|
||||
cosmos_client: CosmosClient | None = None,
|
||||
container_client: ContainerProxy | None = None,
|
||||
env_file_path: str | None = None,
|
||||
env_file_encoding: str | None = None,
|
||||
allowed_checkpoint_types: list[str] | None = None,
|
||||
) -> None:
|
||||
"""Initialize the Azure Cosmos DB checkpoint storage.
|
||||
|
||||
Supports multiple authentication modes:
|
||||
|
||||
- **Container client** (``container_client``): Use a pre-created
|
||||
Cosmos async container proxy. No client lifecycle is managed.
|
||||
- **Cosmos client** (``cosmos_client``): Use a pre-created Cosmos
|
||||
async client. The caller is responsible for closing it.
|
||||
- **Endpoint + credential**: Create a new Cosmos client. The storage
|
||||
owns the client and closes it on ``close()``.
|
||||
- **Environment variables**: Falls back to ``AZURE_COSMOS_ENDPOINT``,
|
||||
``AZURE_COSMOS_DATABASE_NAME``, ``AZURE_COSMOS_CONTAINER_NAME``,
|
||||
and ``AZURE_COSMOS_KEY``.
|
||||
|
||||
Args:
|
||||
endpoint: Cosmos DB account endpoint.
|
||||
Can be set via ``AZURE_COSMOS_ENDPOINT``.
|
||||
database_name: Cosmos DB database name.
|
||||
Can be set via ``AZURE_COSMOS_DATABASE_NAME``.
|
||||
container_name: Cosmos DB container name.
|
||||
Can be set via ``AZURE_COSMOS_CONTAINER_NAME``.
|
||||
credential: Credential to authenticate with Cosmos DB.
|
||||
For **managed identity / RBAC**, pass an Azure credential object
|
||||
such as ``DefaultAzureCredential()`` or
|
||||
``ManagedIdentityCredential()``.
|
||||
For **key-based auth**, pass the account key as a string,
|
||||
or set ``AZURE_COSMOS_KEY`` in the environment.
|
||||
cosmos_client: Pre-created Cosmos async client.
|
||||
container_client: Pre-created Cosmos container client.
|
||||
env_file_path: Path to environment file for loading settings.
|
||||
env_file_encoding: Encoding of the environment file.
|
||||
allowed_checkpoint_types: Additional types (beyond the built-in safe set
|
||||
and framework types) that are permitted during checkpoint
|
||||
deserialization. Each entry should be a ``"module:qualname"``
|
||||
string (e.g., ``"my_app.models:MyState"``).
|
||||
"""
|
||||
self._cosmos_client: CosmosClient | None = cosmos_client
|
||||
self._container_proxy: ContainerProxy | None = container_client
|
||||
self._owns_client = False
|
||||
self._allowed_types: frozenset[str] = frozenset(allowed_checkpoint_types or [])
|
||||
|
||||
if self._container_proxy is not None:
|
||||
self.database_name: str = database_name or ""
|
||||
self.container_name: str = container_name or ""
|
||||
return
|
||||
|
||||
required_fields: list[str] = ["database_name", "container_name"]
|
||||
if cosmos_client is None:
|
||||
required_fields.append("endpoint")
|
||||
if credential is None:
|
||||
required_fields.append("key")
|
||||
|
||||
settings = load_settings(
|
||||
AzureCosmosCheckpointSettings,
|
||||
env_prefix="AZURE_COSMOS_",
|
||||
required_fields=required_fields,
|
||||
endpoint=endpoint,
|
||||
database_name=database_name,
|
||||
container_name=container_name,
|
||||
key=credential if isinstance(credential, str) else None,
|
||||
env_file_path=env_file_path,
|
||||
env_file_encoding=env_file_encoding,
|
||||
)
|
||||
self.database_name = settings["database_name"] # type: ignore[assignment]
|
||||
self.container_name = settings["container_name"] # type: ignore[assignment]
|
||||
|
||||
if self._cosmos_client is None:
|
||||
self._cosmos_client = CosmosClient(
|
||||
url=settings["endpoint"], # type: ignore[arg-type]
|
||||
credential=credential or settings["key"].get_secret_value(), # type: ignore[arg-type,union-attr]
|
||||
user_agent_suffix=get_user_agent(),
|
||||
)
|
||||
self._owns_client = True
|
||||
|
||||
async def save(self, checkpoint: WorkflowCheckpoint) -> CheckpointID:
|
||||
"""Save a checkpoint to Cosmos DB and return its ID.
|
||||
|
||||
The checkpoint is encoded to a JSON-compatible form (using pickle for
|
||||
non-JSON-native values) and stored as a Cosmos DB document with the
|
||||
``workflow_name`` as the partition key.
|
||||
|
||||
The document ``id`` is a composite of ``workflow_name`` and
|
||||
``checkpoint_id`` to ensure global uniqueness across partitions.
|
||||
|
||||
Args:
|
||||
checkpoint: The WorkflowCheckpoint object to save.
|
||||
|
||||
Returns:
|
||||
The unique ID of the saved checkpoint.
|
||||
"""
|
||||
await self._ensure_container_proxy()
|
||||
|
||||
checkpoint_dict = checkpoint.to_dict()
|
||||
encoded = encode_checkpoint_value(checkpoint_dict)
|
||||
|
||||
document: dict[str, Any] = {
|
||||
"id": self._make_document_id(checkpoint.workflow_name, checkpoint.checkpoint_id),
|
||||
"workflow_name": checkpoint.workflow_name,
|
||||
**encoded,
|
||||
}
|
||||
|
||||
await self._container_proxy.upsert_item(body=document) # type: ignore[union-attr]
|
||||
logger.info("Saved checkpoint %s to Cosmos DB", checkpoint.checkpoint_id)
|
||||
return checkpoint.checkpoint_id
|
||||
|
||||
async def load(self, checkpoint_id: CheckpointID) -> WorkflowCheckpoint:
|
||||
"""Load a checkpoint from Cosmos DB by ID.
|
||||
|
||||
Args:
|
||||
checkpoint_id: The unique ID of the checkpoint to load.
|
||||
|
||||
Returns:
|
||||
The WorkflowCheckpoint object corresponding to the given ID.
|
||||
|
||||
Raises:
|
||||
WorkflowCheckpointException: If no checkpoint with the given ID exists,
|
||||
or if multiple checkpoints share the same ID across workflows.
|
||||
"""
|
||||
await self._ensure_container_proxy()
|
||||
|
||||
query = "SELECT * FROM c WHERE c.checkpoint_id = @checkpoint_id"
|
||||
parameters: list[dict[str, object]] = [
|
||||
{"name": "@checkpoint_id", "value": checkpoint_id},
|
||||
]
|
||||
|
||||
items = self._container_proxy.query_items( # type: ignore[union-attr]
|
||||
query=query,
|
||||
parameters=parameters,
|
||||
)
|
||||
|
||||
results: list[dict[str, Any]] = []
|
||||
async for item in items:
|
||||
results.append(item)
|
||||
|
||||
if not results:
|
||||
raise WorkflowCheckpointException(f"No checkpoint found with ID {checkpoint_id}")
|
||||
|
||||
if len(results) > 1:
|
||||
workflow_names = [r.get("workflow_name", "unknown") for r in results]
|
||||
raise WorkflowCheckpointException(
|
||||
f"Multiple checkpoints found with ID {checkpoint_id} across workflows: "
|
||||
f"{workflow_names}. Use list_checkpoints(workflow_name=...) to query "
|
||||
f"by workflow instead."
|
||||
)
|
||||
|
||||
return self._document_to_checkpoint(results[0])
|
||||
|
||||
async def list_checkpoints(self, *, workflow_name: str) -> list[WorkflowCheckpoint]:
|
||||
"""List checkpoint objects for a given workflow name.
|
||||
|
||||
Args:
|
||||
workflow_name: The name of the workflow to list checkpoints for.
|
||||
|
||||
Returns:
|
||||
A list of WorkflowCheckpoint objects for the specified workflow name.
|
||||
"""
|
||||
await self._ensure_container_proxy()
|
||||
|
||||
query = "SELECT * FROM c WHERE c.workflow_name = @workflow_name ORDER BY c.timestamp ASC"
|
||||
parameters: list[dict[str, object]] = [
|
||||
{"name": "@workflow_name", "value": workflow_name},
|
||||
]
|
||||
|
||||
items = self._container_proxy.query_items( # type: ignore[union-attr]
|
||||
query=query,
|
||||
parameters=parameters,
|
||||
partition_key=workflow_name,
|
||||
)
|
||||
|
||||
checkpoints: list[WorkflowCheckpoint] = []
|
||||
async for item in items:
|
||||
try:
|
||||
checkpoints.append(self._document_to_checkpoint(item))
|
||||
except Exception as e:
|
||||
logger.warning("Failed to decode checkpoint document: %s", e)
|
||||
return checkpoints
|
||||
|
||||
async def delete(self, checkpoint_id: CheckpointID) -> bool:
|
||||
"""Delete a checkpoint from Cosmos DB by ID.
|
||||
|
||||
Args:
|
||||
checkpoint_id: The unique ID of the checkpoint to delete.
|
||||
|
||||
Returns:
|
||||
True if the checkpoint was successfully deleted, False if not found.
|
||||
"""
|
||||
await self._ensure_container_proxy()
|
||||
|
||||
query = "SELECT c.id, c.workflow_name FROM c WHERE c.checkpoint_id = @checkpoint_id"
|
||||
parameters: list[dict[str, object]] = [
|
||||
{"name": "@checkpoint_id", "value": checkpoint_id},
|
||||
]
|
||||
|
||||
items = self._container_proxy.query_items( # type: ignore[union-attr]
|
||||
query=query,
|
||||
parameters=parameters,
|
||||
)
|
||||
|
||||
async for item in items:
|
||||
try:
|
||||
await self._container_proxy.delete_item( # type: ignore[union-attr]
|
||||
item=item["id"],
|
||||
partition_key=item["workflow_name"],
|
||||
)
|
||||
logger.info("Deleted checkpoint %s from Cosmos DB", checkpoint_id)
|
||||
return True
|
||||
except CosmosResourceNotFoundError:
|
||||
return False
|
||||
|
||||
return False
|
||||
|
||||
async def get_latest(self, *, workflow_name: str) -> WorkflowCheckpoint | None:
|
||||
"""Get the latest checkpoint for a given workflow name.
|
||||
|
||||
Args:
|
||||
workflow_name: The name of the workflow to get the latest checkpoint for.
|
||||
|
||||
Returns:
|
||||
The latest WorkflowCheckpoint, or None if no checkpoints exist.
|
||||
"""
|
||||
await self._ensure_container_proxy()
|
||||
|
||||
query = "SELECT * FROM c WHERE c.workflow_name = @workflow_name ORDER BY c.timestamp DESC OFFSET 0 LIMIT 1"
|
||||
parameters: list[dict[str, object]] = [
|
||||
{"name": "@workflow_name", "value": workflow_name},
|
||||
]
|
||||
|
||||
items = self._container_proxy.query_items( # type: ignore[union-attr]
|
||||
query=query,
|
||||
parameters=parameters,
|
||||
partition_key=workflow_name,
|
||||
)
|
||||
|
||||
async for item in items:
|
||||
checkpoint = self._document_to_checkpoint(item)
|
||||
logger.debug(
|
||||
"Latest checkpoint for workflow %s is %s",
|
||||
workflow_name,
|
||||
checkpoint.checkpoint_id,
|
||||
)
|
||||
return checkpoint
|
||||
|
||||
return None
|
||||
|
||||
async def list_checkpoint_ids(self, *, workflow_name: str) -> list[CheckpointID]:
|
||||
"""List checkpoint IDs for a given workflow name.
|
||||
|
||||
Args:
|
||||
workflow_name: The name of the workflow to list checkpoint IDs for.
|
||||
|
||||
Returns:
|
||||
A list of checkpoint IDs for the specified workflow name.
|
||||
"""
|
||||
await self._ensure_container_proxy()
|
||||
|
||||
query = "SELECT c.checkpoint_id FROM c WHERE c.workflow_name = @workflow_name ORDER BY c.timestamp ASC"
|
||||
parameters: list[dict[str, object]] = [
|
||||
{"name": "@workflow_name", "value": workflow_name},
|
||||
]
|
||||
|
||||
items = self._container_proxy.query_items( # type: ignore[union-attr]
|
||||
query=query,
|
||||
parameters=parameters,
|
||||
partition_key=workflow_name,
|
||||
)
|
||||
|
||||
checkpoint_ids: list[CheckpointID] = []
|
||||
async for item in items:
|
||||
cid = item.get("checkpoint_id")
|
||||
if isinstance(cid, str):
|
||||
checkpoint_ids.append(cid)
|
||||
return checkpoint_ids
|
||||
|
||||
async def close(self) -> None:
|
||||
"""Close the underlying Cosmos client when this storage owns it."""
|
||||
if self._owns_client and self._cosmos_client is not None:
|
||||
await self._cosmos_client.close()
|
||||
|
||||
async def __aenter__(self) -> CosmosCheckpointStorage:
|
||||
"""Async context manager entry."""
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc_val: BaseException | None,
|
||||
exc_tb: Any,
|
||||
) -> None:
|
||||
"""Async context manager exit."""
|
||||
try:
|
||||
await self.close()
|
||||
except Exception:
|
||||
if exc_type is None:
|
||||
raise
|
||||
|
||||
async def _ensure_container_proxy(self) -> None:
|
||||
"""Get or create the Cosmos DB database and container for storing checkpoints."""
|
||||
if self._container_proxy is not None:
|
||||
return
|
||||
if self._cosmos_client is None:
|
||||
raise RuntimeError("Cosmos client is not initialized.")
|
||||
|
||||
database = await self._cosmos_client.create_database_if_not_exists(id=self.database_name)
|
||||
self._container_proxy = await database.create_container_if_not_exists(
|
||||
id=self.container_name,
|
||||
partition_key=PartitionKey(path="/workflow_name"),
|
||||
)
|
||||
|
||||
def _document_to_checkpoint(self, document: dict[str, Any]) -> WorkflowCheckpoint:
|
||||
"""Convert a Cosmos DB document back to a WorkflowCheckpoint.
|
||||
|
||||
Strips Cosmos DB system properties (``_rid``, ``_self``, ``_etag``,
|
||||
``_attachments``, ``_ts``) before decoding.
|
||||
"""
|
||||
# Remove Cosmos DB system properties and the composite 'id' field
|
||||
# (checkpoints use 'checkpoint_id', not 'id')
|
||||
cosmos_keys = {"id", "_rid", "_self", "_etag", "_attachments", "_ts"}
|
||||
cleaned = {k: v for k, v in document.items() if k not in cosmos_keys}
|
||||
|
||||
decoded = decode_checkpoint_value(cleaned, allowed_types=self._allowed_types)
|
||||
return WorkflowCheckpoint.from_dict(decoded)
|
||||
|
||||
@staticmethod
|
||||
def _make_document_id(workflow_name: str, checkpoint_id: str) -> str:
|
||||
"""Create a composite Cosmos DB document ID.
|
||||
|
||||
Combines ``workflow_name`` and ``checkpoint_id`` to ensure global
|
||||
uniqueness across partitions.
|
||||
"""
|
||||
return f"{workflow_name}_{checkpoint_id}"
|
||||
@@ -0,0 +1,286 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Azure Cosmos DB history provider."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
import uuid
|
||||
from collections.abc import Sequence
|
||||
from typing import Any, ClassVar, TypedDict
|
||||
|
||||
from agent_framework import Message
|
||||
from agent_framework._sessions import HistoryProvider
|
||||
from agent_framework._settings import SecretString, load_settings
|
||||
from agent_framework._telemetry import get_user_agent
|
||||
from azure.core.credentials import TokenCredential
|
||||
from azure.core.credentials_async import AsyncTokenCredential
|
||||
from azure.cosmos import PartitionKey
|
||||
from azure.cosmos.aio import ContainerProxy, CosmosClient, DatabaseProxy
|
||||
|
||||
AzureCredentialTypes = TokenCredential | AsyncTokenCredential
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AzureCosmosHistorySettings(TypedDict, total=False):
|
||||
"""Settings for CosmosHistoryProvider resolved from args and environment."""
|
||||
|
||||
endpoint: str | None
|
||||
database_name: str | None
|
||||
container_name: str | None
|
||||
key: SecretString | None
|
||||
|
||||
|
||||
class CosmosHistoryProvider(HistoryProvider):
|
||||
"""Azure Cosmos DB-backed history provider using HistoryProvider hooks."""
|
||||
|
||||
DEFAULT_SOURCE_ID: ClassVar[str] = "azure_cosmos_history"
|
||||
_BATCH_OPERATION_LIMIT: ClassVar[int] = 100
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
source_id: str = DEFAULT_SOURCE_ID,
|
||||
*,
|
||||
load_messages: bool = True,
|
||||
store_outputs: bool = True,
|
||||
store_inputs: bool = True,
|
||||
store_context_messages: bool = False,
|
||||
store_context_from: set[str] | None = None,
|
||||
endpoint: str | None = None,
|
||||
database_name: str | None = None,
|
||||
container_name: str | None = None,
|
||||
credential: str | AzureCredentialTypes | None = None,
|
||||
cosmos_client: CosmosClient | None = None,
|
||||
container_client: ContainerProxy | None = None,
|
||||
env_file_path: str | None = None,
|
||||
env_file_encoding: str | None = None,
|
||||
) -> None:
|
||||
"""Initialize the Azure Cosmos DB history provider.
|
||||
|
||||
Args:
|
||||
source_id: Unique identifier for this provider instance.
|
||||
load_messages: Whether to load messages before invocation.
|
||||
store_outputs: Whether to store response messages.
|
||||
store_inputs: Whether to store input messages.
|
||||
store_context_messages: Whether to store context from other providers.
|
||||
store_context_from: If set, only store context from these source_ids.
|
||||
endpoint: Cosmos DB account endpoint.
|
||||
Can be set via ``AZURE_COSMOS_ENDPOINT``.
|
||||
database_name: Cosmos DB database name.
|
||||
Can be set via ``AZURE_COSMOS_DATABASE_NAME``.
|
||||
container_name: Cosmos DB container name.
|
||||
Can be set via ``AZURE_COSMOS_CONTAINER_NAME``.
|
||||
credential: Credential to authenticate with Cosmos DB.
|
||||
Supports key string and Azure credential objects.
|
||||
Can be set via ``AZURE_COSMOS_KEY`` when omitted.
|
||||
cosmos_client: Pre-created Cosmos async client.
|
||||
container_client: Pre-created Cosmos container client for fixed-container usage.
|
||||
env_file_path: Path to environment file for loading settings.
|
||||
env_file_encoding: Encoding of the environment file.
|
||||
"""
|
||||
super().__init__(
|
||||
source_id,
|
||||
load_messages=load_messages,
|
||||
store_outputs=store_outputs,
|
||||
store_inputs=store_inputs,
|
||||
store_context_messages=store_context_messages,
|
||||
store_context_from=store_context_from,
|
||||
)
|
||||
|
||||
self._cosmos_client: CosmosClient | None = cosmos_client
|
||||
self._container_proxy: ContainerProxy | None = container_client
|
||||
self._owns_client = False
|
||||
self._database_client: DatabaseProxy | None = None
|
||||
|
||||
if self._container_proxy is not None:
|
||||
self.database_name: str = database_name or ""
|
||||
self.container_name: str = container_name or ""
|
||||
return
|
||||
|
||||
required_fields: list[str] = ["database_name", "container_name"]
|
||||
if cosmos_client is None:
|
||||
required_fields.append("endpoint")
|
||||
if credential is None:
|
||||
required_fields.append("key")
|
||||
|
||||
settings = load_settings(
|
||||
AzureCosmosHistorySettings,
|
||||
env_prefix="AZURE_COSMOS_",
|
||||
required_fields=required_fields,
|
||||
endpoint=endpoint,
|
||||
database_name=database_name,
|
||||
container_name=container_name,
|
||||
key=credential if isinstance(credential, str) else None,
|
||||
env_file_path=env_file_path,
|
||||
env_file_encoding=env_file_encoding,
|
||||
)
|
||||
self.database_name = settings["database_name"] # type: ignore[assignment]
|
||||
self.container_name = settings["container_name"] # type: ignore[assignment]
|
||||
if self._cosmos_client is None:
|
||||
self._cosmos_client = CosmosClient(
|
||||
url=settings["endpoint"], # type: ignore[arg-type]
|
||||
credential=credential or settings["key"].get_secret_value(), # type: ignore[arg-type,union-attr]
|
||||
user_agent_suffix=get_user_agent(),
|
||||
)
|
||||
self._owns_client = True
|
||||
|
||||
self._database_client = self._cosmos_client.get_database_client(self.database_name)
|
||||
|
||||
async def get_messages(
|
||||
self,
|
||||
session_id: str | None,
|
||||
*,
|
||||
state: dict[str, Any] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> list[Message]:
|
||||
"""Retrieve stored messages for this session from Azure Cosmos DB."""
|
||||
await self._ensure_container_proxy()
|
||||
session_key = self._session_partition_key(session_id)
|
||||
|
||||
query = (
|
||||
"SELECT c.message FROM c "
|
||||
"WHERE c.session_id = @session_id AND c.source_id = @source_id "
|
||||
"ORDER BY c.sort_key ASC"
|
||||
)
|
||||
parameters: list[dict[str, object]] = [
|
||||
{"name": "@session_id", "value": session_key},
|
||||
{"name": "@source_id", "value": self.source_id},
|
||||
]
|
||||
items = self._container_proxy.query_items( # type: ignore[union-attr]
|
||||
query=query, parameters=parameters, partition_key=session_key
|
||||
)
|
||||
|
||||
messages: list[Message] = []
|
||||
async for item in items:
|
||||
message_payload = item.get("message")
|
||||
if not isinstance(message_payload, dict):
|
||||
logger.warning("Skipping Cosmos DB item with non-mapping message payload.")
|
||||
continue
|
||||
try:
|
||||
msg = Message.from_dict(message_payload) # pyright: ignore[reportUnknownArgumentType]
|
||||
except ValueError as e:
|
||||
logger.warning("Failed to deserialize message from Cosmos DB item: %s", e)
|
||||
continue
|
||||
messages.append(msg)
|
||||
|
||||
return messages
|
||||
|
||||
async def save_messages(
|
||||
self,
|
||||
session_id: str | None,
|
||||
messages: Sequence[Message],
|
||||
*,
|
||||
state: dict[str, Any] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> None:
|
||||
"""Persist messages for this session to Azure Cosmos DB."""
|
||||
if not messages:
|
||||
return
|
||||
|
||||
await self._ensure_container_proxy()
|
||||
session_key = self._session_partition_key(session_id)
|
||||
|
||||
base_sort_key = time.time_ns()
|
||||
operations: list[tuple[str, tuple[dict[str, Any]]]] = []
|
||||
for index, message in enumerate(messages):
|
||||
document = {
|
||||
"id": str(uuid.uuid4()),
|
||||
"session_id": session_key,
|
||||
"sort_key": base_sort_key + index,
|
||||
"source_id": self.source_id,
|
||||
"message": message.to_dict(),
|
||||
}
|
||||
operations.append(("upsert", (document,)))
|
||||
|
||||
for start in range(0, len(operations), self._BATCH_OPERATION_LIMIT):
|
||||
batch = operations[start : start + self._BATCH_OPERATION_LIMIT]
|
||||
await self._container_proxy.execute_item_batch( # type: ignore[union-attr]
|
||||
batch_operations=batch, partition_key=session_key
|
||||
)
|
||||
|
||||
async def clear(self, session_id: str | None) -> None:
|
||||
"""Clear all messages for a session from Azure Cosmos DB."""
|
||||
await self._ensure_container_proxy()
|
||||
session_key = self._session_partition_key(session_id)
|
||||
query = "SELECT c.id FROM c WHERE c.session_id = @session_id AND c.source_id = @source_id"
|
||||
parameters: list[dict[str, object]] = [
|
||||
{"name": "@session_id", "value": session_key},
|
||||
{"name": "@source_id", "value": self.source_id},
|
||||
]
|
||||
items = self._container_proxy.query_items( # type: ignore[union-attr]
|
||||
query=query, parameters=parameters, partition_key=session_key
|
||||
)
|
||||
|
||||
delete_operations: list[tuple[str, tuple[str]]] = []
|
||||
async for item in items:
|
||||
item_id = item.get("id")
|
||||
if isinstance(item_id, str):
|
||||
delete_operations.append(("delete", (item_id,)))
|
||||
|
||||
for start in range(0, len(delete_operations), self._BATCH_OPERATION_LIMIT):
|
||||
batch = delete_operations[start : start + self._BATCH_OPERATION_LIMIT]
|
||||
await self._container_proxy.execute_item_batch( # type: ignore[union-attr]
|
||||
batch_operations=batch, partition_key=session_key
|
||||
)
|
||||
|
||||
async def list_sessions(self) -> list[str]:
|
||||
"""List all session IDs stored in this provider's Cosmos container."""
|
||||
await self._ensure_container_proxy()
|
||||
query = "SELECT DISTINCT VALUE c.session_id FROM c WHERE c.source_id = @source_id"
|
||||
parameters: list[dict[str, object]] = [{"name": "@source_id", "value": self.source_id}]
|
||||
# without a partition key, it is automatically a cross-partition query
|
||||
items = self._container_proxy.query_items(query=query, parameters=parameters) # type: ignore[union-attr]
|
||||
|
||||
session_ids: set[str] = set()
|
||||
async for item in items:
|
||||
if isinstance(item, str):
|
||||
session_ids.add(item)
|
||||
return sorted(session_ids)
|
||||
|
||||
async def close(self) -> None:
|
||||
"""Close the underlying Cosmos client when this provider owns it."""
|
||||
if self._owns_client and self._cosmos_client is not None:
|
||||
await self._cosmos_client.close()
|
||||
|
||||
async def __aenter__(self) -> CosmosHistoryProvider:
|
||||
"""Async context manager entry."""
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc_val: BaseException | None,
|
||||
exc_tb: Any,
|
||||
) -> None:
|
||||
"""Async context manager exit."""
|
||||
try:
|
||||
await self.close()
|
||||
except Exception:
|
||||
if exc_type is None:
|
||||
raise
|
||||
|
||||
async def _ensure_container_proxy(self) -> None:
|
||||
"""Get or create the Cosmos DB container for storing messages."""
|
||||
if self._container_proxy is not None:
|
||||
return
|
||||
if self._database_client is None:
|
||||
raise RuntimeError("Cosmos database client is not initialized.")
|
||||
|
||||
self._container_proxy = await self._database_client.create_container_if_not_exists(
|
||||
id=self.container_name,
|
||||
partition_key=PartitionKey(path="/session_id"),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _session_partition_key(session_id: str | None) -> str:
|
||||
if session_id:
|
||||
return session_id
|
||||
|
||||
generated_session_id = str(uuid.uuid4())
|
||||
logger.warning(
|
||||
"Received empty session_id; generated temporary session id '%s' for Cosmos partition key.",
|
||||
generated_session_id,
|
||||
)
|
||||
return generated_session_id
|
||||
@@ -0,0 +1,101 @@
|
||||
[project]
|
||||
name = "agent-framework-azure-cosmos"
|
||||
description = "Azure Cosmos DB history provider integration for Microsoft Agent Framework."
|
||||
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
version = "1.0.0b260521"
|
||||
license-files = ["LICENSE"]
|
||||
urls.homepage = "https://aka.ms/agent-framework"
|
||||
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
|
||||
urls.release_notes = "https://github.com/microsoft/agent-framework/releases?q=tag%3Apython-1&expanded=true"
|
||||
urls.issues = "https://github.com/microsoft/agent-framework/issues"
|
||||
classifiers = [
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Development Status :: 4 - Beta",
|
||||
"Intended Audience :: Developers",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Programming Language :: Python :: 3.14",
|
||||
"Typing :: Typed",
|
||||
]
|
||||
dependencies = [
|
||||
"agent-framework-core>=1.6.0,<2",
|
||||
"azure-cosmos>=4.3.0,<5",
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
prerelease = "if-necessary-or-explicit"
|
||||
environments = [
|
||||
"sys_platform == 'darwin'",
|
||||
"sys_platform == 'linux'",
|
||||
"sys_platform == 'win32'"
|
||||
]
|
||||
|
||||
[tool.uv-dynamic-versioning]
|
||||
fallback-version = "0.0.0"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = 'tests'
|
||||
addopts = "-ra -q -r fEX"
|
||||
asyncio_mode = "auto"
|
||||
asyncio_default_fixture_loop_scope = "function"
|
||||
filterwarnings = [
|
||||
"ignore:Support for class-based `config` is deprecated:DeprecationWarning:pydantic.*"
|
||||
]
|
||||
timeout = 120
|
||||
markers = [
|
||||
"integration: marks tests as integration tests that require external services",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
extend = "../../pyproject.toml"
|
||||
|
||||
[tool.coverage.run]
|
||||
omit = [
|
||||
"**/__init__.py"
|
||||
]
|
||||
|
||||
[tool.pyright]
|
||||
extends = "../../pyproject.toml"
|
||||
include = ["agent_framework_azure_cosmos"]
|
||||
|
||||
[tool.mypy]
|
||||
plugins = ['pydantic.mypy']
|
||||
strict = true
|
||||
python_version = "3.10"
|
||||
ignore_missing_imports = true
|
||||
disallow_untyped_defs = true
|
||||
no_implicit_optional = true
|
||||
check_untyped_defs = true
|
||||
warn_return_any = true
|
||||
show_error_codes = true
|
||||
warn_unused_ignores = false
|
||||
disallow_incomplete_defs = true
|
||||
disallow_untyped_decorators = true
|
||||
|
||||
[tool.bandit]
|
||||
targets = ["agent_framework_azure_cosmos"]
|
||||
exclude_dirs = ["tests"]
|
||||
|
||||
[tool.poe]
|
||||
executor.type = "uv"
|
||||
include = "../../shared_tasks.toml"
|
||||
[tool.poe.tasks.mypy]
|
||||
help = "Run MyPy for this package."
|
||||
cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azure_cosmos"
|
||||
|
||||
[tool.poe.tasks.test]
|
||||
help = "Run the default unit test suite for this package."
|
||||
cmd = "pytest -m \"not integration\" --cov=agent_framework_azure_cosmos --cov-report=term-missing:skip-covered tests"
|
||||
|
||||
[tool.poe.tasks.integration-tests]
|
||||
help = "Run the package integration test suite."
|
||||
cmd = "pytest tests/test_cosmos_history_provider.py -m integration"
|
||||
|
||||
[build-system]
|
||||
requires = ["flit-core >= 3.11,<4.0"]
|
||||
build-backend = "flit_core.buildapi"
|
||||
@@ -0,0 +1,737 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import uuid
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import suppress
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from agent_framework._workflows._checkpoint import WorkflowCheckpoint
|
||||
from agent_framework._workflows._checkpoint_encoding import encode_checkpoint_value
|
||||
from agent_framework.exceptions import SettingNotFoundError, WorkflowCheckpointException
|
||||
from azure.cosmos.aio import CosmosClient
|
||||
from azure.cosmos.exceptions import CosmosResourceNotFoundError
|
||||
|
||||
import agent_framework_azure_cosmos._checkpoint_storage as checkpoint_storage_module
|
||||
from agent_framework_azure_cosmos._checkpoint_storage import CosmosCheckpointStorage
|
||||
|
||||
skip_if_cosmos_integration_tests_disabled = pytest.mark.skipif(
|
||||
any(
|
||||
os.getenv(name, "") == ""
|
||||
for name in (
|
||||
"AZURE_COSMOS_ENDPOINT",
|
||||
"AZURE_COSMOS_KEY",
|
||||
"AZURE_COSMOS_DATABASE_NAME",
|
||||
"AZURE_COSMOS_CONTAINER_NAME",
|
||||
)
|
||||
),
|
||||
reason=(
|
||||
"AZURE_COSMOS_ENDPOINT, AZURE_COSMOS_KEY, AZURE_COSMOS_DATABASE_NAME, and "
|
||||
"AZURE_COSMOS_CONTAINER_NAME are required for Cosmos integration tests."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _to_async_iter(items: list[Any]) -> AsyncIterator[Any]:
|
||||
async def _iterator() -> AsyncIterator[Any]:
|
||||
for item in items:
|
||||
yield item
|
||||
|
||||
return _iterator()
|
||||
|
||||
|
||||
def _make_checkpoint(
|
||||
workflow_name: str = "test-workflow",
|
||||
checkpoint_id: str | None = None,
|
||||
previous_checkpoint_id: str | None = None,
|
||||
timestamp: str | None = None,
|
||||
) -> WorkflowCheckpoint:
|
||||
"""Create a minimal WorkflowCheckpoint for testing."""
|
||||
return WorkflowCheckpoint(
|
||||
workflow_name=workflow_name,
|
||||
graph_signature_hash="abc123",
|
||||
checkpoint_id=checkpoint_id or str(uuid.uuid4()),
|
||||
previous_checkpoint_id=previous_checkpoint_id,
|
||||
timestamp=timestamp or "2025-01-01T00:00:00+00:00",
|
||||
state={"counter": 42},
|
||||
iteration_count=1,
|
||||
)
|
||||
|
||||
|
||||
def _checkpoint_to_cosmos_document(checkpoint: WorkflowCheckpoint) -> dict[str, Any]:
|
||||
"""Simulate what a Cosmos DB document looks like after save."""
|
||||
encoded = encode_checkpoint_value(checkpoint.to_dict())
|
||||
doc: dict[str, Any] = {
|
||||
"id": f"{checkpoint.workflow_name}_{checkpoint.checkpoint_id}",
|
||||
"workflow_name": checkpoint.workflow_name,
|
||||
**encoded,
|
||||
# Cosmos system properties
|
||||
"_rid": "abc",
|
||||
"_self": "dbs/abc/colls/def/docs/ghi",
|
||||
"_etag": '"00000000-0000-0000-0000-000000000000"',
|
||||
"_attachments": "attachments/",
|
||||
"_ts": 1700000000,
|
||||
}
|
||||
return doc
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_container() -> MagicMock:
|
||||
container = MagicMock()
|
||||
container.query_items = MagicMock(return_value=_to_async_iter([]))
|
||||
container.upsert_item = AsyncMock(return_value={})
|
||||
container.delete_item = AsyncMock(return_value={})
|
||||
return container
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_cosmos_client(mock_container: MagicMock) -> MagicMock:
|
||||
database_client = MagicMock()
|
||||
database_client.create_container_if_not_exists = AsyncMock(return_value=mock_container)
|
||||
|
||||
client = MagicMock()
|
||||
client.create_database_if_not_exists = AsyncMock(return_value=database_client)
|
||||
client.close = AsyncMock()
|
||||
return client
|
||||
|
||||
|
||||
# --- Tests for initialization ---
|
||||
|
||||
|
||||
async def test_init_uses_provided_container_client(mock_container: MagicMock) -> None:
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
assert storage.database_name == ""
|
||||
assert storage.container_name == ""
|
||||
|
||||
|
||||
async def test_init_uses_provided_cosmos_client(mock_cosmos_client: MagicMock) -> None:
|
||||
storage = CosmosCheckpointStorage(
|
||||
cosmos_client=mock_cosmos_client,
|
||||
database_name="db1",
|
||||
container_name="checkpoints",
|
||||
)
|
||||
assert storage.database_name == "db1"
|
||||
assert storage.container_name == "checkpoints"
|
||||
|
||||
|
||||
async def test_init_missing_required_settings_raises(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.delenv("AZURE_COSMOS_ENDPOINT", raising=False)
|
||||
monkeypatch.delenv("AZURE_COSMOS_DATABASE_NAME", raising=False)
|
||||
monkeypatch.delenv("AZURE_COSMOS_CONTAINER_NAME", raising=False)
|
||||
monkeypatch.delenv("AZURE_COSMOS_KEY", raising=False)
|
||||
|
||||
with pytest.raises(SettingNotFoundError, match="database_name"):
|
||||
CosmosCheckpointStorage()
|
||||
|
||||
|
||||
async def test_init_constructs_client_with_credential(
|
||||
monkeypatch: pytest.MonkeyPatch, mock_cosmos_client: MagicMock
|
||||
) -> None:
|
||||
"""Uses key-based auth when a key string is provided, otherwise falls back to Azure credential (RBAC)."""
|
||||
mock_factory = MagicMock(return_value=mock_cosmos_client)
|
||||
monkeypatch.setattr(checkpoint_storage_module, "CosmosClient", mock_factory)
|
||||
monkeypatch.delenv("AZURE_COSMOS_KEY", raising=False)
|
||||
|
||||
# Simulate real-world pattern: use key if available, else RBAC credential
|
||||
cosmos_key = os.getenv("AZURE_COSMOS_KEY")
|
||||
credential: Any = cosmos_key if cosmos_key else MagicMock() # MagicMock simulates DefaultAzureCredential()
|
||||
|
||||
CosmosCheckpointStorage(
|
||||
endpoint="https://account.documents.azure.com:443/",
|
||||
credential=credential,
|
||||
database_name="db1",
|
||||
container_name="checkpoints",
|
||||
)
|
||||
|
||||
mock_factory.assert_called_once()
|
||||
kwargs = mock_factory.call_args.kwargs
|
||||
assert kwargs["url"] == "https://account.documents.azure.com:443/"
|
||||
assert kwargs["credential"] is credential
|
||||
|
||||
|
||||
async def test_init_creates_database_and_container(mock_cosmos_client: MagicMock) -> None:
|
||||
storage = CosmosCheckpointStorage(
|
||||
cosmos_client=mock_cosmos_client,
|
||||
database_name="db1",
|
||||
container_name="custom-checkpoints",
|
||||
)
|
||||
|
||||
await storage.list_checkpoint_ids(workflow_name="wf")
|
||||
|
||||
mock_cosmos_client.create_database_if_not_exists.assert_awaited_once_with(id="db1")
|
||||
database_client = mock_cosmos_client.create_database_if_not_exists.return_value
|
||||
assert database_client.create_container_if_not_exists.await_count == 1
|
||||
kwargs = database_client.create_container_if_not_exists.await_args.kwargs
|
||||
assert kwargs["id"] == "custom-checkpoints"
|
||||
|
||||
|
||||
# --- Tests for save ---
|
||||
|
||||
|
||||
async def test_save_upserts_document(mock_container: MagicMock) -> None:
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
checkpoint = _make_checkpoint()
|
||||
|
||||
result = await storage.save(checkpoint)
|
||||
|
||||
assert result == checkpoint.checkpoint_id
|
||||
mock_container.upsert_item.assert_awaited_once()
|
||||
document = mock_container.upsert_item.await_args.kwargs["body"]
|
||||
assert document["id"] == f"test-workflow_{checkpoint.checkpoint_id}"
|
||||
assert document["workflow_name"] == "test-workflow"
|
||||
assert document["graph_signature_hash"] == "abc123"
|
||||
assert document["state"]["counter"] == 42
|
||||
|
||||
|
||||
async def test_save_returns_checkpoint_id(mock_container: MagicMock) -> None:
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
checkpoint = _make_checkpoint(checkpoint_id="cp-123")
|
||||
|
||||
result = await storage.save(checkpoint)
|
||||
|
||||
assert result == "cp-123"
|
||||
|
||||
|
||||
# --- Tests for load ---
|
||||
|
||||
|
||||
async def test_load_returns_checkpoint(mock_container: MagicMock) -> None:
|
||||
checkpoint = _make_checkpoint(checkpoint_id="cp-load")
|
||||
doc = _checkpoint_to_cosmos_document(checkpoint)
|
||||
mock_container.query_items.return_value = _to_async_iter([doc])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
loaded = await storage.load("cp-load")
|
||||
|
||||
assert loaded.checkpoint_id == "cp-load"
|
||||
assert loaded.workflow_name == "test-workflow"
|
||||
assert loaded.graph_signature_hash == "abc123"
|
||||
assert loaded.state["counter"] == 42
|
||||
|
||||
|
||||
async def test_load_nonexistent_raises(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
|
||||
with pytest.raises(WorkflowCheckpointException, match="No checkpoint found"):
|
||||
await storage.load("nonexistent-id")
|
||||
|
||||
|
||||
async def test_load_queries_without_partition_key(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
with suppress(WorkflowCheckpointException):
|
||||
await storage.load("cp-id")
|
||||
|
||||
kwargs = mock_container.query_items.call_args.kwargs
|
||||
assert "partition_key" not in kwargs
|
||||
|
||||
|
||||
async def test_load_multiple_workflows_same_checkpoint_id_raises(mock_container: MagicMock) -> None:
|
||||
cp1 = _make_checkpoint(checkpoint_id="shared-id", workflow_name="workflow-a")
|
||||
cp2 = _make_checkpoint(checkpoint_id="shared-id", workflow_name="workflow-b")
|
||||
mock_container.query_items.return_value = _to_async_iter([
|
||||
_checkpoint_to_cosmos_document(cp1),
|
||||
_checkpoint_to_cosmos_document(cp2),
|
||||
])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
|
||||
with pytest.raises(WorkflowCheckpointException, match="Multiple checkpoints found"):
|
||||
await storage.load("shared-id")
|
||||
|
||||
|
||||
# --- Tests for list_checkpoints ---
|
||||
|
||||
|
||||
async def test_list_checkpoints_returns_checkpoints_for_workflow(mock_container: MagicMock) -> None:
|
||||
cp1 = _make_checkpoint(checkpoint_id="cp-1", timestamp="2025-01-01T00:00:00+00:00")
|
||||
cp2 = _make_checkpoint(checkpoint_id="cp-2", timestamp="2025-01-02T00:00:00+00:00")
|
||||
mock_container.query_items.return_value = _to_async_iter([
|
||||
_checkpoint_to_cosmos_document(cp1),
|
||||
_checkpoint_to_cosmos_document(cp2),
|
||||
])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
results = await storage.list_checkpoints(workflow_name="test-workflow")
|
||||
|
||||
assert len(results) == 2
|
||||
assert results[0].checkpoint_id == "cp-1"
|
||||
assert results[1].checkpoint_id == "cp-2"
|
||||
|
||||
|
||||
async def test_list_checkpoints_uses_partition_key(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
await storage.list_checkpoints(workflow_name="my-workflow")
|
||||
|
||||
kwargs = mock_container.query_items.call_args.kwargs
|
||||
assert kwargs["partition_key"] == "my-workflow"
|
||||
|
||||
|
||||
async def test_list_checkpoints_empty_returns_empty(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
results = await storage.list_checkpoints(workflow_name="test-workflow")
|
||||
|
||||
assert results == []
|
||||
|
||||
|
||||
async def test_list_checkpoints_skips_malformed_documents(mock_container: MagicMock) -> None:
|
||||
valid_cp = _make_checkpoint(checkpoint_id="cp-valid")
|
||||
mock_container.query_items.return_value = _to_async_iter([
|
||||
{"id": "bad_doc", "workflow_name": "test-workflow", "not_a_checkpoint": True},
|
||||
_checkpoint_to_cosmos_document(valid_cp),
|
||||
])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
results = await storage.list_checkpoints(workflow_name="test-workflow")
|
||||
|
||||
assert len(results) == 1
|
||||
assert results[0].checkpoint_id == "cp-valid"
|
||||
|
||||
|
||||
# --- Tests for delete ---
|
||||
|
||||
|
||||
async def test_delete_existing_returns_true(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([
|
||||
{"id": "test-workflow_cp-del", "workflow_name": "test-workflow"},
|
||||
])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
result = await storage.delete("cp-del")
|
||||
|
||||
assert result is True
|
||||
mock_container.delete_item.assert_awaited_once_with(
|
||||
item="test-workflow_cp-del",
|
||||
partition_key="test-workflow",
|
||||
)
|
||||
|
||||
|
||||
async def test_delete_nonexistent_returns_false(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
result = await storage.delete("nonexistent")
|
||||
|
||||
assert result is False
|
||||
mock_container.delete_item.assert_not_awaited()
|
||||
|
||||
|
||||
async def test_delete_cosmos_not_found_returns_false(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([
|
||||
{"id": "test-workflow_cp-del", "workflow_name": "test-workflow"},
|
||||
])
|
||||
mock_container.delete_item = AsyncMock(side_effect=CosmosResourceNotFoundError)
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
result = await storage.delete("cp-del")
|
||||
|
||||
assert result is False
|
||||
|
||||
|
||||
# --- Tests for get_latest ---
|
||||
|
||||
|
||||
async def test_get_latest_returns_latest_checkpoint(mock_container: MagicMock) -> None:
|
||||
cp = _make_checkpoint(checkpoint_id="cp-latest", timestamp="2025-06-01T00:00:00+00:00")
|
||||
mock_container.query_items.return_value = _to_async_iter([
|
||||
_checkpoint_to_cosmos_document(cp),
|
||||
])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
result = await storage.get_latest(workflow_name="test-workflow")
|
||||
|
||||
assert result is not None
|
||||
assert result.checkpoint_id == "cp-latest"
|
||||
|
||||
|
||||
async def test_get_latest_returns_none_when_empty(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
result = await storage.get_latest(workflow_name="test-workflow")
|
||||
|
||||
assert result is None
|
||||
|
||||
|
||||
async def test_get_latest_uses_order_by_desc_with_limit(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
await storage.get_latest(workflow_name="test-workflow")
|
||||
|
||||
kwargs = mock_container.query_items.call_args.kwargs
|
||||
assert "ORDER BY c.timestamp DESC" in kwargs["query"]
|
||||
assert "OFFSET 0 LIMIT 1" in kwargs["query"]
|
||||
|
||||
|
||||
# --- Tests for list_checkpoint_ids ---
|
||||
|
||||
|
||||
async def test_list_checkpoint_ids_returns_ids(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([
|
||||
{"checkpoint_id": "cp-1"},
|
||||
{"checkpoint_id": "cp-2"},
|
||||
])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
ids = await storage.list_checkpoint_ids(workflow_name="test-workflow")
|
||||
|
||||
assert ids == ["cp-1", "cp-2"]
|
||||
|
||||
|
||||
async def test_list_checkpoint_ids_empty_returns_empty(mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
ids = await storage.list_checkpoint_ids(workflow_name="test-workflow")
|
||||
|
||||
assert ids == []
|
||||
|
||||
|
||||
# --- Tests for close and context manager ---
|
||||
|
||||
|
||||
async def test_close_closes_owned_client(monkeypatch: pytest.MonkeyPatch, mock_cosmos_client: MagicMock) -> None:
|
||||
mock_factory = MagicMock(return_value=mock_cosmos_client)
|
||||
monkeypatch.setattr(checkpoint_storage_module, "CosmosClient", mock_factory)
|
||||
|
||||
storage = CosmosCheckpointStorage(
|
||||
endpoint="https://account.documents.azure.com:443/",
|
||||
credential="key-123",
|
||||
database_name="db1",
|
||||
container_name="checkpoints",
|
||||
)
|
||||
|
||||
await storage.close()
|
||||
|
||||
mock_cosmos_client.close.assert_awaited_once()
|
||||
|
||||
|
||||
async def test_close_does_not_close_external_client(mock_cosmos_client: MagicMock) -> None:
|
||||
storage = CosmosCheckpointStorage(
|
||||
cosmos_client=mock_cosmos_client,
|
||||
database_name="db1",
|
||||
container_name="checkpoints",
|
||||
)
|
||||
|
||||
await storage.close()
|
||||
|
||||
mock_cosmos_client.close.assert_not_awaited()
|
||||
|
||||
|
||||
async def test_context_manager_closes_owned_client(
|
||||
monkeypatch: pytest.MonkeyPatch, mock_cosmos_client: MagicMock
|
||||
) -> None:
|
||||
mock_factory = MagicMock(return_value=mock_cosmos_client)
|
||||
monkeypatch.setattr(checkpoint_storage_module, "CosmosClient", mock_factory)
|
||||
|
||||
async with CosmosCheckpointStorage(
|
||||
endpoint="https://account.documents.azure.com:443/",
|
||||
credential="key-123",
|
||||
database_name="db1",
|
||||
container_name="checkpoints",
|
||||
) as storage:
|
||||
assert storage is not None
|
||||
|
||||
mock_cosmos_client.close.assert_awaited_once()
|
||||
|
||||
|
||||
async def test_context_manager_preserves_original_exception(mock_container: MagicMock) -> None:
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
|
||||
with (
|
||||
patch.object(storage, "close", AsyncMock(side_effect=RuntimeError("close failed"))),
|
||||
pytest.raises(ValueError, match="inner error"),
|
||||
):
|
||||
async with storage:
|
||||
raise ValueError("inner error")
|
||||
|
||||
|
||||
async def test_context_manager_reraises_close_error(mock_container: MagicMock) -> None:
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
|
||||
with (
|
||||
patch.object(storage, "close", AsyncMock(side_effect=RuntimeError("close failed"))),
|
||||
pytest.raises(RuntimeError, match="close failed"),
|
||||
):
|
||||
async with storage:
|
||||
pass # no inner exception — close error should propagate
|
||||
|
||||
|
||||
# --- Tests for save/load round-trip ---
|
||||
|
||||
|
||||
async def test_round_trip_preserves_data(mock_container: MagicMock) -> None:
|
||||
checkpoint = _make_checkpoint(
|
||||
checkpoint_id="cp-roundtrip",
|
||||
previous_checkpoint_id="cp-parent",
|
||||
)
|
||||
checkpoint.state = {"key": "value", "nested": {"a": 1}}
|
||||
checkpoint.metadata = {"superstep": 3}
|
||||
checkpoint.iteration_count = 5
|
||||
|
||||
saved_doc: dict[str, Any] = {}
|
||||
|
||||
async def capture_upsert(body: dict[str, Any]) -> dict[str, Any]:
|
||||
saved_doc.update(body)
|
||||
return body
|
||||
|
||||
mock_container.upsert_item = AsyncMock(side_effect=capture_upsert)
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
await storage.save(checkpoint)
|
||||
|
||||
returned_doc = {
|
||||
**saved_doc,
|
||||
"_rid": "abc",
|
||||
"_self": "dbs/abc/colls/def/docs/ghi",
|
||||
"_etag": '"etag"',
|
||||
"_attachments": "attachments/",
|
||||
"_ts": 1700000000,
|
||||
}
|
||||
mock_container.query_items.return_value = _to_async_iter([returned_doc])
|
||||
|
||||
loaded = await storage.load("cp-roundtrip")
|
||||
|
||||
assert loaded.checkpoint_id == checkpoint.checkpoint_id
|
||||
assert loaded.workflow_name == checkpoint.workflow_name
|
||||
assert loaded.graph_signature_hash == checkpoint.graph_signature_hash
|
||||
assert loaded.previous_checkpoint_id == "cp-parent"
|
||||
assert loaded.state == {"key": "value", "nested": {"a": 1}}
|
||||
assert loaded.metadata == {"superstep": 3}
|
||||
assert loaded.iteration_count == 5
|
||||
assert loaded.version == "1.0"
|
||||
|
||||
|
||||
# --- Integration test ---
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@skip_if_cosmos_integration_tests_disabled
|
||||
async def test_cosmos_checkpoint_storage_roundtrip_with_emulator() -> None:
|
||||
endpoint = os.getenv("AZURE_COSMOS_ENDPOINT", "")
|
||||
key = os.getenv("AZURE_COSMOS_KEY", "")
|
||||
database_prefix = os.getenv("AZURE_COSMOS_DATABASE_NAME", "")
|
||||
container_prefix = os.getenv("AZURE_COSMOS_CONTAINER_NAME", "")
|
||||
unique = uuid.uuid4().hex[:8]
|
||||
database_name = f"{database_prefix}-cp-{unique}"
|
||||
container_name = f"{container_prefix}-cp-{unique}"
|
||||
|
||||
async with CosmosClient(url=endpoint, credential=key) as cosmos_client:
|
||||
await cosmos_client.create_database_if_not_exists(id=database_name)
|
||||
|
||||
storage = CosmosCheckpointStorage(
|
||||
cosmos_client=cosmos_client,
|
||||
database_name=database_name,
|
||||
container_name=container_name,
|
||||
)
|
||||
|
||||
try:
|
||||
# Save two checkpoints for the same workflow
|
||||
cp1 = _make_checkpoint(
|
||||
checkpoint_id="cp-int-1",
|
||||
workflow_name="integration-wf",
|
||||
timestamp="2025-01-01T00:00:00+00:00",
|
||||
)
|
||||
cp2 = _make_checkpoint(
|
||||
checkpoint_id="cp-int-2",
|
||||
workflow_name="integration-wf",
|
||||
previous_checkpoint_id="cp-int-1",
|
||||
timestamp="2025-01-02T00:00:00+00:00",
|
||||
)
|
||||
cp2.state = {"step": 2}
|
||||
|
||||
await storage.save(cp1)
|
||||
await storage.save(cp2)
|
||||
|
||||
# Load by ID
|
||||
loaded = await storage.load("cp-int-1")
|
||||
assert loaded.checkpoint_id == "cp-int-1"
|
||||
assert loaded.workflow_name == "integration-wf"
|
||||
|
||||
# List all checkpoints for workflow
|
||||
all_cps = await storage.list_checkpoints(workflow_name="integration-wf")
|
||||
assert len(all_cps) == 2
|
||||
|
||||
# List checkpoint IDs
|
||||
ids = await storage.list_checkpoint_ids(workflow_name="integration-wf")
|
||||
assert "cp-int-1" in ids
|
||||
assert "cp-int-2" in ids
|
||||
|
||||
# Get latest
|
||||
latest = await storage.get_latest(workflow_name="integration-wf")
|
||||
assert latest is not None
|
||||
assert latest.checkpoint_id == "cp-int-2"
|
||||
assert latest.state == {"step": 2}
|
||||
|
||||
# Delete
|
||||
assert await storage.delete("cp-int-1") is True
|
||||
assert await storage.delete("cp-int-1") is False
|
||||
|
||||
remaining = await storage.list_checkpoint_ids(workflow_name="integration-wf")
|
||||
assert remaining == ["cp-int-2"]
|
||||
|
||||
# Cross-workflow isolation
|
||||
other_cp = _make_checkpoint(
|
||||
checkpoint_id="cp-other",
|
||||
workflow_name="other-wf",
|
||||
)
|
||||
await storage.save(other_cp)
|
||||
wf_cps = await storage.list_checkpoints(workflow_name="integration-wf")
|
||||
assert len(wf_cps) == 1
|
||||
assert wf_cps[0].checkpoint_id == "cp-int-2"
|
||||
|
||||
finally:
|
||||
with suppress(Exception):
|
||||
await cosmos_client.delete_database(database_name)
|
||||
|
||||
|
||||
# --- Tests for allowed_checkpoint_types ---
|
||||
|
||||
|
||||
@dataclass
|
||||
class _AppState:
|
||||
"""Application-defined state type used to test allowed_checkpoint_types."""
|
||||
|
||||
label: str
|
||||
count: int
|
||||
|
||||
|
||||
_APP_STATE_TYPE_KEY = f"{_AppState.__module__}:{_AppState.__qualname__}"
|
||||
|
||||
|
||||
def _make_checkpoint_with_state(state: dict[str, Any]) -> WorkflowCheckpoint:
|
||||
"""Create a checkpoint with custom state for serialization tests."""
|
||||
return WorkflowCheckpoint(
|
||||
workflow_name="test-workflow",
|
||||
graph_signature_hash="abc123",
|
||||
timestamp="2025-01-01T00:00:00+00:00",
|
||||
state=state,
|
||||
iteration_count=1,
|
||||
)
|
||||
|
||||
|
||||
async def test_init_accepts_allowed_checkpoint_types(mock_container: MagicMock) -> None:
|
||||
"""CosmosCheckpointStorage.__init__ accepts allowed_checkpoint_types."""
|
||||
storage = CosmosCheckpointStorage(
|
||||
container_client=mock_container,
|
||||
allowed_checkpoint_types=["some.module:SomeType"],
|
||||
)
|
||||
assert storage is not None
|
||||
|
||||
|
||||
async def test_load_allows_builtin_safe_types(mock_container: MagicMock) -> None:
|
||||
"""Built-in safe types load without opt-in via allowed_checkpoint_types."""
|
||||
from datetime import datetime, timezone
|
||||
|
||||
checkpoint = _make_checkpoint_with_state({
|
||||
"ts": datetime(2025, 1, 1, tzinfo=timezone.utc),
|
||||
"tags": {1, 2, 3},
|
||||
})
|
||||
doc = _checkpoint_to_cosmos_document(checkpoint)
|
||||
mock_container.query_items.return_value = _to_async_iter([doc])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
loaded = await storage.load(checkpoint.checkpoint_id)
|
||||
|
||||
assert loaded.state["ts"] == datetime(2025, 1, 1, tzinfo=timezone.utc)
|
||||
assert loaded.state["tags"] == {1, 2, 3}
|
||||
|
||||
|
||||
async def test_load_blocks_unlisted_app_type(mock_container: MagicMock) -> None:
|
||||
"""Application types are blocked when not listed in allowed_checkpoint_types."""
|
||||
checkpoint = _make_checkpoint_with_state({"data": _AppState(label="x", count=1)})
|
||||
doc = _checkpoint_to_cosmos_document(checkpoint)
|
||||
mock_container.query_items.return_value = _to_async_iter([doc])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
|
||||
with pytest.raises(WorkflowCheckpointException, match="deserialization blocked"):
|
||||
await storage.load(checkpoint.checkpoint_id)
|
||||
|
||||
|
||||
async def test_load_allows_listed_app_type(mock_container: MagicMock) -> None:
|
||||
"""Application types are allowed when listed in allowed_checkpoint_types."""
|
||||
checkpoint = _make_checkpoint_with_state({"data": _AppState(label="ok", count=7)})
|
||||
doc = _checkpoint_to_cosmos_document(checkpoint)
|
||||
mock_container.query_items.return_value = _to_async_iter([doc])
|
||||
|
||||
storage = CosmosCheckpointStorage(
|
||||
container_client=mock_container,
|
||||
allowed_checkpoint_types=[_APP_STATE_TYPE_KEY],
|
||||
)
|
||||
loaded = await storage.load(checkpoint.checkpoint_id)
|
||||
|
||||
assert isinstance(loaded.state["data"], _AppState)
|
||||
assert loaded.state["data"].label == "ok"
|
||||
assert loaded.state["data"].count == 7
|
||||
|
||||
|
||||
async def test_list_checkpoints_blocks_unlisted_app_type(mock_container: MagicMock) -> None:
|
||||
"""list_checkpoints skips documents with unlisted application types."""
|
||||
checkpoint = _make_checkpoint_with_state({"data": _AppState(label="x", count=1)})
|
||||
doc = _checkpoint_to_cosmos_document(checkpoint)
|
||||
mock_container.query_items.return_value = _to_async_iter([doc])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
results = await storage.list_checkpoints(workflow_name="test-workflow")
|
||||
|
||||
# The document is skipped (logged as warning) because the type is blocked
|
||||
assert len(results) == 0
|
||||
|
||||
|
||||
async def test_list_checkpoints_allows_listed_app_type(mock_container: MagicMock) -> None:
|
||||
"""list_checkpoints decodes documents with listed application types."""
|
||||
checkpoint = _make_checkpoint_with_state({"data": _AppState(label="ok", count=3)})
|
||||
doc = _checkpoint_to_cosmos_document(checkpoint)
|
||||
mock_container.query_items.return_value = _to_async_iter([doc])
|
||||
|
||||
storage = CosmosCheckpointStorage(
|
||||
container_client=mock_container,
|
||||
allowed_checkpoint_types=[_APP_STATE_TYPE_KEY],
|
||||
)
|
||||
results = await storage.list_checkpoints(workflow_name="test-workflow")
|
||||
|
||||
assert len(results) == 1
|
||||
assert isinstance(results[0].state["data"], _AppState)
|
||||
|
||||
|
||||
async def test_get_latest_blocks_unlisted_app_type(mock_container: MagicMock) -> None:
|
||||
"""get_latest raises when the checkpoint contains an unlisted application type."""
|
||||
checkpoint = _make_checkpoint_with_state({"data": _AppState(label="x", count=1)})
|
||||
doc = _checkpoint_to_cosmos_document(checkpoint)
|
||||
mock_container.query_items.return_value = _to_async_iter([doc])
|
||||
|
||||
storage = CosmosCheckpointStorage(container_client=mock_container)
|
||||
|
||||
with pytest.raises(WorkflowCheckpointException, match="deserialization blocked"):
|
||||
await storage.get_latest(workflow_name="test-workflow")
|
||||
|
||||
|
||||
async def test_get_latest_allows_listed_app_type(mock_container: MagicMock) -> None:
|
||||
"""get_latest decodes checkpoints with listed application types."""
|
||||
checkpoint = _make_checkpoint_with_state({"data": _AppState(label="latest", count=9)})
|
||||
doc = _checkpoint_to_cosmos_document(checkpoint)
|
||||
mock_container.query_items.return_value = _to_async_iter([doc])
|
||||
|
||||
storage = CosmosCheckpointStorage(
|
||||
container_client=mock_container,
|
||||
allowed_checkpoint_types=[_APP_STATE_TYPE_KEY],
|
||||
)
|
||||
result = await storage.get_latest(workflow_name="test-workflow")
|
||||
|
||||
assert result is not None
|
||||
assert isinstance(result.state["data"], _AppState)
|
||||
assert result.state["data"].label == "latest"
|
||||
@@ -0,0 +1,417 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import uuid
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import suppress
|
||||
from typing import Any, cast
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from agent_framework import AgentResponse, Message
|
||||
from agent_framework._sessions import AgentSession, SessionContext
|
||||
from agent_framework.exceptions import SettingNotFoundError
|
||||
from azure.cosmos.aio import CosmosClient
|
||||
from azure.cosmos.exceptions import CosmosResourceNotFoundError
|
||||
|
||||
import agent_framework_azure_cosmos._history_provider as history_provider_module
|
||||
from agent_framework_azure_cosmos._history_provider import CosmosHistoryProvider
|
||||
|
||||
skip_if_cosmos_integration_tests_disabled = pytest.mark.skipif(
|
||||
any(
|
||||
os.getenv(name, "") == ""
|
||||
for name in (
|
||||
"AZURE_COSMOS_ENDPOINT",
|
||||
"AZURE_COSMOS_KEY",
|
||||
"AZURE_COSMOS_DATABASE_NAME",
|
||||
"AZURE_COSMOS_CONTAINER_NAME",
|
||||
)
|
||||
),
|
||||
reason=(
|
||||
"AZURE_COSMOS_ENDPOINT, AZURE_COSMOS_KEY, AZURE_COSMOS_DATABASE_NAME, and "
|
||||
"AZURE_COSMOS_CONTAINER_NAME are required for Cosmos integration tests."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _to_async_iter(items: list[Any]) -> AsyncIterator[Any]:
|
||||
async def _iterator() -> AsyncIterator[Any]:
|
||||
for item in items:
|
||||
yield item
|
||||
|
||||
return _iterator()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_container() -> MagicMock:
|
||||
container = MagicMock()
|
||||
container.query_items = MagicMock(return_value=_to_async_iter([]))
|
||||
container.execute_item_batch = AsyncMock(return_value=[])
|
||||
return container
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_cosmos_client(mock_container: MagicMock) -> MagicMock:
|
||||
database_client = MagicMock()
|
||||
database_client.create_container_if_not_exists = AsyncMock(return_value=mock_container)
|
||||
|
||||
client = MagicMock()
|
||||
client.get_database_client.return_value = database_client
|
||||
client.close = AsyncMock()
|
||||
return client
|
||||
|
||||
|
||||
class TestCosmosHistoryProviderInit:
|
||||
def test_uses_provided_container_client(self, mock_container: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
assert provider.source_id == "mem"
|
||||
assert provider.load_messages is True
|
||||
assert provider.store_outputs is True
|
||||
assert provider.store_inputs is True
|
||||
assert provider.database_name == ""
|
||||
assert provider.container_name == ""
|
||||
|
||||
def test_uses_provided_cosmos_client(self, mock_cosmos_client: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(
|
||||
source_id="mem",
|
||||
cosmos_client=mock_cosmos_client,
|
||||
database_name="db1",
|
||||
container_name="history",
|
||||
)
|
||||
|
||||
mock_cosmos_client.get_database_client.assert_called_once_with("db1")
|
||||
assert provider.database_name == "db1"
|
||||
assert provider.container_name == "history"
|
||||
|
||||
def test_missing_required_settings_raises(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.delenv("AZURE_COSMOS_ENDPOINT", raising=False)
|
||||
monkeypatch.delenv("AZURE_COSMOS_DATABASE_NAME", raising=False)
|
||||
monkeypatch.delenv("AZURE_COSMOS_CONTAINER_NAME", raising=False)
|
||||
monkeypatch.delenv("AZURE_COSMOS_KEY", raising=False)
|
||||
|
||||
with pytest.raises(SettingNotFoundError, match="database_name"):
|
||||
CosmosHistoryProvider()
|
||||
|
||||
def test_constructs_client_with_string_credential(
|
||||
self, monkeypatch: pytest.MonkeyPatch, mock_cosmos_client: MagicMock
|
||||
) -> None:
|
||||
mock_factory = MagicMock(return_value=mock_cosmos_client)
|
||||
monkeypatch.setattr(history_provider_module, "CosmosClient", mock_factory)
|
||||
|
||||
CosmosHistoryProvider(
|
||||
endpoint="https://account.documents.azure.com:443/",
|
||||
credential="key-123",
|
||||
database_name="db1",
|
||||
container_name="history",
|
||||
)
|
||||
|
||||
mock_factory.assert_called_once()
|
||||
kwargs = mock_factory.call_args.kwargs
|
||||
assert kwargs["url"] == "https://account.documents.azure.com:443/"
|
||||
assert kwargs["credential"] == "key-123"
|
||||
|
||||
|
||||
class TestCosmosHistoryProviderContainerConfig:
|
||||
async def test_provider_container_name_is_used(self, mock_cosmos_client: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(
|
||||
source_id="mem",
|
||||
cosmos_client=mock_cosmos_client,
|
||||
database_name="db1",
|
||||
container_name="custom-history",
|
||||
)
|
||||
|
||||
await provider.get_messages("session-123")
|
||||
|
||||
database_client = mock_cosmos_client.get_database_client.return_value
|
||||
assert database_client.create_container_if_not_exists.await_count == 1
|
||||
kwargs = database_client.create_container_if_not_exists.await_args.kwargs
|
||||
assert kwargs["id"] == "custom-history"
|
||||
|
||||
|
||||
class TestCosmosHistoryProviderGetMessages:
|
||||
async def test_returns_deserialized_messages(self, mock_container: MagicMock) -> None:
|
||||
msg1 = Message(role="user", contents=["Hello"])
|
||||
msg2 = Message(role="assistant", contents=["Hi"])
|
||||
mock_container.query_items.return_value = _to_async_iter([
|
||||
{"message": msg1.to_dict()},
|
||||
{"message": msg2.to_dict()},
|
||||
])
|
||||
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
messages = await provider.get_messages("s1")
|
||||
|
||||
assert len(messages) == 2
|
||||
assert messages[0].role == "user"
|
||||
assert messages[0].text == "Hello"
|
||||
assert messages[1].role == "assistant"
|
||||
assert messages[1].text == "Hi"
|
||||
query_kwargs = mock_container.query_items.call_args.kwargs
|
||||
assert query_kwargs["partition_key"] == "s1"
|
||||
assert query_kwargs["query"] == (
|
||||
"SELECT c.message FROM c "
|
||||
"WHERE c.session_id = @session_id AND c.source_id = @source_id "
|
||||
"ORDER BY c.sort_key ASC"
|
||||
)
|
||||
assert query_kwargs["parameters"] == [
|
||||
{"name": "@session_id", "value": "s1"},
|
||||
{"name": "@source_id", "value": "mem"},
|
||||
]
|
||||
|
||||
async def test_empty_returns_empty(self, mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
messages = await provider.get_messages("s1")
|
||||
|
||||
assert messages == []
|
||||
|
||||
async def test_none_session_id_generates_guid_partition_key(
|
||||
self, mock_container: MagicMock, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([])
|
||||
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
with caplog.at_level("WARNING"):
|
||||
await provider.get_messages(None)
|
||||
|
||||
query_kwargs = mock_container.query_items.call_args.kwargs
|
||||
session_key = query_kwargs["partition_key"]
|
||||
assert isinstance(session_key, str)
|
||||
assert session_key != ""
|
||||
assert session_key != "default"
|
||||
uuid.UUID(session_key)
|
||||
assert query_kwargs["parameters"] == [
|
||||
{"name": "@session_id", "value": session_key},
|
||||
{"name": "@source_id", "value": "mem"},
|
||||
]
|
||||
assert "Received empty session_id" in caplog.text
|
||||
|
||||
async def test_skips_non_dict_message_payload(self, mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([{"message": "bad"}, {"message": None}])
|
||||
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
messages = await provider.get_messages("s1")
|
||||
|
||||
assert messages == []
|
||||
|
||||
|
||||
class TestCosmosHistoryProviderListSessions:
|
||||
async def test_list_sessions_returns_unique_sorted_ids(self, mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter(["s2", "s1", "s1", "s3"])
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
|
||||
sessions = await provider.list_sessions()
|
||||
|
||||
assert sessions == ["s1", "s2", "s3"]
|
||||
kwargs = mock_container.query_items.call_args.kwargs
|
||||
assert kwargs["query"] == "SELECT DISTINCT VALUE c.session_id FROM c WHERE c.source_id = @source_id"
|
||||
assert kwargs["parameters"] == [{"name": "@source_id", "value": "mem"}]
|
||||
|
||||
|
||||
class TestCosmosHistoryProviderSaveMessages:
|
||||
async def test_saves_messages(self, mock_container: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
messages = [Message(role="user", contents=["Hello"]), Message(role="assistant", contents=["Hi"])]
|
||||
|
||||
await provider.save_messages("s1", messages)
|
||||
|
||||
mock_container.execute_item_batch.assert_awaited_once()
|
||||
batch_operations = mock_container.execute_item_batch.await_args.kwargs["batch_operations"]
|
||||
assert len(batch_operations) == 2
|
||||
first_operation, first_args = batch_operations[0]
|
||||
assert first_operation == "upsert"
|
||||
first_document = first_args[0]
|
||||
assert first_document["session_id"] == "s1"
|
||||
assert first_document["message"]["role"] == "user"
|
||||
assert mock_container.execute_item_batch.await_args.kwargs["partition_key"] == "s1"
|
||||
|
||||
async def test_empty_messages_noop(self, mock_container: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
|
||||
await provider.save_messages("s1", [])
|
||||
|
||||
mock_container.execute_item_batch.assert_not_awaited()
|
||||
|
||||
async def test_batches_when_message_count_exceeds_limit(self, mock_container: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
messages = [Message(role="user", contents=[f"msg-{index}"]) for index in range(101)]
|
||||
|
||||
await provider.save_messages("s1", messages)
|
||||
|
||||
assert mock_container.execute_item_batch.await_count == 2
|
||||
first_call = mock_container.execute_item_batch.await_args_list[0].kwargs
|
||||
second_call = mock_container.execute_item_batch.await_args_list[1].kwargs
|
||||
assert len(first_call["batch_operations"]) == 100
|
||||
assert len(second_call["batch_operations"]) == 1
|
||||
assert first_call["partition_key"] == "s1"
|
||||
assert second_call["partition_key"] == "s1"
|
||||
|
||||
|
||||
class TestCosmosHistoryProviderClear:
|
||||
async def test_clear_deletes_all_session_items(self, mock_container: MagicMock) -> None:
|
||||
mock_container.query_items.return_value = _to_async_iter([{"id": "1"}, {"id": "2"}])
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
|
||||
await provider.clear("s1")
|
||||
|
||||
mock_container.execute_item_batch.assert_awaited_once()
|
||||
batch_operations = mock_container.execute_item_batch.await_args.kwargs["batch_operations"]
|
||||
assert len(batch_operations) == 2
|
||||
assert batch_operations[0] == ("delete", ("1",))
|
||||
assert batch_operations[1] == ("delete", ("2",))
|
||||
assert mock_container.execute_item_batch.await_args.kwargs["partition_key"] == "s1"
|
||||
query_kwargs = mock_container.query_items.call_args.kwargs
|
||||
assert query_kwargs["query"] == (
|
||||
"SELECT c.id FROM c WHERE c.session_id = @session_id AND c.source_id = @source_id"
|
||||
)
|
||||
assert query_kwargs["parameters"] == [
|
||||
{"name": "@session_id", "value": "s1"},
|
||||
{"name": "@source_id", "value": "mem"},
|
||||
]
|
||||
|
||||
|
||||
class TestCosmosHistoryProviderBeforeAfterRun:
|
||||
async def test_before_run_loads_history(self, mock_container: MagicMock) -> None:
|
||||
msg = Message(role="user", contents=["old msg"])
|
||||
mock_container.query_items.return_value = _to_async_iter([{"message": msg.to_dict()}])
|
||||
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
session = AgentSession(session_id="test")
|
||||
context = SessionContext(input_messages=[Message(role="user", contents=["new msg"])], session_id="s1")
|
||||
|
||||
await provider.before_run(
|
||||
agent=cast(Any, None),
|
||||
session=session,
|
||||
context=context,
|
||||
state=session.state.setdefault(provider.source_id, {}),
|
||||
)
|
||||
|
||||
assert "mem" in context.context_messages
|
||||
assert context.context_messages["mem"][0].text == "old msg"
|
||||
|
||||
async def test_after_run_stores_input_and_response(self, mock_container: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
session = AgentSession(session_id="test")
|
||||
context = SessionContext(input_messages=[Message(role="user", contents=["hi"])], session_id="s1")
|
||||
context._response = AgentResponse(messages=[Message(role="assistant", contents=["hello"])])
|
||||
|
||||
await provider.after_run(
|
||||
agent=cast(Any, None),
|
||||
session=session,
|
||||
context=context,
|
||||
state=session.state.setdefault(provider.source_id, {}),
|
||||
)
|
||||
|
||||
mock_container.execute_item_batch.assert_awaited_once()
|
||||
batch_operations = mock_container.execute_item_batch.await_args.kwargs["batch_operations"]
|
||||
assert len(batch_operations) == 2
|
||||
input_doc = batch_operations[0][1][0]
|
||||
response_doc = batch_operations[1][1][0]
|
||||
assert input_doc["message"]["role"] == "user"
|
||||
assert input_doc["message"]["contents"][0]["text"] == "hi"
|
||||
assert response_doc["message"]["role"] == "assistant"
|
||||
assert response_doc["message"]["contents"][0]["text"] == "hello"
|
||||
|
||||
|
||||
class TestCosmosHistoryProviderClose:
|
||||
async def test_close_closes_owned_client(
|
||||
self, monkeypatch: pytest.MonkeyPatch, mock_cosmos_client: MagicMock
|
||||
) -> None:
|
||||
mock_factory = MagicMock(return_value=mock_cosmos_client)
|
||||
monkeypatch.setattr(history_provider_module, "CosmosClient", mock_factory)
|
||||
|
||||
provider = CosmosHistoryProvider(
|
||||
endpoint="https://account.documents.azure.com:443/",
|
||||
credential="key-123",
|
||||
database_name="db1",
|
||||
container_name="history",
|
||||
)
|
||||
|
||||
await provider.close()
|
||||
|
||||
mock_cosmos_client.close.assert_awaited_once()
|
||||
|
||||
async def test_close_does_not_close_external_client(self, mock_cosmos_client: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(
|
||||
source_id="mem",
|
||||
cosmos_client=mock_cosmos_client,
|
||||
database_name="db1",
|
||||
container_name="history",
|
||||
)
|
||||
|
||||
await provider.close()
|
||||
|
||||
mock_cosmos_client.close.assert_not_awaited()
|
||||
|
||||
async def test_async_context_manager_closes_owned_client(
|
||||
self, monkeypatch: pytest.MonkeyPatch, mock_cosmos_client: MagicMock
|
||||
) -> None:
|
||||
mock_factory = MagicMock(return_value=mock_cosmos_client)
|
||||
monkeypatch.setattr(history_provider_module, "CosmosClient", mock_factory)
|
||||
|
||||
async with CosmosHistoryProvider(
|
||||
endpoint="https://account.documents.azure.com:443/",
|
||||
credential="key-123",
|
||||
database_name="db1",
|
||||
container_name="history",
|
||||
) as provider:
|
||||
assert provider is not None
|
||||
|
||||
mock_cosmos_client.close.assert_awaited_once()
|
||||
|
||||
async def test_async_context_manager_preserves_original_exception(self, mock_container: MagicMock) -> None:
|
||||
provider = CosmosHistoryProvider(source_id="mem", container_client=mock_container)
|
||||
|
||||
with (
|
||||
patch.object(provider, "close", AsyncMock(side_effect=RuntimeError("close failed"))),
|
||||
pytest.raises(ValueError, match="inner error"),
|
||||
):
|
||||
async with provider:
|
||||
raise ValueError("inner error")
|
||||
|
||||
|
||||
@pytest.mark.flaky
|
||||
@pytest.mark.integration
|
||||
@skip_if_cosmos_integration_tests_disabled
|
||||
async def test_cosmos_history_provider_roundtrip_with_emulator() -> None:
|
||||
endpoint = os.getenv("AZURE_COSMOS_ENDPOINT", "")
|
||||
key = os.getenv("AZURE_COSMOS_KEY", "")
|
||||
database_prefix = os.getenv("AZURE_COSMOS_DATABASE_NAME", "")
|
||||
container_prefix = os.getenv("AZURE_COSMOS_CONTAINER_NAME", "")
|
||||
unique = uuid.uuid4().hex[:8]
|
||||
database_name = f"{database_prefix}-{unique}"
|
||||
container_name = f"{container_prefix}-{unique}"
|
||||
session_id = f"session-{unique}"
|
||||
|
||||
async with CosmosClient(url=endpoint, credential=key) as cosmos_client:
|
||||
await cosmos_client.create_database_if_not_exists(id=database_name)
|
||||
provider = CosmosHistoryProvider(
|
||||
source_id="cosmos_integration",
|
||||
cosmos_client=cosmos_client,
|
||||
database_name=database_name,
|
||||
container_name=container_name,
|
||||
)
|
||||
|
||||
try:
|
||||
await provider.save_messages(
|
||||
session_id,
|
||||
[
|
||||
Message(role="user", contents=["Hello Cosmos"]),
|
||||
Message(role="assistant", contents=["Hi from Cosmos"]),
|
||||
],
|
||||
)
|
||||
|
||||
stored_messages = await provider.get_messages(session_id)
|
||||
assert [message.role for message in stored_messages] == ["user", "assistant"]
|
||||
assert [message.text for message in stored_messages] == ["Hello Cosmos", "Hi from Cosmos"]
|
||||
|
||||
sessions = await provider.list_sessions()
|
||||
assert session_id in sessions
|
||||
|
||||
await provider.clear(session_id)
|
||||
assert await provider.get_messages(session_id) == []
|
||||
finally:
|
||||
with suppress(CosmosResourceNotFoundError):
|
||||
await cosmos_client.delete_database(database_name)
|
||||
Reference in New Issue
Block a user