chore: import upstream snapshot with attribution
CodeQL / Analyze (csharp) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:25 +08:00
commit db620d33df
5151 changed files with 925932 additions and 0 deletions
@@ -0,0 +1,13 @@
# Azure OpenAI Configuration
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_MODEL=your-deployment-name
# Optional: Use Azure CLI authentication if not provided
# AZURE_OPENAI_API_KEY=your-api-key
# Durable Task Scheduler Configuration
ENDPOINT=http://localhost:8080
TASKHUB=default
# Redis Configuration (for streaming tests)
REDIS_CONNECTION_STRING=redis://localhost:6379
REDIS_STREAM_TTL_MINUTES=10
@@ -0,0 +1,110 @@
# Sample Integration Tests
Integration tests that validate the Durable Agent Framework samples by running them against a Durable Task Scheduler (DTS) instance.
## Setup
### 1. Create `.env` file
Copy `.env.example` to `.env` and fill in your Azure credentials:
```bash
cp .env.example .env
```
Required variables:
- `AZURE_OPENAI_ENDPOINT`
- `AZURE_OPENAI_MODEL`
- `AZURE_OPENAI_API_KEY` (optional if using Azure CLI authentication)
- `ENDPOINT` (default: http://localhost:8080)
- `TASKHUB` (default: default)
Optional variables (for streaming tests):
- `REDIS_CONNECTION_STRING` (default: redis://localhost:6379)
- `REDIS_STREAM_TTL_MINUTES` (default: 10)
### 2. Start required services
**Durable Task Scheduler:**
```bash
docker run -d --name dts-emulator -p 8080:8080 -p 8082:8082 mcr.microsoft.com/dts/dts-emulator:latest
```
- Port 8080: gRPC endpoint (used by tests)
- Port 8082: Web dashboard (optional, for monitoring)
**Redis (for streaming tests):**
```bash
docker run -d --name redis -p 6379:6379 redis:latest
```
- Port 6379: Redis server endpoint
## Running Tests
The tests automatically start and stop worker processes for each sample.
### Run all sample tests
```bash
uv run pytest packages/durabletask/tests/integration_tests -v
```
### Run specific sample
```bash
uv run pytest packages/durabletask/tests/integration_tests/test_01_single_agent.py -v
```
### Run with verbose output
```bash
uv run pytest packages/durabletask/tests/integration_tests -sv
```
## How It Works
Each test file uses pytest markers to automatically configure and start the worker process:
```python
pytestmark = [
pytest.mark.sample("03_single_agent_streaming"),
pytest.mark.integration_test,
pytest.mark.requires_azure_openai,
pytest.mark.requires_dts,
pytest.mark.requires_redis,
]
```
## Troubleshooting
**Tests are skipped:**
Ensure the required environment variables (e.g., `AZURE_OPENAI_ENDPOINT`) are set in your `.env` file.
**DTS connection failed:**
Check that the DTS emulator container is running: `docker ps | grep dts-emulator`
**Redis connection failed:**
Check that Redis is running: `docker ps | grep redis`
**Missing environment variables:**
Ensure your `.env` file contains all required variables from `.env.example`.
**Tests timeout:**
Check that Azure OpenAI credentials are valid and the service is accessible.
If you see "DTS emulator is not available":
- Ensure Docker container is running: `docker ps | grep dts-emulator`
- Check port 8080 is not in use by another process
- Restart the container if needed
### Azure OpenAI Errors
If you see authentication or deployment errors:
- Verify your `AZURE_OPENAI_ENDPOINT` is correct
- Confirm `AZURE_OPENAI_MODEL` matches your deployment
- If using API key, check `AZURE_OPENAI_API_KEY` is valid
- If using Azure CLI, ensure you're logged in: `az login`
## CI/CD
For automated testing in CI/CD pipelines:
1. Use Docker Compose to start DTS emulator
2. Set environment variables via CI/CD secrets
3. Run tests with appropriate markers: `pytest -m integration_test`
@@ -0,0 +1,512 @@
# Copyright (c) Microsoft. All rights reserved.
"""Pytest configuration and fixtures for durabletask integration tests."""
import asyncio
import json
import logging
import os
import socket
import subprocess
import sys
import time
import uuid
from collections.abc import Generator
from pathlib import Path
from typing import Any, Protocol, cast
from urllib.parse import urlparse
import pytest
import redis.asyncio as aioredis
from dotenv import load_dotenv
from durabletask.azuremanaged.client import DurableTaskSchedulerClient
from durabletask.client import OrchestrationStatus
from agent_framework_durabletask import DurableAIAgentClient, DurableWorkflowClient
# Load environment variables from .env file
load_dotenv(Path(__file__).parent / ".env")
# Configure logging to reduce noise during tests
logging.basicConfig(level=logging.WARNING)
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[DurableTaskSchedulerClient, DurableAIAgentClient]: ...
# =============================================================================
# Environment and Service Checks
# =============================================================================
def _get_dts_endpoint() -> str:
"""Get the DTS endpoint from environment or use default."""
return os.getenv("ENDPOINT", "http://localhost:8080")
def _check_dts_available(endpoint: str | None = None) -> bool:
"""Check if DTS emulator is available at the given endpoint."""
try:
resolved_endpoint: str = _get_dts_endpoint() if endpoint is None else endpoint
parsed = urlparse(resolved_endpoint)
host = parsed.hostname or "localhost"
port = parsed.port or 8080
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.settimeout(2)
return sock.connect_ex((host, port)) == 0
except Exception:
return False
def _check_redis_available() -> bool:
"""Check if Redis is available at the default connection string."""
try:
async def test_connection() -> bool:
redis_url = os.getenv("REDIS_CONNECTION_STRING", "redis://localhost:6379")
try:
client = aioredis.from_url(redis_url, socket_timeout=2) # type: ignore[reportUnknownMemberType]
await client.ping() # type: ignore[reportUnknownMemberType]
await client.aclose() # type: ignore[reportUnknownMemberType]
return True
except Exception:
return False
return asyncio.run(test_connection())
except Exception:
return False
# =============================================================================
# Client Factory Functions
# =============================================================================
def create_dts_client(endpoint: str, taskhub: str) -> DurableTaskSchedulerClient:
"""Create a DurableTaskSchedulerClient with common configuration.
Args:
endpoint: The DTS endpoint address
taskhub: The task hub name
Returns:
A configured DurableTaskSchedulerClient instance
"""
return DurableTaskSchedulerClient(
host_address=endpoint,
secure_channel=False,
taskhub=taskhub,
token_credential=None,
)
def create_agent_client(
endpoint: str,
taskhub: str,
max_poll_retries: int = 90,
) -> tuple[DurableTaskSchedulerClient, DurableAIAgentClient]:
"""Create a DurableAIAgentClient with the underlying DTS client.
Args:
endpoint: The DTS endpoint address
taskhub: The task hub name
max_poll_retries: Max poll retries for the agent client
Returns:
A tuple of (DurableTaskSchedulerClient, DurableAIAgentClient)
"""
dts_client = create_dts_client(endpoint, taskhub)
agent_client = DurableAIAgentClient(dts_client, max_poll_retries=max_poll_retries)
return dts_client, agent_client
# =============================================================================
# Orchestration Helper Class
# =============================================================================
class OrchestrationHelper:
"""Helper class for orchestration-related test operations."""
def __init__(self, dts_client: DurableTaskSchedulerClient):
"""Initialize the orchestration helper.
Args:
dts_client: The DurableTaskSchedulerClient instance to use
"""
self.client = dts_client
def wait_for_orchestration(
self,
instance_id: str,
timeout: float = 60.0,
) -> Any:
"""Wait for an orchestration to complete.
Args:
instance_id: The orchestration instance ID
timeout: Maximum time to wait in seconds
Returns:
The final OrchestrationMetadata
Raises:
TimeoutError: If the orchestration doesn't complete within timeout
RuntimeError: If the orchestration fails
"""
# Use the built-in wait_for_orchestration_completion method
metadata = self.client.wait_for_orchestration_completion(
instance_id=instance_id,
timeout=int(timeout),
)
if metadata is None:
raise TimeoutError(f"Orchestration {instance_id} did not complete within {timeout} seconds")
# Check if failed or terminated
if metadata.runtime_status == OrchestrationStatus.FAILED:
raise RuntimeError(f"Orchestration {instance_id} failed: {metadata.serialized_custom_status}")
if metadata.runtime_status == OrchestrationStatus.TERMINATED:
raise RuntimeError(f"Orchestration {instance_id} was terminated")
return metadata
def wait_for_orchestration_with_output(
self,
instance_id: str,
timeout: float = 60.0,
) -> tuple[Any, Any]:
"""Wait for an orchestration to complete and return its output.
Args:
instance_id: The orchestration instance ID
timeout: Maximum time to wait in seconds
Returns:
A tuple of (OrchestrationMetadata, output)
Raises:
TimeoutError: If the orchestration doesn't complete within timeout
RuntimeError: If the orchestration fails
"""
metadata = self.wait_for_orchestration(instance_id, timeout)
# The output should be available in the metadata
return metadata, metadata.serialized_output
def get_orchestration_status(self, instance_id: str) -> Any | None:
"""Get the current status of an orchestration.
Args:
instance_id: The orchestration instance ID
Returns:
The OrchestrationMetadata or None if not found
"""
try:
# Try to wait with a short timeout to get current status
return self.client.wait_for_orchestration_completion(
instance_id=instance_id,
timeout=1, # Very short timeout, just checking status
)
except Exception:
return None
def raise_event(
self,
instance_id: str,
event_name: str,
event_data: Any = None,
) -> None:
"""Raise an external event to an orchestration.
Args:
instance_id: The orchestration instance ID
event_name: The name of the event
event_data: The event data payload
"""
self.client.raise_orchestration_event(instance_id, event_name, data=event_data)
def wait_for_notification(self, instance_id: str, timeout_seconds: int = 30) -> bool:
"""Wait for the orchestration to reach a notification point.
Polls the orchestration status until it appears to be waiting for approval.
Args:
instance_id: The orchestration instance ID
timeout_seconds: Maximum time to wait
Returns:
True if notification detected, False if timeout
"""
start_time = time.time()
while time.time() - start_time < timeout_seconds:
try:
metadata = self.client.get_orchestration_state(
instance_id=instance_id,
)
if metadata:
# Check if we're waiting for approval by examining custom status
if metadata.serialized_custom_status:
try:
custom_status = json.loads(metadata.serialized_custom_status)
# Handle both string and dict custom status
status_str = custom_status if isinstance(custom_status, str) else str(custom_status)
if status_str.lower().startswith("requesting human feedback"):
return True
except (json.JSONDecodeError, AttributeError):
# If it's not JSON, treat as plain string
if metadata.serialized_custom_status.lower().startswith("requesting human feedback"):
return True
# Check for terminal states
if metadata.runtime_status.name == "COMPLETED" or metadata.runtime_status.name == "FAILED":
return False
except Exception:
# Silently ignore transient errors during polling (e.g., network issues, service unavailable).
# The loop will retry until timeout, allowing the service to recover.
pass
time.sleep(1)
return False
# =============================================================================
# Pytest Configuration
# =============================================================================
def pytest_configure(config: pytest.Config) -> None:
"""Register custom markers."""
config.addinivalue_line("markers", "integration_test: mark test as integration test")
config.addinivalue_line("markers", "requires_dts: mark test as requiring DTS emulator")
config.addinivalue_line("markers", "requires_azure_openai: mark test as requiring Azure OpenAI")
config.addinivalue_line("markers", "requires_redis: mark test as requiring Redis")
config.addinivalue_line(
"markers",
"sample(path): specify the sample directory name for the test (e.g., @pytest.mark.sample('01_single_agent'))",
)
def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item]) -> None:
"""Skip tests based on markers and environment availability."""
foundry_vars = ["FOUNDRY_PROJECT_ENDPOINT", "FOUNDRY_MODEL"]
foundry_available = all(os.getenv(var) for var in foundry_vars)
azure_openai_vars = ["AZURE_OPENAI_ENDPOINT", "AZURE_OPENAI_MODEL"]
azure_openai_available = all(os.getenv(var) for var in azure_openai_vars)
skip_foundry = pytest.mark.skip(reason=f"Missing required environment variables: {', '.join(foundry_vars)}")
skip_azure_openai = pytest.mark.skip(
reason=f"Missing required environment variables: {', '.join(azure_openai_vars)}"
)
# Check DTS availability
dts_available = _check_dts_available()
skip_dts = pytest.mark.skip(reason=f"DTS emulator is not available at {_get_dts_endpoint()}")
# Check Redis availability
redis_available = _check_redis_available()
skip_redis = pytest.mark.skip(reason="Redis is not available at redis://localhost:6379")
for item in items:
if "requires_azure_openai" in item.keywords and not foundry_available:
item.add_marker(skip_foundry)
sample_marker = item.get_closest_marker("sample")
sample_name = sample_marker.args[0] if sample_marker and sample_marker.args else None
if sample_name == "06_multi_agent_orchestration_conditionals" and not azure_openai_available:
item.add_marker(skip_azure_openai)
if "requires_dts" in item.keywords and not dts_available:
item.add_marker(skip_dts)
if "requires_redis" in item.keywords and not redis_available:
item.add_marker(skip_redis)
# =============================================================================
# Pytest Fixtures
# =============================================================================
@pytest.fixture(scope="session")
def dts_endpoint() -> str:
"""Get the DTS endpoint from environment or use default."""
return _get_dts_endpoint()
@pytest.fixture(scope="session")
def dts_available(dts_endpoint: str) -> bool:
"""Check if DTS emulator is available and responding."""
if _check_dts_available(dts_endpoint):
return True
pytest.skip(f"DTS emulator is not available at {dts_endpoint}")
return False
@pytest.fixture(scope="module")
def check_sample_env(request: pytest.FixtureRequest) -> None:
"""Verify the environment variables required by the current sample are set."""
sample_marker = request.node.get_closest_marker("sample") # type: ignore[union-attr]
if not sample_marker:
pytest.fail("Test class must have @pytest.mark.sample() marker")
sample_name = cast(str, sample_marker.args[0]) # type: ignore[union-attr]
# Samples that host no AI agents need no model credentials (only the DTS emulator).
no_llm_samples = {"12_subworkflow_hitl"}
if sample_name in no_llm_samples:
return
if sample_name == "06_multi_agent_orchestration_conditionals":
required_vars = ["AZURE_OPENAI_ENDPOINT", "AZURE_OPENAI_MODEL"]
else:
required_vars = ["FOUNDRY_PROJECT_ENDPOINT", "FOUNDRY_MODEL"]
missing = [var for var in required_vars if not os.getenv(var)]
if missing:
pytest.skip(f"Missing required environment variables: {', '.join(missing)}")
@pytest.fixture(scope="module")
def unique_taskhub() -> str:
"""Generate a unique task hub name for test isolation."""
# Use a shorter UUID to avoid naming issues
return f"test-{uuid.uuid4().hex[:8]}"
@pytest.fixture(scope="module")
def worker_process(
dts_available: bool,
check_sample_env: None,
dts_endpoint: str,
unique_taskhub: str,
request: pytest.FixtureRequest,
) -> Generator[dict[str, Any], None, None]:
"""Start a worker process for the current test module by running the sample worker.py.
This fixture:
1. Determines which sample to run from @pytest.mark.sample()
2. Starts the sample's worker.py as a subprocess
3. Waits for the worker to be ready
4. Tears down the worker after tests complete
Usage:
@pytest.mark.sample("01_single_agent")
class TestSingleAgent:
...
"""
# Get sample path from marker
sample_marker = request.node.get_closest_marker("sample") # type: ignore[union-attr]
if not sample_marker:
pytest.fail("Test class must have @pytest.mark.sample() marker")
sample_name: str = cast(str, sample_marker.args[0]) # type: ignore[union-attr]
sample_path: Path = Path(__file__).parents[4] / "samples" / "04-hosting" / "durabletask" / sample_name
worker_file: Path = sample_path / "worker.py"
if not worker_file.exists():
pytest.fail(f"Sample worker not found: {worker_file}")
# Set up environment for worker subprocess
env = os.environ.copy()
env["ENDPOINT"] = dts_endpoint
env["TASKHUB"] = unique_taskhub
# Start worker subprocess
try:
# On Windows, use CREATE_NEW_PROCESS_GROUP to allow proper termination
# shell=True only on Windows to handle PATH resolution
if sys.platform == "win32":
process = subprocess.Popen(
[sys.executable, str(worker_file)],
cwd=str(sample_path),
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
shell=True,
env=env,
text=True,
)
# On Unix, don't use shell=True to avoid shell wrapper issues
else:
process = subprocess.Popen(
[sys.executable, str(worker_file)],
cwd=str(sample_path),
env=env,
text=True,
)
except Exception as e:
pytest.fail(f"Failed to start worker subprocess: {e}")
# Wait for worker to initialize
# The worker needs time to:
# 1. Start Python and import modules
# 2. Create Azure OpenAI clients
# 3. Register agents with the DTS worker
# 4. Connect to DTS and be ready to receive signals
#
# We use a generous wait time because CI environments can be slow,
# and the first test that runs depends on the worker being fully ready.
time.sleep(8)
# Check if process is still running
if process.poll() is not None:
stderr_output = process.stderr.read() if process.stderr else ""
pytest.fail(f"Worker process exited prematurely. stderr: {stderr_output}")
# Provide worker info to tests
worker_info = {
"process": process,
"endpoint": dts_endpoint,
"taskhub": unique_taskhub,
}
try:
yield worker_info
finally:
# Cleanup: terminate worker subprocess
try:
process.terminate()
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
except Exception as e:
logging.warning(f"Error during worker process cleanup: {e}")
@pytest.fixture(scope="module")
def orchestration_helper(worker_process: dict[str, Any]) -> OrchestrationHelper:
"""Create an OrchestrationHelper for the current test module."""
dts_client = create_dts_client(worker_process["endpoint"], worker_process["taskhub"])
return OrchestrationHelper(dts_client)
@pytest.fixture(scope="module")
def agent_client_factory(worker_process: dict[str, Any]) -> type[AgentClientFactoryProtocol]:
"""Return a factory class for creating agent clients.
Usage in tests:
def test_example(self, agent_client_factory):
dts_client, agent_client = agent_client_factory.create(max_poll_retries=90)
"""
class AgentClientFactory:
"""Factory for creating DTS and Agent client pairs."""
endpoint = worker_process["endpoint"]
taskhub = worker_process["taskhub"]
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[DurableTaskSchedulerClient, DurableAIAgentClient]:
"""Create a DTS client and Agent client pair."""
return create_agent_client(cls.endpoint, cls.taskhub, max_poll_retries)
return AgentClientFactory
@pytest.fixture(scope="module")
def workflow_client(worker_process: dict[str, Any]) -> DurableWorkflowClient:
"""Create a DurableWorkflowClient bound to the current sample worker's task hub."""
dts_client = create_dts_client(worker_process["endpoint"], worker_process["taskhub"])
return DurableWorkflowClient(dts_client)
@@ -0,0 +1,97 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for single agent functionality.
Tests basic agent operations including:
- Agent registration and retrieval
- Single agent interactions
- Conversation continuity across multiple messages
- Multi-threaded agent usage
- Empty thread ID handling
"""
from typing import Any, Protocol
import pytest
from agent_framework_durabletask import DurableAIAgentClient
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ...
# Module-level markers - applied to all tests in this module
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("01_single_agent"),
pytest.mark.integration_test,
pytest.mark.requires_azure_openai,
pytest.mark.requires_dts,
]
class TestSingleAgent:
"""Test suite for single agent functionality."""
@pytest.fixture(autouse=True)
def setup(self, agent_client_factory: type[AgentClientFactoryProtocol]) -> None:
"""Setup test fixtures."""
# Create agent client using the factory fixture
_, self.agent_client = agent_client_factory.create()
def test_agent_registration(self) -> None:
"""Test that the Joker agent is registered and accessible."""
agent = self.agent_client.get_agent("Joker")
assert agent is not None
assert agent.name == "Joker"
def test_single_interaction(self):
"""Test a single interaction with the agent."""
agent = self.agent_client.get_agent("Joker")
session = agent.create_session()
response = agent.run("Tell me a short joke about programming.", session=session)
assert response is not None
assert response.text is not None
assert len(response.text) > 0
def test_conversation_continuity(self):
"""Test that conversation context is maintained across turns."""
agent = self.agent_client.get_agent("Joker")
session = agent.create_session()
# First turn: Ask for a joke about a specific topic
response1 = agent.run("Tell me a joke about cats.", session=session)
assert response1 is not None
assert len(response1.text) > 0
# Second turn: Ask a follow-up that requires context
response2 = agent.run("Can you make it funnier?", session=session)
assert response2 is not None
assert len(response2.text) > 0
# The agent should understand "it" refers to the previous joke
def test_multiple_sessions(self):
"""Test that different sessions maintain separate contexts."""
agent = self.agent_client.get_agent("Joker")
# Create two separate sessions
session1 = agent.create_session()
session2 = agent.create_session()
assert session1.durable_session_id != session2.durable_session_id
# Send different messages to each session
response1 = agent.run("Tell me a joke about dogs.", session=session1)
response2 = agent.run("Tell me a joke about birds.", session=session2)
assert response1 is not None
assert response2 is not None
assert response1.text != response2.text
@@ -0,0 +1,113 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for multi-agent functionality.
Tests operations with multiple specialized agents:
- Multiple agent registration
- Agent-specific tool usage
- Independent thread management per agent
- Concurrent agent operations
- Agent isolation and tool routing
"""
from typing import Any, Protocol
import pytest
from agent_framework_durabletask import DurableAIAgentClient
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ...
# Agent names from the 02_multi_agent sample
WEATHER_AGENT_NAME: str = "WeatherAgent"
MATH_AGENT_NAME: str = "MathAgent"
# Module-level markers - applied to all tests in this module
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("02_multi_agent"),
pytest.mark.integration_test,
pytest.mark.requires_azure_openai,
pytest.mark.requires_dts,
]
class TestMultiAgent:
"""Test suite for multi-agent functionality."""
@pytest.fixture(autouse=True)
def setup(self, agent_client_factory: type[AgentClientFactoryProtocol]) -> None:
"""Setup test fixtures."""
# Create agent client using the factory fixture
_, self.agent_client = agent_client_factory.create()
def test_multiple_agents_registered(self) -> None:
"""Test that both agents are registered and accessible."""
weather_agent = self.agent_client.get_agent(WEATHER_AGENT_NAME)
math_agent = self.agent_client.get_agent(MATH_AGENT_NAME)
assert weather_agent is not None
assert weather_agent.name == WEATHER_AGENT_NAME
assert math_agent is not None
assert math_agent.name == MATH_AGENT_NAME
@pytest.mark.skip(reason="Flaky in CI: times out / crashes the xdist runner; temporarily disabled.")
def test_weather_agent_with_tool(self):
"""Test weather agent with weather tool execution."""
agent = self.agent_client.get_agent(WEATHER_AGENT_NAME)
session = agent.create_session()
response = agent.run("What's the weather in Seattle?", session=session)
assert response is not None
assert response.text is not None
# Should contain weather information from the tool
assert len(response.text) > 0
# Verify that the get_weather tool was actually invoked
tool_calls = [
content for msg in response.messages for content in msg.contents if content.type == "function_call"
]
assert len(tool_calls) > 0, "Expected at least one tool call"
assert any(call.name == "get_weather" for call in tool_calls), "Expected get_weather tool to be called"
@pytest.mark.skip(reason="Flaky in CI: times out / crashes the xdist runner; temporarily disabled.")
def test_math_agent_with_tool(self):
"""Test math agent with calculation tool execution."""
agent = self.agent_client.get_agent(MATH_AGENT_NAME)
session = agent.create_session()
response = agent.run("Calculate a 20% tip on a $50 bill.", session=session)
assert response is not None
assert response.text is not None
# Should contain calculation results from the tool
assert len(response.text) > 0
# Verify that the calculate_tip tool was actually invoked
tool_calls = [
content for msg in response.messages for content in msg.contents if content.type == "function_call"
]
assert len(tool_calls) > 0, "Expected at least one tool call"
assert any(call.name == "calculate_tip" for call in tool_calls), "Expected calculate_tip tool to be called"
def test_multiple_calls_to_same_agent(self):
"""Test multiple sequential calls to the same agent."""
agent = self.agent_client.get_agent(WEATHER_AGENT_NAME)
session = agent.create_session()
# Multiple weather queries
response1 = agent.run("What's the weather in Chicago?", session=session)
response2 = agent.run("And what about Los Angeles?", session=session)
assert response1 is not None
assert response2 is not None
assert len(response1.text) > 0
assert len(response2.text) > 0
@@ -0,0 +1,236 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Reliable Streaming Sample
Tests the reliable streaming sample using Redis Streams for persistent message delivery.
The worker process is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/durabletask/tests/integration_tests/.env.example)
- DTS emulator running (docker run -d -p 8080:8080 mcr.microsoft.com/durabletask/emulator:latest)
- Redis running (docker run -d --name redis -p 6379:6379 redis:latest)
Usage:
uv run pytest packages/durabletask/tests/integration_tests/test_03_single_agent_streaming.py -v
"""
import asyncio
import os
import sys
import time
from datetime import timedelta
from pathlib import Path
from typing import Any, Protocol
import pytest
import redis.asyncio as aioredis
from agent_framework_durabletask import DurableAIAgentClient
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ...
# Add sample directory to path to import RedisStreamResponseHandler
SAMPLE_DIR = Path(__file__).parents[4] / "samples" / "04-hosting" / "durabletask" / "03_single_agent_streaming"
sys.path.insert(0, str(SAMPLE_DIR))
from redis_stream_response_handler import ( # type: ignore[reportMissingImports] # pyrefly: ignore[missing-import] # ty: ignore[unresolved-import] # noqa: E402
RedisStreamResponseHandler,
)
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("03_single_agent_streaming"),
pytest.mark.integration_test,
pytest.mark.requires_azure_openai,
pytest.mark.requires_dts,
pytest.mark.requires_redis,
]
class TestSampleReliableStreaming:
"""Tests for 03_single_agent_streaming sample."""
@pytest.fixture(autouse=True)
def setup(self, agent_client_factory: type[AgentClientFactoryProtocol], orchestration_helper) -> None:
"""Setup test fixtures."""
# Create agent client using the factory fixture
_, self.agent_client = agent_client_factory.create()
self.helper = orchestration_helper
# Redis configuration
self.redis_connection_string = os.environ.get("REDIS_CONNECTION_STRING", "redis://localhost:6379")
self.redis_stream_ttl_minutes = int(os.environ.get("REDIS_STREAM_TTL_MINUTES", "10"))
async def _get_stream_handler(self) -> RedisStreamResponseHandler: # type: ignore[reportMissingTypeStubs]
"""Create a new Redis stream handler for each request."""
redis_client = aioredis.from_url( # type: ignore[reportUnknownMemberType]
self.redis_connection_string,
encoding="utf-8",
decode_responses=False,
)
return RedisStreamResponseHandler( # type: ignore[reportUnknownMemberType]
redis_client=redis_client,
stream_ttl=timedelta(minutes=self.redis_stream_ttl_minutes),
)
async def _stream_from_redis(
self,
session_key: str,
cursor: str | None = None,
timeout: float = 30.0,
) -> tuple[str, bool, str]:
"""
Stream responses from Redis using the sample's RedisStreamResponseHandler.
Args:
session_key: The conversation/thread ID to stream from
cursor: Optional cursor to resume from
timeout: Maximum time to wait for stream completion
Returns:
Tuple of (accumulated text, completion status, last entry_id)
"""
accumulated_text = ""
is_complete = False
last_entry_id = cursor if cursor else "0-0"
start_time = time.time()
async with await self._get_stream_handler() as stream_handler: # type: ignore[reportUnknownMemberType]
try:
async for chunk in stream_handler.read_stream(session_key, cursor): # type: ignore[reportUnknownMemberType]
if time.time() - start_time > timeout:
break
last_entry_id = chunk.entry_id # type: ignore[reportUnknownMemberType]
if chunk.error: # type: ignore[reportUnknownMemberType]
# Stream not found or timeout - this is expected if agent hasn't written yet
# Don't raise an error, just return what we have
break
if chunk.is_done: # type: ignore[reportUnknownMemberType]
is_complete = True
break
if chunk.text: # type: ignore[reportUnknownMemberType]
accumulated_text += chunk.text # type: ignore[reportUnknownMemberType]
except Exception as ex:
# For test purposes, we catch exceptions and return what we have
if "timed out" not in str(ex).lower():
raise
return accumulated_text, is_complete, last_entry_id # type: ignore[reportReturnType]
def test_agent_run_and_stream(self) -> None:
"""Test agent execution with Redis streaming."""
# Get the TravelPlanner agent
travel_planner = self.agent_client.get_agent("TravelPlanner")
assert travel_planner is not None
assert travel_planner.name == "TravelPlanner"
# Create a new session
session = travel_planner.create_session()
assert session.durable_session_id is not None
assert session.durable_session_id.key is not None
session_key = str(session.durable_session_id.key)
# Start agent run with wait_for_response=False for non-blocking execution
travel_planner.run(
"Plan a 1-day trip to Seattle in 1 sentence", session=session, options={"wait_for_response": False}
)
# Poll Redis stream with retries to handle race conditions
# The agent may take a few seconds to process and start writing to Redis
# We use cursor-based resumption to continue reading from where we left off
max_retries = 20
retry_count = 0
accumulated_text = ""
is_complete = False
cursor: str | None = None
while retry_count < max_retries and not is_complete:
text, is_complete, last_cursor = asyncio.run(
self._stream_from_redis(session_key, cursor=cursor, timeout=10.0)
)
accumulated_text += text
cursor = last_cursor # Resume from last position on next read
if is_complete:
# Stream completed successfully
break
if len(accumulated_text) > 0:
# Got content but not completion marker yet - keep reading without delay
# The agent may still be streaming or about to write completion marker
continue
# No content yet - wait before retrying
time.sleep(2)
retry_count += 1
# Verify we got content
assert len(accumulated_text) > 0, (
f"Expected text content but got empty string for session_key: {session_key} after {retry_count} retries"
)
assert "seattle" in accumulated_text.lower(), f"Expected 'seattle' in response but got: {accumulated_text}"
assert is_complete, "Expected stream to be complete"
def test_stream_with_cursor_resumption(self) -> None:
"""Test streaming with cursor-based resumption."""
# Get the TravelPlanner agent
travel_planner = self.agent_client.get_agent("TravelPlanner")
session = travel_planner.create_session()
assert session.durable_session_id is not None
assert session.durable_session_id.key is not None
session_key = str(session.durable_session_id.key)
# Start agent run
travel_planner.run("What's the weather like?", session=session, options={"wait_for_response": False})
# Wait for agent to start writing
time.sleep(3)
# Read partial stream to get a cursor
async def get_partial_stream() -> tuple[str, str]:
async with await self._get_stream_handler() as stream_handler: # type: ignore[reportUnknownMemberType]
accumulated_text = ""
last_entry_id = "0-0"
chunk_count = 0
# Read just first 2 chunks
async for chunk in stream_handler.read_stream(session_key): # type: ignore[reportUnknownMemberType]
last_entry_id = chunk.entry_id # type: ignore[reportUnknownMemberType]
if chunk.text: # type: ignore[reportUnknownMemberType]
accumulated_text += chunk.text # type: ignore[reportUnknownMemberType]
chunk_count += 1
if chunk_count >= 2:
break
return accumulated_text, last_entry_id # type: ignore[reportReturnType]
partial_text, cursor = asyncio.run(get_partial_stream())
# Resume from cursor
remaining_text, _, _ = asyncio.run(self._stream_from_redis(session_key, cursor=cursor))
# Verify we got some initial content
assert len(partial_text) > 0
# Combined text should be coherent
full_text = partial_text + remaining_text
assert len(full_text) > 0
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,111 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for single agent orchestration with chaining.
Tests orchestration patterns with sequential agent calls:
- Orchestration registration and execution
- Sequential agent calls on same thread
- Conversation continuity in orchestrations
- Thread context preservation
"""
import json
import logging
from typing import Any, Protocol
import pytest
from durabletask.client import OrchestrationStatus
from agent_framework_durabletask import DurableAIAgentClient
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ...
# Agent name from the 04_single_agent_orchestration_chaining sample
WRITER_AGENT_NAME: str = "WriterAgent"
# Configure logging
logging.basicConfig(level=logging.WARNING)
# Module-level markers - applied to all tests in this module
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("04_single_agent_orchestration_chaining"),
pytest.mark.integration_test,
pytest.mark.requires_azure_openai,
pytest.mark.requires_dts,
]
class TestSingleAgentOrchestrationChaining:
"""Test suite for single agent orchestration with chaining."""
@pytest.fixture(autouse=True)
def setup(self, agent_client_factory: type[AgentClientFactoryProtocol], orchestration_helper) -> None:
"""Setup test fixtures."""
# Create agent client using the factory fixture
self.dts_client, self.agent_client = agent_client_factory.create()
self.orch_helper = orchestration_helper
def test_agent_registered(self):
"""Test that the Writer agent is registered."""
agent = self.agent_client.get_agent(WRITER_AGENT_NAME)
assert agent is not None
assert agent.name == WRITER_AGENT_NAME
def test_chaining_context_preserved(self):
"""Test that context is preserved across agent runs in orchestration."""
# Start the orchestration
instance_id = self.dts_client.schedule_new_orchestration(
orchestrator="single_agent_chaining_orchestration",
input="",
)
# Wait for completion with output
metadata, output = self.orch_helper.wait_for_orchestration_with_output(
instance_id=instance_id,
timeout=120.0,
)
assert metadata is not None
assert output is not None
# The final output should be a refined sentence
final_text = json.loads(output)
# Should be a meaningful sentence (not empty or error message)
assert len(final_text) > 10
assert not final_text.startswith("Error")
def test_multiple_orchestration_instances(self):
"""Test that multiple orchestration instances can run independently."""
# Start two orchestrations
instance_id_1 = self.dts_client.schedule_new_orchestration(
orchestrator="single_agent_chaining_orchestration",
input="",
)
instance_id_2 = self.dts_client.schedule_new_orchestration(
orchestrator="single_agent_chaining_orchestration",
input="",
)
assert instance_id_1 != instance_id_2
# Both should complete
metadata_1 = self.orch_helper.wait_for_orchestration(
instance_id=instance_id_1,
timeout=120.0,
)
metadata_2 = self.orch_helper.wait_for_orchestration(
instance_id=instance_id_2,
timeout=120.0,
)
assert metadata_1.runtime_status == OrchestrationStatus.COMPLETED
assert metadata_2.runtime_status == OrchestrationStatus.COMPLETED
@@ -0,0 +1,87 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for multi-agent orchestration with concurrency.
Tests concurrent execution patterns:
- Parallel agent execution
- Concurrent orchestration tasks
- Independent thread management in parallel
- Result aggregation from concurrent calls
"""
import json
import logging
from typing import Any, Protocol
import pytest
from durabletask.client import OrchestrationStatus
from agent_framework_durabletask import DurableAIAgentClient
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ...
# Agent names from the 05_multi_agent_orchestration_concurrency sample
PHYSICIST_AGENT_NAME: str = "PhysicistAgent"
CHEMIST_AGENT_NAME: str = "ChemistAgent"
# Configure logging
logging.basicConfig(level=logging.WARNING)
# Module-level markers
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("05_multi_agent_orchestration_concurrency"),
pytest.mark.integration_test,
pytest.mark.requires_dts,
]
class TestMultiAgentOrchestrationConcurrency:
"""Test suite for multi-agent orchestration with concurrency."""
@pytest.fixture(autouse=True)
def setup(self, agent_client_factory: type[AgentClientFactoryProtocol], orchestration_helper) -> None:
"""Setup test fixtures."""
# Create agent client using the factory fixture
self.dts_client, self.agent_client = agent_client_factory.create()
self.orch_helper = orchestration_helper
def test_agents_registered(self):
"""Test that both agents are registered."""
physicist = self.agent_client.get_agent(PHYSICIST_AGENT_NAME)
chemist = self.agent_client.get_agent(CHEMIST_AGENT_NAME)
assert physicist is not None
assert physicist.name == PHYSICIST_AGENT_NAME
assert chemist is not None
assert chemist.name == CHEMIST_AGENT_NAME
def test_different_prompts(self):
"""Test concurrent orchestration with different prompts."""
prompts = [
"What is temperature?",
"Explain molecules.",
]
for prompt in prompts:
instance_id = self.dts_client.schedule_new_orchestration(
orchestrator="multi_agent_concurrent_orchestration",
input=prompt,
)
metadata, output = self.orch_helper.wait_for_orchestration_with_output(
instance_id=instance_id,
timeout=120.0,
)
assert metadata.runtime_status == OrchestrationStatus.COMPLETED
result = json.loads(output)
assert "physicist" in result
assert "chemist" in result
@@ -0,0 +1,86 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for multi-agent orchestration with conditionals.
Tests conditional orchestration patterns:
- Conditional branching in orchestrations
- Agent-based decision making
- Activity function execution
- Structured output handling
- Conditional routing based on agent responses
"""
import logging
from typing import Any, Protocol
import pytest
from durabletask.client import OrchestrationStatus
from agent_framework_durabletask import DurableAIAgentClient
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ...
# Agent names from the 06_multi_agent_orchestration_conditionals sample
SPAM_AGENT_NAME: str = "SpamDetectionAgent"
EMAIL_AGENT_NAME: str = "EmailAssistantAgent"
# Configure logging
logging.basicConfig(level=logging.WARNING)
# Module-level markers
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("06_multi_agent_orchestration_conditionals"),
pytest.mark.integration_test,
pytest.mark.requires_dts,
]
class TestMultiAgentOrchestrationConditionals:
"""Test suite for multi-agent orchestration with conditionals."""
@pytest.fixture(autouse=True)
def setup(self, agent_client_factory: type[AgentClientFactoryProtocol], orchestration_helper) -> None:
"""Setup test fixtures."""
# Create agent client using the factory fixture
self.dts_client, self.agent_client = agent_client_factory.create()
self.orch_helper = orchestration_helper
def test_agents_registered(self):
"""Test that both agents are registered."""
spam_agent = self.agent_client.get_agent(SPAM_AGENT_NAME)
email_agent = self.agent_client.get_agent(EMAIL_AGENT_NAME)
assert spam_agent is not None
assert spam_agent.name == SPAM_AGENT_NAME
assert email_agent is not None
assert email_agent.name == EMAIL_AGENT_NAME
@pytest.mark.skip(reason="Flaky in CI: times out / crashes the xdist runner; temporarily disabled.")
def test_conditional_branching(self):
"""Test that conditional branching works correctly."""
# Test with obvious spam
spam_payload = {
"email_id": "spam-001",
"email_content": "Buy cheap medications online! No prescription needed! Limited time offer!",
}
spam_instance_id = self.dts_client.schedule_new_orchestration(
orchestrator="spam_detection_orchestration",
input=spam_payload,
)
# Both should complete successfully (different branches)
spam_metadata = self.orch_helper.wait_for_orchestration(
instance_id=spam_instance_id,
timeout=120.0,
)
assert spam_metadata.runtime_status == OrchestrationStatus.COMPLETED
@@ -0,0 +1,174 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for single agent orchestration with human-in-the-loop.
Tests human-in-the-loop (HITL) patterns:
- External event waiting and handling
- Timeout handling in orchestrations
- Iterative refinement with human feedback
- Activity function integration
- Approval workflow patterns
"""
import logging
from typing import Any, Protocol
import pytest
from durabletask.client import OrchestrationStatus
from agent_framework_durabletask import DurableAIAgentClient
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ...
# Constants from the 07_single_agent_orchestration_hitl sample
WRITER_AGENT_NAME: str = "WriterAgent"
HUMAN_APPROVAL_EVENT: str = "HumanApproval"
# Configure logging
logging.basicConfig(level=logging.WARNING)
# Module-level markers
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("07_single_agent_orchestration_hitl"),
pytest.mark.integration_test,
pytest.mark.requires_dts,
]
class TestSingleAgentOrchestrationHITL:
"""Test suite for single agent orchestration with human-in-the-loop."""
@pytest.fixture(autouse=True)
def setup(self, agent_client_factory: type[AgentClientFactoryProtocol], orchestration_helper) -> None:
"""Setup test fixtures."""
# Create agent client using the factory fixture
self.dts_client, self.agent_client = agent_client_factory.create()
self.orch_helper = orchestration_helper
def test_agent_registered(self):
"""Test that the Writer agent is registered."""
agent = self.agent_client.get_agent(WRITER_AGENT_NAME)
assert agent is not None
assert agent.name == WRITER_AGENT_NAME
def test_hitl_orchestration_with_approval(self):
"""Test HITL orchestration with immediate approval."""
payload = {
"topic": "The benefits of continuous learning",
"max_review_attempts": 3,
"approval_timeout_seconds": 60,
}
# Start the orchestration
instance_id = self.dts_client.schedule_new_orchestration(
orchestrator="content_generation_hitl_orchestration",
input=payload,
)
assert instance_id is not None
# Wait for orchestration to reach notification point
notification_received = self.orch_helper.wait_for_notification(instance_id, timeout_seconds=90)
assert notification_received, "Failed to receive notification from orchestration"
# Send approval event
approval_data = {"approved": True, "feedback": ""}
self.orch_helper.raise_event(
instance_id=instance_id,
event_name=HUMAN_APPROVAL_EVENT,
event_data=approval_data,
)
# Wait for completion
metadata = self.orch_helper.wait_for_orchestration(
instance_id=instance_id,
timeout=90.0,
)
assert metadata is not None
assert metadata.runtime_status == OrchestrationStatus.COMPLETED
def test_hitl_orchestration_with_rejection_and_feedback(self):
"""Test HITL orchestration with rejection and iterative refinement."""
payload = {
"topic": "Artificial Intelligence in healthcare",
"max_review_attempts": 3,
"approval_timeout_seconds": 60,
}
# Start the orchestration
instance_id = self.dts_client.schedule_new_orchestration(
orchestrator="content_generation_hitl_orchestration",
input=payload,
)
# Wait for orchestration to reach notification point
notification_received = self.orch_helper.wait_for_notification(instance_id, timeout_seconds=90)
assert notification_received, "Failed to receive notification from orchestration"
# First rejection with feedback
rejection_data = {
"approved": False,
"feedback": "Please make it more concise and add specific examples.",
}
self.orch_helper.raise_event(
instance_id=instance_id,
event_name=HUMAN_APPROVAL_EVENT,
event_data=rejection_data,
)
# Wait for orchestration to refine and reach notification point again
notification_received = self.orch_helper.wait_for_notification(instance_id, timeout_seconds=90)
assert notification_received, "Failed to receive notification after refinement"
# Second approval
approval_data = {"approved": True, "feedback": ""}
self.orch_helper.raise_event(
instance_id=instance_id,
event_name=HUMAN_APPROVAL_EVENT,
event_data=approval_data,
)
# Wait for completion
metadata = self.orch_helper.wait_for_orchestration(
instance_id=instance_id,
timeout=90.0,
)
assert metadata is not None
assert metadata.runtime_status == OrchestrationStatus.COMPLETED
def test_hitl_orchestration_timeout(self):
"""Test HITL orchestration timeout behavior."""
payload = {
"topic": "Cloud computing fundamentals",
"max_review_attempts": 1,
"approval_timeout_seconds": 0.1, # Short timeout for testing
}
# Start the orchestration
instance_id = self.dts_client.schedule_new_orchestration(
orchestrator="content_generation_hitl_orchestration",
input=payload,
)
# Don't send any approval - let it timeout
# The orchestration should fail due to timeout
try:
metadata = self.orch_helper.wait_for_orchestration(
instance_id=instance_id,
timeout=90.0,
)
# If it completes, it should be failed status due to timeout
assert metadata.runtime_status == OrchestrationStatus.FAILED
except (RuntimeError, TimeoutError):
# Expected - orchestration should timeout and fail
pass
@@ -0,0 +1,90 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for the standalone durabletask workflow sample (08_workflow).
Exercises the standalone (non-Azure-Functions) workflow path:
- ``DurableAIAgentWorker.configure_workflow`` auto-registers the agent entities,
non-agent executor activities, and the workflow orchestrator.
- A client starts the workflow by scheduling its ``dafx-{workflow_name}`` orchestration.
- Conditional routing sends spam to a non-agent handler and legitimate email
through a second agent and a sender executor.
"""
import logging
from typing import Any, Protocol
import pytest
from durabletask.client import OrchestrationStatus
from agent_framework_durabletask import DurableAIAgentClient, workflow_orchestrator_name
# Must match the workflow name in samples/04-hosting/durabletask/08_workflow/worker.py
WORKFLOW_NAME = "email_triage"
logging.basicConfig(level=logging.WARNING)
class AgentClientFactoryProtocol(Protocol):
"""Protocol for the agent client factory fixture."""
@classmethod
def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ...
# Module-level markers
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("08_workflow"),
pytest.mark.integration_test,
pytest.mark.requires_dts,
]
class TestStandaloneWorkflow:
"""Standalone (non-Azure-Functions) workflow execution on a durabletask worker."""
@pytest.fixture(autouse=True)
def setup(self, agent_client_factory: type[AgentClientFactoryProtocol], orchestration_helper) -> None:
"""Provide a DTS client and orchestration helper for each test."""
self.dts_client, self.agent_client = agent_client_factory.create()
self.orch_helper = orchestration_helper
def test_legitimate_email_drafts_response(self) -> None:
"""A legitimate email routes through the email agent and is 'sent'."""
instance_id = self.dts_client.schedule_new_orchestration(
orchestrator=workflow_orchestrator_name(WORKFLOW_NAME),
input=(
"Hi team, just a reminder about our sprint planning meeting tomorrow at 10 AM. "
"Please review the agenda in Jira."
),
)
metadata, output = self.orch_helper.wait_for_orchestration_with_output(
instance_id=instance_id,
timeout=180.0,
)
assert metadata.runtime_status == OrchestrationStatus.COMPLETED
assert output is not None
assert "Email sent" in str(output)
def test_spam_email_handled(self) -> None:
"""A spam email routes to the non-agent spam handler."""
instance_id = self.dts_client.schedule_new_orchestration(
orchestrator=workflow_orchestrator_name(WORKFLOW_NAME),
input="URGENT! You've won $1,000,000! Click here now to claim your prize! Limited time offer!",
)
metadata, output = self.orch_helper.wait_for_orchestration_with_output(
instance_id=instance_id,
timeout=180.0,
)
assert metadata.runtime_status == OrchestrationStatus.COMPLETED
assert output is not None
assert "spam" in str(output).lower()
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,112 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for the standalone durabletask HITL workflow sample (09_workflow_hitl).
Exercises the human-in-the-loop workflow path on a standalone durabletask worker:
- The ``InputRouter`` start executor receives a typed ``ContentSubmission`` that the
shared engine reconstructs from the client's JSON payload (no manual parsing).
- An analysis agent produces a recommendation, then the workflow pauses for human
approval via ``request_info``.
- The client retrieves the pending request, replies with ``send_hitl_response``, and
the workflow resumes to an approved/rejected outcome read via ``await_workflow_output``.
"""
import logging
import time
from typing import Any
import pytest
from agent_framework_durabletask import DurableWorkflowClient
logging.basicConfig(level=logging.WARNING)
# Must match the workflow name in samples/04-hosting/durabletask/09_workflow_hitl/worker.py
WORKFLOW_NAME = "content_moderation"
# Module-level markers
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("09_workflow_hitl"),
pytest.mark.integration_test,
pytest.mark.requires_dts,
pytest.mark.requires_azure_openai,
]
def _wait_for_hitl_request(
client: DurableWorkflowClient, instance_id: str, timeout_seconds: int = 90
) -> list[dict[str, Any]]:
"""Poll until the workflow records at least one pending HITL request."""
deadline = time.time() + timeout_seconds
while time.time() < deadline:
pending = client.get_pending_hitl_requests(instance_id, workflow_name=WORKFLOW_NAME)
if pending:
return pending
time.sleep(2)
raise AssertionError(f"Timed out waiting for a HITL request on instance {instance_id}")
class TestStandaloneWorkflowHITL:
"""Human-in-the-loop workflow execution on a standalone durabletask worker."""
@pytest.fixture(autouse=True)
def setup(self, workflow_client: DurableWorkflowClient) -> None:
"""Bind the DurableWorkflowClient for the current sample worker."""
self.client = workflow_client
def _run_case(self, submission: dict[str, Any], *, approve: bool) -> Any:
"""Start a moderation case, answer the HITL pause, and return the final output."""
instance_id = self.client.start_workflow(input=submission, workflow_name=WORKFLOW_NAME)
pending = _wait_for_hitl_request(self.client, instance_id)
request = pending[0]
assert request["request_id"]
assert request["source_executor_id"]
self.client.send_hitl_response(
instance_id,
request["request_id"],
{"approved": approve, "reviewer_notes": "Looks good." if approve else "Violates content policy."},
workflow_name=WORKFLOW_NAME,
)
return self.client.await_workflow_output(instance_id, workflow_name=WORKFLOW_NAME, timeout_seconds=180)
def test_hitl_workflow_approval(self) -> None:
"""Appropriate content is approved after the reviewer says yes."""
output = self._run_case(
{
"content_id": "article-001",
"title": "Introduction to AI in Healthcare",
"body": (
"Artificial intelligence is improving healthcare by enabling faster diagnosis, "
"personalized treatment plans, and better patient outcomes."
),
"author": "Dr. Jane Smith",
},
approve=True,
)
assert output is not None
assert "APPROVED" in str(output).upper()
def test_hitl_workflow_rejection(self) -> None:
"""Spammy content is rejected after the reviewer says no."""
output = self._run_case(
{
"content_id": "article-002",
"title": "Get Rich Quick",
"body": "Click here NOW to make $10,000 overnight! GUARANTEED! Limited time offer!",
"author": "Definitely Not Spam",
},
approve=False,
)
assert output is not None
assert "REJECTED" in str(output).upper()
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,72 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for the composed sub-workflow sample (11_subworkflow).
Exercises workflow *composition* on a standalone durabletask worker:
- An outer ``review_pipeline`` embeds an inner ``sentiment_analysis`` workflow via a
``WorkflowExecutor`` node (``sentiment_sub``).
- ``DurableAIAgentWorker.configure_workflow`` walks the composition and registers a
durable orchestration for each workflow; the inner workflow runs as a child
orchestration when the outer reaches the ``WorkflowExecutor`` node.
- The inner workflow's output (a sentiment summary) is forwarded to the outer
``reporter`` executor, which produces the final result.
The inner workflow hosts an AI agent, so these tests require model credentials.
"""
import logging
from typing import Any
import pytest
from agent_framework_durabletask import DurableWorkflowClient
logging.basicConfig(level=logging.WARNING)
# Must match the outer workflow name in samples/04-hosting/durabletask/11_subworkflow/worker.py
WORKFLOW_NAME = "review_pipeline"
# Module-level markers
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("11_subworkflow"),
pytest.mark.integration_test,
pytest.mark.requires_dts,
pytest.mark.requires_azure_openai,
]
class TestSubworkflowComposition:
"""Composed (outer + inner) workflow execution on a standalone durabletask worker."""
@pytest.fixture(autouse=True)
def setup(self, workflow_client: DurableWorkflowClient) -> None:
"""Bind the DurableWorkflowClient for the current sample worker."""
self.client = workflow_client
def _run(self, review: str) -> Any:
"""Run the composed workflow with a review and return its final output."""
instance_id = self.client.start_workflow(input=review, workflow_name=WORKFLOW_NAME)
return self.client.await_workflow_output(instance_id, workflow_name=WORKFLOW_NAME, timeout_seconds=180)
def test_positive_review_runs_through_subworkflow(self) -> None:
"""A positive review flows through the embedded sentiment sub-workflow to a report."""
output = self._run(
"Absolutely love this espresso machine - it heats up fast and the coffee is consistently great."
)
assert output is not None
# The outer reporter wraps the inner sub-workflow's forwarded sentiment summary.
assert "sentiment" in str(output).lower()
def test_negative_review_runs_through_subworkflow(self) -> None:
"""A negative review also completes the composed pipeline end-to-end."""
output = self._run("Disappointed. The device stopped working after two weeks and support never replied.")
assert output is not None
assert "sentiment" in str(output).lower()
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,152 @@
# Copyright (c) Microsoft. All rights reserved.
"""Integration tests for the composed sub-workflow HITL sample (12_subworkflow_hitl).
Exercises human-in-the-loop **inside a nested sub-workflow** on a standalone
durabletask worker:
- An outer ``moderation_pipeline`` embeds an inner ``human_review`` workflow via a
``WorkflowExecutor`` node (``review_sub``); on the durable host the inner workflow
runs as a child orchestration.
- The inner ``review_gate`` pauses via ``request_info``. The pending request surfaces
at the top-level instance with a **qualified** id ``review_sub~0~{requestId}`` (the
``~{ordinal}~`` hop addresses the specific child the node dispatched).
- The client responds with that qualified id against the *top-level* instance and the
host routes it to the owning child orchestration, resuming to an approved/rejected
outcome.
This sample hosts **no AI agents**, so it needs only the DTS emulator (no model
credentials), which makes it a deterministic end-to-end check of the nested-HITL
addressing.
"""
import logging
import time
from typing import Any
import pytest
from agent_framework_durabletask import DurableWorkflowClient
from agent_framework_durabletask._workflows.naming import SUBWORKFLOW_REQUEST_SEPARATOR
logging.basicConfig(level=logging.WARNING)
# Must match the outer workflow name in samples/04-hosting/durabletask/12_subworkflow_hitl/worker.py
WORKFLOW_NAME = "moderation_pipeline"
# The WorkflowExecutor node id that embeds the inner HITL workflow.
SUBWORKFLOW_NODE_ID = "review_sub"
# Module-level markers. No requires_azure_openai: the sample hosts no agents.
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("12_subworkflow_hitl"),
pytest.mark.integration_test,
pytest.mark.requires_dts,
]
def _wait_for_hitl_request(
client: DurableWorkflowClient, instance_id: str, timeout_seconds: int = 90
) -> list[dict[str, Any]]:
"""Poll until the workflow (or a nested sub-workflow) records a pending HITL request."""
deadline = time.time() + timeout_seconds
while time.time() < deadline:
pending = client.get_pending_hitl_requests(instance_id, workflow_name=WORKFLOW_NAME)
if pending:
return pending
time.sleep(2)
raise AssertionError(f"Timed out waiting for a nested HITL request on instance {instance_id}")
class TestSubworkflowHITL:
"""Nested (sub-workflow) human-in-the-loop on a standalone durabletask worker."""
@pytest.fixture(autouse=True)
def setup(self, workflow_client: DurableWorkflowClient) -> None:
"""Bind the DurableWorkflowClient for the current sample worker."""
self.client = workflow_client
def _run_case(self, submission: dict[str, Any], *, approve: bool) -> tuple[dict[str, Any], Any]:
"""Start a moderation case, answer the nested HITL pause, return (request, output)."""
instance_id = self.client.start_workflow(input=submission, workflow_name=WORKFLOW_NAME)
pending = _wait_for_hitl_request(self.client, instance_id)
request = pending[0]
self.client.send_hitl_response(
instance_id,
request["request_id"],
{"approved": approve, "reviewer_notes": "Looks good." if approve else "Violates content policy."},
workflow_name=WORKFLOW_NAME,
)
output = self.client.await_workflow_output(instance_id, workflow_name=WORKFLOW_NAME, timeout_seconds=180)
return request, output
def test_nested_request_id_is_qualified_with_ordinal(self) -> None:
"""The nested pending request surfaces with a ``review_sub~0~{id}`` qualified id."""
instance_id = self.client.start_workflow(
input={
"content_id": "article-100",
"title": "Quarterly Roadmap",
"body": "A summary of the upcoming features planned for the next quarter.",
},
workflow_name=WORKFLOW_NAME,
)
pending = _wait_for_hitl_request(self.client, instance_id)
assert len(pending) == 1
request = pending[0]
# The qualifier carries the node id and the child's ordinal (0 for the single
# dispatch), then the inner bare request id: ``review_sub~0~{requestId}``.
expected_prefix = f"{SUBWORKFLOW_NODE_ID}{SUBWORKFLOW_REQUEST_SEPARATOR}0{SUBWORKFLOW_REQUEST_SEPARATOR}"
assert request["request_id"].startswith(expected_prefix), request["request_id"]
# The bare inner id is non-empty after the qualifier.
assert request["request_id"][len(expected_prefix) :]
# The originating executor is the inner workflow's review gate.
assert request["source_executor_id"] == "review_gate"
# Drain the pause so the worker does not leave the instance hanging.
self.client.send_hitl_response(
instance_id,
request["request_id"],
{"approved": True, "reviewer_notes": "ok"},
workflow_name=WORKFLOW_NAME,
)
self.client.await_workflow_output(instance_id, workflow_name=WORKFLOW_NAME, timeout_seconds=180)
def test_nested_hitl_approval(self) -> None:
"""Responding 'approved' to the nested request resumes the outer workflow to APPROVED."""
_request, output = self._run_case(
{
"content_id": "article-001",
"title": "Introduction to AI in Healthcare",
"body": (
"Artificial intelligence is improving healthcare by enabling faster diagnosis, "
"personalized treatment plans, and better patient outcomes."
),
},
approve=True,
)
assert output is not None
assert "APPROVED" in str(output).upper()
def test_nested_hitl_rejection(self) -> None:
"""Responding 'rejected' to the nested request resumes the outer workflow to REJECTED."""
_request, output = self._run_case(
{
"content_id": "article-002",
"title": "Get Rich Quick",
"body": "Click here NOW to make $10,000 overnight! GUARANTEED! Limited time offer!",
},
approve=False,
)
assert output is not None
assert "REJECTED" in str(output).upper()
if __name__ == "__main__":
pytest.main([__file__, "-v"])