chore: import upstream snapshot with attribution
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
CodeQL / Analyze (csharp) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:25 +08:00
commit db620d33df
5151 changed files with 925932 additions and 0 deletions
@@ -0,0 +1,10 @@
# Azure OpenAI Configuration
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_MODEL=your-deployment-name
FUNCTIONS_WORKER_RUNTIME=python
# Azure Functions Configuration
AzureWebJobsStorage=UseDevelopmentStorage=true
DURABLE_TASK_SCHEDULER_CONNECTION_STRING=Endpoint=http://localhost:8080;Authentication=None
# Note: TASKHUB_NAME is not required for integration tests; it is auto-generated per test run.
@@ -0,0 +1,81 @@
# Sample Integration Tests
Integration tests that validate the Durable Agent Framework samples by running them as Azure Functions.
## Setup
### 1. Create `.env` file
Copy `.env.example` to `.env` and fill in your Azure credentials:
```bash
cp .env.example .env
```
Required variables:
- `AZURE_OPENAI_ENDPOINT`
- `AZURE_OPENAI_MODEL`
- `AZURE_OPENAI_API_KEY`
- `AzureWebJobsStorage`
- `DURABLE_TASK_SCHEDULER_CONNECTION_STRING`
- `FUNCTIONS_WORKER_RUNTIME`
### 2. Start required services
**Azurite (for orchestration tests):**
```bash
docker run -d -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite
```
**Durable Task Scheduler:**
```bash
docker run -d -p 8080:8080 -p 8082:8082 -e DTS_USE_DYNAMIC_TASK_HUBS=true mcr.microsoft.com/dts/dts-emulator:latest
```
## Running Tests
The tests automatically start and stop the Azure Functions app for each sample.
### Run all sample tests
```bash
uv run pytest packages/azurefunctions/tests/integration_tests -v
```
### Run specific sample
```bash
uv run pytest packages/azurefunctions/tests/integration_tests/test_01_single_agent.py -v
```
### Run with verbose output
```bash
uv run pytest packages/azurefunctions/tests/integration_tests -sv
```
## How It Works
Each test file uses pytest markers to automatically configure and start the function app:
```python
pytestmark = [
pytest.mark.sample("01_single_agent"),
pytest.mark.usefixtures("function_app_for_test"),
skip_if_azure_functions_integration_tests_disabled,
]
```
The `function_app_for_test` fixture:
1. Loads environment variables from `.env`
2. Validates required variables are present
3. Starts the function app on a dynamically allocated port
4. Waits for the app to be ready
5. Runs your tests
6. Tears down the function app
## Troubleshooting
**Missing environment variables:**
Ensure your `.env` file contains all required variables from `.env.example`.
**Tests timeout:**
Check that Azure OpenAI credentials are valid and the service is accessible.
@@ -0,0 +1,613 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Pytest configuration for Azure Functions integration tests.
This module provides fixtures, configuration, and test utilities for pytest.
"""
import os
import shutil
import socket
import subprocess
import sys
import time
import uuid
from collections.abc import Iterator, Mapping
from contextlib import suppress
from pathlib import Path
from typing import Any
import pytest
import requests
# =============================================================================
# Configuration Constants
# =============================================================================
TIMEOUT = 30 # seconds
ORCHESTRATION_TIMEOUT = 180 # seconds for orchestrations
_DEFAULT_HOST = "localhost"
# Emulator ports (match CI workflow configuration)
_AZURITE_BLOB_PORT = 10000
_DTS_EMULATOR_PORT = 8080
# =============================================================================
# Exceptions
# =============================================================================
class FunctionAppStartupError(RuntimeError):
"""Raised when the Azure Functions host fails to start reliably."""
pass
# =============================================================================
# Environment and Service Checks
# =============================================================================
def _load_env_file_if_present() -> None:
"""Load environment variables from the local .env file when available."""
env_file = Path(__file__).parent / ".env"
if not env_file.exists():
return
try:
from dotenv import load_dotenv
load_dotenv(env_file)
except ImportError:
# python-dotenv not available; rely on existing environment
pass
def _check_func_cli_available() -> bool:
"""Check if Azure Functions Core Tools (func) is installed and available."""
return shutil.which("func") is not None
def _check_port_listening(port: int, host: str = _DEFAULT_HOST) -> bool:
"""Check if a service is listening on the given port."""
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.settimeout(1)
return sock.connect_ex((host, port)) == 0
def _check_azurite_available() -> bool:
"""Check if Azurite (Azure Storage emulator) is available on the expected port."""
return _check_port_listening(_AZURITE_BLOB_PORT)
def _check_dts_emulator_available() -> bool:
"""Check if Durable Task Scheduler emulator is available on the expected port."""
return _check_port_listening(_DTS_EMULATOR_PORT)
def _should_skip_azure_functions_integration_tests() -> tuple[bool, str]:
"""Determine whether Azure Functions integration tests should be skipped."""
_load_env_file_if_present()
# Check for Azure Functions Core Tools
if not _check_func_cli_available():
return (
True,
"Azure Functions Core Tools (func) not installed. Install with: npm install -g azure-functions-core-tools@4", # noqa: E501
)
# Check for Azurite (Azure Storage emulator)
if not _check_azurite_available():
return (
True,
f"Azurite not running on port {_AZURITE_BLOB_PORT}. Start with: docker run -d -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite", # noqa: E501
)
# Check for Durable Task Scheduler emulator
if not _check_dts_emulator_available():
return (
True,
f"Durable Task Scheduler emulator not running on port {_DTS_EMULATOR_PORT}. Start with: docker run -d -p 8080:8080 -p 8082:8082 mcr.microsoft.com/dts/dts-emulator:latest", # noqa: E501
)
has_foundry_config = bool(os.getenv("FOUNDRY_PROJECT_ENDPOINT", "").strip()) and bool(
os.getenv("FOUNDRY_MODEL", "").strip()
)
has_azure_openai_config = bool(os.getenv("AZURE_OPENAI_ENDPOINT", "").strip()) and bool(
os.getenv("AZURE_OPENAI_MODEL", "").strip()
)
if not has_foundry_config and not has_azure_openai_config:
return (
True,
"No real FOUNDRY_* or AZURE_OPENAI_* configuration provided; skipping integration tests.",
)
return False, "Integration tests enabled."
_SKIP_AZURE_FUNCTIONS_INTEGRATION_TESTS, _AZURE_FUNCTIONS_SKIP_REASON = _should_skip_azure_functions_integration_tests()
skip_if_azure_functions_integration_tests_disabled = pytest.mark.skipif(
_SKIP_AZURE_FUNCTIONS_INTEGRATION_TESTS,
reason=_AZURE_FUNCTIONS_SKIP_REASON,
)
# =============================================================================
# Test Helper Class
# =============================================================================
class SampleTestHelper:
"""Helper class for testing samples."""
@staticmethod
def post_json(url: str, data: dict[str, Any], timeout: int = TIMEOUT) -> requests.Response:
"""POST JSON data to a URL."""
return requests.post(url, json=data, headers={"Content-Type": "application/json"}, timeout=timeout)
@staticmethod
def post_text(url: str, text: str, timeout: int = TIMEOUT) -> requests.Response:
"""POST plain text to a URL."""
return requests.post(url, data=text, headers={"Content-Type": "text/plain"}, timeout=timeout)
@staticmethod
def get(url: str, timeout: int = TIMEOUT) -> requests.Response:
"""GET request to a URL."""
return requests.get(url, timeout=timeout)
@staticmethod
def wait_for_orchestration(
status_url: str, max_wait: int = ORCHESTRATION_TIMEOUT, poll_interval: int = 2
) -> dict[str, Any]:
"""Wait for an orchestration to complete.
Args:
status_url: URL to poll for orchestration status
max_wait: Maximum seconds to wait
poll_interval: Seconds between polls
Returns:
Final orchestration status
Raises:
TimeoutError: If orchestration doesn't complete in time
"""
start_time = time.time()
while time.time() - start_time < max_wait:
response = requests.get(status_url, timeout=TIMEOUT)
response.raise_for_status()
status = response.json()
runtime_status = status.get("runtimeStatus", "")
if runtime_status in ["Completed", "Failed", "Terminated"]:
return status
time.sleep(poll_interval)
raise TimeoutError(f"Orchestration did not complete within {max_wait} seconds")
@staticmethod
def wait_for_orchestration_with_output(
status_url: str, max_wait: int = ORCHESTRATION_TIMEOUT, poll_interval: int = 2
) -> dict[str, Any]:
"""Wait for an orchestration to complete and have output available.
This is a specialized version of wait_for_orchestration that also
ensures the output field is present, handling timing race conditions.
Args:
status_url: URL to poll for orchestration status
max_wait: Maximum seconds to wait
poll_interval: Seconds between polls
Returns:
Final orchestration status with output
Raises:
TimeoutError: If orchestration doesn't complete with output in time
"""
start_time = time.time()
while time.time() - start_time < max_wait:
response = requests.get(status_url, timeout=TIMEOUT)
response.raise_for_status()
status = response.json()
runtime_status = status.get("runtimeStatus", "")
if runtime_status in ["Failed", "Terminated"]:
return status
if runtime_status == "Completed" and status.get("output"):
return status
# If completed but no output, continue polling for a bit more to
# handle the race condition where output has not been persisted yet.
time.sleep(poll_interval)
# Provide detailed error message based on final status
final_response = requests.get(status_url, timeout=TIMEOUT)
final_response.raise_for_status()
final_status = final_response.json()
final_runtime_status = final_status.get("runtimeStatus", "Unknown")
if final_runtime_status == "Completed":
if "output" not in final_status:
raise TimeoutError(
"Orchestration completed but 'output' field is missing after "
f"{max_wait} seconds. Final status: {final_status}"
)
if not final_status["output"]:
raise TimeoutError(
"Orchestration completed but output is empty after "
f"{max_wait} seconds. Final status: {final_status}"
)
raise TimeoutError(
"Orchestration completed with output but validation failed after "
f"{max_wait} seconds. Final status: {final_status}"
)
raise TimeoutError(
"Orchestration did not complete within "
f"{max_wait} seconds. Final status: {final_runtime_status}, "
f"Full status: {final_status}"
)
# =============================================================================
# Function App Lifecycle Management
# =============================================================================
def _resolve_repo_root() -> Path:
"""Resolve the repository root, preferring GITHUB_WORKSPACE when available."""
workspace = os.getenv("GITHUB_WORKSPACE")
if workspace:
candidate = Path(workspace).expanduser()
if not (candidate / "samples").exists() and (candidate / "python" / "samples").exists():
return (candidate / "python").resolve()
return candidate.resolve()
# If `GITHUB_WORKSPACE` is not set,
# go up from conftest.py -> integration_tests -> tests -> azurefunctions -> packages -> python
return Path(__file__).resolve().parents[4]
def _get_sample_path_from_marker(request: pytest.FixtureRequest) -> tuple[Path | None, str | None]:
"""Get sample path from @pytest.mark.sample() marker.
Returns a tuple of (sample_path, error_message).
If successful, error_message is None.
If failed, sample_path is None and error_message contains the reason.
"""
marker = request.node.get_closest_marker("sample")
if not marker:
return (
None,
(
"No @pytest.mark.sample() marker found on test. Add pytestmark with "
"@pytest.mark.sample('sample_name') to the test module."
),
)
if not marker.args:
return (
None,
"@pytest.mark.sample() marker found but no sample name provided. Use @pytest.mark.sample('sample_name').",
)
sample_name = marker.args[0]
repo_root = _resolve_repo_root()
sample_path = repo_root / "samples" / "04-hosting" / "azure_functions" / sample_name
if not sample_path.exists():
return None, f"Sample directory does not exist: {sample_path}"
return sample_path, None
def _find_available_port(host: str = _DEFAULT_HOST) -> int:
"""Find an available TCP port on the given host."""
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind((host, 0))
return sock.getsockname()[1]
def _build_base_url(port: int, host: str = _DEFAULT_HOST) -> str:
"""Construct a base URL for the Azure Functions host."""
return f"http://{host}:{port}"
def _is_port_in_use(port: int, host: str = _DEFAULT_HOST) -> bool:
"""Check if a port is already in use.
Returns True if the port is in use, False otherwise.
"""
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
return sock.connect_ex((host, port)) == 0
def _load_and_validate_env(sample_path: Path) -> None:
"""Load .env file from current directory if it exists, then validate required environment variables.
Raises pytest.fail if required environment variables are missing.
"""
_load_env_file_if_present()
# Required environment variables for Azure Functions samples
required_env_vars = [
"AzureWebJobsStorage",
"DURABLE_TASK_SCHEDULER_CONNECTION_STRING",
"FUNCTIONS_WORKER_RUNTIME",
]
# Samples that host no AI agents need no model credentials (only the DTS emulator
# and Azurite). The suite-level gate still requires *some* LLM config to be present.
no_llm_samples = {"13_subworkflow_hitl"}
if sample_path.name in no_llm_samples:
pass
elif sample_path.name == "11_workflow_parallel":
required_env_vars.extend(["AZURE_OPENAI_ENDPOINT", "AZURE_OPENAI_MODEL"])
else:
required_env_vars.extend(["FOUNDRY_PROJECT_ENDPOINT", "FOUNDRY_MODEL"])
# Check if required env vars are set
missing_vars = [var for var in required_env_vars if not os.environ.get(var)]
if missing_vars:
pytest.fail(
f"Missing required environment variables: {', '.join(missing_vars)}. "
"Please create a .env file in tests/integration_tests/ based on .env.example or "
"set these variables in your environment."
)
def _start_function_app(sample_path: Path, port: int) -> subprocess.Popen[Any]:
"""Start a function app in the specified sample directory.
Returns the subprocess.Popen object for the running process.
"""
env = os.environ.copy()
# Use a unique TASKHUB_NAME for each test run to ensure test isolation.
# This prevents conflicts between parallel or repeated test runs, as Durable Functions
# use the task hub name to separate orchestration state.
env["TASKHUB_NAME"] = f"test{uuid.uuid4().hex[:8]}"
# The Azure Functions Python worker's dependency isolation mechanism crashes
# on Python 3.13 with a SIGSEGV in the protobuf C extension (google._upb).
# Disabling isolation lets the worker load dependencies from the app's own
# environment, which avoids the crash.
# See: https://github.com/Azure/azure-functions-python-worker/issues/1797
if sys.version_info >= (3, 13):
env.setdefault("PYTHON_ISOLATE_WORKER_DEPENDENCIES", "0")
# On Windows, use CREATE_NEW_PROCESS_GROUP to allow proper termination
# shell=True only on Windows to handle PATH resolution
if sys.platform == "win32":
return subprocess.Popen(
["func", "start", "--port", str(port)],
cwd=str(sample_path),
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
shell=True,
env=env,
)
# On Unix, use start_new_session=True to isolate the process group from the
# pytest-xdist worker. Without this, signals (e.g. from test-timeout) can
# propagate to the func host and vice-versa, potentially killing the worker.
return subprocess.Popen(
["func", "start", "--port", str(port)],
cwd=str(sample_path),
env=env,
start_new_session=True,
)
def _wait_for_function_app_ready(func_process: subprocess.Popen[Any], port: int, max_wait: int = 60) -> None:
"""Block until the Azure Functions host responds healthy or fail fast."""
start_time = time.time()
health_url = f"{_build_base_url(port)}/api/health"
last_error: Exception | None = None
while time.time() - start_time < max_wait:
# If the process exited early, capture any previously seen error and fail fast.
if func_process.poll() is not None:
raise FunctionAppStartupError(
f"Function app process exited with code {func_process.returncode} before becoming healthy"
) from last_error
if _is_port_in_use(port):
try:
response = requests.get(health_url, timeout=5)
if response.status_code == 200:
return
last_error = RuntimeError(f"Health check returned {response.status_code}")
except requests.RequestException as exc:
last_error = exc
time.sleep(1)
raise FunctionAppStartupError(
f"Function app did not become healthy on port {port} within {max_wait} seconds"
) from last_error
def _cleanup_function_app(func_process: subprocess.Popen[Any]) -> None:
"""Clean up the function app process and all its children.
Uses psutil if available for more thorough cleanup, falls back to basic termination.
"""
try:
import psutil
if func_process.poll() is None: # Process still running
# Get parent process
parent = psutil.Process(func_process.pid)
# Get all child processes recursively
children = parent.children(recursive=True)
# Kill children first
for child in children:
with suppress(psutil.NoSuchProcess, psutil.AccessDenied):
child.kill()
# Kill parent
with suppress(psutil.NoSuchProcess, psutil.AccessDenied):
parent.kill()
# Wait for all to terminate
_gone, alive = psutil.wait_procs(children + [parent], timeout=3)
# Force kill any remaining
for proc in alive:
with suppress(psutil.NoSuchProcess, psutil.AccessDenied):
proc.kill()
except ImportError:
# Fallback if psutil not available
try:
if func_process.poll() is None:
func_process.kill()
func_process.wait()
except Exception:
# Ignore all exceptions during fallback cleanup; best effort to terminate process.
pass
except Exception:
pass # Best effort cleanup
# Give the port time to be released
time.sleep(2)
# =============================================================================
# Pytest Configuration
# =============================================================================
def pytest_configure(config: pytest.Config) -> None:
"""Register custom markers."""
config.addinivalue_line("markers", "orchestration: marks tests that use orchestrations (require Azurite)")
config.addinivalue_line(
"markers",
"sample(path): specify the sample directory path for the test (e.g., @pytest.mark.sample('01_single_agent'))",
)
def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item]) -> None:
"""Skip integration tests in this directory if prerequisites are not met."""
should_skip, reason = _should_skip_azure_functions_integration_tests()
if should_skip:
skip_marker = pytest.mark.skip(reason=reason)
for item in items:
# Only skip items that are in this integration_tests directory
if "integration_tests" in str(item.fspath):
item.add_marker(skip_marker)
# =============================================================================
# Pytest Fixtures
# =============================================================================
@pytest.fixture(scope="session")
def function_app_running() -> bool:
"""Check if the function app is running on localhost:7071.
This fixture can be used to skip tests if the function app is not available.
"""
try:
response = requests.get("http://localhost:7071/api/health", timeout=2)
return response.status_code == 200
except requests.exceptions.RequestException:
return False
@pytest.fixture(scope="session")
def skip_if_no_function_app(function_app_running: bool) -> None:
"""Skip test if function app is not running."""
if not function_app_running:
pytest.skip("Function app is not running on http://localhost:7071")
@pytest.fixture(scope="module")
def function_app_for_test(request: pytest.FixtureRequest) -> Iterator[dict[str, int | str]]:
"""Start the function app for the corresponding sample based on marker.
This fixture:
1. Determines which sample to run from @pytest.mark.sample()
2. Validates environment variables
3. Starts the function app using 'func start'
4. Waits for the app to be ready
5. Tears down the app after tests complete
Usage:
@pytest.mark.sample("01_single_agent")
@pytest.mark.usefixtures("function_app_for_test")
class TestSample01SingleAgent:
...
"""
# Get sample path from marker
sample_path, error_message = _get_sample_path_from_marker(request)
if error_message:
pytest.fail(error_message)
assert sample_path is not None, "Sample path must be resolved before starting the function app"
# Load .env file if it exists and validate required env vars
_load_and_validate_env(sample_path)
max_attempts = 3
# The overall budget MUST be shorter than the pytest-timeout value
# (--timeout=120 by default) so that the fixture finishes cleanly instead
# of being killed by os._exit() which crashes the xdist worker.
overall_budget = 100 # seconds leaves headroom below the 120 s test timeout
last_error: Exception | None = None
func_process: subprocess.Popen[Any] | None = None
base_url = ""
port = 0
overall_start = time.monotonic()
attempts_made = 0
for _ in range(max_attempts):
remaining = overall_budget - (time.monotonic() - overall_start)
if remaining < 10:
# Not enough time for another attempt; bail out.
break
attempts_made += 1
port = _find_available_port()
base_url = _build_base_url(port)
func_process = _start_function_app(sample_path, port)
try:
# Cap each attempt's wait to the remaining budget minus a small
# buffer for cleanup.
per_attempt_wait = min(60, int(remaining) - 5)
_wait_for_function_app_ready(func_process, port, max_wait=max(per_attempt_wait, 10))
last_error = None
break
except FunctionAppStartupError as exc:
last_error = exc
_cleanup_function_app(func_process)
func_process = None
if func_process is None:
elapsed = int(time.monotonic() - overall_start)
error_message = f"Function app failed to start after {attempts_made} attempt(s) ({elapsed}s elapsed)."
if last_error is not None:
error_message += f" Last error: {last_error}"
pytest.fail(error_message)
try:
yield {"base_url": base_url, "port": port}
finally:
if func_process is not None:
_cleanup_function_app(func_process)
@pytest.fixture(scope="module")
def base_url(function_app_for_test: Mapping[str, int | str]) -> str:
"""Expose the function app's base URL to tests."""
return str(function_app_for_test["base_url"])
@pytest.fixture(scope="session")
def sample_helper() -> type[SampleTestHelper]:
"""Provide the SampleTestHelper class for tests."""
return SampleTestHelper
@@ -0,0 +1,115 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Single Agent Sample
Tests the single agent sample with various message formats and session management.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite or Azure Storage account configured
Usage:
uv run pytest packages/azurefunctions/tests/integration_tests/test_01_single_agent.py -v
"""
import pytest
from agent_framework_durabletask import THREAD_ID_HEADER
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("01_single_agent"),
pytest.mark.usefixtures("function_app_for_test"),
]
class TestSampleSingleAgent:
"""Tests for 01_single_agent sample."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Provide agent-specific base URL and helper for the tests."""
self.base_url = f"{base_url}/api/agents/Joker"
self.helper = sample_helper
def test_health_check(self, base_url: str, sample_helper) -> None:
"""Test health check endpoint."""
response = sample_helper.get(f"{base_url}/api/health")
assert response.status_code == 200
data = response.json()
assert data["status"] == "healthy"
def test_simple_message_json(self) -> None:
"""Test sending a simple message with JSON payload."""
response = self.helper.post_json(
f"{self.base_url}/run",
{"message": "Tell me a short joke about cloud computing.", "thread_id": "test-simple-json"},
)
# Agent can return 200 (immediate) or 202 (async with wait_for_response=false)
assert response.status_code in [200, 202]
data = response.json()
if response.status_code == 200:
# Synchronous response - check result directly
assert data["status"] == "success"
assert "response" in data
assert data["message_count"] >= 1
else:
# Async response - check we got correlation info
assert "correlation_id" in data or "thread_id" in data
def test_simple_message_plain_text(self) -> None:
"""Test sending a message with plain text payload."""
response = self.helper.post_text(f"{self.base_url}/run", "Tell me a short joke about networking.")
assert response.status_code in [200, 202]
# Agent responded with plain text when the request body was text/plain.
assert response.text.strip()
assert response.headers.get(THREAD_ID_HEADER) is not None
def test_thread_id_in_query(self) -> None:
"""Test using thread_id in query parameter."""
response = self.helper.post_text(
f"{self.base_url}/run?thread_id=test-query-thread", "Tell me a short joke about weather in Texas."
)
assert response.status_code in [200, 202]
assert response.text.strip()
assert response.headers.get(THREAD_ID_HEADER) == "test-query-thread"
def test_conversation_continuity(self) -> None:
"""Test conversation context is maintained across requests."""
thread_id = "test-continuity"
# First message
response1 = self.helper.post_json(
f"{self.base_url}/run",
{"message": "Tell me a short joke about weather in Seattle.", "thread_id": thread_id},
)
assert response1.status_code in [200, 202]
if response1.status_code == 200:
data1 = response1.json()
assert data1["message_count"] == 2 # Initial + reply
# Second message in same session
response2 = self.helper.post_json(
f"{self.base_url}/run", {"message": "What about San Francisco?", "thread_id": thread_id}
)
assert response2.status_code == 200
data2 = response2.json()
assert data2["message_count"] == 4
else:
# In async mode, we can't easily test message count
# Just verify we can make multiple calls
response2 = self.helper.post_json(
f"{self.base_url}/run", {"message": "What about Texas?", "thread_id": thread_id}
)
assert response2.status_code == 202
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,65 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Multi-Agent Sample
Tests the multi-agent sample with different agent endpoints.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite or Azure Storage account configured
Usage:
uv run pytest packages/azurefunctions/tests/integration_tests/test_02_multi_agent.py -v
"""
import pytest
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("02_multi_agent"),
pytest.mark.usefixtures("function_app_for_test"),
]
class TestSampleMultiAgent:
"""Tests for 02_multi_agent sample."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Configure base URLs for Weather and Math agents."""
self.weather_base_url = f"{base_url}/api/agents/WeatherAgent"
self.math_base_url = f"{base_url}/api/agents/MathAgent"
self.helper = sample_helper
@pytest.mark.skip(reason="Flaky in CI: times out / crashes the xdist runner; temporarily disabled.")
def test_weather_agent(self) -> None:
"""Test WeatherAgent endpoint."""
response = self.helper.post_json(
f"{self.weather_base_url}/run",
{"message": "What is the weather in Seattle?"},
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
assert "response" in data
def test_math_agent(self) -> None:
"""Test MathAgent endpoint."""
response = self.helper.post_json(
f"{self.math_base_url}/run",
{"message": "Calculate a 20% tip on a $50 bill", "wait_for_response": False},
)
assert response.status_code == 202
data = response.json()
assert data["status"] == "accepted"
assert "correlation_id" in data
assert "thread_id" in data
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,121 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Reliable Streaming Sample
Tests the reliable streaming sample using Redis Streams for persistent message delivery.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite or Azure Storage account configured
- Redis running (docker run -d --name redis -p 6379:6379 redis:latest)
Usage:
uv run pytest packages/azurefunctions/tests/integration_tests/test_03_reliable_streaming.py -v
"""
import time
import pytest
import requests
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("03_reliable_streaming"),
pytest.mark.usefixtures("function_app_for_test"),
]
class TestSampleReliableStreaming:
"""Tests for 03_reliable_streaming sample."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Provide the base URL and helper for each test."""
self.base_url = base_url
self.agent_url = f"{base_url}/api/agents/TravelPlanner"
self.stream_url = f"{base_url}/api/agent/stream"
self.helper = sample_helper
def test_agent_run_and_stream(self) -> None:
"""Test agent execution with Redis streaming."""
# Start agent run
response = self.helper.post_json(
f"{self.agent_url}/run",
{"message": "Plan a 1-day trip to Seattle in 1 sentence", "wait_for_response": False},
)
assert response.status_code == 202
data = response.json()
thread_id = data.get("thread_id")
# Wait a moment for the agent to start writing to Redis
time.sleep(2)
# Stream response from Redis with longer timeout to account for LLM latency
stream_response = requests.get(
f"{self.stream_url}/{thread_id}",
headers={"Accept": "text/plain"},
timeout=60,
)
assert stream_response.status_code == 200
def test_stream_with_sse_format(self) -> None:
"""Test streaming with Server-Sent Events format."""
# Start agent run
response = self.helper.post_json(
f"{self.agent_url}/run",
{"message": "What's the weather like?", "wait_for_response": False},
)
assert response.status_code == 202
data = response.json()
thread_id = data.get("thread_id")
# Wait for agent to start writing
time.sleep(2)
# Stream with SSE format
stream_response = requests.get(
f"{self.stream_url}/{thread_id}",
headers={"Accept": "text/event-stream"},
timeout=60,
)
assert stream_response.status_code == 200
content_type = stream_response.headers.get("content-type", "")
assert "text/event-stream" in content_type
# Check for SSE event markers if we got content
content = stream_response.text
if content:
assert "event:" in content or "data:" in content
def test_stream_nonexistent_conversation(self) -> None:
"""Test streaming from a non-existent conversation.
The endpoint will wait for data in Redis, but since the conversation
doesn't exist, it will timeout. This is expected behavior.
"""
fake_id = "nonexistent-conversation-12345"
# Should timeout since the conversation doesn't exist
with pytest.raises(requests.exceptions.ReadTimeout):
requests.get(
f"{self.stream_url}/{fake_id}",
headers={"Accept": "text/plain"},
timeout=10, # Short timeout for non-existent ID
)
def test_health_endpoint(self) -> None:
"""Test health check endpoint."""
response = self.helper.get(f"{self.base_url}/api/health")
assert response.status_code == 200
data = response.json()
assert data["status"] == "healthy"
assert "agents" in data
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,57 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Orchestration Chaining Sample
Tests the orchestration chaining sample for sequential agent execution.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite running for durable orchestrations (or Azure Storage account configured)
Usage:
# Start Azurite (if not already running)
azurite &
# Run tests
uv run pytest packages/azurefunctions/tests/integration_tests/test_04_single_agent_orchestration_chaining.py -v
"""
import pytest
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("04_single_agent_orchestration_chaining"),
pytest.mark.usefixtures("function_app_for_test"),
]
@pytest.mark.orchestration
class TestSampleOrchestrationChaining:
"""Tests for 04_single_agent_orchestration_chaining sample."""
@pytest.fixture(autouse=True)
def _setup(self, sample_helper) -> None:
"""Provide the helper for each test."""
self.helper = sample_helper
def test_orchestration_chaining(self, base_url: str) -> None:
"""Test sequential agent calls in orchestration."""
# Start orchestration
response = self.helper.post_json(f"{base_url}/api/singleagent/run", {})
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
# Wait for completion with output available
status = self.helper.wait_for_orchestration_with_output(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "output" in status
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,59 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for MultiAgent Concurrency Sample
Tests the multi-agent concurrency sample for parallel agent execution.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite running for durable orchestrations (or Azure Storage account configured)
Usage:
# Start Azurite (if not already running)
azurite &
# Run tests
uv run pytest packages/azurefunctions/tests/integration_tests/test_05_multi_agent_orchestration_concurrency.py -v
"""
import pytest
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.orchestration,
pytest.mark.sample("05_multi_agent_orchestration_concurrency"),
pytest.mark.usefixtures("function_app_for_test"),
]
class TestSampleMultiAgentConcurrency:
"""Tests for 05_multi_agent_orchestration_concurrency sample."""
@pytest.fixture(autouse=True)
def _setup(self, sample_helper) -> None:
"""Provide the helper for each test."""
self.helper = sample_helper
def test_concurrent_agents(self, base_url: str) -> None:
"""Test multiple agents running concurrently."""
# Start orchestration
response = self.helper.post_text(f"{base_url}/api/multiagent/run", "What is temperature?")
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
# Wait for completion
status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
output = status["output"]
assert "physicist" in output
assert "chemist" in output
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,77 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for MultiAgent Conditionals Sample
Tests the multi-agent conditionals sample for conditional orchestration logic.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite running for durable orchestrations (or Azure Storage account configured)
Usage:
# Start Azurite (if not already running)
azurite &
# Run tests
uv run pytest packages/azurefunctions/tests/integration_tests/test_06_multi_agent_orchestration_conditionals.py -v
"""
import pytest
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.orchestration,
pytest.mark.sample("06_multi_agent_orchestration_conditionals"),
pytest.mark.usefixtures("function_app_for_test"),
]
class TestSampleMultiAgentConditionals:
"""Tests for 06_multi_agent_orchestration_conditionals sample."""
@pytest.fixture(autouse=True)
def _setup(self, sample_helper) -> None:
"""Provide the helper for each test."""
self.helper = sample_helper
def test_legitimate_email(self, base_url: str) -> None:
"""Test conditional logic with legitimate email."""
response = self.helper.post_json(
f"{base_url}/api/spamdetection/run",
{
"email_id": "email-test-001",
"email_content": "Hi John, I hope you are doing well. Can you send me the report?",
},
)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
# Wait for completion
status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "Email sent:" in status["output"]
def test_spam_email(self, base_url: str) -> None:
"""Test conditional logic with spam email."""
response = self.helper.post_json(
f"{base_url}/api/spamdetection/run",
{"email_id": "email-test-002", "email_content": "URGENT! You have won $1,000,000! Click here now!"},
)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
# Wait for completion
status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "Email marked as spam:" in status["output"]
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,185 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Human-in-the-Loop (HITL) Orchestration Sample
Tests the HITL orchestration sample for content generation with human approval workflow.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite running for durable orchestrations (or Azure Storage account configured)
Usage:
# Start Azurite (if not already running)
azurite &
# Run tests
uv run pytest packages/azurefunctions/tests/integration_tests/test_07_single_agent_orchestration_hitl.py -v
"""
import time
import pytest
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("07_single_agent_orchestration_hitl"),
pytest.mark.usefixtures("function_app_for_test"),
]
@pytest.mark.orchestration
class TestSampleHITLOrchestration:
"""Tests for 07_single_agent_orchestration_hitl sample."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Provide the helper and base URL for each test."""
self.hitl_base_url = f"{base_url}/api/hitl"
self.helper = sample_helper
def test_hitl_orchestration_approval(self) -> None:
"""Test HITL orchestration with human approval."""
# Start orchestration
response = self.helper.post_json(
f"{self.hitl_base_url}/run",
{"topic": "artificial intelligence", "max_review_attempts": 3, "approval_timeout_hours": 1.0},
)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
assert data["topic"] == "artificial intelligence"
instance_id = data["instanceId"]
# Wait a bit for the orchestration to generate initial content
time.sleep(5)
# Check status to ensure it's waiting for approval
status_response = self.helper.get(data["statusQueryGetUri"])
assert status_response.status_code == 200
status = status_response.json()
assert status["runtimeStatus"] in ["Running", "Pending"]
# Send approval
approval_response = self.helper.post_json(
f"{self.hitl_base_url}/approve/{instance_id}", {"approved": True, "feedback": ""}
)
assert approval_response.status_code == 200
approval_data = approval_response.json()
assert approval_data["approved"] is True
# Wait for orchestration to complete
status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "output" in status
assert "content" in status["output"]
def test_hitl_orchestration_rejection_with_feedback(self) -> None:
"""Test HITL orchestration with rejection and subsequent approval."""
# Start orchestration
response = self.helper.post_json(
f"{self.hitl_base_url}/run",
{"topic": "machine learning", "max_review_attempts": 3, "approval_timeout_hours": 1.0},
)
assert response.status_code == 202
data = response.json()
instance_id = data["instanceId"]
# Wait for initial content generation
time.sleep(5)
# Send rejection with feedback
rejection_response = self.helper.post_json(
f"{self.hitl_base_url}/approve/{instance_id}",
{"approved": False, "feedback": "Please make it more concise and focus on practical applications."},
)
assert rejection_response.status_code == 200
# Wait for regeneration
time.sleep(5)
# Check status - should still be running
status_response = self.helper.get(data["statusQueryGetUri"])
assert status_response.status_code == 200
status = status_response.json()
assert status["runtimeStatus"] in ["Running", "Pending"]
# Now approve the revised content
approval_response = self.helper.post_json(
f"{self.hitl_base_url}/approve/{instance_id}", {"approved": True, "feedback": ""}
)
assert approval_response.status_code == 200
# Wait for completion
status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "output" in status
def test_hitl_orchestration_missing_topic(self) -> None:
"""Test HITL orchestration with missing topic."""
response = self.helper.post_json(f"{self.hitl_base_url}/run", {"max_review_attempts": 3})
assert response.status_code == 400
data = response.json()
assert "error" in data
def test_hitl_get_status(self) -> None:
"""Test getting orchestration status."""
# Start orchestration
response = self.helper.post_json(
f"{self.hitl_base_url}/run",
{"topic": "quantum computing", "max_review_attempts": 2, "approval_timeout_hours": 1.0},
)
assert response.status_code == 202
data = response.json()
instance_id = data["instanceId"]
# Get status
status_response = self.helper.get(f"{self.hitl_base_url}/status/{instance_id}")
assert status_response.status_code == 200
status = status_response.json()
assert "instanceId" in status
assert "runtimeStatus" in status
assert status["instanceId"] == instance_id
# Cleanup: approve to complete orchestration
time.sleep(5)
self.helper.post_json(f"{self.hitl_base_url}/approve/{instance_id}", {"approved": True, "feedback": ""})
def test_hitl_approval_invalid_payload(self) -> None:
"""Test sending approval with invalid payload."""
# Start orchestration first
response = self.helper.post_json(
f"{self.hitl_base_url}/run",
{"topic": "test topic", "max_review_attempts": 1, "approval_timeout_hours": 1.0},
)
assert response.status_code == 202
data = response.json()
instance_id = data["instanceId"]
time.sleep(3)
# Send approval without 'approved' field
approval_response = self.helper.post_json(
f"{self.hitl_base_url}/approve/{instance_id}", {"feedback": "Some feedback"}
)
assert approval_response.status_code == 400
error_data = approval_response.json()
assert "error" in error_data
# Cleanup
self.helper.post_json(f"{self.hitl_base_url}/approve/{instance_id}", {"approved": True, "feedback": ""})
def test_hitl_status_invalid_instance(self) -> None:
"""Test getting status for non-existent instance."""
response = self.helper.get(f"{self.hitl_base_url}/status/invalid-instance-id")
assert response.status_code == 404
data = response.json()
assert "error" in data
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,100 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Workflow Shared State Sample
Tests the workflow shared state sample for conditional email processing
with shared state management.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite running for durable orchestrations (or Azure Storage account configured)
Usage:
# Start Azurite (if not already running)
azurite &
# Run tests
uv run pytest packages/azurefunctions/tests/integration_tests/test_09_workflow_shared_state.py -v
"""
import pytest
# Must match the workflow name in samples/04-hosting/azure_functions/09_workflow_shared_state/function_app.py
WORKFLOW_NAME = "email_triage_shared_state"
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("09_workflow_shared_state"),
pytest.mark.usefixtures("function_app_for_test"),
]
@pytest.mark.orchestration
class TestWorkflowSharedState:
"""Tests for 09_workflow_shared_state sample."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Provide the helper and base URL for each test."""
self.base_url = base_url
self.helper = sample_helper
def test_workflow_with_spam_email(self) -> None:
"""Test workflow with spam email content - should be detected and handled as spam."""
spam_content = "URGENT! You have won $1,000,000! Click here to claim your prize now before it expires!"
# Start orchestration with spam email
response = self.helper.post_text(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", spam_content)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
# Wait for completion
status = self.helper.wait_for_orchestration_with_output(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "output" in status
def test_workflow_with_legitimate_email(self) -> None:
"""Test workflow with legitimate email content - should generate response."""
legitimate_content = (
"Hi team, just a reminder about the sprint planning meeting tomorrow at 10 AM. "
"Please review the agenda items in Jira before the call."
)
# Start orchestration with legitimate email
response = self.helper.post_text(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", legitimate_content)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
# Wait for completion
status = self.helper.wait_for_orchestration_with_output(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "output" in status
def test_workflow_with_phishing_email(self) -> None:
"""Test workflow with phishing email - should be detected as spam."""
phishing_content = (
"Dear Customer, Your account has been compromised! "
"Click this link immediately to secure your account: http://totallylegit.suspicious.com/secure"
)
# Start orchestration with phishing email
response = self.helper.post_text(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", phishing_content)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
# Wait for completion
status = self.helper.wait_for_orchestration_with_output(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,116 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Workflow No Shared State Sample
Tests the workflow sample that runs without shared state,
demonstrating conditional routing with spam detection and email response.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite running for durable orchestrations (or Azure Storage account configured)
Usage:
# Start Azurite (if not already running)
azurite &
# Run tests
uv run pytest packages/azurefunctions/tests/integration_tests/test_10_workflow_no_shared_state.py -v
"""
import pytest
# Must match the workflow name in samples/04-hosting/azure_functions/10_workflow_no_shared_state/function_app.py
WORKFLOW_NAME = "email_triage"
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("10_workflow_no_shared_state"),
pytest.mark.usefixtures("function_app_for_test"),
]
@pytest.mark.orchestration
class TestWorkflowNoSharedState:
"""Tests for 10_workflow_no_shared_state sample."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Provide the helper and base URL for each test."""
self.base_url = base_url
self.helper = sample_helper
def test_workflow_with_spam_email(self) -> None:
"""Test workflow with spam email - should detect and handle as spam."""
payload = {
"email_id": "email-test-001",
"email_content": (
"URGENT! You've won $1,000,000! Click here immediately to claim your prize! "
"Limited time offer - act now!"
),
}
# Start orchestration
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
# Wait for completion
status = self.helper.wait_for_orchestration_with_output(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "output" in status
def test_workflow_with_legitimate_email(self) -> None:
"""Test workflow with legitimate email - should draft a response."""
payload = {
"email_id": "email-test-002",
"email_content": (
"Hi team, just a reminder about our sprint planning meeting tomorrow at 10 AM. "
"Please review the agenda in Jira."
),
}
# Start orchestration
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
# Wait for completion
status = self.helper.wait_for_orchestration_with_output(data["statusQueryGetUri"])
assert status["runtimeStatus"] == "Completed"
assert "output" in status
def test_workflow_status_endpoint(self) -> None:
"""Test that the status endpoint works correctly."""
payload = {
"email_id": "email-test-003",
"email_content": "Quick question: When is the next team meeting scheduled?",
}
# Start orchestration
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
data = response.json()
instance_id = data["instanceId"]
# Check status using the workflow status endpoint
status_response = self.helper.get(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/status/{instance_id}")
assert status_response.status_code == 200
status = status_response.json()
assert "instanceId" in status
assert status["instanceId"] == instance_id
assert "runtimeStatus" in status
# Wait for completion to clean up
self.helper.wait_for_orchestration(data["statusQueryGetUri"])
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,87 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Parallel Workflow Sample
Tests the parallel workflow execution sample demonstrating:
- Two executors running concurrently (fan-out to activities)
- Two agents running concurrently (fan-out to entities)
- Mixed agent + executor running concurrently
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite running for durable orchestrations (or Azure Storage account configured)
Usage:
# Start Azurite (if not already running)
azurite &
# Run tests
uv run pytest packages/azurefunctions/tests/integration_tests/test_11_workflow_parallel.py -v
"""
import pytest
# Must match the workflow name in samples/04-hosting/azure_functions/11_workflow_parallel/function_app.py
WORKFLOW_NAME = "parallel_review"
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("11_workflow_parallel"),
pytest.mark.usefixtures("function_app_for_test"),
]
@pytest.mark.orchestration
class TestWorkflowParallel:
"""Tests for 11_workflow_parallel sample."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Provide the helper and base URL for each test."""
self.base_url = base_url
self.helper = sample_helper
@pytest.mark.skip(reason="Flaky in CI: times out / crashes the xdist runner; temporarily disabled.")
def test_parallel_workflow_end_to_end(self) -> None:
"""Run the parallel workflow end-to-end: start, check status, verify completion.
Consolidated into a single test on purpose: the work-stealing xdist scheduler
distributes tests (not modules) across workers, and the module-scoped
``function_app_for_test`` fixture is created per worker -- so multiple tests in
this module would each spawn a separate ``func`` host for this resource-heavy
parallel sample. One test keeps it to a single host while still covering the
fan-out path end-to-end.
"""
payload = {
"document_id": "doc-test-001",
"content": (
"The quarterly earnings report shows strong growth in our cloud services division. "
"Revenue increased by 25% compared to last year, driven by enterprise adoption. "
"Customer satisfaction remains high at 92%."
),
}
# Start the orchestration.
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
data = response.json()
instance_id = data["instanceId"]
assert "statusQueryGetUri" in data
# The status endpoint reflects the started instance.
status_response = self.helper.get(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/status/{instance_id}")
assert status_response.status_code == 200
assert status_response.json()["instanceId"] == instance_id
# Fan-out to parallel processors and agents completes with an aggregated output.
status = self.helper.wait_for_orchestration_with_output(data["statusQueryGetUri"], max_wait=300)
assert status["runtimeStatus"] == "Completed"
assert "output" in status
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,219 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for Workflow Human-in-the-Loop (HITL) Sample
Tests the workflow HITL sample demonstrating content moderation with human approval
using the MAF request_info / @response_handler pattern.
The function app is automatically started by the test fixture.
Prerequisites:
- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example)
- Azurite running for durable orchestrations (or Azure Storage account configured)
Usage:
# Start Azurite (if not already running)
azurite &
# Run tests
uv run pytest packages/azurefunctions/tests/integration_tests/test_12_workflow_hitl.py -v
"""
import time
import pytest
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("12_workflow_hitl"),
pytest.mark.usefixtures("function_app_for_test"),
]
# Must match the workflow name in samples/04-hosting/azure_functions/12_workflow_hitl/function_app.py
WORKFLOW_NAME = "content_moderation"
@pytest.mark.orchestration
class TestWorkflowHITL:
"""Tests for 12_workflow_hitl sample."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Provide the helper and base URL for each test."""
self.base_url = base_url
self.helper = sample_helper
def _wait_for_hitl_request(self, instance_id: str, timeout: int = 40) -> dict:
"""Polls for a pending HITL request."""
start_time = time.time()
while time.time() - start_time < timeout:
status_response = self.helper.get(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/status/{instance_id}")
if status_response.status_code == 200:
status = status_response.json()
pending_requests = status.get("pendingHumanInputRequests", [])
if pending_requests:
return status
time.sleep(2)
raise AssertionError(f"Timed out waiting for HITL request for instance {instance_id}")
def test_hitl_workflow_approval(self) -> None:
"""Test HITL workflow with human approval."""
payload = {
"content_id": "article-test-001",
"title": "Introduction to AI in Healthcare",
"body": (
"Artificial intelligence is revolutionizing healthcare by enabling faster diagnosis, "
"personalized treatment plans, and improved patient outcomes. Machine learning algorithms "
"can analyze medical images with remarkable accuracy."
),
"author": "Dr. Jane Smith",
}
# Start orchestration
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
data = response.json()
assert "instanceId" in data
assert "statusQueryGetUri" in data
instance_id = data["instanceId"]
# Wait for the workflow to reach the HITL pause point
status = self._wait_for_hitl_request(instance_id)
# Confirm status is valid
assert status["runtimeStatus"] in ["Running", "Pending"]
# Get the request ID from pending requests
pending_requests = status.get("pendingHumanInputRequests", [])
assert len(pending_requests) > 0, "Expected pending HITL request"
request_id = pending_requests[0]["requestId"]
# Send approval
approval_response = self.helper.post_json(
f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/respond/{instance_id}/{request_id}",
{"approved": True, "reviewer_notes": "Content is appropriate and well-written."},
)
assert approval_response.status_code == 200
# Wait for orchestration to complete
final_status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert final_status["runtimeStatus"] == "Completed"
assert "output" in final_status
def test_hitl_workflow_rejection(self) -> None:
"""Test HITL workflow with human rejection."""
payload = {
"content_id": "article-test-002",
"title": "Get Rich Quick Scheme",
"body": (
"Click here NOW to make $10,000 overnight! This SECRET method is GUARANTEED to work! "
"Limited time offer - act NOW before it's too late!"
),
"author": "Definitely Not Spam",
}
# Start orchestration
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
data = response.json()
instance_id = data["instanceId"]
# Wait for the workflow to reach the HITL pause point
status = self._wait_for_hitl_request(instance_id)
# Get the request ID from pending requests
pending_requests = status.get("pendingHumanInputRequests", [])
assert len(pending_requests) > 0, "Expected pending HITL request"
request_id = pending_requests[0]["requestId"]
# Send rejection
rejection_response = self.helper.post_json(
f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/respond/{instance_id}/{request_id}",
{"approved": False, "reviewer_notes": "Content appears to be spam/scam material."},
)
assert rejection_response.status_code == 200
# Wait for orchestration to complete
final_status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert final_status["runtimeStatus"] == "Completed"
assert "output" in final_status
# The output should indicate rejection
output = final_status["output"]
assert "rejected" in str(output).lower()
def test_hitl_workflow_status_endpoint(self) -> None:
"""Test that the workflow status endpoint shows pending HITL requests."""
payload = {
"content_id": "article-test-003",
"title": "Test Article",
"body": "This is a test article for checking status endpoint functionality.",
"author": "Test Author",
}
# Start orchestration
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
data = response.json()
instance_id = data["instanceId"]
# Wait for HITL pause
status = self._wait_for_hitl_request(instance_id)
# Check status
assert "instanceId" in status
assert status["instanceId"] == instance_id
assert "runtimeStatus" in status
assert "pendingHumanInputRequests" in status
# Clean up: approve to complete
pending_requests = status.get("pendingHumanInputRequests", [])
if pending_requests:
request_id = pending_requests[0]["requestId"]
self.helper.post_json(
f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/respond/{instance_id}/{request_id}",
{"approved": True, "reviewer_notes": ""},
)
# Wait for completion
self.helper.wait_for_orchestration(data["statusQueryGetUri"])
def test_hitl_workflow_with_neutral_content(self) -> None:
"""Test HITL workflow with neutral content that should get medium risk."""
payload = {
"content_id": "article-test-004",
"title": "Product Review",
"body": (
"This product works as advertised. The build quality is average and the price "
"is reasonable. I would recommend it for basic use cases but not for professional work."
),
"author": "Regular User",
}
# Start orchestration
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
data = response.json()
instance_id = data["instanceId"]
# Wait for HITL pause
status = self._wait_for_hitl_request(instance_id)
pending_requests = status.get("pendingHumanInputRequests", [])
assert len(pending_requests) > 0
request_id = pending_requests[0]["requestId"]
# Approve
self.helper.post_json(
f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/respond/{instance_id}/{request_id}",
{"approved": True, "reviewer_notes": "Approved after review."},
)
# Wait for completion
final_status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert final_status["runtimeStatus"] == "Completed"
if __name__ == "__main__":
pytest.main([__file__, "-v"])
@@ -0,0 +1,150 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Integration Tests for the Sub-workflow HITL Sample (13_subworkflow_hitl)
Tests nested human-in-the-loop through the Azure Functions host: the HITL pause
lives inside an inner workflow embedded via ``WorkflowExecutor``, so the pending
request surfaces at the top-level instance with a **qualified** request id
(``review_sub~0~{requestId}``). The caller responds against the top-level instance
and the host routes it to the owning child orchestration.
This sample hosts no AI agents, so it exercises the AF nested-HITL plumbing
deterministically (no model latency / variability).
The function app is automatically started by the test fixture.
Prerequisites:
- Azurite running for durable orchestrations
- Durable Task Scheduler emulator running on localhost:8080
Usage:
uv run pytest packages/azurefunctions/tests/integration_tests/test_13_workflow_subworkflow_hitl.py -v
"""
import time
import pytest
# Module-level markers - applied to all tests in this file
pytestmark = [
pytest.mark.flaky,
pytest.mark.integration,
pytest.mark.sample("13_subworkflow_hitl"),
pytest.mark.usefixtures("function_app_for_test"),
]
# Must match the outer workflow name in samples/.../13_subworkflow_hitl/function_app.py
WORKFLOW_NAME = "moderation_pipeline"
# The WorkflowExecutor node id that embeds the inner HITL workflow.
SUBWORKFLOW_NODE_ID = "review_sub"
@pytest.mark.orchestration
class TestSubworkflowHITL:
"""Tests for the 13_subworkflow_hitl sample (nested HITL behind one surface)."""
@pytest.fixture(autouse=True)
def _setup(self, base_url: str, sample_helper) -> None:
"""Provide the helper and base URL for each test."""
self.base_url = base_url
self.helper = sample_helper
def _wait_for_hitl_request(self, instance_id: str, timeout: int = 40) -> dict:
"""Poll the top-level status endpoint until a (nested) HITL request appears."""
start_time = time.time()
while time.time() - start_time < timeout:
status_response = self.helper.get(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/status/{instance_id}")
if status_response.status_code == 200:
status = status_response.json()
if status.get("pendingHumanInputRequests"):
return status
time.sleep(2)
raise AssertionError(f"Timed out waiting for a nested HITL request for instance {instance_id}")
def _start(self, payload: dict) -> dict:
"""Start the outer workflow and return the run response JSON."""
response = self.helper.post_json(f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/run", payload)
assert response.status_code == 202
return response.json()
def test_nested_request_surfaces_with_qualified_id(self) -> None:
"""The nested pending request is surfaced with a ``review_sub~0~{id}`` qualified id."""
data = self._start({
"content_id": "article-100",
"title": "Quarterly Roadmap",
"body": "A summary of the upcoming features planned for the next quarter.",
})
instance_id = data["instanceId"]
status = self._wait_for_hitl_request(instance_id)
pending = status.get("pendingHumanInputRequests", [])
assert len(pending) == 1
request_id = pending[0]["requestId"]
# The qualifier carries the node id and the child's ordinal (0 for the single
# dispatch), then the inner bare request id: ``review_sub~0~{requestId}``.
expected_prefix = f"{SUBWORKFLOW_NODE_ID}~0~"
assert request_id.startswith(expected_prefix), request_id
assert request_id[len(expected_prefix) :] # non-empty inner id
# The respondUrl always targets the top-level instance.
assert f"/api/workflow/{WORKFLOW_NAME}/respond/{instance_id}/" in pending[0]["respondUrl"]
# Drain the pause so the instance does not hang.
approve = self.helper.post_json(
f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/respond/{instance_id}/{request_id}",
{"approved": True, "reviewer_notes": "ok"},
)
assert approve.status_code == 200
self.helper.wait_for_orchestration(data["statusQueryGetUri"])
def test_nested_hitl_approval(self) -> None:
"""Responding 'approved' to the nested request resumes the outer workflow to APPROVED."""
data = self._start({
"content_id": "article-001",
"title": "Introduction to AI in Healthcare",
"body": (
"Artificial intelligence is improving healthcare by enabling faster diagnosis, "
"personalized treatment plans, and better patient outcomes."
),
})
instance_id = data["instanceId"]
status = self._wait_for_hitl_request(instance_id)
request_id = status["pendingHumanInputRequests"][0]["requestId"]
approval = self.helper.post_json(
f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/respond/{instance_id}/{request_id}",
{"approved": True, "reviewer_notes": "Looks good."},
)
assert approval.status_code == 200
final_status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert final_status["runtimeStatus"] == "Completed"
assert "APPROVED" in str(final_status.get("output")).upper()
def test_nested_hitl_rejection(self) -> None:
"""Responding 'rejected' to the nested request resumes the outer workflow to REJECTED."""
data = self._start({
"content_id": "article-002",
"title": "Get Rich Quick",
"body": "Click here NOW to make $10,000 overnight! GUARANTEED! Limited time offer!",
})
instance_id = data["instanceId"]
status = self._wait_for_hitl_request(instance_id)
request_id = status["pendingHumanInputRequests"][0]["requestId"]
rejection = self.helper.post_json(
f"{self.base_url}/api/workflow/{WORKFLOW_NAME}/respond/{instance_id}/{request_id}",
{"approved": False, "reviewer_notes": "Violates content policy."},
)
assert rejection.status_code == 200
final_status = self.helper.wait_for_orchestration(data["statusQueryGetUri"])
assert final_status["runtimeStatus"] == "Completed"
assert "REJECTED" in str(final_status.get("output")).upper()
if __name__ == "__main__":
pytest.main([__file__, "-v"])