chore: import upstream snapshot with attribution
CodeQL / Analyze (csharp) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
CodeQL / Analyze (csharp) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,44 @@
|
||||
# Purview Package (agent-framework-purview)
|
||||
|
||||
Integration with Microsoft Purview for data governance and policy enforcement.
|
||||
|
||||
## Main Classes
|
||||
|
||||
### Middleware
|
||||
|
||||
- **`PurviewPolicyMiddleware`** - Agent middleware for Purview policy enforcement
|
||||
- **`PurviewChatPolicyMiddleware`** - Chat-level middleware for policy enforcement
|
||||
|
||||
### Configuration
|
||||
|
||||
- **`PurviewSettings`** - Pydantic settings for Purview configuration
|
||||
- **`PurviewAppLocation`** / **`PurviewLocationType`** - Location configuration
|
||||
|
||||
### Caching
|
||||
|
||||
- **`CacheProvider`** - Cache provider for Purview policy caching
|
||||
|
||||
### Exceptions
|
||||
|
||||
- **`PurviewAuthenticationError`** - Authentication failures (inherits from `IntegrationInvalidAuthException`)
|
||||
- **`PurviewRateLimitError`** - Rate limit exceeded (inherits from `IntegrationException` via `PurviewServiceError`)
|
||||
- **`PurviewRequestError`** / **`PurviewServiceError`** - Request/service errors (inherit from `IntegrationException`)
|
||||
- **`PurviewPaymentRequiredError`** - Payment required (inherits from `IntegrationException` via `PurviewServiceError`)
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from agent_framework.microsoft import PurviewPolicyMiddleware, PurviewSettings
|
||||
|
||||
settings = PurviewSettings(...)
|
||||
middleware = PurviewPolicyMiddleware(settings=settings)
|
||||
agent = Agent(..., middleware=[middleware])
|
||||
```
|
||||
|
||||
## Import Path
|
||||
|
||||
```python
|
||||
from agent_framework.microsoft import PurviewPolicyMiddleware
|
||||
# or directly:
|
||||
from agent_framework_purview import PurviewPolicyMiddleware
|
||||
```
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) Microsoft Corporation.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE
|
||||
@@ -0,0 +1,324 @@
|
||||
## Microsoft Agent Framework – Purview Integration (Python)
|
||||
|
||||
`agent-framework-purview` adds Microsoft Purview (Microsoft Graph dataSecurityAndGovernance) policy evaluation to the Microsoft Agent Framework. It lets you enforce data security / governance policies on both the *prompt* (user input + conversation history) and the *model response* before they proceed further in your workflow.
|
||||
|
||||
> Status: **Preview**
|
||||
|
||||
### Key Features
|
||||
|
||||
- Middleware-based policy enforcement (agent-level and chat-client level)
|
||||
- Blocks or allows content at both ingress (prompt) and egress (response)
|
||||
- Works with any `Agent` / agent orchestration using the standard Agent Framework middleware pipeline
|
||||
- Supports both synchronous `TokenCredential` and `AsyncTokenCredential` from `azure-identity`
|
||||
- Configuration via `PurviewSettings` / `PurviewAppLocation`
|
||||
- Built-in caching with configurable TTL and size limits for protection scopes in `PurviewSettings`
|
||||
- Background processing for content activities and offline policy evaluation
|
||||
|
||||
### When to Use
|
||||
Add Purview when you need to:
|
||||
|
||||
- **Prevent sensitive data leaks**: Inline blocking of sensitive content based on Data Loss Prevention (DLP) policies.
|
||||
- **Enable governance**: Log AI interactions in Purview for Audit, Communication Compliance, Insider Risk Management, eDiscovery, and Data Lifecycle Management.
|
||||
- Prevent sensitive or disallowed content from being sent to an LLM
|
||||
- Prevent model output containing disallowed data from leaving the system
|
||||
- Apply centrally managed policies without rewriting agent logic
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Microsoft Azure subscription with Microsoft Purview configured.
|
||||
- Microsoft 365 subscription with an E5 license and pay-as-you-go billing setup.
|
||||
- For testing, you can use a Microsoft 365 Developer Program tenant. For more information, see [Join the Microsoft 365 Developer Program](https://learn.microsoft.com/en-us/office/developer-program/microsoft-365-developer-program).
|
||||
|
||||
### Authentication
|
||||
|
||||
`PurviewClient` uses the `azure-identity` library for token acquisition. You can use any `TokenCredential` or `AsyncTokenCredential` implementation.
|
||||
|
||||
- **Entra registration**: Register your agent and add the required Microsoft Graph permissions (`dataSecurityAndGovernance`) to the Service Principal. For more information, see [Register an application in Microsoft Entra ID](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app) and [dataSecurityAndGovernance resource type](https://learn.microsoft.com/en-us/graph/api/resources/datasecurityandgovernance). You'll need the Microsoft Entra app ID in the next step.
|
||||
|
||||
- **Graph Permissions**:
|
||||
- ProtectionScopes.Compute.All : [userProtectionScopeContainer](https://learn.microsoft.com/en-us/graph/api/userprotectionscopecontainer-compute)
|
||||
- Content.Process.All : [processContent](https://learn.microsoft.com/en-us/graph/api/userdatasecurityandgovernance-processcontent)
|
||||
- ContentActivity.Write : [contentActivity](https://learn.microsoft.com/en-us/graph/api/activitiescontainer-post-contentactivities)
|
||||
|
||||
- **Purview policies**: Configure Purview policies using the Microsoft Entra app ID to enable agent communications data to flow into Purview. For more information, see [Configure Microsoft Purview](https://learn.microsoft.com/purview/developer/configurepurview).
|
||||
|
||||
#### Scopes
|
||||
`PurviewSettings.get_scopes()` derives the Graph scope list (currently `https://graph.microsoft.com/.default` style).
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from agent_framework import Agent, Message, Role
|
||||
from agent_framework.openai import OpenAIChatCompletionClient
|
||||
from agent_framework.microsoft import PurviewPolicyMiddleware, PurviewSettings
|
||||
from azure.identity import InteractiveBrowserCredential
|
||||
|
||||
async def main():
|
||||
client = OpenAIChatCompletionClient() # uses environment for endpoint + deployment
|
||||
|
||||
purview_middleware = PurviewPolicyMiddleware(
|
||||
credential=InteractiveBrowserCredential(),
|
||||
settings=PurviewSettings(app_name="My Sample App")
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
client=client,
|
||||
instructions="You are a helpful assistant.",
|
||||
middleware=[purview_middleware]
|
||||
)
|
||||
|
||||
response = await agent.run(Message("user", ["Summarize zero trust in one sentence."]))
|
||||
print(response)
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
If a policy violation is detected on the prompt, the middleware terminates the run and substitutes a system message: `"Prompt blocked by policy"`. If on the response, the result becomes `"Response blocked by policy"`.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### `PurviewSettings`
|
||||
|
||||
```python
|
||||
PurviewSettings(
|
||||
app_name="My App", # Required: Display / logical name
|
||||
app_version=None, # Optional: Version string of the application
|
||||
tenant_id=None, # Optional: Tenant id (guid), used mainly for auth context
|
||||
purview_app_location=None, # Optional: PurviewAppLocation for scoping
|
||||
graph_base_uri="https://graph.microsoft.com/v1.0/",
|
||||
blocked_prompt_message="Prompt blocked by policy", # Custom message for blocked prompts
|
||||
blocked_response_message="Response blocked by policy", # Custom message for blocked responses
|
||||
ignore_exceptions=False, # If True, non-payment exceptions are logged but not thrown
|
||||
ignore_payment_required=False, # If True, 402 payment required errors are logged but not thrown
|
||||
cache_ttl_seconds=14400, # Cache TTL in seconds (default 4 hours)
|
||||
max_cache_size_bytes=200 * 1024 * 1024 # Max cache size in bytes (default 200MB)
|
||||
)
|
||||
```
|
||||
|
||||
### Caching
|
||||
|
||||
The Purview integration includes built-in caching for protection scopes responses to improve performance and reduce API calls:
|
||||
|
||||
- **Default TTL**: 4 hours (14400 seconds)
|
||||
- **Default Cache Size**: 200MB
|
||||
- **Cache Provider**: `InMemoryCacheProvider` is used by default, but you can provide a custom implementation via the `CacheProvider` protocol
|
||||
- **Cache Invalidation**: Cache is automatically invalidated when protection scope state is modified
|
||||
- **Exception Caching**: 402 Payment Required errors are cached to avoid repeated failed API calls
|
||||
|
||||
You can customize caching behavior in `PurviewSettings`:
|
||||
|
||||
```python
|
||||
from agent_framework.microsoft import PurviewSettings
|
||||
|
||||
settings = PurviewSettings(
|
||||
app_name="My App",
|
||||
cache_ttl_seconds=14400, # 4 hours
|
||||
max_cache_size_bytes=200 * 1024 * 1024 # 200MB
|
||||
)
|
||||
```
|
||||
|
||||
Or provide your own cache provider:
|
||||
|
||||
```python
|
||||
from typing import Any
|
||||
from agent_framework.microsoft import PurviewPolicyMiddleware, PurviewSettings, CacheProvider
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
class MyCustomCache(CacheProvider):
|
||||
async def get(self, key: str) -> Any | None:
|
||||
# Your implementation
|
||||
pass
|
||||
|
||||
async def set(self, key: str, value: Any, ttl_seconds: int | None = None) -> None:
|
||||
# Your implementation
|
||||
pass
|
||||
|
||||
async def remove(self, key: str) -> None:
|
||||
# Your implementation
|
||||
pass
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
settings = PurviewSettings(app_name="MyApp")
|
||||
|
||||
middleware = PurviewPolicyMiddleware(
|
||||
credential=credential,
|
||||
settings=settings,
|
||||
cache_provider=MyCustomCache()
|
||||
)
|
||||
```
|
||||
|
||||
To scope evaluation by location (application, URL, or domain):
|
||||
|
||||
```python
|
||||
from agent_framework.microsoft import (
|
||||
PurviewAppLocation,
|
||||
PurviewLocationType,
|
||||
PurviewSettings,
|
||||
)
|
||||
|
||||
settings = PurviewSettings(
|
||||
app_name="Contoso Support",
|
||||
purview_app_location=PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION,
|
||||
location_value="<app-client-id>"
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
### Customizing Blocked Messages
|
||||
|
||||
By default, when Purview blocks a prompt or response, the middleware returns a generic system message. You can customize these messages by providing your own text in the `PurviewSettings`:
|
||||
|
||||
```python
|
||||
from agent_framework.microsoft import PurviewSettings
|
||||
|
||||
settings = PurviewSettings(
|
||||
app_name="My App",
|
||||
blocked_prompt_message="Your request contains content that violates our policies. Please rephrase and try again.",
|
||||
blocked_response_message="The response was blocked due to policy restrictions. Please contact support if you need assistance."
|
||||
)
|
||||
```
|
||||
|
||||
### Exception Handling Controls
|
||||
|
||||
The Purview integration provides fine-grained control over exception handling to support graceful degradation scenarios:
|
||||
|
||||
```python
|
||||
from agent_framework.microsoft import PurviewSettings
|
||||
|
||||
# Ignore all non-payment exceptions (continue execution even if policy check fails)
|
||||
settings = PurviewSettings(
|
||||
app_name="My App",
|
||||
ignore_exceptions=True # Log errors but don't throw
|
||||
)
|
||||
|
||||
# Ignore only 402 Payment Required errors (useful for tenants without proper licensing)
|
||||
settings = PurviewSettings(
|
||||
app_name="My App",
|
||||
ignore_payment_required=True # Continue even without Purview Consumptive Billing Setup
|
||||
)
|
||||
|
||||
# Both can be combined
|
||||
settings = PurviewSettings(
|
||||
app_name="My App",
|
||||
ignore_exceptions=True,
|
||||
ignore_payment_required=True
|
||||
)
|
||||
```
|
||||
|
||||
### Selecting Agent vs Chat Middleware
|
||||
|
||||
Use the agent middleware when you already have / want the full agent pipeline:
|
||||
|
||||
```python
|
||||
from agent_framework import Agent
|
||||
from agent_framework.openai import OpenAIChatCompletionClient
|
||||
from agent_framework.microsoft import PurviewPolicyMiddleware, PurviewSettings
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
client = OpenAIChatCompletionClient()
|
||||
|
||||
agent = Agent(
|
||||
client=client,
|
||||
instructions="You are helpful.",
|
||||
middleware=[PurviewPolicyMiddleware(credential, PurviewSettings(app_name="My App"))]
|
||||
)
|
||||
```
|
||||
|
||||
Use the chat middleware when you attach directly to a chat client (e.g. minimal agent shell or custom orchestration):
|
||||
|
||||
```python
|
||||
import os
|
||||
from agent_framework import Agent
|
||||
from agent_framework.openai import OpenAIChatCompletionClient
|
||||
from agent_framework.microsoft import PurviewChatPolicyMiddleware, PurviewSettings
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
client = OpenAIChatCompletionClient(
|
||||
model=os.environ["AZURE_OPENAI_MODEL"],
|
||||
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
|
||||
credential=credential,
|
||||
middleware=[
|
||||
PurviewChatPolicyMiddleware(credential, PurviewSettings(app_name="My App (Chat)"))
|
||||
],
|
||||
)
|
||||
|
||||
agent = Agent(client=client, instructions="You are helpful.")
|
||||
```
|
||||
|
||||
The policy logic is identical; the difference is only the hook point in the pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Middleware Lifecycle
|
||||
|
||||
1. **Before agent execution** (`prompt phase`): all `context.messages` are evaluated.
|
||||
- If no valid user_id is found, processing is skipped (no policy evaluation)
|
||||
- Protection scopes are retrieved (with caching)
|
||||
- Applicable scopes are checked to determine execution mode
|
||||
- In inline mode: content is evaluated immediately
|
||||
- In offline mode: evaluation is queued in background
|
||||
2. **If blocked**: `context.result` is replaced with a system message and `context.terminate = True`.
|
||||
3. **After successful agent execution** (`response phase`): the produced messages are evaluated using the same user_id from the prompt phase.
|
||||
4. **If blocked**: result messages are replaced with a blocking notice.
|
||||
|
||||
The user identifier is discovered from `Message.additional_properties['user_id']` during the prompt phase and reused for the response phase, ensuring both evaluations map consistently to the same user. If no user_id is present, policy evaluation is skipped entirely.
|
||||
|
||||
You can customize the blocking messages using the `blocked_prompt_message` and `blocked_response_message` fields in `PurviewSettings`. For more advanced scenarios, you can wrap the middleware or post-process `context.result` in later middleware.
|
||||
|
||||
---
|
||||
|
||||
## Exceptions
|
||||
|
||||
| Exception | Scenario |
|
||||
|-----------|----------|
|
||||
| `PurviewPaymentRequiredError` | 402 Payment Required - tenant lacks proper Purview licensing or consumptive billing setup |
|
||||
| `PurviewAuthenticationError` | Token acquisition / validation issues |
|
||||
| `PurviewRateLimitError` | 429 responses from service |
|
||||
| `PurviewRequestError` | 4xx client errors (bad input, unauthorized, forbidden) |
|
||||
| `PurviewServiceError` | 5xx or unexpected service errors |
|
||||
|
||||
### Exception Handling
|
||||
|
||||
All exceptions inherit from `PurviewServiceError`. You can catch specific exceptions or use the base class:
|
||||
|
||||
```python
|
||||
from agent_framework.microsoft import (
|
||||
PurviewPaymentRequiredError,
|
||||
PurviewAuthenticationError,
|
||||
PurviewRateLimitError,
|
||||
PurviewRequestError,
|
||||
PurviewServiceError
|
||||
)
|
||||
|
||||
try:
|
||||
# Your code here
|
||||
pass
|
||||
except PurviewPaymentRequiredError as ex:
|
||||
# Handle licensing issues specifically
|
||||
print(f"Purview licensing required: {ex}")
|
||||
except (PurviewAuthenticationError, PurviewRateLimitError, PurviewRequestError, PurviewServiceError) as ex:
|
||||
# Handle other errors
|
||||
print(f"Purview enforcement skipped: {ex}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
- **User Identification**: When the configured credential resolves to a user token, that token's `user_id` is used for per-user policy scoping. For app-token credentials, provide a `user_id` per request (e.g. in `Message(..., additional_properties={"user_id": "<guid>"})`). If no user_id is provided or inferred, policy evaluation is skipped.
|
||||
- **Blocking Messages**: Can be customized via `blocked_prompt_message` and `blocked_response_message` in `PurviewSettings`. By default, they are "Prompt blocked by policy" and "Response blocked by policy" respectively.
|
||||
- **Streaming Responses**: Post-response policy evaluation presently applies only to non-streaming chat responses.
|
||||
- **Error Handling**: Use `ignore_exceptions` and `ignore_payment_required` settings for graceful degradation. When enabled, errors are logged but don't fail the request.
|
||||
- **Caching**: Protection scopes responses and 402 errors are cached by default with a 4-hour TTL. Cache is automatically invalidated when protection scope state changes.
|
||||
- **Cold-cache parallelization**: On a `ProtectionScopes` cache miss, scopes are refreshed in the background while `ProcessContent` runs in the foreground.
|
||||
- **Background Processing**: Content Activities and offline Process Content requests are handled asynchronously using background tasks to avoid blocking the main execution flow.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from ._cache import CacheProvider
|
||||
from ._exceptions import (
|
||||
PurviewAuthenticationError,
|
||||
PurviewPaymentRequiredError,
|
||||
PurviewRateLimitError,
|
||||
PurviewRequestError,
|
||||
PurviewServiceError,
|
||||
)
|
||||
from ._middleware import PurviewChatPolicyMiddleware, PurviewPolicyMiddleware
|
||||
from ._settings import PurviewAppLocation, PurviewLocationType, PurviewSettings, get_purview_scopes
|
||||
|
||||
__all__ = [
|
||||
"CacheProvider",
|
||||
"PurviewAppLocation",
|
||||
"PurviewAuthenticationError",
|
||||
"PurviewChatPolicyMiddleware",
|
||||
"PurviewLocationType",
|
||||
"PurviewPaymentRequiredError",
|
||||
"PurviewPolicyMiddleware",
|
||||
"PurviewRateLimitError",
|
||||
"PurviewRequestError",
|
||||
"PurviewServiceError",
|
||||
"PurviewSettings",
|
||||
"get_purview_scopes",
|
||||
]
|
||||
@@ -0,0 +1,188 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
"""Cache provider for Purview data."""
|
||||
|
||||
import hashlib
|
||||
import heapq
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
from typing import Any, Protocol
|
||||
|
||||
from ._models import ProtectionScopesRequest
|
||||
|
||||
|
||||
class CacheProvider(Protocol):
|
||||
"""Protocol for cache providers used by Purview integration."""
|
||||
|
||||
async def get(self, key: str) -> Any | None:
|
||||
"""Get a value from the cache.
|
||||
|
||||
Args:
|
||||
key: The cache key.
|
||||
|
||||
Returns:
|
||||
The cached value or None if not found or expired.
|
||||
"""
|
||||
...
|
||||
|
||||
async def set(self, key: str, value: Any, ttl_seconds: int | None = None) -> None:
|
||||
"""Set a value in the cache.
|
||||
|
||||
Args:
|
||||
key: The cache key.
|
||||
value: The value to cache.
|
||||
ttl_seconds: Time to live in seconds. If None, uses provider default.
|
||||
"""
|
||||
...
|
||||
|
||||
async def remove(self, key: str) -> None:
|
||||
"""Remove a value from the cache.
|
||||
|
||||
Args:
|
||||
key: The cache key.
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
class InMemoryCacheProvider:
|
||||
"""Simple in-memory cache implementation for Purview data.
|
||||
|
||||
This implementation uses a dictionary with expiration tracking and size limits.
|
||||
"""
|
||||
|
||||
def __init__(self, default_ttl_seconds: int = 1800, max_size_bytes: int = 200 * 1024 * 1024):
|
||||
"""Initialize the in-memory cache.
|
||||
|
||||
Args:
|
||||
default_ttl_seconds: Default time to live in seconds (default 1800 = 30 minutes).
|
||||
max_size_bytes: Maximum cache size in bytes (default 200MB).
|
||||
"""
|
||||
self._cache: dict[str, tuple[Any, float, int]] = {} # key -> (value, expiry, size)
|
||||
self._expiry_heap: list[tuple[float, str]] = [] # min-heap of (expiry_time, key)
|
||||
self._default_ttl = default_ttl_seconds
|
||||
self._max_size_bytes = max_size_bytes
|
||||
self._current_size_bytes = 0
|
||||
|
||||
def _estimate_size(self, value: Any) -> int:
|
||||
"""Estimate the size of a cached value in bytes.
|
||||
|
||||
Args:
|
||||
value: The value to estimate size for.
|
||||
|
||||
Returns:
|
||||
Estimated size in bytes.
|
||||
"""
|
||||
try:
|
||||
if hasattr(value, "model_dump_json"):
|
||||
return len(value.model_dump_json().encode("utf-8"))
|
||||
|
||||
return len(json.dumps(value, default=str).encode("utf-8"))
|
||||
except Exception:
|
||||
# Fallback to sys.getsizeof if JSON serialization fails
|
||||
try:
|
||||
return sys.getsizeof(value)
|
||||
except Exception:
|
||||
# Conservative fallback estimate
|
||||
return 1024
|
||||
|
||||
def _evict_if_needed(self, required_size: int) -> None:
|
||||
"""Evict oldest entries if needed to make room for new entry.
|
||||
|
||||
Uses a min-heap to efficiently find and evict entries with earliest expiry times.
|
||||
Also cleans up stale heap entries for keys that no longer exist in cache.
|
||||
|
||||
Args:
|
||||
required_size: Size in bytes needed for new entry.
|
||||
"""
|
||||
if self._current_size_bytes + required_size <= self._max_size_bytes:
|
||||
return
|
||||
|
||||
while self._expiry_heap and self._current_size_bytes + required_size > self._max_size_bytes:
|
||||
expiry_time, key = heapq.heappop(self._expiry_heap)
|
||||
|
||||
if key in self._cache:
|
||||
_, cached_expiry, size = self._cache[key]
|
||||
if cached_expiry == expiry_time:
|
||||
del self._cache[key]
|
||||
self._current_size_bytes -= size
|
||||
# else: stale heap entry, already updated/removed, skip it
|
||||
|
||||
async def get(self, key: str) -> Any | None:
|
||||
"""Get a value from the cache.
|
||||
|
||||
Args:
|
||||
key: The cache key.
|
||||
|
||||
Returns:
|
||||
The cached value or None if not found or expired.
|
||||
"""
|
||||
if key not in self._cache:
|
||||
return None
|
||||
|
||||
value, expiry, size = self._cache[key]
|
||||
if time.time() > expiry:
|
||||
del self._cache[key]
|
||||
self._current_size_bytes -= size
|
||||
return None
|
||||
|
||||
return value
|
||||
|
||||
async def set(self, key: str, value: Any, ttl_seconds: int | None = None) -> None:
|
||||
"""Set a value in the cache.
|
||||
|
||||
Args:
|
||||
key: The cache key.
|
||||
value: The value to cache.
|
||||
ttl_seconds: Time to live in seconds. If None, uses default TTL.
|
||||
"""
|
||||
ttl = ttl_seconds if ttl_seconds is not None else self._default_ttl
|
||||
expiry = time.time() + ttl
|
||||
size = self._estimate_size(value)
|
||||
|
||||
# Remove old entry if exists
|
||||
if key in self._cache:
|
||||
old_size = self._cache[key][2]
|
||||
self._current_size_bytes -= old_size
|
||||
|
||||
# Evict if needed
|
||||
self._evict_if_needed(size)
|
||||
|
||||
self._cache[key] = (value, expiry, size)
|
||||
self._current_size_bytes += size
|
||||
|
||||
heapq.heappush(self._expiry_heap, (expiry, key))
|
||||
|
||||
async def remove(self, key: str) -> None:
|
||||
"""Remove a value from the cache.
|
||||
|
||||
Args:
|
||||
key: The cache key.
|
||||
"""
|
||||
entry = self._cache.pop(key, None)
|
||||
if entry is not None:
|
||||
self._current_size_bytes -= entry[2]
|
||||
|
||||
|
||||
def create_protection_scopes_cache_key(request: ProtectionScopesRequest) -> str:
|
||||
"""Create a cache key for a ProtectionScopesRequest.
|
||||
|
||||
The key is based on the serialized request content (excluding correlation_id).
|
||||
|
||||
Args:
|
||||
request: The protection scopes request.
|
||||
|
||||
Returns:
|
||||
A string cache key.
|
||||
"""
|
||||
data = request.to_dict(exclude_none=True)
|
||||
|
||||
for field in ["correlation_id"]:
|
||||
data.pop(field, None)
|
||||
|
||||
json_str = json.dumps(data, sort_keys=True)
|
||||
return f"purview:protection_scopes:{hashlib.sha256(json_str.encode()).hexdigest()}"
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CacheProvider",
|
||||
]
|
||||
@@ -0,0 +1,233 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import inspect
|
||||
import json
|
||||
import logging
|
||||
from collections.abc import Awaitable, Callable
|
||||
from typing import Any, Literal, TypeVar, Union, overload
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
from agent_framework._telemetry import get_user_agent
|
||||
from agent_framework.observability import get_tracer
|
||||
from azure.core.credentials import TokenCredential
|
||||
from azure.core.credentials_async import AsyncTokenCredential
|
||||
from opentelemetry import trace
|
||||
|
||||
from ._exceptions import (
|
||||
PurviewAuthenticationError,
|
||||
PurviewPaymentRequiredError,
|
||||
PurviewRateLimitError,
|
||||
PurviewRequestError,
|
||||
PurviewServiceError,
|
||||
)
|
||||
from ._models import (
|
||||
ContentActivitiesRequest,
|
||||
ContentActivitiesResponse,
|
||||
ProcessContentRequest,
|
||||
ProcessContentResponse,
|
||||
ProtectionScopesRequest,
|
||||
ProtectionScopesResponse,
|
||||
)
|
||||
from ._settings import PurviewSettings, get_purview_scopes
|
||||
|
||||
AzureCredentialTypes = Union[TokenCredential, AsyncTokenCredential]
|
||||
AzureTokenProvider = Callable[[], Union[str, Awaitable[str]]]
|
||||
|
||||
logger = logging.getLogger("agent_framework.purview")
|
||||
|
||||
ResponseT = TypeVar("ResponseT")
|
||||
|
||||
|
||||
class PurviewClient:
|
||||
"""Async client for calling Graph Purview endpoints.
|
||||
|
||||
Supports synchronous TokenCredential, asynchronous AsyncTokenCredential,
|
||||
or callable token providers. A sync credential will be invoked in a thread
|
||||
to avoid blocking the event loop.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
credential: AzureCredentialTypes | AzureTokenProvider,
|
||||
settings: PurviewSettings,
|
||||
*,
|
||||
timeout: float | None = 10.0,
|
||||
):
|
||||
self._credential: AzureCredentialTypes | AzureTokenProvider = credential
|
||||
self._settings = settings
|
||||
self._graph_uri = (settings.get("graph_base_uri") or "https://graph.microsoft.com/v1.0/").rstrip("/")
|
||||
self._timeout = timeout
|
||||
self._client = httpx.AsyncClient(timeout=timeout)
|
||||
|
||||
async def close(self) -> None:
|
||||
await self._client.aclose()
|
||||
|
||||
async def _get_token(self, *, tenant_id: str | None = None) -> str:
|
||||
"""Acquire an access token using either async or sync credential, or callable token provider."""
|
||||
cred = self._credential
|
||||
# Callable token provider — returns a token string directly
|
||||
if callable(cred) and not isinstance(cred, (TokenCredential, AsyncTokenCredential)):
|
||||
result = cred()
|
||||
return await result if inspect.isawaitable(result) else result
|
||||
scopes = get_purview_scopes(self._settings)
|
||||
token = cred.get_token(*scopes, tenant_id=tenant_id)
|
||||
token = await token if inspect.isawaitable(token) else token
|
||||
return token.token
|
||||
|
||||
@staticmethod
|
||||
def _extract_token_info(token: str) -> dict[str, Any]:
|
||||
parts = token.split(".")
|
||||
if len(parts) < 2:
|
||||
raise ValueError("Invalid JWT token format")
|
||||
payload = parts[1]
|
||||
rem = len(payload) % 4
|
||||
if rem:
|
||||
payload += "=" * (4 - rem)
|
||||
decoded = base64.urlsafe_b64decode(payload)
|
||||
data = json.loads(decoded.decode("utf-8"))
|
||||
return {
|
||||
"user_id": data.get("oid") if data.get("idtyp") == "user" else None,
|
||||
"tenant_id": data.get("tid"),
|
||||
"client_id": data.get("appid"),
|
||||
}
|
||||
|
||||
async def get_user_info_from_token(self, *, tenant_id: str | None = None) -> dict[str, Any]:
|
||||
token = await self._get_token(tenant_id=tenant_id)
|
||||
return self._extract_token_info(token)
|
||||
|
||||
async def process_content(self, request: ProcessContentRequest) -> ProcessContentResponse:
|
||||
with get_tracer().start_as_current_span("purview.process_content"):
|
||||
token = await self._get_token(tenant_id=request.tenant_id)
|
||||
url = f"{self._graph_uri}/users/{request.user_id}/dataSecurityAndGovernance/processContent"
|
||||
headers: dict[str, str] = {}
|
||||
# Add If-None-Match header if scope_identifier is present
|
||||
if hasattr(request, "scope_identifier") and request.scope_identifier:
|
||||
headers["If-None-Match"] = request.scope_identifier
|
||||
# Add Prefer: evaluateInline header if process_inline is True
|
||||
if hasattr(request, "process_inline") and request.process_inline:
|
||||
headers["Prefer"] = "evaluateInline"
|
||||
|
||||
response: ProcessContentResponse | tuple[ProcessContentResponse, httpx.Headers] = await self._post(
|
||||
url, request, ProcessContentResponse, token, headers=headers, return_response=True
|
||||
)
|
||||
|
||||
if isinstance(response, tuple) and len(response) == 2:
|
||||
response_obj, _ = response
|
||||
return response_obj
|
||||
|
||||
return response
|
||||
|
||||
async def get_protection_scopes(self, request: ProtectionScopesRequest) -> ProtectionScopesResponse:
|
||||
with get_tracer().start_as_current_span("purview.get_protection_scopes"):
|
||||
token = await self._get_token()
|
||||
url = f"{self._graph_uri}/users/{request.user_id}/dataSecurityAndGovernance/protectionScopes/compute"
|
||||
response: ProtectionScopesResponse | tuple[ProtectionScopesResponse, httpx.Headers] = await self._post(
|
||||
url, request, ProtectionScopesResponse, token, return_response=True
|
||||
)
|
||||
|
||||
# Extract etag from response headers
|
||||
if isinstance(response, tuple) and len(response) == 2:
|
||||
response_obj, headers = response
|
||||
if "etag" in headers:
|
||||
etag_value = headers["etag"].strip('"')
|
||||
response_obj.scope_identifier = etag_value
|
||||
return response_obj
|
||||
|
||||
return response
|
||||
|
||||
async def send_content_activities(self, request: ContentActivitiesRequest) -> ContentActivitiesResponse:
|
||||
with get_tracer().start_as_current_span("purview.send_content_activities"):
|
||||
token = await self._get_token()
|
||||
url = f"{self._graph_uri}/users/{request.user_id}/dataSecurityAndGovernance/activities/contentActivities"
|
||||
return await self._post(url, request, ContentActivitiesResponse, token)
|
||||
|
||||
@overload
|
||||
async def _post(
|
||||
self,
|
||||
url: str,
|
||||
model: Any,
|
||||
response_type: type[ResponseT],
|
||||
token: str,
|
||||
headers: dict[str, str] | None = None,
|
||||
return_response: Literal[False] = False,
|
||||
) -> ResponseT: ...
|
||||
|
||||
@overload
|
||||
async def _post(
|
||||
self,
|
||||
url: str,
|
||||
model: Any,
|
||||
response_type: type[ResponseT],
|
||||
token: str,
|
||||
headers: dict[str, str] | None = None,
|
||||
return_response: Literal[True] = True,
|
||||
) -> tuple[ResponseT, httpx.Headers]: ...
|
||||
|
||||
async def _post(
|
||||
self,
|
||||
url: str,
|
||||
model: Any,
|
||||
response_type: type[ResponseT],
|
||||
token: str,
|
||||
headers: dict[str, str] | None = None,
|
||||
return_response: bool = False,
|
||||
) -> ResponseT | tuple[ResponseT, httpx.Headers]:
|
||||
if hasattr(model, "correlation_id") and not model.correlation_id:
|
||||
model.correlation_id = str(uuid4())
|
||||
|
||||
correlation_id = getattr(model, "correlation_id", None)
|
||||
if correlation_id:
|
||||
span = trace.get_current_span()
|
||||
if span and span.is_recording():
|
||||
span.set_attribute("correlation_id", correlation_id)
|
||||
logger.info(f"Purview request to {url} with correlation_id: {correlation_id}")
|
||||
|
||||
payload = model.model_dump(by_alias=True, exclude_none=True, mode="json")
|
||||
request_headers = {
|
||||
"Authorization": f"Bearer {token}",
|
||||
"User-Agent": get_user_agent(),
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
if correlation_id:
|
||||
request_headers["client-request-id"] = correlation_id
|
||||
|
||||
if headers:
|
||||
request_headers.update(headers)
|
||||
resp = await self._client.post(url, json=payload, headers=request_headers)
|
||||
|
||||
if resp.status_code in (401, 403):
|
||||
raise PurviewAuthenticationError(f"Auth failure {resp.status_code}: {resp.text}")
|
||||
if resp.status_code == 402:
|
||||
if self._settings.get("ignore_payment_required", False):
|
||||
return response_type()
|
||||
raise PurviewPaymentRequiredError(f"Payment required {resp.status_code}: {resp.text}")
|
||||
if resp.status_code == 429:
|
||||
raise PurviewRateLimitError(f"Rate limited {resp.status_code}: {resp.text}")
|
||||
if resp.status_code not in (200, 201, 202):
|
||||
raise PurviewRequestError(f"Purview request failed {resp.status_code}: {resp.text}")
|
||||
try:
|
||||
data = resp.json()
|
||||
except ValueError:
|
||||
data = {}
|
||||
|
||||
try:
|
||||
# Prefer pydantic-style model_validate if present, else fall back to constructor.
|
||||
model_validate = getattr(response_type, "model_validate", None)
|
||||
response_obj = model_validate(data) if callable(model_validate) else response_type(**data)
|
||||
|
||||
# Extract correlation_id from response headers if response object supports it
|
||||
if "client-request-id" in resp.headers and hasattr(response_obj, "correlation_id"):
|
||||
response_correlation_id = resp.headers["client-request-id"]
|
||||
response_obj.correlation_id = response_correlation_id # pyright: ignore[reportAttributeAccessIssue]
|
||||
logger.info(f"Purview response from {url} with correlation_id: {response_correlation_id}")
|
||||
|
||||
typed_response_obj = response_obj if isinstance(response_obj, response_type) else response_type(**data)
|
||||
if return_response:
|
||||
return (typed_response_obj, resp.headers)
|
||||
return typed_response_obj
|
||||
except Exception as ex:
|
||||
raise PurviewServiceError(f"Failed to deserialize Purview response: {ex}") from ex
|
||||
@@ -0,0 +1,32 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
"""Purview specific exceptions mapped to the Integration exception hierarchy."""
|
||||
|
||||
from agent_framework.exceptions import IntegrationException, IntegrationInvalidAuthException
|
||||
|
||||
__all__ = [
|
||||
"PurviewAuthenticationError",
|
||||
"PurviewPaymentRequiredError",
|
||||
"PurviewRateLimitError",
|
||||
"PurviewRequestError",
|
||||
"PurviewServiceError",
|
||||
]
|
||||
|
||||
|
||||
class PurviewServiceError(IntegrationException):
|
||||
"""Base exception for Purview errors."""
|
||||
|
||||
|
||||
class PurviewAuthenticationError(IntegrationInvalidAuthException):
|
||||
"""Authentication / authorization failure (401/403)."""
|
||||
|
||||
|
||||
class PurviewPaymentRequiredError(PurviewServiceError):
|
||||
"""Payment required (402)."""
|
||||
|
||||
|
||||
class PurviewRateLimitError(PurviewServiceError):
|
||||
"""Rate limiting or throttling (429)."""
|
||||
|
||||
|
||||
class PurviewRequestError(PurviewServiceError):
|
||||
"""Other non-success HTTP errors."""
|
||||
@@ -0,0 +1,249 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import logging
|
||||
from collections.abc import Awaitable, Callable
|
||||
from typing import Union
|
||||
|
||||
from agent_framework import AgentContext, AgentMiddleware, ChatContext, ChatMiddleware, MiddlewareTermination
|
||||
from azure.core.credentials import TokenCredential
|
||||
from azure.core.credentials_async import AsyncTokenCredential
|
||||
|
||||
from ._cache import CacheProvider
|
||||
from ._client import PurviewClient
|
||||
from ._exceptions import PurviewPaymentRequiredError
|
||||
from ._models import Activity
|
||||
from ._processor import ScopedContentProcessor
|
||||
from ._settings import PurviewSettings
|
||||
|
||||
AzureCredentialTypes = Union[TokenCredential, AsyncTokenCredential]
|
||||
AzureTokenProvider = Callable[[], Union[str, Awaitable[str]]]
|
||||
|
||||
logger = logging.getLogger("agent_framework.purview")
|
||||
|
||||
|
||||
class PurviewPolicyMiddleware(AgentMiddleware):
|
||||
"""Agent middleware that enforces Purview policies on prompt and response.
|
||||
|
||||
Accepts a TokenCredential, AsyncTokenCredential, or callable token provider.
|
||||
|
||||
Usage:
|
||||
|
||||
.. code-block:: python
|
||||
from agent_framework.microsoft import PurviewPolicyMiddleware, PurviewSettings
|
||||
from agent_framework import Agent
|
||||
|
||||
credential = ... # TokenCredential, AsyncTokenCredential, or callable
|
||||
settings = PurviewSettings(app_name="My App")
|
||||
agent = Agent(client=client, instructions="...", middleware=[PurviewPolicyMiddleware(credential, settings)])
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
credential: AzureCredentialTypes | AzureTokenProvider,
|
||||
settings: PurviewSettings,
|
||||
cache_provider: CacheProvider | None = None,
|
||||
) -> None:
|
||||
self._client = PurviewClient(credential, settings)
|
||||
self._processor = ScopedContentProcessor(self._client, settings, cache_provider)
|
||||
self._settings = settings
|
||||
|
||||
@staticmethod
|
||||
def _get_agent_session_id(context: AgentContext) -> str | None:
|
||||
"""Resolve a session/conversation id from the agent run context.
|
||||
|
||||
Resolution order:
|
||||
1. session.service_session_id
|
||||
2. First message whose additional_properties contains 'conversation_id'
|
||||
3. None: the downstream processor will generate a new UUID
|
||||
"""
|
||||
if context.session:
|
||||
service_session_id = context.session.service_session_id
|
||||
if isinstance(service_session_id, str) and service_session_id:
|
||||
return service_session_id
|
||||
|
||||
for message in context.messages:
|
||||
conversation_id = message.additional_properties.get("conversation_id")
|
||||
if conversation_id is not None:
|
||||
return str(conversation_id)
|
||||
|
||||
return None
|
||||
|
||||
async def process(
|
||||
self,
|
||||
context: AgentContext,
|
||||
call_next: Callable[[], Awaitable[None]],
|
||||
) -> None:
|
||||
resolved_user_id: str | None = None
|
||||
session_id: str | None = None
|
||||
try:
|
||||
# Pre (prompt) check
|
||||
session_id = self._get_agent_session_id(context)
|
||||
should_block_prompt, resolved_user_id = await self._processor.process_messages(
|
||||
context.messages, Activity.UPLOAD_TEXT, session_id=session_id
|
||||
)
|
||||
if should_block_prompt:
|
||||
from agent_framework import AgentResponse, Message
|
||||
|
||||
msg = self._settings.get("blocked_prompt_message", None) or "Prompt blocked by policy"
|
||||
|
||||
context.result = AgentResponse(
|
||||
messages=[
|
||||
Message(
|
||||
role="system",
|
||||
contents=[msg],
|
||||
)
|
||||
]
|
||||
)
|
||||
raise MiddlewareTermination
|
||||
except MiddlewareTermination:
|
||||
raise
|
||||
except PurviewPaymentRequiredError as ex:
|
||||
logger.error(f"Purview payment required error in policy pre-check: {ex}")
|
||||
if not self._settings.get("ignore_payment_required", False):
|
||||
raise
|
||||
except Exception as ex:
|
||||
logger.error(f"Error in Purview policy pre-check: {ex}")
|
||||
if not self._settings.get("ignore_exceptions", False):
|
||||
raise
|
||||
|
||||
await call_next()
|
||||
|
||||
try:
|
||||
# Post (response) check only if we have a normal AgentResponse
|
||||
# Use the same user_id from the request for the response evaluation
|
||||
session_id_response = self._get_agent_session_id(context)
|
||||
if session_id_response is None:
|
||||
session_id_response = session_id
|
||||
if context.result and not context.stream:
|
||||
should_block_response, _ = await self._processor.process_messages(
|
||||
context.result.messages, # type: ignore[union-attr]
|
||||
Activity.DOWNLOAD_TEXT,
|
||||
session_id=session_id_response,
|
||||
user_id=resolved_user_id,
|
||||
)
|
||||
if should_block_response:
|
||||
from agent_framework import AgentResponse, Message
|
||||
|
||||
msg = self._settings.get("blocked_response_message", None) or "Response blocked by policy"
|
||||
|
||||
context.result = AgentResponse(
|
||||
messages=[
|
||||
Message(
|
||||
role="system",
|
||||
contents=[msg],
|
||||
)
|
||||
]
|
||||
)
|
||||
else:
|
||||
# Streaming responses are not supported for post-checks
|
||||
logger.debug("Streaming responses are not supported for Purview policy post-checks")
|
||||
except PurviewPaymentRequiredError as ex:
|
||||
logger.error(f"Purview payment required error in policy post-check: {ex}")
|
||||
if not self._settings.get("ignore_payment_required", False):
|
||||
raise
|
||||
except Exception as ex:
|
||||
logger.error(f"Error in Purview policy post-check: {ex}")
|
||||
if not self._settings.get("ignore_exceptions", False):
|
||||
raise
|
||||
|
||||
|
||||
class PurviewChatPolicyMiddleware(ChatMiddleware):
|
||||
"""Chat middleware variant for Purview policy evaluation.
|
||||
|
||||
This allows users to attach Purview enforcement directly to a chat client
|
||||
|
||||
Behavior:
|
||||
* Pre-chat: evaluates outgoing (user + context) messages as an upload activity
|
||||
and can terminate execution if blocked.
|
||||
* Post-chat: evaluates the received response messages (streaming is not presently supported)
|
||||
and can replace them with a blocked message. Uses the same user_id from the request
|
||||
to ensure consistent user identity throughout the evaluation.
|
||||
|
||||
Usage:
|
||||
|
||||
.. code-block:: python
|
||||
from agent_framework.microsoft import PurviewChatPolicyMiddleware, PurviewSettings
|
||||
from agent_framework import ChatClient
|
||||
|
||||
credential = ... # TokenCredential, AsyncTokenCredential, or callable
|
||||
settings = PurviewSettings(app_name="My App")
|
||||
client = ChatClient(..., middleware=[PurviewChatPolicyMiddleware(credential, settings)])
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
credential: AzureCredentialTypes | AzureTokenProvider,
|
||||
settings: PurviewSettings,
|
||||
cache_provider: CacheProvider | None = None,
|
||||
) -> None:
|
||||
self._client = PurviewClient(credential, settings)
|
||||
self._processor = ScopedContentProcessor(self._client, settings, cache_provider)
|
||||
self._settings = settings
|
||||
|
||||
async def process(
|
||||
self,
|
||||
context: ChatContext,
|
||||
call_next: Callable[[], Awaitable[None]],
|
||||
) -> None:
|
||||
resolved_user_id: str | None = None
|
||||
session_id: str | None = None
|
||||
try:
|
||||
session_id = context.options.get("conversation_id") if context.options else None
|
||||
should_block_prompt, resolved_user_id = await self._processor.process_messages(
|
||||
context.messages, Activity.UPLOAD_TEXT, session_id=session_id
|
||||
)
|
||||
if should_block_prompt:
|
||||
from agent_framework import ChatResponse, Message
|
||||
|
||||
blocked_message = Message(
|
||||
role="system",
|
||||
contents=[self._settings.get("blocked_prompt_message", None) or "Prompt blocked by policy"],
|
||||
)
|
||||
context.result = ChatResponse(messages=[blocked_message])
|
||||
raise MiddlewareTermination
|
||||
except MiddlewareTermination:
|
||||
raise
|
||||
except PurviewPaymentRequiredError as ex:
|
||||
logger.error(f"Purview payment required error in policy pre-check: {ex}")
|
||||
if not self._settings.get("ignore_payment_required", False):
|
||||
raise
|
||||
except Exception as ex:
|
||||
logger.error(f"Error in Purview policy pre-check: {ex}")
|
||||
if not self._settings.get("ignore_exceptions", False):
|
||||
raise
|
||||
|
||||
await call_next()
|
||||
|
||||
try:
|
||||
# Post (response) evaluation only if non-streaming and we have messages result shape
|
||||
# Use the same user_id from the request for the response evaluation
|
||||
session_id_response = context.options.get("conversation_id") if context.options else None
|
||||
if session_id_response is None:
|
||||
session_id_response = session_id
|
||||
if context.result and not context.stream:
|
||||
result_obj = context.result
|
||||
messages = getattr(result_obj, "messages", None)
|
||||
if messages:
|
||||
should_block_response, _ = await self._processor.process_messages(
|
||||
messages, Activity.DOWNLOAD_TEXT, session_id=session_id_response, user_id=resolved_user_id
|
||||
)
|
||||
if should_block_response:
|
||||
from agent_framework import ChatResponse, Message
|
||||
|
||||
blocked_message = Message(
|
||||
role="system",
|
||||
contents=[
|
||||
self._settings.get("blocked_response_message", None) or "Response blocked by policy"
|
||||
],
|
||||
)
|
||||
context.result = ChatResponse(messages=[blocked_message])
|
||||
else:
|
||||
logger.debug("Streaming responses are not supported for Purview policy post-checks")
|
||||
except PurviewPaymentRequiredError as ex:
|
||||
logger.error(f"Purview payment required error in policy post-check: {ex}")
|
||||
if not self._settings.get("ignore_payment_required", False):
|
||||
raise
|
||||
except Exception as ex:
|
||||
logger.error(f"Error in Purview policy post-check: {ex}")
|
||||
if not self._settings.get("ignore_exceptions", False):
|
||||
raise
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,388 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import time
|
||||
import uuid
|
||||
from collections.abc import Iterable, MutableMapping
|
||||
from typing import Any
|
||||
|
||||
from agent_framework import Message
|
||||
|
||||
from ._cache import CacheProvider, InMemoryCacheProvider, create_protection_scopes_cache_key
|
||||
from ._client import PurviewClient
|
||||
from ._exceptions import PurviewPaymentRequiredError
|
||||
from ._models import (
|
||||
Activity,
|
||||
ActivityMetadata,
|
||||
ContentActivitiesRequest,
|
||||
ContentToProcess,
|
||||
DeviceMetadata,
|
||||
DlpAction,
|
||||
DlpActionInfo,
|
||||
ExecutionMode,
|
||||
IntegratedAppMetadata,
|
||||
OperatingSystemSpecifications,
|
||||
PolicyLocation,
|
||||
ProcessContentRequest,
|
||||
ProcessContentResponse,
|
||||
ProcessConversationMetadata,
|
||||
ProtectedAppMetadata,
|
||||
ProtectionScopesRequest,
|
||||
ProtectionScopesResponse,
|
||||
ProtectionScopeState,
|
||||
PurviewTextContent,
|
||||
RestrictionAction,
|
||||
translate_activity,
|
||||
)
|
||||
from ._settings import PurviewSettings
|
||||
|
||||
logger = logging.getLogger("agent_framework.purview")
|
||||
|
||||
|
||||
def _is_valid_guid(value: str | None) -> bool:
|
||||
"""Check if a string is a valid GUID/UUID format using uuid module."""
|
||||
if not value:
|
||||
return False
|
||||
try:
|
||||
uuid.UUID(value)
|
||||
return True
|
||||
except (ValueError, AttributeError):
|
||||
return False
|
||||
|
||||
|
||||
class ScopedContentProcessor:
|
||||
"""Combine protection scopes, process content, and content activities logic."""
|
||||
|
||||
def __init__(self, client: PurviewClient, settings: PurviewSettings, cache_provider: CacheProvider | None = None):
|
||||
self._client = client
|
||||
self._settings = settings
|
||||
cache_ttl = settings.get("cache_ttl_seconds")
|
||||
max_cache = settings.get("max_cache_size_bytes")
|
||||
self._cache: CacheProvider = cache_provider or InMemoryCacheProvider(
|
||||
default_ttl_seconds=cache_ttl if cache_ttl is not None else 14400,
|
||||
max_size_bytes=max_cache if max_cache is not None else 200 * 1024 * 1024,
|
||||
)
|
||||
self._background_tasks: set[asyncio.Task[Any]] = set()
|
||||
|
||||
async def process_messages(
|
||||
self,
|
||||
messages: Iterable[Message],
|
||||
activity: Activity,
|
||||
session_id: str | None = None,
|
||||
user_id: str | None = None,
|
||||
) -> tuple[bool, str | None]:
|
||||
"""Process messages for policy evaluation.
|
||||
|
||||
Args:
|
||||
messages: The messages to process
|
||||
activity: The activity type (e.g., UPLOAD_TEXT)
|
||||
session_id: Optional session/conversation id. Else, a new GUID is generated.
|
||||
user_id: Optional user_id to use for all messages. If provided, this is the fallback.
|
||||
|
||||
Returns:
|
||||
A tuple of (should_block: bool, resolved_user_id: str | None).
|
||||
The resolved_user_id can be stored and passed back when processing the response
|
||||
to ensure the same user context is maintained throughout the request/response cycle.
|
||||
"""
|
||||
pc_requests, resolved_user_id = await self._map_messages(messages, activity, session_id, user_id)
|
||||
should_block = False
|
||||
for req in pc_requests:
|
||||
resp = await self._process_with_scopes(req)
|
||||
if resp.policy_actions:
|
||||
for act in resp.policy_actions:
|
||||
if act.action == DlpAction.BLOCK_ACCESS or act.restriction_action == RestrictionAction.BLOCK:
|
||||
should_block = True
|
||||
break
|
||||
if should_block:
|
||||
break
|
||||
return should_block, resolved_user_id
|
||||
|
||||
async def _map_messages(
|
||||
self,
|
||||
messages: Iterable[Message],
|
||||
activity: Activity,
|
||||
session_id: str | None = None,
|
||||
provided_user_id: str | None = None,
|
||||
) -> tuple[list[ProcessContentRequest], str | None]:
|
||||
"""Map messages to ProcessContentRequests.
|
||||
|
||||
Args:
|
||||
messages: The messages to map
|
||||
activity: The activity type
|
||||
session_id: Optional session/conversation id to use for correlation
|
||||
provided_user_id: Optional user_id to use. If provided, this is the fallback.
|
||||
|
||||
Returns:
|
||||
A tuple of (requests, resolved_user_id)
|
||||
"""
|
||||
results: list[ProcessContentRequest] = []
|
||||
token_info = await self._client.get_user_info_from_token(tenant_id=self._settings.get("tenant_id"))
|
||||
|
||||
tenant_id = (token_info or {}).get("tenant_id") or self._settings.get("tenant_id")
|
||||
if not tenant_id or not _is_valid_guid(tenant_id):
|
||||
raise ValueError("Tenant id required or must be inferable from credential")
|
||||
|
||||
resolved_user_id = (token_info or {}).get("user_id")
|
||||
resolved_author_name = None
|
||||
if not resolved_user_id:
|
||||
resolved_user_id = provided_user_id if provided_user_id and _is_valid_guid(provided_user_id) else None
|
||||
|
||||
if not resolved_user_id:
|
||||
for m in messages:
|
||||
if m.additional_properties:
|
||||
potential_user_id = m.additional_properties.get("user_id")
|
||||
if _is_valid_guid(potential_user_id):
|
||||
resolved_user_id = potential_user_id
|
||||
break
|
||||
if m.author_name and _is_valid_guid(m.author_name) and not resolved_author_name:
|
||||
resolved_author_name = m.author_name
|
||||
|
||||
if not resolved_user_id and resolved_author_name:
|
||||
resolved_user_id = resolved_author_name
|
||||
|
||||
# Return empty results if user_id is empty
|
||||
if not resolved_user_id or not _is_valid_guid(resolved_user_id):
|
||||
return results, None
|
||||
|
||||
for m in messages:
|
||||
message_id = m.message_id or str(uuid.uuid4())
|
||||
content = PurviewTextContent(data=m.text or "")
|
||||
correlation_id = (session_id or str(uuid.uuid4())) + "@AF"
|
||||
meta = ProcessConversationMetadata(
|
||||
identifier=message_id,
|
||||
content=content,
|
||||
name=f"Agent Framework Message {message_id}",
|
||||
is_truncated=False,
|
||||
correlation_id=correlation_id,
|
||||
# This would be c# ticks equivalent and needs to fit inside c# long
|
||||
sequence_number=time.time_ns() // 100 + 621355968000000000,
|
||||
)
|
||||
activity_meta = ActivityMetadata(activity=activity)
|
||||
|
||||
purview_app_location = self._settings.get("purview_app_location")
|
||||
if purview_app_location:
|
||||
policy_location = PolicyLocation(
|
||||
data_type=purview_app_location.get_policy_location()["@odata.type"],
|
||||
value=purview_app_location.location_value,
|
||||
)
|
||||
elif token_info and token_info.get("client_id"):
|
||||
policy_location = PolicyLocation(
|
||||
data_type="microsoft.graph.policyLocationApplication",
|
||||
value=token_info["client_id"],
|
||||
)
|
||||
else:
|
||||
raise ValueError("App location not provided or inferable")
|
||||
|
||||
app_name = self._settings.get("app_name") or "Unknown"
|
||||
protected_app = ProtectedAppMetadata(
|
||||
name=app_name,
|
||||
version=self._settings.get("app_version", "Unknown"),
|
||||
application_location=policy_location,
|
||||
)
|
||||
integrated_app = IntegratedAppMetadata(name=app_name, version=self._settings.get("app_version", "Unknown"))
|
||||
device_meta = DeviceMetadata(
|
||||
operating_system_specifications=OperatingSystemSpecifications(
|
||||
operating_system_platform="Unknown", operating_system_version="Unknown"
|
||||
)
|
||||
)
|
||||
|
||||
ctp = ContentToProcess(
|
||||
content_entries=[meta],
|
||||
activity_metadata=activity_meta,
|
||||
device_metadata=device_meta,
|
||||
integrated_app_metadata=integrated_app,
|
||||
protected_app_metadata=protected_app,
|
||||
)
|
||||
req = ProcessContentRequest(
|
||||
content_to_process=ctp,
|
||||
user_id=resolved_user_id, # Use the resolved user_id for all messages
|
||||
tenant_id=tenant_id,
|
||||
correlation_id=meta.correlation_id,
|
||||
process_inline=None, # Will be set based on execution mode
|
||||
)
|
||||
results.append(req)
|
||||
return results, resolved_user_id
|
||||
|
||||
async def _process_with_scopes(self, pc_request: ProcessContentRequest) -> ProcessContentResponse:
|
||||
app_location = pc_request.content_to_process.protected_app_metadata.application_location
|
||||
locations: list[PolicyLocation | MutableMapping[str, Any]] = [app_location] if app_location is not None else []
|
||||
|
||||
ps_req = ProtectionScopesRequest(
|
||||
user_id=pc_request.user_id,
|
||||
tenant_id=pc_request.tenant_id,
|
||||
activities=translate_activity(pc_request.content_to_process.activity_metadata.activity),
|
||||
locations=locations,
|
||||
device_metadata=pc_request.content_to_process.device_metadata,
|
||||
integrated_app_metadata=pc_request.content_to_process.integrated_app_metadata,
|
||||
correlation_id=pc_request.correlation_id,
|
||||
)
|
||||
|
||||
# Check for tenant-level 402 exception cache first
|
||||
tenant_payment_cache_key = f"purview:payment_required:{pc_request.tenant_id}"
|
||||
cached_payment_exception = await self._cache.get(tenant_payment_cache_key)
|
||||
if isinstance(cached_payment_exception, PurviewPaymentRequiredError):
|
||||
raise cached_payment_exception
|
||||
|
||||
cache_key = create_protection_scopes_cache_key(ps_req)
|
||||
cached_ps_resp = await self._cache.get(cache_key)
|
||||
|
||||
if cached_ps_resp is not None and isinstance(cached_ps_resp, ProtectionScopesResponse):
|
||||
return await self._process_with_cached_scopes(pc_request, cached_ps_resp, cache_key)
|
||||
|
||||
task = asyncio.create_task(self._refresh_protection_scopes_background(ps_req, cache_key, pc_request))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
return await self._call_process_content(pc_request, cache_key, dlp_actions=[])
|
||||
|
||||
async def _process_with_cached_scopes(
|
||||
self,
|
||||
pc_request: ProcessContentRequest,
|
||||
ps_resp: ProtectionScopesResponse,
|
||||
cache_key: str,
|
||||
) -> ProcessContentResponse:
|
||||
if ps_resp.scope_identifier:
|
||||
pc_request.scope_identifier = ps_resp.scope_identifier
|
||||
|
||||
should_process, dlp_actions, execution_mode = self._check_applicable_scopes(pc_request, ps_resp)
|
||||
|
||||
if should_process:
|
||||
# Set process_inline based on execution mode
|
||||
pc_request.process_inline = execution_mode == ExecutionMode.EVALUATE_INLINE
|
||||
|
||||
# If execution mode is offline, queue the PC request in background
|
||||
if execution_mode != ExecutionMode.EVALUATE_INLINE:
|
||||
task = asyncio.create_task(self._process_content_background(pc_request, cache_key))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
return ProcessContentResponse(id="204", correlation_id=pc_request.correlation_id)
|
||||
|
||||
return await self._call_process_content(pc_request, cache_key, dlp_actions=dlp_actions)
|
||||
|
||||
# No applicable scopes - send content activities in background
|
||||
ca_req = ContentActivitiesRequest(
|
||||
user_id=pc_request.user_id,
|
||||
tenant_id=pc_request.tenant_id,
|
||||
content_to_process=pc_request.content_to_process,
|
||||
correlation_id=pc_request.correlation_id,
|
||||
)
|
||||
|
||||
task = asyncio.create_task(self._send_content_activities_background(ca_req))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
# Respond with HttpStatusCode 204(No Content)
|
||||
return ProcessContentResponse(id="204", correlation_id=pc_request.correlation_id)
|
||||
|
||||
async def _call_process_content(
|
||||
self,
|
||||
pc_request: ProcessContentRequest,
|
||||
cache_key: str,
|
||||
dlp_actions: list[DlpActionInfo],
|
||||
) -> ProcessContentResponse:
|
||||
pc_resp = await self._client.process_content(pc_request)
|
||||
|
||||
if pc_request.scope_identifier and pc_resp.protection_scope_state == ProtectionScopeState.MODIFIED:
|
||||
await self._cache.remove(cache_key)
|
||||
|
||||
if dlp_actions:
|
||||
pc_resp.policy_actions = self._combine_policy_actions(pc_resp.policy_actions, dlp_actions)
|
||||
return pc_resp
|
||||
|
||||
async def _refresh_protection_scopes_background(
|
||||
self, ps_req: ProtectionScopesRequest, cache_key: str, pc_request: ProcessContentRequest
|
||||
) -> None:
|
||||
"""Fetch protection scopes and warm the cache without blocking the foreground call."""
|
||||
ttl = self._settings.get("cache_ttl_seconds")
|
||||
ttl_seconds = ttl if ttl is not None else 14400
|
||||
try:
|
||||
ps_resp = await self._client.get_protection_scopes(ps_req)
|
||||
await self._cache.set(cache_key, ps_resp, ttl_seconds=ttl_seconds)
|
||||
should_process, _, _ = self._check_applicable_scopes(pc_request, ps_resp)
|
||||
if not should_process:
|
||||
ca_req = ContentActivitiesRequest(
|
||||
user_id=pc_request.user_id,
|
||||
tenant_id=pc_request.tenant_id,
|
||||
content_to_process=pc_request.content_to_process,
|
||||
correlation_id=pc_request.correlation_id,
|
||||
)
|
||||
await self._send_content_activities_background(ca_req)
|
||||
except PurviewPaymentRequiredError as ex:
|
||||
tenant_payment_cache_key = f"purview:payment_required:{ps_req.tenant_id}"
|
||||
await self._cache.set(tenant_payment_cache_key, ex, ttl_seconds=ttl_seconds)
|
||||
logger.warning("Background protection scopes refresh failed with payment required: %s", ex)
|
||||
except Exception as ex:
|
||||
logger.warning("Background protection scopes refresh failed: %s", ex)
|
||||
|
||||
async def _process_content_background(self, pc_request: ProcessContentRequest, cache_key: str) -> None:
|
||||
"""Process content in background for offline execution mode."""
|
||||
try:
|
||||
pc_resp = await self._client.process_content(pc_request)
|
||||
|
||||
# If protection scopes changed, invalidate cache and retry once.
|
||||
if pc_request.scope_identifier and pc_resp.protection_scope_state == ProtectionScopeState.MODIFIED:
|
||||
await self._cache.remove(cache_key)
|
||||
await self._client.process_content(pc_request)
|
||||
except Exception as ex:
|
||||
# Log errors but don't propagate since this is fire-and-forget
|
||||
logger.warning(f"Background process content request failed: {ex}")
|
||||
|
||||
async def _send_content_activities_background(self, ca_req: ContentActivitiesRequest) -> None:
|
||||
"""Send content activities in background without blocking."""
|
||||
try:
|
||||
await self._client.send_content_activities(ca_req)
|
||||
except Exception as ex:
|
||||
# Log errors but don't propagate since this is fire-and-forget
|
||||
logger.warning(f"Background content activities request failed: {ex}")
|
||||
|
||||
@staticmethod
|
||||
def _combine_policy_actions(
|
||||
existing: list[DlpActionInfo] | None, new_actions: list[DlpActionInfo]
|
||||
) -> list[DlpActionInfo]:
|
||||
combined: dict[tuple[DlpAction | None, RestrictionAction | None], DlpActionInfo] = {}
|
||||
for action_info in (existing or []) + new_actions:
|
||||
combined.setdefault((action_info.action, action_info.restriction_action), action_info)
|
||||
return list(combined.values())
|
||||
|
||||
@staticmethod
|
||||
def _check_applicable_scopes(
|
||||
pc_request: ProcessContentRequest, ps_response: ProtectionScopesResponse
|
||||
) -> tuple[bool, list[DlpActionInfo], ExecutionMode]:
|
||||
"""Check if any scopes are applicable to the request.
|
||||
|
||||
Args:
|
||||
pc_request: The process content request
|
||||
ps_response: The protection scopes response
|
||||
|
||||
Returns:
|
||||
A tuple of (should_process, dlp_actions, execution_mode)
|
||||
"""
|
||||
req_activity = translate_activity(pc_request.content_to_process.activity_metadata.activity)
|
||||
location = pc_request.content_to_process.protected_app_metadata.application_location
|
||||
should_process: bool = False
|
||||
dlp_actions: list[DlpActionInfo] = []
|
||||
execution_mode: ExecutionMode = ExecutionMode.EVALUATE_OFFLINE # Default to offline
|
||||
|
||||
for scope in ps_response.scopes or []:
|
||||
# Check if all activities in req_activity are present in scope.activities using bitwise flags.
|
||||
activity_match = bool(scope.activities and (scope.activities & req_activity) == req_activity)
|
||||
location_match = False
|
||||
if location is not None:
|
||||
for loc in scope.locations or []:
|
||||
if (
|
||||
loc.data_type
|
||||
and location.data_type
|
||||
and loc.data_type.lower().endswith(location.data_type.split(".")[-1].lower())
|
||||
and loc.value == location.value
|
||||
):
|
||||
location_match = True
|
||||
break
|
||||
if activity_match and location_match:
|
||||
should_process = True
|
||||
|
||||
# If any scope has EvaluateInline, upgrade to inline mode
|
||||
if scope.execution_mode == ExecutionMode.EVALUATE_INLINE:
|
||||
execution_mode = ExecutionMode.EVALUATE_INLINE
|
||||
|
||||
if scope.policy_actions:
|
||||
dlp_actions.extend(scope.policy_actions)
|
||||
return should_process, dlp_actions, execution_mode
|
||||
@@ -0,0 +1,84 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import sys
|
||||
from enum import Enum
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
if sys.version_info >= (3, 11):
|
||||
from typing import TypedDict # pragma: no cover
|
||||
else:
|
||||
from typing_extensions import TypedDict # pragma: no cover
|
||||
|
||||
|
||||
class PurviewLocationType(str, Enum):
|
||||
"""The type of location for Purview policy evaluation."""
|
||||
|
||||
APPLICATION = "application"
|
||||
URI = "uri"
|
||||
DOMAIN = "domain"
|
||||
|
||||
|
||||
class PurviewAppLocation(BaseModel):
|
||||
"""Identifier representing the app's location for Purview policy evaluation."""
|
||||
|
||||
location_type: PurviewLocationType
|
||||
location_value: str
|
||||
|
||||
def get_policy_location(self) -> dict[str, str]:
|
||||
ns = "microsoft.graph"
|
||||
if self.location_type == PurviewLocationType.APPLICATION:
|
||||
dt = f"{ns}.policyLocationApplication"
|
||||
elif self.location_type == PurviewLocationType.URI:
|
||||
dt = f"{ns}.policyLocationUrl"
|
||||
elif self.location_type == PurviewLocationType.DOMAIN:
|
||||
dt = f"{ns}.policyLocationDomain"
|
||||
else: # pragma: no cover - defensive
|
||||
raise ValueError("Invalid Purview location type")
|
||||
return {"@odata.type": dt, "value": self.location_value}
|
||||
|
||||
|
||||
class PurviewSettings(TypedDict, total=False):
|
||||
"""Settings for Purview integration mirroring .NET PurviewSettings.
|
||||
|
||||
Attributes:
|
||||
app_name: Public app name.
|
||||
app_version: Optional version string of the application.
|
||||
tenant_id: Optional tenant id (guid) of the user making the request.
|
||||
purview_app_location: Optional app location for policy evaluation.
|
||||
graph_base_uri: Base URI for Microsoft Graph.
|
||||
blocked_prompt_message: Custom message to return when a prompt is blocked by policy.
|
||||
blocked_response_message: Custom message to return when a response is blocked by policy.
|
||||
ignore_exceptions: If True, all Purview exceptions will be logged but not thrown in middleware.
|
||||
ignore_payment_required: If True, 402 payment required errors will be logged but not thrown.
|
||||
cache_ttl_seconds: Time to live for cache entries in seconds (default 14400 = 4 hours).
|
||||
max_cache_size_bytes: Maximum cache size in bytes (default 200MB).
|
||||
"""
|
||||
|
||||
app_name: str | None
|
||||
app_version: str | None
|
||||
tenant_id: str | None
|
||||
purview_app_location: PurviewAppLocation | None
|
||||
graph_base_uri: str | None
|
||||
blocked_prompt_message: str | None
|
||||
blocked_response_message: str | None
|
||||
ignore_exceptions: bool | None
|
||||
ignore_payment_required: bool | None
|
||||
cache_ttl_seconds: int | None
|
||||
max_cache_size_bytes: int | None
|
||||
|
||||
|
||||
def get_purview_scopes(settings: PurviewSettings) -> list[str]:
|
||||
"""Get the OAuth scopes for the Purview Graph API.
|
||||
|
||||
Args:
|
||||
settings: The Purview settings containing graph_base_uri.
|
||||
|
||||
Returns:
|
||||
A list of OAuth scope strings.
|
||||
"""
|
||||
from urllib.parse import urlparse
|
||||
|
||||
graph_base_uri = settings.get("graph_base_uri", "https://graph.microsoft.com/v1.0/")
|
||||
host = urlparse(str(graph_base_uri)).hostname or "graph.microsoft.com"
|
||||
return [f"https://{host}/.default"]
|
||||
@@ -0,0 +1,97 @@
|
||||
[project]
|
||||
name = "agent-framework-purview"
|
||||
description = "Microsoft Purview (Graph dataSecurityAndGovernance) integration for Microsoft Agent Framework."
|
||||
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
version = "1.0.0b260709"
|
||||
license-files = ["LICENSE"]
|
||||
urls.homepage = "https://github.com/microsoft/agent-framework"
|
||||
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
|
||||
urls.release_notes = "https://github.com/microsoft/agent-framework/releases"
|
||||
urls.issues = "https://github.com/microsoft/agent-framework/issues"
|
||||
classifiers = [
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Development Status :: 4 - Beta",
|
||||
"Intended Audience :: Developers",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Programming Language :: Python :: 3.14",
|
||||
"Framework :: Pydantic :: 2",
|
||||
"Typing :: Typed",
|
||||
]
|
||||
dependencies = [
|
||||
"agent-framework-core>=1.11.0,<2",
|
||||
"azure-core>=1.30.0,<2",
|
||||
"httpx>=0.27.0,<0.29",
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
prerelease = "if-necessary-or-explicit"
|
||||
environments = [
|
||||
"sys_platform == 'darwin'",
|
||||
"sys_platform == 'linux'",
|
||||
"sys_platform == 'win32'"
|
||||
]
|
||||
|
||||
[tool.uv-dynamic-versioning]
|
||||
fallback-version = "0.0.0"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = 'tests'
|
||||
addopts = "-ra -q -r fEX"
|
||||
asyncio_mode = "auto"
|
||||
asyncio_default_fixture_loop_scope = "function"
|
||||
filterwarnings = []
|
||||
markers = [
|
||||
"integration: marks tests as integration tests that require external services",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
extend = "../../pyproject.toml"
|
||||
|
||||
[tool.coverage.run]
|
||||
omit = [
|
||||
"**/__init__.py"
|
||||
]
|
||||
|
||||
[tool.pyright]
|
||||
extends = "../../pyproject.toml"
|
||||
include = ["agent_framework_purview"]
|
||||
|
||||
[tool.mypy]
|
||||
plugins = ['pydantic.mypy']
|
||||
strict = true
|
||||
python_version = "3.10"
|
||||
ignore_missing_imports = true
|
||||
disallow_untyped_defs = true
|
||||
no_implicit_optional = true
|
||||
check_untyped_defs = true
|
||||
warn_return_any = true
|
||||
show_error_codes = true
|
||||
warn_unused_ignores = false
|
||||
disallow_incomplete_defs = true
|
||||
disallow_untyped_decorators = true
|
||||
|
||||
[tool.bandit]
|
||||
targets = ["agent_framework_purview"]
|
||||
exclude_dirs = ["tests"]
|
||||
|
||||
[tool.poe]
|
||||
executor.type = "uv"
|
||||
include = "../../shared_tasks.toml"
|
||||
|
||||
[tool.poe.tasks.mypy]
|
||||
help = "Run MyPy for this package."
|
||||
cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_purview"
|
||||
|
||||
[tool.poe.tasks.test]
|
||||
help = "Run the default unit test suite for this package."
|
||||
cmd = 'pytest -m "not integration" --cov=agent_framework_purview --cov-report=term-missing:skip-covered tests'
|
||||
|
||||
[build-system]
|
||||
requires = ["flit-core >= 3.9,<4.0"]
|
||||
build-backend = "flit_core.buildapi"
|
||||
@@ -0,0 +1,68 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
"""Shared pytest fixtures for Purview tests."""
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_framework_purview._models import (
|
||||
Activity,
|
||||
ActivityMetadata,
|
||||
ContentToProcess,
|
||||
DeviceMetadata,
|
||||
IntegratedAppMetadata,
|
||||
OperatingSystemSpecifications,
|
||||
PolicyLocation,
|
||||
ProcessContentRequest,
|
||||
ProcessConversationMetadata,
|
||||
ProtectedAppMetadata,
|
||||
PurviewTextContent,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def content_to_process_factory():
|
||||
"""Factory fixture to create ContentToProcess objects with test data."""
|
||||
|
||||
def _create_content(text: str = "Test") -> ContentToProcess:
|
||||
text_content = PurviewTextContent(data=text)
|
||||
metadata = ProcessConversationMetadata(
|
||||
identifier="msg-1",
|
||||
content=text_content,
|
||||
name="Test",
|
||||
is_truncated=False,
|
||||
)
|
||||
activity_meta = ActivityMetadata(activity=Activity.UPLOAD_TEXT)
|
||||
device_meta = DeviceMetadata(
|
||||
operating_system_specifications=OperatingSystemSpecifications(
|
||||
operating_system_platform="Windows", operating_system_version="10"
|
||||
)
|
||||
)
|
||||
integrated_app = IntegratedAppMetadata(name="App", version="1.0")
|
||||
location = PolicyLocation(data_type="microsoft.graph.policyLocationApplication", value="app-id")
|
||||
protected_app = ProtectedAppMetadata(name="Protected", version="1.0", application_location=location)
|
||||
|
||||
return ContentToProcess(
|
||||
content_entries=[metadata],
|
||||
activity_metadata=activity_meta,
|
||||
device_metadata=device_meta,
|
||||
integrated_app_metadata=integrated_app,
|
||||
protected_app_metadata=protected_app,
|
||||
)
|
||||
|
||||
return _create_content
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def process_content_request_factory(content_to_process_factory):
|
||||
"""Factory fixture to create ProcessContentRequest objects with test data."""
|
||||
|
||||
def _create_request(
|
||||
text: str = "Test", user_id: str = "user-123", tenant_id: str = "tenant-456"
|
||||
) -> ProcessContentRequest:
|
||||
content = content_to_process_factory(text)
|
||||
return ProcessContentRequest(
|
||||
content_to_process=content,
|
||||
user_id=user_id,
|
||||
tenant_id=tenant_id,
|
||||
)
|
||||
|
||||
return _create_request
|
||||
@@ -0,0 +1,215 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for Purview cache provider."""
|
||||
|
||||
import asyncio
|
||||
|
||||
from agent_framework_purview._cache import (
|
||||
InMemoryCacheProvider,
|
||||
create_protection_scopes_cache_key,
|
||||
)
|
||||
from agent_framework_purview._models import PolicyLocation, ProtectionScopesRequest
|
||||
|
||||
|
||||
class TestInMemoryCacheProvider:
|
||||
"""Test InMemoryCacheProvider functionality."""
|
||||
|
||||
async def test_cache_set_and_get(self) -> None:
|
||||
"""Test basic set and get operations."""
|
||||
cache = InMemoryCacheProvider()
|
||||
|
||||
await cache.set("key1", "value1")
|
||||
result = await cache.get("key1")
|
||||
|
||||
assert result == "value1"
|
||||
|
||||
async def test_cache_get_nonexistent_key(self) -> None:
|
||||
"""Test get returns None for non-existent key."""
|
||||
cache = InMemoryCacheProvider()
|
||||
|
||||
result = await cache.get("nonexistent")
|
||||
|
||||
assert result is None
|
||||
|
||||
async def test_cache_expiration(self) -> None:
|
||||
"""Test that cached values expire after TTL."""
|
||||
cache = InMemoryCacheProvider(default_ttl_seconds=1)
|
||||
|
||||
await cache.set("key1", "value1")
|
||||
result = await cache.get("key1")
|
||||
assert result == "value1"
|
||||
|
||||
await asyncio.sleep(1.1)
|
||||
result = await cache.get("key1")
|
||||
assert result is None
|
||||
|
||||
async def test_cache_custom_ttl(self) -> None:
|
||||
"""Test that custom TTL overrides default."""
|
||||
cache = InMemoryCacheProvider(default_ttl_seconds=10)
|
||||
|
||||
await cache.set("key1", "value1", ttl_seconds=1)
|
||||
result = await cache.get("key1")
|
||||
assert result == "value1"
|
||||
|
||||
await asyncio.sleep(1.1)
|
||||
result = await cache.get("key1")
|
||||
assert result is None
|
||||
|
||||
async def test_cache_update_existing_key(self) -> None:
|
||||
"""Test updating an existing cache entry."""
|
||||
cache = InMemoryCacheProvider()
|
||||
|
||||
await cache.set("key1", "value1")
|
||||
await cache.set("key1", "value2")
|
||||
result = await cache.get("key1")
|
||||
|
||||
assert result == "value2"
|
||||
|
||||
async def test_cache_remove(self) -> None:
|
||||
"""Test removing a cache entry."""
|
||||
cache = InMemoryCacheProvider()
|
||||
|
||||
await cache.set("key1", "value1")
|
||||
await cache.remove("key1")
|
||||
result = await cache.get("key1")
|
||||
|
||||
assert result is None
|
||||
|
||||
async def test_cache_remove_nonexistent_key(self) -> None:
|
||||
"""Test removing non-existent key does not raise error."""
|
||||
cache = InMemoryCacheProvider()
|
||||
|
||||
await cache.remove("nonexistent")
|
||||
|
||||
async def test_cache_size_limit_eviction(self) -> None:
|
||||
"""Test that cache evicts old entries when size limit is reached."""
|
||||
cache = InMemoryCacheProvider(max_size_bytes=200)
|
||||
|
||||
await cache.set("key1", "a" * 50)
|
||||
await cache.set("key2", "b" * 50)
|
||||
await cache.set("key3", "c" * 50)
|
||||
|
||||
await cache.set("key4", "d" * 100)
|
||||
|
||||
result1 = await cache.get("key1")
|
||||
assert result1 is None
|
||||
|
||||
async def test_estimate_size_with_pydantic_model(self) -> None:
|
||||
"""Test size estimation with Pydantic models."""
|
||||
cache = InMemoryCacheProvider()
|
||||
|
||||
location = PolicyLocation(**{"@odata.type": "microsoft.graph.policyLocationApplication", "value": "app-id"})
|
||||
request = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1", locations=[location])
|
||||
|
||||
await cache.set("key1", request)
|
||||
result = await cache.get("key1")
|
||||
|
||||
assert result == request
|
||||
|
||||
async def test_estimate_size_fallback(self) -> None:
|
||||
"""Test size estimation fallback for non-serializable objects."""
|
||||
cache = InMemoryCacheProvider()
|
||||
|
||||
class CustomObject:
|
||||
pass
|
||||
|
||||
obj = CustomObject()
|
||||
await cache.set("key1", obj)
|
||||
result = await cache.get("key1")
|
||||
|
||||
assert result == obj
|
||||
|
||||
async def test_estimate_size_conservative_fallback_when_all_size_methods_fail(self, monkeypatch) -> None:
|
||||
"""Test that the cache returns a conservative size estimate when all strategies fail."""
|
||||
cache = InMemoryCacheProvider()
|
||||
|
||||
class BadString:
|
||||
def __str__(self) -> str:
|
||||
raise RuntimeError("boom")
|
||||
|
||||
def raise_getsizeof(_: object) -> int:
|
||||
raise RuntimeError("no sizeof")
|
||||
|
||||
monkeypatch.setattr("agent_framework_purview._cache.sys.getsizeof", raise_getsizeof)
|
||||
|
||||
# Arrange/Act
|
||||
size = cache._estimate_size(BadString())
|
||||
|
||||
# Assert
|
||||
assert size == 1024
|
||||
|
||||
async def test_cache_multiple_updates(self) -> None:
|
||||
"""Test that updating a key multiple times maintains correct size tracking."""
|
||||
cache = InMemoryCacheProvider(max_size_bytes=1000)
|
||||
|
||||
await cache.set("key1", "a" * 100)
|
||||
initial_size = cache._current_size_bytes
|
||||
|
||||
await cache.set("key1", "b" * 200)
|
||||
|
||||
assert cache._current_size_bytes != initial_size
|
||||
|
||||
async def test_eviction_with_stale_heap_entries(self) -> None:
|
||||
"""Test that eviction correctly handles stale heap entries."""
|
||||
cache = InMemoryCacheProvider(max_size_bytes=500)
|
||||
|
||||
await cache.set("key1", "a" * 100, ttl_seconds=10)
|
||||
await cache.set("key2", "b" * 100, ttl_seconds=10)
|
||||
await cache.set("key1", "c" * 100, ttl_seconds=20)
|
||||
|
||||
await cache.set("key3", "d" * 300)
|
||||
|
||||
result = await cache.get("key1")
|
||||
assert result is not None
|
||||
|
||||
|
||||
class TestCreateProtectionScopesCacheKey:
|
||||
"""Test cache key generation for ProtectionScopesRequest."""
|
||||
|
||||
def test_cache_key_deterministic(self) -> None:
|
||||
"""Test that same request generates same cache key."""
|
||||
location = PolicyLocation(**{"@odata.type": "microsoft.graph.policyLocationApplication", "value": "app-id"})
|
||||
request1 = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1", locations=[location])
|
||||
request2 = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1", locations=[location])
|
||||
|
||||
key1 = create_protection_scopes_cache_key(request1)
|
||||
key2 = create_protection_scopes_cache_key(request2)
|
||||
|
||||
assert key1 == key2
|
||||
|
||||
def test_cache_key_different_for_different_requests(self) -> None:
|
||||
"""Test that different requests generate different cache keys."""
|
||||
location1 = PolicyLocation(**{"@odata.type": "microsoft.graph.policyLocationApplication", "value": "app-id1"})
|
||||
location2 = PolicyLocation(**{"@odata.type": "microsoft.graph.policyLocationApplication", "value": "app-id2"})
|
||||
request1 = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1", locations=[location1])
|
||||
request2 = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1", locations=[location2])
|
||||
|
||||
key1 = create_protection_scopes_cache_key(request1)
|
||||
key2 = create_protection_scopes_cache_key(request2)
|
||||
|
||||
assert key1 != key2
|
||||
|
||||
def test_cache_key_excludes_correlation_id(self) -> None:
|
||||
"""Test that correlation_id is excluded from cache key."""
|
||||
location = PolicyLocation(**{"@odata.type": "microsoft.graph.policyLocationApplication", "value": "app-id"})
|
||||
request1 = ProtectionScopesRequest(
|
||||
user_id="user1", tenant_id="tenant1", locations=[location], correlation_id="corr1"
|
||||
)
|
||||
request2 = ProtectionScopesRequest(
|
||||
user_id="user1", tenant_id="tenant1", locations=[location], correlation_id="corr2"
|
||||
)
|
||||
|
||||
key1 = create_protection_scopes_cache_key(request1)
|
||||
key2 = create_protection_scopes_cache_key(request2)
|
||||
|
||||
assert key1 == key2
|
||||
|
||||
def test_cache_key_format(self) -> None:
|
||||
"""Test that cache key has expected format."""
|
||||
location = PolicyLocation(**{"@odata.type": "microsoft.graph.policyLocationApplication", "value": "app-id"})
|
||||
request = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1", locations=[location])
|
||||
|
||||
key = create_protection_scopes_cache_key(request)
|
||||
|
||||
assert key.startswith("purview:protection_scopes:")
|
||||
assert len(key) > len("purview:protection_scopes:")
|
||||
@@ -0,0 +1,425 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
"""Tests for Purview chat middleware."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, cast
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from agent_framework import ChatContext, ChatResponse, Message, MiddlewareTermination
|
||||
from azure.core.credentials import AccessToken
|
||||
|
||||
from agent_framework_purview import PurviewChatPolicyMiddleware, PurviewSettings
|
||||
from agent_framework_purview._models import Activity
|
||||
|
||||
|
||||
@dataclass
|
||||
class DummyChatClient:
|
||||
name: str = "dummy"
|
||||
|
||||
|
||||
class TestPurviewChatPolicyMiddleware:
|
||||
@pytest.fixture
|
||||
def mock_credential(self) -> AsyncMock:
|
||||
credential = AsyncMock()
|
||||
credential.get_token = AsyncMock(return_value=AccessToken("fake-token", 9999999999))
|
||||
return credential
|
||||
|
||||
@pytest.fixture
|
||||
def settings(self) -> PurviewSettings:
|
||||
return PurviewSettings(app_name="Test App", tenant_id="test-tenant")
|
||||
|
||||
@pytest.fixture
|
||||
def middleware(self, mock_credential: AsyncMock, settings: PurviewSettings) -> PurviewChatPolicyMiddleware:
|
||||
return PurviewChatPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
@pytest.fixture
|
||||
def chat_context(self) -> ChatContext:
|
||||
client = DummyChatClient()
|
||||
chat_options = MagicMock()
|
||||
chat_options.model = "test-model"
|
||||
return ChatContext(
|
||||
client=cast(Any, client), messages=[Message(role="user", contents=["Hello"])], options=chat_options
|
||||
)
|
||||
|
||||
async def test_initialization(self, middleware: PurviewChatPolicyMiddleware) -> None:
|
||||
assert middleware._client is not None
|
||||
assert middleware._processor is not None
|
||||
|
||||
async def test_allows_clean_prompt(
|
||||
self, middleware: PurviewChatPolicyMiddleware, chat_context: ChatContext
|
||||
) -> None:
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
next_called = False
|
||||
|
||||
async def mock_next() -> None:
|
||||
nonlocal next_called
|
||||
next_called = True
|
||||
|
||||
chat_context.result = ChatResponse(messages=[Message(role="assistant", contents=["Hi there"])])
|
||||
|
||||
await middleware.process(chat_context, mock_next)
|
||||
assert next_called
|
||||
assert mock_proc.call_count == 2
|
||||
result = cast(ChatResponse[Any], chat_context.result)
|
||||
assert result.messages[0].role == "assistant"
|
||||
|
||||
async def test_blocks_prompt(self, middleware: PurviewChatPolicyMiddleware, chat_context: ChatContext) -> None:
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(True, "user-123")):
|
||||
|
||||
async def mock_next() -> None: # should not run
|
||||
raise AssertionError("next should not be called when prompt blocked")
|
||||
|
||||
with pytest.raises(MiddlewareTermination):
|
||||
await middleware.process(chat_context, mock_next)
|
||||
assert chat_context.result
|
||||
assert hasattr(chat_context.result, "messages")
|
||||
result = cast(ChatResponse[Any], chat_context.result)
|
||||
msg = result.messages[0]
|
||||
assert msg.role in ("system", "system")
|
||||
assert "blocked" in msg.text.lower()
|
||||
|
||||
async def test_blocks_response(self, middleware: PurviewChatPolicyMiddleware, chat_context: ChatContext) -> None:
|
||||
call_state = {"count": 0}
|
||||
|
||||
async def side_effect(messages, activity, session_id=None, user_id=None):
|
||||
call_state["count"] += 1
|
||||
should_block = call_state["count"] == 2
|
||||
return (should_block, "user-123")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=side_effect):
|
||||
|
||||
async def mock_next() -> None:
|
||||
chat_context.result = ChatResponse(messages=[Message(role="assistant", contents=["Sensitive output"])])
|
||||
|
||||
await middleware.process(chat_context, mock_next)
|
||||
assert call_state["count"] == 2
|
||||
result = cast(ChatResponse[Any], chat_context.result)
|
||||
msgs = result.messages
|
||||
first_msg = msgs[0]
|
||||
assert first_msg.role in ("system", "system")
|
||||
assert "blocked" in first_msg.text.lower()
|
||||
|
||||
async def test_streaming_skips_post_check(self, middleware: PurviewChatPolicyMiddleware) -> None:
|
||||
client = DummyChatClient()
|
||||
chat_options = MagicMock()
|
||||
chat_options.model = "test-model"
|
||||
streaming_context = ChatContext(
|
||||
client=cast(Any, client),
|
||||
messages=[Message(role="user", contents=["Hello"])],
|
||||
options=chat_options,
|
||||
stream=True,
|
||||
)
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
streaming_context.result = MagicMock()
|
||||
|
||||
await middleware.process(streaming_context, mock_next)
|
||||
assert mock_proc.call_count == 1
|
||||
|
||||
async def test_chat_middleware_handles_post_check_exception(
|
||||
self, middleware: PurviewChatPolicyMiddleware, chat_context: ChatContext
|
||||
) -> None:
|
||||
"""Test that exceptions in post-check are logged but don't affect result when ignore_exceptions=True."""
|
||||
# Set ignore_exceptions to True to test exception suppression
|
||||
middleware._settings["ignore_exceptions"] = True
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def mock_process_messages(*args, **kwargs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return (False, "user-123") # Pre-check succeeds
|
||||
raise Exception("Post-check error") # Post-check fails
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["Response"])]
|
||||
chat_context.result = result
|
||||
|
||||
await middleware.process(chat_context, mock_next)
|
||||
|
||||
# Should have been called twice (pre and post)
|
||||
assert call_count == 2
|
||||
# Result should still be set
|
||||
assert chat_context.result is not None
|
||||
|
||||
async def test_chat_middleware_uses_consistent_user_id(
|
||||
self, middleware: PurviewChatPolicyMiddleware, chat_context: ChatContext
|
||||
) -> None:
|
||||
"""Test that the same user_id from pre-check is used in post-check."""
|
||||
captured_user_ids: list[str | None] = []
|
||||
|
||||
async def mock_process_messages(messages, activity, session_id=None, user_id=None):
|
||||
captured_user_ids.append(user_id)
|
||||
return (False, "resolved-user-123")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["Response"])]
|
||||
chat_context.result = result
|
||||
|
||||
await middleware.process(chat_context, mock_next)
|
||||
|
||||
# Should have been called twice
|
||||
assert len(captured_user_ids) == 2
|
||||
# First call should have None (no user_id provided yet)
|
||||
assert captured_user_ids[0] is None
|
||||
# Second call should have the resolved user_id from first call
|
||||
assert captured_user_ids[1] == "resolved-user-123"
|
||||
|
||||
async def test_chat_middleware_handles_payment_required_pre_check(self, mock_credential: AsyncMock) -> None:
|
||||
"""Test that 402 in pre-check is handled based on settings."""
|
||||
from agent_framework_purview._exceptions import PurviewPaymentRequiredError
|
||||
|
||||
# Test with ignore_payment_required=False
|
||||
settings = PurviewSettings(app_name="Test App", ignore_payment_required=False)
|
||||
middleware = PurviewChatPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
client = DummyChatClient()
|
||||
chat_options = MagicMock()
|
||||
chat_options.model = "test-model"
|
||||
context = ChatContext(
|
||||
client=cast(Any, client), messages=[Message(role="user", contents=["Hello"])], options=chat_options
|
||||
)
|
||||
|
||||
async def mock_process_messages(*args, **kwargs):
|
||||
raise PurviewPaymentRequiredError("Payment required")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next() -> None:
|
||||
raise AssertionError("next should not be called")
|
||||
|
||||
# Should raise the exception
|
||||
with pytest.raises(PurviewPaymentRequiredError):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
async def test_chat_middleware_handles_payment_required_post_check(self, mock_credential: AsyncMock) -> None:
|
||||
"""Test that 402 in post-check is raised when ignore_payment_required=False."""
|
||||
from agent_framework_purview._exceptions import PurviewPaymentRequiredError
|
||||
|
||||
settings = PurviewSettings(app_name="Test App", ignore_payment_required=False)
|
||||
middleware = PurviewChatPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
client = DummyChatClient()
|
||||
chat_options = MagicMock()
|
||||
chat_options.model = "test-model"
|
||||
context = ChatContext(
|
||||
client=cast(Any, client), messages=[Message(role="user", contents=["Hello"])], options=chat_options
|
||||
)
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def side_effect(*args, **kwargs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return (False, "user-123")
|
||||
raise PurviewPaymentRequiredError("Payment required")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=side_effect):
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["OK"])]
|
||||
context.result = result
|
||||
|
||||
with pytest.raises(PurviewPaymentRequiredError):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
async def test_chat_middleware_ignores_payment_required_when_configured(self, mock_credential: AsyncMock) -> None:
|
||||
"""Test that 402 is ignored when ignore_payment_required=True."""
|
||||
from agent_framework_purview._exceptions import PurviewPaymentRequiredError
|
||||
|
||||
settings = PurviewSettings(app_name="Test App", ignore_payment_required=True)
|
||||
middleware = PurviewChatPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
client = DummyChatClient()
|
||||
chat_options = MagicMock()
|
||||
chat_options.model = "test-model"
|
||||
context = ChatContext(
|
||||
client=cast(Any, client), messages=[Message(role="user", contents=["Hello"])], options=chat_options
|
||||
)
|
||||
|
||||
async def mock_process_messages(*args, **kwargs):
|
||||
raise PurviewPaymentRequiredError("Payment required")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["Response"])]
|
||||
context.result = result
|
||||
|
||||
# Should not raise, just log
|
||||
await middleware.process(context, mock_next)
|
||||
# Next should have been called
|
||||
assert context.result is not None
|
||||
|
||||
async def test_chat_middleware_handles_result_without_messages_attribute(
|
||||
self, middleware: PurviewChatPolicyMiddleware, chat_context: ChatContext
|
||||
) -> None:
|
||||
"""Test middleware handles result that doesn't have messages attribute."""
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")):
|
||||
|
||||
async def mock_next() -> None:
|
||||
# Set result to something without messages attribute
|
||||
chat_context.result = cast(Any, "Some string result")
|
||||
|
||||
await middleware.process(chat_context, mock_next)
|
||||
|
||||
# Should not crash, result should be unchanged
|
||||
assert chat_context.result == "Some string result"
|
||||
|
||||
async def test_chat_middleware_with_ignore_exceptions(self, mock_credential: AsyncMock) -> None:
|
||||
"""Test that middleware respects ignore_exceptions setting."""
|
||||
settings = PurviewSettings(app_name="Test App", ignore_exceptions=True)
|
||||
middleware = PurviewChatPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
client = DummyChatClient()
|
||||
chat_options = MagicMock()
|
||||
chat_options.model = "test-model"
|
||||
context = ChatContext(
|
||||
client=cast(Any, client), messages=[Message(role="user", contents=["Hello"])], options=chat_options
|
||||
)
|
||||
|
||||
async def mock_process_messages(*args, **kwargs):
|
||||
raise ValueError("Some error")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["Response"])]
|
||||
context.result = result
|
||||
|
||||
# Should not raise, just log
|
||||
await middleware.process(context, mock_next)
|
||||
# Next should have been called
|
||||
assert context.result is not None
|
||||
|
||||
async def test_chat_middleware_raises_on_pre_check_exception_when_ignore_exceptions_false(
|
||||
self, mock_credential: AsyncMock
|
||||
) -> None:
|
||||
"""Test that exceptions are propagated by default when ignore_exceptions=False."""
|
||||
settings = PurviewSettings(app_name="Test App", ignore_exceptions=False)
|
||||
middleware = PurviewChatPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
client = DummyChatClient()
|
||||
chat_options = MagicMock()
|
||||
chat_options.model = "test-model"
|
||||
context = ChatContext(
|
||||
client=cast(Any, client), messages=[Message(role="user", contents=["Hello"])], options=chat_options
|
||||
)
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=ValueError("boom")):
|
||||
|
||||
async def mock_next() -> None:
|
||||
raise AssertionError("next should not be called")
|
||||
|
||||
with pytest.raises(ValueError, match="boom"):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
async def test_chat_middleware_raises_on_post_check_exception_when_ignore_exceptions_false(
|
||||
self, mock_credential: AsyncMock
|
||||
) -> None:
|
||||
"""Test that post-check exceptions are propagated by default."""
|
||||
settings = PurviewSettings(app_name="Test App", ignore_exceptions=False)
|
||||
middleware = PurviewChatPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
client = DummyChatClient()
|
||||
chat_options = MagicMock()
|
||||
chat_options.model = "test-model"
|
||||
context = ChatContext(
|
||||
client=cast(Any, client), messages=[Message(role="user", contents=["Hello"])], options=chat_options
|
||||
)
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def side_effect(*args, **kwargs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return (False, "user-123")
|
||||
raise ValueError("post")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=side_effect):
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["OK"])]
|
||||
context.result = result
|
||||
|
||||
with pytest.raises(ValueError, match="post"):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
async def test_chat_middleware_uses_conversation_id_from_options(
|
||||
self, middleware: PurviewChatPolicyMiddleware
|
||||
) -> None:
|
||||
"""Test that session_id is extracted from context.options['conversation_id']."""
|
||||
chat_client = DummyChatClient()
|
||||
messages = [Message(role="user", contents=["Hello"])]
|
||||
options = {"conversation_id": "conv-123", "model": "test-model"}
|
||||
context = ChatContext(client=cast(Any, chat_client), messages=messages, options=options)
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["Hi"])]
|
||||
context.result = result
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Verify session_id is passed to both pre-check and post-check
|
||||
assert mock_proc.call_count == 2
|
||||
mock_proc.assert_any_call(messages, Activity.UPLOAD_TEXT, session_id="conv-123")
|
||||
|
||||
async def test_chat_middleware_passes_none_session_id_when_options_missing(
|
||||
self, middleware: PurviewChatPolicyMiddleware
|
||||
) -> None:
|
||||
"""Test that session_id is None when options don't contain conversation_id."""
|
||||
chat_client = DummyChatClient()
|
||||
messages = [Message(role="user", contents=["Hello"])]
|
||||
context = ChatContext(client=cast(Any, chat_client), messages=messages, options=None)
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["Hi"])]
|
||||
context.result = result
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Verify session_id=None is passed
|
||||
mock_proc.assert_any_call(messages, Activity.UPLOAD_TEXT, session_id=None)
|
||||
|
||||
async def test_chat_middleware_session_id_used_in_post_check(self, middleware: PurviewChatPolicyMiddleware) -> None:
|
||||
"""Test that session_id is passed to post-check process_messages call."""
|
||||
chat_client = DummyChatClient()
|
||||
messages = [Message(role="user", contents=["Hello"])]
|
||||
options = {"conversation_id": "conv-999"}
|
||||
context = ChatContext(client=cast(Any, chat_client), messages=messages, options=options)
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
result = MagicMock()
|
||||
result.messages = [Message(role="assistant", contents=["Response"])]
|
||||
context.result = result
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Verify both calls include session_id
|
||||
assert mock_proc.call_count == 2
|
||||
# Check post-check call includes session_id
|
||||
post_check_call = mock_proc.call_args_list[1]
|
||||
assert post_check_call[1]["session_id"] == "conv-999"
|
||||
@@ -0,0 +1,47 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for Purview exceptions."""
|
||||
|
||||
from agent_framework.exceptions import IntegrationException, IntegrationInvalidAuthException
|
||||
|
||||
from agent_framework_purview import (
|
||||
PurviewAuthenticationError,
|
||||
PurviewPaymentRequiredError,
|
||||
PurviewRateLimitError,
|
||||
PurviewRequestError,
|
||||
PurviewServiceError,
|
||||
)
|
||||
|
||||
|
||||
class TestPurviewExceptions:
|
||||
"""Test custom Purview exception classes."""
|
||||
|
||||
def test_purview_service_error(self) -> None:
|
||||
"""Test PurviewServiceError base exception."""
|
||||
error = PurviewServiceError("Service error occurred")
|
||||
assert str(error) == "Service error occurred"
|
||||
assert isinstance(error, IntegrationException)
|
||||
|
||||
def test_purview_authentication_error(self) -> None:
|
||||
"""Test PurviewAuthenticationError exception."""
|
||||
error = PurviewAuthenticationError("Authentication failed")
|
||||
assert str(error) == "Authentication failed"
|
||||
assert isinstance(error, IntegrationInvalidAuthException)
|
||||
|
||||
def test_purview_payment_required_error(self) -> None:
|
||||
"""Test PurviewPaymentRequiredError exception."""
|
||||
error = PurviewPaymentRequiredError("Payment required")
|
||||
assert str(error) == "Payment required"
|
||||
assert isinstance(error, IntegrationException)
|
||||
|
||||
def test_purview_rate_limit_error(self) -> None:
|
||||
"""Test PurviewRateLimitError exception."""
|
||||
error = PurviewRateLimitError("Rate limit exceeded")
|
||||
assert str(error) == "Rate limit exceeded"
|
||||
assert isinstance(error, IntegrationException)
|
||||
|
||||
def test_purview_request_error(self) -> None:
|
||||
"""Test PurviewRequestError exception."""
|
||||
error = PurviewRequestError("Request failed")
|
||||
assert str(error) == "Request failed"
|
||||
assert isinstance(error, IntegrationException)
|
||||
@@ -0,0 +1,433 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for Purview middleware."""
|
||||
|
||||
from typing import Any, cast
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from agent_framework import AgentContext, AgentResponse, AgentSession, Message, MiddlewareTermination
|
||||
from azure.core.credentials import AccessToken
|
||||
|
||||
from agent_framework_purview import PurviewPolicyMiddleware, PurviewSettings
|
||||
from agent_framework_purview._models import Activity
|
||||
|
||||
|
||||
class TestPurviewPolicyMiddleware:
|
||||
"""Test PurviewPolicyMiddleware functionality."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_credential(self) -> AsyncMock:
|
||||
"""Create a mock async credential."""
|
||||
credential = AsyncMock()
|
||||
credential.get_token = AsyncMock(return_value=AccessToken("fake-token", 9999999999))
|
||||
return credential
|
||||
|
||||
@pytest.fixture
|
||||
def settings(self) -> PurviewSettings:
|
||||
"""Create test settings."""
|
||||
return PurviewSettings(app_name="Test App", tenant_id="test-tenant")
|
||||
|
||||
@pytest.fixture
|
||||
def middleware(self, mock_credential: AsyncMock, settings: PurviewSettings) -> PurviewPolicyMiddleware:
|
||||
"""Create PurviewPolicyMiddleware instance."""
|
||||
return PurviewPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
@pytest.fixture
|
||||
def mock_agent(self) -> MagicMock:
|
||||
"""Create a mock agent."""
|
||||
agent = MagicMock()
|
||||
agent.name = "test-agent"
|
||||
return agent
|
||||
|
||||
def test_middleware_initialization(self, mock_credential: AsyncMock, settings: PurviewSettings) -> None:
|
||||
"""Test PurviewPolicyMiddleware initialization."""
|
||||
middleware = PurviewPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
assert middleware._client is not None
|
||||
assert middleware._processor is not None
|
||||
|
||||
async def test_middleware_allows_clean_prompt(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test middleware allows prompt that passes policy check."""
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello, how are you?"])])
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")):
|
||||
next_called = False
|
||||
|
||||
async def mock_next() -> None:
|
||||
nonlocal next_called
|
||||
next_called = True
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["I'm good, thanks!"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
assert next_called
|
||||
assert context.result is not None
|
||||
|
||||
async def test_middleware_blocks_prompt_on_policy_violation(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test middleware blocks prompt that violates policy."""
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Sensitive information"])])
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(True, "user-123")):
|
||||
next_called = False
|
||||
|
||||
async def mock_next() -> None:
|
||||
nonlocal next_called
|
||||
next_called = True
|
||||
|
||||
with pytest.raises(MiddlewareTermination):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
assert not next_called
|
||||
assert context.result is not None
|
||||
result = cast(AgentResponse[Any], context.result)
|
||||
assert len(result.messages) == 1
|
||||
assert result.messages[0].role == "system"
|
||||
assert "blocked by policy" in result.messages[0].text.lower()
|
||||
|
||||
async def test_middleware_checks_response(self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock) -> None:
|
||||
"""Test middleware checks agent response for policy violations."""
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])])
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def mock_process_messages(messages, activity, session_id=None, user_id=None):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
should_block = call_count != 1
|
||||
return (should_block, "user-123")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(
|
||||
messages=[Message(role="assistant", contents=["Here's some sensitive information"])]
|
||||
)
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
assert call_count == 2
|
||||
assert context.result is not None
|
||||
result = cast(AgentResponse[Any], context.result)
|
||||
assert len(result.messages) == 1
|
||||
assert result.messages[0].role == "system"
|
||||
assert "blocked by policy" in result.messages[0].text.lower()
|
||||
|
||||
async def test_middleware_handles_result_without_messages(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test middleware handles result that doesn't have messages attribute."""
|
||||
# Set ignore_exceptions to True so AttributeError is caught and logged
|
||||
middleware._settings["ignore_exceptions"] = True
|
||||
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])])
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")):
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = cast(Any, "Some non-standard result")
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
assert context.result == "Some non-standard result"
|
||||
|
||||
async def test_middleware_processor_receives_correct_activity(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test middleware passes correct activity type to processor."""
|
||||
from agent_framework_purview._models import Activity
|
||||
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Test"])])
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_process:
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Response"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
assert mock_process.call_count == 2
|
||||
# First call (pre-check) should be UPLOAD_TEXT for user prompt
|
||||
assert mock_process.call_args_list[0][0][1] == Activity.UPLOAD_TEXT
|
||||
# Second call (post-check) should be DOWNLOAD_TEXT for agent response
|
||||
assert mock_process.call_args_list[1][0][1] == Activity.DOWNLOAD_TEXT
|
||||
|
||||
async def test_middleware_streaming_skips_post_check(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that streaming results skip post-check evaluation."""
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])])
|
||||
context.stream = True
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["streaming"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
assert mock_proc.call_count == 1
|
||||
|
||||
async def test_middleware_payment_required_in_pre_check_raises_by_default(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that 402 in pre-check is raised when ignore_payment_required=False."""
|
||||
from agent_framework_purview._exceptions import PurviewPaymentRequiredError
|
||||
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])])
|
||||
|
||||
with patch.object(
|
||||
middleware._processor,
|
||||
"process_messages",
|
||||
side_effect=PurviewPaymentRequiredError("Payment required"),
|
||||
):
|
||||
|
||||
async def mock_next() -> None:
|
||||
raise AssertionError("next should not be called")
|
||||
|
||||
with pytest.raises(PurviewPaymentRequiredError):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
async def test_middleware_payment_required_in_post_check_raises_by_default(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that 402 in post-check is raised when ignore_payment_required=False."""
|
||||
from agent_framework_purview._exceptions import PurviewPaymentRequiredError
|
||||
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])])
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def side_effect(*args, **kwargs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return (False, "user-123")
|
||||
raise PurviewPaymentRequiredError("Payment required")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=side_effect):
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["OK"])])
|
||||
|
||||
with pytest.raises(PurviewPaymentRequiredError):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
async def test_middleware_post_check_exception_raises_when_ignore_exceptions_false(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that post-check exceptions are propagated when ignore_exceptions=False."""
|
||||
middleware._settings["ignore_exceptions"] = False
|
||||
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])])
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def side_effect(*args, **kwargs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return (False, "user-123")
|
||||
raise ValueError("Post-check blew up")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=side_effect):
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["OK"])])
|
||||
|
||||
with pytest.raises(ValueError, match="Post-check blew up"):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
async def test_middleware_handles_pre_check_exception(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that exceptions in pre-check are logged but don't stop processing when ignore_exceptions=True."""
|
||||
# Set ignore_exceptions to True
|
||||
middleware._settings["ignore_exceptions"] = True
|
||||
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Test"])])
|
||||
|
||||
with patch.object(
|
||||
middleware._processor, "process_messages", side_effect=Exception("Pre-check error")
|
||||
) as mock_process:
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Response"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Should have been called twice (pre-check raises, then post-check also raises)
|
||||
assert mock_process.call_count == 2
|
||||
# Result should be set by mock_next
|
||||
assert context.result is not None
|
||||
|
||||
async def test_middleware_handles_post_check_exception(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that exceptions in post-check are logged but don't affect result when ignore_exceptions=True."""
|
||||
# Set ignore_exceptions to True
|
||||
middleware._settings["ignore_exceptions"] = True
|
||||
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Test"])])
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def mock_process_messages(*args, **kwargs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return (False, "user-123") # Pre-check succeeds
|
||||
raise Exception("Post-check error") # Post-check fails
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Response"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Should have been called twice (pre and post)
|
||||
assert call_count == 2
|
||||
# Result should still be set
|
||||
assert context.result is not None
|
||||
assert hasattr(context.result, "messages")
|
||||
|
||||
async def test_middleware_with_ignore_exceptions_true(self, mock_credential: AsyncMock) -> None:
|
||||
"""Test that middleware logs but doesn't throw when ignore_exceptions is True."""
|
||||
settings = PurviewSettings(app_name="Test App", ignore_exceptions=True)
|
||||
middleware = PurviewPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
mock_agent = MagicMock()
|
||||
mock_agent.name = "test-agent"
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Test"])])
|
||||
|
||||
# Mock processor to raise an exception
|
||||
async def mock_process_messages(*args, **kwargs):
|
||||
raise ValueError("Test error")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next():
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Response"])])
|
||||
|
||||
# Should not raise, just log
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Result should be set because next was called despite the error
|
||||
assert context.result is not None
|
||||
|
||||
async def test_middleware_with_ignore_exceptions_false(self, mock_credential: AsyncMock) -> None:
|
||||
"""Test that middleware throws exceptions when ignore_exceptions is False."""
|
||||
settings = PurviewSettings(app_name="Test App", ignore_exceptions=False)
|
||||
middleware = PurviewPolicyMiddleware(mock_credential, settings)
|
||||
|
||||
mock_agent = MagicMock()
|
||||
mock_agent.name = "test-agent"
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Test"])])
|
||||
|
||||
# Mock processor to raise an exception
|
||||
async def mock_process_messages(*args, **kwargs):
|
||||
raise ValueError("Test error")
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", side_effect=mock_process_messages):
|
||||
|
||||
async def mock_next():
|
||||
pass
|
||||
|
||||
# Should raise the exception
|
||||
with pytest.raises(ValueError, match="Test error"):
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
async def test_middleware_uses_session_service_session_id_as_session_id(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that session_id is extracted from session.service_session_id."""
|
||||
session = AgentSession(service_session_id="thread-123")
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])], session=session)
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Hi"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Verify session_id is passed to both pre-check and post-check
|
||||
assert mock_proc.call_count == 2
|
||||
mock_proc.assert_any_call(context.messages, Activity.UPLOAD_TEXT, session_id="thread-123")
|
||||
|
||||
async def test_middleware_uses_message_conversation_id_as_session_id(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that session_id is extracted from message.additional_properties['conversation_id']."""
|
||||
messages = [Message(role="user", contents=["Hello"], additional_properties={"conversation_id": "conv-456"})]
|
||||
context = AgentContext(agent=mock_agent, messages=messages)
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Hi"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Verify session_id is passed to both pre-check and post-check
|
||||
assert mock_proc.call_count == 2
|
||||
mock_proc.assert_any_call(messages, Activity.UPLOAD_TEXT, session_id="conv-456")
|
||||
|
||||
async def test_middleware_session_id_takes_precedence_over_message_conversation_id(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that session.service_session_id takes precedence over message conversation_id."""
|
||||
session = AgentSession(service_session_id="thread-789")
|
||||
messages = [Message(role="user", contents=["Hello"], additional_properties={"conversation_id": "conv-456"})]
|
||||
context = AgentContext(agent=mock_agent, messages=messages, session=session)
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Hi"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Verify session ID is used, not message conversation_id
|
||||
mock_proc.assert_any_call(messages, Activity.UPLOAD_TEXT, session_id="thread-789")
|
||||
|
||||
async def test_middleware_passes_none_session_id_when_not_available(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that session_id is None when no session or conversation_id is available."""
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])])
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Hi"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Verify session_id=None is passed
|
||||
mock_proc.assert_any_call(context.messages, Activity.UPLOAD_TEXT, session_id=None)
|
||||
|
||||
async def test_middleware_session_id_used_in_post_check(
|
||||
self, middleware: PurviewPolicyMiddleware, mock_agent: MagicMock
|
||||
) -> None:
|
||||
"""Test that session_id is passed to post-check process_messages call."""
|
||||
session = AgentSession(service_session_id="thread-999")
|
||||
context = AgentContext(agent=mock_agent, messages=[Message(role="user", contents=["Hello"])], session=session)
|
||||
|
||||
with patch.object(middleware._processor, "process_messages", return_value=(False, "user-123")) as mock_proc:
|
||||
|
||||
async def mock_next() -> None:
|
||||
context.result = AgentResponse(messages=[Message(role="assistant", contents=["Response"])])
|
||||
|
||||
await middleware.process(context, mock_next)
|
||||
|
||||
# Verify both calls include session_id
|
||||
assert mock_proc.call_count == 2
|
||||
# Check post-check call includes session_id
|
||||
post_check_call = mock_proc.call_args_list[1]
|
||||
assert post_check_call[1]["session_id"] == "thread-999"
|
||||
@@ -0,0 +1,894 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for Purview processor."""
|
||||
|
||||
import asyncio
|
||||
from typing import Any, cast
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from agent_framework import Message
|
||||
|
||||
from agent_framework_purview import PurviewAppLocation, PurviewLocationType, PurviewSettings
|
||||
from agent_framework_purview._models import (
|
||||
Activity,
|
||||
DlpAction,
|
||||
DlpActionInfo,
|
||||
ExecutionMode,
|
||||
PolicyLocation,
|
||||
PolicyScope,
|
||||
ProcessContentResponse,
|
||||
ProtectionScopeActivities,
|
||||
ProtectionScopeState,
|
||||
RestrictionAction,
|
||||
)
|
||||
from agent_framework_purview._processor import ScopedContentProcessor, _is_valid_guid
|
||||
|
||||
|
||||
class TestGuidValidation:
|
||||
"""Test GUID validation helper."""
|
||||
|
||||
def test_valid_guid(self) -> None:
|
||||
"""Test _is_valid_guid with valid GUIDs."""
|
||||
assert _is_valid_guid("12345678-1234-1234-1234-123456789012")
|
||||
assert _is_valid_guid("a1b2c3d4-e5f6-4a5b-8c9d-0e1f2a3b4c5d")
|
||||
|
||||
def test_invalid_guid(self) -> None:
|
||||
"""Test _is_valid_guid with invalid GUIDs."""
|
||||
assert not _is_valid_guid("not-a-guid")
|
||||
assert not _is_valid_guid("")
|
||||
assert not _is_valid_guid(None)
|
||||
|
||||
|
||||
class TestScopedContentProcessor:
|
||||
"""Test ScopedContentProcessor functionality."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_client(self) -> AsyncMock:
|
||||
"""Create a mock Purview client."""
|
||||
client = AsyncMock()
|
||||
client.get_user_info_from_token = AsyncMock(
|
||||
return_value={
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"user_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
)
|
||||
return client
|
||||
|
||||
@pytest.fixture
|
||||
def settings_with_defaults(self) -> PurviewSettings:
|
||||
"""Create settings with default values."""
|
||||
app_location = PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION, location_value="12345678-1234-1234-1234-123456789012"
|
||||
)
|
||||
return PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=app_location,
|
||||
)
|
||||
|
||||
@pytest.fixture
|
||||
def settings_without_defaults(self) -> PurviewSettings:
|
||||
"""Create settings without default values (requiring token info)."""
|
||||
return PurviewSettings(app_name="Test App")
|
||||
|
||||
@pytest.fixture
|
||||
def processor(self, mock_client: AsyncMock, settings_with_defaults: PurviewSettings) -> ScopedContentProcessor:
|
||||
"""Create a ScopedContentProcessor with mock client."""
|
||||
return ScopedContentProcessor(mock_client, settings_with_defaults)
|
||||
|
||||
async def test_processor_initialization(
|
||||
self, mock_client: AsyncMock, settings_with_defaults: PurviewSettings
|
||||
) -> None:
|
||||
"""Test ScopedContentProcessor initialization."""
|
||||
processor = ScopedContentProcessor(mock_client, settings_with_defaults)
|
||||
|
||||
assert processor._client == mock_client
|
||||
assert processor._settings == settings_with_defaults
|
||||
|
||||
async def test_process_messages_with_defaults(self, processor: ScopedContentProcessor) -> None:
|
||||
"""Test process_messages with settings that have defaults."""
|
||||
messages = [
|
||||
Message(role="user", contents=["Hello"]),
|
||||
Message(role="assistant", contents=["Hi there"]),
|
||||
]
|
||||
|
||||
with patch.object(processor, "_map_messages", return_value=([], None)) as mock_map:
|
||||
should_block, user_id = await processor.process_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
assert should_block is False
|
||||
assert user_id is None
|
||||
mock_map.assert_called_once_with(messages, Activity.UPLOAD_TEXT, None, None)
|
||||
|
||||
async def test_process_messages_blocks_content(
|
||||
self, processor: ScopedContentProcessor, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test process_messages returns True when content should be blocked."""
|
||||
messages = [Message(role="user", contents=["Sensitive content"])]
|
||||
|
||||
mock_request = process_content_request_factory("Sensitive content")
|
||||
|
||||
mock_response = ProcessContentResponse(
|
||||
policy_actions=cast(
|
||||
Any, [DlpActionInfo(action=DlpAction.BLOCK_ACCESS, restrictionAction=RestrictionAction.BLOCK)]
|
||||
)
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(processor, "_map_messages", return_value=([mock_request], "user-123")),
|
||||
patch.object(processor, "_process_with_scopes", return_value=mock_response),
|
||||
):
|
||||
should_block, user_id = await processor.process_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
assert should_block is True
|
||||
assert user_id == "user-123"
|
||||
|
||||
async def test_map_messages_creates_requests(
|
||||
self, processor: ScopedContentProcessor, mock_client: AsyncMock
|
||||
) -> None:
|
||||
"""Test _map_messages creates ProcessContentRequest objects."""
|
||||
messages = [
|
||||
Message(
|
||||
role="user",
|
||||
contents=["Test message"],
|
||||
message_id="msg-123",
|
||||
author_name="12345678-1234-1234-1234-123456789012",
|
||||
),
|
||||
]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
assert len(requests) == 1
|
||||
assert requests[0].user_id == "12345678-1234-1234-1234-123456789012"
|
||||
assert requests[0].tenant_id == "12345678-1234-1234-1234-123456789012"
|
||||
assert user_id == "12345678-1234-1234-1234-123456789012"
|
||||
|
||||
async def test_map_messages_without_defaults_gets_token_info(self, mock_client: AsyncMock) -> None:
|
||||
"""Test _map_messages gets token info when settings lack some defaults."""
|
||||
settings = PurviewSettings(app_name="Test App", tenant_id="12345678-1234-1234-1234-123456789012")
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
messages = [Message(role="user", contents=["Test"], message_id="msg-123")]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
mock_client.get_user_info_from_token.assert_called_once()
|
||||
assert len(requests) == 1
|
||||
assert user_id is not None
|
||||
|
||||
async def test_map_messages_raises_on_missing_tenant_id(self, mock_client: AsyncMock) -> None:
|
||||
"""Test _map_messages raises ValueError when tenant_id cannot be determined."""
|
||||
settings = PurviewSettings(app_name="Test App") # No tenant_id
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
|
||||
mock_client.get_user_info_from_token = AsyncMock(
|
||||
return_value={"user_id": "test-user", "client_id": "test-client"}
|
||||
)
|
||||
|
||||
messages = [Message(role="user", contents=["Test"], message_id="msg-123")]
|
||||
|
||||
with pytest.raises(ValueError, match="Tenant id required"):
|
||||
await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
async def test_check_applicable_scopes_no_scopes(
|
||||
self, processor: ScopedContentProcessor, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test _check_applicable_scopes when no scopes are returned."""
|
||||
from agent_framework_purview._models import ProtectionScopesResponse
|
||||
|
||||
request = process_content_request_factory()
|
||||
response = ProtectionScopesResponse(scopes=None)
|
||||
|
||||
should_process, actions, execution_mode = processor._check_applicable_scopes(request, response)
|
||||
|
||||
assert should_process is False
|
||||
assert actions == []
|
||||
|
||||
async def test_check_applicable_scopes_with_block_action(
|
||||
self, processor: ScopedContentProcessor, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test _check_applicable_scopes identifies block actions."""
|
||||
from agent_framework_purview._models import (
|
||||
PolicyLocation,
|
||||
PolicyScope,
|
||||
ProtectionScopeActivities,
|
||||
ProtectionScopesResponse,
|
||||
)
|
||||
|
||||
request = process_content_request_factory()
|
||||
|
||||
block_action = DlpActionInfo(action=DlpAction.BLOCK_ACCESS, restrictionAction=RestrictionAction.BLOCK)
|
||||
scope_location = PolicyLocation(
|
||||
data_type="microsoft.graph.policyLocationApplication",
|
||||
value="app-id",
|
||||
)
|
||||
scope = PolicyScope(
|
||||
**cast(
|
||||
Any,
|
||||
{
|
||||
"policyActions": [block_action],
|
||||
"activities": ProtectionScopeActivities.UPLOAD_TEXT,
|
||||
"locations": [scope_location],
|
||||
},
|
||||
)
|
||||
)
|
||||
response = ProtectionScopesResponse(scopes=cast(Any, [scope]))
|
||||
|
||||
should_process, actions, execution_mode = processor._check_applicable_scopes(request, response)
|
||||
|
||||
assert should_process is True
|
||||
assert len(actions) == 1
|
||||
assert actions[0].action == DlpAction.BLOCK_ACCESS
|
||||
|
||||
async def test_combine_policy_actions(self, processor: ScopedContentProcessor) -> None:
|
||||
"""Test _combine_policy_actions merges action lists."""
|
||||
action1 = DlpActionInfo(action=DlpAction.BLOCK_ACCESS, restrictionAction=RestrictionAction.BLOCK)
|
||||
action2 = DlpActionInfo(action=DlpAction.OTHER, restrictionAction=RestrictionAction.OTHER)
|
||||
|
||||
combined = processor._combine_policy_actions([action1], [action2])
|
||||
|
||||
assert len(combined) == 2
|
||||
assert action1 in combined
|
||||
assert action2 in combined
|
||||
|
||||
async def test_combine_policy_actions_preserves_restriction_only_actions(
|
||||
self, processor: ScopedContentProcessor
|
||||
) -> None:
|
||||
"""Test _combine_policy_actions keeps actions that only set restrictionAction."""
|
||||
existing_action = DlpActionInfo(action=DlpAction.OTHER, restrictionAction=RestrictionAction.OTHER)
|
||||
restriction_only_action = DlpActionInfo(restriction_action=RestrictionAction.BLOCK)
|
||||
|
||||
combined = processor._combine_policy_actions([existing_action], [restriction_only_action])
|
||||
|
||||
assert combined == [existing_action, restriction_only_action]
|
||||
|
||||
async def test_combine_policy_actions_deduplicates_by_action_and_restriction(
|
||||
self, processor: ScopedContentProcessor
|
||||
) -> None:
|
||||
"""Test _combine_policy_actions removes exact duplicate actions."""
|
||||
block_action = DlpActionInfo(action=DlpAction.BLOCK_ACCESS, restriction_action=RestrictionAction.BLOCK)
|
||||
duplicate_block_action = DlpActionInfo(
|
||||
action=DlpAction.BLOCK_ACCESS, restriction_action=RestrictionAction.BLOCK
|
||||
)
|
||||
restriction_only_action = DlpActionInfo(restriction_action=RestrictionAction.BLOCK)
|
||||
|
||||
combined = processor._combine_policy_actions(
|
||||
[block_action],
|
||||
[duplicate_block_action, restriction_only_action],
|
||||
)
|
||||
|
||||
assert combined == [block_action, restriction_only_action]
|
||||
|
||||
async def test_process_with_scopes_calls_client_methods(
|
||||
self, processor: ScopedContentProcessor, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test _process_with_scopes calls process_content immediately and warms scopes in background on cache miss."""
|
||||
from agent_framework_purview._models import (
|
||||
ContentActivitiesResponse,
|
||||
ProtectionScopesResponse,
|
||||
)
|
||||
|
||||
request = process_content_request_factory()
|
||||
|
||||
mock_client.get_protection_scopes = AsyncMock(return_value=ProtectionScopesResponse(scopes=[]))
|
||||
mock_client.process_content = AsyncMock(
|
||||
return_value=ProcessContentResponse(
|
||||
id="response-123", protection_scope_state=ProtectionScopeState.NOT_MODIFIED
|
||||
)
|
||||
)
|
||||
mock_client.send_content_activities = AsyncMock(return_value=ContentActivitiesResponse(**{"error": None}))
|
||||
|
||||
response = await processor._process_with_scopes(request)
|
||||
|
||||
# On cache miss, ProcessContent runs in the foreground and the response is returned.
|
||||
assert response.id == "response-123"
|
||||
mock_client.process_content.assert_called_once()
|
||||
|
||||
# Protection scopes are refreshed in a background task.
|
||||
await asyncio.gather(*list(processor._background_tasks))
|
||||
mock_client.get_protection_scopes.assert_called_once()
|
||||
mock_client.send_content_activities.assert_called_once()
|
||||
|
||||
async def test_process_with_scopes_preserves_restriction_only_policy_actions(
|
||||
self, processor: ScopedContentProcessor, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test cold-cache ProcessContent actions are not dropped when they only contain restrictionAction."""
|
||||
from agent_framework_purview._models import ProtectionScopesResponse
|
||||
|
||||
request = process_content_request_factory()
|
||||
restriction_only_action = DlpActionInfo(restriction_action=RestrictionAction.BLOCK)
|
||||
|
||||
mock_client.get_protection_scopes = AsyncMock(return_value=ProtectionScopesResponse(scopes=[]))
|
||||
mock_client.process_content = AsyncMock(
|
||||
return_value=ProcessContentResponse(
|
||||
id="response-123",
|
||||
protection_scope_state=ProtectionScopeState.NOT_MODIFIED,
|
||||
policy_actions=[restriction_only_action],
|
||||
)
|
||||
)
|
||||
|
||||
response = await processor._process_with_scopes(request)
|
||||
|
||||
assert response.policy_actions == [restriction_only_action]
|
||||
await asyncio.gather(*list(processor._background_tasks))
|
||||
|
||||
async def test_process_with_cached_scopes_preserves_restriction_only_policy_actions(
|
||||
self, processor: ScopedContentProcessor, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test cached ProtectionScopes actions are not dropped when they only contain restrictionAction."""
|
||||
from agent_framework_purview._models import (
|
||||
ExecutionMode,
|
||||
PolicyLocation,
|
||||
PolicyScope,
|
||||
ProtectionScopeActivities,
|
||||
ProtectionScopesResponse,
|
||||
)
|
||||
|
||||
request = process_content_request_factory()
|
||||
restriction_only_action = DlpActionInfo(restriction_action=RestrictionAction.BLOCK)
|
||||
process_content_action = DlpActionInfo(action=DlpAction.OTHER, restriction_action=RestrictionAction.OTHER)
|
||||
scope_location = PolicyLocation(
|
||||
data_type="microsoft.graph.policyLocationApplication",
|
||||
value="app-id",
|
||||
)
|
||||
scope = PolicyScope(
|
||||
activities=ProtectionScopeActivities.UPLOAD_TEXT,
|
||||
locations=[scope_location],
|
||||
policy_actions=[restriction_only_action],
|
||||
execution_mode=ExecutionMode.EVALUATE_INLINE,
|
||||
)
|
||||
|
||||
cast(Any, processor._cache).get = AsyncMock(
|
||||
side_effect=[
|
||||
None,
|
||||
ProtectionScopesResponse(scope_identifier="scope-123", scopes=[scope]),
|
||||
]
|
||||
)
|
||||
mock_client.process_content = AsyncMock(
|
||||
return_value=ProcessContentResponse(
|
||||
id="response-123",
|
||||
protection_scope_state=ProtectionScopeState.NOT_MODIFIED,
|
||||
policy_actions=[process_content_action],
|
||||
)
|
||||
)
|
||||
|
||||
response = await processor._process_with_scopes(request)
|
||||
|
||||
assert response.policy_actions == [process_content_action, restriction_only_action]
|
||||
|
||||
async def test_process_with_scopes_ignores_unexpected_cached_value_type(
|
||||
self, processor: ScopedContentProcessor, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test that a corrupted cache entry does not crash processing."""
|
||||
from agent_framework_purview._models import ProtectionScopesResponse
|
||||
|
||||
request = process_content_request_factory()
|
||||
|
||||
mock_client.get_protection_scopes = AsyncMock(return_value=ProtectionScopesResponse(scopes=[]))
|
||||
# Return a valid, inline scope so we stay on the normal (non-background) path.
|
||||
scope_location = PolicyLocation(
|
||||
data_type="microsoft.graph.policyLocationApplication",
|
||||
value="app-id",
|
||||
)
|
||||
scope = PolicyScope(
|
||||
**cast(
|
||||
Any,
|
||||
{
|
||||
"activities": ProtectionScopeActivities.UPLOAD_TEXT,
|
||||
"locations": [scope_location],
|
||||
"execution_mode": ExecutionMode.EVALUATE_INLINE,
|
||||
},
|
||||
)
|
||||
)
|
||||
mock_client.get_protection_scopes = AsyncMock(return_value=ProtectionScopesResponse(scopes=cast(Any, [scope])))
|
||||
mock_client.process_content = AsyncMock(
|
||||
return_value=ProcessContentResponse(id="ok", protection_scope_state=ProtectionScopeState.NOT_MODIFIED)
|
||||
)
|
||||
|
||||
# First cache read is the tenant payment key (None). Second is the scopes cache (corrupt value).
|
||||
cast(Any, processor._cache).get = AsyncMock(side_effect=[None, "corrupt-value"])
|
||||
cast(Any, processor._cache).set = AsyncMock()
|
||||
|
||||
response = await processor._process_with_scopes(request)
|
||||
|
||||
assert response.id == "ok"
|
||||
mock_client.process_content.assert_called_once()
|
||||
await asyncio.gather(*list(processor._background_tasks))
|
||||
mock_client.get_protection_scopes.assert_called_once()
|
||||
|
||||
async def test_process_with_scopes_uses_tenant_payment_exception_cache(
|
||||
self, processor: ScopedContentProcessor, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test that a cached 402 exception short-circuits all subsequent requests for the tenant."""
|
||||
from agent_framework_purview._exceptions import PurviewPaymentRequiredError
|
||||
|
||||
request = process_content_request_factory()
|
||||
|
||||
cast(Any, processor._cache).get = AsyncMock(return_value=PurviewPaymentRequiredError("Payment required"))
|
||||
|
||||
with pytest.raises(PurviewPaymentRequiredError):
|
||||
await processor._process_with_scopes(request)
|
||||
|
||||
mock_client.get_protection_scopes.assert_not_called()
|
||||
|
||||
async def test_process_content_background_retries_on_modified_state(
|
||||
self, processor: ScopedContentProcessor, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test offline background processing invalidates cache and retries when scope state changes."""
|
||||
request = process_content_request_factory()
|
||||
request.scope_identifier = "etag-1"
|
||||
|
||||
mock_client.process_content = AsyncMock(
|
||||
side_effect=[
|
||||
ProcessContentResponse(id="r1", protection_scope_state=ProtectionScopeState.MODIFIED),
|
||||
ProcessContentResponse(id="r2", protection_scope_state=ProtectionScopeState.NOT_MODIFIED),
|
||||
]
|
||||
)
|
||||
cast(Any, processor._cache).remove = AsyncMock()
|
||||
|
||||
await processor._process_content_background(request, cache_key="purview:protection_scopes:abc")
|
||||
|
||||
cast(Any, processor._cache).remove.assert_called_once_with("purview:protection_scopes:abc")
|
||||
assert mock_client.process_content.call_count == 2
|
||||
|
||||
async def test_background_scope_refresh_caches_payment_required(
|
||||
self, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""402 raised during background scope refresh is cached at the tenant level."""
|
||||
from agent_framework_purview._cache import InMemoryCacheProvider
|
||||
from agent_framework_purview._exceptions import PurviewPaymentRequiredError
|
||||
|
||||
settings = PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION, location_value="app-id"
|
||||
),
|
||||
)
|
||||
|
||||
cache = InMemoryCacheProvider()
|
||||
processor = ScopedContentProcessor(mock_client, settings, cache_provider=cache)
|
||||
|
||||
mock_client.get_protection_scopes = AsyncMock(side_effect=PurviewPaymentRequiredError("nope"))
|
||||
mock_client.process_content = AsyncMock(
|
||||
return_value=ProcessContentResponse(id="pc-1", protection_scope_state=ProtectionScopeState.NOT_MODIFIED)
|
||||
)
|
||||
|
||||
request = process_content_request_factory()
|
||||
await processor._process_with_scopes(request)
|
||||
await asyncio.gather(*list(processor._background_tasks))
|
||||
|
||||
cached = await cache.get(f"purview:payment_required:{request.tenant_id}")
|
||||
assert isinstance(cached, PurviewPaymentRequiredError)
|
||||
|
||||
async def test_map_messages_with_user_id_in_additional_properties(self, mock_client: AsyncMock) -> None:
|
||||
"""Test user_id extraction from message additional_properties."""
|
||||
settings = PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION, location_value="app-id"
|
||||
),
|
||||
)
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [
|
||||
Message(
|
||||
role="user",
|
||||
contents=["Test message"],
|
||||
additional_properties={"user_id": "22345678-1234-1234-1234-123456789012"},
|
||||
),
|
||||
]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
assert len(requests) == 1
|
||||
assert user_id == "22345678-1234-1234-1234-123456789012"
|
||||
assert requests[0].user_id == "22345678-1234-1234-1234-123456789012"
|
||||
|
||||
async def test_map_messages_with_provided_user_id_fallback(self, mock_client: AsyncMock) -> None:
|
||||
"""Test using provided_user_id when no other source is available."""
|
||||
settings = PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION, location_value="app-id"
|
||||
),
|
||||
)
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [Message(role="user", contents=["Test message"])]
|
||||
|
||||
requests, user_id = await processor._map_messages(
|
||||
messages, Activity.UPLOAD_TEXT, provided_user_id="32345678-1234-1234-1234-123456789012"
|
||||
)
|
||||
|
||||
assert len(requests) == 1
|
||||
assert user_id == "32345678-1234-1234-1234-123456789012"
|
||||
assert requests[0].user_id == "32345678-1234-1234-1234-123456789012"
|
||||
|
||||
async def test_map_messages_returns_empty_when_no_user_id(self, mock_client: AsyncMock) -> None:
|
||||
"""Test that empty results are returned when user_id cannot be resolved."""
|
||||
settings = PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION, location_value="app-id"
|
||||
),
|
||||
)
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [Message(role="user", contents=["Test message"])]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
assert len(requests) == 0
|
||||
assert user_id is None
|
||||
|
||||
async def test_process_content_sends_activities_when_not_applicable(
|
||||
self, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test that response is returned when scopes don't apply (activities sent in background)."""
|
||||
from agent_framework_purview._models import ProtectionScopesResponse
|
||||
|
||||
settings = PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION, location_value="app-id"
|
||||
),
|
||||
)
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
|
||||
pc_request = process_content_request_factory()
|
||||
|
||||
mock_ps_response = ProtectionScopesResponse(scopes=[])
|
||||
cast(Any, processor._cache).get = AsyncMock(side_effect=[None, mock_ps_response])
|
||||
|
||||
# Mock send_content_activities to return success (called in background)
|
||||
mock_ca_response = MagicMock()
|
||||
mock_ca_response.error = None
|
||||
mock_client.send_content_activities.return_value = mock_ca_response
|
||||
|
||||
response = await processor._process_with_scopes(pc_request)
|
||||
|
||||
mock_client.get_protection_scopes.assert_not_called()
|
||||
mock_client.process_content.assert_not_called()
|
||||
await asyncio.gather(*list(processor._background_tasks))
|
||||
mock_client.send_content_activities.assert_called_once()
|
||||
# Response should have id=204 when no scopes apply
|
||||
assert response.id == "204"
|
||||
|
||||
async def test_process_content_handles_activities_error(
|
||||
self, mock_client: AsyncMock, process_content_request_factory
|
||||
) -> None:
|
||||
"""Test that errors in background activities don't affect the response."""
|
||||
from agent_framework_purview._models import ProtectionScopesResponse
|
||||
|
||||
settings = PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION, location_value="app-id"
|
||||
),
|
||||
)
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
|
||||
pc_request = process_content_request_factory()
|
||||
|
||||
mock_ps_response = ProtectionScopesResponse(scopes=[])
|
||||
cast(Any, processor._cache).get = AsyncMock(side_effect=[None, mock_ps_response])
|
||||
|
||||
# Mock send_content_activities to return error (called in background task)
|
||||
mock_ca_response = MagicMock()
|
||||
mock_ca_response.error = "Test error message"
|
||||
mock_client.send_content_activities.return_value = mock_ca_response
|
||||
|
||||
response = await processor._process_with_scopes(pc_request)
|
||||
|
||||
# Since activities are sent in background, errors don't affect the response
|
||||
# Response should have id=204 when no scopes apply
|
||||
assert response.id == "204"
|
||||
await asyncio.gather(*list(processor._background_tasks))
|
||||
mock_client.send_content_activities.assert_called_once()
|
||||
|
||||
|
||||
class TestUserIdResolution:
|
||||
"""Test user ID resolution from various sources."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_client(self) -> AsyncMock:
|
||||
"""Create a mock Purview client."""
|
||||
client = AsyncMock()
|
||||
client.get_user_info_from_token = AsyncMock(
|
||||
return_value={
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"user_id": "11111111-1111-1111-1111-111111111111",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
)
|
||||
return client
|
||||
|
||||
@pytest.fixture
|
||||
def settings(self) -> PurviewSettings:
|
||||
"""Create settings."""
|
||||
return PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=PurviewAppLocation(
|
||||
location_type=PurviewLocationType.APPLICATION, location_value="app-id"
|
||||
),
|
||||
)
|
||||
|
||||
async def test_user_id_from_token_when_no_other_source(self, mock_client: AsyncMock) -> None:
|
||||
"""Test user_id is extracted from token when no other source available."""
|
||||
settings = PurviewSettings(app_name="Test App") # No tenant_id or app_location
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
|
||||
messages = [Message(role="user", contents=["Test"])]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
mock_client.get_user_info_from_token.assert_called_once()
|
||||
assert user_id == "11111111-1111-1111-1111-111111111111"
|
||||
|
||||
async def test_user_id_from_token_takes_priority_over_additional_properties(
|
||||
self, mock_client: AsyncMock, settings: PurviewSettings
|
||||
) -> None:
|
||||
"""Test token user_id takes priority over message additional_properties."""
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
|
||||
messages = [
|
||||
Message(
|
||||
role="user",
|
||||
contents=["Test"],
|
||||
additional_properties={"user_id": "22222222-2222-2222-2222-222222222222"},
|
||||
)
|
||||
]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
mock_client.get_user_info_from_token.assert_called_once()
|
||||
assert user_id == "11111111-1111-1111-1111-111111111111"
|
||||
assert all(req.user_id == "11111111-1111-1111-1111-111111111111" for req in requests)
|
||||
|
||||
async def test_user_id_from_author_name_as_fallback(
|
||||
self, mock_client: AsyncMock, settings: PurviewSettings
|
||||
) -> None:
|
||||
"""Test user_id is extracted from author_name when it's a valid GUID."""
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [
|
||||
Message(
|
||||
role="user",
|
||||
contents=["Test"],
|
||||
author_name="33333333-3333-3333-3333-333333333333",
|
||||
)
|
||||
]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
assert user_id == "33333333-3333-3333-3333-333333333333"
|
||||
|
||||
async def test_author_name_ignored_if_not_valid_guid(
|
||||
self, mock_client: AsyncMock, settings: PurviewSettings
|
||||
) -> None:
|
||||
"""Test author_name is ignored if it's not a valid GUID."""
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [
|
||||
Message(
|
||||
role="user",
|
||||
contents=["Test"],
|
||||
author_name="John Doe", # Not a GUID
|
||||
)
|
||||
]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
# Should return empty since author_name is not a valid GUID
|
||||
assert user_id is None
|
||||
assert len(requests) == 0
|
||||
|
||||
async def test_provided_user_id_used_as_last_resort(
|
||||
self, mock_client: AsyncMock, settings: PurviewSettings
|
||||
) -> None:
|
||||
"""Test provided_user_id parameter is used as last resort."""
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [Message(role="user", contents=["Test"])]
|
||||
|
||||
requests, user_id = await processor._map_messages(
|
||||
messages, Activity.UPLOAD_TEXT, provided_user_id="44444444-4444-4444-4444-444444444444"
|
||||
)
|
||||
|
||||
assert user_id == "44444444-4444-4444-4444-444444444444"
|
||||
|
||||
async def test_invalid_provided_user_id_ignored(self, mock_client: AsyncMock, settings: PurviewSettings) -> None:
|
||||
"""Test invalid provided_user_id is ignored."""
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [Message(role="user", contents=["Test"])]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT, provided_user_id="not-a-guid")
|
||||
|
||||
assert user_id is None
|
||||
assert len(requests) == 0
|
||||
|
||||
async def test_multiple_messages_same_user_id(self, mock_client: AsyncMock, settings: PurviewSettings) -> None:
|
||||
"""Test that all messages use the same resolved user_id."""
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [
|
||||
Message(
|
||||
role="user",
|
||||
contents=["First"],
|
||||
additional_properties={"user_id": "55555555-5555-5555-5555-555555555555"},
|
||||
),
|
||||
Message(role="assistant", contents=["Response"]),
|
||||
Message(role="user", contents=["Second"]),
|
||||
]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
assert user_id == "55555555-5555-5555-5555-555555555555"
|
||||
# All requests should have the same user_id
|
||||
assert all(req.user_id == "55555555-5555-5555-5555-555555555555" for req in requests)
|
||||
|
||||
async def test_first_valid_user_id_in_messages_is_used(
|
||||
self, mock_client: AsyncMock, settings: PurviewSettings
|
||||
) -> None:
|
||||
"""Test that the first valid user_id found in messages is used for all."""
|
||||
processor = ScopedContentProcessor(mock_client, settings)
|
||||
mock_client.get_user_info_from_token.return_value = {
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
|
||||
messages = [
|
||||
Message(role="user", contents=["First"], author_name="Not a GUID"),
|
||||
Message(
|
||||
role="assistant",
|
||||
contents=["Response"],
|
||||
additional_properties={"user_id": "66666666-6666-6666-6666-666666666666"},
|
||||
),
|
||||
Message(
|
||||
role="user",
|
||||
contents=["Third"],
|
||||
additional_properties={"user_id": "77777777-7777-7777-7777-777777777777"},
|
||||
),
|
||||
]
|
||||
|
||||
requests, user_id = await processor._map_messages(messages, Activity.UPLOAD_TEXT)
|
||||
|
||||
# First valid user_id (from second message) should be used
|
||||
assert user_id == "66666666-6666-6666-6666-666666666666"
|
||||
assert all(req.user_id == "66666666-6666-6666-6666-666666666666" for req in requests)
|
||||
|
||||
|
||||
class TestScopedContentProcessorCaching:
|
||||
"""Test caching functionality in ScopedContentProcessor."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_client(self) -> AsyncMock:
|
||||
"""Create a mock Purview client."""
|
||||
client = AsyncMock()
|
||||
client.get_user_info_from_token = AsyncMock(
|
||||
return_value={
|
||||
"tenant_id": "12345678-1234-1234-1234-123456789012",
|
||||
"user_id": "12345678-1234-1234-1234-123456789012",
|
||||
"client_id": "12345678-1234-1234-1234-123456789012",
|
||||
}
|
||||
)
|
||||
client.get_protection_scopes = AsyncMock()
|
||||
return client
|
||||
|
||||
@pytest.fixture
|
||||
def settings(self) -> PurviewSettings:
|
||||
"""Create test settings."""
|
||||
location = PurviewAppLocation(location_type=PurviewLocationType.APPLICATION, location_value="app-id")
|
||||
return PurviewSettings(
|
||||
app_name="Test App",
|
||||
tenant_id="12345678-1234-1234-1234-123456789012",
|
||||
purview_app_location=location,
|
||||
)
|
||||
|
||||
async def test_protection_scopes_cached_on_first_call(
|
||||
self, mock_client: AsyncMock, settings: PurviewSettings
|
||||
) -> None:
|
||||
"""Test that protection scopes response is cached after first call."""
|
||||
from agent_framework_purview._cache import InMemoryCacheProvider
|
||||
from agent_framework_purview._models import ProtectionScopesResponse
|
||||
|
||||
cache_provider = InMemoryCacheProvider()
|
||||
processor = ScopedContentProcessor(mock_client, settings, cache_provider=cache_provider)
|
||||
|
||||
mock_client.get_protection_scopes.return_value = ProtectionScopesResponse(
|
||||
scope_identifier="scope-123", scopes=[]
|
||||
)
|
||||
mock_client.process_content.return_value = ProcessContentResponse(
|
||||
id="ok", protection_scope_state=ProtectionScopeState.NOT_MODIFIED
|
||||
)
|
||||
|
||||
messages = [Message(role="user", contents=["Test"])]
|
||||
|
||||
await processor.process_messages(messages, Activity.UPLOAD_TEXT, user_id="12345678-1234-1234-1234-123456789012")
|
||||
await asyncio.gather(*list(processor._background_tasks))
|
||||
|
||||
mock_client.get_protection_scopes.assert_called_once()
|
||||
|
||||
await processor.process_messages(messages, Activity.UPLOAD_TEXT, user_id="12345678-1234-1234-1234-123456789012")
|
||||
|
||||
mock_client.get_protection_scopes.assert_called_once()
|
||||
|
||||
async def test_payment_required_exception_cached_at_tenant_level(
|
||||
self, mock_client: AsyncMock, settings: PurviewSettings
|
||||
) -> None:
|
||||
"""Test that background scope 402 returns once, then throws from the tenant-level cache."""
|
||||
from agent_framework_purview._cache import InMemoryCacheProvider
|
||||
from agent_framework_purview._exceptions import PurviewPaymentRequiredError
|
||||
|
||||
cache_provider = InMemoryCacheProvider()
|
||||
processor = ScopedContentProcessor(mock_client, settings, cache_provider=cache_provider)
|
||||
|
||||
mock_client.get_protection_scopes.side_effect = PurviewPaymentRequiredError("Payment required")
|
||||
mock_client.process_content.return_value = ProcessContentResponse(
|
||||
id="ok", protection_scope_state=ProtectionScopeState.NOT_MODIFIED
|
||||
)
|
||||
|
||||
messages = [Message(role="user", contents=["Test"])]
|
||||
|
||||
await processor.process_messages(messages, Activity.UPLOAD_TEXT, user_id="12345678-1234-1234-1234-123456789012")
|
||||
await asyncio.gather(*list(processor._background_tasks))
|
||||
|
||||
mock_client.get_protection_scopes.assert_called_once()
|
||||
|
||||
with pytest.raises(PurviewPaymentRequiredError):
|
||||
await processor.process_messages(
|
||||
messages, Activity.UPLOAD_TEXT, user_id="12345678-1234-1234-1234-123456789012"
|
||||
)
|
||||
|
||||
mock_client.get_protection_scopes.assert_called_once()
|
||||
|
||||
async def test_custom_cache_provider_used(self, mock_client: AsyncMock, settings: PurviewSettings) -> None:
|
||||
"""Test that custom cache provider is used when provided."""
|
||||
from agent_framework_purview._cache import InMemoryCacheProvider
|
||||
|
||||
custom_cache = InMemoryCacheProvider(default_ttl_seconds=60)
|
||||
processor = ScopedContentProcessor(mock_client, settings, cache_provider=custom_cache)
|
||||
|
||||
assert processor._cache is custom_cache
|
||||
assert isinstance(processor._cache, InMemoryCacheProvider)
|
||||
assert processor._cache._default_ttl == 60
|
||||
@@ -0,0 +1,610 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for Purview client."""
|
||||
|
||||
from collections.abc import AsyncGenerator
|
||||
from typing import Any, cast
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from azure.core.credentials import AccessToken
|
||||
|
||||
from agent_framework_purview import PurviewSettings
|
||||
from agent_framework_purview._client import PurviewClient
|
||||
from agent_framework_purview._exceptions import (
|
||||
PurviewAuthenticationError,
|
||||
PurviewPaymentRequiredError,
|
||||
PurviewRateLimitError,
|
||||
PurviewRequestError,
|
||||
PurviewServiceError,
|
||||
)
|
||||
from agent_framework_purview._models import (
|
||||
ContentActivitiesRequest,
|
||||
ContentActivitiesResponse,
|
||||
PolicyLocation,
|
||||
ProcessContentRequest,
|
||||
ProtectionScopesRequest,
|
||||
)
|
||||
|
||||
|
||||
class TestPurviewClient:
|
||||
"""Test PurviewClient functionality."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_credential(self) -> MagicMock:
|
||||
"""Create a mock async credential."""
|
||||
from azure.core.credentials_async import AsyncTokenCredential
|
||||
|
||||
credential = MagicMock(spec=AsyncTokenCredential)
|
||||
mock_token = AccessToken("fake-token", 9999999999)
|
||||
|
||||
async def mock_get_token(*args, **kwargs):
|
||||
return mock_token
|
||||
|
||||
credential.get_token = mock_get_token
|
||||
return credential
|
||||
|
||||
@pytest.fixture
|
||||
def settings(self) -> PurviewSettings:
|
||||
"""Create test settings."""
|
||||
return PurviewSettings(app_name="Test App", tenant_id="test-tenant")
|
||||
|
||||
@pytest.fixture
|
||||
async def client(
|
||||
self, mock_credential: MagicMock, settings: PurviewSettings
|
||||
) -> AsyncGenerator[PurviewClient, None]:
|
||||
"""Create a PurviewClient with mock credential."""
|
||||
client = PurviewClient(mock_credential, settings, timeout=10.0)
|
||||
yield client
|
||||
await client.close()
|
||||
|
||||
async def test_client_initialization(self, mock_credential: MagicMock, settings: PurviewSettings) -> None:
|
||||
"""Test PurviewClient initialization."""
|
||||
client = PurviewClient(mock_credential, settings)
|
||||
|
||||
assert client._credential == mock_credential
|
||||
assert client._settings == settings
|
||||
assert client._graph_uri == "https://graph.microsoft.com/v1.0"
|
||||
assert client._timeout == 10.0
|
||||
|
||||
await client.close()
|
||||
|
||||
async def test_get_token_async_credential(self, client: PurviewClient, mock_credential: MagicMock) -> None:
|
||||
"""Test _get_token with async credential."""
|
||||
token = await client._get_token(tenant_id="test-tenant")
|
||||
|
||||
assert token == "fake-token"
|
||||
|
||||
async def test_get_token_sync_credential(self, settings: PurviewSettings) -> None:
|
||||
"""Test _get_token with sync credential."""
|
||||
sync_credential = MagicMock()
|
||||
sync_credential.get_token = MagicMock(return_value=AccessToken("sync-token", 9999999999))
|
||||
|
||||
client = PurviewClient(sync_credential, settings)
|
||||
|
||||
with patch("asyncio.get_running_loop") as mock_loop:
|
||||
mock_executor = AsyncMock()
|
||||
mock_executor.return_value = AccessToken("sync-token", 9999999999)
|
||||
mock_loop.return_value.run_in_executor = mock_executor
|
||||
|
||||
token = await client._get_token(tenant_id="test-tenant")
|
||||
|
||||
assert token == "sync-token"
|
||||
|
||||
await client.close()
|
||||
|
||||
async def test_get_user_info_from_token(self, client: PurviewClient) -> None:
|
||||
"""Test get_user_info_from_token extracts user info."""
|
||||
import base64
|
||||
import json
|
||||
|
||||
payload = {"tid": "test-tenant", "oid": "test-user", "idtyp": "user"}
|
||||
payload_str = json.dumps(payload)
|
||||
payload_bytes = payload_str.encode("utf-8")
|
||||
payload_b64 = base64.urlsafe_b64encode(payload_bytes).decode("utf-8").rstrip("=")
|
||||
fake_token = f"header.{payload_b64}.signature"
|
||||
|
||||
with patch.object(client, "_get_token", return_value=fake_token):
|
||||
user_info = await client.get_user_info_from_token(tenant_id="test-tenant")
|
||||
|
||||
assert user_info["tenant_id"] == "test-tenant"
|
||||
assert user_info["user_id"] == "test-user"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"status_code,exception_type",
|
||||
[
|
||||
(401, PurviewAuthenticationError),
|
||||
(403, PurviewAuthenticationError),
|
||||
(429, PurviewRateLimitError),
|
||||
(400, PurviewRequestError),
|
||||
(404, PurviewRequestError),
|
||||
(500, PurviewServiceError),
|
||||
(502, PurviewServiceError),
|
||||
],
|
||||
)
|
||||
async def test_post_error_handling(
|
||||
self, client: PurviewClient, content_to_process_factory, status_code: int, exception_type: type[Exception]
|
||||
) -> None:
|
||||
"""Test _post method handles different HTTP errors correctly."""
|
||||
from agent_framework_purview._models import ProcessContentResponse
|
||||
|
||||
content = content_to_process_factory()
|
||||
request = ProcessContentRequest(
|
||||
content_to_process=content,
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
)
|
||||
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = status_code
|
||||
mock_response.text = "Error message"
|
||||
mock_response.raise_for_status.side_effect = httpx.HTTPStatusError(
|
||||
"Error", request=MagicMock(), response=mock_response
|
||||
)
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response), pytest.raises(exception_type):
|
||||
await client._post(
|
||||
"https://graph.microsoft.com/v1.0/test",
|
||||
request,
|
||||
ProcessContentResponse,
|
||||
"fake-token",
|
||||
)
|
||||
|
||||
async def test_process_content_success(
|
||||
self, client: PurviewClient, content_to_process_factory, mock_credential: MagicMock
|
||||
) -> None:
|
||||
"""Test process_content method success path."""
|
||||
content = content_to_process_factory("Test message")
|
||||
request = ProcessContentRequest(
|
||||
content_to_process=content,
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
)
|
||||
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {}
|
||||
mock_response.json.return_value = {"id": "response-123", "protectionScopeState": "notModified"}
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response):
|
||||
response = await client.process_content(request)
|
||||
|
||||
assert response.id == "response-123"
|
||||
assert response.protection_scope_state == "notModified"
|
||||
|
||||
async def test_get_protection_scopes_success(self, client: PurviewClient) -> None:
|
||||
"""Test get_protection_scopes method success path."""
|
||||
location = PolicyLocation(**{"@odata.type": "microsoft.graph.policyLocationApplication", "value": "app-id"})
|
||||
request = ProtectionScopesRequest(
|
||||
user_id="user-123", tenant_id="tenant-456", locations=[location], correlation_id="corr-789"
|
||||
)
|
||||
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {} # Add headers attribute
|
||||
mock_response.json.return_value = {"scopeIdentifier": "scope-123", "value": []}
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response):
|
||||
response = await client.get_protection_scopes(request)
|
||||
|
||||
assert response.scope_identifier == "scope-123"
|
||||
assert response.scopes == []
|
||||
|
||||
async def test_get_protection_scopes_uses_etag_header_when_present(self, client: PurviewClient) -> None:
|
||||
"""Test that get_protection_scopes prefers the HTTP ETag header when present."""
|
||||
from agent_framework_purview._models import ProtectionScopesResponse
|
||||
|
||||
location = PolicyLocation(**{"@odata.type": "microsoft.graph.policyLocationApplication", "value": "app-id"})
|
||||
request = ProtectionScopesRequest(
|
||||
user_id="user-123", tenant_id="tenant-456", locations=[location], correlation_id="corr-789"
|
||||
)
|
||||
|
||||
response_obj = ProtectionScopesResponse(scope_identifier="scope-from-body", scopes=[])
|
||||
|
||||
with patch.object(
|
||||
client,
|
||||
"_post",
|
||||
return_value=(response_obj, {"etag": '"etag-from-header"'}),
|
||||
):
|
||||
response = await client.get_protection_scopes(request)
|
||||
|
||||
assert response.scope_identifier == "etag-from-header"
|
||||
|
||||
async def test_post_402_returns_empty_response_when_ignore_payment_required_enabled(
|
||||
self, mock_credential: MagicMock
|
||||
) -> None:
|
||||
"""Test that 402 is suppressed when ignore_payment_required=True."""
|
||||
from agent_framework_purview._models import ProcessContentResponse
|
||||
|
||||
settings = PurviewSettings(app_name="Test App", ignore_payment_required=True)
|
||||
client = PurviewClient(mock_credential, settings)
|
||||
|
||||
request = ProcessContentRequest(user_id="user-123", tenant_id="tenant-456", content_to_process=cast(Any, []))
|
||||
|
||||
resp = httpx.Response(402, text="Payment required", request=httpx.Request("POST", "http://test"))
|
||||
|
||||
with patch.object(client._client, "post", return_value=resp):
|
||||
result = await client._post("http://test", request, ProcessContentResponse, token="fake-token")
|
||||
|
||||
assert isinstance(result, ProcessContentResponse)
|
||||
await client.close()
|
||||
|
||||
async def test_post_sets_request_and_response_correlation_id(self, client: PurviewClient) -> None:
|
||||
"""Test that correlation_id is injected into request headers and hydrated from response headers."""
|
||||
from agent_framework_purview._models import ProcessContentResponse
|
||||
|
||||
# correlation_id is optional and should be auto-generated when empty
|
||||
request = ProcessContentRequest(user_id="user-123", tenant_id="tenant-456", content_to_process=cast(Any, []))
|
||||
request.correlation_id = "" # force auto-generation branch
|
||||
|
||||
captured_headers: dict[str, str] = {}
|
||||
|
||||
async def fake_post(url: str, json=None, headers=None):
|
||||
nonlocal captured_headers
|
||||
captured_headers = dict(headers or {})
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={"id": "resp-1", "protectionScopeState": "notModified"},
|
||||
headers={"client-request-id": "corr-from-response"},
|
||||
request=httpx.Request("POST", url),
|
||||
)
|
||||
|
||||
with patch.object(client._client, "post", side_effect=fake_post):
|
||||
result_obj, result_headers = await client._post(
|
||||
"http://test",
|
||||
request,
|
||||
ProcessContentResponse,
|
||||
token="fake-token",
|
||||
return_response=True,
|
||||
)
|
||||
|
||||
assert "client-request-id" in captured_headers
|
||||
assert captured_headers["client-request-id"]
|
||||
assert result_headers["client-request-id"] == "corr-from-response"
|
||||
assert result_obj.correlation_id == "corr-from-response"
|
||||
|
||||
async def test_process_content_402_returns_empty_when_ignored(self, mock_credential: MagicMock) -> None:
|
||||
"""Test that process_content returns an empty response (non-tuple path) when 402 is ignored."""
|
||||
from agent_framework_purview._models import ProcessContentResponse
|
||||
|
||||
settings = PurviewSettings(app_name="Test App", ignore_payment_required=True)
|
||||
client = PurviewClient(mock_credential, settings)
|
||||
|
||||
req = ProcessContentRequest(user_id="user-123", tenant_id="tenant-456", content_to_process=cast(Any, []))
|
||||
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 402
|
||||
mock_response.text = "Payment required"
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response):
|
||||
response = await client.process_content(req)
|
||||
|
||||
assert isinstance(response, ProcessContentResponse)
|
||||
await client.close()
|
||||
|
||||
async def test_post_sets_correlation_id_attribute_on_recording_span(self, client: PurviewClient) -> None:
|
||||
"""Test that correlation_id is added to the active span when recording is enabled."""
|
||||
from agent_framework_purview._models import ProcessContentResponse
|
||||
|
||||
request = ProcessContentRequest(user_id="user-123", tenant_id="tenant-456", content_to_process=cast(Any, []))
|
||||
request.correlation_id = "corr-123"
|
||||
|
||||
class RecordingSpan:
|
||||
def __init__(self) -> None:
|
||||
self.attributes: dict[str, str] = {}
|
||||
|
||||
def is_recording(self) -> bool:
|
||||
return True
|
||||
|
||||
def set_attribute(self, key: str, value: str) -> None:
|
||||
self.attributes[key] = value
|
||||
|
||||
span = RecordingSpan()
|
||||
|
||||
with (
|
||||
patch("agent_framework_purview._client.trace.get_current_span", return_value=span),
|
||||
patch.object(
|
||||
client._client,
|
||||
"post",
|
||||
return_value=httpx.Response(
|
||||
200,
|
||||
json={"id": "resp-1", "protectionScopeState": "notModified"},
|
||||
headers={},
|
||||
request=httpx.Request("POST", "http://test"),
|
||||
),
|
||||
),
|
||||
):
|
||||
await client._post("http://test", request, ProcessContentResponse, token="fake-token")
|
||||
|
||||
assert span.attributes["correlation_id"] == "corr-123"
|
||||
|
||||
async def test_post_uses_constructor_when_response_type_has_no_model_validate(self, client: PurviewClient) -> None:
|
||||
"""Test that _post falls back to the response type constructor when model_validate is absent."""
|
||||
|
||||
class DummyResponse:
|
||||
def __init__(self, **data):
|
||||
self.data = data
|
||||
|
||||
request = ProcessContentRequest(user_id="user-123", tenant_id="tenant-456", content_to_process=cast(Any, []))
|
||||
request.correlation_id = "corr-123"
|
||||
|
||||
with patch.object(
|
||||
client._client,
|
||||
"post",
|
||||
return_value=httpx.Response(
|
||||
200,
|
||||
json={"hello": "world"},
|
||||
headers={},
|
||||
request=httpx.Request("POST", "http://test"),
|
||||
),
|
||||
):
|
||||
result = await client._post("http://test", request, DummyResponse, token="fake-token")
|
||||
|
||||
assert isinstance(result, DummyResponse)
|
||||
assert result.data == {"hello": "world"}
|
||||
|
||||
async def test_send_content_activities_success(self, client: PurviewClient, content_to_process_factory) -> None:
|
||||
"""Test send_content_activities success path."""
|
||||
request = ContentActivitiesRequest(
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
content_to_process=content_to_process_factory("hello"),
|
||||
correlation_id="corr-1",
|
||||
)
|
||||
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {}
|
||||
mock_response.json.return_value = {"error": None}
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response):
|
||||
resp = await client.send_content_activities(request)
|
||||
|
||||
assert isinstance(resp, ContentActivitiesResponse)
|
||||
|
||||
async def test_post_handles_invalid_json_response_body(self, client: PurviewClient) -> None:
|
||||
"""Test that invalid JSON bodies fall back to an empty dict."""
|
||||
request = ProcessContentRequest(user_id="user-123", tenant_id="tenant-456", content_to_process=cast(Any, []))
|
||||
request.correlation_id = "corr-123"
|
||||
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {}
|
||||
mock_response.json.side_effect = ValueError("not json")
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response):
|
||||
result = await client._post("http://test", request, ContentActivitiesResponse, token="fake-token")
|
||||
|
||||
assert isinstance(result, ContentActivitiesResponse)
|
||||
|
||||
async def test_post_deserialization_failure_raises_purview_service_error(self, client: PurviewClient) -> None:
|
||||
"""Test that response deserialization errors are wrapped as PurviewServiceError."""
|
||||
|
||||
class BadResponseType:
|
||||
@classmethod
|
||||
def model_validate(cls, value):
|
||||
raise RuntimeError("boom")
|
||||
|
||||
request = ProcessContentRequest(user_id="user-123", tenant_id="tenant-456", content_to_process=cast(Any, []))
|
||||
request.correlation_id = "corr-123"
|
||||
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {}
|
||||
mock_response.json.return_value = {"any": "data"}
|
||||
|
||||
with (
|
||||
patch.object(client._client, "post", return_value=mock_response),
|
||||
pytest.raises(PurviewServiceError, match="Failed to deserialize Purview response"),
|
||||
):
|
||||
await client._post("http://test", request, BadResponseType, token="fake-token")
|
||||
|
||||
async def test_client_close(self, mock_credential: AsyncMock, settings: PurviewSettings) -> None:
|
||||
"""Test client properly closes HTTP client."""
|
||||
client = PurviewClient(mock_credential, settings)
|
||||
|
||||
with patch.object(client._client, "aclose", new_callable=AsyncMock) as mock_close:
|
||||
await client.close()
|
||||
mock_close.assert_called_once()
|
||||
|
||||
async def test_invalid_jwt_token_format(self, client: PurviewClient) -> None:
|
||||
"""Test that invalid JWT token format raises ValueError."""
|
||||
with pytest.raises(ValueError, match="Invalid JWT token format"):
|
||||
client._extract_token_info("invalid-token-without-dots")
|
||||
|
||||
async def test_rate_limit_error(self, client: PurviewClient) -> None:
|
||||
"""Test that 429 status code raises PurviewRateLimitError."""
|
||||
request = ProcessContentRequest(
|
||||
user_id="test-user",
|
||||
tenant_id="test-tenant",
|
||||
content_to_process=cast(Any, []),
|
||||
correlation_id="test-correlation-id",
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(client, "_get_token", return_value="fake-token"),
|
||||
patch.object(
|
||||
client._client,
|
||||
"post",
|
||||
return_value=httpx.Response(429, text="Rate limited", request=httpx.Request("POST", "http://test")),
|
||||
),
|
||||
pytest.raises(PurviewRateLimitError, match="Rate limited"),
|
||||
):
|
||||
await client.process_content(request)
|
||||
|
||||
async def test_generic_request_error(self, client: PurviewClient) -> None:
|
||||
"""Test that non-200/201/202 status codes raise PurviewRequestError."""
|
||||
request = ProcessContentRequest(
|
||||
user_id="test-user",
|
||||
tenant_id="test-tenant",
|
||||
content_to_process=cast(Any, []),
|
||||
correlation_id="test-correlation-id",
|
||||
)
|
||||
|
||||
with (
|
||||
patch.object(client, "_get_token", return_value="fake-token"),
|
||||
patch.object(
|
||||
client._client,
|
||||
"post",
|
||||
return_value=httpx.Response(
|
||||
500, text="Internal server error", request=httpx.Request("POST", "http://test")
|
||||
),
|
||||
),
|
||||
pytest.raises(PurviewRequestError, match="Purview request failed"),
|
||||
):
|
||||
await client.process_content(request)
|
||||
|
||||
async def test_prefer_header_sent_when_process_inline_true(
|
||||
self, client: PurviewClient, content_to_process_factory
|
||||
) -> None:
|
||||
"""Test that Prefer: evaluateInline header is sent when process_inline is True."""
|
||||
content = content_to_process_factory()
|
||||
request = ProcessContentRequest(
|
||||
content_to_process=content,
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
process_inline=True,
|
||||
)
|
||||
|
||||
posted_headers: dict[str, str] = {}
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {}
|
||||
mock_response.json.return_value = {}
|
||||
|
||||
async def capture_post(url, json, headers):
|
||||
posted_headers.update(headers)
|
||||
return mock_response
|
||||
|
||||
with patch.object(client._client, "post", side_effect=capture_post):
|
||||
await client.process_content(request)
|
||||
|
||||
assert "Prefer" in posted_headers
|
||||
assert posted_headers["Prefer"] == "evaluateInline"
|
||||
|
||||
async def test_prefer_header_not_sent_when_process_inline_false(
|
||||
self, client: PurviewClient, content_to_process_factory
|
||||
) -> None:
|
||||
"""Test that Prefer header is not sent when process_inline is False."""
|
||||
content = content_to_process_factory()
|
||||
request = ProcessContentRequest(
|
||||
content_to_process=content,
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
process_inline=False,
|
||||
)
|
||||
|
||||
posted_headers: dict[str, str] = {}
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {}
|
||||
mock_response.json.return_value = {}
|
||||
|
||||
async def capture_post(url, json, headers):
|
||||
posted_headers.update(headers)
|
||||
return mock_response
|
||||
|
||||
with patch.object(client._client, "post", side_effect=capture_post):
|
||||
await client.process_content(request)
|
||||
|
||||
assert "Prefer" not in posted_headers
|
||||
|
||||
async def test_prefer_header_not_sent_when_process_inline_none(
|
||||
self, client: PurviewClient, content_to_process_factory
|
||||
) -> None:
|
||||
"""Test that Prefer header is not sent when process_inline is None."""
|
||||
content = content_to_process_factory()
|
||||
request = ProcessContentRequest(
|
||||
content_to_process=content,
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
process_inline=None,
|
||||
)
|
||||
|
||||
posted_headers: dict[str, str] = {}
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {}
|
||||
mock_response.json.return_value = {}
|
||||
|
||||
async def capture_post(url, json, headers):
|
||||
posted_headers.update(headers)
|
||||
return mock_response
|
||||
|
||||
with patch.object(client._client, "post", side_effect=capture_post):
|
||||
await client.process_content(request)
|
||||
|
||||
assert "Prefer" not in posted_headers
|
||||
|
||||
async def test_scope_identifier_extraction_from_etag(self, client: PurviewClient) -> None:
|
||||
"""Test that scope_identifier is extracted from ETag header."""
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {"etag": '"test-scope-id"'}
|
||||
mock_response.json.return_value = {"value": []}
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response):
|
||||
req = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1")
|
||||
response = await client.get_protection_scopes(req)
|
||||
|
||||
assert response.scope_identifier == "test-scope-id"
|
||||
|
||||
async def test_scope_identifier_sent_as_if_none_match_header(
|
||||
self, client: PurviewClient, content_to_process_factory
|
||||
) -> None:
|
||||
"""Test that scope_identifier is sent as If-None-Match header."""
|
||||
content = content_to_process_factory()
|
||||
request = ProcessContentRequest(
|
||||
content_to_process=content,
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
scope_identifier="test-scope-id",
|
||||
)
|
||||
|
||||
posted_headers: dict[str, str] = {}
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 200
|
||||
mock_response.headers = {}
|
||||
mock_response.json.return_value = {}
|
||||
|
||||
async def capture_post(url, json, headers):
|
||||
posted_headers.update(headers)
|
||||
return mock_response
|
||||
|
||||
with patch.object(client._client, "post", side_effect=capture_post):
|
||||
await client.process_content(request)
|
||||
|
||||
assert "If-None-Match" in posted_headers
|
||||
assert posted_headers["If-None-Match"] == "test-scope-id"
|
||||
|
||||
async def test_402_payment_required_raises_exception_by_default(self, client: PurviewClient) -> None:
|
||||
"""Test that 402 raises exception when ignore_payment_required is False."""
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 402
|
||||
mock_response.text = "Payment required"
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response):
|
||||
req = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1")
|
||||
|
||||
with pytest.raises(PurviewPaymentRequiredError):
|
||||
await client.get_protection_scopes(req)
|
||||
|
||||
async def test_402_payment_required_returns_empty_when_ignored(self, mock_credential: MagicMock) -> None:
|
||||
"""Test that 402 returns empty response when ignore_payment_required is True."""
|
||||
settings = PurviewSettings(app_name="Test App", ignore_payment_required=True)
|
||||
client = PurviewClient(mock_credential, settings)
|
||||
|
||||
mock_response = MagicMock(spec=httpx.Response)
|
||||
mock_response.status_code = 402
|
||||
mock_response.text = "Payment required"
|
||||
|
||||
with patch.object(client._client, "post", return_value=mock_response):
|
||||
req = ProtectionScopesRequest(user_id="user1", tenant_id="tenant1")
|
||||
response = await client.get_protection_scopes(req)
|
||||
|
||||
# Should return empty response without raising
|
||||
assert response is not None
|
||||
assert response.scopes is None or response.scopes == []
|
||||
|
||||
await client.close()
|
||||
@@ -0,0 +1,246 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for Purview models and serialization."""
|
||||
|
||||
from agent_framework_purview._models import (
|
||||
Activity,
|
||||
ActivityMetadata,
|
||||
ContentToProcess,
|
||||
DeviceMetadata,
|
||||
IntegratedAppMetadata,
|
||||
OperatingSystemSpecifications,
|
||||
PolicyLocation,
|
||||
ProcessContentRequest,
|
||||
ProcessContentResponse,
|
||||
ProcessConversationMetadata,
|
||||
ProtectedAppMetadata,
|
||||
ProtectionScopeActivities,
|
||||
ProtectionScopesRequest,
|
||||
ProtectionScopesResponse,
|
||||
PurviewTextContent,
|
||||
deserialize_flag,
|
||||
serialize_flag,
|
||||
)
|
||||
|
||||
|
||||
class TestFlagOperations:
|
||||
"""Test flag serialization and deserialization operations."""
|
||||
|
||||
def test_protection_scope_activities_flag_combination(self) -> None:
|
||||
"""Test combining flags."""
|
||||
combined = ProtectionScopeActivities.UPLOAD_TEXT | ProtectionScopeActivities.UPLOAD_FILE
|
||||
assert combined.value == 3
|
||||
assert ProtectionScopeActivities.UPLOAD_TEXT in combined
|
||||
assert ProtectionScopeActivities.UPLOAD_FILE in combined
|
||||
|
||||
def test_deserialize_flag_with_string(self) -> None:
|
||||
"""Test deserializing flag from comma-separated string."""
|
||||
mapping = {
|
||||
"uploadText": ProtectionScopeActivities.UPLOAD_TEXT,
|
||||
"uploadFile": ProtectionScopeActivities.UPLOAD_FILE,
|
||||
}
|
||||
|
||||
result = deserialize_flag("uploadText,uploadFile", mapping, ProtectionScopeActivities)
|
||||
assert result is not None
|
||||
assert ProtectionScopeActivities.UPLOAD_TEXT in result
|
||||
assert ProtectionScopeActivities.UPLOAD_FILE in result
|
||||
|
||||
def test_deserialize_flag_with_none(self) -> None:
|
||||
"""Test deserializing None returns None."""
|
||||
mapping = {"uploadText": ProtectionScopeActivities.UPLOAD_TEXT}
|
||||
result = deserialize_flag(None, mapping, ProtectionScopeActivities)
|
||||
assert result is None
|
||||
|
||||
def test_serialize_flag_with_none(self) -> None:
|
||||
"""Test serializing None returns None."""
|
||||
result = serialize_flag(None, [])
|
||||
assert result is None
|
||||
|
||||
def test_serialize_flag_with_values(self) -> None:
|
||||
"""Test serializing flag with values."""
|
||||
flag = ProtectionScopeActivities.UPLOAD_TEXT | ProtectionScopeActivities.UPLOAD_FILE
|
||||
ordered = [
|
||||
("uploadText", ProtectionScopeActivities.UPLOAD_TEXT),
|
||||
("uploadFile", ProtectionScopeActivities.UPLOAD_FILE),
|
||||
]
|
||||
result = serialize_flag(flag, ordered)
|
||||
assert result == "uploadText,uploadFile"
|
||||
|
||||
|
||||
class TestComplexModels:
|
||||
"""Test complex models with nested structures."""
|
||||
|
||||
def test_content_to_process_with_nested_structures(self) -> None:
|
||||
"""Test ContentToProcess with all nested structures."""
|
||||
text_content = PurviewTextContent(data="Test")
|
||||
metadata = ProcessConversationMetadata(
|
||||
identifier="msg-1",
|
||||
content=text_content,
|
||||
name="Test",
|
||||
is_truncated=False,
|
||||
)
|
||||
|
||||
activity_meta = ActivityMetadata(activity=Activity.UPLOAD_TEXT)
|
||||
device_meta = DeviceMetadata(
|
||||
operating_system_specifications=OperatingSystemSpecifications(
|
||||
operating_system_platform="Windows", operating_system_version="10"
|
||||
)
|
||||
)
|
||||
integrated_app = IntegratedAppMetadata(name="App", version="1.0")
|
||||
location = PolicyLocation(data_type="microsoft.graph.policyLocationApplication", value="app-id")
|
||||
protected_app = ProtectedAppMetadata(name="Protected", version="1.0", application_location=location)
|
||||
|
||||
content = ContentToProcess(
|
||||
content_entries=[metadata],
|
||||
activity_metadata=activity_meta,
|
||||
device_metadata=device_meta,
|
||||
integrated_app_metadata=integrated_app,
|
||||
protected_app_metadata=protected_app,
|
||||
)
|
||||
|
||||
assert len(content.content_entries) == 1
|
||||
assert content.activity_metadata.activity == Activity.UPLOAD_TEXT
|
||||
os_specs = content.device_metadata.operating_system_specifications
|
||||
assert os_specs is not None
|
||||
assert os_specs.operating_system_platform == "Windows"
|
||||
assert content.integrated_app_metadata.name == "App"
|
||||
assert content.protected_app_metadata.name == "Protected"
|
||||
|
||||
|
||||
class TestRequestResponseSerialization:
|
||||
"""Test request/response serialization with aliases."""
|
||||
|
||||
def test_protection_scopes_request_serialization(self) -> None:
|
||||
"""Test ProtectionScopesRequest serializes activities correctly."""
|
||||
location = PolicyLocation(data_type="microsoft.graph.policyLocationApplication", value="app-id")
|
||||
|
||||
request = ProtectionScopesRequest(
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
activities=ProtectionScopeActivities.UPLOAD_TEXT | ProtectionScopeActivities.UPLOAD_FILE,
|
||||
locations=[location],
|
||||
)
|
||||
|
||||
dumped = request.model_dump(by_alias=True, exclude_none=True, mode="json")
|
||||
|
||||
assert "activities" in dumped
|
||||
assert isinstance(dumped["activities"], str)
|
||||
assert "uploadText" in dumped["activities"]
|
||||
|
||||
|
||||
class TestModelDeserialization:
|
||||
"""Test model deserialization from API responses."""
|
||||
|
||||
def test_protection_scopes_response_deserialization(self) -> None:
|
||||
"""Test ProtectionScopesResponse deserializes 'value' to 'scopes'."""
|
||||
api_data = {
|
||||
"scopeIdentifier": "scope-123",
|
||||
"value": [
|
||||
{
|
||||
"activities": "uploadText,downloadText",
|
||||
"locations": [{"@odata.type": "location.type", "value": "/path"}],
|
||||
"policyActions": [{"action": "warn", "restrictionAction": "blockAccess"}],
|
||||
"executionMode": "evaluateInline",
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
response = ProtectionScopesResponse.model_validate(api_data)
|
||||
|
||||
assert response.scope_identifier == "scope-123"
|
||||
assert response.scopes is not None
|
||||
assert len(response.scopes) == 1
|
||||
assert response.scopes[0].execution_mode == "evaluateInline"
|
||||
|
||||
def test_process_content_response_deserialization(self) -> None:
|
||||
"""Test ProcessContentResponse deserializes aliased fields correctly."""
|
||||
api_data = {
|
||||
"id": "response-123",
|
||||
"protectionScopeState": "blocked",
|
||||
"policyActions": [{"action": "block", "restrictionAction": "blockAccess"}],
|
||||
}
|
||||
|
||||
response = ProcessContentResponse.model_validate(api_data)
|
||||
|
||||
assert response.id == "response-123"
|
||||
assert response.protection_scope_state == "blocked"
|
||||
assert response.policy_actions is not None
|
||||
assert len(response.policy_actions) == 1
|
||||
|
||||
def test_content_serialization_uses_aliases(self) -> None:
|
||||
"""Test ContentToProcess serializes with camelCase aliases."""
|
||||
text_content = PurviewTextContent(data="Test")
|
||||
metadata = ProcessConversationMetadata(
|
||||
identifier="msg-1",
|
||||
content=text_content,
|
||||
name="Test",
|
||||
is_truncated=False,
|
||||
)
|
||||
|
||||
activity_meta = ActivityMetadata(activity=Activity.UPLOAD_TEXT)
|
||||
device_meta = DeviceMetadata(
|
||||
operating_system_specifications=OperatingSystemSpecifications(
|
||||
operating_system_platform="Windows", operating_system_version="10"
|
||||
)
|
||||
)
|
||||
integrated_app = IntegratedAppMetadata(name="App", version="1.0")
|
||||
location = PolicyLocation(data_type="microsoft.graph.policyLocationApplication", value="app-id")
|
||||
protected_app = ProtectedAppMetadata(name="Protected", version="1.0", application_location=location)
|
||||
|
||||
content = ContentToProcess(
|
||||
content_entries=[metadata],
|
||||
activity_metadata=activity_meta,
|
||||
device_metadata=device_meta,
|
||||
integrated_app_metadata=integrated_app,
|
||||
protected_app_metadata=protected_app,
|
||||
)
|
||||
|
||||
dumped = content.model_dump(by_alias=True, exclude_none=True, mode="json")
|
||||
|
||||
assert "contentEntries" in dumped
|
||||
assert "activityMetadata" in dumped
|
||||
assert "deviceMetadata" in dumped
|
||||
assert "integratedAppMetadata" in dumped
|
||||
assert "protectedAppMetadata" in dumped
|
||||
|
||||
def test_process_content_request_excludes_private_fields(self) -> None:
|
||||
"""Test ProcessContentRequest excludes private fields when serializing."""
|
||||
text_content = PurviewTextContent(data="Test")
|
||||
metadata = ProcessConversationMetadata(
|
||||
identifier="msg-1",
|
||||
content=text_content,
|
||||
name="Test",
|
||||
is_truncated=False,
|
||||
)
|
||||
|
||||
activity_meta = ActivityMetadata(activity=Activity.UPLOAD_TEXT)
|
||||
device_meta = DeviceMetadata(
|
||||
operating_system_specifications=OperatingSystemSpecifications(
|
||||
operating_system_platform="Windows", operating_system_version="10"
|
||||
)
|
||||
)
|
||||
integrated_app = IntegratedAppMetadata(name="App", version="1.0")
|
||||
location = PolicyLocation(data_type="microsoft.graph.policyLocationApplication", value="app-id")
|
||||
protected_app = ProtectedAppMetadata(name="Protected", version="1.0", application_location=location)
|
||||
|
||||
content = ContentToProcess(
|
||||
content_entries=[metadata],
|
||||
activity_metadata=activity_meta,
|
||||
device_metadata=device_meta,
|
||||
integrated_app_metadata=integrated_app,
|
||||
protected_app_metadata=protected_app,
|
||||
)
|
||||
|
||||
request = ProcessContentRequest(
|
||||
content_to_process=content,
|
||||
user_id="user-123",
|
||||
tenant_id="tenant-456",
|
||||
correlation_id="corr-789",
|
||||
)
|
||||
|
||||
dumped = request.model_dump(by_alias=True, exclude_none=True, mode="json")
|
||||
|
||||
assert "user_id" in dumped
|
||||
assert "tenant_id" in dumped
|
||||
assert "correlation_id" not in dumped
|
||||
assert "contentToProcess" in dumped
|
||||
@@ -0,0 +1,83 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for Purview settings."""
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_framework_purview import PurviewAppLocation, PurviewLocationType, PurviewSettings, get_purview_scopes
|
||||
|
||||
|
||||
class TestPurviewSettings:
|
||||
"""Test PurviewSettings configuration."""
|
||||
|
||||
def test_settings_defaults(self) -> None:
|
||||
"""Test PurviewSettings with default values."""
|
||||
settings = PurviewSettings(app_name="Test App")
|
||||
|
||||
assert settings["app_name"] == "Test App"
|
||||
assert settings.get("graph_base_uri") is None
|
||||
assert settings.get("tenant_id") is None
|
||||
assert settings.get("purview_app_location") is None
|
||||
|
||||
def test_settings_with_custom_values(self) -> None:
|
||||
"""Test PurviewSettings with custom values."""
|
||||
app_location = PurviewAppLocation(location_type=PurviewLocationType.APPLICATION, location_value="app-123")
|
||||
|
||||
settings = PurviewSettings(
|
||||
app_name="Test App",
|
||||
graph_base_uri="https://graph.microsoft-ppe.com",
|
||||
tenant_id="test-tenant-id",
|
||||
purview_app_location=app_location,
|
||||
)
|
||||
|
||||
assert settings["graph_base_uri"] == "https://graph.microsoft-ppe.com"
|
||||
assert settings["tenant_id"] == "test-tenant-id"
|
||||
assert settings["purview_app_location"] is not None
|
||||
assert settings["purview_app_location"].location_value == "app-123"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"graph_uri,expected_scope",
|
||||
[
|
||||
("https://graph.microsoft.com/v1.0/", "https://graph.microsoft.com/.default"),
|
||||
("https://graph.microsoft-ppe.com/v1.0/", "https://graph.microsoft-ppe.com/.default"),
|
||||
],
|
||||
)
|
||||
def test_get_scopes(self, graph_uri: str, expected_scope: str) -> None:
|
||||
"""Test get_scopes returns correct scope for different URIs."""
|
||||
settings = PurviewSettings(app_name="Test App", graph_base_uri=graph_uri)
|
||||
scopes = get_purview_scopes(settings)
|
||||
|
||||
assert len(scopes) == 1
|
||||
assert expected_scope in scopes
|
||||
|
||||
|
||||
class TestPurviewAppLocation:
|
||||
"""Test PurviewAppLocation configuration."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"location_type,location_value,expected_odata_type",
|
||||
[
|
||||
(PurviewLocationType.APPLICATION, "app-123", "microsoft.graph.policyLocationApplication"),
|
||||
(PurviewLocationType.URI, "https://example.com", "microsoft.graph.policyLocationUrl"),
|
||||
(PurviewLocationType.DOMAIN, "example.com", "microsoft.graph.policyLocationDomain"),
|
||||
],
|
||||
)
|
||||
def test_get_policy_location(
|
||||
self, location_type: PurviewLocationType, location_value: str, expected_odata_type: str
|
||||
) -> None:
|
||||
"""Test get_policy_location returns correct structure for all location types."""
|
||||
location = PurviewAppLocation(location_type=location_type, location_value=location_value)
|
||||
policy_location = location.get_policy_location()
|
||||
|
||||
assert policy_location["@odata.type"] == expected_odata_type
|
||||
assert policy_location["value"] == location_value
|
||||
|
||||
|
||||
class TestPurviewLocationType:
|
||||
"""Test PurviewLocationType enum."""
|
||||
|
||||
def test_location_type_values(self) -> None:
|
||||
"""Test PurviewLocationType enum has expected values."""
|
||||
assert PurviewLocationType.APPLICATION == "application"
|
||||
assert PurviewLocationType.URI == "uri"
|
||||
assert PurviewLocationType.DOMAIN == "domain"
|
||||
Reference in New Issue
Block a user