wehub-resource-sync
2379 lines
76 KiB
Markdown
2379 lines
76 KiB
Markdown
# MCP Development Best Practices
|
|
|
|
[](https://youtu.be/W56H9W7x-ao)
|
|
|
|
_(Click the image above to view video of this lesson)_
|
|
|
|
## Overview
|
|
|
|
This lesson focuses on advanced best practices for developing, testing, and deploying MCP servers and features in production environments. As MCP ecosystems grow in complexity and importance, following established patterns ensures reliability, maintainability, and interoperability. This lesson consolidates practical wisdom gained from real-world MCP implementations to guide you in creating robust, efficient servers with effective resources, prompts, and tools.
|
|
|
|
## Learning Objectives
|
|
|
|
By the end of this lesson, you will be able to:
|
|
|
|
- Apply industry best practices in MCP server and feature design
|
|
- Create comprehensive testing strategies for MCP servers
|
|
- Design efficient, reusable workflow patterns for complex MCP applications
|
|
- Implement proper error handling, logging, and observability in MCP servers
|
|
- Optimize MCP implementations for performance, security, and maintainability
|
|
|
|
## MCP Core Principles
|
|
|
|
Before diving into specific implementation practices, it's important to understand the core principles that guide effective MCP development:
|
|
|
|
1. **Standardized Communication**: MCP uses JSON-RPC 2.0 as its foundation, providing a consistent format for requests, responses, and error handling across all implementations.
|
|
|
|
2. **User-Centric Design**: Always prioritize user consent, control, and transparency in your MCP implementations.
|
|
|
|
3. **Security First**: Implement robust security measures including authentication, authorization, validation, and rate limiting.
|
|
|
|
4. **Modular Architecture**: Design your MCP servers with a modular approach, where each tool and resource has a clear, focused purpose.
|
|
|
|
5. **Stateful Connections**: Leverage MCP's ability to maintain state across multiple requests for more coherent and context-aware interactions.
|
|
|
|
## Official MCP Best Practices
|
|
|
|
The following best practices are derived from the official Model Context Protocol documentation:
|
|
|
|
### Security Best Practices
|
|
|
|
1. **User Consent and Control**: Always require explicit user consent before accessing data or performing operations. Provide clear control over what data is shared and which actions are authorized.
|
|
|
|
2. **Data Privacy**: Only expose user data with explicit consent and protect it with appropriate access controls. Safeguard against unauthorized data transmission.
|
|
|
|
3. **Tool Safety**: Require explicit user consent before invoking any tool. Ensure users understand each tool's functionality and enforce robust security boundaries.
|
|
|
|
4. **Tool Permission Control**: Configure which tools a model is allowed to use during a session, ensuring only explicitly authorized tools are accessible.
|
|
|
|
5. **Authentication**: Require proper authentication before granting access to tools, resources, or sensitive operations using API keys, OAuth tokens, or other secure authentication methods.
|
|
|
|
6. **Parameter Validation**: Enforce validation for all tool invocations to prevent malformed or malicious input from reaching tool implementations.
|
|
|
|
7. **Rate Limiting**: Implement rate limiting to prevent abuse and ensure fair usage of server resources.
|
|
|
|
### Implementation Best Practices
|
|
|
|
1. **Capability Negotiation**: During connection setup, exchange information about supported features, protocol versions, available tools, and resources.
|
|
|
|
2. **Tool Design**: Create focused tools that do one thing well, rather than monolithic tools that handle multiple concerns.
|
|
|
|
3. **Error Handling**: Implement standardized error messages and codes to help diagnose issues, handle failures gracefully, and provide actionable feedback.
|
|
|
|
4. **Logging**: Configure structured logs for auditing, debugging, and monitoring protocol interactions.
|
|
|
|
5. **Progress Tracking**: For long-running operations, report progress updates to enable responsive user interfaces.
|
|
|
|
6. **Request Cancellation**: Allow clients to cancel in-flight requests that are no longer needed or taking too long.
|
|
|
|
## Additional References
|
|
|
|
For the most up-to-date information on MCP best practices, refer to:
|
|
|
|
- [MCP Documentation](https://modelcontextprotocol.io/)
|
|
- [MCP Specification (2025-11-25)](https://spec.modelcontextprotocol.io/specification/2025-11-25/)
|
|
- [GitHub Repository](https://github.com/modelcontextprotocol)
|
|
- [Security Best Practices](https://modelcontextprotocol.io/specification/draft/basic/security_best_practices)
|
|
- [OWASP MCP Top 10](https://microsoft.github.io/mcp-azure-security-guide/mcp/) - Security risks and mitigations
|
|
- [MCP Security Summit Workshop (Sherpa)](https://azure-samples.github.io/sherpa/) - Hands-on security training
|
|
|
|
## Practical Implementation Examples
|
|
|
|
### Tool Design Best Practices
|
|
|
|
#### 1. Single Responsibility Principle
|
|
|
|
Each MCP tool should have a clear, focused purpose. Rather than creating monolithic tools that attempt to handle multiple concerns, develop specialized tools that excel at specific tasks.
|
|
|
|
```csharp
|
|
// A focused tool that does one thing well
|
|
public class WeatherForecastTool : ITool
|
|
{
|
|
private readonly IWeatherService _weatherService;
|
|
|
|
public WeatherForecastTool(IWeatherService weatherService)
|
|
{
|
|
_weatherService = weatherService;
|
|
}
|
|
|
|
public string Name => "weatherForecast";
|
|
public string Description => "Gets weather forecast for a specific location";
|
|
|
|
public ToolDefinition GetDefinition()
|
|
{
|
|
return new ToolDefinition
|
|
{
|
|
Name = Name,
|
|
Description = Description,
|
|
Parameters = new Dictionary<string, ParameterDefinition>
|
|
{
|
|
["location"] = new ParameterDefinition
|
|
{
|
|
Type = ParameterType.String,
|
|
Description = "City or location name"
|
|
},
|
|
["days"] = new ParameterDefinition
|
|
{
|
|
Type = ParameterType.Integer,
|
|
Description = "Number of forecast days",
|
|
Default = 3
|
|
}
|
|
},
|
|
Required = new[] { "location" }
|
|
};
|
|
}
|
|
|
|
public async Task<ToolResponse> ExecuteAsync(IDictionary<string, object> parameters)
|
|
{
|
|
var location = parameters["location"].ToString();
|
|
var days = parameters.ContainsKey("days")
|
|
? Convert.ToInt32(parameters["days"])
|
|
: 3;
|
|
|
|
var forecast = await _weatherService.GetForecastAsync(location, days);
|
|
|
|
return new ToolResponse
|
|
{
|
|
Content = new List<ContentItem>
|
|
{
|
|
new TextContent(JsonSerializer.Serialize(forecast))
|
|
}
|
|
};
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 2. Consistent Error Handling
|
|
|
|
Implement robust error handling with informative error messages and appropriate recovery mechanisms.
|
|
|
|
```python
|
|
# Python example with comprehensive error handling
|
|
class DataQueryTool:
|
|
def get_name(self):
|
|
return "dataQuery"
|
|
|
|
def get_description(self):
|
|
return "Queries data from specified database tables"
|
|
|
|
async def execute(self, parameters):
|
|
try:
|
|
# Parameter validation
|
|
if "query" not in parameters:
|
|
raise ToolParameterError("Missing required parameter: query")
|
|
|
|
query = parameters["query"]
|
|
|
|
# Security validation
|
|
if self._contains_unsafe_sql(query):
|
|
raise ToolSecurityError("Query contains potentially unsafe SQL")
|
|
|
|
try:
|
|
# Database operation with timeout
|
|
async with timeout(10): # 10 second timeout
|
|
result = await self._database.execute_query(query)
|
|
|
|
return ToolResponse(
|
|
content=[TextContent(json.dumps(result))]
|
|
)
|
|
except asyncio.TimeoutError:
|
|
raise ToolExecutionError("Database query timed out after 10 seconds")
|
|
except DatabaseConnectionError as e:
|
|
# Connection errors might be transient
|
|
self._log_error("Database connection error", e)
|
|
raise ToolExecutionError(f"Database connection error: {str(e)}")
|
|
except DatabaseQueryError as e:
|
|
# Query errors are likely client errors
|
|
self._log_error("Database query error", e)
|
|
raise ToolExecutionError(f"Invalid query: {str(e)}")
|
|
|
|
except ToolError:
|
|
# Let tool-specific errors pass through
|
|
raise
|
|
except Exception as e:
|
|
# Catch-all for unexpected errors
|
|
self._log_error("Unexpected error in DataQueryTool", e)
|
|
raise ToolExecutionError(f"An unexpected error occurred: {str(e)}")
|
|
|
|
def _contains_unsafe_sql(self, query):
|
|
# Implementation of SQL injection detection
|
|
pass
|
|
|
|
def _log_error(self, message, error):
|
|
# Implementation of error logging
|
|
pass
|
|
```
|
|
|
|
#### 3. Parameter Validation
|
|
|
|
Always validate parameters thoroughly to prevent malformed or malicious input.
|
|
|
|
```javascript
|
|
// JavaScript/TypeScript example with detailed parameter validation
|
|
class FileOperationTool {
|
|
getName() {
|
|
return "fileOperation";
|
|
}
|
|
|
|
getDescription() {
|
|
return "Performs file operations like read, write, and delete";
|
|
}
|
|
|
|
getDefinition() {
|
|
return {
|
|
name: this.getName(),
|
|
description: this.getDescription(),
|
|
parameters: {
|
|
operation: {
|
|
type: "string",
|
|
description: "Operation to perform",
|
|
enum: ["read", "write", "delete"]
|
|
},
|
|
path: {
|
|
type: "string",
|
|
description: "File path (must be within allowed directories)"
|
|
},
|
|
content: {
|
|
type: "string",
|
|
description: "Content to write (only for write operation)",
|
|
optional: true
|
|
}
|
|
},
|
|
required: ["operation", "path"]
|
|
};
|
|
}
|
|
|
|
async execute(parameters) {
|
|
// 1. Validate parameter presence
|
|
if (!parameters.operation) {
|
|
throw new ToolError("Missing required parameter: operation");
|
|
}
|
|
|
|
if (!parameters.path) {
|
|
throw new ToolError("Missing required parameter: path");
|
|
}
|
|
|
|
// 2. Validate parameter types
|
|
if (typeof parameters.operation !== "string") {
|
|
throw new ToolError("Parameter 'operation' must be a string");
|
|
}
|
|
|
|
if (typeof parameters.path !== "string") {
|
|
throw new ToolError("Parameter 'path' must be a string");
|
|
}
|
|
|
|
// 3. Validate parameter values
|
|
const validOperations = ["read", "write", "delete"];
|
|
if (!validOperations.includes(parameters.operation)) {
|
|
throw new ToolError(`Invalid operation. Must be one of: ${validOperations.join(", ")}`);
|
|
}
|
|
|
|
// 4. Validate content presence for write operation
|
|
if (parameters.operation === "write" && !parameters.content) {
|
|
throw new ToolError("Content parameter is required for write operation");
|
|
}
|
|
|
|
// 5. Path safety validation
|
|
if (!this.isPathWithinAllowedDirectories(parameters.path)) {
|
|
throw new ToolError("Access denied: path is outside of allowed directories");
|
|
}
|
|
|
|
// Implementation based on validated parameters
|
|
// ...
|
|
}
|
|
|
|
isPathWithinAllowedDirectories(path) {
|
|
// Implementation of path safety check
|
|
// ...
|
|
}
|
|
}
|
|
```
|
|
|
|
### Security Implementation Examples
|
|
|
|
#### 1. Authentication and Authorization
|
|
|
|
```java
|
|
// Java example with authentication and authorization
|
|
public class SecureDataAccessTool implements Tool {
|
|
private final AuthenticationService authService;
|
|
private final AuthorizationService authzService;
|
|
private final DataService dataService;
|
|
|
|
// Dependency injection
|
|
public SecureDataAccessTool(
|
|
AuthenticationService authService,
|
|
AuthorizationService authzService,
|
|
DataService dataService) {
|
|
this.authService = authService;
|
|
this.authzService = authzService;
|
|
this.dataService = dataService;
|
|
}
|
|
|
|
@Override
|
|
public String getName() {
|
|
return "secureDataAccess";
|
|
}
|
|
|
|
@Override
|
|
public ToolResponse execute(ToolRequest request) {
|
|
// 1. Extract authentication context
|
|
String authToken = request.getContext().getAuthToken();
|
|
|
|
// 2. Authenticate user
|
|
UserIdentity user;
|
|
try {
|
|
user = authService.validateToken(authToken);
|
|
} catch (AuthenticationException e) {
|
|
return ToolResponse.error("Authentication failed: " + e.getMessage());
|
|
}
|
|
|
|
// 3. Check authorization for the specific operation
|
|
String dataId = request.getParameters().get("dataId").getAsString();
|
|
String operation = request.getParameters().get("operation").getAsString();
|
|
|
|
boolean isAuthorized = authzService.isAuthorized(user, "data:" + dataId, operation);
|
|
if (!isAuthorized) {
|
|
return ToolResponse.error("Access denied: Insufficient permissions for this operation");
|
|
}
|
|
|
|
// 4. Proceed with authorized operation
|
|
try {
|
|
switch (operation) {
|
|
case "read":
|
|
Object data = dataService.getData(dataId, user.getId());
|
|
return ToolResponse.success(data);
|
|
case "update":
|
|
JsonNode newData = request.getParameters().get("newData");
|
|
dataService.updateData(dataId, newData, user.getId());
|
|
return ToolResponse.success("Data updated successfully");
|
|
default:
|
|
return ToolResponse.error("Unsupported operation: " + operation);
|
|
}
|
|
} catch (Exception e) {
|
|
return ToolResponse.error("Operation failed: " + e.getMessage());
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 2. Rate Limiting
|
|
|
|
```csharp
|
|
// C# rate limiting implementation
|
|
public class RateLimitingMiddleware
|
|
{
|
|
private readonly RequestDelegate _next;
|
|
private readonly IMemoryCache _cache;
|
|
private readonly ILogger<RateLimitingMiddleware> _logger;
|
|
|
|
// Configuration options
|
|
private readonly int _maxRequestsPerMinute;
|
|
|
|
public RateLimitingMiddleware(
|
|
RequestDelegate next,
|
|
IMemoryCache cache,
|
|
ILogger<RateLimitingMiddleware> logger,
|
|
IConfiguration config)
|
|
{
|
|
_next = next;
|
|
_cache = cache;
|
|
_logger = logger;
|
|
_maxRequestsPerMinute = config.GetValue<int>("RateLimit:MaxRequestsPerMinute", 60);
|
|
}
|
|
|
|
public async Task InvokeAsync(HttpContext context)
|
|
{
|
|
// 1. Get client identifier (API key or user ID)
|
|
string clientId = GetClientIdentifier(context);
|
|
|
|
// 2. Get rate limiting key for this minute
|
|
string cacheKey = $"rate_limit:{clientId}:{DateTime.UtcNow:yyyyMMddHHmm}";
|
|
|
|
// 3. Check current request count
|
|
if (!_cache.TryGetValue(cacheKey, out int requestCount))
|
|
{
|
|
requestCount = 0;
|
|
}
|
|
|
|
// 4. Enforce rate limit
|
|
if (requestCount >= _maxRequestsPerMinute)
|
|
{
|
|
_logger.LogWarning("Rate limit exceeded for client {ClientId}", clientId);
|
|
|
|
context.Response.StatusCode = StatusCodes.Status429TooManyRequests;
|
|
context.Response.Headers.Add("Retry-After", "60");
|
|
|
|
await context.Response.WriteAsJsonAsync(new
|
|
{
|
|
error = "Rate limit exceeded",
|
|
message = "Too many requests. Please try again later.",
|
|
retryAfterSeconds = 60
|
|
});
|
|
|
|
return;
|
|
}
|
|
|
|
// 5. Increment request count
|
|
_cache.Set(cacheKey, requestCount + 1, TimeSpan.FromMinutes(2));
|
|
|
|
// 6. Add rate limit headers
|
|
context.Response.Headers.Add("X-RateLimit-Limit", _maxRequestsPerMinute.ToString());
|
|
context.Response.Headers.Add("X-RateLimit-Remaining", (_maxRequestsPerMinute - requestCount - 1).ToString());
|
|
|
|
// 7. Continue with the request
|
|
await _next(context);
|
|
}
|
|
|
|
private string GetClientIdentifier(HttpContext context)
|
|
{
|
|
// Implementation to extract API key or user ID
|
|
// ...
|
|
}
|
|
}
|
|
```
|
|
|
|
## Testing Best Practices
|
|
|
|
### 1. Unit Testing MCP Tools
|
|
|
|
Always test your tools in isolation, mocking external dependencies:
|
|
|
|
```typescript
|
|
// TypeScript example of a tool unit test
|
|
describe('WeatherForecastTool', () => {
|
|
let tool: WeatherForecastTool;
|
|
let mockWeatherService: jest.Mocked<IWeatherService>;
|
|
|
|
beforeEach(() => {
|
|
// Create a mock weather service
|
|
mockWeatherService = {
|
|
getForecasts: jest.fn()
|
|
} as any;
|
|
|
|
// Create the tool with the mock dependency
|
|
tool = new WeatherForecastTool(mockWeatherService);
|
|
});
|
|
|
|
it('should return weather forecast for a location', async () => {
|
|
// Arrange
|
|
const mockForecast = {
|
|
location: 'Seattle',
|
|
forecasts: [
|
|
{ date: '2025-07-16', temperature: 72, conditions: 'Sunny' },
|
|
{ date: '2025-07-17', temperature: 68, conditions: 'Partly Cloudy' },
|
|
{ date: '2025-07-18', temperature: 65, conditions: 'Rain' }
|
|
]
|
|
};
|
|
|
|
mockWeatherService.getForecasts.mockResolvedValue(mockForecast);
|
|
|
|
// Act
|
|
const response = await tool.execute({
|
|
location: 'Seattle',
|
|
days: 3
|
|
});
|
|
|
|
// Assert
|
|
expect(mockWeatherService.getForecasts).toHaveBeenCalledWith('Seattle', 3);
|
|
expect(response.content[0].text).toContain('Seattle');
|
|
expect(response.content[0].text).toContain('Sunny');
|
|
});
|
|
|
|
it('should handle errors from the weather service', async () => {
|
|
// Arrange
|
|
mockWeatherService.getForecasts.mockRejectedValue(new Error('Service unavailable'));
|
|
|
|
// Act & Assert
|
|
await expect(tool.execute({
|
|
location: 'Seattle',
|
|
days: 3
|
|
})).rejects.toThrow('Weather service error: Service unavailable');
|
|
});
|
|
});
|
|
```
|
|
|
|
### 2. Integration Testing
|
|
|
|
Test the complete flow from client requests to server responses:
|
|
|
|
```python
|
|
# Python integration test example
|
|
@pytest.mark.asyncio
|
|
async def test_mcp_server_integration():
|
|
# Start a test server
|
|
server = McpServer()
|
|
server.register_tool(WeatherForecastTool(MockWeatherService()))
|
|
await server.start(port=5000)
|
|
|
|
try:
|
|
# Create a client
|
|
client = McpClient("http://localhost:5000")
|
|
|
|
# Test tool discovery
|
|
tools = await client.discover_tools()
|
|
assert "weatherForecast" in [t.name for t in tools]
|
|
|
|
# Test tool execution
|
|
response = await client.execute_tool("weatherForecast", {
|
|
"location": "Seattle",
|
|
"days": 3
|
|
})
|
|
|
|
# Verify response
|
|
assert response.status_code == 200
|
|
assert "Seattle" in response.content[0].text
|
|
assert len(json.loads(response.content[0].text)["forecasts"]) == 3
|
|
|
|
finally:
|
|
# Clean up
|
|
await server.stop()
|
|
```
|
|
|
|
## Performance Optimization
|
|
|
|
### 1. Caching Strategies
|
|
|
|
Implement appropriate caching to reduce latency and resource usage:
|
|
|
|
```csharp
|
|
// C# example with caching
|
|
public class CachedWeatherTool : ITool
|
|
{
|
|
private readonly IWeatherService _weatherService;
|
|
private readonly IDistributedCache _cache;
|
|
private readonly ILogger<CachedWeatherTool> _logger;
|
|
|
|
public CachedWeatherTool(
|
|
IWeatherService weatherService,
|
|
IDistributedCache cache,
|
|
ILogger<CachedWeatherTool> logger)
|
|
{
|
|
_weatherService = weatherService;
|
|
_cache = cache;
|
|
_logger = logger;
|
|
}
|
|
|
|
public string Name => "weatherForecast";
|
|
|
|
public async Task<ToolResponse> ExecuteAsync(IDictionary<string, object> parameters)
|
|
{
|
|
var location = parameters["location"].ToString();
|
|
var days = Convert.ToInt32(parameters.GetValueOrDefault("days", 3));
|
|
|
|
// Create cache key
|
|
string cacheKey = $"weather:{location}:{days}";
|
|
|
|
// Try to get from cache
|
|
string cachedForecast = await _cache.GetStringAsync(cacheKey);
|
|
if (!string.IsNullOrEmpty(cachedForecast))
|
|
{
|
|
_logger.LogInformation("Cache hit for weather forecast: {Location}", location);
|
|
return new ToolResponse
|
|
{
|
|
Content = new List<ContentItem>
|
|
{
|
|
new TextContent(cachedForecast)
|
|
}
|
|
};
|
|
}
|
|
|
|
// Cache miss - get from service
|
|
_logger.LogInformation("Cache miss for weather forecast: {Location}", location);
|
|
var forecast = await _weatherService.GetForecastAsync(location, days);
|
|
string forecastJson = JsonSerializer.Serialize(forecast);
|
|
|
|
// Store in cache (weather forecasts valid for 1 hour)
|
|
await _cache.SetStringAsync(
|
|
cacheKey,
|
|
forecastJson,
|
|
new DistributedCacheEntryOptions
|
|
{
|
|
AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1)
|
|
});
|
|
|
|
return new ToolResponse
|
|
{
|
|
Content = new List<ContentItem>
|
|
{
|
|
new TextContent(forecastJson)
|
|
}
|
|
};
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 2. Dependency Injection and Testability
|
|
|
|
Design tools to receive their dependencies through constructor injection, making them testable and configurable:
|
|
|
|
```java
|
|
// Java example with dependency injection
|
|
public class CurrencyConversionTool implements Tool {
|
|
private final ExchangeRateService exchangeService;
|
|
private final CacheService cacheService;
|
|
private final Logger logger;
|
|
|
|
// Dependencies injected through constructor
|
|
public CurrencyConversionTool(
|
|
ExchangeRateService exchangeService,
|
|
CacheService cacheService,
|
|
Logger logger) {
|
|
this.exchangeService = exchangeService;
|
|
this.cacheService = cacheService;
|
|
this.logger = logger;
|
|
}
|
|
|
|
// Tool implementation
|
|
// ...
|
|
}
|
|
```
|
|
|
|
#### 3. Composable Tools
|
|
|
|
Design tools that can be composed together to create more complex workflows:
|
|
|
|
```python
|
|
# Python example showing composable tools
|
|
class DataFetchTool(Tool):
|
|
def get_name(self):
|
|
return "dataFetch"
|
|
|
|
# Implementation...
|
|
|
|
class DataAnalysisTool(Tool):
|
|
def get_name(self):
|
|
return "dataAnalysis"
|
|
|
|
# This tool can use results from the dataFetch tool
|
|
async def execute_async(self, request):
|
|
# Implementation...
|
|
pass
|
|
|
|
class DataVisualizationTool(Tool):
|
|
def get_name(self):
|
|
return "dataVisualize"
|
|
|
|
# This tool can use results from the dataAnalysis tool
|
|
async def execute_async(self, request):
|
|
# Implementation...
|
|
pass
|
|
|
|
# These tools can be used independently or as part of a workflow
|
|
```
|
|
|
|
### Schema Design Best Practices
|
|
|
|
The schema is the contract between the model and your tool. Well-designed schemas lead to better tool usability.
|
|
|
|
#### 1. Clear Parameter Descriptions
|
|
|
|
Always include descriptive information for each parameter:
|
|
|
|
```csharp
|
|
public object GetSchema()
|
|
{
|
|
return new {
|
|
type = "object",
|
|
properties = new {
|
|
query = new {
|
|
type = "string",
|
|
description = "Search query text. Use precise keywords for better results."
|
|
},
|
|
filters = new {
|
|
type = "object",
|
|
description = "Optional filters to narrow down search results",
|
|
properties = new {
|
|
dateRange = new {
|
|
type = "string",
|
|
description = "Date range in format YYYY-MM-DD:YYYY-MM-DD"
|
|
},
|
|
category = new {
|
|
type = "string",
|
|
description = "Category name to filter by"
|
|
}
|
|
}
|
|
},
|
|
limit = new {
|
|
type = "integer",
|
|
description = "Maximum number of results to return (1-50)",
|
|
default = 10
|
|
}
|
|
},
|
|
required = new[] { "query" }
|
|
};
|
|
}
|
|
```
|
|
|
|
#### 2. Validation Constraints
|
|
|
|
Include validation constraints to prevent invalid inputs:
|
|
|
|
```java
|
|
Map<String, Object> getSchema() {
|
|
Map<String, Object> schema = new HashMap<>();
|
|
schema.put("type", "object");
|
|
|
|
Map<String, Object> properties = new HashMap<>();
|
|
|
|
// Email property with format validation
|
|
Map<String, Object> email = new HashMap<>();
|
|
email.put("type", "string");
|
|
email.put("format", "email");
|
|
email.put("description", "User email address");
|
|
|
|
// Age property with numeric constraints
|
|
Map<String, Object> age = new HashMap<>();
|
|
age.put("type", "integer");
|
|
age.put("minimum", 13);
|
|
age.put("maximum", 120);
|
|
age.put("description", "User age in years");
|
|
|
|
// Enumerated property
|
|
Map<String, Object> subscription = new HashMap<>();
|
|
subscription.put("type", "string");
|
|
subscription.put("enum", Arrays.asList("free", "basic", "premium"));
|
|
subscription.put("default", "free");
|
|
subscription.put("description", "Subscription tier");
|
|
|
|
properties.put("email", email);
|
|
properties.put("age", age);
|
|
properties.put("subscription", subscription);
|
|
|
|
schema.put("properties", properties);
|
|
schema.put("required", Arrays.asList("email"));
|
|
|
|
return schema;
|
|
}
|
|
```
|
|
|
|
#### 3. Consistent Return Structures
|
|
|
|
Maintain consistency in your response structures to make it easier for models to interpret results:
|
|
|
|
```python
|
|
async def execute_async(self, request):
|
|
try:
|
|
# Process request
|
|
results = await self._search_database(request.parameters["query"])
|
|
|
|
# Always return a consistent structure
|
|
return ToolResponse(
|
|
result={
|
|
"matches": [self._format_item(item) for item in results],
|
|
"totalCount": len(results),
|
|
"queryTime": calculation_time_ms,
|
|
"status": "success"
|
|
}
|
|
)
|
|
except Exception as e:
|
|
return ToolResponse(
|
|
result={
|
|
"matches": [],
|
|
"totalCount": 0,
|
|
"queryTime": 0,
|
|
"status": "error",
|
|
"error": str(e)
|
|
}
|
|
)
|
|
|
|
def _format_item(self, item):
|
|
"""Ensures each item has a consistent structure"""
|
|
return {
|
|
"id": item.id,
|
|
"title": item.title,
|
|
"summary": item.summary[:100] + "..." if len(item.summary) > 100 else item.summary,
|
|
"url": item.url,
|
|
"relevance": item.score
|
|
}
|
|
```
|
|
|
|
### Error Handling
|
|
|
|
Robust error handling is crucial for MCP tools to maintain reliability.
|
|
|
|
#### 1. Graceful Error Handling
|
|
|
|
Handle errors at appropriate levels and provide informative messages:
|
|
|
|
```csharp
|
|
public async Task<ToolResponse> ExecuteAsync(ToolRequest request)
|
|
{
|
|
try
|
|
{
|
|
string fileId = request.Parameters.GetProperty("fileId").GetString();
|
|
|
|
try
|
|
{
|
|
var fileData = await _fileService.GetFileAsync(fileId);
|
|
return new ToolResponse {
|
|
Result = JsonSerializer.SerializeToElement(fileData)
|
|
};
|
|
}
|
|
catch (FileNotFoundException)
|
|
{
|
|
throw new ToolExecutionException($"File not found: {fileId}");
|
|
}
|
|
catch (UnauthorizedAccessException)
|
|
{
|
|
throw new ToolExecutionException("You don't have permission to access this file");
|
|
}
|
|
catch (Exception ex) when (ex is IOException || ex is TimeoutException)
|
|
{
|
|
_logger.LogError(ex, "Error accessing file {FileId}", fileId);
|
|
throw new ToolExecutionException("Error accessing file: The service is temporarily unavailable");
|
|
}
|
|
}
|
|
catch (JsonException)
|
|
{
|
|
throw new ToolExecutionException("Invalid file ID format");
|
|
}
|
|
catch (Exception ex)
|
|
{
|
|
_logger.LogError(ex, "Unexpected error in FileAccessTool");
|
|
throw new ToolExecutionException("An unexpected error occurred");
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 2. Structured Error Responses
|
|
|
|
Return structured error information when possible:
|
|
|
|
```java
|
|
@Override
|
|
public ToolResponse execute(ToolRequest request) {
|
|
try {
|
|
// Implementation
|
|
} catch (Exception ex) {
|
|
Map<String, Object> errorResult = new HashMap<>();
|
|
|
|
errorResult.put("success", false);
|
|
|
|
if (ex instanceof ValidationException) {
|
|
ValidationException validationEx = (ValidationException) ex;
|
|
|
|
errorResult.put("errorType", "validation");
|
|
errorResult.put("errorMessage", validationEx.getMessage());
|
|
errorResult.put("validationErrors", validationEx.getErrors());
|
|
|
|
return new ToolResponse.Builder()
|
|
.setResult(errorResult)
|
|
.build();
|
|
}
|
|
|
|
// Re-throw other exceptions as ToolExecutionException
|
|
throw new ToolExecutionException("Tool execution failed: " + ex.getMessage(), ex);
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 3. Retry Logic
|
|
|
|
Implement appropriate retry logic for transient failures:
|
|
|
|
```python
|
|
async def execute_async(self, request):
|
|
max_retries = 3
|
|
retry_count = 0
|
|
base_delay = 1 # seconds
|
|
|
|
while retry_count < max_retries:
|
|
try:
|
|
# Call external API
|
|
return await self._call_api(request.parameters)
|
|
except TransientError as e:
|
|
retry_count += 1
|
|
if retry_count >= max_retries:
|
|
raise ToolExecutionException(f"Operation failed after {max_retries} attempts: {str(e)}")
|
|
|
|
# Exponential backoff
|
|
delay = base_delay * (2 ** (retry_count - 1))
|
|
logging.warning(f"Transient error, retrying in {delay}s: {str(e)}")
|
|
await asyncio.sleep(delay)
|
|
except Exception as e:
|
|
# Non-transient error, don't retry
|
|
raise ToolExecutionException(f"Operation failed: {str(e)}")
|
|
```
|
|
|
|
### Performance Optimization
|
|
|
|
#### 1. Caching
|
|
|
|
Implement caching for expensive operations:
|
|
|
|
```csharp
|
|
public class CachedDataTool : IMcpTool
|
|
{
|
|
private readonly IDatabase _database;
|
|
private readonly IMemoryCache _cache;
|
|
|
|
public CachedDataTool(IDatabase database, IMemoryCache cache)
|
|
{
|
|
_database = database;
|
|
_cache = cache;
|
|
}
|
|
|
|
public async Task<ToolResponse> ExecuteAsync(ToolRequest request)
|
|
{
|
|
var query = request.Parameters.GetProperty("query").GetString();
|
|
|
|
// Create cache key based on parameters
|
|
var cacheKey = $"data_query_{ComputeHash(query)}";
|
|
|
|
// Try to get from cache first
|
|
if (_cache.TryGetValue(cacheKey, out var cachedResult))
|
|
{
|
|
return new ToolResponse { Result = cachedResult };
|
|
}
|
|
|
|
// Cache miss - perform actual query
|
|
var result = await _database.QueryAsync(query);
|
|
|
|
// Store in cache with expiration
|
|
var cacheOptions = new MemoryCacheEntryOptions()
|
|
.SetAbsoluteExpiration(TimeSpan.FromMinutes(15));
|
|
|
|
_cache.Set(cacheKey, JsonSerializer.SerializeToElement(result), cacheOptions);
|
|
|
|
return new ToolResponse { Result = JsonSerializer.SerializeToElement(result) };
|
|
}
|
|
|
|
private string ComputeHash(string input)
|
|
{
|
|
// Implementation to generate stable hash for cache key
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 2. Asynchronous Processing
|
|
|
|
Use asynchronous programming patterns for I/O-bound operations:
|
|
|
|
```java
|
|
public class AsyncDocumentProcessingTool implements Tool {
|
|
private final DocumentService documentService;
|
|
private final ExecutorService executorService;
|
|
|
|
@Override
|
|
public ToolResponse execute(ToolRequest request) {
|
|
String documentId = request.getParameters().get("documentId").asText();
|
|
|
|
// For long-running operations, return a processing ID immediately
|
|
String processId = UUID.randomUUID().toString();
|
|
|
|
// Start async processing
|
|
CompletableFuture.runAsync(() -> {
|
|
try {
|
|
// Perform long-running operation
|
|
documentService.processDocument(documentId);
|
|
|
|
// Update status (would typically be stored in a database)
|
|
processStatusRepository.updateStatus(processId, "completed");
|
|
} catch (Exception ex) {
|
|
processStatusRepository.updateStatus(processId, "failed", ex.getMessage());
|
|
}
|
|
}, executorService);
|
|
|
|
// Return immediate response with process ID
|
|
Map<String, Object> result = new HashMap<>();
|
|
result.put("processId", processId);
|
|
result.put("status", "processing");
|
|
result.put("estimatedCompletionTime", ZonedDateTime.now().plusMinutes(5));
|
|
|
|
return new ToolResponse.Builder().setResult(result).build();
|
|
}
|
|
|
|
// Companion status check tool
|
|
public class ProcessStatusTool implements Tool {
|
|
@Override
|
|
public ToolResponse execute(ToolRequest request) {
|
|
String processId = request.getParameters().get("processId").asText();
|
|
ProcessStatus status = processStatusRepository.getStatus(processId);
|
|
|
|
return new ToolResponse.Builder().setResult(status).build();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 3. Resource Throttling
|
|
|
|
Implement resource throttling to prevent overload:
|
|
|
|
```python
|
|
class ThrottledApiTool(Tool):
|
|
def __init__(self):
|
|
self.rate_limiter = TokenBucketRateLimiter(
|
|
tokens_per_second=5, # Allow 5 requests per second
|
|
bucket_size=10 # Allow bursts up to 10 requests
|
|
)
|
|
|
|
async def execute_async(self, request):
|
|
# Check if we can proceed or need to wait
|
|
delay = self.rate_limiter.get_delay_time()
|
|
|
|
if delay > 0:
|
|
if delay > 2.0: # If wait is too long
|
|
raise ToolExecutionException(
|
|
f"Rate limit exceeded. Please try again in {delay:.1f} seconds."
|
|
)
|
|
else:
|
|
# Wait for the appropriate delay time
|
|
await asyncio.sleep(delay)
|
|
|
|
# Consume a token and proceed with the request
|
|
self.rate_limiter.consume()
|
|
|
|
# Call API
|
|
result = await self._call_api(request.parameters)
|
|
return ToolResponse(result=result)
|
|
|
|
class TokenBucketRateLimiter:
|
|
def __init__(self, tokens_per_second, bucket_size):
|
|
self.tokens_per_second = tokens_per_second
|
|
self.bucket_size = bucket_size
|
|
self.tokens = bucket_size
|
|
self.last_refill = time.time()
|
|
self.lock = asyncio.Lock()
|
|
|
|
async def get_delay_time(self):
|
|
async with self.lock:
|
|
self._refill()
|
|
if self.tokens >= 1:
|
|
return 0
|
|
|
|
# Calculate time until next token available
|
|
return (1 - self.tokens) / self.tokens_per_second
|
|
|
|
async def consume(self):
|
|
async with self.lock:
|
|
self._refill()
|
|
self.tokens -= 1
|
|
|
|
def _refill(self):
|
|
now = time.time()
|
|
elapsed = now - self.last_refill
|
|
|
|
# Add new tokens based on elapsed time
|
|
new_tokens = elapsed * self.tokens_per_second
|
|
self.tokens = min(self.bucket_size, self.tokens + new_tokens)
|
|
self.last_refill = now
|
|
```
|
|
|
|
### Security Best Practices
|
|
|
|
#### 1. Input Validation
|
|
|
|
Always validate input parameters thoroughly:
|
|
|
|
```csharp
|
|
public async Task<ToolResponse> ExecuteAsync(ToolRequest request)
|
|
{
|
|
// Validate parameters exist
|
|
if (!request.Parameters.TryGetProperty("query", out var queryProp))
|
|
{
|
|
throw new ToolExecutionException("Missing required parameter: query");
|
|
}
|
|
|
|
// Validate correct type
|
|
if (queryProp.ValueKind != JsonValueKind.String)
|
|
{
|
|
throw new ToolExecutionException("Query parameter must be a string");
|
|
}
|
|
|
|
var query = queryProp.GetString();
|
|
|
|
// Validate string content
|
|
if (string.IsNullOrWhiteSpace(query))
|
|
{
|
|
throw new ToolExecutionException("Query parameter cannot be empty");
|
|
}
|
|
|
|
if (query.Length > 500)
|
|
{
|
|
throw new ToolExecutionException("Query parameter exceeds maximum length of 500 characters");
|
|
}
|
|
|
|
// Check for SQL injection attacks if applicable
|
|
if (ContainsSqlInjection(query))
|
|
{
|
|
throw new ToolExecutionException("Invalid query: contains potentially unsafe SQL");
|
|
}
|
|
|
|
// Proceed with execution
|
|
// ...
|
|
}
|
|
```
|
|
|
|
#### 2. Authorization Checks
|
|
|
|
Implement proper authorization checks:
|
|
|
|
```java
|
|
@Override
|
|
public ToolResponse execute(ToolRequest request) {
|
|
// Get user context from request
|
|
UserContext user = request.getContext().getUserContext();
|
|
|
|
// Check if user has required permissions
|
|
if (!authorizationService.hasPermission(user, "documents:read")) {
|
|
throw new ToolExecutionException("User does not have permission to access documents");
|
|
}
|
|
|
|
// For specific resources, check access to that resource
|
|
String documentId = request.getParameters().get("documentId").asText();
|
|
if (!documentService.canUserAccess(user.getId(), documentId)) {
|
|
throw new ToolExecutionException("Access denied to the requested document");
|
|
}
|
|
|
|
// Proceed with tool execution
|
|
// ...
|
|
}
|
|
```
|
|
|
|
#### 3. Sensitive Data Handling
|
|
|
|
Handle sensitive data carefully:
|
|
|
|
```python
|
|
class SecureDataTool(Tool):
|
|
def get_schema(self):
|
|
return {
|
|
"type": "object",
|
|
"properties": {
|
|
"userId": {"type": "string"},
|
|
"includeSensitiveData": {"type": "boolean", "default": False}
|
|
},
|
|
"required": ["userId"]
|
|
}
|
|
|
|
async def execute_async(self, request):
|
|
user_id = request.parameters["userId"]
|
|
include_sensitive = request.parameters.get("includeSensitiveData", False)
|
|
|
|
# Get user data
|
|
user_data = await self.user_service.get_user_data(user_id)
|
|
|
|
# Filter sensitive fields unless explicitly requested AND authorized
|
|
if not include_sensitive or not self._is_authorized_for_sensitive_data(request):
|
|
user_data = self._redact_sensitive_fields(user_data)
|
|
|
|
return ToolResponse(result=user_data)
|
|
|
|
def _is_authorized_for_sensitive_data(self, request):
|
|
# Check authorization level in request context
|
|
auth_level = request.context.get("authorizationLevel")
|
|
return auth_level == "admin"
|
|
|
|
def _redact_sensitive_fields(self, user_data):
|
|
# Create a copy to avoid modifying the original
|
|
redacted = user_data.copy()
|
|
|
|
# Redact specific sensitive fields
|
|
sensitive_fields = ["ssn", "creditCardNumber", "password"]
|
|
for field in sensitive_fields:
|
|
if field in redacted:
|
|
redacted[field] = "REDACTED"
|
|
|
|
# Redact nested sensitive data
|
|
if "financialInfo" in redacted:
|
|
redacted["financialInfo"] = {"available": True, "accessRestricted": True}
|
|
|
|
return redacted
|
|
```
|
|
|
|
## Testing Best Practices for MCP Tools
|
|
|
|
Comprehensive testing ensures that MCP tools function correctly, handle edge cases, and integrate properly with the rest of the system.
|
|
|
|
### Unit Testing
|
|
|
|
#### 1. Test Each Tool in Isolation
|
|
|
|
Create focused tests for each tool's functionality:
|
|
|
|
```csharp
|
|
[Fact]
|
|
public async Task WeatherTool_ValidLocation_ReturnsCorrectForecast()
|
|
{
|
|
// Arrange
|
|
var mockWeatherService = new Mock<IWeatherService>();
|
|
mockWeatherService
|
|
.Setup(s => s.GetForecastAsync("Seattle", 3))
|
|
.ReturnsAsync(new WeatherForecast(/* test data */));
|
|
|
|
var tool = new WeatherForecastTool(mockWeatherService.Object);
|
|
|
|
var request = new ToolRequest(
|
|
toolName: "weatherForecast",
|
|
parameters: JsonSerializer.SerializeToElement(new {
|
|
location = "Seattle",
|
|
days = 3
|
|
})
|
|
);
|
|
|
|
// Act
|
|
var response = await tool.ExecuteAsync(request);
|
|
|
|
// Assert
|
|
Assert.NotNull(response);
|
|
var result = JsonSerializer.Deserialize<WeatherForecast>(response.Result);
|
|
Assert.Equal("Seattle", result.Location);
|
|
Assert.Equal(3, result.DailyForecasts.Count);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task WeatherTool_InvalidLocation_ThrowsToolExecutionException()
|
|
{
|
|
// Arrange
|
|
var mockWeatherService = new Mock<IWeatherService>();
|
|
mockWeatherService
|
|
.Setup(s => s.GetForecastAsync("InvalidLocation", It.IsAny<int>()))
|
|
.ThrowsAsync(new LocationNotFoundException("Location not found"));
|
|
|
|
var tool = new WeatherForecastTool(mockWeatherService.Object);
|
|
|
|
var request = new ToolRequest(
|
|
toolName: "weatherForecast",
|
|
parameters: JsonSerializer.SerializeToElement(new {
|
|
location = "InvalidLocation",
|
|
days = 3
|
|
})
|
|
);
|
|
|
|
// Act & Assert
|
|
var exception = await Assert.ThrowsAsync<ToolExecutionException>(
|
|
() => tool.ExecuteAsync(request)
|
|
);
|
|
|
|
Assert.Contains("Location not found", exception.Message);
|
|
}
|
|
```
|
|
|
|
#### 2. Schema Validation Testing
|
|
|
|
Test that schemas are valid and properly enforce constraints:
|
|
|
|
```java
|
|
@Test
|
|
public void testSchemaValidation() {
|
|
// Create tool instance
|
|
SearchTool searchTool = new SearchTool();
|
|
|
|
// Get schema
|
|
Object schema = searchTool.getSchema();
|
|
|
|
// Convert schema to JSON for validation
|
|
String schemaJson = objectMapper.writeValueAsString(schema);
|
|
|
|
// Validate schema is valid JSONSchema
|
|
JsonSchemaFactory factory = JsonSchemaFactory.byDefault();
|
|
JsonSchema jsonSchema = factory.getJsonSchema(schemaJson);
|
|
|
|
// Test valid parameters
|
|
JsonNode validParams = objectMapper.createObjectNode()
|
|
.put("query", "test query")
|
|
.put("limit", 5);
|
|
|
|
ProcessingReport validReport = jsonSchema.validate(validParams);
|
|
assertTrue(validReport.isSuccess());
|
|
|
|
// Test missing required parameter
|
|
JsonNode missingRequired = objectMapper.createObjectNode()
|
|
.put("limit", 5);
|
|
|
|
ProcessingReport missingReport = jsonSchema.validate(missingRequired);
|
|
assertFalse(missingReport.isSuccess());
|
|
|
|
// Test invalid parameter type
|
|
JsonNode invalidType = objectMapper.createObjectNode()
|
|
.put("query", "test")
|
|
.put("limit", "not-a-number");
|
|
|
|
ProcessingReport invalidReport = jsonSchema.validate(invalidType);
|
|
assertFalse(invalidReport.isSuccess());
|
|
}
|
|
```
|
|
|
|
#### 3. Error Handling Tests
|
|
|
|
Create specific tests for error conditions:
|
|
|
|
```python
|
|
@pytest.mark.asyncio
|
|
async def test_api_tool_handles_timeout():
|
|
# Arrange
|
|
tool = ApiTool(timeout=0.1) # Very short timeout
|
|
|
|
# Mock a request that will time out
|
|
with aioresponses() as mocked:
|
|
mocked.get(
|
|
"https://api.example.com/data",
|
|
callback=lambda *args, **kwargs: asyncio.sleep(0.5) # Longer than timeout
|
|
)
|
|
|
|
request = ToolRequest(
|
|
tool_name="apiTool",
|
|
parameters={"url": "https://api.example.com/data"}
|
|
)
|
|
|
|
# Act & Assert
|
|
with pytest.raises(ToolExecutionException) as exc_info:
|
|
await tool.execute_async(request)
|
|
|
|
# Verify exception message
|
|
assert "timed out" in str(exc_info.value).lower()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_api_tool_handles_rate_limiting():
|
|
# Arrange
|
|
tool = ApiTool()
|
|
|
|
# Mock a rate-limited response
|
|
with aioresponses() as mocked:
|
|
mocked.get(
|
|
"https://api.example.com/data",
|
|
status=429,
|
|
headers={"Retry-After": "2"},
|
|
body=json.dumps({"error": "Rate limit exceeded"})
|
|
)
|
|
|
|
request = ToolRequest(
|
|
tool_name="apiTool",
|
|
parameters={"url": "https://api.example.com/data"}
|
|
)
|
|
|
|
# Act & Assert
|
|
with pytest.raises(ToolExecutionException) as exc_info:
|
|
await tool.execute_async(request)
|
|
|
|
# Verify exception contains rate limit information
|
|
error_msg = str(exc_info.value).lower()
|
|
assert "rate limit" in error_msg
|
|
assert "try again" in error_msg
|
|
```
|
|
|
|
### Integration Testing
|
|
|
|
#### 1. Tool Chain Testing
|
|
|
|
Test tools working together in expected combinations:
|
|
|
|
```csharp
|
|
[Fact]
|
|
public async Task DataProcessingWorkflow_CompletesSuccessfully()
|
|
{
|
|
// Arrange
|
|
var dataFetchTool = new DataFetchTool(mockDataService.Object);
|
|
var analysisTools = new DataAnalysisTool(mockAnalysisService.Object);
|
|
var visualizationTool = new DataVisualizationTool(mockVisualizationService.Object);
|
|
|
|
var toolRegistry = new ToolRegistry();
|
|
toolRegistry.RegisterTool(dataFetchTool);
|
|
toolRegistry.RegisterTool(analysisTools);
|
|
toolRegistry.RegisterTool(visualizationTool);
|
|
|
|
var workflowExecutor = new WorkflowExecutor(toolRegistry);
|
|
|
|
// Act
|
|
var result = await workflowExecutor.ExecuteWorkflowAsync(new[] {
|
|
new ToolCall("dataFetch", new { source = "sales2023" }),
|
|
new ToolCall("dataAnalysis", ctx => new {
|
|
data = ctx.GetResult("dataFetch"),
|
|
analysis = "trend"
|
|
}),
|
|
new ToolCall("dataVisualize", ctx => new {
|
|
analysisResult = ctx.GetResult("dataAnalysis"),
|
|
type = "line-chart"
|
|
})
|
|
});
|
|
|
|
// Assert
|
|
Assert.NotNull(result);
|
|
Assert.True(result.Success);
|
|
Assert.NotNull(result.GetResult("dataVisualize"));
|
|
Assert.Contains("chartUrl", result.GetResult("dataVisualize").ToString());
|
|
}
|
|
```
|
|
|
|
#### 2. MCP Server Testing
|
|
|
|
Test the MCP server with full tool registration and execution:
|
|
|
|
```java
|
|
@SpringBootTest
|
|
@AutoConfigureMockMvc
|
|
public class McpServerIntegrationTest {
|
|
|
|
@Autowired
|
|
private MockMvc mockMvc;
|
|
|
|
@Autowired
|
|
private ObjectMapper objectMapper;
|
|
|
|
@Test
|
|
public void testToolDiscovery() throws Exception {
|
|
// Test the discovery endpoint
|
|
mockMvc.perform(get("/mcp/tools"))
|
|
.andExpect(status().isOk())
|
|
.andExpect(jsonPath("$.tools").isArray())
|
|
.andExpect(jsonPath("$.tools[*].name").value(hasItems(
|
|
"weatherForecast", "calculator", "documentSearch"
|
|
)));
|
|
}
|
|
|
|
@Test
|
|
public void testToolExecution() throws Exception {
|
|
// Create tool request
|
|
Map<String, Object> request = new HashMap<>();
|
|
request.put("toolName", "calculator");
|
|
|
|
Map<String, Object> parameters = new HashMap<>();
|
|
parameters.put("operation", "add");
|
|
parameters.put("a", 5);
|
|
parameters.put("b", 7);
|
|
request.put("parameters", parameters);
|
|
|
|
// Send request and verify response
|
|
mockMvc.perform(post("/mcp/execute")
|
|
.contentType(MediaType.APPLICATION_JSON)
|
|
.content(objectMapper.writeValueAsString(request)))
|
|
.andExpect(status().isOk())
|
|
.andExpect(jsonPath("$.result.value").value(12));
|
|
}
|
|
|
|
@Test
|
|
public void testToolValidation() throws Exception {
|
|
// Create invalid tool request
|
|
Map<String, Object> request = new HashMap<>();
|
|
request.put("toolName", "calculator");
|
|
|
|
Map<String, Object> parameters = new HashMap<>();
|
|
parameters.put("operation", "divide");
|
|
parameters.put("a", 10);
|
|
// Missing parameter "b"
|
|
request.put("parameters", parameters);
|
|
|
|
// Send request and verify error response
|
|
mockMvc.perform(post("/mcp/execute")
|
|
.contentType(MediaType.APPLICATION_JSON)
|
|
.content(objectMapper.writeValueAsString(request)))
|
|
.andExpect(status().isBadRequest())
|
|
.andExpect(jsonPath("$.error").exists());
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 3. End-to-End Testing
|
|
|
|
Test complete workflows from model prompt to tool execution:
|
|
|
|
```python
|
|
@pytest.mark.asyncio
|
|
async def test_model_interaction_with_tool():
|
|
# Arrange - Set up MCP client and mock model
|
|
mcp_client = McpClient(server_url="http://localhost:5000")
|
|
|
|
# Mock model responses
|
|
mock_model = MockLanguageModel([
|
|
MockResponse(
|
|
"What's the weather in Seattle?",
|
|
tool_calls=[{
|
|
"tool_name": "weatherForecast",
|
|
"parameters": {"location": "Seattle", "days": 3}
|
|
}]
|
|
),
|
|
MockResponse(
|
|
"Here's the weather forecast for Seattle:\n- Today: 65°F, Partly Cloudy\n- Tomorrow: 68°F, Sunny\n- Day after: 62°F, Rain",
|
|
tool_calls=[]
|
|
)
|
|
])
|
|
|
|
# Mock weather tool response
|
|
with aioresponses() as mocked:
|
|
mocked.post(
|
|
"http://localhost:5000/mcp/execute",
|
|
payload={
|
|
"result": {
|
|
"location": "Seattle",
|
|
"forecast": [
|
|
{"date": "2023-06-01", "temperature": 65, "conditions": "Partly Cloudy"},
|
|
{"date": "2023-06-02", "temperature": 68, "conditions": "Sunny"},
|
|
{"date": "2023-06-03", "temperature": 62, "conditions": "Rain"}
|
|
]
|
|
}
|
|
}
|
|
)
|
|
|
|
# Act
|
|
response = await mcp_client.send_prompt(
|
|
"What's the weather in Seattle?",
|
|
model=mock_model,
|
|
allowed_tools=["weatherForecast"]
|
|
)
|
|
|
|
# Assert
|
|
assert "Seattle" in response.generated_text
|
|
assert "65" in response.generated_text
|
|
assert "Sunny" in response.generated_text
|
|
assert "Rain" in response.generated_text
|
|
assert len(response.tool_calls) == 1
|
|
assert response.tool_calls[0].tool_name == "weatherForecast"
|
|
```
|
|
|
|
### Performance Testing
|
|
|
|
#### 1. Load Testing
|
|
|
|
Test how many concurrent requests your MCP server can handle:
|
|
|
|
```csharp
|
|
[Fact]
|
|
public async Task McpServer_HandlesHighConcurrency()
|
|
{
|
|
// Arrange
|
|
var server = new McpServer(
|
|
name: "TestServer",
|
|
version: "1.0",
|
|
maxConcurrentRequests: 100
|
|
);
|
|
|
|
server.RegisterTool(new FastExecutingTool());
|
|
await server.StartAsync();
|
|
|
|
var client = new McpClient("http://localhost:5000");
|
|
|
|
// Act
|
|
var tasks = new List<Task<McpResponse>>();
|
|
for (int i = 0; i < 1000; i++)
|
|
{
|
|
tasks.Add(client.ExecuteToolAsync("fastTool", new { iteration = i }));
|
|
}
|
|
|
|
var results = await Task.WhenAll(tasks);
|
|
|
|
// Assert
|
|
Assert.Equal(1000, results.Length);
|
|
Assert.All(results, r => Assert.NotNull(r));
|
|
}
|
|
```
|
|
|
|
#### 2. Stress Testing
|
|
|
|
Test the system under extreme load:
|
|
|
|
```java
|
|
@Test
|
|
public void testServerUnderStress() {
|
|
int maxUsers = 1000;
|
|
int rampUpTimeSeconds = 60;
|
|
int testDurationSeconds = 300;
|
|
|
|
// Set up JMeter for stress testing
|
|
StandardJMeterEngine jmeter = new StandardJMeterEngine();
|
|
|
|
// Configure JMeter test plan
|
|
HashTree testPlanTree = new HashTree();
|
|
|
|
// Create test plan, thread group, samplers, etc.
|
|
TestPlan testPlan = new TestPlan("MCP Server Stress Test");
|
|
testPlanTree.add(testPlan);
|
|
|
|
ThreadGroup threadGroup = new ThreadGroup();
|
|
threadGroup.setNumThreads(maxUsers);
|
|
threadGroup.setRampUp(rampUpTimeSeconds);
|
|
threadGroup.setScheduler(true);
|
|
threadGroup.setDuration(testDurationSeconds);
|
|
|
|
testPlanTree.add(threadGroup);
|
|
|
|
// Add HTTP sampler for tool execution
|
|
HTTPSampler toolExecutionSampler = new HTTPSampler();
|
|
toolExecutionSampler.setDomain("localhost");
|
|
toolExecutionSampler.setPort(5000);
|
|
toolExecutionSampler.setPath("/mcp/execute");
|
|
toolExecutionSampler.setMethod("POST");
|
|
toolExecutionSampler.addArgument("toolName", "calculator");
|
|
toolExecutionSampler.addArgument("parameters", "{\"operation\":\"add\",\"a\":5,\"b\":7}");
|
|
|
|
threadGroup.add(toolExecutionSampler);
|
|
|
|
// Add listeners
|
|
SummaryReport summaryReport = new SummaryReport();
|
|
threadGroup.add(summaryReport);
|
|
|
|
// Run test
|
|
jmeter.configure(testPlanTree);
|
|
jmeter.run();
|
|
|
|
// Validate results
|
|
assertEquals(0, summaryReport.getErrorCount());
|
|
assertTrue(summaryReport.getAverage() < 200); // Average response time < 200ms
|
|
assertTrue(summaryReport.getPercentile(90.0) < 500); // 90th percentile < 500ms
|
|
}
|
|
```
|
|
|
|
#### 3. Monitoring and Profiling
|
|
|
|
Set up monitoring for long-term performance analysis:
|
|
|
|
```python
|
|
# Configure monitoring for an MCP server
|
|
def configure_monitoring(server):
|
|
# Set up Prometheus metrics
|
|
prometheus_metrics = {
|
|
"request_count": Counter("mcp_requests_total", "Total MCP requests"),
|
|
"request_latency": Histogram(
|
|
"mcp_request_duration_seconds",
|
|
"Request duration in seconds",
|
|
buckets=[0.01, 0.05, 0.1, 0.5, 1.0, 2.5, 5.0, 10.0]
|
|
),
|
|
"tool_execution_count": Counter(
|
|
"mcp_tool_executions_total",
|
|
"Tool execution count",
|
|
labelnames=["tool_name"]
|
|
),
|
|
"tool_execution_latency": Histogram(
|
|
"mcp_tool_duration_seconds",
|
|
"Tool execution duration in seconds",
|
|
labelnames=["tool_name"],
|
|
buckets=[0.01, 0.05, 0.1, 0.5, 1.0, 2.5, 5.0, 10.0]
|
|
),
|
|
"tool_errors": Counter(
|
|
"mcp_tool_errors_total",
|
|
"Tool execution errors",
|
|
labelnames=["tool_name", "error_type"]
|
|
)
|
|
}
|
|
|
|
# Add middleware for timing and recording metrics
|
|
server.add_middleware(PrometheusMiddleware(prometheus_metrics))
|
|
|
|
# Expose metrics endpoint
|
|
@server.router.get("/metrics")
|
|
async def metrics():
|
|
return generate_latest()
|
|
|
|
return server
|
|
```
|
|
|
|
## MCP Workflow Design Patterns
|
|
|
|
Well-designed MCP workflows improve efficiency, reliability, and maintainability. Here are key patterns to follow:
|
|
|
|
### 1. Chain of Tools Pattern
|
|
|
|
Connect multiple tools in a sequence where each tool's output becomes the input for the next:
|
|
|
|
```python
|
|
# Python Chain of Tools implementation
|
|
class ChainWorkflow:
|
|
def __init__(self, tools_chain):
|
|
self.tools_chain = tools_chain # List of tool names to execute in sequence
|
|
|
|
async def execute(self, mcp_client, initial_input):
|
|
current_result = initial_input
|
|
all_results = {"input": initial_input}
|
|
|
|
for tool_name in self.tools_chain:
|
|
# Execute each tool in the chain, passing previous result
|
|
response = await mcp_client.execute_tool(tool_name, current_result)
|
|
|
|
# Store result and use as input for next tool
|
|
all_results[tool_name] = response.result
|
|
current_result = response.result
|
|
|
|
return {
|
|
"final_result": current_result,
|
|
"all_results": all_results
|
|
}
|
|
|
|
# Example usage
|
|
data_processing_chain = ChainWorkflow([
|
|
"dataFetch",
|
|
"dataCleaner",
|
|
"dataAnalyzer",
|
|
"dataVisualizer"
|
|
])
|
|
|
|
result = await data_processing_chain.execute(
|
|
mcp_client,
|
|
{"source": "sales_database", "table": "transactions"}
|
|
)
|
|
```
|
|
|
|
### 2. Dispatcher Pattern
|
|
|
|
Use a central tool that dispatches to specialized tools based on input:
|
|
|
|
```csharp
|
|
public class ContentDispatcherTool : IMcpTool
|
|
{
|
|
private readonly IMcpClient _mcpClient;
|
|
|
|
public ContentDispatcherTool(IMcpClient mcpClient)
|
|
{
|
|
_mcpClient = mcpClient;
|
|
}
|
|
|
|
public string Name => "contentProcessor";
|
|
public string Description => "Processes content of various types";
|
|
|
|
public object GetSchema()
|
|
{
|
|
return new {
|
|
type = "object",
|
|
properties = new {
|
|
content = new { type = "string" },
|
|
contentType = new {
|
|
type = "string",
|
|
enum = new[] { "text", "html", "markdown", "csv", "code" }
|
|
},
|
|
operation = new {
|
|
type = "string",
|
|
enum = new[] { "summarize", "analyze", "extract", "convert" }
|
|
}
|
|
},
|
|
required = new[] { "content", "contentType", "operation" }
|
|
};
|
|
}
|
|
|
|
public async Task<ToolResponse> ExecuteAsync(ToolRequest request)
|
|
{
|
|
var content = request.Parameters.GetProperty("content").GetString();
|
|
var contentType = request.Parameters.GetProperty("contentType").GetString();
|
|
var operation = request.Parameters.GetProperty("operation").GetString();
|
|
|
|
// Determine which specialized tool to use
|
|
string targetTool = DetermineTargetTool(contentType, operation);
|
|
|
|
// Forward to the specialized tool
|
|
var specializedResponse = await _mcpClient.ExecuteToolAsync(
|
|
targetTool,
|
|
new { content, options = GetOptionsForTool(targetTool, operation) }
|
|
);
|
|
|
|
return new ToolResponse { Result = specializedResponse.Result };
|
|
}
|
|
|
|
private string DetermineTargetTool(string contentType, string operation)
|
|
{
|
|
return (contentType, operation) switch
|
|
{
|
|
("text", "summarize") => "textSummarizer",
|
|
("text", "analyze") => "textAnalyzer",
|
|
("html", _) => "htmlProcessor",
|
|
("markdown", _) => "markdownProcessor",
|
|
("csv", _) => "csvProcessor",
|
|
("code", _) => "codeAnalyzer",
|
|
_ => throw new ToolExecutionException($"No tool available for {contentType}/{operation}")
|
|
};
|
|
}
|
|
|
|
private object GetOptionsForTool(string toolName, string operation)
|
|
{
|
|
// Return appropriate options for each specialized tool
|
|
return toolName switch
|
|
{
|
|
"textSummarizer" => new { length = "medium" },
|
|
"htmlProcessor" => new { cleanUp = true, operation },
|
|
// Options for other tools...
|
|
_ => new { }
|
|
};
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3. Parallel Processing Pattern
|
|
|
|
Execute multiple tools simultaneously for efficiency:
|
|
|
|
```java
|
|
public class ParallelDataProcessingWorkflow {
|
|
private final McpClient mcpClient;
|
|
|
|
public ParallelDataProcessingWorkflow(McpClient mcpClient) {
|
|
this.mcpClient = mcpClient;
|
|
}
|
|
|
|
public WorkflowResult execute(String datasetId) {
|
|
// Step 1: Fetch dataset metadata (synchronous)
|
|
ToolResponse metadataResponse = mcpClient.executeTool("datasetMetadata",
|
|
Map.of("datasetId", datasetId));
|
|
|
|
// Step 2: Launch multiple analyses in parallel
|
|
CompletableFuture<ToolResponse> statisticalAnalysis = CompletableFuture.supplyAsync(() ->
|
|
mcpClient.executeTool("statisticalAnalysis", Map.of(
|
|
"datasetId", datasetId,
|
|
"type", "comprehensive"
|
|
))
|
|
);
|
|
|
|
CompletableFuture<ToolResponse> correlationAnalysis = CompletableFuture.supplyAsync(() ->
|
|
mcpClient.executeTool("correlationAnalysis", Map.of(
|
|
"datasetId", datasetId,
|
|
"method", "pearson"
|
|
))
|
|
);
|
|
|
|
CompletableFuture<ToolResponse> outlierDetection = CompletableFuture.supplyAsync(() ->
|
|
mcpClient.executeTool("outlierDetection", Map.of(
|
|
"datasetId", datasetId,
|
|
"sensitivity", "medium"
|
|
))
|
|
);
|
|
|
|
// Wait for all parallel tasks to complete
|
|
CompletableFuture<Void> allAnalyses = CompletableFuture.allOf(
|
|
statisticalAnalysis, correlationAnalysis, outlierDetection
|
|
);
|
|
|
|
allAnalyses.join(); // Wait for completion
|
|
|
|
// Step 3: Combine results
|
|
Map<String, Object> combinedResults = new HashMap<>();
|
|
combinedResults.put("metadata", metadataResponse.getResult());
|
|
combinedResults.put("statistics", statisticalAnalysis.join().getResult());
|
|
combinedResults.put("correlations", correlationAnalysis.join().getResult());
|
|
combinedResults.put("outliers", outlierDetection.join().getResult());
|
|
|
|
// Step 4: Generate summary report
|
|
ToolResponse summaryResponse = mcpClient.executeTool("reportGenerator",
|
|
Map.of("analysisResults", combinedResults));
|
|
|
|
// Return complete workflow result
|
|
WorkflowResult result = new WorkflowResult();
|
|
result.setDatasetId(datasetId);
|
|
result.setAnalysisResults(combinedResults);
|
|
result.setSummaryReport(summaryResponse.getResult());
|
|
|
|
return result;
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4. Error Recovery Pattern
|
|
|
|
Implement graceful fallbacks for tool failures:
|
|
|
|
```python
|
|
class ResilientWorkflow:
|
|
def __init__(self, mcp_client):
|
|
self.client = mcp_client
|
|
|
|
async def execute_with_fallback(self, primary_tool, fallback_tool, parameters):
|
|
try:
|
|
# Try primary tool first
|
|
response = await self.client.execute_tool(primary_tool, parameters)
|
|
return {
|
|
"result": response.result,
|
|
"source": "primary",
|
|
"tool": primary_tool
|
|
}
|
|
except ToolExecutionException as e:
|
|
# Log the failure
|
|
logging.warning(f"Primary tool '{primary_tool}' failed: {str(e)}")
|
|
|
|
# Fall back to secondary tool
|
|
try:
|
|
# Might need to transform parameters for fallback tool
|
|
fallback_params = self._adapt_parameters(parameters, primary_tool, fallback_tool)
|
|
|
|
response = await self.client.execute_tool(fallback_tool, fallback_params)
|
|
return {
|
|
"result": response.result,
|
|
"source": "fallback",
|
|
"tool": fallback_tool,
|
|
"primaryError": str(e)
|
|
}
|
|
except ToolExecutionException as fallback_error:
|
|
# Both tools failed
|
|
logging.error(f"Both primary and fallback tools failed. Fallback error: {str(fallback_error)}")
|
|
raise WorkflowExecutionException(
|
|
f"Workflow failed: primary error: {str(e)}; fallback error: {str(fallback_error)}"
|
|
)
|
|
|
|
def _adapt_parameters(self, params, from_tool, to_tool):
|
|
"""Adapt parameters between different tools if needed"""
|
|
# This implementation would depend on the specific tools
|
|
# For this example, we'll just return the original parameters
|
|
return params
|
|
|
|
# Example usage
|
|
async def get_weather(workflow, location):
|
|
return await workflow.execute_with_fallback(
|
|
"premiumWeatherService", # Primary (paid) weather API
|
|
"basicWeatherService", # Fallback (free) weather API
|
|
{"location": location}
|
|
)
|
|
```
|
|
|
|
### 5. Workflow Composition Pattern
|
|
|
|
Build complex workflows by composing simpler ones:
|
|
|
|
```csharp
|
|
public class CompositeWorkflow : IWorkflow
|
|
{
|
|
private readonly List<IWorkflow> _workflows;
|
|
|
|
public CompositeWorkflow(IEnumerable<IWorkflow> workflows)
|
|
{
|
|
_workflows = new List<IWorkflow>(workflows);
|
|
}
|
|
|
|
public async Task<WorkflowResult> ExecuteAsync(WorkflowContext context)
|
|
{
|
|
var results = new Dictionary<string, object>();
|
|
|
|
foreach (var workflow in _workflows)
|
|
{
|
|
var workflowResult = await workflow.ExecuteAsync(context);
|
|
|
|
// Store each workflow's result
|
|
results[workflow.Name] = workflowResult;
|
|
|
|
// Update context with the result for the next workflow
|
|
context = context.WithResult(workflow.Name, workflowResult);
|
|
}
|
|
|
|
return new WorkflowResult(results);
|
|
}
|
|
|
|
public string Name => "CompositeWorkflow";
|
|
public string Description => "Executes multiple workflows in sequence";
|
|
}
|
|
|
|
// Example usage
|
|
var documentWorkflow = new CompositeWorkflow(new IWorkflow[] {
|
|
new DocumentFetchWorkflow(),
|
|
new DocumentProcessingWorkflow(),
|
|
new InsightGenerationWorkflow(),
|
|
new ReportGenerationWorkflow()
|
|
});
|
|
|
|
var result = await documentWorkflow.ExecuteAsync(new WorkflowContext {
|
|
Parameters = new { documentId = "12345" }
|
|
});
|
|
```
|
|
|
|
# Testing MCP Servers: Best Practices and Top Tips
|
|
|
|
## Overview
|
|
|
|
Testing is a critical aspect of developing reliable, high-quality MCP servers. This guide provides comprehensive best practices and tips for testing your MCP servers throughout the development lifecycle, from unit tests to integration tests and end-to-end validation.
|
|
|
|
## Why Testing Matters for MCP Servers
|
|
|
|
MCP servers serve as crucial middleware between AI models and client applications. Thorough testing ensures:
|
|
|
|
- Reliability in production environments
|
|
- Accurate handling of requests and responses
|
|
- Proper implementation of MCP specifications
|
|
- Resilience against failures and edge cases
|
|
- Consistent performance under various loads
|
|
|
|
## Unit Testing for MCP Servers
|
|
|
|
### Unit Testing (Foundation)
|
|
|
|
Unit tests verify individual components of your MCP server in isolation.
|
|
|
|
#### What to Test
|
|
|
|
1. **Resource Handlers**: Test each resource handler's logic independently
|
|
2. **Tool Implementations**: Verify tool behavior with various inputs
|
|
3. **Prompt Templates**: Ensure prompt templates render correctly
|
|
4. **Schema Validation**: Test parameter validation logic
|
|
5. **Error Handling**: Verify error responses for invalid inputs
|
|
|
|
#### Best Practices for Unit Testing
|
|
|
|
```csharp
|
|
// Example unit test for a calculator tool in C#
|
|
[Fact]
|
|
public async Task CalculatorTool_Add_ReturnsCorrectSum()
|
|
{
|
|
// Arrange
|
|
var calculator = new CalculatorTool();
|
|
var parameters = new Dictionary<string, object>
|
|
{
|
|
["operation"] = "add",
|
|
["a"] = 5,
|
|
["b"] = 7
|
|
};
|
|
|
|
// Act
|
|
var response = await calculator.ExecuteAsync(parameters);
|
|
var result = JsonSerializer.Deserialize<CalculationResult>(response.Content[0].ToString());
|
|
|
|
// Assert
|
|
Assert.Equal(12, result.Value);
|
|
}
|
|
```
|
|
|
|
```python
|
|
# Example unit test for a calculator tool in Python
|
|
def test_calculator_tool_add():
|
|
# Arrange
|
|
calculator = CalculatorTool()
|
|
parameters = {
|
|
"operation": "add",
|
|
"a": 5,
|
|
"b": 7
|
|
}
|
|
|
|
# Act
|
|
response = calculator.execute(parameters)
|
|
result = json.loads(response.content[0].text)
|
|
|
|
# Assert
|
|
assert result["value"] == 12
|
|
```
|
|
|
|
### Integration Testing (Middle Layer)
|
|
|
|
Integration tests verify interactions between components of your MCP server.
|
|
|
|
#### What to Test
|
|
|
|
1. **Server Initialization**: Test server startup with various configurations
|
|
2. **Route Registration**: Verify all endpoints are correctly registered
|
|
3. **Request Processing**: Test the full request-response cycle
|
|
4. **Error Propagation**: Ensure errors are properly handled across components
|
|
5. **Authentication & Authorization**: Test security mechanisms
|
|
|
|
#### Best Practices for Integration Testing
|
|
|
|
```csharp
|
|
// Example integration test for MCP server in C#
|
|
[Fact]
|
|
public async Task Server_ProcessToolRequest_ReturnsValidResponse()
|
|
{
|
|
// Arrange
|
|
var server = new McpServer();
|
|
server.RegisterTool(new CalculatorTool());
|
|
await server.StartAsync();
|
|
|
|
var request = new McpRequest
|
|
{
|
|
Tool = "calculator",
|
|
Parameters = new Dictionary<string, object>
|
|
{
|
|
["operation"] = "multiply",
|
|
["a"] = 6,
|
|
["b"] = 7
|
|
}
|
|
};
|
|
|
|
// Act
|
|
var response = await server.ProcessRequestAsync(request);
|
|
|
|
// Assert
|
|
Assert.NotNull(response);
|
|
Assert.Equal(McpStatusCodes.Success, response.StatusCode);
|
|
// Additional assertions for response content
|
|
|
|
// Cleanup
|
|
await server.StopAsync();
|
|
}
|
|
```
|
|
|
|
### End-to-End Testing (Top Layer)
|
|
|
|
End-to-end tests verify the complete system behavior from client to server.
|
|
|
|
#### What to Test
|
|
|
|
1. **Client-Server Communication**: Test complete request-response cycles
|
|
2. **Real Client SDKs**: Test with actual client implementations
|
|
3. **Performance Under Load**: Verify behavior with multiple concurrent requests
|
|
4. **Error Recovery**: Test system recovery from failures
|
|
5. **Long-Running Operations**: Verify handling of streaming and long operations
|
|
|
|
#### Best Practices for E2E Testing
|
|
|
|
```typescript
|
|
// Example E2E test with a client in TypeScript
|
|
describe('MCP Server E2E Tests', () => {
|
|
let client: McpClient;
|
|
|
|
beforeAll(async () => {
|
|
// Start server in test environment
|
|
await startTestServer();
|
|
client = new McpClient('http://localhost:5000');
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await stopTestServer();
|
|
});
|
|
|
|
test('Client can invoke calculator tool and get correct result', async () => {
|
|
// Act
|
|
const response = await client.invokeToolAsync('calculator', {
|
|
operation: 'divide',
|
|
a: 20,
|
|
b: 4
|
|
});
|
|
|
|
// Assert
|
|
expect(response.statusCode).toBe(200);
|
|
expect(response.content[0].text).toContain('5');
|
|
});
|
|
});
|
|
```
|
|
|
|
## Mocking Strategies for MCP Testing
|
|
|
|
Mocking is essential for isolating components during testing.
|
|
|
|
### Components to Mock
|
|
|
|
1. **External AI Models**: Mock model responses for predictable testing
|
|
2. **External Services**: Mock API dependencies (databases, third-party services)
|
|
3. **Authentication Services**: Mock identity providers
|
|
4. **Resource Providers**: Mock expensive resource handlers
|
|
|
|
### Example: Mocking an AI Model Response
|
|
|
|
```csharp
|
|
// C# example with Moq
|
|
var mockModel = new Mock<ILanguageModel>();
|
|
mockModel
|
|
.Setup(m => m.GenerateResponseAsync(
|
|
It.IsAny<string>(),
|
|
It.IsAny<McpRequestContext>()))
|
|
.ReturnsAsync(new ModelResponse {
|
|
Text = "Mocked model response",
|
|
FinishReason = FinishReason.Completed
|
|
});
|
|
|
|
var server = new McpServer(modelClient: mockModel.Object);
|
|
```
|
|
|
|
```python
|
|
# Python example with unittest.mock
|
|
@patch('mcp_server.models.OpenAIModel')
|
|
def test_with_mock_model(mock_model):
|
|
# Configure mock
|
|
mock_model.return_value.generate_response.return_value = {
|
|
"text": "Mocked model response",
|
|
"finish_reason": "completed"
|
|
}
|
|
|
|
# Use mock in test
|
|
server = McpServer(model_client=mock_model)
|
|
# Continue with test
|
|
```
|
|
|
|
## Performance Testing
|
|
|
|
Performance testing is crucial for production MCP servers.
|
|
|
|
### What to Measure
|
|
|
|
1. **Latency**: Response time for requests
|
|
2. **Throughput**: Requests handled per second
|
|
3. **Resource Utilization**: CPU, memory, network usage
|
|
4. **Concurrency Handling**: Behavior under parallel requests
|
|
5. **Scaling Characteristics**: Performance as load increases
|
|
|
|
### Tools for Performance Testing
|
|
|
|
- **k6**: Open-source load testing tool
|
|
- **JMeter**: Comprehensive performance testing
|
|
- **Locust**: Python-based load testing
|
|
- **Azure Load Testing**: Cloud-based performance testing
|
|
|
|
### Example: Basic Load Test with k6
|
|
|
|
```javascript
|
|
// k6 script for load testing MCP server
|
|
import http from 'k6/http';
|
|
import { check, sleep } from 'k6';
|
|
|
|
export const options = {
|
|
vus: 10, // 10 virtual users
|
|
duration: '30s',
|
|
};
|
|
|
|
export default function () {
|
|
const payload = JSON.stringify({
|
|
tool: 'calculator',
|
|
parameters: {
|
|
operation: 'add',
|
|
a: Math.floor(Math.random() * 100),
|
|
b: Math.floor(Math.random() * 100)
|
|
}
|
|
});
|
|
|
|
const params = {
|
|
headers: {
|
|
'Content-Type': 'application/json',
|
|
'Authorization': 'Bearer test-token'
|
|
},
|
|
};
|
|
|
|
const res = http.post('http://localhost:5000/api/tools/invoke', payload, params);
|
|
|
|
check(res, {
|
|
'status is 200': (r) => r.status === 200,
|
|
'response time < 500ms': (r) => r.timings.duration < 500,
|
|
});
|
|
|
|
sleep(1);
|
|
}
|
|
```
|
|
|
|
## Test Automation for MCP Servers
|
|
|
|
Automating your tests ensures consistent quality and faster feedback loops.
|
|
|
|
### CI/CD Integration
|
|
|
|
1. **Run Unit Tests on Pull Requests**: Ensure code changes don't break existing functionality
|
|
2. **Integration Tests in Staging**: Run integration tests in pre-production environments
|
|
3. **Performance Baselines**: Maintain performance benchmarks to catch regressions
|
|
4. **Security Scans**: Automate security testing as part of the pipeline
|
|
|
|
### Example CI Pipeline (GitHub Actions)
|
|
|
|
```yaml
|
|
name: MCP Server Tests
|
|
|
|
on:
|
|
push:
|
|
branches: [ main ]
|
|
pull_request:
|
|
branches: [ main ]
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v2
|
|
|
|
- name: Set up Runtime
|
|
uses: actions/setup-dotnet@v1
|
|
with:
|
|
dotnet-version: '8.0.x'
|
|
|
|
- name: Restore dependencies
|
|
run: dotnet restore
|
|
|
|
- name: Build
|
|
run: dotnet build --no-restore
|
|
|
|
- name: Unit Tests
|
|
run: dotnet test --no-build --filter Category=Unit
|
|
|
|
- name: Integration Tests
|
|
run: dotnet test --no-build --filter Category=Integration
|
|
|
|
- name: Performance Tests
|
|
run: dotnet run --project tests/PerformanceTests/PerformanceTests.csproj
|
|
```
|
|
|
|
## Testing for Compliance with MCP Specification
|
|
|
|
Verify your server correctly implements the MCP specification.
|
|
|
|
### Key Compliance Areas
|
|
|
|
1. **API Endpoints**: Test required endpoints (/resources, /tools, etc.)
|
|
2. **Request/Response Format**: Validate schema compliance
|
|
3. **Error Codes**: Verify correct status codes for various scenarios
|
|
4. **Content Types**: Test handling of different content types
|
|
5. **Authentication Flow**: Verify spec-compliant auth mechanisms
|
|
|
|
### Compliance Test Suite
|
|
|
|
```csharp
|
|
[Fact]
|
|
public async Task Server_ResourceEndpoint_ReturnsCorrectSchema()
|
|
{
|
|
// Arrange
|
|
var client = new HttpClient();
|
|
client.DefaultRequestHeaders.Add("Authorization", "Bearer test-token");
|
|
|
|
// Act
|
|
var response = await client.GetAsync("http://localhost:5000/api/resources");
|
|
var content = await response.Content.ReadAsStringAsync();
|
|
var resources = JsonSerializer.Deserialize<ResourceList>(content);
|
|
|
|
// Assert
|
|
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
|
|
Assert.NotNull(resources);
|
|
Assert.All(resources.Resources, resource =>
|
|
{
|
|
Assert.NotNull(resource.Id);
|
|
Assert.NotNull(resource.Type);
|
|
// Additional schema validation
|
|
});
|
|
}
|
|
```
|
|
|
|
## Top 10 Tips for Effective MCP Server Testing
|
|
|
|
1. **Test Tool Definitions Separately**: Verify schema definitions independently from tool logic
|
|
2. **Use Parameterized Tests**: Test tools with a variety of inputs, including edge cases
|
|
3. **Check Error Responses**: Verify proper error handling for all possible error conditions
|
|
4. **Test Authorization Logic**: Ensure proper access control for different user roles
|
|
5. **Monitor Test Coverage**: Aim for high coverage of critical path code
|
|
6. **Test Streaming Responses**: Verify proper handling of streaming content
|
|
7. **Simulate Network Issues**: Test behavior under poor network conditions
|
|
8. **Test Resource Limits**: Verify behavior when reaching quotas or rate limits
|
|
9. **Automate Regression Tests**: Build a suite that runs on every code change
|
|
10. **Document Test Cases**: Maintain clear documentation of test scenarios
|
|
|
|
## Common Testing Pitfalls
|
|
|
|
- **Over-reliance on happy path testing**: Make sure to test error cases thoroughly
|
|
- **Ignoring performance testing**: Identify bottlenecks before they affect production
|
|
- **Testing in isolation only**: Combine unit, integration, and E2E tests
|
|
- **Incomplete API coverage**: Ensure all endpoints and features are tested
|
|
- **Inconsistent test environments**: Use containers to ensure consistent test environments
|
|
|
|
## Conclusion
|
|
|
|
A comprehensive testing strategy is essential for developing reliable, high-quality MCP servers. By implementing the best practices and tips outlined in this guide, you can ensure your MCP implementations meet the highest standards of quality, reliability, and performance.
|
|
|
|
|
|
## Key Takeaways
|
|
|
|
1. **Tool Design**: Follow single responsibility principle, use dependency injection, and design for composability
|
|
2. **Schema Design**: Create clear, well-documented schemas with proper validation constraints
|
|
3. **Error Handling**: Implement graceful error handling, structured error responses, and retry logic
|
|
4. **Performance**: Use caching, asynchronous processing, and resource throttling
|
|
5. **Security**: Apply thorough input validation, authorization checks, and sensitive data handling
|
|
6. **Testing**: Create comprehensive unit, integration, and end-to-end tests
|
|
7. **Workflow Patterns**: Apply established patterns like chains, dispatchers, and parallel processing
|
|
|
|
## Exercise
|
|
|
|
Design an MCP tool and workflow for a document processing system that:
|
|
|
|
1. Accepts documents in multiple formats (PDF, DOCX, TXT)
|
|
2. Extracts text and key information from the documents
|
|
3. Classifies documents by type and content
|
|
4. Generates a summary of each document
|
|
|
|
Implement the tool schemas, error handling, and a workflow pattern that best suits this scenario. Consider how you would test this implementation.
|
|
|
|
## Resources
|
|
|
|
1. Join the MCP community on the [Microsoft Foundry Discord Community](https://aka.ms/foundrydevs) to stay updated on the latest developments
|
|
2. Contribute to open-source [MCP projects](https://github.com/modelcontextprotocol)
|
|
3. Apply MCP principles in your own organization's AI initiatives
|
|
4. Explore specialized MCP implementations for your industry.
|
|
5. Consider taking advanced courses on specific MCP topics, such as multi-modal integration or enterprise application integration.
|
|
6. Experiment with building your own MCP tools and workflows using the principles learned through the [Hands on Lab](../10-StreamliningAIWorkflowsBuildingAnMCPServerWithAIToolkit/README.md)
|
|
|
|
## What's Next
|
|
|
|
Next: [Case Studies](../09-CaseStudy/README.md)
|
|
|
|
|