# Claude API Grading Benchmark

This benchmark integrates Claude 3 Sonnet for grading benchmark results with proper API access through the local database.

## Features

- Uses Claude 3 Sonnet for grading benchmark results
- Accesses API keys from the local database
- Supports SimpleQA and BrowseComp benchmarks
- Provides composite scoring with customizable weights
- Comprehensive metrics and accuracy reports

## Usage

From the project root directory:

```bash
# Run with default settings (source_based strategy, 1 iteration, 5 examples)
./examples/benchmarks/claude_grading/run_benchmark.sh

# Run with custom parameters
./examples/benchmarks/claude_grading/run_benchmark.sh --strategy source_based --iterations 2 --examples 200
```

## How It Works

The benchmark integrates with the evaluation system by patching the grading module to use the local `get_llm` function, which properly retrieves API keys from the database and configures the Claude model for grading.

This approach ensures accurate grading of benchmark results and enables comparison between different strategies and configurations.

## Requirements

- Valid Claude API key stored in the local database
- SearXNG search engine running locally
- Python dependencies installed

## Output

Results are saved in the `benchmark_results` directory with comprehensive metrics:
- Accuracy scores
- Processing times
- Grading confidence
- Detailed evaluation reports