# Gemini Benchmark Examples

This directory contains example scripts for running benchmarks with Gemini models via OpenRouter.

## Scripts Included

### run_gemini_benchmark_fixed.py
A comprehensive benchmark script that runs both SimpleQA and BrowseComp evaluations
using Google's Gemini 2.0 Flash model via the OpenRouter API.

Key features:
- Patches the LLM configuration to use Gemini for all evaluations
- Supports both SimpleQA and BrowseComp benchmarks
- Properly handles result collection and reporting

## Usage

To run the benchmark with Gemini:

```bash
# Run with default settings (1 example)
python run_gemini_benchmark_fixed.py

# Run with custom number of examples
python run_gemini_benchmark_fixed.py --examples 5
```

## Notes

These scripts assume you have:
1. An OpenRouter API key configured in your LDR database
2. The correct access permissions for the Gemini model