learningcircuit--local-deep-research/examples/optimization/README.md

# Optimization Tools for Local Deep Research

This directory contains scripts for optimizing Local Deep Research's parameters.

## Parameter Optimization

Optimization helps find the best settings for different use cases:
- **Balanced**: Optimizes for a good balance of speed and quality
- **Speed-focused**: Prioritizes faster responses
- **Quality-focused**: Prioritizes more accurate, comprehensive answers
- **Efficiency**: Balances quality, speed, and resource usage

## Available Scripts

### Main Optimization Runner

`run_optimization.py` provides a command-line interface for running different types of optimization:

```bash
python run_optimization.py "What are the latest developments in fusion energy?" --mode quality --trials 20
```

Options:
- `query`: The research query to use for optimization
- `--output-dir`: Directory to save results (default: "optimization_results")
- `--search-tool`: Search tool to use (default: "searxng")
- `--model`: Model name for the LLM (e.g., 'claude-3-sonnet-20240229')
- `--provider`: Provider for the LLM (e.g., 'anthropic', 'openai', 'openai_endpoint')
- `--endpoint-url`: Custom endpoint URL (e.g., 'https://openrouter.ai/api/v1' for OpenRouter)
- `--api-key`: API key for the LLM provider
- `--temperature`: Temperature for the LLM (default: 0.7)
- `--trials`: Number of parameter combinations to try (default: 30)
- `--mode`: Optimization mode ("balanced", "speed", "quality", "efficiency")
- `--weights`: Custom weights as JSON string, e.g., '{"quality": 0.7, "speed": 0.3}'

### Example Scripts

- `example_optimization.py`: Full example with all optimization modes
- `example_quick_optimization.py`: Simplified example for quick testing
- `gemini_optimization.py`: Example using Gemini 2.0 Flash via OpenRouter
- `llm_multi_benchmark.py`: Example with multi-benchmark optimization and custom LLM settings

### Utility Scripts

- `update_llm_config.py`: Update LLM configuration in the database
  ```bash
  python update_llm_config.py --model "google/gemini-2.0-flash" --provider "openai_endpoint" --endpoint "https://openrouter.ai/api/v1" --api-key "your-api-key"
  ```

- `run_gemini_benchmark.py`: Run benchmarks with Gemini 2.0 Flash via OpenRouter
  ```bash
  python run_gemini_benchmark.py --api-key "your-api-key" --examples 10
  ```

**Important**: Always update the LLM configuration in the database before running benchmarks or optimization to ensure consistent behavior. The utility scripts above help you do this.

## How Optimization Works

The optimization process:
1. Defines a parameter space to explore (iterations, questions per iteration, search strategy, etc.)
2. Runs multiple trials with different parameter combinations
3. Evaluates each combination using benchmarks
4. Uses Optuna to efficiently search for the best parameters
5. Returns the optimal parameters and stores detailed results

## Example Parameter Space

Optimization explores parameters such as:
- `iterations`: Number of search iterations
- `questions_per_iteration`: Number of questions to generate per iteration
- `search_strategy`: Search strategy to use ("standard", "rapid", "iterdrag", etc.)
- `max_results`: Maximum number of search results to consider
- Other system-specific parameters

## Using Custom LLM Models

The optimization tools support different LLM providers and models:

### Via OpenRouter

To use models like Gemini or other models via OpenRouter:

```bash
python run_optimization.py "Research query" --model "google/gemini-2.0-flash-001" --provider "openai_endpoint" --endpoint-url "https://openrouter.ai/api/v1" --api-key "your-openrouter-api-key"
```

Or use the dedicated example:

```bash
python gemini_optimization.py --api-key "your-openrouter-api-key"
```

### Direct Provider Access

To use models directly from providers like Anthropic or OpenAI:

```bash
python run_optimization.py "Research query" --model "claude-3-sonnet-20240229" --provider "anthropic" --api-key "your-anthropic-api-key"
```

Or for OpenAI:

```bash
python run_optimization.py "Research query" --model "gpt-4-turbo" --provider "openai" --api-key "your-openai-api-key"
```

## Using Optimization Results

After running optimization, you can use the resulting parameters by updating your configuration:

```python
from local_deep_research.api import quick_summary

results = quick_summary(
    query="What are the latest developments in fusion energy?",
    iterations=best_params["iterations"],
    questions_per_iteration=best_params["questions_per_iteration"],
    search_strategy=best_params["search_strategy"],
    # Other optimized parameters
    # You can also use custom LLM configuration:
    model_name="your-model",
    provider="your-provider"
)
```