learningcircuit--local-deep-research/docs/analytics-dashboard.md

# Metrics Dashboard Guide

The Local Deep Research metrics dashboard provides insights into your research activities, costs, and system performance.

> **Note**: This documentation is maintained by the community and may contain inaccuracies. While we strive to keep it up-to-date, please verify critical information and report any errors via [GitHub Issues](https://github.com/LearningCircuit/local-deep-research/issues).

## Overview

The metrics dashboard tracks:
- Token usage and costs across LLM providers
- Search engine performance and health
- Research session statistics
- User satisfaction ratings
- Rate limiting status

## Accessing Metrics

- **Web Interface**: Navigate to `/metrics` in your browser
- **Direct URL**: `http://localhost:5000/metrics`

## Dashboard Components

### System Overview Cards

The main dashboard displays key metrics:

- **Total Tokens Used**: Shows total token consumption with expandable breakdown by model
- **Total Researches**: Number of research sessions conducted
- **Average Response Time**: System performance metric
- **Success Rate**: Percentage of successful operations
- **User Satisfaction**: Average star rating from user feedback
- **Estimated Costs**: Token-based cost estimation with provider breakdown

### Time-based Filtering

Filter analytics by time period:
- Last 7 days
- Last 30 days
- Last 3 months
- Last year
- All time

Additional filters:
- Research mode (Quick Summary / Detailed / All)

## Detailed Analytics Pages

### Star Reviews Analytics

Access via: `/metrics/star-reviews`

**Features:**
- 5-star rating distribution
- Average ratings by time period
- Rating trends visualization
- Breakdown by model and search engine
- User feedback analysis

### Cost Analytics

Access via: `/metrics/costs`

**Tracked Metrics:**
- Cost breakdown by provider (OpenAI, Anthropic, etc.)
- Token usage details (input/output/total)
- Cost trends over time
- Model-specific cost analysis
- Research type cost comparison

### Rate Limiting Dashboard

**Real-time monitoring of:**
- Search engine rate limit status
- Success/failure rates per engine
- Wait time tracking
- Engine health indicators:
  - 🟢 Healthy: >95% success rate
  - 🟡 Degraded: 70-95% success rate
  - 🔴 Poor: <70% success rate

## Metrics Tracked

### Token Metrics
- Total tokens (input + output)
- Token usage by model
- Average tokens per research
- Token consumption trends

### Search Metrics
- Search engine usage frequency
- Response times per engine
- Success/failure rates
- Results count statistics

### Research Metrics
- Total research sessions
- Research duration
- Completion status
- Strategy usage
- Query complexity

### Performance Metrics
- API response times
- System latency
- Error rates
- Throughput statistics

## Data Export

### Research Reports
Export individual research results as:
- **PDF**: Formatted reports with citations
- **Markdown**: Raw markdown with formatting
- **JSON**: Structured data via API

### Analytics Data
Access analytics data via API:

```bash
# Get overall metrics
curl http://localhost:5000/api/metrics

# Get specific research metrics
curl http://localhost:5000/api/metrics/research/<research_id>

# Get enhanced tracking data
curl http://localhost:5000/api/metrics/enhanced

# Get rating analytics
curl http://localhost:5000/api/star-reviews

# Get cost analytics
curl http://localhost:5000/api/cost-analytics

# Get rate limiting status
curl http://localhost:5000/api/rate-limiting
```

## Visualizations

The dashboard uses Chart.js to display:

- **Line Charts**: Token usage and search activity over time
- **Bar Charts**: Model usage comparison, cost breakdown
- **Pie Charts**: Provider distribution, search engine usage
- **Progress Indicators**: Success rates, health status

## Cost Tracking

### Automatic Cost Calculation

Costs are calculated based on:
- Provider pricing (OpenAI, Anthropic, etc.)
- Actual token usage
- Model-specific rates
- Both input and output tokens

### Supported Providers
- OpenAI (GPT-3.5, GPT-4)
- Anthropic (Claude models)
- Google (Gemini models)
- Local models (shown as $0)

## Rate Limiting Analytics

### Monitoring Features
- Real-time rate limit status
- Historical rate limit events
- Automatic wait time optimization
- Per-engine performance tracking

### Managing Rate Limits

Via command line:
```bash
# Check status
python -m local_deep_research.web_search_engines.rate_limiting status

# Reset rate limits
python -m local_deep_research.web_search_engines.rate_limiting reset
```

## Privacy and Data Storage

- All analytics data is stored locally
- No external analytics services used
- Data stored in SQLite databases
- Configurable data retention

## Using Analytics for Optimization

### Identify Cost Drivers
1. Review high-token queries in cost analytics
2. Compare model costs vs. quality ratings
3. Optimize model selection for different tasks

### Improve Search Performance
1. Monitor search engine health status
2. Identify frequently rate-limited engines
3. Adjust search strategy based on success rates

### Enhance Research Quality
1. Analyze user ratings by research type
2. Review low-rated sessions for patterns
3. Adjust parameters based on feedback

## Benchmarking Integration

For advanced users, analytics integrates with the benchmarking system:

- Track performance across different configurations
- Visualize optimization results (when matplotlib available)
- Compare quality vs. speed trade-offs
- Export benchmark data for analysis

## API Reference

### Metrics Endpoints

```bash
# General metrics with optional time filter
curl 'http://localhost:5000/api/metrics?days=30&mode=quick'

# Research-specific metrics
curl http://localhost:5000/api/metrics/research/<id>

# Enhanced metrics (detailed tracking)
curl http://localhost:5000/api/metrics/enhanced

# Star ratings data
curl 'http://localhost:5000/api/star-reviews?days=30'

# Cost breakdown
curl 'http://localhost:5000/api/cost-analytics?provider=openai'

# Rate limit status
curl http://localhost:5000/api/rate-limiting
```

## Related Documentation

- [Features Documentation](features.md)
- [Configuration Guide](env_configuration.md)
- [Full Configuration Reference](CONFIGURATION.md)
- [API Documentation](api-quickstart.md)
- [Benchmarking Guide](BENCHMARKING.md)