learningcircuit--local-deep-research/docs/MIGRATION_GUIDE_v1.md

# Migration Guide: LDR v0.x to v1.0

## Overview

Local Deep Research v1.0 introduces significant security and architectural improvements:

- **User Authentication**: All access now requires authentication
- **Per-User Encrypted Databases**: Each user has their own encrypted SQLCipher database
- **Settings Snapshots**: Thread-safe settings management for concurrent operations
- **New API Structure**: Reorganized endpoints under blueprint prefixes

## Breaking Changes

### 1. Authentication Required

**v0.x**: Open access, no authentication
```python
# Direct API access
from local_deep_research.api import quick_summary
result = quick_summary("query")
```

**v1.0**: Authentication required
```python
from local_deep_research.api import quick_summary
from local_deep_research.settings import SettingsManager
from local_deep_research.database.session_context import get_user_db_session

# Must authenticate first
with get_user_db_session(username="user", password="pass") as session:
    settings_manager = SettingsManager(session)
    settings_snapshot = settings_manager.get_all_settings()

    result = quick_summary(
        "query",
        settings_snapshot=settings_snapshot  # Required parameter
    )
```

### 2. HTTP API Changes

#### Endpoint Structure
- **v0.x**: `/api/v1/quick_summary`
- **v1.0**: `/api/start_research`

#### Authentication Flow
```python
import requests

# v1.0 requires session-based authentication
session = requests.Session()

# 1. Login
session.post(
    "http://localhost:5000/auth/login",
    json={"username": "user", "password": "pass"}
)

# 2. Get CSRF token for state-changing operations
csrf = session.get("http://localhost:5000/auth/csrf-token").json()["csrf_token"]

# 3. Make API requests with CSRF token
response = session.post(
    "http://localhost:5000/api/start_research",
    json={"query": "test"},
    headers={"X-CSRF-Token": csrf}
)
```

### 3. Database Changes

#### v0.x
- Single shared database: `ldr.db`
- No encryption
- Direct database access from any thread

#### v1.0
- Per-user databases: `encrypted_databases/{username}.db`
- SQLCipher encryption with user passwords
- Thread-local session management
- In-memory queue tracking (no more service_db)

### 4. Settings Management

#### v0.x
```python
# Direct settings access
from local_deep_research.config import get_db_setting
value = get_db_setting("llm.provider")
```

#### v1.0
```python
# Settings require context
from local_deep_research.settings import SettingsManager

# Within authenticated session
settings_manager = SettingsManager(session)
value = settings_manager.get_setting("llm.provider")

# Or use settings snapshot for thread safety
settings_snapshot = settings_manager.get_all_settings()
```

## Migration Steps

### 1. Update Dependencies

```bash
pip install --upgrade local-deep-research
```

### 2. Create User Accounts

Users must create accounts through the web interface:

1. Start the server: `python -m local_deep_research.web.app`
2. Open http://localhost:5000
3. Click "Register" and create an account
4. Configure LLM providers and API keys in Settings

### 3. Update Programmatic Code

#### Before (v0.x):
```python
from local_deep_research.api import (
    quick_summary,
    detailed_research,
    generate_report
)

# Direct usage
result = quick_summary("What is AI?")
```

#### After (v1.0):
```python
from local_deep_research.api import quick_summary
from local_deep_research.settings import SettingsManager
from local_deep_research.database.session_context import get_user_db_session

def run_research(username, password, query):
    with get_user_db_session(username, password) as session:
        settings_manager = SettingsManager(session)
        settings_snapshot = settings_manager.get_all_settings()

        return quick_summary(
            query=query,
            settings_snapshot=settings_snapshot,
            # Other parameters remain the same
            iterations=2,
            questions_per_iteration=3
        )
```

### 4. Update HTTP API Calls

Create a wrapper for authenticated requests:

```python
class LDRClient:
    def __init__(self, base_url="http://localhost:5000"):
        self.base_url = base_url
        self.session = requests.Session()
        self.csrf_token = None

    def login(self, username, password):
        response = self.session.post(
            f"{self.base_url}/auth/login",
            json={"username": username, "password": password}
        )
        if response.status_code == 200:
            self.csrf_token = self.session.get(
                f"{self.base_url}/auth/csrf-token"
            ).json()["csrf_token"]
        return response

    def start_research(self, query, **kwargs):
        return self.session.post(
            f"{self.base_url}/api/start_research",
            json={"query": query, **kwargs},
            headers={"X-CSRF-Token": self.csrf_token}
        )

# Usage
client = LDRClient()
client.login("user", "pass")
result = client.start_research("What is quantum computing?")
```

### 5. Update Configuration

#### API Keys
API keys are now stored encrypted in per-user databases. Users must:
1. Login to the web interface
2. Go to Settings
3. Re-enter API keys for LLM providers

#### Custom LLMs
Custom LLM registrations now require settings context:

```python
# v1.0 custom LLM with settings support
def create_custom_llm(model_name=None, temperature=None, settings_snapshot=None):
    # Access settings from snapshot if needed
    api_key = settings_snapshot.get("llm.custom.api_key", {}).get("value")
    return CustomLLM(api_key=api_key, model=model_name, temperature=temperature)
```

## Common Issues and Solutions

### Issue: "No settings context available in thread"
**Solution**: Pass `settings_snapshot` parameter to all API calls

### Issue: "Encrypted database requires password"
**Solution**: Ensure you're using `get_user_db_session()` with credentials

### Issue: CSRF token errors
**Solution**: Get fresh CSRF token before state-changing requests

### Issue: Old endpoints return 404
**Solution**: Update to new endpoint structure (see mapping above)

### Issue: Rate limiting not working
**Solution**: Rate limits are now per-user; ensure proper authentication

## Backward Compatibility

For temporary backward compatibility, you can:

1. Set environment variable: `LDR_USE_SHARED_DB=1` (not recommended)
2. Create a compatibility wrapper for your existing code

```python
# compatibility.py
import os
os.environ["LDR_USE_SHARED_DB"] = "1"  # Use at your own risk

def quick_summary_compat(query, **kwargs):
    # Minimal compatibility wrapper
    # Note: This bypasses security features!
    from local_deep_research.api import quick_summary
    return quick_summary(query, settings_snapshot={}, **kwargs)
```

⚠️ **Warning**: Compatibility mode bypasses security features and is not recommended for production use.

## Benefits of v1.0

1. **Security**: Encrypted databases protect sensitive API keys and research data
2. **Multi-User**: Multiple users can work simultaneously without conflicts
3. **Performance**: Cached settings and thread-local sessions improve speed
4. **Reliability**: Thread-safe operations prevent race conditions
5. **Privacy**: User data is completely isolated

## Getting Help

- Check the [API Quick Start Guide](api-quickstart.md)
- See [examples/api_usage](../examples/api_usage/) for updated examples
- Join our [Discord](https://discord.gg/ttcqQeFcJ3) for migration support
- Report issues on [GitHub](https://github.com/LearningCircuit/local-deep-research/issues)