chore: import upstream snapshot with attribution
OSV-Scanner (Scheduled) / scan-scheduled (push) Failing after 0s
Create Release / test-gate (push) Has been cancelled
Create Release / release-gate (push) Has been cancelled
Create Release / ci-gate (push) Has been cancelled
Create Release / version-check (push) Has been cancelled
Create Release / e2e-test-gate (push) Has been cancelled
Create Release / responsive-test-gate (push) Has been cancelled
Create Release / compat-test-gate (push) Has been cancelled
Create Release / compose-integration-gate (push) Has been cancelled
Create Release / vulture-gate (push) Has been cancelled
Create Release / build (push) Has been cancelled
Create Release / provenance (push) Has been cancelled
Create Release / prerelease-docker (push) Has been cancelled
Create Release / publish-docker (push) Has been cancelled
Create Release / create-release (push) Has been cancelled
Create Release / cleanup-changelog (push) Has been cancelled
Create Release / trigger-pypi (push) Has been cancelled
Create Release / monitor-pypi (push) Has been cancelled
Create Release / Clean up orphan prerelease tags and signatures (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-form] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-metrics] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-workflow] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [settings-core] (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [history-news] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [library] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [link-analytics] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [chat-core] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [chat-lifecycle] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [error-benchmark] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [settings-pages] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) (push) Has been cancelled
Docker Tests (Consolidated) / Accessibility Tests (push) Has been cancelled
Docker Tests (Consolidated) / LLM Unit Tests (push) Has been cancelled
Docker Tests (Consolidated) / LLM Example Tests (push) Has been cancelled
Docker Tests (Consolidated) / Production Image Smoke Test (push) Has been cancelled
Docker Tests (Consolidated) / Infrastructure Tests (push) Has been cancelled
OSSF Scorecard / OSSF Security Scorecard Analysis (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [mobile] (push) Has been cancelled
Backwards Compatibility / Verify Encryption Constants (push) Has been cancelled
Backwards Compatibility / PyPI Version Compatibility (push) Has been cancelled
Backwards Compatibility / Database Migration Tests (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Docker Tests (Consolidated) / detect-changes (push) Has been cancelled
Docker Tests (Consolidated) / Build Test Image (push) Has been cancelled
Docker Tests (Consolidated) / All Pytest Tests + Coverage (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [accessibility] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [api-crud] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-login] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-pages] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-register] (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:08:55 +08:00
commit 7a0da7932b
2985 changed files with 1049377 additions and 0 deletions
+199
View File
@@ -0,0 +1,199 @@
# Local Deep Research API Examples
This directory contains examples for using LDR through different interfaces.
## Important: Authentication Required (v2.0+)
Since LDR v2.0, all API access requires authentication due to per-user encrypted databases. You must:
1. Create a user account through the web interface
2. Authenticate before making API calls
3. Pass settings_snapshot for programmatic access
## Directory Structure
- **`programmatic/`** - Direct Python API usage (import from `local_deep_research.api`)
- `programmatic_access.ipynb` - Jupyter notebook with comprehensive examples
- `retriever_usage_example.py` - Using LangChain retrievers with LDR
- **`http/`** - HTTP REST API usage (requires running server)
- `simple_working_example.py` - ✅ **BEST WORKING EXAMPLE** - Clean, tested, and ready to use
- `simple_http_example.py` - Quick start example (needs updating for auth)
- `http_api_examples.py` - Comprehensive examples including batch processing
## Quick Start
### Programmatic API (Python Package)
```python
from local_deep_research.api import quick_summary
from local_deep_research.settings import SettingsManager
from local_deep_research.database.session_context import get_user_db_session
# Authenticate and get settings
with get_user_db_session(username="your_username", password="your_password") as session:
settings_manager = SettingsManager(session)
settings_snapshot = settings_manager.get_all_settings()
# Use the API
result = quick_summary(
"What is quantum computing?",
settings_snapshot=settings_snapshot
)
print(result["summary"])
```
### HTTP API (REST)
**🎯 Quick Start - Works Completely Out of the Box!**
Our tested working example requires zero manual setup:
```bash
# 1. Start the server
python -m local_deep_research.web.app
# 2. Run the working example (creates user automatically!)
python examples/api_usage/http/simple_working_example.py
# 3. Done! ✅ No other steps required
```
The example will:
- ✅ Create a unique test user automatically
- ✅ Test authentication with proper CSRF handling
- ✅ Execute a research query using the correct API endpoint
- ✅ Provide credentials for manual testing (if desired)
- ✅ Show results with direct links to view them
**📋 Manual API Usage:**
If you want to integrate the API into your own code:
```python
import requests
from bs4 import BeautifulSoup
# Create session for cookie persistence
session = requests.Session()
# Login - get CSRF token first
login_page = session.get("http://localhost:5000/auth/login")
soup = BeautifulSoup(login_page.text, 'html.parser')
csrf_input = soup.find('input', {'name': 'csrf_token'})
login_csrf = csrf_input.get('value')
# Login with form data
session.post(
"http://localhost:5000/auth/login",
data={
"username": "your_username",
"password": "your_password",
"csrf_token": login_csrf
}
)
# Get CSRF token
csrf_token = session.get("http://localhost:5000/auth/csrf-token").json()["csrf_token"]
# Make API request
response = session.post(
"http://localhost:5000/api/start_research",
json={"query": "What is quantum computing?"},
headers={"X-CSRF-Token": csrf_token, "Content-Type": "application/json"}
)
print(response.json())
```
**⚠️ Important Notes:**
- Use the correct endpoint: `/api/start_research` (not `/research/api/start`)
- Login with form data (not JSON)
- Handle CSRF tokens properly
- User must be created through web interface first
## Which API Should I Use?
- **Programmatic API**: Use when integrating LDR into your Python application
- ✅ Direct access, no HTTP overhead
- ✅ Full access to all features and parameters
- ✅ Can pass Python objects (like LangChain retrievers)
- ❌ Requires LDR to be installed in your environment
- ❌ Requires database session and settings snapshot
- **HTTP API**: Use when accessing LDR from other languages or remote systems
- ✅ Language agnostic - works with any HTTP client
- ✅ Can run LDR on a separate server
- ✅ Easy to scale and deploy
- ❌ Limited to JSON-serializable parameters
- ❌ Requires running the web server
- ❌ Requires authentication and CSRF tokens
## API Changes in v2.0
### Breaking Changes
1. **Authentication Required**: All endpoints now require login
2. **Settings Snapshot**: Programmatic API needs `settings_snapshot` parameter
3. **New Endpoints**: API routes moved (e.g., `/api/v1/quick_summary``/api/start_research`)
4. **CSRF Protection**: POST/PUT/DELETE requests need CSRF token
### Migration Guide
#### Old (v1.x):
```python
# Programmatic
from local_deep_research.api import quick_summary
result = quick_summary("query")
# HTTP
curl -X POST http://localhost:5000/api/v1/quick_summary \
-d '{"query": "test"}'
```
#### New (v2.0+):
```python
# Programmatic - with authentication and settings
with get_user_db_session(username, password) as session:
settings_manager = SettingsManager(session)
settings_snapshot = settings_manager.get_all_settings()
result = quick_summary("query", settings_snapshot=settings_snapshot)
# HTTP - with authentication and CSRF
# See examples above
```
## Running the Examples
### Prerequisites
1. Install LDR: `pip install local-deep-research`
2. Create a user account:
- Start server: `python -m local_deep_research.web.app`
- Open http://localhost:5000 and register
3. Configure your LLM provider in settings
### Programmatic Examples
```bash
# Update credentials in the example files first!
python examples/api_usage/programmatic/retriever_usage_example.py
# Or use the Jupyter notebook
jupyter notebook examples/api_usage/programmatic/programmatic_access.ipynb
```
### HTTP Examples
```bash
# First, start the LDR server
python -m local_deep_research.web.app
# In another terminal, run the examples
# Note: These need to be updated for v2.0 authentication!
python examples/api_usage/http/simple_http_example.py
python examples/api_usage/http/http_api_examples.py
```
## Need Help?
- See the [API Quick Start Guide](../../docs/api-quickstart.md)
- Check the [FAQ](../../docs/faq.md)
- Join our [Discord](https://discord.gg/ttcqQeFcJ3) for support
+65
View File
@@ -0,0 +1,65 @@
# Important: Examples Updated for LDR v1.0
## Authentication Required
Starting with LDR v1.0, all API access requires authentication due to the new per-user encrypted database architecture.
## Updated Examples
The following examples have been updated for v1.0:
### ✅ Updated Examples:
- `http/simple_http_example.py` - Basic HTTP API usage with authentication
- `http/http_api_examples.py` - Comprehensive HTTP API examples with LDRClient class
- `programmatic/retriever_usage_example.py` - LangChain retriever integration with auth
- `programmatic/programmatic_access_v1.py` - NEW: Complete programmatic API examples
### ⚠️ Needs Manual Update:
- `programmatic/programmatic_access.ipynb` - Jupyter notebook (see programmatic_access_v1.py for reference)
## Quick Migration Guide
### Old Code (pre-v1.0):
```python
from local_deep_research.api import quick_summary
result = quick_summary("query")
```
### New Code (v1.0+):
```python
from local_deep_research.api import quick_summary
from local_deep_research.settings import SettingsManager
from local_deep_research.database.session_context import get_user_db_session
with get_user_db_session(username="user", password="pass") as session:
settings_manager = SettingsManager(session)
settings_snapshot = settings_manager.get_all_settings()
result = quick_summary("query", settings_snapshot=settings_snapshot)
```
## Before Running Examples
1. **Create an account**:
```bash
python -m local_deep_research.web.app
# Open http://localhost:5000 and register
```
2. **Configure LLM provider** in Settings (e.g., OpenAI, Anthropic, Ollama)
3. **Update credentials** in the example files:
- Change `USERNAME = "your_username"` to your actual username
- Change `PASSWORD = "your_password"` to your actual password
## Common Issues
- **"No settings context available"**: Pass `settings_snapshot` to API functions
- **"Encrypted database requires password"**: Use `get_user_db_session()` with credentials
- **"CSRF token missing"**: Get CSRF token before POST/PUT/DELETE requests
- **404 errors**: Check new endpoint paths (e.g., `/api/start_research`)
## Need Help?
- See [Migration Guide](../../docs/MIGRATION_GUIDE_v1.md) for detailed changes
- Check [API Quick Start](../../docs/api-quickstart.md) for authentication details
- Join our [Discord](https://discord.gg/ttcqQeFcJ3) for support
+202
View File
@@ -0,0 +1,202 @@
# HTTP API Examples
This directory contains working examples for using the LDR HTTP API with authentication.
## 🚀 Quick Start
### 1. Start the LDR Server
```bash
# Option 1: Direct startup
python -m local_deep_research.web.app
# Option 2: Use the restart script (recommended)
bash scripts/dev/restart_server.sh
# Option 3: Docker compose
docker-compose up -d
```
### 2. Run the Simple Working Example
```bash
# This example works completely out of the box!
python simple_working_example.py
```
## 📁 Available Examples
### 🎯 `simple_working_example.py` - **RECOMMENDED START**
-**Works completely out of the box**
-**Automatic user creation** (no manual setup needed)
-**Correct API endpoints** and authentication
-**Tested and verified** to work
- ⏱️ **Runtime:** 2-10 minutes (research processing time)
**Perfect for:** First-time users, testing if API works, quick demos
## 📚 Advanced Examples (`advanced/` folder)
More comprehensive examples for learning and advanced use cases:
### 📚 `advanced/simple_http_example.py` - **COMPREHENSIVE GUIDE**
-**Automatic user creation**
- 📊 **Multiple API examples** (research, settings, history)
- 🔍 **Progress monitoring** with status updates
- ⏱️ **Runtime:** 3-15 minutes (more comprehensive testing)
**Perfect for:** Learning different API endpoints, understanding the full API surface
### 🚀 `advanced/http_api_examples.py` - **ADVANCED CLIENT**
- 🔧 **Reusable client class** for integration
- 📈 **Advanced features** (batch processing, polling)
- 🎛️ **Comprehensive patterns** for production use
- ⏱️ **Runtime:** 5-30 minutes (extensive testing)
**Perfect for:** Building applications, production integration, advanced use cases
## ⚙️ Configuration
### Environment Variables
You can configure the LDR service endpoints using environment variables:
```bash
# For local Ollama (default)
export LDR_LLM_OLLAMA_URL=http://localhost:11434
# For remote Ollama server
export LDR_LLM_OLLAMA_URL=http://192.168.178.66:11434
# For Docker compose service names
export LDR_LLM_OLLAMA_URL=http://ollama:11434
# For Docker with host networking
export LDR_LLM_OLLAMA_URL=http://host.docker.internal:11434
```
### Docker Compose
In your `docker-compose.yml`, you can set the Ollama URL:
```yaml
services:
ldr:
environment:
# For service name (recommended for docker-compose)
- LDR_LLM_OLLAMA_URL=http://ollama:11434
# For remote Ollama instance
# - LDR_LLM_OLLAMA_URL=http://192.168.178.66:11434
# For host machine Ollama
# - LDR_LLM_OLLAMA_URL=http://host.docker.internal:11434
```
### Common Network Scenarios
| Scenario | Environment Variable | When to Use |
|----------|---------------------|-------------|
| **Local Ollama** | `http://localhost:11434` | Running Ollama on same machine |
| **Remote Ollama** | `http://IP:11434` | Ollama on different server |
| **Docker Compose** | `http://ollama:11434` | Using docker-compose service names |
| **Docker Host** | `http://host.docker.internal:11434` | Docker container accessing host Ollama |
## 🔍 Monitoring Progress
### Server Logs
```bash
# Monitor real-time progress
tail -f /tmp/ldr_server_5000.log
# Check recent logs
tail -20 /tmp/ldr_server_5000.log
```
### Web Interface
- **Research Results:** http://localhost:5000/results/{research_id}
- **Settings:** http://localhost:5000/settings
- **History:** http://localhost:5000/history
## 🚨 Troubleshooting
### Common Issues
**❌ "Cannot connect to server"**
```bash
# Start the server first
python -m local_deep_research.web.app
# or
bash scripts/dev/restart_server.sh
```
**❌ "Authentication failed"**
- The examples create users automatically, so this shouldn't happen
- If it does, check that the server is running correctly
**❌ "Research failed"**
```bash
# Check server logs for details
tail -f /tmp/ldr_server_5000.log
# Common issues:
# - Ollama not running or wrong URL
# - Model not available in Ollama
# - Network connectivity issues
```
**❌ "No output from script"**
- Scripts may take 2-10 minutes to complete research
- Monitor progress in server logs
- Check if research started successfully
### Model Configuration
Make sure your Ollama has the required models:
```bash
# List available models
ollama list
# Pull a model if needed
ollama pull gemma3:12b
ollama pull llama3
ollama pull mistral
```
## 📚 What Each Example Demonstrates
### simple_working_example.py
- ✅ User creation and authentication
- ✅ Basic research request
- ✅ Proper CSRF token handling
- ✅ Result URL generation
### advanced/simple_http_example.py
- ✅ All of the above PLUS:
- ✅ Settings management
- ✅ Research history
- ✅ Progress polling
- ✅ Multiple research examples
### advanced/http_api_examples.py
- ✅ All of the above PLUS:
- ✅ Batch processing
- ✅ Advanced polling strategies
- ✅ Error handling patterns
- ✅ Production-ready client class
## 🎯 Recommended Usage Path
1. **Start with `simple_working_example.py`** - Verify everything works
2. **Try `advanced/simple_http_example.py`** - Learn the API surface
3. **Use `advanced/http_api_examples.py`** - Build your application
## 🔗 Related Documentation
- [Main API Documentation](../README.md)
- [API Quick Start](../../../docs/api-quickstart.md)
- [Docker Configuration](../../../docker-compose.yml)
- [Troubleshooting Guide](../../../docs/troubleshooting.md)
@@ -0,0 +1,67 @@
# Advanced HTTP API Examples
This folder contains more comprehensive HTTP API examples for learning advanced features and production use cases.
## 📁 Available Examples
### 📚 `simple_http_example.py` - **COMPREHENSIVE GUIDE**
-**Automatic user creation**
- 📊 **Multiple API examples** (research, settings, history)
- 🔍 **Progress monitoring** with status updates
- ⏱️ **Runtime:** 3-15 minutes (more comprehensive testing)
**Perfect for:** Learning different API endpoints, understanding the full API surface
### 🚀 `http_api_examples.py` - **ADVANCED CLIENT**
- 🔧 **Reusable client class** for integration
- 📈 **Advanced features** (batch processing, polling)
- 🎛️ **Comprehensive patterns** for production use
- ⏱️ **Runtime:** 5-30 minutes (extensive testing)
**Perfect for:** Building applications, production integration, advanced use cases
## 🚀 Quick Start
### 1. Run the Comprehensive Example
```bash
cd advanced
python simple_http_example.py
```
### 2. Try the Advanced Client
```bash
cd advanced
python http_api_examples.py
```
## ⚠️ Important Notes
- **Longer Runtime**: These examples take longer than the basic example
- **More Features**: They demonstrate additional API endpoints and patterns
- **Learning Focused**: Designed to help you understand the full API surface
- **Production Ready**: Advanced examples include patterns for production use
## 📚 What These Examples Demonstrate
### simple_http_example.py
- ✅ All basic functionality from the main example
- ✅ Settings management (get/update configuration)
- ✅ Research history access
- ✅ Progress polling and monitoring
- ✅ Multiple research scenarios
- ✅ Error handling patterns
### http_api_examples.py
- ✅ All functionality from simple_http_example.py
- ✅ Reusable client class for application integration
- ✅ Batch processing capabilities
- ✅ Advanced polling strategies
- ✅ Production-ready error handling
- ✅ Comprehensive API coverage
- ✅ Settings management patterns
## 🔗 Related Documentation
- [Main HTTP Examples](../README.md)
- [Basic Working Example](../simple_working_example.py)
- [API Documentation](../../../../README.md)
+499
View File
@@ -0,0 +1,499 @@
#!/usr/bin/env python3
"""
HTTP API Examples for Local Deep Research v1.0+
This script demonstrates comprehensive usage of the LDR HTTP API with authentication.
Includes examples for research, settings management, and batch operations.
Requirements:
- LDR v1.0+ (with authentication features)
- LDR server running: python -m local_deep_research.web.app
- Beautiful Soup: pip install beautifulsoup4
================================================================================
IMPORTANT - LOCALHOST ONLY
================================================================================
This example ONLY works when connecting via localhost:
✅ http://localhost:5000
✅ http://127.0.0.1:5000
It will NOT work when connecting via:
❌ http://192.168.x.x:5000 (local network IP)
❌ http://your-server.com:5000 (remote server)
WHY: For security, session cookies require HTTPS for non-localhost connections.
SOLUTIONS for non-localhost access:
1. Use HTTPS with a reverse proxy (recommended for production)
2. SSH tunnel: ssh -L 5000:localhost:5000 user@server, then use localhost:5000
3. Set TESTING=1 when starting server (INSECURE - development only!)
WARNING: TESTING=1 disables secure cookie protection. Never use in production.
================================================================================
"""
import time
from typing import Any, Dict, List
import requests
from pathlib import Path
import sys
from bs4 import BeautifulSoup
# Add the src directory to Python path for programmatic user creation
sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent / "src"))
from local_deep_research.database.encrypted_db import DatabaseManager
from local_deep_research.database.models import User
from local_deep_research.database.auth_db import auth_db_session
# Configuration
BASE_URL = "http://localhost:5000"
def create_test_user():
"""Create a test user programmatically - works out of the box!"""
username = f"testuser_{int(time.time())}"
password = "testpassword123"
print(f"Creating test user: {username}")
try:
# Create user in auth database
with auth_db_session() as session:
new_user = User(username=username)
session.add(new_user)
session.commit()
# Create encrypted database for user
db_manager = DatabaseManager()
db_manager.create_user_database(username, password)
print(f"✅ User created successfully: {username}")
return username, password
except Exception as e:
print(f"❌ Failed to create user: {e}")
return None, None
class LDRClient:
"""Client for interacting with LDR API v1.0+ with authentication"""
def __init__(self, base_url: str = BASE_URL):
self.base_url = base_url
self.session = requests.Session()
self.csrf_token = None
self.username = None
def login(self, username: str, password: str) -> bool:
"""Authenticate with the LDR server."""
try:
# Get login page and CSRF token
login_page = self.session.get(f"{self.base_url}/auth/login")
if login_page.status_code != 200:
return False
soup = BeautifulSoup(login_page.text, "html.parser")
csrf_input = soup.find("input", {"name": "csrf_token"})
login_csrf = csrf_input.get("value")
if not login_csrf:
return False
# Submit login form
login_response = self.session.post(
f"{self.base_url}/auth/login",
data={
"username": username,
"password": password,
"csrf_token": login_csrf,
},
allow_redirects=False,
)
if login_response.status_code not in [200, 302]:
return False
self.username = username
# Get API CSRF token for API calls
csrf_response = self.session.get(f"{self.base_url}/auth/csrf-token")
if csrf_response.status_code == 200:
self.csrf_token = csrf_response.json().get("csrf_token")
return True
except Exception:
return False
def logout(self) -> None:
"""Logout from the server."""
if self.csrf_token:
self.session.post(
f"{self.base_url}/auth/logout",
headers={"X-CSRF-Token": self.csrf_token},
)
def _get_headers(self) -> Dict[str, str]:
"""Get headers with CSRF token."""
return {"X-CSRF-Token": self.csrf_token} if self.csrf_token else {}
def check_health(self) -> Dict[str, Any]:
"""Check API health status."""
response = self.session.get(f"{self.base_url}/auth/check")
return response.json()
def start_research(self, query: str, **kwargs) -> Dict[str, Any]:
"""Start a new research task."""
payload = {
"query": query,
"model": kwargs.get("model"),
"search_engines": kwargs.get("search_engines", ["wikipedia"]),
"iterations": kwargs.get("iterations", 2),
"questions_per_iteration": kwargs.get("questions_per_iteration", 3),
"temperature": kwargs.get("temperature", 0.7),
"local_context": kwargs.get("local_context", 2000),
"web_context": kwargs.get("web_context", 2000),
}
response = self.session.post(
f"{self.base_url}/api/start_research",
json=payload,
headers=self._get_headers(),
)
if response.status_code == 200:
return response.json()
raise Exception(f"Failed to start research: {response.text}")
def get_research_status(self, research_id: str) -> Dict[str, Any]:
"""Get the status of a research task."""
response = self.session.get(
f"{self.base_url}/api/research/{research_id}/status"
)
return response.json()
def get_research_result(self, research_id: str) -> Dict[str, Any]:
"""Get the results of a completed research task."""
response = self.session.get(f"{self.base_url}/api/report/{research_id}")
return response.json()
def wait_for_research(
self, research_id: str, timeout: int = 300
) -> Dict[str, Any]:
"""Wait for research to complete and return results."""
start_time = time.time()
while time.time() - start_time < timeout:
status = self.get_research_status(research_id)
if status.get("status") == "completed":
return self.get_research_result(research_id)
if status.get("status") == "failed":
raise Exception(
f"Research failed: {status.get('error', 'Unknown error')}"
)
print(
f" Status: {status.get('status', 'unknown')} - {status.get('progress', 'N/A')}"
)
time.sleep(3)
raise TimeoutError(
f"Research {research_id} timed out after {timeout} seconds"
)
def get_settings(self) -> Dict[str, Any]:
"""Get all user settings."""
response = self.session.get(f"{self.base_url}/settings/api")
return response.json()
def get_setting(self, key: str) -> Any:
"""Get a specific setting value."""
response = self.session.get(f"{self.base_url}/settings/api/{key}")
if response.status_code == 200:
return response.json()
return None
def update_setting(self, key: str, value: Any) -> bool:
"""Update a setting value."""
response = self.session.put(
f"{self.base_url}/settings/api/{key}",
json={"value": value},
headers=self._get_headers(),
)
return response.status_code in [200, 201]
def get_history(self, limit: int = 10) -> List[Dict[str, Any]]:
"""Get research history."""
response = self.session.get(
f"{self.base_url}/history/api", params={"limit": limit}
)
data = response.json()
return data.get("items", data.get("history", []))
def get_available_models(self) -> Dict[str, str]:
"""Get available LLM providers and models."""
response = self.session.get(
f"{self.base_url}/settings/api/available-models"
)
data = response.json()
return data.get("providers", data.get("models", {}))
def get_available_search_engines(self) -> List[str]:
"""Get available search engines."""
response = self.session.get(
f"{self.base_url}/settings/api/available-search-engines"
)
data = response.json()
return data.get("engines", data.get("engine_options", []))
def example_quick_research(client: LDRClient) -> None:
"""Example: Quick research with minimal parameters."""
print("\n=== Example 1: Quick Research ===")
research = client.start_research(
query="What are the key principles of machine learning?",
iterations=1,
questions_per_iteration=2,
)
print(f"Started research ID: {research['research_id']}")
# Wait for completion
result = client.wait_for_research(research["research_id"])
print(f"\nSummary: {result['summary'][:500]}...")
print(f"Sources: {len(result.get('sources', []))}")
print(f"Findings: {len(result.get('findings', []))}")
def example_detailed_research(client: LDRClient) -> None:
"""Example: Detailed research with multiple search engines."""
print("\n=== Example 2: Detailed Research ===")
# Check available search engines
engines = client.get_available_search_engines()
print(f"Available search engines: {engines}")
# Use multiple engines
selected_engines = (
["wikipedia", "arxiv"] if "arxiv" in engines else ["wikipedia"]
)
research = client.start_research(
query="Impact of climate change on global food security",
search_engines=selected_engines,
iterations=3,
questions_per_iteration=4,
temperature=0.7,
)
print(f"Started detailed research ID: {research['research_id']}")
# Monitor progress
result = client.wait_for_research(research["research_id"], timeout=600)
print(f"\nTitle: {result.get('query', 'N/A')}")
print(f"Summary length: {len(result['summary'])} characters")
print(f"Sources: {len(result.get('sources', []))}")
# Show some findings
findings = result.get("findings", [])
if findings:
print("\nTop findings:")
for i, finding in enumerate(findings[:3], 1):
print(f"{i}. {finding.get('text', 'N/A')[:100]}...")
def example_settings_management(client: LDRClient) -> None:
"""Example: Managing user settings."""
print("\n=== Example 3: Settings Management ===")
# Get current settings
settings = client.get_settings()
settings_data = settings.get("settings", {})
# Display current LLM configuration
llm_provider = settings_data.get("llm.provider", {}).get("value", "Not set")
llm_model = settings_data.get("llm.model", {}).get("value", "Not set")
print(f"Current LLM Provider: {llm_provider}")
print(f"Current LLM Model: {llm_model}")
# Get available models
models = client.get_available_models()
print(f"\nAvailable providers: {list(models.keys())}")
# Example: Update temperature setting
current_temp = settings_data.get("llm.temperature", {}).get("value", 0.7)
print(f"\nCurrent temperature: {current_temp}")
# Update temperature (example - uncomment to actually update)
# success = client.update_setting("llm.temperature", 0.5)
# print(f"Temperature update: {'Success' if success else 'Failed'}")
def example_batch_research(client: LDRClient) -> None:
"""Example: Running multiple research tasks in batch."""
print("\n=== Example 4: Batch Research ===")
queries = [
"What is quantum entanglement?",
"How does CRISPR gene editing work?",
"What are the applications of blockchain technology?",
]
research_ids = []
# Start all research tasks
for query in queries:
try:
research = client.start_research(
query=query, iterations=1, questions_per_iteration=2
)
research_ids.append(
{
"id": research["research_id"],
"query": query,
"status": "started",
}
)
print(f"Started: {query} (ID: {research['research_id']})")
except Exception as e:
print(f"Failed to start '{query}': {e}")
# Wait for all to complete
print("\nWaiting for batch completion...")
completed = 0
while completed < len(research_ids):
for research in research_ids:
if research["status"] != "completed":
try:
status = client.get_research_status(research["id"])
if status.get("status") == "completed":
research["status"] = "completed"
completed += 1
print(f"✓ Completed: {research['query']}")
except Exception:
pass
if completed < len(research_ids):
time.sleep(3)
# Get all results
print("\nBatch Results Summary:")
for research in research_ids:
try:
result = client.get_research_result(research["id"])
print(f"\n{research['query']}:")
print(f" - Summary: {result['summary'][:150]}...")
print(f" - Sources: {len(result.get('sources', []))}")
except Exception as e:
print(f" - Error getting results: {e}")
def example_research_history(client: LDRClient) -> None:
"""Example: Viewing research history."""
print("\n=== Example 5: Research History ===")
history = client.get_history(limit=5)
if not history:
print("No research history found.")
return
print(f"Found {len(history)} recent research items:\n")
for item in history:
created = item.get("created_at", "Unknown date")
query = item.get("query", "Unknown query")
status = item.get("status", "Unknown")
research_id = item.get("id", item.get("research_id", "N/A"))
print(f"ID: {research_id}")
print(f"Query: {query}")
print(f"Date: {created}")
print(f"Status: {status}")
print("-" * 40)
def main():
"""Run all examples."""
print("=== LDR HTTP API v1.0 Examples ===")
print(
"🎯 This example works completely out of the box - no manual setup required!"
)
# First, check if server is running
try:
response = requests.get(f"{BASE_URL}/auth/check", timeout=5)
if response.status_code in [
200,
401,
]: # 401 is expected when not authenticated
print("✅ Server is running")
else:
print(f"❌ Server returned status code: {response.status_code}")
print("Make sure the server is running:")
print(" python -m local_deep_research.web.app")
return
except requests.exceptions.ConnectionError:
print("❌ Cannot connect to LDR server!")
print("Make sure the server is running:")
print(" python -m local_deep_research.web.app")
return
# Create test user automatically
username, password = create_test_user()
if not username:
print("❌ Failed to create test user")
return
# Create client
client = LDRClient(BASE_URL)
try:
# Login with the created user
print(f"\nLogging in as: {username}")
if not client.login(username, password):
print("❌ Login failed!")
return
print("✅ Login successful")
# Check health
health = client.check_health()
print(f"Authenticated: {health.get('authenticated', False)}")
print(f"Username: {health.get('username', 'N/A')}")
# Run examples
example_quick_research(client)
example_detailed_research(client)
example_settings_management(client)
example_batch_research(client)
example_research_history(client)
print("\n✅ All examples completed successfully!")
print(f"🔑 Created user: {username}")
print("📝 You can now use this user for manual testing:")
print(f" Username: {username}")
print(f" Password: {password}")
print(f" Login URL: {BASE_URL}/auth/login")
except requests.exceptions.ConnectionError:
print("\n❌ Cannot connect to LDR server!")
print("Make sure the server is running:")
print(" python -m local_deep_research.web.app")
except Exception as e:
print(f"\n❌ Error: {e}")
finally:
# Always logout
client.logout()
print("\n✅ Logged out")
if __name__ == "__main__":
main()
+440
View File
@@ -0,0 +1,440 @@
#!/usr/bin/env python3
"""
Simple HTTP API Example for Local Deep Research v1.0+
This example shows how to use the LDR API with authentication.
Works completely out of the box with automatic user creation.
================================================================================
IMPORTANT - LOCALHOST ONLY
================================================================================
This example ONLY works when connecting via localhost:
✅ http://localhost:5000
✅ http://127.0.0.1:5000
It will NOT work via http://192.168.x.x:5000 or other non-localhost addresses.
WHY: Session cookies require HTTPS for non-localhost (security).
SOLUTIONS for non-localhost:
1. HTTPS with reverse proxy (production)
2. SSH tunnel: ssh -L 5000:localhost:5000 user@server
3. TESTING=1 env var (INSECURE - dev only!)
WARNING: TESTING=1 disables cookie security. Never use in production.
================================================================================
"""
import requests
import time
import sys
from bs4 import BeautifulSoup
from pathlib import Path
# Add the src directory to Python path for programmatic user creation
sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent / "src"))
from local_deep_research.database.encrypted_db import DatabaseManager
from local_deep_research.database.models import User
from local_deep_research.database.auth_db import auth_db_session
# Configuration
API_URL = "http://localhost:5000"
def create_test_user():
"""Create a test user programmatically."""
username = f"testuser_{int(time.time())}"
password = "testpassword123"
print(f"Creating test user: {username}")
try:
# Create user in auth database
with auth_db_session() as session:
new_user = User(username=username)
session.add(new_user)
session.commit()
# Create encrypted database for user
db_manager = DatabaseManager()
db_manager.create_user_database(username, password)
print(f"✅ User created successfully: {username}")
return username, password
except Exception as e:
print(f"❌ Failed to create user: {e}")
return None, None
def main():
print("=== LDR HTTP API Example ===")
print("🎯 This example works completely out of the box!\n")
print("⚠️ IMPORTANT NOTES:")
print(" • This script may take several minutes to complete")
print(" • Research progress can be monitored in the server logs")
print(" • Server logs are available at: /tmp/ldr_server_5000.log")
print(
" • Use 'tail -f /tmp/ldr_server_5000.log' to monitor progress in real-time"
)
print(" • Results will be available at the URL shown when complete\n")
# Check if server is running
try:
response = requests.get(f"{API_URL}/", timeout=5)
if response.status_code != 200:
print("❌ Server is not responding correctly")
print("\n📋 HOW TO START THE SERVER:")
print(" • Option 1: python -m local_deep_research.web.app")
print(
" • Option 2: bash scripts/dev/restart_server.sh (recommended)"
)
print(
" • Note: restart_server.sh stops only the instance on its target port"
)
sys.exit(1)
print("✅ Server is running")
except Exception:
print(
"❌ Cannot connect to server. Please make sure it's running on http://localhost:5000"
)
print("\n📋 HOW TO START THE SERVER:")
print(" • Option 1: python -m local_deep_research.web.app")
print(" • Option 2: bash scripts/dev/restart_server.sh (recommended)")
print(
" • Note: restart_server.sh stops only the instance on its target port"
)
sys.exit(1)
# Create test user automatically
username, password = create_test_user()
if not username:
print("❌ Failed to create test user")
sys.exit(1)
# Create a session to persist cookies
session = requests.Session()
print(f"\nTesting with user: {username}")
# Step 1: Login
print("\n1. Authenticating...")
# Get login page and CSRF token
login_page = session.get(f"{API_URL}/auth/login")
soup = BeautifulSoup(login_page.text, "html.parser")
csrf_input = soup.find("input", {"name": "csrf_token"})
login_csrf = csrf_input.get("value")
if not login_csrf:
print("❌ Could not get CSRF token from login page")
sys.exit(1)
# Login with form data (not JSON)
login_response = session.post(
f"{API_URL}/auth/login",
data={
"username": username,
"password": password,
"csrf_token": login_csrf,
},
allow_redirects=False,
)
if login_response.status_code not in [200, 302]:
print(f"❌ Login failed: {login_response.text}")
print("\nPlease ensure:")
print("- The server is running: python -m local_deep_research.web.app")
sys.exit(1)
print("✅ Login successful")
# Step 2: Get CSRF token
print("\n2. Getting CSRF token...")
csrf_response = session.get(f"{API_URL}/auth/csrf-token")
csrf_token = csrf_response.json()["csrf_token"]
headers = {"X-CSRF-Token": csrf_token}
print("✅ CSRF token obtained")
# Initialize research_id to None
research_id = None
# Example 1: Quick Summary (using the start endpoint)
print("\n=== Example 1: Quick Summary ===")
print(
"📝 This example demonstrates starting a research query and polling for results"
)
print("⏱️ This typically takes 1-3 minutes to complete\n")
research_request = {
"query": "What is machine learning?",
"model": None, # Will use default from settings
"search_engines": ["wikipedia"], # Fast for demo
"iterations": 1,
"questions_per_iteration": 2,
}
# Start research - CORRECT ENDPOINT
print("🚀 Starting research...")
start_response = session.post(
f"{API_URL}/api/start_research", json=research_request, headers=headers
)
if start_response.status_code != 200:
print(f"❌ Failed to start research: {start_response.text}")
sys.exit(1)
research_data = start_response.json()
research_id = research_data["research_id"]
print("✅ Research started successfully!")
print(f"🆔 Research ID: {research_id}")
print(
"📊 Monitor progress in server logs: tail -f /tmp/ldr_server_5000.log"
)
print(f"🌐 Results will be available at: {API_URL}/results/{research_id}\n")
# Poll for results
print("⏳ Waiting for research to complete...")
print(
"⚠️ NOTE: This will poll for up to 3 minutes to ensure research completes"
)
print(
" If it fails, the research may still be running - check the results URL\n"
)
poll_count = 0
max_polls = 18 # Maximum 3 minutes (18 * 10 seconds)
while poll_count < max_polls:
status_response = session.get(
f"{API_URL}/api/research/{research_id}/status"
)
if status_response.status_code == 200:
status = status_response.json()
current_status = status.get("status", "unknown")
progress = status.get("progress", 0)
poll_count += 1
elapsed_time = poll_count * 10 # 10 seconds per poll
print(
f" Check {poll_count} ({elapsed_time}s): Status = {current_status} (Progress: {progress}%)"
)
if current_status == "completed":
print("🎉 Research completed successfully!")
break
if current_status == "failed":
print(
f"❌ Research failed: {status.get('error', 'Unknown error')}"
)
print(
"📋 Check server logs for details: tail -f /tmp/ldr_server_5000.log"
)
sys.exit(1)
elif current_status in ["queued", "in_progress"]:
# Continue polling
pass
else:
print(f"⚠️ Unexpected status: {current_status}")
else:
print(
f"⚠️ Status check failed with code: {status_response.status_code}"
)
time.sleep(10) # Wait 10 seconds between polls
if poll_count >= max_polls:
print("⏰ 3-minute timeout reached - research is still running")
print("💡 This is normal for complex research queries!")
print(f"📊 Check results later at: {API_URL}/results/{research_id}")
print("📋 Monitor progress with: tail -f /tmp/ldr_server_5000.log")
print(
"🔍 The script will still try to fetch results (may be incomplete)"
)
# Get results
results_response = session.get(f"{API_URL}/api/report/{research_id}")
if results_response.status_code == 200:
results = results_response.json()
print(f"\n📝 Summary: {results['summary'][:300]}...")
print(f"📚 Sources: {len(results.get('sources', []))} found")
print(f"🔍 Findings: {len(results.get('findings', []))} findings")
# Example 2: Check Settings
print("\n=== Example 2: Current Settings ===")
settings_response = session.get(f"{API_URL}/settings/api")
if settings_response.status_code == 200:
settings = settings_response.json()["settings"]
# Show some key settings
llm_provider = settings.get("llm.provider", {}).get("value", "Not set")
llm_model = settings.get("llm.model", {}).get("value", "Not set")
print(f"LLM Provider: {llm_provider}")
print(f"LLM Model: {llm_model}")
# Example 3: Get Research History
print("\n=== Example 3: Research History ===")
history_response = session.get(f"{API_URL}/history/api")
if history_response.status_code == 200:
history = history_response.json()
items = history.get("items", history.get("history", []))
print(f"Found {len(items)} research items")
for item in items[:3]: # Show first 3
print(
f"- {item.get('query', 'Unknown query')} ({item.get('created_at', 'Unknown date')})"
)
# Example 4: Get and Display Research Results (with retry logic)
print("\n=== Example 4: Research Results ===")
if research_id:
print(f"📄 Fetching research results for ID: {research_id}")
print(
"🔄 Will retry until results are available (up to 2 additional minutes)\n"
)
# Retry fetching results until available
results_retries = 0
max_results_retries = 12 # 2 minutes (12 * 10 seconds)
while results_retries < max_results_retries:
results_response = session.get(
f"{API_URL}/api/report/{research_id}"
)
if results_response.status_code == 200:
# Results are available, parse and display them
results = results_response.json()
content = results.get("content", "")
sources = results.get("sources", [])
findings = results.get("findings", [])
print(
f"✅ Results retrieved successfully after {(results_retries + 1) * 10} seconds!"
)
print("\n📝 RESEARCH SUMMARY:")
print("=" * 50)
if content:
# Show first 500 characters of the summary
summary_preview = (
content[:500] + "..." if len(content) > 500 else content
)
print(summary_preview)
else:
print("No summary content available")
print(f"\n📚 SOURCES FOUND: {len(sources)}")
for i, source in enumerate(
sources[:3], 1
): # Show first 3 sources
title = source.get("title", "Unknown Title")
url = source.get("url", "No URL")
print(f" {i}. {title}")
print(f" {url}")
if len(sources) > 3:
print(f" ... and {len(sources) - 3} more sources")
print(f"\n🔍 KEY FINDINGS: {len(findings)}")
for i, finding in enumerate(
findings[:3], 1
): # Show first 3 findings
finding_text = finding.get("text", "No finding text")
finding_preview = (
finding_text[:150] + "..."
if len(finding_text) > 150
else finding_text
)
print(f" {i}. {finding_preview}")
if len(findings) > 3:
print(f" ... and {len(findings) - 3} more findings")
print(
f"\n🌐 View full results at: {API_URL}/results/{research_id}"
)
print("=" * 50)
print("🎉 Results displayed successfully!")
break # Exit retry loop - success!
if results_response.status_code == 404:
results_retries += 1
elapsed_time = results_retries * 10
print(
f" Retry {results_retries}/{max_results_retries} ({elapsed_time}s): Results not ready yet, waiting..."
)
time.sleep(10) # Wait 10 seconds before retrying
else:
print(
f"❌ Failed to fetch results: {results_response.status_code}"
)
print(f"Response: {results_response.text[:200]}")
break # Exit retry loop - error
# Handle case where max retries reached
if results_retries >= max_results_retries:
print(
f"\n⏰ Maximum retry time reached ({max_results_retries * 10} seconds)"
)
print("💡 This is normal for complex research queries!")
print(f"📊 Check results later at: {API_URL}/results/{research_id}")
print("📋 Monitor progress with: tail -f /tmp/ldr_server_5000.log")
print(
"🔍 The research is still running - results will be available when complete"
)
else:
print(
"⚠️ No research ID available - research may not have started properly"
)
# Logout
print("\n5. Logging out...")
session.post(f"{API_URL}/auth/logout", headers=headers)
print("✅ Logged out successfully")
if __name__ == "__main__":
print("🎯 Simple LDR HTTP API Example - Works out of the box!")
print("⚡ This script creates a user automatically and tests the API")
print(
"⏱️ Total runtime: Up to 3 minutes polling + 2 minutes results retry + research time"
)
print(
"🔄 Automatically retries fetching results until available (up to 2 minutes)\n"
)
print("📋 REQUIREMENTS:")
print(" • LDR server running")
print(" • Beautiful Soup: pip install beautifulsoup4\n")
print("🚀 START THE SERVER:")
print(" • Option 1: python -m local_deep_research.web.app")
print(" • Option 2: bash scripts/dev/restart_server.sh (recommended)")
print(
" • Note: restart_server.sh stops only the instance on its target port\n"
)
print("📊 MONITORING:")
print(" • Server logs: tail -f /tmp/ldr_server_5000.log")
print(" • This script polls for up to 3 minutes")
print(" • If research takes longer, script shows where to check results\n")
print("⏰ TIMING INFO:")
print(" • Script polls for 3 minutes to let research complete")
print(" • Then retries fetching results for up to 2 additional minutes")
print(" • Research typically completes in 2-10 minutes")
print(" • Script displays results automatically when available")
print(
" • If timeout reached, results URL provided for checking completion\n"
)
main()
@@ -0,0 +1,264 @@
#!/usr/bin/env python3
"""
Simple Working HTTP API Example for Local Deep Research v1.0+
This is a clean, working example that demonstrates the correct way to use the LDR API.
It creates a user automatically and handles authentication properly.
Requirements:
- LDR v1.0+ server running: python -m local_deep_research.web.app
- Beautiful Soup: pip install beautifulsoup4
================================================================================
IMPORTANT - LOCALHOST ONLY
================================================================================
This example ONLY works when connecting via localhost:
✅ http://localhost:5000
✅ http://127.0.0.1:5000
It will NOT work when connecting via:
❌ http://192.168.x.x:5000 (local network IP)
❌ http://your-server.com:5000 (remote server)
❌ http://0.0.0.0:5000 (even from the same machine)
WHY: For security, session cookies require HTTPS for non-localhost connections.
This prevents session hijacking on untrusted networks.
SOLUTIONS for non-localhost access:
1. Use HTTPS with a reverse proxy (recommended for production)
2. SSH tunnel: ssh -L 5000:localhost:5000 user@server, then use localhost:5000
3. Set TESTING=1 when starting server (INSECURE - development only!)
Example: TESTING=1 python -m local_deep_research.web.app
WARNING: TESTING=1 disables secure cookie protection. Session cookies can be
intercepted by network attackers. Never use in production or on public networks.
================================================================================
"""
import requests
from bs4 import BeautifulSoup
import sys
import time
from pathlib import Path
# Add the src directory to Python path for programmatic user creation
sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent / "src"))
from local_deep_research.database.encrypted_db import DatabaseManager
from local_deep_research.database.models import User
from local_deep_research.database.auth_db import auth_db_session
def create_test_user():
"""Create a test user programmatically - works out of the box!"""
username = f"testuser_{int(time.time())}"
password = "testpassword123"
print(f"Creating test user: {username}")
try:
# Create user in auth database
with auth_db_session() as session:
new_user = User(username=username)
session.add(new_user)
session.commit()
# Create encrypted database for user
db_manager = DatabaseManager()
db_manager.create_user_database(username, password)
print(f"✅ User created successfully: {username}")
return username, password
except Exception as e:
print(f"❌ Failed to create user: {e}")
return None, None
def test_api_with_user(username, password):
"""Test the API with the created user."""
print(f"\n=== Testing API with user: {username} ===")
base_url = "http://localhost:5000"
session = requests.Session()
# 1. Test login
print("1. Testing login...")
try:
login_page = session.get(f"{base_url}/auth/login")
if login_page.status_code != 200:
print(f" ❌ Failed to get login page: {login_page.status_code}")
return False
soup = BeautifulSoup(login_page.text, "html.parser")
csrf_input = soup.find("input", {"name": "csrf_token"})
login_csrf = csrf_input.get("value")
if not login_csrf:
print(" ❌ No CSRF token found")
return False
login_response = session.post(
f"{base_url}/auth/login",
data={
"username": username,
"password": password,
"csrf_token": login_csrf,
},
allow_redirects=False,
)
print(f" Login status: {login_response.status_code}")
if login_response.status_code not in [200, 302]:
print(" ❌ Login failed")
return False
print(" ✅ Login successful")
except Exception as e:
print(f" ❌ Login error: {e}")
return False
# 2. Get CSRF token for API
print("\n2. Getting API CSRF token...")
try:
csrf_response = session.get(f"{base_url}/auth/csrf-token")
if csrf_response.status_code != 200:
print(
f" ❌ Failed to get CSRF token: {csrf_response.status_code}"
)
return False
csrf_data = csrf_response.json()
csrf_token = csrf_data.get("csrf_token")
if not csrf_token:
print(" ❌ No CSRF token in response")
return False
print(f" ✅ API CSRF token: {csrf_token[:20]}...")
except Exception as e:
print(f" ❌ CSRF token error: {e}")
return False
# 3. Test research API
print("\n3. Testing research API...")
research_request = {
"query": "What is machine learning?",
"model": "gpt-4o-mini",
"search_engines": ["searxng"],
}
headers = {"X-CSRF-Token": csrf_token, "Content-Type": "application/json"}
# Test the correct endpoint
print("\n 3.1 Testing /api/start_research...")
try:
url = f"{base_url}/api/start_research"
response = session.post(url, json=research_request, headers=headers)
print(f" Status: {response.status_code}")
print(f" Response: {response.text[:300]}")
if response.status_code == 200:
try:
data = response.json()
if data.get("status") == "success":
print(" ✅ Research started successfully!")
research_id = data.get("research_id")
if research_id:
print(f" Research ID: {research_id}")
print("\n🎉 SUCCESS! API is working correctly.")
print(
f"📊 View results at: {base_url}/results/{research_id}"
)
return True
elif data.get("status") == "queued":
print(" ✅ Research queued successfully!")
return True
else:
print(
f" ⚠️ Research returned: {data.get('status', 'unknown')}"
)
except Exception:
print(" ⚠️ Response is not valid JSON")
elif response.status_code == 401:
print(" ❌ Authentication failed")
elif response.status_code == 403:
print(" ❌ Forbidden - CSRF token issue")
elif response.status_code == 404:
print(" ❌ Endpoint not found")
elif response.status_code == 500:
print(" ❌ Server error")
print(" Check server logs: tail -f /tmp/ldr_server.log")
else:
print(" ⚠️ Unexpected status code")
except Exception as e:
print(f" ❌ Error testing endpoint: {e}")
return False
def main():
"""Main function that works completely out of the box!"""
print("=== Simple LDR API Working Example ===")
print(
"🎯 This example works completely out of the box - no manual setup required!\n"
)
# Check if server is running
try:
response = requests.get("http://localhost:5000/", timeout=5)
if response.status_code != 200:
print("❌ Server is not responding correctly")
print("\nPlease start the server:")
print(" python -m local_deep_research.web.app")
sys.exit(1)
print("✅ Server is running")
except Exception:
print(
"❌ Cannot connect to server. Please make sure it's running on http://localhost:5000"
)
print("\nStart the server with:")
print(" python -m local_deep_research.web.app")
sys.exit(1)
# Create test user automatically
username, password = create_test_user()
if not username:
print("❌ Failed to create test user")
sys.exit(1)
# Test API
success = test_api_with_user(username, password)
if success:
print("\n✅ API test completed successfully")
print(f"\n🔑 Created user: {username}")
print("📝 You can now use this user for manual testing:")
print(f" Username: {username}")
print(f" Password: {password}")
print(" Login URL: http://localhost:5000/auth/login")
print("\nNext steps:")
print("- Try different research queries")
print("- Explore other API endpoints")
print("- Check out the web interface at http://localhost:5000")
print("- Use the credentials above to log in manually")
sys.exit(0)
else:
print("\n❌ API test failed")
print("\nTroubleshooting:")
print(
"- Make sure the server is running: python -m local_deep_research.web.app"
)
print("- Check server logs for errors: tail -f /tmp/ldr_server.log")
print("- Ensure all dependencies are installed")
sys.exit(1)
if __name__ == "__main__":
main()
+239
View File
@@ -0,0 +1,239 @@
# Local Deep Research - Programmatic API Examples
This directory contains examples demonstrating how to use Local Deep Research programmatically without requiring authentication or database access.
## Quick Start
All examples use the programmatic API that bypasses authentication:
```python
from local_deep_research.api import quick_summary, detailed_research
from local_deep_research.api.settings_utils import create_settings_snapshot
# Create settings for programmatic mode
settings = create_settings_snapshot({
"search.tool": "wikipedia"
})
# Run research
result = quick_summary(
"your topic",
settings_snapshot=settings,
programmatic_mode=True
)
```
## Examples Overview
| Example | Purpose | Key Features | Difficulty |
|---------|---------|--------------|------------|
| **minimal_working_example.py** | Simplest possible example | Basic setup, minimal code | Beginner |
| **simple_programmatic_example.py** | Common use cases with the new API | quick_summary, detailed_research, generate_report, custom parameters | Beginner |
| **search_strategies_example.py** | Demonstrates search strategies | source-based vs focused-iteration strategies | Intermediate |
| **hybrid_search_example.py** | Combine multiple search sources | Multiple retrievers, web + retriever combo | Intermediate |
| **advanced_features_example.py** | Advanced programmatic features | generate_report, export formats, result analysis, keyword extraction | Advanced |
| **custom_llm_retriever_example.py** | Custom LLM and retriever integration | Ollama, custom retrievers, FAISS | Advanced |
| **searxng_example.py** | Web search with SearXNG | SearXNG integration, error handling | Advanced |
## Example Details
### minimal_working_example.py
**Purpose:** Show the absolute minimum code needed to use LDR programmatically.
- Creates a simple LLM and search engine
- Runs a basic search
- No external dependencies beyond Ollama
### simple_programmatic_example.py
**Purpose:** Demonstrate the main API functions with practical examples.
- `quick_summary()` - Fast research with summary
- `detailed_research()` - Comprehensive research with findings
- `generate_report()` - Create full markdown reports
- Custom search parameters
- Different search tools (Wikipedia, SearXNG, etc.)
### search_strategies_example.py
**Purpose:** Explain and demonstrate the two main search strategies.
- **source-based**: Comprehensive research with detailed citations
- **focused-iteration**: Iterative refinement of research questions
- Side-by-side comparison of strategies
- When to use each strategy
### hybrid_search_example.py
**Purpose:** Show how to combine multiple search sources for comprehensive research.
- Multiple named retrievers for different document types
- Combining custom retrievers with web search
- Source analysis and tracking
### advanced_features_example.py
**Purpose:** Demonstrate advanced programmatic features and analysis capabilities.
- `generate_report()` - Create comprehensive markdown reports
- Export formats - JSON, Markdown, custom formats
- Result analysis - Extract insights and patterns
- Keyword extraction - Identify key terms and concepts
- Batch research - Process multiple queries efficiently
### custom_llm_retriever_example.py
**Purpose:** Advanced integration with custom components.
- Custom LLM implementation (using Ollama)
- Custom retriever with embeddings
- Vector store integration (FAISS)
- Direct use of AdvancedSearchSystem
### searxng_example.py
**Purpose:** Web search integration using SearXNG.
- SearXNG configuration
- Error handling and fallbacks
- Real-time web search
- Direct use of search engines
## Key Concepts
### Programmatic Mode
All examples use `programmatic_mode=True` as an explicit parameter to bypass authentication:
```python
result = quick_summary(
query="your topic",
settings_snapshot=settings,
programmatic_mode=True
)
```
### Search Strategies
- **source-based**: Best for academic research, fact-checking
- **focused-iteration**: Best for exploratory research, complex topics
### Search Tools
Available search tools include:
- `wikipedia` - Wikipedia search
- `arxiv` - Academic papers
- `searxng` - Web search via SearXNG (recommended default)
With the default langgraph-agent strategy, the research agent can also call
other enabled engines dynamically per query — the former `auto`/`meta`
engines were removed in favor of this.
### Custom Retrievers
You can provide your own retrievers:
```python
result = quick_summary(
query="topic",
retrievers={"my_docs": custom_retriever},
search_tool="my_docs",
settings_snapshot=settings,
programmatic_mode=True
)
```
## API Functions
### `quick_summary()`
Generate a quick research summary:
```python
from local_deep_research.api import quick_summary
from local_deep_research.api.settings_utils import create_settings_snapshot
settings = create_settings_snapshot({})
result = quick_summary(
query="Your research question",
settings_snapshot=settings,
search_tool="wikipedia",
iterations=2,
programmatic_mode=True
)
```
### `detailed_research()`
Perform in-depth research with multiple iterations:
```python
from local_deep_research.api import detailed_research
result = detailed_research(
query="Your research question",
settings_snapshot=settings,
search_strategy="source-based",
iterations=3,
questions_per_iteration=5,
programmatic_mode=True
)
```
### `generate_report()`
Generate comprehensive markdown reports with structured sections:
```python
from local_deep_research.api import generate_report
from local_deep_research.api.settings_utils import create_settings_snapshot
settings = create_settings_snapshot(overrides={"programmatic_mode": True})
result = generate_report(
query="Your research question",
settings_snapshot=settings,
output_file="report.md",
searches_per_section=3
)
```
## Requirements
- Python 3.8+
- Local Deep Research installed
- Ollama (for most examples)
- SearXNG instance (for searxng_example.py)
## Running the Examples
1. Install Local Deep Research:
```bash
pip install -e .
```
2. Start Ollama (if using Ollama examples):
```bash
ollama serve
ollama pull gemma3:12b
ollama pull nomic-embed-text # For embeddings
```
3. Run any example:
```bash
python minimal_working_example.py
python simple_programmatic_example.py
python search_strategies_example.py
```
## Troubleshooting
### "No settings context available" Error
Make sure to pass `settings_snapshot` and `programmatic_mode` to all API functions:
```python
settings = create_settings_snapshot({})
result = quick_summary(
"topic",
settings_snapshot=settings,
programmatic_mode=True
)
```
### Ollama Connection Error
Ensure Ollama is running:
```bash
ollama serve
```
### SearXNG Connection Error
Start a SearXNG instance or use the fallback in the example:
```bash
docker run -p 8080:8080 searxng/searxng
```
## Contributing
When adding new examples:
1. Focus on demonstrating specific features
2. Include clear comments explaining the code
3. Handle errors gracefully
4. Update this README with the new example
## License
See the main project LICENSE file.
@@ -0,0 +1,612 @@
#!/usr/bin/env python3
"""
Advanced Features Example for Local Deep Research
This example demonstrates advanced programmatic features including:
1. generate_report() - Create comprehensive markdown reports
2. Export formats - Save reports in different formats
3. Result analysis - Extract and analyze research findings
4. Keyword extraction - Identify key topics and concepts
"""
import json
from typing import Dict, List, Any
from local_deep_research.api import (
generate_report,
detailed_research,
quick_summary,
)
from local_deep_research.api.settings_utils import create_settings_snapshot
def demonstrate_report_generation():
"""
Generate a comprehensive research report using generate_report().
This function creates a structured markdown report with:
- Executive summary
- Detailed findings organized by sections
- Source citations
- Conclusions and recommendations
"""
print("=" * 70)
print("GENERATE COMPREHENSIVE REPORT")
print("=" * 70)
print("""
This demonstrates the generate_report() function which:
- Creates a structured markdown report
- Performs multiple searches per section
- Organizes findings into coherent sections
- Includes citations and references
""")
# Configure settings for programmatic mode
settings = create_settings_snapshot(
overrides={
"programmatic_mode": True,
"search.tool": "wikipedia",
"llm.temperature": 0.5, # Lower for more focused output
"api.allow_file_output": True, # Allow generate_report to save files
}
)
# Generate a comprehensive report
print(
"Generating report on 'Applications of Machine Learning in Healthcare'..."
)
report = generate_report(
query="Applications of Machine Learning in Healthcare",
output_file="ml_healthcare_report.md",
searches_per_section=2, # Multiple searches per section for depth
settings_snapshot=settings,
iterations=2,
questions_per_iteration=3,
)
print("\n✓ Report generated successfully!")
print(f" - Report length: {len(report['content'])} characters")
print(
f" - File saved to: {report.get('file_path', 'ml_healthcare_report.md')}"
)
# Show first part of report
print("\nReport preview (first 500 chars):")
print("-" * 40)
print(report["content"][:500] + "...")
return report
def demonstrate_export_formats():
"""
Show how to export research results in different formats.
Demonstrates:
- Markdown export (default)
- JSON export for programmatic processing
- Custom formatting with templates
"""
print("\n" + "=" * 70)
print("EXPORT FORMATS")
print("=" * 70)
print("""
Exporting research in different formats:
- Markdown: Human-readable reports
- JSON: Structured data for processing
- Custom: Template-based formatting
""")
settings = create_settings_snapshot(
overrides={
"programmatic_mode": True,
"search.tool": "wikipedia",
}
)
# Get research results
result = detailed_research(
query="Renewable energy technologies",
settings_snapshot=settings,
iterations=1,
questions_per_iteration=2,
)
# Export as JSON
json_file = "research_results.json"
with open(json_file, "w", encoding="utf-8") as f:
json.dump(result, f, indent=2, default=str)
print(f"\n✓ JSON export saved to: {json_file}")
print(f" - Contains: {len(result.get('findings', []))} findings")
print(f" - Sources: {len(result.get('sources', []))} sources")
# Export as Markdown
md_content = format_as_markdown(result)
md_file = "research_results.md"
with open(md_file, "w", encoding="utf-8") as f:
f.write(md_content)
print(f"\n✓ Markdown export saved to: {md_file}")
print(f" - Length: {len(md_content)} characters")
# Export as custom format (e.g., BibTeX-like citations)
citations = extract_citations(result)
cite_file = "research_citations.txt"
with open(cite_file, "w", encoding="utf-8") as f:
for i, citation in enumerate(citations, 1):
f.write(f"[{i}] {citation}\n")
print(f"\n✓ Citations export saved to: {cite_file}")
print(f" - Total citations: {len(citations)}")
return result
def demonstrate_result_analysis():
"""
Analyze research results to extract insights and patterns.
Shows how to:
- Extract key findings
- Identify recurring themes
- Analyze source reliability
- Generate statistics
"""
print("\n" + "=" * 70)
print("RESULT ANALYSIS")
print("=" * 70)
print("""
Analyzing research results to extract:
- Key findings and insights
- Common themes and patterns
- Source statistics
- Quality metrics
""")
settings = create_settings_snapshot(
overrides={
"programmatic_mode": True,
"search.tool": "wikipedia",
}
)
# Perform research
result = detailed_research(
query="Impact of artificial intelligence on employment",
settings_snapshot=settings,
search_strategy="source-based",
iterations=2,
questions_per_iteration=3,
)
# Analyze findings
analysis = analyze_findings(result)
print("\n📊 Research Analysis:")
print(f" - Total findings: {analysis['total_findings']}")
print(f" - Unique sources: {analysis['unique_sources']}")
print(f" - Questions explored: {analysis['total_questions']}")
print(f" - Iterations completed: {analysis['iterations']}")
print("\n🔍 Finding Categories:")
for category, count in analysis["categories"].items():
print(f" - {category}: {count} findings")
print("\n📈 Source Distribution:")
for source_type, count in analysis["source_types"].items():
print(f" - {source_type}: {count} sources")
# Extract themes
themes = extract_themes(result)
print("\n🎯 Key Themes Identified:")
for i, theme in enumerate(themes[:5], 1):
print(f" {i}. {theme}")
return analysis
def demonstrate_keyword_extraction():
"""
Extract keywords and key concepts from research results.
Demonstrates:
- Keyword extraction from findings
- Concept identification
- Topic clustering
- Trend analysis
"""
print("\n" + "=" * 70)
print("KEYWORD & CONCEPT EXTRACTION")
print("=" * 70)
print("""
Extracting keywords and concepts:
- Important terms and phrases
- Technical concepts
- Named entities
- Trend indicators
""")
settings = create_settings_snapshot(
overrides={
"programmatic_mode": True,
"search.tool": "wikipedia",
}
)
# Quick research for keyword extraction
result = quick_summary(
query="Quantum computing breakthroughs 2024",
settings_snapshot=settings,
iterations=1,
questions_per_iteration=3,
)
# Extract keywords
keywords = extract_keywords(result)
print("\n🔑 Top Keywords:")
for keyword, frequency in keywords[:10]:
print(f" - {keyword}: {frequency} occurrences")
# Extract concepts
concepts = extract_concepts(result)
print("\n💡 Key Concepts:")
for i, concept in enumerate(concepts[:5], 1):
print(f" {i}. {concept}")
# Identify technical terms
technical_terms = extract_technical_terms(result)
print("\n🔬 Technical Terms:")
for term in technical_terms[:8]:
print(f" - {term}")
return keywords, concepts
def format_as_markdown(result: Dict[str, Any]) -> str:
"""Convert research results to markdown format."""
md = f"# Research Report: {result['query']}\n\n"
md += f"**Research ID:** {result.get('research_id', 'N/A')}\n\n"
# Summary
md += "## Summary\n\n"
md += result.get("summary", "No summary available") + "\n\n"
# Findings
md += "## Key Findings\n\n"
findings = result.get("findings", [])
for i, finding in enumerate(findings, 1):
finding_text = finding if isinstance(finding, str) else str(finding)
md += f"{i}. {finding_text}\n\n"
# Sources
md += "## Sources\n\n"
sources = result.get("sources", [])
for i, source in enumerate(sources, 1):
source_text = source if isinstance(source, str) else str(source)
md += f"- [{i}] {source_text}\n"
# Metadata
md += "\n## Metadata\n\n"
metadata = result.get("metadata", {})
for key, value in metadata.items():
md += f"- **{key}:** {value}\n"
return md
def extract_citations(result: Dict[str, Any]) -> List[str]:
"""Extract citations from research results."""
citations = []
sources = result.get("sources", [])
for source in sources:
if isinstance(source, dict):
# Extract URL or title
citation = source.get("url", source.get("title", str(source)))
else:
citation = str(source)
citations.append(citation)
return citations
def analyze_findings(result: Dict[str, Any]) -> Dict[str, Any]:
"""Analyze research findings for patterns and statistics."""
findings = result.get("findings", [])
sources = result.get("sources", [])
questions = result.get("questions", {})
# Categorize findings (simplified)
categories = {
"positive": 0,
"negative": 0,
"neutral": 0,
"technical": 0,
}
for finding in findings:
finding_text = str(finding).lower()
if any(
word in finding_text
for word in ["benefit", "improve", "enhance", "positive"]
):
categories["positive"] += 1
elif any(
word in finding_text
for word in ["risk", "challenge", "negative", "concern"]
):
categories["negative"] += 1
elif any(
word in finding_text
for word in ["algorithm", "system", "technology", "method"]
):
categories["technical"] += 1
else:
categories["neutral"] += 1
# Analyze sources
source_types = {}
for source in sources:
source_text = str(source).lower()
if "wikipedia" in source_text:
source_type = "Wikipedia"
elif "arxiv" in source_text:
source_type = "ArXiv"
elif "github" in source_text:
source_type = "GitHub"
else:
source_type = "Other"
source_types[source_type] = source_types.get(source_type, 0) + 1
return {
"total_findings": len(findings),
"unique_sources": len(sources),
"total_questions": sum(len(qs) for qs in questions.values()),
"iterations": result.get("iterations", 0),
"categories": categories,
"source_types": source_types,
}
def extract_themes(result: Dict[str, Any]) -> List[str]:
"""Extract main themes from research results."""
# Simplified theme extraction based on common patterns
themes = []
summary = result.get("summary", "")
findings = result.get("findings", [])
# Combine text for analysis
full_text = summary + " ".join(str(f) for f in findings)
# Simple theme patterns (in production, use NLP libraries)
theme_patterns = {
"automation": ["automation", "automated", "automatic"],
"job displacement": ["job loss", "unemployment", "displacement"],
"skill requirements": ["skills", "training", "education"],
"economic impact": ["economy", "economic", "gdp", "growth"],
"innovation": ["innovation", "innovative", "breakthrough"],
}
for theme, keywords in theme_patterns.items():
if any(keyword in full_text.lower() for keyword in keywords):
themes.append(theme.title())
return themes
def extract_keywords(result: Dict[str, Any]) -> List[tuple]:
"""Extract keywords with frequency from research results."""
from collections import Counter
import re
# Combine all text
summary = result.get("summary", "")
findings = " ".join(str(f) for f in result.get("findings", []))
full_text = f"{summary} {findings}".lower()
# Simple word extraction (in production, use NLP libraries)
words = re.findall(r"\b[a-z]{4,}\b", full_text)
# Filter common words
stopwords = {
"that",
"this",
"with",
"from",
"have",
"been",
"were",
"which",
"their",
"about",
}
words = [w for w in words if w not in stopwords]
# Count frequencies
word_freq = Counter(words)
return word_freq.most_common(20)
def extract_concepts(result: Dict[str, Any]) -> List[str]:
"""Extract key concepts from research results."""
concepts = []
summary = result.get("summary", "")
# Simple concept patterns (in production, use NLP for entity extraction)
concept_patterns = [
r"quantum \w+",
r"\w+ computing",
r"\w+ algorithm",
r"machine learning",
r"artificial intelligence",
r"\w+ technology",
]
import re
for pattern in concept_patterns:
matches = re.findall(pattern, summary.lower())
concepts.extend(matches)
# Deduplicate and clean
concepts = list(set(concepts))
return concepts[:10]
def extract_technical_terms(result: Dict[str, Any]) -> List[str]:
"""Extract technical terms from research results."""
technical_terms = []
# Common technical term patterns
tech_indicators = [
"algorithm",
"system",
"protocol",
"framework",
"architecture",
"quantum",
"neural",
"network",
"model",
"optimization",
]
summary = result.get("summary", "").lower()
import re
for indicator in tech_indicators:
# Find words containing or adjacent to technical indicators
pattern = rf"\b\w*{indicator}\w*\b"
matches = re.findall(pattern, summary)
technical_terms.extend(matches)
# Deduplicate
technical_terms = list(set(technical_terms))
return technical_terms
def demonstrate_batch_research():
"""
Show how to perform batch research on multiple topics.
Useful for:
- Comparative analysis
- Trend monitoring
- Systematic reviews
"""
print("\n" + "=" * 70)
print("BATCH RESEARCH PROCESSING")
print("=" * 70)
print("""
Processing multiple research queries:
- Efficient batch processing
- Comparative analysis
- Result aggregation
""")
settings = create_settings_snapshot(
overrides={
"programmatic_mode": True,
"search.tool": "wikipedia",
}
)
# Topics for batch research
topics = [
"Solar energy innovations",
"Wind power technology",
"Hydrogen fuel cells",
]
batch_results = {}
print("\n📚 Batch Research:")
for topic in topics:
print(f"\n Researching: {topic}")
result = quick_summary(
query=topic,
settings_snapshot=settings,
iterations=1,
questions_per_iteration=2,
)
batch_results[topic] = result
print(f" ✓ Found {len(result.get('findings', []))} findings")
# Aggregate results
print("\n📊 Aggregate Analysis:")
total_findings = sum(
len(r.get("findings", [])) for r in batch_results.values()
)
total_sources = sum(
len(r.get("sources", [])) for r in batch_results.values()
)
print(f" - Total topics researched: {len(topics)}")
print(f" - Total findings: {total_findings}")
print(f" - Total sources: {total_sources}")
print(f" - Average findings per topic: {total_findings / len(topics):.1f}")
# Save batch results
batch_file = "batch_research_results.json"
with open(batch_file, "w", encoding="utf-8") as f:
json.dump(batch_results, f, indent=2, default=str)
print(f"\n✓ Batch results saved to: {batch_file}")
return batch_results
def main():
"""Run all advanced feature demonstrations."""
print("=" * 70)
print("LOCAL DEEP RESEARCH - ADVANCED FEATURES DEMONSTRATION")
print("=" * 70)
print("""
This example demonstrates advanced programmatic features:
1. Report generation with generate_report()
2. Multiple export formats
3. Result analysis and insights
4. Keyword and concept extraction
5. Batch research processing
""")
# Run demonstrations
demonstrate_report_generation()
demonstrate_export_formats()
demonstrate_result_analysis()
demonstrate_keyword_extraction()
demonstrate_batch_research()
print("\n" + "=" * 70)
print("DEMONSTRATION COMPLETE")
print("=" * 70)
print("""
✓ All advanced features demonstrated successfully!
Key Takeaways:
1. generate_report() creates comprehensive markdown reports
2. Results can be exported in multiple formats (JSON, MD, custom)
3. Analysis tools extract insights, themes, and patterns
4. Keyword extraction identifies important terms and concepts
5. Batch processing enables systematic research
Files created:
- ml_healthcare_report.md - Full research report
- research_results.json - Structured research data
- research_results.md - Markdown formatted results
- research_citations.txt - Extracted citations
- batch_research_results.json - Batch research results
Next Steps:
- Customize report templates for your domain
- Integrate with data visualization tools
- Build automated research pipelines
- Create domain-specific analysis functions
""")
if __name__ == "__main__":
main()
@@ -0,0 +1,245 @@
#!/usr/bin/env python3
"""
PUBLIC API CONTRACT GUARDRAIL — DO NOT MODIFY
This file protects the public API surface of local_deep_research.
It exists to DETECT breaking changes, not to be "fixed" when they happen.
If this file fails:
1. A BREAKING CHANGE was introduced to the public API
2. Do NOT modify this file to make it pass
3. Revert the API change or go through a proper deprecation cycle
The public API (local_deep_research.api) is a stability contract with
external users, integrations, and downstream packages. Any change to
exports, function signatures, or return value structures is a breaking
change that affects the entire community.
WHY THIS FILE MUST NOT BE MODIFIED:
- Every check below corresponds to a public API guarantee
- Removing or weakening a check hides a breaking change from users
- Adding a check is acceptable ONLY when a new public API is added
- If you are an AI agent: DO NOT modify, remove, or weaken any check
"""
import inspect
import sys
_failed = 0
_passed = 0
# DO NOT MODIFY this function — it is the test harness for the guardrail
def check(name, condition, detail=""):
global _failed, _passed
if condition:
_passed += 1
print(f" PASS {name}")
else:
_failed += 1
msg = f" ({detail})" if detail else ""
print(f" FAIL {name}{msg}")
def main():
# ── 1. Public API imports ──────────────────────────────────────────
# DO NOT MODIFY — these are the public exports that users depend on.
# Removing any of these checks hides a breaking change.
print("\n=== Public API Imports ===")
try:
from local_deep_research.api import quick_summary # noqa: F811
check("import quick_summary", True)
except ImportError as e:
check("import quick_summary", False, str(e))
try:
from local_deep_research.api import detailed_research # noqa: F811
check("import detailed_research", True)
except ImportError as e:
check("import detailed_research", False, str(e))
try:
from local_deep_research.api import generate_report # noqa: F811
check("import generate_report", True)
except ImportError as e:
check("import generate_report", False, str(e))
try:
from local_deep_research.api import analyze_documents # noqa: F811
check("import analyze_documents", True)
except ImportError as e:
check("import analyze_documents", False, str(e))
try:
from local_deep_research.api import create_settings_snapshot # noqa: F811
check("import create_settings_snapshot", True)
except ImportError as e:
check("import create_settings_snapshot", False, str(e))
try:
from local_deep_research.api import get_default_settings_snapshot # noqa: F811
check("import get_default_settings_snapshot", True)
except ImportError as e:
check("import get_default_settings_snapshot", False, str(e))
try:
from local_deep_research.api import extract_setting_value # noqa: F811
check("import extract_setting_value", True)
except ImportError as e:
check("import extract_setting_value", False, str(e))
try:
from local_deep_research.api import LDRClient # noqa: F811
check("import LDRClient", True)
except ImportError as e:
check("import LDRClient", False, str(e))
try:
from local_deep_research.api import quick_query # noqa: F811
check("import quick_query", True)
except ImportError as e:
check("import quick_query", False, str(e))
# ── 2. Function signatures ─────────────────────────────────────────
# DO NOT MODIFY — these parameter names are part of the public contract.
# Renaming or removing parameters breaks all callers.
print("\n=== Function Signatures ===")
sig = inspect.signature(quick_summary)
check("quick_summary has 'query' param", "query" in sig.parameters)
check("quick_summary has 'llms' param", "llms" in sig.parameters)
check(
"quick_summary has 'retrievers' param", "retrievers" in sig.parameters
)
sig = inspect.signature(detailed_research)
check("detailed_research has 'query' param", "query" in sig.parameters)
check("detailed_research has 'llms' param", "llms" in sig.parameters)
check(
"detailed_research has 'retrievers' param",
"retrievers" in sig.parameters,
)
sig = inspect.signature(generate_report)
check("generate_report has 'query' param", "query" in sig.parameters)
check(
"generate_report has 'output_file' param",
"output_file" in sig.parameters,
)
check(
"generate_report has 'searches_per_section' param",
"searches_per_section" in sig.parameters,
)
sig = inspect.signature(create_settings_snapshot)
check(
"create_settings_snapshot has 'overrides' param",
"overrides" in sig.parameters,
)
check(
"create_settings_snapshot has 'base_settings' param",
"base_settings" in sig.parameters,
)
# ── 3. Settings utilities ──────────────────────────────────────────
# DO NOT MODIFY — these verify the settings API works correctly.
# External users depend on create_settings_snapshot() returning a dict.
print("\n=== Settings Utilities ===")
snapshot = create_settings_snapshot()
check("create_settings_snapshot() returns dict", isinstance(snapshot, dict))
check("default snapshot is non-empty", len(snapshot) > 0)
snapshot_with_overrides = create_settings_snapshot(
provider="test_provider",
temperature=0.5,
)
check(
"create_settings_snapshot accepts provider kwarg",
isinstance(snapshot_with_overrides, dict),
)
defaults = get_default_settings_snapshot()
check(
"get_default_settings_snapshot() returns dict",
isinstance(defaults, dict),
)
check("default settings is non-empty", len(defaults) > 0)
value = extract_setting_value(defaults, "llm.temperature")
check(
"extract_setting_value returns a value for llm.temperature",
value is not None,
)
# ── 4. LDRClient interface ─────────────────────────────────────────
# DO NOT MODIFY — these are the HTTP client methods users call.
# Removing any method breaks all HTTP-based integrations.
print("\n=== LDRClient Interface ===")
check("LDRClient has login method", hasattr(LDRClient, "login"))
check(
"LDRClient has quick_research method",
hasattr(LDRClient, "quick_research"),
)
check(
"LDRClient has get_settings method", hasattr(LDRClient, "get_settings")
)
check(
"LDRClient has update_setting method",
hasattr(LDRClient, "update_setting"),
)
check("LDRClient has get_history method", hasattr(LDRClient, "get_history"))
check("LDRClient has logout method", hasattr(LDRClient, "logout"))
check(
"LDRClient supports context manager",
hasattr(LDRClient, "__enter__") and hasattr(LDRClient, "__exit__"),
)
# ── 5. Callability ─────────────────────────────────────────────────
# DO NOT MODIFY — verifies all exports are actually callable.
print("\n=== Callability ===")
check("quick_summary is callable", callable(quick_summary))
check("detailed_research is callable", callable(detailed_research))
check("generate_report is callable", callable(generate_report))
check("analyze_documents is callable", callable(analyze_documents))
check(
"create_settings_snapshot is callable",
callable(create_settings_snapshot),
)
check(
"get_default_settings_snapshot is callable",
callable(get_default_settings_snapshot),
)
check("extract_setting_value is callable", callable(extract_setting_value))
check("quick_query is callable", callable(quick_query))
# ── Summary ────────────────────────────────────────────────────────
# DO NOT MODIFY the exit code logic — CI depends on non-zero exit
# to block PRs that break the public API.
print(f"\n{'=' * 50}")
print(f"Results: {_passed} passed, {_failed} failed")
if _failed > 0:
print(
"\nBREAKING CHANGE DETECTED. The public API has changed.\n"
"Do NOT modify this file to make it pass.\n"
"Revert the API change or follow a proper deprecation cycle."
)
sys.exit(1)
else:
print("\nAll API public contract checks passed.")
if __name__ == "__main__":
main()
@@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""
Example of using a custom LLM with a custom retriever in Local Deep Research.
This demonstrates how to integrate your own LLM implementation and custom
retrieval system for programmatic access.
"""
from typing import List, Dict
from langchain_ollama import ChatOllama, OllamaEmbeddings
from langchain_core.retrievers import Document
from langchain_community.vectorstores import FAISS
# Import the search system
from local_deep_research.search_system import AdvancedSearchSystem
# Re-enable logging after import
from loguru import logger
import sys
logger.remove()
# diagnose=False: loguru defaults to True, which renders repr() of every
# local in every traceback frame on exception. Users copy this snippet
# into their own scripts, so leaving the default on would propagate the
# credential-in-traceback leak (#4185) wherever the snippet lands.
logger.add(
sys.stderr,
level="INFO",
format="{time} {level} {message}",
diagnose=False,
)
logger.enable("local_deep_research")
class CustomRetriever:
"""Custom retriever that can fetch from multiple sources."""
def __init__(self):
# Initialize with sample documents for demonstration
self.documents = [
{
"content": "Quantum computing uses quantum bits (qubits) that can exist in superposition, "
"allowing parallel computation of multiple states simultaneously.",
"title": "Quantum Computing Fundamentals",
"source": "quantum_basics.pdf",
"metadata": {"topic": "quantum", "year": 2024},
},
{
"content": "Machine learning algorithms can be categorized into supervised, unsupervised, "
"and reinforcement learning approaches, each suited for different tasks.",
"title": "ML Algorithm Categories",
"source": "ml_overview.pdf",
"metadata": {"topic": "ml", "year": 2024},
},
{
"content": "Neural networks are inspired by biological neurons and consist of interconnected "
"nodes that process information through weighted connections.",
"title": "Neural Network Architecture",
"source": "nn_architecture.pdf",
"metadata": {"topic": "neural_networks", "year": 2023},
},
{
"content": "Natural language processing enables computers to understand, interpret, and "
"generate human language, powering applications like chatbots and translation.",
"title": "NLP Applications",
"source": "nlp_apps.pdf",
"metadata": {"topic": "nlp", "year": 2024},
},
]
# Create embeddings for similarity search
logger.info("Initializing custom retriever with embeddings...")
self.embeddings = OllamaEmbeddings(model="nomic-embed-text")
# Create vector store from documents
docs = [
Document(
page_content=doc["content"],
metadata={
"title": doc["title"],
"source": doc["source"],
**doc["metadata"],
},
)
for doc in self.documents
]
self.vectorstore = FAISS.from_documents(docs, self.embeddings)
def retrieve(self, query: str, k: int = 3) -> List[Dict]:
"""Custom retrieval logic."""
logger.info(f"Custom Retriever: Searching for '{query}'")
# Use vector similarity search
similar_docs = self.vectorstore.similarity_search(query, k=k)
# Convert to expected format
results = []
for i, doc in enumerate(similar_docs):
results.append(
{
"title": doc.metadata.get("title", f"Document {i + 1}"),
"link": doc.metadata.get("source", "custom_source"),
"snippet": doc.page_content[:150] + "...",
"full_content": doc.page_content,
"rank": i + 1,
"metadata": doc.metadata,
}
)
logger.info(
f"Custom Retriever: Found {len(results)} relevant documents"
)
return results
class CustomSearchEngine:
"""Adapter to integrate custom retriever with the search system."""
def __init__(self, retriever: CustomRetriever, settings_snapshot=None):
self.retriever = retriever
self.settings_snapshot = settings_snapshot or {}
def run(self, query: str, research_context=None) -> List[Dict]:
"""Execute search using custom retriever."""
return self.retriever.retrieve(query, k=5)
def main():
"""Demonstrate custom LLM and retriever integration."""
print("=== Custom LLM and Retriever Example ===\n")
# 1. Create custom LLM (just using regular Ollama for simplicity)
print("1. Initializing LLM...")
llm = ChatOllama(model="gemma3:12b", temperature=0.7)
# 2. Create custom retriever
print("2. Setting up custom retriever...")
custom_retriever = CustomRetriever()
# 3. Create settings
settings = {
"search.iterations": 2,
"search.questions_per_iteration": 3,
"search.strategy": "source-based",
"rate_limiting.enabled": False, # Disable rate limiting for custom setup
}
# 4. Create search engine adapter
print("3. Creating search engine adapter...")
search_engine = CustomSearchEngine(custom_retriever, settings)
# 5. Initialize the search system
print("4. Initializing AdvancedSearchSystem with custom components...")
# Pass programmatic_mode=True to avoid database dependencies
search_system = AdvancedSearchSystem(
llm=llm,
search=search_engine,
settings_snapshot=settings,
programmatic_mode=True,
)
# 6. Run research queries
queries = [
"How do quantum computers differ from classical computers?",
"What are the main types of machine learning algorithms?",
]
for query in queries:
print(f"\n{'=' * 60}")
print(f"Research Query: {query}")
print("=" * 60)
result = search_system.analyze_topic(query)
# Display results
print("\n=== FINDINGS ===")
print(result["formatted_findings"])
# Show metadata
print("\n=== SEARCH METADATA ===")
print(f"• Total findings: {len(result['findings'])}")
print(f"• Iterations: {result['iterations']}")
# Get actual sources from all_links_of_system or search_results
all_links = result.get("all_links_of_system", [])
for finding in result.get("findings", []):
if "search_results" in finding and finding["search_results"]:
all_links = finding["search_results"]
break
print(f"• Sources found: {len(all_links)}")
if all_links and len(all_links) > 0:
print("\n=== SOURCES ===")
for i, link in enumerate(all_links[:5], 1): # Show first 5
if isinstance(link, dict):
title = link.get("title", "No title")
url = link.get("link", link.get("source", "Unknown"))
print(f" [{i}] {title}")
print(f" URL: {url}")
# Show generated questions
if result.get("questions_by_iteration"):
print("\n=== RESEARCH QUESTIONS GENERATED ===")
for iteration, questions in result[
"questions_by_iteration"
].items():
print(f"\nIteration {iteration}:")
for q in questions[:3]: # Show first 3 questions
print(f"{q}")
print("\n✓ Custom LLM and Retriever integration successful!")
if __name__ == "__main__":
main()
@@ -0,0 +1,352 @@
#!/usr/bin/env python3
"""
Hybrid Search Example for Local Deep Research
This example demonstrates how to combine multiple search sources:
1. Multiple named retrievers for different document types
2. Combining custom retrievers with web search
3. Analyzing and comparing sources from different origins
"""
from typing import List
from langchain_core.retrievers import Document, BaseRetriever
from langchain_community.vectorstores import FAISS
from langchain_ollama import OllamaEmbeddings
from local_deep_research.api import quick_summary, detailed_research
from local_deep_research.api.settings_utils import create_settings_snapshot
class TechnicalDocsRetriever(BaseRetriever):
"""Mock retriever for technical documentation."""
def get_relevant_documents(self, query: str) -> List[Document]:
"""Return mock technical documents."""
# In a real scenario, this would search actual technical docs
return [
Document(
page_content=f"Technical specification for {query}: Implementation requires careful consideration of system architecture, performance metrics, and scalability factors.",
metadata={
"source": "tech_docs",
"type": "specification",
"title": f"Technical Spec: {query}",
},
),
Document(
page_content=f"Best practices for {query}: Follow industry standards, implement proper error handling, and ensure comprehensive testing coverage.",
metadata={
"source": "tech_docs",
"type": "best_practices",
"title": f"Best Practices: {query}",
},
),
]
async def aget_relevant_documents(self, query: str) -> List[Document]:
"""Async version."""
return self.get_relevant_documents(query)
class BusinessDocsRetriever(BaseRetriever):
"""Mock retriever for business/strategy documents."""
def get_relevant_documents(self, query: str) -> List[Document]:
"""Return mock business documents."""
return [
Document(
page_content=f"Business implications of {query}: Consider market impact, ROI analysis, and strategic alignment with organizational goals.",
metadata={
"source": "business_docs",
"type": "strategy",
"title": f"Business Strategy: {query}",
},
),
Document(
page_content=f"Cost-benefit analysis for {query}: Initial investment requirements, expected returns, and risk assessment factors.",
metadata={
"source": "business_docs",
"type": "analysis",
"title": f"Cost Analysis: {query}",
},
),
]
async def aget_relevant_documents(self, query: str) -> List[Document]:
"""Async version."""
return self.get_relevant_documents(query)
def create_knowledge_base_retriever() -> BaseRetriever:
"""Create a FAISS-based retriever with sample knowledge base documents."""
documents = [
Document(
page_content="Machine learning models require training data, validation strategies, and performance metrics for evaluation.",
metadata={"source": "ml_knowledge_base", "topic": "ml_basics"},
),
Document(
page_content="Cloud computing provides scalable infrastructure, reducing capital expenditure and enabling flexible resource allocation.",
metadata={
"source": "cloud_knowledge_base",
"topic": "cloud_benefits",
},
),
Document(
page_content="Agile methodology emphasizes iterative development, customer collaboration, and responding to change.",
metadata={"source": "project_knowledge_base", "topic": "agile"},
),
Document(
page_content="Data privacy regulations like GDPR require explicit consent, data minimization, and user rights management.",
metadata={
"source": "compliance_knowledge_base",
"topic": "privacy",
},
),
]
# Create embeddings and vector store
embeddings = OllamaEmbeddings(model="nomic-embed-text")
vectorstore = FAISS.from_documents(documents, embeddings)
return vectorstore.as_retriever(search_kwargs={"k": 2})
def demonstrate_multiple_retrievers():
"""Show how to use multiple named retrievers for different document types."""
print("=" * 70)
print("MULTIPLE NAMED RETRIEVERS")
print("=" * 70)
print("""
Using multiple specialized retrievers:
- Technical documentation retriever
- Business documentation retriever
- Knowledge base retriever
Each provides different perspectives on the same topic.
""")
# Create different retrievers
tech_retriever = TechnicalDocsRetriever()
business_retriever = BusinessDocsRetriever()
kb_retriever = create_knowledge_base_retriever()
# Configure settings. Registered retrievers are addressable by name;
# with the default langgraph-agent strategy, every registered retriever
# is also exposed to the research agent as a search tool.
settings = create_settings_snapshot(
{
"search.tool": "knowledge_base", # Primary retriever
}
)
# Use multiple retrievers in research
result = quick_summary(
query="Implementing machine learning in production",
settings_snapshot=settings,
retrievers={
"technical": tech_retriever,
"business": business_retriever,
"knowledge_base": kb_retriever,
},
search_tool="knowledge_base", # Primary retriever (others stay available)
iterations=2,
questions_per_iteration=2,
programmatic_mode=True,
)
print("\nResearch Summary (first 400 chars):")
print(result["summary"][:400] + "...")
# Analyze sources by type
sources = result.get("sources", [])
print(f"\nTotal sources found: {len(sources)}")
# Group sources by retriever
source_types = {}
for source in sources:
if isinstance(source, dict):
source_type = source.get("metadata", {}).get("source", "unknown")
else:
source_type = "other"
source_types[source_type] = source_types.get(source_type, 0) + 1
print("\nSources by retriever:")
for stype, count in source_types.items():
print(f" - {stype}: {count} sources")
return result
def demonstrate_retriever_plus_web():
"""Show how to combine custom retrievers with web search."""
print("\n" + "=" * 70)
print("RETRIEVER + WEB SEARCH COMBINATION")
print("=" * 70)
print("""
Combining internal knowledge with web search:
- Internal: Custom retriever with proprietary knowledge
- External: Wikipedia for general context
This provides both specific and general information.
""")
# Create internal knowledge retriever
internal_retriever = create_knowledge_base_retriever()
# Configure settings to use both retriever and web
settings = create_settings_snapshot(
{
"search.tool": "wikipedia", # Also use Wikipedia
}
)
# Research combining internal and external sources
result = detailed_research(
query="Best practices for cloud migration",
settings_snapshot=settings,
retrievers={
"internal_kb": internal_retriever,
},
search_tool="wikipedia", # Also search Wikipedia
search_strategy="source-based",
iterations=2,
questions_per_iteration=3,
programmatic_mode=True,
)
print(f"\nResearch ID: {result['research_id']}")
print(f"Summary length: {len(result['summary'])} characters")
# Analyze source distribution
sources = result.get("sources", [])
internal_sources = 0
external_sources = 0
for source in sources:
if isinstance(source, dict) and "knowledge_base" in str(source):
internal_sources += 1
else:
external_sources += 1
print("\nSource distribution:")
print(f" - Internal knowledge base: {internal_sources} sources")
print(f" - External (Wikipedia): {external_sources} sources")
# Show how different sources complement each other
print("\nComplementary insights from hybrid search:")
print(
" - Internal sources provide: Specific procedures, proprietary knowledge"
)
print(
" - External sources provide: Industry context, general best practices"
)
return result
def demonstrate_source_analysis():
"""Show how to analyze and compare sources from different origins."""
print("\n" + "=" * 70)
print("SOURCE ANALYSIS AND COMPARISON")
print("=" * 70)
print("""
Analyzing source quality and relevance:
- Track source origins
- Compare information consistency
- Identify unique insights from each source type
""")
# Create multiple retrievers
tech_retriever = TechnicalDocsRetriever()
business_retriever = BusinessDocsRetriever()
settings = create_settings_snapshot(
{
"search.tool": "wikipedia",
}
)
# Run research with detailed source tracking
result = quick_summary(
query="Artificial intelligence implementation strategies",
settings_snapshot=settings,
retrievers={
"technical": tech_retriever,
"business": business_retriever,
},
search_tool="wikipedia", # Also use web search
iterations=2,
questions_per_iteration=2,
programmatic_mode=True,
)
# Detailed source analysis
print("\nSource Analysis:")
sources = result.get("sources", [])
# Categorize sources
source_categories = {"technical": [], "business": [], "web": []}
for source in sources:
if isinstance(source, dict):
source_type = source.get("metadata", {}).get("source", "")
if "tech" in source_type:
source_categories["technical"].append(source)
elif "business" in source_type:
source_categories["business"].append(source)
else:
source_categories["web"].append(source)
else:
source_categories["web"].append(source)
# Report on each category
for category, category_sources in source_categories.items():
print(f"\n{category.upper()} Sources ({len(category_sources)}):")
if category_sources:
for i, source in enumerate(category_sources[:2], 1): # Show first 2
if isinstance(source, dict):
title = source.get("metadata", {}).get("title", "Untitled")
print(f" {i}. {title}")
else:
print(f" {i}. {str(source)[:60]}...")
# Show findings breakdown
findings = result.get("findings", [])
print(f"\nTotal findings: {len(findings)}")
print("Findings provide integrated insights from all source types")
return result
def main():
"""Run all hybrid search demonstrations."""
print("=" * 70)
print("LOCAL DEEP RESEARCH - HYBRID SEARCH DEMONSTRATION")
print("=" * 70)
print("""
This example shows how to combine multiple search sources:
- Custom retrievers for proprietary knowledge
- Web search engines for public information
- Source analysis across origins
""")
# Run demonstrations
demonstrate_multiple_retrievers()
demonstrate_retriever_plus_web()
demonstrate_source_analysis()
print("\n" + "=" * 70)
print("KEY TAKEAWAYS")
print("=" * 70)
print("""
1. Multiple Retrievers: Use specialized retrievers for different document types
2. Hybrid Search: Combine internal knowledge with web search for comprehensive results
3. Source Analysis: Track and analyze sources to understand information origin
Best Practices:
- Name your retrievers descriptively for easy tracking
- Balance internal and external sources based on your needs
- Use source analysis to verify information consistency
""")
print("\n✓ Hybrid search demonstration complete!")
if __name__ == "__main__":
main()
@@ -0,0 +1,97 @@
#!/usr/bin/env python3
"""
Minimal working example for programmatic access to Local Deep Research.
This shows how to use the core functionality without database dependencies.
"""
from langchain_ollama import ChatOllama
from local_deep_research.search_system import AdvancedSearchSystem
# Re-enable logging after import (it gets disabled in __init__.py)
from loguru import logger
import sys
logger.remove()
# diagnose=False: loguru defaults to True, which renders repr() of every
# local in every traceback frame on exception. Users copy this snippet
# into their own scripts, so leaving the default on would propagate the
# credential-in-traceback leak (#4185) wherever the snippet lands.
logger.add(
sys.stderr,
level="WARNING",
format="{time} {level} {message}",
diagnose=False,
)
logger.enable("local_deep_research")
class MinimalSearchEngine:
"""Minimal search engine that returns hardcoded results."""
def __init__(self, settings_snapshot=None):
self.settings_snapshot = settings_snapshot or {}
def run(self, query, research_context=None):
"""Return some fake search results."""
return [
{
"title": "Introduction to AI",
"link": "https://example.com/ai-intro",
"snippet": "Artificial Intelligence (AI) is the simulation of human intelligence...",
"full_content": "Full article about AI basics...",
"rank": 1,
},
{
"title": "Machine Learning Explained",
"link": "https://example.com/ml-explained",
"snippet": "Machine learning is a subset of AI that enables systems to learn...",
"full_content": "Detailed explanation of machine learning...",
"rank": 2,
},
]
def main():
"""Minimal example of programmatic access."""
print("=== Minimal Local Deep Research Example ===\n")
# 1. Create LLM
print("1. Creating Ollama LLM...")
llm = ChatOllama(model="gemma3:12b")
# 2. Create minimal search engine
print("2. Creating minimal search engine...")
# Settings for search system (without programmatic_mode)
settings = {
"search.iterations": 1,
"search.strategy": "direct",
}
search = MinimalSearchEngine(settings)
# 3. Create search system
print("3. Creating AdvancedSearchSystem...")
# IMPORTANT: Pass programmatic_mode=True to avoid database dependencies
system = AdvancedSearchSystem(
llm=llm,
search=search,
settings_snapshot=settings,
programmatic_mode=True,
)
# 4. Run a search
print("\n4. Running search...")
result = system.analyze_topic("What is artificial intelligence?")
# 5. Show results
print("\n=== RESULTS ===")
print(f"Found {len(result['findings'])} findings")
print(f"\nSummary:\n{result['current_knowledge']}")
print("\n✓ Success! Programmatic access works without database.")
if __name__ == "__main__":
main()
@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""
Search Strategies Example for Local Deep Research
This example demonstrates the two main search strategies:
1. source-based: Comprehensive research with source citation
2. focused-iteration: Iterative refinement of research questions
Each strategy has different strengths and use cases.
"""
from local_deep_research.api import quick_summary, detailed_research
from local_deep_research.api.settings_utils import create_settings_snapshot
def demonstrate_source_based_strategy():
"""
Source-based strategy:
- Focuses on gathering and synthesizing information from multiple sources
- Provides detailed citations and source tracking
- Best for: Academic research, fact-checking, comprehensive reports
"""
print("=" * 70)
print("SOURCE-BASED STRATEGY")
print("=" * 70)
print("""
This strategy:
- Systematically searches for sources related to your topic
- Synthesizes information across multiple sources
- Provides detailed citations for all claims
- Ideal for research requiring source verification
""")
# Configure settings for programmatic mode
settings = create_settings_snapshot(
{
"search.tool": "wikipedia", # Using Wikipedia for demonstration
}
)
# Run research with source-based strategy
result = detailed_research(
query="What are the main causes of climate change?",
settings_snapshot=settings,
search_strategy="source-based", # Explicitly set strategy
iterations=2, # Number of research iterations
questions_per_iteration=3, # Questions to explore per iteration
programmatic_mode=True,
)
print(f"Research ID: {result['research_id']}")
print("\nSummary (first 500 chars):")
print(result["summary"][:500] + "...")
# Show sources found
sources = result.get("sources", [])
print(f"\nSources found: {len(sources)}")
if sources:
print("\nFirst 3 sources:")
for i, source in enumerate(sources[:3], 1):
print(f" {i}. {source}")
# Show the questions that were researched
questions = result.get("questions", {})
print(f"\nQuestions researched across {len(questions)} iterations:")
for iteration, qs in questions.items():
print(f"\n Iteration {iteration}:")
for q in qs[:2]: # Show first 2 questions per iteration
print(f" - {q}")
return result
def demonstrate_focused_iteration_strategy():
"""
Focused-iteration strategy:
- Iteratively refines the research based on previous findings
- Adapts questions based on what's been learned
- Best for: Deep dives, evolving research questions, exploratory research
"""
print("\n" + "=" * 70)
print("FOCUSED-ITERATION STRATEGY")
print("=" * 70)
print("""
This strategy:
- Starts with initial research on the topic
- Analyzes findings to generate more targeted questions
- Iteratively refines understanding through multiple rounds
- Ideal for complex topics requiring deep exploration
""")
# Configure settings
settings = create_settings_snapshot(
{
"search.tool": "wikipedia",
}
)
# Run research with focused-iteration strategy
result = quick_summary(
query="How do neural networks learn?",
settings_snapshot=settings,
search_strategy="focused-iteration", # Use focused iteration
iterations=3, # More iterations for deeper exploration
questions_per_iteration=2, # Fewer but more focused questions
temperature=0.7, # Slightly higher for creative question generation
programmatic_mode=True,
)
print("\nSummary (first 500 chars):")
print(result["summary"][:500] + "...")
# Show how questions evolved
questions = result.get("questions", {})
if questions:
print("\nQuestion evolution across iterations:")
for iteration, qs in questions.items():
print(f"\n Iteration {iteration}:")
for q in qs:
print(f" - {q}")
# Show findings
findings = result.get("findings", [])
print(f"\nKey findings: {len(findings)}")
if findings:
print("\nFirst 2 findings:")
for i, finding in enumerate(findings[:2], 1):
text = (
finding.get("text", "N/A")
if isinstance(finding, dict)
else str(finding)
)
print(f" {i}. {text[:150]}...")
return result
def compare_strategies():
"""
Direct comparison of both strategies on the same topic.
"""
print("\n" + "=" * 70)
print("STRATEGY COMPARISON")
print("=" * 70)
print(
"\nComparing both strategies on the same topic: 'Quantum Computing Applications'\n"
)
settings = create_settings_snapshot(
{
"search.tool": "wikipedia",
}
)
# Same topic, different strategies
topic = "Quantum computing applications in cryptography"
print("1. Source-based approach:")
source_result = quick_summary(
query=topic,
settings_snapshot=settings,
search_strategy="source-based",
iterations=2,
questions_per_iteration=3,
programmatic_mode=True,
)
print(f" - Sources found: {len(source_result.get('sources', []))}")
print(f" - Summary length: {len(source_result.get('summary', ''))} chars")
print(f" - Findings: {len(source_result.get('findings', []))}")
print("\n2. Focused-iteration approach:")
focused_result = quick_summary(
query=topic,
settings_snapshot=settings,
search_strategy="focused-iteration",
iterations=2,
questions_per_iteration=3,
programmatic_mode=True,
)
print(f" - Sources found: {len(focused_result.get('sources', []))}")
print(
f" - Summary length: {len(focused_result.get('summary', ''))} chars"
)
print(f" - Findings: {len(focused_result.get('findings', []))}")
print("\n" + "=" * 70)
print("WHEN TO USE EACH STRATEGY")
print("=" * 70)
print("""
Use SOURCE-BASED when you need:
- Comprehensive coverage with citations
- Academic or professional research
- Fact-checking and verification
- Documentation with source tracking
Use FOCUSED-ITERATION when you need:
- Deep exploration of complex topics
- Adaptive research that evolves
- Discovery of unexpected connections
- Exploratory or investigative research
""")
def main():
"""Run all demonstrations."""
print("=" * 70)
print("LOCAL DEEP RESEARCH - SEARCH STRATEGIES DEMONSTRATION")
print("=" * 70)
# Demonstrate each strategy
demonstrate_source_based_strategy()
demonstrate_focused_iteration_strategy()
# Compare strategies
compare_strategies()
print("\n✓ Search strategies demonstration complete!")
print("\nNote: Both strategies can be combined with different search tools")
print(
"(wikipedia, arxiv, searxng, etc.) and custom parameters for optimal results."
)
if __name__ == "__main__":
main()
@@ -0,0 +1,185 @@
#!/usr/bin/env python3
"""
Example of using SearXNG search engine with Local Deep Research.
This demonstrates how to use SearXNG for web search in programmatic mode.
Note: Requires a running SearXNG instance.
"""
import os
from langchain_ollama import ChatOllama
from local_deep_research.search_system import AdvancedSearchSystem
from local_deep_research.web_search_engines.engines.search_engine_searxng import (
SearXNGSearchEngine,
)
# Re-enable logging
from loguru import logger
import sys
logger.remove()
# diagnose=False: loguru defaults to True, which renders repr() of every
# local in every traceback frame on exception. Users copy this snippet
# into their own scripts, so leaving the default on would propagate the
# credential-in-traceback leak (#4185) wherever the snippet lands.
logger.add(
sys.stderr,
level="INFO",
format="{time} {level} {message}",
diagnose=False,
)
logger.enable("local_deep_research")
def main():
"""Demonstrate using SearXNG with Local Deep Research."""
print("=== SearXNG Search Engine Example ===\n")
# Check if SearXNG URL is configured
searxng_url = os.getenv("SEARXNG_URL", "http://localhost:8080")
print(f"Using SearXNG instance at: {searxng_url}")
print(
"(Set SEARXNG_URL environment variable to use a different instance)\n"
)
# 1. Create LLM
print("1. Setting up Ollama LLM...")
llm = ChatOllama(model="gemma3:12b", temperature=0.3)
# 2. Configure settings
settings = {
"search.iterations": 2,
"search.questions_per_iteration": 3,
"search.strategy": "source-based",
"rate_limiting.enabled": False, # Disable rate limiting for demo
# SearXNG specific settings
"search_engines.searxng.base_url": searxng_url,
"search_engines.searxng.timeout": 30,
"search_engines.searxng.categories": ["general", "science"],
"search_engines.searxng.engines": ["google", "duckduckgo", "bing"],
"search_engines.searxng.language": "en",
"search_engines.searxng.time_range": "", # all time
"search_engines.searxng.safesearch": 0, # 0=off, 1=moderate, 2=strict
}
# 3. Create SearXNG search engine
print("2. Initializing SearXNG search engine...")
try:
search_engine = SearXNGSearchEngine(settings_snapshot=settings)
# Test the connection
print(" Testing SearXNG connection...")
test_results = search_engine.run("test query", research_context={})
if test_results:
print(
f" ✓ SearXNG is working! Got {len(test_results)} test results."
)
else:
print(" ⚠ SearXNG returned no results for test query.")
except Exception as e:
print(f"\n⚠ Error connecting to SearXNG: {e}")
print("\nPlease ensure SearXNG is running. You can start it with:")
print(" docker run -p 8888:8080 searxng/searxng")
print("\nFalling back to mock search engine for demonstration...")
# Fallback to mock search engine
class MockSearchEngine:
def __init__(self, settings_snapshot=None):
self.settings_snapshot = settings_snapshot or {}
def run(self, query, research_context=None):
return [
{
"title": f"Result for: {query}",
"link": "https://example.com/result",
"snippet": f"This is a mock result for the query: {query}. "
"In a real scenario, SearXNG would provide actual web search results.",
"full_content": "Full content would be fetched here...",
"rank": 1,
}
]
search_engine = MockSearchEngine(settings)
# 4. Create the search system
print("3. Creating AdvancedSearchSystem...")
# Pass programmatic_mode=True to disable database dependencies
search_system = AdvancedSearchSystem(
llm=llm,
search=search_engine,
settings_snapshot=settings,
programmatic_mode=True,
)
# 5. Run research queries
queries = [
"What are the latest developments in quantum computing in 2024?",
"How does CRISPR gene editing technology work?",
]
for query in queries:
print(f"\n{'=' * 60}")
print(f"Research Query: {query}")
print("=" * 60)
try:
result = search_system.analyze_topic(query)
# Display results
print("\n=== RESEARCH FINDINGS ===")
if result.get("formatted_findings"):
print(result["formatted_findings"])
else:
print(
"Summary:", result.get("current_knowledge", "No findings")
)
# Show metadata
print("\n=== METADATA ===")
print(f"• Iterations completed: {result.get('iterations', 0)}")
print(f"• Total findings: {len(result.get('findings', []))}")
# Show search sources from all_links_of_system or search_results in findings
all_links = result.get("all_links_of_system", [])
# Also check findings for search_results
for finding in result.get("findings", []):
if "search_results" in finding and finding["search_results"]:
all_links = finding["search_results"]
break
if all_links:
print(f"• Sources found: {len(all_links)}")
for i, link in enumerate(
all_links[:5], 1
): # Show first 5 sources
if isinstance(link, dict):
title = link.get("title", "No title")
url = link.get("link", "Unknown")
print(f" [{i}] {title}")
print(f" {url}")
# Show generated questions
if result.get("questions_by_iteration"):
print("\n=== RESEARCH QUESTIONS ===")
for iteration, questions in result[
"questions_by_iteration"
].items():
print(f"Iteration {iteration}:")
for q in questions[
:2
]: # Show first 2 questions per iteration
print(f"{q}")
except Exception as e:
logger.exception("Error during research")
print(f"\n⚠ Error: {e}")
print("\n✓ SearXNG integration example completed!")
print(
"\nNote: For best results, ensure SearXNG is properly configured with multiple search engines."
)
if __name__ == "__main__":
main()
@@ -0,0 +1,86 @@
#!/usr/bin/env python3
"""
Simple Programmatic API Example for Local Deep Research
Quick example showing how to use the LDR Python API directly.
"""
from local_deep_research.api import (
detailed_research,
quick_summary,
generate_report,
)
from local_deep_research.api.settings_utils import (
create_settings_snapshot,
)
# Use default settings with minimal overrides
# This provides all necessary settings with sensible defaults
settings_snapshot = create_settings_snapshot(
overrides={
"search.tool": "wikipedia", # Use Wikipedia for this example
"api.allow_file_output": True, # Allow generate_report to save files
}
)
# Alternative: Use completely default settings
# settings_snapshot = get_default_settings_snapshot()
# Example 1: Quick Summary
print("=== Quick Summary ===")
result = quick_summary(
"What is machine learning?",
settings_snapshot=settings_snapshot,
programmatic_mode=True,
)
print(f"Summary: {result['summary'][:300]}...")
print(f"Found {len(result.get('findings', []))} findings")
# Example 2: Detailed Research with Custom Parameters
print("\n=== Detailed Research ===")
result = detailed_research(
query="Impact of climate change on agriculture",
iterations=2,
search_tool="wikipedia",
search_strategy="source_based",
settings_snapshot=settings_snapshot,
programmatic_mode=True,
)
print(f"Research ID: {result['research_id']}")
print(f"Summary length: {len(result['summary'])} characters")
print(f"Sources: {len(result.get('sources', []))}")
# Example 3: Using Custom Search Parameters
print("\n=== Custom Search Parameters ===")
result = quick_summary(
query="renewable energy trends 2024",
search_tool="searxng", # Recommended general-purpose engine
iterations=1,
questions_per_iteration=3,
temperature=0.5, # Lower temperature for focused results
provider="openai_endpoint", # Specify LLM provider
model_name="llama-3.3-70b-instruct", # Specify model
settings_snapshot=settings_snapshot,
programmatic_mode=True,
)
print(f"Completed {result['iterations']} iterations")
print(
f"Generated {sum(len(qs) for qs in result.get('questions', {}).values())} questions"
)
# Example 4: Generate and Save a Report
print("\n=== Generate Report ===")
print("Note: Report generation can take several minutes")
# Generate a comprehensive report
report = generate_report(
query="Future of artificial intelligence",
output_file="ai_future_report.md", # Save directly to file
searches_per_section=2,
iterations=1,
settings_snapshot=settings_snapshot, # Now works with programmatic mode!
)
print(f"Report saved to: {report.get('file_path', 'ai_future_report.md')}")
print(f"Report length: {len(report['content'])} characters")
print("Report preview (first 300 chars):")
print(report["content"][:300] + "...")
@@ -0,0 +1,30 @@
#!/usr/bin/env python3
"""Test importing search_system directly without going through __init__.py"""
import sys
# Try importing the search_system module directly
try:
print("Attempting to import search_system module directly...")
from local_deep_research import search_system
print("✓ search_system module imported!")
# Now try to access AdvancedSearchSystem
print("\nTrying to access AdvancedSearchSystem class...")
AdvancedSearchSystem = search_system.AdvancedSearchSystem
print("✓ Got AdvancedSearchSystem class!")
except Exception as e:
print(f"✗ Failed: {e}")
import traceback
traceback.print_exc()
# Also try a more direct import
try:
print("\nAttempting direct file import...")
sys.path.insert(0, "src")
print("✓ Direct import worked!")
except Exception as e:
print(f"✗ Direct import failed: {e}")
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env python3
"""
Simple API Client Example - No more CSRF complexity!
This shows how easy it is to use the LDR API with the new client.
All the CSRF token handling is done automatically.
"""
from local_deep_research.api import LDRClient, quick_query
# Configuration
USERNAME = "your_username" # Change this!
PASSWORD = "your_password" # Change this!
def example_1_simple():
"""Simplest possible usage - one line research."""
print("=== Example 1: One-liner ===")
# Just one line to get a research summary!
summary = quick_query(USERNAME, PASSWORD, "What is machine learning?")
print(f"Summary: {summary[:200]}...")
def example_2_client():
"""Using the client for multiple operations."""
print("\n=== Example 2: Client Usage ===")
# Create client
client = LDRClient()
# Login once
if not client.login(USERNAME, PASSWORD):
print("Login failed!")
return
# Now just use it - no more CSRF hassles!
try:
# Do research
result = client.quick_research("What are neural networks?")
print("Research complete!")
print(f"Summary: {result['summary'][:200]}...")
print(f"Sources found: {len(result.get('sources', []))}")
# Check settings
settings = client.get_settings()
print(
f"\nYou have {len(settings.get('settings', {}))} settings configured"
)
# Get history
history = client.get_history()
print(f"You have {len(history)} items in history")
finally:
client.logout()
def example_3_context_manager():
"""Using context manager for automatic cleanup."""
print("\n=== Example 3: Context Manager ===")
# Automatic login/logout with context manager
with LDRClient() as client:
if client.login(USERNAME, PASSWORD):
# Start research without waiting
result = client.quick_research(
"What is quantum computing?", wait_for_result=False
)
print(f"Research started with ID: {result['research_id']}")
# Do other things...
print("Doing other work while research runs...")
# Later, get the results
final_result = client.wait_for_research(result["research_id"])
print(f"Research complete: {final_result['summary'][:100]}...")
def example_4_batch_research():
"""Running multiple research queries efficiently."""
print("\n=== Example 4: Batch Research ===")
questions = [
"What is DNA?",
"How do vaccines work?",
"What causes earthquakes?",
]
with LDRClient() as client:
if not client.login(USERNAME, PASSWORD):
print("Login failed!")
return
# Start all research tasks
research_ids = []
for question in questions:
result = client.quick_research(question, wait_for_result=False)
research_ids.append((question, result["research_id"]))
print(f"Started: {question}")
print("\nWaiting for all results...")
# Collect all results
for question, research_id in research_ids:
try:
result = client.wait_for_research(research_id, timeout=120)
print(f"\n{question}")
print(f"{result['summary'][:150]}...")
except Exception as e:
print(f"\n{question}")
print(f"→ Error: {e}")
if __name__ == "__main__":
print("LDR Simple Client Examples")
print("=" * 50)
print("\nBefore: Complex CSRF handling, HTML parsing, manual polling...")
print("After: Just login() and quick_research()!")
print("\nMake sure:")
print("1. LDR server is running: python -m local_deep_research.web.app")
print("2. You've updated USERNAME and PASSWORD in this script")
print("=" * 50)
# Uncomment the examples you want to run:
# example_1_simple()
# example_2_client()
# example_3_context_manager()
# example_4_batch_research()
print("\nUncomment the examples in the script to run them!")