7a0da7932b
OSV-Scanner (Scheduled) / scan-scheduled (push) Failing after 0s
Create Release / test-gate (push) Has been cancelled
Create Release / release-gate (push) Has been cancelled
Create Release / ci-gate (push) Has been cancelled
Create Release / version-check (push) Has been cancelled
Create Release / e2e-test-gate (push) Has been cancelled
Create Release / responsive-test-gate (push) Has been cancelled
Create Release / compat-test-gate (push) Has been cancelled
Create Release / compose-integration-gate (push) Has been cancelled
Create Release / vulture-gate (push) Has been cancelled
Create Release / build (push) Has been cancelled
Create Release / provenance (push) Has been cancelled
Create Release / prerelease-docker (push) Has been cancelled
Create Release / publish-docker (push) Has been cancelled
Create Release / create-release (push) Has been cancelled
Create Release / cleanup-changelog (push) Has been cancelled
Create Release / trigger-pypi (push) Has been cancelled
Create Release / monitor-pypi (push) Has been cancelled
Create Release / Clean up orphan prerelease tags and signatures (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-form] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-metrics] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-workflow] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [settings-core] (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [history-news] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [library] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [link-analytics] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [chat-core] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [chat-lifecycle] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [error-benchmark] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [settings-pages] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) (push) Has been cancelled
Docker Tests (Consolidated) / Accessibility Tests (push) Has been cancelled
Docker Tests (Consolidated) / LLM Unit Tests (push) Has been cancelled
Docker Tests (Consolidated) / LLM Example Tests (push) Has been cancelled
Docker Tests (Consolidated) / Production Image Smoke Test (push) Has been cancelled
Docker Tests (Consolidated) / Infrastructure Tests (push) Has been cancelled
OSSF Scorecard / OSSF Security Scorecard Analysis (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [mobile] (push) Has been cancelled
Backwards Compatibility / Verify Encryption Constants (push) Has been cancelled
Backwards Compatibility / PyPI Version Compatibility (push) Has been cancelled
Backwards Compatibility / Database Migration Tests (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Docker Tests (Consolidated) / detect-changes (push) Has been cancelled
Docker Tests (Consolidated) / Build Test Image (push) Has been cancelled
Docker Tests (Consolidated) / All Pytest Tests + Coverage (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [accessibility] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [api-crud] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-login] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-pages] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-register] (push) Has been cancelled
368 lines
8.7 KiB
Markdown
368 lines
8.7 KiB
Markdown
# Features Documentation
|
|
|
|
This comprehensive guide covers all features available in Local Deep Research (LDR).
|
|
|
|
> **Note**: This documentation is maintained by the community and may contain inaccuracies. While we strive to keep it up-to-date, please verify critical information and report any errors via [GitHub Issues](https://github.com/LearningCircuit/local-deep-research/issues).
|
|
|
|
## Table of Contents
|
|
|
|
1. [Research Modes](#research-modes)
|
|
2. [Search Capabilities](#search-capabilities)
|
|
3. [LLM Integration](#llm-integration)
|
|
4. [User Interface Features](#user-interface-features)
|
|
5. [Advanced Features](#advanced-features)
|
|
6. [Developer Features](#developer-features)
|
|
7. [Performance Features](#performance-features)
|
|
|
|
## Research Modes
|
|
|
|
### Quick Summary Mode
|
|
|
|
Fast research mode that provides concise answers with citations.
|
|
|
|
**Features:**
|
|
- Automatic query decomposition
|
|
- Parallel search execution
|
|
- Smart result synthesis
|
|
- Citation tracking
|
|
- Structured output with tables when relevant
|
|
|
|
**Usage:**
|
|
```python
|
|
from local_deep_research.api import quick_summary
|
|
|
|
result = quick_summary(
|
|
query="Your research question",
|
|
iterations=2, # Number of research iterations
|
|
questions_per_iteration=3 # Sub-questions per iteration
|
|
)
|
|
```
|
|
|
|
### Detailed Research Mode
|
|
|
|
Comprehensive analysis mode for in-depth exploration of topics.
|
|
|
|
**Features:**
|
|
- Section-based research organization
|
|
- Multiple research cycles
|
|
- Cross-reference validation
|
|
- Extended context windows
|
|
- Detailed citation management
|
|
|
|
### Report Generation Mode
|
|
|
|
Creates professional research reports with proper structure.
|
|
|
|
**Features:**
|
|
- Automatic table of contents
|
|
- Section headers and organization
|
|
- Executive summary generation
|
|
- Bibliography management
|
|
- Export to PDF/Markdown
|
|
|
|
### Document Analysis Mode
|
|
|
|
Searches and analyzes your private document collections.
|
|
|
|
**Features:**
|
|
- Multiple document formats supported
|
|
- Vector-based semantic search
|
|
- Collection management
|
|
- Incremental indexing
|
|
- Privacy-preserved processing
|
|
|
|
### Chat Mode
|
|
|
|
> **Experimental** — interface and behavior may change before GA.
|
|
|
|
Interactive multi-turn research conversations. Each session accumulates context across turns and supports streaming progress and follow-up refinement via the sidebar **Chat** link or `/chat/`. Designed for exploring a topic progressively rather than one-off lookups; for single queries, use a research mode directly from the home page.
|
|
|
|
**Features:**
|
|
- Multi-turn conversation with accumulated context (entities, topics, source count)
|
|
- Live streaming of research steps and citations as the answer is built
|
|
- Persistent sessions in your per-user database (encrypted by default; survive logout)
|
|
- Session lifecycle: archive, reactivate, permanently delete
|
|
- Optional LLM-generated session titles (toggle via `chat.llm_title_generation`)
|
|
- Export a session as Markdown
|
|
- Always uses "quick" research mode (v1); one in-flight research per session
|
|
|
|
## Search Capabilities
|
|
|
|
### Multi-Engine Search
|
|
|
|
Simultaneously query multiple search engines for comprehensive results.
|
|
|
|
**Supported Engines:**
|
|
- Academic: arXiv, PubMed, Semantic Scholar
|
|
- General: Wikipedia, SearXNG, DuckDuckGo
|
|
- Technical: GitHub, Elasticsearch
|
|
- Custom: Local documents, LangChain retrievers
|
|
|
|
### Intelligent Query Routing
|
|
|
|
The default langgraph-agent strategy selects appropriate search engines dynamically based on query type:
|
|
- Scientific queries → Academic engines
|
|
- Code questions → GitHub + technical sources
|
|
- General knowledge → Wikipedia + web search
|
|
|
|
### Adaptive Rate Limiting
|
|
|
|
**Features:**
|
|
- Learns optimal wait times per engine
|
|
- Automatic retry with exponential backoff
|
|
- Fallback engine selection
|
|
- Rate limit status monitoring
|
|
|
|
### Search Strategies
|
|
|
|
Search strategies:
|
|
- `source-based`: Comprehensive research with detailed source tracking
|
|
- `focused-iteration`: Iterative refinement, quick Q&A (highest factual accuracy)
|
|
- `focused-iteration-standard`: Comprehensive variant with broader exploration
|
|
- `topic-organization`: Clusters sources into topics for structured output
|
|
- `mcp`: Agentic ReAct-pattern research using MCP tools
|
|
- `langgraph-agent`: Autonomous agentic research
|
|
|
|
See [Architecture Overview](architecture/OVERVIEW.md) for details.
|
|
|
|
## LLM Integration
|
|
|
|
### Local Models (via Ollama)
|
|
|
|
**Supported Models:**
|
|
- Llama 3 (8B, 70B)
|
|
- Mistral (7B, 8x7B)
|
|
- Gemma (7B, 12B)
|
|
- DeepSeek Coder
|
|
- Custom GGUF models
|
|
|
|
**Features:**
|
|
- Complete privacy
|
|
- No API costs
|
|
- Model hot-swapping
|
|
- GPU acceleration support
|
|
|
|
### Cloud Models
|
|
|
|
**Providers:**
|
|
- OpenAI (GPT-3.5, GPT-4)
|
|
- Anthropic (Claude 3 family)
|
|
- Google (Gemini models)
|
|
- OpenRouter (100+ models)
|
|
|
|
**Features:**
|
|
- Automatic fallback
|
|
- Cost tracking per model
|
|
- Token usage monitoring
|
|
- Model comparison tools
|
|
|
|
## User Interface Features
|
|
|
|
### Web Interface
|
|
|
|
**Core Features:**
|
|
- Real-time research progress
|
|
- Interactive result exploration
|
|
- Settings management
|
|
- Research history
|
|
- Export capabilities
|
|
|
|
### Keyboard Shortcuts
|
|
|
|
- `ESC`: Cancel current operation
|
|
- `Ctrl+Shift+1`: Quick Summary mode
|
|
- `Ctrl+Shift+2`: Detailed Research mode
|
|
- `Ctrl+Shift+3`: Report Generation
|
|
- `Ctrl+Shift+4`: Settings
|
|
- `Ctrl+Shift+5`: Analytics
|
|
|
|
### Real-time Updates
|
|
|
|
**WebSocket Features:**
|
|
- Live research progress
|
|
- Streaming results
|
|
- Status notifications
|
|
- Error handling
|
|
- Connection management
|
|
|
|
### Export Options
|
|
|
|
**Formats:**
|
|
- PDF with formatting
|
|
- Markdown with citations
|
|
- JSON for programmatic use
|
|
- Plain text
|
|
- HTML with styling
|
|
|
|
## Advanced Features
|
|
|
|
### LangChain Integration
|
|
|
|
Connect any LangChain-compatible retriever:
|
|
|
|
```python
|
|
from local_deep_research.api import quick_summary
|
|
|
|
result = quick_summary(
|
|
query="Internal documentation query",
|
|
retrievers={"company_docs": your_retriever},
|
|
search_tool="company_docs"
|
|
)
|
|
```
|
|
|
|
**Supported Vector Stores:**
|
|
- FAISS
|
|
- Chroma
|
|
- Pinecone
|
|
- Weaviate
|
|
- Elasticsearch
|
|
- Custom implementations
|
|
|
|
### MCP Server (Claude Integration)
|
|
|
|
Use LDR as a research tool directly from Claude Desktop or other MCP-compatible AI assistants.
|
|
|
|
**Features:**
|
|
- 8 tools (5 research, 3 discovery) accessible via Model Context Protocol
|
|
- STDIO transport for secure local operation
|
|
- Per-call settings overrides
|
|
- Autonomous agentic research (langgraph-agent strategy) with dynamic tool selection
|
|
- Document analysis with RAG pipeline
|
|
|
|
See [MCP Server Guide](mcp-server.md) for setup and usage.
|
|
|
|
### REST API
|
|
|
|
Full HTTP API for language-agnostic access:
|
|
|
|
```bash
|
|
# Quick summary
|
|
POST /api/v1/quick_summary
|
|
|
|
# Detailed research
|
|
POST /api/v1/detailed_research
|
|
|
|
# Report generation
|
|
POST /api/v1/generate_report
|
|
```
|
|
|
|
**Features:**
|
|
- OpenAPI specification
|
|
- Authentication support
|
|
- Rate limiting
|
|
- Webhook callbacks
|
|
- Batch processing
|
|
|
|
### Analytics Dashboard
|
|
|
|
**Metrics Tracked:**
|
|
- Cost per research/model
|
|
- Token usage patterns
|
|
- Response times
|
|
- Success rates
|
|
- Search engine health
|
|
- User ratings
|
|
|
|
**Time Ranges:**
|
|
- Last 7 days
|
|
- Last 30 days
|
|
- Last 90 days
|
|
- All time
|
|
|
|
### Research History
|
|
|
|
**Features:**
|
|
- Full research archive
|
|
- Search within results
|
|
- Tagging system
|
|
- Sharing capabilities
|
|
- Version tracking
|
|
|
|
## Developer Features
|
|
|
|
### Python SDK
|
|
|
|
```python
|
|
from local_deep_research import ResearchClient
|
|
|
|
client = ResearchClient(
|
|
llm_provider="ollama",
|
|
llm_model="llama3:8b",
|
|
search_engines=["searxng", "arxiv"]
|
|
)
|
|
|
|
result = client.research(
|
|
query="Your question",
|
|
strategy="focused_iteration"
|
|
)
|
|
```
|
|
|
|
### Benchmarking System
|
|
|
|
**Features:**
|
|
- SimpleQA dataset support
|
|
- Custom dataset creation
|
|
- Performance metrics
|
|
- A/B testing framework
|
|
- Configuration optimization
|
|
|
|
**Usage:**
|
|
```bash
|
|
python -m local_deep_research.benchmarks.cli.benchmark_commands simpleqa \
|
|
--examples 100 \
|
|
--search-tool searxng
|
|
```
|
|
|
|
|
|
### Command Line Tools
|
|
|
|
```bash
|
|
# Run benchmarks from CLI
|
|
python -m local_deep_research.benchmarks.cli.benchmark_commands simpleqa --examples 50
|
|
|
|
# Manage rate limiting
|
|
python -m local_deep_research.web_search_engines.rate_limiting status
|
|
python -m local_deep_research.web_search_engines.rate_limiting reset --engine SearXNGSearchEngine
|
|
```
|
|
|
|
## Performance Features
|
|
|
|
### Caching System
|
|
|
|
**Document Embedding Cache:**
|
|
- Caches document embeddings for faster subsequent searches
|
|
|
|
### Parallel Processing
|
|
|
|
**Optimization:**
|
|
- Concurrent search queries
|
|
- Parallel LLM calls
|
|
- Async result processing
|
|
- Thread pool management
|
|
|
|
### Resource Management
|
|
|
|
**Features:**
|
|
- Token budget enforcement
|
|
- Request queuing
|
|
- Graceful degradation
|
|
|
|
|
|
## Security Features
|
|
|
|
### Privacy Protection
|
|
|
|
- Local processing options
|
|
- No telemetry by default
|
|
- Secure credential storage
|
|
|
|
|
|
## Related Documentation
|
|
|
|
- [Search Engines Guide](search-engines.md)
|
|
- [API Quickstart](api-quickstart.md)
|
|
- [Configuration Guide](env_configuration.md)
|
|
- [Full Configuration Reference](CONFIGURATION.md)
|
|
- [Troubleshooting](troubleshooting.md)
|
|
- [Analytics Dashboard](analytics-dashboard.md)
|