chore: import upstream snapshot with attribution
OSV-Scanner (Scheduled) / scan-scheduled (push) Failing after 0s
Create Release / test-gate (push) Has been cancelled
Create Release / release-gate (push) Has been cancelled
Create Release / ci-gate (push) Has been cancelled
Create Release / version-check (push) Has been cancelled
Create Release / e2e-test-gate (push) Has been cancelled
Create Release / responsive-test-gate (push) Has been cancelled
Create Release / compat-test-gate (push) Has been cancelled
Create Release / compose-integration-gate (push) Has been cancelled
Create Release / vulture-gate (push) Has been cancelled
Create Release / build (push) Has been cancelled
Create Release / provenance (push) Has been cancelled
Create Release / prerelease-docker (push) Has been cancelled
Create Release / publish-docker (push) Has been cancelled
Create Release / create-release (push) Has been cancelled
Create Release / cleanup-changelog (push) Has been cancelled
Create Release / trigger-pypi (push) Has been cancelled
Create Release / monitor-pypi (push) Has been cancelled
Create Release / Clean up orphan prerelease tags and signatures (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-form] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-metrics] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-workflow] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [settings-core] (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [history-news] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [library] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [link-analytics] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [chat-core] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [chat-lifecycle] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [error-benchmark] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [settings-pages] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) (push) Has been cancelled
Docker Tests (Consolidated) / Accessibility Tests (push) Has been cancelled
Docker Tests (Consolidated) / LLM Unit Tests (push) Has been cancelled
Docker Tests (Consolidated) / LLM Example Tests (push) Has been cancelled
Docker Tests (Consolidated) / Production Image Smoke Test (push) Has been cancelled
Docker Tests (Consolidated) / Infrastructure Tests (push) Has been cancelled
OSSF Scorecard / OSSF Security Scorecard Analysis (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [mobile] (push) Has been cancelled
Backwards Compatibility / Verify Encryption Constants (push) Has been cancelled
Backwards Compatibility / PyPI Version Compatibility (push) Has been cancelled
Backwards Compatibility / Database Migration Tests (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Docker Tests (Consolidated) / detect-changes (push) Has been cancelled
Docker Tests (Consolidated) / Build Test Image (push) Has been cancelled
Docker Tests (Consolidated) / All Pytest Tests + Coverage (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [accessibility] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [api-crud] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-login] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-pages] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-register] (push) Has been cancelled
OSV-Scanner (Scheduled) / scan-scheduled (push) Failing after 0s
Create Release / test-gate (push) Has been cancelled
Create Release / release-gate (push) Has been cancelled
Create Release / ci-gate (push) Has been cancelled
Create Release / version-check (push) Has been cancelled
Create Release / e2e-test-gate (push) Has been cancelled
Create Release / responsive-test-gate (push) Has been cancelled
Create Release / compat-test-gate (push) Has been cancelled
Create Release / compose-integration-gate (push) Has been cancelled
Create Release / vulture-gate (push) Has been cancelled
Create Release / build (push) Has been cancelled
Create Release / provenance (push) Has been cancelled
Create Release / prerelease-docker (push) Has been cancelled
Create Release / publish-docker (push) Has been cancelled
Create Release / create-release (push) Has been cancelled
Create Release / cleanup-changelog (push) Has been cancelled
Create Release / trigger-pypi (push) Has been cancelled
Create Release / monitor-pypi (push) Has been cancelled
Create Release / Clean up orphan prerelease tags and signatures (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-form] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-metrics] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [research-workflow] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [settings-core] (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [history-news] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [library] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [link-analytics] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [chat-core] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [chat-lifecycle] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [error-benchmark] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [settings-pages] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) (push) Has been cancelled
Docker Tests (Consolidated) / Accessibility Tests (push) Has been cancelled
Docker Tests (Consolidated) / LLM Unit Tests (push) Has been cancelled
Docker Tests (Consolidated) / LLM Example Tests (push) Has been cancelled
Docker Tests (Consolidated) / Production Image Smoke Test (push) Has been cancelled
Docker Tests (Consolidated) / Infrastructure Tests (push) Has been cancelled
OSSF Scorecard / OSSF Security Scorecard Analysis (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [mobile] (push) Has been cancelled
Backwards Compatibility / Verify Encryption Constants (push) Has been cancelled
Backwards Compatibility / PyPI Version Compatibility (push) Has been cancelled
Backwards Compatibility / Database Migration Tests (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Docker Tests (Consolidated) / detect-changes (push) Has been cancelled
Docker Tests (Consolidated) / Build Test Image (push) Has been cancelled
Docker Tests (Consolidated) / All Pytest Tests + Coverage (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [accessibility] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [api-crud] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-login] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-pages] (push) Has been cancelled
Docker Tests (Consolidated) / UI Tests (Puppeteer) [auth-register] (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,367 @@
|
||||
# Features Documentation
|
||||
|
||||
This comprehensive guide covers all features available in Local Deep Research (LDR).
|
||||
|
||||
> **Note**: This documentation is maintained by the community and may contain inaccuracies. While we strive to keep it up-to-date, please verify critical information and report any errors via [GitHub Issues](https://github.com/LearningCircuit/local-deep-research/issues).
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Research Modes](#research-modes)
|
||||
2. [Search Capabilities](#search-capabilities)
|
||||
3. [LLM Integration](#llm-integration)
|
||||
4. [User Interface Features](#user-interface-features)
|
||||
5. [Advanced Features](#advanced-features)
|
||||
6. [Developer Features](#developer-features)
|
||||
7. [Performance Features](#performance-features)
|
||||
|
||||
## Research Modes
|
||||
|
||||
### Quick Summary Mode
|
||||
|
||||
Fast research mode that provides concise answers with citations.
|
||||
|
||||
**Features:**
|
||||
- Automatic query decomposition
|
||||
- Parallel search execution
|
||||
- Smart result synthesis
|
||||
- Citation tracking
|
||||
- Structured output with tables when relevant
|
||||
|
||||
**Usage:**
|
||||
```python
|
||||
from local_deep_research.api import quick_summary
|
||||
|
||||
result = quick_summary(
|
||||
query="Your research question",
|
||||
iterations=2, # Number of research iterations
|
||||
questions_per_iteration=3 # Sub-questions per iteration
|
||||
)
|
||||
```
|
||||
|
||||
### Detailed Research Mode
|
||||
|
||||
Comprehensive analysis mode for in-depth exploration of topics.
|
||||
|
||||
**Features:**
|
||||
- Section-based research organization
|
||||
- Multiple research cycles
|
||||
- Cross-reference validation
|
||||
- Extended context windows
|
||||
- Detailed citation management
|
||||
|
||||
### Report Generation Mode
|
||||
|
||||
Creates professional research reports with proper structure.
|
||||
|
||||
**Features:**
|
||||
- Automatic table of contents
|
||||
- Section headers and organization
|
||||
- Executive summary generation
|
||||
- Bibliography management
|
||||
- Export to PDF/Markdown
|
||||
|
||||
### Document Analysis Mode
|
||||
|
||||
Searches and analyzes your private document collections.
|
||||
|
||||
**Features:**
|
||||
- Multiple document formats supported
|
||||
- Vector-based semantic search
|
||||
- Collection management
|
||||
- Incremental indexing
|
||||
- Privacy-preserved processing
|
||||
|
||||
### Chat Mode
|
||||
|
||||
> **Experimental** — interface and behavior may change before GA.
|
||||
|
||||
Interactive multi-turn research conversations. Each session accumulates context across turns and supports streaming progress and follow-up refinement via the sidebar **Chat** link or `/chat/`. Designed for exploring a topic progressively rather than one-off lookups; for single queries, use a research mode directly from the home page.
|
||||
|
||||
**Features:**
|
||||
- Multi-turn conversation with accumulated context (entities, topics, source count)
|
||||
- Live streaming of research steps and citations as the answer is built
|
||||
- Persistent sessions in your per-user database (encrypted by default; survive logout)
|
||||
- Session lifecycle: archive, reactivate, permanently delete
|
||||
- Optional LLM-generated session titles (toggle via `chat.llm_title_generation`)
|
||||
- Export a session as Markdown
|
||||
- Always uses "quick" research mode (v1); one in-flight research per session
|
||||
|
||||
## Search Capabilities
|
||||
|
||||
### Multi-Engine Search
|
||||
|
||||
Simultaneously query multiple search engines for comprehensive results.
|
||||
|
||||
**Supported Engines:**
|
||||
- Academic: arXiv, PubMed, Semantic Scholar
|
||||
- General: Wikipedia, SearXNG, DuckDuckGo
|
||||
- Technical: GitHub, Elasticsearch
|
||||
- Custom: Local documents, LangChain retrievers
|
||||
|
||||
### Intelligent Query Routing
|
||||
|
||||
The default langgraph-agent strategy selects appropriate search engines dynamically based on query type:
|
||||
- Scientific queries → Academic engines
|
||||
- Code questions → GitHub + technical sources
|
||||
- General knowledge → Wikipedia + web search
|
||||
|
||||
### Adaptive Rate Limiting
|
||||
|
||||
**Features:**
|
||||
- Learns optimal wait times per engine
|
||||
- Automatic retry with exponential backoff
|
||||
- Fallback engine selection
|
||||
- Rate limit status monitoring
|
||||
|
||||
### Search Strategies
|
||||
|
||||
Search strategies:
|
||||
- `source-based`: Comprehensive research with detailed source tracking
|
||||
- `focused-iteration`: Iterative refinement, quick Q&A (highest factual accuracy)
|
||||
- `focused-iteration-standard`: Comprehensive variant with broader exploration
|
||||
- `topic-organization`: Clusters sources into topics for structured output
|
||||
- `mcp`: Agentic ReAct-pattern research using MCP tools
|
||||
- `langgraph-agent`: Autonomous agentic research
|
||||
|
||||
See [Architecture Overview](architecture/OVERVIEW.md) for details.
|
||||
|
||||
## LLM Integration
|
||||
|
||||
### Local Models (via Ollama)
|
||||
|
||||
**Supported Models:**
|
||||
- Llama 3 (8B, 70B)
|
||||
- Mistral (7B, 8x7B)
|
||||
- Gemma (7B, 12B)
|
||||
- DeepSeek Coder
|
||||
- Custom GGUF models
|
||||
|
||||
**Features:**
|
||||
- Complete privacy
|
||||
- No API costs
|
||||
- Model hot-swapping
|
||||
- GPU acceleration support
|
||||
|
||||
### Cloud Models
|
||||
|
||||
**Providers:**
|
||||
- OpenAI (GPT-3.5, GPT-4)
|
||||
- Anthropic (Claude 3 family)
|
||||
- Google (Gemini models)
|
||||
- OpenRouter (100+ models)
|
||||
|
||||
**Features:**
|
||||
- Automatic fallback
|
||||
- Cost tracking per model
|
||||
- Token usage monitoring
|
||||
- Model comparison tools
|
||||
|
||||
## User Interface Features
|
||||
|
||||
### Web Interface
|
||||
|
||||
**Core Features:**
|
||||
- Real-time research progress
|
||||
- Interactive result exploration
|
||||
- Settings management
|
||||
- Research history
|
||||
- Export capabilities
|
||||
|
||||
### Keyboard Shortcuts
|
||||
|
||||
- `ESC`: Cancel current operation
|
||||
- `Ctrl+Shift+1`: Quick Summary mode
|
||||
- `Ctrl+Shift+2`: Detailed Research mode
|
||||
- `Ctrl+Shift+3`: Report Generation
|
||||
- `Ctrl+Shift+4`: Settings
|
||||
- `Ctrl+Shift+5`: Analytics
|
||||
|
||||
### Real-time Updates
|
||||
|
||||
**WebSocket Features:**
|
||||
- Live research progress
|
||||
- Streaming results
|
||||
- Status notifications
|
||||
- Error handling
|
||||
- Connection management
|
||||
|
||||
### Export Options
|
||||
|
||||
**Formats:**
|
||||
- PDF with formatting
|
||||
- Markdown with citations
|
||||
- JSON for programmatic use
|
||||
- Plain text
|
||||
- HTML with styling
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### LangChain Integration
|
||||
|
||||
Connect any LangChain-compatible retriever:
|
||||
|
||||
```python
|
||||
from local_deep_research.api import quick_summary
|
||||
|
||||
result = quick_summary(
|
||||
query="Internal documentation query",
|
||||
retrievers={"company_docs": your_retriever},
|
||||
search_tool="company_docs"
|
||||
)
|
||||
```
|
||||
|
||||
**Supported Vector Stores:**
|
||||
- FAISS
|
||||
- Chroma
|
||||
- Pinecone
|
||||
- Weaviate
|
||||
- Elasticsearch
|
||||
- Custom implementations
|
||||
|
||||
### MCP Server (Claude Integration)
|
||||
|
||||
Use LDR as a research tool directly from Claude Desktop or other MCP-compatible AI assistants.
|
||||
|
||||
**Features:**
|
||||
- 8 tools (5 research, 3 discovery) accessible via Model Context Protocol
|
||||
- STDIO transport for secure local operation
|
||||
- Per-call settings overrides
|
||||
- Autonomous agentic research (langgraph-agent strategy) with dynamic tool selection
|
||||
- Document analysis with RAG pipeline
|
||||
|
||||
See [MCP Server Guide](mcp-server.md) for setup and usage.
|
||||
|
||||
### REST API
|
||||
|
||||
Full HTTP API for language-agnostic access:
|
||||
|
||||
```bash
|
||||
# Quick summary
|
||||
POST /api/v1/quick_summary
|
||||
|
||||
# Detailed research
|
||||
POST /api/v1/detailed_research
|
||||
|
||||
# Report generation
|
||||
POST /api/v1/generate_report
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- OpenAPI specification
|
||||
- Authentication support
|
||||
- Rate limiting
|
||||
- Webhook callbacks
|
||||
- Batch processing
|
||||
|
||||
### Analytics Dashboard
|
||||
|
||||
**Metrics Tracked:**
|
||||
- Cost per research/model
|
||||
- Token usage patterns
|
||||
- Response times
|
||||
- Success rates
|
||||
- Search engine health
|
||||
- User ratings
|
||||
|
||||
**Time Ranges:**
|
||||
- Last 7 days
|
||||
- Last 30 days
|
||||
- Last 90 days
|
||||
- All time
|
||||
|
||||
### Research History
|
||||
|
||||
**Features:**
|
||||
- Full research archive
|
||||
- Search within results
|
||||
- Tagging system
|
||||
- Sharing capabilities
|
||||
- Version tracking
|
||||
|
||||
## Developer Features
|
||||
|
||||
### Python SDK
|
||||
|
||||
```python
|
||||
from local_deep_research import ResearchClient
|
||||
|
||||
client = ResearchClient(
|
||||
llm_provider="ollama",
|
||||
llm_model="llama3:8b",
|
||||
search_engines=["searxng", "arxiv"]
|
||||
)
|
||||
|
||||
result = client.research(
|
||||
query="Your question",
|
||||
strategy="focused_iteration"
|
||||
)
|
||||
```
|
||||
|
||||
### Benchmarking System
|
||||
|
||||
**Features:**
|
||||
- SimpleQA dataset support
|
||||
- Custom dataset creation
|
||||
- Performance metrics
|
||||
- A/B testing framework
|
||||
- Configuration optimization
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
python -m local_deep_research.benchmarks.cli.benchmark_commands simpleqa \
|
||||
--examples 100 \
|
||||
--search-tool searxng
|
||||
```
|
||||
|
||||
|
||||
### Command Line Tools
|
||||
|
||||
```bash
|
||||
# Run benchmarks from CLI
|
||||
python -m local_deep_research.benchmarks.cli.benchmark_commands simpleqa --examples 50
|
||||
|
||||
# Manage rate limiting
|
||||
python -m local_deep_research.web_search_engines.rate_limiting status
|
||||
python -m local_deep_research.web_search_engines.rate_limiting reset --engine SearXNGSearchEngine
|
||||
```
|
||||
|
||||
## Performance Features
|
||||
|
||||
### Caching System
|
||||
|
||||
**Document Embedding Cache:**
|
||||
- Caches document embeddings for faster subsequent searches
|
||||
|
||||
### Parallel Processing
|
||||
|
||||
**Optimization:**
|
||||
- Concurrent search queries
|
||||
- Parallel LLM calls
|
||||
- Async result processing
|
||||
- Thread pool management
|
||||
|
||||
### Resource Management
|
||||
|
||||
**Features:**
|
||||
- Token budget enforcement
|
||||
- Request queuing
|
||||
- Graceful degradation
|
||||
|
||||
|
||||
## Security Features
|
||||
|
||||
### Privacy Protection
|
||||
|
||||
- Local processing options
|
||||
- No telemetry by default
|
||||
- Secure credential storage
|
||||
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Search Engines Guide](search-engines.md)
|
||||
- [API Quickstart](api-quickstart.md)
|
||||
- [Configuration Guide](env_configuration.md)
|
||||
- [Full Configuration Reference](CONFIGURATION.md)
|
||||
- [Troubleshooting](troubleshooting.md)
|
||||
- [Analytics Dashboard](analytics-dashboard.md)
|
||||
Reference in New Issue
Block a user