From 78e4b455c4e0c1800dd10fb8acfb97dd3bf24dca Mon Sep 17 00:00:00 2001 From: wehub-resource-sync Date: Mon, 13 Jul 2026 10:15:14 +0000 Subject: [PATCH] docs: preserve upstream English README --- README.en.md | 581 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 581 insertions(+) create mode 100644 README.en.md diff --git a/README.en.md b/README.en.md new file mode 100644 index 0000000..4586a70 --- /dev/null +++ b/README.en.md @@ -0,0 +1,581 @@ +# Local Deep Research + +
+ +[![GitHub stars](https://img.shields.io/github/stars/LearningCircuit/local-deep-research?style=for-the-badge)](https://github.com/LearningCircuit/local-deep-research/stargazers) +[![Docker Pulls](https://img.shields.io/docker/pulls/localdeepresearch/local-deep-research?style=for-the-badge)](https://hub.docker.com/r/localdeepresearch/local-deep-research) +[![PyPI Downloads](https://img.shields.io/pypi/dm/local-deep-research?style=for-the-badge)](https://pypi.org/project/local-deep-research/) + +[![Trendshift](https://trendshift.io/api/badge/repositories/14116)](https://trendshift.io/repositories/14116) + +[![Commits](https://img.shields.io/github/commit-activity/m/LearningCircuit/local-deep-research?style=for-the-badge)](https://github.com/LearningCircuit/local-deep-research/commits/main) +[![Last Commit](https://img.shields.io/github/last-commit/LearningCircuit/local-deep-research?style=for-the-badge)](https://github.com/LearningCircuit/local-deep-research/commits/main) + +[![SimpleQA Accuracy](https://img.shields.io/badge/SimpleQA-~95%25_Accuracy-gold?style=for-the-badge)](https://huggingface.co/datasets/local-deep-research/ldr-benchmarks) +[![SQLCipher](https://img.shields.io/badge/Database-SQLCipher_Encrypted-red?style=for-the-badge&logo=sqlite&logoColor=white)](docs/SQLCIPHER_INSTALL.md) + + +[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/LearningCircuit/local-deep-research/badge)](https://securityscorecards.dev/viewer/?uri=github.com/LearningCircuit/local-deep-research) +[![CodeQL](https://github.com/LearningCircuit/local-deep-research/actions/workflows/codeql.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/security/code-scanning) +[![Semgrep](https://github.com/LearningCircuit/local-deep-research/actions/workflows/semgrep.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/semgrep.yml) + +[![๐Ÿ”ง Pre-commit](https://github.com/LearningCircuit/local-deep-research/actions/workflows/pre-commit.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/pre-commit.yml) + +[![๐Ÿณ Docker Publish](https://github.com/LearningCircuit/local-deep-research/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/docker-publish.yml) +[![๐Ÿ“ฆ PyPI Publish](https://github.com/LearningCircuit/local-deep-research/actions/workflows/publish.yml/badge.svg)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/publish.yml) + +[![Discord](https://img.shields.io/discord/1352043059562680370?style=for-the-badge&logo=discord)](https://discord.gg/ttcqQeFcJ3) +[![Reddit](https://img.shields.io/badge/Reddit-r/LocalDeepResearch-FF4500?style=for-the-badge&logo=reddit)](https://www.reddit.com/r/LocalDeepResearch/) +[![YouTube](https://img.shields.io/badge/YouTube-Channel-red?style=for-the-badge&logo=youtube)](https://www.youtube.com/@local-deep-research) + + +**AI-powered research assistant for deep, agentic research** + +*Performs deep, agentic research using multiple LLMs and search engines with proper citations* + +๐Ÿงช **First open-source project โ€” fully-local on a single RTX 3090 (Qwen3.6-27B) โ€” to report ~95% SimpleQA (n=500) and 77% xbench-DeepSearch (n=100) on local hardware.** See the [r/LocalLLaMA announcement](https://www.reddit.com/r/LocalLLaMA/comments/1t1n6o8/we_are_finally_there_qwen3627b_agentic_search_957/) and the [benchmark dataset](https://huggingface.co/datasets/local-deep-research/ldr-benchmarks). + + + โ–ถ๏ธ Watch Review by The Art Of The Terminal + + +
+ +## ๐Ÿš€ What is Local Deep Research? + +AI research assistant you control. Run locally for privacy, use any LLM and build your own searchable knowledge base. You own your data and see exactly how it works. + +## โšก Quick Start + +**Option 1: Docker Run (Linux)** +```bash +# Step 1: Pull and run Ollama +docker run -d -p 11434:11434 --name ollama ollama/ollama +docker exec ollama ollama pull gpt-oss:20b + +# Step 2: Pull and run SearXNG for optimal search results +docker run -d -p 8080:8080 --name searxng searxng/searxng + +# Step 3: Pull and run Local Deep Research +docker run -d --network host \ + --name local-deep-research \ + --volume "deep-research:/data" \ + -e LDR_DATA_DIR=/data \ + localdeepresearch/local-deep-research +``` + +> **Mac / Windows / WSL2 users:** `--network host` only works on native Linux. On Docker Desktop it silently fails to publish port 5000 *and* leaves `localhost` pointing at the LDR container itself (so it can't reach Ollama/SearXNG). Use **Option 2** below, or see the [Windows/WSL2 FAQ entry](docs/faq.md#port-5000-not-accessible-on-windows) for a working `docker run` recipe. + +**Option 2: Docker Compose** + +CPU-only (all platforms): +```bash +curl -O https://raw.githubusercontent.com/LearningCircuit/local-deep-research/main/docker-compose.yml && docker compose up -d +``` + +With NVIDIA GPU (Linux): +```bash +curl -O https://raw.githubusercontent.com/LearningCircuit/local-deep-research/main/docker-compose.yml && \ +curl -O https://raw.githubusercontent.com/LearningCircuit/local-deep-research/main/docker-compose.gpu.override.yml && \ +docker compose -f docker-compose.yml -f docker-compose.gpu.override.yml up -d +``` + +Open http://localhost:5000 after ~30 seconds. For GPU setup, environment variables, and more, see the [Docker Compose Guide](docs/docker-compose-guide.md). + +**Option 3: pip install** +```bash +pip install local-deep-research +python -m local_deep_research.web.app # starts the web UI on http://localhost:5000 +``` +> You'll also need Ollama (or any OpenAI-compatible LLM endpoint) and SearXNG running โ€” see the [pip install guide](docs/install-pip.md) for the full recipe. +> Works on Windows, macOS, and Linux. SQLCipher encryption is included via pre-built wheels โ€” no compilation needed. +> PDF export on Windows requires Pango ([setup guide](https://doc.courtbouillon.org/weasyprint/stable/first_steps.html)). +> If you encounter issues with encryption, set `export LDR_BOOTSTRAP_ALLOW_UNENCRYPTED=true` to use standard SQLite instead. + +**Detailed install guides:** [Docker](docs/installation.md#docker) ยท [Docker Compose](docs/docker-compose-guide.md) ยท [pip](docs/install-pip.md) ยท [Unraid](docs/deployment/unraid.md) ยท [full install reference](docs/installation.md) + +> **Older CPU (x86-64)?** LDR needs an AVX-capable CPU โ€” Intel Sandy Bridge / AMD Bulldozer (2011) or newer. Several scientific Python dependencies (pandas, scikit-learn) ship wheels that crash with `Illegal instruction` on older CPUs. ARM64 (aarch64) is fully supported. Every release is smoke-tested against this floor, including AVX-without-AVX2 CPUs ([#4480](https://github.com/LearningCircuit/local-deep-research/issues/4480)). + +## ๐Ÿ—๏ธ How It Works + +### Research + +You ask a complex question. Local Deep Research (LDR): +- Does the research for you automatically +- Searches across web, academic papers, and your own documents +- Synthesizes everything into a report with proper citations + +Choose the research strategy that fits: quick pipeline modes for fast facts, or fully agentic deep research for complex analysis and academic work. + +**LangGraph Agent Strategy** โ€” An autonomous agentic research mode where the LLM decides what to search, which specialized engines to use (arXiv, PubMed, Semantic Scholar, etc.), and when to synthesize. It adaptively switches between search engines based on what it finds and collects significantly more sources than pipeline-based strategies โ€” this is the strategy behind the ~95% SimpleQA result above. Select `langgraph-agent` in Settings. + +### Build Your Knowledge Base + +```mermaid +flowchart LR + R[Research] --> D[Download Sources] + D --> L[(Library)] + L --> I[Index & Embed] + I --> S[Search Your Docs] + S -.-> R +``` + +Every research session finds valuable sources. Download them directly into your encrypted library โ€” academic papers from ArXiv, PubMed articles, web pages. LDR extracts text, indexes everything, and makes it searchable. Next time you research, ask questions across your own documents and the live web together. Your knowledge compounds over time. + +## ๐Ÿ›ก๏ธ Security + +
+ + +[![DevSkim](https://github.com/LearningCircuit/local-deep-research/actions/workflows/devskim.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/devskim.yml) +[![Bearer](https://github.com/LearningCircuit/local-deep-research/actions/workflows/bearer.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/bearer.yml) + + +[![OSV-Scanner](https://github.com/LearningCircuit/local-deep-research/actions/workflows/osv-scanner.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/osv-scanner.yml) +[![npm-audit](https://github.com/LearningCircuit/local-deep-research/actions/workflows/npm-audit.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/npm-audit.yml) +[![Retire.js](https://github.com/LearningCircuit/local-deep-research/actions/workflows/retirejs.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/retirejs.yml) + + +[![Container Security](https://github.com/LearningCircuit/local-deep-research/actions/workflows/container-security.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/container-security.yml) +[![Dockle](https://github.com/LearningCircuit/local-deep-research/actions/workflows/dockle.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/dockle.yml) +[![Hadolint](https://github.com/LearningCircuit/local-deep-research/actions/workflows/hadolint.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/hadolint.yml) +[![Checkov](https://github.com/LearningCircuit/local-deep-research/actions/workflows/checkov.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/checkov.yml) + + +[![Zizmor](https://github.com/LearningCircuit/local-deep-research/actions/workflows/zizmor-security.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/zizmor-security.yml) +[![OWASP ZAP](https://github.com/LearningCircuit/local-deep-research/actions/workflows/owasp-zap-scan.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/owasp-zap-scan.yml) +[![Security Tests](https://github.com/LearningCircuit/local-deep-research/actions/workflows/security-tests.yml/badge.svg?branch=main)](https://github.com/LearningCircuit/local-deep-research/actions/workflows/security-tests.yml) + +
+ +```mermaid +flowchart LR + U1[User A] --> D1[(Encrypted DB)] + U2[User B] --> D2[(Encrypted DB)] +``` + +Your data stays yours. Each user gets their own isolated SQLCipher database encrypted with AES-256, with the key derived from your password. Your password is never stored โ€” login works by attempting to decrypt your database, so the database files on their own are unusable to anyone who obtains them. Per-user LLM API keys live encrypted inside the same personal database rather than in a shared server-level store. + +The [Docker setup](docker-compose.yml) ships with `cap_drop: ALL`, `no-new-privileges`, and a non-root runtime, with the bundled Ollama and SearXNG images pinned by digest. Or run fully local with Ollama + SearXNG and nothing ever leaves your machine. + +**In-memory credentials**: Like any application that uses secrets at runtime, credentials are held in process memory during active sessions โ€” mitigated with session-scoped credential lifetimes and core dump exclusion. See the [Security Policy](SECURITY.md#in-memory-credentials) for the full threat model. + +**Supply Chain Security**: Docker images are signed with [Cosign](https://github.com/sigstore/cosign) using GitHub's keyless OIDC flow, include SLSA provenance attestations, and ship with attested SPDX SBOMs. See [Verifying images and SBOMs](SECURITY.md#verifying-images-and-sboms) for the step-by-step verification commands. + +**Security Transparency**: Scanner suppressions are documented with justifications in [Security Alerts Assessment](.github/SECURITY_ALERTS.md), [Scorecard Compliance](.github/SECURITY_SCORECARD.md), [Container CVE Suppressions](.trivyignore), and [SAST Rule Rationale](bearer.yml). Some alerts (Dependabot, code scanning) can only be dismissed or are very difficult to suppress outside the [GitHub Security tab](https://docs.github.com/en/code-security/dependabot/dependabot-alerts/viewing-and-updating-dependabot-alerts), so the files above do not cover every dismissed finding. + +[Detailed Architecture โ†’](docs/architecture.md) | [Security Policy โ†’](SECURITY.md) | [Security Review Process โ†’](docs/processes/security-review-process/) + +### ๐Ÿ”’ Privacy & Data + +Local Deep Research contains **no telemetry, no analytics, and no tracking**. We do not collect, transmit, or store any data about you or your usage. No analytics SDKs, no phone-home calls, no crash reporting, no external scripts. Usage metrics stay in your local encrypted database. + +The only network calls LDR makes are ones **you** initiate: search queries (to engines you configure), LLM API calls (to your chosen provider), and notifications (only if you set up Apprise). + +Since we don't collect any usage data, we rely on you to tell us what works, what's broken, and what you'd like to see next โ€” [bug reports](https://github.com/LearningCircuit/local-deep-research/issues), feature ideas, and even which features you love or never use all help us improve LDR. + +## ๐Ÿ“Š Benchmarks + +Headline results from the [community benchmarks](https://huggingface.co/datasets/local-deep-research/ldr-benchmarks) using the `langgraph-agent` strategy with Serper search, fully local via Ollama: + +| Model | SimpleQA | xbench-DeepSearch | +|---|---|---| +| Qwen3.6-27B | 95.7% (287/300) | 77.0% (77/100) | +| Qwen3.5-9B | 91.2% (182/200) | 59.0% (59/100) | +| gpt-oss-20B | 85.4% (295/346) | โ€“ | + +Caveats: small samples, LLM-grader noise, and SimpleQA contamination risk on newer base models. + +Picking a local model? The same community-maintained dataset tracks accuracy across models, search engines, and research strategies โ€” the fastest way to see which Ollama / LM Studio / llama.cpp models actually work well for deep research before you download multi-GB weights. **[Browse the full leaderboard on Hugging Face โ†’](https://huggingface.co/datasets/local-deep-research/ldr-benchmarks)** + +[Submit your own results โ†’](https://github.com/LearningCircuit/ldr-benchmarks) (contributors are listed in [CONTRIBUTORS.md](https://github.com/LearningCircuit/ldr-benchmarks/blob/main/CONTRIBUTORS.md)), or [run benchmarks locally โ†’](docs/BENCHMARKING.md). + +## โœจ Key Features + +### ๐Ÿ” Research Modes +- **Quick Summary** - Get answers in 30 seconds to 3 minutes with citations +- **Detailed Research** - Comprehensive analysis with structured findings +- **Report Generation** - Professional reports with sections and table of contents +- **Document Analysis** - Search your private documents with AI + +### ๐Ÿ› ๏ธ Advanced Capabilities +- **[LangChain Integration](docs/LANGCHAIN_RETRIEVER_INTEGRATION.md)** - Use any vector store as a search engine +- **[REST API](docs/api-quickstart.md)** - Authenticated HTTP access with per-user databases +- **[Benchmarking](docs/BENCHMARKING.md)** - Test and optimize your configuration +- **[Analytics Dashboard](docs/analytics-dashboard.md)** - Track costs, performance, and usage metrics +- **[Journal Quality System](docs/journal-quality.md)** - Automatic journal reputation scoring with 212K+ indexed sources, predatory detection, and quality dashboard. Powered by [OpenAlex](https://openalex.org) (CC0), [DOAJ](https://doaj.org) (CC0), and [Stop Predatory Journals](https://predatoryjournals.org) (MIT). See the [v1.6.0 announcement](https://www.reddit.com/r/LocalDeepResearch/comments/1svndjb/v160_highly_anticipated_journal_quality_filter/). +- **Real-time Updates** - WebSocket support for live research progress +- **[Chat Mode](docs/features.md#chat-mode)** - Multi-turn research conversations with streaming progress and accumulated context across turns +- **Export Options** - Download results as PDF or Markdown +- **Research History** - Save, search, and revisit past research +- **Adaptive Rate Limiting** - Intelligent retry system that learns optimal wait times +- **Keyboard Shortcuts** - Navigate efficiently (ESC, Ctrl+Shift+1-4) + +### ๐Ÿ“ฐ News & Research Subscriptions +- **Automated Research Digests** - Subscribe to topics or specific queries; AI filters and summarizes only the most relevant developments +- **Customizable Delivery** - Daily, weekly, or custom schedules, as markdown reports or structured summaries + +### ๐ŸŒ Search Sources + +#### Free Search Engines +- **Academic**: arXiv, PubMed, Semantic Scholar +- **General**: Wikipedia, SearXNG +- **Technical**: GitHub, Elasticsearch +- **Historical**: Wayback Machine +- **News**: The Guardian, Wikinews + +#### Premium Search Engines +- **Tavily** - AI-powered search +- **Google** - Via SerpAPI or Programmable Search Engine +- **Brave Search** - Privacy-focused web search + +#### Custom Sources +- **Local Documents** - Search your files with AI +- **LangChain Retrievers** - Any vector store or database +- **Meta Search** - Combine multiple engines intelligently + +LDR respects `robots.txt` and identifies itself honestly when fetching web pages โ€” no stealth or anti-detection techniques. In rare cases this means a page that blocks automated access won't be fetched, which we consider the right trade-off. + +[Full Search Engines Guide โ†’](docs/search-engines.md) + +## ๐Ÿ’ป Usage Examples + +### Python API +```python +from local_deep_research.api import LDRClient, quick_query + +# Option 1: Simplest - one line research +summary = quick_query("username", "password", "What is quantum computing?") +print(summary) + +# Option 2: Client for multiple operations +client = LDRClient() +client.login("username", "password") +result = client.quick_research("What are the latest advances in quantum computing?") +print(result["summary"]) +``` + +### HTTP API + +*The code example below shows the basic API structure - for working examples, see the link below* + +```python +import requests +from bs4 import BeautifulSoup + +# Create session and authenticate +session = requests.Session() +login_page = session.get("http://localhost:5000/auth/login") +soup = BeautifulSoup(login_page.text, "html.parser") +login_csrf = soup.find("input", {"name": "csrf_token"}).get("value") + +# Login and get API CSRF token +session.post("http://localhost:5000/auth/login", + data={"username": "user", "password": "pass", "csrf_token": login_csrf}) +csrf = session.get("http://localhost:5000/auth/csrf-token").json()["csrf_token"] + +# Make API request +response = session.post("http://localhost:5000/api/start_research", + json={"query": "Your research question"}, + headers={"X-CSRF-Token": csrf}) +``` + +๐Ÿš€ **[Ready-to-use HTTP API Examples โ†’ examples/api_usage/http/](examples/api_usage/http/)** +- โœ… **Automatic user creation** - works out of the box +- โœ… **Complete authentication** with CSRF handling +- โœ… **Result retry logic** - waits until research completes +- โœ… **Progress monitoring** and error handling + +### Command Line Tools + +```bash +# Run benchmarks from CLI +python -m local_deep_research.benchmarks.cli.benchmark_commands simpleqa --examples 50 + +# Manage rate limiting +python -m local_deep_research.web_search_engines.rate_limiting status +python -m local_deep_research.web_search_engines.rate_limiting reset --engine SearXNGSearchEngine +``` + +See the [Command Line Tools guide](docs/cli-tools.md) for the full reference. + +## ๐Ÿ”— Bring Your Own Knowledge Base + +Connect LDR to your existing knowledge base. Unlike the HTTP client above, `quick_summary()` runs LDR in-process โ€” no server needed โ€” so you can pass it live Python objects such as LangChain retrievers: + +```python +from local_deep_research.api import quick_summary + +# Use your existing LangChain retriever +result = quick_summary( + query="What are our deployment procedures?", + retrievers={"company_kb": your_retriever}, + search_tool="company_kb" +) +``` + +Works with: FAISS, Chroma, Pinecone, Weaviate, Elasticsearch, and any LangChain-compatible retriever. + +[Integration Guide โ†’](docs/LANGCHAIN_RETRIEVER_INTEGRATION.md) + +## ๐Ÿ”Œ MCP Server (Claude Integration) + +LDR provides an MCP (Model Context Protocol) server that allows AI assistants like Claude Desktop and Claude Code to perform deep research. Full setup details in the [MCP Server guide](docs/mcp-server.md). + +> โš ๏ธ **Security Note**: This MCP server is designed for **local use only** via STDIO transport (e.g., Claude Desktop). It has no built-in authentication or rate limiting. Do not expose over a network without implementing proper security controls. See the [MCP Security Best Practices](https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices) for network deployment requirements. + +### Installation + +```bash +# Install with MCP extras +pip install "local-deep-research[mcp]" +``` + +### Claude Desktop Configuration + +Add to your `claude_desktop_config.json`: + +```json +{ + "mcpServers": { + "local-deep-research": { + "command": "ldr-mcp", + "env": { + "LDR_LLM_PROVIDER": "openai", + "LDR_LLM_OPENAI_API_KEY": "sk-..." + } + } + } +} +``` + +### Claude Code Configuration + +Add to your `.mcp.json` (project-level) or `~/.claude/mcp.json` (global): + +```json +{ + "mcpServers": { + "local-deep-research": { + "command": "ldr-mcp", + "env": { + "LDR_LLM_PROVIDER": "ollama", + "LDR_LLM_OLLAMA_URL": "http://localhost:11434" + } + } + } +} +``` + +Any LDR setting can be passed in `env` as an `LDR_*` variable โ€” see the auto-generated [Full Configuration Reference](docs/CONFIGURATION.md) for the complete list. + +### Available Tools + +| Tool | Description | Duration | LLM Cost | +|------|-------------|----------|----------| +| `search` | Raw results from a specific engine (arxiv, pubmed, wikipedia, ...) | 5-30s | None | +| `quick_research` | Fast research summary | 1-5 min | Yes | +| `detailed_research` | Comprehensive analysis | 5-15 min | Yes | +| `generate_report` | Full markdown report | 10-30 min | Yes | +| `analyze_documents` | Search local collections | 30s-2 min | Yes | +| `list_search_engines` | List available search engines | instant | None | +| `list_strategies` | List research strategies | instant | None | +| `get_configuration` | Get current config | instant | None | + +### Individual Search Engines + +The `search` tool lets you query specific search engines directly and get raw results (title, link, snippet) โ€” no LLM processing, no cost, fast. This is especially useful for **monitoring and subscriptions** where you want to check for new content regularly without burning LLM tokens. + +``` +# Search arXiv for recent papers +search(query="transformer architecture improvements", engine="arxiv") + +# Search PubMed for medical literature +search(query="CRISPR clinical trials 2024", engine="pubmed") + +# Search Wikipedia for quick facts +search(query="quantum error correction", engine="wikipedia") + +# Search GitHub for code and repositories +search(query="agentic research frameworks", engine="github") + +# Use list_search_engines() to see all available engines +``` + +### Example Usage + +``` +"Use quick_research to find information about quantum computing applications" +"Search arxiv for recent papers on diffusion models" +"Generate a detailed research report on renewable energy trends" +``` + + +## ๐Ÿค– Supported LLMs + +### Local Models +- **Ollama** โ€” connect to its native API (default `http://localhost:11434`) +- **LM Studio** โ€” connect to its OpenAI-compatible server (default `http://localhost:1234/v1`) +- **llama.cpp** โ€” connect to `llama-server`'s OpenAI-compatible endpoint (default `http://localhost:8080/v1`); start with `llama-server -m ` +- Common models: Llama, Mistral, Gemma, DeepSeek, Qwen +- LLM processing stays local (search queries still go to web). No API costs. + +> ๐Ÿ’ก **Which local model should I pick?** See the [community benchmarks](#-benchmarks) โ€” community-submitted accuracy numbers across local and cloud models, so you can compare before downloading. + +### Cloud Models +- OpenAI +- Anthropic Claude +- Google Gemini +- 100+ models via OpenRouter + +### Custom Endpoints +- **OpenAI-Compatible Endpoint** โ€” any service speaking the OpenAI chat-completions API (vLLM, llama.cpp, gateways) +- **Anthropic-Compatible Endpoint** โ€” a self-hosted service speaking the Anthropic Messages API (`/v1/messages`); set `llm.anthropic_endpoint.url` + +[Model Setup โ†’](docs/env_configuration.md) + +## ๐Ÿ”„ Upgrading from Earlier Versions + +- **`llm.model` no longer has a default.** Pre-1.6.3 installs auto-filled `gemma3:12b` (Ollama) when no model was configured, which silently downloaded a multi-GB binary. The field is now empty by default โ€” pick a model in Settings โ†’ LLM, or research will fail loudly with a clear error. +- **The `auto` and `parallel` meta search engines were removed.** The default langgraph-agent strategy selects engines dynamically per query, which replaces them. Stored settings are migrated automatically (removed values become `searxng`); update any explicit `search_tool="auto"` API calls or `LDR_SEARCH_TOOL=auto` env overrides to a concrete engine such as `searxng`. +- **The `llamacpp` provider now uses HTTP instead of in-process loading.** If you previously set `llm.llamacpp_model_path` to a local `.gguf` file, that setting is no longer read. Instead, run `llama-server -m ` (it ships with every modern llama.cpp build) and the default `llm.llamacpp.url` of `http://localhost:8080/v1` will pick it up. Optional API key support is available via `llm.llamacpp.api_key` if you put `llama-server` behind an auth proxy. + +## ๐Ÿ“š Documentation + +### Getting Started +- [Installation Guide](docs/installation.md) +- [Frequently Asked Questions](docs/faq.md) +- [API Quickstart](docs/api-quickstart.md) +- [Configuration Guide](docs/env_configuration.md) +- [Full Configuration Reference](docs/CONFIGURATION.md) + +### Core Features +- [All Features Guide](docs/features.md) +- [Search Engines Guide](docs/search-engines.md) +- [Analytics Dashboard](docs/analytics-dashboard.md) + +### Advanced Features +- [LangChain Integration](docs/LANGCHAIN_RETRIEVER_INTEGRATION.md) +- [Benchmarking System](docs/BENCHMARKING.md) +- [Elasticsearch Setup](docs/elasticsearch_search_engine.md) +- [SearXNG Setup](docs/SearXNG-Setup.md) + +### Development +- [Docker Compose Guide](docs/docker-compose-guide.md) +- [Development Guide](docs/developing.md) +- [CodeQL Guide](docs/security/CODEQL_GUIDE.md) +- [Release Guide](docs/RELEASE_GUIDE.md) +- [CI/CD Infrastructure](docs/CI_CD_INFRASTRUCTURE.md) โ€” pre-commit hooks, workflows, security scanning +- [Workflow Status Dashboard](docs/ci/workflow-status.md) โ€” live health of every GitHub Actions workflow +- [Test Coverage Report](https://learningcircuit.github.io/local-deep-research/) โ€” pytest coverage published to GitHub Pages + +### Examples & Tutorials +- [API Examples](examples/api_usage/) +- [Benchmark Examples](examples/benchmarks/) +- [Optimization Examples](examples/optimization/) + +## ๐Ÿ“ฐ Featured In + +> "Local Deep Research **deserves special mention** for those who prioritize privacy... **tuned to use open-source LLMs** that can run on consumer GPUs or even CPUs. Journalists, researchers, or companies with sensitive topics can investigate information **without queries ever hitting an external server**." +> +> โ€” [Medium: Open-Source Deep Research AI Assistants](https://medium.com/@leucopsis/open-source-deep-research-ai-assistants-157462a59c14) + +### News & Articles +- [Korben.info](https://korben.info/local-deep-research-alternative-gratuite-recherche-ia-sourcee.html) - French tech blog ("Sherlock Holmes numรฉrique") +- [Roboto.fr](https://www.roboto.fr/blog/local-deep-research-l-alternative-open-source-gratuite-deep-research-d-openai) - "L'alternative open-source gratuite ร  Deep Research d'OpenAI" +- [KDJingPai AI Tools](https://www.kdjingpai.com/en/local-deep-research/) - AI productivity tools coverage +- [AI Sharing Circle](https://aisharenet.com/en/local-deep-research/) - AI resources coverage + +### Community Discussions +- [r/LocalLLaMA: 95.7% SimpleQA on a single 3090, fully local](https://www.reddit.com/r/LocalLLaMA/comments/1t1n6o8/we_are_finally_there_qwen3627b_agentic_search_957/) - Qwen3.6-27B benchmark announcement (161K views) +- [r/WebAfterAI: LDR as the agentic layer on top of Ollama / LM Studio / LocalAI](https://www.reddit.com/r/WebAfterAI/comments/1t18wr6/local_deep_research_opensource_ai_research/) - Stack-positioning explainer +- [Hacker News](https://news.ycombinator.com/item?id=43330164) - 190+ points, community discussion +- [LangChain Twitter/X](https://x.com/LangChainAI/status/1901347759757902038) - Official LangChain promotion +- [LangChain LinkedIn](https://www.linkedin.com/posts/langchain_local-deep-research-an-ai-research-activity-7307113456095137792-cXRH) - 400+ likes + +### International Coverage + +#### ๐Ÿ‡จ๐Ÿ‡ณ Chinese +- [Juejin (ๆŽ˜้‡‘)](https://juejin.cn/post/7481604667589885991) - Developer community +- [Cnblogs (ๅšๅฎขๅ›ญ)](https://www.cnblogs.com/qife122/p/18955032) - Developer blogs +- [GitHubDaily (Twitter/X)](https://x.com/GitHub_Daily/status/1900169979313741846) - Influential tech account +- [Zhihu (็ŸฅไนŽ)](https://zhuanlan.zhihu.com/p/30886269290) - Tech community +- [Aๅงๅˆ†ไบซ](https://www.ahhhhfs.com/68713/) - AI resources +- [CSDN](https://blog.csdn.net/gitblog_01198/article/details/147061415) - Installation guide +- [NetEase (็ฝ‘ๆ˜“)](https://www.163.com/dy/article/JQKAS50205567BLV.html) - Tech news portal + +#### ๐Ÿ‡ฏ๐Ÿ‡ต Japanese +- [note.com: ่ชฟๆŸป้ฉๅ‘ฝ๏ผšLocal Deep Researchๅพนๅบ•ๆดป็”จๆณ•](https://note.com/r7038xx/n/nb3b74debbb30) - Comprehensive tutorial +- [Qiita: Local Deep Researchใ‚’่ฉฆใ™](https://qiita.com/orca13/items/635f943287c45388d48f) - Docker setup guide +- [LangChainJP (Twitter/X)](https://x.com/LangChainJP/status/1902918110073807073) - Japanese LangChain community + +#### ๐Ÿ‡ฐ๐Ÿ‡ท Korean +- [PyTorch Korea Forum](https://discuss.pytorch.kr/t/local-deep-research/6476) - Korean ML community +- [GeekNews (Hada.io)](https://news.hada.io/topic?id=19707) - Korean tech news + +### Reviews & Analysis +- [BSAIL Lab: How useful is Deep Research in Academia?](https://uflbsail.net/uncategorized/how-useful-is-deep-research-in-academia/) - Academic review by contributor [@djpetti](https://github.com/djpetti) +- [The Art Of The Terminal: Use Local LLMs Already!](https://youtu.be/pfxgLX-MxMY?t=1999) - Comprehensive review of local AI tools, featuring LDR's research capabilities +- [Fahd Mirza: Local Deep Research + Ollama โ€” Free AI Research Assistant You Control](https://youtu.be/Q6kygd04sFI) - Installation walkthrough (349K-subscriber channel). The SearXNG startup issue shown in the video has since been fixed in [#3881](https://github.com/LearningCircuit/local-deep-research/pull/3881) โ€” clean Docker installs now start out of the box. +- [BC Adventure Tech: Run ChatGPT Deep Research Locally (Unraid AI Agent Setup)](https://youtu.be/bhy5TqmoLYo) - Unraid setup and showcase + +### Related Projects +- [SearXNG LDR-Academic](https://github.com/porespellar/searxng-LDR-academic) - Academic-focused SearXNG fork with 12 research engines (arXiv, Google Scholar, PubMed, etc.) designed for LDR +- [DeepWiki Documentation](https://deepwiki.com/LearningCircuit/local-deep-research) - Third-party documentation and guides + +> **Note:** Third-party projects and articles are independently maintained. We link to them as useful resources but cannot guarantee their code quality or security. + +## ๐Ÿค Community & Support + +- [Discord](https://discord.gg/ttcqQeFcJ3) - Get help and share research techniques +- [Reddit](https://www.reddit.com/r/LocalDeepResearch/) - Updates and showcases +- [GitHub Issues](https://github.com/LearningCircuit/local-deep-research/issues) - Bug reports + +## ๐Ÿง‘โ€๐Ÿ’ป Contributing + +We welcome contributions of all sizes โ€” from typo fixes to new features. The key rule: **keep PRs small and atomic** (one change per PR). For larger changes, please open an issue or start a discussion first โ€” we want to protect your time and make sure your effort leads to a successful merge rather than a misaligned PR. See our [Contributing Guide](CONTRIBUTING.md) to get started. + +## Acknowledgements + +Local Deep Research is built on the work of many open-access initiatives, academic databases, and open-source projects. We are grateful to: + +### Academic & Research Data + +| Source | What It Provides | License | +|--------|-----------------|---------| +| [OpenAlex](https://openalex.org) | Academic metadata for ~280K sources and ~120K institutions, including DOAJ status | CC0 | +| [DOAJ](https://doaj.org) | Directory of Open Access Journals โ€” open-access verification (via OpenAlex) | CC0 | +| [arXiv](https://arxiv.org) | Preprints in physics, mathematics, CS, and more | Various (see arXiv license) | +| [PubMed / NCBI](https://pubmed.ncbi.nlm.nih.gov) | Biomedical and life sciences literature | Public domain (US Gov) | +| [Semantic Scholar](https://www.semanticscholar.org) | Cross-discipline academic search with citation data | [Terms](https://www.semanticscholar.org/product/api/license) | +| [NASA ADS](https://ui.adsabs.harvard.edu) | Astrophysics, physics, and astronomy papers | [Terms](https://ui.adsabs.harvard.edu/help/terms/) | +| [Zenodo](https://zenodo.org) | Open research data, datasets, and software | Various per record | +| [PubChem](https://pubchem.ncbi.nlm.nih.gov) | Chemistry and biochemistry database | Public domain (US Gov) | +| [Stop Predatory Journals](https://predatoryjournals.org) | Predatory journal/publisher blacklist | MIT | +| [JabRef](https://github.com/JabRef/abbrv.jabref.org) | Journal abbreviation database | CC0 | + +### Knowledge & Content Sources + +[Wikipedia](https://www.wikipedia.org) • [OpenLibrary](https://openlibrary.org) • [Project Gutenberg](https://www.gutenberg.org) • [GitHub](https://github.com) • [Stack Exchange](https://stackexchange.com) • [The Guardian](https://www.theguardian.com) • [Wayback Machine](https://web.archive.org) + +### Infrastructure & Frameworks + +[LangChain](https://github.com/hwchase17/langchain) • [Ollama](https://ollama.ai) • [SearXNG](https://searxng.org/) • [FAISS](https://github.com/facebookresearch/faiss) + +### Support Open Access + +These projects run on donations and grants, not paywalls. If Local Deep Research is useful to you, consider giving back to the open-access ecosystem that makes it possible: + +- [arXiv](https://arxiv.org/about/give) โ€” free preprints for physics, math, CS, and more +- [PubMed / NLM](https://www.nlm.nih.gov/pubs/donations/donations.html) โ€” open biomedical literature +- [Wikipedia / Wikimedia](https://donate.wikimedia.org) โ€” the free encyclopedia +- [Internet Archive](https://archive.org/donate) โ€” the Wayback Machine and open digital library +- [DOAJ](https://doaj.org/support) โ€” curating and verifying open-access journals worldwide +- [OpenAlex](https://openalex.org) โ€” open scholarly metadata (sponsored by [OurResearch](https://ourresearch.org)) +- [Project Gutenberg](https://www.gutenberg.org/donate/) โ€” free ebooks since 1971 + +## ๐Ÿ“„ License + +MIT License - see [LICENSE](LICENSE) file. + +**Dependencies:** All third-party packages use permissive licenses (MIT, Apache-2.0, BSD, etc.) - see [allowlist](.github/workflows/dependency-review.yml)