learningcircuit--local-deep-research/docs/SearXNG-Setup.md

SearXNG Integration for Local Deep Research

This document explains how to configure and use the SearXNG integration with Local Deep Research.

Configuring SearXNG Access

The SearXNG search engine is disabled by default until you provide an instance URL. This ensures the system doesn't attempt to use public instances without explicit configuration.

Setting Up Access

You have two ways to enable the SearXNG search engine:

1. Environment Variable (Recommended):

   # Add to your .env file or set in your environment
   SEARXNG_INSTANCE=http://localhost:8080
   
   # Optional: Set custom delay between requests (in seconds)
   SEARXNG_DELAY=2.0
   

2. Configuration Parameter: Add to your config.py:

   # In config.py
   SEARXNG_CONFIG = {
       "instance_url": "http://localhost:8080",
       "delay_between_requests": 2.0
   }
   

Self-Hosting SearXNG (Recommended)

For the most ethical usage, we strongly recommend self-hosting your own SearXNG instance:

Using Docker (easiest method)

# Pull the SearXNG Docker image
docker pull searxng/searxng

# Run SearXNG (will be available at http://localhost:8080)
docker run -d -p 8080:8080 --name searxng searxng/searxng

Using Docker Compose (recommended for production)

1. Create a file named docker-compose.yml with the following content:

version: '3'
services:
  searxng:
    container_name: searxng
    image: searxng/searxng
    ports:
      - "8080:8080"
    volumes:
      - ./searxng:/etc/searxng
    environment:
      - SEARXNG_BASE_URL=http://localhost:8080/
    restart: unless-stopped

2. Run with Docker Compose:

docker-compose up -d

Using Public Instances

If you must use a public instance:

1. Get Permission: Always contact the administrator of any public instance
2. Respect Resources: Use a longer delay (4-5 seconds minimum) between requests
3. Limited Usage: Keep your research volume reasonable

Example configuration for a public instance:

SEARXNG_INSTANCE=https://instance.example.com
SEARXNG_DELAY=5.0

Checking Configuration

To verify if SearXNG is properly configured:

from web_search_engines.search_engine_factory import create_search_engine

# Create the engine
engine = create_search_engine("searxng")

# Check if available
if engine and hasattr(engine, 'is_available') and engine.is_available:
    print(f"SearXNG configured with instance: {engine.instance_url}")
    print(f"Delay between requests: {engine.delay_between_requests} seconds")
else:
    print("SearXNG is not properly configured or is disabled")

Network Security

SearXNG is designed for self-hosting, so Local Deep Research allows SearXNG to access private network IPs by default. This means you can run SearXNG on:

• Localhost: http://127.0.0.1:8080 or http://localhost:8080
• LAN IPs: http://192.168.1.100:8080, http://10.0.0.5:8080, http://172.16.0.2:8080
• Docker networks: http://172.17.0.2:8080
• Local hostnames: http://searxng.local:8080 (if configured in DNS/hosts)

This is intentional and secure because:

1. The SearXNG URL is admin-configured, not user input
2. Private IPs are only accessible from your local network
3. Cloud metadata endpoints (AWS IMDS / ECS, Azure, OCI, DigitalOcean, AlibabaCloud, Tencent Cloud — see ssrf_validator.ALWAYS_BLOCKED_METADATA_IPS) are always blocked to prevent credential theft in cloud environments

IPv6-only deployments (NAT64)

The "private IPs allowed" exception above does not cover IPv6 transition prefixes. On IPv6-only Kubernetes / cloud deployments (AWS / GKE / Azure IPv6-only nodes) where outbound IPv4 traffic is synthesized through NAT64 (64:ff9b::/96 RFC 6052 well-known or 64:ff9b:1::/48 RFC 8215 local-use), reaching a SearXNG instance through these prefixes is blocked by default. To opt in, set:

LDR_SECURITY_ALLOW_NAT64=true

The opt-in is scoped strictly to the two NAT64 prefixes — 6to4 (2002::/16), Teredo (2001::/32), the discard prefix (100::/64), and the deprecated IPv4-Compatible IPv6 form (::/96) remain blocked, and cloud-metadata IPs stay unreachable through any NAT64 wrap. See SECURITY.md for the full rationale.

Troubleshooting

If you encounter errors:

1. Check that your instance is running
2. Verify the URL is correct in your environment variables
3. Ensure you can access the instance in your browser
4. Check firewall settings and network connectivity

Resources

• SearXNG Documentation
• SearXNG GitHub Repository
• SearXNG Docker Hub