learningcircuit--local-deep-research/docs/SearXNG-Setup.md

# SearXNG Integration for Local Deep Research

This document explains how to configure and use the SearXNG integration with Local Deep Research.

## Configuring SearXNG Access

The SearXNG search engine is **disabled by default** until you provide an instance URL. This ensures the system doesn't attempt to use public instances without explicit configuration.

### Setting Up Access

You have two ways to enable the SearXNG search engine:

1. **Environment Variable (Recommended)**:
   ```bash
   # Add to your .env file or set in your environment
   SEARXNG_INSTANCE=http://localhost:8080

   # Optional: Set custom delay between requests (in seconds)
   SEARXNG_DELAY=2.0
   ```

2. **Configuration Parameter**: Add to your `config.py`:
   ```python
   # In config.py
   SEARXNG_CONFIG = {
       "instance_url": "http://localhost:8080",
       "delay_between_requests": 2.0
   }
   ```

## Self-Hosting SearXNG (Recommended)

For the most ethical usage, we strongly recommend self-hosting your own SearXNG instance:

### Using Docker (easiest method)

```bash
# Pull the SearXNG Docker image
docker pull searxng/searxng

# Run SearXNG (will be available at http://localhost:8080)
docker run -d -p 8080:8080 --name searxng searxng/searxng
```

### Using Docker Compose (recommended for production)

1. Create a file named `docker-compose.yml` with the following content:

```yaml
version: '3'
services:
  searxng:
    container_name: searxng
    image: searxng/searxng
    ports:
      - "8080:8080"
    volumes:
      - ./searxng:/etc/searxng
    environment:
      - SEARXNG_BASE_URL=http://localhost:8080/
    restart: unless-stopped
```

2. Run with Docker Compose:

```bash
docker-compose up -d
```

## Using Public Instances

If you must use a public instance:

1. **Get Permission**: Always contact the administrator of any public instance
2. **Respect Resources**: Use a longer delay (4-5 seconds minimum) between requests
3. **Limited Usage**: Keep your research volume reasonable

Example configuration for a public instance:
```bash
SEARXNG_INSTANCE=https://instance.example.com
SEARXNG_DELAY=5.0
```

## Checking Configuration

To verify if SearXNG is properly configured:

```python
from web_search_engines.search_engine_factory import create_search_engine

# Create the engine
engine = create_search_engine("searxng")

# Check if available
if engine and hasattr(engine, 'is_available') and engine.is_available:
    print(f"SearXNG configured with instance: {engine.instance_url}")
    print(f"Delay between requests: {engine.delay_between_requests} seconds")
else:
    print("SearXNG is not properly configured or is disabled")
```

## Network Security

SearXNG is designed for self-hosting, so Local Deep Research allows SearXNG to access **private network IPs** by default. This means you can run SearXNG on:

- **Localhost**: `http://127.0.0.1:8080` or `http://localhost:8080`
- **LAN IPs**: `http://192.168.1.100:8080`, `http://10.0.0.5:8080`, `http://172.16.0.2:8080`
- **Docker networks**: `http://172.17.0.2:8080`
- **Local hostnames**: `http://searxng.local:8080` (if configured in DNS/hosts)

This is intentional and secure because:
1. The SearXNG URL is **admin-configured**, not user input
2. Private IPs are only accessible from your local network
3. **Cloud metadata endpoints** (AWS IMDS / ECS, Azure, OCI, DigitalOcean, AlibabaCloud, Tencent Cloud — see `ssrf_validator.ALWAYS_BLOCKED_METADATA_IPS`) are always blocked to prevent credential theft in cloud environments

### IPv6-only deployments (NAT64)

The "private IPs allowed" exception above does **not** cover IPv6 transition prefixes. On IPv6-only Kubernetes / cloud deployments (AWS / GKE / Azure IPv6-only nodes) where outbound IPv4 traffic is synthesized through NAT64 (`64:ff9b::/96` RFC 6052 well-known or `64:ff9b:1::/48` RFC 8215 local-use), reaching a SearXNG instance through these prefixes is blocked by default. To opt in, set:

```bash
LDR_SECURITY_ALLOW_NAT64=true
```

The opt-in is scoped strictly to the two NAT64 prefixes — 6to4 (`2002::/16`), Teredo (`2001::/32`), the discard prefix (`100::/64`), and the deprecated IPv4-Compatible IPv6 form (`::/96`) remain blocked, and cloud-metadata IPs stay unreachable through any NAT64 wrap. See [SECURITY.md](../SECURITY.md#ipv6-transition-prefix-block-list) for the full rationale.

## Troubleshooting

If you encounter errors:

1. Check that your instance is running
2. Verify the URL is correct in your environment variables
3. Ensure you can access the instance in your browser
4. Check firewall settings and network connectivity

## Resources

- [SearXNG Documentation](https://searxng.github.io/searxng/)
- [SearXNG GitHub Repository](https://github.com/searxng/searxng)
- [SearXNG Docker Hub](https://hub.docker.com/r/searxng/searxng)