625 Searxng Setup B5889c65

# SearXNG Integration for Local Deep Research

This document explains how to configure and use the SearXNG integration with Local Deep Research.

## Configuring SearXNG Access

The SearXNG search engine is **disabled by default** until you provide an instance URL. This ensures the system doesn't attempt to use public instances without explicit configuration.

### Setting Up Access

You have two ways to enable the SearXNG search engine:

1. **Environment Variable (Recommended)**:
   ```bash
   # Add to your .env file or set in your environment
   SEARXNG_INSTANCE=http://localhost:8080

   # Optional: Set custom delay between requests (in seconds)
   SEARXNG_DELAY=2.0
   ```

2. **Configuration Parameter**: Add to your `config.py`:
   ```python
   # In config.py
   SEARXNG_CONFIG = {
       "instance_url": "http://localhost:8080",
       "delay_between_requests": 2.0
   }
   ```

## Self-Hosting SearXNG (Recommended)

For the most ethical usage, we strongly recommend self-hosting your own SearXNG instance:

### Using Docker (easiest method)

```bash
# Pull the SearXNG Docker image
docker pull searxng/searxng

# Run SearXNG (will be available at http://localhost:8080)
docker run -d -p 8080:8080 --name searxng searxng/searxng
```

### Using Docker Compose (recommended for production)

1. Create a file named `docker-compose.yml` with the following content:

```yaml
version: '3'
services:
  searxng:
    container_name: searxng
    image: searxng/searxng
    ports:
      - "8080:8080"
    volumes:
      - ./searxng:/etc/searxng
    environment:
      - SEARXNG_BASE_URL=http://localhost:8080/
    restart: unless-stopped
```

2. Run with Docker Compose:

```bash
docker-compose up -d
```

## Using Public Instances

If you must use a public instance:

1. **Get Permission**: Always contact the administrator of any public instance
2. **Respect Resources**: Use a longer delay (4-5 seconds minimum) between requests
3. **Limited Usage**: Keep your research volume reasonable

Example configuration for a public instance:
```bash
SEARXNG_INSTANCE=https://instance.example.com
SEARXNG_DELAY=5.0
```

## Checking Configuration

To verify if SearXNG is properly configured:

```python
from web_search_engines.search_engine_factory import create_search_engine

# Create the engine
engine = create_search_engine("searxng")

# Check if available
if engine and hasattr(engine, 'is_available') and engine.is_available:
    print(f"SearXNG configured with instance: {engine.instance_url}")
    print(f"Delay between requests: {engine.delay_between_requests} seconds")
else:
    print("SearXNG is not properly configured or is disabled")
```

## Network Security

SearXNG is designed for self-hosting, so Local Deep Research allows SearXNG to access **private network IPs** by default. This means you can run SearXNG on:

- **Localhost**: `http://127.0.0.1:8080` or `http://localhost:8080`
- **LAN IPs**: `http://192.168.1.100:8080`, `http://10.0.0.5:8080`, `http://172.16.0.2:8080`
- **Docker networks**: `http://172.17.0.2:8080`
- **Local hostnames**: `http://searxng.local:8080` (if configured in DNS/hosts)

This is intentional and secure because:
1. The SearXNG URL is **admin-configured**, not user input
2. Private IPs are only accessible from your local network
3. The **AWS metadata endpoint** (169.254.169.254) is always blocked to prevent credential theft in cloud environments

## Troubleshooting

If you encounter errors:

1. Check that your instance is running
2. Verify the URL is correct in your environment variables
3. Ensure you can access the instance in your browser
4. Check firewall settings and network connectivity

## Resources

- [SearXNG Documentation](https://searxng.github.io/searxng/)
- [SearXNG GitHub Repository](https://github.com/searxng/searxng)
- [SearXNG Docker Hub](https://hub.docker.com/r/searxng/searxng)