chore: import upstream snapshot with attribution

docs/security/CODEQL_GUIDE.md

@@ -0,0 +1,193 @@
+# Local CodeQL Analysis Guide
+
+This guide explains how to set up and use CodeQL for local security analysis of the Local Deep Research project, covering both Python backend and JavaScript frontend code.
+
+## Prerequisites
+
+1. **CodeQL CLI**
+   - Download the latest CodeQL bundle from [GitHub](https://github.com/github/codeql-action/releases)
+   - Extract it to a permanent location (e.g., `C:\codeql` on Windows or `/opt/codeql` on Linux)
+   - Add the CodeQL executable to your system PATH
+
+2. **Ollama**
+   - Install Ollama from [ollama.ai](https://ollama.ai)
+   - Pull the required model: `ollama pull deepseek-r1:32b`
+
+3. **Analysis Scripts**
+   - Copy the appropriate analysis script from `docs/security/` to your project root:
+     - Windows: `analyze_sarif.ps1`
+     - Linux/Mac: `analyze_sarif.sh`
+
+## Setup Instructions
+
+### Windows
+
+1. **Install CodeQL**
+   ```powershell
+   # Download and extract CodeQL bundle
+   Invoke-WebRequest -Uri "https://github.com/github/codeql-action/releases/latest/download/codeql-bundle-win64.tar.gz" -OutFile "codeql.zip"
+   tar -xvzf codeql.zip -C C:\codeql
+
+   # Add to PATH (run as administrator)
+   [Environment]::SetEnvironmentVariable("Path", "$env:Path;C:\codeql\codeql", "Machine")
+   ```
+
+2. **Create CodeQL Databases**
+   ```powershell
+   # Navigate to project root/src (src is recommended for codeql analysis, to avoid analyzing venv folder)
+   cd path/to/local-deep-research/src
+
+   # Create Python database
+   codeql database create --language=python --source-root . ./python-db
+
+   # Create JavaScript database (for frontend code)
+   codeql database create --language=javascript --source-root . ./js-db
+   ```
+
+3. **Run Analysis**
+   ```powershell
+   # Run Python CodeQL analysis
+   codeql database analyze ./python-db python-security-and-quality.qls --format=sarif-latest --output=python-results.sarif
+
+   # Run JavaScript CodeQL analysis
+   codeql database analyze ./js-db javascript-security-extended.qls --format=sarif-latest --output=js-results.sarif
+
+   # Merge results (optional)
+   codeql dataset merge --output=combined-db --source=python-db --source=js-db
+   codeql database analyze ./combined-db --format=sarif-latest --output=combined-results.sarif
+
+   # Analyze results with Ollama
+   .\analyze_sarif.ps1
+   ```
+
+### Linux/Mac
+
+1. **Install CodeQL**
+   ```bash
+   # Download and extract CodeQL bundle
+   wget https://github.com/github/codeql-action/releases/latest/download/codeql-bundle-linux64.tar.gz
+   tar -xvzf codeql-bundle-linux64.tar.gz -C /opt/codeql
+
+   # Add to PATH (add to ~/.bashrc or ~/.zshrc)
+   export PATH=$PATH:/opt/codeql/codeql
+   ```
+
+2. **Create CodeQL Databases**
+   ```bash
+   # Navigate to project root/src
+   cd path/to/local-deep-research/src
+
+   # Create Python database
+   codeql database create --language=python --source-root . ./python-db
+
+   # Create JavaScript database
+   codeql database create --language=javascript --source-root . ./js-db
+   ```
+
+3. **Run Analysis**
+   ```bash
+   # Make script executable
+   chmod +x analyze_sarif.sh
+
+   # Run Python CodeQL analysis
+   codeql database analyze ./python-db python-security-and-quality.qls --format=sarif-latest --output=python-results.sarif
+
+   # Run JavaScript CodeQL analysis
+   codeql database analyze ./js-db javascript-security-extended.qls --format=sarif-latest --output=js-results.sarif
+
+   # Merge results (optional)
+   codeql dataset merge --output=combined-db --source=python-db --source=js-db
+   codeql database analyze ./combined-db --format=sarif-latest --output=combined-results.sarif
+
+   # Analyze results with Ollama
+   ./analyze_sarif.sh
+   ```
+
+## Important Queries
+
+The following CodeQL queries are particularly relevant for our project:
+
+1. **Python Security**
+   - `python-security-and-quality.qls`: General security and code quality
+   - `python/sql-injection`: SQL injection vulnerabilities
+   - `python/hardcoded-credentials`: Hardcoded secrets
+   - `python/unsafe-deserialization`: Insecure deserialization
+
+2. **JavaScript Security**
+   - `javascript-security-extended.qls`: Comprehensive security checks
+   - `javascript/xss`: Cross-site scripting vulnerabilities
+   - `javascript/prototype-pollution`: Prototype pollution issues
+   - `javascript/express-misconfigured-cors`: CORS misconfigurations
+   - `javascript/unsafe-dynamic-import`: Unsafe dynamic imports
+   - `javascript/unsafe-eval`: Unsafe eval() usage
+
+3. **Custom Queries**
+   - Logging injection vulnerabilities
+   - Uninitialized variables
+   - API security issues
+   - Frontend security best practices
+
+## Analysis Script Features
+
+The analysis scripts (`analyze_sarif.ps1` and `analyze_sarif.sh`) provide:
+
+1. **Input Validation**
+   - Checks if Ollama is running
+   - Validates SARIF file format
+   - Verifies required dependencies
+
+2. **Error Handling**
+   - Graceful error messages
+   - Color-coded output
+   - Detailed error reporting
+
+3. **Output**
+   - Human-readable analysis
+   - Prioritized findings
+   - Recommended fixes
+   - Language-specific recommendations
+
+## Troubleshooting
+
+1. **Ollama Connection Issues**
+   - Ensure Ollama is running: `ollama serve`
+   - Check if the model is pulled: `ollama list`
+   - Verify the endpoint in the script matches your setup
+
+2. **CodeQL Database Creation**
+   - Clean previous databases: `rm -rf ./python-db ./js-db`
+   - Ensure dependencies are installed (Python and Node.js)
+   - Check for sufficient disk space
+   - For JavaScript: Ensure node_modules is present
+
+3. **Analysis Script Issues**
+   - Verify script permissions (Linux/Mac)
+   - Check PowerShell execution policy (Windows)
+   - Ensure all required tools are installed
+
+## Best Practices
+
+1. **Regular Analysis**
+   - Run analysis before major commits
+   - Schedule regular security scans
+   - Review and address findings promptly
+   - Run both frontend and backend scans
+
+2. **Database Management**
+   - Clean old databases regularly
+   - Keep CodeQL CLI updated
+   - Use appropriate query suites
+   - Consider using RAM disk for large projects
+
+3. **Results Handling**
+   - Document resolved issues
+   - Track false positives
+   - Share findings with the team
+   - Prioritize critical frontend and backend issues
+
+## Additional Resources
+
+- [CodeQL Documentation](https://codeql.github.com/docs/)
+- [Python CodeQL Queries](https://github.com/github/codeql/tree/main/python/ql/src)
+- [JavaScript CodeQL Queries](https://github.com/github/codeql/tree/main/javascript/ql/src)
+- [Ollama Documentation](https://ollama.ai/docs)

docs/security/analyze_sarif.sh

@@ -0,0 +1,111 @@
+#!/bin/bash
+
+# CodeQL SARIF Analysis Script for Unix/Linux
+# This script analyzes CodeQL SARIF results using Ollama for human-readable explanations
+
+# Configuration
+SARIF_PATH="python-results.sarif"
+OUTPUT_PATH="codeql_analysis_results.txt"
+OLLAMA_ENDPOINT="http://localhost:11434/api/generate"
+MODEL="deepseek-r1:32b"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+CYAN='\033[0;36m'
+NC='\033[0m' # No Color
+
+# Function to check if Ollama is running
+check_ollama() {
+    echo -n "Checking Ollama connection... "
+    if curl -s "$OLLAMA_ENDPOINT" > /dev/null; then
+        echo -e "${GREEN}OK${NC}"
+        return 0
+    else
+        echo -e "${RED}FAILED${NC}"
+        echo -e "${RED}Error: Ollama is not running. Please start Ollama first.${NC}"
+        return 1
+    fi
+}
+
+# Function to validate SARIF file
+validate_sarif() {
+    echo -n "Validating SARIF file... "
+    if [ ! -f "$SARIF_PATH" ]; then
+        echo -e "${RED}FAILED${NC}"
+        echo -e "${RED}Error: SARIF file not found at: $SARIF_PATH${NC}"
+        return 1
+    fi
+
+    if jq empty "$SARIF_PATH" 2>/dev/null; then
+        echo -e "${GREEN}OK${NC}"
+        return 0
+    else
+        echo -e "${RED}FAILED${NC}"
+        echo -e "${RED}Error: Invalid SARIF file format${NC}"
+        return 1
+    fi
+}
+
+# Main script
+echo -e "${CYAN}CodeQL SARIF Analysis Tool${NC}"
+echo -e "${CYAN}=========================${NC}"
+
+# Check if required tools are installed
+command -v curl >/dev/null 2>&1 || { echo -e "${RED}Error: curl is required but not installed.${NC}"; exit 1; }
+command -v jq >/dev/null 2>&1 || { echo -e "${RED}Error: jq is required but not installed.${NC}"; exit 1; }
+
+# Check if Ollama is running
+check_ollama || exit 1
+
+# Validate SARIF file
+validate_sarif || exit 1
+
+# Read SARIF content
+echo -n "Reading SARIF file... "
+if SARIF_CONTENT=$(cat "$SARIF_PATH"); then
+    echo -e "${GREEN}OK${NC}"
+else
+    echo -e "${RED}FAILED${NC}"
+    echo -e "${RED}Error: Failed to read SARIF file${NC}"
+    exit 1
+fi
+
+# Prepare the prompt
+PROMPT="Please analyze these security findings from CodeQL analysis.
+Focus on:
+1. Critical vulnerabilities
+2. High priority issues
+3. Potential impact
+4. Recommended fixes
+
+Here is the analysis data:
+$SARIF_CONTENT"
+
+# Prepare the request body
+REQUEST_BODY=$(jq -n \
+    --arg model "$MODEL" \
+    --arg prompt "$PROMPT" \
+    '{"model": $model, "prompt": $prompt}')
+
+# Make the request to Ollama and process the streaming response
+echo -n "Analyzing results with Ollama... "
+if curl -s -X POST "$OLLAMA_ENDPOINT" \
+    -H "Content-Type: application/json" \
+    -d "$REQUEST_BODY" | \
+    while IFS= read -r line; do
+        if [ -n "$line" ]; then
+            echo "$line" | jq -r '.response // empty'
+        fi
+    done > "$OUTPUT_PATH"; then
+    echo -e "${GREEN}OK${NC}"
+else
+    echo -e "${RED}FAILED${NC}"
+    echo -e "${RED}Error: Failed to analyze with Ollama${NC}"
+    exit 1
+fi
+
+echo -e "\n${GREEN}Analysis complete!${NC}"
+echo -e "${YELLOW}Results saved to: $OUTPUT_PATH${NC}"
+echo -e "${YELLOW}You can view the results by opening: $OUTPUT_PATH${NC}"

docs/security/database-backup.md

@@ -0,0 +1,92 @@
+# Database Backup System
+
+Local Deep Research includes an automatic database backup system that creates encrypted backups of your user database after each successful login.
+
+## Overview
+
+- **Automatic**: Backups run in the background after login without blocking the UI
+- **Encrypted**: Backups use the same encryption as your main database
+- **Safe**: Uses SQLCipher's `sqlcipher_export()` for atomic backups that work correctly with WAL mode
+- **Configurable**: Enable/disable and configure retention via settings
+- **Pre-migration**: A backup is automatically created before any database schema migration
+
+## How It Works
+
+1. When you log in successfully, a background backup is scheduled
+2. Only one backup per calendar day is created — subsequent logins the same day are skipped to prevent a corrupted database from overwriting all good backups
+3. The backup runs in a separate thread (non-blocking)
+4. Uses `sqlcipher_export()` to create an encrypted copy preserving all cipher settings
+5. Old backups are automatically cleaned up based on your retention settings
+6. Before database migrations, a backup is always created regardless of the daily limit
+
+## Backup Location
+
+Backups are stored in:
+```
+{data_directory}/encrypted_databases/backups/{user_hash}/
+```
+
+Where `{user_hash}` is the first 16 hex characters of the SHA-256 hash of your username.
+
+Each backup file is named with a timestamp:
+```
+ldr_backup_20250125_143022.db
+```
+
+## Settings
+
+Configure backup behavior in Settings > Backup:
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| **Enable Auto-Backup** | `true` | Enable/disable automatic backups on login. Disable if disk space is limited. |
+| **Max Backups** | `1` | Maximum number of backup files to keep (1-30) |
+| **Backup Retention (days)** | `7` | Delete backups older than this many days |
+
+**Note**: Each backup is a full encrypted copy of your database and cannot be compressed. With the default of 1 backup, disk usage equals your database size. Users with large databases (e.g., containing uploaded PDFs) should monitor disk usage and can reduce the backup count or disable backups entirely via the **Enable Auto-Backup** setting if disk space is limited.
+
+## Why sqlcipher_export()?
+
+We use SQLCipher's `sqlcipher_export()` instead of `VACUUM INTO` or simple file copy because:
+
+1. **Encryption Preservation**: `VACUUM INTO` does not preserve SQLCipher encryption settings. `sqlcipher_export()` correctly copies data while maintaining the same encryption key and cipher configuration
+2. **WAL Safety**: Regular file copy can corrupt databases using WAL (Write-Ahead Logging) mode
+3. **Atomic Operation**: The backup uses ATTACH + export + DETACH, and is written to a temporary file then atomically renamed
+4. **Integrity Verification**: Each backup is verified with `PRAGMA quick_check` before being finalized
+
+## Restoring from Backup
+
+To restore from a backup:
+
+1. Stop the application
+2. Locate your backup in the backup directory
+3. Copy it to replace your current database file (keep the `.salt` file alongside it)
+4. Restart the application
+
+**Important**: The backup uses the same password as when it was created. If you've changed your password since the backup, you'll need to use the old password to access it.
+
+## Troubleshooting
+
+### Backups not being created
+
+1. Check if backups are enabled in Settings
+2. Check the logs for backup-related errors
+3. Verify sufficient disk space (requires 2x database size)
+
+### Disk space issues
+
+The system checks for available disk space before creating a backup. If you see "Insufficient disk space" errors:
+
+1. Free up disk space
+2. Reduce the max backup count setting
+3. Reduce the retention days setting
+
+### Backup verification failed
+
+If you see "Backup verification failed" in logs, the backup may be corrupted. This can happen if:
+
+1. The disk ran out of space during backup
+2. There was a system crash during backup
+3. The source database is corrupted
+
+In this case, the corrupted backup file is automatically deleted.