chore: import upstream snapshot with attribution

lightrag/evaluation/sample_documents/01_lightrag_overview.md

@@ -0,0 +1,17 @@
+# LightRAG Framework Overview
+
+## What is LightRAG?
+
+**LightRAG** is a Simple and Fast Retrieval-Augmented Generation framework. LightRAG was developed by HKUDS (Hong Kong University Data Science Lab). The framework provides developers with tools to build RAG applications efficiently.
+
+## Problem Statement
+
+Large language models face several limitations. LLMs have a knowledge cutoff date that prevents them from accessing recent information. Large language models generate hallucinations when providing responses without factual grounding. LLMs lack domain-specific expertise in specialized fields.
+
+## How LightRAG Solves These Problems
+
+LightRAG solves the hallucination problem by combining large language models with external knowledge retrieval. The framework ensures accurate responses by grounding LLM outputs in actual documents. LightRAG provides contextual responses that reduce hallucinations significantly. The system enables efficient retrieval from external knowledge bases to supplement LLM capabilities.
+
+## Core Benefits
+
+LightRAG offers accuracy through document-grounded responses. The framework provides up-to-date information without model retraining. LightRAG enables domain expertise through specialized document collections. The system delivers cost-effectiveness by avoiding expensive model fine-tuning. LightRAG ensures transparency by showing source documents for each response.

lightrag/evaluation/sample_documents/02_rag_architecture.md

@@ -0,0 +1,21 @@
+# RAG System Architecture
+
+## Main Components of RAG Systems
+
+A RAG system consists of three main components that work together to provide intelligent responses.
+
+### Component 1: Retrieval System
+
+The retrieval system is the first component of a RAG system. A retrieval system finds relevant documents from large document collections. Vector databases serve as the primary storage for the retrieval system. Search engines can also function as retrieval systems in RAG architectures.
+
+### Component 2: Embedding Model
+
+The embedding model is the second component of a RAG system. An embedding model converts text into vector representations for similarity search. The embedding model transforms documents and queries into numerical vectors. These vector representations enable semantic similarity matching between queries and documents.
+
+### Component 3: Large Language Model
+
+The large language model is the third component of a RAG system. An LLM generates responses based on retrieved context from documents. The large language model synthesizes information from multiple sources into coherent answers. LLMs provide natural language generation capabilities for the RAG system.
+
+## How Components Work Together
+
+The retrieval system fetches relevant documents for a user query. The embedding model enables similarity matching between query and documents. The LLM generates the final response using retrieved context. These three components collaborate to provide accurate, contextual responses.

lightrag/evaluation/sample_documents/03_lightrag_improvements.md

@@ -0,0 +1,25 @@
+# LightRAG Improvements Over Traditional RAG
+
+## Key Improvements
+
+LightRAG improves upon traditional RAG approaches in several significant ways.
+
+### Simpler API Design
+
+LightRAG offers a simpler API compared to traditional RAG frameworks. The framework provides intuitive interfaces for developers. Traditional RAG systems often require complex configuration and setup. LightRAG focuses on ease of use while maintaining functionality.
+
+### Faster Retrieval Performance
+
+LightRAG delivers faster retrieval performance than traditional RAG approaches. The framework optimizes document retrieval operations for speed. Traditional RAG systems often suffer from slow query response times. LightRAG achieves high quality results with improved performance.
+
+### Better Vector Database Integration
+
+LightRAG provides better integration with various vector databases. The framework supports multiple vector database backends seamlessly. Traditional RAG approaches typically lock developers into specific database choices. LightRAG enables flexible storage backend selection.
+
+### Optimized Prompting Strategies
+
+LightRAG implements optimized prompting strategies for better results. The framework uses refined prompt templates for accurate responses. Traditional RAG systems often use generic prompting approaches. LightRAG balances simplicity with high quality output.
+
+## Design Philosophy
+
+LightRAG prioritizes ease of use without sacrificing quality. The framework combines speed with accuracy in retrieval operations. LightRAG maintains flexibility in database and model selection.

lightrag/evaluation/sample_documents/04_supported_databases.md

@@ -0,0 +1,37 @@
+# LightRAG Vector Database Support
+
+## Supported Vector Databases
+
+LightRAG supports multiple vector databases for flexible deployment options.
+
+### ChromaDB
+
+ChromaDB is a vector database supported by LightRAG. ChromaDB provides simple deployment for development environments. The database offers efficient vector similarity search capabilities.
+
+### Neo4j
+
+Neo4j is a graph database supported by LightRAG. Neo4j enables graph-based knowledge representation alongside vector search. The database combines relationship modeling with vector capabilities.
+
+### Milvus
+
+Milvus is a vector database supported by LightRAG. Milvus provides high-performance vector search at scale. The database handles large-scale vector collections efficiently.
+
+### Qdrant
+
+Qdrant is a vector database supported by LightRAG. Qdrant offers fast similarity search with filtering capabilities. The database provides production-ready vector search infrastructure.
+
+### MongoDB Atlas Vector Search
+
+MongoDB Atlas Vector Search is supported by LightRAG. MongoDB Atlas combines document storage with vector search capabilities. The database enables unified data management for RAG applications.
+
+### Redis
+
+Redis is supported by LightRAG for vector search operations. Redis provides in-memory vector search with low latency. The database offers fast retrieval for real-time applications.
+
+### Built-in Nano-VectorDB
+
+LightRAG includes a built-in nano-vectordb for simple deployments. Nano-vectordb eliminates external database dependencies for small projects. The built-in database provides basic vector search functionality without additional setup.
+
+## Database Selection Benefits
+
+The multiple database support enables developers to choose appropriate storage backends. LightRAG adapts to different deployment scenarios from development to production. Users can select databases based on scale, performance, and infrastructure requirements.

lightrag/evaluation/sample_documents/05_evaluation_and_deployment.md

@@ -0,0 +1,41 @@
+# RAG Evaluation Metrics and Deployment
+
+## Key RAG Evaluation Metrics
+
+RAG system quality is measured through four key metrics.
+
+### Faithfulness Metric
+
+Faithfulness measures whether answers are factually grounded in retrieved context. The faithfulness metric detects hallucinations in LLM responses. High faithfulness scores indicate answers based on actual document content. The metric evaluates factual accuracy of generated responses.
+
+### Answer Relevance Metric
+
+Answer Relevance measures how well answers address the user question. The answer relevance metric evaluates response quality and appropriateness. High answer relevance scores show responses that directly answer user queries. The metric assesses the connection between questions and generated answers.
+
+### Context Recall Metric
+
+Context Recall measures completeness of retrieval from documents. The context recall metric evaluates whether all relevant information was retrieved. High context recall scores indicate comprehensive document retrieval. The metric assesses retrieval system effectiveness.
+
+### Context Precision Metric
+
+Context Precision measures quality and relevance of retrieved documents. The context precision metric evaluates retrieval accuracy without noise. High context precision scores show clean retrieval without irrelevant content. The metric measures retrieval system selectivity.
+
+## LightRAG Deployment Options
+
+LightRAG can be deployed in production through multiple approaches.
+
+### Docker Container Deployment
+
+Docker containers enable consistent LightRAG deployment across environments. Docker provides isolated runtime environments for the framework. Container deployment simplifies dependency management and scaling.
+
+### REST API Server with FastAPI
+
+FastAPI serves as the REST API framework for LightRAG deployment. The FastAPI server exposes LightRAG functionality through HTTP endpoints. REST API deployment enables client-server architecture for RAG applications.
+
+### Direct Python Integration
+
+Direct Python integration embeds LightRAG into Python applications. Python integration provides programmatic access to RAG capabilities. Direct integration supports custom application workflows and pipelines.
+
+### Deployment Features
+
+LightRAG supports environment-based configuration for different deployment scenarios. The framework integrates with multiple LLM providers for flexibility. LightRAG enables horizontal scaling for production workloads.

lightrag/evaluation/sample_documents/README.md

@@ -0,0 +1,21 @@
+# Sample Documents for Evaluation
+
+These markdown files correspond to test questions in `../sample_dataset.json`.
+
+## Usage
+
+1. **Index documents** into LightRAG (via WebUI, API, or Python)
+2. **Run evaluation**: `python lightrag/evaluation/eval_rag_quality.py`
+3. **Expected results**: ~91-100% RAGAS score per question
+
+## Files
+
+- `01_lightrag_overview.md` - LightRAG framework and hallucination problem
+- `02_rag_architecture.md` - RAG system components
+- `03_lightrag_improvements.md` - LightRAG vs traditional RAG
+- `04_supported_databases.md` - Vector database support
+- `05_evaluation_and_deployment.md` - Metrics and deployment
+
+## Note
+
+Documents use clear entity-relationship patterns for LightRAG's default entity extraction prompts. For better results with your data, customize `lightrag/prompt.py`.