chore: import upstream snapshot with attribution

site/docs/providers/echo.md

@@ -0,0 +1,132 @@
+---
+sidebar_label: Echo
+description: Configure Echo Provider for testing and debugging LLM integrations with zero-cost pass-through responses, perfect for validating pre-generated outputs locally
+---
+
+# Echo Provider
+
+The Echo Provider is a simple utility provider that returns the input prompt as the output. It's particularly useful for testing, debugging, and validating pre-generated outputs without making any external API calls.
+
+## Configuration
+
+To use the Echo Provider, set the provider ID to `echo` in your configuration file:
+
+```yaml
+providers:
+  - echo
+  # or
+  - id: echo
+    label: pass through provider
+```
+
+## Response Format
+
+The Echo Provider returns a complete `ProviderResponse` object with the following fields:
+
+- `output`: The original input string
+- `cost`: Always 0
+- `cached`: Always false
+- `tokenUsage`: Set to `{ total: 0, prompt: 0, completion: 0 }`
+- `isRefusal`: Always false
+- `metadata`: Any additional metadata provided in the context
+
+## Usage
+
+The Echo Provider requires no additional configuration and returns the input after performing any variable substitutions.
+
+### Example
+
+```yaml
+providers:
+  - echo
+  - openai:chat:gpt-5-mini
+
+prompts:
+  - 'Summarize this: {{text}}'
+
+tests:
+  - vars:
+      text: 'The quick brown fox jumps over the lazy dog.'
+    assert:
+      - type: contains
+        value: 'quick brown fox'
+      - type: similar
+        value: '{{text}}'
+        threshold: 0.75
+```
+
+In this example, the Echo Provider returns the exact input after variable substitution, while the OpenAI provider generates a summary.
+
+## Use Cases and Working with Pre-generated Outputs
+
+The Echo Provider is useful for:
+
+- **Debugging and Testing Prompts**: Ensure prompts and variable substitutions work correctly before using complex providers.
+
+- **Assertion and Pre-generated Output Evaluation**: Test assertion logic on known inputs and validate pre-generated outputs without new API calls.
+
+- **Testing Transformations**: Test how transformations affect the output without the variability of an LLM response.
+
+- **Mocking in Test Environments**: Use as a drop-in replacement for other providers in test environments when you don't want to make actual API calls.
+
+### Evaluating Logged Production Outputs
+
+A common pattern is evaluating LLM outputs that were already generated in production. This allows you to run assertions against real production data without making new API calls.
+
+Use your logged output directly as the prompt:
+
+```yaml
+prompts:
+  - '{{logged_output}}'
+
+providers:
+  - echo
+
+tests:
+  - vars:
+      logged_output: 'Paris is the capital of France.'
+    assert:
+      - type: llm-rubric
+        value: 'Answer is factually correct'
+      - type: contains
+        value: 'Paris'
+```
+
+The echo provider returns the prompt as-is, so your logged output flows directly to assertions without any API calls.
+
+For JSON-formatted production logs, use a default transform to extract specific fields:
+
+```yaml
+prompts:
+  - '{{logged_output}}'
+
+providers:
+  - echo
+
+defaultTest:
+  options:
+    # Extract just the response field from all logged outputs
+    transform: 'JSON.parse(output).response'
+
+tests:
+  - vars:
+      # Production logs often contain JSON strings
+      logged_output: '{"response": "Paris is the capital of France.", "confidence": 0.95, "model": "gpt-5"}'
+    assert:
+      - type: llm-rubric
+        value: 'Answer is factually correct'
+  - vars:
+      logged_output: '{"response": "London is in England.", "confidence": 0.98, "model": "gpt-5"}'
+    assert:
+      - type: contains
+        value: 'London'
+```
+
+This pattern is particularly useful for:
+
+- Post-deployment evaluation of production prompts
+- Regression testing against known outputs
+- A/B testing assertion strategies on historical data
+- Validating system behavior without API costs
+
+For loading large volumes of logged outputs, test cases can be generated dynamically from [CSV files, Python scripts, JavaScript functions, or JSON](/docs/configuration/test-cases).