chore: import upstream snapshot with attribution

examples/integration-docker/README.md

@@ -0,0 +1,8 @@
+# integration-docker (Docker)
+
+Examples for using promptfoo with Docker containers.
+
+## Examples
+
+- [basic](./basic/) - Basic Docker provider setup
+- [code-generation-sandbox](./code-generation-sandbox/) - Sandboxed code generation and execution in Docker

examples/integration-docker/basic/README.md

@@ -0,0 +1,46 @@
+# integration-docker/basic (Docker Comparison)
+
+[Docker Model Runner](https://docs.docker.com/ai/model-runner/) makes it easy to manage, run, and deploy AI models using Docker. Designed for developers, Docker Model Runner streamlines the process of pulling, running, and serving large language models (LLMs) and other AI models directly from Docker Hub or any OCI-compliant registry.
+
+## Getting Started
+
+1. Create a promptfoo project with docker examples
+
+```bash
+npx promptfoo@latest init --example integration-docker/basic
+cd integration-docker/basic
+```
+
+2. Enable Docker Model Runner in Docker Desktop or Docker Engine per https://docs.docker.com/ai/model-runner/#enable-docker-model-runner.
+3. Use the Docker Model Runner CLI to pull the models
+
+**For the simple example:**
+
+```bash
+docker model pull ai/llama3.2:3B-Q4_K_M
+```
+
+**For the advanced example:**
+
+```bash
+docker model pull ai/llama3.2:3B-Q4_K_M
+docker model pull ai/gemma3:4B-Q4_K_M
+docker model pull ai/phi4:14B-Q4_K_M
+docker model pull ai/deepseek-r1-distill-llama:8B-Q4_K_M
+docker model pull ai/smollm3:Q4_K_M
+docker model pull ai/mxbai-embed-large:335M-F16
+```
+
+Note: These six models together require ~20 GiB of disk storage.
+
+## Simple Example
+
+```bash
+promptfoo eval -c promptfooconfig.comparison.simple.yaml
+```
+
+## Advanced Example
+
+```bash
+promptfoo eval -c promptfooconfig.comparison.advanced.yaml
+```

examples/integration-docker/basic/promptfooconfig.comparison.advanced.yaml

@@ -0,0 +1,32 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Compare facts about a topic with llm-rubric and similar assertions
+
+prompts:
+  - 'What are three concise facts about {{topic}}?'
+
+providers:
+  - id: docker:ai/llama3.2:3B-Q4_K_M
+  - id: docker:ai/gemma3:4B-Q4_K_M
+  - id: docker:ai/phi4:14B-Q4_K_M
+  - id: docker:ai/deepseek-r1-distill-llama:8B-Q4_K_M
+    config:
+      temperature: 0.1
+      max_tokens: 512
+
+tests:
+  - vars:
+      topic: 'whales'
+    assert:
+      - type: llm-rubric
+        value: 'Provide at least two of these three facts: Whales are (a) mammals, (b) live in the ocean, and (c) communicate with sound.'
+      - type: similar
+        value: 'whales are the largest animals in the world'
+        threshold: 0.6
+
+# Use local models for grading and embeddings for similarity instead of OpenAI
+defaultTest:
+  options:
+    provider:
+      id: docker:ai/smollm3:Q4_K_M
+      embedding:
+        id: docker:embeddings:ai/mxbai-embed-large:335M-F16

examples/integration-docker/basic/promptfooconfig.comparison.simple.yaml

@@ -0,0 +1,15 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Compare facts about a topic
+
+prompts:
+  - 'What are three concise facts about {{topic}}?'
+
+providers:
+  - id: docker:ai/llama3.2:3B-Q4_K_M
+
+tests:
+  - vars:
+      topic: 'whales'
+    assert:
+      - type: contains
+        value: 'mammals'

examples/integration-docker/code-generation-sandbox/README.md

@@ -0,0 +1,12 @@
+# integration-docker/code-generation-sandbox (Docker Code Generation Sandbox)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example integration-docker/code-generation-sandbox
+cd integration-docker/code-generation-sandbox
+```
+
+## Usage
+
+See https://promptfoo.dev/docs/guides/sandboxed-code-evals

examples/integration-docker/code-generation-sandbox/code_generation_prompt.txt

@@ -0,0 +1,7 @@
+You are a Python code generator. Write a Python function to solve the following problem:
+
+{{problem}}
+
+Use the following function name: {{function_name}}
+
+Only provide the function code, without any explanations or additional text. Wrap your code in triple backticks.

examples/integration-docker/code-generation-sandbox/promptfooconfig.yaml

@@ -0,0 +1,31 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: 'Generate code with Claude in a Docker sandbox'
+
+prompts:
+  - file://code_generation_prompt.txt
+
+providers:
+  - anthropic:claude-sonnet-4-6
+  # - openai:gpt-4.1
+
+tests:
+  - vars:
+      problem: 'Write a Python function to calculate the factorial of a number'
+      function_name: 'factorial'
+      test_input: '5'
+      expected_output: '120'
+  - vars:
+      problem: 'Write a Python function to check if a string is a palindrome'
+      function_name: 'is_palindrome'
+      test_input: "'racecar'"
+      expected_output: 'True'
+  - vars:
+      problem: 'Write a Python function to find the largest element in a list'
+      function_name: 'find_largest'
+      test_input: '[1, 5, 3, 9, 2]'
+      expected_output: '9'
+
+defaultTest:
+  assert:
+    - type: python
+      value: file://validate_and_run_code.py

examples/integration-docker/code-generation-sandbox/requirements.txt

@@ -0,0 +1,10 @@
+certifi==2025.10.5
+charset-normalizer==3.3.2
+docker==7.1.0
+epicbox==1.1.0
+idna==3.15
+python-dateutil==2.9.0.post0
+requests==2.33.0
+six==1.16.0
+structlog==24.2.0
+urllib3==2.7.0

examples/integration-docker/code-generation-sandbox/validate_and_run_code.py

@@ -0,0 +1,60 @@
+import re
+
+import epicbox
+
+# Replace with your preferred Docker image
+DOCKER_IMAGE = "python:3.9-alpine"
+
+
+def get_assert(output, context):
+    # Extract the Python function from the LLM output
+    function_match = re.search(r"```python\s*\n(def\s+.*?)\n```", output, re.DOTALL)
+    if not function_match:
+        return {"pass": False, "score": 0, "reason": "No function definition found"}
+
+    function_code = function_match.group(1)
+
+    # Configure epicbox
+    epicbox.configure(profiles=[epicbox.Profile("python", DOCKER_IMAGE)])
+
+    # Get the function name, test input, and expected output from the context
+    function_name = context["vars"]["function_name"]
+    test_input = context["vars"]["test_input"]
+    expected_output = context["vars"]["expected_output"]
+
+    # Prepare the code to run in the sandbox
+    test_code = f"""
+{function_code}
+
+# Test the function
+result = {function_name}({test_input})
+print(result)
+"""
+
+    files = [{"name": "main.py", "content": test_code.encode("utf-8")}]
+    limits = {"cputime": 1, "memory": 64}
+
+    # Run the code in the sandbox
+    result = epicbox.run("python", "python main.py", files=files, limits=limits)
+
+    if result["exit_code"] != 0:
+        return {
+            "pass": False,
+            "score": 0,
+            "reason": f"Execution error: {result['stderr'].decode('utf-8')}",
+        }
+
+    # Compare the output with the expected result
+    actual_output = result["stdout"].decode("utf-8").strip()
+    if actual_output == str(expected_output):
+        return {
+            "pass": True,
+            "score": 1,
+            "reason": f"Correct output: got {expected_output}",
+        }
+    else:
+        return {
+            "pass": False,
+            "score": 0,
+            "reason": f"Incorrect output. Expected: {expected_output}, Got: {actual_output}",
+        }