chore: import upstream snapshot with attribution

examples/google-aistudio-gemini/README.md

@@ -0,0 +1,82 @@
+# google-aistudio-gemini (Google AI Studio (Gemini and Gemma) Example)
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example google-aistudio-gemini
+cd google-aistudio-gemini
+```
+
+This example demonstrates using Google's Gemini and Gemma models with promptfoo to evaluate math puzzle-solving capabilities.
+
+## Prerequisites
+
+- promptfoo CLI installed (`npm install -g promptfoo` or `brew install promptfoo`)
+- Google AI Studio API key set as `GOOGLE_API_KEY`
+
+## Available Models
+
+The example tests across multiple Gemini and Gemma models:
+
+- **Gemma 4 31B IT** - Open model with strong reasoning, coding, and agentic capabilities
+- **Gemma 4 26B A4B IT** - Smaller open Gemma 4 model for lower-latency reasoning and coding evals
+- **Gemini 3.5 Flash** - Newest, fastest frontier Flash model with high-effort thinking
+- **Gemini 3.1 Pro** - Frontier model with improved reasoning and multimodal understanding
+- **Gemini 3 Flash** - Frontier Flash model with strong reasoning at lower latency
+- **Gemini 3.1 Flash-Lite** - Low-latency, cost-efficient model for high-volume tasks
+- **Gemini 2.5 Pro** - Stable model with strong reasoning, coding, and multimodal understanding; also used with structured JSON output and function calling
+- **Gemini 2.5 Flash** - Stable Flash model with enhanced reasoning and thinking capabilities
+- **Gemini 2.5 Flash-Lite** - Cost-efficient and fast 2.5 model, optimized for high-volume, latency-sensitive tasks
+- **gemini-embedding-001** - Embedding model used for similarity-based assertions
+
+## System Instructions from File
+
+This example also demonstrates how to load system instructions from an external file using the `file://` prefix:
+
+```yaml
+providers:
+  - id: google:gemini-2.5-pro
+    label: gemini-with-system-instruction-file
+    config:
+      systemInstruction: file://system-instruction.txt
+```
+
+The `system-instruction.txt` file contains reusable instructions that can be:
+
+- Shared across multiple configurations
+- Version controlled separately
+- Used to manage complex or lengthy system prompts
+
+## Image Understanding Example
+
+This example also includes an image understanding configuration (`promptfooconfig.image.yaml`) that demonstrates:
+
+- **Multimodal capabilities**: Using Gemini models to analyze and compare images
+- **Image file handling**: Loading images using the `file://` prefix
+- **Visual comparison**: Testing the model's ability to identify and compare different images
+
+Images should be placed on separate lines in the prompt. The `file://` prefix automatically handles loading and encoding the images for the Gemini API.
+
+## Running the Eval
+
+1. Get a local copy of the configuration:
+
+```sh
+promptfoo init --example google-aistudio-gemini
+```
+
+2. Run the examples:
+
+```sh
+# Basic math puzzle evaluation across multiple Gemini and Gemma models
+promptfoo eval
+
+# Image understanding and comparison evaluation
+promptfoo eval -c promptfooconfig.image.yaml
+```
+
+3. View the results:
+
+```sh
+promptfoo view
+```

examples/google-aistudio-gemini/promptfooconfig.image.yaml

@@ -0,0 +1,21 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Image captioning with Gemini
+
+prompts:
+  - |
+    {{imageFile1}}
+    {{imageFile2}}
+    Compare the two images.
+
+providers:
+  - id: google:gemini-2.5-flash
+
+tests:
+  - vars:
+      imageFile1: file://assets/red-panda.jpg
+      imageFile2: file://assets/raccoon.jpg
+    assert:
+      - type: contains-all
+        value:
+          - red panda
+          - raccoon

examples/google-aistudio-gemini/promptfooconfig.yaml

@@ -0,0 +1,146 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Google AI Studio Gemini model evaluation
+
+prompts:
+  - |
+    Given this math puzzle: {{puzzle}}
+    Please solve this step by step, showing your reasoning process.
+
+providers:
+  - id: google:gemma-4-31b-it
+    config:
+      temperature: 0.7
+      maxOutputTokens: 2048
+
+  - id: google:gemma-4-26b-a4b-it
+    config:
+      temperature: 0.7
+      maxOutputTokens: 2048
+
+  - id: google:gemini-3.1-pro-preview
+    config:
+      generationConfig:
+        temperature: 0.7
+        maxOutputTokens: 4096
+        thinkingConfig:
+          thinkingLevel: HIGH # Improved reasoning and performance
+
+  - id: google:gemini-3.5-flash
+    config:
+      generationConfig:
+        temperature: 0.7
+        maxOutputTokens: 2048
+        thinkingConfig:
+          thinkingLevel: HIGH # Newest, fastest frontier Flash model
+
+  - id: google:gemini-3-flash-preview
+    config:
+      generationConfig:
+        temperature: 0.7
+        maxOutputTokens: 2048
+        thinkingConfig:
+          thinkingLevel: HIGH # Frontier Flash reasoning with preview features
+
+  - id: google:gemini-3.1-flash-lite
+    config:
+      generationConfig:
+        temperature: 0.7
+        maxOutputTokens: 2048
+        # Low-latency Flash-Lite model
+
+  - id: google:gemini-2.5-pro
+    config:
+      generationConfig:
+        temperature: 0.7
+        maxOutputTokens: 4096
+        thinkingConfig:
+          thinkingBudget: 2048 # Enhanced thinking for complex reasoning
+
+  - id: google:gemini-2.5-flash
+    config:
+      generationConfig:
+        temperature: 0.7
+        maxOutputTokens: 2048
+        thinkingConfig:
+          thinkingBudget: 1024 # Enhanced reasoning and thinking capabilities
+
+  - id: google:gemini-2.5-flash-lite
+    config:
+      generationConfig:
+        temperature: 0.7
+        maxOutputTokens: 1024
+        thinkingConfig:
+          thinkingBudget: 512 # Most cost-efficient and fastest 2.5 model
+
+  - id: google:gemini-2.5-pro
+    config:
+      temperature: 0.7
+      maxOutputTokens: 8192
+      topP: 0.9
+      topK: 40
+
+  # System instruction from file example
+  - id: google:gemini-2.5-pro
+    label: gemini-with-system-instruction-file
+    config:
+      temperature: 0.3
+      maxOutputTokens: 8192
+      systemInstruction: file://system-instruction.txt
+
+  # Structured output example
+  - id: google:gemini-2.5-pro
+    config:
+      generationConfig:
+        temperature: 0
+        maxOutputTokens: 8192
+        response_mime_type: 'application/json'
+        response_schema:
+          type: 'object'
+          properties:
+            answer:
+              type: 'string'
+              description: 'The numerical or text answer'
+            confidence:
+              type: 'string'
+              enum: ['high', 'medium', 'low']
+            method_used:
+              type: 'string'
+              description: 'One-word description of solving method'
+          required: ['answer', 'confidence', 'method_used']
+
+  # Function calling example
+  - id: google:gemini-2.5-pro
+    config:
+      tools:
+        function_declarations:
+          - name: 'verify_answer'
+            description: 'Verify if the answer is correct'
+            parameters:
+              type: 'object'
+              properties:
+                answer:
+                  type: 'string'
+                  description: 'The proposed answer'
+                explanation:
+                  type: 'string'
+                  description: 'Brief explanation'
+              required: ['answer']
+      tool_config:
+        function_calling_config:
+          mode: 'auto'
+defaultTest:
+  options:
+    provider:
+      text:
+        id: google:gemini-2.5-flash
+      embedding:
+        id: google:embedding:gemini-embedding-001
+tests:
+  - vars:
+      puzzle: 'If a train travels at 60 mph for 2.5 hours, then at 40 mph for 1.5 hours, what is the average speed for the entire journey?'
+    assert:
+      - type: contains
+        value: 52.5
+      - type: llm-rubric
+        value: |
+          The answer should be 52.5 mph.

examples/google-aistudio-gemini/system-instruction.txt

@@ -0,0 +1,5 @@
+You are a helpful math tutor. When solving problems:
+1. Show your work step by step
+2. Explain your reasoning clearly
+3. Double-check your calculations
+4. Always provide the final answer in a clear format