chore: import upstream snapshot with attribution

examples/eval-tool-use/README.md

@@ -0,0 +1,90 @@
+# eval-tool-use (Function and Tool Calling)
+
+This example demonstrates how to evaluate LLM function/tool calling capabilities using promptfoo.
+
+You can run this example with:
+
+```bash
+npx promptfoo@latest init --example eval-tool-use
+cd eval-tool-use
+```
+
+## Overview
+
+This example shows how to configure and test function/tool calling capabilities across multiple LLM providers:
+
+- OpenAI (with native function calling)
+- Anthropic (with Claude's tool use)
+- AWS Bedrock models
+- Groq (with function calling)
+
+Each provider has slightly different syntax and requirements for implementing function/tool calling.
+
+## Environment Variables
+
+This example requires the following environment variables:
+
+- `OPENAI_API_KEY` - Your OpenAI API key
+- `ANTHROPIC_API_KEY` - Your Anthropic API key
+- `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` - For AWS Bedrock (if using the Bedrock example)
+- `GROQ_API_KEY` - If using Groq's LLaMA models
+
+You can set these in a `.env` file or directly in your environment.
+
+## Provider Documentation
+
+Each provider implements tool use with different syntax:
+
+- [OpenAI Function Calling Guide](https://platform.openai.com/docs/guides/function-calling)
+- [Anthropic Tool Use Guide](https://docs.anthropic.com/en/docs/tool-use)
+- [AWS Bedrock Claude Tool Use](https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html)
+- [Groq Function Calling](https://console.groq.com/docs/tools)
+
+## Running the Example
+
+The configuration for this example is in:
+
+- `promptfooconfig.yaml` - Main example with OpenAI, Anthropic, and Groq
+- `promptfooconfig.bedrock.yaml` - Example specifically for AWS Bedrock models
+
+To run the main example:
+
+```bash
+promptfoo eval
+```
+
+To run the Bedrock example:
+
+```bash
+promptfoo eval -c promptfooconfig.bedrock.yaml
+```
+
+After running the evaluation, view the results with:
+
+```bash
+promptfoo view
+```
+
+## Example Tool: Weather Function
+
+This example uses a simple weather lookup function that takes a location and optionally a temperature unit. The example illustrates how different providers handle the same function definition with different syntaxes.
+
+External tools can also be loaded from separate files, as demonstrated with `external_tools.yaml`.
+
+### Anthropic Strict Mode
+
+The Anthropic provider includes an example with `strict: true` enabled, which uses Anthropic's structured outputs feature to guarantee that tool parameters exactly match your schema. This is useful for:
+
+- Building reliable agentic workflows
+- Ensuring type-safe function calls
+- Production systems that require guaranteed schema conformance
+
+When `strict: true` is enabled, Claude will always return tool inputs that strictly follow your `input_schema`, with no type mismatches or missing required fields. See the [Anthropic structured outputs example](../anthropic/structured-outputs) for more details.
+
+## Finish Reason Assertions
+
+This example also demonstrates the use of `finish-reason` assertions to validate why a model stopped generating:
+
+- **`tool_calls`**: Verifies the model stopped to make a function/tool call (e.g., weather lookup for cities)
+
+The example shows that when models are asked about weather in real cities (Boston, New York, Paris), they correctly stop generation to make tool calls, resulting in a `tool_calls` finish reason. This helps ensure your models are using tools appropriately when they should be.

examples/eval-tool-use/external_tools.yaml

@@ -0,0 +1,17 @@
+- type: function
+  function:
+    name: get_weather
+    description: Get the current weather in a given location
+    input_schema:
+      type: object
+      properties:
+        location:
+          type: string
+          description: The city and state, e.g. San Francisco, CA
+        unit:
+          type: string
+          enum:
+            - celsius
+            - fahrenheit
+      required:
+        - location

examples/eval-tool-use/promptfooconfig.bedrock.yaml

@@ -0,0 +1,48 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Bedrock Tool Use
+prompts:
+  - Tell me about the weather in {{city}}.
+providers:
+  - id: bedrock:us.anthropic.claude-sonnet-4-6
+    label: Claude Sonnet 4.6
+    config:
+      region: us-east-2
+      # optional block to force specific tool to be used
+      tool_choice:
+        type: tool
+        name: get_weather
+      tools:
+        - name: get_weather
+          description: Get the current weather in a given location
+          input_schema:
+            type: object
+            properties:
+              location:
+                type: string
+                description: The city and state, e.g. San Francisco, CA
+              unit:
+                type: string
+                enum:
+                  - celsius
+                  - fahrenheit
+            required:
+              - location
+        # irrelevant tool
+        - name: add
+          description: Add two numbers
+          input_schema:
+            type: object
+            properties:
+              a:
+                type: number
+              b:
+                type: number
+tests:
+  - vars:
+      city: Boston
+  - vars:
+      city: New York
+  - vars:
+      city: Paris
+  - vars:
+      city: Mars

examples/eval-tool-use/promptfooconfig.yaml

@@ -0,0 +1,115 @@
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+description: Tool use evaluation with function calling capabilities
+
+prompts:
+  - Tell me about the weather in {{city}} in the default unit for the location.
+
+providers:
+  - id: anthropic:messages:claude-sonnet-4-6
+    config:
+      tools:
+        - name: get_weather
+          description: Get the current weather in a given location
+          input_schema:
+            type: object
+            properties:
+              location:
+                type: string
+                description: The city and state, e.g. San Francisco, CA
+              unit:
+                type: string
+                enum:
+                  - celsius
+                  - fahrenheit
+            required:
+              - location
+  - id: anthropic:messages:claude-sonnet-4-6
+    label: With Strict Mode
+    config:
+      tools:
+        - name: get_weather
+          description: Get the current weather in a given location
+          strict: true # Enable strict mode for guaranteed schema validation
+          input_schema:
+            type: object
+            properties:
+              location:
+                type: string
+                description: The city and state, e.g. San Francisco, CA
+              unit:
+                type: string
+                enum:
+                  - celsius
+                  - fahrenheit
+            required:
+              - location
+            additionalProperties: false
+  - id: openai:chat:gpt-5.4-mini
+    config:
+      tools:
+        - type: function
+          function:
+            name: get_weather
+            description: Get the current weather in a given location
+            input_schema:
+              type: object
+              properties:
+                location:
+                  type: string
+                  description: The city and state, e.g. San Francisco, CA
+                unit:
+                  type: string
+                  enum:
+                    - celsius
+                    - fahrenheit
+              required:
+                - location
+  - id: openai:chat:gpt-5.4-mini
+    label: External Tools
+    config:
+      tools: file://external_tools.yaml
+  - id: groq:llama-4-scout-17b-16e-instruct
+    config:
+      tools:
+        - type: function
+          function:
+            name: get_weather
+            description: 'Get the current weather in a given location'
+            parameters:
+              type: object
+              properties:
+                location:
+                  type: string
+                  description: 'The city and state, e.g. San Francisco, CA'
+                unit:
+                  type: string
+                  enum:
+                    - celsius
+                    - fahrenheit
+              required:
+                - 'location'
+      tool_choice: auto
+
+tests:
+  # Test that tool calls result in 'tool_calls' finish reason
+  - vars:
+      city: Boston
+    assert:
+      - type: finish-reason
+        value: tool_calls # Should trigger tool use for weather lookup
+
+  - vars:
+      city: New York
+    assert:
+      - type: finish-reason
+        value: tool_calls
+
+  - vars:
+      city: Paris
+    assert:
+      - type: finish-reason
+        value: tool_calls
+
+  # Negative test case - Mars is not a real city, behavior may vary by provider
+  - vars:
+      city: Mars