wehub-resource-sync
0d3cb498a3
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
openai-tools-call (OpenAI Tools Call Example)
This example demonstrates how to use promptfoo to evaluate OpenAI's tools calling capabilities. It shows how to define and test tool usage with the Chat Completions API.
Features Demonstrated
- Defining tools for AI models to use
- Testing tool call outputs
- Validating AI-generated function arguments
- Transforming outputs for assertions
Environment Variables
This example requires the following environment variables:
OPENAI_API_KEY- Your OpenAI API key
You can set this in a .env file or directly in your environment.
Running the Example
You can run this example with:
npx promptfoo@latest init --example openai-tools-call
# and then
cd openai-tools-call
# Run the evaluation
npx promptfoo eval
# View the results
npx promptfoo view
What This Example Does
The configuration defines a custom tool for getting weather information. It then tests the model's ability to:
- Correctly call the weather function when asked about weather
- Pass the correct location parameter based on the city mentioned
- Handle various cities, including international ones
- Format responses consistently
Key Features
- Uses
is-valid-openai-tools-callassertion to validate the function call structure - Demonstrates output transformation to isolate and test specific parts of the response
- Shows how to use JavaScript assertions for detailed validation
- Tests with a variety of locations to ensure robust behavior
Documentation
For more details, see: