chore: import upstream snapshot with attribution
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,100 @@
|
||||
# google-vertex-tools (Google Vertex Tools)
|
||||
|
||||
Example configurations for testing Google Vertex AI models with function calling and tool callbacks.
|
||||
|
||||
You can run this example with:
|
||||
|
||||
```bash
|
||||
npx promptfoo@latest init --example google-vertex-tools
|
||||
cd google-vertex-tools
|
||||
```
|
||||
|
||||
## Purpose
|
||||
|
||||
This example demonstrates how to use [Vertex AI models](https://www.promptfoo.dev/docs/providers/vertex/) with:
|
||||
|
||||
- Function calling and tool declarations
|
||||
- Function callback execution with local implementations
|
||||
- Different configuration approaches (YAML vs JavaScript)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Install the Google Auth Library:
|
||||
|
||||
```sh
|
||||
npm install google-auth-library
|
||||
```
|
||||
|
||||
2. Enable the Vertex AI API in your Google Cloud project
|
||||
|
||||
3. Configure your Google Cloud project:
|
||||
|
||||
```sh
|
||||
gcloud config set project PROJECT_ID
|
||||
```
|
||||
|
||||
4. Set up authentication using one of these methods:
|
||||
- Authenticate with your Google account:
|
||||
|
||||
```sh
|
||||
gcloud auth application-default login
|
||||
```
|
||||
|
||||
- Use a machine with an authorized service account
|
||||
- Use service account credentials file:
|
||||
1. Download your service account JSON
|
||||
2. Set the credentials path:
|
||||
|
||||
```sh
|
||||
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json
|
||||
```
|
||||
|
||||
## Configurations
|
||||
|
||||
This example includes two different approaches:
|
||||
|
||||
### Basic Tool Declaration (`promptfooconfig.yaml`)
|
||||
|
||||
Uses external tool definitions and validates function calls without execution:
|
||||
|
||||
- `promptfooconfig.yaml` - YAML configuration with external tools
|
||||
- `tools.json` - Function definitions for weather lookup
|
||||
|
||||
### Function Callbacks (`promptfooconfig-callback.js`)
|
||||
|
||||
Demonstrates actual function execution with local callbacks:
|
||||
|
||||
- `promptfooconfig-callback.js` - JavaScript configuration with inline tools and callbacks
|
||||
- Includes local function implementation for adding numbers
|
||||
|
||||
## Running the Examples
|
||||
|
||||
1. **Basic tool declaration example:**
|
||||
|
||||
```sh
|
||||
promptfoo eval -c promptfooconfig.yaml
|
||||
```
|
||||
|
||||
2. **Function callback example:**
|
||||
|
||||
```sh
|
||||
promptfoo eval -c promptfooconfig-callback.js
|
||||
```
|
||||
|
||||
3. **View results:**
|
||||
|
||||
```sh
|
||||
promptfoo view
|
||||
```
|
||||
|
||||
## Expected Results
|
||||
|
||||
- **Basic example**: Validates that the model correctly calls the weather function with proper parameters
|
||||
- **Callback example**: Actually executes the addition function and validates the computed results
|
||||
|
||||
## Learn More
|
||||
|
||||
- [Vertex AI Provider Documentation](https://www.promptfoo.dev/docs/providers/vertex/)
|
||||
- [Google Cloud Vertex AI Documentation](https://cloud.google.com/vertex-ai/docs)
|
||||
- [Function Calling Documentation](https://www.promptfoo.dev/docs/providers/vertex/#function-calling)
|
||||
- [promptfoo Documentation](https://www.promptfoo.dev/docs/)
|
||||
Reference in New Issue
Block a user