wehub-resource-sync
0d3cb498a3
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Waiting to run
Test and Publish Multi-arch Docker Image / test (push) Waiting to run
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Blocked by required conditions
CI / Shell Format Check (push) Waiting to run
CI / Check Ruby (3.4) (push) Waiting to run
CI / CI Config (push) Waiting to run
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Blocked by required conditions
CI / Build on Node ${{ matrix.node }} (push) Blocked by required conditions
CI / Style Check (push) Waiting to run
CI / Generate Assets (push) Waiting to run
CI / Check Python (3.14) (push) Waiting to run
CI / Check Python (3.9) (push) Waiting to run
CI / Build Docs (push) Waiting to run
CI / Code Scan Action (push) Waiting to run
CI / Site tests (push) Waiting to run
CI / webui tests (push) Waiting to run
CI / Run Integration Tests (push) Waiting to run
CI / Run Smoke Tests (push) Waiting to run
CI / Go Tests (push) Waiting to run
CI / Share Test (push) Waiting to run
CI / Redteam (Production API) (push) Waiting to run
CI / Redteam (Staging API) (push) Waiting to run
CI / GitHub Actions Lint (push) Waiting to run
CI / Check Ruby (3.0) (push) Waiting to run
release-please / release-please (push) Waiting to run
release-please / build (push) Blocked by required conditions
release-please / publish-npm (push) Blocked by required conditions
release-please / publish-npm-backfill (push) Waiting to run
release-please / docker (push) Blocked by required conditions
release-please / publish-code-scan-action (push) Blocked by required conditions
release-please / attest-code-scan-action (push) Blocked by required conditions
Validate Renovate Config / Validate Renovate Configuration (push) Waiting to run
compare-claude-vs-gpt-image (Image Analysis Example)
You can run this example with:
npx promptfoo@latest init --example compare-claude-vs-gpt-image
cd compare-claude-vs-gpt-image
Usage
This example compares an image analysis task using:
- gpt-4.1 via OpenAI
- claude-sonnet-4-6 via Amazon Bedrock
- claude-sonnet-4-6 via Anthropic
- gemini-3.1-flash-image-preview (Nano Banana 2) via Google AI Studio
GPT-4.1, Claude, and Gemini have different prompt formats. We use custom provider functions in Python and JavaScript to dynamically format the prompt based on context about the provider. The responses are scored using llm-rubric with a vision-capable OpenAI model.
To get started, set your environment variables:
OPENAI_API_KEYANTHROPIC_API_KEYGEMINI_API_KEYAWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEY
If you do not have access to all of these providers, simply comment out the providers you do not have access to in promptfooconfig.yaml.
Then run:
npx promptfoo@latest eval
Afterwards, you can view the results by running:
npx promptfoo@latest view