wehub-resource-sync
0d3cb498a3
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Waiting to run
Test and Publish Multi-arch Docker Image / test (push) Waiting to run
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Blocked by required conditions
CI / Shell Format Check (push) Waiting to run
CI / Check Ruby (3.4) (push) Waiting to run
CI / CI Config (push) Waiting to run
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Blocked by required conditions
CI / Build on Node ${{ matrix.node }} (push) Blocked by required conditions
CI / Style Check (push) Waiting to run
CI / Generate Assets (push) Waiting to run
CI / Check Python (3.14) (push) Waiting to run
CI / Check Python (3.9) (push) Waiting to run
CI / Build Docs (push) Waiting to run
CI / Code Scan Action (push) Waiting to run
CI / Site tests (push) Waiting to run
CI / webui tests (push) Waiting to run
CI / Run Integration Tests (push) Waiting to run
CI / Run Smoke Tests (push) Waiting to run
CI / Go Tests (push) Waiting to run
CI / Share Test (push) Waiting to run
CI / Redteam (Production API) (push) Waiting to run
CI / Redteam (Staging API) (push) Waiting to run
CI / GitHub Actions Lint (push) Waiting to run
CI / Check Ruby (3.0) (push) Waiting to run
release-please / release-please (push) Waiting to run
release-please / build (push) Blocked by required conditions
release-please / publish-npm (push) Blocked by required conditions
release-please / publish-npm-backfill (push) Waiting to run
release-please / docker (push) Blocked by required conditions
release-please / publish-code-scan-action (push) Blocked by required conditions
release-please / attest-code-scan-action (push) Blocked by required conditions
Validate Renovate Config / Validate Renovate Configuration (push) Waiting to run
provider-github-models (GitHub Models Provider)
You can run this example with:
npx promptfoo@latest init --example provider-github-models
cd provider-github-models
Setup
Set your GITHUB_TOKEN environment variable. You can create a Personal Access Token at https://github.com/settings/tokens.
export GITHUB_TOKEN=ghp_your_token_here
What This Tests
This example tests the GitHub Models API integration:
- Direct Provider Usage: Uses GitHub Models as the main provider (gpt-4o-mini and gpt-5-mini)
- GitHub as Grader: Uses GitHub Models for
llm-rubricassertions via thedefaultTest.options.provider.textconfiguration
Run the Evaluation
promptfoo eval
View Results
promptfoo view
Available Models
GitHub Models supports various models including:
OpenAI Models:
github:openai/gpt-4o,github:openai/gpt-4o-minigithub:openai/gpt-5,github:openai/gpt-5-mini,github:openai/gpt-5-nanogithub:openai/o4-mini,github:openai/o3-mini
Other Providers:
github:meta/llama-4-scout-17b-16e-instructgithub:deepseek/DeepSeek-V3-0324github:mistral-ai/mistral-large
See the GitHub Models marketplace for the full list.