wehub-resource-sync
0d3cb498a3
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Waiting to run
Test and Publish Multi-arch Docker Image / test (push) Waiting to run
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Blocked by required conditions
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Blocked by required conditions
Validate Renovate Config / Validate Renovate Configuration (push) Waiting to run
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
2.5 KiB
2.5 KiB
anthropic/opus-4-8-coding (Claude Opus 4.8 Advanced Coding)
This example exercises Claude Opus 4.8 on hard coding tasks using the xhigh effort level with adaptive thinking.
You can run this example with:
npx promptfoo@latest init --example anthropic/opus-4-8-coding
cd opus-4-8-coding
What This Tests
Claude Opus 4.8 is Anthropic's most capable model for complex reasoning and agentic coding. This example evaluates:
- Bug diagnosis across multiple system boundaries
- Production-quality code generation with proper error handling
- Code review with nuanced, prioritized feedback
Working with Opus 4.8
- Builds on Opus 4.7. Opus 4.8 supports the same feature set as 4.7 (no breaking API changes) and improves capability on complex reasoning and long-horizon agentic coding.
- Adaptive thinking is opt-in. Set
thinking: { type: adaptive }(as this example does) to let the model decide when and how much to reason per request. Without an explicitthinkingblock the model runs without extended thinking, even at high effort. effortdefaults tohigh;xhighis available. Settingeffort: highbehaves the same as omitting it. Start withxhighfor coding and agentic work, and pair high effort with a largemax_tokens.- Sampling controls are managed for you. Opus 4.8 rejects
temperature,top_p, andtop_kat the model level; promptfoo omits them automatically (don't set them in config).
Running the Example
# Set your API key
export ANTHROPIC_API_KEY=your_api_key_here
# Run the evaluation
npx promptfoo@latest eval
# View results
npx promptfoo@latest view
Other providers
Opus 4.8 is also reachable through:
- AWS Bedrock —
bedrock:us.anthropic.claude-opus-4-8(orbedrock:converse:us.anthropic.claude-opus-4-8) - Google Vertex —
vertex:claude-opus-4-8withconfig.region: global - Azure AI Foundry — point
anthropic:messages:claude-opus-4-8athttps://<resource>.services.ai.azure.com/anthropicviaapiBaseUrl
Across all four providers, promptfoo automatically omits the unsupported sampling parameters (temperature, top_p, top_k) for Opus 4.8. The Anthropic Messages provider also logs a one-time warning if you set them explicitly; the Bedrock, Vertex, and Azure paths omit them silently.