wehub-resource-sync
0d3cb498a3
anthropic/claude-code-session (Authenticate via Claude Code session)
This example shows how to run Promptfoo evals against the Anthropic Messages
API — including llm-rubric model-graded assertions — by reusing an existing
local Claude Code session instead of creating a separate Anthropic Console API
key.
It's meant for Claude Pro / Max subscribers who already use the
claude CLI on the
same machine.
You can run this example with:
npx promptfoo@latest init --example anthropic/claude-code-session
cd anthropic/claude-code-session
Prerequisites
-
Install and log in to Claude Code:
claude /loginPromptfoo reads the OAuth credential that Claude Code stores in either the macOS keychain (
Claude Code-credentials) or$HOME/.claude/.credentials.json. -
Make sure
ANTHROPIC_API_KEYis unset if you specifically want Promptfoo to use your Claude Code session. If the env var is set, Promptfoo will prefer it over the OAuth credential.
How it works
The provider config sets apiKeyRequired: false, which tells Promptfoo to:
- Skip its upfront API key check.
- Load the Claude Code OAuth credential from the local session.
- Authenticate Messages API requests with a Bearer token plus the
claude-code-20250219,oauth-2025-04-20beta headers. - Prepend the required Claude Code identity system block
(
"You are Claude Code, Anthropic's official CLI for Claude.") before your own system prompt.
Requests made this way are expected to count against your Claude
subscription the same way calls from the claude CLI do. Check
Anthropic's documentation
for current billing behavior.
Run it
promptfoo eval
promptfoo view
Configuration
See promptfooconfig.yaml — both the main provider and the llm-rubric
grader point at anthropic:messages:claude-sonnet-4-6 with
apiKeyRequired: false so the entire eval runs on your Claude subscription.