promptfoo--promptfoo/site/docs/providers/opencode-sdk.md

sidebar_position, title, description

sidebar_position	title	description
42	OpenCode SDK	Use OpenCode SDK for evals with 75+ providers, built-in tools, and terminal-native AI agent

OpenCode SDK

This provider integrates OpenCode, an open-source AI coding agent for the terminal with support for 75+ LLM providers.

Provider IDs

• opencode:sdk - Uses OpenCode's configured model
• opencode - Same as opencode:sdk

The model is configured via the OpenCode CLI or ~/.opencode/config.yaml.

Installation

The OpenCode SDK provider requires both the OpenCode CLI and the SDK package.

1. Install OpenCode CLI

curl -fsSL https://opencode.ai/install | bash

Or via other package managers - see opencode.ai for options.

2. Install SDK Package

npm install @opencode-ai/sdk

:::note

Promptfoo treats the SDK package as an optional runtime dependency, so it only needs to be installed if you want to use the OpenCode SDK provider.

:::

Setup

Configure your LLM provider credentials. For Anthropic:

export ANTHROPIC_API_KEY=your_api_key_here

For OpenAI:

export OPENAI_API_KEY=your_api_key_here

If promptfoo starts the OpenCode server for you, you can also set config.apiKey together with config.provider_id in your provider config.

:::note

If you connect to an existing OpenCode server with baseUrl, that server is responsible for authentication, MCP setup, and custom agents. Promptfoo can still send per-request options like model, tools, format, and workspace, but it cannot reconfigure the remote server.

:::

OpenCode supports 75+ providers - see Supported Providers for the full list.

Quick Start

Basic Usage

Use opencode:sdk to access OpenCode's configured model:

providers:
  - opencode:sdk

prompts:
  - 'Write a Python function that validates email addresses'

Configure your model via the OpenCode CLI: opencode config set model openai/gpt-4o

By default, OpenCode SDK runs in a temporary directory with no tools enabled. When your test cases finish, the temporary directory is deleted.

With Inline Model Configuration

Specify the provider and model directly in your config:

providers:
  - id: opencode:sdk
    config:
      provider_id: anthropic
      model: claude-sonnet-4-20250514

prompts:
  - 'Write a Python function that validates email addresses'

This overrides the model configured via the OpenCode CLI for this specific eval.

With Working Directory

Specify a working directory to enable read-only file tools:

providers:
  - id: opencode:sdk
    config:
      working_dir: ./src

prompts:
  - 'Review the TypeScript files and identify potential bugs'

By default, when you specify a working directory, OpenCode SDK has access to these read-only tools: read, grep, glob, list. Relative working_dir values are resolved from the directory containing the config file.

Structured Output

Use the OpenCode format request option for JSON Schema-constrained responses:

providers:
  - id: opencode:sdk
    config:
      provider_id: openai
      model: gpt-4o-mini
      format:
        type: json_schema
        schema:
          type: object
          additionalProperties: false
          properties:
            summary:
              type: string
            severity:
              type: string
          required:
            - summary
            - severity

prompts:
  - 'Summarize the issue as JSON'

With Workspace

OpenCode workspace support lets you target a specific workspace-aware server context. This requires either working_dir or baseUrl:

providers:
  - id: opencode:sdk
    config:
      working_dir: ./repo
      workspace: feature-branch

prompts:
  - 'Review the files in this workspace'

With Full Tool Access

Enable additional tools for file modifications and shell access:

providers:
  - id: opencode:sdk
    config:
      working_dir: ./project
      tools:
        read: true
        grep: true
        glob: true
        list: true
        write: true
        edit: true
        bash: true
      permission:
        bash: allow
        edit: allow

:::warning

When enabling write/edit/bash tools, consider how you will reset files after each test. See Managing Side Effects.

:::

Supported Parameters

Parameter	Type	Description	Default
apiKey	string	Inject API key into a spawned OpenCode server for provider_id	Environment variable
baseUrl	string	URL for an existing OpenCode server	Auto-start server
hostname	string	Server hostname when starting a new server	127.0.0.1
port	number	Server port when starting a new server	Auto-select
timeout	number	Server startup timeout in milliseconds	30000
log_level	string	OpenCode server log level (debug, info, warn, error, off)	Provider default
working_dir	string	Directory for file operations and read-only default tools	Temporary directory
workspace	string	Workspace identifier for workspace-aware OpenCode requests	None
provider_id	string	LLM provider (anthropic, openai, google, ollama, etc.)	OpenCode default
model	string	Model to use for this request	OpenCode default
format	object	Output format, including JSON Schema structured output	Text
variant	string	Provider/model variant defined in OpenCode config	Default variant
tools	object	Tool configuration	Read-only with working_dir
permission	object	Permission configuration for tools	Ask for dangerous tools
agent	string	Built-in or preconfigured agent to use	Default agent
custom_agent	object	Custom agent configuration when promptfoo starts the OpenCode server	None
session_id	string	Resume an existing session	Create new session
parent_session_id	string	Fork from an existing session (v2 server only); inherits compacted history	None
persist_sessions	boolean	Reuse the same session for repeated calls with the same provider config	false
mcp	object	MCP server configuration when promptfoo starts the OpenCode server	None
cache_mcp	boolean	Enable caching when MCP is configured	false

Supported Providers

OpenCode supports 75+ LLM providers through Models.dev:

Cloud Providers:

• Anthropic (Claude)
• OpenAI
• Google AI Studio / Vertex AI
• Amazon Bedrock
• Azure OpenAI
• Groq
• Together AI
• Fireworks AI
• DeepSeek
• Perplexity
• Cohere
• Mistral
• And many more...

Local Models:

• Ollama
• LM Studio
• llama.cpp

Configure your preferred model using the OpenCode CLI:

# Set your default model
opencode config set model anthropic/claude-sonnet-4-20250514

# Or for OpenAI
opencode config set model openai/gpt-4o

# Or for local models
opencode config set model ollama/llama3

Tools and Permissions

Default Tools

With no working_dir specified, OpenCode runs in a temp directory with no tools.

With working_dir specified, these read-only tools are enabled by default:

Tool	Purpose
read	Read file contents
grep	Search file contents with regex
glob	Find files by pattern
list	List directory contents

All Available Tools

Tool	Purpose	Default
bash	Execute shell commands	false
edit	Modify existing files	false
write	Create/overwrite files	false
read	Read file contents	true*
grep	Search file contents with regex	true*
glob	Find files by pattern	true*
list	List directory contents	true*
patch	Apply diff patches	false
todowrite	Create task lists	false
todoread	Read task lists	false
webfetch	Fetch web content	false
question	Prompt user for input during execution	false
skill	Load SKILL.md files into conversation	false
lsp	Code intelligence queries (experimental)	false

* Only enabled when working_dir is specified.

Tool Configuration

Customize available tools:

# Enable additional tools
providers:
  - id: opencode:sdk
    config:
      working_dir: ./project
      tools:
        read: true
        grep: true
        glob: true
        list: true
        write: true # Enable file writing
        edit: true # Enable file editing
        bash: true # Enable shell commands
        patch: true # Enable patch application
        webfetch: true # Enable web fetching
        question: true # Enable user prompts
        skill: true # Enable SKILL.md loading

# Disable specific tools
providers:
  - id: opencode:sdk
    config:
      working_dir: ./project
      tools:
        bash: false # Disable shell

Permissions

Configure tool permissions using simple values or pattern-based rules:

# Simple permissions
providers:
  - id: opencode:sdk
    config:
      permission:
        bash: allow # or 'ask' or 'deny'
        edit: allow
        webfetch: deny
        doom_loop: deny # Prevent infinite agent loops
        external_directory: deny # Block access outside working dir

# Pattern-based permissions
providers:
  - id: opencode:sdk
    config:
      permission:
        bash:
          'git *': allow # Allow git commands
          'rm *': deny # Deny rm commands
          '*': ask # Ask for everything else
        edit:
          '*.md': allow # Allow editing markdown
          'src/**': ask # Ask for src directory

Permission	Purpose
bash	Shell command execution
edit	File editing
read	Reading files
glob	Finding files by pattern
grep	Searching file contents
list	Listing directories
task	Subtask execution
lsp	Code intelligence queries
skill	Loading SKILL.md files
webfetch	Web fetching
websearch	Web search
codesearch	Codebase search
todowrite	Writing to todo list
question	Interactive user prompts
doom_loop	Prevents infinite agent loops
external_directory	Access outside working directory

Additional tools added by future OpenCode releases can be configured using the same shape — unknown keys are forwarded unchanged.

:::note

Promptfoo converts the object form above into the PermissionRuleset array the OpenCode v2 API expects ([{ permission: "bash", pattern: "*", action: "allow" }, ...]). You configure permissions in the friendly object form and the provider handles the conversion per request.

:::

:::tip Security Recommendation

For security-conscious deployments, set doom_loop: deny and external_directory: deny to prevent infinite agent loops and restrict file access to the working directory.

:::

Skills

OpenCode loads Agent Skills through its native skill tool. Enable the tool for the eval, point working_dir at a repo that contains skills OpenCode can discover, and allow the skill permission if you want a non-interactive run:

providers:
  - id: opencode:sdk
    config:
      working_dir: ./project
      tools:
        skill: true
      permission:
        skill: allow

tests:
  - assert:
      - type: skill-used
        value: review-standards

Promptfoo normalizes OpenCode's native skill tool parts into response.metadata.skillCalls, so skill-used works the same way it does for Claude Agent SDK. Each normalized entry keeps the requested skill name and tool input, records tool failures with is_error: true, and includes the loaded SKILL.md path when OpenCode returns the skill directory in its result metadata. Those errored entries remain available for diagnostics, but they do not count as successful skill-used matches.

Session Management

Ephemeral Sessions (Default)

Creates a new session for each call and deletes it when the call completes:

providers:
  - opencode:sdk

Persistent Sessions

Reuse the same session between calls that use the same provider config:

providers:
  - id: opencode:sdk
    config:
      persist_sessions: true

This reuse is independent of the promptfoo response cache. It is scoped to the lifetime of the provider instance. If you need to continue a session later, capture its sessionId and pass it back with session_id.

Session Resumption

Resume a specific session:

providers:
  - id: opencode:sdk
    config:
      session_id: previous-session-id

Forked Sessions

Fork a new session off an existing one. The child inherits the parent's compacted history and starts as a fresh conversation. Requires the v2 OpenCode server (silently ignored on v1):

providers:
  - id: opencode:sdk
    config:
      parent_session_id: parent-session-id

parent_session_id is independent of session_id: use session_id to continue the same conversation, or parent_session_id to branch a new one. Setting both is supported, but session_id wins because resumed sessions are not re-forked on create.

Custom Agents

Define custom agents with specific configurations:

providers:
  - id: opencode:sdk
    config:
      custom_agent:
        description: Security-focused code reviewer
        mode: primary # 'primary', 'subagent', or 'all'
        model: claude-sonnet-4-20250514
        temperature: 0.3
        top_p: 0.9 # Nucleus sampling parameter
        steps: 10 # Max iterations before text-only response
        color: '#ff5500' # Visual identification
        tools:
          read: true
          grep: true
          write: false
          bash: false
        permission:
          edit: deny
          external_directory: deny
        prompt: |
          You are a security-focused code reviewer.
          Analyze code for vulnerabilities and report findings.

custom_agent is applied when promptfoo starts the OpenCode server itself. If you use baseUrl, define that agent on the target server and use agent to select it.

Parameter	Type	Description
description	string	Required. Explains the agent's purpose
mode	string	'primary', 'subagent', or 'all'
model	string	Model ID (overrides global)
temperature	number	Response randomness (0.0-1.0)
top_p	number	Nucleus sampling (0.0-1.0)
steps	number	Max iterations before text-only response
color	string	Hex color for visual identification
tools	object	Tool configuration
permission	object	Permission configuration
prompt	string	Custom system prompt
disable	boolean	Disable this agent
hidden	boolean	Hide from @ autocomplete (subagents only)

MCP Integration

OpenCode supports MCP (Model Context Protocol) servers:

providers:
  - id: opencode:sdk
    config:
      mcp:
        # Local MCP server
        weather-server:
          type: local
          command: ['node', 'mcp-weather-server.js']
          environment:
            API_KEY: '{{env.WEATHER_API_KEY}}'
          timeout: 30000
          enabled: true

        # Remote MCP server with headers
        api-server:
          type: remote
          url: https://api.example.com/mcp
          headers:
            Authorization: 'Bearer {{env.API_TOKEN}}'

        # Remote MCP server with OAuth
        oauth-server:
          type: remote
          url: https://secure.example.com/mcp
          oauth:
            clientId: '{{env.OAUTH_CLIENT_ID}}'
            clientSecret: '{{env.OAUTH_CLIENT_SECRET}}'
            scope: 'read write'

Like custom_agent, mcp is server configuration. It applies when promptfoo starts the OpenCode server, not when you connect to an already-running server with baseUrl.

Caching Behavior

This provider automatically caches responses based on:

• Prompt content
• Working directory fingerprint (if specified)
• Workspace and output format configuration
• Provider and model configuration
• Tool configuration

When MCP servers are configured, caching is disabled by default because MCP tools typically interact with external state. To opt back into caching for deterministic MCP tools, set cache_mcp: true:

providers:
  - id: opencode:sdk
    config:
      cache_mcp: true
      mcp:
        my-server:
          type: local
          command: ['node', 'my-deterministic-mcp-server.js']

To disable caching:

export PROMPTFOO_CACHE_ENABLED=false

To bust the cache for a specific test:

tests:
  - vars: {}
    options:
      bustCache: true

Managing Side Effects

When using tools that allow side effects (write, edit, bash), consider:

• Serial execution: Set evaluateOptions.maxConcurrency: 1 to prevent race conditions
• Git reset: Use git to reset files after each test
• Extension hooks: Use promptfoo hooks for setup/cleanup
• Containers: Run tests in containers for isolation

Example with serial execution:

providers:
  - id: opencode:sdk
    config:
      working_dir: ./project
      tools:
        write: true
        edit: true

evaluateOptions:
  maxConcurrency: 1

Comparison with Other Agentic Providers

Feature	OpenCode SDK	Claude Agent SDK	Codex SDK
Provider flexibility	75+ providers	Anthropic only	OpenAI only
Architecture	Client-server	Direct API	Thread-based
Local models	Ollama, LM Studio	No	No
Tool ecosystem	Native + MCP	Native + MCP	Native
Working dir isolation	Yes	Yes	Git required

Choose based on your use case:

• Multiple providers / local models → OpenCode SDK
• Anthropic-specific features → Claude Agent SDK
• OpenAI-specific features → Codex SDK

Examples

See the examples directory for complete implementations:

• Basic usage - Simple chat-only mode
• Working directory - Read-only local file access
• Structured output - JSON Schema-constrained responses

See Also

• OpenCode Documentation
• OpenCode SDK Reference
• Claude Agent SDK Provider - Alternative agentic provider
• OpenAI Codex SDK Provider - Alternative agentic provider