567-labs--instructor/instructor/v2/README.md

V2 Core Architecture

This document covers the v2 core infrastructure, including the registry-based design, exception handling, and component interactions.

Overview

The v2 architecture uses a hierarchical registry system for managing provider modes and their corresponding handlers. It replaces the monolithic v1 approach with modular, composable components:

• Registry: Central mode/handler management
• Handlers: Pluggable request/response/reask handlers per mode
• Patch: Unified function patching mechanism
• Retry: Intelligent retry with registry-based handling
• Exceptions: Organized, centralized error handling

Ownership Rules

• Provider request preparation, response parsing, reask formatting, and wire-format helpers belong under instructor/v2/providers/<provider>/.
• Shared core modules should contain provider-agnostic primitives only.
• ResponseSchema.parse_* helpers are deprecated compatibility shims that delegate into the registry; they are not homes for new provider behavior.
• ResponseSchema.openai_schema, .anthropic_schema, and .gemini_schema are compatibility shims too. Provider wire-format builders live with their provider packages; shared schema exports only forward to them.
• Provider-specific templating and usage setup belong with provider modules; shared orchestration should only dispatch into them.
• Shared multimodal models keep the public compatibility methods, but provider wire-format encoders live with provider modules.
• Shared routing still owns provider detection, compatibility-mode normalization, and registry bootstrap tables; those are orchestration surfaces rather than provider implementations.
• Provider capabilities, public factory bindings, handler modules, and alias relationships live in core/provider_specs.py; tests and runtime dispatch should consume that manifest instead of maintaining parallel provider tables.
• Public modules under instructor/core, instructor/processing, instructor/dsl, and instructor/validation are compatibility facades over v2-owned implementations.

Core Components

Protocols (instructor/v2/core/protocols.py)

Type-safe interfaces for handlers:

• RequestHandler - Prepares request kwargs for a mode
• ResponseParser - Parses API response into Pydantic model
• ReaskHandler - Handles validation failures for retry
• StreamExtractor - Extracts JSON chunks from streaming responses
• AsyncStreamExtractor - Async version of the stream extractor
• MessageConverter - Converts multimodal messages for a provider
• TemplateHandler - Applies template context to provider payloads

Mode Registry (instructor/v2/core/registry.py)

The mode registry manages all available modes for each provider. It maps (Provider, Mode) tuples to their handler implementations.

Key Features:

• Provider/mode combination lookup
• Handler registration and retrieval
• Mode listing and discovery
• Fast O(1) lookups for handler dispatch

Registry API:

from instructor.v2.core.registry import mode_registry
from instructor import Provider, Mode

# Get handlers (preferred)
handlers = mode_registry.get_handlers(Provider.ANTHROPIC, Mode.TOOLS)

# Query
modes = mode_registry.get_modes_for_provider(Provider.ANTHROPIC)
is_registered = mode_registry.is_registered(Provider.ANTHROPIC, Mode.TOOLS)

Handlers are registered via @register_mode_handler decorator (see Handler Registration).

Patch Mechanism (instructor/v2/core/patch.py)

Wraps provider API functions to add structured output support. Auto-detects sync/async, validates mode registration, injects default models, and integrates with registry handlers.

from instructor.v2.core.patch import patch_v2

patched_create = patch_v2(
    client.messages.create,
    provider=Provider.ANTHROPIC,
    mode=Mode.TOOLS,
    default_model="claude-3-5-sonnet-20241022"
)

Retry Logic (instructor/v2/core/retry.py)

Handles retries with registry-based reask logic. On ValidationError, uses registry handlers to generate reask prompts and retries up to max_retries times.

Exception Handling

V2 exceptions inherit from instructor.core.exceptions.InstructorError:

• RegistryError - Mode not registered or handler lookup failure
• ValidationContextError - Conflicting context/validation_context parameters
• InstructorRetryException - Max retries exceeded with full attempt context

RegistryValidationMixin provides validation utilities used internally.

Handler System

Handlers are pluggable components that implement provider-specific logic. They can be implemented as classes (using ModeHandler ABC) or as standalone functions (using Protocols).

Handler Base Class (instructor/v2/core/handler.py)

The ModeHandler abstract base class provides a structured way to implement handlers:

from instructor.v2.core.handler import ModeHandler
from pydantic import BaseModel
from typing import Any

class MyModeHandler(ModeHandler):
    """Handler for a specific mode."""

    def prepare_request(
        self,
        response_model: type[BaseModel] | None,
        kwargs: dict[str, Any],
    ) -> tuple[type[BaseModel] | None, dict[str, Any]]:
        """Prepare request kwargs for this mode."""
        # Modify kwargs for mode-specific requirements
        return response_model, kwargs

    def handle_reask(
        self,
        kwargs: dict[str, Any],
        response: Any,
        exception: Exception,
    ) -> dict[str, Any]:
        """Handle validation failure and prepare retry."""
        # Modify kwargs for retry attempt
        return kwargs

    def parse_response(
        self,
        response: Any,
        response_model: type[BaseModel],
        validation_context: dict[str, Any] | None = None,
        strict: bool | None = None,
    ) -> BaseModel:
        """Parse API response into validated Pydantic model."""
        # Extract and validate response
        return response_model.model_validate(...)

Handler Registration

All handlers must be registered using the @register_mode_handler decorator. This is the only supported way to register handlers in v2.

from instructor.v2.core.decorators import register_mode_handler
from instructor import Provider, Mode
from instructor.v2.core.handler import ModeHandler

@register_mode_handler(Provider.ANTHROPIC, Mode.TOOLS)
class AnthropicToolsHandler(ModeHandler):
    """Handler automatically registered on import.

    The decorator internally calls mode_registry.register() with the
    handler methods mapped to the protocol functions.
    """

    def prepare_request(self, response_model, kwargs):
        # Implementation
        return response_model, kwargs

    def handle_reask(self, kwargs, response, exception):
        # Implementation
        return kwargs

    def parse_response(self, response, response_model, **kwargs):
        # Implementation
        return response_model.model_validate(...)

How it works: The decorator instantiates the handler class and calls mode_registry.register() with the handler's methods mapped to the protocol functions:

• handler.prepare_request → request_handler
• handler.handle_reask → reask_handler
• handler.parse_response → response_parser

Benefits:

• Automatic registration on import (no manual calls needed)
• Clean, declarative syntax
• Type-safe and consistent with the codebase pattern
• Used by all v2 providers (see instructor/v2/providers/anthropic/handlers.py)

Important: Direct calls to mode_registry.register() are not supported. All handlers must use the @register_mode_handler decorator.

Execution Flow

Sync Execution Path

Client.create() with response_model
  ↓
patch_v2() [registry validation]
  ↓
new_create_sync()
  ├─ handle_context() [parameter validation]
  └─ retry_sync_v2() [retry logic]
      ├─ validate_mode_registration()
      ├─ For each attempt:
      │  ├─ Call original API
      │  ├─ Get handlers from registry
      │  ├─ Parse response via handler
      │  ├─ On success → return
      │  └─ On ValidationError:
      │     ├─ Record attempt
      │     ├─ Get reask via handler
      │     └─ Retry
      └─ Max retries exceeded → InstructorRetryException

Async Execution Path

AsyncClient.create() with response_model
  ↓
patch_v2() [registry validation]
  ↓
new_create_async()
  ├─ handle_context() [parameter validation]
  └─ retry_async_v2() [async retry logic]
      ├─ validate_mode_registration()
      ├─ For each attempt:
      │  ├─ Await API call
      │  ├─ Get handlers from registry
      │  ├─ Parse response via handler
      │  ├─ On success → return
      │  └─ On ValidationError:
      │     ├─ Record attempt
      │     ├─ Get reask via handler
      │     └─ Retry
      └─ Max retries exceeded → InstructorRetryException

Error Handling Strategy

• Fail fast: Mode validation at patch time
• Context validation: context/validation_context conflict detection
• Comprehensive logging: All stages logged with attempt numbers
• Exception chaining: Full context preserved in exception chain

Configuration

• Mode: Specified when creating client (from_anthropic(client, mode=Mode.TOOLS))
• Default Model: Injected via patch_v2(..., default_model="...") if not provided in request
• Max Retries: Per-request via max_retries=3 or Retrying(...) instance

Adding a New Provider

1. Add Provider Enum (instructor/utils.py):

class Provider(Enum):
    YOUR_PROVIDER = "your_provider"

2. Create Handler (instructor/v2/providers/your_provider/handlers.py):

from instructor.v2.core.handler import ModeHandler
from instructor.v2.core.decorators import register_mode_handler
from instructor import Provider, Mode

@register_mode_handler(Provider.YOUR_PROVIDER, Mode.TOOLS)
class YourProviderToolsHandler(ModeHandler):
    def prepare_request(self, response_model, kwargs):
        # Convert response_model to provider tools format
        return response_model, kwargs

    def parse_response(self, response, response_model, **kwargs):
        # Extract and validate response
        return response_model.model_validate(...)

    def handle_reask(self, kwargs, response, exception):
        # Add error message for retry
        return kwargs

3. Create Factory (instructor/v2/providers/your_provider/client.py):

from instructor.v2.providers.your_provider import handlers  # noqa: F401
from instructor.v2.core.patch import patch_v2
from instructor import Instructor, AsyncInstructor, Mode, Provider

@overload
def from_your_provider(client: YourProviderClient, mode=Mode.TOOLS) -> Instructor: ...

def from_your_provider(client, mode=Mode.TOOLS):
    patched_create = patch_v2(
        client.messages.create,
        provider=Provider.YOUR_PROVIDER,
        mode=mode,
    )
    return Instructor(client=client, create=patched_create, mode=mode)

4. Export (instructor/v2/providers/your_provider/__init__.py):

from . import handlers  # noqa: F401
from .client import from_your_provider
__all__ = ["from_your_provider"]

See instructor/v2/providers/anthropic/ for a complete example.

Comprehensive Migration Guide: V1 to V2

This guide walks through migrating a provider from v1 to v2 architecture.

Understanding V1 vs V2 Architecture

Compatibility Architecture:

• Legacy public entry points remain as thin shims for free upgrades.
• Compatibility methods delegate into the v2 registry instead of owning live provider logic.
• Legacy mode enums are still accepted and normalized in the registry.

V2 Architecture:

• Centralized registry-based handler system
• Pluggable handlers per provider/mode combination
• Compile-time mode validation
• Generic mode enums only (e.g., TOOLS, JSON)

Step-by-Step Migration Process

Step 1: Analyze Your V1 Implementation

Before migrating, understand your current v1 provider:

1. Locate provider files:

   • instructor/v2/providers/<provider>/client.py - provider factories
   • instructor/v2/providers/<provider>/handlers.py - request, response, and reask behavior
   • instructor/v2/auto_client.py - unified from_provider() routing
   • instructor/v2/core/client.py - shared client wrapper
   • instructor/v2/core/patch.py / instructor/v2/core/retry.py - shared orchestration
   • instructor/core/*, instructor/processing/*, instructor/dsl/*, instructor/validation/* - compatibility facades over v2 internals

   Current migration footprint: provider-specific runtime behavior belongs in instructor/v2/providers/*. The remaining non-v2 modules are shared public wrappers or compatibility facades, not parallel provider implementations. Provider-shaped schema builders follow the same rule: live implementations stay with the matching provider package, while shared exports remain forwarding compatibility APIs. from_provider() keeps the public routing entrypoint in instructor/v2/auto_client.py, but dispatch is table-driven and every accepted alias must have an explicit builder.

   Typing contract: public factories should infer sync versus async clients from concrete client classes or literal flags, and response helpers should preserve the caller's response-model type. Use create_iterable() and create_partial() when you want precise streaming inference; direct create(response_model=Iterable[...]) and create(response_model=Partial[...]) remain compatibility forms. Keep the executable assertions in tests/typing/test_public_surface.py up to date when changing public APIs.

2. Identify key components:

   • What modes does your provider support?
   • What's the main API function being patched? (e.g., client.chat, client.messages.create)
   • How does request preparation work? (converting response_model to provider format)
   • How does response parsing work? (extracting structured data from raw response)
   • How does reask/retry work? (handling validation failures)

3. Example V1 structure (from instructor/core/client.py):

# V1: Factory normalizes mode + applies patching
def from_openai(client, mode=Mode.TOOLS, **kwargs):
    normalized_mode = normalize_mode_for_provider(mode, Provider.OPENAI)

    # Uses instructor.patch() which delegates to the registry handlers
    return Instructor(
        client=client,
        create=instructor.patch(
            create=client.chat.completions.create,
            mode=normalized_mode,
        ),
        provider=Provider.OPENAI,
        mode=normalized_mode,
        **kwargs,
    )

Step 2: Create V2 Provider Directory Structure

Create the v2 provider directory:

mkdir -p instructor/v2/providers/your_provider
touch instructor/v2/providers/your_provider/__init__.py
touch instructor/v2/providers/your_provider/client.py
touch instructor/v2/providers/your_provider/handlers.py

Step 3: Map V1 Modes to V2 Modes

Determine which generic v2 modes your provider supports:

• Mode.TOOLS - Function calling / tool use
• Mode.JSON - JSON mode with schema instructions
• Mode.JSON_SCHEMA - Native structured outputs (if supported)
• Mode.PARALLEL_TOOLS - Parallel tool calling (if supported) Provider-specific legacy modes are deprecated in v2. They emit warnings and normalize to generic modes. Use the generic modes directly.

Step 4: Extract Handler Logic from V1

Identify the three handler methods needed:

1. Request Preparation (prepare_request):

   • Look for functions like handle_cohere_modes(), handle_anthropic_json()
   • These convert response_model to provider-specific format
   • Modify request kwargs (e.g., add tools parameter)

2. Response Parsing (parse_response):

   • Look for functions in v1 utils or response parsers used by the registry
   • Extract structured data from raw API response
   • Validate against response_model using Pydantic

3. Reask Handling (handle_reask):

   • Look for functions like reask_cohere_tools(), reask_anthropic_json()
   • Modify kwargs to include error context for retry

Example V1 handler functions (from instructor/processing/function_calls.py and instructor/processing/response.py):

# Compatibility: deprecated helper delegates through the registry
@classmethod
def parse_cohere_json_schema(cls, completion, validation_context=None, strict=None):
    return cls._parse_with_registry(
        completion,
        mode=Mode.JSON_SCHEMA,
        provider=Provider.COHERE,
        validation_context=validation_context,
        strict=strict,
    )

def handle_reask_kwargs(kwargs, mode, response, exception, provider=Provider.OPENAI):
    # Dispatch to provider-specific reask handler for retries
    return handlers.reask_handler(kwargs, response, exception)

Step 5: Implement V2 Handlers

Create handler classes using the @register_mode_handler decorator:

# instructor/v2/providers/your_provider/handlers.py
from instructor.v2.core.handler import ModeHandler
from instructor.v2.core.decorators import register_mode_handler
from instructor import Provider, Mode
from pydantic import BaseModel
from typing import Any

@register_mode_handler(Provider.COHERE, Mode.TOOLS)
class CohereToolsHandler(ModeHandler):
    """Handler for Cohere TOOLS mode."""

    def prepare_request(
        self,
        response_model: type[BaseModel] | None,
        kwargs: dict[str, Any],
    ) -> tuple[type[BaseModel] | None, dict[str, Any]]:
        """Convert response_model to Cohere tools format."""
        if response_model is None:
            return None, kwargs

        # Convert response_model to Cohere function/tool format
        # (extract logic from v1 handle_cohere_modes)
        tool_schema = convert_to_cohere_tools(response_model)
        kwargs["tools"] = [tool_schema]

        return response_model, kwargs

    def parse_response(
        self,
        response: Any,
        response_model: type[BaseModel],
        validation_context: dict[str, Any] | None = None,
        strict: bool | None = None,
    ) -> BaseModel:
        """Extract and validate structured data from Cohere response."""
        # Extract logic from v1 handlers or utils
        tool_calls = response.tool_calls or []
        if not tool_calls:
            raise ValueError("No tool calls in response")

        # Parse first tool call
        tool_call = tool_calls[0]
        return response_model.model_validate_json(
            tool_call.parameters,
            context=validation_context,
            strict=strict,
        )

    def handle_reask(
        self,
        kwargs: dict[str, Any],
        response: Any,
        exception: Exception,
    ) -> dict[str, Any]:
        """Handle validation failure and prepare retry."""
        # Extract logic from v1 reask_cohere_tools
        kwargs = kwargs.copy()
        error_msg = f"Validation Error: {exception}\nPlease fix and retry."
        kwargs["messages"].append({"role": "user", "content": error_msg})
        return kwargs

@register_mode_handler(Provider.COHERE, Mode.JSON)
class CohereJSONHandler(ModeHandler):
    """Handler for Cohere JSON mode."""
    # Similar structure for JSON mode
    ...

Key Migration Patterns:

1. Request Preparation: Move logic from handle_*_modes() functions
2. Response Parsing: Extract from v1 handlers or response utils
3. Reask Handling: Move from reask_*() functions
4. Error Handling: Use Pydantic ValidationError for retries

Step 6: Create V2 Factory Function

Create the factory function using patch_v2:

# instructor/v2/providers/your_provider/client.py
from instructor.v2.core.patch import patch_v2
from instructor import Instructor, AsyncInstructor, Mode, Provider
from instructor.v2.core.registry import mode_registry
from typing import overload, Any

# Ensure handlers are registered (import triggers decorators)
from . import handlers  # noqa: F401

@overload
def from_cohere(
    client: cohere.Client,
    mode: Mode = Mode.TOOLS,
    **kwargs: Any,
) -> Instructor: ...

@overload
def from_cohere(
    client: cohere.AsyncClient,
    mode: Mode = Mode.TOOLS,
    **kwargs: Any,
) -> AsyncInstructor: ...

def from_cohere(
    client: cohere.Client | cohere.AsyncClient,
    mode: Mode = Mode.TOOLS,
    **kwargs: Any,
) -> Instructor | AsyncInstructor:
    """Create v2 Instructor instance from Cohere client.

    Args:
        client: Cohere client instance (sync or async)
        mode: Mode to use (defaults to Mode.TOOLS)
        **kwargs: Additional kwargs for Instructor constructor

    Returns:
        Instructor instance (sync or async)
    """
    # Validate mode is registered
    if not mode_registry.is_registered(Provider.COHERE, mode):
        from instructor.v2.core.errors import ModeError
        available_modes = mode_registry.get_modes_for_provider(Provider.COHERE)
        raise ModeError(
            mode=mode.value,
            provider=Provider.COHERE.value,
            valid_modes=[m.value for m in available_modes],
        )

    # Determine sync/async
    is_async = isinstance(client, cohere.AsyncClient)

    # Get the API function to patch
    create_func = client.chat

    # Patch using v2 registry
    patched_create = patch_v2(
        func=create_func,
        provider=Provider.COHERE,
        mode=mode,
    )

    # Return appropriate instructor type
    if is_async:
        return AsyncInstructor(
            client=client,
            create=patched_create,
            provider=Provider.COHERE,
            mode=mode,
            **kwargs,
        )
    else:
        return Instructor(
            client=client,
            create=patched_create,
            provider=Provider.COHERE,
            mode=mode,
            **kwargs,
        )

Key Differences from V1:

• Uses patch_v2() instead of instructor.patch()
• Validates mode registration via registry
• Uses generic Mode enum values

Step 7: Export Provider

Update __init__.py to export the factory:

# instructor/v2/providers/your_provider/__init__.py
from . import handlers  # noqa: F401 - triggers registration
from .client import from_cohere

__all__ = ["from_cohere"]

Update main v2 exports:

# instructor/v2/__init__.py
try:
    from instructor.v2.providers.cohere import from_cohere
except ImportError:
    from_cohere = None  # type: ignore

__all__ = [
    # ... existing exports ...
    "from_cohere",
]

Step 8: Write Comprehensive Tests

Create tests following the testing guide (see "Testing Guide" section):

# tests/v2/test_cohere_provider.py
import pytest
from pydantic import BaseModel
from instructor import Mode
from instructor.v2 import Provider, mode_registry

class TestModel(BaseModel):
    value: str

def test_mode_registration():
    """Verify modes are registered."""
    assert mode_registry.is_registered(Provider.COHERE, Mode.TOOLS)
    assert mode_registry.is_registered(Provider.COHERE, Mode.JSON)

@pytest.mark.requires_api_key
def test_basic_extraction():
    """Test end-to-end extraction."""
    from instructor.v2.providers.cohere import from_cohere
    import cohere

    client = cohere.Client(api_key="...")
    instructor_client = from_cohere(client, mode=Mode.TOOLS)

    result = instructor_client.create(
        response_model=TestModel,
        messages=[{"role": "user", "content": "Return value='test'"}],
    )

    assert isinstance(result, TestModel)
    assert result.value == "test"

Step 9: Update Integration Points

1. Update from_provider() routing (if applicable):

   • Ensure instructor.from_provider("cohere/model") routes to v2

2. Add deprecation warnings to v1 entry points:

   # instructor/core/client.py
   def from_openai(...):
       warnings.warn(
           "from_openai() is deprecated. Use instructor.v2.providers.openai.from_openai()",
           DeprecationWarning,
           stacklevel=2,
       )
       # ... existing v1 code ...
   

3. Update documentation:

   • Add provider to migration checklist
   • Update examples to use v2

Common Migration Patterns

Pattern 1: Simple Provider (No Custom Utils)

V1: Provider uses standard instructor.patch() with minimal customization.

V2: Create handlers that delegate to standard processing:

@register_mode_handler(Provider.SIMPLE, Mode.TOOLS)
class SimpleToolsHandler(ModeHandler):
    def prepare_request(self, response_model, kwargs):
        # Minimal customization
        return response_model, kwargs

    def parse_response(self, response, response_model, **kwargs):
        # Use standard parsing
        return response_model.model_validate(response.data)

    def handle_reask(self, kwargs, response, exception):
        # Standard reask pattern
        kwargs["messages"].append({
            "role": "user",
            "content": f"Error: {exception}. Please fix."
        })
        return kwargs

Pattern 2: Provider with Complex Utils

V1: Provider has extensive utility functions in utils.py.

V2: Import and adapt existing utilities:

from instructor.v2.providers.cohere import handlers as cohere_handlers

@register_mode_handler(Provider.COHERE, Mode.JSON)
class CohereJSONHandler(ModeHandler):
    def prepare_request(self, response_model, kwargs):
        # Reuse v1 utility function
        return cohere_utils.handle_cohere_json_schema(response_model, kwargs)

    def handle_reask(self, kwargs, response, exception):
        # Reuse v1 reask function
        return cohere_utils.reask_cohere_tools(kwargs, response, exception)

Pattern 3: Provider with Multiple API Functions

V1: Provider patches different functions based on client type.

V2: Handle in factory function:

def from_provider(client, mode=Mode.TOOLS):
    # Determine which function to patch
    if isinstance(client, SyncClient):
        create_func = client.chat
    elif isinstance(client, AsyncClient):
        create_func = client.chat_async
    else:
        raise ClientError("Invalid client type")

    patched_create = patch_v2(
        func=create_func,
        provider=Provider.YOUR_PROVIDER,
        mode=mode,
    )
    # ...

Pattern 4: Provider with Streaming Support

V1: Streaming handled by v1 DSL helpers and registry handlers.

V2: Check for streaming in handler:

from collections.abc import Generator, Iterable
from typing import Any

from instructor.v2.core.handler import ModeHandler


class ProviderToolsHandler(ModeHandler):
    def prepare_request(self, response_model, kwargs):
        # Register streaming model if stream=True
        if kwargs.get("stream") and response_model:
            self._streaming_models[response_model] = None
        return response_model, kwargs

    def extract_streaming_json(
        self, completion: Iterable[Any]
    ) -> Generator[str, None, None]:
        # Yield JSON chunks from the provider stream
        for chunk in completion:
            yield chunk.delta.text

    def parse_response(self, response, response_model, **kwargs):
        # Check if this is a streaming response
        if response_model in self._streaming_models:
            return response_model.from_streaming_response(
                response,
                stream_extractor=self.extract_streaming_json,
            )
        # Normal parsing
        return response_model.model_validate(...)

Migration Checklist

Use this checklist when migrating a provider:

Pre-Migration:

• Understand v1 implementation structure
• Identify all supported modes
• Map v1 modes to v2 generic modes
• Identify request preparation logic
• Identify response parsing logic
• Identify reask/retry logic

Implementation:

• Create v2 provider directory structure
• Implement handler classes with @register_mode_handler
• Implement prepare_request() method
• Implement parse_response() method
• Implement handle_reask() method
• Create factory function using patch_v2()
• Add proper type hints and overloads
• Export provider in __init__.py

Testing:

• Test mode registration
• Test basic extraction (sync)
• Test basic extraction (async)
• Test all supported modes
• Test error handling
• Test retry logic
• Test streaming (if applicable)
• Test edge cases

Integration:

• Update from_provider() routing (if needed)
• Add deprecation warnings to v1 factory
• Update migration checklist in README
• Update documentation
• Verify backward compatibility

Post-Migration:

• Monitor for issues
• Collect user feedback
• Plan v1 deprecation timeline

Troubleshooting Common Issues

Issue: Mode not found in registry

• Solution: Ensure handlers module is imported before using factory (use # noqa: F401 import)

Issue: Handler methods not being called

• Solution: Verify @register_mode_handler decorator is applied correctly and module is imported

Issue: Provider-specific modes not working

• Solution: Use v2 generic modes only (legacy modes are deprecated and normalize with warnings)

Issue: Tests failing with import errors

• Solution: Ensure provider handlers are imported in test files or use from . import handlers

Issue: Async client not working

• Solution: Verify is_async() check and use AsyncInstructor for async clients

Migration Example: Complete Cohere Migration

See instructor/v2/providers/anthropic/ and instructor/v2/providers/genai/ for complete reference implementations.

Key Differences Summary

Aspect	V1	V2
Mode Handling	Registry adapters in v1	Registry-based handler lookup
Mode Validation	Runtime (in factory function)	Compile-time (in patch_v2)
Handler Organization	Scattered utility functions	Centralized handler classes
Mode Enums	Provider-specific (ANTHROPIC_TOOLS)	Generic (TOOLS, JSON, JSON_SCHEMA)
Registration	Manual function calls	Decorator-based auto-registration
Testing	Test entire flow	Test handlers independently

V1 code continues to work during transition period, but new code should use v2.

How the System Works

Request Flow

When a user calls client.create(response_model=MyModel, ...), the following happens:

1. Patch Time (patch_v2):

   • Validates that the mode is registered for the provider
   • Creates a wrapper function that intercepts calls
   • Injects default model if provided

2. Request Preparation (prepare_request):

   • Handler receives response_model and request kwargs
   • Converts response_model to provider-specific format (e.g., tools schema for TOOLS mode)
   • Modifies kwargs to include provider-specific parameters
   • Returns modified response_model and kwargs

3. API Call:

   • Original provider API function is called with modified kwargs
   • Returns raw provider response object

4. Response Parsing (parse_response):

   • Handler extracts structured data from raw response
   • Validates against response_model using Pydantic
   • Returns validated Pydantic model instance

5. Retry on Failure (handle_reask):

   • If validation fails, handler modifies kwargs with error context
   • Retry logic calls API again with updated kwargs
   • Process repeats up to max_retries times

Mode Usage

V2 expects generic modes (e.g., Mode.TOOLS, Mode.JSON, Mode.JSON_SCHEMA). Provider-specific legacy modes are normalized with deprecation warnings.

Handler Lifecycle

1. Registration: Handler classes decorated with @register_mode_handler are instantiated and registered when the module is imported
2. Lookup: When a request is made, handlers are retrieved from the registry using (Provider, Mode) tuple
3. Execution: Handler methods are called during request preparation, response parsing, and retry handling
4. Caching: Handlers are cached in the registry after first lookup for performance

Registry Internals

The registry stores handlers in a dictionary keyed by (Provider, Mode) tuples:

{
    (Provider.ANTHROPIC, Mode.TOOLS): ModeHandlers(...),
    (Provider.ANTHROPIC, Mode.JSON): ModeHandlers(...),
    (Provider.GENAI, Mode.TOOLS): ModeHandlers(...),
    ...
}

Each ModeHandlers object contains:

• request_handler: Function to prepare request kwargs
• reask_handler: Function to handle validation failures
• response_parser: Function to parse API responses

Testing Guide

Writing Tests for V2 Providers

Tests for v2 providers should verify:

1. Mode registration in the registry
2. Handler functionality (request preparation, response parsing, reask handling)
3. End-to-end extraction with real API calls
4. Error handling and retry logic

Test Structure

Create tests in tests/v2/ directory following this pattern:

"""Tests for YourProvider v2 implementation."""

import pytest
from pydantic import BaseModel
from instructor import Mode
from instructor.v2 import Provider, mode_registry

class SimpleModel(BaseModel):
    """Simple test model."""
    value: str

# Test mode registration
def test_mode_is_registered():
    """Verify mode is registered in the v2 registry."""
    assert mode_registry.is_registered(Provider.YOUR_PROVIDER, Mode.TOOLS)

    handlers = mode_registry.get_handlers(Provider.YOUR_PROVIDER, Mode.TOOLS)
    assert handlers.request_handler is not None
    assert handlers.reask_handler is not None
    assert handlers.response_parser is not None

# Test basic extraction
@pytest.mark.requires_api_key
def test_basic_extraction():
    """Test basic extraction with real API call."""
    from instructor.v2.providers.your_provider import from_your_provider
    from your_provider_sdk import Client

    client = Client(api_key="...")
    instructor_client = from_your_provider(client, mode=Mode.TOOLS)

    result = instructor_client.create(
        response_model=SimpleModel,
        messages=[{"role": "user", "content": "Return value='test'"}],
    )

    assert isinstance(result, SimpleModel)
    assert result.value == "test"

# Test async extraction
@pytest.mark.asyncio
@pytest.mark.requires_api_key
async def test_async_extraction():
    """Test async extraction."""
    from instructor.v2.providers.your_provider import from_your_provider
    from your_provider_sdk import AsyncClient

    client = AsyncClient(api_key="...")
    instructor_client = from_your_provider(client, mode=Mode.TOOLS)

    result = await instructor_client.create(
        response_model=SimpleModel,
        messages=[{"role": "user", "content": "Return value='async'"}],
    )

    assert isinstance(result, SimpleModel)
    assert result.value == "async"

Parametrized Tests

Use pytest parametrization to test multiple modes:

@pytest.mark.parametrize(
    "provider,mode",
    [
        (Provider.YOUR_PROVIDER, Mode.TOOLS),
        (Provider.YOUR_PROVIDER, Mode.JSON),
    ],
)
@pytest.mark.requires_api_key
def test_all_modes(provider: Provider, mode: Mode):
    """Test all registered modes."""
    # Test implementation
    pass

Testing Handler Methods Directly

You can test handler methods in isolation:

def test_handler_prepare_request():
    """Test request preparation logic."""
    from instructor.v2.providers.your_provider.handlers import YourProviderToolsHandler

    handler = YourProviderToolsHandler()
    response_model, kwargs = handler.prepare_request(
        response_model=SimpleModel,
        kwargs={"messages": [{"role": "user", "content": "test"}]},
    )

    assert "tools" in kwargs  # Verify tools were added
    assert response_model == SimpleModel

def test_handler_parse_response():
    """Test response parsing logic."""
    from instructor.v2.providers.your_provider.handlers import YourProviderToolsHandler

    handler = YourProviderToolsHandler()
    # Mock response object
    mock_response = create_mock_response(...)

    result = handler.parse_response(
        response=mock_response,
        response_model=SimpleModel,
    )

    assert isinstance(result, SimpleModel)

Test Coverage Checklist

For each provider mode, ensure tests cover:

• Mode registration verification
• Basic extraction (sync)
• Basic extraction (async)
• Request preparation (handler method)
• Response parsing (handler method)
• Reask handling (handler method)
• Error handling (invalid responses)
• Retry logic (validation failures)
• Streaming support (if applicable)
• Mode-specific features (e.g., parallel tools, thinking)

Running Tests

# Run all v2 tests
pytest tests/v2/ -v

# Run tests for specific provider
pytest tests/v2/test_provider_modes.py -v

# Run with API key (requires environment variable)
ANTHROPIC_API_KEY=... pytest tests/v2/ -v -m requires_api_key

Provider Migration Checklist

This checklist tracks which providers have been migrated to v2:

Completed Migrations

• OpenAI (Provider.OPENAI)

  • Location: instructor/v2/providers/openai/
  • Modes: TOOLS, JSON, JSON_SCHEMA, MD_JSON, PARALLEL_TOOLS, RESPONSES_TOOLS
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• OpenAI-Compatible (Provider.ANYSCALE, Provider.TOGETHER, Provider.DATABRICKS, Provider.DEEPSEEK)

  • Location: instructor/v2/providers/openai/
  • Modes: TOOLS, JSON, JSON_SCHEMA, MD_JSON, PARALLEL_TOOLS
  • Tests: tests/v2/test_handlers_parametrized.py, tests/v2/test_client_unified.py
  • Status: ✅ Complete

• OpenRouter (Provider.OPENROUTER)

  • Location: instructor/v2/providers/openrouter/
  • Modes: TOOLS, JSON, MD_JSON, PARALLEL_TOOLS, JSON_SCHEMA
  • Tests: tests/v2/test_handlers_parametrized.py, tests/v2/test_client_unified.py
  • Status: ✅ Complete

• Anthropic (Provider.ANTHROPIC)

  • Location: instructor/v2/providers/anthropic/
  • Modes: TOOLS, JSON, JSON_SCHEMA, PARALLEL_TOOLS
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• Google GenAI (Provider.GENAI)

  • Location: instructor/v2/providers/genai/
  • Modes: TOOLS, JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• Google Gemini (Provider.GEMINI)

  • Location: instructor/v2/providers/gemini/
  • Modes: TOOLS, MD_JSON
  • Tests: tests/v2/test_handlers_parametrized.py, tests/v2/test_client_unified.py
  • Status: ✅ Complete

• Vertex AI (Provider.VERTEXAI)

  • Location: instructor/v2/providers/vertexai/
  • Modes: TOOLS, MD_JSON, PARALLEL_TOOLS
  • Tests: tests/v2/test_handlers_parametrized.py, tests/v2/test_client_unified.py
  • Status: ✅ Complete

• Cohere (Provider.COHERE)

  • Location: instructor/v2/providers/cohere/
  • Modes: TOOLS, JSON_SCHEMA, MD_JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• Mistral (Provider.MISTRAL)

  • Location: instructor/v2/providers/mistral/
  • Modes: TOOLS, JSON_SCHEMA, MD_JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• Groq (Provider.GROQ)

  • Location: instructor/v2/providers/groq/
  • Modes: TOOLS, MD_JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• Fireworks (Provider.FIREWORKS)

  • Location: instructor/v2/providers/fireworks/
  • Modes: TOOLS, MD_JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• Cerebras (Provider.CEREBRAS)

  • Location: instructor/v2/providers/cerebras/
  • Modes: TOOLS, MD_JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• Writer (Provider.WRITER)

  • Location: instructor/v2/providers/writer/
  • Modes: TOOLS, MD_JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• xAI (Provider.XAI)

  • Location: instructor/v2/providers/xai/
  • Modes: TOOLS, JSON_SCHEMA, MD_JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

• Perplexity (Provider.PERPLEXITY)

  • Location: instructor/v2/providers/perplexity/
  • Modes: MD_JSON
  • Tests: tests/v2/test_handlers_parametrized.py, tests/v2/test_client_unified.py
  • Status: ✅ Complete

• Bedrock (Provider.BEDROCK)

  • Location: instructor/v2/providers/bedrock/
  • Modes: TOOLS, MD_JSON
  • Tests: tests/v2/test_provider_modes.py, tests/v2/test_handlers_parametrized.py
  • Status: ✅ Complete

Pending Migrations

All current providers have v2 implementations in instructor/v2/providers/. The remaining V1 surface area is concentrated in instructor/core/* and instructor/auto_client.py (see the Step 1 note above).

Migration Steps

To migrate a provider to v2:

1. Create provider directory: instructor/v2/providers/your_provider/
2. Implement handlers: Create handlers.py with @register_mode_handler decorators
3. Create factory function: Create client.py with from_your_provider() function
4. Export: Update __init__.py to export the factory function
5. Add to v2 exports: Update instructor/v2/__init__.py to import provider
6. Write tests: Create tests in tests/v2/ following the testing guide above
7. Update checklist: Mark provider as complete in this document

Migration Notes

• Providers can coexist in v1 and v2 during migration
• Use instructor.from_provider() which routes to v2 when available
• Test both sync and async clients
• Verify all modes work correctly
• Ensure backward compatibility with existing code

Best Practices

• New Modes: Define in instructor.Mode enum, create handler, register via decorator
• Error Handling: Validate early, provide context, preserve exception chains
• Testing: Test both success and failure paths, verify registry registration
• Documentation: Document provider-specific behavior in handler docstrings
• Type Safety: Use type hints throughout handler implementations

Module Organization

instructor/v2/
├── __init__.py              # V2 exports (ModeHandler, Protocols, Registry, Providers)
├── README.md               # This document
├── core/
│   ├── __init__.py         # Core exports (Protocols, Registry)
│   ├── decorators.py       # @register_mode_handler decorator
│   ├── exceptions.py       # Exception classes & validation utilities
│   ├── handler.py          # ModeHandler abstract base class
│   ├── patch.py           # Patching mechanism
│   ├── protocols.py       # Protocol definitions (RequestHandler, etc.)
│   ├── registry.py        # Mode registry implementation
│   └── retry.py           # Retry logic (sync & async)
└── providers/
    ├── __init__.py         # Provider exports
    └── anthropic/          # Anthropic provider implementation
        ├── __init__.py     # Provider exports
        ├── client.py       # from_anthropic factory function
        └── handlers.py     # Handler implementations (TOOLS, JSON, etc.)

Module Exports

• instructor.v2: ModeHandler, mode_registry, RequestHandler, ReaskHandler, ResponseParser, from_anthropic
• instructor.v2.core: Core types and registry
• instructor.v2.providers.anthropic: from_anthropic