567-labs--instructor/docs/faq.md

title, description

title	description
Frequently Asked Questions	Common questions and answers about using Instructor

Frequently Asked Questions

This page answers common questions about using Instructor with various LLM providers.

General Questions

What is Instructor?

Instructor is a library that makes it easy to get structured data from Large Language Models (LLMs). It uses Pydantic to define output schemas and provides a consistent interface across different LLM providers.

How does Instructor work?

Instructor "patches" LLM clients to add a response_model parameter that accepts a Pydantic model. When you make a request, Instructor:

1. Converts your Pydantic model to a schema the LLM can understand
2. Formats the prompt appropriately for the provider
3. Validates the LLM's response against your model
4. Retries automatically if validation fails
5. Returns a properly typed Pydantic object

Which LLM providers does Instructor support?

Instructor supports many providers, including:

• OpenAI (GPT models)
• Anthropic (Claude models)
• Google (Gemini models)
• Cohere
• Mistral AI
• Groq
• LiteLLM (meta-provider)
• TrueFoundry AI Gateway
• Various open-source models via Ollama, llama.cpp, etc.

See the Integrations section for the complete list.

What's the difference between various modes?

Instructor supports generic modes across providers:

• Mode.TOOLS - Tool/function calling when supported
• Mode.JSON - JSON generation for providers that support it (GenAI)
• Mode.JSON_SCHEMA - JSON schema enforcement (OpenAI, Mistral, Cohere)
• Mode.MD_JSON - JSON embedded in markdown
• Mode.PARALLEL_TOOLS - Parallel tool calls where supported

The optimal mode depends on your provider and use case. See Patching for details.

Installation and Setup

How do I install Instructor?

Basic installation:

pip install instructor

For specific providers:

pip install "instructor[anthropic]"  # For Anthropic
pip install "instructor[google-generativeai]"  # For Google/Gemini

What environment variables do I need?

This depends on your provider:

• OpenAI: OPENAI_API_KEY
• Anthropic: ANTHROPIC_API_KEY
• Google: GOOGLE_API_KEY

Each provider has specific requirements documented in their integration guide.

Common Issues

Why is my model not returning structured data?

Common reasons include:

1. Using the wrong mode for your provider
2. Complex schema that confuses the model
3. Insufficient context in your prompt
4. Using a model that doesn't support function/tool calling

Try simplifying your schema or providing clearer instructions in your prompt.

How do I handle validation errors?

Instructor automatically retries when validation fails. You can customize this behavior:

from tenacity import stop_after_attempt

result = client.create(
    response_model=MyModel,
    max_retries=stop_after_attempt(5),  # Retry up to 5 times
    messages=[...]
)

Can I see the raw response from the LLM?

Yes, use create_with_completion:

result, completion = client.create_with_completion(
    response_model=MyModel,
    messages=[...]
)

result is your Pydantic model, and completion is the raw response.

How do I stream large responses?

Use create_partial for partial updates as the response is generated:

stream = client.create_partial(
    response_model=MyModel,
    messages=[...]
)

for partial in stream:
    print(partial)  # Partial model with fields filled in as they're generated

Performance and Costs

How can I optimize token usage?

1. Use concise prompts
2. Use smaller models for simpler tasks
3. Use the MD_JSON or JSON mode for simple schemas
4. Cache responses for repeated queries

How do I handle rate limits?

Instructor uses the tenacity library for retries, which you can configure:

from tenacity import retry_if_exception_type, wait_exponential
from openai.error import RateLimitError

result = client.create(
    response_model=MyModel,
    max_retries=retry_if_exception_type(RateLimitError),
    messages=[...],
)

Advanced Usage

How do I use Instructor with FastAPI?

Instructor works seamlessly with FastAPI:

from fastapi import FastAPI
from pydantic import BaseModel
import instructor
app = FastAPI()
client = instructor.from_provider("openai/gpt-5-nano")

class UserInfo(BaseModel):
    name: str
    age: int

@app.post("/extract")
async def extract_user_info(text: str) -> UserInfo:
    return client.create(
        model="gpt-5.4-mini",
        response_model=UserInfo,
        messages=[{"role": "user", "content": text}]
    )

How do I use Instructor with async code?

Use the async client:

import instructor
import asyncio
client = instructor.from_provider("openai/gpt-5-nano", async_client=True)

async def extract_data():
    result = await client.create(
        response_model=MyModel,
        messages=[...]
    )
    return result

asyncio.run(extract_data())

Where can I get more help?

• Discord community
• GitHub issues
• Twitter @jxnl