chore: import upstream snapshot with attribution

docs/learning/patterns/field_validation.md

@@ -0,0 +1,385 @@
+# Field Validation
+
+This guide covers how to add validation to fields when extracting structured data with Instructor. Field validation ensures that your extracted data meets specific criteria and constraints.
+
+## Why Field Validation Matters
+
+Field validation helps you:
+
+1. Ensure data quality and consistency
+2. Enforce business rules
+3. Prevent errors in downstream processing
+4. Provide clear feedback for invalid data
+
+Instructor uses Pydantic's validation system, which is applied automatically during extraction.
+
+## Basic Field Constraints
+
+You can add basic constraints to fields using Pydantic's `Field` function:
+
+```python
+from pydantic import BaseModel, Field
+import instructor
+from openai import OpenAI
+
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class User(BaseModel):
+    name: str = Field(..., min_length=2, max_length=50)
+    age: int = Field(..., ge=0, le=120)  # greater than or equal to 0, less than or equal to 120
+    email: str = Field(..., pattern=r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$')
+
+# Extract with validation
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": "I'm John Smith, 35 years old, with email john@example.com"}
+    ],
+    response_model=User
+)
+```
+
+Common Field constraints include:
+
+| Constraint | Description | Example |
+|------------|-------------|---------|
+| `min_length` | Minimum string length | `min_length=2` |
+| `max_length` | Maximum string length | `max_length=50` |
+| `pattern` | Regex pattern to match | `pattern=r'^[0-9]+$'` |
+| `gt` | Greater than | `gt=0` (for numbers) |
+| `ge` | Greater than or equal | `ge=18` |
+| `lt` | Less than | `lt=100` |
+| `le` | Less than or equal | `le=120` |
+| `min_items` | Minimum list items | `min_items=1` |
+| `max_items` | Maximum list items | `max_items=10` |
+
+For more information on field definitions, see the [Fields](../../concepts/fields.md) concepts page.
+
+## Validation with Field Validators
+
+For more complex validation logic, use Pydantic's `field_validator` decorator:
+
+```python
+from pydantic import BaseModel, Field, field_validator
+import instructor
+from openai import OpenAI
+import re
+
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Product(BaseModel):
+    name: str
+    sku: str
+    price: float
+
+    @field_validator('name')
+    @classmethod
+    def validate_name(cls, v):
+        if len(v.strip()) < 3:
+            raise ValueError("Product name must be at least 3 characters")
+        return v.strip()
+
+    @field_validator('sku')
+    @classmethod
+    def validate_sku(cls, v):
+        if not re.match(r'^[A-Z]{3}-\d{4}$', v):
+            raise ValueError("SKU must be in format XXX-0000")
+        return v
+
+    @field_validator('price')
+    @classmethod
+    def validate_price(cls, v):
+        if v <= 0:
+            raise ValueError("Price must be greater than zero")
+        if v > 10000:
+            raise ValueError("Price exceeds maximum allowed value")
+        return v
+
+# Extract validated data
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": "Product: Wireless Headphones, SKU: ABC-1234, Price: $79.99"}
+    ],
+    response_model=Product
+)
+```
+
+Field validators can:
+- Perform complex validation logic
+- Clean and normalize data
+- Transform values
+- Check values against external data sources
+
+For more on custom validators, see the [Custom Validators](../validation/custom_validators.md) guide.
+
+## Model-level Validation
+
+Sometimes validation needs to check relationships between fields. For this, use `model_validator`:
+
+```python
+from pydantic import BaseModel, Field, model_validator
+import instructor
+from openai import OpenAI
+from datetime import date
+
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class DateRange(BaseModel):
+    start_date: date
+    end_date: date
+
+    @model_validator(mode='after')
+    def validate_date_range(self):
+        if self.end_date < self.start_date:
+            raise ValueError("End date must be after start date")
+        return self
+```
+
+## Validation in Nested Structures
+
+You can apply validation at any level in nested structures:
+
+```python
+from pydantic import BaseModel, Field, field_validator
+import instructor
+from openai import OpenAI
+from typing import List
+
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Address(BaseModel):
+    street: str
+    city: str
+    state: str
+    zip_code: str
+
+    @field_validator('state')
+    @classmethod
+    def validate_state(cls, v):
+        valid_states = {"CA", "NY", "TX", "FL"}  # Example: just a few states
+        if v not in valid_states:
+            raise ValueError(f"State must be one of: {', '.join(valid_states)}")
+        return v
+
+    @field_validator('zip_code')
+    @classmethod
+    def validate_zip(cls, v):
+        if not v.isdigit() or len(v) != 5:
+            raise ValueError("ZIP code must be 5 digits")
+        return v
+
+class Person(BaseModel):
+    name: str
+    addresses: List[Address]  # Nested structure with validation
+```
+
+For more on nested structures, see the [Nested Structure](nested_structure.md) guide.
+
+## List Item Validation
+
+You can validate items in a list:
+
+```python
+from typing import List
+from pydantic import BaseModel, Field, field_validator
+import instructor
+from openai import OpenAI
+
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class TagList(BaseModel):
+    tags: List[str] = Field(..., min_items=1, max_items=5)
+
+    @field_validator('tags')
+    @classmethod
+    def validate_tags(cls, tags):
+        # Convert all tags to lowercase
+        tags = [tag.lower() for tag in tags]
+
+        # Check for minimum length of each tag
+        for tag in tags:
+            if len(tag) < 2:
+                raise ValueError("Each tag must be at least 2 characters")
+
+        # Check for duplicates
+        if len(tags) != len(set(tags)):
+            raise ValueError("Tags must be unique")
+
+        return tags
+```
+
+For more on lists, see the [List Extraction](list_extraction.md) guide.
+
+## Using Enumerations for Validation
+
+Enums provide a way to validate fields against a predefined set of values:
+
+```python
+from enum import Enum
+from pydantic import BaseModel
+import instructor
+from openai import OpenAI
+
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Status(str, Enum):
+    PENDING = "pending"
+    APPROVED = "approved"
+    REJECTED = "rejected"
+
+class Priority(str, Enum):
+    LOW = "low"
+    MEDIUM = "medium"
+    HIGH = "high"
+
+class Task(BaseModel):
+    title: str
+    description: str
+    status: Status  # Must be one of the enum values
+    priority: Priority  # Must be one of the enum values
+
+# Extract with enum validation
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": "Task: Update website, Description: Refresh content on homepage, Status: pending, Priority: high"}
+    ],
+    response_model=Task
+)
+```
+
+For more information on enums, see the [Enums](../../concepts/enums.md) concepts page.
+
+## Custom Error Messages
+
+You can customize validation error messages for better feedback:
+
+```python
+from pydantic import BaseModel, Field
+import instructor
+from openai import OpenAI
+
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class CreditCard(BaseModel):
+    number: str = Field(
+        ...,
+        pattern=r'^\d{16}$',
+        json_schema_extra={"error_msg": "Credit card number must be exactly 16 digits"}
+    )
+    expiry_month: int = Field(
+        ...,
+        ge=1,
+        le=12,
+        json_schema_extra={"error_msg": "Expiry month must be between 1 and 12"}
+    )
+    expiry_year: int = Field(
+        ...,
+        ge=2023,
+        le=2030,
+        json_schema_extra={"error_msg": "Expiry year must be between 2023 and 2030"}
+    )
+    cvv: str = Field(
+        ...,
+        pattern=r'^\d{3,4}$',
+        json_schema_extra={"error_msg": "CVV must be 3 or 4 digits"}
+    )
+```
+
+## Handling Validation Failures
+
+When validation fails, Instructor will:
+
+1. Capture the validation error
+2. Add the error message to the context
+3. Retry the request with this feedback (if retries are enabled)
+
+To control retry behavior:
+
+```python
+client = instructor.from_provider(
+    "openai/gpt-4o",
+    max_retries=2,  # Number of retries after the initial attempt
+    throw_error=True  # Whether to raise an exception on validation failure
+)
+```
+
+For more on retries, see the [Retry Mechanisms](../validation/retry_mechanisms.md) guide.
+
+## Real-world Example: Form Data Validation
+
+Here's a more complete example validating form inputs:
+
+```python
+from pydantic import BaseModel, Field, field_validator, model_validator
+import instructor
+import re
+from datetime import date, datetime
+from typing import Optional
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class RegistrationForm(BaseModel):
+    username: str = Field(..., min_length=3, max_length=20)
+    email: str
+    password: str
+    confirm_password: str
+    birth_date: date
+
+    @field_validator('username')
+    @classmethod
+    def validate_username(cls, v):
+        if not re.match(r'^[a-zA-Z0-9_]+$', v):
+            raise ValueError("Username can only contain letters, numbers, and underscores")
+        return v
+
+    @field_validator('email')
+    @classmethod
+    def validate_email(cls, v):
+        if not re.match(r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$', v):
+            raise ValueError("Invalid email format")
+        return v
+
+    @field_validator('password')
+    @classmethod
+    def validate_password(cls, v):
+        if len(v) < 8:
+            raise ValueError("Password must be at least 8 characters")
+        if not re.search(r'[A-Z]', v):
+            raise ValueError("Password must contain at least one uppercase letter")
+        if not re.search(r'[a-z]', v):
+            raise ValueError("Password must contain at least one lowercase letter")
+        if not re.search(r'[0-9]', v):
+            raise ValueError("Password must contain at least one number")
+        return v
+
+    @field_validator('birth_date')
+    @classmethod
+    def validate_age(cls, v):
+        today = date.today()
+        age = today.year - v.year - ((today.month, today.day) < (v.month, v.day))
+        if age < 18:
+            raise ValueError("You must be at least 18 years old to register")
+        return v
+
+    @model_validator(mode='after')
+    def passwords_match(self):
+        if self.password != self.confirm_password:
+            raise ValueError("Passwords do not match")
+        return self
+```
+
+## Related Resources
+
+- [Validation Basics](../validation/basics.md) - Core validation concepts
+- [Custom Validators](../validation/custom_validators.md) - Creating custom validation logic
+- [Field-level Validation](../validation/field_level_validation.md) - Advanced field validation
+- [Retry Mechanisms](../validation/retry_mechanisms.md) - Handling validation failures
+- [Fields](../../concepts/fields.md) - Understanding field definitions
+- [Enums](../../concepts/enums.md) - Using enumeration types
+
+## Next Steps
+
+- Learn about [Optional Fields](optional_fields.md) for handling missing data
+- Explore [Custom Validators](../validation/custom_validators.md) for complex validation
+- Check out [Nested Structure](nested_structure.md) for complex data relationships

docs/learning/patterns/list_extraction.md

@@ -0,0 +1,291 @@
+---
+title: List Extraction from LLMs Tutorial
+description: Master extracting multiple structured objects from language models using Instructor with type-safe list validation.
+---
+
+# List Extraction Tutorial: Extract Multiple Objects from LLMs
+
+Master the art of extracting lists and arrays from LLMs in this comprehensive tutorial. Learn how to use Instructor to extract multiple structured objects from language models like GPT-4, Claude, and Gemini with type-safe validation.
+
+## Basic List Extraction
+
+To extract a list of items, you define a model for a single item and then use Python's typing system to specify you want a list of that type:
+
+```python
+from typing import List
+from pydantic import BaseModel, Field
+import instructor
+# Initialize the client
+client = instructor.from_provider("openai/gpt-5-nano")
+
+# Define a single item model
+class Person(BaseModel):
+    name: str = Field(..., description="The person's full name")
+    age: int = Field(..., description="The person's age in years")
+
+# Define a wrapper model for the list
+class PeopleList(BaseModel):
+    people: List[Person] = Field(..., description="List of people mentioned in the text")
+
+# Extract the list
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Here's information about some people:
+        - John Smith is 35 years old
+        - Mary Johnson is 28 years old
+        - Robert Davis is 42 years old
+        """}
+    ],
+    response_model=PeopleList
+)
+
+# Access the extracted data
+for i, person in enumerate(response.people):
+    print(f"Person {i+1}: {person.name}, {person.age} years old")
+```
+
+This example shows how to:
+1. Define a model for a single item (`Person`)
+2. Create a wrapper model that contains a list of items (`PeopleList`)
+3. Access each item in the list through the response
+
+## Direct List Extraction
+
+You can also extract a list directly without a wrapper model:
+
+```python
+from typing import List
+from pydantic import BaseModel, Field
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Book(BaseModel):
+    title: str
+    author: str
+    publication_year: int
+
+# Extract a list directly
+books = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Classic novels:
+        1. To Kill a Mockingbird by Harper Lee (1960)
+        2. 1984 by George Orwell (1949)
+        3. The Great Gatsby by F. Scott Fitzgerald (1925)
+        """}
+    ],
+    response_model=List[Book]  # Direct list extraction
+)
+
+# Access the extracted data
+for book in books:
+    print(f"{book.title} by {book.author} ({book.publication_year})")
+```
+
+## Nested Lists
+
+You can extract nested lists by combining list types:
+
+```python
+from typing import List
+from pydantic import BaseModel, Field
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Author(BaseModel):
+    name: str
+    nationality: str
+
+class Book(BaseModel):
+    title: str
+    authors: List[Author]  # Nested list of authors
+    publication_year: int
+
+# Extract data with nested lists
+books = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Book 1: "Good Omens" (1990)
+        Authors: Terry Pratchett (British), Neil Gaiman (British)
+
+        Book 2: "The Talisman" (1984)
+        Authors: Stephen King (American), Peter Straub (American)
+        """}
+    ],
+    response_model=List[Book]
+)
+
+# Access the nested data
+for book in books:
+    author_names = ", ".join([author.name for author in book.authors])
+    print(f"{book.title} ({book.publication_year}) by {author_names}")
+```
+
+## Using Streaming with Lists
+
+You can stream list extraction results using Instructor's streaming capabilities:
+
+```python
+from typing import List
+import instructor
+from pydantic import BaseModel, Field
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Task(BaseModel):
+    description: str
+    priority: str
+    deadline: str
+
+# Stream a list of tasks
+for task in client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": "Generate a list of 5 sample tasks for a project manager"}
+    ],
+    response_model=List[Task],
+    stream=True
+):
+    print(f"Received task: {task.description} (Priority: {task.priority}, Deadline: {task.deadline})")
+```
+
+For more information on streaming, see the [Streaming Basics](../streaming/basics.md) and [Streaming Lists](../streaming/lists.md) guides.
+
+## List Validation
+
+You can add validation for both individual items and the entire list:
+
+```python
+from typing import List
+from pydantic import BaseModel, Field, field_validator, model_validator
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Product(BaseModel):
+    name: str
+    price: float
+
+    @field_validator('price')
+    @classmethod
+    def validate_price(cls, v):
+        if v <= 0:
+            raise ValueError("Price must be greater than zero")
+        return v
+
+class ProductList(BaseModel):
+    products: List[Product] = Field(..., min_items=1)
+
+    @model_validator(mode='after')
+    def validate_unique_names(self):
+        names = [p.name for p in self.products]
+        if len(names) != len(set(names)):
+            raise ValueError("All product names must be unique")
+        return self
+
+# Extract list with validation
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": "List of products: Headphones ($50), Speakers ($80), Earbuds ($30)"}
+    ],
+    response_model=ProductList
+)
+```
+
+For more on validation, see [Field Validation](./field_validation.md) and [Validation Basics](../validation/basics.md).
+
+## List Constraints
+
+You can add constraints to lists using Pydantic's Field:
+
+```python
+from typing import List
+from pydantic import BaseModel, Field
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Ingredient(BaseModel):
+    name: str
+    amount: str
+
+class Recipe(BaseModel):
+    title: str
+    ingredients: List[Ingredient] = Field(
+        ...,
+        min_items=2,         # Minimum 2 ingredients
+        max_items=10,        # Maximum 10 ingredients
+        description="List of ingredients needed for the recipe"
+    )
+    steps: List[str] = Field(
+        ...,
+        min_items=1,
+        description="Step-by-step instructions to prepare the recipe"
+    )
+```
+
+## Real-world Example: Task Extraction
+
+Here's a more complete example for extracting a list of tasks from a meeting transcript:
+
+```python
+from typing import List, Optional
+from pydantic import BaseModel, Field
+import instructor
+from datetime import date
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Assignee(BaseModel):
+    name: str
+    email: Optional[str] = None
+
+class ActionItem(BaseModel):
+    description: str = Field(..., description="The task that needs to be completed")
+    assignee: Assignee = Field(..., description="The person responsible for the task")
+    due_date: Optional[date] = Field(None, description="The deadline for the task")
+    priority: str = Field(..., description="Priority level: Low, Medium, or High")
+
+# Extract action items from meeting notes
+action_items = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Meeting Notes - Project Kickoff
+        Date: 2023-05-15
+
+        Attendees: John (john@example.com), Sarah (sarah@example.com), Mike
+
+        Discussion points:
+        1. John will prepare the project timeline by next Friday. This is high priority.
+        2. Sarah needs to contact the client for requirements clarification by Wednesday. Medium priority.
+        3. Mike is responsible for setting up the development environment. Due by tomorrow, high priority.
+        """}
+    ],
+    response_model=List[ActionItem]
+)
+
+# Process the extracted action items
+for item in action_items:
+    due_str = item.due_date.isoformat() if item.due_date else "Not specified"
+    print(f"Task: {item.description}")
+    print(f"Assignee: {item.assignee.name} ({item.assignee.email or 'No email'})")
+    print(f"Due: {due_str}, Priority: {item.priority}")
+    print("---")
+```
+
+For a more detailed example, see the [Action Items Extraction](../../examples/action_items.md) example.
+
+## Related Resources
+
+- [Simple Object Extraction](./simple_object.md) - Extracting single objects
+- [Nested Structure](./nested_structure.md) - Working with complex nested data
+- [Streaming Lists](../streaming/lists.md) - Streaming list results
+- [Lists and Arrays](../../concepts/lists.md) - Concepts related to list extraction
+
+## Next Steps
+
+- Learn about [Nested Structure](./nested_structure.md) for complex data
+- Explore [Streaming Lists](../streaming/lists.md) for handling large lists
+- Check out [Field Validation](./field_validation.md) for validation techniques

docs/learning/patterns/nested_structure.md

@@ -0,0 +1,357 @@
+---
+title: Nested Structure Extraction with Instructor
+description: Learn how to extract complex nested data structures from LLMs using hierarchical Pydantic models.
+---
+
+# Simple Nested Structure
+
+This guide explains how to extract nested structured data using Instructor. Nested structures allow you to represent complex, hierarchical data relationships.
+
+## Understanding Nested Structures
+
+Nested structures are objects that contain other objects as fields. They're useful for representing:
+
+1. Parent-child relationships
+2. Complex entities with sub-components
+3. Hierarchical data
+4. Related data that belongs together
+
+## Basic Nested Structure Example
+
+Here's a simple example of extracting a nested structure:
+
+```python
+from pydantic import BaseModel, Field
+import instructor
+from typing import List, Optional
+# Initialize the client
+client = instructor.from_provider("openai/gpt-5-nano")
+
+# Define nested models
+class Address(BaseModel):
+    street: str
+    city: str
+    state: str
+    zip_code: str
+
+class Person(BaseModel):
+    name: str
+    age: int
+    address: Address  # Nested structure
+
+# Extract the nested data
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        John Smith is 35 years old.
+        He lives at 123 Main Street, Boston, MA 02108.
+        """}
+    ],
+    response_model=Person
+)
+
+# Access the nested data
+print(f"Name: {response.name}")
+print(f"Age: {response.age}")
+print(f"Address: {response.address.street}, {response.address.city}, "
+      f"{response.address.state} {response.address.zip_code}")
+```
+
+## Multiple Levels of Nesting
+
+You can use multiple levels of nesting for more complex structures:
+
+```python
+from pydantic import BaseModel, Field
+import instructor
+from typing import List, Optional
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class EmployeeDetails(BaseModel):
+    department: str
+    position: str
+    start_date: str
+
+class ContactInfo(BaseModel):
+    phone: str
+    email: str
+
+class Address(BaseModel):
+    street: str
+    city: str
+    state: str
+    zip_code: str
+
+class Person(BaseModel):
+    name: str
+    age: int
+    contact: ContactInfo  # First level nesting
+    address: Address      # First level nesting
+    employment: Optional[EmployeeDetails] = None  # Optional nested structure
+
+# Extract deeply nested data
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Employee Profile:
+        Name: Jane Doe
+        Age: 32
+        Phone: (555) 123-4567
+        Email: jane.doe@example.com
+        Address: 456 Oak Avenue, Chicago, IL 60601
+        Department: Engineering
+        Position: Senior Developer
+        Start Date: 2021-03-15
+        """}
+    ],
+    response_model=Person
+)
+```
+
+## Nested Lists
+
+You can combine nesting with lists to represent complex collections:
+
+```python
+from pydantic import BaseModel, Field
+import instructor
+from typing import List
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Ingredient(BaseModel):
+    name: str
+    amount: str
+    unit: str
+
+class Recipe(BaseModel):
+    title: str
+    description: str
+    ingredients: List[Ingredient]  # Nested list of ingredients
+    steps: List[str]  # List of strings
+
+# Extract nested list data
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Recipe: Chocolate Chip Cookies
+
+        Description: Classic homemade chocolate chip cookies that are soft in the middle and crispy on the edges.
+
+        Ingredients:
+        - 2 1/4 cups all-purpose flour
+        - 1 teaspoon baking soda
+        - 1 teaspoon salt
+        - 1 cup butter
+        - 3/4 cup white sugar
+        - 3/4 cup brown sugar
+        - 2 eggs
+        - 2 teaspoons vanilla extract
+        - 2 cups chocolate chips
+
+        Instructions:
+        1. Preheat oven to 375°F (190°C)
+        2. Mix flour, baking soda, and salt
+        3. Cream butter and sugars, then add eggs and vanilla
+        4. Gradually add dry ingredients
+        5. Stir in chocolate chips
+        6. Drop by rounded tablespoons onto ungreased baking sheets
+        7. Bake for 9 to 11 minutes or until golden brown
+        8. Cool on wire racks
+        """}
+    ],
+    response_model=Recipe
+)
+```
+
+For more information on working with lists, see the [List Extraction](list_extraction.md) guide.
+
+## Handling Optional Nested Fields
+
+Sometimes parts of a nested structure might be missing. Use Optional to handle this:
+
+```python
+from pydantic import BaseModel, Field
+import instructor
+from typing import Optional
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class SocialMedia(BaseModel):
+    twitter: Optional[str] = None
+    linkedin: Optional[str] = None
+    instagram: Optional[str] = None
+
+class ContactInfo(BaseModel):
+    email: str
+    phone: Optional[str] = None
+    social: Optional[SocialMedia] = None  # Optional nested structure
+
+class Person(BaseModel):
+    name: str
+    contact: ContactInfo
+```
+
+For more information on optional fields, see the [Optional Fields](optional_fields.md) guide.
+
+## Nested Structure Validation
+
+You can add validation to nested structures at any level:
+
+```python
+from pydantic import BaseModel, Field, field_validator, model_validator
+import instructor
+import re
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class EmailContact(BaseModel):
+    email: str
+
+    @field_validator('email')
+    @classmethod
+    def validate_email(cls, v):
+        pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
+        if not re.match(pattern, v):
+            raise ValueError("Invalid email format")
+        return v
+
+class Customer(BaseModel):
+    name: str
+    contact: EmailContact  # Nested structure with its own validation
+
+    @model_validator(mode='after')
+    def validate_name_email_match(self):
+        name_part = self.name.lower().split()[0]
+        if name_part not in self.contact.email.lower():
+            print(f"Warning: Email {self.contact.email} may not match name {self.name}")
+        return self
+```
+
+For more on validation, see [Field Validation](field_validation.md) and [Validation Basics](../validation/basics.md).
+
+## Working with Recursive Structures
+
+For more complex hierarchical data, you can use recursive structures:
+
+```python
+from typing import List, Optional
+from pydantic import BaseModel, Field
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Comment(BaseModel):
+    text: str
+    author: str
+    replies: List["Comment"] = []  # Recursive structure
+
+# Update the Comment class reference for Pydantic
+Comment.model_rebuild()
+
+class Post(BaseModel):
+    title: str
+    content: str
+    author: str
+    comments: List[Comment] = []
+
+# Extract recursive nested data
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Blog Post: "Python Tips and Tricks"
+        Author: John Smith
+        Content: Here are some helpful Python tips for beginners...
+
+        Comments:
+        1. Alice: "Great post! Very helpful."
+           - Bob: "I agree, I learned a lot."
+             - Alice: "Bob, did you try the last example?"
+           - Charlie: "Thanks for sharing this."
+        2. David: "Could you explain the second tip more?"
+           - John: "Sure, I'll add more details."
+        """}
+    ],
+    response_model=Post
+)
+```
+
+For more advanced recursive structures, see the [Recursive Structures](../../examples/recursive.md) guide.
+
+## Real-world Example: Organization Structure
+
+Here's a more complete example extracting an organization structure:
+
+```python
+from typing import List, Optional
+from pydantic import BaseModel, Field
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Employee(BaseModel):
+    name: str
+    title: str
+
+class Department(BaseModel):
+    name: str
+    head: Employee
+    employees: List[Employee]
+    sub_departments: List["Department"] = []
+
+# Update for Pydantic's recursive model support
+Department.model_rebuild()
+
+class Organization(BaseModel):
+    name: str
+    ceo: Employee
+    departments: List[Department]
+
+# Extract organization structure
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Acme Corporation
+        CEO: Jane Smith, Chief Executive Officer
+
+        Departments:
+
+        1. Engineering
+           Head: Bob Johnson, CTO
+           Employees:
+           - Sarah Lee, Senior Engineer
+           - Tom Brown, Software Developer
+
+           Sub-departments:
+           - Frontend Team
+             Head: Lisa Wang, Frontend Lead
+             Employees:
+             - Mike Chen, UI Developer
+             - Ana Garcia, UX Designer
+
+           - Backend Team
+             Head: David Kim, Backend Lead
+             Employees:
+             - James Wright, Database Engineer
+             - Rachel Patel, API Developer
+
+        2. Marketing
+           Head: Michael Davis, CMO
+           Employees:
+           - Jennifer Miller, Marketing Specialist
+           - Robert Chen, Content Creator
+        """}
+    ],
+    response_model=Organization
+)
+```
+
+
+## Related Resources
+
+- [Simple Object Extraction](./simple_object.md) - Extracting basic objects
+- [List Extraction](./list_extraction.md) - Working with lists of objects
+- [Optional Fields](./optional_fields.md) - Handling optional data
+- [Recursive Structures](../../examples/recursive.md) - Building more complex hierarchies
+- [Field Validation](./field_validation.md) - Adding validation to your fields
+

docs/learning/patterns/optional_fields.md

@@ -0,0 +1,191 @@
+---
+title: Working with Optional Fields in Instructor
+description: Learn how to use optional fields in Pydantic models to handle missing or uncertain information from LLM outputs.
+---
+
+# Optional Fields
+
+This guide explains how to work with optional fields in your data models. Optional fields allow the model to skip fields when information is unavailable or uncertain.
+
+## Why Use Optional Fields?
+
+Optional fields are useful when:
+
+1. Some information is missing from the input text
+2. Certain fields are only relevant in specific contexts
+3. The LLM can't confidently extract all fields
+4. You want to allow partial success instead of complete failure
+
+## Basic Optional Fields
+
+To make a field optional, use Python's `Optional` type and provide a default value:
+
+```python
+from typing import Optional
+from pydantic import BaseModel
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Person(BaseModel):
+    name: str  # Required field
+    age: Optional[int] = None  # Optional field with None default
+    occupation: Optional[str] = None  # Optional field with None default
+```
+
+Here, `name` is required, while `age` and `occupation` are optional and will default to `None` if not found.
+
+## Using Default Values
+
+You can provide meaningful default values for optional fields:
+
+```python
+from typing import List
+from pydantic import BaseModel
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Product(BaseModel):
+    name: str
+    price: float
+    currency: str = "USD"  # Default value
+    in_stock: bool = True  # Default value
+    tags: List[str] = []  # Default empty list
+```
+
+## Optional Fields with Validation
+
+You can add the `Field` class for more control and validation:
+
+```python
+from typing import Optional
+from pydantic import BaseModel, Field
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class UserProfile(BaseModel):
+    username: str
+    email: str
+    bio: Optional[str] = Field(
+        None,  # Default value
+        max_length=200,  # Validation applies if present
+        description="User's biography, limited to 200 characters"
+    )
+```
+
+## Optional Nested Structures
+
+Entire nested structures can be optional:
+
+```python
+from typing import Optional
+from pydantic import BaseModel
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Address(BaseModel):
+    street: str
+    city: str
+    state: str
+    zip_code: str
+
+class Contact(BaseModel):
+    email: str
+    phone: Optional[str] = None
+    address: Optional[Address] = None  # Optional nested structure
+
+class Person(BaseModel):
+    name: str
+    contact: Contact
+```
+
+When using nested optional structures, check if they exist before accessing:
+
+```python
+# Access nested data safely
+if person.contact.address:
+    print(f"Address: {person.contact.address.city}")
+else:
+    print("No address information available")
+```
+
+## Using `Maybe` for Uncertain Fields
+
+Instructor provides a `Maybe` type for uncertain or ambiguous fields:
+
+```python
+from pydantic import BaseModel
+import instructor
+from instructor.types import Maybe
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class PersonInfo(BaseModel):
+    name: str
+    age: Maybe[int] = None  # Maybe type for uncertain fields
+```
+
+Check if a `Maybe` field contains uncertain information:
+
+```python
+if person.age and person.age.is_uncertain:
+    print(f"Uncertain age: approximately {person.age.value}")
+elif person.age:
+    print(f"Age: {person.age.value}")
+else:
+    print("Age: Unknown")
+```
+
+For more about the `Maybe` type, see the [Missing Concepts](../../concepts/maybe.md) page.
+
+## Handling Optional Values
+
+Always handle the possibility of `None` values in your code:
+
+```python
+# Check for None before using
+if person.age is not None:
+    drinking_age = "Legal" if person.age >= 21 else "Underage"
+else:
+    drinking_age = "Unknown"
+
+# Use conditional expressions
+price_display = f"${product.price}" if product.price is not None else "Price unavailable"
+
+# Provide defaults with 'or'
+display_name = user.nickname or user.username
+```
+
+## Validation with Optional Fields
+
+Optional fields can still have validation when they're present:
+
+```python
+from typing import Optional
+from pydantic import BaseModel, field_validator
+import instructor
+import re
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class ContactInfo(BaseModel):
+    email: str
+    phone: Optional[str] = None
+
+    @field_validator('phone')
+    @classmethod
+    def validate_phone(cls, v):
+        if v is not None and not re.match(r'^\+?[1-9]\d{1,14}$', v):
+            raise ValueError("Invalid phone format")
+        return v
+```
+
+## Related Resources
+
+- [Simple Object Extraction](./simple_object.md) - Extracting basic objects
+- [Field Validation](./field_validation.md) - Adding validation to fields
+- [Nested Structure](./nested_structure.md) - Working with complex data
+- [Missing Concepts](../../concepts/maybe.md) - Using the Maybe type for uncertain fields
+
+## Next Steps
+
+- Learn about [Field Validation](./field_validation.md)
+- Explore [Nested Structure](./nested_structure.md) for complex data
+- Check out [Prompt Templates](./prompt_templates.md) for crafting prompts

docs/learning/patterns/prompt_templates.md

@@ -0,0 +1,174 @@
+---
+title: Using Prompt Templates with Instructor
+description: Learn how to create reusable prompt templates for consistent structured output extraction across different use cases.
+---
+
+# Prompt Templates
+
+This guide covers how to use prompt templates with Instructor to create reusable, parameterized prompts for structured data extraction.
+
+## Why Prompt Templates Matter
+
+Good prompts are essential for effective structured data extraction. Prompt templates help you:
+
+1. Create consistent and reusable prompts
+2. Parameterize prompts with dynamic values
+3. Separate prompt engineering from application logic
+4. Standardize prompt patterns for different use cases
+
+## Basic Prompt Templates
+
+The simplest form of a prompt template is a string with placeholders for variables:
+
+```python
+from pydantic import BaseModel, Field
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class Person(BaseModel):
+    name: str
+    age: int
+    occupation: str
+
+# Define a template with parameters
+prompt_template = """
+Extract information about the person mentioned in the following {document_type}:
+
+{content}
+
+Please provide their name, age, and occupation.
+"""
+
+# Use the template with specific values
+document_type = "email"
+content = "Hi team, I'm introducing our new project manager, Sarah Johnson. She's 34 and has been in project management for 8 years."
+
+prompt = prompt_template.format(
+    document_type=document_type,
+    content=content
+)
+
+# Extract structured data using the formatted prompt
+response = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": prompt}
+    ],
+    response_model=Person
+)
+```
+
+## Using f-strings for Simple Templates
+
+For simple cases, you can use f-strings to create prompt templates:
+
+```python
+def extract_person(content, document_type="text"):
+    prompt = f"""
+    Extract information about the person mentioned in the following {document_type}:
+
+    {content}
+
+    Please provide their name, age, and occupation.
+    """
+
+    return client.create(
+        model="gpt-5.4-mini",
+        messages=[
+            {"role": "user", "content": prompt}
+        ],
+        response_model=Person
+    )
+
+# Use the function
+person = extract_person(
+    "According to his resume, John Smith (42) works as a software developer.",
+    document_type="resume"
+)
+```
+
+## Template Functions
+
+For more complex templates, create dedicated template functions:
+
+```python
+from typing import List, Optional
+from pydantic import BaseModel
+import instructor
+client = instructor.from_provider("openai/gpt-5-nano")
+
+class ProductReview(BaseModel):
+    product_name: str
+    rating: int
+    pros: List[str]
+    cons: List[str]
+    summary: str
+
+def create_review_extraction_prompt(
+    review_text: str,
+    product_category: str,
+    include_sentiment: bool = False
+) -> str:
+    sentiment_instruction = """
+    Also include a brief sentiment analysis of the review.
+    """ if include_sentiment else ""
+
+    return f"""
+    Extract product review information from the following {product_category} review:
+
+    {review_text}
+
+    Please identify:
+    - The name of the product being reviewed
+    - The numerical rating (1-5)
+    - A list of pros/positive points
+    - A list of cons/negative points
+    - A brief summary of the review
+    {sentiment_instruction}
+    """
+
+# Use the template function
+review_text = """
+I recently purchased the UltraSound X300 headphones, and I'm mostly satisfied.
+The sound quality is amazing and the battery lasts for days. They're also very
+comfortable to wear for long periods. However, they're a bit pricey at $299, and
+the Bluetooth occasionally disconnects. Overall, I'd give them 4 out of 5 stars.
+"""
+
+prompt = create_review_extraction_prompt(
+    review_text=review_text,
+    product_category="headphone",
+    include_sentiment=True
+)
+
+review = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": prompt}
+    ],
+    response_model=ProductReview
+)
+```
+
+## Best Practices for Prompt Templates
+
+1. **Be explicit about the output format**: Clearly specify what fields you need and in what format
+2. **Use consistent language**: Maintain consistent terminology throughout the template
+3. **Keep it concise**: Avoid unnecessary verbosity that could confuse the model
+4. **Parameterize only what varies**: Only make template parameters for parts that need to change
+5. **Include examples for complex tasks**: Provide few-shot examples for more complex extractions
+6. **Test with different inputs**: Ensure your template works well with a variety of inputs
+
+## Related Resources
+
+- [Simple Object Extraction](./simple_object.md) - Extracting basic objects
+- [List Extraction](./list_extraction.md) - Working with lists of objects
+- [Optional Fields](./optional_fields.md) - Handling optional data
+- [Prompting](../../concepts/prompting.md) - General prompting concepts
+- [Templating](../../concepts/templating.md) - Advanced template techniques
+
+## Next Steps
+
+- Explore [Field Validation](./field_validation.md) for ensuring data quality
+- Try [List Extraction](./list_extraction.md) for extracting multiple items
+- Learn about [Nested Structure](./nested_structure.md) for complex data

docs/learning/patterns/simple_object.md

@@ -0,0 +1,145 @@
+---
+title: Simple Object Extraction Pattern
+description: Learn the fundamental pattern of extracting simple objects from text using Instructor with type-safe validation.
+---
+
+# Simple Object Extraction: LLM Tutorial for Structured Data
+
+Learn how to extract structured objects from text using LLMs in this comprehensive tutorial. We'll cover the fundamental pattern of transforming unstructured text into validated Python objects using Instructor with GPT-4, Claude, and other language models.
+
+## Basic LLM Object Extraction Tutorial
+
+```python
+from pydantic import BaseModel
+import instructor
+# Define your LLM extraction schema
+class Person(BaseModel):
+    name: str
+    age: int
+    occupation: str
+
+# Extract structured data from LLM
+client = instructor.from_provider("openai/gpt-5-nano")
+person = client.create(
+    model="gpt-5.4-mini",  # Works with GPT-4, Claude, Gemini
+    messages=[
+        {"role": "user", "content": "John Smith is a 35-year-old software engineer."}
+    ],
+    response_model=Person  # Type-safe LLM extraction
+)
+
+print(f"Name: {person.name}")
+print(f"Age: {person.age}")
+print(f"Occupation: {person.occupation}")
+```
+
+```
+┌───────────────┐            ┌───────────────┐
+│ Define Model  │            │ Extracted     │
+│ name: str     │  Extract   │ name: "John"  │
+│ age: int      │ ─────────> │ age: 35       │
+│ occupation: str│            │ occupation:   │
+└───────────────┘            │ "software..." │
+                             └───────────────┘
+```
+
+## Enhance LLM Extraction with Field Descriptions
+
+Guide your LLM with clear field descriptions for more accurate extraction:
+
+```python
+from pydantic import BaseModel, Field
+
+class Book(BaseModel):
+    title: str = Field(description="The full title of the book")
+    author: str = Field(description="The author's full name")
+    publication_year: int = Field(description="The year the book was published")
+```
+
+Field descriptions serve as prompts for the LLM, improving extraction accuracy and reducing errors in your structured outputs.
+
+## Handle Missing Data in LLM Responses
+
+Real-world LLM extractions often encounter missing information. Here's how to handle it gracefully:
+
+```python
+from typing import Optional
+from pydantic import BaseModel
+
+class MovieReview(BaseModel):
+    title: str
+    director: Optional[str] = None  # Optional field
+    rating: float
+```
+
+Using `Optional` fields ensures your LLM extraction remains robust when dealing with incomplete or partial information.
+
+## Validate LLM Outputs with Pydantic
+
+Ensure LLM outputs meet your requirements with built-in validation:
+
+```python
+from pydantic import BaseModel, Field
+
+class Product(BaseModel):
+    name: str
+    price: float = Field(gt=0, description="The product price in USD")
+    in_stock: bool
+```
+
+Pydantic validation ensures your LLM outputs are not just structured, but also correct and business-rule compliant.
+
+## Production-Ready LLM Extraction Example
+
+Here's a complete example showing nested object extraction from LLMs:
+
+```python
+from pydantic import BaseModel
+from typing import Optional
+
+class Address(BaseModel):
+    street: str
+    city: str
+    state: str
+    zip_code: str
+
+class ContactInfo(BaseModel):
+    name: str
+    email: str
+    phone: Optional[str] = None
+    address: Optional[Address] = None
+
+# Extract structured data
+client = instructor.from_provider("openai/gpt-5-nano")
+contact = client.create(
+    model="gpt-5.4-mini",
+    messages=[
+        {"role": "user", "content": """
+        Contact information:
+        Name: Sarah Johnson
+        Email: sarah.j@example.com
+        Phone: (555) 123-4567
+        Address: 123 Main St, Boston, MA 02108
+        """}
+    ],
+    response_model=ContactInfo
+)
+
+print(f"Name: {contact.name}")
+print(f"Email: {contact.email}")
+```
+
+## Common LLM Object Extraction Use Cases
+
+- **Contact Information**: Extract names, emails, phones from unstructured text
+- **Product Details**: Parse product descriptions into structured catalogs
+- **Event Information**: Extract dates, locations, attendees from event descriptions
+- **Entity Recognition**: Identify and structure people, places, organizations
+
+## Continue Your LLM Tutorial Journey
+
+- **[List Extraction Tutorial](list_extraction.md)** - Extract multiple objects from LLM responses
+- **[Nested Structures](nested_structure.md)** - Handle complex hierarchical data from LLMs
+- **[Advanced Validation](field_validation.md)** - Implement business rules for LLM outputs
+
+Master these patterns to build production-ready LLM applications with reliable structured outputs!