chore: import upstream snapshot with attribution

examples/batch_api/README.md

@@ -0,0 +1,223 @@
+# Batch API Examples
+
+This directory contains examples and test scripts for Instructor's batch processing capabilities, including both traditional file-based and new in-memory processing.
+
+## Examples
+
+### 1. In-Memory Batch Processing (`in_memory_batch_example.py`)
+
+Demonstrates the new in-memory batch processing feature, perfect for serverless deployments:
+
+```bash
+python in_memory_batch_example.py
+```
+
+**Key Features:**
+- No disk I/O required - ideal for serverless environments
+- BytesIO buffers instead of temporary files  
+- Automatic cleanup - no file management needed
+- Security benefits - no temporary files on disk
+
+### 2. Unified Test Script (`run_batch_test.py`)
+
+Tests the unified BatchProcessor with all supported providers: OpenAI, Anthropic, and Google Gemini.
+
+The script creates a batch job to extract structured `User(name: str, age: int)` data from 10 text examples and saves the batch ID for later checking. Since batch jobs can take time to complete, the script returns immediately after creation.
+
+## Unified Test Script (`run_batch_test.py`)
+
+Tests the unified BatchProcessor with any supported provider/model combination.
+
+### Usage
+
+```bash
+# Test OpenAI
+export OPENAI_API_KEY="your-openai-api-key"
+python run_batch_test.py create --model "openai/gpt-4o-mini"
+
+# Test Anthropic  
+export ANTHROPIC_API_KEY="your-anthropic-api-key"
+python run_batch_test.py create --model "anthropic/claude-3-5-sonnet-20241022"
+
+# Test Google (simulation mode)
+python run_batch_test.py create --model "google/gemini-2.0-flash-001"
+```
+
+### Supported Models
+
+Use the `list-models` command to see all supported models:
+
+```bash
+python run_batch_test.py list-models
+```
+
+**OpenAI Models:**
+- `openai/gpt-4o-mini`
+- `openai/gpt-4o`
+- `openai/gpt-4-turbo`
+
+**Anthropic Models:**
+- `anthropic/claude-3-5-sonnet-20241022`
+- `anthropic/claude-3-opus-20240229`
+- `anthropic/claude-3-haiku-20240307`
+
+**Google Models:**
+- `google/gemini-2.0-flash-001`
+- `google/gemini-pro`
+- `google/gemini-pro-vision`
+
+### What the Script Does
+
+1. **Creates test messages**: 10 prompts containing user information
+2. **Uses BatchProcessor**: Leverages the unified API with provider detection
+3. **Generates batch file**: Provider-specific format with JSON schema
+4. **Submits batch job**: Actual API call to create the batch
+5. **Saves batch ID**: Stores ID in `{provider}_batch_id.txt`
+6. **Returns immediately**: No waiting for completion
+
+### API Keys Required
+
+| Provider | Environment Variable | Required |
+|----------|---------------------|----------|
+| OpenAI | `OPENAI_API_KEY` | Yes |
+| Anthropic | `ANTHROPIC_API_KEY` | Yes |
+| Google | `GOOGLE_API_KEY` | No (simulation mode) |
+
+### Output Files
+
+Each run creates:
+- `{provider}_batch_id.txt` - Contains the batch ID for status checking
+- Temporary batch files (automatically cleaned up)
+
+### Test Data
+
+All providers use the same 10 test prompts:
+
+1. "Hi there! My name is Alice and I'm 28 years old. I work as a software engineer."
+2. "Hello, I'm Bob, 35 years old, and I love hiking and photography."
+3. "This is Sarah speaking. I'm 42 and I'm a graphic designer."
+4. "Hey! John here, I'm 29 years old and I teach high school math."
+5. "I'm Emma, 33 years old, currently working as a marketing manager."
+6. "My name is Michael and I'm 45 years old. I'm a chef at a downtown restaurant."
+7. "I'm Lisa, 31 years old, working as a nurse at the local hospital."
+8. "This is David, 38 years old, I'm a freelance photographer."
+9. "Hello, I'm Jessica, 26 years old, and I'm a data scientist."
+10. "I'm Ryan, 41 years old, working in software development for a tech startup."
+
+### Expected Results
+
+Each batch job should extract `User` objects:
+
+```python
+class User(BaseModel):
+    name: str
+    age: int
+```
+
+Expected extractions:
+- Alice, 28 | Bob, 35 | Sarah, 42 | John, 29 | Emma, 33
+- Michael, 45 | Lisa, 31 | David, 38 | Jessica, 26 | Ryan, 41
+
+## Checking Batch Status
+
+After creating batch jobs, use the CLI to check their status:
+
+```bash
+# List all batch jobs for a provider
+instructor batch list --model "openai/gpt-4o-mini"
+instructor batch list --model "anthropic/claude-3-5-sonnet-20241022"
+
+# Check specific batch status
+instructor batch status --batch-id "batch_123" --model "openai/gpt-4o-mini"
+
+# Get results when completed
+instructor batch results \
+  --batch-id "batch_123" \
+  --output-file "results.jsonl" \
+  --model "openai/gpt-4o-mini"
+```
+
+## Processing Times
+
+- **OpenAI**: Usually completes within a few hours, guaranteed within 24h
+- **Anthropic**: Most batches complete in under 1 hour
+- **Google**: Varies (simulation only in this test)
+
+## Running Tests for All Providers
+
+```bash
+# Test all providers (requires API keys)
+python run_batch_test.py create --model "openai/gpt-4o-mini"
+python run_batch_test.py create --model "anthropic/claude-3-5-sonnet-20241022" 
+python run_batch_test.py create --model "google/gemini-2.0-flash-001"
+
+# Check what was created
+ls *_batch_id.txt
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **API Key Not Set**
+   ```
+   ❌ Error: OPENAI_API_KEY environment variable is not set
+   ```
+   Solution: Set the appropriate environment variable.
+
+2. **Invalid Model Format**
+   ```
+   ❌ Error: Model must be in format 'provider/model-name'
+   ```
+   Solution: Use the format `provider/model-name`, e.g., `openai/gpt-4o-mini`.
+
+3. **Unsupported Provider**
+   ```
+   ❌ Unsupported provider: xyz
+   ```
+   Solution: Use `openai`, `anthropic`, or `google` as the provider.
+
+### Provider-Specific Notes
+
+**OpenAI:**
+- Requires valid API key with sufficient credits
+- Supports both individual and organization accounts
+- Rate limits are separate for batch vs regular API
+
+**Anthropic:**
+- Uses beta API endpoints (`client.beta.messages.batches`)
+- Requires Anthropic API access
+- May have different availability by region
+
+**Google:**
+- Runs in simulation mode by default
+- Full implementation requires Google Cloud Storage setup
+- Would need proper GCS authentication for real batch jobs
+
+## Integration with CLI
+
+This test validates that the unified BatchProcessor works correctly, which powers the CLI commands:
+
+```bash
+# Create batch using CLI directly
+instructor batch create \
+  --messages-file messages.jsonl \
+  --model "openai/gpt-4o-mini" \
+  --response-model "examples.User" \
+  --output-file batch_requests.jsonl
+
+# Submit the batch
+instructor batch create-from-file \
+  --file-path batch_requests.jsonl \
+  --model "openai/gpt-4o-mini"
+```
+
+## Development
+
+To modify the test:
+1. Update `create_test_messages()` to change test data
+2. Modify the `User` model if needed
+3. Add new providers in the provider detection logic
+4. Adjust batch creation functions for new provider-specific behavior
+
+The test demonstrates that the same code works across all providers thanks to the unified BatchProcessor abstraction!

examples/batch_api/in_memory_batch_example.py

@@ -0,0 +1,244 @@
+#!/usr/bin/env python3
+"""Example of using in-memory batching for serverless deployments.
+
+This example shows how to create and submit batch requests without writing to disk
+"""
+
+import time
+from pydantic import BaseModel
+from instructor.batch.processor import BatchProcessor
+
+
+class User(BaseModel):
+    """User model for extraction."""
+
+    name: str
+    age: int
+    email: str
+
+
+def main():
+    """Demonstrate in-memory batch processing."""
+    print("In-Memory Batch Processing Example")
+    print("===================================\n")
+
+    # Initialize batch processor
+    # Note: Use gpt-4o-mini for JSON schema support in batch API
+    processor = BatchProcessor("openai/gpt-4o-mini", User)
+
+    # Sample messages for batch processing
+    messages_list = [
+        [
+            {"role": "system", "content": "Extract user information from the text."},
+            {
+                "role": "user",
+                "content": "John Doe is 25 years old and his email is john@example.com",
+            },
+        ],
+        [
+            {"role": "system", "content": "Extract user information from the text."},
+            {
+                "role": "user",
+                "content": "Jane Smith, age 30, can be reached at jane.smith@company.com",
+            },
+        ],
+        [
+            {"role": "system", "content": "Extract user information from the text."},
+            {
+                "role": "user",
+                "content": "Bob Wilson (bob.wilson@email.com) is 28 years old",
+            },
+        ],
+    ]
+
+    print("Creating batch requests in memory...")
+
+    # Create batch in memory (no file_path specified)
+    batch_buffer = processor.create_batch_from_messages(
+        messages_list,
+        file_path=None,  # This triggers in-memory mode
+        max_tokens=150,
+        temperature=0.1,
+    )
+
+    print(f"Created batch buffer: {type(batch_buffer)}")
+    print(f"Buffer size: {len(batch_buffer.getvalue())} bytes\n")
+
+    # Show the content of the buffer (first 200 chars)
+    batch_buffer.seek(0)
+    content_preview = batch_buffer.read(200).decode("utf-8")
+    print("Buffer content preview:")
+    print(f"{content_preview}...\n")
+
+    # Reset buffer position for submission
+    batch_buffer.seek(0)
+
+    print("Submitting batch job...")
+
+    try:
+        # Submit the batch using the in-memory buffer
+        batch_id = processor.submit_batch(
+            batch_buffer, metadata={"description": "In-memory batch example"}
+        )
+
+        print(f"Batch submitted successfully!")
+        print(f"Batch ID: {batch_id}")
+
+        # Poll for completion
+        print("\nWaiting for batch to complete...")
+        max_wait_time = 300  # 5 minutes max
+        start_time = time.time()
+        status = {}
+
+        while time.time() - start_time < max_wait_time:
+            status = processor.get_batch_status(batch_id)
+            current_status = status.get("status", "unknown")
+
+            # Update status on the same line
+            print(f"\rCurrent status: {current_status.ljust(20)}", end="")
+
+            if current_status in ["completed", "failed", "cancelled", "expired"]:
+                break
+
+            time.sleep(10)
+
+        print()  # Newline after polling is done
+
+        # Use the last fetched status
+        final_status = status
+        print(f"\nFinal status: {final_status.get('status', 'unknown')}")
+
+        if final_status.get("status") == "completed":
+            print("\nBatch completed! Retrieving results...")
+
+            # Retrieve and process results
+            results = processor.get_results(batch_id)
+
+            print(f"\nResults Summary:")
+            print(f"   Total results: {len(results)}")
+
+            successful_results = [r for r in results if hasattr(r, "result")]
+            error_results = [r for r in results if hasattr(r, "error_message")]
+
+            print(f"   Successful: {len(successful_results)}")
+            print(f"   Errors: {len(error_results)}")
+
+            # Show successful extractions
+            if successful_results:
+                print("\nExtracted Users:")
+                for result in successful_results:
+                    user = result.result
+                    print(f"   - {user.name}, {user.age} years old, {user.email}")
+
+            # Show any errors
+            if error_results:
+                print("\nErrors encountered:")
+                for error in error_results:
+                    print(f"   - {error.custom_id}: {error.error_message}")
+
+        elif final_status.get("status") == "failed":
+            print("\nBatch failed to complete")
+            print("   Check your API usage and batch format")
+
+        else:
+            print(f"\nBatch did not complete within {max_wait_time} seconds")
+            print(f"   Current status: {final_status.get('status', 'unknown')}")
+            print(
+                "   You can check status later with processor.get_batch_status(batch_id)"
+            )
+
+    except Exception as e:
+        print(f"Error during batch processing: {e}")
+        print("\nThis is expected if you don't have OpenAI API credentials set up.")
+        print(
+            "   The important part is that the in-memory buffer was created successfully!"
+        )
+
+    print("\nIn-memory batch processing demo complete!")
+    print("\nKey benefits of in-memory batching:")
+    print("   - No disk I/O required - perfect for serverless")
+    print("   - Faster processing - no file system overhead")
+    print("   - Better security - no temporary files on disk")
+    print("   - Cleaner code - no file cleanup required")
+
+
+def compare_file_vs_memory():
+    """Compare file-based vs in-memory batch creation."""
+    print("\nComparing File-based vs In-Memory Batching")
+    print("===========================================\n")
+
+    processor = BatchProcessor("openai/gpt-4o-mini", User)
+
+    messages_list = [
+        [{"role": "user", "content": "Extract: John, 25, john@example.com"}],
+        [{"role": "user", "content": "Extract: Jane, 30, jane@example.com"}],
+    ]
+
+    # File-based approach (traditional)
+    print("File-based approach:")
+    file_path = processor.create_batch_from_messages(
+        messages_list,
+        file_path="temp_batch.jsonl",  # Specify file path
+    )
+    print(f"   Created file: {file_path}")
+
+    # Clean up the file
+    import os
+
+    if os.path.exists(file_path):
+        os.remove(file_path)
+        print("   File cleaned up")
+
+    # In-memory approach (new)
+    print("\nIn-memory approach:")
+    buffer = processor.create_batch_from_messages(
+        messages_list,
+        file_path=None,  # No file path = in-memory
+    )
+    print(f"   Created buffer: {type(buffer).__name__}")
+    print(f"   Buffer size: {len(buffer.getvalue())} bytes")
+    print("   No cleanup required!")
+
+
+def demo_polling_logic():
+    """Demonstrate how to properly poll for batch completion."""
+    print("\nBatch Polling Best Practices")
+    print("============================\n")
+
+    print("When working with real batches, follow this pattern:")
+    print("")
+    print("```python")
+    print("import time")
+    print("")
+    print("# Submit your batch")
+    print("batch_id = processor.submit_batch(buffer)")
+    print("")
+    print("# Poll for completion")
+    print("while True:")
+    print("    status = processor.get_batch_status(batch_id)")
+    print("    current_status = status.get('status')")
+    print("    ")
+    print("    if current_status == 'completed':")
+    print("        results = processor.get_results(batch_id)")
+    print("        break")
+    print("    elif current_status in ['failed', 'cancelled', 'expired']:")
+    print("        print(f'Batch failed with status: {current_status}')")
+    print("        break")
+    print("    else:")
+    print("        print(f'Status: {current_status}, waiting...')")
+    print("        time.sleep(10)  # Wait 10 seconds before checking again")
+    print("```")
+    print("")
+    print("Typical batch statuses:")
+    print("   - validating - Checking request format")
+    print("   - in_progress - Processing requests")
+    print("   - finalizing - Preparing results")
+    print("   - completed - Ready for download")
+    print("   - failed - Something went wrong")
+    print("   - cancelled - Manually cancelled")
+    print("   - expired - Took too long to process")
+
+
+if __name__ == "__main__":
+    main()
+    compare_file_vs_memory()

examples/batch_api/run_batch_test.py

@@ -0,0 +1,851 @@
+#!/usr/bin/env python3
+"""Unified Batch API Test Script
+
+Test script to verify the unified BatchProcessor works correctly with all supported providers.
+Creates a batch job to extract User(name: str, age: int) data from text examples.
+
+Supports:
+- OpenAI: openai/gpt-4o-mini, openai/gpt-4o, etc.
+- Anthropic: anthropic/claude-3-5-sonnet-20241022, anthropic/claude-3-opus-20240229, etc.
+- Google: google/gemini-2.5-flash, google/gemini-pro, etc.
+
+Usage:
+    # Default (Google Gemini 2.5 Flash)
+    export GOOGLE_API_KEY="your-key"
+    python run_batch_test.py
+
+    # OpenAI
+    export OPENAI_API_KEY="your-key"
+    python run_batch_test.py --model "openai/gpt-4o-mini"
+
+    # Anthropic
+    export ANTHROPIC_API_KEY="your-key"
+    python run_batch_test.py --model "anthropic/claude-3-5-sonnet-20241022"
+
+    # Google with specific model
+    export GOOGLE_API_KEY="your-key"
+    python run_batch_test.py --model "google/gemini-2.5-flash"
+"""
+
+import os
+import sys
+from typing import Optional
+import typer
+from pydantic import BaseModel
+
+# Add parent directory to path for imports
+sys.path.append(os.path.join(os.path.dirname(__file__), "..", ".."))
+from instructor.batch import (
+    BatchProcessor,
+    BatchStatus,
+    filter_successful,
+    filter_errors,
+    extract_results,
+)
+
+app = typer.Typer(help="Unified Batch API Test for all providers")
+
+
+class User(BaseModel):
+    name: str
+    age: int
+
+
+def create_test_messages() -> list[list[dict]]:
+    """Create test message conversations for user extraction"""
+    test_prompts = [
+        "Hi there! My name is Alice and I'm 28 years old. I work as a software engineer.",
+    ]
+
+    messages_list = []
+    for prompt in test_prompts:
+        messages = [
+            {
+                "role": "system",
+                "content": "You are an expert at extracting structured user information from text. Extract the person's name and age.",
+            },
+            {"role": "user", "content": prompt},
+        ]
+        messages_list.append(messages)
+
+    return messages_list
+
+
+def get_expected_results() -> list[User]:
+    """Get the expected User objects for validation"""
+    return [
+        User(name="Alice", age=28),
+    ]
+
+
+def check_api_key(provider: str) -> bool:
+    """Check if the required API key is set for the provider"""
+    key_map = {
+        "openai": "OPENAI_API_KEY",
+        "anthropic": "ANTHROPIC_API_KEY",
+        "google": "GOOGLE_API_KEY",
+    }
+
+    required_key = key_map.get(provider)
+    if not required_key:
+        return True  # Unknown provider, let it fail later
+
+    if provider == "google":
+        # Google is optional since we simulate
+        if not os.getenv(required_key):
+            typer.echo(f"Warning: {required_key} not set - will run in simulation mode")
+        return True
+
+    if not os.getenv(required_key):
+        typer.echo(f"Error: {required_key} environment variable is not set", err=True)
+        typer.echo(
+            f"Please set your API key: export {required_key}='your-api-key-here'",
+            err=True,
+        )
+        return False
+
+    return True
+
+
+def create_openai_batch(model: str, messages_list: list[list[dict]]) -> Optional[str]:
+    """Create OpenAI batch job using BatchProcessor"""
+    processor = BatchProcessor(model, User)
+
+    # Create batch file
+    batch_filename = "test_batch.jsonl"
+    processor.create_batch_from_messages(
+        file_path=batch_filename,
+        messages_list=messages_list,
+        max_tokens=200,
+        temperature=0.1,
+    )
+
+    try:
+        typer.echo("Submitting batch job...")
+        batch_id = processor.submit_batch(
+            file_path=batch_filename,
+            metadata={"description": "Unified BatchProcessor test"},
+        )
+        return batch_id
+
+    finally:
+        if os.path.exists(batch_filename):
+            os.remove(batch_filename)
+
+
+def create_anthropic_batch(
+    model: str, messages_list: list[list[dict]]
+) -> Optional[str]:
+    """Create Anthropic batch job using BatchProcessor"""
+    processor = BatchProcessor(model, User)
+
+    # Create batch file
+    batch_filename = "test_batch.jsonl"
+    processor.create_batch_from_messages(
+        file_path=batch_filename,
+        messages_list=messages_list,
+        max_tokens=200,
+        temperature=0.1,
+    )
+
+    try:
+        typer.echo("Submitting batch job...")
+        batch_id = processor.submit_batch(file_path=batch_filename)
+        return batch_id
+
+    finally:
+        if os.path.exists(batch_filename):
+            os.remove(batch_filename)
+
+
+def create_google_batch(model: str, messages_list: list[list[dict]]) -> Optional[str]:
+    """Create Google batch job using BatchProcessor (inline only)"""
+    processor = BatchProcessor(model, User)
+
+    typer.echo("Submitting Google inline batch...")
+    batch_id = processor.submit_batch(
+        messages_list=messages_list,
+        metadata={"description": "Unified BatchProcessor test"},
+        use_inline=True,
+        max_tokens=200,
+        temperature=0.1,
+    )
+
+    typer.echo(f"Inline batch job created: {batch_id}")
+    return batch_id
+
+
+@app.command()
+def create(
+    model: str = typer.Option(
+        "openai/gpt-4o-mini",
+        help="Model in format 'provider/model-name' (e.g., 'google/gemini-2.5-flash', 'openai/gpt-4o-mini', 'anthropic/claude-3-5-sonnet-20241022')",
+    ),
+    save_id: bool = typer.Option(True, help="Save batch ID to file"),
+):
+    """Create a batch job for the specified model"""
+
+    typer.echo(f"Creating Batch Job for {model}")
+    typer.echo("=" * 50)
+
+    # Parse provider from model
+    try:
+        provider, model_name = model.split("/", 1)
+    except ValueError:
+        typer.echo("Error: Model must be in format 'provider/model-name'", err=True)
+        typer.echo(
+            "Examples: 'openai/gpt-4o-mini', 'anthropic/claude-3-5-sonnet-20241022'",
+            err=True,
+        )
+        raise typer.Exit(1) from None
+
+    # Check API key
+    if not check_api_key(provider):
+        raise typer.Exit(1)
+
+    # Create test messages
+    messages_list = create_test_messages()
+    typer.echo(f"Created {len(messages_list)} test message conversations")
+
+    try:
+        # Create batch job based on provider
+        batch_id = None
+
+        if provider == "openai":
+            batch_id = create_openai_batch(model, messages_list)
+        elif provider == "anthropic":
+            batch_id = create_anthropic_batch(model, messages_list)
+        else:
+            typer.echo(f"Unsupported provider: {provider}", err=True)
+            raise typer.Exit(1)
+
+        if batch_id:
+            typer.echo(f"Batch job created with ID: {batch_id}")
+
+            if save_id:
+                filename = f"{provider}_batch_id.txt"
+                with open(filename, "w") as f:
+                    f.write(batch_id)
+                typer.echo(f"Batch ID saved to {filename}")
+
+            # Validate expected results
+            expected_results = get_expected_results()
+            typer.echo(f"Expected results validated: {len(expected_results)} users")
+            for i, user in enumerate(expected_results):
+                typer.echo(f"   {i + 1}. {user.name}, age {user.age}")
+
+            # Show how to check status
+            typer.echo(f"Check status with:")
+            typer.echo(f"   instructor batch list --model {model}")
+
+            typer.echo(f"Cost savings: 50% vs regular API")
+            typer.echo(f"\nSuccess! Batch ID: {batch_id}")
+
+        else:
+            typer.echo("Failed to create batch job", err=True)
+            raise typer.Exit(1)
+
+    except Exception as e:
+        typer.echo(f"Error creating batch: {e}", err=True)
+        raise typer.Exit(1) from e
+
+
+@app.command()
+def list_batches():
+    """List saved batch IDs for all providers"""
+    typer.echo("Saved Batch IDs:")
+    typer.echo("=" * 30)
+
+    providers = ["openai", "anthropic"]
+    found_any = False
+
+    for provider in providers:
+        filename = f"{provider}_batch_id.txt"
+        if os.path.exists(filename):
+            with open(filename) as f:
+                batch_id = f.read().strip()
+
+            typer.echo(f"{provider.upper()}: {batch_id}")
+            found_any = True
+
+    if not found_any:
+        typer.echo("No batch IDs found. Run 'create' command first.")
+        typer.echo(
+            "Usage: python run_batch_test.py create --model 'provider/model-name'"
+        )
+    else:
+        typer.echo()
+        typer.echo(
+            "To fetch results: python run_batch_test.py fetch --provider <provider>"
+        )
+
+
+@app.command()
+def fetch(
+    provider: str = typer.Option(
+        help="Provider to fetch results from (openai, anthropic, google)"
+    ),
+    validate: bool = typer.Option(
+        True, help="Validate extracted data against expected results"
+    ),
+    poll: bool = typer.Option(
+        False, help="Poll every 30 seconds until batch completes"
+    ),
+    max_wait: int = typer.Option(
+        600, help="Maximum time to wait in seconds (default: 10 minutes)"
+    ),
+):
+    """Fetch and validate batch results from a provider"""
+
+    if provider not in ["openai", "anthropic"]:
+        typer.echo("Error: Provider must be one of: openai, anthropic", err=True)
+        raise typer.Exit(1)
+
+    # Check if batch ID file exists
+    filename = f"{provider}_batch_id.txt"
+    if not os.path.exists(filename):
+        typer.echo(
+            f"Error: No batch ID found for {provider}. Run 'create' command first.",
+            err=True,
+        )
+        raise typer.Exit(1)
+
+    # Read batch ID
+    with open(filename) as f:
+        batch_id = f.read().strip()
+
+    typer.echo(f"Fetching results for {provider.upper()} batch: {batch_id}")
+    typer.echo("=" * 60)
+
+    # Check API key
+    if not check_api_key(provider):
+        raise typer.Exit(1)
+
+    try:
+        if poll:
+            results = poll_for_results(provider, batch_id, validate, max_wait)
+        else:
+            if provider == "openai":
+                results = fetch_openai_results(batch_id, validate)
+            elif provider == "anthropic":
+                results = fetch_anthropic_results(batch_id, validate)
+
+        if results:
+            typer.echo(f"Successfully fetched and validated {len(results)} results!")
+            if validate:
+                # Assert that the results match the expected results
+                assert validate_results(results, provider.capitalize()), (
+                    f"Test failed: {provider} results do not match expected results."
+                )
+        else:
+            typer.echo("No results available yet or batch still processing")
+            if not poll:
+                typer.echo("Use --poll to automatically wait for completion")
+
+    except AssertionError as ae:
+        typer.echo(f"AssertionError: {ae}", err=True)
+        raise typer.Exit(1) from ae
+    except Exception as e:
+        typer.echo(f"Error fetching results: {e}", err=True)
+        raise typer.Exit(1) from e
+
+
+@app.command()
+def show_results(
+    provider: str = typer.Option(
+        help="Provider to show detailed results from (openai, anthropic, google)"
+    ),
+):
+    """Show detailed parsed Pydantic objects from batch results"""
+
+    if provider not in ["openai", "anthropic"]:
+        typer.echo("Error: Provider must be one of: openai, anthropic", err=True)
+        raise typer.Exit(1)
+
+    # Check if batch ID file exists
+    filename = f"{provider}_batch_id.txt"
+    if not os.path.exists(filename):
+        typer.echo(
+            f"Error: No batch ID found for {provider}. Run 'create' command first.",
+            err=True,
+        )
+        raise typer.Exit(1)
+
+    # Read batch ID
+    with open(filename) as f:
+        batch_id = f.read().strip()
+
+    typer.echo(f"{provider.upper()} BATCH RESULTS")
+    typer.echo("=" * 50)
+    typer.echo(f"Batch ID: {batch_id}")
+
+    # Check API key
+    if not check_api_key(provider):
+        raise typer.Exit(1)
+
+    try:
+        # Get results using BatchProcessor
+        if provider == "openai":
+            processor = BatchProcessor("openai/gpt-4o-mini", User)
+        elif provider == "anthropic":
+            processor = BatchProcessor("anthropic/claude-3-5-sonnet-20241022", User)
+
+        # Get batch info using list_batches to find our batch
+        all_batches = processor.list_batches(limit=100)
+        batch_info = None
+        for batch in all_batches:
+            if batch.id == batch_id:
+                batch_info = batch
+                break
+
+        if not batch_info:
+            typer.echo(f"Batch {batch_id} not found")
+            return
+
+        typer.echo(f"Status: {batch_info.status.value}")
+        typer.echo(f"Raw Status: {batch_info.raw_status}")
+
+        if batch_info.status != BatchStatus.COMPLETED:
+            typer.echo(f"Batch not completed yet: {batch_info.status.value}")
+            return
+
+        # Get all results using the new get_results method
+        all_results = processor.get_results(batch_id)
+        typer.echo(f"Total results: {len(all_results)}")
+
+        # Show each result with detailed info
+        for i, result in enumerate(all_results):
+            typer.echo(f"\n--- Result {i + 1} ---")
+            typer.echo(f"Custom ID: {result.custom_id}")
+            typer.echo(f"Success: {result.success}")
+
+            if result.success:
+                user = result.result
+                typer.echo(f"PARSED USER OBJECT:")
+                typer.echo(f"   Type: {type(user)}")
+                typer.echo(f"   Name: {user.name}")
+                typer.echo(f"   Age: {user.age}")
+                typer.echo(f"   JSON: {user.model_dump_json()}")
+                typer.echo(f"   Dict: {user.model_dump()}")
+
+                # Test that it's a real Pydantic object
+                typer.echo(f"   Is BaseModel: {isinstance(user, BaseModel)}")
+                typer.echo(f"   Is User: {isinstance(user, User)}")
+
+                # Test Pydantic methods
+                try:
+                    validated = User.model_validate(user.model_dump())
+                    typer.echo(f"   Re-validation: Works")
+                    typer.echo(f"   Re-validated: {validated}")
+                except Exception as e:
+                    typer.echo(f"   Re-validation: Failed - {e}")
+            else:
+                typer.echo(f"ERROR:")
+                typer.echo(f"   Type: {result.error_type}")
+                typer.echo(f"   Message: {result.error_message}")
+
+        # Test the utility functions
+        successful_results = filter_successful(all_results)
+        error_results = filter_errors(all_results)
+        extracted_users = extract_results(all_results)
+
+        typer.echo(f"\nUTILITY FUNCTIONS:")
+        typer.echo(f"Successful results: {len(successful_results)}")
+        typer.echo(f"Error results: {len(error_results)}")
+        typer.echo(f"Extracted users: {len(extracted_users)}")
+
+        if extracted_users:
+            typer.echo(f"\nEXTRACTED USER OBJECTS:")
+            for user in extracted_users:
+                typer.echo(
+                    f"  • {user.name}, age {user.age} (type: {type(user).__name__})"
+                )
+
+    except Exception as e:
+        typer.echo(f"Error showing results: {e}", err=True)
+        raise typer.Exit(1) from e
+
+
+def poll_for_results(
+    provider: str, batch_id: str, validate: bool, max_wait: int
+) -> list[User]:
+    """Poll for batch results until completion or timeout"""
+    import time
+
+    typer.echo(f"Polling {provider.upper()} batch every 30 seconds...")
+    typer.echo(f"Max wait time: {max_wait} seconds ({max_wait // 60} minutes)")
+    typer.echo(f"Batch ID: {batch_id}")
+    typer.echo()
+
+    start_time = time.time()
+    attempt = 1
+
+    while time.time() - start_time < max_wait:
+        typer.echo(f"Attempt {attempt} - Checking batch status...")
+
+        try:
+            if provider == "openai":
+                status, results = fetch_openai_results_with_status(batch_id, validate)
+            elif provider == "anthropic":
+                status, results = fetch_anthropic_results_with_status(
+                    batch_id, validate
+                )
+
+            if status == "completed" or status == "ended":
+                typer.echo(
+                    f"Batch completed after {int(time.time() - start_time)} seconds!"
+                )
+                return results
+            elif status in ["failed", "expired", "cancelled"]:
+                typer.echo(f"Batch {status}")
+                return []
+            else:
+                elapsed = int(time.time() - start_time)
+                remaining = max_wait - elapsed
+                typer.echo(
+                    f"Status: {status} | Elapsed: {elapsed}s | Remaining: {remaining}s"
+                )
+
+                if remaining > 30:
+                    typer.echo("Waiting 30 seconds before next check...")
+                    time.sleep(30)
+                else:
+                    typer.echo(f"Waiting {remaining} seconds...")
+                    time.sleep(remaining)
+                    break
+
+        except Exception as e:
+            typer.echo(f"Error during polling: {e}")
+            time.sleep(30)
+
+        attempt += 1
+
+    typer.echo(f"Timeout reached after {max_wait} seconds")
+    return []
+
+
+def fetch_openai_results_with_status(
+    batch_id: str, validate: bool
+) -> tuple[str, list[User]]:
+    """Fetch OpenAI batch results and return status"""
+    processor = BatchProcessor("openai/gpt-4o-mini", User)
+
+    # Get batch info
+    all_batches = processor.list_batches(limit=100)
+    batch_info = None
+    for batch in all_batches:
+        if batch.id == batch_id:
+            batch_info = batch
+            break
+
+    if not batch_info:
+        return "not_found", []
+
+    if batch_info.status != BatchStatus.COMPLETED:
+        return batch_info.raw_status, []
+
+    # Get results using the new get_results method
+    all_results = processor.get_results(batch_id)
+
+    successful_results = filter_successful(all_results)
+    error_results = filter_errors(all_results)
+    extracted_results = extract_results(all_results)
+
+    typer.echo(f"Successful extractions: {len(successful_results)}")
+    if error_results:
+        typer.echo(f"Failed extractions: {len(error_results)}")
+        # Show first few errors for debugging
+        for error in error_results[:3]:
+            typer.echo(f"   Error ({error.custom_id}): {error.error_message}")
+
+    if validate and extracted_results:
+        validate_results(extracted_results, "OpenAI")
+
+    return "completed", extracted_results
+
+
+def fetch_anthropic_results_with_status(
+    batch_id: str, validate: bool
+) -> tuple[str, list[User]]:
+    """Fetch Anthropic batch results and return status"""
+    processor = BatchProcessor("anthropic/claude-3-5-sonnet-20241022", User)
+
+    # Get batch info
+    all_batches = processor.list_batches(limit=100)
+    batch_info = None
+    for batch in all_batches:
+        if batch.id == batch_id:
+            batch_info = batch
+            break
+
+    if not batch_info:
+        return "not_found", []
+
+    # Check for various terminal states
+    if batch_info.status in [
+        BatchStatus.FAILED,
+        BatchStatus.CANCELLED,
+        BatchStatus.EXPIRED,
+    ]:
+        return batch_info.raw_status, []
+
+    if batch_info.status != BatchStatus.COMPLETED:
+        return batch_info.raw_status, []
+
+    # Get results using the new get_results method
+    all_results = processor.get_results(batch_id)
+
+    successful_results = filter_successful(all_results)
+    error_results = filter_errors(all_results)
+    extracted_results = extract_results(all_results)
+
+    typer.echo(f"Successful extractions: {len(successful_results)}")
+    if error_results:
+        typer.echo(f"Failed extractions: {len(error_results)}")
+        # Show first few errors for debugging
+        for error in error_results[:3]:
+            typer.echo(f"   Error ({error.custom_id}): {error.error_message}")
+
+    if validate and extracted_results:
+        validate_results(extracted_results, "Anthropic")
+
+    return "ended", extracted_results
+
+
+def fetch_openai_results(batch_id: str, validate: bool) -> list[User]:
+    """Fetch OpenAI batch results using BatchProcessor"""
+    processor = BatchProcessor("openai/gpt-4o-mini", User)
+
+    # Get batch info
+    all_batches = processor.list_batches(limit=100)
+    batch_info = None
+    for batch in all_batches:
+        if batch.id == batch_id:
+            batch_info = batch
+            break
+
+    if not batch_info:
+        typer.echo(f"Batch {batch_id} not found")
+        return []
+
+    typer.echo(f"Batch Status: {batch_info.status.value}")
+
+    if batch_info.status != BatchStatus.COMPLETED:
+        typer.echo(
+            f"Batch is still {batch_info.status.value}. Please wait and try again."
+        )
+        return []
+
+    # Get results using the new get_results method
+    all_results = processor.get_results(batch_id)
+
+    successful_results = filter_successful(all_results)
+    error_results = filter_errors(all_results)
+    extracted_results = extract_results(all_results)
+
+    typer.echo(f"Successful extractions: {len(successful_results)}")
+    if error_results:
+        typer.echo(f"Failed extractions: {len(error_results)}")
+        # Show first few errors for debugging
+        for error in error_results[:3]:
+            typer.echo(f"   Error ({error.custom_id}): {error.error_message}")
+
+    if validate and extracted_results:
+        validate_results(extracted_results, "OpenAI")
+
+    return extracted_results
+
+
+def fetch_anthropic_results(batch_id: str, validate: bool) -> list[User]:
+    """Fetch Anthropic batch results using BatchProcessor"""
+    processor = BatchProcessor("anthropic/claude-3-5-sonnet-20241022", User)
+
+    # Get batch info
+    all_batches = processor.list_batches(limit=100)
+    batch_info = None
+    for batch in all_batches:
+        if batch.id == batch_id:
+            batch_info = batch
+            break
+
+    if not batch_info:
+        typer.echo(f"Batch {batch_id} not found")
+        return []
+
+    typer.echo(f"Batch Status: {batch_info.status.value}")
+
+    if batch_info.status != BatchStatus.COMPLETED:
+        typer.echo(
+            f"Batch is still {batch_info.status.value}. Please wait and try again."
+        )
+        return []
+
+    # Get results using the new get_results method
+    all_results = processor.get_results(batch_id)
+
+    successful_results = filter_successful(all_results)
+    error_results = filter_errors(all_results)
+    extracted_results = extract_results(all_results)
+
+    typer.echo(f"Successful extractions: {len(successful_results)}")
+    if error_results:
+        typer.echo(f"Failed extractions: {len(error_results)}")
+        # Show first few errors for debugging
+        for error in error_results[:3]:
+            typer.echo(f"   Error ({error.custom_id}): {error.error_message}")
+
+    if validate and extracted_results:
+        validate_results(extracted_results, "Anthropic")
+
+    return extracted_results
+
+
+def fetch_google_results(batch_job_name: str, validate: bool) -> list[User]:
+    """Fetch Google batch results using BatchProcessor"""
+    try:
+        processor = BatchProcessor("google/gemini-2.5-flash", User)
+
+        # Get batch info
+        all_batches = processor.list_batches(limit=100)
+        batch_info = None
+        for batch in all_batches:
+            if batch.id == batch_job_name:
+                batch_info = batch
+                break
+
+        if not batch_info:
+            typer.echo(f"Batch {batch_job_name} not found")
+            return []
+
+        typer.echo(f"Batch Status: {batch_info.status.value}")
+
+        if batch_info.status != BatchStatus.COMPLETED:
+            typer.echo(
+                f"Batch is still {batch_info.status.value}. Please wait and try again."
+            )
+            return []
+
+        # Get results using the new get_results method
+        all_results = processor.get_results(batch_job_name)
+
+        successful_results = filter_successful(all_results)
+        error_results = filter_errors(all_results)
+        extracted_results = extract_results(all_results)
+
+        typer.echo(f"Successful extractions: {len(successful_results)}")
+        if error_results:
+            typer.echo(f"Failed extractions: {len(error_results)}")
+
+        if validate and extracted_results:
+            validate_results(extracted_results, "Google GenAI")
+
+        return extracted_results
+
+    except Exception as e:
+        typer.echo(f"Error fetching Google batch results: {e}")
+        return []
+
+
+def validate_results(results: list[User], provider_name: str) -> bool:
+    """Validate extracted results against expected results"""
+    expected_results = get_expected_results()
+
+    typer.echo(f"\nValidating {provider_name} Results:")
+    typer.echo("-" * 40)
+
+    if len(results) != len(expected_results):
+        typer.echo(f"Expected {len(expected_results)} results, got {len(results)}")
+        return False
+
+    # Sort both lists by name for comparison
+    results_sorted = sorted(results, key=lambda x: x.name)
+    expected_sorted = sorted(expected_results, key=lambda x: x.name)
+
+    all_correct = True
+    for i, (actual, expected) in enumerate(zip(results_sorted, expected_sorted)):
+        if actual.name == expected.name and actual.age == expected.age:
+            typer.echo(f"{i + 1}. {actual.name}, age {actual.age} - CORRECT")
+        else:
+            typer.echo(f"{i + 1}. Expected: {expected.name}, age {expected.age}")
+            typer.echo(f"    Got: {actual.name}, age {actual.age}")
+            all_correct = False
+
+    if all_correct:
+        typer.echo(f"\nAll {provider_name} extractions are correct!")
+    else:
+        typer.echo(f"\nSome {provider_name} extractions have errors")
+
+    return all_correct
+
+
+@app.command()
+def help():
+    """Show all available commands and usage examples"""
+    typer.echo("Unified Batch API Test Commands")
+    typer.echo("=" * 40)
+    typer.echo()
+
+    typer.echo("Available Commands:")
+    typer.echo("  • create         - Create a new batch job")
+    typer.echo("  • list-batches   - List all saved batch IDs")
+    typer.echo("  • fetch          - Fetch and validate batch results")
+    typer.echo("  • show-results   - Show detailed parsed Pydantic objects")
+    typer.echo("  • list-models    - Show supported models")
+    typer.echo("  • help           - Show this help message")
+    typer.echo()
+
+    typer.echo("Usage Examples:")
+    typer.echo("  # Create batch job (default: Google Gemini 2.5 Flash)")
+    typer.echo("  python run_batch_test.py create")
+    typer.echo()
+    typer.echo("  # Create batch job with specific model")
+    typer.echo("  python run_batch_test.py create --model 'openai/gpt-4o-mini'")
+    typer.echo()
+    typer.echo("  # List saved batch IDs")
+    typer.echo("  python run_batch_test.py list-batches")
+    typer.echo()
+    typer.echo("  # Fetch results with validation")
+    typer.echo("  python run_batch_test.py fetch --provider openai")
+    typer.echo()
+    typer.echo("  # Show detailed parsed objects")
+    typer.echo("  python run_batch_test.py show-results --provider anthropic")
+    typer.echo()
+    typer.echo("  # Poll every 30 seconds until batch completes (max 10 minutes)")
+    typer.echo("  python run_batch_test.py fetch --provider openai --poll")
+    typer.echo()
+    typer.echo("  # Poll with custom timeout (20 minutes)")
+    typer.echo(
+        "  python run_batch_test.py fetch --provider openai --poll --max-wait 1200"
+    )
+    typer.echo()
+
+
+@app.command()
+def list_models():
+    """List example models for each provider"""
+    typer.echo("Supported Models by Provider:")
+    typer.echo()
+
+    typer.echo("OpenAI:")
+    typer.echo("  • openai/gpt-4o-mini")
+    typer.echo("  • openai/gpt-4o")
+    typer.echo("  • openai/gpt-4-turbo")
+    typer.echo()
+
+    typer.echo("Anthropic:")
+    typer.echo("  • anthropic/claude-3-5-sonnet-20241022")
+    typer.echo("  • anthropic/claude-3-opus-20240229")
+    typer.echo("  • anthropic/claude-3-haiku-20240307")
+    typer.echo()
+
+    typer.echo("Google:")
+    typer.echo("  • google/gemini-2.5-flash")
+    typer.echo("  • google/gemini-2.0-flash-001")
+    typer.echo("  • google/gemini-pro")
+    typer.echo()
+
+    typer.echo("Usage: python run_batch_test.py create --model 'provider/model-name'")
+
+
+if __name__ == "__main__":
+    app()