--- title: Contribute to Instructor: Evals, Issues, and Pull Requests description: Join us in enhancing the Instructor library with evals, report issues, and submit pull requests on GitHub. Collaborate and contribute! --- # Contributing to Instructor We welcome contributions to Instructor! This page covers the different ways you can help improve the library. ## Ways to Contribute ### Evaluation Tests (Evals) Evals help us monitor the quality of both the OpenAI models and the Instructor library. To contribute: 1. **Explore Existing Evals**: Check out [our evals directory](https://github.com/instructor-ai/instructor/tree/main/tests/llm/test_openai/evals) 2. **Create a New Eval**: Add new pytest tests that evaluate specific capabilities or edge cases 3. **Follow the Pattern**: Structure your eval similar to existing ones 4. **Submit a PR**: We'll review and incorporate your eval Evals are run weekly, and results are tracked to monitor performance over time. ### Reporting Issues If you encounter a bug or problem, please [file an issue on GitHub](https://github.com/instructor-ai/instructor/issues) with: 1. A clear, descriptive title 2. Detailed information including: - The `response_model` you're using - The `messages` you sent - The `model` you're using - Steps to reproduce the issue - Expected vs. actual behavior - Your environment details (Python version, OS, package versions) ### Contributing Code We welcome pull requests! Here's the process: 1. **For Small Changes**: Feel free to submit a PR directly 2. **For Larger Changes**: [Start with an issue](https://github.com/instructor-ai/instructor/issues) to discuss approach 3. **Looking for Ideas?** Check issues labeled [help wanted](https://github.com/instructor-ai/instructor/labels/help%20wanted) or [good first issue](https://github.com/instructor-ai/instructor/labels/good%20first%20issue) ## Setting Up Your Development Environment ### Using UV (Recommended) UV is a fast Python package installer and resolver that makes development easier. 1. **Install UV** (official method): ```bash # macOS/Linux curl -LsSf https://astral.sh/uv/install.sh | sh # Windows PowerShell powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" ``` 2. **Install Project in Development Mode**: ```bash # Clone the repository git clone https://github.com/YOUR-USERNAME/instructor.git cd instructor # Install with development dependencies uv pip install -e ".[dev,docs]" ``` 3. **Adding New Dependencies**: ```bash # Add a regular dependency uv pip install some-package # Install a specific version uv pip install "some-package>=1.0.0,<2.0.0" ``` 4. **Common UV Commands**: ```bash # Update UV itself uv self update # Create a requirements file uv pip freeze > requirements.txt ``` ### Using Poetry Poetry provides comprehensive dependency management and packaging. 1. **Install Poetry**: ```bash curl -sSL https://install.python-poetry.org | python3 - ``` 2. **Install Dependencies**: ```bash # Clone the repository git clone https://github.com/YOUR-USERNAME/instructor.git cd instructor # Install with development dependencies poetry install --with dev,docs ``` 3. **Working with Poetry**: ```bash # Activate virtual environment poetry shell # Run a command in the virtual environment poetry run pytest # Add a dependency poetry add package-name # Add a development dependency poetry add --group dev package-name ``` ## Adding Support for New LLM Providers Instructor uses optional dependencies to support different LLM providers. Provider-specific utilities live in the `instructor/utils` directory. To add a new provider: 1. **Add Dependencies to pyproject.toml**: ```toml [project.optional-dependencies] # Add your provider my-provider = ["my-provider-sdk>=1.0.0,<2.0.0"] [dependency-groups] # Mirror in dependency groups my-provider = ["my-provider-sdk>=1.0.0,<2.0.0"] ``` 2. **Create Provider Client**: - Create a new file at `instructor/clients/client_myprovider.py` - Implement `from_myprovider` function that patches the provider's client 3. **Add Tests**: Create tests in `tests/llm/test_myprovider/` 4. **Document Installation**: ```bash # Installation command for your provider uv pip install "instructor[my-provider]" # or with poetry poetry install --with my-provider ``` 5. **Create Provider Utilities and Handlers**: - Add `instructor/utils/myprovider.py` with `reask` and `handle_*` helpers - Define `MYPROVIDER_HANDLERS` mapping `Mode` values to these functions 6. **Register the Provider**: - Update `instructor/utils/providers.py` with your provider enum value - Extend `get_provider` detection for your base URL 7. **Update `process_response.py`**: - Import your handlers and add them to `mode_handlers` - This script uses the handlers to prepare kwargs and parse results 8. **Write Documentation**: - Add a new markdown file in `docs/integrations/` for your provider - Update `mkdocs.yml` to include your new page - Make sure to include a complete example ## Development Workflow 1. **Fork the Repository**: Create your own fork of the project 2. **Clone and Set Up**: ```bash git clone https://github.com/YOUR-USERNAME/instructor.git cd instructor git remote add upstream https://github.com/instructor-ai/instructor.git ``` 3. **Create a Branch**: ```bash git checkout -b feature/your-feature-name ``` 4. **Make Changes, Test, and Commit**: ```bash # Run tests pytest tests/ -k 'not llm and not openai' # Skip LLM tests for faster local dev # Commit changes git add . git commit -m "Your descriptive commit message" ``` 5. **Keep Updated and Push**: ```bash git fetch upstream git rebase upstream/main git push origin feature/your-feature-name ``` 6. **Create a Pull Request**: Submit your PR with a clear description of changes ## Utility Scripts The `scripts/` directory contains utility scripts that help maintain code quality and documentation. These scripts are integrated into pre-commit hooks and can also be run manually. ### Available Scripts #### `make_clean.py` - Markdown File Cleaner Cleans markdown files by removing special whitespace characters and replacing em dashes with regular dashes. ```bash # Clean all markdown files python scripts/make_clean.py # Preview changes without modifying files python scripts/make_clean.py --dry-run ``` #### `check_blog_excerpts.py` - Blog Post Excerpt Validator Ensures all blog posts contain the `` tag for proper excerpt handling. ```bash # Check all blog posts python scripts/check_blog_excerpts.py ``` #### `make_sitemap.py` - Enhanced Documentation Sitemap Generator Generates an enhanced sitemap (`sitemap.yaml`) with AI-powered content analysis and cross-link suggestions. ```bash # Generate sitemap with default settings python scripts/make_sitemap.py # Customize settings python scripts/make_sitemap.py \ --root-dir docs \ --output-file sitemap.yaml \ --max-concurrency 10 ``` **Requirements for sitemap generation**: - OpenAI API key (set as `OPENAI_API_KEY` environment variable) - Additional dependencies: `openai`, `typer`, `rich`, `tenacity`, `pyyaml` ### Pre-commit Integration These scripts run automatically during the commit process: - **Markdown cleaning**: Runs on commits with markdown files in `docs/` - **Blog excerpt validation**: Runs on commits with blog post files ### Manual Usage You can run scripts manually for testing or one-time operations: ```bash # Test markdown cleaning python scripts/make_clean.py --dry-run # Check blog excerpts python scripts/check_blog_excerpts.py # Generate fresh sitemap python scripts/make_sitemap.py ``` For detailed documentation on each script, see the `scripts/README.md` file in the project repository. ## Using Cursor to Build PRs [Cursor](https://cursor.sh) is an AI-powered code editor that can help you contribute to Instructor. 1. **Getting Started with Cursor**: - Download Cursor from [cursor.sh](https://cursor.sh) - Open the Instructor project in Cursor - Cursor will automatically detect our rules in `.cursor/rules/` 2. **Using Cursor Rules**: - `new-features-planning`: Helps plan and structure new features - `simple-language`: Guidelines for writing clear documentation - `documentation-sync`: Ensures documentation stays in sync with code changes 3. **Creating PRs with Cursor**: - Use Cursor's Git integration to create a new branch - Make your changes with AI assistance - Create a PR with: ```bash # Use GitHub CLI to create the PR gh pr create -t "Your feature title" -b "Description of your changes" -r jxnl,ivanleomk ``` - Add `This PR was written by [Cursor](https://cursor.sh)` to your PR description 4. **Benefits of Using Cursor**: - AI helps generate code that follows our style guidelines - Simplifies PR creation process - Helps maintain documentation standards ## Code Style Guidelines We use the following tools to maintain code quality: - **Ruff**: For linting and formatting - **ty**: For type checking - **Pre-commit**: For automatic checks before committing ```bash # Install pre-commit hooks pip install pre-commit pre-commit install ``` Key style guidelines: - Use strict typing - Follow import order: standard lib → third-party → local - Use snake_case for functions/variables, PascalCase for classes - Write comprehensive docstrings for public API functions ### Conventional Comments When reviewing code or writing commit messages, we use conventional comments to make feedback clearer: ```