open-metadata--openmetadata/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## About OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance. This is a multi-module project with Java backend services, React frontend, Python ingestion framework, and comprehensive Docker infrastructure.

For architecture deep dives, entity/repository/resource patterns, and end-to-end checklists for adding new entities or connectors, see [DEVELOPER.md](DEVELOPER.md).

## Architecture Overview

- **Backend**: Java 21 + Dropwizard REST API framework, multi-module Maven project
- **Frontend**: React + TypeScript, built with Webpack and Yarn; component library via `openmetadata-ui-core-components` (Tailwind CSS v4 with `tw:` prefix, react-aria-components foundation)
- **Ingestion**: Python 3.10-3.11 with Pydantic 2.x, 75+ data source connectors
- **Database**: MySQL (default) or PostgreSQL with Flyway migrations
- **Search**: Elasticsearch 7.17+ or OpenSearch 2.6+ for metadata discovery
- **Infrastructure**: Apache Airflow for workflow orchestration

## Environment Setup

### Python Virtual Environment (REQUIRED)

**You MUST activate the Python venv before any Python work.** OpenMetadata supports Python 3.10-3.11; 3.11 is recommended.

```bash
# First-time setup (creates venv at repo root):
# python3.11 -m venv env

# ALWAYS activate before running Python, make generate, make install_dev, etc:
source env/bin/activate

# Verify:
python --version   # Should show Python 3.10.x or 3.11.x
```

**In worktrees**: When Claude Code creates a Git worktree, the venv from the main repo is NOT copied. You need to either:
- Create a new venv in the worktree: `python3.11 -m venv env && source env/bin/activate && cd ingestion && make install_dev`
- Or symlink the main repo's venv: `ln -s /path/to/main-repo/env env`

### Initial Dev Environment Setup

After activating the venv, install all dependencies:

```bash
source env/bin/activate

# Install ingestion module with all dev dependencies (required before make generate)
cd ingestion
make install_dev_env            # Full dev environment (edit mode + all extras)
# OR for lighter install:
make install_dev                # Just dev dependencies
cd ..

# Generate Pydantic models from JSON schemas (required after schema changes)
make generate

# Install UI dependencies
make yarn_install_cache
```

### Other Environment Notes

- **Java**: Java 21 required. Use `mvn` (Maven) for backend builds.
- **Node/Yarn**: Use `yarn` (not `npm`) for frontend. Frontend root is `openmetadata-ui/src/main/resources/ui/`.
- **Docker services**: Development services (MySQL, Elasticsearch, etc.) run via `docker/development/docker-compose.yml`:
  ```bash
  docker compose -f docker/development/docker-compose.yml up -d
  ```

## Essential Development Commands

### Prerequisites and Setup
```bash
make prerequisites              # Check system requirements
source env/bin/activate         # ALWAYS activate venv first
cd ingestion && make install_dev_env  # Install Python dev dependencies
make generate                  # Generate Pydantic models from JSON schemas
make yarn_install_cache        # Install UI dependencies
```

### Frontend Development
```bash
cd openmetadata-ui/src/main/resources/ui
yarn start                     # Start development server on localhost:3000
yarn test                      # Run Jest unit tests
yarn test path/to/test.spec.ts # Run a specific test file
yarn test:watch               # Run tests in watch mode
yarn playwright:run            # Run E2E tests
yarn lint                      # ESLint check
yarn lint:fix                  # ESLint with auto-fix
yarn build                     # Production build
```

### Frontend CI Checkstyle (run before PR to match CI)
```bash
cd openmetadata-ui/src/main/resources/ui
yarn ui-checkstyle:changed         # One-shot checkstyle for changed files (excludes tsc)
yarn organize-imports:cli <files>  # Sort and organize imports
yarn lint:fix                      # ESLint auto-fix
yarn pretty:base --write <files>   # Prettier formatting
yarn license-header-fix <files>    # Add Apache 2.0 license headers
yarn i18n                          # Sync all 17 locale files with en-us.json
yarn generate:app-docs             # Regenerate application documentation
npx tsc --noEmit                   # TypeScript type check (catches errors early)
```

### Backend Development
```bash
mvn clean package -DskipTests  # Build without tests
mvn clean package -DonlyBackend -pl !openmetadata-ui  # Backend only
mvn test                       # Run unit tests
mvn verify                     # Run integration tests
mvn spotless:apply             # Format Java code
```

### Python Ingestion Development
```bash
cd ingestion
make install_dev_env           # Install in development mode
make generate                  # Generate Pydantic models from JSON schemas
make unit_ingestion_dev_env    # Run unit tests
make py_format                 # Apply ruff lint-fix + format
make py_format_check           # Verify lint + format (matches CI; catches non-auto-fixable issues)
make static-checks             # Run type checking with basedpyright
```

### Full Local Environment
```bash
./docker/run_local_docker.sh -m ui -d mysql        # Complete local setup with UI
./docker/run_local_docker.sh -m no-ui -d postgresql # Backend only with PostgreSQL
./docker/run_local_docker.sh -s true               # Skip Maven build step
```

### Testing
```bash
make run_e2e_tests             # Full E2E test suite
make unit_ingestion            # Python unit tests with coverage
yarn test:coverage             # Frontend test coverage
```

### Backend Integration Tests
All backend API integration tests MUST be placed in `openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/` directory. Tests should:
- Use naming convention `*IT.java` (Integration Test)
- Extend `BaseEntityIT<T, K>` for entity CRUD tests
- Be designed to run concurrently (use `@Execution(ExecutionMode.CONCURRENT)`)
- Use `TestNamespace` for test isolation
- Use `SdkClients` for API calls (e.g., `SdkClients.adminClient().tables().create(...)`)

```bash
# Run a specific integration test
mvn test -pl openmetadata-integration-tests -Dtest=TaskResourceIT

# Run all integration tests
mvn test -pl openmetadata-integration-tests
```

## Code Generation and Schemas

OpenMetadata uses a schema-first approach with JSON Schema definitions driving code generation:

```bash
make generate                  # Generate all models from schemas
make py_antlr                  # Generate Python ANTLR parsers
make js_antlr                  # Generate JavaScript ANTLR parsers
yarn parse-schema              # Parse JSON schemas for frontend (connection and ingestion schemas)
```

### Schema Architecture
- **Source schemas** in `openmetadata-spec/` define the canonical data models
- **Connection schemas** are pre-processed at build time via `parseSchemas.js` to resolve all `$ref` references
- **Application schemas** in `openmetadata-ui/.../ApplicationSchemas/` are resolved at runtime using `schemaResolver.ts`
- JSON schemas with `$ref` references to external files require resolution before use in forms

## Key Directories

- `openmetadata-service/` - Core Java backend services and REST APIs
- `openmetadata-ui/src/main/resources/ui/` - React frontend application
- `ingestion/` - Python ingestion framework with connectors
- `openmetadata-spec/` - JSON Schema specifications for all entities
- `bootstrap/sql/` - Database schema migrations and sample data
- `conf/` - Configuration files for different environments
- `docker/` - Docker configurations for local and production deployment

## Development Workflow

1. **Schema Changes**: Modify JSON schemas in `openmetadata-spec/`, then run `mvn clean install` on openmetadata-spec to update models
2. **Backend**: Develop in Java using Dropwizard patterns, test with `mvn test`, format with `mvn spotless:apply`
3. **Frontend**: Use React/TypeScript with components from `openmetadata-ui-core-components`, test with Jest/Playwright
4. **Ingestion**: Python connectors follow plugin pattern, use `make install_dev_env` for development
5. **Full Testing**: Use `make run_e2e_tests` before major changes

## Frontend Architecture Patterns

### React Component Patterns
- **File Naming**: Components use `ComponentName.component.tsx`, interfaces use `ComponentName.interface.ts`
- **State Management**: Use `useState` with proper typing, avoid `any`
- **Side Effects**: Use `useEffect` with proper dependency arrays
- **Performance**: Use `useCallback` for event handlers, `useMemo` for expensive computations
- **Custom Hooks**: Prefix with `use`, place in `src/hooks/`, return typed objects
- **Internationalization**: Use `useTranslation` hook from react-i18next, access with `t('key')`
- **Component Structure**: Functional components only, no class components
- **Props**: Define interfaces for all component props, place in `.interface.ts` files
- **Loading States**: Use object state for multiple loading states: `useState<Record<string, boolean>>({})`
- **Error Handling**: Use `showErrorToast` and `showSuccessToast` utilities from ToastUtils
- **Navigation**: Use `useNavigate` from react-router-dom, not direct history manipulation
- **Data Fetching**: Async functions with try-catch blocks, update loading states appropriately

### State Management
- Use Zustand stores for global state (e.g., `useLimitStore`, `useWelcomeStore`)
- Keep component state local when possible with `useState`
- Use context providers for feature-specific shared state (e.g., `ApplicationsProvider`)

### Forms

- **Building forms**: New forms use the `react-hook-form` + `react-aria` stack from `@openmetadata/ui-core-components` (`getField`/`FieldProp`/`FieldTypes`/`HookForm`/`FormFields`). A form is `FieldProp[]` config objects + RHF state + a pure values→payload transform. The full reference is [`openmetadata-ui/src/main/resources/ui/docs/formutils.md`](openmetadata-ui/src/main/resources/ui/docs/formutils.md). Do not use the legacy Ant Design `getField`/`generateFormFields` from `@utils/formUtils` for new forms.

### Styling

- **Component Library**: Use components from `openmetadata-ui-core-components` for all new UI work. This is the canonical component library — do not use MUI or introduce new MUI dependencies.
- **Available Components**: Button, Input, Select, Modal, Table, Tabs, Pagination, Badge, Avatar, Checkbox, Dropdown, Form, Card, Tooltip, Toggle, Slider, Textarea, Tags, and more — all in `openmetadata-ui-core-components/src/main/resources/ui/src/components/`
- **Tailwind Classes**: All Tailwind utility classes must use the `tw:` prefix (e.g., `tw:flex`, `tw:text-sm`, `tw:bg-blue-500`) to avoid conflicts with existing Ant Design/Less styles
- **Design Tokens**: Use CSS custom properties defined in `openmetadata-ui-core-components/src/main/resources/ui/src/styles/globals.css`. Never use hardcoded color or spacing values. Semantic tokens include:
  - Text: `--color-text-primary`, `--color-text-secondary`, `--color-text-tertiary`, `--color-text-error-primary`, etc.
  - Border: `--color-border-primary`, `--color-border-secondary`, `--color-border-error`, `--color-border-brand`, etc.
  - Background: `--color-bg-primary`, `--color-bg-secondary`, `--color-bg-error-primary`, `--color-bg-brand-solid`, etc.
  - Shadows: `--shadow-xs` through `--shadow-3xl`
  - Border radius: `--radius-none` through `--radius-full`
- **Color Usage**: Full token reference, dark mode guide, and anti-pattern cheat sheet: [`openmetadata-ui/src/main/resources/ui/docs/colors.md`](openmetadata-ui/src/main/resources/ui/docs/colors.md). Always consult this before choosing any color class.
- **MUI**: Do not use MUI — we are actively removing MUI from the codebase. Do not import from `@mui/*` or `@emotion/*`
- **Legacy**: Ant Design components remain in existing code but should be replaced with `openmetadata-ui-core-components` equivalents when refactoring
- Do not add unnecessary spacing between logs and code.
- In Java, avoid wildcards imports (e.g., use `import java.util.List;` instead of `import java.util.*;`)
- Custom styles in `.less` files with component-specific naming (legacy pattern, avoid for new code)
- Follow BEM naming convention for custom CSS classes when writing raw CSS

### UI considerations

- Do not use string literals at any place. You should use useTranslation hook and use it like const {t} = useTranslation(). And for example if you want to have "Run" as string, you should be using { t('label.run') }, this label is defined in locales.


### Application Configuration
- Applications use `ApplicationsClassBase` for schema loading and configuration
- Dynamic imports handle application-specific schemas and assets
- Form schemas use React JSON Schema Form (RJSF) with custom UI widgets

### Service Utilities
- Each service type has dedicated utility files (e.g., `DatabaseServiceUtils.tsx`)
- Connection schemas are imported statically and pre-resolved
- Service configurations use switch statements to map types to schemas

### Type Safety
- All API responses have generated TypeScript interfaces in `generated/`
- Custom types extend base interfaces when needed
- Avoid type assertions unless absolutely necessary
- Use discriminated unions for action types and state variants

## Database and Migrations

- Flyway handles schema migrations in `bootstrap/sql/migrations/`
- Use Docker containers for local database setup
- Default MySQL, PostgreSQL supported as alternative
- Sample data loaded automatically in development environment

## Security and Authentication

- JWT-based authentication with OAuth2/SAML support
- Role-based access control defined in Java entities
- Security configurations in `conf/openmetadata.yaml`
- Never commit secrets - use environment variables or secure vaults

## Code Generation Standards

### Comments Policy
- **Do NOT add unnecessary comments** - write self-documenting code
- **NEVER add single-line comments that describe what the code obviously does**
- Only include comments for:
    - Complex business logic that isn't obvious
    - Non-obvious algorithms or workarounds
    - Public API JavaDoc documentation
    - TODO/FIXME with ticket references
- Bad examples (NEVER do this):
    - `// Create user` before `createUser()`
    - `// Get client` before `SdkClients.adminClient()`
    - `// Verify domain is set` before `assertNotNull(entity.getDomain())`
    - `// User names are lowercased` when the code `toLowerCase()` makes it obvious
- If the code needs a comment to be understood, refactor the code to be clearer instead

### Java Code Requirements

**Always run `mvn spotless:apply` before you finish any task that touched
`.java` files.** CI runs `mvn spotless:check` and will fail the PR otherwise —
the bot's exact suggestion is "Please run `mvn spotless:apply` in the root of
your repository and commit the changes to this PR." Scope the run with
`-pl <module>` for speed if only one module changed. When asked to "fix
checkstyle" / "fix Java formatting" / "apply spotless", invoke the
`java-checkstyle` skill (see `.claude/skills/java-checkstyle/`) rather than
hand-editing formatting.

#### Method Size and Complexity (Kafka-Grade Standards)
- **Methods must be small and focused — aim for 15 lines or fewer** (excluding blank lines and braces). A method longer than that is almost always hiding multiple responsibilities; break it into smaller methods with descriptive names. "Meaningful" means each method does one nameable thing — if you can't fit the body comfortably on a screen, it's too big.
- **One return statement per method, placed at the end.** No early-return guard clauses, no scattered returns in the middle. Initialize a `result` variable, structure the work as `if/else`, or extract a helper — the control flow then stays linear and easy to reason about. (Returns inside `lambda` bodies, `switch` expressions, and anonymous classes are scoped to those constructs and don't count against the outer method.)
  ```java
  // BAD: four scattered early returns
  Map<UUID, X> compute(List<EntityInterface> entities) {
    if (entities == null) return Collections.emptyMap();
    if (entities.isEmpty()) return Collections.emptyMap();
    if (!supportsX(entities.get(0))) return null;
    Map<UUID, X> prefetched = doWork(entities);
    if (prefetched.isEmpty()) return null;
    return prefetched;
  }

  // GOOD: single trailing return; guards become extracted helpers + a result variable
  Map<UUID, X> compute(List<EntityInterface> entities) {
    Map<UUID, X> result = null;
    if (entities != null && !entities.isEmpty() && supportsX(entities.get(0))) {
      Map<UUID, X> prefetched = doWork(entities);
      if (!prefetched.isEmpty()) {
        result = prefetched;
      }
    }
    return result;
  }
  ```
- **Maximum 3 levels of nesting.** Don't flatten by sprinkling early returns — extract a named helper or combine conditions into a single boolean:
  ```java
  // BAD: deeply nested
  if (entity != null) {
      if (entity.isActive()) {
          if (hasPermission(entity)) {
              process(entity);
          }
      }
  }

  // GOOD: extract the eligibility check
  if (isEligibleForProcessing(entity)) {
    process(entity);
  }

  private boolean isEligibleForProcessing(Entity entity) {
    return entity != null && entity.isActive() && hasPermission(entity);
  }
  ```
- **Maximum 10 cyclomatic complexity.** Extract complex conditions into named methods:
  ```java
  // BAD: complex inline boolean
  if (entity.getStatus() == ACTIVE && entity.getOwner() != null
      && !entity.isDeleted() && entity.getVersion() > 0.1) { ... }

  // GOOD: self-documenting
  if (isEligibleForProcessing(entity)) { ... }
  ```
- **Maximum 5 parameters.** Introduce a parameter object or builder for more.
- **Each method does one thing.** If you can describe what a method does using "and" or "then", it should be two methods.

#### Naming and Readability
- Names should make code read like prose — if you need a comment, the name isn't good enough
- **Methods**: verb phrases — `calculateScore()`, `findByName()`, `isValid()`
- **Booleans**: question-form — `isActive`, `hasPermission`, `canRetry` (never `flag`, `status`, `check`)
- **Variables**: descriptive, no abbreviations — `entityReference` not `er`, `retryCount` not `rc`
- **Constants**: `UPPER_SNAKE_CASE` — `MAX_RETRY_COUNT`, `DEFAULT_PAGE_SIZE`
- **No single-letter variables** except in short lambdas or loop indices

#### Immutability and Defensive Design
- Use `final` on local variables and parameters that don't change (which is most of them)
- Use `final` on fields set in the constructor
- Return `Collections.unmodifiableList()` / `List.copyOf()` from public methods, never expose internal mutable collections
- Utility classes must be `final` with a private constructor
- Prefer `record` for immutable data carriers where appropriate

#### Error Handling
- **No empty catch blocks** — at minimum, log the exception
- **No `catch (Exception e)`** — catch the specific type you expect
- **No `e.printStackTrace()`** — use the logger
- **Error messages must include context**: `"Table '%s' not found in database '%s'"` not just `"Not found"`
- **No `throw` or `return` inside `finally` blocks** — they mask the original exception
- **No exceptions for flow control** — use conditionals for expected cases

#### No Magic Strings — Define Constants
- **Never use raw string literals in `.equals()`, `.contains()`, or `switch` cases** — define a constant or use an existing enum
- If an enum already exists in `openmetadata-spec/` schemas for those values, use it
- If the same string appears in more than one place, it must be a named constant
- **One definition, one location** — don't define the same constant in multiple classes
- Prefer enums over string constants when the values form a closed set:
  ```java
  // BAD: magic strings scattered everywhere
  if (taskStatus.equals("Open")) { ... }
  if (config.getResources().get(0).equals("all")) { ... }

  // GOOD: use existing enums or define constants
  if (taskStatus == TaskStatus.OPEN) { ... }
  private static final String RESOURCE_ALL = "all";
  ```

#### No Convoluted if/else Chains
- **More than 3 `else if` branches means the structure is wrong — refactor:**
  - `else if` chain on `instanceof` → `switch` with pattern matching (Java 21)
  - `else if` chain on enum values → `switch` expression
  - `else if` chain on `.equals("string")` → `Map` dispatch or enum lookup
  - `else if` chain on `.contains("string")` → `Map` or list of predicates
- **Repeated compound conditions** (same multi-part `&&`/`||` expression in multiple places) → extract into a named method or `Set.contains()`
  ```java
  // BAD: 3-part condition repeated 3 times across the file
  if (!tenantId.equals("common") && !tenantId.equals("organizations")
      && !tenantId.equals("consumers")) { ... }

  // GOOD: define once, use everywhere
  private static final Set<String> MULTI_TENANT_IDS =
      Set.of("common", "organizations", "consumers");

  private boolean isSingleTenant(String tenantId) {
      return !MULTI_TENANT_IDS.contains(tenantId);
  }
  ```

#### No Code Duplication
- If the same logic exists in two places, extract to a shared method
- Near-identical methods (e.g., same logic for OpenSearch and ElasticSearch) should share a common implementation with only the engine-specific parts varying
- Copy-pasted blocks within the same file should be extracted into a parameterized method

#### Class Size
- **Classes should be under 500 lines.** Over 1000 lines is a design problem.
- If a class is large, look for clusters of methods that operate on the same subset of fields — extract them into a new focused class
- Resource classes should be thin orchestrators
- Repository classes handle data access, not business logic

#### Modern Java (Java 21)
- Use try-with-resources for all `AutoCloseable` objects
- Use diamond operator `<>` — `new ArrayList<>()` not `new ArrayList<String>()`
- Use pattern matching: `if (obj instanceof String s)` instead of cast
- Use `switch` expressions instead of `if/else if` chains on enums or types
- Use `List.of()`, `Map.of()`, `Set.of()` for immutable collection literals
- Use `Optional` correctly: never as a field type, never as a parameter, never assign `null` to it
- Use text blocks `"""` for multi-line strings
- **Use `SequencedCollection` accessors on Lists/Deques** — `list.getFirst()` / `list.getLast()` (Java 21) instead of `list.get(0)` / `list.get(list.size() - 1)`. Same for `removeFirst()` / `removeLast()`. Reads more clearly and avoids off-by-one indexing.
- **Collection emptiness: use the project's `nullOrEmpty(...)` helper** from `org.openmetadata.common.utils.CommonUtil` instead of hand-rolling `coll != null && !coll.isEmpty()` (or its negation). It's the established idiom across this codebase, handles `null` correctly, and reads as a single semantic check. Same applies to `String` checks — use `nullOrEmpty(str)` not `str != null && !str.isEmpty()`.
  ```java
  // BAD
  if (entities != null && !entities.isEmpty()) {
    process(entities.get(0));
  }

  // GOOD
  if (!nullOrEmpty(entities)) {
    process(entities.getFirst());
  }
  ```

#### Common Bug Patterns to Avoid
- `equals()` without `hashCode()` (or vice versa)
- `equals()` on arrays — use `Arrays.equals()`
- Ignoring return values of `String.replace()`, `File.delete()`
- `collection.size() == 0` — use `collection.isEmpty()`
- String concatenation inside loops — use `StringBuilder`
- `synchronized` on non-final fields — the lock reference can change
- `toLowerCase()` without `Locale` — always use `toLowerCase(Locale.ROOT)`
- Double map lookups — use `computeIfAbsent()` or `getOrDefault()`

#### Testing
- Generate production-ready code, not tutorial code
- Create integration tests in `openmetadata-integration-tests` for new API endpoints
- **Never use `Thread.sleep()` in tests** — use condition-based waiting or `Awaitility`
- Bug fixes must include a test that fails without the fix
- 90% line coverage target on changed classes

#### Structure
- Do not use Fully Qualified Names in code (e.g., `org.openmetadata.schema.type.Status`) — import the class instead
- Do not import wildcard packages — import exactly the required classes
- No commented-out code — version control maintains history
- No TODOs without a ticket reference
- One statement per line — no `if (x) return y;` on one line

### TypeScript/Frontend Code Requirements

**Always run the UI checkstyle sequence before you finish any task that
touched `.ts`/`.tsx`/`.js`/`.jsx`/`.json` under
`openmetadata-ui/src/main/resources/ui/src/`, `.../playwright/`, or
`openmetadata-ui-core-components/src/main/resources/ui/src/`.** CI's
`UI Checkstyle / lint-src|lint-playwright|lint-core-components` jobs fail the
PR otherwise. The order matters — run `organize-imports-cli`, then
`eslint --fix`, then `prettier --write`; reversing organize-imports and
prettier leaves a dirty diff (organize-imports uses 4-space indentation,
prettier uses 2 + trailing commas). When asked to "fix UI checkstyle" / "run
prettier" / "fix UI lint", invoke the `ui-checkstyle` skill (see
`.claude/skills/ui-checkstyle/`) rather than hand-editing formatting.

- **NEVER use `any` type** in TypeScript code - always use proper types
- Use `unknown` when the type is truly unknown and add type guards
- Import types from existing type definitions (e.g., `RJSFSchema` from `@rjsf/utils`)
- Add `// eslint-disable-next-line` comments only when absolutely necessary
- **Import Organization** — use `yarn organize-imports:cli` to auto-sort. Order:
  1. External libraries (React, etc.)
  2. Internal absolute imports from `generated/`, `constants/`, `hooks/`, etc.
  3. Relative imports for utilities and components
  4. Asset imports (SVGs, styles)
  5. Type imports grouped separately when needed

#### CI Checkstyle Rules (enforced on every PR)
These checks run automatically in CI. Code that violates them **will not merge**.
- **No `console.log/warn/error`** — `no-console` rule is enforced. Use the logger or remove.
- **Use `===` not `==`** — `eqeqeq` (smart mode, except for `null` checks)
- **Max 200 characters per line** — break long lines
- **Self-closing components** — `<Div />` not `<Div></Div>`
- **Sort JSX props alphabetically** — callbacks last
- **Space after `//` in comments** — `// comment` not `//comment`
- **Blank lines** before `function`, `class`, `export`, `return` statements
- **Use `it()` consistently in tests** — don't mix `test()` and `it()`
- **Blank lines around `describe`, `it`, `beforeEach`** in test files
- **JSON keys sorted alphabetically** in locale files (`src/locale/**/*.json`)
- **Apache 2.0 license header** on every new source file — run `yarn license-header-fix`
- **i18n keys synced** — after adding keys to `en-us.json`, run `yarn i18n` to sync all 17 locales
- **Prettier formatting** — 2-space indent, single quotes, strict HTML whitespace

#### Playwright Test Rules (lint-playwright)
- **No `waitForLoadState('networkidle')`** — flaky, use web-first assertions
- **No `page.pause()`** — remove before committing
- **No `.only` on tests** — blocks all other tests in CI
- Prefer `expect(locator).toBeVisible()` over manual `waitForSelector` checks
- Don't use `{ force: true }` — fix the locator instead
- Use locators, not element handles

### Python Code Requirements
- **Use pytest, not unittest** - write tests using pytest style with plain `assert` statements
- Use pytest fixtures for test setup instead of `setUp`/`tearDown` methods
- Use `unittest.mock` for mocking (MagicMock, patch) - this is compatible with pytest
- Test classes should not inherit from `TestCase` - use plain classes prefixed with `Test`
- Use `assert x == y` instead of `self.assertEqual(x, y)`
- Use `assert x is None` instead of `self.assertIsNone(x)`
- Use `assert "text" in string` instead of `self.assertIn("text", string)`

### Python Ingestion Connector Guidelines
- **Keep connector-specific logic in connector-specific files**, not in generic/shared files like `builders.py`
- Example: Redshift IAM auth should be in `ingestion/src/metadata/ingestion/source/database/redshift/connection.py`, not in `ingestion/src/metadata/ingestion/connections/builders.py`
- This keeps the codebase modular and prevents generic utilities from becoming cluttered with connector-specific edge cases
- **Use `model_str()` for Pydantic RootModel to string conversion** — OpenMetadata schema types like `ColumnName`, `EntityName`, `FullyQualifiedEntityName`, and `UUID` are Pydantic `RootModel[str]` subclasses where `str()` returns `"root='value'"` instead of the raw value. Always use `model_str()` from `metadata.ingestion.ometa.utils` instead of manual `hasattr(x, "root")` / `str(x.root)` checks.

### Caching
- **All caches MUST be bounded.** Never use a bare `dict` / `HashMap` / `Map` as a cache without an explicit size cap — they grow with the input and cause OOMs on large catalogs/ingestions. The only exception is when the user explicitly asks for an unbounded cache for a specific case.
- Pick a sane default (typically 100–1000 entries depending on entity size); if you're unsure, ask the user.
- **Python**: use `collections.OrderedDict` with `popitem(last=False)` eviction after insert, `@functools.lru_cache(maxsize=N)`, or `cachetools.LRUCache`. Cache both hits and misses (negative caching) — repeated unresolvable lookups are a common hot path.
- **Java**: use Caffeine (`Caffeine.newBuilder().maximumSize(N).build()`) or Guava `CacheBuilder.newBuilder().maximumSize(N).build()`. Never a bare `HashMap`.
- **TypeScript**: use `lru-cache` — never a bare `Map` or plain object.
- **Before adding a cache, check whether the underlying call is already cached at a lower layer.** Example: `OpenMetadata._search_es_entity` is `@lru_cache(maxsize=512)`, so wrapping `get_entity_from_es` / `es_search_container_by_path` calls in a local dict cache is redundant — drop the local cache and rely on the existing LRU.

### Testing Philosophy
- **Test real behavior, not mock wiring** - if a test requires mocking 3+ classes just to verify a method call, it's testing the wrong thing
- **Prefer integration tests** over heavily-mocked unit tests. This project has full integration test infrastructure (OpenMetadataApplicationTest, Docker containers, real OpenSearch). Use it.
- **Mocks are for boundaries, not internals** - mock external services (HTTP clients, third-party APIs), not your own classes. If you're mocking static methods left and right to test internal plumbing, write an integration test instead.
- **A test that mocks everything proves nothing** - it only verifies that your mocks are wired correctly, not that the system works
- **Ask "what breaks if this test passes but the code is wrong?"** - if the answer is "nothing, because everything real is mocked out", delete the test and write a better one
- **Test the outcome, not the implementation** - assert on observable results (API responses, database state, stats values) rather than verifying internal method calls with `verify()`

### Response Format
- Provide clean code blocks without unnecessary explanations
- Assume readers are experienced developers
- Focus on functionality over education