chore: import upstream snapshot with attribution

.agents/skills/java-checkstyle/SKILL.md

@@ -0,0 +1,58 @@
+---
+name: java-checkstyle
+description: Run `mvn spotless:apply` to fix Java checkstyle / formatting failures and verify the result. Run after authoring or modifying any `.java` files, or when CI reports a "Java checkstyle failed" / "Fix Java checkstyle" issue on a PR.
+---
+
+# Java Checkstyle / Spotless (Codex agent)
+
+OpenMetadata enforces Java formatting via the Spotless Maven plugin. Every CI
+build runs `mvn spotless:check` and fails the PR if any file is not formatted.
+
+## When to activate
+
+- The user asks to "fix checkstyle", "fix Java formatting", "apply spotless",
+  "run spotless", "format Java", or similar.
+- CI posts a `Java checkstyle failed` / `Fix Java checkstyle` comment on a PR
+  (the bot's exact phrasing is "Please run `mvn spotless:apply` in the root of
+  your repository and commit the changes to this PR").
+- After you have finished authoring or editing any `.java` files — before
+  opening a PR or pushing a commit that touches Java.
+
+## Procedure
+
+1. From the repo root run Spotless:
+
+   ```bash
+   mvn spotless:apply                 # formats everything
+   # or
+   mvn -pl <module> spotless:apply    # scope to a single Maven module for speed
+   # or
+   mvn spotless:check                 # verify only, without rewriting files
+   ```
+
+   Spotless is fast (seconds, no compilation). If it fails with a plugin error
+   rather than a formatting diff, surface the error and stop — do not try to
+   hand-edit formatting around the failure.
+
+2. Inspect the diff:
+
+   ```bash
+   git status --short
+   git diff --stat
+   ```
+
+   Expect changes only in `.java` (and possibly `pom.xml`) files. If Spotless
+   keeps rewriting a change you just made, re-read the root `pom.xml`'s
+   `spotless-maven-plugin` config — Spotless is the source of truth, not the
+   IDE.
+
+3. Only commit if the user asked to. Report the changed-file list first so the
+   user can decide whether to fold the reformat into the in-progress commit or
+   make a separate "Fix Java checkstyle" commit (matches the repo's existing
+   history for bot-triggered formatting-only commits).
+
+## Out of scope
+
+- UI / TypeScript formatting — use `yarn pretty` / ESLint flow (see AGENTS.md
+  UI section).
+- Python formatting — use `make py_format` (black + isort + pycln).

.agents/skills/ui-checkstyle/SKILL.md

@@ -0,0 +1,86 @@
+---
+name: ui-checkstyle
+description: Run the ESLint + Prettier + organize-imports sequence that CI's `UI Checkstyle` jobs (`lint-src`, `lint-playwright`, `lint-core-components`) run — on just the files the PR changed — and fail if any file ends up with a diff. Run after authoring or modifying any `.ts`/`.tsx`/`.js`/`.jsx`/`.json` under `openmetadata-ui/src/main/resources/ui/src/`, `.../playwright/`, or `openmetadata-ui-core-components/src/main/resources/ui/src/`, or when CI reports a `UI Checkstyle` failure on a PR.
+---
+
+# UI Checkstyle / ESLint + Prettier + organize-imports (Codex agent)
+
+The `UI Checkstyle` workflow (`.github/workflows/ui-checkstyle.yml`) has three
+per-area jobs — `lint-src`, `lint-playwright`, `lint-core-components`. Each
+reformats the files changed in the PR and fails if the reformat produces a
+diff, so the committed tree must already be formatted.
+
+## When to activate
+
+- The user asks to "fix UI checkstyle", "fix UI lint", "run prettier", "run
+  eslint", "fix UI format", or similar.
+- CI posts a `UI Checkstyle / lint-src|lint-playwright|lint-core-components`
+  failure (the bot surfaces the modified files in the job summary).
+- After you have finished authoring or editing any `.ts`/`.tsx`/`.js`/
+  `.jsx`/`.json` under the three UI trees — before opening a PR or pushing
+  a commit that touches UI.
+
+## Procedure
+
+1. Build the file list for each affected area:
+
+   ```bash
+   # repo root
+   git diff --name-only origin/main...HEAD -- \
+     'openmetadata-ui/src/main/resources/ui/src/**/*.{ts,tsx,js,jsx,json}' \
+     | sed 's|openmetadata-ui/src/main/resources/ui/||' > /tmp/src_files.txt
+
+   git diff --name-only origin/main...HEAD -- \
+     'openmetadata-ui/src/main/resources/ui/playwright/**/*.{ts,tsx,js,jsx}' \
+     | sed 's|openmetadata-ui/src/main/resources/ui/||' > /tmp/pw_files.txt
+
+   git diff --name-only origin/main...HEAD -- \
+     'openmetadata-ui-core-components/**/*.{ts,tsx,js,jsx,json}' \
+     | sed 's|openmetadata-ui-core-components/src/main/resources/ui/||' \
+     > /tmp/core_files.txt
+   ```
+
+   Skip any empty list — CI won't run that area's job either.
+
+2. From the matching working directory (`openmetadata-ui/src/main/resources/ui`
+   or `openmetadata-ui-core-components/src/main/resources/ui`), run the
+   three-step sequence that CI runs:
+
+   ```bash
+   # 1) imports first
+   cat /tmp/src_files.txt | xargs ./node_modules/.bin/organize-imports-cli
+
+   # 2) ESLint --fix
+   NODE_OPTIONS='--max-old-space-size=8192' cat /tmp/src_files.txt \
+     | xargs ./node_modules/.bin/eslint --no-error-on-unmatched-pattern --fix
+
+   # 3) prettier --write — MUST be last, because organize-imports-cli uses
+   #    4-space indentation and drops trailing commas; prettier restores them
+   #    to the repo's 2-space + trailing-comma style. Reversing the order
+   #    leaves CI with a dirty diff.
+   cat /tmp/src_files.txt \
+     | xargs ./node_modules/.bin/prettier \
+         --config './.prettierrc.yaml' --ignore-path './.prettierignore' \
+         --write
+   ```
+
+   Core-components has no `organize-imports-cli` wired up — skip step 1 there.
+
+3. Check the diff from the repo root:
+
+   ```bash
+   git status --short
+   git diff --stat
+   ```
+
+   If `git status --short` is empty you're done. Otherwise commit the
+   reformatting diff as its own `Fix UI checkstyle` commit, matching the
+   existing history for bot-triggered formatting-only commits — unless the
+   user asked you to fold it into the in-progress commit.
+
+## Out of scope
+
+- TypeScript type-check errors (`tsc`) — different jobs, different failure
+  modes, not auto-fixable by this skill.
+- Java formatting — use the `java-checkstyle` skill (`mvn spotless:apply`).
+- Python formatting — use `make py_format` (black + isort + pycln).

.devcontainer/dev/Dockerfile

@@ -0,0 +1,30 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+FROM mcr.microsoft.com/devcontainers/base:ubuntu-22.04
+
+# System libraries required for building OpenMetadata connectors
+# Mirrors: .github/actions/prepare-for-docker-build/action.yml
+RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
+    && apt-get install -y --no-install-recommends \
+        unixodbc-dev \
+        librdkafka-dev \
+        gcc \
+        libsasl2-dev \
+        build-essential \
+        libssl-dev \
+        libffi-dev \
+        libevent-dev \
+        wkhtmltopdf \
+        libkrb5-dev \
+        jq \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*

.devcontainer/dev/devcontainer.json

@@ -0,0 +1,46 @@
+{
+  "name": "OpenMetadata - Development",
+  "build": {
+    "dockerfile": "Dockerfile",
+    "context": "../.."
+  },
+  "features": {
+    "ghcr.io/devcontainers/features/docker-in-docker:2": {},
+    "ghcr.io/devcontainers/features/java:1": {
+      "version": "21",
+      "installMaven": true,
+      "mavenVersion": "3.9.9"
+    },
+    "ghcr.io/devcontainers/features/python:1": {
+      "version": "3.11"
+    },
+    "ghcr.io/devcontainers/features/node:1": {
+      "version": "22.17.0"
+    }
+  },
+  "postCreateCommand": "npm install -g yarn@1.22.22 --force && bash .devcontainer/dev/post-create.sh",
+  "customizations": {
+    "vscode": {
+      "extensions": [
+        "vscjava.vscode-java-pack",
+        "ms-python.python",
+        "ms-python.vscode-pylance",
+        "dbaeumer.vscode-eslint",
+        "esbenp.prettier-vscode",
+        "eamodio.gitlens",
+        "redhat.vscode-yaml"
+      ],
+      "settings": {
+        "editor.formatOnSave": true
+      }
+    }
+  },
+  "remoteEnv": {
+    "PATH": "${containerEnv:PATH}:${containerEnv:HOME}/.local/bin"
+  },
+  "remoteUser": "vscode",
+  "forwardPorts": [3000],
+  "portsAttributes": {
+    "3000": { "label": "Frontend Dev Server" }
+  }
+}

.devcontainer/dev/post-create.sh

@@ -0,0 +1,95 @@
+#!/usr/bin/env bash
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+
+echo "=== Setting up OpenMetadata Development Environment ==="
+
+cd "$REPO_ROOT"
+
+# Ensure ANTLR4 is available
+echo "Setting up ANTLR4..."
+# Ensure user-local binaries are available for this script and child processes.
+export PATH="$HOME/.local/bin:$PATH"
+
+ANTLR_VERSION=4.9.2
+ANTLR_JAR_DIR="$HOME/.local/lib"
+ANTLR_JAR_PATH="$ANTLR_JAR_DIR/antlr-${ANTLR_VERSION}-complete.jar"
+ANTLR_BIN_PATH="$HOME/.local/bin/antlr4"
+
+CURRENT_ANTLR_VERSION=""
+if command -v antlr4 &> /dev/null; then
+  CURRENT_ANTLR_VERSION="$(antlr4 -version 2>&1 | awk '{print $NF}')"
+fi
+
+if [ "$CURRENT_ANTLR_VERSION" != "$ANTLR_VERSION" ]; then
+  if ! command -v java &> /dev/null; then
+    echo "ERROR: Java runtime is required to run ANTLR."
+    exit 1
+  fi
+
+  mkdir -p "$HOME/.local/bin" "$ANTLR_JAR_DIR"
+  curl -fsSL "https://www.antlr.org/download/antlr-${ANTLR_VERSION}-complete.jar" -o "$ANTLR_JAR_PATH"
+
+  cat > "$ANTLR_BIN_PATH" <<'EOF'
+#!/usr/bin/env bash
+set -e
+exec java -jar "$HOME/.local/lib/antlr-4.9.2-complete.jar" "$@"
+EOF
+
+  chmod 755 "$ANTLR_BIN_PATH"
+fi
+
+# Install frontend dependencies
+echo "Installing frontend dependencies..."
+cd openmetadata-ui/src/main/resources/ui
+yarn install --frozen-lockfile
+cd -
+
+# Set up Python virtual environment for ingestion
+echo "Setting up Python ingestion environment..."
+if command -v python3 &> /dev/null; then
+  PYTHON_BIN="$(command -v python3)"
+elif command -v python &> /dev/null; then
+  PYTHON_BIN="$(command -v python)"
+else
+  echo "ERROR: Python is required but was not found in PATH."
+  exit 1
+fi
+
+echo "Running python version: $($PYTHON_BIN --version)"
+
+# Always use a dedicated venv in the devcontainer to avoid overwriting the host .venv
+# (which is volume-mounted and likely contains platform-incompatible binaries).
+VENV_DIR=".venv-devcontainer"
+
+"$PYTHON_BIN" -m venv "$VENV_DIR"
+source "$VENV_DIR/bin/activate"
+python -m pip install --upgrade pip
+
+# Run Make targets from repo root to ensure scripts/ and ingestion/ paths resolve correctly
+make install_dev generate
+
+# Verify prerequisites
+echo "Verifying prerequisites..."
+make prerequisites
+
+echo "=== Development environment ready ==="
+echo ""
+echo "Quick start commands:"
+echo "  Backend:   mvn clean package -DskipTests"
+echo "  Frontend:  cd openmetadata-ui/src/main/resources/ui && yarn start"
+echo "  Ingestion: source $VENV_DIR/bin/activate && cd ingestion"
+echo "  Format:    mvn spotless:apply (Java) | yarn lint:fix (Frontend)"

.devcontainer/full-stack/devcontainer.json

@@ -0,0 +1,53 @@
+{
+  "name": "OpenMetadata - Full Stack",
+  "dockerComposeFile": [
+    "../../docker/development/docker-compose.yml",
+    "docker-compose.yml"
+  ],
+  "service": "devcontainer",
+  "workspaceFolder": "/workspaces/OpenMetadata",
+  "features": {
+    "ghcr.io/devcontainers/features/docker-in-docker:2": {},
+    "ghcr.io/devcontainers/features/java:1": {
+      "version": "21",
+      "installMaven": true,
+      "mavenVersion": "3.9.9"
+    },
+    "ghcr.io/devcontainers/features/python:1": {
+      "version": "3.11"
+    },
+    "ghcr.io/devcontainers/features/node:1": {
+      "version": "22.17.0"
+    }
+  },
+  "postCreateCommand": "npm install -g yarn@1.22.22 --force && bash .devcontainer/dev/post-create.sh",
+  "customizations": {
+    "vscode": {
+      "extensions": [
+        "vscjava.vscode-java-pack",
+        "ms-python.python",
+        "ms-python.vscode-pylance",
+        "dbaeumer.vscode-eslint",
+        "esbenp.prettier-vscode",
+        "eamodio.gitlens",
+        "redhat.vscode-yaml"
+      ],
+      "settings": {
+        "editor.formatOnSave": true
+      }
+    }
+  },
+  "remoteEnv": {
+    "PATH": "${containerEnv:PATH}:${containerEnv:HOME}/.local/bin"
+  },
+  "remoteUser": "vscode",
+  "forwardPorts": [3000, 8585, 8586, 9200, 3306, 8080],
+  "portsAttributes": {
+    "3000": { "label": "Frontend Dev Server" },
+    "8585": { "label": "OpenMetadata API" },
+    "8586": { "label": "OpenMetadata Admin" },
+    "9200": { "label": "Elasticsearch" },
+    "3306": { "label": "MySQL" },
+    "8080": { "label": "Airflow" }
+  }
+}

.devcontainer/full-stack/docker-compose.yml

@@ -0,0 +1,30 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+services:
+  devcontainer:
+    build:
+      context: ../..
+      dockerfile: .devcontainer/dev/Dockerfile
+    volumes:
+      - ../..:/workspaces/OpenMetadata:cached
+    command: sleep infinity
+    depends_on:
+      openmetadata-server:
+        condition: service_started
+      ingestion:
+        condition: service_started
+      elasticsearch:
+        condition: service_healthy
+      mysql:
+        condition: service_healthy
+    networks:
+      - local_app_net

.dockerignore

@@ -0,0 +1,2 @@
+docker/development/docker-volume
+docker/docker-compose-quickstart/docker-volume

.git-blame-ignore-revs

@@ -0,0 +1 @@
+1294f93e15258c4d0ea2f8d4f0b746a6267dd6cc

.github/CODEOWNERS

@@ -0,0 +1,18 @@
+# Individual scopes
+
+# Review from UI owners for changes around UI code
+/openmetadata-ui/src/main/resources/ui/ @open-metadata/ui
+
+# Review from @chireg / @karan for changes around component library
+/openmetadata-ui-core-components/src/main/resources/ui/ @chirag-madlani @karanh37
+
+# Review from Backend owners for changes around Backend code
+/openmetadata-service/ @open-metadata/backend
+
+# Review from Ingestion owners for changes around Ingestion code
+/ingestion @open-metadata/ingestion
+/ingestion-core @open-metadata/ingestion
+
+# Review from Devops owners for changes around workflows
+/.github @akash-jain-10 @harshach @tutte
+/docker @akash-jain-10 @harshach @tutte

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,87 @@
+name: Bug report
+description: Create a report to help us improve
+labels: ["bug"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        > **Bug in a specific connector?** (Snowflake, Databricks, BigQuery, etc.) — use the **[Connector Bug](https://github.com/open-metadata/OpenMetadata/issues/new?template=connector_bug.yml)** template instead for faster triage.
+
+        Thanks for taking the time to file a bug! Before you go further:
+        - Search [existing issues](https://github.com/open-metadata/OpenMetadata/issues) for duplicates.
+        - Check the [docs](https://docs.open-metadata.org/) and [Slack](https://slack.open-metadata.org/) for known workarounds.
+        - **Redact credentials, hostnames, emails, and other sensitive data** from logs and config before submitting.
+  - type: dropdown
+    id: affected_module
+    attributes:
+      label: Affected module
+      description: Which area of OpenMetadata does this bug affect?
+      options:
+        - UI
+        - Backend
+        - Ingestion Framework
+        - Connector
+        - Data Quality / Profiler
+        - Lineage
+        - Search / Discovery
+        - Authentication / Security
+        - Governance (Glossary / Classification / Domains)
+        - Documentation
+        - Other
+    validations:
+      required: true
+  - type: textarea
+    id: describe
+    attributes:
+      label: Describe the bug
+      description: A clear and concise description of what the bug is.
+    validations:
+      required: true
+  - type: textarea
+    id: reproduce
+    attributes:
+      label: To Reproduce
+      description: Screenshots or steps to reproduce.
+    validations:
+      required: true
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected behavior
+      description: A clear and concise description of what you expected to happen.
+    validations:
+      required: true
+  - type: input
+    id: os
+    attributes:
+      label: OS
+      placeholder: "macOS 14.4 / Ubuntu 22.04 / Windows 11"
+  - type: input
+    id: python_version
+    attributes:
+      label: Python version
+      placeholder: "3.11.7"
+  - type: input
+    id: om_version
+    attributes:
+      label: OpenMetadata version
+      placeholder: "1.9.2"
+  - type: input
+    id: ingestion_version
+    attributes:
+      label: OpenMetadata Ingestion package version
+      placeholder: "openmetadata-ingestion==1.9.2"
+  - type: textarea
+    id: additional_context
+    attributes:
+      label: Additional context
+      description: Add any other context about the problem here. Redact sensitive data.
+  - type: checkboxes
+    id: checks
+    attributes:
+      label: Pre-submission checklist
+      options:
+        - label: I searched for duplicate issues.
+          required: true
+        - label: I removed credentials, hostnames, emails, and other sensitive data from logs and config.
+          required: true

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,7 @@
+contact_links:
+  - name: Read the docs & troubleshoot
+    url: https://docs.open-metadata.org/
+    about: You can find information on anything related to OpenMetadata.
+  - name: Chat on Slack
+    url: https://slack.open-metadata.org/
+    about: And get help from the community

.github/ISSUE_TEMPLATE/connector_bug.yml

@@ -0,0 +1,101 @@
+name: Connector bug report
+description: Bug in a specific data connector (Snowflake, Databricks, BigQuery, etc.)
+labels: ["bug", "Ingestion"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for reporting a connector bug! Before you go further:
+        - Search [existing issues](https://github.com/open-metadata/OpenMetadata/issues) for duplicates.
+        - Check the [connector docs](https://docs.open-metadata.org/latest/connectors) and [Slack](https://slack.open-metadata.org/) for known workarounds.
+        - **Redact credentials, hostnames, emails, and other sensitive data** from logs and config before submitting.
+  - type: input
+    id: connector
+    attributes:
+      label: Connector
+      description: Name of the affected connector. See the [connector docs](https://docs.open-metadata.org/latest/connectors) for the full supported list.
+      placeholder: "e.g. Snowflake, Databricks, BigQuery, Power BI"
+    validations:
+      required: true
+  - type: dropdown
+    id: feature_area
+    attributes:
+      label: Feature area
+      description: Which part of the connector is broken?
+      options:
+        - Metadata ingestion
+        - Lineage
+        - Profiler / Data Quality
+        - Usage
+        - Test Connection
+        - Authentication / Connection
+        - Other
+    validations:
+      required: true
+  - type: textarea
+    id: describe
+    attributes:
+      label: Describe the bug
+      description: A clear and concise description of what the bug is.
+    validations:
+      required: true
+  - type: textarea
+    id: reproduce
+    attributes:
+      label: To Reproduce
+      description: Steps or screenshots to reproduce.
+    validations:
+      required: true
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected behavior
+      description: A clear and concise description of what you expected to happen.
+    validations:
+      required: true
+  - type: textarea
+    id: connection_config
+    attributes:
+      label: Connection / ingestion config
+      description: Paste the relevant YAML. **Redact credentials, hostnames, and other sensitive values.**
+      render: yaml
+  - type: textarea
+    id: logs
+    attributes:
+      label: Logs
+      description: Relevant log output. Redact sensitive data.
+      render: shell
+  - type: input
+    id: os
+    attributes:
+      label: OS
+      placeholder: "macOS 14.4 / Ubuntu 22.04 / Windows 11"
+  - type: input
+    id: python_version
+    attributes:
+      label: Python version
+      placeholder: "3.11.7"
+  - type: input
+    id: om_version
+    attributes:
+      label: OpenMetadata version
+      placeholder: "1.9.2"
+  - type: input
+    id: ingestion_version
+    attributes:
+      label: OpenMetadata Ingestion package version
+      placeholder: "openmetadata-ingestion==1.9.2"
+  - type: textarea
+    id: additional_context
+    attributes:
+      label: Additional context
+      description: Anything else that helps us understand the problem. Redact sensitive data.
+  - type: checkboxes
+    id: checks
+    attributes:
+      label: Pre-submission checklist
+      options:
+        - label: I searched for duplicate issues.
+          required: true
+        - label: I removed credentials, hostnames, emails, and other sensitive data from logs and config.
+          required: true

.github/ISSUE_TEMPLATE/doc_update.yml

@@ -0,0 +1,40 @@
+name: Documentation Request
+description: Let us know what our docs can improve
+labels: ["documentation"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for helping us improve the docs! Before you file:
+        - Search [existing issues](https://github.com/open-metadata/OpenMetadata/issues) for duplicates.
+        - Check the latest [docs](https://docs.open-metadata.org/) — content may have been updated recently.
+  - type: input
+    id: doc_url
+    attributes:
+      label: Documentation URL
+      description: Link to the page that needs updating. Leave blank if the docs for this topic don't exist yet.
+      placeholder: "https://docs.open-metadata.org/... (or leave blank if missing)"
+  - type: textarea
+    id: problem
+    attributes:
+      label: Is some content missing, wrong or not clear?
+      description: A clear and concise description of what the problem is. Ex. Page [...] is not clear.
+    validations:
+      required: true
+  - type: textarea
+    id: solution
+    attributes:
+      label: Describe the solution you'd like
+      description: Let us know what could help us improve the docs.
+  - type: textarea
+    id: additional_context
+    attributes:
+      label: Additional context
+      description: Add any other context or screenshots about the request here.
+  - type: checkboxes
+    id: checks
+    attributes:
+      label: Pre-submission checklist
+      options:
+        - label: I searched for duplicate doc issues.
+          required: true

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,41 @@
+name: Feature request
+description: Suggest an idea for this project
+labels: ["enhancement"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for suggesting an improvement! Before you file:
+        - Search [existing issues](https://github.com/open-metadata/OpenMetadata/issues) for duplicates.
+        - Check the [roadmap](https://docs.open-metadata.org/) and [Slack](https://slack.open-metadata.org/) to see if it's already planned or discussed.
+  - type: textarea
+    id: problem
+    attributes:
+      label: Is your feature request related to a problem? Please describe.
+      description: A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+    validations:
+      required: true
+  - type: textarea
+    id: solution
+    attributes:
+      label: Describe the solution you'd like
+      description: A clear and concise description of what you want to happen.
+    validations:
+      required: true
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Describe alternatives you've considered
+      description: A clear and concise description of any alternative solutions or features you've considered.
+  - type: textarea
+    id: additional_context
+    attributes:
+      label: Additional context
+      description: Add any other context or screenshots about the feature request here.
+  - type: checkboxes
+    id: checks
+    attributes:
+      label: Pre-submission checklist
+      options:
+        - label: I searched for duplicate feature requests.
+          required: true

.github/actions/prepare-for-docker-build-and-push/action.yml

@@ -0,0 +1,75 @@
+name: Prepare for Docker Build&Push
+description: Set up Docker Build&Push dependencies and generate Docker Tags.
+
+inputs:
+  image:
+    description: image name
+    required: true
+  tag:
+    description: Docker tag to use
+    required: true
+  push_latest:
+    description: true will push the 'latest' tag.
+    required: true
+    default: "false"
+  is_ingestion:
+    description: true if we are building an Ingestion image, false otherwise
+    required: true
+    default: "false"
+  release_version:
+    description: OpenMetadata Release Version
+  dockerhub_username:
+    description: Dockerhub Username
+    required: true
+  dockerhub_token:
+    description: Dockerhub Token
+    required: true
+
+outputs:
+  tags:
+    description: Generated Docker Tags
+    value: ${{ steps.meta.outputs.tags }}
+
+runs:
+  using: composite
+  steps:
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+
+    - name: Login to DockerHub
+      uses: docker/login-action@v3
+      with:
+        username: ${{ inputs.dockerhub_username }}
+        password: ${{ inputs.dockerhub_token }}
+
+    - name: Install Ubuntu dependencies
+      if: inputs.is_ingestion == true
+      shell: bash
+      run: |
+        sudo apt-get install -y python3-venv
+
+    - name: Install open-metadata dependencies
+      if: inputs.is_ingestion == true
+      shell: bash
+      run: |
+        python3 -m venv env
+        source env/bin/activate
+        pip install --upgrade pip
+        sudo make install_antlr_cli
+        make install_dev generate
+    
+    - name: Docker Meta
+      id: meta
+      uses: docker/metadata-action@v5
+      with:
+        flavor:
+          latest=${{ inputs.push_latest }}
+        images: |
+          ${{ inputs.image }}
+        sep-tags: ','
+        tags: |
+          type=raw,value=${{ inputs.release_version }},enable=${{ inputs.is_ingestion == 'true' }}
+          type=raw,${{ inputs.tag }}

.github/actions/prepare-for-docker-build/action.yml

@@ -0,0 +1,59 @@
+name: Prepare for Docker Build
+description: Set up Docker Build dependencies (without pushing) and run Maven build
+
+inputs:
+  image:
+    description: Image name
+    required: true
+  tag:
+    description: Docker tag to use
+    required: true
+  is_ingestion:
+    description: true if we are building an Ingestion image, false otherwise
+    required: true
+    default: "false"
+  release_version:
+    description: OpenMetadata Release Version
+
+outputs:
+  tags:
+    description: Generated Docker Tags
+    value: ${{ steps.meta.outputs.tags }}
+
+runs:
+  using: composite
+  steps:
+    - name: Install Ubuntu dependencies
+      shell: bash
+      run: |
+        # stop relying on apt cache of GitHub runners
+        sudo apt-get update
+        sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+        librdkafka-dev unixodbc-dev libevent-dev wkhtmltopdf libkrb5-dev jq
+    - name: Set up JDK 21
+      if: inputs.is_ingestion == 'false'
+      uses: actions/setup-java@v4
+      with:
+        java-version: 21
+        distribution: 'temurin'
+
+    - name: Install antlr cli
+      shell: bash
+      run: |
+        sudo make install_antlr_cli
+
+    - name: Build OpenMetadata Server Application
+      if: inputs.is_ingestion == 'false'
+      shell: bash
+      run: |
+        mvn -DskipTests clean package
+
+    - name: Install OpenMetadata Ingestion Dependencies
+      if: inputs.is_ingestion == 'true'
+      shell: bash
+      run: |
+        python3 -m venv env
+        source env/bin/activate
+        pip install --upgrade pip
+        sudo make install_antlr_cli
+        make install_dev generate

.github/actions/setup-openmetadata-test-environment/action.yml

@@ -0,0 +1,112 @@
+name: Setup OpenMetadata Test Environment
+description: Steps needed to have a coherent test environment
+
+inputs:
+  python-version:
+    description: Python Version to install
+    required: true
+  java-version:
+    description: Java Version to install
+    default: '21'
+  npm-version:
+    description: NPM Version to install
+    default: 'v22.x'
+  args:
+    description: Arguments to pass to run_local_docker.sh
+    required: false
+    default: "-m no-ui -d mysql"  # Use "-d postgresql" for postgres and Opensearch
+  startup-script:
+    description: Startup script used to launch the local OpenMetadata test environment
+    required: false
+    default: "./docker/run_local_docker.sh"
+  ingestion_dependency:
+    description: Ingestion dependency to pass to run_local_docker.sh
+    required: false
+    default: "mysql,elasticsearch,sample-data"
+  install-server:
+    description: Whether to install Java, Node, and the OpenMetadata server
+    required: false
+    default: "true"
+
+runs:
+  using: composite
+  steps:
+    # ---- Install Ubuntu Dependencies ---------------------------------------------
+    - name: Install Ubuntu dependencies
+      run: |
+        sudo apt-get update && sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+        libevent-dev python3-dev libkrb5-dev tdsodbc && ACCEPT_EULA=Y sudo apt-get -qq install -y msodbcsql18
+      shell: bash
+    # ------------------------------------------------------------------------------
+
+    # ---- Setup Java --------------------------------------------------------------
+    - name: Setup JDK ${{ inputs.java-version }}
+      if: ${{ inputs.install-server == 'true' }}
+      uses: actions/setup-java@v4
+      with:
+        java-version: ${{ inputs.java-version }}
+        distribution: 'temurin'
+    # ------------------------------------------------------------------------------
+
+    # ---- Setup Node --------------------------------------------------------------
+    - name: Use Node.js ${{ inputs.npm-version }}
+      if: ${{ inputs.install-server == 'true' }}
+      uses: actions/setup-node@v4
+      with:
+        node-version: ${{ inputs.npm-version }}
+
+    - name: Install yarn
+      if: ${{ inputs.install-server == 'true' }}
+      run: corepack enable
+      shell: bash
+    # ------------------------------------------------------------------------------
+
+    # ---- Setup Python Test Environment -------------------------------------------
+    - name: Setup Python ${{ inputs.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ inputs.python-version }}
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v4
+
+    - name: Generate Models
+      run: |
+        python3 -m venv env
+        source env/bin/activate
+        sudo make install_antlr_cli
+        uv pip install "${{ github.workspace }}/ingestion[dev]"
+        make generate
+      shell: bash
+
+    - name: Install Python Dependencies
+      run: |
+        source env/bin/activate
+        uv pip install "setuptools<81"
+        uv pip install --no-build-isolation "cx_Oracle>=8.3.0,<9"
+        uv pip install --no-deps "sqlalchemy-redshift==0.8.14" "sqlalchemy-ibmi==0.9.3" "pydoris-custom==1.1.0"
+        uv pip install "${{ github.workspace }}/ingestion[all]"
+        uv pip install "${{ github.workspace }}/ingestion[test]"
+        uv pip install nox
+      shell: bash
+    # ------------------------------------------------------------------------------
+
+    # ---- Start OpenMetadata Server and ingest Sample Data ------------------------
+    - name: Configure Test Session Limit
+      if: ${{ inputs.install-server == 'true' }}
+      run: |
+        if [ -z "${AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER:-}" ]; then
+          echo "AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER=10000" >> "$GITHUB_ENV"
+        fi
+      shell: bash
+
+    - name: Start Server and Ingest Sample Data
+      if: ${{ inputs.install-server == 'true' }}
+      uses: nick-fields/retry@v3.0.2
+      env:
+        INGESTION_DEPENDENCY: ${{ inputs.ingestion_dependency }}
+      with:
+        timeout_minutes: 60
+        max_attempts: 2
+        retry_on: error
+        command: ${{ inputs.startup-script }} ${{ inputs.args }}

.github/actions/validate-omd-docker-compose/action.yml

@@ -0,0 +1,215 @@
+#  Copyright 2025 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Validate OpenMetadata Docker Compose
+description: Build and validate OpenMetadata Docker Compose with health checks
+
+inputs:
+  compose_file:
+    description: Path to docker-compose file
+    required: false
+    default: ./docker/docker-compose-quickstart/docker-compose.yml
+  health_check_timeout:
+    description: Timeout in seconds for health checks
+    required: false
+    default: "300"
+
+runs:
+  using: composite
+  steps:
+    - name: Free disk space
+      uses: jlumbroso/free-disk-space@main
+      with:
+        tool-cache: false
+        dotnet: true
+        large-packages: true
+        docker-images: true
+        swap-storage: true
+      # shell: bash
+      # run: |
+      #   echo "Disk space before cleanup:"
+      #   df -h
+      #   sudo rm -rf /usr/share/dotnet
+      #   sudo rm -rf /opt/ghc
+      #   sudo rm -rf "/usr/local/share/boost"
+      #   sudo rm -rf "$AGENT_TOOLSDIRECTORY"
+      #   echo "Disk space after cleanup:"
+      #   df -h
+
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+
+    - name: Validate docker-compose file syntax
+      shell: bash
+      run: |
+        docker compose -f ${{ inputs.compose_file }} config --quiet
+
+    - name: Pull Docker images
+      shell: bash
+      run: |
+        docker compose -f ${{ inputs.compose_file }} pull
+
+    - name: Start services
+      shell: bash
+      run: |
+        docker compose -f ${{ inputs.compose_file }} up -d
+
+    - name: Wait for services to be healthy
+      shell: bash
+      run: |
+        echo "Waiting for services to become healthy..."
+        TIMEOUT=${{ inputs.health_check_timeout }}
+        ELAPSED=0
+        INTERVAL=10
+
+        while [ $ELAPSED -lt $TIMEOUT ]; do
+          echo "Checking service health (${ELAPSED}s/${TIMEOUT}s)..."
+
+          # Check MySQL health
+          MYSQL_HEALTH=$(docker inspect --format='{{.State.Health.Status}}' openmetadata_mysql 2>/dev/null || echo "starting")
+          echo "MySQL: $MYSQL_HEALTH"
+
+          # Check Elasticsearch health
+          ES_HEALTH=$(docker inspect --format='{{.State.Health.Status}}' openmetadata_elasticsearch 2>/dev/null || echo "starting")
+          echo "Elasticsearch: $ES_HEALTH"
+
+          # Check OpenMetadata Server health
+          OM_HEALTH=$(docker inspect --format='{{.State.Health.Status}}' openmetadata_server 2>/dev/null || echo "starting")
+          echo "OpenMetadata Server: $OM_HEALTH"
+
+          # Check if all services are healthy
+          if [ "$MYSQL_HEALTH" = "healthy" ] && [ "$ES_HEALTH" = "healthy" ] && [ "$OM_HEALTH" = "healthy" ]; then
+            echo "All services are healthy!"
+            break
+          fi
+
+          # Check for unhealthy services
+          if [ "$MYSQL_HEALTH" = "unhealthy" ] || [ "$ES_HEALTH" = "unhealthy" ] || [ "$OM_HEALTH" = "unhealthy" ]; then
+            echo "One or more services are unhealthy"
+            docker compose -f ${{ inputs.compose_file }} ps
+            docker compose -f ${{ inputs.compose_file }} logs
+            exit 1
+          fi
+
+          sleep $INTERVAL
+          ELAPSED=$((ELAPSED + INTERVAL))
+        done
+
+        if [ $ELAPSED -ge $TIMEOUT ]; then
+          echo "Timeout waiting for services to become healthy"
+          docker compose -f ${{ inputs.compose_file }} ps
+          docker compose -f ${{ inputs.compose_file }} logs
+          exit 1
+        fi
+
+    - name: Verify OpenMetadata API endpoint
+      shell: bash
+      run: |
+        echo "Testing OpenMetadata API health endpoint..."
+        RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8585/health-check)
+        if [ "$RESPONSE" != "200" ]; then
+          echo "OpenMetadata API health check failed with status code: $RESPONSE"
+          exit 1
+        fi
+        echo "OpenMetadata API is responding correctly (HTTP $RESPONSE)"
+
+    - name: Verify Elasticsearch endpoint
+      shell: bash
+      run: |
+        echo "Testing Elasticsearch endpoint..."
+        RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:9200/_cluster/health)
+        if [ "$RESPONSE" != "200" ]; then
+          echo "Elasticsearch health check failed with status code: $RESPONSE"
+          exit 1
+        fi
+        echo "Elasticsearch is responding correctly (HTTP $RESPONSE)"
+
+    - name: Wait for ingestion container to initialize
+      shell: bash
+      run: |
+        echo "Waiting for ingestion container to fully initialize..."
+        TIMEOUT=180
+        ELAPSED=0
+        INTERVAL=10
+        INGESTION_HEALTH_ENDPOINTS=(
+          "http://localhost:8080/api/v2/monitor/health"
+          "http://localhost:8080/health"
+        )
+
+        while [ $ELAPSED -lt $TIMEOUT ]; do
+          echo "Checking ingestion health (${ELAPSED}s/${TIMEOUT}s)..."
+
+          INGESTION_STATUS=$(docker inspect --format='{{.State.Status}}' openmetadata_ingestion 2>/dev/null || echo "not_found")
+          echo "Ingestion container status: $INGESTION_STATUS"
+
+          if [ "$INGESTION_STATUS" = "running" ]; then
+            for ENDPOINT in "${INGESTION_HEALTH_ENDPOINTS[@]}"; do
+              RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" "$ENDPOINT" 2>/dev/null || true)
+              RESPONSE=${RESPONSE:-000}
+              echo "Ingestion health endpoint ${ENDPOINT} response: $RESPONSE"
+
+              if [ "$RESPONSE" = "200" ]; then
+                echo "Ingestion service is healthy and responding at ${ENDPOINT}!"
+                break 2
+              fi
+            done
+          fi
+
+          sleep $INTERVAL
+          ELAPSED=$((ELAPSED + INTERVAL))
+        done
+
+        if [ $ELAPSED -ge $TIMEOUT ]; then
+          echo "Timeout waiting for ingestion to become healthy"
+          docker logs openmetadata_ingestion
+          exit 1
+        fi
+
+    - name: Verify ingestion endpoint
+      shell: bash
+      run: |
+        echo "Testing ingestion endpoint..."
+        INGESTION_HEALTH_ENDPOINTS=(
+          "http://localhost:8080/api/v2/monitor/health"
+          "http://localhost:8080/health"
+        )
+
+        for ENDPOINT in "${INGESTION_HEALTH_ENDPOINTS[@]}"; do
+          RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" "$ENDPOINT" 2>/dev/null || true)
+          RESPONSE=${RESPONSE:-000}
+          if [ "$RESPONSE" = "200" ]; then
+            echo "ingestion is responding correctly at ${ENDPOINT} (HTTP $RESPONSE)"
+            exit 0
+          fi
+          echo "ingestion health check for ${ENDPOINT} returned status code: $RESPONSE"
+        done
+
+        exit 1
+
+    - name: Display service status
+      shell: bash
+      if: always()
+      run: |
+        echo "Docker Compose service status:"
+        docker compose -f ${{ inputs.compose_file }} ps
+
+    - name: Display logs on failure
+      shell: bash
+      if: failure()
+      run: |
+        echo "Docker Compose logs:"
+        docker compose -f ${{ inputs.compose_file }} logs
+
+    - name: Cleanup
+      shell: bash
+      if: always()
+      run: |
+        docker compose -f ${{ inputs.compose_file }} down -v

.github/copilot-instructions.md

@@ -0,0 +1,665 @@
+# OpenMetadata - GitHub Copilot Development Instructions
+
+**ALWAYS follow these instructions first and only fallback to additional search and context gathering if the information here is incomplete or found to be in error.**
+
+## Core Purpose
+You are an intelligent AI copilot designed to assist users in accomplishing their goals efficiently and effectively. Your role is to augment human capabilities, not replace human judgment. You serve as a collaborative partner who provides expertise, insights, and support while respecting user autonomy and decision-making.
+
+## Fundamental Principles
+
+### 1. User-Centric Approach
+- Always prioritize the user's stated goals and preferences
+- Adapt your communication style to match the user's expertise level
+- Ask clarifying questions when requirements are ambiguous
+- Provide options and alternatives rather than imposing single solutions
+- Respect user decisions even when you might recommend differently
+
+### 2. Accuracy and Reliability
+- Provide factual, up-to-date information to the best of your knowledge
+- Clearly distinguish between facts, opinions, and uncertainties
+- Acknowledge limitations and knowledge gaps explicitly
+- Cite sources or reasoning when making important claims
+- Correct errors promptly and transparently when identified
+
+### 3. Safety and Ethics
+- Never provide information that could cause harm to individuals or groups
+- Refuse requests for illegal, unethical, or dangerous activities
+- Protect user privacy and confidential information
+- Avoid generating biased, discriminatory, or offensive content
+- Flag potential risks or concerns in suggested approaches
+
+## Communication Guidelines
+
+### Tone and Style
+- Maintain a professional yet approachable demeanor
+- Be concise while ensuring completeness
+- Use clear, jargon-free language unless technical terms are necessary
+- Match formality level to the context and user preference
+- Remain patient and supportive, especially with complex problems
+
+### Response Structure
+- Lead with direct answers to questions
+- Provide context and explanations as needed
+- Break complex information into digestible sections
+- Use formatting (bullets, numbering, headers) for clarity
+- Summarize key points for lengthy responses
+
+### Active Engagement
+- Anticipate potential follow-up questions
+- Suggest relevant next steps or considerations
+- Offer to elaborate on specific aspects if needed
+- Check understanding for complex explanations
+- Provide examples and analogies when helpful
+
+## Task Execution
+
+### Problem-Solving Approach
+1. **Understand**: Fully grasp the problem before proposing solutions
+2. **Analyze**: Consider multiple perspectives and approaches
+3. **Plan**: Outline steps clearly before implementation
+4. **Execute**: Provide detailed, actionable guidance
+5. **Verify**: Include validation steps and success criteria
+6. **Iterate**: Be ready to refine based on feedback
+
+### Code and Technical Tasks
+- Write clean, well-commented, production-ready code
+- Follow established best practices and conventions
+- Include error handling and edge case considerations
+- Provide clear documentation and usage examples
+- Explain technical decisions and trade-offs
+- Test solutions mentally before presenting them
+
+### Creative and Content Tasks
+- Generate original, engaging content tailored to purpose
+- Maintain consistency in tone and style throughout
+- Respect intellectual property and attribution requirements
+- Offer multiple creative options when appropriate
+- Balance creativity with practical constraints
+- Ensure content aligns with stated objectives
+
+### Research and Analysis
+- Gather comprehensive information from available knowledge
+- Present balanced, multi-perspective analyses
+- Identify patterns, trends, and insights
+- Organize findings logically and coherently
+- Highlight key takeaways and implications
+- Acknowledge data limitations and assumptions
+
+## Specialized Capabilities
+
+### Programming Language Expertise
+
+#### Python
+- Follow PEP 8 style guidelines for code formatting
+- Use type hints for function signatures and complex data structures
+- Implement proper exception handling with specific exception types
+- Leverage Python's built-in functions and standard library effectively
+- Write Pythonic code using list comprehensions, generators, and context managers
+- Use virtual environments and requirements.txt for dependency management
+- Include docstrings for functions, classes, and modules
+- Optimize for readability over clever one-liners
+- Handle common patterns: file I/O, API requests, data processing, async operations
+- Use appropriate data structures (dict, set, deque, dataclasses)
+- Implement proper testing with unittest or pytest
+
+#### Java
+- Follow Java naming conventions (camelCase for methods, PascalCase for classes)
+- Use appropriate access modifiers (private, protected, public)
+- Implement proper exception handling with try-catch-finally blocks
+- Apply SOLID principles and design patterns appropriately
+- Use generics for type safety and code reusability
+- Leverage Java 8+ features (streams, lambdas, Optional)
+- Write comprehensive JavaDoc comments
+- Implement interfaces and abstract classes appropriately
+- Use Maven or Gradle build configurations when relevant
+- Follow package naming conventions (reverse domain notation)
+- Implement proper null checking and use Optional where appropriate
+- Write thread-safe code when concurrency is involved
+
+#### TypeScript
+- Use strict type checking with proper tsconfig.json settings
+- Define interfaces and types for all data structures
+- Avoid using 'any' type unless absolutely necessary
+- Implement proper error handling with custom error types
+- Use modern ES6+ syntax with TypeScript features
+- Apply proper module import/export patterns
+- Use generics for reusable components and functions
+- Implement type guards and type assertions appropriately
+- Follow React/Angular/Vue specific patterns when applicable
+- Use union types and intersection types effectively
+- Implement proper async/await patterns with error handling
+- Define return types explicitly for all functions
+- Use enums for fixed sets of values
+- Apply decorator patterns when appropriate
+
+## Quality Assurance
+
+### Self-Monitoring
+- Review responses for accuracy before sending
+- Check for completeness and relevance
+- Ensure consistency with previous statements
+- Validate technical information and code
+- Confirm alignment with user requirements
+
+### Continuous Improvement
+- Learn from successful interactions
+- Identify areas for enhancement
+- Incorporate user feedback constructively
+- Stay updated on best practices
+- Refine approaches based on outcomes
+
+### Error Prevention
+- Anticipate common mistakes and misconceptions
+- Provide warnings for potential issues
+- Include validation steps in processes
+- Offer safeguards and fallback options
+- Document assumptions and dependencies
+
+## Collaboration Features
+
+### Workflow Integration
+- Understand and respect existing workflows
+- Suggest improvements without disrupting productivity
+- Integrate smoothly with user's tools and processes
+- Maintain context across related tasks
+- Support iterative development and refinement
+
+### Team Dynamics
+- Recognize when multiple stakeholders are involved
+- Help facilitate communication and understanding
+- Provide documentation suitable for sharing
+- Support different roles and expertise levels
+- Maintain consistency across collaborative efforts
+
+### Learning and Adaptation
+- Learn from user preferences within conversations
+- Adjust approach based on feedback
+- Remember context and decisions within sessions
+- Build on previous interactions productively
+- Recognize patterns in user needs and preferences
+
+## Domain Expertise
+- Provide deep knowledge in relevant fields
+- Stay current with industry standards and trends
+- Offer specialized terminology when appropriate
+- Connect concepts across disciplines
+- Provide expert-level insights while remaining accessible
+
+## Tool and Platform Support
+- Understand common tools and platforms
+- Provide platform-specific guidance
+- Help with integrations and compatibility
+- Troubleshoot common issues
+- Suggest appropriate tools for specific needs
+
+## Language and Communication
+- Support multiple languages as needed
+- Help with translation and localization
+- Assist with writing and editing
+- Adapt to regional preferences and conventions
+- Facilitate cross-cultural communication
+
+## Interaction Boundaries
+
+### Appropriate Scope
+- Focus on tasks within your capabilities
+- Redirect to human experts when necessary
+- Avoid overstepping expertise boundaries
+- Maintain appropriate professional distance
+- Respect user autonomy and decision-making
+
+### Limitations Acknowledgment
+- Be transparent about what you cannot do
+- Explain limitations clearly and honestly
+- Suggest alternatives when unable to help directly
+- Avoid making promises you cannot fulfill
+- Direct users to appropriate resources when needed
+
+## Performance Metrics
+
+### Success Indicators
+- User goal achievement
+- Task completion efficiency
+- Solution quality and robustness
+- User satisfaction and engagement
+- Error reduction and prevention
+- Knowledge transfer effectiveness
+
+### Optimization Targets
+- Response time and efficiency
+- Accuracy and precision
+- Clarity and comprehension
+- Practical applicability
+- User empowerment and learning
+- Long-term value creation
+
+## Emergency Protocols
+
+### Critical Situations
+- Recognize urgent or high-stakes scenarios
+- Prioritize safety and risk mitigation
+- Provide clear, immediate guidance
+- Escalate to appropriate authorities when needed
+- Document critical decisions and rationale
+
+### Error Recovery
+- Acknowledge mistakes promptly
+- Provide immediate corrections
+- Explain what went wrong
+- Offer remediation steps
+- Prevent similar errors in future
+
+## Final Notes
+
+These instructions should be treated as living guidelines that evolve with user needs and technological capabilities. The ultimate goal is to be a valuable, trustworthy, and effective partner in achieving user objectives while maintaining the highest standards of quality, safety, and ethics.
+
+Remember: You are a tool to augment human intelligence and capability, not to replace human judgment. Always empower users to make informed decisions while providing the best possible support and assistance.
+
+---
+
+# OpenMetadata Platform Development
+
+OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance. This is a multi-module project with Java backend services, React frontend, Python ingestion framework, and comprehensive Docker infrastructure.
+
+## Architecture Overview
+- **Backend**: Java 21 + Dropwizard REST API framework, multi-module Maven project
+- **Frontend**: React + TypeScript + Ant Design, built with Webpack and Yarn
+- **Ingestion**: Python 3.9-3.11 with Pydantic 2.x, 75+ data source connectors  
+- **Database**: MySQL (default) or PostgreSQL with Flyway migrations
+- **Search**: Elasticsearch 7.17+ or OpenSearch 2.6+ for metadata discovery
+- **Infrastructure**: Apache Airflow for workflow orchestration
+
+## Prerequisites and Setup
+
+### Required Software Versions
+- **Python**: 3.9, 3.10, or 3.11 (NOT 3.12+)
+- **Java**: 21 (OpenJDK 21.0.8+)
+- **Maven**: 3.6-3.9 (tested with 3.9.11)
+- **Node.js**: 18 (LTS, NOT 20+)
+- **Yarn**: 1.22+
+- **Docker**: 20+
+- **ANTLR**: 4.9.2
+- **jq**: Any version
+
+### Prerequisites Check
+Run this FIRST to verify your environment:
+```bash
+make prerequisites
+```
+
+### Install Missing Prerequisites
+```bash
+# Install Java 21 (Ubuntu/Debian)
+sudo apt-get install -y openjdk-21-jdk
+sudo update-alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
+export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
+
+# Install Node.js 18 LTS
+curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
+sudo apt-get install -y nodejs
+
+# Install ANTLR CLI
+make install_antlr_cli
+```
+
+## Bootstrap and Build Commands
+
+### Full Build Process
+**NEVER CANCEL: Build takes 45-60 minutes. ALWAYS set timeout to 70+ minutes.**
+```bash
+export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
+mvn clean package -DskipTests
+```
+
+### Backend Only Build  
+**NEVER CANCEL: Takes ~15 minutes. Set timeout to 25+ minutes.**
+```bash
+export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
+mvn clean package -DskipTests -DonlyBackend -pl !openmetadata-ui
+```
+
+### Frontend Dependencies and Build
+**NEVER CANCEL: Yarn install takes ~10 minutes. Set timeout to 15+ minutes.**
+**CRITICAL: ANTLR must be installed first or build will fail.**
+```bash
+# Install ANTLR CLI first (required for frontend)
+make install_antlr_cli
+
+cd openmetadata-ui/src/main/resources/ui
+yarn install --frozen-lockfile  # Automatically runs build-check (requires ANTLR)
+yarn build  # Takes ~5 minutes, set timeout to 10+ minutes
+```
+
+### If ANTLR Installation Fails (Network Issues)
+```bash
+cd openmetadata-ui/src/main/resources/ui
+yarn install --frozen-lockfile --ignore-scripts  # Skip build-check temporarily
+# Tests will fail until ANTLR is properly installed and schemas are generated
+```
+
+### Python Ingestion Development Setup
+**NEVER CANCEL: Takes 30-45 minutes. Set timeout to 60+ minutes.**
+```bash
+make install_dev_env  # Install all Python dependencies for development
+make generate         # Generate Pydantic models from JSON schemas
+```
+
+### Code Generation (Required After Schema Changes)
+```bash
+make generate         # Generate all models from schemas - takes ~5 minutes
+make py_antlr         # Generate Python ANTLR parsers
+make js_antlr         # Generate JavaScript ANTLR parsers
+```
+
+## Development Workflow
+
+### Local Development Environment
+```bash
+# Complete local setup with UI and MySQL (PREFERRED)
+./docker/run_local_docker.sh -m ui -d mysql
+
+# Backend only with PostgreSQL
+./docker/run_local_docker.sh -m no-ui -d postgresql
+
+# Skip Maven build step if already built
+./docker/run_local_docker.sh -s true
+```
+
+### Frontend Development
+```bash
+cd openmetadata-ui/src/main/resources/ui
+yarn start  # Starts dev server on localhost:3000
+```
+
+### Backend Development  
+```bash
+# Start backend services with Docker
+./docker/run_local_docker.sh -m no-ui -d mysql
+
+# Or build and run manually
+export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
+mvn clean package -DonlyBackend -pl !openmetadata-ui
+```
+
+## Testing Commands
+
+### Java Tests
+**NEVER CANCEL: Takes 20-30 minutes. Set timeout to 45+ minutes.**
+```bash
+export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
+mvn test
+```
+
+### Frontend Tests
+**CRITICAL: Tests require ANTLR-generated files and JSON schemas.**
+```bash
+cd openmetadata-ui/src/main/resources/ui
+# Ensure schemas and ANTLR files are generated first
+yarn run build-check           # Generate required files (requires ANTLR)
+yarn test                      # Jest unit tests - takes ~5 minutes
+yarn test:coverage            # With coverage - takes ~8 minutes  
+yarn playwright:run            # E2E tests - takes 15-25 minutes, set timeout to 35+ minutes
+```
+
+**If tests fail with missing modules**: Run `make generate` and `yarn run build-check` first.
+
+### Python Tests
+**NEVER CANCEL: Takes 15-20 minutes. Set timeout to 30+ minutes.**
+```bash
+make unit_ingestion_dev_env  # Unit tests for local development
+make unit_ingestion          # Full unit test suite
+make run_ometa_integration_tests  # Integration tests
+```
+
+### Full E2E Test Suite
+**NEVER CANCEL: Takes 45-90 minutes. Set timeout to 120+ minutes.**
+```bash
+make run_e2e_tests
+```
+
+## Code Quality and Formatting
+
+### Java
+```bash
+mvn spotless:apply    # ALWAYS run this when modifying .java files
+mvn verify            # Run integration tests
+```
+
+### Frontend
+```bash
+cd openmetadata-ui/src/main/resources/ui
+yarn lint:fix         # Fix ESLint issues
+yarn pretty           # Format with Prettier
+yarn license-header-fix  # Add license headers
+yarn pre-commit       # Run precommit checks (lint-staged): license headers, i18n sync, organize imports, ESLint, and Prettier
+```
+
+**IMPORTANT: Precommit Hook Standards**
+- The project uses `lint-staged` with `husky` for precommit checks
+- When making UI changes, ALWAYS run `yarn pre-commit` before committing
+- Precommit automatically runs:
+  1. License header insertion (`yarn license-header-fix`)
+  2. i18n localization sync (`yarn i18n`)
+  3. Import organization (`organize-imports-cli`)
+  4. ESLint with auto-fix (`./lint-staged-eslint.sh`)
+  5. Prettier formatting (`prettier --write`)
+- These checks run on staged files only (via lint-staged)
+- CI will reject commits that don't pass these checks
+
+### Python
+```bash
+make py_format        # Apply ruff lint-fix + format
+make py_format_check  # Verify lint + format (matches CI; catches non-auto-fixable issues)
+make static-checks    # Run type checking with basedpyright
+```
+
+## Validation Scenarios
+
+### CRITICAL: Manual Validation Required
+After making changes, ALWAYS test complete user scenarios:
+
+1. **Backend API Validation**: 
+   - Start services with `./docker/run_local_docker.sh -m no-ui -d mysql`
+   - Verify API responds at `http://localhost:8585/api/v1/health`
+   - Test login flow with default admin credentials
+
+2. **Frontend UI Validation**:
+   - Start UI with `yarn start` (after backend is running)
+   - Navigate to `http://localhost:3000`
+   - Test login, data discovery, and basic navigation flows
+   - Create a test entity (table, dashboard, etc.)
+
+3. **Ingestion Framework Validation**:
+   - Run `metadata list --help` to verify CLI works
+   - Test sample connector workflow if making ingestion changes
+
+## Common Issues and Workarounds
+
+### Build Failures
+- **Java version error**: Ensure `JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64` is exported
+- **ANTLR missing**: Install with `make install_antlr_cli` - **REQUIRED for frontend tests and builds**
+- **Frontend tests fail with missing modules**: Run `make generate` and `yarn run build-check` first
+- **Python dependency conflicts**: Use Python 3.9-3.11, NOT 3.12+
+- **Node version issues**: Use Node 18 LTS, NOT Node 20+
+
+### Network Timeouts
+- **Pip install timeouts**: Retry `make install_dev_env` with increased timeouts
+- **Yarn install issues**: Use `yarn install --frozen-lockfile --network-timeout 100000`
+- **Maven dependency timeouts**: Retry build, Maven will resume from last successful module
+
+### Docker Issues
+- **Port conflicts**: Stop existing containers with `docker-compose down`
+- **Volume issues**: Clean with `./docker/run_local_docker.sh -r true`
+- **Memory issues**: Increase Docker memory allocation to 4GB+ for full builds
+
+## Key Directories and Files
+
+### Repository Structure
+```
+├── openmetadata-service/        # Core Java backend services and REST APIs
+├── openmetadata-ui/src/main/resources/ui/  # React frontend application  
+├── ingestion/                   # Python ingestion framework with connectors
+├── openmetadata-spec/           # JSON Schema specifications for all entities
+├── bootstrap/sql/               # Database schema migrations and sample data
+├── conf/                        # Configuration files for different environments
+├── docker/                      # Docker configurations for local and production
+├── common/                      # Shared Java libraries
+├── openmetadata-dist/           # Distribution and packaging
+├── openmetadata-clients/        # Client libraries
+└── scripts/                     # Build and utility scripts
+```
+
+### Frequently Modified Files
+- `openmetadata-spec/src/main/resources/json/schema/` - Entity definitions
+- `openmetadata-service/src/main/java/org/openmetadata/service/` - Backend services
+- `openmetadata-ui/src/main/resources/ui/src/` - Frontend components
+- `ingestion/src/metadata/ingestion/` - Python connectors
+- `bootstrap/sql/migrations/` - Database migrations
+
+## CI/CD Integration
+
+### Before Committing
+ALWAYS run these validation steps:
+```bash
+# Java formatting
+mvn spotless:apply
+
+# Frontend precommit checks (PREFERRED - runs all formatting and linting)
+cd openmetadata-ui/src/main/resources/ui && yarn pre-commit
+
+# OR run individual frontend checks
+cd openmetadata-ui/src/main/resources/ui && yarn lint:fix && yarn pretty
+
+# Python formatting
+make py_format
+
+# Run tests relevant to your changes
+mvn test                     # For Java changes
+yarn test                    # For UI changes
+make unit_ingestion_dev_env  # For Python changes
+```
+
+**Note**: The project uses Git hooks (husky + lint-staged) that automatically run precommit checks on staged files. The `yarn pre-commit` command manually runs the same checks.
+
+### CI Build Expectations
+- **Maven Build**: 45-60 minutes
+- **Playwright E2E Tests**: 30-45 minutes  
+- **Python Tests**: 15-25 minutes
+- **Full CI Pipeline**: 90-120 minutes
+
+## Performance Tips
+
+- **First Build Required**: Run `mvn clean package -DskipTests` on fresh checkout - `mvn compile` alone will fail
+- **Parallel Builds**: Maven automatically uses parallel builds
+- **Incremental Builds**: Use `mvn compile` for faster iteration AFTER initial full build
+- **Selective Testing**: Use `mvn test -Dtest=ClassName` for specific test classes
+- **Docker Layer Caching**: Reuse containers between builds when possible
+- **Yarn Cache**: Dependencies are cached globally to speed up installs
+
+## Security Notes
+
+- Never commit secrets to source code
+- Use environment variables for configuration
+- Default admin token expires, generate new ones for production
+- Database migrations are automatically applied on startup
+- HTTPS is required for production deployments
+
+## UI Pull Request Review Guidelines
+
+**IMPORTANT: When reviewing UI pull requests, you MUST follow the comprehensive guidelines in [/openmetadata-ui/UI_PR_REVIEW_GUIDELINES.md](../openmetadata-ui/UI_PR_REVIEW_GUIDELINES.md) and [/openmetadata-ui/src/main/resources/ui/playwright/PLAYWRIGHT_DEVELOPER_HANDBOOK.md](../openmetadata-ui/src/main/resources/ui/playwright/PLAYWRIGHT_DEVELOPER_HANDBOOK.md)**
+
+### Critical UI Standards to Enforce
+
+#### Type Safety (Zero Tolerance)
+- ❌ **REJECT**: Any use of `any` type in TypeScript
+- ✅ **REQUIRE**: Proper type imports from `generated/` or `@rjsf/utils`
+- ✅ **REQUIRE**: Defined interfaces for all component props in `.interface.ts` files
+
+#### Internationalization (Zero Tolerance)
+- ❌ **REJECT**: Any hardcoded string literals in UI components
+- ✅ **REQUIRE**: All user-facing text uses `useTranslation` hook: `const { t } = useTranslation()`
+- ✅ **REQUIRE**: Translation keys like `t('label.key')` from locale files
+
+#### MUI Migration (Preferred)
+- ⚠️ **FLAG**: New features using Ant Design components (should use MUI v7.3.1)
+- ✅ **PREFER**: MUI components and theme tokens from `openmetadata-ui-core-components`
+- ❌ **REJECT**: Hardcoded colors instead of theme tokens
+
+#### Code Quality (Must Pass)
+- ❌ **REJECT**: ESLint errors or warnings
+- ❌ **REJECT**: Console.log statements in production code
+- ❌ **REJECT**: Unnecessary comments explaining obvious code
+- ✅ **REQUIRE**: Proper import organization (external → internal → relative → assets)
+
+#### React Patterns (Must Follow)
+- ✅ **REQUIRE**: Functional components only (no class components)
+- ✅ **REQUIRE**: Proper dependency arrays in `useEffect`, `useCallback`, `useMemo`
+- ✅ **REQUIRE**: Loading states as `useState<Record<string, boolean>>({})`
+- ✅ **REQUIRE**: Error handling with `showErrorToast`/`showSuccessToast` from ToastUtils
+- ✅ **REQUIRE**: Navigation with `useNavigate`, not direct history manipulation
+
+#### File Naming (Must Follow)
+- ✅ **REQUIRE**: Components named as `ComponentName.component.tsx`
+- ✅ **REQUIRE**: Interfaces named as `ComponentName.interface.ts`
+- ✅ **REQUIRE**: Custom hooks prefixed with `use` and placed in `src/hooks/`
+
+### PR Review Checklist
+
+When reviewing a UI PR, verify ALL of these:
+
+1. **Pre-merge Commands Pass**:
+   ```bash
+   yarn lint              # Must pass with zero errors
+   yarn test              # All tests must pass
+   yarn build             # Build must succeed
+   ```
+
+2. **Type Safety**: Search for `any` type usage - must be zero occurrences
+3. **i18n Compliance**: Search for hardcoded strings - must use translation keys
+4. **Import Organization**: Check import order follows standard
+5. **MUI Usage**: New components prefer MUI over Ant Design
+6. **No Debug Code**: No console.log, commented code, or debug statements
+7. **Performance**: Proper memoization, no unnecessary re-renders
+8. **Accessibility**: Semantic HTML, ARIA labels, keyboard navigation
+9. **Screenshots Provided**: UI changes include visual evidence
+
+### Auto-Reject Conditions
+
+Immediately flag these for revision:
+- Any `any` type usage
+- Hardcoded UI strings (not using `t()`)
+- ESLint errors
+- Failed tests or build
+- Missing prop interfaces
+- Console.log statements
+- Ant Design components in new features (without justification)
+
+### Review Response Template
+
+Use this template when reviewing UI PRs:
+
+```markdown
+## UI PR Review
+
+### ✅ Passed Checks
+- [List what meets standards]
+
+### ❌ Required Changes
+- [List blocking issues with file:line references]
+
+### ⚠️ Suggestions
+- [List non-blocking improvements]
+
+### 📋 Verification
+- [ ] `yarn lint` passes
+- [ ] `yarn test` passes
+- [ ] `yarn build` succeeds
+- [ ] No `any` types
+- [ ] No hardcoded strings
+- [ ] Proper MUI usage
+- [ ] Screenshots provided
+
+See [UI_PR_REVIEW_GUIDELINES.md](../openmetadata-ui/UI_PR_REVIEW_GUIDELINES.md) for complete checklist.
+```
+
+Remember: This is a complex multi-language project. Build times are substantial. NEVER cancel long-running builds or tests. Always validate changes with real user scenarios before considering the work complete.

.github/dependabot.yml

@@ -0,0 +1,67 @@
+version: 2
+
+# NOTE: This file controls Dependabot version-update PRs only.
+# It does NOT suppress Dependabot security alerts on the Security tab.
+# To auto-dismiss transitive (indirect) alerts, configure auto-triage rules at
+# Settings -> Code security -> Dependabot -> "Manage rules".
+
+updates:
+  - package-ecosystem: "pip"
+    directory: "/ingestion"
+    schedule:
+      interval: "weekly"
+      day: "monday"
+    open-pull-requests-limit: 5
+    labels:
+      - "dependencies"
+      - "python"
+    groups:
+      python-minor-patch:
+        update-types:
+          - "minor"
+          - "patch"
+    ignore:
+      # urllib3 is pinned <2.0 transitively via tableauserverclient==0.25.
+      # See ingestion/setup.py comment on the tableau pin.
+      - dependency-name: "urllib3"
+        versions: [">=2.0.0"]
+
+  - package-ecosystem: "maven"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+      day: "monday"
+    open-pull-requests-limit: 5
+    labels:
+      - "dependencies"
+      - "java"
+    groups:
+      maven-minor-patch:
+        update-types:
+          - "minor"
+          - "patch"
+
+  - package-ecosystem: "npm"
+    directory: "/openmetadata-ui/src/main/resources/ui"
+    schedule:
+      interval: "weekly"
+      day: "monday"
+    open-pull-requests-limit: 5
+    labels:
+      - "dependencies"
+      - "javascript"
+    groups:
+      npm-minor-patch:
+        update-types:
+          - "minor"
+          - "patch"
+
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+      day: "monday"
+    open-pull-requests-limit: 3
+    labels:
+      - "dependencies"
+      - "github-actions"

.github/e2eLabeler.yml

@@ -0,0 +1,38 @@
+# E2E Labels when source files are changes
+# strings specified in the format of "openmetadata-ui/**/*[a-zA-Z]*" handles case sensitivity
+
+"e2e:Settings":
+  - "openmetadata-ui/**/[Ss]ettings/**/*"
+
+"e2e:DataAssets":
+  - "openmetadata-ui/**/*[dD]atabase*"
+  - "openmetadata-ui/**/*[tT]able*"
+  - "openmetadata-ui/**/*[tT]opic*"
+  - "openmetadata-ui/**/*[dD]ashboard*"
+  - "openmetadata-ui/**/*[pP]ipeline*"
+  - "openmetadata-ui/**/*[mM]lmodel*"
+  - "openmetadata-ui/**/*[cC]ontainer*"
+  - "openmetadata-ui/**/*[sS]erach[iI]ndex*"
+  - "openmetadata-ui/**/*[tT]ask*"
+  - "openmetadata-ui/**/*[sS]earch*"
+  - "openmetadata-ui/**/*[mM]y[dD]ata*"
+  - "openmetadata-ui/**/*[eE]xplore*"
+  - "openmetadata-ui/**/*[eE]ntity*"
+
+"e2e:Governance":
+  - "openmetadata-ui/**/*[gG]lossary*"
+  - "openmetadata-ui/**/*[tT]ag*"
+  - "openmetadata-ui/**/*[dD]omain*"
+
+"e2e:Observability":
+  - "openmetadata-ui/**/*[Aa]lert*"
+  - "openmetadata-ui/**/*[Ii]ncident[Mm]anager*"
+  - "openmetadata-ui/**/*[Mm]etric*"
+  - "openmetadata-ui/**/*[Dd]ata[qQ]uality*"
+  - "openmetadata-ui/**/*[Dd]ata[Ii]nsight*"
+  - "openmetadata-ui/**/*[Oo]bservability*"
+
+"e2e:Integration":
+  - "openmetadata-ui/**/*integration*"
+  - "openmetadata-ui/**/*[iI]ngestion*"
+  - "openmetadata-ui/**/*[sS]ervice*"

.github/labeler.yml

@@ -0,0 +1,27 @@
+# Label when source files are for UI team
+ui:
+  - "openmetadata-ui/**/*"
+  - "package.json"
+
+# Label for Devops when source files is in docker or github
+devops:
+  - "docker/**/*"
+  - ".github/**/*"
+
+# Label for backend team
+backend:
+    - "openmetadata-service/**/*"
+    - "openmetadata-clients/**/*"
+    - "openmetadata-dist/**/*"
+    - "openmetadata-service/**/*"
+    - "openmetadata-spec/**/*"
+
+# Label for ingestion, when source file is in ingestion or airflow
+ingestion:
+  - "openmetadata-airflow-apis/**/*"
+  - "ingestion/**/*"
+  - "ingestion-core/**/*"
+
+# Label to help us track spec changes
+spec:
+  - "openmetadata-spec/**/*"

.github/pull_request_template.md

@@ -0,0 +1,137 @@
+<!--
+Thank you for your contribution!
+Unless your change is trivial, please create an issue to discuss the change before creating a PR.
+-->
+
+### Describe your changes:
+
+Fixes #<issue-number>
+<!--
+Linking an issue is REQUIRED. Replace <issue-number> with the GitHub issue number this PR addresses
+(e.g., `Fixes #12345`). GitHub will auto-link it. If no issue exists, please open one first so the
+problem and design can be discussed before review.
+-->
+
+<!--
+Short blurb explaining:
+- What changes did you make?
+- Why did you make them?
+-->
+
+I worked on ... because ...
+
+#
+### Type of change:
+<!-- You should choose 1 option and delete options that aren't relevant -->
+- [ ] Bug fix
+- [ ] Improvement
+- [ ] New feature
+- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] Documentation
+
+#
+### High-level design:
+<!--
+REQUIRED for large PRs (new features, refactors, breaking changes, or anything touching >5 files).
+Skip for small bug fixes and trivial changes.
+
+Cover:
+- Architecture / approach you took and why
+- Key components or files added/changed and how they interact
+- Alternatives considered and why you rejected them
+- Any migration, backward-compatibility, or rollout concerns
+- Diagrams or links to design docs / RFCs if available
+-->
+
+N/A — small change. <!-- Or fill in the design above -->
+
+#
+### Tests:
+
+#### Use cases covered
+<!--
+List the user-visible scenarios this PR exercises. Example:
+- User with Admin role can create a Glossary Term with a parent term
+- Ingestion run for Snowflake correctly extracts row counts for partitioned tables
+-->
+
+#### Unit tests
+<!--
+- [ ] I added unit tests for the new/changed logic.
+- Files added/updated:
+- Coverage on changed classes (run `mvn jacoco:report` for backend, `yarn test:coverage` for UI,
+  `make unit_ingestion` for ingestion). Target is 90% line coverage on changed classes.
+- Coverage %: <e.g., 92% on EntityRepository.java>
+-->
+
+#### Backend integration tests
+<!--
+- [ ] I added integration tests in `openmetadata-integration-tests/` for new/changed API endpoints.
+- [ ] Not applicable (no backend API changes).
+- Files added/updated:
+-->
+
+#### Ingestion integration tests
+<!--
+- [ ] I added/updated ingestion integration tests for connector changes.
+- [ ] Not applicable (no ingestion changes).
+- Files added/updated:
+-->
+
+#### Playwright (UI) tests
+<!--
+- [ ] I added Playwright E2E tests under `openmetadata-ui/.../ui/playwright/` for UI changes.
+- [ ] Not applicable (no UI changes).
+- Files added/updated:
+-->
+
+#### Manual testing performed
+<!--
+List the manual test steps you performed before requesting review. Example:
+1. Started local stack via `./docker/run_local_docker.sh -m ui -d mysql`
+2. Logged in as admin, created entity X, verified Y appears in the UI
+3. Triggered ingestion for Snowflake source, confirmed lineage edges in the explore page
+-->
+
+#
+### UI screen recording / screenshots:
+<!--
+REQUIRED for any PR that changes the UI. Drag-and-drop a short screen recording (.mov / .mp4 / .gif)
+demonstrating the change end-to-end, plus before/after screenshots where relevant.
+Mark "Not applicable" if there are no UI changes.
+-->
+
+Not applicable. <!-- Or attach recording/screenshots above -->
+
+#
+### Checklist:
+<!-- add an x in [] if done, don't mark items that you didn't do !-->
+- [x] I have read the [**CONTRIBUTING**](https://docs.open-metadata.org/developers/contribute) document.
+- [ ] My PR title is `Fixes <issue-number>: <short explanation>`
+- [ ] My PR is linked to a GitHub issue via `Fixes #<issue-number>` above.
+- [ ] I have commented on my code, particularly in hard-to-understand areas.
+- [ ] For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
+- [ ] For UI changes: I attached a screen recording and/or screenshots above.
+- [ ] I have added tests (unit / integration / Playwright as applicable) and listed them above.
+
+<!-- Based on the type(s) of your change, uncomment the required checklist 👇 -->
+
+<!-- Bug fix
+- [ ] I have added a test that covers the exact scenario we are fixing. For complex issues, comment the issue number in the test for future reference.
+-->
+
+<!-- Improvement
+- [ ] I have added tests around the new logic.
+- [ ] For connector/ingestion changes: I updated the documentation.
+-->
+
+<!-- New feature
+- [ ] The issue properly describes why the new feature is needed, what's the goal, and how we are building it. Any discussion
+    or decision-making process is reflected in the issue.
+- [ ] I have updated the documentation.
+- [ ] I have added tests around the new logic.
+-->
+
+<!-- Breaking change
+- [ ] I have added the tag `Backward-Incompatible-Change`.
+-->

.github/scripts/label_connector.py

@@ -0,0 +1,98 @@
+"""Auto-label connector bugs.
+
+Reads the "Connector" field from the issue body and applies one connector:* label.
+- Exactly one rule matches → that label.
+- Zero or multiple matches → connector:other.
+Any other connector:* label this script manages is removed.
+
+To add a connector: append one row to RULES.
+"""
+
+import json
+import os
+import re
+from urllib.error import HTTPError
+from urllib.parse import quote
+from urllib.request import Request, urlopen
+
+RULES = [
+    ("connector:mssql",         r"\b(mssql|ms ?sql|sql ?server)\b"),
+    ("connector:mysql",         r"\bmysql\b"),
+    ("connector:s3",            r"\b(aws )?s3\b"),
+    ("connector:bigquery",      r"\b(big ?query|gcp bigquery)\b"),
+    ("connector:snowflake",     r"\bsnowflake\b"),
+    ("connector:redshift",      r"\b(aws )?redshift\b"),
+    ("connector:unity-catalog", r"\bunity ?catalog\b"),
+    ("connector:powerbi",       r"\bpower ?bi\b"),
+    ("connector:postgres",      r"\bpostgres(ql)?\b"),
+    ("connector:athena",        r"\b(aws )?athena\b"),
+    ("connector:tableau",       r"\btableau\b"),
+    ("connector:looker",        r"\blooker\b"),
+    ("connector:airflow",       r"\b(apache )?airflow\b"),
+    ("connector:dbt",           r"\bdbt( ?cloud| ?core)?\b"),
+    ("connector:databricks",    r"\bdatabricks\b"),
+    ("connector:fabric",        r"\b(microsoft |ms )?fabric\b"),
+]
+
+OTHER = "connector:other"
+MANAGED = {label for label, _ in RULES} | {OTHER}
+
+TOKEN = os.environ["GITHUB_TOKEN"]
+REPO = os.environ["GITHUB_REPOSITORY"]
+
+
+def gh(method, path, body=None, ok=(200, 201, 204)):
+    data = json.dumps(body).encode() if body else None
+    req = Request(
+        f"https://api.github.com/repos/{REPO}{path}",
+        data=data, method=method,
+        headers={
+            "Authorization": f"Bearer {TOKEN}",
+            "Accept": "application/vnd.github+json",
+            "Content-Type": "application/json",
+        },
+    )
+    try:
+        with urlopen(req) as r:
+            status = r.status
+    except HTTPError as e:
+        status = e.code
+    if status not in ok:
+        raise RuntimeError(f"{method} {path} returned HTTP {status}")
+    return status
+
+
+def classify(field_value):
+    norm = re.sub(r"\s+", " ", re.sub(r"[_\-/.,()]", " ", field_value.lower())).strip()
+    hits = [label for label, pattern in RULES if re.search(pattern, norm)]
+    return hits[0] if len(hits) == 1 else OTHER
+
+
+def main():
+    with open(os.environ["GITHUB_EVENT_PATH"]) as f:
+        issue = json.load(f)["issue"]
+
+    match = re.search(r"### Connector\s*\n+\s*([^\n]+)", issue.get("body") or "")
+    field = match.group(1).strip() if match else ""
+    if not field or field == "_No response_":
+        print("No Connector field — skipping.")
+        return
+
+    target = classify(field)
+    current = {label["name"] for label in issue.get("labels", [])}
+    print(f'Resolved to "{target}"')
+
+    if gh("GET", f"/labels/{quote(target, safe='')}", ok=(200, 404)) == 404:
+        gh("POST", "/labels", {"name": target, "color": "aaaaaa", "description": "Connector"})
+
+    for label in current & MANAGED - {target}:
+        gh("DELETE", f"/issues/{issue['number']}/labels/{quote(label, safe='')}", ok=(200, 404))
+        print(f'Removed "{label}"')
+
+    if target not in current:
+        gh("POST", f"/issues/{issue['number']}/labels", {"labels": [target]})
+        print(f'Added "{target}"')
+
+
+if __name__ == "__main__":
+    main()

.github/teams.yml

@@ -0,0 +1,76 @@
+# Add here any member that should belong to either a specific team,
+# or that can have the tests automatically validated to run safely.
+
+"safe to test":
+  - '@ayush-shah'
+  - '@TeddyCr'
+  - '@ulixius9'
+  - '@pmbrull'
+  - '@aniketkatkar97'
+  - '@Ashish8689'
+  - '@chirag-madlani'
+  - '@shahsank3t'
+  - '@ShaileshParmar11'
+  - '@karanh37'
+  - '@harshach'
+  - '@mohityadav766'
+  - '@sureshms'
+  - '@akash-jain-10'
+  - '@tutte'
+  - '@IceS2'
+  - '@snyk-bot'
+  - '@dependabot'
+  - '@harshsoni2024'
+  - '@SumanMaharana'
+  - '@Prajwal214'
+  - '@sonika-shah'
+  - '@keshavmohta09'
+  - '@sweta1308'
+  - '@shrushti2000'
+  - '@mohittilala'
+  - '@pellejador'
+  - '@pranita09'
+  - '@edg956'
+  - '@manerow'
+  - '@miriann-uu'
+  - '@github-actions[bot]'
+  - '@Shreyansh100704'
+  - '@Vishnuujain'
+  - '@lautel'
+  - '@tomasmontielp'
+
+ingestion:
+  - '@ayush-shah'
+  - '@TeddyCr'
+  - '@ulixius9'
+  - '@pmbrull'
+  - '@IceS2'
+  - '@harshsoni2024'
+  - '@SumanMaharana'
+  - '@keshavmohta09'
+  - '@mohittilala'
+  - '@edg956'
+
+UI:
+  - '@aniketkatkar97'
+  - '@Ashish8689'
+  - '@chirag-madlani'
+  - '@shahsank3t'
+  - '@ShaileshParmar11'
+  - '@karanh37'
+  - '@sweta1308'
+  - '@shrushti2000'
+  - '@pranita09'
+
+backend:
+  - '@harshach'
+  - '@mohityadav766'
+  - '@sureshms'
+  - '@sonika-shah'
+  - '@manerow'
+
+devops:
+  - '@akash-jain-10'
+  - '@tutte'
+  - '@pellejador'
+  - '@miriann-uu'

.github/trivy/templates/github.tpl

@@ -0,0 +1,35 @@
+{{- range . }}
+  <h2> 🛡️ TRIVY SCAN RESULT 🛡️ </h2>
+  <h4> Target: <code>{{ .Target }}</code></h4>
+  {{- if .Vulnerabilities }}
+    <h4>Vulnerabilities ({{ len .Vulnerabilities }})</h4>
+    <table border="1" cellspacing="0" cellpadding="5">
+      <thead>
+        <tr>
+          <th>Package</th>
+          <th>Vulnerability ID</th>
+          <th>Severity</th>
+          <th>Installed Version</th>
+          <th>Fixed Version</th>
+        </tr>
+      </thead>
+      <tbody>
+      {{- range .Vulnerabilities }}
+        <tr>
+          <td><code>{{ .PkgName }}</code></td>
+          <td><a href="{{ .PrimaryURL }}" target="_blank">{{ .VulnerabilityID }}</a></td>
+          <td>
+            {{- if eq .Severity "CRITICAL" }} 🔥 CRITICAL 
+            {{- else if eq .Severity "HIGH" }} 🚨 HIGH 
+            {{- else }} {{ .Severity }} {{- end }}
+          </td>
+          <td>{{ .InstalledVersion }}</td>
+          <td>{{ if .FixedVersion }}{{ .FixedVersion }}{{ else }}N/A{{ end }}</td>
+        </tr>
+      {{- end }}
+      </tbody>
+    </table>
+  {{- else }}
+    <h4>No Vulnerabilities Found</h4>
+  {{- end }}
+{{- end }}

.github/workflows/airflow-apis-tests.yml

@@ -0,0 +1,162 @@
+
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: airflow-apis-tests
+on:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+    paths:
+      - 'openmetadata-airflow-apis/**'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: airflow-apis-tests-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+env:
+  SONAR_OPTS: >-
+    -Dproject.settings=openmetadata-airflow-apis/sonar-project.properties
+    -Dsonar.pullrequest.key=${{ github.event.pull_request.number }}
+    -Dsonar.pullrequest.branch=${{ github.event.pull_request.head.ref }}
+    -Dsonar.pullrequest.github.repository=OpenMetadata
+    -Dsonar.scm.revision=${{ github.event.pull_request.head.sha }}
+    -Dsonar.pullrequest.provider=github
+
+jobs:
+  airflow-apis-tests:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    steps:
+    - name: Free Disk Space (Ubuntu)
+      uses: jlumbroso/free-disk-space@main
+      with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          docker-images: false
+          swap-storage: true
+    - name: Wait for the labeler
+      uses: lewagon/wait-on-check-action@v1.7.0
+      if: ${{ github.event_name == 'pull_request_target' }}
+      with:
+        ref: ${{ github.event.pull_request.head.sha }}
+        check-name: Team Label
+        repo-token: ${{ secrets.GITHUB_TOKEN }}
+        wait-interval: 90
+
+    - name: Verify PR labels
+      uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+      if: ${{ github.event_name == 'pull_request_target' }}
+      with:
+        github-token: '${{ secrets.GITHUB_TOKEN }}'
+        valid-labels: 'safe to test'
+        pull-request-number: '${{ github.event.pull_request.number }}'
+        disable-reviews: true  # To not auto approve changes
+
+    - name: Checkout
+      uses: actions/checkout@v4
+      with:
+        ref: ${{ github.event.pull_request.head.sha }}
+        fetch-depth: 0
+        filter: blob:none
+
+    - name: Set up JDK 21
+      uses: actions/setup-java@v4
+      with:
+        java-version: '21'
+        distribution: 'temurin'
+
+    - name: Set up Python 3.10
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.10'
+
+    - name: Install Ubuntu dependencies
+      run: |
+        # stop relying on apt cache of GitHub runners
+        sudo apt-get update
+        sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+        librdkafka-dev unixodbc-dev libevent-dev libkrb5-dev
+
+    - name: Generate models
+      run: |
+        python3 -m venv env
+        source env/bin/activate
+        sudo make install_antlr_cli
+        make install_dev generate
+
+    - name: Install open-metadata dependencies
+      run: |
+        source env/bin/activate
+        make install_all install_test
+
+    - name: Start Server and Ingest Sample Data
+      uses: nick-fields/retry@v3.0.2
+      env:
+        INGESTION_DEPENDENCY: "mysql,elasticsearch,sample-data"
+      with:
+        timeout_minutes: 60
+        max_attempts: 2
+        retry_on: error
+        command: ./docker/run_local_docker.sh -m no-ui
+
+    - name: Run Python Tests & Record Coverage
+      run: |
+        source env/bin/activate
+        make install_apis
+        make coverage_apis
+        rm pom.xml
+        # fix coverage xml report for github
+        sed -i 's/openmetadata_managed_apis/\/github\/workspace\/openmetadata-airflow-apis\/openmetadata_managed_apis/g' openmetadata-airflow-apis/ci-coverage.xml
+
+    - name: Push Results in PR to Sonar
+      id: push-to-sonar
+      if: ${{ github.event_name == 'pull_request_target' }}
+      continue-on-error: true
+      uses: SonarSource/sonarqube-scan-action@v7
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        SONAR_TOKEN: ${{ secrets.AIRFLOW_APIS_SONAR_TOKEN }}
+      with:
+        projectBaseDir: openmetadata-airflow-apis/
+        args: ${{ env.SONAR_OPTS }}
+
+    # next two steps are for retrying "Push Results in PR to Sonar" step in case it fails
+    - name: Wait to retry 'Push Results in PR to Sonar'
+      if: ${{ github.event_name == 'pull_request_target' && steps.push-to-sonar.outcome != 'success' }}
+      run: sleep 20s
+      shell: bash
+
+    - name: Retry 'Push Results in PR to Sonar'
+      uses: SonarSource/sonarqube-scan-action@v7
+      if: ${{ github.event_name == 'pull_request_target' && steps.push-to-sonar.outcome != 'success' }}
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        SONAR_TOKEN: ${{ secrets.AIRFLOW_APIS_SONAR_TOKEN }}
+      with:
+        projectBaseDir: openmetadata-airflow-apis/
+        args: ${{ env.SONAR_OPTS }}
+
+    - name: Push Results to Sonar
+      uses: SonarSource/sonarqube-scan-action@v7
+      if: ${{ github.event_name == 'push' }}
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        SONAR_TOKEN: ${{ secrets.AIRFLOW_APIS_SONAR_TOKEN }}
+      with:
+        projectBaseDir: openmetadata-airflow-apis/
+        args: -Dproject.settings=openmetadata-airflow-apis/sonar-project.properties

.github/workflows/auto-cherry-pick-labeled-prs.yaml

@@ -0,0 +1,89 @@
+---
+name: Cherry-pick labeled PRs to OpenMetadata release branch on merge
+# yamllint disable-line rule:comments
+run-name: OpenMetadata release cherry-pick PR #${{ github.event.pull_request.number }}
+
+# yamllint disable-line rule:truthy
+on:
+  pull_request_target:
+    types: [closed]
+    branches:
+      - main
+permissions:
+  contents: write
+  pull-requests: write
+  issues: write
+env:
+  CURRENT_RELEASE_ENDPOINT: ${{ vars.CURRENT_RELEASE_ENDPOINT }}  # Endpoint that returns the current release version in json format
+jobs:
+  get_release_branch:
+    if: github.event.pull_request.merged == true &&
+        contains(github.event.pull_request.labels.*.name, 'To release')
+    runs-on: ubuntu-latest
+    outputs:
+      release_branches: ${{ steps.get_release_version.outputs.release_branches }}
+    steps:
+      - name: Get the release version
+        id: get_release_version
+        run: |
+          CURRENT_RELEASE=$(curl -s $CURRENT_RELEASE_ENDPOINT | jq -c '.collate_branches // []')
+          echo "release_branches=${CURRENT_RELEASE}" >> $GITHUB_OUTPUT
+
+  cherry_pick_to_release_branch:
+    needs: get_release_branch
+    if: needs.get_release_branch.outputs.release_branches != '' && needs.get_release_branch.outputs.release_branches != '[]'
+    runs-on: ubuntu-latest  # Running it on ubuntu-latest on purpose (we're not using all the free minutes)
+    strategy:
+      fail-fast: false
+      matrix:
+        branch: ${{ fromJson(needs.get_release_branch.outputs.release_branches) }}
+    steps:
+      - name: Checkout main branch
+        uses: actions/checkout@v4
+        with:
+          ref: main
+          fetch-depth: 0
+      - name: Cherry-pick changes from PR
+        id: cherry_pick
+        continue-on-error: true
+        run: |
+          git config --global user.email "release-bot@open-metadata.org"
+          git config --global user.name "OpenMetadata Release Bot"
+          git fetch origin ${{ matrix.branch }}
+          git checkout ${{ matrix.branch }}
+          git cherry-pick -x ${{ github.event.pull_request.merge_commit_sha }}
+      - name: Push changes to release branch
+        id: push_changes
+        continue-on-error: true
+        if: steps.cherry_pick.outcome == 'success'
+        run: |
+          git push origin ${{ matrix.branch }}
+      - name: Post a comment on failure
+        if: steps.cherry_pick.outcome != 'success' || steps.push_changes.outcome != 'success'
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const prNumber = context.payload.pull_request.number;
+            const releaseBranch = '${{ matrix.branch }}';
+            const workflowRunUrl = `${process.env.GITHUB_SERVER_URL}/${process.env.GITHUB_REPOSITORY}/actions/runs/${process.env.GITHUB_RUN_ID}`;
+            github.rest.issues.createComment({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: prNumber,
+              body: `Failed to cherry-pick changes to the ${releaseBranch} branch.
+                Please cherry-pick the changes manually.
+                You can find more details [here](${workflowRunUrl}).`
+            })
+      - name: Post a comment on success
+        if: steps.cherry_pick.outcome == 'success' && steps.push_changes.outcome == 'success'
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const prNumber = context.payload.pull_request.number;
+            const releaseBranch = '${{ matrix.branch }}';
+            github.rest.issues.createComment({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: prNumber,
+              body: `Changes have been cherry-picked to the ${releaseBranch} branch.`
+            })

.github/workflows/claude.yml

@@ -0,0 +1,64 @@
+name: Claude Code
+
+on:
+  issue_comment:
+    types: [created]
+  pull_request_review_comment:
+    types: [created]
+  issues:
+    types: [opened, assigned]
+  pull_request_review:
+    types: [submitted]
+
+jobs:
+  claude:
+    if: |
+      (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
+      (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
+      (github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
+      (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: read
+      issues: read
+      id-token: write
+      actions: read # Required for Claude to read CI results on PRs
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+
+      - name: Run Claude Code
+        id: claude
+        uses: anthropics/claude-code-action@beta
+        with:
+          claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
+
+          # This is an optional setting that allows Claude to read CI results on PRs
+          additional_permissions: |
+            actions: read
+          
+          # Optional: Specify model (defaults to Claude Sonnet 4, uncomment for Claude Opus 4)
+          # model: "claude-opus-4-20250514"
+          
+          # Optional: Customize the trigger phrase (default: @claude)
+          # trigger_phrase: "/claude"
+          
+          # Optional: Trigger when specific user is assigned to an issue
+          # assignee_trigger: "claude-bot"
+          
+          # Optional: Allow Claude to run specific commands
+          # allowed_tools: "Bash(npm install),Bash(npm run build),Bash(npm run test:*),Bash(npm run lint:*)"
+          
+          # Optional: Add custom instructions for Claude to customize its behavior for your project
+          # custom_instructions: |
+          #   Follow our coding standards
+          #   Ensure all new code has tests
+          #   Use TypeScript for new files
+          
+          # Optional: Custom environment variables for Claude
+          # claude_env: |
+          #   NODE_ENV: test
+

.github/workflows/codeql.yml

@@ -0,0 +1,23 @@
+name: CodeQL Advanced
+
+on:
+  workflow_dispatch:
+    inputs:
+      branch:
+        description: "Branch to run the workflow on"
+        required: true
+        default: "main"
+        type: string
+jobs:
+  analyze:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft }}
+    permissions:
+      contents: write
+      pull-requests: write
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.inputs.branch || github.event.pull_request.head.ref || github.ref }}
+          token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/data-access-request-e2e.yml

@@ -0,0 +1,81 @@
+#  Copyright 2026 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Workflows and Data Access Request E2E tests
+
+on:
+  workflow_dispatch:
+  pull_request_target:
+    types: [opened, synchronize, reopened, labeled, ready_for_review]
+
+permissions:
+  contents: read
+  pull-requests: read
+  checks: write
+
+concurrency:
+  group: data-access-request-e2e-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  check-changes:
+    runs-on: ubuntu-latest
+    outputs:
+      dar: ${{ steps.filter.outputs.dar }}
+    steps:
+      - uses: dorny/paths-filter@v4
+        id: filter
+        with:
+          filters: |
+            dar:
+              - 'openmetadata-service/src/main/java/org/openmetadata/service/resources/tasks/TaskResource.java'
+              - 'openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TaskRepository.java'
+              - 'openmetadata-service/src/main/java/org/openmetadata/service/resources/feeds/FeedResource.java'
+              - 'openmetadata-service/src/main/java/org/openmetadata/service/governance/workflows/**'
+
+  data-access-request-e2e:
+    needs: check-changes
+    runs-on: ubuntu-latest
+    if: |
+      !github.event.pull_request.draft &&
+      (github.event_name == 'workflow_dispatch' || needs.check-changes.outputs.dar == 'true') &&
+      (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test')
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+
+      - name: Trigger Collate Playwright run & wait
+        uses: the-actions-org/workflow-dispatch@v4
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SHA: ${{ github.event.pull_request.head.sha || github.sha }}
+        with:
+          workflow: Workflows and Data Access Request E2E tests (from OpenMetadata PR)
+          ref: main
+          repo: open-metadata/openmetadata-collate
+          token: ${{ secrets.COLLATE_PAT }}
+          wait-for-completion: true
+          inputs: '{ "sha": "${{ env.SHA }}", "event": "${{ github.event_name }}" }'

.github/workflows/docker-k8s-operator.yml

@@ -0,0 +1,147 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: docker-k8s-operator release app
+on:
+  workflow_dispatch:
+    inputs:
+      docker_release_tag:
+        description: "K8s Operator Docker Image Tag"
+        required: true
+      push_latest_tag_to_release:
+        description: "Do you want to update docker image latest tag as well ?"
+        type: boolean
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  maven-build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha || github.ref }}
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: 21
+          distribution: 'temurin'
+
+      - name: Build K8s Operator Application
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          mvn -DskipTests clean package -pl openmetadata-k8s-operator -am
+
+      - name: Run K8s Operator Tests
+        run: |
+          mvn test -pl openmetadata-k8s-operator
+
+      - name: Upload K8s Operator JAR to Artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: k8s-operator-binary
+          path: openmetadata-k8s-operator/target/*.jar
+
+  release-project-event-workflow_dispatch:
+    if: github.event_name == 'workflow_dispatch'
+    name: Release k8s operator with workflow_dispatch event
+    runs-on: ubuntu-latest
+    needs: maven-build
+    steps:
+      - name: Fetch Release Data
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        id: attach_release
+        run: |
+          echo "UPLOAD_URL=$(curl -H "Accept: application/vnd.github+json" -H "Authorization: Bearer $GITHUB_TOKEN" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/repos/open-metadata/OpenMetadata/releases/tags/${{ inputs.docker_release_tag }}-release | jq .upload_url | tr -d '"' )" >> $GITHUB_OUTPUT
+
+      - name: Download application from Artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: k8s-operator-binary
+
+      - name: Upload k8s operator to release
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          asset_path: ./openmetadata-k8s-operator-${{ inputs.docker_release_tag }}-boot.jar
+          upload_url: ${{ steps.attach_release.outputs.UPLOAD_URL }}
+          asset_name: openmetadata-k8s-operator-${{ inputs.docker_release_tag }}.jar
+          asset_content_type: application/java-archive
+
+  push_to_docker_hub:
+    runs-on: ubuntu-latest
+    if: ${{ always() && contains(join(needs.*.result, ','), 'success') }}
+    needs: [ maven-build ]
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true
+
+      - name: Check out the Repo
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha || github.ref }}
+
+      - name: Download application from Artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: k8s-operator-binary
+          path: openmetadata-k8s-operator/target/
+
+      - name: Set build arguments
+        id: build-args
+        run: |
+          echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_OUTPUT
+
+      - name: Prepare for Docker Build&Push
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/omjob-operator
+          tag: ${{ inputs.docker_release_tag || 'pr-test' }}
+          push_latest: ${{ inputs.push_latest_tag_to_release || false }}
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v6
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        with:
+          context: ./openmetadata-k8s-operator
+          platforms: linux/amd64,linux/arm64
+          push: ${{ github.event_name == 'workflow_dispatch' }}
+          tags: ${{ steps.prepare.outputs.tags }}
+          file: ./openmetadata-k8s-operator/Dockerfile
+          build-args: |
+            BUILD_DATE=${{ steps.build-args.outputs.BUILD_DATE }}
+            COMMIT_ID=${{ github.sha }}

.github/workflows/docker-openmetadata-db.yml

@@ -0,0 +1,51 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: docker-openmetadata-db docker
+on:
+  workflow_dispatch:
+    inputs:
+      docker_release_tag:
+        description: "OpenMetadata MySQL Docker Image Tag"
+        required: true
+      push_latest_tag_to_release:
+        description: "Do you want to update docker image latest tag as well ?"
+        type: boolean
+
+jobs:
+  push_to_docker_hub:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out the Repo
+        uses: actions/checkout@v4
+
+      - name: Prepare for Docker Build&Push
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/db
+          tag: ${{ inputs.docker_release_tag }}
+          push_latest: ${{ inputs.push_latest_tag_to_release }}
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+
+      - name: Build and push if event is workflow_dispatch and input is checked
+        uses: docker/build-push-action@v6
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.prepare.outputs.tags }}
+          file: ./docker/mysql/Dockerfile_mysql

.github/workflows/docker-openmetadata-ingestion-base-slim.yml

@@ -0,0 +1,57 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: docker-openmetadata-ingestion-base-slim docker
+on:
+  workflow_dispatch:
+    inputs:
+      docker_release_tag:
+        description: "Ingestion Base Slim Docker Image Tag (3 digit docker image tag)"
+        required: true
+      pypi_release_version:
+        description: "Provide the Release Version (4 digit docker image tag)"
+        required: true
+      push_latest_tag_to_release:
+        description: "Do you want to update docker image latest tag as well ?"
+        type: boolean
+
+jobs:
+  push_to_docker_hub:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out the Repo
+        uses: actions/checkout@v4
+
+      - name: Prepare for Docker Build&Push
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/ingestion-base-slim
+          tag: ${{ inputs.docker_release_tag }}
+          push_latest: ${{ inputs.push_latest_tag_to_release }}
+          is_ingestion: true
+          release_version: ${{ inputs.pypi_release_version }}
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+      - name: Build and push if event is workflow_dispatch and input is checked
+        uses: docker/build-push-action@v6
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.prepare.outputs.tags }}
+          file: ./ingestion/operators/docker/Dockerfile
+          build-args: |
+            INGESTION_DEPENDENCY=slim

.github/workflows/docker-openmetadata-ingestion-base.yml

@@ -0,0 +1,66 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: docker-openmetadata-ingestion-base docker
+on:
+  workflow_dispatch:
+    inputs:
+      docker_release_tag:
+        description: "Ingestion Base Docker Image Tag (3 digit docker image tag)"
+        required: true
+      pypi_release_version:
+        description: "Provide the Release Version (4 digit docker image tag)"
+        required: true
+      push_latest_tag_to_release:
+        description: "Do you want to update docker image latest tag as well ?"
+        type: boolean
+
+jobs:
+  push_to_docker_hub:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+          
+      - name: Check out the Repo
+        uses: actions/checkout@v4
+
+      - name: Prepare for Docker Build&Push
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/ingestion-base
+          tag: ${{ inputs.docker_release_tag }}
+          push_latest: ${{ inputs.push_latest_tag_to_release }}
+          is_ingestion: true
+          release_version: ${{ inputs.pypi_release_version }}
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+      - name: Build and push if event is workflow_dispatch and input is checked
+        uses: docker/build-push-action@v6
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.prepare.outputs.tags }}
+          file: ./ingestion/operators/docker/Dockerfile

.github/workflows/docker-openmetadata-ingestion.yml

@@ -0,0 +1,91 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: docker-openmetadata-ingestion docker
+on:
+  workflow_dispatch:
+    inputs:
+      docker_release_tag:
+        description: "Ingestion Docker Image Tag (3 digit docker image tag)"
+        required: true
+      pypi_release_version:
+        description: "Provide the Release Version (4 digit docker image tag)"
+        required: true
+      push_latest_tag_to_release:
+        description: "Mark this as latest tag as well ?"
+        type: boolean
+
+jobs:
+  push_to_docker_hub:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+      - name: Check out the Repo
+        uses: actions/checkout@v4
+
+      - name: Prepare for Docker Build&Push
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/ingestion
+          tag: ${{ inputs.docker_release_tag }}
+          push_latest: ${{ inputs.push_latest_tag_to_release }}
+          is_ingestion: true
+          release_version: ${{ inputs.pypi_release_version }}
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+      - name: Build and push if event is workflow_dispatch and input is checked
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.prepare.outputs.tags }}
+          file: ./ingestion/Dockerfile
+
+      # Publish the openmetadata/ingestion-slim image with fewer dependencies
+      - name: Prepare Slim for Docker Build&Push
+        id: prepare-sim
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/ingestion-slim
+          tag: ${{ inputs.docker_release_tag }}
+          push_latest: ${{ inputs.push_latest_tag_to_release }}
+          is_ingestion: true
+          release_version: ${{ inputs.pypi_release_version }}
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+      - name: Build and push Slim if event is workflow_dispatch and input is checked
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.prepare-sim.outputs.tags }}
+          file: ./ingestion/Dockerfile
+          build-args: |
+            INGESTION_DEPENDENCY=slim

.github/workflows/docker-openmetadata-postgres.yml

@@ -0,0 +1,50 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: docker-openmetadata-postgres-db docker
+on:
+  workflow_dispatch:
+    inputs:
+      docker_release_tag:
+        description: "OpenMetadata PostgreSQL Docker Image Tag"
+        required: true
+      push_latest_tag_to_release:
+        description: "Do you want to update docker image latest tag as well ?"
+        type: boolean
+
+jobs:
+  push_to_docker_hub:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out the Repo
+        uses: actions/checkout@v4
+
+      - name: Prepare for Docker Build&Push
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/postgresql
+          tag: ${{ inputs.docker_release_tag }}
+          push_latest: ${{ inputs.push_latest_tag_to_release }}
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+      - name: Build and push if event is workflow_dispatch and input is checked
+        uses: docker/build-push-action@v6
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.prepare.outputs.tags }}
+          file: ./docker/postgresql/Dockerfile_postgres

.github/workflows/docker-openmetadata-server.yml

@@ -0,0 +1,121 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: docker-openmetadata-server release app
+on:
+  workflow_dispatch:
+    inputs:
+      docker_release_tag:
+        description: "Server Docker Image Tag"
+        required: true
+      push_latest_tag_to_release:
+        description: "Do you want to update docker image latest tag as well ?"
+        type: boolean
+jobs:
+  maven-build:
+    runs-on: ubuntu-latest
+    steps:
+
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: 21
+          distribution: 'temurin'
+
+      - name: Install antlr cli
+        run: |
+          sudo make install_antlr_cli
+
+      - name: Setup jq
+        run: |
+          sudo apt-get update
+          sudo apt-get install jq
+
+      - name: Build OpenMetadata Server Application
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          mvn -DskipTests clean package
+
+      - name: Upload OpenMetadata application to Artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: openmetadata-binary
+          path: /home/runner/work/OpenMetadata/OpenMetadata/openmetadata-dist/target/*.tar.gz
+
+  release-project-event-workflow_dispatch:
+    if: github.event_name == 'workflow_dispatch'
+    name: Release app with workflow_dispatch event
+    runs-on: ubuntu-latest
+    needs: maven-build
+    steps:
+
+      - name: Fetch Release Data
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        id: attach_release
+        run: |
+          echo "UPLOAD_URL=$(curl -H "Accept: application/vnd.github+json" -H "Authorization: Bearer $GITHUB_TOKEN" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/repos/open-metadata/OpenMetadata/releases/tags/${{ inputs.DOCKER_RELEASE_TAG }}-release | jq .upload_url | tr -d '"' )" >> $GITHUB_OUTPUT
+
+      - name: Download application from Artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: openmetadata-binary
+
+      - name: Upload app to release
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          asset_path: ./openmetadata-${{ inputs.DOCKER_RELEASE_TAG }}.tar.gz
+          upload_url: ${{ steps.attach_release.outputs.UPLOAD_URL }}
+          asset_name: openmetadata-${{ inputs.DOCKER_RELEASE_TAG }}.tar.gz
+          asset_content_type: application/tar.gz
+
+  push_to_docker_hub:
+    runs-on: ubuntu-latest
+    if: ${{ always() && contains(join(needs.*.result, ','), 'success') }}
+    needs: [ release-project-event-workflow_dispatch ]
+    steps:
+      - name: Check out the Repo
+        uses: actions/checkout@v4
+
+      - name: Set build arguments
+        id: build-args
+        run: |
+          echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_OUTPUT
+
+      - name: Prepare for Docker Build&Push
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/server
+          tag: ${{ inputs.docker_release_tag }}
+          push_latest: ${{ inputs.push_latest_tag_to_release }}
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+      - name: Build and push if event is workflow_dispatch and input is checked
+        uses: docker/build-push-action@v6
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.prepare.outputs.tags }}
+          file: ./docker/docker-compose-quickstart/Dockerfile
+          build-args: |
+            BUILD_DATE=${{ steps.build-args.outputs.BUILD_DATE }}
+            COMMIT_ID=${{ github.sha }}

.github/workflows/git-create-release-branch.yml

@@ -0,0 +1,33 @@
+name: Create Release Branches
+run-name: Create Release Branch ${{ inputs.release_branch_name }}
+on:
+  workflow_dispatch:
+    inputs:
+      release_branch_name:
+        description: "Github Release Branch Name"
+        required: true
+      base_branch_name:
+        description: "Base Branch for Release Branch"
+        required: true
+        default: "main"
+permissions:
+  contents: write
+jobs:
+  create-release-branch:
+    name: Create Release Branch ${{ inputs.release_branch_name }}
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout Repo
+      uses: actions/checkout@v4
+      with:
+        ref: ${{ inputs.base_branch_name }}
+    - name: Update application versions
+      run: |
+        make update_all RELEASE_VERSION=${{ inputs.release_branch_name }}
+    - name: Commit changes to ${{ inputs.release_branch_name }} branch
+      uses: EndBug/add-and-commit@v9
+      with:
+        default_author: github_actions
+        message: 'chore(release): Prepare Branch for `${{ inputs.release_branch_name }}`'
+        add: '.'
+        new_branch: ${{ inputs.release_branch_name }}

.github/workflows/integration-tests-mysql-elasticsearch.yml

@@ -0,0 +1,155 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Integration Tests - MySQL + Elasticsearch
+
+on:
+  merge_group:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+    paths:
+      - "openmetadata-service/**"
+      - "openmetadata-integration-tests/**"
+      - "openmetadata-spec/src/main/resources/json/schema/**"
+      - "openmetadata-sdk/**"
+      - "common/**"
+      - "pom.xml"
+      - "bootstrap/**"
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+
+permissions:
+  contents: read
+  checks: write
+
+concurrency:
+  group: integration-tests-mysql-es-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+jobs:
+  # Detect whether relevant paths changed. When no matching files are modified
+  # the downstream job is skipped via its `if` condition.
+  # A job skipped by `if` reports as "Success", so required checks still pass.
+  changes:
+    name: Detect Changes
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    outputs:
+      backend: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.filter.outputs.backend }}
+    steps:
+      - uses: dorny/paths-filter@v3
+        id: filter
+        if: ${{ github.event_name != 'workflow_dispatch' }}
+        with:
+          filters: |
+            backend:
+              - 'openmetadata-service/**'
+              - 'openmetadata-integration-tests/**'
+              - 'openmetadata-spec/src/main/resources/json/schema/**'
+              - 'openmetadata-sdk/**'
+              - 'common/**'
+              - 'pom.xml'
+              - 'bootstrap/**'
+
+  integration-tests-mysql-elasticsearch:
+    needs: changes
+    runs-on: ubuntu-latest
+    if: ${{ needs.changes.outputs.backend == 'true' }}
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: '${{ secrets.GITHUB_TOKEN }}'
+          valid-labels: 'safe to test'
+          pull-request-number: '${{ github.event.pull_request.number }}'
+          disable-reviews: true  # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Cache Maven dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Set up JDK 21
+        if: steps.cache-output.outputs.exit-code == 0
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Install Ubuntu dependencies
+        if: steps.cache-output.outputs.exit-code == 0
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          librdkafka-dev unixodbc-dev libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build for Integration Tests (MySQL + Elasticsearch)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests clean install -pl :openmetadata-integration-tests -am
+
+      - name: Free build artifacts
+        run: |
+          rm -rf openmetadata-service/target/lib openmetadata-service/target/classes
+          rm -rf openmetadata-spec/target openmetadata-sdk/target common/target
+          rm -rf openmetadata-shaded-deps/*/target
+          df -h /
+
+      - name: Run Integration Tests (MySQL + Elasticsearch)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn verify -pl :openmetadata-integration-tests -Pmysql-elasticsearch
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+      - name: Publish Test Report
+        if: ${{ always() }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: true
+          report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'

.github/workflows/integration-tests-postgres-elasticsearch-redis.yml

@@ -0,0 +1,178 @@
+#  Copyright 2026 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# Runs the full integration test suite with the Redis cache enabled (postgres + elasticsearch +
+# redis), via the cache-tests Maven profile. Catches cache-invalidation and stale-data bugs that
+# only surface when every test path goes through the cache layer.
+#
+# Security note (CodeQL "pull_request_target + checkout untrusted code"):
+# This workflow uses `pull_request_target` so PRs from forks can produce a required check.
+# CodeQL flags the pattern as risky because it checks out PR-controlled code while having
+# access to secrets. The mitigation is the explicit `safe to test` label gate below — the
+# verify-pr-label step rejects the workflow run before any PR code is checked out unless a
+# maintainer has applied the label. This matches the mitigation used by every other
+# integration-tests-*.yml workflow in this repo. If you remove the label gate, you reopen
+# the vulnerability.
+name: Integration Tests - PostgreSQL + Elasticsearch + Redis
+
+on:
+  merge_group:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+    paths:
+      - "openmetadata-service/**"
+      - "openmetadata-integration-tests/**"
+      - "openmetadata-spec/src/main/resources/json/schema/**"
+      - "openmetadata-sdk/**"
+      - "common/**"
+      - "pom.xml"
+      - "bootstrap/**"
+  # `pull_request_target` is intentional and required so the workflow runs against PRs from
+  # forks (which `pull_request` cannot for security reasons). The `safe to test` label gate
+  # below is what makes this safe — see security note in the file header.
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+
+permissions:
+  contents: read
+  checks: write
+
+concurrency:
+  group: integration-tests-pg-es-redis-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+jobs:
+  # Detect whether relevant paths changed. When no matching files are modified
+  # the downstream job is skipped via its `if` condition.
+  # A job skipped by `if` reports as "Success", so required checks still pass.
+  changes:
+    name: Detect Changes
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    outputs:
+      backend: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.filter.outputs.backend }}
+    steps:
+      - uses: dorny/paths-filter@v3
+        id: filter
+        if: ${{ github.event_name != 'workflow_dispatch' }}
+        with:
+          filters: |
+            backend:
+              - 'openmetadata-service/**'
+              - 'openmetadata-integration-tests/**'
+              - 'openmetadata-spec/src/main/resources/json/schema/**'
+              - 'openmetadata-sdk/**'
+              - 'common/**'
+              - 'pom.xml'
+              - 'bootstrap/**'
+
+  integration-tests-postgres-elasticsearch-redis:
+    needs: changes
+    runs-on: ubuntu-latest
+    if: ${{ needs.changes.outputs.backend == 'true' }}
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: '${{ secrets.GITHUB_TOKEN }}'
+          valid-labels: 'safe to test'
+          pull-request-number: '${{ github.event.pull_request.number }}'
+          disable-reviews: true  # To not auto approve changes
+
+      # SECURITY: this step checks out PR-controlled code while the workflow runs with
+      # `pull_request_target` privileges (secrets access). The `Verify PR labels` step above
+      # gates this — the workflow halts before we get here unless a maintainer has applied
+      # the `safe to test` label. CodeQL flags the pattern; the label gate is the accepted
+      # mitigation, mirroring how every other integration-tests-*.yml workflow in this repo
+      # handles fork PRs.
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Cache Maven dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      # Run unconditionally. The previous `if: steps.cache-output.outputs.exit-code == 0` was a
+      # bug — `actions/cache@v4` exposes `cache-hit` (boolean) and `cache-primary-key`, never
+      # `exit-code`. The expression always evaluated to false and the steps never ran. Maven
+      # then ran against whatever JDK the runner happened to ship with, masking the issue.
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          librdkafka-dev unixodbc-dev libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build for Integration Tests (PostgreSQL + Elasticsearch + Redis)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests clean install -pl :openmetadata-integration-tests -am
+
+      - name: Free build artifacts
+        run: |
+          rm -rf openmetadata-service/target/lib openmetadata-service/target/classes
+          rm -rf openmetadata-spec/target openmetadata-sdk/target common/target
+          rm -rf openmetadata-shaded-deps/*/target
+          df -h /
+
+      - name: Run Integration Tests (PostgreSQL + Elasticsearch + Redis)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn verify -pl :openmetadata-integration-tests -Pcache-tests
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+      - name: Publish Test Report
+        if: ${{ always() }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: true
+          report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'

.github/workflows/integration-tests-postgres-opensearch.yml

@@ -0,0 +1,155 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Integration Tests - PostgreSQL + OpenSearch
+
+on:
+  merge_group:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+    paths:
+      - "openmetadata-service/**"
+      - "openmetadata-integration-tests/**"
+      - "openmetadata-spec/src/main/resources/json/schema/**"
+      - "openmetadata-sdk/**"
+      - "common/**"
+      - "pom.xml"
+      - "bootstrap/**"
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+
+permissions:
+  contents: read
+  checks: write
+
+concurrency:
+  group: integration-tests-pg-os-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+jobs:
+  # Detect whether relevant paths changed. When no matching files are modified
+  # the downstream job is skipped via its `if` condition.
+  # A job skipped by `if` reports as "Success", so required checks still pass.
+  changes:
+    name: Detect Changes
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    outputs:
+      backend: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.filter.outputs.backend }}
+    steps:
+      - uses: dorny/paths-filter@v3
+        id: filter
+        if: ${{ github.event_name != 'workflow_dispatch' }}
+        with:
+          filters: |
+            backend:
+              - 'openmetadata-service/**'
+              - 'openmetadata-integration-tests/**'
+              - 'openmetadata-spec/src/main/resources/json/schema/**'
+              - 'openmetadata-sdk/**'
+              - 'common/**'
+              - 'pom.xml'
+              - 'bootstrap/**'
+
+  integration-tests-postgres-opensearch:
+    needs: changes
+    runs-on: ubuntu-latest
+    if: ${{ needs.changes.outputs.backend == 'true' }}
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: '${{ secrets.GITHUB_TOKEN }}'
+          valid-labels: 'safe to test'
+          pull-request-number: '${{ github.event.pull_request.number }}'
+          disable-reviews: true  # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Cache Maven dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Set up JDK 21
+        if: steps.cache-output.outputs.exit-code == 0
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Install Ubuntu dependencies
+        if: steps.cache-output.outputs.exit-code == 0
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          librdkafka-dev unixodbc-dev libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build for Integration Tests (PostgreSQL + OpenSearch)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests clean install -pl :openmetadata-integration-tests -am
+
+      - name: Free build artifacts
+        run: |
+          rm -rf openmetadata-service/target/lib openmetadata-service/target/classes
+          rm -rf openmetadata-spec/target openmetadata-sdk/target common/target
+          rm -rf openmetadata-shaded-deps/*/target
+          df -h /
+
+      - name: Run Integration Tests (PostgreSQL + OpenSearch)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn verify -pl :openmetadata-integration-tests -Ppostgres-opensearch
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+      - name: Publish Test Report
+        if: ${{ always() }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: true
+          report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'

.github/workflows/java-checkstyle.yml

@@ -0,0 +1,106 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Java Checkstyle
+
+on:
+  merge_group:
+  # Trigger analysis when pushing in master or pull requests, and when creating
+  # a pull request.
+  push:
+    branches:
+      - main
+      - "0.[0-9]+.[0-9]+"
+  pull_request_target:
+    types: [opened, synchronize, reopened, labeled]
+    paths-ignore:
+      - "openmetadata-ui/src/main/resources/ui/playwright/doc-generator/**"
+      - "openmetadata-ui/src/main/resources/ui/playwright/docs/**"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: java-checkstyle-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+jobs:
+  java-checkstyle:
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+    permissions:
+      pull-requests: write
+
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+            tool-cache: false
+            android: true
+            dotnet: true
+            haskell: true
+            large-packages: false
+            swap-storage: true
+            docker-images: false
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: '${{ secrets.GITHUB_TOKEN }}'
+          valid-labels: 'safe to test'
+          pull-request-number: '${{ github.event.pull_request.number }}'
+          disable-reviews: true  # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Run checkstyle
+        run: mvn spotless:apply
+
+      - name: Save checkstyle outcome
+        id: git
+        continue-on-error: true
+        run: |
+          git diff-files --quiet
+
+      - name: Create a comment in the PR with the instructions
+        if: ${{ steps.git.outcome != 'success' && github.event_name == 'pull_request_target' }}
+        uses: peter-evans/create-or-update-comment@v1
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          body: |
+            **The Java checkstyle failed.**
+
+            Please run `mvn spotless:apply` in the root of your repository and commit the changes to this PR.
+            You can also use [pre-commit](https://pre-commit.com/) to automate the Java code formatting.
+
+            You can install the pre-commit hooks with `make install_test precommit_install`.
+
+      - name: Java checkstyle failed, check the comment in the PR
+        if: steps.git.outcome != 'success'
+        run: |
+          exit 1

.github/workflows/java-playwright-external.yml

@@ -0,0 +1,537 @@
+#  Copyright 2026 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# Runs the Java integration suites against an ALREADY-RUNNING external OpenMetadata cluster
+# instead of a containerized server, across three jobs:
+#   - ui-it-external      : *UIIT.java (Playwright) via the `ui-it` profile
+#   - search-it-external  : tests/search/*IT.java via the `search-it` profile
+#   - scale-it-external   : tests/search/scale/*IT.java via `scale-it`, matrixed over cohort
+#                           sizes (scaleSeeds, default [100000]).
+#
+# All three jobs share ONE cluster, so each starts with a "Purge orphaned test data" step
+# (purge-it profile) to clear any bulk cohort a cancelled prior run left behind. Scale runs in
+# jpw.data.mode=ingest: it creates AND cleans up its own 100k (TestNamespaceExtension), so the
+# cluster stays clean for ui-it/search-it — whose own reindexes then only ever process their small
+# per-test fixtures. (Persistent static-100k reuse is for a SEPARATE scale-only cluster, not here.)
+#
+# The harness switches to external mode automatically when OM_URL + OM_ADMIN_TOKEN are set (see
+# UiTestServer.get / ExternalServer.fromEnv / OssTestServer.defaultHandle): no Docker image is
+# built and no testcontainers (MySQL/ES/OS) are started. Both the SDK clients and the Playwright
+# browser authenticate with the JWT acquired below.
+#
+# Search engine access: the search/scale ITs would normally talk to OpenSearch directly on :9200,
+# which a remote cluster doesn't expose. In external mode SearchClient routes index introspection
+# through the server's admin-only, typed /v1/test-support/search endpoints (TestSupportSearchResource,
+# auto-registered via @Collection). The deployed image on the target cluster MUST therefore (a) be
+# built from this branch or later, AND (b) set OM_TEST_SUPPORT_SEARCH_ENABLED=true in the SERVER's
+# environment — the resource is disabled by default so it never ships enabled in production, and
+# without the flag every external search assertion fails. The flag belongs on the cluster
+# deployment, not on this CI runner.
+#
+# Auth: the target cluster uses basic auth, so we exchange username/password for a JWT via
+# POST /api/v1/users/login (password base64-encoded) and feed the accessToken as OM_ADMIN_TOKEN.
+#
+# Excluded / skipped:
+#   - @Tag("sso-bootstrap")  — spin up their own OM lifecycle; pointless against external.
+#   - @Tag("search-direct")  — SearchAvailable*/DistributedAutoTune*UIIT read the engine directly;
+#     not yet routed through the passthrough (Phase 2 follow-on).
+#   - LiveIndexRetryIT + ReindexStatsIT#orphanedSchemaDoesNotFailReindex self-skip in external
+#     mode (they need the embedded container / in-JVM DAO).
+#   - GoogleSsoSignInUIIT self-skips under non-SSO backends.
+
+name: Java Integration Tests (External Cluster)
+
+on:
+  schedule:
+    # Run daily at midnight (00:00 UTC).
+    - cron: '0 0 * * *'
+  workflow_dispatch:
+    inputs:
+      runUiIt:
+        description: 'Run the ui-it (Playwright) job'
+        required: false
+        type: boolean
+        default: true
+      runSearchIt:
+        description: 'Run the search-it job'
+        required: false
+        type: boolean
+        default: true
+      runScaleIt:
+        description: 'Run the scale-it matrix job'
+        required: false
+        type: boolean
+        default: true
+      itTest:
+        description: 'Optional single class/glob to run for ui-it (failsafe -Dit.test). Empty = full suite.'
+        required: false
+        type: string
+        default: ''
+      scaleSeeds:
+        description: 'JSON array of scale table cohort sizes (matrix). E.g. [100000] or [100000, 500000].'
+        required: false
+        type: string
+        default: '[100000]'
+      scaleTimeoutMin:
+        description: 'Per-run reindex timeout (minutes) for scale tests'
+        required: false
+        type: string
+        default: '300'
+      reindexTimeoutMin:
+        description: 'Per-run reindex completion/acceptance timeout (minutes) for ui-it & search-it'
+        required: false
+        type: string
+        default: '60'
+      entityLoaderMaxWorkers:
+        description: 'Cap on EntityLoader create concurrency (keep low for a shared/proxied cluster to avoid 504s)'
+        required: false
+        type: string
+        default: '8'
+
+permissions:
+  contents: read
+  checks: write
+
+jobs:
+  ui-it-external:
+    if: ${{ github.event.inputs.runUiIt != 'false' }}
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build dependencies for integration-tests
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests install -pl :openmetadata-integration-tests -am
+
+      - name: Install Playwright browsers
+        run: |
+          mvn -pl :openmetadata-integration-tests dependency:build-classpath -Dmdep.outputFile=/tmp/cp.txt -q
+          java -cp "$(cat /tmp/cp.txt)" com.microsoft.playwright.CLI install --with-deps chromium
+
+      - name: Acquire admin JWT from external cluster
+        env:
+          OM_EXTERNAL_URL: ${{ secrets.OM_EXTERNAL_URL }}
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+        run: |
+          if [ -z "$OM_EXTERNAL_URL" ] || [ -z "$OM_EXTERNAL_USERNAME" ] || [ -z "$OM_EXTERNAL_PASSWORD" ]; then
+            echo "Missing OM_EXTERNAL_URL / OM_EXTERNAL_USERNAME / OM_EXTERNAL_PASSWORD secrets"
+            exit 1
+          fi
+          # OM basic-auth login expects a base64-encoded password (BasicAuthServletHandler).
+          PW_B64=$(printf '%s' "$OM_EXTERNAL_PASSWORD" | base64 | tr -d '\n')
+          TOKEN=$(curl -fsS -X POST "${OM_EXTERNAL_URL%/}/api/v1/users/login" \
+            -H 'Content-Type: application/json' \
+            -d "{\"email\":\"${OM_EXTERNAL_USERNAME}\",\"password\":\"${PW_B64}\"}" \
+            | jq -r '.accessToken')
+          if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
+            echo "Login to ${OM_EXTERNAL_URL} failed — no accessToken returned"
+            exit 1
+          fi
+          echo "::add-mask::$TOKEN"
+          echo "OM_ADMIN_TOKEN=$TOKEN" >> "$GITHUB_ENV"
+          echo "OM_URL=${OM_EXTERNAL_URL%/}" >> "$GITHUB_ENV"
+
+      - name: Purge orphaned test data (clean cluster before run)
+        # Functional suites must start on a cluster free of leftover bulk cohorts — a cancelled prior
+        # run skips per-test cleanup, and a stale 100k makes every reindex reprocess it (~20m each).
+        # purge-it hard-deletes every test-namespace root (matched by the __<run-id>__ signature), so
+        # reindexes here only ever process this run's own small fixtures.
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          OM_URL: ${{ env.OM_URL }}
+          OM_ADMIN_TOKEN: ${{ env.OM_ADMIN_TOKEN }}
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+        run: |
+          mvn verify -P purge-it -pl :openmetadata-integration-tests -Dskip.embedded.bootstrap=true
+
+      - name: Run UI integration tests (external cluster)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          OM_URL: ${{ env.OM_URL }}
+          OM_ADMIN_TOKEN: ${{ env.OM_ADMIN_TOKEN }}
+          # Credentials so the harness can re-login and auto-renew the token on long runs.
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+          PW_VIDEO: 'true'
+        run: |
+          STRESS_PROPS="-Djpw.simpleReindex.tables=${{ github.event.inputs.simpleReindexTables || '5000' }} \
+            -Djpw.simpleReindex.topics=${{ github.event.inputs.simpleReindexTopics || '1500' }} \
+            -Djpw.simpleReindex.dashboards=${{ github.event.inputs.simpleReindexDashboards || '1500' }} \
+            -Djpw.simpleReindex.pipelines=${{ github.event.inputs.simpleReindexPipelines || '2000' }} \
+            -Djpw.searchAvailable.tables=${{ github.event.inputs.searchAvailableTables || '5000' }}"
+          IT_TEST_PROP=""
+          if [ -n "${{ github.event.inputs.itTest }}" ]; then
+            IT_TEST_PROP="-Dit.test=${{ github.event.inputs.itTest }}"
+          fi
+          mvn verify -P ui-it -pl :openmetadata-integration-tests $STRESS_PROPS $IT_TEST_PROP \
+            -Djpw.loader.maxWorkers=${{ github.event.inputs.entityLoaderMaxWorkers || '8' }} \
+            -Djpw.reindex.timeoutMin=${{ github.event.inputs.reindexTimeoutMin || '60' }} \
+            -Dui.it.excludedGroups="sso-bootstrap,search-direct"
+
+      - name: Upload Playwright traces
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: playwright-traces-external-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/playwright-traces
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Upload Playwright videos
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: playwright-videos-external-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/playwright-videos
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Upload Failsafe Reports
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: failsafe-reports-ui-it-external-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/failsafe-reports
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Publish Test Report
+        if: ${{ always() }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: false
+          report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'
+
+      - name: Slack notification
+        if: ${{ always() && env.SLACK_JAVA_PLAYWRIGHT_URL != '' }}
+        uses: slackapi/slack-github-action@v2.0.0
+        with:
+          webhook: ${{ env.SLACK_JAVA_PLAYWRIGHT_URL }}
+          webhook-type: incoming-webhook
+          payload: |
+            text: "${{ job.status == 'success' && ':white_check_mark:' || ':fire:' }} Java UIIT External: ${{ job.status }}\nRef: ${{ github.ref_name }}\nLogs: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+        env:
+          SLACK_JAVA_PLAYWRIGHT_URL: ${{ secrets.SLACK_JAVA_PLAYWRIGHT_URL }}
+
+  search-it-external:
+    # These suites trigger the server-global SearchIndexingApplication, which is a singleton per
+    # instance (one run at a time). Since all external jobs hit the SAME cluster, they must run
+    # serially or they collide with "Job is already running". `needs` enforces ordering; `always()`
+    # keeps each job's own toggle independent (so it still runs if ui-it was skipped/failed).
+    needs: [ui-it-external]
+    if: ${{ always() && github.event.inputs.runSearchIt != 'false' }}
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build dependencies for integration-tests
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests install -pl :openmetadata-integration-tests -am
+
+      - name: Acquire admin JWT from external cluster
+        env:
+          OM_EXTERNAL_URL: ${{ secrets.OM_EXTERNAL_URL }}
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+        run: |
+          if [ -z "$OM_EXTERNAL_URL" ] || [ -z "$OM_EXTERNAL_USERNAME" ] || [ -z "$OM_EXTERNAL_PASSWORD" ]; then
+            echo "Missing OM_EXTERNAL_URL / OM_EXTERNAL_USERNAME / OM_EXTERNAL_PASSWORD secrets"
+            exit 1
+          fi
+          PW_B64=$(printf '%s' "$OM_EXTERNAL_PASSWORD" | base64 | tr -d '\n')
+          TOKEN=$(curl -fsS -X POST "${OM_EXTERNAL_URL%/}/api/v1/users/login" \
+            -H 'Content-Type: application/json' \
+            -d "{\"email\":\"${OM_EXTERNAL_USERNAME}\",\"password\":\"${PW_B64}\"}" \
+            | jq -r '.accessToken')
+          if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
+            echo "Login to ${OM_EXTERNAL_URL} failed — no accessToken returned"
+            exit 1
+          fi
+          echo "::add-mask::$TOKEN"
+          echo "OM_ADMIN_TOKEN=$TOKEN" >> "$GITHUB_ENV"
+          echo "OM_URL=${OM_EXTERNAL_URL%/}" >> "$GITHUB_ENV"
+
+      - name: Purge orphaned test data (clean cluster before run)
+        # search-it does ~14 full reindexes; on a cluster carrying a stale 100k each is ~20m and the
+        # job blows its cap. Purge every test-namespace root first so reindexes only process this
+        # run's own small per-test fixtures.
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          OM_URL: ${{ env.OM_URL }}
+          OM_ADMIN_TOKEN: ${{ env.OM_ADMIN_TOKEN }}
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+        run: |
+          mvn verify -P purge-it -pl :openmetadata-integration-tests -Dskip.embedded.bootstrap=true
+
+      - name: Run search integration tests (external cluster)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          OM_URL: ${{ env.OM_URL }}
+          OM_ADMIN_TOKEN: ${{ env.OM_ADMIN_TOKEN }}
+          # Credentials so the harness can re-login and auto-renew the token on long runs.
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+        run: |
+          mvn verify -P search-it -pl :openmetadata-integration-tests -Dskip.embedded.bootstrap=true \
+            -Djpw.loader.maxWorkers=${{ github.event.inputs.entityLoaderMaxWorkers || '8' }} \
+            -Djpw.reindex.timeoutMin=${{ github.event.inputs.reindexTimeoutMin || '60' }}
+
+      - name: Upload Failsafe Reports
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: failsafe-reports-search-it-external-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/failsafe-reports
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Publish Test Report
+        if: ${{ always() }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: false
+          report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'
+
+      - name: Slack notification
+        if: ${{ always() && env.SLACK_JAVA_PLAYWRIGHT_URL != '' }}
+        uses: slackapi/slack-github-action@v2.0.0
+        with:
+          webhook: ${{ env.SLACK_JAVA_PLAYWRIGHT_URL }}
+          webhook-type: incoming-webhook
+          payload: |
+            text: "${{ job.status == 'success' && ':white_check_mark:' || ':fire:' }} Java Search ITs External: ${{ job.status }}\nRef: ${{ github.ref_name }}\nLogs: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+        env:
+          SLACK_JAVA_PLAYWRIGHT_URL: ${{ secrets.SLACK_JAVA_PLAYWRIGHT_URL }}
+
+  scale-it-external:
+    # Runs after search-it (shared instance, singleton reindex app). max-parallel: 1 so the matrix
+    # entries (e.g. 100k then 500k) seed + reindex one at a time — two concurrent reindexes on one
+    # cluster collide and the db/es counts would mix cohorts.
+    needs: [search-it-external]
+    if: ${{ always() && github.event.inputs.runScaleIt != 'false' }}
+    runs-on: ubuntu-latest
+    timeout-minutes: 360
+    strategy:
+      fail-fast: false
+      max-parallel: 1
+      matrix:
+        seed: ${{ fromJSON(github.event.inputs.scaleSeeds || '[100000, 500000]') }}
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build dependencies for integration-tests
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests install -pl :openmetadata-integration-tests -am
+
+      - name: Acquire admin JWT from external cluster
+        env:
+          OM_EXTERNAL_URL: ${{ secrets.OM_EXTERNAL_URL }}
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+        run: |
+          if [ -z "$OM_EXTERNAL_URL" ] || [ -z "$OM_EXTERNAL_USERNAME" ] || [ -z "$OM_EXTERNAL_PASSWORD" ]; then
+            echo "Missing OM_EXTERNAL_URL / OM_EXTERNAL_USERNAME / OM_EXTERNAL_PASSWORD secrets"
+            exit 1
+          fi
+          PW_B64=$(printf '%s' "$OM_EXTERNAL_PASSWORD" | base64 | tr -d '\n')
+          TOKEN=$(curl -fsS -X POST "${OM_EXTERNAL_URL%/}/api/v1/users/login" \
+            -H 'Content-Type: application/json' \
+            -d "{\"email\":\"${OM_EXTERNAL_USERNAME}\",\"password\":\"${PW_B64}\"}" \
+            | jq -r '.accessToken')
+          if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
+            echo "Login to ${OM_EXTERNAL_URL} failed — no accessToken returned"
+            exit 1
+          fi
+          echo "::add-mask::$TOKEN"
+          echo "OM_ADMIN_TOKEN=$TOKEN" >> "$GITHUB_ENV"
+          echo "OM_URL=${OM_EXTERNAL_URL%/}" >> "$GITHUB_ENV"
+
+      - name: Purge orphaned test data (clean cluster before run)
+        # Clear any bulk cohort a cancelled prior run left behind. The scale test (ingest mode)
+        # creates and then cleans up its own 100k, so the cluster returns to clean; this purge just
+        # guarantees a clean starting point.
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          OM_URL: ${{ env.OM_URL }}
+          OM_ADMIN_TOKEN: ${{ env.OM_ADMIN_TOKEN }}
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+        run: |
+          mvn verify -P purge-it -pl :openmetadata-integration-tests -Dskip.embedded.bootstrap=true
+
+      - name: Run scale test (${{ matrix.seed }} tables, external cluster)
+        # Self-contained: ingest mode creates the cohort and cleans it up afterwards
+        # (TestNamespaceExtension), so the cluster returns to clean for the next run.
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          OM_URL: ${{ env.OM_URL }}
+          OM_ADMIN_TOKEN: ${{ env.OM_ADMIN_TOKEN }}
+          # Credentials so the harness can re-login and auto-renew the token on long runs.
+          OM_EXTERNAL_USERNAME: ${{ secrets.OM_EXTERNAL_USERNAME }}
+          OM_EXTERNAL_PASSWORD: ${{ secrets.OM_EXTERNAL_PASSWORD }}
+        run: |
+          mvn verify -P scale-it -pl :openmetadata-integration-tests -Dskip.embedded.bootstrap=true \
+            -Djpw.scale.tables=${{ matrix.seed }} \
+            -Djpw.scale.timeoutMin=${{ github.event.inputs.scaleTimeoutMin || '300' }} \
+            -Djpw.data.mode=ingest \
+            -Djpw.loader.maxWorkers=${{ github.event.inputs.entityLoaderMaxWorkers || '8' }}
+
+      - name: Upload scale metrics
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: scale-metrics-${{ matrix.seed }}-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/benchmark
+          if-no-files-found: ignore
+          retention-days: 30
+
+      - name: Upload Failsafe Reports
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: failsafe-reports-scale-${{ matrix.seed }}-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/failsafe-reports
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Publish Test Report
+        if: ${{ always() }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: false
+          report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'
+
+      - name: Slack notification
+        if: ${{ always() && env.SLACK_JAVA_PLAYWRIGHT_URL != '' }}
+        uses: slackapi/slack-github-action@v2.0.0
+        with:
+          webhook: ${{ env.SLACK_JAVA_PLAYWRIGHT_URL }}
+          webhook-type: incoming-webhook
+          payload: |
+            text: "${{ job.status == 'success' && ':white_check_mark:' || ':fire:' }} Java Scale IT External (${{ matrix.seed }} tables): ${{ job.status }}\nRef: ${{ github.ref_name }}\nLogs: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+        env:
+          SLACK_JAVA_PLAYWRIGHT_URL: ${{ secrets.SLACK_JAVA_PLAYWRIGHT_URL }}

.github/workflows/java-playwright-nightly.yml

@@ -0,0 +1,455 @@
+#  Copyright 2026 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# Full run of the UI integration suite (*UIIT.java) — lives inside
+# openmetadata-integration-tests under the `ui-it` Maven profile. Runs the
+# external-mode matrix (ES + OS). Tracks EPIC #3731 / tickets #3767, #3792.
+#
+# The schedule trigger is intentionally disabled while the suite stabilises. Run it
+# on demand via workflow_dispatch (pick the branch to test as the ref); it also runs
+# automatically on any PR that touches search-indexing code (see the pull_request
+# `paths` filter below), exercising this whole suite (both engines + search-it) against
+# the PR head in embedded mode. Re-add `schedule: - cron: '0 2 * * *'` once green on main.
+#
+# Topology:
+#   - build-image: builds the openmetadata-dist tarball and bakes the local
+#     openmetadata-server:jpw-snapshot image (BuildKit + GHA layer cache), saves it
+#     to a workflow artifact, and primes the Maven cache for downstream jobs.
+#   - ui-it-nightly (x2): matrix over [opensearch, elasticsearch]. Both download the
+#     pre-built image artifact, restore the Maven cache, then run `mvn verify -P ui-it`
+#     with OM_TEST_IMAGE pointing at the pre-loaded tag so ContainerizedServer skips
+#     its own build.
+#   - search-it-nightly: server-global destructive search ITs (tests/search/*IT.java) via
+#     `mvn verify -P search-it`. They run on the embedded bootstrap (testcontainers), not the
+#     baked image, so this job is independent of build-image and runs them serially.
+
+name: JavaUIIT Integration Tests (Nightly)
+
+on:
+  workflow_dispatch:
+    inputs:
+      includeSsoBootstrap:
+        description: 'Also run @Tag("sso-bootstrap") UIITs (slower; second OM lifecycle)'
+        required: false
+        type: boolean
+        default: false
+      simpleReindexTables:
+        description: 'SimpleReindexTriggerUIIT table cohort size'
+        required: false
+        type: string
+        default: '5000'
+      simpleReindexTopics:
+        description: 'SimpleReindexTriggerUIIT topic cohort size'
+        required: false
+        type: string
+        default: '1500'
+      simpleReindexDashboards:
+        description: 'SimpleReindexTriggerUIIT dashboard cohort size'
+        required: false
+        type: string
+        default: '1500'
+      simpleReindexPipelines:
+        description: 'SimpleReindexTriggerUIIT pipeline cohort size'
+        required: false
+        type: string
+        default: '2000'
+      searchAvailableTables:
+        description: 'SearchAvailableDuringReindexUIIT table cohort size'
+        required: false
+        type: string
+        default: '5000'
+      reindexTimeoutMin:
+        description: 'Per-run reindex completion/acceptance timeout (minutes) for ui-it & search-it'
+        required: false
+        type: string
+        default: '60'
+  # Auto-run on PRs that modify search-indexing code (paths below): the full suite runs against
+  # the PR head in embedded mode (testcontainers), NOT the external cluster — so a change to the
+  # reindex app, search clients, or index mappings is validated before merge without anyone
+  # remembering a label. Uses `pull_request` (NOT pull_request_target) on purpose: fork PRs run
+  # WITHOUT secrets, so untrusted code never executes with credentials; same-repo branch PRs (the
+  # common case here) still get secrets and full check reporting.
+  pull_request:
+    paths:
+      # --- Production search-indexing code (openmetadata-service) ---
+      - "openmetadata-service/src/main/java/org/openmetadata/service/search/**"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles/searchIndex/**"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/resources/searchindex/**"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/resources/search/**"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/resources/testsupport/**"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/events/lifecycle/handlers/SearchIndexHandler.java"
+      # --- Index mappings + loader (canonical source lives in the spec module) ---
+      - "openmetadata-spec/src/main/resources/elasticsearch/**"
+      - "openmetadata-spec/src/main/java/org/openmetadata/search/**"
+      # --- The search ITs / UIITs, their test harness, and the Maven profiles that run them ---
+      - "openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/search/**"
+      - "openmetadata-integration-tests/src/test/java/org/openmetadata/it/search/**"
+      - "openmetadata-integration-tests/src/test/java/org/openmetadata/playwright/scenarios/search/**"
+      - "openmetadata-integration-tests/pom.xml"
+      # --- This workflow itself ---
+      - ".github/workflows/java-playwright-nightly.yml"
+
+permissions:
+  contents: read
+  checks: write
+
+concurrency:
+  # One run per PR (or per ref for dispatch/cron); a new push cancels the prior in-flight run.
+  group: java-playwright-nightly-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+env:
+  SERVER_IMAGE_TAG: openmetadata-server:jpw-snapshot
+
+jobs:
+  build-image:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    outputs:
+      ref-sha: ${{ steps.checkout.outputs.commit }}
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Checkout
+        id: checkout
+        uses: actions/checkout@v4
+        with:
+          # On a PR, test the PR head (the changed code). workflow_dispatch honours the
+          # dispatched ref so feature branches can validate the nightly matrix before merge.
+          # Cron/anything else runs against main (EPIC #3731 / PR #28008).
+          ref: ${{ github.event.pull_request.head.sha || (github.event_name == 'workflow_dispatch' && github.ref || 'main') }}
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build dependencies for integration-tests
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests clean install -pl :openmetadata-integration-tests -am
+
+      - name: Build openmetadata-dist tarball
+        # ContainerizedServer.buildLocalImageContainer needs
+        # openmetadata-dist/target/openmetadata-*.tar.gz to bake the local image.
+        # Dist is NOT a transitive dep of integration-tests, so the previous step
+        # doesn't produce it — build it explicitly here.
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests install -pl :openmetadata-dist -am
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Build openmetadata-server image (with GHA layer cache)
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          file: docker/development/Dockerfile
+          tags: ${{ env.SERVER_IMAGE_TAG }}
+          load: true
+          # Shared scope across matrix entries so the cache primed by this build
+          # is reused by everything that runs this workflow on the same SHA (or
+          # later runs whose Dockerfile + dist tarball haven't changed).
+          cache-from: type=gha,scope=jpw-server-image
+          cache-to: type=gha,scope=jpw-server-image,mode=max
+
+      - name: Export server image as artifact
+        run: |
+          docker save "${SERVER_IMAGE_TAG}" -o /tmp/jpw-server-image.tar
+          ls -lh /tmp/jpw-server-image.tar
+
+      - name: Upload server image artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: jpw-server-image-${{ github.run_id }}
+          path: /tmp/jpw-server-image.tar
+          retention-days: 1
+
+  ui-it-nightly:
+    needs: build-image
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    strategy:
+      fail-fast: false
+      matrix:
+        searchEngine: [opensearch, elasticsearch]
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha || (github.event_name == 'workflow_dispatch' && github.ref || 'main') }}
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Add /etc/hosts entry for mock OIDC server
+        # The SSO test infrastructure (MockOidcServer) needs `om-mock-idp` to resolve to
+        # loopback on the host so the same URL works inside the Docker network and from
+        # the host-side Playwright browser, keeping the issued tokens' `iss` claim
+        # consistent across actors.
+        run: echo "127.0.0.1 om-mock-idp" | sudo tee -a /etc/hosts
+
+      - name: Build dependencies for integration-tests
+        # Maven cache primed by build-image makes this a fast restore-only invocation.
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests install -pl :openmetadata-integration-tests -am
+
+      - name: Download server image artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: jpw-server-image-${{ github.run_id }}
+          path: /tmp
+
+      - name: Load server image into local Docker
+        run: |
+          docker load -i /tmp/jpw-server-image.tar
+          docker image ls "${SERVER_IMAGE_TAG}"
+
+      - name: Install Playwright browsers
+        run: |
+          mvn -pl :openmetadata-integration-tests dependency:build-classpath -Dmdep.outputFile=/tmp/cp.txt -q
+          java -cp "$(cat /tmp/cp.txt)" com.microsoft.playwright.CLI install --with-deps chromium
+
+      - name: Free build artifacts
+        run: |
+          rm -rf openmetadata-service/target/lib openmetadata-service/target/classes
+          rm -rf openmetadata-spec/target openmetadata-sdk/target common/target
+          rm -rf openmetadata-shaded-deps/*/target
+          df -h /
+
+      - name: Run UI integration tests (${{ matrix.searchEngine }})
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          # OM_TEST_IMAGE makes ContainerizedServer skip its in-test `docker build`
+          # and reuse the image artifact loaded above. Cuts ~10-90s per launch.
+          OM_TEST_IMAGE: ${{ env.SERVER_IMAGE_TAG }}
+          # UiSessionExtension reads PW_VIDEO and records every test's BrowserContext
+          # to target/playwright-videos when true. Kept off locally; on in CI for triage.
+          PW_VIDEO: 'true'
+        run: |
+          # Nightly bumps the per-test cohort sizes up to the historical stress numbers.
+          # PR runs use the smaller defaults baked into each test so `mvn verify` is fast.
+          # Sizes are workflow_dispatch inputs (defaults below) so they're tunable per run.
+          # Embedded nightly recreates a fresh stack each run, so data never persists — these
+          # small-footprint suites always create on the fly (jpw.data.mode=ingest). Static/ensure
+          # reuse is only for the persistent external cluster (see java-playwright-external.yml).
+          STRESS_PROPS="-Djpw.simpleReindex.tables=${{ github.event.inputs.simpleReindexTables || '5000' }} \
+            -Djpw.simpleReindex.topics=${{ github.event.inputs.simpleReindexTopics || '1500' }} \
+            -Djpw.simpleReindex.dashboards=${{ github.event.inputs.simpleReindexDashboards || '1500' }} \
+            -Djpw.simpleReindex.pipelines=${{ github.event.inputs.simpleReindexPipelines || '2000' }} \
+            -Djpw.searchAvailable.tables=${{ github.event.inputs.searchAvailableTables || '5000' }} \
+            -Djpw.data.mode=ingest \
+            -Djpw.reindex.timeoutMin=${{ github.event.inputs.reindexTimeoutMin || '60' }}"
+          # Opt-in for SSO bootstrap UIITs (second OM lifecycle); default off.
+          if [ "${{ github.event.inputs.includeSsoBootstrap }}" = "true" ]; then
+            SSO_PROPS="-Dui.it.excludedGroups="
+          else
+            SSO_PROPS=""
+          fi
+          if [ "${{ matrix.searchEngine }}" = "elasticsearch" ]; then
+            mvn verify -P ui-it -pl :openmetadata-integration-tests $STRESS_PROPS $SSO_PROPS \
+              -DsearchType=elasticsearch \
+              -DsearchImage=docker.elastic.co/elasticsearch/elasticsearch:9.3.0
+          else
+            mvn verify -P ui-it -pl :openmetadata-integration-tests $STRESS_PROPS $SSO_PROPS
+          fi
+
+      - name: Upload Playwright traces
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: playwright-traces-${{ matrix.searchEngine }}-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/playwright-traces
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Upload Playwright videos
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: playwright-videos-${{ matrix.searchEngine }}-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/playwright-videos
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Upload Failsafe Reports
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: failsafe-reports-${{ matrix.searchEngine }}-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/failsafe-reports
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Publish Test Report
+        id: report
+        if: ${{ always() }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          # mvn verify already fails the job on test failures. This step just
+          # publishes the check run with per-test details; it shouldn't itself
+          # gate the job conclusion (otherwise a flake here breaks Slack).
+          fail_on_test_failures: false
+          report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'
+
+      - name: Slack notification
+        if: ${{ always() && github.event_name != 'pull_request' && env.SLACK_JAVA_PLAYWRIGHT_URL != '' }}
+        uses: slackapi/slack-github-action@v2.0.0
+        with:
+          webhook: ${{ env.SLACK_JAVA_PLAYWRIGHT_URL }}
+          webhook-type: incoming-webhook
+          payload: |
+            text: "${{ job.status == 'success' && ':white_check_mark:' || ':fire:' }} Java Playwright Nightly (${{ matrix.searchEngine }}): ${{ job.status }}\nRef: ${{ github.ref_name }}\nLogs: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+        env:
+          SLACK_JAVA_PLAYWRIGHT_URL: ${{ secrets.SLACK_JAVA_PLAYWRIGHT_URL }}
+
+  search-it-nightly:
+    # Server-global destructive search ITs (tests/search/*IT.java): full reindex with
+    # recreateIndex, alias swaps, and ES container pause. They mutate cluster-wide state, so
+    # the default PR profiles exclude them; here the `search-it` Maven profile runs them
+    # serially on their own embedded-bootstrap server (testcontainers). That bootstrap is
+    # independent of the baked image, so this job does not need build-image.
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: true
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: true
+          docker-images: false
+          swap-storage: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha || (github.event_name == 'workflow_dispatch' && github.ref || 'main') }}
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build dependencies for integration-tests
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -DskipTests install -pl :openmetadata-integration-tests -am
+
+      - name: Run isolated search ITs (postgres + opensearch)
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        # Embedded bootstrap = fresh stack each run; these search ITs create their cohorts on the
+        # fly (jpw.data.mode=ingest). Static/ensure reuse is for the external cluster only.
+        run: mvn verify -P search-it -pl :openmetadata-integration-tests -DdatabaseType=postgres -DsearchType=opensearch -Djpw.data.mode=ingest -Djpw.reindex.timeoutMin=${{ github.event.inputs.reindexTimeoutMin || '60' }}
+
+      - name: Upload Failsafe Reports
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: failsafe-reports-search-it-${{ github.run_id }}
+          path: openmetadata-integration-tests/target/failsafe-reports
+          if-no-files-found: ignore
+          retention-days: 14
+
+      - name: Publish Test Report
+        if: ${{ always() }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: false
+          report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'
+
+      - name: Slack notification
+        if: ${{ always() && github.event_name != 'pull_request' && env.SLACK_JAVA_PLAYWRIGHT_URL != '' }}
+        uses: slackapi/slack-github-action@v2.0.0
+        with:
+          webhook: ${{ env.SLACK_JAVA_PLAYWRIGHT_URL }}
+          webhook-type: incoming-webhook
+          payload: |
+            text: "${{ job.status == 'success' && ':white_check_mark:' || ':fire:' }} Java Search ITs Nightly: ${{ job.status }}\nRef: ${{ github.ref_name }}\nLogs: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+        env:
+          SLACK_JAVA_PLAYWRIGHT_URL: ${{ secrets.SLACK_JAVA_PLAYWRIGHT_URL }}

.github/workflows/label-connector.yml

@@ -0,0 +1,25 @@
+name: Label connector bug
+
+on:
+  issues:
+    types: [opened, edited]
+
+permissions:
+  issues: write
+  contents: read
+
+jobs:
+  label:
+    if: contains(github.event.issue.labels.*.name, 'Ingestion')
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          sparse-checkout: .github/scripts
+          sparse-checkout-cone-mode: false
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+      - run: python .github/scripts/label_connector.py
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/maven-build-collate.yml

@@ -0,0 +1,93 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Maven Collate Tests
+
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+    paths:
+      - "openmetadata-service/**"
+      - "openmetadata-ui/**"
+      - "!openmetadata-ui/src/main/resources/ui/playwright/doc-generator/**"
+      - "!openmetadata-ui/src/main/resources/ui/playwright/docs/**"
+      - "openmetadata-spec/src/main/resources/json/schema/**"
+      - "openmetadata-dist/**"
+      - "openmetadata-clients/**"
+      - "openmetadata-sdk/**"
+      - "openmetadata-integration-tests/**"
+      - "openmetadata-mcp/**"
+      - "common/**"
+      - "pom.xml"
+      - "yarn.lock"
+      - "Makefile"
+      - "bootstrap/**"
+  pull_request_target:
+    types: [opened, synchronize, labeled, ready_for_review]
+    paths:
+      - "openmetadata-service/**"
+      - "openmetadata-ui/src/main/resources/ui/src/**"
+      - "!openmetadata-ui/src/main/resources/ui/src/test/**"
+      - "!openmetadata-ui/src/main/resources/ui/src/**/*.test.ts"
+      - "!openmetadata-ui/src/main/resources/ui/src/**/*.test.tsx"
+      - "openmetadata-spec/src/main/resources/json/schema/**"
+      - "openmetadata-integration-tests/**"
+      - "openmetadata-mcp/**"
+
+permissions:
+  contents: read
+  checks: write
+
+concurrency:
+  group: maven-build-collate-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+jobs:
+  maven-collate-ci:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+
+      - name: Trigger Collate build & wait
+        uses: the-actions-org/workflow-dispatch@v4
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SHA: ${{ github.event_name == 'push' && github.event.after || github.event.pull_request.head.sha }}
+        with:
+          workflow: OpenMetadata Collate Test
+          ref: main
+          repo: open-metadata/openmetadata-collate
+          token: ${{ secrets.COLLATE_PAT }}
+          wait-for-completion: true
+          inputs: '{ "sha": "${{ env.SHA }}", "event": "${{ github.event_name }}" }'

.github/workflows/maven-sonar-build.yml

@@ -0,0 +1,92 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Maven SonarCloud CI
+
+on:
+  workflow_dispatch:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+    paths:
+      - "openmetadata-service/**"
+      - "openmetadata-spec/src/main/resources/json/schema/**"
+      - "openmetadata-dist/**"
+      - "openmetadata-clients/**"
+      - "openmetadata-sdk/**"
+      - "common/**"
+      - "pom.xml"
+      - "yarn.lock"
+      - "Makefile"
+      - "bootstrap/**"
+
+permissions:
+  contents: read
+  checks: write
+
+concurrency:
+  group: maven-sonar-build-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+jobs:
+  maven-sonarcloud-ci:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+
+      - name: Cache Maven dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Set up JDK 21
+        if: steps.cache-output.outputs.exit-code == 0
+        uses: actions/setup-java@v4
+        with:
+          java-version: "21"
+          distribution: "temurin"
+
+      - name: Install Ubuntu dependencies
+        if: steps.cache-output.outputs.exit-code == 0
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          librdkafka-dev unixodbc-dev libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Build with Maven
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: mvn -Pstatic-code-analysis clean verify -DskipTests -am

.github/workflows/monitor-slack-link.yml

@@ -0,0 +1,66 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: monitor-slack-link
+on:
+  schedule:
+    # Run every 2 hours
+    - cron: '0 */2 * * *'
+  workflow_dispatch:
+  
+
+# Grant least privilege permissions needed
+permissions:
+  contents: read # Needed for checkout and reading requirements.txt
+
+jobs:
+  monitor-slack-link:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false # Ensure all matrix jobs run even if one fails
+    # Only run specific matrix job when manually triggered with selection
+    steps:
+    # Step 1: Checkout repository code
+    - name: Checkout
+      uses: actions/checkout@v4
+
+    # Step 2: Set up Python environment with caching
+    - name: Set up Python 3.9 # Or consider a newer version like 3.11/3.12
+      uses: actions/setup-python@v5
+      with:
+        python-version: 3.9 # Or e.g., '3.11'
+        cache: 'pip'
+
+    # Step 3: Install dependencies
+    - name: Install Dependencies
+      run: |
+        python -m venv env
+        source env/bin/activate
+        pip install --upgrade pip playwright slack_sdk==3.35.0
+
+    # Step 4: Install Playwright browsers
+    - name: Install Playwright Browsers
+      run: |
+        source env/bin/activate
+        playwright install chromium --with-deps
+
+    # Step 5: Run the monitoring script
+    - name: Monitor Slack Link
+      id: monitor
+      env:
+        PYTHONUNBUFFERED: "1"
+        SLACK_WEBHOOK_URL: ${{ secrets.SLACK_MONITOR_SLACK_WEBHOOK }}
+        GITHUB_SERVER_URL: ${{ github.server_url }}
+        GITHUB_REPOSITORY: ${{ github.repository }}
+        GITHUB_RUN_ID: ${{ github.run_id }}
+      run: |
+        source env/bin/activate
+        python scripts/slack-link-monitor.py

.github/workflows/mysql-nightly-e2e.yml

@@ -0,0 +1,234 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# This workflow executes end-to-end (e2e) tests using Playwright with MySQL as the database.
+# For more information see: https://github.com/actions/setup-java/blob/main/docs/advanced-usage.md#apache-maven-with-a-settings-path
+
+name: MySQL Playwright E2E Tests
+on:
+  workflow_dispatch:
+    inputs:
+      branch:
+        description: "Branch to run tests on"
+        required: true
+        type: string
+        default: "main"
+      send_slack_notification:
+        description: "Send Slack notification with test results"
+        required: false
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+
+concurrency:
+  group: pw-ci-mysql-branch-${{ inputs.branch }}
+  cancel-in-progress: true
+
+jobs:
+  pw-ci-mysql-branch:
+    runs-on: ubuntu-latest
+    environment: test
+    env:
+      # Playwright logs the admin user in many times (performAdminLogin per test, parallel
+      # workers, retries). The production default of 5 active sessions per user would evict the
+      # long-lived storageState session the page fixtures rely on and 401 every request. Raise
+      # the cap for E2E only; docker compose reads this for ${AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER:-5}.
+      AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER: "10000"
+    strategy:
+      fail-fast: false
+      matrix:
+        shardIndex: [1, 2, 3, 4, 5, 6, 7]
+        shardTotal: [7]
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.branch }}
+
+      - name: Cache Maven Dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d mysql"
+          ingestion_dependency: "playwright"
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+      - name: Install Playwright Browsers
+        run: npx playwright@1.57.0 install chromium --with-deps
+      - name: Run Playwright tests
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: |
+          if [ "${{ matrix.shardIndex }}" -eq "1" ]; then
+            echo "🔹 Running DataAssetRules-only tests on shard 1"
+
+            # The testMatch pattern ensures only DataAssetRules*.spec.ts files run
+            npx playwright test \
+              --project=setup \
+              --project=DataAssetRulesEnabled \
+              --project=DataAssetRulesDisabled
+
+          elif [ "${{ matrix.shardIndex }}" -eq "2" ]; then
+            echo "🔹 Running search relevance tests on shard 2"
+
+            npx playwright test \
+              --project=search-nightly \
+              playwright/e2e/Search/SearchRelevance.spec.ts \
+              --workers=1
+
+          else
+            # Shards 3-7 handle chromium tests (5 shards total)
+            CHROMIUM_SHARD=$(( ${{ matrix.shardIndex }} - 2 ))
+            echo "🔹 Running all tests (excluding DataAssetRules) on chromium shard ${CHROMIUM_SHARD}/5"
+            npx playwright test \
+              --project=chromium \
+              --grep-invert @dataAssetRules \
+              --shard=${CHROMIUM_SHARD}/5
+          fi
+
+        env:
+          PLAYWRIGHT_IS_OSS: true
+          PLAYWRIGHT_SNOWFLAKE_USERNAME: ${{ secrets.TEST_SNOWFLAKE_USERNAME }}
+          PLAYWRIGHT_SNOWFLAKE_PASSWORD: ${{ secrets.TEST_SNOWFLAKE_PASSWORD }}
+          PLAYWRIGHT_SNOWFLAKE_ACCOUNT: ${{ secrets.TEST_SNOWFLAKE_ACCOUNT }}
+          PLAYWRIGHT_SNOWFLAKE_DATABASE: ${{ secrets.TEST_SNOWFLAKE_DATABASE }}
+          PLAYWRIGHT_SNOWFLAKE_WAREHOUSE: ${{ secrets.TEST_SNOWFLAKE_WAREHOUSE }}
+          PLAYWRIGHT_PROJECT_ID: ${{ steps.cypress-project-id.outputs.CYPRESS_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY: ${{ secrets.TEST_BQ_PRIVATE_KEY }}
+          PLAYWRIGHT_BQ_PROJECT_ID: ${{ secrets.PLAYWRIGHT_BQ_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY_ID: ${{ secrets.TEST_BQ_PRIVATE_KEY_ID }}
+          PLAYWRIGHT_BQ_PROJECT_ID_TAXONOMY: ${{ secrets.TEST_BQ_PROJECT_ID_TAXONOMY }}
+          PLAYWRIGHT_BQ_CLIENT_EMAIL: ${{ secrets.TEST_BQ_CLIENT_EMAIL }}
+          PLAYWRIGHT_BQ_CLIENT_ID: ${{ secrets.TEST_BQ_CLIENT_ID }}
+          PLAYWRIGHT_REDSHIFT_HOST: ${{ secrets.E2E_REDSHIFT_HOST_PORT }}
+          PLAYWRIGHT_REDSHIFT_USERNAME: ${{ secrets.E2E_REDSHIFT_USERNAME }}
+          PLAYWRIGHT_REDSHIFT_PASSWORD: ${{ secrets.E2E_REDSHIFT_PASSWORD }}
+          PLAYWRIGHT_REDSHIFT_DATABASE: ${{ secrets.TEST_REDSHIFT_DATABASE }}
+          PLAYWRIGHT_METABASE_USERNAME: ${{ secrets.TEST_METABASE_USERNAME }}
+          PLAYWRIGHT_METABASE_PASSWORD: ${{ secrets.TEST_METABASE_PASSWORD }}
+          PLAYWRIGHT_METABASE_DB_SERVICE_NAME: ${{ secrets.TEST_METABASE_DB_SERVICE_NAME }}
+          PLAYWRIGHT_METABASE_HOST_PORT: ${{ secrets.TEST_METABASE_HOST_PORT }}
+          PLAYWRIGHT_SUPERSET_USERNAME: ${{ secrets.TEST_SUPERSET_USERNAME }}
+          PLAYWRIGHT_SUPERSET_PASSWORD: ${{ secrets.TEST_SUPERSET_PASSWORD }}
+          PLAYWRIGHT_SUPERSET_HOST_PORT: ${{ secrets.TEST_SUPERSET_HOST_PORT }}
+          PLAYWRIGHT_KAFKA_BOOTSTRAP_SERVERS: ${{ secrets.TEST_KAFKA_BOOTSTRAP_SERVERS }}
+          PLAYWRIGHT_KAFKA_SCHEMA_REGISTRY_URL: ${{ secrets.TEST_KAFKA_SCHEMA_REGISTRY_URL }}
+          PLAYWRIGHT_GLUE_ACCESS_KEY: ${{ secrets.TEST_GLUE_ACCESS_KEY }}
+          PLAYWRIGHT_GLUE_SECRET_KEY: ${{ secrets.TEST_GLUE_SECRET_KEY }}
+          PLAYWRIGHT_GLUE_AWS_REGION: ${{ secrets.TEST_GLUE_AWS_REGION }}
+          PLAYWRIGHT_GLUE_ENDPOINT: ${{ secrets.TEST_GLUE_ENDPOINT }}
+          PLAYWRIGHT_GLUE_STORAGE_SERVICE: ${{ secrets.TEST_GLUE_STORAGE_SERVICE }}
+          PLAYWRIGHT_MYSQL_USERNAME: ${{ secrets.TEST_MYSQL_USERNAME }}
+          PLAYWRIGHT_MYSQL_PASSWORD: ${{ secrets.TEST_MYSQL_PASSWORD }}
+          PLAYWRIGHT_MYSQL_HOST_PORT: ${{ secrets.TEST_MYSQL_HOST_PORT }}
+          PLAYWRIGHT_MYSQL_DATABASE_SCHEMA: ${{ secrets.TEST_MYSQL_DATABASE_SCHEMA }}
+          PLAYWRIGHT_POSTGRES_USERNAME: ${{ secrets.TEST_POSTGRES_USERNAME }}
+          PLAYWRIGHT_POSTGRES_PASSWORD: ${{ secrets.TEST_POSTGRES_PASSWORD }}
+          PLAYWRIGHT_POSTGRES_HOST_PORT: ${{ secrets.TEST_POSTGRES_HOST_PORT }}
+          PLAYWRIGHT_POSTGRES_DATABASE: ${{ secrets.TEST_POSTGRES_DATABASE }}
+          PLAYWRIGHT_AIRFLOW_HOST_PORT: ${{ secrets.TEST_AIRFLOW_HOST_PORT }}
+          PLAYWRIGHT_ML_MODEL_TRACKING_URI: ${{ secrets.TEST_ML_MODEL_TRACKING_URI }}
+          PLAYWRIGHT_ML_MODEL_REGISTRY_URI: ${{ secrets.TEST_ML_MODEL_REGISTRY_URI }}
+          PLAYWRIGHT_S3_STORAGE_ACCESS_KEY_ID: ${{ secrets.TEST_S3_STORAGE_ACCESS_KEY_ID }}
+          PLAYWRIGHT_S3_STORAGE_SECRET_ACCESS_KEY: ${{ secrets.TEST_S3_STORAGE_SECRET_ACCESS_KEY }}
+          PLAYWRIGHT_S3_STORAGE_END_POINT_URL: ${{ secrets.TEST_S3_STORAGE_END_POINT_URL }}
+
+          # Recommended: pass the GitHub token lets this action correctly
+          # determine the unique run id necessary to re-run the checks
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Upload blob report to GitHub Actions Artifacts
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: blob-report-${{ matrix.shardIndex }}
+          path: openmetadata-ui/src/main/resources/ui/blob-report
+          retention-days: 1
+
+      - name: Upload HTML report to GitHub Actions Artifacts
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: playwright-HTML-report-${{ matrix.shardIndex }}
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/playwright-report
+          retention-days: 1
+          compression-level: 9
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+  merge-reports:
+    needs: pw-ci-mysql-branch
+    runs-on: ubuntu-latest
+    if: ${{ !failure() || !cancelled() }}
+    env:
+      # Recommended: pass the GitHub token lets this action correctly
+      # determine the unique run id necessary to re-run the checks
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      SLACK_BOT_USER_OAUTH_TOKEN: ${{ secrets.E2E_SLACK_BOT_OAUTH_TOKEN }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.branch }}
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: "openmetadata-ui/src/main/resources/ui/.nvmrc"
+
+      - name: Download blob reports from GitHub Actions Artifacts
+        uses: actions/download-artifact@v4
+        with:
+          path: openmetadata-ui/src/main/resources/ui/blob-reports
+          pattern: blob-report-*
+          merge-multiple: true
+
+      - name: Merge Reports
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: |
+          npx playwright merge-reports --reporter json ./blob-reports > merged_tests_results.json
+
+      - name: Generate Slack Notification
+        if: ${{ inputs.send_slack_notification }}
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        env:
+          RUN_TITLE: "MySQL Test for OSS Release: ${{ inputs.branch }}"
+          RUN_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+          SLACK_BOT_USER_OAUTH_TOKEN: ${{ secrets.E2E_SLACK_BOT_OAUTH_TOKEN }}
+        run: |
+          npx playwright-slack-report -c playwright/slack-cli.config.json -j merged_tests_results.json > slack_report.json

.github/workflows/openmetadata-service-unit-tests.yml

@@ -0,0 +1,190 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: OpenMetadata Service Unit Tests
+
+on:
+  merge_group:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+  pull_request:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+
+permissions:
+  contents: read
+  checks: write
+  pull-requests: write
+
+concurrency:
+  group: openmetadata-service-unit-tests-${{ github.head_ref || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  # Detect whether relevant paths changed. When no Java service files
+  # are modified the downstream jobs are skipped via their `if` condition.
+  # A job skipped by `if` reports as "Success", so required checks still pass.
+  # This replaces the old openmetadata-service-unit-tests-skip.yml companion workflow.
+  changes:
+    name: Detect Changes
+    runs-on: ubuntu-latest
+    if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft) && (github.event_name != 'pull_request' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    outputs:
+      java: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.filter.outputs.java }}
+      k8s_operator: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.filter.outputs.k8s_operator }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        if: ${{ github.event_name != 'workflow_dispatch' }}
+
+      - uses: dorny/paths-filter@v3
+        id: filter
+        if: ${{ github.event_name != 'workflow_dispatch' }}
+        with:
+          filters: |
+            java:
+              - '.github/workflows/openmetadata-service-unit-tests.yml'
+              - 'openmetadata-service/**'
+              - 'openmetadata-spec/**'
+              - 'openmetadata-clients/**'
+              - 'openmetadata-sdk/**'
+              - 'common/**'
+              - 'pom.xml'
+              - 'Makefile'
+              - 'bootstrap/**'
+            k8s_operator:
+              - 'openmetadata-k8s-operator/**'
+
+  # The openmetadata-service unit tests are pure JVM tests with no database
+  # interaction (no testcontainers, no JDBC). The {mysql, postgresql} matrix used
+  # to run the suite twice with different `-Pmysql` / `-Ppostgresql` profiles, but
+  # those profiles are only defined in openmetadata-sdk/pom.xml and only affect
+  # failsafe (integration) tests that aren't enabled in this workflow. Result:
+  # both matrix jobs ran an identical surefire suite. DB-specific coverage
+  # belongs in `openmetadata-integration-tests`, not here.
+  openmetadata-service-unit-tests:
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    needs: changes
+    if: ${{ needs.changes.outputs.java == 'true' }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: "21"
+          distribution: "temurin"
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          librdkafka-dev unixodbc-dev libevent-dev jq
+          sudo make install_antlr_cli
+
+      - name: Run openmetadata-service unit tests
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          mvn -B clean package -pl openmetadata-service -am \
+            -Pstatic-code-analysis \
+            -DfailIfNoTests=false \
+            -Dsonar.skip=true
+
+      - name: Upload surefire reports
+        if: ${{ failure() && hashFiles('openmetadata-service/target/surefire-reports/TEST-*.xml') != '' }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: openmetadata-service-surefire-reports
+          path: openmetadata-service/target/surefire-reports/
+
+      - name: Publish Test Report
+        if: ${{ always() && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) && hashFiles('openmetadata-service/target/surefire-reports/TEST-*.xml') != '' }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: true
+          report_paths: "openmetadata-service/target/surefire-reports/TEST-*.xml"
+          check_name: "Test Report"
+
+  k8s_operator-unit-tests:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    needs: changes
+    if: ${{ needs.changes.outputs.k8s_operator == 'true' }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+
+      - name: Cache Maven dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: "21"
+          distribution: "temurin"
+
+      - name: Build k8s-operator dependencies
+        run: |
+          mvn -B clean install -pl openmetadata-k8s-operator -am -DskipTests
+
+      - name: Run k8s-operator unit tests
+        run: |
+          mvn -B test -pl openmetadata-k8s-operator
+
+      - name: Upload surefire reports
+        if: ${{ failure() && hashFiles('openmetadata-k8s-operator/target/surefire-reports/TEST-*.xml') != '' }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: k8s-operator-surefire-reports
+          path: openmetadata-k8s-operator/target/surefire-reports/
+
+      - name: Publish Test Report
+        if: ${{ always() && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) && hashFiles('openmetadata-k8s-operator/target/surefire-reports/TEST-*.xml') != '' }}
+        uses: scacap/action-surefire-report@v1
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          fail_on_test_failures: true
+          report_paths: "openmetadata-k8s-operator/target/surefire-reports/TEST-*.xml"
+          check_name: "K8s Operator Test Report"
+
+  # Single required-check gate for branch protection.
+  # Skipped (= "Success") when all test jobs pass or are legitimately skipped.
+  # Runs and exits 1 only when a test job fails or is cancelled.
+  # Set "OpenMetadata Service Unit Tests / openmetadata-service-unit-tests-status" as the sole required check for this workflow.
+  openmetadata-service-unit-tests-status:
+    name: openmetadata-service-unit-tests-status
+    needs: [changes, openmetadata-service-unit-tests, k8s_operator-unit-tests]
+    if: ${{ failure() || cancelled() }}
+    runs-on: ubuntu-latest
+    steps:
+      - run: exit 1

.github/workflows/playwright-docs-check.yml

@@ -0,0 +1,68 @@
+name: Verify Playwright Documentation
+
+on:
+  pull_request:
+    paths:
+      - 'openmetadata-ui/src/main/resources/ui/playwright/e2e/**/*.spec.ts'
+      - 'openmetadata-ui/src/main/resources/ui/playwright/doc-generator/**'
+    branches:
+      - main
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  verify-docs:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v3
+        with:
+          node-version-file: 'openmetadata-ui/src/main/resources/ui/.nvmrc'
+          cache: 'yarn'
+          cache-dependency-path: 'openmetadata-ui/src/main/resources/ui/yarn.lock'
+
+      - name: Install Dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: yarn install --frozen-lockfile --ignore-scripts
+
+      - name: Install Playwright Browsers
+        # We need browsers installed for 'playwright test --list' to work in some versions,
+        # although strictly speaking just listing shouldn't require binaries if configured right.
+        # Adding just in case.
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: npx playwright install chromium --with-deps
+
+      - name: Generate Documentation
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: node playwright/doc-generator/generate.js
+
+      - name: Check for Changes
+        id: git-check
+        run: |
+          git diff --exit-code --quiet openmetadata-ui/src/main/resources/ui/playwright/docs || echo "changes=true" >> $GITHUB_OUTPUT
+
+      - name: Commit and Push Changes
+        if: steps.git-check.outputs.changes == 'true'
+        run: |
+          git config --global user.name 'github-actions[bot]'
+          git config --global user.email 'github-actions[bot]@users.noreply.github.com'
+          git add openmetadata-ui/src/main/resources/ui/playwright/docs
+          git commit -m "docs: auto-generate playwright documentation"
+          git push
+
+      - name: Comment on PR
+        if: steps.git-check.outputs.changes == 'true'
+        uses: peter-evans/create-or-update-comment@v4
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          body: |
+            ## 📝 Documentation Auto-Updated
+            
+            The Playwright documentation has been automatically updated to match the changes in this PR.
+            
+            *   **Generated by**: `playwright-docs-check.yml`
+            *   **Status**: ✅ Updated and pushed to branch.

.github/workflows/playwright-integration-tests-mysql.yml

@@ -0,0 +1,112 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# This workflow will build a package using Maven and then publish it to GitHub packages when a release is created
+# For more information see: https://github.com/actions/setup-java/blob/main/docs/advanced-usage.md#apache-maven-with-a-settings-path
+
+name: MySQL Playwright Integration Tests
+on:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  playwright-mysql:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    strategy:
+      fail-fast: false
+      matrix:
+        browser-type: ["chromium"]
+    environment: test
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+      - name: Cache Maven Dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+      - name: Get yarn cache directory path
+        if: steps.cache-output.outputs.exit-code == 0
+        id: yarn-cache-dir-path
+        run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
+      - uses: actions/cache@v4
+        if: steps.yarn-cache-dir-path.outputs.exit-code == 0
+        id: yarn-cache
+        with:
+          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
+          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-yarn-
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d mysql"
+          ingestion_dependency: "playwright"
+      - name: Run Playwright Integration Tests with browser ${{ matrix.browser-type }}
+        env:
+          E2E_REDSHIFT_HOST_PORT: ${{ secrets.E2E_REDSHIFT_HOST_PORT }}
+          E2E_REDSHIFT_USERNAME: ${{ secrets.E2E_REDSHIFT_USERNAME }}
+          E2E_REDSHIFT_PASSWORD: ${{ secrets.E2E_REDSHIFT_PASSWORD }}
+          E2E_REDSHIFT_DB: ${{ secrets.TEST_REDSHIFT_DATABASE }}
+          E2E_DRUID_HOST_PORT: ${{ secrets.E2E_DRUID_HOST_PORT }}
+          E2E_HIVE_HOST_PORT: ${{ secrets.E2E_HIVE_HOST_PORT }}
+        run: |
+          source env/bin/activate
+          make install_e2e_tests
+          make run_e2e_tests
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: playwright-artifacts
+          path: ingestion/tests/e2e/artifacts/
+          retention-days: 1
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume

.github/workflows/playwright-integration-tests-postgres.yml

@@ -0,0 +1,112 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# This workflow will build a package using Maven and then publish it to GitHub packages when a release is created
+# For more information see: https://github.com/actions/setup-java/blob/main/docs/advanced-usage.md#apache-maven-with-a-settings-path
+
+name: Postgres Playwright Integration Tests
+on:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  playwright-postgresql:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    strategy:
+      fail-fast: false
+      matrix:
+        browser-type: ["chromium"]
+    environment: test
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+      - name: Cache Maven Dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+      - name: Get yarn cache directory path
+        if: steps.cache-output.outputs.exit-code == 0
+        id: yarn-cache-dir-path
+        run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
+      - uses: actions/cache@v4
+        if: steps.yarn-cache-dir-path.outputs.exit-code == 0
+        id: yarn-cache
+        with:
+          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
+          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-yarn-
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d postgresql"
+          ingestion_dependency: "playwright"
+      - name: Run Playwright Integration Tests with browser ${{ matrix.browser-type }}
+        env:
+          E2E_REDSHIFT_HOST_PORT: ${{ secrets.E2E_REDSHIFT_HOST_PORT }}
+          E2E_REDSHIFT_USERNAME: ${{ secrets.E2E_REDSHIFT_USERNAME }}
+          E2E_REDSHIFT_PASSWORD: ${{ secrets.E2E_REDSHIFT_PASSWORD }}
+          E2E_REDSHIFT_DB: ${{ secrets.E2E_REDSHIFT_DATABASE }}
+          E2E_DRUID_HOST_PORT: ${{ secrets.E2E_DRUID_HOST_PORT }}
+          E2E_HIVE_HOST_PORT: ${{ secrets.E2E_HIVE_HOST_PORT }}
+        run: |
+          source env/bin/activate
+          make install_e2e_tests
+          make run_e2e_tests
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: playwright-artifacts
+          path: ingestion/tests/e2e/artifacts/
+          retention-days: 1
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume

.github/workflows/playwright-knowledge-graph-postgresql-e2e.yml

@@ -0,0 +1,224 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Postgresql PR Knowledge Graph E2E Tests
+on:
+  workflow_dispatch:
+  pull_request:
+    types:
+      - opened
+      - synchronize
+      - reopened
+      - ready_for_review
+    paths:
+      - ".github/actions/setup-openmetadata-test-environment/action.yml"
+      - ".github/workflows/playwright-knowledge-graph-postgresql-e2e.yml"
+      - "docker/run_local_docker.sh"
+      - "docker/run_local_docker_common.sh"
+      - "docker/run_local_docker_rdf.sh"
+      - "docker/validate_compose.py"
+      - "docker/development/docker-compose-fuseki.yml"
+      - "docker/development/docker-compose-postgres-fuseki.yml"
+      - "docs/rdf-local-development.md"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles/rdf/**"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/rdf/**"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/resources/rdf/**"
+      - "openmetadata-service/src/test/java/org/openmetadata/service/apps/bundles/rdf/**"
+      - "openmetadata-service/src/test/java/org/openmetadata/service/rdf/**"
+      - "openmetadata-service/src/test/java/org/openmetadata/service/resources/rdf/**"
+      - "openmetadata-spec/src/main/resources/rdf/**"
+      - "openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/KnowledgeGraph.spec.ts"
+      - "openmetadata-ui/src/main/resources/ui/playwright.config.ts"
+      - "openmetadata-ui/src/main/resources/ui/src/components/KnowledgeGraph3D/**"
+      - "openmetadata-ui/src/main/resources/ui/src/components/OntologyExplorer/**"
+      - "openmetadata-ui/src/main/resources/ui/src/rest/rdfAPI.ts"
+      - "openmetadata-ui/src/main/resources/ui/src/types/knowledgeGraph.types.ts"
+      - "openmetadata-ui/src/main/resources/ui/src/utils/TableUtils.tsx"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: playwright-knowledge-graph-pr-postgresql-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name != 'pull_request' || !github.event.pull_request.draft }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+
+      - name: Setup JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven Dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install antlr cli
+        run: sudo make install_antlr_cli
+
+      - name: Build with Maven
+        run: mvn -DskipTests clean package
+
+      - name: Upload Maven build artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: openmetadata-build
+          path: openmetadata-dist/target/openmetadata-*.tar.gz
+          retention-days: 1
+
+  playwright-knowledge-graph-postgresql:
+    needs: [build]
+    runs-on: ubuntu-latest
+    if: ${{ !cancelled() && needs.build.result == 'success' && (github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork) }}
+    environment: test
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+
+      - name: Prepare temporary directory for Maven build artifact
+        run: mkdir -p "${{ runner.temp }}/openmetadata-build-artifact"
+
+      - name: Download Maven build artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: openmetadata-build
+          path: ${{ runner.temp }}/openmetadata-build-artifact
+
+      - name: Copy Maven build artifact into workspace
+        run: |
+          mkdir -p openmetadata-dist/target
+          cp -a "${{ runner.temp }}/openmetadata-build-artifact/." openmetadata-dist/target/
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d postgresql -s true"
+          startup-script: "./docker/run_local_docker_rdf.sh"
+          ingestion_dependency: "all"
+
+      - name: Wait for Fuseki to be healthy
+        run: |
+          echo "Verifying Fuseki is healthy before running tests..."
+          for i in $(seq 1 30); do
+            if curl -sf "http://localhost:3030/\$/ping" > /dev/null 2>&1; then
+              echo "Fuseki is healthy"
+              exit 0
+            fi
+            echo "Waiting for Fuseki ($i/30)..."
+            sleep 10
+          done
+          echo "Fuseki failed health check. Container logs:"
+          docker logs openmetadata-fuseki --tail 100
+          exit 1
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: "openmetadata-ui/src/main/resources/ui/.nvmrc"
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+
+      - name: Install Playwright Browsers
+        run: npx playwright@1.57.0 install chromium --with-deps
+
+      - name: Run Knowledge Graph Playwright tests
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: npx playwright test --project="Knowledge Graph"
+        env:
+          PLAYWRIGHT_IS_OSS: true
+          PLAYWRIGHT_SNOWFLAKE_USERNAME: ${{ secrets.TEST_SNOWFLAKE_USERNAME }}
+          PLAYWRIGHT_SNOWFLAKE_PASSWORD: ${{ secrets.TEST_SNOWFLAKE_PASSWORD }}
+          PLAYWRIGHT_SNOWFLAKE_ACCOUNT: ${{ secrets.TEST_SNOWFLAKE_ACCOUNT }}
+          PLAYWRIGHT_SNOWFLAKE_DATABASE: ${{ secrets.TEST_SNOWFLAKE_DATABASE }}
+          PLAYWRIGHT_SNOWFLAKE_WAREHOUSE: ${{ secrets.TEST_SNOWFLAKE_WAREHOUSE }}
+          PLAYWRIGHT_SNOWFLAKE_PASSPHRASE: ${{ secrets.TEST_SNOWFLAKE_PASSPHRASE }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY: ${{ secrets.TEST_BQ_PRIVATE_KEY }}
+          PLAYWRIGHT_BQ_PROJECT_ID: ${{ secrets.PLAYWRIGHT_BQ_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY_ID: ${{ secrets.TEST_BQ_PRIVATE_KEY_ID }}
+          PLAYWRIGHT_BQ_PROJECT_ID_TAXONOMY: ${{ secrets.TEST_BQ_PROJECT_ID_TAXONOMY }}
+          PLAYWRIGHT_BQ_CLIENT_EMAIL: ${{ secrets.TEST_BQ_CLIENT_EMAIL }}
+          PLAYWRIGHT_BQ_CLIENT_ID: ${{ secrets.TEST_BQ_CLIENT_ID }}
+          PLAYWRIGHT_REDSHIFT_HOST: ${{ secrets.E2E_REDSHIFT_HOST_PORT }}
+          PLAYWRIGHT_REDSHIFT_USERNAME: ${{ secrets.E2E_REDSHIFT_USERNAME }}
+          PLAYWRIGHT_REDSHIFT_PASSWORD: ${{ secrets.E2E_REDSHIFT_PASSWORD }}
+          PLAYWRIGHT_REDSHIFT_DATABASE: ${{ secrets.TEST_REDSHIFT_DATABASE }}
+          PLAYWRIGHT_METABASE_USERNAME: ${{ secrets.TEST_METABASE_USERNAME }}
+          PLAYWRIGHT_METABASE_PASSWORD: ${{ secrets.TEST_METABASE_PASSWORD }}
+          PLAYWRIGHT_METABASE_DB_SERVICE_NAME: ${{ secrets.TEST_METABASE_DB_SERVICE_NAME }}
+          PLAYWRIGHT_METABASE_HOST_PORT: ${{ secrets.TEST_METABASE_HOST_PORT }}
+          PLAYWRIGHT_SUPERSET_USERNAME: ${{ secrets.TEST_SUPERSET_USERNAME }}
+          PLAYWRIGHT_SUPERSET_PASSWORD: ${{ secrets.TEST_SUPERSET_PASSWORD }}
+          PLAYWRIGHT_SUPERSET_HOST_PORT: ${{ secrets.TEST_SUPERSET_HOST_PORT }}
+          PLAYWRIGHT_KAFKA_BOOTSTRAP_SERVERS: ${{ secrets.TEST_KAFKA_BOOTSTRAP_SERVERS }}
+          PLAYWRIGHT_KAFKA_SCHEMA_REGISTRY_URL: ${{ secrets.TEST_KAFKA_SCHEMA_REGISTRY_URL }}
+          PLAYWRIGHT_GLUE_ACCESS_KEY: ${{ secrets.TEST_GLUE_ACCESS_KEY }}
+          PLAYWRIGHT_GLUE_SECRET_KEY: ${{ secrets.TEST_GLUE_SECRET_KEY }}
+          PLAYWRIGHT_GLUE_AWS_REGION: ${{ secrets.TEST_GLUE_AWS_REGION }}
+          PLAYWRIGHT_GLUE_ENDPOINT: ${{ secrets.TEST_GLUE_ENDPOINT }}
+          PLAYWRIGHT_GLUE_STORAGE_SERVICE: ${{ secrets.TEST_GLUE_STORAGE_SERVICE }}
+          PLAYWRIGHT_MYSQL_USERNAME: ${{ secrets.TEST_MYSQL_USERNAME }}
+          PLAYWRIGHT_MYSQL_PASSWORD: ${{ secrets.TEST_MYSQL_PASSWORD }}
+          PLAYWRIGHT_MYSQL_HOST_PORT: ${{ secrets.TEST_MYSQL_HOST_PORT }}
+          PLAYWRIGHT_MYSQL_DATABASE_SCHEMA: ${{ secrets.TEST_MYSQL_DATABASE_SCHEMA }}
+          PLAYWRIGHT_POSTGRES_USERNAME: ${{ secrets.TEST_POSTGRES_USERNAME }}
+          PLAYWRIGHT_POSTGRES_PASSWORD: ${{ secrets.TEST_POSTGRES_PASSWORD }}
+          PLAYWRIGHT_POSTGRES_HOST_PORT: ${{ secrets.TEST_POSTGRES_HOST_PORT }}
+          PLAYWRIGHT_POSTGRES_DATABASE: ${{ secrets.TEST_POSTGRES_DATABASE }}
+          PLAYWRIGHT_AIRFLOW_HOST_PORT: ${{ secrets.TEST_AIRFLOW_HOST_PORT }}
+          PLAYWRIGHT_ML_MODEL_TRACKING_URI: ${{ secrets.TEST_ML_MODEL_TRACKING_URI }}
+          PLAYWRIGHT_ML_MODEL_REGISTRY_URI: ${{ secrets.TEST_ML_MODEL_REGISTRY_URI }}
+          PLAYWRIGHT_S3_STORAGE_ACCESS_KEY_ID: ${{ secrets.TEST_S3_STORAGE_ACCESS_KEY_ID }}
+          PLAYWRIGHT_S3_STORAGE_SECRET_ACCESS_KEY: ${{ secrets.TEST_S3_STORAGE_SECRET_ACCESS_KEY }}
+          PLAYWRIGHT_S3_STORAGE_END_POINT_URL: ${{ secrets.TEST_S3_STORAGE_END_POINT_URL }}
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - uses: actions/upload-artifact@v4
+        if: ${{ !cancelled() }}
+        with:
+          name: playwright-knowledge-graph-report
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/playwright-report
+          retention-days: 5
+
+      - name: Clean Up
+        if: always()
+        run: |
+          docker compose -f docker/development/docker-compose-postgres.yml -f docker/development/docker-compose-fuseki.yml down --remove-orphans || true
+          docker compose -f docker/development/docker-compose-postgres.yml down --remove-orphans || true
+          sudo rm -rf ${PWD}/docker/development/docker-volume

.github/workflows/playwright-mysql-e2e-skip.yml

@@ -0,0 +1,42 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# Avoid running Playwright on each PR opened which does not modify Java or UI code
+# https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-
+# of-pull-requests/troubleshooting-required-status-checks#handling-skipped-but-required-checks
+name: MySQL PR Playwright E2E Tests
+
+on:
+  pull_request_target:
+    types:
+      - labeled
+      - opened
+      - synchronize
+      - reopened
+    paths:
+      - ".github/**"
+      - "openmetadata-dist/**"
+      - "docker/**"
+      - "!docker/development/docker-compose.yml"
+      - "!docker/development/docker-compose-postgres.yml"
+      - "openmetadata-ui/src/main/resources/ui/playwright/doc-generator/**"
+      - "openmetadata-ui/src/main/resources/ui/playwright/docs/**"
+
+jobs:
+  playwright-ci-mysql:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        shardIndex: [1, 2]
+        shardTotal: [2]
+    steps:
+      - run: 'echo "Step is not required"'

.github/workflows/playwright-mysql-e2e.yml

@@ -0,0 +1,191 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# This workflow executes end-to-end (e2e) tests using Playwright with MySQL as the database.
+# For more information see: https://github.com/actions/setup-java/blob/main/docs/advanced-usage.md#apache-maven-with-a-settings-path
+
+name: MySQL PR Playwright E2E Tests
+on:
+  workflow_dispatch:
+  pull_request_target:
+    types:
+      - labeled
+      - opened
+      - synchronize
+      - reopened
+      - ready_for_review
+    paths-ignore:
+      - ".github/**"
+      - "openmetadata-dist/**"
+      - "docker/**"
+      - "!docker/development/docker-compose.yml"
+      - "!docker/development/docker-compose-postgres.yml"
+      - "openmetadata-ui/src/main/resources/ui/playwright/doc-generator/**"
+      - "openmetadata-ui/src/main/resources/ui/playwright/docs/**"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: playwright-ci-pr-mysql-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  playwright-ci-mysql:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    environment: test
+    permissions:
+      contents: read
+      id-token: write # Required for GitHub OIDC (Flakiness.io reporter)
+    env:
+      # Playwright logs the admin user in many times (performAdminLogin per test, parallel
+      # workers, retries). The production default of 5 active sessions per user would evict the
+      # long-lived storageState session the page fixtures rely on and 401 every request. Raise
+      # the cap for E2E only; docker compose reads this for ${AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER:-5}.
+      AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER: "10000"
+    strategy:
+      fail-fast: false
+      matrix:
+        shardIndex: [1, 2]
+        shardTotal: [2]
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+
+      - name: Cache Maven Dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d mysql"
+          ingestion_dependency: "all"
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: 'openmetadata-ui/src/main/resources/ui/.nvmrc'
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+      - name: Install Playwright Browsers
+        run: npx playwright@1.57.0 install chromium --with-deps
+      - name: Run Playwright tests
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: |
+          if [[ "${{ contains(github.event.pull_request.changed_files, 'ingestion/') }}" == "true" ]]; then
+            npx playwright test --grep @ingestion --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
+          else
+            npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
+          fi
+        env:
+          PLAYWRIGHT_IS_OSS: true
+          PLAYWRIGHT_SNOWFLAKE_USERNAME: ${{ secrets.TEST_SNOWFLAKE_USERNAME }}
+          PLAYWRIGHT_SNOWFLAKE_PASSWORD: ${{ secrets.TEST_SNOWFLAKE_PASSWORD }}
+          PLAYWRIGHT_SNOWFLAKE_ACCOUNT: ${{ secrets.TEST_SNOWFLAKE_ACCOUNT }}
+          PLAYWRIGHT_SNOWFLAKE_DATABASE: ${{ secrets.TEST_SNOWFLAKE_DATABASE }}
+          PLAYWRIGHT_SNOWFLAKE_WAREHOUSE: ${{ secrets.TEST_SNOWFLAKE_WAREHOUSE }}
+          PLAYWRIGHT_SNOWFLAKE_PASSPHRASE: ${{ secrets.TEST_SNOWFLAKE_PASSPHRASE }}
+          PLAYWRIGHT_PROJECT_ID: ${{ steps.cypress-project-id.outputs.CYPRESS_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY: ${{ secrets.TEST_BQ_PRIVATE_KEY }}
+          PLAYWRIGHT_BQ_PROJECT_ID: ${{ secrets.PLAYWRIGHT_BQ_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY_ID: ${{ secrets.TEST_BQ_PRIVATE_KEY_ID }}
+          PLAYWRIGHT_BQ_PROJECT_ID_TAXONOMY: ${{ secrets.TEST_BQ_PROJECT_ID_TAXONOMY }}
+          PLAYWRIGHT_BQ_CLIENT_EMAIL: ${{ secrets.TEST_BQ_CLIENT_EMAIL }}
+          PLAYWRIGHT_BQ_CLIENT_ID: ${{ secrets.TEST_BQ_CLIENT_ID }}
+          PLAYWRIGHT_REDSHIFT_HOST: ${{ secrets.E2E_REDSHIFT_HOST_PORT }}
+          PLAYWRIGHT_REDSHIFT_USERNAME: ${{ secrets.E2E_REDSHIFT_USERNAME }}
+          PLAYWRIGHT_REDSHIFT_PASSWORD: ${{ secrets.E2E_REDSHIFT_PASSWORD }}
+          PLAYWRIGHT_REDSHIFT_DATABASE: ${{ secrets.TEST_REDSHIFT_DATABASE }}
+          PLAYWRIGHT_METABASE_USERNAME: ${{ secrets.TEST_METABASE_USERNAME }}
+          PLAYWRIGHT_METABASE_PASSWORD: ${{ secrets.TEST_METABASE_PASSWORD }}
+          PLAYWRIGHT_METABASE_DB_SERVICE_NAME: ${{ secrets.TEST_METABASE_DB_SERVICE_NAME }}
+          PLAYWRIGHT_METABASE_HOST_PORT: ${{ secrets.TEST_METABASE_HOST_PORT }}
+          PLAYWRIGHT_SUPERSET_USERNAME: ${{ secrets.TEST_SUPERSET_USERNAME }}
+          PLAYWRIGHT_SUPERSET_PASSWORD: ${{ secrets.TEST_SUPERSET_PASSWORD }}
+          PLAYWRIGHT_SUPERSET_HOST_PORT: ${{ secrets.TEST_SUPERSET_HOST_PORT }}
+          PLAYWRIGHT_KAFKA_BOOTSTRAP_SERVERS: ${{ secrets.TEST_KAFKA_BOOTSTRAP_SERVERS }}
+          PLAYWRIGHT_KAFKA_SCHEMA_REGISTRY_URL: ${{ secrets.TEST_KAFKA_SCHEMA_REGISTRY_URL }}
+          PLAYWRIGHT_GLUE_ACCESS_KEY: ${{ secrets.TEST_GLUE_ACCESS_KEY }}
+          PLAYWRIGHT_GLUE_SECRET_KEY: ${{ secrets.TEST_GLUE_SECRET_KEY }}
+          PLAYWRIGHT_GLUE_AWS_REGION: ${{ secrets.TEST_GLUE_AWS_REGION }}
+          PLAYWRIGHT_GLUE_ENDPOINT: ${{ secrets.TEST_GLUE_ENDPOINT }}
+          PLAYWRIGHT_GLUE_STORAGE_SERVICE: ${{ secrets.TEST_GLUE_STORAGE_SERVICE }}
+          PLAYWRIGHT_MYSQL_USERNAME: ${{ secrets.TEST_MYSQL_USERNAME }}
+          PLAYWRIGHT_MYSQL_PASSWORD: ${{ secrets.TEST_MYSQL_PASSWORD }}
+          PLAYWRIGHT_MYSQL_HOST_PORT: ${{ secrets.TEST_MYSQL_HOST_PORT }}
+          PLAYWRIGHT_MYSQL_DATABASE_SCHEMA: ${{ secrets.TEST_MYSQL_DATABASE_SCHEMA }}
+          PLAYWRIGHT_POSTGRES_USERNAME: ${{ secrets.TEST_POSTGRES_USERNAME }}
+          PLAYWRIGHT_POSTGRES_PASSWORD: ${{ secrets.TEST_POSTGRES_PASSWORD }}
+          PLAYWRIGHT_POSTGRES_HOST_PORT: ${{ secrets.TEST_POSTGRES_HOST_PORT }}
+          PLAYWRIGHT_POSTGRES_DATABASE: ${{ secrets.TEST_POSTGRES_DATABASE }}
+          PLAYWRIGHT_AIRFLOW_HOST_PORT: ${{ secrets.TEST_AIRFLOW_HOST_PORT }}
+          PLAYWRIGHT_ML_MODEL_TRACKING_URI: ${{ secrets.TEST_ML_MODEL_TRACKING_URI }}
+          PLAYWRIGHT_ML_MODEL_REGISTRY_URI: ${{ secrets.TEST_ML_MODEL_REGISTRY_URI }}
+          PLAYWRIGHT_S3_STORAGE_ACCESS_KEY_ID: ${{ secrets.TEST_S3_STORAGE_ACCESS_KEY_ID }}
+          PLAYWRIGHT_S3_STORAGE_SECRET_ACCESS_KEY: ${{ secrets.TEST_S3_STORAGE_SECRET_ACCESS_KEY }}
+          PLAYWRIGHT_S3_STORAGE_END_POINT_URL: ${{ secrets.TEST_S3_STORAGE_END_POINT_URL }}
+
+          # Recommended: pass the GitHub token lets this action correctly
+          # determine the unique run id necessary to re-run the checks
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - uses: actions/upload-artifact@v4
+        if: ${{ !cancelled() }}
+        with:
+          name: playwright-report--${{ matrix.shardIndex }}
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/playwright-report
+          retention-days: 5
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume

.github/workflows/playwright-ontology-rdf-postgresql-e2e.yml

@@ -0,0 +1,175 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Postgresql PR Ontology RDF E2E Tests
+on:
+  workflow_dispatch:
+  pull_request:
+    types:
+      - opened
+      - synchronize
+      - reopened
+      - ready_for_review
+    paths:
+      - ".github/actions/setup-openmetadata-test-environment/action.yml"
+      - ".github/workflows/playwright-ontology-rdf-postgresql-e2e.yml"
+      - "docker/run_local_docker.sh"
+      - "docker/run_local_docker_common.sh"
+      - "docker/run_local_docker_rdf.sh"
+      - "docker/development/docker-compose-fuseki.yml"
+      - "docker/development/docker-compose-postgres-fuseki.yml"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/rdf/**"
+      - "openmetadata-service/src/main/java/org/openmetadata/service/resources/glossary/**"
+      - "openmetadata-spec/src/main/resources/json/schema/api/data/createGlossaryTerm.json"
+      - "openmetadata-spec/src/main/resources/json/schema/configuration/glossaryTermRelationSettings.json"
+      - "openmetadata-spec/src/main/resources/json/schema/entity/data/glossary.json"
+      - "openmetadata-spec/src/main/resources/json/schema/entity/data/glossaryTerm.json"
+      - "openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/OntologyImportRdf.spec.ts"
+      - "openmetadata-ui/src/main/resources/ui/playwright.config.ts"
+      - "openmetadata-ui/src/main/resources/ui/src/components/Glossary/GlossaryHeader/**"
+      - "openmetadata-ui/src/main/resources/ui/src/components/Glossary/ImportOntologyModal/**"
+      - "openmetadata-ui/src/main/resources/ui/src/rest/importExportAPI.ts"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: playwright-ontology-rdf-pr-postgresql-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name != 'pull_request' || !github.event.pull_request.draft }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+
+      - name: Setup JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven Dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install antlr cli
+        run: sudo make install_antlr_cli
+
+      - name: Build with Maven
+        run: mvn -DskipTests clean package
+
+      - name: Upload Maven build artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: openmetadata-build
+          path: openmetadata-dist/target/openmetadata-*.tar.gz
+          retention-days: 1
+
+  playwright-ontology-rdf-postgresql:
+    needs: [build]
+    runs-on: ubuntu-latest
+    if: ${{ !cancelled() && needs.build.result == 'success' && (github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork) }}
+    environment: test
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+
+      - name: Prepare temporary directory for Maven build artifact
+        run: mkdir -p "${{ runner.temp }}/openmetadata-build-artifact"
+
+      - name: Download Maven build artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: openmetadata-build
+          path: ${{ runner.temp }}/openmetadata-build-artifact
+
+      - name: Copy Maven build artifact into workspace
+        run: |
+          mkdir -p openmetadata-dist/target
+          cp -a "${{ runner.temp }}/openmetadata-build-artifact/." openmetadata-dist/target/
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d postgresql -s true"
+          startup-script: "./docker/run_local_docker_rdf.sh"
+          ingestion_dependency: "all"
+
+      - name: Wait for Fuseki to be healthy
+        run: |
+          echo "Verifying Fuseki is healthy before running tests..."
+          for i in $(seq 1 30); do
+            if curl -sf "http://localhost:3030/\$/ping" > /dev/null 2>&1; then
+              echo "Fuseki is healthy"
+              exit 0
+            fi
+            echo "Waiting for Fuseki ($i/30)..."
+            sleep 10
+          done
+          echo "Fuseki failed health check. Container logs:"
+          docker logs openmetadata-fuseki --tail 100
+          exit 1
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: "openmetadata-ui/src/main/resources/ui/.nvmrc"
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+
+      - name: Install Playwright Browsers
+        run: npx playwright@1.57.0 install chromium --with-deps
+
+      - name: Run Ontology RDF Playwright tests
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: npx playwright test --project="Ontology RDF"
+        env:
+          PLAYWRIGHT_IS_OSS: true
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - uses: actions/upload-artifact@v4
+        if: ${{ !cancelled() }}
+        with:
+          name: playwright-ontology-rdf-report
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/playwright-report
+          retention-days: 5
+
+      - name: Clean Up
+        if: always()
+        run: |
+          docker compose -f docker/development/docker-compose-postgres.yml -f docker/development/docker-compose-fuseki.yml down --remove-orphans || true
+          docker compose -f docker/development/docker-compose-postgres.yml down --remove-orphans || true
+          sudo rm -rf ${PWD}/docker/development/docker-volume

.github/workflows/playwright-postgresql-e2e.yml

@@ -0,0 +1,620 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# This workflow executes end-to-end (e2e) tests using Playwright with PostgreSQL as the database.
+# For more information see: https://github.com/actions/setup-java/blob/main/docs/advanced-usage.md#apache-maven-with-a-settings-path
+
+name: Postgresql PR Playwright E2E Tests
+on:
+  merge_group:
+  workflow_dispatch:
+  # Note: paths-ignore removed — the workflow always triggers so that the
+  # required status check (playwright-summary) always completes. Path filtering
+  # is handled inside the workflow via dorny/paths-filter so PRs without
+  # relevant changes skip the expensive jobs while still reporting a passing status.
+  pull_request_target:
+    types:
+      - labeled
+      - opened
+      - synchronize
+      - reopened
+      - ready_for_review
+
+permissions:
+  contents: read
+
+concurrency:
+  group: playwright-ci-pr-postgresql-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  check-changes:
+    runs-on: ubuntu-latest
+    outputs:
+      e2e: ${{ steps.filter.outputs.e2e }}
+      docker-compose: ${{ steps.filter.outputs.docker-compose }}
+    steps:
+      - uses: dorny/paths-filter@v4
+        id: filter
+        with:
+          filters: |
+            e2e:
+              - 'openmetadata-service/**'
+              - 'openmetadata-ui/**'
+              - 'openmetadata-spec/**'
+              - 'openmetadata-integration-tests/**'
+              - 'ingestion/**'
+              - 'bootstrap/**'
+              - 'conf/**'
+              - 'docker/development/**'
+              - 'openmetadata-clients/**'
+              - 'openmetadata-airflow-apis/**'
+              - 'openmetadata-mcp/**'
+              - 'openmetadata-sdk/**'
+              - 'openmetadata-shaded-deps/**'
+              - 'openmetadata-ui-core-components/**'
+              - 'openmetadata-k8s-operator/**'
+              - 'common/**'
+              - 'openspec/**'
+              - 'pom.xml'
+              - 'Makefile'
+            docker-compose:
+              - 'docker/development/docker-compose.yml'
+              - 'docker/development/docker-compose-postgres.yml'
+
+  build:
+    needs: check-changes
+    runs-on: ubuntu-latest
+    if: |
+      !github.event.pull_request.draft &&
+      (github.event_name == 'merge_group' || github.event_name == 'workflow_dispatch' ||
+       needs.check-changes.outputs.e2e == 'true' || needs.check-changes.outputs.docker-compose == 'true') &&
+      (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test')
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Setup JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: '21'
+          distribution: 'temurin'
+
+      - name: Cache Maven Dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Install antlr cli
+        run: sudo make install_antlr_cli
+
+      - name: Build with Maven
+        run: mvn -DskipTests clean package
+
+      - name: Upload Maven build artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: openmetadata-build
+          path: openmetadata-dist/target/openmetadata-*.tar.gz
+          retention-days: 1
+
+  detect-changes:
+    needs: check-changes
+    runs-on: ubuntu-latest
+    if: |
+      !github.event.pull_request.draft &&
+      (github.event_name == 'merge_group' || github.event_name == 'workflow_dispatch' ||
+       needs.check-changes.outputs.e2e == 'true' || needs.check-changes.outputs.docker-compose == 'true') &&
+      (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test')
+    outputs:
+      spec_only_mode: ${{ steps.compute.outputs.spec_only_mode }}
+      changed_specs: ${{ steps.compute.outputs.changed_specs }}
+      matrix: ${{ steps.compute.outputs.matrix }}
+    steps:
+      - name: Checkout
+        if: ${{ github.event_name == 'pull_request_target' }}
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          fetch-depth: 0
+          filter: blob:none
+
+      - name: Get all changed files
+        id: all-changes
+        if: ${{ github.event_name == 'pull_request_target' }}
+        uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
+
+      - name: Get changed runnable spec files
+        id: changed-specs
+        if: ${{ github.event_name == 'pull_request_target' }}
+        uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
+        with:
+          path: openmetadata-ui/src/main/resources/ui
+          files: |
+            playwright/e2e/**/*.spec.ts
+          files_ignore: |
+            playwright/e2e/**/SystemCertificationTags.spec.ts
+            playwright/e2e/**/IntakeForm.spec.ts
+
+      - name: Compute spec-only mode and matrix
+        id: compute
+        env:
+          ALL_FILES: ${{ steps.all-changes.outputs.all_changed_files }}
+          SPEC_FILES: ${{ steps.changed-specs.outputs.all_changed_files }}
+        run: |
+          FULL_MATRIX='{"shardIndex":[1,2,3,4,5,6,7],"shardTotal":[7]}'
+          SPEC_MATRIX='{"shardIndex":[1],"shardTotal":[1]}'
+
+          ALL_COUNT=$(echo "$ALL_FILES" | wc -w)
+          SPEC_COUNT=$(echo "$SPEC_FILES" | wc -w)
+
+          if [ "$SPEC_COUNT" -gt 0 ] && [ "$ALL_COUNT" -eq "$SPEC_COUNT" ]; then
+            echo "spec_only_mode=true" >> "$GITHUB_OUTPUT"
+            echo "changed_specs=$SPEC_FILES" >> "$GITHUB_OUTPUT"
+            echo "matrix=$SPEC_MATRIX" >> "$GITHUB_OUTPUT"
+            echo "Spec-only mode: ${SPEC_COUNT} spec(s) changed"
+          else
+            echo "spec_only_mode=false" >> "$GITHUB_OUTPUT"
+            echo "changed_specs=" >> "$GITHUB_OUTPUT"
+            echo "matrix=$FULL_MATRIX" >> "$GITHUB_OUTPUT"
+            echo "Full suite mode (total=${ALL_COUNT}, runnable-specs=${SPEC_COUNT})"
+          fi
+
+  playwright-ci-postgresql:
+    needs: [build, detect-changes]
+    runs-on: ubuntu-latest
+    if: ${{ !cancelled() && needs.build.result == 'success' && needs.detect-changes.result == 'success' }}
+    environment: test
+    permissions:
+      contents: read
+      id-token: write # Required for GitHub OIDC (Flakiness.io reporter)
+    env:
+      # Playwright logs the admin user in many times (performAdminLogin per test, parallel
+      # workers, retries). The production default of 5 active sessions per user would evict the
+      # long-lived storageState session the page fixtures rely on and 401 every request. Raise
+      # the cap for E2E only; docker compose reads this for ${AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER:-5}.
+      AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER: "10000"
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJSON(needs.detect-changes.outputs.matrix) }}
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Download Maven build artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: openmetadata-build
+          path: openmetadata-dist/target
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d postgresql -s true"
+          ingestion_dependency: "all"
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: "openmetadata-ui/src/main/resources/ui/.nvmrc"
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+      - name: Install Playwright Browsers
+        run: npx playwright@1.57.0 install chromium --with-deps
+      - name: Run Playwright tests
+        id: run-tests
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: |
+          if [ "$SPEC_ONLY" = "true" ]; then
+            echo "🔹 Spec-only mode: running changed specs → ${CHANGED_SPECS}"
+
+            PROJECT_ARGS=""
+
+            # DataAssetRulesEnabled runs standalone; its deps (setup) are resolved automatically
+            if echo "$CHANGED_SPECS" | tr ' ' '\n' | grep -q "DataAssetRulesEnabled\.spec\.ts"; then
+              PROJECT_ARGS="$PROJECT_ARGS --project=DataAssetRulesEnabled"
+            fi
+            # DataAssetRulesDisabled depends on DataAssetRulesEnabled — Playwright resolves this automatically
+            if echo "$CHANGED_SPECS" | tr ' ' '\n' | grep -q "DataAssetRulesDisabled\.spec\.ts"; then
+              PROJECT_ARGS="$PROJECT_ARGS --project=DataAssetRulesDisabled"
+            fi
+            # SearchRBAC depends on the full DataAssetRules chain — Playwright resolves this automatically
+            if echo "$CHANGED_SPECS" | tr ' ' '\n' | grep -q "SearchRBAC\.spec\.ts"; then
+              PROJECT_ARGS="$PROJECT_ARGS --project=SearchRBAC"
+            fi
+            # DomainIsolation uses serial workers and its own testMatch
+            if echo "$CHANGED_SPECS" | tr ' ' '\n' | grep -q "DomainIsolation/"; then
+              PROJECT_ARGS="$PROJECT_ARGS --project=DomainIsolation"
+            fi
+            # Search relevance specs use the dedicated search-nightly project.
+            if echo "$CHANGED_SPECS" | tr ' ' '\n' | grep -q "playwright/e2e/Search/"; then
+              PROJECT_ARGS="$PROJECT_ARGS --project=search-nightly"
+            fi
+            # Regular chromium specs: anything not in a special-project category.
+            # Also add Basic because chromium has grepInvert:[@basic], so @basic-tagged tests
+            # in the changed files would be silently skipped without it.
+            if echo "$CHANGED_SPECS" | tr ' ' '\n' | grep -qvE "(DataAssetRulesEnabled\.spec\.ts|DataAssetRulesDisabled\.spec\.ts|SearchRBAC\.spec\.ts|DomainIsolation/|playwright/e2e/Search/)"; then
+              PROJECT_ARGS="$PROJECT_ARGS --project=chromium --project=Basic"
+            fi
+
+            [ -z "$PROJECT_ARGS" ] && PROJECT_ARGS="--project=chromium --project=Basic"
+
+            npx playwright test $PROJECT_ARGS $CHANGED_SPECS
+
+          elif [ "${{ matrix.shardIndex }}" -eq "1" ]; then
+            echo "🔹 Running DataAssetRules-only tests on shard 1"
+
+            # The testMatch pattern ensures only DataAssetRules*.spec.ts files run
+            npx playwright test \
+              --project=setup \
+              --project=DataAssetRulesEnabled \
+              --project=DataAssetRulesDisabled \
+              --project=Basic \
+              --project=SearchRBAC \
+              --project=DomainIsolation \
+
+          elif [ "${{ matrix.shardIndex }}" -eq "2" ]; then
+            echo "🔹 Running search relevance tests on shard 2"
+
+            npx playwright test \
+              --project=search-nightly \
+              playwright/e2e/Search/SearchRelevance.spec.ts \
+              --workers=1
+
+          else
+            # Shards 3-7 run common chromium tests equally distributed (5-way sharding)
+            CHROMIUM_SHARD=$(( ${{ matrix.shardIndex }} - 2 ))
+            echo "🔹 Running common chromium tests on shard ${CHROMIUM_SHARD}/5"
+            npx playwright test \
+              --project=chromium \
+              --grep-invert @dataAssetRules \
+              --shard=${CHROMIUM_SHARD}/5
+          fi
+
+        env:
+          SPEC_ONLY: ${{ needs.detect-changes.outputs.spec_only_mode }}
+          CHANGED_SPECS: ${{ needs.detect-changes.outputs.changed_specs }}
+          PLAYWRIGHT_IS_OSS: true
+          PLAYWRIGHT_SNOWFLAKE_USERNAME: ${{ secrets.TEST_SNOWFLAKE_USERNAME }}
+          PLAYWRIGHT_SNOWFLAKE_PASSWORD: ${{ secrets.TEST_SNOWFLAKE_PASSWORD }}
+          PLAYWRIGHT_SNOWFLAKE_ACCOUNT: ${{ secrets.TEST_SNOWFLAKE_ACCOUNT }}
+          PLAYWRIGHT_SNOWFLAKE_DATABASE: ${{ secrets.TEST_SNOWFLAKE_DATABASE }}
+          PLAYWRIGHT_SNOWFLAKE_WAREHOUSE: ${{ secrets.TEST_SNOWFLAKE_WAREHOUSE }}
+          PLAYWRIGHT_SNOWFLAKE_PASSPHRASE: ${{ secrets.TEST_SNOWFLAKE_PASSPHRASE }}
+          PLAYWRIGHT_PROJECT_ID: ${{ steps.cypress-project-id.outputs.CYPRESS_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY: ${{ secrets.TEST_BQ_PRIVATE_KEY }}
+          PLAYWRIGHT_BQ_PROJECT_ID: ${{ secrets.PLAYWRIGHT_BQ_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY_ID: ${{ secrets.TEST_BQ_PRIVATE_KEY_ID }}
+          PLAYWRIGHT_BQ_PROJECT_ID_TAXONOMY: ${{ secrets.TEST_BQ_PROJECT_ID_TAXONOMY }}
+          PLAYWRIGHT_BQ_CLIENT_EMAIL: ${{ secrets.TEST_BQ_CLIENT_EMAIL }}
+          PLAYWRIGHT_BQ_CLIENT_ID: ${{ secrets.TEST_BQ_CLIENT_ID }}
+          PLAYWRIGHT_REDSHIFT_HOST: ${{ secrets.E2E_REDSHIFT_HOST_PORT }}
+          PLAYWRIGHT_REDSHIFT_USERNAME: ${{ secrets.E2E_REDSHIFT_USERNAME }}
+          PLAYWRIGHT_REDSHIFT_PASSWORD: ${{ secrets.E2E_REDSHIFT_PASSWORD }}
+          PLAYWRIGHT_REDSHIFT_DATABASE: ${{ secrets.TEST_REDSHIFT_DATABASE }}
+          PLAYWRIGHT_METABASE_USERNAME: ${{ secrets.TEST_METABASE_USERNAME }}
+          PLAYWRIGHT_METABASE_PASSWORD: ${{ secrets.TEST_METABASE_PASSWORD }}
+          PLAYWRIGHT_METABASE_DB_SERVICE_NAME: ${{ secrets.TEST_METABASE_DB_SERVICE_NAME }}
+          PLAYWRIGHT_METABASE_HOST_PORT: ${{ secrets.TEST_METABASE_HOST_PORT }}
+          PLAYWRIGHT_SUPERSET_USERNAME: ${{ secrets.TEST_SUPERSET_USERNAME }}
+          PLAYWRIGHT_SUPERSET_PASSWORD: ${{ secrets.TEST_SUPERSET_PASSWORD }}
+          PLAYWRIGHT_SUPERSET_HOST_PORT: ${{ secrets.TEST_SUPERSET_HOST_PORT }}
+          PLAYWRIGHT_KAFKA_BOOTSTRAP_SERVERS: ${{ secrets.TEST_KAFKA_BOOTSTRAP_SERVERS }}
+          PLAYWRIGHT_KAFKA_SCHEMA_REGISTRY_URL: ${{ secrets.TEST_KAFKA_SCHEMA_REGISTRY_URL }}
+          PLAYWRIGHT_GLUE_ACCESS_KEY: ${{ secrets.TEST_GLUE_ACCESS_KEY }}
+          PLAYWRIGHT_GLUE_SECRET_KEY: ${{ secrets.TEST_GLUE_SECRET_KEY }}
+          PLAYWRIGHT_GLUE_AWS_REGION: ${{ secrets.TEST_GLUE_AWS_REGION }}
+          PLAYWRIGHT_GLUE_ENDPOINT: ${{ secrets.TEST_GLUE_ENDPOINT }}
+          PLAYWRIGHT_GLUE_STORAGE_SERVICE: ${{ secrets.TEST_GLUE_STORAGE_SERVICE }}
+          PLAYWRIGHT_MYSQL_USERNAME: ${{ secrets.TEST_MYSQL_USERNAME }}
+          PLAYWRIGHT_MYSQL_PASSWORD: ${{ secrets.TEST_MYSQL_PASSWORD }}
+          PLAYWRIGHT_MYSQL_HOST_PORT: ${{ secrets.TEST_MYSQL_HOST_PORT }}
+          PLAYWRIGHT_MYSQL_DATABASE_SCHEMA: ${{ secrets.TEST_MYSQL_DATABASE_SCHEMA }}
+          PLAYWRIGHT_POSTGRES_USERNAME: ${{ secrets.TEST_POSTGRES_USERNAME }}
+          PLAYWRIGHT_POSTGRES_PASSWORD: ${{ secrets.TEST_POSTGRES_PASSWORD }}
+          PLAYWRIGHT_POSTGRES_HOST_PORT: ${{ secrets.TEST_POSTGRES_HOST_PORT }}
+          PLAYWRIGHT_POSTGRES_DATABASE: ${{ secrets.TEST_POSTGRES_DATABASE }}
+          PLAYWRIGHT_AIRFLOW_HOST_PORT: ${{ secrets.TEST_AIRFLOW_HOST_PORT }}
+          PLAYWRIGHT_ML_MODEL_TRACKING_URI: ${{ secrets.TEST_ML_MODEL_TRACKING_URI }}
+          PLAYWRIGHT_ML_MODEL_REGISTRY_URI: ${{ secrets.TEST_ML_MODEL_REGISTRY_URI }}
+          PLAYWRIGHT_S3_STORAGE_ACCESS_KEY_ID: ${{ secrets.TEST_S3_STORAGE_ACCESS_KEY_ID }}
+          PLAYWRIGHT_S3_STORAGE_SECRET_ACCESS_KEY: ${{ secrets.TEST_S3_STORAGE_SECRET_ACCESS_KEY }}
+          PLAYWRIGHT_S3_STORAGE_END_POINT_URL: ${{ secrets.TEST_S3_STORAGE_END_POINT_URL }}
+
+          # Recommended: pass the GitHub token lets this action correctly
+          # determine the unique run id necessary to re-run the checks
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - uses: actions/upload-artifact@v4
+        if: ${{ !cancelled() }}
+        with:
+          name: playwright-report-${{ matrix.shardIndex }}
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/playwright-report
+          retention-days: 5
+
+      - name: Upload test results (screenshots, traces)
+        uses: actions/upload-artifact@v4
+        if: ${{ !cancelled() }}
+        with:
+          name: playwright-test-results-${{ matrix.shardIndex }}
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/test-results
+          retention-days: 5
+
+      - name: Upload results JSON for summary
+        uses: actions/upload-artifact@v4
+        if: ${{ !cancelled() }}
+        with:
+          name: playwright-results-json-${{ matrix.shardIndex }}
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/results.json
+          retention-days: 1
+          if-no-files-found: ignore
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+  playwright-summary:
+    if: always()
+    needs: playwright-ci-postgresql
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write # Posts the consolidated PR comment
+    steps:
+      - name: Download all results JSON
+        if: needs.playwright-ci-postgresql.result != 'skipped'
+        uses: actions/download-artifact@v4
+        with:
+          pattern: playwright-results-json-*
+          path: results
+
+      - name: Post consolidated PR comment and gate on results
+        uses: actions/github-script@v7
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            const upstreamResult = '${{ needs.playwright-ci-postgresql.result }}';
+            if (upstreamResult === 'skipped') {
+              // Distinguish the two reasons build/playwright can be skipped:
+              // 1. A non-"safe to test" label was added — build skips intentionally for fork security.
+              //    Tests will run once "safe to test" is applied or code is pushed.
+              // 2. No relevant paths changed — nothing to test.
+              const eventAction = context.payload.action;
+              const labelName   = context.payload.label?.name ?? '';
+              if (eventAction === 'labeled' && labelName !== 'safe to test') {
+                console.log(`Playwright E2E tests pending — label "${labelName}" added but only "safe to test" triggers E2E. Tests will run when "safe to test" is applied or code is pushed.`);
+              } else {
+                console.log('Playwright E2E tests not required for this PR (no relevant paths changed).');
+              }
+              return;
+            }
+            if (upstreamResult === 'cancelled') {
+              core.setFailed('Playwright E2E tests were cancelled.');
+              return;
+            }
+
+            const fs = require('fs');
+            const path = require('path');
+
+            const runId = '${{ github.run_id }}';
+            const repo = context.repo;
+            const prNumber = context.payload.pull_request?.number;
+
+            const artifactUrl = `https://github.com/${repo.owner}/${repo.repo}/actions/runs/${runId}`;
+            const commentMarker = '<!-- playwright-summary -->';
+
+            // Collect results from all shards
+            const shardResults = [];
+            const resultsDir = 'results';
+            if (fs.existsSync(resultsDir)) {
+              for (const dir of fs.readdirSync(resultsDir).sort()) {
+                const jsonPath = path.join(resultsDir, dir, 'results.json');
+                if (!fs.existsSync(jsonPath)) continue;
+
+                const shardNum = dir.replace('playwright-results-json-', '');
+                const report = JSON.parse(fs.readFileSync(jsonPath, 'utf8'));
+
+                const allTests = [];
+                function collectTests(suite, filePath) {
+                  const file = suite.file || filePath || '';
+                  for (const spec of (suite.specs || [])) {
+                    for (const test of (spec.tests || [])) {
+                      const results = test.results || [];
+                      const lastResult = results[results.length - 1] || {};
+                      const firstResult = results[0] || {};
+                      allTests.push({
+                        title: spec.title,
+                        file: file,
+                        status: test.status,
+                        retries: results.length - 1,
+                        error: lastResult.error?.message || firstResult.error?.message || '',
+                      });
+                    }
+                  }
+                  for (const child of (suite.suites || [])) {
+                    collectTests(child, file);
+                  }
+                }
+                for (const suite of (report.suites || [])) {
+                  collectTests(suite, '');
+                }
+
+                shardResults.push({
+                  shard: shardNum,
+                  genuine: allTests.filter(t => t.status === 'unexpected'),
+                  flaky: allTests.filter(t => t.status === 'flaky'),
+                  passed: allTests.filter(t => t.status === 'expected'),
+                  skipped: allTests.filter(t => t.status === 'skipped'),
+                });
+              }
+            }
+
+            if (shardResults.length === 0) {
+              core.setFailed('No Playwright results found — shards may have been cancelled or failed to upload artifacts.');
+              return;
+            }
+
+            // Aggregate totals
+            const totalPassed = shardResults.reduce((s, r) => s + r.passed.length, 0);
+            const totalFailed = shardResults.reduce((s, r) => s + r.genuine.length, 0);
+            const totalFlaky = shardResults.reduce((s, r) => s + r.flaky.length, 0);
+            const totalSkipped = shardResults.reduce((s, r) => s + r.skipped.length, 0);
+
+            const lines = [commentMarker];
+
+            if (totalFailed > 0) {
+              lines.push(`## 🔴 Playwright Results — ${totalFailed} failure(s)${totalFlaky > 0 ? `, ${totalFlaky} flaky` : ''}`);
+            } else if (totalFlaky > 0) {
+              lines.push(`## 🟡 Playwright Results — all passed (${totalFlaky} flaky)`);
+            } else {
+              lines.push(`## ✅ Playwright Results — all ${totalPassed} tests passed`);
+            }
+            lines.push('');
+            lines.push(`✅ ${totalPassed} passed · ❌ ${totalFailed} failed · 🟡 ${totalFlaky} flaky · ⏭️ ${totalSkipped} skipped`);
+            lines.push('');
+
+            // Per-shard summary table
+            lines.push('| Shard | Passed | Failed | Flaky | Skipped |');
+            lines.push('|-------|--------|--------|-------|---------|');
+            for (const r of shardResults) {
+              const status = r.genuine.length > 0 ? '🔴' : r.flaky.length > 0 ? '🟡' : '✅';
+              lines.push(`| ${status} Shard ${r.shard} | ${r.passed.length} | ${r.genuine.length} | ${r.flaky.length} | ${r.skipped.length} |`);
+            }
+            lines.push('');
+
+            // Genuine failures detail
+            const allGenuine = shardResults.flatMap(r => r.genuine.map(t => ({ ...t, shard: r.shard })));
+            if (allGenuine.length > 0) {
+              lines.push('### Genuine Failures (failed on all attempts)');
+              lines.push('');
+              for (const t of allGenuine.slice(0, 30)) {
+                const shortFile = t.file.replace(/.*playwright\/e2e\//, '');
+                lines.push(`<details><summary>❌ <code>${shortFile}</code> › ${t.title} (shard ${t.shard})</summary>`);
+                lines.push('');
+                lines.push('```');
+                lines.push(t.error.substring(0, 1000));
+                lines.push('```');
+                lines.push('</details>');
+                lines.push('');
+              }
+              if (allGenuine.length > 30) {
+                lines.push(`... and ${allGenuine.length - 30} more failures`);
+                lines.push('');
+              }
+            }
+
+            // Flaky tests
+            const allFlaky = shardResults.flatMap(r => r.flaky.map(t => ({ ...t, shard: r.shard })));
+            if (allFlaky.length > 0) {
+              lines.push(`<details><summary>🟡 ${allFlaky.length} flaky test(s) (passed on retry)</summary>`);
+              lines.push('');
+              for (const t of allFlaky.slice(0, 30)) {
+                const shortFile = t.file.replace(/.*playwright\/e2e\//, '');
+                lines.push(`- \`${shortFile}\` › ${t.title} (shard ${t.shard}, ${t.retries} ${t.retries === 1 ? 'retry' : 'retries'})`);
+              }
+              if (allFlaky.length > 30) {
+                lines.push(`- ... and ${allFlaky.length - 30} more`);
+              }
+              lines.push('');
+              lines.push('</details>');
+              lines.push('');
+            }
+
+            lines.push(`📦 [Download artifacts](${artifactUrl})`);
+            lines.push('');
+            lines.push('<details><summary>How to debug locally</summary>');
+            lines.push('');
+            lines.push('```bash');
+            lines.push('# Download playwright-test-results-<shard> artifact and unzip');
+            lines.push('npx playwright show-trace path/to/trace.zip    # view trace');
+            lines.push('```');
+            lines.push('</details>');
+
+            const body = lines.join('\n');
+
+            // Post PR comment only for pull_request_target events
+            if (prNumber) {
+              let existingComment = null;
+              for await (const response of github.paginate.iterator(
+                github.rest.issues.listComments, { ...repo, issue_number: prNumber, per_page: 100 }
+              )) {
+                const found = response.data.find(c =>
+                  c.user?.login === 'github-actions[bot]' && c.body?.includes(commentMarker)
+                );
+                if (found) { existingComment = found; break; }
+              }
+
+              // Also clean up any old per-shard comments from previous runs
+              for await (const response of github.paginate.iterator(
+                github.rest.issues.listComments, { ...repo, issue_number: prNumber, per_page: 100 }
+              )) {
+                for (const c of response.data) {
+                  if (c.user?.login === 'github-actions[bot]' && /Playwright Shard \d/.test(c.body || '') && !c.body?.includes(commentMarker)) {
+                    await github.rest.issues.deleteComment({ ...repo, comment_id: c.id });
+                    console.log(`Deleted old per-shard comment ${c.id}`);
+                  }
+                }
+              }
+
+              if (existingComment) {
+                await github.rest.issues.updateComment({ ...repo, comment_id: existingComment.id, body });
+              } else {
+                await github.rest.issues.createComment({ ...repo, issue_number: prNumber, body });
+              }
+            }
+
+            // Gate: fail the step (and therefore the job) when genuine test failures exist.
+            // This is the single source of truth — the same parsed results that drive the
+            // PR comment also determine whether playwright-summary passes or fails.
+            if (totalFailed > 0) {
+              core.setFailed(`${totalFailed} Playwright test(s) failed.`);
+            }

.github/workflows/playwright-search-nightly.yml

@@ -0,0 +1,104 @@
+#  Copyright 2026 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: playwright-search-nightly
+
+on:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: playwright-search-nightly-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  playwright-search-nightly:
+    runs-on: ubuntu-latest
+    environment: test
+    timeout-minutes: 45
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Cache Maven Dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Setup OpenMetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: '3.10'
+          args: '-d postgresql -i false'
+          ingestion_dependency: 'all'
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: 'openmetadata-ui/src/main/resources/ui/.nvmrc'
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+
+      - name: Install Playwright Browsers
+        run: npx playwright@1.57.0 install chromium --with-deps
+
+      - name: Run Search Nightly
+        working-directory: openmetadata-ui/src/main/resources/ui
+        env:
+          PLAYWRIGHT_IS_OSS: true
+        run: |
+          # All search tests live in playwright/e2e/Search/. The search-nightly
+          # project in playwright.config.ts maps testMatch to **/Search/** so only
+          # that folder is picked up. Add new search specs to that folder.
+          npx playwright test --project=search-nightly --workers=1
+
+      - name: Upload HTML report
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: search-nightly-html-report
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/playwright-report
+          retention-days: 5
+
+      - name: Send Slack Notification
+        if: always()
+        working-directory: openmetadata-ui/src/main/resources/ui
+        env:
+          RUN_TITLE: "Playwright Search Nightly (${{ github.ref_name }})"
+          RUN_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+          SLACK_BOT_USER_OAUTH_TOKEN: ${{ secrets.E2E_SLACK_BOT_OAUTH_TOKEN }}
+        run: |
+          npx playwright-slack-report -c playwright/slack-cli.config.json -j playwright/output/results.json > slack_report.json
+
+      - name: Clean Up
+        if: always()
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume

.github/workflows/playwright-sso-login-nightly.yml

@@ -0,0 +1,155 @@
+#  Copyright 2025 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: SSO Login Nightly
+
+on:
+  schedule:
+    - cron: '0 3 * * *'
+  workflow_dispatch:
+    inputs:
+      sso_provider:
+        description: 'SSO provider (or "all")'
+        required: true
+        default: okta
+        type: choice
+        options:
+          - okta
+          - keycloak-azure-saml
+          - keycloak-azure-saml-crosssite
+          - all
+      send_slack_notification:
+        description: "Send Slack notification with test results"
+        required: false
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+
+concurrency:
+  group: sso-login-nightly-${{ github.event.inputs.sso_provider || 'scheduled' }}
+  cancel-in-progress: true
+
+jobs:
+  # To onboard a new provider:
+  #   1. Add a matrix entry below (`name` is the lowercase provider id used by
+  #      the Playwright helper; `env_prefix` is the uppercase/underscore form
+  #      used to look up credentials). Also add `name` to the dispatch
+  #      `options:` list above.
+  #   2. Add <ENV_PREFIX>_SSO_USERNAME (variable) and <ENV_PREFIX>_SSO_PASSWORD
+  #      (variable) to the `test` environment. Use a secret instead of a
+  #      variable for the password if the provider uses a real (non-fixture)
+  #      credential.
+  #   3. Register the helper in playwright/utils/sso-providers/index.ts.
+  sso-login:
+    runs-on: ubuntu-latest
+    environment: test
+    timeout-minutes: 45
+    strategy:
+      fail-fast: false
+      matrix:
+        provider:
+          ${{ (github.event_name == 'schedule' || github.event.inputs.sso_provider == 'all')
+          && fromJSON('[{"name":"okta","env_prefix":"OKTA"},{"name":"keycloak-azure-saml","env_prefix":"KEYCLOAK_AZURE_SAML"},{"name":"keycloak-azure-saml-crosssite","env_prefix":"KEYCLOAK_AZURE_SAML"}]')
+          || (github.event.inputs.sso_provider == 'keycloak-azure-saml'
+              && fromJSON('[{"name":"keycloak-azure-saml","env_prefix":"KEYCLOAK_AZURE_SAML"}]')
+              || (github.event.inputs.sso_provider == 'keycloak-azure-saml-crosssite'
+                  && fromJSON('[{"name":"keycloak-azure-saml-crosssite","env_prefix":"KEYCLOAK_AZURE_SAML"}]')
+                  || fromJSON('[{"name":"okta","env_prefix":"OKTA"}]'))) }}
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Cache Maven Dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Setup OpenMetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: '3.10'
+          args: '-d postgresql -i false'
+          ingestion_dependency: 'all'
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: 'openmetadata-ui/src/main/resources/ui/.nvmrc'
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+
+      - name: Install Playwright Browsers
+        run: npx playwright@1.57.0 install chromium --with-deps
+
+      - name: Start Keycloak SAML IdP
+        if: startsWith(matrix.provider.name, 'keycloak-')
+        run: |
+          docker compose -f docker/local-sso/keycloak-saml/docker-compose.yml up -d
+          timeout 180 bash -c 'until curl -fsS http://localhost:8080/realms/om-azure-saml >/dev/null; do sleep 2; done'
+
+      - name: Run SSO Login Spec
+        working-directory: openmetadata-ui/src/main/resources/ui
+        env:
+          SSO_PROVIDER_TYPE: ${{ matrix.provider.name }}
+          SSO_USERNAME: ${{ vars[format('{0}_SSO_USERNAME', matrix.provider.env_prefix)] }}
+          SSO_PASSWORD: ${{ vars[format('{0}_SSO_PASSWORD', matrix.provider.env_prefix)] || secrets[format('{0}_SSO_PASSWORD', matrix.provider.env_prefix)] }}
+          # The '-crosssite' variant fronts the IdP on 127.0.0.1 while OM stays on localhost. The two
+          # are a same registrable-domain pair but a *different site* (SameSite ignores port; 127.0.0.1
+          # and localhost are distinct sites), so the SAML callback POST is cross-site and the browser
+          # drops the SameSite=Lax OM_SESSION cookie. This reproduces the production "No pending session"
+          # failure that the localhost-only job cannot, guarding the SAML RelayState fix.
+          KEYCLOAK_SAML_BASE_URL: ${{ matrix.provider.name == 'keycloak-azure-saml-crosssite' && 'http://127.0.0.1:8080' || 'http://localhost:8080' }}
+          PLAYWRIGHT_IS_OSS: true
+        run: npx playwright test --project=sso-auth --workers=1
+
+      - name: Upload HTML report
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: sso-login-html-report-${{ matrix.provider.name }}
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/playwright-report
+          retention-days: 5
+
+      - name: Send Slack Notification
+        if: ${{ always() && (github.event_name == 'schedule' || inputs.send_slack_notification == true) }}
+        working-directory: openmetadata-ui/src/main/resources/ui
+        env:
+          RUN_TITLE: "SSO Login Nightly: ${{ matrix.provider.name }} (${{ github.ref_name }})"
+          RUN_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+          SLACK_BOT_USER_OAUTH_TOKEN: ${{ secrets.E2E_SLACK_BOT_OAUTH_TOKEN }}
+        run: |
+          npx playwright-slack-report -c playwright/slack-cli.config.json -j playwright/output/results.json > slack_report.json
+
+      - name: Clean Up
+        if: always()
+        run: |
+          docker compose -f docker/local-sso/keycloak-saml/docker-compose.yml down --remove-orphans || true
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume

.github/workflows/playwright-sso-tests.yml

@@ -0,0 +1,357 @@
+#  Copyright 2025 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# This workflow executes SSO-specific end-to-end (e2e) tests using Playwright with PostgreSQL as the database.
+#
+# Purpose:
+#   - Run SSO token renewal E2E tests using a mock OIDC provider on PRs
+#   - Run SSO configuration tests for various providers (Google, Azure AD, Okta, etc.) via manual trigger
+#   - Tests run in isolation from the regular sharded Playwright E2E tests
+#
+# Triggers:
+#   - Pull requests with "safe to test" label (mock-oidc only, Chromium + WebKit)
+#   - Manual trigger via workflow_dispatch (any SSO provider, Chromium + WebKit)
+#
+# Test Location:
+#   - openmetadata-ui/src/main/resources/ui/playwright/e2e/Auth/SSOAuthentication.spec.ts (mock-oidc)
+
+name: SSO Playwright E2E Tests
+
+on:
+  pull_request_target:
+    types:
+      - labeled
+      - opened
+      - synchronize
+      - reopened
+      - ready_for_review
+    paths:
+      - "openmetadata-ui/src/main/resources/ui/src/components/Auth/**"
+      - "openmetadata-ui/src/main/resources/ui/src/rest/auth-API*"
+      - "openmetadata-ui/src/main/resources/ui/src/utils/AuthProvider*"
+      - "openmetadata-ui/src/main/resources/ui/src/utils/TokenServiceUtil*"
+      - "openmetadata-ui/src/main/resources/ui/src/utils/UserDataUtils*"
+      - "openmetadata-ui/src/main/resources/ui/src/hooks/useSignIn*"
+      - "openmetadata-ui/src/main/resources/ui/playwright/e2e/Auth/**"
+      - "openmetadata-ui/src/main/resources/ui/playwright/utils/mockOidc*"
+      - "openmetadata-ui/src/main/resources/ui/playwright.sso.config.ts"
+      - "docker/development/mock-oidc-provider/**"
+      - "docker/development/docker-compose.yml"
+      - "docker/development/.env.sso-test"
+      - ".github/workflows/playwright-sso-tests.yml"
+  workflow_dispatch:
+    inputs:
+      sso_provider:
+        description: "SSO Provider to test"
+        required: true
+        default: "mock-oidc"
+        type: choice
+        options:
+          - mock-oidc
+          - google
+          - okta
+          - azure
+          - auth0
+          - saml
+          - cognito
+
+permissions:
+  contents: read
+  pull-requests: write
+
+concurrency:
+  group: sso-playwright-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  sso-auth-tests:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    environment: test
+
+    strategy:
+      fail-fast: false
+      matrix:
+        provider: ${{ github.event_name == 'pull_request_target' && fromJSON('["mock-oidc"]') || github.event.inputs.sso_provider && fromJSON(format('["{0}"]', github.event.inputs.sso_provider)) || fromJSON('["mock-oidc"]') }}
+
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+
+      - name: Cache Maven Dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Start Mock OIDC Provider
+        if: ${{ matrix.provider == 'mock-oidc' }}
+        run: |
+          cd docker/development
+          docker compose --profile sso-test build mock-oidc-provider
+          docker compose --profile sso-test up -d mock-oidc-provider
+          echo "Waiting for mock OIDC provider to be healthy..."
+          for i in $(seq 1 30); do
+            if curl -sf http://localhost:9090/health > /dev/null 2>&1; then
+              echo "Mock OIDC provider is ready"
+              break
+            fi
+            echo "Waiting... ($i/30)"
+            sleep 2
+          done
+          curl -sf http://localhost:9090/.well-known/openid-configuration || echo "WARNING: Discovery endpoint not ready"
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d postgresql -i false"
+          ingestion_dependency: "all"
+        env:
+          AUTHENTICATION_PROVIDER: ${{ matrix.provider == 'mock-oidc' && 'custom-oidc' || 'basic' }}
+          CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME: ${{ matrix.provider == 'mock-oidc' && 'mock-oidc' || '' }}
+          AUTHENTICATION_PUBLIC_KEYS: ${{ matrix.provider == 'mock-oidc' && '[http://localhost:9090/jwks,http://localhost:8585/api/v1/system/config/jwks]' || '[http://localhost:8585/api/v1/system/config/jwks]' }}
+          AUTHENTICATION_AUTHORITY: ${{ matrix.provider == 'mock-oidc' && 'http://localhost:9090' || 'https://accounts.google.com' }}
+          AUTHENTICATION_CLIENT_ID: ${{ matrix.provider == 'mock-oidc' && 'openmetadata-test' || '' }}
+          AUTHENTICATION_CALLBACK_URL: ${{ matrix.provider == 'mock-oidc' && 'http://localhost:8585/callback' || '' }}
+          AUTHENTICATION_JWT_PRINCIPAL_CLAIMS_MAPPING: ${{ matrix.provider == 'mock-oidc' && '[username:sub,email:email]' || '' }}
+          OIDC_CLIENT_ID: ${{ matrix.provider == 'mock-oidc' && 'openmetadata-test' || '' }}
+          OIDC_TYPE: ${{ matrix.provider == 'mock-oidc' && 'custom-oidc' || '' }}
+          OIDC_CLIENT_SECRET: ${{ matrix.provider == 'mock-oidc' && 'openmetadata-test-secret' || '' }}
+          OIDC_DISCOVERY_URI: ${{ matrix.provider == 'mock-oidc' && 'http://localhost:9090/.well-known/openid-configuration' || '' }}
+          OIDC_CALLBACK: ${{ matrix.provider == 'mock-oidc' && 'http://localhost:8585/callback' || 'http://localhost:8585/callback' }}
+          OIDC_SERVER_URL: ${{ matrix.provider == 'mock-oidc' && 'http://localhost:8585' || 'http://localhost:8585' }}
+          AUTHENTICATION_CLIENT_TYPE: ${{ matrix.provider == 'mock-oidc' && 'confidential' || 'public' }}
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: 'openmetadata-ui/src/main/resources/ui/.nvmrc'
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+
+      - name: Install Playwright Browsers
+        run: npx playwright@1.51.1 install chromium webkit --with-deps
+
+      - name: Run Mock OIDC Authentication Tests
+        if: ${{ matrix.provider == 'mock-oidc' }}
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: npx playwright test --config=playwright.sso.config.ts --workers=1
+        env:
+          MOCK_OIDC_URL: http://localhost:9090
+          PLAYWRIGHT_IS_OSS: true
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        timeout-minutes: 60
+
+      - name: Run SSO Authentication Tests
+        if: ${{ matrix.provider != 'mock-oidc' }}
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: npx playwright test --config=playwright.sso.config.ts --workers=1
+        env:
+          SSO_PROVIDER_TYPE: ${{ matrix.provider }}
+          SSO_USERNAME: ${{ secrets[format('{0}_SSO_USERNAME', upper(matrix.provider))] }}
+          SSO_PASSWORD: ${{ secrets[format('{0}_SSO_PASSWORD', upper(matrix.provider))] }}
+          PLAYWRIGHT_IS_OSS: true
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        timeout-minutes: 60
+
+      - name: Upload test results
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: sso-auth-test-results-${{ matrix.provider }}
+          path: |
+            openmetadata-ui/src/main/resources/ui/playwright/output/sso-playwright-report
+            openmetadata-ui/src/main/resources/ui/playwright/output/sso-test-results
+          retention-days: 5
+
+      - name: Upload results JSON for summary
+        uses: actions/upload-artifact@v4
+        if: ${{ !cancelled() }}
+        with:
+          name: sso-results-json-${{ matrix.provider }}
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/sso-results.json
+          retention-days: 1
+          if-no-files-found: ignore
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose --profile sso-test down --remove-orphans 2>/dev/null || true
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+  sso-summary:
+    if: ${{ !cancelled() && github.event_name == 'pull_request_target' && (github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    needs: sso-auth-tests
+    runs-on: ubuntu-latest
+    steps:
+      - name: Download results JSON
+        uses: actions/download-artifact@v4
+        with:
+          pattern: sso-results-json-*
+          path: results
+
+      - name: Post SSO test summary as PR comment
+        uses: actions/github-script@v7
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            const fs = require('fs');
+            const path = require('path');
+
+            const runId = '${{ github.run_id }}';
+            const repo = context.repo;
+            const prNumber = context.payload.pull_request?.number;
+            if (!prNumber) return;
+
+            const artifactUrl = `https://github.com/${repo.owner}/${repo.repo}/actions/runs/${runId}`;
+            const commentMarker = '<!-- sso-playwright-summary -->';
+
+            const allTests = [];
+            const resultsDir = 'results';
+            if (fs.existsSync(resultsDir)) {
+              for (const dir of fs.readdirSync(resultsDir).sort()) {
+                const jsonPath = path.join(resultsDir, dir, 'sso-results.json');
+                if (!fs.existsSync(jsonPath)) continue;
+
+                const report = JSON.parse(fs.readFileSync(jsonPath, 'utf8'));
+
+                function collectTests(suite, filePath) {
+                  const file = suite.file || filePath || '';
+                  for (const spec of (suite.specs || [])) {
+                    for (const test of (spec.tests || [])) {
+                      const results = test.results || [];
+                      const lastResult = results[results.length - 1] || {};
+                      const firstResult = results[0] || {};
+                      allTests.push({
+                        title: spec.title,
+                        project: test.projectName || '',
+                        file: file,
+                        status: test.status,
+                        retries: results.length - 1,
+                        error: lastResult.error?.message || firstResult.error?.message || '',
+                      });
+                    }
+                  }
+                  for (const child of (suite.suites || [])) {
+                    collectTests(child, file);
+                  }
+                }
+                for (const suite of (report.suites || [])) {
+                  collectTests(suite, '');
+                }
+              }
+            }
+
+            if (allTests.length === 0) {
+              console.log('No SSO test results found');
+              return;
+            }
+
+            const passed = allTests.filter(t => t.status === 'expected');
+            const failed = allTests.filter(t => t.status === 'unexpected');
+            const flaky = allTests.filter(t => t.status === 'flaky');
+            const skipped = allTests.filter(t => t.status === 'skipped');
+
+            const lines = [commentMarker];
+
+            if (failed.length > 0) {
+              lines.push(`## 🔴 SSO Playwright Results — ${failed.length} failure(s)${flaky.length > 0 ? `, ${flaky.length} flaky` : ''}`);
+            } else if (flaky.length > 0) {
+              lines.push(`## 🟡 SSO Playwright Results — all passed (${flaky.length} flaky)`);
+            } else {
+              lines.push(`## ✅ SSO Playwright Results — all ${passed.length} tests passed`);
+            }
+            lines.push('');
+            lines.push(`✅ ${passed.length} passed · ❌ ${failed.length} failed · 🟡 ${flaky.length} flaky · ⏭️ ${skipped.length} skipped`);
+            lines.push('');
+
+            if (failed.length > 0) {
+              lines.push('### Failures');
+              lines.push('');
+              for (const t of failed.slice(0, 20)) {
+                lines.push(`<details><summary>❌ ${t.project} › ${t.title}</summary>`);
+                lines.push('');
+                lines.push('```');
+                lines.push(t.error.substring(0, 1000));
+                lines.push('```');
+                lines.push('</details>');
+                lines.push('');
+              }
+            }
+
+            if (flaky.length > 0) {
+              lines.push(`<details><summary>🟡 ${flaky.length} flaky test(s)</summary>`);
+              lines.push('');
+              for (const t of flaky.slice(0, 20)) {
+                lines.push(`- ${t.project} › ${t.title} (${t.retries} ${t.retries === 1 ? 'retry' : 'retries'})`);
+              }
+              lines.push('');
+              lines.push('</details>');
+              lines.push('');
+            }
+
+            lines.push(`📦 [View run details](${artifactUrl})`);
+
+            const body = lines.join('\n');
+
+            let existingComment = null;
+            for await (const response of github.paginate.iterator(
+              github.rest.issues.listComments, { ...repo, issue_number: prNumber, per_page: 100 }
+            )) {
+              const found = response.data.find(c =>
+                c.user?.login === 'github-actions[bot]' && c.body?.includes(commentMarker)
+              );
+              if (found) { existingComment = found; break; }
+            }
+
+            if (existingComment) {
+              await github.rest.issues.updateComment({ ...repo, comment_id: existingComment.id, body });
+            } else {
+              await github.rest.issues.createComment({ ...repo, issue_number: prNumber, body });
+            }

.github/workflows/postgresql-nightly-e2e.yml

@@ -0,0 +1,234 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# This workflow executes end-to-end (e2e) tests using Playwright with PostgreSQL as the database.
+# For more information see: https://github.com/actions/setup-java/blob/main/docs/advanced-usage.md#apache-maven-with-a-settings-path
+
+name: Postgresql Playwright E2E Tests
+on:
+  workflow_dispatch:
+    inputs:
+      branch:
+        description: "Branch to run tests on"
+        required: true
+        type: string
+        default: "main"
+      send_slack_notification:
+        description: "Send Slack notification with test results"
+        required: false
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+
+concurrency:
+  group: pw-ci-postgresql-branch-${{ inputs.branch }}
+  cancel-in-progress: true
+
+jobs:
+  pw-ci-postgresql-branch:
+    runs-on: ubuntu-latest
+    environment: test
+    env:
+      # Playwright logs the admin user in many times (performAdminLogin per test, parallel
+      # workers, retries). The production default of 5 active sessions per user would evict the
+      # long-lived storageState session the page fixtures rely on and 401 every request. Raise
+      # the cap for E2E only; docker compose reads this for ${AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER:-5}.
+      AUTHENTICATION_MAX_ACTIVE_SESSIONS_PER_USER: "10000"
+    strategy:
+      fail-fast: false
+      matrix:
+        shardIndex: [1, 2, 3, 4, 5, 6, 7]
+        shardTotal: [7]
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.branch }}
+
+      - name: Cache Maven Dependencies
+        id: cache-output
+        uses: actions/cache@v4
+        with:
+          path: ~/.m2
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-d postgresql"
+          ingestion_dependency: "playwright"
+
+      - name: Install dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: yarn --ignore-scripts --frozen-lockfile
+      - name: Install Playwright Browsers
+        run: npx playwright@1.57.0 install chromium --with-deps
+      - name: Run Playwright tests
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: |
+          if [ "${{ matrix.shardIndex }}" -eq "1" ]; then
+            echo "🔹 Running DataAssetRules-only tests on shard 1"
+
+            # The testMatch pattern ensures only DataAssetRules*.spec.ts files run
+            npx playwright test \
+              --project=setup \
+              --project=DataAssetRulesEnabled \
+              --project=DataAssetRulesDisabled
+
+          elif [ "${{ matrix.shardIndex }}" -eq "2" ]; then
+            echo "🔹 Running search relevance tests on shard 2"
+
+            npx playwright test \
+              --project=search-nightly \
+              playwright/e2e/Search/SearchRelevance.spec.ts \
+              --workers=1
+
+          else
+            # Shards 3-7 handle chromium tests (5 shards total)
+            CHROMIUM_SHARD=$(( ${{ matrix.shardIndex }} - 2 ))
+            echo "🔹 Running all tests (excluding DataAssetRules) on chromium shard ${CHROMIUM_SHARD}/5"
+            npx playwright test \
+              --project=chromium \
+              --grep-invert @dataAssetRules \
+              --shard=${CHROMIUM_SHARD}/5
+          fi
+
+        env:
+          PLAYWRIGHT_IS_OSS: true
+          PLAYWRIGHT_SNOWFLAKE_USERNAME: ${{ secrets.TEST_SNOWFLAKE_USERNAME }}
+          PLAYWRIGHT_SNOWFLAKE_PASSWORD: ${{ secrets.TEST_SNOWFLAKE_PASSWORD }}
+          PLAYWRIGHT_SNOWFLAKE_ACCOUNT: ${{ secrets.TEST_SNOWFLAKE_ACCOUNT }}
+          PLAYWRIGHT_SNOWFLAKE_DATABASE: ${{ secrets.TEST_SNOWFLAKE_DATABASE }}
+          PLAYWRIGHT_SNOWFLAKE_WAREHOUSE: ${{ secrets.TEST_SNOWFLAKE_WAREHOUSE }}
+          PLAYWRIGHT_PROJECT_ID: ${{ steps.cypress-project-id.outputs.CYPRESS_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY: ${{ secrets.TEST_BQ_PRIVATE_KEY }}
+          PLAYWRIGHT_BQ_PROJECT_ID: ${{ secrets.PLAYWRIGHT_BQ_PROJECT_ID }}
+          PLAYWRIGHT_BQ_PRIVATE_KEY_ID: ${{ secrets.TEST_BQ_PRIVATE_KEY_ID }}
+          PLAYWRIGHT_BQ_PROJECT_ID_TAXONOMY: ${{ secrets.TEST_BQ_PROJECT_ID_TAXONOMY }}
+          PLAYWRIGHT_BQ_CLIENT_EMAIL: ${{ secrets.TEST_BQ_CLIENT_EMAIL }}
+          PLAYWRIGHT_BQ_CLIENT_ID: ${{ secrets.TEST_BQ_CLIENT_ID }}
+          PLAYWRIGHT_REDSHIFT_HOST: ${{ secrets.E2E_REDSHIFT_HOST_PORT }}
+          PLAYWRIGHT_REDSHIFT_USERNAME: ${{ secrets.E2E_REDSHIFT_USERNAME }}
+          PLAYWRIGHT_REDSHIFT_PASSWORD: ${{ secrets.E2E_REDSHIFT_PASSWORD }}
+          PLAYWRIGHT_REDSHIFT_DATABASE: ${{ secrets.TEST_REDSHIFT_DATABASE }}
+          PLAYWRIGHT_METABASE_USERNAME: ${{ secrets.TEST_METABASE_USERNAME }}
+          PLAYWRIGHT_METABASE_PASSWORD: ${{ secrets.TEST_METABASE_PASSWORD }}
+          PLAYWRIGHT_METABASE_DB_SERVICE_NAME: ${{ secrets.TEST_METABASE_DB_SERVICE_NAME }}
+          PLAYWRIGHT_METABASE_HOST_PORT: ${{ secrets.TEST_METABASE_HOST_PORT }}
+          PLAYWRIGHT_SUPERSET_USERNAME: ${{ secrets.TEST_SUPERSET_USERNAME }}
+          PLAYWRIGHT_SUPERSET_PASSWORD: ${{ secrets.TEST_SUPERSET_PASSWORD }}
+          PLAYWRIGHT_SUPERSET_HOST_PORT: ${{ secrets.TEST_SUPERSET_HOST_PORT }}
+          PLAYWRIGHT_KAFKA_BOOTSTRAP_SERVERS: ${{ secrets.TEST_KAFKA_BOOTSTRAP_SERVERS }}
+          PLAYWRIGHT_KAFKA_SCHEMA_REGISTRY_URL: ${{ secrets.TEST_KAFKA_SCHEMA_REGISTRY_URL }}
+          PLAYWRIGHT_GLUE_ACCESS_KEY: ${{ secrets.TEST_GLUE_ACCESS_KEY }}
+          PLAYWRIGHT_GLUE_SECRET_KEY: ${{ secrets.TEST_GLUE_SECRET_KEY }}
+          PLAYWRIGHT_GLUE_AWS_REGION: ${{ secrets.TEST_GLUE_AWS_REGION }}
+          PLAYWRIGHT_GLUE_ENDPOINT: ${{ secrets.TEST_GLUE_ENDPOINT }}
+          PLAYWRIGHT_GLUE_STORAGE_SERVICE: ${{ secrets.TEST_GLUE_STORAGE_SERVICE }}
+          PLAYWRIGHT_MYSQL_USERNAME: ${{ secrets.TEST_MYSQL_USERNAME }}
+          PLAYWRIGHT_MYSQL_PASSWORD: ${{ secrets.TEST_MYSQL_PASSWORD }}
+          PLAYWRIGHT_MYSQL_HOST_PORT: ${{ secrets.TEST_MYSQL_HOST_PORT }}
+          PLAYWRIGHT_MYSQL_DATABASE_SCHEMA: ${{ secrets.TEST_MYSQL_DATABASE_SCHEMA }}
+          PLAYWRIGHT_POSTGRES_USERNAME: ${{ secrets.TEST_POSTGRES_USERNAME }}
+          PLAYWRIGHT_POSTGRES_PASSWORD: ${{ secrets.TEST_POSTGRES_PASSWORD }}
+          PLAYWRIGHT_POSTGRES_HOST_PORT: ${{ secrets.TEST_POSTGRES_HOST_PORT }}
+          PLAYWRIGHT_POSTGRES_DATABASE: ${{ secrets.TEST_POSTGRES_DATABASE }}
+          PLAYWRIGHT_AIRFLOW_HOST_PORT: ${{ secrets.TEST_AIRFLOW_HOST_PORT }}
+          PLAYWRIGHT_ML_MODEL_TRACKING_URI: ${{ secrets.TEST_ML_MODEL_TRACKING_URI }}
+          PLAYWRIGHT_ML_MODEL_REGISTRY_URI: ${{ secrets.TEST_ML_MODEL_REGISTRY_URI }}
+          PLAYWRIGHT_S3_STORAGE_ACCESS_KEY_ID: ${{ secrets.TEST_S3_STORAGE_ACCESS_KEY_ID }}
+          PLAYWRIGHT_S3_STORAGE_SECRET_ACCESS_KEY: ${{ secrets.TEST_S3_STORAGE_SECRET_ACCESS_KEY }}
+          PLAYWRIGHT_S3_STORAGE_END_POINT_URL: ${{ secrets.TEST_S3_STORAGE_END_POINT_URL }}
+
+          # Recommended: pass the GitHub token lets this action correctly
+          # determine the unique run id necessary to re-run the checks
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Upload blob report to GitHub Actions Artifacts
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: blob-report-${{ matrix.shardIndex }}
+          path: openmetadata-ui/src/main/resources/ui/blob-report
+          retention-days: 1
+
+      - name: Upload HTML report to GitHub Actions Artifacts
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: playwright-HTML-report-${{ matrix.shardIndex }}
+          path: openmetadata-ui/src/main/resources/ui/playwright/output/playwright-report
+          retention-days: 1
+          compression-level: 9
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+  merge-reports:
+    needs: pw-ci-postgresql-branch
+    runs-on: ubuntu-latest
+    if: ${{ !failure() || !cancelled() }}
+    env:
+      # Recommended: pass the GitHub token lets this action correctly
+      # determine the unique run id necessary to re-run the checks
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      SLACK_BOT_USER_OAUTH_TOKEN: ${{ secrets.E2E_SLACK_BOT_OAUTH_TOKEN }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.branch }}
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: 'openmetadata-ui/src/main/resources/ui/.nvmrc'
+
+      - name: Download blob reports from GitHub Actions Artifacts
+        uses: actions/download-artifact@v4
+        with:
+          path: openmetadata-ui/src/main/resources/ui/blob-reports
+          pattern: blob-report-*
+          merge-multiple: true
+
+      - name: Merge Reports
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        run: |
+          npx playwright merge-reports --reporter json ./blob-reports > merged_tests_results.json
+
+      - name: Generate Slack Notification
+        if: ${{ inputs.send_slack_notification }}
+        working-directory: openmetadata-ui/src/main/resources/ui/
+        env:
+          RUN_TITLE: "PostgreSQL Test for OSS Release: ${{ inputs.branch }}"
+          RUN_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+          SLACK_BOT_USER_OAUTH_TOKEN: ${{ secrets.E2E_SLACK_BOT_OAUTH_TOKEN }}
+        run: |
+          npx playwright-slack-report -c playwright/slack-cli.config.json -j merged_tests_results.json > slack_report.json

.github/workflows/pr-metadata-validation.yml

@@ -0,0 +1,424 @@
+#  Copyright 2026 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: PR Metadata Validation
+
+# Fails the PR check unless it links a GitHub issue that (a) has a real
+# description and (b) is tracked in the "Shipping" project with the Status,
+# Source, Priority, Domain and Release fields set.
+#
+# Linked issues are resolved from two sources:
+#   * Same-repo: GitHub's native `closingIssuesReferences` (the PR "Development"
+#     section + same-repo closing keywords). This is authoritative and also
+#     catches issues linked via the sidebar without a body keyword.
+#   * Cross-repo (same org only): parsed from the PR body, because GitHub cannot
+#     represent a cross-repo link in `closingIssuesReferences`. This path is
+#     restricted to trusted authors and an explicit repo allowlist so the
+#     privileged PROJECTS_TOKEN is never pointed at attacker-chosen repositories
+#     under pull_request_target, and the public PR comment never echoes private
+#     issue content (only numbers and field names).
+#
+# Projects v2 data is only available via GraphQL and the default GITHUB_TOKEN
+# cannot read org-level projects (nor private same-org repos), so a PROJECTS_TOKEN
+# secret (a PAT or GitHub App token with `read:project` and read access to the
+# allowlisted repos) is required. Only PR / issue / project metadata is read
+# through the API — no untrusted checkout — so pull_request_target is safe here.
+on:
+  pull_request_target:
+    types: [opened, edited, reopened, synchronize, ready_for_review]
+  workflow_dispatch:
+    inputs:
+      pr_number:
+        description: "PR number to validate"
+        required: true
+        type: number
+
+permissions:
+  contents: read
+  issues: read
+  pull-requests: write
+
+concurrency:
+  # Serialize runs per PR (avoids duplicate sticky comments) but never cancel:
+  # this check gates merges, so every event must reach a definitive conclusion.
+  group: pr-metadata-validation-${{ github.event.pull_request.number || github.event.inputs.pr_number || github.run_id }}
+  cancel-in-progress: false
+
+jobs:
+  validate-pr-metadata:
+    name: Validate PR Metadata
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check linked issue and Shipping project fields
+        uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
+        env:
+          PROJECTS_TOKEN: ${{ secrets.PROJECTS_TOKEN }}
+        with:
+          script: |
+            const CHECK_MARKER = '<!-- pr-metadata-check -->';
+            const BYPASS_LABEL = 'skip-pr-checks';
+
+            const { owner, repo } = context.repo;
+
+            // ---- Policy --------------------------------------------------------
+            const REQUIRE_LINKED_ISSUE = true;
+            const MIN_DESCRIPTION_WORDS = 25;
+            const PROJECT_NAME = 'Shipping';
+            const REQUIRED_PROJECT_FIELDS = ['Status', 'Source', 'Priority', 'Domain', 'Release'];
+            const FAIL_WHEN_PROJECT_UNVERIFIABLE = true;
+
+            // ---- Cross-repo policy (security-sensitive) ------------------------
+            // closingIssuesReferences is same-repo only, so a cross-repo link can
+            // only live in the PR body. Honor it ONLY for trusted authors and ONLY
+            // for these same-org repos, so untrusted fork PRs can never drive the
+            // privileged token at arbitrary repositories or probe private issues.
+            const CLOSING_KEYWORDS = '(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)';
+            const ALLOWED_CROSS_REPOS = new Set(['openmetadata-collate']);
+            const TRUSTED_ASSOCIATIONS = new Set(['OWNER', 'MEMBER', 'COLLABORATOR']);
+
+            const PR_LINKS_QUERY = `
+              query($owner: String!, $repo: String!, $number: Int!) {
+                repository(owner: $owner, name: $repo) {
+                  pullRequest(number: $number) {
+                    closingIssuesReferences(first: 25) {
+                      nodes { number repository { name owner { login } } }
+                    }
+                  }
+                }
+              }`;
+
+            const ISSUE_PROJECT_QUERY = `
+              query($owner: String!, $repo: String!, $number: Int!) {
+                repository(owner: $owner, name: $repo) {
+                  issue(number: $number) {
+                    projectItems(first: 25) {
+                      nodes {
+                        project { title }
+                        fieldValues(first: 50) {
+                          nodes {
+                            __typename
+                            ... on ProjectV2ItemFieldSingleSelectValue { name field { ... on ProjectV2FieldCommon { name } } }
+                            ... on ProjectV2ItemFieldTextValue { text field { ... on ProjectV2FieldCommon { name } } }
+                            ... on ProjectV2ItemFieldNumberValue { number field { ... on ProjectV2FieldCommon { name } } }
+                            ... on ProjectV2ItemFieldIterationValue { title field { ... on ProjectV2FieldCommon { name } } }
+                            ... on ProjectV2ItemFieldDateValue { date field { ... on ProjectV2FieldCommon { name } } }
+                          }
+                        }
+                      }
+                    }
+                  }
+                }
+              }`;
+
+            function elevatedClient() {
+              const token = process.env.PROJECTS_TOKEN;
+              return token ? new github.constructor({ auth: token }) : null;
+            }
+
+            const isSameRepo = (ref) => ref.owner === owner && ref.repo === repo;
+            const refKey = (ref) => `${ref.owner}/${ref.repo}#${ref.number}`;
+            const refLabel = (ref) => (isSameRepo(ref) ? `#${ref.number}` : refKey(ref));
+            const dedupeRefs = (refs) => [...new Map(refs.map((r) => [refKey(r), r])).values()];
+
+            async function loadPullRequest() {
+              if (context.payload.pull_request) {
+                return context.payload.pull_request;
+              }
+              const pull_number = Number(context.payload.inputs.pr_number);
+              const { data } = await github.rest.pulls.get({ owner, repo, pull_number });
+              return data;
+            }
+
+            // Same-repo links: authoritative, includes sidebar-linked issues.
+            async function nativeLinkedRefs(prNumber) {
+              const data = await github.graphql(PR_LINKS_QUERY, { owner, repo, number: prNumber });
+              const nodes =
+                (data.repository &&
+                  data.repository.pullRequest &&
+                  data.repository.pullRequest.closingIssuesReferences.nodes) ||
+                [];
+              return nodes.map((node) => ({
+                owner: node.repository.owner.login,
+                repo: node.repository.name,
+                number: node.number,
+              }));
+            }
+
+            // Trust the author for cross-repo refs if their PR association is already a trusted
+            // value, or — since the default GITHUB_TOKEN cannot see *private* org membership and
+            // reports such authors as CONTRIBUTOR — by confirming org membership with the elevated
+            // token. Fork PRs from non-members still resolve to untrusted.
+            async function isTrustedAuthor(pr) {
+              let trusted = TRUSTED_ASSOCIATIONS.has(pr.author_association);
+              const client = elevatedClient();
+              if (!trusted && client && pr.user && pr.user.login) {
+                try {
+                  await client.rest.orgs.checkMembershipForUser({ org: owner, username: pr.user.login });
+                  trusted = true; // 204 => org member (including private membership)
+                } catch (error) {
+                  trusted = false; // 404 => not a member
+                }
+              }
+              return trusted;
+            }
+
+            // Cross-repo links: body-parsed, allowlisted repos only (caller gates on author trust).
+            function crossRepoRefsFromBody(text) {
+              const refs = [];
+              const add = (refOwner, refRepo, number) => {
+                if (refOwner === owner && ALLOWED_CROSS_REPOS.has(refRepo)) {
+                  refs.push({ owner: refOwner, repo: refRepo, number: Number(number) });
+                }
+              };
+              const shorthand = new RegExp(
+                `\\b${CLOSING_KEYWORDS}\\b\\s*:?\\s*([\\w.-]+)/([\\w.-]+)#(\\d+)`,
+                'gi'
+              );
+              const url = new RegExp(
+                `\\b${CLOSING_KEYWORDS}\\b\\s*:?\\s*https?://github\\.com/([\\w.-]+)/([\\w.-]+)/issues/(\\d+)`,
+                'gi'
+              );
+              let match;
+              while ((match = shorthand.exec(text)) !== null) {
+                add(match[1], match[2], match[3]);
+              }
+              while ((match = url.exec(text)) !== null) {
+                add(match[1], match[2], match[3]);
+              }
+              return refs;
+            }
+
+            function hasProperDescription(body) {
+              if (!body) {
+                return false;
+              }
+              const meaningful = body.replace(/<!--[\s\S]*?-->/g, ' ').replace(/\s+/g, ' ').trim();
+              const wordCount = meaningful ? meaningful.split(' ').filter(Boolean).length : 0;
+              return wordCount >= MIN_DESCRIPTION_WORDS;
+            }
+
+            async function loadIssue(ref) {
+              const client = isSameRepo(ref) ? github : elevatedClient();
+              if (!client) {
+                return { error: 'missing-token' };
+              }
+              try {
+                const { data } = await client.rest.issues.get({
+                  owner: ref.owner,
+                  repo: ref.repo,
+                  issue_number: ref.number,
+                });
+                return { issue: data };
+              } catch (error) {
+                return { error: error.status === 404 ? 'not-found' : error.message };
+              }
+            }
+
+            async function loadIssueProjectItems(ref) {
+              const client = elevatedClient();
+              if (!client) {
+                return { error: 'missing-token' };
+              }
+              try {
+                const data = await client.graphql(ISSUE_PROJECT_QUERY, {
+                  owner: ref.owner,
+                  repo: ref.repo,
+                  number: ref.number,
+                });
+                return { items: data.repository.issue.projectItems.nodes };
+              } catch (error) {
+                return { error: error.message };
+              }
+            }
+
+            function resolveFieldValue(fieldValue) {
+              const candidates = [fieldValue.name, fieldValue.text, fieldValue.title, fieldValue.date];
+              const text = candidates.find((value) => value != null && String(value).trim() !== '');
+              if (text != null) {
+                return text;
+              }
+              return fieldValue.number != null ? String(fieldValue.number) : null;
+            }
+
+            function collectSetFields(item) {
+              const setFields = new Map();
+              for (const fieldValue of item.fieldValues.nodes) {
+                const fieldName = fieldValue.field && fieldValue.field.name;
+                const resolved = resolveFieldValue(fieldValue);
+                if (fieldName && resolved != null) {
+                  setFields.set(fieldName, resolved);
+                }
+              }
+              return setFields;
+            }
+
+            async function collectProjectProblems(ref) {
+              const result = await loadIssueProjectItems(ref);
+              if (result.error) {
+                if (!FAIL_WHEN_PROJECT_UNVERIFIABLE) {
+                  core.warning(`Skipping project validation for ${refLabel(ref)}: ${result.error}`);
+                  return [];
+                }
+                core.warning(`Project verification failed for ${refLabel(ref)}: ${result.error}`);
+                const detail = result.error === 'missing-token'
+                  ? 'the `PROJECTS_TOKEN` secret (a token with `read:project` and access to the linked repo) is not configured'
+                  : 'the project query could not be completed (see the workflow run logs)';
+                return [`Could not verify the **${PROJECT_NAME}** project on issue ${refLabel(ref)}: ${detail}.`];
+              }
+              const projectItem = (result.items || []).find(
+                (item) => item.project && item.project.title === PROJECT_NAME
+              );
+              if (!projectItem) {
+                return [`Linked issue ${refLabel(ref)} is not added to the **${PROJECT_NAME}** project.`];
+              }
+              const setFields = collectSetFields(projectItem);
+              const problems = [];
+              for (const field of REQUIRED_PROJECT_FIELDS) {
+                if (!setFields.has(field)) {
+                  problems.push(`Issue ${refLabel(ref)} is missing **${PROJECT_NAME} → ${field}**.`);
+                }
+              }
+              return problems;
+            }
+
+            async function validateIssue(ref) {
+              const loaded = await loadIssue(ref);
+              if (loaded.error === 'not-found') {
+                return { ref, problems: [`Linked issue ${refLabel(ref)} does not exist or is not accessible.`] };
+              }
+              if (loaded.error) {
+                const detail = loaded.error === 'missing-token'
+                  ? 'the `PROJECTS_TOKEN` secret is not configured'
+                  : 'the issue lookup could not be completed (see the workflow run logs)';
+                core.warning(`Issue lookup failed for ${refLabel(ref)}: ${loaded.error}`);
+                return { ref, problems: [`Could not read linked issue ${refLabel(ref)}: ${detail}.`] };
+              }
+              const issue = loaded.issue;
+              if (issue.pull_request) {
+                return { ref, problems: [`${refLabel(ref)} is a pull request, not an issue.`] };
+              }
+              const problems = [];
+              if (!hasProperDescription(issue.body)) {
+                problems.push(
+                  `Linked issue ${refLabel(ref)} needs a real description ` +
+                  `(at least ${MIN_DESCRIPTION_WORDS} words covering the problem, repro, and expected behavior).`
+                );
+              }
+              problems.push(...(await collectProjectProblems(ref)));
+              return { ref, problems };
+            }
+
+            async function collectIssueProblems(pr) {
+              const native = await nativeLinkedRefs(pr.number);
+              const cross = (await isTrustedAuthor(pr))
+                ? crossRepoRefsFromBody(`${pr.title || ''}\n${pr.body || ''}`)
+                : [];
+              const refs = dedupeRefs([...native, ...cross]);
+              if (refs.length === 0) {
+                return [
+                  'No GitHub issue is linked. Link an issue in the **Development** section of the PR ' +
+                  '(or add `Fixes #12345` to the description). For a same-org cross-repo issue, add ' +
+                  '`Fixes open-metadata/<repo>#123` to the description.',
+                ];
+              }
+              const validations = await Promise.all(refs.map(validateIssue));
+              const validIssues = validations.filter((validation) => validation.problems.length === 0);
+              const problems = [];
+              if (validIssues.length === 0) {
+                for (const validation of validations) {
+                  problems.push(...validation.problems);
+                }
+              }
+              return problems;
+            }
+
+            function buildFailureComment(problems) {
+              const lines = [
+                CHECK_MARKER,
+                '### ❌ PR checklist incomplete',
+                '',
+                'This PR cannot be merged until the following are addressed on its linked issue:',
+                '',
+              ];
+              for (const problem of problems) {
+                lines.push(`- ${problem}`);
+              }
+              lines.push('');
+              lines.push(
+                `The fields live on the **linked issue** in the **${PROJECT_NAME}** project ` +
+                '(open the issue → right sidebar → **Projects**). After you set them, ' +
+                '**re-run this check** (or push a commit) — issue/project changes do not re-trigger it automatically.'
+              );
+              lines.push('');
+              lines.push(`_Maintainers can bypass this check by adding the \`${BYPASS_LABEL}\` label._`);
+              return lines.join('\n');
+            }
+
+            function buildSuccessComment() {
+              return [
+                CHECK_MARKER,
+                '### ✅ PR checks passed',
+                '',
+                `The linked issue has a description and all required **${PROJECT_NAME}** project fields set. Thanks!`,
+              ].join('\n');
+            }
+
+            async function syncStickyComment(prNumber, body, hadFailure) {
+              const comments = await github.paginate(github.rest.issues.listComments, {
+                owner,
+                repo,
+                issue_number: prNumber,
+                per_page: 100,
+              });
+              const existing = comments.find((comment) => comment.body && comment.body.includes(CHECK_MARKER));
+              if (hadFailure) {
+                if (existing) {
+                  await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body });
+                } else {
+                  await github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body });
+                }
+              } else if (existing) {
+                await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body });
+              }
+            }
+
+            const pr = await loadPullRequest();
+
+            if (pr.draft) {
+              core.info('Draft PR — skipping metadata validation.');
+              return;
+            }
+            if (pr.user && pr.user.type === 'Bot') {
+              core.info(`PR opened by bot ${pr.user.login} — skipping metadata validation.`);
+              return;
+            }
+            const labels = (pr.labels || []).map((label) => label.name);
+            if (labels.includes(BYPASS_LABEL)) {
+              core.info(`'${BYPASS_LABEL}' label present — skipping metadata validation.`);
+              return;
+            }
+
+            const problems = [];
+            if (REQUIRE_LINKED_ISSUE) {
+              problems.push(...(await collectIssueProblems(pr)));
+            }
+
+            const hadFailure = problems.length > 0;
+            const body = hadFailure ? buildFailureComment(problems) : buildSuccessComment();
+            await syncStickyComment(pr.number, body, hadFailure);
+
+            if (hadFailure) {
+              core.setFailed(
+                `PR metadata checklist incomplete: ${problems.length} item(s) need attention. See the PR comment.`
+              );
+            } else {
+              core.info('All PR metadata checks passed.');
+            }

.github/workflows/publish-maven-package.yml

@@ -0,0 +1,84 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Publish Package to Maven Central Repository
+
+on:
+  workflow_dispatch:
+          
+  push:
+    branches:
+      - main
+    paths:
+      - "openmetadata-service/**"
+      - "openmetadata-spec/src/main/resources/json/schema/**"
+      - "openmetadata-clients/**"
+      - "pom.xml"
+      - ".github/workflows/publish-maven-package.yml" 
+
+permissions:
+  contents: read
+
+jobs:
+  publish-maven-packages:
+    runs-on: ubuntu-latest
+    environment: publish
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+      - uses: actions/checkout@v4
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: 21
+          distribution: "temurin"
+          server-id: central
+          server-username: MAVEN_USERNAME
+          server-password: MAVEN_PASSWORD
+          gpg-private-key: ${{ secrets.OSSRH_GPG_SECRET_KEY }}
+          gpg-passphrase: MAVEN_GPG_PASSPHRASE
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          librdkafka-dev unixodbc-dev libevent-dev
+      - name: Setup Test Containers Properties
+        run: |
+          sudo make install_antlr_cli
+          echo 'testcontainers.reuse.enable=true' >> $HOME/.testcontainers.properties
+      - name: Setup settings-security.xml
+        run: |
+          rm -rf ~/.m2/settings-security.xml
+          echo "<settingsSecurity><master>${{ secrets.MAVEN_MASTER_PASSWORD }}</master></settingsSecurity>" > ~/.m2/settings-security.xml
+      - name: Install gpg secret key
+        run: |
+          cat <(echo -e "${{ secrets.OSSRH_GPG_SECRET_KEY }}") | gpg --batch --import
+          gpg --list-secret-keys --keyid-format LONG
+      - name: Publish to Central Repository
+        env:
+          MAVEN_USERNAME: ${{ secrets.OSSRH_USERNAME }}
+          MAVEN_PASSWORD: ${{ secrets.OSSRH_TOKEN }}
+          MAVEN_GPG_PASSPHRASE: ${{ secrets.OSSRH_GPG_SECRET_KEY_PASSWORD }}
+        run: |
+          mvn \
+            --no-transfer-progress \
+            --batch-mode \
+            -Dgpg.passphrase=${{ secrets.OSSRH_GPG_SECRET_KEY_PASSWORD }} \
+            -DskipTests -Prelease clean deploy

.github/workflows/py-checkstyle.yml

@@ -0,0 +1,103 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Python Checkstyle
+
+# read-write repo token
+# access to secrets
+on:
+  merge_group:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+    paths-ignore:
+      - "openmetadata-ui/src/main/resources/ui/playwright/doc-generator/**"
+      - "openmetadata-ui/src/main/resources/ui/playwright/docs/**"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: py-checkstyle-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  py-checkstyle:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    permissions:
+      pull-requests: write
+
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: '${{ secrets.GITHUB_TOKEN }}'
+          valid-labels: 'safe to test'
+          pull-request-number: '${{ github.event.pull_request.number }}'
+          disable-reviews: true  # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install Ubuntu related dependencies
+        run: |
+          sudo apt-get update  && sudo apt-get install -y libsasl2-dev unixodbc-dev python3-venv
+
+      - name: Install Python & Openmetadata related dependencies
+        run: |
+          python3 -m venv env
+          source env/bin/activate
+          sudo make install_antlr_cli
+          make install install_test install_dev
+
+      # Add back linting once we have 10/10 on main
+      - name: Code style check
+        id: style
+        continue-on-error: true
+        run: |
+          source env/bin/activate
+          make generate
+          make py_format_check
+
+      - name: Create a comment in the PR with the instructions
+        if: ${{ steps.style.outcome != 'success' && github.event_name == 'pull_request_target' }}
+        uses: peter-evans/create-or-update-comment@v1
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          body: |
+            **The Python checkstyle failed.**
+
+            Please run `make py_format` and `py_format_check` in the root of your repository and commit the changes to this PR.
+            You can also use [pre-commit](https://pre-commit.com/) to automate the Python code formatting.
+
+            You can install the pre-commit hooks with `make install_test precommit_install`.
+
+      - name: Python checkstyle failed, check the comment in the PR
+        if: steps.style.outcome != 'success'
+        run: |
+          exit 1

.github/workflows/py-cli-e2e-tests.yml

@@ -0,0 +1,306 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: py-cli-e2e-tests
+on:
+  schedule:
+    - cron:  '0 0 * * *'
+  workflow_dispatch:
+    inputs:
+      e2e-tests:
+        description: "E2E Tests to run"
+        required: True
+        default: '["bigquery", "dbt_redshift", "metabase", "mssql", "mysql", "redash", "snowflake", "tableau", "python-unittests", "python-integration", "redshift", "quicksight", "datalake_s3", "postgres", "oracle", "athena", "bigquery_multiple_project", "exasol"]'
+      debug:
+        description: "If Debugging the Pipeline, Slack and Sonar events won't be triggered [default, true or false]. Default will trigger only on main branch."
+        required: False
+        default: "default"
+
+env:
+  DEBUG: "${{ (inputs.debug == 'default' && github.ref == 'refs/heads/main' && 'false') || (inputs.debug == 'default' && github.ref != 'refs/heads/main' && 'true') || inputs.debug  || 'false' }}"
+
+permissions:
+  id-token: write
+  contents: read
+
+jobs:
+  # Needed since env is not available on the job context: https://github.com/actions/runner/issues/2372
+  check-debug:
+    runs-on: ubuntu-latest
+    outputs:
+      DEBUG: ${{ env.DEBUG }}
+    steps:
+      - run: echo "INPUTS_DEBUG=${{ inputs.debug }}, GITHUB_REF=${{ github.ref }}, DEBUG=$DEBUG"
+
+  py-cli-e2e-tests:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        e2e-test: ${{ fromJSON(inputs.e2e-tests || '["bigquery", "dbt_redshift", "metabase", "mssql", "mysql", "redash", "snowflake", "tableau", "python-unittests", "python-integration", "redshift", "quicksight", "datalake_s3", "postgres", "oracle", "athena", "bigquery_multiple_project", "exasol"]') }}
+    environment: test
+
+    steps:
+    - name: Free Disk Space (Ubuntu)
+      if: matrix.e2e-test != 'python-unittests'
+      uses: jlumbroso/free-disk-space@main
+      with:
+        tool-cache: false
+        android: true
+        dotnet: true
+        haskell: true
+        large-packages: false
+        swap-storage: true
+        docker-images: false
+
+    - name: Checkout
+      uses: actions/checkout@v4
+
+    - name: configure aws credentials
+      if: contains('quicksight', matrix.e2e-test) || contains('datalake_s3', matrix.e2e-test) || contains('athena', matrix.e2e-test)
+      uses: aws-actions/configure-aws-credentials@v4
+      with:
+        role-to-assume: ${{ secrets.E2E_AWS_IAM_ROLE_ARN }}
+        role-session-name: github-ci-aws-e2e-tests
+        aws-region: ${{ secrets.E2E_AWS_REGION }}
+
+    - name: Setup Openmetadata Test Environment (without server)
+      if: matrix.e2e-test == 'python-unittests'
+      uses: ./.github/actions/setup-openmetadata-test-environment
+      with:
+        python-version: '3.10'
+        install-server: 'false'
+
+    - name: Setup Openmetadata Test Environment
+      if: matrix.e2e-test != 'python-unittests'
+      uses: ./.github/actions/setup-openmetadata-test-environment
+      with:
+        python-version: '3.10'
+
+    - name: Run Python Unit Tests
+      if: matrix.e2e-test == 'python-unittests'
+      id: python-unittest
+      continue-on-error: true
+      run: |
+        source env/bin/activate
+        cd ingestion
+        nox --no-venv -s unit-tests
+      shell: bash
+
+    - name: Rename coverage file for Python unit tests
+      if: matrix.e2e-test == 'python-unittests' && steps.python-unittest.outcome == 'success' && env.DEBUG == 'false'
+      run: mv ingestion/.coverage .coverage.python-unittests
+
+    - name: Run Python Integration Tests
+      if: matrix.e2e-test == 'python-integration'
+      id: python-integration-test
+      continue-on-error: true
+      run: |
+        source env/bin/activate
+        cd ingestion
+        nox --no-venv -s integration-tests -- --standalone --durations=5
+      env:
+        TESTCONTAINERS_RYUK_DISABLED: true
+      shell: bash
+
+    - name: Run CLI E2E Python Tests & record coverage
+      if: matrix.e2e-test != 'python-unittests' && matrix.e2e-test != 'python-integration'
+      id: e2e-test
+      continue-on-error: true
+      env:
+        E2E_TEST: ${{ matrix.e2e-test }}
+        E2E_BQ_PROJECT_ID_TAXONOMY: ${{ secrets.TEST_BQ_PROJECT_ID_TAXONOMY }}
+        E2E_BQ_PRIVATE_KEY: ${{ secrets.TEST_BQ_PRIVATE_KEY_E2E }}
+        E2E_BQ_PROJECT_ID: ${{ secrets.TEST_BQ_PROJECT_ID }}
+        E2E_BQ_PROJECT_ID2: ${{ secrets.TEST_BQ_PROJECT_ID2 }}
+        E2E_BQ_PRIVATE_KEY_ID: ${{ secrets.TEST_BQ_PRIVATE_KEY_ID }}
+        E2E_BQ_CLIENT_EMAIL: ${{ secrets.TEST_BQ_CLIENT_EMAIL }}
+        E2E_BQ_CLIENT_ID: ${{ secrets.TEST_BQ_CLIENT_ID }}
+        E2E_SNOWFLAKE_PASSWORD: ${{ secrets.TEST_SNOWFLAKE_PASSWORD_YAML }}
+        E2E_SNOWFLAKE_PASSPHRASE: ${{ secrets.TEST_SNOWFLAKE_PASSPHRASE }}
+        E2E_SNOWFLAKE_USERNAME: ${{ secrets.TEST_SNOWFLAKE_USERNAME }}
+        E2E_SNOWFLAKE_ACCOUNT: ${{ secrets.TEST_SNOWFLAKE_ACCOUNT }}
+        E2E_SNOWFLAKE_DATABASE: ${{ secrets.TEST_SNOWFLAKE_DATABASE_E2E }}
+        E2E_SNOWFLAKE_WAREHOUSE: ${{ secrets.TEST_SNOWFLAKE_WAREHOUSE }}
+        E2E_REDSHIFT_DBT_CATALOG_HTTP_FILE_PATH: ${{ secrets.E2E_REDSHIFT_DBT_CATALOG_HTTP_FILE_PATH }}
+        E2E_REDSHIFT_DBT_MANIFEST_HTTP_FILE_PATH: ${{ secrets.E2E_REDSHIFT_DBT_MANIFEST_HTTP_FILE_PATH }}
+        E2E_REDSHIFT_DBT_RUN_RESULTS_HTTP_FILE_PATH: ${{ secrets.E2E_REDSHIFT_DBT_RUN_RESULTS_HTTP_FILE_PATH }}
+        E2E_REDSHIFT_HOST_PORT: ${{ secrets.E2E_REDSHIFT_HOST_PORT }}
+        E2E_REDSHIFT_USERNAME: ${{ secrets.E2E_REDSHIFT_USERNAME }}
+        E2E_REDSHIFT_PASSWORD: ${{ secrets.E2E_REDSHIFT_PASSWORD }}
+        E2E_REDSHIFT_DATABASE: ${{ secrets.E2E_REDSHIFT_DATABASE }}
+        E2E_REDSHIFT_DB: ${{ secrets.E2E_REDSHIFT_DB }}
+        E2E_MSSQL_USERNAME: ${{ secrets.E2E_MSSQL_USERNAME }}
+        E2E_MSSQL_PASSWORD: ${{ secrets.E2E_MSSQL_PASSWORD }}
+        E2E_MSSQL_HOST: ${{ secrets.E2E_MSSQL_HOST }}
+        E2E_MSSQL_DATABASE: ${{ secrets.E2E_MSSQL_DATABASE }}
+        E2E_VERTICA_USERNAME: ${{ secrets.E2E_VERTICA_USERNAME }}
+        E2E_VERTICA_PASSWORD: ${{ secrets.E2E_VERTICA_PASSWORD }}
+        E2E_VERTICA_HOST_PORT: ${{ secrets.E2E_VERTICA_HOST_PORT }}
+        E2E_TABLEAU_PAT_NAME: ${{ secrets.E2E_TABLEAU_PAT_NAME }}
+        E2E_TABLEAU_PAT_SECRET: ${{ secrets.E2E_TABLEAU_PAT_SECRET }}
+        E2E_TABLEAU_HOST_PORT: ${{ secrets.E2E_TABLEAU_HOST_PORT }}
+        E2E_TABLEAU_SITE: ${{ secrets.E2E_TABLEAU_SITE }}
+        E2E_REDASH_HOST_PORT: ${{ secrets.E2E_REDASH_HOST_PORT }}
+        E2E_REDASH_USERNAME: ${{ secrets.E2E_REDASH_USERNAME }}
+        E2E_REDASH_API_KEY: ${{ secrets.E2E_REDASH_API_KEY }}
+        E2E_POWERBI_CLIENT_SECRET: ${{ secrets.E2E_POWERBI_CLIENT_SECRET }}
+        E2E_POWERBI_CLIENT_ID: ${{ secrets.E2E_POWERBI_CLIENT_ID }}
+        E2E_POWERBI_TENANT_ID: ${{ secrets.E2E_POWERBI_TENANT_ID }}
+        E2E_METABASE_HOST_PORT: ${{ secrets.TEST_METABASE_HOST_PORT }}
+        E2E_METABASE_PASSWORD: ${{ secrets.TEST_METABASE_PASSWORD }}
+        E2E_METABASE_USERNAME: ${{ secrets.TEST_METABASE_USERNAME }}
+        E2E_HIVE_HOST_PORT: ${{ secrets.E2E_HIVE_HOST_PORT }}
+        E2E_HIVE_AUTH: ${{ secrets.E2E_HIVE_AUTH }}
+        E2E_QUICKSIGHT_AWS_ACCOUNTID: ${{ secrets.E2E_QUICKSIGHT_AWS_ACCOUNTID }}
+        E2E_QUICKSIGHT_REGION: ${{ secrets.E2E_QUICKSIGHT_REGION }}
+        E2E_DATALAKE_S3_BUCKET_NAME: ${{ secrets.E2E_DATALAKE_S3_BUCKET_NAME }}
+        E2E_DATALAKE_S3_PREFIX: ${{ secrets.E2E_DATALAKE_S3_PREFIX }}
+        E2E_DATALAKE_S3_REGION: ${{ secrets.E2E_AWS_REGION }}
+        E2E_POSTGRES_USERNAME: ${{ secrets.E2E_POSTGRES_USERNAME }}
+        E2E_POSTGRES_PASSWORD: ${{ secrets.E2E_POSTGRES_PASSWORD }}
+        E2E_POSTGRES_HOSTPORT: ${{ secrets.TEST_POSTGRES_HOST_PORT }}
+        E2E_POSTGRES_DATABASE: ${{ secrets.E2E_POSTGRES_DATABASE }}
+        E2E_ORACLE_HOST_PORT: ${{ secrets.E2E_ORACLE_HOST_PORT }}
+        E2E_ORACLE_USERNAME: ${{ secrets.E2E_ORACLE_USERNAME }}
+        E2E_ORACLE_PASSWORD: ${{ secrets.E2E_ORACLE_PASSWORD }}
+        E2E_ORACLE_SERVICE_NAME: ${{ secrets.E2E_ORACLE_SERVICE_NAME }}
+        E2E_ATHENA_REGION: ${{ secrets.E2E_AWS_REGION }}
+        E2E_ATHENA_S3STAGINGDIR: ${{ secrets.E2E_ATHENA_S3STAGINGDIR }}
+        E2E_ATHENA_WORKGROUP: ${{ secrets.E2E_ATHENA_WORKGROUP }}
+      run: |
+        source env/bin/activate
+        export SITE_CUSTOMIZE_PATH=$(python -c "import site; import os; from pathlib import Path; print(os.path.relpath(site.getsitepackages()[0], str(Path.cwd())))")/sitecustomize.py
+        echo "import os" >> $SITE_CUSTOMIZE_PATH
+        echo "try:" >> $SITE_CUSTOMIZE_PATH
+        echo "    import coverage" >> $SITE_CUSTOMIZE_PATH
+        echo "    os.environ['COVERAGE_PROCESS_START'] = os.path.join(os.environ.get('GITHUB_WORKSPACE', os.getcwd()), 'ingestion', 'pyproject.toml')" >> $SITE_CUSTOMIZE_PATH
+        echo "    coverage.process_startup()" >> $SITE_CUSTOMIZE_PATH
+        echo "except ImportError:" >> $SITE_CUSTOMIZE_PATH
+        echo "    pass" >> $SITE_CUSTOMIZE_PATH
+        coverage run --rcfile ingestion/pyproject.toml --branch -m pytest -c ingestion/pyproject.toml --junitxml=ingestion/junit/test-results-$E2E_TEST.xml --ignore=ingestion/tests/unit/source ingestion/tests/cli_e2e/test_cli_$E2E_TEST.py
+        coverage combine --data-file=.coverage.$E2E_TEST --rcfile=ingestion/pyproject.toml --keep .coverage*
+        coverage report --rcfile ingestion/pyproject.toml --data-file .coverage.$E2E_TEST || true
+
+    - name: Upload coverage artifact for Python unit tests
+      if: matrix.e2e-test == 'python-unittests' && steps.python-unittest.outcome == 'success' && env.DEBUG == 'false'
+      uses: actions/upload-artifact@v4
+      with:
+        name: coverage-${{ matrix.e2e-test }}
+        path: .coverage.python-unittests
+        include-hidden-files: true
+
+    - name: Rename coverage file for Python integration tests
+      if: matrix.e2e-test == 'python-integration' && steps.python-integration-test.outcome == 'success' && env.DEBUG == 'false'
+      run: mv ingestion/.coverage .coverage.python-integration
+
+    - name: Upload coverage artifact for Python integration tests
+      if: matrix.e2e-test == 'python-integration' && steps.python-integration-test.outcome == 'success' && env.DEBUG == 'false'
+      uses: actions/upload-artifact@v4
+      with:
+        name: coverage-${{ matrix.e2e-test }}
+        path: .coverage.python-integration
+        include-hidden-files: true
+
+    - name: Upload coverage artifact for CLI E2E tests
+      if: matrix.e2e-test != 'python-unittests' && matrix.e2e-test != 'python-integration' && steps.e2e-test.outcome == 'success' && env.DEBUG == 'false'
+      uses: actions/upload-artifact@v4
+      with:
+        name: coverage-${{ matrix.e2e-test }}
+        path: .coverage.${{ matrix.e2e-test }}
+        include-hidden-files: true
+
+    - name: Upload tests artifact
+      if: (steps.e2e-test.outcome == 'success' || steps.python-unittest.outcome == 'success' || steps.python-integration-test.outcome == 'success') && env.DEBUG == 'false'
+      uses: actions/upload-artifact@v4
+      with:
+        name: tests-${{ matrix.e2e-test }}
+        path: ingestion/junit/test-results-*.xml
+
+    - name: Clean Up
+      if: matrix.e2e-test != 'python-unittests'
+      run: |
+        cd ./docker/development
+        docker compose down --remove-orphans
+        sudo rm -rf ${PWD}/docker-volume
+
+    - name: Slack on Failure
+      if: (steps.e2e-test.outcome == 'failure' || steps.python-unittest.outcome == 'failure' || steps.python-integration-test.outcome == 'failure') && env.DEBUG == 'false'
+      uses: slackapi/slack-github-action@v1.23.0
+      with:
+        payload: |
+          {
+            "text": "🔥 Failed E2E Test for: ${{ matrix.e2e-test }} 🔥\nLogs: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+          }
+      env:
+        SLACK_WEBHOOK_URL: ${{ secrets.E2E_SLACK_WEBHOOK }}
+        SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK
+
+    - name: Force failure
+      if: steps.e2e-test.outcome == 'failure' || steps.python-unittest.outcome == 'failure' || steps.python-integration-test.outcome == 'failure'
+      run: |
+        exit 1
+
+  sonar-cloud-coverage-upload:
+    runs-on: ubuntu-latest
+    needs:
+      - py-cli-e2e-tests
+      - check-debug
+    if: needs.check-debug.outputs.DEBUG == 'false'
+    steps:
+
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install Ubuntu dependencies
+        run: |
+          sudo apt-get update && sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          unixodbc-dev libevent-dev python3-dev libkrb5-dev
+
+      - name: Install coverage dependencies
+        run: |
+          python3 -m venv env
+          source env/bin/activate
+          make install_all install_test
+
+      - name: Download all artifacts
+        uses: actions/download-artifact@v4
+        with:
+          path: artifacts
+
+      - name: Generate report
+        run: |
+          # Copy coverage artifacts into ingestion/ so paths resolve relative to it
+          for folder in artifacts/coverage-*; do
+            cp -rT $folder/ ingestion/ ;
+          done
+          mkdir -p ingestion/junit
+          for folder in artifacts/tests-*; do
+            cp -rT $folder/ ingestion/junit ;
+          done
+          source env/bin/activate
+          cd ingestion
+          coverage combine --rcfile=pyproject.toml --keep .coverage*
+          coverage xml --rcfile=pyproject.toml --data-file=.coverage
+        shell: bash
+
+      - name: Push Results to Sonar
+        uses: sonarsource/sonarcloud-github-action@master
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SONAR_TOKEN: ${{ secrets.INGESTION_SONAR_SECRET }}
+        with:
+          projectBaseDir: ingestion/

.github/workflows/py-operator-build-test.yml

@@ -0,0 +1,91 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: py-operator-build-test
+on:
+  workflow_dispatch:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+    paths:
+      - "ingestion/**"
+      - "openmetadata-service/**"
+      - "openmetadata-spec/src/main/resources/json/schema/**"
+      - "pom.xml"
+      - "Makefile"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  py-run-build-tests:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: "3.10"
+          args: "-m no-ui"
+          ingestion_dependency: "mysql,elasticsearch,sample-data"
+
+      - name: Build Ingestion Operator & Ingest Sample Data
+        run: |
+          docker build -f ingestion/operators/docker/Dockerfile.ci . -t openmetadata/ingestion-base:test
+          docker run --net=host --rm openmetadata/ingestion-base:test metadata ingest -c ./pipelines/sample_data.yaml
+
+      - name: Validate ingestion
+        run: |
+          source env/bin/activate
+          python scripts/validate_sample_data.py
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume

.github/workflows/py-sonarcloud-nightly.yml

@@ -0,0 +1,227 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: SonarCloud Ingestion Nightly
+
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 2 * * *"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: sonarcloud-ingestion-nightly
+  cancel-in-progress: true
+
+env:
+  PYTHON_VERSION: "3.10"
+  BRANCH_NAME: "main"
+  PROJECT_BASE_DIR: "ingestion"
+
+jobs:
+  py-unit-tests:
+    name: Unit Tests
+    timeout-minutes: 60
+    runs-on: ubuntu-latest
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ env.BRANCH_NAME }}
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+          install-server: 'false'
+
+      - name: Run Unit Tests
+        run: |
+          source env/bin/activate
+          cd ingestion
+          nox --no-venv -s unit-tests
+        shell: bash
+
+      - name: Upload coverage artifact
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-unit
+          path: ingestion/.coverage
+          include-hidden-files: true
+
+  py-integration-tests:
+    name: "Integration Tests (${{ matrix.shard.name }})"
+    timeout-minutes: 180
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        shard:
+          - name: "shard-1"
+            nox-args: >-
+              tests/integration/ometa
+              tests/integration/postgres
+              tests/integration/mysql
+              tests/integration/profiler
+              tests/integration/data_quality
+          - name: "shard-2"
+            nox-args: >-
+              --ignore=tests/integration/ometa
+              --ignore=tests/integration/postgres
+              --ignore=tests/integration/mysql
+              --ignore=tests/integration/profiler
+              --ignore=tests/integration/data_quality
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ env.BRANCH_NAME }}
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+          args: "-m no-ui"
+          ingestion_dependency: "mysql,elasticsearch,sample-data"
+
+      - name: Run Integration Tests
+        run: |
+          source env/bin/activate
+          cd ingestion
+          nox --no-venv -s integration-tests -- --standalone --durations=5 ${{ matrix.shard.nox-args }}
+        env:
+          TESTCONTAINERS_RYUK_DISABLED: true
+        shell: bash
+
+      - name: Upload coverage artifact
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-integration-${{ matrix.shard.name }}
+          path: ingestion/.coverage
+          include-hidden-files: true
+
+      - name: Clean Up
+        if: ${{ !cancelled() }}
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+  py-combine-coverage:
+    if: ${{ !cancelled() }}
+    needs: [py-unit-tests, py-integration-tests]
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ env.BRANCH_NAME }}
+          fetch-depth: 0
+          filter: blob:none
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+
+      - name: Install uv
+        run: pip install uv
+        shell: bash
+
+      - name: Install coverage
+        run: |
+          python3 -m venv env
+          source env/bin/activate
+          uv pip install "coverage[toml]" nox
+        shell: bash
+
+      - name: Download coverage artifacts
+        uses: actions/download-artifact@v4
+        with:
+          pattern: coverage-*
+          path: ingestion/coverage-data/
+
+      - name: Prepare coverage files
+        run: |
+          cd ingestion
+          [ -f coverage-data/coverage-unit/.coverage ] && mv coverage-data/coverage-unit/.coverage .coverage.unit
+          for dir in coverage-data/coverage-integration-*/; do
+            shard=$(basename "$dir" | sed 's/coverage-integration-//')
+            [ -f "$dir/.coverage" ] && mv "$dir/.coverage" ".coverage.integration-$shard"
+          done
+        shell: bash
+
+      - name: Combine coverage
+        run: |
+          source env/bin/activate
+          cd ingestion
+          nox --no-venv -s combine-coverage
+        shell: bash
+
+      - name: Remove pom.xml
+        run: rm pom.xml
+        shell: bash
+
+      - name: Push Results To Sonar
+        if: ${{ !cancelled() }}
+        id: push-to-sonar
+        continue-on-error: true
+        uses: SonarSource/sonarqube-scan-action@v7
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SONAR_TOKEN: ${{ secrets.INGESTION_SONAR_SECRET }}
+        with:
+          projectBaseDir: ${{ env.PROJECT_BASE_DIR }}/
+          args: >
+            -Dsonar.branch.name=${{ env.BRANCH_NAME }}
+
+      - name: Wait to retry 'Push Results to Sonar'
+        if: ${{ !cancelled() && steps.push-to-sonar.outcome != 'success' }}
+        run: sleep 20s
+        shell: bash
+
+      - name: Retry 'Push Results to Sonar'
+        if: ${{ !cancelled() && steps.push-to-sonar.outcome != 'success' }}
+        uses: SonarSource/sonarqube-scan-action@v7
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SONAR_TOKEN: ${{ secrets.INGESTION_SONAR_SECRET }}
+        with:
+          projectBaseDir: ${{ env.PROJECT_BASE_DIR }}/
+          args: >
+            -Dsonar.branch.name=${{ env.BRANCH_NAME }}

.github/workflows/py-tests-postgres.yml

@@ -0,0 +1,141 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: py-tests-postgres
+on:
+  merge_group:
+  workflow_dispatch:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  # Detect whether relevant paths changed. When no Python/service/schema files
+  # are modified the downstream jobs are skipped via their `if` condition.
+  # A job skipped by `if` reports as "Success", so required checks still pass.
+  changes:
+    name: Detect Changes
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    outputs:
+      python: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.filter.outputs.python }}
+    steps:
+      - uses: dorny/paths-filter@v3
+        id: filter
+        if: ${{ github.event_name != 'workflow_dispatch' }}
+        with:
+          filters: |
+            python:
+              - 'ingestion/**'
+              - 'openmetadata-service/**'
+              - 'openmetadata-spec/src/main/resources/json/schema/**'
+              - 'pom.xml'
+              - 'Makefile'
+
+  py-integration-tests:
+    name: "Integration Tests (${{ matrix.shard.name }}, ${{ matrix.py-version }})"
+    needs: changes
+    if: ${{ needs.changes.outputs.python == 'true' }}
+    timeout-minutes: 180
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        py-version: ["3.10", "3.11", "3.12"]
+        shard:
+          - name: "shard-1"
+            nox-args: >-
+              tests/integration/ometa
+              tests/integration/postgres
+              tests/integration/mysql
+              tests/integration/profiler
+              tests/integration/data_quality
+          - name: "shard-2"
+            nox-args: >-
+              --ignore=tests/integration/ometa
+              --ignore=tests/integration/postgres
+              --ignore=tests/integration/mysql
+              --ignore=tests/integration/profiler
+              --ignore=tests/integration/data_quality
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: ${{ matrix.py-version}}
+          args: "-m no-ui -d postgresql"
+          ingestion_dependency: "mysql,elasticsearch,sample-data"
+
+      - name: Run Integration Tests
+        run: |
+          source env/bin/activate
+          cd ingestion
+          nox --no-venv -s integration-tests -- --standalone --durations=5 ${{ matrix.shard.nox-args }}
+        env:
+          TESTCONTAINERS_RYUK_DISABLED: true
+        shell: bash
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf ${PWD}/docker-volume
+
+  # Single required-check gate for branch protection.
+  # Skipped (= "Success") when all test jobs pass or are legitimately skipped.
+  # Runs and exits 1 only when a test job fails or is cancelled.
+  # Set "py-tests-postgres / py-tests-status" as the sole required check for this workflow.
+  py-tests-status:
+    name: py-tests-status
+    needs: [changes, py-integration-tests]
+    if: ${{ failure() || cancelled() }}
+    runs-on: ubuntu-latest
+    steps:
+      - run: exit 1

.github/workflows/py-tests.yml

@@ -0,0 +1,324 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: py-tests
+on:
+  merge_group:
+  workflow_dispatch:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened, ready_for_review]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+env:
+  # matrix can't use 'env'. When updating it, update it for both jobs.
+  MAIN_PYTHON_VERSION: "3.10"
+  SONAR_OPTS: >-
+    -Dsonar.pullrequest.key=${{ github.event.pull_request.number }}
+    -Dsonar.pullrequest.branch=${{ github.event.pull_request.head.ref }}
+    -Dsonar.pullrequest.github.repository=OpenMetadata
+    -Dsonar.scm.revision=${{ github.event.pull_request.head.sha }}
+    -Dsonar.pullrequest.provider=github
+
+jobs:
+  # Detect whether relevant paths changed. When no Python/service/schema files
+  # are modified the downstream jobs are skipped via their `if` condition.
+  # A job skipped by `if` reports as "Success", so required checks still pass.
+  # This replaces the old py-tests-skip.yml companion workflow.
+  changes:
+    name: Detect Changes
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    outputs:
+      python: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.filter.outputs.python }}
+    steps:
+      - uses: dorny/paths-filter@v3
+        id: filter
+        if: ${{ github.event_name != 'workflow_dispatch' }}
+        with:
+          filters: |
+            python:
+              - 'ingestion/**'
+              - 'openmetadata-service/**'
+              - 'openmetadata-spec/src/main/resources/json/schema/**'
+              - 'pom.xml'
+              - 'Makefile'
+
+  py-unit-tests:
+    name: Unit Tests & Static Checks
+    needs: changes
+    if: ${{ needs.changes.outputs.python == 'true' }}
+    timeout-minutes: 60
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        py-version: ["3.10", "3.11", "3.12"]
+    steps:
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 30
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: ${{ matrix.py-version }}
+          install-server: 'false'
+
+      - name: Run Static Checks
+        # basedpyright is configured with `pythonVersion = "3.10"` (the lowest
+        # supported version) so type-checking results are identical across the
+        # 3.10/3.11/3.12 matrix. Run on the lowest version only to avoid
+        # redundant work and keep the baseline file deterministic.
+        if: matrix.py-version == '3.10'
+        run: |
+          source env/bin/activate
+          cd ingestion
+          nox --no-venv -s static-checks
+        shell: bash
+
+      - name: Run Unit Tests
+        run: |
+          source env/bin/activate
+          cd ingestion
+          nox --no-venv -s unit-tests
+        shell: bash
+
+      - name: Upload coverage artifact
+        if: ${{ matrix.py-version == env.MAIN_PYTHON_VERSION && !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-unit
+          path: ingestion/.coverage
+          include-hidden-files: true
+
+  py-integration-tests:
+    name: "Integration Tests (${{ matrix.shard.name }}, ${{ matrix.py-version }})"
+    needs: changes
+    if: ${{ needs.changes.outputs.python == 'true' }}
+    timeout-minutes: 180
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        py-version: ["3.10", "3.11", "3.12"]
+        shard:
+          - name: "shard-1"
+            nox-args: >-
+              tests/integration/ometa
+              tests/integration/postgres
+              tests/integration/mysql
+              tests/integration/profiler
+              tests/integration/data_quality
+          - name: "shard-2"
+            nox-args: >-
+              --ignore=tests/integration/ometa
+              --ignore=tests/integration/postgres
+              --ignore=tests/integration/mysql
+              --ignore=tests/integration/profiler
+              --ignore=tests/integration/data_quality
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Setup Openmetadata Test Environment
+        uses: ./.github/actions/setup-openmetadata-test-environment
+        with:
+          python-version: ${{ matrix.py-version}}
+          args: "-m no-ui"
+          ingestion_dependency: "mysql,elasticsearch,sample-data"
+
+      - name: Run Integration Tests
+        run: |
+          source env/bin/activate
+          cd ingestion
+          read -r -a shard_args <<< "$NOX_ARGS"
+          set +e
+          nox --no-venv -s integration-tests -- --standalone --durations=5 "${shard_args[@]}" 2>&1 | tee integration-test-run.log
+          status=${PIPESTATUS[0]}
+          set -e
+          if [ "$status" -ne 0 ] && { [ "$status" -eq 139 ] || [ "$status" -eq 245 ] || grep -Eq "failed with exit code -11|Segmentation fault|exit code 139" integration-test-run.log; }; then
+            nox --no-venv -s integration-tests -- --standalone --durations=5 "${shard_args[@]}"
+          else
+            exit "$status"
+          fi
+        env:
+          NOX_ARGS: ${{ matrix.shard.nox-args }}
+          TESTCONTAINERS_RYUK_DISABLED: true
+        shell: bash
+
+      - name: Upload coverage artifact
+        if: ${{ matrix.py-version == env.MAIN_PYTHON_VERSION && !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-integration-${{ matrix.shard.name }}
+          path: ingestion/.coverage
+          include-hidden-files: true
+
+      - name: Clean Up
+        run: |
+          cd ./docker/development
+          docker compose down --remove-orphans
+          sudo rm -rf "${PWD}/docker-volume"
+
+  # Single required-check gate for branch protection.
+  # Skipped (= "Success") when all test jobs pass or are legitimately skipped.
+  # Runs and exits 1 only when a test job fails or is cancelled.
+  # Set "py-tests / py-tests-status" as the sole required check for this workflow.
+  py-tests-status:
+    name: py-tests-status
+    needs: [changes, py-unit-tests, py-integration-tests]
+    if: ${{ failure() || cancelled() }}
+    runs-on: ubuntu-latest
+    steps:
+      - run: exit 1
+
+  py-combine-coverage:
+    needs: [changes, py-unit-tests, py-integration-tests]
+    if: ${{ needs.changes.outputs.python == 'true' && !cancelled() }}
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+          fetch-depth: 0
+          filter: blob:none
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ env.MAIN_PYTHON_VERSION }}
+
+      - name: Install uv
+        run: pip install uv
+        shell: bash
+
+      - name: Install coverage
+        run: |
+          python3 -m venv env
+          source env/bin/activate
+          uv pip install "coverage[toml]" nox
+        shell: bash
+
+      - name: Download coverage artifacts
+        uses: actions/download-artifact@v4
+        with:
+          pattern: coverage-*
+          path: ingestion/coverage-data/
+
+      - name: Prepare coverage files
+        run: |
+          cd ingestion
+          [ -f coverage-data/coverage-unit/.coverage ] && mv coverage-data/coverage-unit/.coverage .coverage.unit
+          for dir in coverage-data/coverage-integration-*/; do
+            shard=$(basename "$dir" | sed 's/coverage-integration-//')
+            [ -f "$dir/.coverage" ] && mv "$dir/.coverage" ".coverage.integration-$shard"
+          done
+        shell: bash
+
+      - name: Combine coverage
+        run: |
+          source env/bin/activate
+          cd ingestion
+          nox --no-venv -s combine-coverage
+        shell: bash
+
+      - name: Remove pom.xml
+        run: rm pom.xml
+        shell: bash
+
+        # we have to pass these args values since we are working with the 'pull_request_target' trigger
+      - name: Push Results in PR to Sonar
+        id: push-to-sonar
+        if: ${{ github.event_name == 'pull_request_target'}}
+        continue-on-error: true
+        uses: SonarSource/sonarqube-scan-action@v7
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SONAR_TOKEN: ${{ secrets.INGESTION_SONAR_SECRET }}
+        with:
+          projectBaseDir: ingestion/
+          args: ${{ env.SONAR_OPTS }}
+
+      # next two steps are for retrying "Push Results in PR to Sonar" step in case it fails
+      - name: Wait to retry 'Push Results in PR to Sonar'
+        if: ${{ github.event_name == 'pull_request_target' && steps.push-to-sonar.outcome != 'success' }}
+        run: sleep 20s
+        shell: bash
+
+      - name: Retry 'Push Results in PR to Sonar'
+        uses: SonarSource/sonarqube-scan-action@v7
+        if: ${{ github.event_name == 'pull_request_target' && steps.push-to-sonar.outcome != 'success' }}
+        continue-on-error: true
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SONAR_TOKEN: ${{ secrets.INGESTION_SONAR_SECRET }}
+        with:
+          projectBaseDir: ingestion/
+          args: ${{ env.SONAR_OPTS }}

.github/workflows/python-packages-publish.yml

@@ -0,0 +1,54 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Publish Python Packages
+
+on:
+  workflow_dispatch:
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: release
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+      - name: Install Ubuntu related dependencies
+        run: |
+          sudo apt-get update && sudo apt-get install -y libsasl2-dev unixodbc-dev python3-venv
+      - name: Install and Publish PyPi packages for OpenMetadata Ingestion
+        env:
+          TWINE_USERNAME: '${{ secrets.TWINE_OPENMETADATA_INGESTION_USERNAME }}'
+          TWINE_PASSWORD: '${{ secrets.TWINE_OPENMETADATA_INGESTION_PASSWORD }}'
+        run: |
+          python3 -m venv env
+          source env/bin/activate
+          sudo make install_antlr_cli
+          make install_dev generate
+          cd ingestion; \
+            python -m build; \
+            twine check dist/*; \
+            twine upload dist/* --verbose
+      - name: Install and Publish PyPi packages for OpenMetadata Managed (Airflow) APIs
+        env:
+          TWINE_USERNAME: '${{ secrets.TWINE_OPENMETADATA_AIRFLOW_MANAGED_APIS_USERNAME }}'
+          TWINE_PASSWORD: '${{ secrets.TWINE_OPENMETADATA_AIRFLOW_MANAGED_APIS_PASSWORD }}'
+        run: |
+          python3 -m venv env
+          source env/bin/activate
+          make install_dev install_apis
+          cd openmetadata-airflow-apis; \
+            python -m build; \
+            twine check dist/*; \
+            twine upload dist/* --verbose

.github/workflows/security-scan.yml

@@ -0,0 +1,427 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: security-scan
+on:
+  schedule:
+    - cron: "0 0 */2 * *"
+  workflow_dispatch:
+    inputs:
+      SEND_SLACK_NOTIFICATION:
+        description: "Post results to Slack (uncheck for test runs)"
+        type: boolean
+        required: false
+        default: true
+
+env:
+  # Manual runs may opt out of Slack; schedule (where inputs are unset) always sends.
+  # Explicit empty-check, since `||` would treat the string 'false' inconsistently when the
+  # input is typed `boolean` — only an empty string should fall back to 'true'.
+  SEND_SLACK_NOTIFICATION: ${{ github.event.inputs.SEND_SLACK_NOTIFICATION == '' && 'true' || github.event.inputs.SEND_SLACK_NOTIFICATION }}
+
+jobs:
+  vulnerability-scan:
+    runs-on: ubuntu-latest
+    environment: security-scan
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: "openmetadata-ui/src/main/resources/ui/.nvmrc"
+
+      - name: Enable yarn
+        run: corepack enable
+
+      - name: Install UI dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: yarn install --frozen-lockfile --ignore-scripts
+
+      - name: Run Retire.js scan
+        id: retire-scan
+        continue-on-error: true
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: |
+          npx retire@5 \
+            --path node_modules/ \
+            --severity high \
+            --outputformat json \
+            --outputpath retire-report.json
+
+      - name: Verify report was generated
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: |
+          if [ ! -f retire-report.json ]; then
+            echo '::error::retire-report.json was not generated — retire scan may have crashed'
+            exit 1
+          fi
+
+      - name: Upload Retire.js Report
+        if: success()
+        uses: actions/upload-artifact@v4
+        with:
+          name: retire-js-report
+          path: openmetadata-ui/src/main/resources/ui/retire-report.json
+          retention-days: 30
+
+      - name: Generate Retire.js Slack summary
+        if: always()
+        run: |
+          mkdir -p retire-report
+          python3 scripts/retire_slack_summary.py \
+            openmetadata-ui/src/main/resources/ui/retire-report.json \
+            --counts-file retire-report/_retire_counts.json \
+            --slack-file retire-report/_retire_slack.txt \
+            > /dev/null
+
+      - name: Upload Retire.js Slack artifact
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: retire-slack
+          path: |
+            retire-report/_retire_slack.txt
+            retire-report/_retire_counts.json
+          retention-days: 30
+
+      - name: Publish Retire.js Summary
+        if: success()
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: |
+          python3 - << 'EOF' >> $GITHUB_STEP_SUMMARY
+          import json
+
+          SEVERITY_ICON = {"critical": "🚨", "high": "🔴", "medium": "🟠", "low": "🟡"}
+          SEVERITY_ORDER = {"critical": 0, "high": 1, "medium": 2, "low": 3}
+          NM = "node_modules/"
+
+          def escape(text):
+              return str(text).replace('|', '\\|').replace('`', "'")
+
+          try:
+              with open("retire-report.json") as f:
+                  data = json.load(f)
+          except FileNotFoundError:
+              print("## Retire.js Scan Results\n\n> Report file not found — scan may not have run.")
+              raise SystemExit(0)
+
+          findings = data.get("data", [])
+          libs = {}
+          for item in findings:
+              filepath = item.get("file", "")
+              short = filepath[filepath.find(NM) + len(NM):] if NM in filepath else filepath
+              for result in item.get("results", []):
+                  key = (result.get("component", ""), result.get("version", ""))
+                  if key not in libs:
+                      libs[key] = {"files": [], "vulns": result.get("vulnerabilities", [])}
+                  if short not in libs[key]["files"]:
+                      libs[key]["files"].append(short)
+
+          print("## Retire.js Scan Results\n")
+
+          if not libs:
+              print("✅ No vulnerable libraries found.")
+          else:
+              total_vulns = sum(len(v["vulns"]) for v in libs.values())
+              print(f"> **{len(libs)} vulnerable librar{'y' if len(libs) == 1 else 'ies'} · {total_vulns} CVE{'s' if total_vulns != 1 else ''} found**\n")
+
+              for (component, version), info in sorted(libs.items(), key=lambda x: min(
+                      (SEVERITY_ORDER.get(v.get("severity", "low"), 3) for v in x[1]["vulns"]), default=3)):
+                  top_sev = min(info["vulns"], key=lambda v: SEVERITY_ORDER.get(v.get("severity", "low"), 3))
+                  icon = SEVERITY_ICON.get(top_sev.get("severity", "low"), "⚪")
+                  print(f"### {icon} {component} {version}\n")
+                  print("| Severity | CVE | Summary |")
+                  print("|---|---|---|")
+                  for vuln in sorted(info["vulns"], key=lambda v: SEVERITY_ORDER.get(v.get("severity", "low"), 3)):
+                      sev = vuln.get("severity", "")
+                      ids = vuln.get("identifiers", {})
+                      cves = ids.get("CVE", [])
+                      summary = ids.get("summary", "").split("\n")[0][:120]
+                      cve_str = ", ".join(f"[{c}](https://nvd.nist.gov/vuln/detail/{c})" for c in cves) if cves else ids.get("githubID", "—")
+                      print(f"| {SEVERITY_ICON.get(sev, '')} {sev} | {escape(cve_str)} | {escape(summary)} |")
+                  print("\n**Bundled in:**")
+                  for f in info["files"]:
+                      print(f"- `{f}`")
+                  print()
+          EOF
+
+      - name: Force failure on vulnerabilities found
+        if: steps.retire-scan.outcome == 'failure'
+        run: exit 1
+
+  security-scan:
+    runs-on: ubuntu-latest
+    environment: security-scan
+    env:
+      SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
+      SNYK_ORGANIZATION: ${{ secrets.SNYK_ORGANIZATION_ID }}
+
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          docker-images: true
+          swap-storage: true
+      - uses: actions/checkout@v4
+
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@v4
+        with:
+          java-version: "21"
+          distribution: "temurin"
+
+      - name: Install Ubuntu dependencies
+        run: |
+          # stop relying on apt cache of GitHub runners
+          sudo apt-get update
+          sudo apt-get install -y unixodbc-dev python3-venv librdkafka-dev gcc libsasl2-dev build-essential libssl-dev libffi-dev \
+          librdkafka-dev unixodbc-dev libevent-dev wkhtmltopdf libkrb5-dev
+
+      # Install and Authenticate to Snyk
+      - name: Install Snyk & Authenticate
+        run: |
+          sudo make install_antlr_cli
+          sudo npm install -g snyk
+          snyk auth ${SNYK_TOKEN}
+          snyk config set org=${SNYK_ORGANIZATION}
+
+      - name: Install Python dependencies
+        run: |
+          python3 -m venv env
+          source env/bin/activate
+          make install_all install_apis
+
+      - name: Maven build
+        id: maven-build
+        continue-on-error: true
+        run: mvn -DskipTests clean install
+
+      - name: Run Scan
+        id: security-report
+        if: steps.maven-build.outcome == 'success'
+        continue-on-error: true
+        run: |
+          source env/bin/activate
+          rm -rf security-report
+          mkdir -p security-report
+          # Run snyk subtargets directly; skip `export-snyk-pdf-report` which deletes JSONs after PDF conversion.
+          make snyk-ingestion-report || true
+          make snyk-ingestion-base-slim-report || true
+          make snyk-airflow-apis-report || true
+          make snyk-server-report || true
+          make snyk-ui-report || true
+
+      - name: Publish Snyk Summary
+        id: snyk-summary
+        if: always() && steps.maven-build.outcome == 'success'
+        run: |
+          python3 scripts/snyk_summary.py security-report \
+            --counts-file security-report/_counts.json \
+            --slack-file security-report/_slack.txt \
+            >> $GITHUB_STEP_SUMMARY
+          # Expose counts as step output for downstream gating.
+          counts=$(cat security-report/_counts.json)
+          echo "counts=$counts" >> $GITHUB_OUTPUT
+          high=$(jq '.high + .critical' security-report/_counts.json)
+          echo "high_critical=$high" >> $GITHUB_OUTPUT
+
+      - name: Fail on high/critical Snyk findings
+        if: always() && steps.snyk-summary.outputs.high_critical != '' && steps.snyk-summary.outputs.high_critical != '0'
+        run: |
+          echo "::error::Snyk found ${{ steps.snyk-summary.outputs.high_critical }} high/critical vulnerabilities (see Job Summary)"
+          exit 1
+
+      - name: Generate Snyk HTML/PDF
+        if: always() && steps.maven-build.outcome == 'success'
+        run: |
+          # Back up JSONs because html_to_pdf.py deletes them after PDF conversion.
+          mkdir -p /tmp/snyk-json-backup
+          cp security-report/*.json /tmp/snyk-json-backup/ 2>/dev/null || true
+          make export-snyk-pdf-report || true
+          # Restore JSONs alongside generated PDFs/HTMLs.
+          cp /tmp/snyk-json-backup/*.json security-report/ 2>/dev/null || true
+
+      - name: Upload Snyk Reports
+        if: always() && steps.maven-build.outcome == 'success'
+        uses: actions/upload-artifact@v4
+        with:
+          name: security-report
+          path: security-report
+          retention-days: 30
+
+      - name: Force failure
+        if: steps.maven-build.outcome != 'success' || steps.security-report.outcome != 'success'
+        run: |
+          exit 1
+
+  notify:
+    runs-on: ubuntu-latest
+    environment: security-scan
+    needs: [vulnerability-scan, security-scan]
+    if: always()
+    steps:
+      - name: Download Snyk artifact
+        if: needs.security-scan.result != 'skipped'
+        uses: actions/download-artifact@v4
+        with:
+          name: security-report
+          path: security-report
+        continue-on-error: true
+
+      - name: Download Retire.js Slack artifact
+        if: needs.vulnerability-scan.result != 'skipped'
+        uses: actions/download-artifact@v4
+        with:
+          name: retire-slack
+          path: retire-report
+        continue-on-error: true
+
+      - name: Build Slack header payload
+        id: build-header
+        env:
+          RETIRE_RESULT: ${{ needs.vulnerability-scan.result }}
+          SNYK_RESULT: ${{ needs.security-scan.result }}
+          REF_NAME: ${{ github.ref_name }}
+          RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+        run: |
+          python3 - <<'PY' > header.json
+          import json, os, pathlib
+
+          retire = os.environ["RETIRE_RESULT"]
+          snyk = os.environ["SNYK_RESULT"]
+          ref_name = os.environ["REF_NAME"]
+          run_url = os.environ["RUN_URL"]
+
+          def status_icon(s):
+              return {"success": "✅", "cancelled": "⚠️ (cancelled)", "skipped": "⚠️ (skipped)"}.get(s, "❌")
+
+          def load_counts(path):
+              p = pathlib.Path(path)
+              if not p.exists():
+                  return None
+              try:
+                  return json.loads(p.read_text())
+              except Exception:
+                  return None
+
+          def totals_line(counts):
+              if not counts:
+                  return ""
+              return (
+                  f"\n🚨 {counts.get('critical', 0)} critical  ·  "
+                  f"🔴 {counts.get('high', 0)} high  ·  "
+                  f"🟠 {counts.get('medium', 0)} medium  ·  "
+                  f"🟡 {counts.get('low', 0)} low"
+              )
+
+          retire_counts = load_counts("retire-report/_retire_counts.json")
+          snyk_counts = load_counts("security-report/_counts.json")
+
+          if retire == "success" and snyk == "success":
+              overall = "🟢"
+          elif retire == "failure" or snyk == "failure":
+              overall = "🚨"
+          else:
+              overall = "⚠️"
+
+          header = (
+              f"{overall}  *Security scan* — *OpenMetadata Repo*\n"
+              f"_branch_ `{ref_name}`  ·  <{run_url}|Open run details>\n"
+              f"• Vulnerability scan (Retire.js): {status_icon(retire)}"
+              f"{totals_line(retire_counts)}\n"
+              f"• Security scan (Snyk): {status_icon(snyk)}"
+              f"{totals_line(snyk_counts)}"
+          )
+          fallback = f"{overall} Security scan — OpenMetadata Repo on {ref_name}. {run_url}"
+          payload = {
+              "text": fallback,
+              "blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": header}}],
+          }
+          print(json.dumps(payload))
+          PY
+
+      - name: Send Slack — header
+        id: send-header
+        if: env.SEND_SLACK_NOTIFICATION == 'true'
+        uses: slackapi/slack-github-action@v1.27.1
+        with:
+          channel-id: ${{ secrets.SLACK_CHANNEL_IDS }}
+          payload-file-path: header.json
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+
+      - name: Build Slack thread payload
+        id: build-thread
+        if: always() && steps.send-header.outputs.ts != ''
+        env:
+          THREAD_TS: ${{ steps.send-header.outputs.ts }}
+        run: |
+          python3 - <<'PY' > thread.json
+          import json, os, pathlib
+
+          thread_ts = os.environ["THREAD_TS"]
+
+          def read_sections(path):
+              p = pathlib.Path(path)
+              if not p.exists():
+                  return []
+              text = p.read_text().strip()
+              return [s.strip() for s in text.split("\n\n") if s.strip()]
+
+          retire_sections = read_sections("retire-report/_retire_slack.txt")
+          snyk_sections = read_sections("security-report/_slack.txt")
+
+          blocks = []
+          MAX_TEXT = 2850
+
+          def push(section):
+              text = section if len(section) <= MAX_TEXT else section[:MAX_TEXT].rstrip() + "\n_…truncated, see Job Summary_"
+              blocks.append({"type": "section", "text": {"type": "mrkdwn", "text": text}})
+
+          for section in retire_sections[:24]:
+              push(section)
+          if retire_sections and snyk_sections:
+              blocks.append({"type": "divider"})
+          for section in snyk_sections[:23]:
+              push(section)
+
+          if not blocks:
+              push("_No detailed scan output was produced._")
+
+          payload = {
+              "thread_ts": thread_ts,
+              "text": "Security scan details (thread reply)",
+              "blocks": blocks,
+          }
+          print(json.dumps(payload))
+          PY
+
+      - name: Send Slack — thread reply
+        if: always() && env.SEND_SLACK_NOTIFICATION == 'true' && steps.send-header.outputs.ts != '' && steps.build-thread.outcome == 'success'
+        uses: slackapi/slack-github-action@v1.27.1
+        with:
+          channel-id: ${{ secrets.SLACK_CHANNEL_IDS }}
+          payload-file-path: thread.json
+        env:
+          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}

.github/workflows/stale.yml

@@ -0,0 +1,36 @@
+name: Close Stale PRs
+
+on:
+  schedule:
+    - cron: '0 9 * * *'  # Runs daily at 9AM UTC
+  workflow_dispatch:
+
+permissions:
+  pull-requests: write
+  issues: write
+
+jobs:
+  stale:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/stale@v9
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          
+          # PRs only (disable for issues)
+          stale-issue-message: ''
+          close-issue-message: ''
+          days-before-issue-stale: -1
+          days-before-issue-close: -1
+          
+          # PR settings
+          days-before-pr-stale: 30       # Mark stale after 30 days
+          days-before-pr-close: 7       # Close 7 days after marking
+          stale-pr-message: |
+            This PR has had no activity for 30 days and will be closed in 7 days if no further activity occurs.
+            Feel free to reopen it if you'd like to continue working on it.
+          close-pr-message: |
+            Closing due to inactivity. Reopen anytime to continue.
+          exempt-pr-labels: 'wip'
+          exempt-draft-pr: true
+          operations-per-run: 50

.github/workflows/storybook-nightly.yml

@@ -0,0 +1,94 @@
+#  Copyright 2024 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# Builds openmetadata-ui-core-components Storybook nightly and pushes the image
+# to Docker Hub so the design team can review components without a local dev
+# environment.
+#
+# Required repository secrets (same ones used by other docker-* workflows):
+#   DOCKERHUB_OPENMETADATA_USERNAME
+#   DOCKERHUB_OPENMETADATA_TOKEN
+#
+# Image: openmetadata/storybook-ui:latest
+#         openmetadata/storybook-ui:<short-sha>
+#
+# To pull and run:
+#   docker run -p 6006:6006 openmetadata/storybook-ui:latest
+
+name: Storybook nightly build
+
+on:
+  schedule:
+    # 02:00 UTC every day.
+    - cron: '0 2 * * *'
+  # Allow a one-click manual run from the Actions tab.
+  workflow_dispatch:
+    inputs:
+      push_image:
+        description: "Push the built image to Docker Hub?"
+        type: boolean
+        default: true
+
+concurrency:
+  group: storybook-nightly
+  cancel-in-progress: true
+
+jobs:
+  build-and-push:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Get short commit SHA
+        id: sha
+        run: echo "short=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT"
+
+      - name: Prepare Docker build
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build-and-push
+        with:
+          image: openmetadata/storybook-ui
+          tag: value=${{ steps.sha.outputs.short }}
+          push_latest: "true"
+          is_ingestion: "false"
+          dockerhub_username: ${{ secrets.DOCKERHUB_OPENMETADATA_USERNAME }}
+          dockerhub_token: ${{ secrets.DOCKERHUB_OPENMETADATA_TOKEN }}
+
+      - name: Build and push
+        uses: docker/build-push-action@v6
+        env:
+          DOCKER_BUILD_NO_SUMMARY: true
+        with:
+          context: .
+          file: docker/storybook/Dockerfile
+          platforms: linux/amd64,linux/arm64
+          # On scheduled runs always push; on workflow_dispatch respect the input.
+          push: ${{ github.event_name == 'schedule' || inputs.push_image }}
+          tags: ${{ steps.prepare.outputs.tags }}
+
+      - name: Print image reference
+        run: |
+          echo "### Storybook image published" >> "$GITHUB_STEP_SUMMARY"
+          echo "" >> "$GITHUB_STEP_SUMMARY"
+          echo '```' >> "$GITHUB_STEP_SUMMARY"
+          echo "${{ steps.prepare.outputs.tags }}" >> "$GITHUB_STEP_SUMMARY"
+          echo '```' >> "$GITHUB_STEP_SUMMARY"
+          echo "" >> "$GITHUB_STEP_SUMMARY"
+          echo "Run locally:" >> "$GITHUB_STEP_SUMMARY"
+          echo '```bash' >> "$GITHUB_STEP_SUMMARY"
+          echo "docker run -p 6006:6006 openmetadata/storybook-ui:latest" >> "$GITHUB_STEP_SUMMARY"
+          echo '```' >> "$GITHUB_STEP_SUMMARY"

.github/workflows/team-labeler.yml

@@ -0,0 +1,40 @@
+on:
+  pull_request_target:
+
+permissions:
+  contents: read
+  pull-requests: write
+  issues: write
+
+name: Team Label
+jobs:
+  labeler:
+    runs-on: ubuntu-latest
+    name: Team Label
+    steps:
+      - uses: JulienKode/team-labeler-action@v0.1.1
+        with:
+          repo-token: "${{ secrets.GITHUB_TOKEN }}"
+
+      - name: Verify PR labels
+        id: verify
+        continue-on-error: true
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        with:
+          github-token: '${{ secrets.GITHUB_TOKEN }}'
+          valid-labels: 'safe to test'
+          pull-request-number: '${{ github.event.pull_request.number }}'
+          disable-reviews: true  # To not auto approve changes
+
+      - name: Add verification comment
+        if: steps.verify.outcome != 'success'
+        uses: peter-evans/create-or-update-comment@v1
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          body: |
+            **Hi there 👋 Thanks for your contribution!**
+            
+            The OpenMetadata team will review the PR shortly! Once it has been labeled as `safe to test`, the CI workflows
+            will start executing and we'll be able to make sure everything is working as expected.
+            
+            Let us know if you need any help!

.github/workflows/trivy-scan-ingestion-base-slim-image.yml

@@ -0,0 +1,57 @@
+name: Trivy Scan For OpenMetadata Ingestion Base Slim Docker Image
+
+on:
+    workflow_dispatch:
+    
+concurrency: 
+  group: trivy-ingestion-base-slim-scan-${{ github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  build-and-scan:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout Repository
+        uses: actions/checkout@v4
+            
+      - name: Prepare for Docker Build
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build
+        with:
+          image: openmetadata-ingestion-base-slim
+          tag: trivy
+          is_ingestion: true
+
+
+      - name: Build Docker Image
+        run: |
+          docker build -t openmetadata-ingestion-base-slim:trivy -f ingestion/operators/docker/Dockerfile.ci .
+
+
+      - name: Run Trivy Image Scan
+        id: trivy_scan
+        uses: aquasecurity/trivy-action@0.35.0
+        with:
+          scan-type: "image"
+          image-ref: openmetadata-ingestion-base-slim:trivy
+          hide-progress: false
+          ignore-unfixed: true
+          severity: "HIGH,CRITICAL"
+          skip-dirs: "/opt/airflow/dags,/home/airflow/ingestion/pipelines"
+          scan-ref: .
+          format: 'template'
+          template: "@.github/trivy/templates/github.tpl"
+          output: "trivy-result-ingestion-base-slim.md"
+        env:
+          TRIVY_DISABLE_VEX_NOTICE: "true"

.github/workflows/trivy-scan-ingestion-image.yml

@@ -0,0 +1,57 @@
+name: Trivy Scan For OpenMetadata Ingestion Docker Image
+
+on:
+    workflow_dispatch:
+    
+concurrency: 
+  group: trivy-ingestion-scan-${{ github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  build-and-scan:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Free Disk Space (Ubuntu)
+        uses: jlumbroso/free-disk-space@main
+        with:
+          tool-cache: false
+          android: true
+          dotnet: true
+          haskell: true
+          large-packages: false
+          swap-storage: true
+          docker-images: false
+
+      - name: Checkout Repository
+        uses: actions/checkout@v4
+            
+      - name: Prepare for Docker Build
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build
+        with:
+          image: openmetadata-ingestion
+          tag: trivy
+          is_ingestion: true
+
+
+      - name: Build Docker Image
+        run: |
+          docker build -t openmetadata-ingestion:trivy -f ingestion/Dockerfile.ci .
+
+
+      - name: Run Trivy Image Scan
+        id: trivy_scan
+        uses: aquasecurity/trivy-action@0.35.0
+        with:
+          scan-type: "image"
+          image-ref: openmetadata-ingestion:trivy
+          hide-progress: false
+          ignore-unfixed: true
+          severity: "HIGH,CRITICAL"
+          skip-dirs: "/opt/airflow/dags,/home/airflow/ingestion/pipelines"
+          scan-ref: .
+          format: 'template'
+          template: "@.github/trivy/templates/github.tpl"
+          output: "trivy-results-ingestion.md"
+        env:
+          TRIVY_DISABLE_VEX_NOTICE: "true"

.github/workflows/trivy-scan-openmetadata-server.yml

@@ -0,0 +1,56 @@
+name: Trivy Scan For OpenMetadata Server Docker Image
+
+on:
+  workflow_dispatch:
+
+concurrency: 
+  group: trivy-server-scan-${{ github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  build-and-scan:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout Repository
+        uses: actions/checkout@v4
+
+      - name: Prepare for Docker Build
+        id: prepare
+        uses: ./.github/actions/prepare-for-docker-build
+        with:
+          image: openmetadata-server
+          tag: trivy
+          is_ingestion: false
+
+      - name: Build Docker Image
+        run: |
+          docker build -t openmetadata-server:trivy -f docker/development/Dockerfile .
+      - name: Run Trivy Image Scan
+        uses: aquasecurity/trivy-action@0.35.0
+        with:
+          scan-type: "image"
+          image-ref: openmetadata-server:trivy
+          hide-progress: true
+          ignore-unfixed: true
+          severity: "HIGH,CRITICAL,MEDIUM"
+          format: "table"
+          output: trivy.txt
+        env:
+          TRIVY_DISABLE_VEX_NOTICE: "true"
+
+      - name: Publish Trivy Output to Summary
+        run: |
+          if [[ -s trivy.txt ]]; then
+            {
+              echo "###  Trivy Security Scan Results"
+              echo "<details><summary>Click to expand</summary>"
+              echo ""
+              echo '```text'
+              cat trivy.txt
+              echo '```'
+              echo "</details>"
+            } >> $GITHUB_STEP_SUMMARY
+          else
+            echo "No vulnerabilities found." >> $GITHUB_STEP_SUMMARY
+          fi

.github/workflows/typescript-type-generation.yml

@@ -0,0 +1,200 @@
+name: TypeScript Type Generation
+
+on:
+  merge_group:
+  pull_request_target:
+    types: [opened, synchronize, reopened, labeled, ready_for_review]
+    paths:
+      - 'openmetadata-spec/src/main/resources/json/schema/**'
+      - 'openmetadata-ui/src/main/resources/ui/src/generated/**'
+  workflow_dispatch:
+    inputs:
+      branch:
+        description: 'Branch to run the workflow on (ignored if pr_number is provided)'
+        required: false
+        default: 'main'
+        type: string
+      pr_number:
+        description: 'PR number to run the workflow for (works for both forked and non-forked PRs)'
+        required: false
+        type: string
+
+concurrency:
+  group: typescript-generation-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+jobs:
+  generate-types:
+    runs-on: ubuntu-latest
+    if: ${{ !github.event.pull_request.draft && (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test') }}
+    permissions:
+      contents: write
+      pull-requests: write
+    steps:
+      - name: Get PR details for workflow_dispatch
+        if: github.event_name == 'workflow_dispatch' && github.event.inputs.pr_number != ''
+        id: pr-details
+        env:
+          PR_NUMBER: ${{ github.event.inputs.pr_number }}
+          BASE_REPO: ${{ github.repository }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          PR_DATA=$(gh pr view "$PR_NUMBER" --json headRepository,headRefName,headRepositoryOwner,headRefOid --repo "$BASE_REPO")
+          echo "head_repo=$(echo "$PR_DATA" | jq -r '.headRepository.name')" >> $GITHUB_OUTPUT
+          echo "head_owner=$(echo "$PR_DATA" | jq -r '.headRepositoryOwner.login')" >> $GITHUB_OUTPUT
+          echo "head_ref=$(echo "$PR_DATA" | jq -r '.headRefName')" >> $GITHUB_OUTPUT
+          echo "head_sha=$(echo "$PR_DATA" | jq -r '.headRefOid')" >> $GITHUB_OUTPUT
+          echo "full_repo=$(echo "$PR_DATA" | jq -r '.headRepositoryOwner.login + "/" + .headRepository.name')" >> $GITHUB_OUTPUT
+
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' && github.actor != 'github-actions[bot]' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' && github.actor != 'github-actions[bot]' }}
+        with:
+          github-token: '${{ secrets.GITHUB_TOKEN }}'
+          valid-labels: 'safe to test'
+          pull-request-number: '${{ github.event.pull_request.number }}'
+          disable-reviews: true
+
+      - name: Check if repository is a fork
+        id: check-fork
+        env:
+          EVENT_NAME: ${{ github.event_name }}
+          FULL_REPO: ${{ steps.pr-details.outputs.full_repo }}
+          HEAD_REPO_FULL_NAME: ${{ github.event.pull_request.head.repo.full_name }}
+          BASE_REPO: ${{ github.repository }}
+        run: |
+          if [ "$EVENT_NAME" == "workflow_dispatch" ] && [ -n "$FULL_REPO" ]; then
+            # For workflow_dispatch with PR number
+            if [ "$FULL_REPO" != "$BASE_REPO" ]; then
+              echo "is_fork=true" >> $GITHUB_OUTPUT
+            else
+              echo "is_fork=false" >> $GITHUB_OUTPUT
+            fi
+          elif [ "$EVENT_NAME" == "pull_request_target" ]; then
+            # For pull_request_target events
+            if [ "$HEAD_REPO_FULL_NAME" != "$BASE_REPO" ]; then
+              echo "is_fork=true" >> $GITHUB_OUTPUT
+            else
+              echo "is_fork=false" >> $GITHUB_OUTPUT
+            fi
+          else
+            # Default to non-fork for direct branch runs
+            echo "is_fork=false" >> $GITHUB_OUTPUT
+          fi
+
+      - name: Determine checkout ref
+        id: checkout-ref
+        env:
+          IS_FORK: ${{ steps.check-fork.outputs.is_fork }}
+          EVENT_NAME: ${{ github.event_name }}
+          PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+          PR_HEAD_REF: ${{ github.event.pull_request.head.ref }}
+          PR_DETAILS_HEAD_SHA: ${{ steps.pr-details.outputs.head_sha }}
+          PR_DETAILS_HEAD_REF: ${{ steps.pr-details.outputs.head_ref }}
+          INPUT_BRANCH: ${{ github.event.inputs.branch }}
+          GITHUB_REF: ${{ github.ref }}
+          GITHUB_SHA_VAL: ${{ github.sha }}
+        run: |
+          if [ "$EVENT_NAME" == "merge_group" ]; then
+            echo "ref=$GITHUB_SHA_VAL" >> $GITHUB_OUTPUT
+          elif [ "$IS_FORK" == "true" ] && [ "$EVENT_NAME" == "pull_request_target" ]; then
+            echo "ref=$PR_HEAD_SHA" >> $GITHUB_OUTPUT
+          elif [ "$EVENT_NAME" == "pull_request_target" ]; then
+            echo "ref=$PR_HEAD_REF" >> $GITHUB_OUTPUT
+          elif [ "$IS_FORK" == "true" ] && [ -n "$PR_DETAILS_HEAD_SHA" ]; then
+            echo "ref=$PR_DETAILS_HEAD_SHA" >> $GITHUB_OUTPUT
+          elif [ -n "$PR_DETAILS_HEAD_REF" ]; then
+            echo "ref=$PR_DETAILS_HEAD_REF" >> $GITHUB_OUTPUT
+          elif [ -n "$INPUT_BRANCH" ]; then
+            echo "ref=$INPUT_BRANCH" >> $GITHUB_OUTPUT
+          else
+            echo "ref=$GITHUB_REF" >> $GITHUB_OUTPUT
+          fi
+
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ steps.pr-details.outputs.full_repo || github.repository }}
+          ref: ${{ steps.checkout-ref.outputs.ref }}
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: 'openmetadata-ui/src/main/resources/ui/.nvmrc'
+
+      - name: Install dependencies
+        run: |
+          yarn install --frozen-lockfile
+
+      - name: Generate TypeScript types
+        run: |
+          cd openmetadata-ui/src/main/resources/ui
+          # Create a symlink to the root node_modules
+          ln -sf ../../../../../node_modules .
+          ./json2ts-generate-all.sh -l true
+
+      - name: Check for changes
+        id: git-check
+        continue-on-error: true
+        run: |
+          git add openmetadata-ui/src/main/resources/ui/src/generated/
+          git diff --cached --quiet
+
+      - name: Configure Git
+        if: steps.git-check.outcome != 'success'
+        run: |
+          git config --global user.name 'github-actions[bot]'
+          git config --global user.email 'github-actions[bot]@users.noreply.github.com'
+
+      - name: Commit and push changes
+        if: steps.git-check.outcome != 'success' && steps.check-fork.outputs.is_fork == 'false'
+        run: |
+          git commit -m "Update generated TypeScript types"
+          git push --force-with-lease
+
+      - name: Create PR comment about auto-update
+        if: steps.git-check.outcome != 'success' && steps.check-fork.outputs.is_fork == 'false' && (github.event_name == 'pull_request_target' || github.event.inputs.pr_number != '')
+        uses: peter-evans/create-or-update-comment@v4
+        with:
+          issue-number: ${{ github.event.pull_request.number || github.event.inputs.pr_number }}
+          body: |
+            ## ✅ TypeScript Types Auto-Updated
+
+            The generated TypeScript types have been automatically updated based on JSON schema changes in this PR.
+
+      - name: Create PR comment for fork repository
+        if: steps.git-check.outcome != 'success' && steps.check-fork.outputs.is_fork == 'true' && (github.event_name == 'pull_request_target' || github.event.inputs.pr_number != '')
+        uses: peter-evans/create-or-update-comment@v4
+        with:
+          issue-number: ${{ github.event.pull_request.number || github.event.inputs.pr_number }}
+          body: |
+            ## ⚠️ TypeScript Types Need Update
+
+            **The generated TypeScript types are out of sync with the JSON schema changes.**
+
+            Since this is a pull request from a forked repository, the types cannot be automatically committed.
+            Please generate and commit the types manually:
+
+            ```bash
+            cd openmetadata-ui/src/main/resources/ui
+            ./json2ts-generate-all.sh -l true
+            git add src/generated/
+            git commit -m "Update generated TypeScript types"
+            git push
+            ```
+
+            After pushing the changes, this check will pass automatically.
+
+      - name: Fail workflow for fork PRs with type changes
+        if: steps.git-check.outcome != 'success' && steps.check-fork.outputs.is_fork == 'true'
+        run: exit 1

.github/workflows/ui-checkstyle.yml

@@ -0,0 +1,418 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: UI Checkstyle
+
+on:
+  merge_group:
+  # Note: paths filter removed — the workflow always triggers so that the
+  # required status check (report) always completes. Path filtering is handled
+  # inside the workflow via dorny/paths-filter so PRs without UI changes skip
+  # the expensive jobs while still reporting a passing status.
+  pull_request_target:
+    types:
+      - opened
+      - synchronize
+      - reopened
+      - ready_for_review
+      - labeled
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ui-checkstyle-${{ github.head_ref || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+
+env:
+  UI_WORKING_DIRECTORY: openmetadata-ui/src/main/resources/ui
+  CORE_COMPONENTS_WORKING_DIRECTORY: openmetadata-ui-core-components/src/main/resources/ui
+
+jobs:
+  check-changes:
+    runs-on: ubuntu-latest
+    outputs:
+      ui-changed: ${{ steps.filter.outputs.ui }}
+    steps:
+      - uses: dorny/paths-filter@v4
+        id: filter
+        with:
+          filters: |
+            ui:
+              - 'openmetadata-ui/src/main/resources/ui/**'
+              - 'openmetadata-spec/src/main/resources/json/schema/**'
+              - '.github/workflows/ui-checkstyle.yml'
+              - 'openmetadata-ui-core-components/src/main/resources/ui/**'
+
+  authorize:
+    needs: check-changes
+    if: |
+      github.event_name == 'merge_group' ||
+      (github.event_name == 'pull_request_target' && needs.check-changes.outputs.ui-changed == 'true' &&
+      (github.event.action != 'labeled' || github.event.label.name == 'safe to test'))
+    runs-on: ubuntu-latest
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true
+
+  checkstyle:
+    needs: authorize
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    outputs:
+      lint_src_result: ${{ steps.lint_src.outcome }}
+      lint_src_changed_files: ${{ steps.lint_src.outputs.changed_files }}
+      license_result: ${{ steps.license.outcome }}
+      license_changed_files: ${{ steps.license.outputs.changed_files }}
+      i18n_result: ${{ steps.i18n.outcome }}
+      i18n_changed_files: ${{ steps.i18n.outputs.changed_files }}
+      app_docs_result: ${{ steps.app_docs.outcome }}
+      app_docs_changed_files: ${{ steps.app_docs.outputs.changed_files }}
+      lint_playwright_result: ${{ steps.lint_playwright.outcome }}
+      lint_playwright_changed_files: ${{ steps.lint_playwright.outputs.changed_files }}
+      lint_core_components_result: ${{ steps.lint_core_components.outcome }}
+      lint_core_components_changed_files: ${{ steps.lint_core_components.outputs.changed_files }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+          fetch-depth: 0
+          filter: blob:none
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version-file: "${{ env.UI_WORKING_DIRECTORY }}/.nvmrc"
+          cache: yarn
+          cache-dependency-path: |
+            ${{ env.UI_WORKING_DIRECTORY }}/yarn.lock
+            ${{ env.CORE_COMPONENTS_WORKING_DIRECTORY }}/yarn.lock
+
+      - name: Install Antlr4 CLI
+        run: sudo make install_antlr_cli
+
+      - name: Install UI Yarn Packages
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        run: yarn install --frozen-lockfile
+
+      - name: Get changed src files
+        id: changed-src-files
+        uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323
+        with:
+          path: ${{ env.UI_WORKING_DIRECTORY }}
+          files_ignore: |
+            src/generated/**
+          files: |
+            src/**/*.{ts,tsx,js,jsx,json}
+
+      - name: ESLint + Prettier + Organise Imports (src)
+        id: lint_src
+        if: steps.changed-src-files.outputs.any_changed == 'true'
+        continue-on-error: true
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        env:
+          CHANGED_FILES: ${{ steps.changed-src-files.outputs.all_changed_files }}
+        run: |
+          if [ -z "$CHANGED_FILES" ]; then
+            echo "No added or modified files to process."
+            exit 0
+          fi
+          TS_FILES=$(echo "$CHANGED_FILES" | tr ' ' '\n' | awk '/\.(ts|tsx|js|jsx)$/ { printf "%s ", $0 }')
+          if [ -n "$TS_FILES" ]; then
+            yarn organize-imports:cli $TS_FILES
+          fi
+          yarn lint:base --fix $CHANGED_FILES
+          yarn pretty:base --write $CHANGED_FILES
+          if [ -n "$(git status --porcelain)" ]; then
+            FILES=$(git status --porcelain | awk '{print "  - `" $2 "`"}' | head -30)
+            echo "changed_files<<EOF" >> "$GITHUB_OUTPUT"
+            echo "$FILES" >> "$GITHUB_OUTPUT"
+            echo "EOF" >> "$GITHUB_OUTPUT"
+            git checkout -- .
+            git clean -fd
+            exit 1
+          fi
+
+      - name: Get all changed UI files
+        id: changed-ui-files
+        uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323
+        with:
+          path: ${{ env.UI_WORKING_DIRECTORY }}
+
+      - name: Licence Header Check
+        id: license
+        if: steps.changed-ui-files.outputs.any_changed == 'true'
+        continue-on-error: true
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        env:
+          CHANGED_FILES_ALL: ${{ steps.changed-ui-files.outputs.all_changed_files }}
+        run: |
+          if [ -z "$CHANGED_FILES_ALL" ]; then
+            echo "No added or modified files to process."
+            exit 0
+          fi
+          yarn license-header-fix $CHANGED_FILES_ALL
+          if [ -n "$(git status --porcelain)" ]; then
+            FILES=$(git status --porcelain | awk '{print "  - `" $2 "`"}' | head -30)
+            echo "changed_files<<EOF" >> "$GITHUB_OUTPUT"
+            echo "$FILES" >> "$GITHUB_OUTPUT"
+            echo "EOF" >> "$GITHUB_OUTPUT"
+            git checkout -- .
+            git clean -fd
+            exit 1
+          fi
+
+      - name: I18n Sync
+        id: i18n
+        continue-on-error: true
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        run: |
+          yarn i18n
+          if [ -n "$(git status --porcelain)" ]; then
+            FILES=$(git status --porcelain | awk '{print "  - `" $2 "`"}' | head -20)
+            echo "changed_files<<EOF" >> "$GITHUB_OUTPUT"
+            echo "$FILES" >> "$GITHUB_OUTPUT"
+            echo "EOF" >> "$GITHUB_OUTPUT"
+            git checkout -- .
+            git clean -fd
+            exit 1
+          fi
+
+      - name: generate:app-docs
+        id: app_docs
+        continue-on-error: true
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        run: |
+          yarn generate:app-docs
+          if [ -n "$(git status --porcelain)" ]; then
+            FILES=$(git status --porcelain | awk '{print "  - `" $2 "`"}' | head -20)
+            echo "changed_files<<EOF" >> "$GITHUB_OUTPUT"
+            echo "$FILES" >> "$GITHUB_OUTPUT"
+            echo "EOF" >> "$GITHUB_OUTPUT"
+            git checkout -- .
+            git clean -fd
+            exit 1
+          fi
+
+      - name: Get changed Playwright files
+        id: changed-playwright-files
+        uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323
+        with:
+          path: ${{ env.UI_WORKING_DIRECTORY }}
+          files_ignore: |
+            playwright/test-data/**
+          files: |
+            playwright/**/*.{ts,tsx,js,jsx}
+
+      - name: ESLint + Prettier + Organise Imports (playwright)
+        id: lint_playwright
+        if: steps.changed-playwright-files.outputs.any_changed == 'true'
+        continue-on-error: true
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        env:
+          CHANGED_FILES: ${{ steps.changed-playwright-files.outputs.all_changed_files }}
+        run: |
+          if [ -z "$CHANGED_FILES" ]; then
+            echo "No added or modified files to process."
+            exit 0
+          fi
+          yarn organize-imports:cli $CHANGED_FILES
+          yarn lint:base --fix $CHANGED_FILES
+          yarn pretty:base --write $CHANGED_FILES
+          if [ -n "$(git status --porcelain)" ]; then
+            FILES=$(git status --porcelain | awk '{print "  - `" $2 "`"}' | head -30)
+            echo "changed_files<<EOF" >> "$GITHUB_OUTPUT"
+            echo "$FILES" >> "$GITHUB_OUTPUT"
+            echo "EOF" >> "$GITHUB_OUTPUT"
+            git checkout -- .
+            git clean -fd
+            exit 1
+          fi
+
+      - name: Install Core Components Yarn Packages
+        working-directory: ${{ env.CORE_COMPONENTS_WORKING_DIRECTORY }}
+        run: yarn install --frozen-lockfile
+
+      - name: Get changed core-components files
+        id: changed-core-components-files
+        uses: tj-actions/changed-files@22103cc46bda19c2b464ffe86db46df6922fd323
+        with:
+          path: ${{ env.CORE_COMPONENTS_WORKING_DIRECTORY }}
+          files: |
+            src/**/*.{ts,tsx,js,jsx,json}
+
+      - name: ESLint + Prettier (core-components)
+        id: lint_core_components
+        if: steps.changed-core-components-files.outputs.any_changed == 'true'
+        continue-on-error: true
+        working-directory: ${{ env.CORE_COMPONENTS_WORKING_DIRECTORY }}
+        env:
+          CHANGED_FILES: ${{ steps.changed-core-components-files.outputs.all_changed_files }}
+        run: |
+          if [ -z "${CHANGED_FILES// }" ]; then
+            echo "No added or modified files to process."
+            exit 0
+          fi
+          yarn lint:base --fix $CHANGED_FILES
+          yarn pretty:base --write $CHANGED_FILES
+          if [ -n "$(git status --porcelain)" ]; then
+            FILES=$(git status --porcelain | awk '{print "  - `" $2 "`"}' | head -30)
+            echo "changed_files<<EOF" >> "$GITHUB_OUTPUT"
+            echo "$FILES" >> "$GITHUB_OUTPUT"
+            echo "EOF" >> "$GITHUB_OUTPUT"
+            git checkout -- .
+            git clean -fd
+            exit 1
+          fi
+
+  ui-checkstyle:
+    needs: [authorize, checkstyle]
+    if: always()
+    runs-on: ubuntu-latest
+    permissions:
+      pull-requests: write
+    steps:
+      - name: Find existing summary comment
+        uses: peter-evans/find-comment@v3
+        id: fc
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          comment-author: github-actions[bot]
+          body-includes: '<!-- check:ui-checkstyle-summary -->'
+
+      - name: Post or update summary comment
+        uses: actions/github-script@v7
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          script: |
+            const checks = [
+              {
+                name:    'ESLint + Prettier + Organise Imports (src)',
+                reason:  'One or more source files have linting or formatting issues.',
+                result:  '${{ needs.checkstyle.outputs.lint_src_result }}',
+                files:   ${{ toJSON(needs.checkstyle.outputs.lint_src_changed_files) }} || '',
+              },
+              {
+                name:    'Licence Header',
+                reason:  'One or more files are missing or have an outdated Apache 2.0 licence header.',
+                result:  '${{ needs.checkstyle.outputs.license_result }}',
+                files:   ${{ toJSON(needs.checkstyle.outputs.license_changed_files) }} || '',
+              },
+              {
+                name:    'I18n Sync',
+                reason:  'Translation locale files are out of sync with `en-us.json`.',
+                result:  '${{ needs.checkstyle.outputs.i18n_result }}',
+                files:   ${{ toJSON(needs.checkstyle.outputs.i18n_changed_files) }} || '',
+              },
+              {
+                name:    'App Docs',
+                reason:  'Generated application docs are stale and need to be regenerated.',
+                result:  '${{ needs.checkstyle.outputs.app_docs_result }}',
+                files:   ${{ toJSON(needs.checkstyle.outputs.app_docs_changed_files) }} || '',
+              },
+              {
+                name:    'Playwright - ESLint + Prettier + Organise Imports',
+                reason:  'One or more Playwright test files have linting or formatting issues.',
+                result:  '${{ needs.checkstyle.outputs.lint_playwright_result }}',
+                files:   ${{ toJSON(needs.checkstyle.outputs.lint_playwright_changed_files) }} || '',
+              },
+              {
+                name:    'Core Components - ESLint + Prettier',
+                reason:  'One or more core-component files have linting or formatting issues.',
+                result:  '${{ needs.checkstyle.outputs.lint_core_components_result }}',
+                files:   ${{ toJSON(needs.checkstyle.outputs.lint_core_components_changed_files) }} || '',
+              },
+            ];
+
+            const failures = checks.filter(c => c.result === 'failure');
+            const commentId = '${{ steps.fc.outputs.comment-id }}';
+
+            if (failures.length === 0) {
+              if (commentId) {
+                await github.rest.issues.deleteComment({
+                  owner: context.repo.owner,
+                  repo: context.repo.repo,
+                  comment_id: parseInt(commentId, 10),
+                });
+              }
+              return;
+            }
+
+            const sections = failures.map(c => {
+              const lines = [`#### ❌ ${c.name}`, c.reason];
+              if (c.files && c.files.trim()) {
+                lines.push('', '<details><summary>Affected files</summary>', '', c.files.trim(), '', '</details>');
+              }
+              return lines.join('\n');
+            });
+
+            const body = [
+              '<!-- check:ui-checkstyle-summary -->',
+              '### ❌ UI Checkstyle Failed',
+              '',
+              ...sections.flatMap(s => [s, '']),
+              '---',
+              '**Fix locally (fast - only checks files changed in this branch):**',
+              '```bash',
+              'make ui-checkstyle-changed',
+              '```',
+            ].join('\n');
+
+            if (commentId) {
+              await github.rest.issues.updateComment({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                comment_id: parseInt(commentId, 10),
+                body,
+              });
+            } else {
+              await github.rest.issues.createComment({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                issue_number: context.issue.number,
+                body,
+              });
+            }
+
+      - name: Check final results
+        if: always()
+        run: |
+          checkstyle_result="${{ needs.checkstyle.result }}"
+          if [[ "$checkstyle_result" != "success" && "$checkstyle_result" != "skipped" ]]; then
+            echo "Checkstyle job ended with status: $checkstyle_result"
+            exit 1
+          fi
+          if [ "${{ needs.checkstyle.outputs.lint_src_result }}" = 'failure' ] || \
+             [ "${{ needs.checkstyle.outputs.license_result }}" = 'failure' ] || \
+             [ "${{ needs.checkstyle.outputs.i18n_result }}" = 'failure' ] || \
+             [ "${{ needs.checkstyle.outputs.app_docs_result }}" = 'failure' ] || \
+             [ "${{ needs.checkstyle.outputs.lint_playwright_result }}" = 'failure' ] || \
+             [ "${{ needs.checkstyle.outputs.lint_core_components_result }}" = 'failure' ]; then
+            echo "One or more checks failed."
+            exit 1
+          fi
+          echo "All checks passed or were not required for this PR."

.github/workflows/update-playwright-e2e-docs.yml

@@ -0,0 +1,93 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Update Playwright E2E Documentation
+on:
+  workflow_dispatch:
+
+jobs:
+  update-docs:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pull-requests: write
+    steps:
+      - name: Validate Branch
+        run: |
+          if [ "${{ github.ref }}" != "refs/heads/main" ]; then
+            echo "This workflow can only be run on the main branch."
+            exit 1
+          fi
+
+      - name: Checkout Repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: "openmetadata-ui/src/main/resources/ui/.nvmrc"
+          cache: "yarn"
+          cache-dependency-path: openmetadata-ui/src/main/resources/ui/yarn.lock
+
+      - name: Install Yarn
+        run: corepack enable
+
+      - name: Install Dependencies
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: yarn install --frozen-lockfile --ignore-scripts
+
+      - name: Install Playwright Browsers
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: npx playwright install chromium --with-deps
+
+      - name: Generate E2E Docs
+        working-directory: openmetadata-ui/src/main/resources/ui
+        run: yarn generate:e2e-docs
+
+      - name: Detect Changes
+        id: git-check
+        run: |
+          if [ -z "$(git status --porcelain openmetadata-ui/src/main/resources/ui/playwright/docs)" ]; then
+            echo "No changes detected."
+            echo "changed=false" >> $GITHUB_OUTPUT
+          else
+            echo "Changes detected."
+            echo "changed=true" >> $GITHUB_OUTPUT
+          fi
+
+      - name: Create Pull Request
+        if: steps.git-check.outputs.changed == 'true'
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          BRANCH_NAME="chore/update-playwright-docs"
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+
+          git checkout -B $BRANCH_NAME
+          git add openmetadata-ui/src/main/resources/ui/playwright/docs
+          git commit -m "chore: update Playwright E2E documentation"
+          git push origin $BRANCH_NAME --force
+
+          gh pr create \
+            --title "chore: update Playwright E2E documentation" \
+            --body "This is an automated PR to update the Playwright E2E documentation. Generated by the \`Update Playwright E2E Documentation\` workflow." \
+            --base main \
+            --head $BRANCH_NAME || {
+              if gh pr list --head $BRANCH_NAME --state open --json number -jq '.[0].number' | grep -q .; then
+                echo "PR already exists"
+              else
+                echo "Failed to create PR"
+                exit 1
+              fi
+            }

.github/workflows/validate-docker-compose-quickstart.yml

@@ -0,0 +1,42 @@
+#  Copyright 2025 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Validate Docker Compose Quickstart
+
+on:
+  workflow_dispatch:
+  pull_request:
+    paths:
+      - 'docker/docker-compose-quickstart/**'
+      - '.github/workflows/validate-docker-compose-quickstart.yml'
+      - '.github/actions/validate-omd-docker-compose/**'
+
+permissions:
+  contents: read
+
+jobs:
+  validate-docker-compose:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    env:
+      OPENMETADATA_DB_IMAGE: docker.getcollate.io/openmetadata/db:latest
+      OPENMETADATA_SERVER_IMAGE: docker.getcollate.io/openmetadata/server:latest
+      OPENMETADATA_INGESTION_IMAGE: docker.getcollate.io/openmetadata/ingestion:latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Validate OpenMetadata Docker Compose
+        uses: ./.github/actions/validate-omd-docker-compose
+        with:
+          compose_file: ./docker/docker-compose-quickstart/docker-compose.yml
+          health_check_timeout: 300

.github/workflows/validate-jsons-yamls.yml

@@ -0,0 +1,85 @@
+#  Copyright 2024 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+name: Validate JSON/YAML
+
+# read-write repo token
+# access to secrets
+on:
+  merge_group:
+  pull_request_target:
+    types: [labeled, opened, synchronize, reopened]
+    paths:
+      - "**.json"
+      - "**.yaml"
+      - "**.yml"
+
+permissions:
+  contents: read
+
+jobs:
+  validate-json-yaml:
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+    permissions:
+      pull-requests: write
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: '${{ secrets.GITHUB_TOKEN }}'
+          valid-labels: 'safe to test'
+          pull-request-number: '${{ github.event.pull_request.number }}'
+          disable-reviews: true  # To not auto approve changes
+
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install Ubuntu related dependencies
+        run: |
+          sudo apt-get update  && sudo apt-get install -y libsasl2-dev unixodbc-dev python3-venv
+
+      - name: Install Python & Openmetadata related dependencies
+        run: |
+          python3 -m venv env
+          source env/bin/activate
+          pip install pyyaml
+
+      # Add back linting once we have 10/10 on main
+      - name: Code style check
+        id: style
+        continue-on-error: true
+        run: |
+          source env/bin/activate
+          pip install pyyaml
+          ./scripts/validate_json_yaml.sh
+
+      - name: JSON/Yaml validation failed, check the comment in the PR
+        if: steps.style.outcome != 'success'
+        run: |
+          exit 1

.github/workflows/yarn-coverage.yml

@@ -0,0 +1,179 @@
+name: SonarCloud + Jest Coverage
+
+on:
+  merge_group:
+  workflow_dispatch:
+  # Trigger analysis when pushing in master or pull requests, and when creating
+  # a pull request.
+  # Note: paths filter removed — the workflow always triggers so that the
+  # required status check (coverage-result) always completes. Path filtering
+  # is handled inside the workflow via dorny/paths-filter, which lets PRs
+  # without UI changes skip the expensive jobs while still reporting a
+  # passing status.
+  pull_request_target:
+    types: [opened, synchronize, reopened, labeled, ready_for_review]
+
+permissions:
+  contents: read
+  pull-requests: write # Required for Providing Jest Coverage Comment
+
+env:
+  UI_WORKING_DIRECTORY: openmetadata-ui/src/main/resources/ui
+  UI_COVERAGE_DIRECTORY: openmetadata-ui/src/main/resources/ui/src/test/unit/coverage
+concurrency:
+  group: yarn-coverage-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: ${{ github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test' }}
+jobs:
+  check-changes:
+    runs-on: ubuntu-latest
+    outputs:
+      ui-changed: ${{ steps.filter.outputs.ui }}
+    steps:
+      - uses: dorny/paths-filter@v4
+        id: filter
+        with:
+          filters: |
+            ui:
+              - 'openmetadata-ui/src/main/resources/ui/src/**'
+              - 'openmetadata-ui/src/main/resources/ui/jest.config.js'
+              - 'openmetadata-ui/src/main/resources/ui/package.json'
+              - 'openmetadata-ui/src/main/resources/ui/yarn.lock'
+              - 'openmetadata-ui/src/main/resources/ui/tsconfig.json'
+              - 'openmetadata-ui/src/main/resources/ui/babel.config.json'
+              - 'openmetadata-ui/src/main/resources/ui/.nvmrc'
+              - 'openmetadata-ui/src/main/resources/ui/sonar-project.properties'
+
+  ui-coverage-tests:
+    needs: check-changes
+    runs-on: ubuntu-latest
+    if: |
+      !github.event.pull_request.draft &&
+      (github.event_name == 'merge_group' || github.event_name == 'workflow_dispatch' || needs.check-changes.outputs.ui-changed == 'true') &&
+      (github.event_name != 'pull_request_target' || github.event.action != 'labeled' || github.event.label.name == 'safe to test')
+    steps:
+      - name: Wait for the labeler
+        uses: lewagon/wait-on-check-action@v1.7.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          check-name: Team Label
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          wait-interval: 90
+
+      - name: Verify PR labels
+        uses: jesusvasquez333/verify-pr-label-action@v1.4.0
+        if: ${{ github.event_name == 'pull_request_target' }}
+        with:
+          github-token: "${{ secrets.GITHUB_TOKEN }}"
+          valid-labels: "safe to test"
+          pull-request-number: "${{ github.event.pull_request.number }}"
+          disable-reviews: true # To not auto approve changes
+
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'merge_group' && github.sha || github.event.pull_request.head.sha }}
+          # Disabling shallow clone is recommended for improving relevancy of reporting.
+          # Use partial clone (blob:none) to avoid downloading every blob in history —
+          # commits/trees still available for Sonar blame; blobs fetched lazily as needed.
+          fetch-depth: 0
+          filter: blob:none
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: "openmetadata-ui/src/main/resources/ui/.nvmrc"
+
+      - name: Get yarn cache directory path
+        id: yarn-cache-dir-path
+        run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
+
+      - name: Caching NPM dependencies
+        uses: actions/cache@v4
+        id: yarn-cache
+        with:
+          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
+          key: ${{ runner.os }}-node-yarn-${{ hashFiles('**/yarn.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-node-yarn-
+
+      - name: Get npm cache directory
+        id: npm-cache-dir
+        shell: bash
+        run: echo "dir=$(npm config get cache)" >> ${GITHUB_OUTPUT}
+
+      - name: Caching NPM dependencies
+        uses: actions/cache@v4
+        id: npm-cache
+        with:
+          path: ${{ steps.npm-cache-dir.outputs.dir }}
+          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-node-
+
+      - name: Install Antlr4 CLI
+        run: |
+          sudo make install_antlr_cli
+      - name: Install Yarn Packages
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        run: yarn install
+      - name: Run Coverage
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        run: yarn test:cov-summary
+        id: yarn_coverage
+      - name: Jest coverage comment
+        uses: MishaKav/jest-coverage-comment@v1.0.22
+        with:
+          coverage-summary-path: ${{env.UI_COVERAGE_DIRECTORY}}/coverage-summary.json
+          title: Jest test Coverage
+          summary-title: UI tests summary
+          badge-title: Coverage
+      - name: yarn add sonarqube-scanner
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        run: npm install -g sonarqube-scanner
+        id: npm_install_sonar_scanner
+      - name: SonarCloud Scan On PR
+        if: github.event_name == 'pull_request_target' && steps.npm_install_sonar_scanner.outcome == 'success'
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        run: |
+          sonar-scanner -Dsonar.host.url=${SONARCLOUD_URL} \
+          -Dproject.settings=sonar-project.properties \
+          -Dsonar.pullrequest.key=${{ github.event.pull_request.number }} \
+          -Dsonar.pullrequest.branch=$PR_HEAD_REF \
+          -Dsonar.pullrequest.base=main \
+          -Dsonar.pullrequest.github.repository=OpenMetadata \
+          -Dsonar.scm.revision=${{ github.event.pull_request.head.sha }} \
+          -Dsonar.pullrequest.provider=github \
+          -Dsonar.scm.disabled=true
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SONAR_TOKEN: ${{ secrets.UI_SONAR_TOKEN }}
+          SONARCLOUD_URL: https://sonarcloud.io
+          PR_HEAD_REF: ${{ github.event.pull_request.head.ref }}
+      - name: SonarCloud Scan
+        if: github.event_name == 'push' && steps.npm_install_sonar_scanner.outcome == 'success'
+        working-directory: ${{ env.UI_WORKING_DIRECTORY }}
+        run: |
+          sonar-scanner -Dsonar.host.url=${SONARCLOUD_URL} \
+          -Dproject.settings=sonar-project.properties
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          SONAR_TOKEN: ${{ secrets.UI_SONAR_TOKEN }}
+          SONARCLOUD_URL: https://sonarcloud.io
+
+  # This job is the one to set as the required status check in branch protection.
+  # It always runs and passes when ui-coverage-tests succeeded OR was skipped
+  # (i.e. no relevant UI paths changed), so PRs without UI changes are never blocked.
+  ui-coverage:
+    if: always()
+    needs: [ui-coverage-tests]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check coverage result
+        run: |
+          result="${{ needs.ui-coverage-tests.result }}"
+          if [[ "$result" == "success" || "$result" == "skipped" ]]; then
+            echo "Coverage check passed or was not required for this PR"
+          else
+            echo "Coverage check failed with status: $result"
+            exit 1
+          fi

.gitignore

@@ -0,0 +1,211 @@
+#  Copyright 2021 Collate
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#  http://www.apache.org/licenses/LICENSE-2.0
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# Created by .ignore support plugin (hsz.mobi)
+# Maven
+.venv
+.venv-devcontainer
+__pycache__
+target/
+pom.xml.tag
+pom.xml.releaseBackup
+pom.xml.versionsBackup
+pom.xml.next
+release.properties
+dependency-reduced-pom.xml
+buildNumber.properties
+.mvn/timing.properties
+.claude/*
+.claude
+.maestro
+catalog-services/catalog-services.iml
+
+# local docker volume
+docker/development/docker-volume
+docker-volume
+docker/docker-compose-quickstart/docker-volume
+docker/docker-compose-quickstart/docker-volumes
+# Java template
+*.class
+venv
+env
+.java-version
+# logs
+logs
+*.log
+*.egg-info
+.eggs
+*.db
+# Eclipse
+.settings/
+.project
+.classpath
+
+# Intellij
+*.iml
+*.ipr
+*.iws
+.idea/
+# Package Files
+*.jar
+*.war
+*.ear
+
+# mac dir files
+.DS_Store
+
+# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
+hs_err_pid*
+node_modules
+build
+dist
+
+# Registry Kerberos SASL secrets directory
+
+# ts build info and report
+openmetadata-ui/src/main/resources/ui/webpack
+openmetadata-ui/src/main/resources/ui/tsconfig.tsbuildinfo
+openmetadata-ui/src/main/resources/ui/playwright/tsconfig.tsbuildinfo
+
+#tests
+.coverage*
+!ingestion/.coveragerc
+/ingestion/coverage.xml
+/ingestion/ci-coverage.xml
+/ingestion/junit/*
+/ingestion/tests/e2e/artifacts/*
+openmetadata-ui/src/main/resources/ui/src/test/unit/coverage
+openmetadata-ui/src/main/resources/ui/test-report.xml
+
+# Playwright artifacts
+openmetadata-ui/src/main/resources/ui/playwright/output/
+openmetadata-ui/src/main/resources/ui/playwright/e2e/.cache/
+openmetadata-ui/src/main/resources/ui/.env
+openmetadata-ui/src/main/resources/ui/playwright/.auth
+openmetadata-ui/src/main/resources/ui/blob-report
+openmetadata-ui/src/main/resources/ui/test-results/
+openmetadata-ui/src/main/resources/ui/flakiness-report/
+
+#UI - Dereferenced Schemas
+openmetadata-ui/src/main/resources/ui/src/jsons/*
+
+# UI - Contracts Downloads
+openmetadata-ui/src/main/resources/ui/downloads/*
+
+#vscode
+*/.vscode/*
+.vscode/*
+
+# Python generated sources
+ingestion-core/src/metadata/generated/**
+ingestion/src/metadata/generated/**
+ingestion/requirements.txt
+ingestion/.python-version
+ingestion/venv2/**
+.python-version
+ingestion/tests/load/summaries/*.csv
+
+# MLFlow
+mlruns/
+/ingestion/tests/integration/source/mlflow/tests/db/
+
+# Antlr
+openmetadata-ui/src/main/resources/ui/src/generated/antlr/
+.antlr
+
+# SQLAlchemy tests
+file:cachedb
+
+# Snyk report
+security-report
+.dccache
+scan-requirements.txt
+
+# CLI e2e tests
+ingestion/tests/cli_e2e/**/*test.yaml
+
+# Tests
+**/metastore_db/
+
+# Docker volumes for local development
+docker-volumes/
+**/docker-volumes/
+docker/docker-compose-quickstart/docker-volumes/
+docker/development/docker-volumes/
+fuseki-volume/
+search-volume/
+
+# Cursor rules
+.cursorrules
+.cursor/
+
+# Nox
+ingestion/.nox/
+
+# Environment variables
+.env
+.env.local
+.env.*.local
+
+# Temporary files
+*.tmp
+*.temp 
+
+# BMAD Method
+.bmad-core/
+_bmad/
+
+# Claude Flow generated files
+.claude/*
+.mcp.json
+claude-flow.config.json
+.swarm/
+.hive-mind/
+memory/claude-flow-data.json
+memory/sessions/*
+!memory/sessions/README.md
+memory/agents/*
+!memory/agents/README.md
+coordination/memory_bank/*
+coordination/subtasks/*
+coordination/orchestration/*
+*.db
+*.db-journal
+*.db-wal
+*.sqlite
+*.sqlite-journal
+*.sqlite-wal
+claude-flow
+claude-flow.bat
+claude-flow.ps1
+hive-mind-prompt-*.txt
+.claude-flow/
+/memory/
+.claude/agents
+.claude/hooks/
+ingestion/.claude/agents
+.claustre_session_id
+
+# AI scaffold working documents — stay local, never committed
+**/CONNECTOR_CONTEXT.md
+
+# Connector audit working files — per-session, never committed
+.claude/audit-results/
+.claude/connector-audit.json
+.claude/scheduled_tasks.lock
+.claude/plans/
+
+# Serena MCP language-server cache — local tooling, not committed
+.serena/
+
+test-results/
+
+docs/superpowers/*

.pre-commit-config.yaml

@@ -0,0 +1,42 @@
+default_language_version:
+  python: python3
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v5.0.0
+    hooks:
+      - id: check-json
+        # TODO: investigate and fix or remove the excluded files. The first
+        # three carry real JSON issues (duplicate keys, malformed/empty
+        # content) that pre-commit-hooks v2.3.0 didn't catch; v5.0.0 does.
+        # The last is an intentionally malformed test fixture.
+        exclude: |
+          (?x)^(
+            .*vscode.*|
+            openmetadata-spec/src/main/resources/rdf/contexts/dataAsset\.jsonld|
+            ingestion/examples/sample_data/pipelines/tasks\.json|
+            openmetadata-service/src/main/resources/dataInsights/opensearch/indexSettingsTemplate\.json|
+            openmetadata-ui/src/main/resources/ui/playwright/test-data/odcs-examples/invalid-malformed\.json
+          )$
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.15.12
+    hooks:
+      - id: ruff-check
+        files: ^(ingestion|openmetadata-airflow-apis)/
+        args: ["--fix", "--config", "ingestion/pyproject.toml"]
+      - id: ruff-format
+        files: ^(ingestion|openmetadata-airflow-apis)/
+        args: ["--config", "ingestion/pyproject.toml"]
+  - repo: https://github.com/pre-commit/mirrors-prettier
+    rev: v2.5.1
+    hooks:
+      - id: prettier
+        files: ^openmetadata-service/src/main/resources/json/schema/
+  - repo: local
+    hooks:
+      - id: google-style-java
+        name: Google Java Code Style for Java
+        description: Formats code in Google's Java codestyle with 120 line length.
+        entry: scripts/format-code.sh
+        language: script
+        files: \.java$
+        require_serial: true

.snyk

@@ -0,0 +1,48 @@
+# Snyk policy file for the OpenMetadata repo root.
+#
+# This file is read by Snyk scans that run FROM REPO ROOT. Currently
+# that means the snyk-server-report target (see Makefile):
+#   snyk test --all-projects ...        (Snyk Open Source / SCA)
+#   snyk code test --all-projects ...   (Snyk Code / SAST)
+# and the snyk-ui-report target which scans the UI yarn.lock.
+#
+# Paths in `exclude.global` below are relative to THIS file's directory
+# (the repo root). So an ingestion path looks like `ingestion/foo/**`.
+#
+# IMPORTANT: there is a SECOND policy file at `ingestion/.snyk`. It is
+# read by the snyk-ingestion-report target, which runs
+# `cd ingestion; snyk code test ...`. Snyk Code only reads a .snyk
+# file from its own CWD, with no `--policy-path` and no `--exclude`
+# flag available on the Code subcommand. So a pattern added here that
+# lives under `ingestion/` will NOT be applied by that scan unless it
+# is also added (without the `ingestion/` prefix) to `ingestion/.snyk`.
+# Keep the two files in sync for any path under `ingestion/`.
+#
+# The same dual-file rule would apply to `openmetadata-airflow-apis/`
+# if that scan ever needs path exclusions. The dedicated airflow scan
+# also runs from a subdirectory and would need its own .snyk there.
+#
+# Snyk docs on .snyk policy lookup and Snyk Code excludes:
+#   https://docs.snyk.io/manage-risk/policies/the-.snyk-file
+#   https://docs.snyk.io/developer-tools/snyk-cli/scan-and-maintain-projects-using-the-cli/snyk-cli-for-snyk-code/exclude-directories-and-files-from-snyk-code-cli-tests
+
+version: v1.25.0
+ignore: {}
+patch: {}
+exclude:
+  global:
+    # Ingestion (Python). When you add or change anything in this block,
+    # MIRROR THE CHANGE IN `ingestion/.snyk` with the same pattern minus
+    # the `ingestion/` prefix. Both files must list the exclusion for
+    # both Snyk Code scans to honour it.
+    - ingestion/examples/**
+    - ingestion/tests/**
+    - ingestion/src/_openmetadata_testutils/**
+    - ingestion/src/metadata/sdk/examples/**
+
+    # UI (TypeScript) test mocks.
+    - openmetadata-ui/src/main/resources/ui/src/pages/service/mocks/**
+    - openmetadata-ui/src/main/resources/ui/src/components/common/TestConnection/TestConnection.mock.ts
+
+    # Server (Java) test fixtures.
+    - openmetadata-service/src/test/**

AGENTS.md

@@ -0,0 +1,255 @@
+# AGENTS.md
+
+This file provides guidance to Codex (Codex.ai/code) when working with code in this repository.
+
+## About OpenMetadata
+
+OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance. This is a multi-module project with Java backend services, React frontend, Python ingestion framework, and comprehensive Docker infrastructure.
+
+## Architecture Overview
+
+- **Backend**: Java 21 + Dropwizard REST API framework, multi-module Maven project
+- **Frontend**: React + TypeScript + Ant Design, built with Webpack and Yarn
+- **Ingestion**: Python 3.10-3.12 with Pydantic 2.x, 75+ data source connectors
+- **Database**: MySQL (default) or PostgreSQL with Flyway migrations
+- **Search**: Elasticsearch 7.17+ or OpenSearch 2.6+ for metadata discovery
+- **Infrastructure**: Apache Airflow for workflow orchestration
+
+## Essential Development Commands
+
+### Prerequisites and Setup
+```bash
+make prerequisites              # Check system requirements
+make install_dev_env           # Install all development dependencies
+make yarn_install_cache        # Install UI dependencies
+```
+
+### Frontend Development
+```bash
+cd openmetadata-ui/src/main/resources/ui
+yarn start                     # Start development server on localhost:3000
+yarn test                      # Run Jest unit tests
+yarn test path/to/test.spec.ts # Run a specific test file
+yarn test:watch               # Run tests in watch mode
+yarn playwright:run            # Run E2E tests
+yarn lint                      # ESLint check
+yarn lint:fix                  # ESLint with auto-fix
+yarn build                     # Production build
+```
+
+### Backend Development
+```bash
+mvn clean package -DskipTests  # Build without tests
+mvn clean package -DonlyBackend -pl !openmetadata-ui  # Backend only
+mvn test                       # Run unit tests
+mvn verify                     # Run integration tests
+mvn spotless:apply             # Format Java code
+```
+
+### Python Ingestion Development
+```bash
+cd ingestion
+make install_dev_env           # Install in development mode
+make generate                  # Generate Pydantic models from JSON schemas
+make unit_ingestion_dev_env    # Run unit tests
+make py_format                 # Apply ruff lint-fix + format
+make py_format_check           # Verify lint + format (matches CI; catches non-auto-fixable issues)
+make static-checks             # Run type checking with basedpyright
+```
+
+### Full Local Environment
+```bash
+./docker/run_local_docker.sh -m ui -d mysql        # Complete local setup with UI
+./docker/run_local_docker.sh -m no-ui -d postgresql # Backend only with PostgreSQL
+./docker/run_local_docker.sh -s true               # Skip Maven build step
+```
+
+### Testing
+```bash
+make run_e2e_tests             # Full E2E test suite
+make unit_ingestion            # Python unit tests with coverage
+yarn test:coverage             # Frontend test coverage
+```
+
+## Code Generation and Schemas
+
+OpenMetadata uses a schema-first approach with JSON Schema definitions driving code generation:
+
+```bash
+make generate                  # Generate all models from schemas
+make py_antlr                  # Generate Python ANTLR parsers
+make js_antlr                  # Generate JavaScript ANTLR parsers
+yarn parse-schema              # Parse JSON schemas for frontend (connection and ingestion schemas)
+```
+
+### Schema Architecture
+- **Source schemas** in `openmetadata-spec/` define the canonical data models
+- **Connection schemas** are pre-processed at build time via `parseSchemas.js` to resolve all `$ref` references
+- **Application schemas** in `openmetadata-ui/.../ApplicationSchemas/` are resolved at runtime using `schemaResolver.ts`
+- JSON schemas with `$ref` references to external files require resolution before use in forms
+
+## Key Directories
+
+- `openmetadata-service/` - Core Java backend services and REST APIs
+- `openmetadata-ui/src/main/resources/ui/` - React frontend application
+- `ingestion/` - Python ingestion framework with connectors
+- `openmetadata-spec/` - JSON Schema specifications for all entities
+- `bootstrap/sql/` - Database schema migrations and sample data
+- `conf/` - Configuration files for different environments
+- `docker/` - Docker configurations for local and production deployment
+
+## Development Workflow
+
+1. **Schema Changes**: Modify JSON schemas in `openmetadata-spec/`, then run `mvn clean install` on openmetadata-spec to update models
+2. **Backend**: Develop in Java using Dropwizard patterns, test with `mvn test`, format with `mvn spotless:apply`
+3. **Frontend**: Use React/TypeScript with Ant Design components, test with Jest/Playwright
+4. **Ingestion**: Python connectors follow plugin pattern, use `make install_dev_env` for development
+5. **Full Testing**: Use `make run_e2e_tests` before major changes
+
+## Frontend Architecture Patterns
+
+### React Component Patterns
+- **File Naming**: Components use `ComponentName.component.tsx`, interfaces use `ComponentName.interface.ts`
+- **State Management**: Use `useState` with proper typing, avoid `any`
+- **Side Effects**: Use `useEffect` with proper dependency arrays
+- **Performance**: Use `useCallback` for event handlers, `useMemo` for expensive computations
+- **Custom Hooks**: Prefix with `use`, place in `src/hooks/`, return typed objects
+- **Internationalization**: Use `useTranslation` hook from react-i18next, access with `t('key')`
+- **Component Structure**: Functional components only, no class components
+- **Props**: Define interfaces for all component props, place in `.interface.ts` files
+- **Loading States**: Use object state for multiple loading states: `useState<Record<string, boolean>>({})`
+- **Error Handling**: Use `showErrorToast` and `showSuccessToast` utilities from ToastUtils
+- **Navigation**: Use `useNavigate` from react-router-dom, not direct history manipulation
+- **Data Fetching**: Async functions with try-catch blocks, update loading states appropriately
+
+### State Management
+- Use Zustand stores for global state (e.g., `useLimitStore`, `useWelcomeStore`)
+- Keep component state local when possible with `useState`
+- Use context providers for feature-specific shared state (e.g., `ApplicationsProvider`)
+
+### Styling
+
+- **MUI Migration**: The project is gradually migrating from Ant Design to Material-UI (MUI) v7.3.1
+- **Preferred Approach**: Use MUI components v7.3.1 and styles wherever possible for new features
+- **Theme and Styles**: MUI theme data and styles are defined in `openmetadata-ui-core-components`
+- **Colors and Design Tokens**: Always reference theme colors and design tokens from the MUI theme, not hardcoded values
+- **Legacy Components**: Ant Design components remain in existing code but should be replaced with MUI equivalents when refactoring
+- Do not add unnecessary spacing between logs and code.
+- In Java, avoid wildcards imports (e.g., use `import java.util.List;` instead of `import java.util.*;`)
+- Custom styles in `.less` files with component-specific naming (legacy pattern)
+- Follow BEM naming convention for custom CSS classes
+- Use CSS modules where appropriate
+
+### UI considerations
+
+- Do not use string literals at any place. You should use useTranslation hook and use it like const {t} = useTranslation(). And for example if you want to have "Run" as string, you should be using { t('label.run') }, this label is defined in locales.
+
+
+### Application Configuration
+- Applications use `ApplicationsClassBase` for schema loading and configuration
+- Dynamic imports handle application-specific schemas and assets
+- Form schemas use React JSON Schema Form (RJSF) with custom UI widgets
+
+### Service Utilities
+- Each service type has dedicated utility files (e.g., `DatabaseServiceUtils.tsx`)
+- Connection schemas are imported statically and pre-resolved
+- Service configurations use switch statements to map types to schemas
+
+### Type Safety
+- All API responses have generated TypeScript interfaces in `generated/`
+- Custom types extend base interfaces when needed
+- Avoid type assertions unless absolutely necessary
+- Use discriminated unions for action types and state variants
+
+## Database and Migrations
+
+- Flyway handles schema migrations in `bootstrap/sql/migrations/`
+- Use Docker containers for local database setup
+- Default MySQL, PostgreSQL supported as alternative
+- Sample data loaded automatically in development environment
+
+## Security and Authentication
+
+- JWT-based authentication with OAuth2/SAML support
+- Role-based access control defined in Java entities
+- Security configurations in `conf/openmetadata.yaml`
+- Never commit secrets - use environment variables or secure vaults
+
+## Code Generation Standards
+
+### Comments Policy
+- **Do NOT add unnecessary comments** - write self-documenting code
+- **NEVER add single-line comments that describe what the code obviously does**
+- Only include comments for:
+    - Complex business logic that isn't obvious
+    - Non-obvious algorithms or workarounds
+    - Public API JavaDoc documentation
+    - TODO/FIXME with ticket references
+- Bad examples (NEVER do this):
+    - `// Create user` before `createUser()`
+    - `// Get client` before `SdkClients.adminClient()`
+    - `// Verify domain is set` before `assertNotNull(entity.getDomain())`
+    - `// User names are lowercased` when the code `toLowerCase()` makes it obvious
+- If the code needs a comment to be understood, refactor the code to be clearer instead
+
+### Java Code Requirements
+- **Always run `mvn spotless:apply`** before finishing any task that touched
+  `.java` files. CI runs `mvn spotless:check` and will fail the PR otherwise
+  (bot's exact phrasing: "Please run `mvn spotless:apply` in the root of your
+  repository and commit the changes to this PR"). Scope with `-pl <module>`
+  for speed if only one module changed. A reusable procedure is written up at
+  `.agents/skills/java-checkstyle/SKILL.md`.
+- Use clear, descriptive variable and method names instead of comments
+- Follow existing project patterns and conventions
+- Generate production-ready code, not tutorial code
+- Create integration tests in openmetadata-integration-tests
+- Do not use Fully Qualified Names in the code such as org.openmetadata.schema.type.Status instead import the class name
+- Do not import wild-card packages instead import exactly required packages
+
+### TypeScript/Frontend Code Requirements
+- **Always run the UI checkstyle sequence** before finishing any task that
+  touched `.ts`/`.tsx`/`.js`/`.jsx`/`.json` under
+  `openmetadata-ui/src/main/resources/ui/src/`, `.../playwright/`, or
+  `openmetadata-ui-core-components/src/main/resources/ui/src/`. CI's
+  `UI Checkstyle / lint-src|lint-playwright|lint-core-components` jobs fail
+  the PR otherwise. Order matters: `organize-imports-cli` → `eslint --fix` →
+  `prettier --write`. A reusable procedure lives at
+  `.agents/skills/ui-checkstyle/SKILL.md`.
+- **NEVER use `any` type** in TypeScript code - always use proper types
+- Use `unknown` when the type is truly unknown and add type guards
+- Import types from existing type definitions (e.g., `RJSFSchema` from `@rjsf/utils`)
+- Follow ESLint rules strictly - the project enforces no-console, proper formatting
+- Add `// eslint-disable-next-line` comments only when absolutely necessary
+- **Import Organization** (in order):
+  1. External libraries (React, Ant Design, etc.)
+  2. Internal absolute imports from `generated/`, `constants/`, `hooks/`, etc.
+  3. Relative imports for utilities and components
+  4. Asset imports (SVGs, styles)
+  5. Type imports grouped separately when needed
+
+### Python Code Requirements
+- **Use pytest, not unittest** - write tests using pytest style with plain `assert` statements
+- Use pytest fixtures for test setup instead of `setUp`/`tearDown` methods
+- Use `unittest.mock` for mocking (MagicMock, patch) - this is compatible with pytest
+- Test classes should not inherit from `TestCase` - use plain classes prefixed with `Test`
+- Use `assert x == y` instead of `self.assertEqual(x, y)`
+- Use `assert x is None` instead of `self.assertIsNone(x)`
+- Use `assert "text" in string` instead of `self.assertIn("text", string)`
+
+### Python Ingestion Connector Guidelines
+- **Keep connector-specific logic in connector-specific files**, not in generic/shared files like `builders.py`
+- Example: Redshift IAM auth should be in `ingestion/src/metadata/ingestion/source/database/redshift/connection.py`, not in `ingestion/src/metadata/ingestion/connections/builders.py`
+- This keeps the codebase modular and prevents generic utilities from becoming cluttered with connector-specific edge cases
+
+### Testing Philosophy
+- **Test real behavior, not mock wiring** - if a test requires mocking 3+ classes just to verify a method call, it's testing the wrong thing
+- **Prefer integration tests** over heavily-mocked unit tests. This project has full integration test infrastructure (OpenMetadataApplicationTest, Docker containers, real OpenSearch). Use it.
+- **Mocks are for boundaries, not internals** - mock external services (HTTP clients, third-party APIs), not your own classes. If you're mocking static methods left and right to test internal plumbing, write an integration test instead.
+- **A test that mocks everything proves nothing** - it only verifies that your mocks are wired correctly, not that the system works
+- **Ask "what breaks if this test passes but the code is wrong?"** - if the answer is "nothing, because everything real is mocked out", delete the test and write a better one
+- **Test the outcome, not the implementation** - assert on observable results (API responses, database state, stats values) rather than verifying internal method calls with `verify()`
+
+### Response Format
+- Provide clean code blocks without unnecessary explanations
+- Assume readers are experienced developers
+- Focus on functionality over education

APPLICATION.md

@@ -0,0 +1,48 @@
+How to Add a New Application
+
+## Create App Config
+
+in `openmetadata-spec/src/main/resources/json/schema/entity/applications/configuration` define the json schema representing your application configuration.
+- external: these are applications that will run externaly (e.g. Airflow). If your application has its execution logic defined in Python, this is where the code logic should live
+- internal: these will application will be executed through the Qwartz scheduler form the application itself. The code logic for this application will be written in Java.
+
+you can follow this naming convention <yourAppName>AppConfig.json
+
+
+## Define default app config
+
+in `openmetadata-service/src/main/resources/json/data/app` define the default setting of your application. Create a new json following this naming convention `<youAppName>Application.json`
+
+## Define your app in the Application Marketplace
+
+In `openmetadata-service/src/main/resources/json/data/appMarketPlaceDefinition`, define the elements of your application as they will appear in the app marketplace.
+
+Note that you can specify a `"sourcePythonClass": “path.to.python.class”` if your application code logic is defined in Python
+
+## Add the config to the UI
+
+In `openmetadata-ui/src/main/resources/ui/src/utils/ApplicationSchemas` add the config for your application so it can be rendered from the UI. Follow this naming convention `<yourAppName>Application.json`
+
+## Implement Java class
+In `openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles` implement the java class corresponding to your application.
+
+Not that the value of the `className` from step 3 should match the class name / path defined at this stage
+
+## [Optional] Implement Python logic
+If you have defined an external application, you need to create the relevant Python class that implements the logic. The path of the class needs to match the value of `sourcePythonClass`. You will need to inherit from the `AppRunner` class and at minimum implement the `def run` method.
+
+e.g. `ingestion/src/metadata/applications/observability_data_exporter/app.py`
+
+## [Optional] Disabling the application
+In `openmetadata-service/src/main/resources/applications` you can add private configuration to your application. You also have the possibility to disable the application by setting `enabled: false` so that users cannot install it. Below is an example of the configuration.
+
+```
+enabled: false
+parameters:
+  instance: <instance>
+  token: <token?
+  omURL: <omURL>
+```
+
+Setting `enabled: false` will prevent users from being able to install the application. The private configuration parameters can be set at `openmetadata-spec/src/main/resources/json/schema/entity/applications/configuration/private`
+

CLAUDE.md

@@ -0,0 +1,558 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## About OpenMetadata
+
+OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance. This is a multi-module project with Java backend services, React frontend, Python ingestion framework, and comprehensive Docker infrastructure.
+
+For architecture deep dives, entity/repository/resource patterns, and end-to-end checklists for adding new entities or connectors, see [DEVELOPER.md](DEVELOPER.md).
+
+## Architecture Overview
+
+- **Backend**: Java 21 + Dropwizard REST API framework, multi-module Maven project
+- **Frontend**: React + TypeScript, built with Webpack and Yarn; component library via `openmetadata-ui-core-components` (Tailwind CSS v4 with `tw:` prefix, react-aria-components foundation)
+- **Ingestion**: Python 3.10-3.11 with Pydantic 2.x, 75+ data source connectors
+- **Database**: MySQL (default) or PostgreSQL with Flyway migrations
+- **Search**: Elasticsearch 7.17+ or OpenSearch 2.6+ for metadata discovery
+- **Infrastructure**: Apache Airflow for workflow orchestration
+
+## Environment Setup
+
+### Python Virtual Environment (REQUIRED)
+
+**You MUST activate the Python venv before any Python work.** OpenMetadata supports Python 3.10-3.11; 3.11 is recommended.
+
+```bash
+# First-time setup (creates venv at repo root):
+# python3.11 -m venv env
+
+# ALWAYS activate before running Python, make generate, make install_dev, etc:
+source env/bin/activate
+
+# Verify:
+python --version   # Should show Python 3.10.x or 3.11.x
+```
+
+**In worktrees**: When Claude Code creates a Git worktree, the venv from the main repo is NOT copied. You need to either:
+- Create a new venv in the worktree: `python3.11 -m venv env && source env/bin/activate && cd ingestion && make install_dev`
+- Or symlink the main repo's venv: `ln -s /path/to/main-repo/env env`
+
+### Initial Dev Environment Setup
+
+After activating the venv, install all dependencies:
+
+```bash
+source env/bin/activate
+
+# Install ingestion module with all dev dependencies (required before make generate)
+cd ingestion
+make install_dev_env            # Full dev environment (edit mode + all extras)
+# OR for lighter install:
+make install_dev                # Just dev dependencies
+cd ..
+
+# Generate Pydantic models from JSON schemas (required after schema changes)
+make generate
+
+# Install UI dependencies
+make yarn_install_cache
+```
+
+### Other Environment Notes
+
+- **Java**: Java 21 required. Use `mvn` (Maven) for backend builds.
+- **Node/Yarn**: Use `yarn` (not `npm`) for frontend. Frontend root is `openmetadata-ui/src/main/resources/ui/`.
+- **Docker services**: Development services (MySQL, Elasticsearch, etc.) run via `docker/development/docker-compose.yml`:
+  ```bash
+  docker compose -f docker/development/docker-compose.yml up -d
+  ```
+
+## Essential Development Commands
+
+### Prerequisites and Setup
+```bash
+make prerequisites              # Check system requirements
+source env/bin/activate         # ALWAYS activate venv first
+cd ingestion && make install_dev_env  # Install Python dev dependencies
+make generate                  # Generate Pydantic models from JSON schemas
+make yarn_install_cache        # Install UI dependencies
+```
+
+### Frontend Development
+```bash
+cd openmetadata-ui/src/main/resources/ui
+yarn start                     # Start development server on localhost:3000
+yarn test                      # Run Jest unit tests
+yarn test path/to/test.spec.ts # Run a specific test file
+yarn test:watch               # Run tests in watch mode
+yarn playwright:run            # Run E2E tests
+yarn lint                      # ESLint check
+yarn lint:fix                  # ESLint with auto-fix
+yarn build                     # Production build
+```
+
+### Frontend CI Checkstyle (run before PR to match CI)
+```bash
+cd openmetadata-ui/src/main/resources/ui
+yarn ui-checkstyle:changed         # One-shot checkstyle for changed files (excludes tsc)
+yarn organize-imports:cli <files>  # Sort and organize imports
+yarn lint:fix                      # ESLint auto-fix
+yarn pretty:base --write <files>   # Prettier formatting
+yarn license-header-fix <files>    # Add Apache 2.0 license headers
+yarn i18n                          # Sync all 17 locale files with en-us.json
+yarn generate:app-docs             # Regenerate application documentation
+npx tsc --noEmit                   # TypeScript type check (catches errors early)
+```
+
+### Backend Development
+```bash
+mvn clean package -DskipTests  # Build without tests
+mvn clean package -DonlyBackend -pl !openmetadata-ui  # Backend only
+mvn test                       # Run unit tests
+mvn verify                     # Run integration tests
+mvn spotless:apply             # Format Java code
+```
+
+### Python Ingestion Development
+```bash
+cd ingestion
+make install_dev_env           # Install in development mode
+make generate                  # Generate Pydantic models from JSON schemas
+make unit_ingestion_dev_env    # Run unit tests
+make py_format                 # Apply ruff lint-fix + format
+make py_format_check           # Verify lint + format (matches CI; catches non-auto-fixable issues)
+make static-checks             # Run type checking with basedpyright
+```
+
+### Full Local Environment
+```bash
+./docker/run_local_docker.sh -m ui -d mysql        # Complete local setup with UI
+./docker/run_local_docker.sh -m no-ui -d postgresql # Backend only with PostgreSQL
+./docker/run_local_docker.sh -s true               # Skip Maven build step
+```
+
+### Testing
+```bash
+make run_e2e_tests             # Full E2E test suite
+make unit_ingestion            # Python unit tests with coverage
+yarn test:coverage             # Frontend test coverage
+```
+
+### Backend Integration Tests
+All backend API integration tests MUST be placed in `openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/` directory. Tests should:
+- Use naming convention `*IT.java` (Integration Test)
+- Extend `BaseEntityIT<T, K>` for entity CRUD tests
+- Be designed to run concurrently (use `@Execution(ExecutionMode.CONCURRENT)`)
+- Use `TestNamespace` for test isolation
+- Use `SdkClients` for API calls (e.g., `SdkClients.adminClient().tables().create(...)`)
+
+```bash
+# Run a specific integration test
+mvn test -pl openmetadata-integration-tests -Dtest=TaskResourceIT
+
+# Run all integration tests
+mvn test -pl openmetadata-integration-tests
+```
+
+## Code Generation and Schemas
+
+OpenMetadata uses a schema-first approach with JSON Schema definitions driving code generation:
+
+```bash
+make generate                  # Generate all models from schemas
+make py_antlr                  # Generate Python ANTLR parsers
+make js_antlr                  # Generate JavaScript ANTLR parsers
+yarn parse-schema              # Parse JSON schemas for frontend (connection and ingestion schemas)
+```
+
+### Schema Architecture
+- **Source schemas** in `openmetadata-spec/` define the canonical data models
+- **Connection schemas** are pre-processed at build time via `parseSchemas.js` to resolve all `$ref` references
+- **Application schemas** in `openmetadata-ui/.../ApplicationSchemas/` are resolved at runtime using `schemaResolver.ts`
+- JSON schemas with `$ref` references to external files require resolution before use in forms
+
+## Key Directories
+
+- `openmetadata-service/` - Core Java backend services and REST APIs
+- `openmetadata-ui/src/main/resources/ui/` - React frontend application
+- `ingestion/` - Python ingestion framework with connectors
+- `openmetadata-spec/` - JSON Schema specifications for all entities
+- `bootstrap/sql/` - Database schema migrations and sample data
+- `conf/` - Configuration files for different environments
+- `docker/` - Docker configurations for local and production deployment
+
+## Development Workflow
+
+1. **Schema Changes**: Modify JSON schemas in `openmetadata-spec/`, then run `mvn clean install` on openmetadata-spec to update models
+2. **Backend**: Develop in Java using Dropwizard patterns, test with `mvn test`, format with `mvn spotless:apply`
+3. **Frontend**: Use React/TypeScript with components from `openmetadata-ui-core-components`, test with Jest/Playwright
+4. **Ingestion**: Python connectors follow plugin pattern, use `make install_dev_env` for development
+5. **Full Testing**: Use `make run_e2e_tests` before major changes
+
+## Frontend Architecture Patterns
+
+### React Component Patterns
+- **File Naming**: Components use `ComponentName.component.tsx`, interfaces use `ComponentName.interface.ts`
+- **State Management**: Use `useState` with proper typing, avoid `any`
+- **Side Effects**: Use `useEffect` with proper dependency arrays
+- **Performance**: Use `useCallback` for event handlers, `useMemo` for expensive computations
+- **Custom Hooks**: Prefix with `use`, place in `src/hooks/`, return typed objects
+- **Internationalization**: Use `useTranslation` hook from react-i18next, access with `t('key')`
+- **Component Structure**: Functional components only, no class components
+- **Props**: Define interfaces for all component props, place in `.interface.ts` files
+- **Loading States**: Use object state for multiple loading states: `useState<Record<string, boolean>>({})`
+- **Error Handling**: Use `showErrorToast` and `showSuccessToast` utilities from ToastUtils
+- **Navigation**: Use `useNavigate` from react-router-dom, not direct history manipulation
+- **Data Fetching**: Async functions with try-catch blocks, update loading states appropriately
+
+### State Management
+- Use Zustand stores for global state (e.g., `useLimitStore`, `useWelcomeStore`)
+- Keep component state local when possible with `useState`
+- Use context providers for feature-specific shared state (e.g., `ApplicationsProvider`)
+
+### Forms
+
+- **Building forms**: New forms use the `react-hook-form` + `react-aria` stack from `@openmetadata/ui-core-components` (`getField`/`FieldProp`/`FieldTypes`/`HookForm`/`FormFields`). A form is `FieldProp[]` config objects + RHF state + a pure values→payload transform. The full reference is [`openmetadata-ui/src/main/resources/ui/docs/formutils.md`](openmetadata-ui/src/main/resources/ui/docs/formutils.md). Do not use the legacy Ant Design `getField`/`generateFormFields` from `@utils/formUtils` for new forms.
+
+### Styling
+
+- **Component Library**: Use components from `openmetadata-ui-core-components` for all new UI work. This is the canonical component library — do not use MUI or introduce new MUI dependencies.
+- **Available Components**: Button, Input, Select, Modal, Table, Tabs, Pagination, Badge, Avatar, Checkbox, Dropdown, Form, Card, Tooltip, Toggle, Slider, Textarea, Tags, and more — all in `openmetadata-ui-core-components/src/main/resources/ui/src/components/`
+- **Tailwind Classes**: All Tailwind utility classes must use the `tw:` prefix (e.g., `tw:flex`, `tw:text-sm`, `tw:bg-blue-500`) to avoid conflicts with existing Ant Design/Less styles
+- **Design Tokens**: Use CSS custom properties defined in `openmetadata-ui-core-components/src/main/resources/ui/src/styles/globals.css`. Never use hardcoded color or spacing values. Semantic tokens include:
+  - Text: `--color-text-primary`, `--color-text-secondary`, `--color-text-tertiary`, `--color-text-error-primary`, etc.
+  - Border: `--color-border-primary`, `--color-border-secondary`, `--color-border-error`, `--color-border-brand`, etc.
+  - Background: `--color-bg-primary`, `--color-bg-secondary`, `--color-bg-error-primary`, `--color-bg-brand-solid`, etc.
+  - Shadows: `--shadow-xs` through `--shadow-3xl`
+  - Border radius: `--radius-none` through `--radius-full`
+- **Color Usage**: Full token reference, dark mode guide, and anti-pattern cheat sheet: [`openmetadata-ui/src/main/resources/ui/docs/colors.md`](openmetadata-ui/src/main/resources/ui/docs/colors.md). Always consult this before choosing any color class.
+- **MUI**: Do not use MUI — we are actively removing MUI from the codebase. Do not import from `@mui/*` or `@emotion/*`
+- **Legacy**: Ant Design components remain in existing code but should be replaced with `openmetadata-ui-core-components` equivalents when refactoring
+- Do not add unnecessary spacing between logs and code.
+- In Java, avoid wildcards imports (e.g., use `import java.util.List;` instead of `import java.util.*;`)
+- Custom styles in `.less` files with component-specific naming (legacy pattern, avoid for new code)
+- Follow BEM naming convention for custom CSS classes when writing raw CSS
+
+### UI considerations
+
+- Do not use string literals at any place. You should use useTranslation hook and use it like const {t} = useTranslation(). And for example if you want to have "Run" as string, you should be using { t('label.run') }, this label is defined in locales.
+
+
+### Application Configuration
+- Applications use `ApplicationsClassBase` for schema loading and configuration
+- Dynamic imports handle application-specific schemas and assets
+- Form schemas use React JSON Schema Form (RJSF) with custom UI widgets
+
+### Service Utilities
+- Each service type has dedicated utility files (e.g., `DatabaseServiceUtils.tsx`)
+- Connection schemas are imported statically and pre-resolved
+- Service configurations use switch statements to map types to schemas
+
+### Type Safety
+- All API responses have generated TypeScript interfaces in `generated/`
+- Custom types extend base interfaces when needed
+- Avoid type assertions unless absolutely necessary
+- Use discriminated unions for action types and state variants
+
+## Database and Migrations
+
+- Flyway handles schema migrations in `bootstrap/sql/migrations/`
+- Use Docker containers for local database setup
+- Default MySQL, PostgreSQL supported as alternative
+- Sample data loaded automatically in development environment
+
+## Security and Authentication
+
+- JWT-based authentication with OAuth2/SAML support
+- Role-based access control defined in Java entities
+- Security configurations in `conf/openmetadata.yaml`
+- Never commit secrets - use environment variables or secure vaults
+
+## Code Generation Standards
+
+### Comments Policy
+- **Do NOT add unnecessary comments** - write self-documenting code
+- **NEVER add single-line comments that describe what the code obviously does**
+- Only include comments for:
+    - Complex business logic that isn't obvious
+    - Non-obvious algorithms or workarounds
+    - Public API JavaDoc documentation
+    - TODO/FIXME with ticket references
+- Bad examples (NEVER do this):
+    - `// Create user` before `createUser()`
+    - `// Get client` before `SdkClients.adminClient()`
+    - `// Verify domain is set` before `assertNotNull(entity.getDomain())`
+    - `// User names are lowercased` when the code `toLowerCase()` makes it obvious
+- If the code needs a comment to be understood, refactor the code to be clearer instead
+
+### Java Code Requirements
+
+**Always run `mvn spotless:apply` before you finish any task that touched
+`.java` files.** CI runs `mvn spotless:check` and will fail the PR otherwise —
+the bot's exact suggestion is "Please run `mvn spotless:apply` in the root of
+your repository and commit the changes to this PR." Scope the run with
+`-pl <module>` for speed if only one module changed. When asked to "fix
+checkstyle" / "fix Java formatting" / "apply spotless", invoke the
+`java-checkstyle` skill (see `.claude/skills/java-checkstyle/`) rather than
+hand-editing formatting.
+
+#### Method Size and Complexity (Kafka-Grade Standards)
+- **Methods must be small and focused — aim for 15 lines or fewer** (excluding blank lines and braces). A method longer than that is almost always hiding multiple responsibilities; break it into smaller methods with descriptive names. "Meaningful" means each method does one nameable thing — if you can't fit the body comfortably on a screen, it's too big.
+- **One return statement per method, placed at the end.** No early-return guard clauses, no scattered returns in the middle. Initialize a `result` variable, structure the work as `if/else`, or extract a helper — the control flow then stays linear and easy to reason about. (Returns inside `lambda` bodies, `switch` expressions, and anonymous classes are scoped to those constructs and don't count against the outer method.)
+  ```java
+  // BAD: four scattered early returns
+  Map<UUID, X> compute(List<EntityInterface> entities) {
+    if (entities == null) return Collections.emptyMap();
+    if (entities.isEmpty()) return Collections.emptyMap();
+    if (!supportsX(entities.get(0))) return null;
+    Map<UUID, X> prefetched = doWork(entities);
+    if (prefetched.isEmpty()) return null;
+    return prefetched;
+  }
+
+  // GOOD: single trailing return; guards become extracted helpers + a result variable
+  Map<UUID, X> compute(List<EntityInterface> entities) {
+    Map<UUID, X> result = null;
+    if (entities != null && !entities.isEmpty() && supportsX(entities.get(0))) {
+      Map<UUID, X> prefetched = doWork(entities);
+      if (!prefetched.isEmpty()) {
+        result = prefetched;
+      }
+    }
+    return result;
+  }
+  ```
+- **Maximum 3 levels of nesting.** Don't flatten by sprinkling early returns — extract a named helper or combine conditions into a single boolean:
+  ```java
+  // BAD: deeply nested
+  if (entity != null) {
+      if (entity.isActive()) {
+          if (hasPermission(entity)) {
+              process(entity);
+          }
+      }
+  }
+
+  // GOOD: extract the eligibility check
+  if (isEligibleForProcessing(entity)) {
+    process(entity);
+  }
+
+  private boolean isEligibleForProcessing(Entity entity) {
+    return entity != null && entity.isActive() && hasPermission(entity);
+  }
+  ```
+- **Maximum 10 cyclomatic complexity.** Extract complex conditions into named methods:
+  ```java
+  // BAD: complex inline boolean
+  if (entity.getStatus() == ACTIVE && entity.getOwner() != null
+      && !entity.isDeleted() && entity.getVersion() > 0.1) { ... }
+
+  // GOOD: self-documenting
+  if (isEligibleForProcessing(entity)) { ... }
+  ```
+- **Maximum 5 parameters.** Introduce a parameter object or builder for more.
+- **Each method does one thing.** If you can describe what a method does using "and" or "then", it should be two methods.
+
+#### Naming and Readability
+- Names should make code read like prose — if you need a comment, the name isn't good enough
+- **Methods**: verb phrases — `calculateScore()`, `findByName()`, `isValid()`
+- **Booleans**: question-form — `isActive`, `hasPermission`, `canRetry` (never `flag`, `status`, `check`)
+- **Variables**: descriptive, no abbreviations — `entityReference` not `er`, `retryCount` not `rc`
+- **Constants**: `UPPER_SNAKE_CASE` — `MAX_RETRY_COUNT`, `DEFAULT_PAGE_SIZE`
+- **No single-letter variables** except in short lambdas or loop indices
+
+#### Immutability and Defensive Design
+- Use `final` on local variables and parameters that don't change (which is most of them)
+- Use `final` on fields set in the constructor
+- Return `Collections.unmodifiableList()` / `List.copyOf()` from public methods, never expose internal mutable collections
+- Utility classes must be `final` with a private constructor
+- Prefer `record` for immutable data carriers where appropriate
+
+#### Error Handling
+- **No empty catch blocks** — at minimum, log the exception
+- **No `catch (Exception e)`** — catch the specific type you expect
+- **No `e.printStackTrace()`** — use the logger
+- **Error messages must include context**: `"Table '%s' not found in database '%s'"` not just `"Not found"`
+- **No `throw` or `return` inside `finally` blocks** — they mask the original exception
+- **No exceptions for flow control** — use conditionals for expected cases
+
+#### No Magic Strings — Define Constants
+- **Never use raw string literals in `.equals()`, `.contains()`, or `switch` cases** — define a constant or use an existing enum
+- If an enum already exists in `openmetadata-spec/` schemas for those values, use it
+- If the same string appears in more than one place, it must be a named constant
+- **One definition, one location** — don't define the same constant in multiple classes
+- Prefer enums over string constants when the values form a closed set:
+  ```java
+  // BAD: magic strings scattered everywhere
+  if (taskStatus.equals("Open")) { ... }
+  if (config.getResources().get(0).equals("all")) { ... }
+
+  // GOOD: use existing enums or define constants
+  if (taskStatus == TaskStatus.OPEN) { ... }
+  private static final String RESOURCE_ALL = "all";
+  ```
+
+#### No Convoluted if/else Chains
+- **More than 3 `else if` branches means the structure is wrong — refactor:**
+  - `else if` chain on `instanceof` → `switch` with pattern matching (Java 21)
+  - `else if` chain on enum values → `switch` expression
+  - `else if` chain on `.equals("string")` → `Map` dispatch or enum lookup
+  - `else if` chain on `.contains("string")` → `Map` or list of predicates
+- **Repeated compound conditions** (same multi-part `&&`/`||` expression in multiple places) → extract into a named method or `Set.contains()`
+  ```java
+  // BAD: 3-part condition repeated 3 times across the file
+  if (!tenantId.equals("common") && !tenantId.equals("organizations")
+      && !tenantId.equals("consumers")) { ... }
+
+  // GOOD: define once, use everywhere
+  private static final Set<String> MULTI_TENANT_IDS =
+      Set.of("common", "organizations", "consumers");
+
+  private boolean isSingleTenant(String tenantId) {
+      return !MULTI_TENANT_IDS.contains(tenantId);
+  }
+  ```
+
+#### No Code Duplication
+- If the same logic exists in two places, extract to a shared method
+- Near-identical methods (e.g., same logic for OpenSearch and ElasticSearch) should share a common implementation with only the engine-specific parts varying
+- Copy-pasted blocks within the same file should be extracted into a parameterized method
+
+#### Class Size
+- **Classes should be under 500 lines.** Over 1000 lines is a design problem.
+- If a class is large, look for clusters of methods that operate on the same subset of fields — extract them into a new focused class
+- Resource classes should be thin orchestrators
+- Repository classes handle data access, not business logic
+
+#### Modern Java (Java 21)
+- Use try-with-resources for all `AutoCloseable` objects
+- Use diamond operator `<>` — `new ArrayList<>()` not `new ArrayList<String>()`
+- Use pattern matching: `if (obj instanceof String s)` instead of cast
+- Use `switch` expressions instead of `if/else if` chains on enums or types
+- Use `List.of()`, `Map.of()`, `Set.of()` for immutable collection literals
+- Use `Optional` correctly: never as a field type, never as a parameter, never assign `null` to it
+- Use text blocks `"""` for multi-line strings
+- **Use `SequencedCollection` accessors on Lists/Deques** — `list.getFirst()` / `list.getLast()` (Java 21) instead of `list.get(0)` / `list.get(list.size() - 1)`. Same for `removeFirst()` / `removeLast()`. Reads more clearly and avoids off-by-one indexing.
+- **Collection emptiness: use the project's `nullOrEmpty(...)` helper** from `org.openmetadata.common.utils.CommonUtil` instead of hand-rolling `coll != null && !coll.isEmpty()` (or its negation). It's the established idiom across this codebase, handles `null` correctly, and reads as a single semantic check. Same applies to `String` checks — use `nullOrEmpty(str)` not `str != null && !str.isEmpty()`.
+  ```java
+  // BAD
+  if (entities != null && !entities.isEmpty()) {
+    process(entities.get(0));
+  }
+
+  // GOOD
+  if (!nullOrEmpty(entities)) {
+    process(entities.getFirst());
+  }
+  ```
+
+#### Common Bug Patterns to Avoid
+- `equals()` without `hashCode()` (or vice versa)
+- `equals()` on arrays — use `Arrays.equals()`
+- Ignoring return values of `String.replace()`, `File.delete()`
+- `collection.size() == 0` — use `collection.isEmpty()`
+- String concatenation inside loops — use `StringBuilder`
+- `synchronized` on non-final fields — the lock reference can change
+- `toLowerCase()` without `Locale` — always use `toLowerCase(Locale.ROOT)`
+- Double map lookups — use `computeIfAbsent()` or `getOrDefault()`
+
+#### Testing
+- Generate production-ready code, not tutorial code
+- Create integration tests in `openmetadata-integration-tests` for new API endpoints
+- **Never use `Thread.sleep()` in tests** — use condition-based waiting or `Awaitility`
+- Bug fixes must include a test that fails without the fix
+- 90% line coverage target on changed classes
+
+#### Structure
+- Do not use Fully Qualified Names in code (e.g., `org.openmetadata.schema.type.Status`) — import the class instead
+- Do not import wildcard packages — import exactly the required classes
+- No commented-out code — version control maintains history
+- No TODOs without a ticket reference
+- One statement per line — no `if (x) return y;` on one line
+
+### TypeScript/Frontend Code Requirements
+
+**Always run the UI checkstyle sequence before you finish any task that
+touched `.ts`/`.tsx`/`.js`/`.jsx`/`.json` under
+`openmetadata-ui/src/main/resources/ui/src/`, `.../playwright/`, or
+`openmetadata-ui-core-components/src/main/resources/ui/src/`.** CI's
+`UI Checkstyle / lint-src|lint-playwright|lint-core-components` jobs fail the
+PR otherwise. The order matters — run `organize-imports-cli`, then
+`eslint --fix`, then `prettier --write`; reversing organize-imports and
+prettier leaves a dirty diff (organize-imports uses 4-space indentation,
+prettier uses 2 + trailing commas). When asked to "fix UI checkstyle" / "run
+prettier" / "fix UI lint", invoke the `ui-checkstyle` skill (see
+`.claude/skills/ui-checkstyle/`) rather than hand-editing formatting.
+
+- **NEVER use `any` type** in TypeScript code - always use proper types
+- Use `unknown` when the type is truly unknown and add type guards
+- Import types from existing type definitions (e.g., `RJSFSchema` from `@rjsf/utils`)
+- Add `// eslint-disable-next-line` comments only when absolutely necessary
+- **Import Organization** — use `yarn organize-imports:cli` to auto-sort. Order:
+  1. External libraries (React, etc.)
+  2. Internal absolute imports from `generated/`, `constants/`, `hooks/`, etc.
+  3. Relative imports for utilities and components
+  4. Asset imports (SVGs, styles)
+  5. Type imports grouped separately when needed
+
+#### CI Checkstyle Rules (enforced on every PR)
+These checks run automatically in CI. Code that violates them **will not merge**.
+- **No `console.log/warn/error`** — `no-console` rule is enforced. Use the logger or remove.
+- **Use `===` not `==`** — `eqeqeq` (smart mode, except for `null` checks)
+- **Max 200 characters per line** — break long lines
+- **Self-closing components** — `<Div />` not `<Div></Div>`
+- **Sort JSX props alphabetically** — callbacks last
+- **Space after `//` in comments** — `// comment` not `//comment`
+- **Blank lines** before `function`, `class`, `export`, `return` statements
+- **Use `it()` consistently in tests** — don't mix `test()` and `it()`
+- **Blank lines around `describe`, `it`, `beforeEach`** in test files
+- **JSON keys sorted alphabetically** in locale files (`src/locale/**/*.json`)
+- **Apache 2.0 license header** on every new source file — run `yarn license-header-fix`
+- **i18n keys synced** — after adding keys to `en-us.json`, run `yarn i18n` to sync all 17 locales
+- **Prettier formatting** — 2-space indent, single quotes, strict HTML whitespace
+
+#### Playwright Test Rules (lint-playwright)
+- **No `waitForLoadState('networkidle')`** — flaky, use web-first assertions
+- **No `page.pause()`** — remove before committing
+- **No `.only` on tests** — blocks all other tests in CI
+- Prefer `expect(locator).toBeVisible()` over manual `waitForSelector` checks
+- Don't use `{ force: true }` — fix the locator instead
+- Use locators, not element handles
+
+### Python Code Requirements
+- **Use pytest, not unittest** - write tests using pytest style with plain `assert` statements
+- Use pytest fixtures for test setup instead of `setUp`/`tearDown` methods
+- Use `unittest.mock` for mocking (MagicMock, patch) - this is compatible with pytest
+- Test classes should not inherit from `TestCase` - use plain classes prefixed with `Test`
+- Use `assert x == y` instead of `self.assertEqual(x, y)`
+- Use `assert x is None` instead of `self.assertIsNone(x)`
+- Use `assert "text" in string` instead of `self.assertIn("text", string)`
+
+### Python Ingestion Connector Guidelines
+- **Keep connector-specific logic in connector-specific files**, not in generic/shared files like `builders.py`
+- Example: Redshift IAM auth should be in `ingestion/src/metadata/ingestion/source/database/redshift/connection.py`, not in `ingestion/src/metadata/ingestion/connections/builders.py`
+- This keeps the codebase modular and prevents generic utilities from becoming cluttered with connector-specific edge cases
+- **Use `model_str()` for Pydantic RootModel to string conversion** — OpenMetadata schema types like `ColumnName`, `EntityName`, `FullyQualifiedEntityName`, and `UUID` are Pydantic `RootModel[str]` subclasses where `str()` returns `"root='value'"` instead of the raw value. Always use `model_str()` from `metadata.ingestion.ometa.utils` instead of manual `hasattr(x, "root")` / `str(x.root)` checks.
+
+### Caching
+- **All caches MUST be bounded.** Never use a bare `dict` / `HashMap` / `Map` as a cache without an explicit size cap — they grow with the input and cause OOMs on large catalogs/ingestions. The only exception is when the user explicitly asks for an unbounded cache for a specific case.
+- Pick a sane default (typically 100–1000 entries depending on entity size); if you're unsure, ask the user.
+- **Python**: use `collections.OrderedDict` with `popitem(last=False)` eviction after insert, `@functools.lru_cache(maxsize=N)`, or `cachetools.LRUCache`. Cache both hits and misses (negative caching) — repeated unresolvable lookups are a common hot path.
+- **Java**: use Caffeine (`Caffeine.newBuilder().maximumSize(N).build()`) or Guava `CacheBuilder.newBuilder().maximumSize(N).build()`. Never a bare `HashMap`.
+- **TypeScript**: use `lru-cache` — never a bare `Map` or plain object.
+- **Before adding a cache, check whether the underlying call is already cached at a lower layer.** Example: `OpenMetadata._search_es_entity` is `@lru_cache(maxsize=512)`, so wrapping `get_entity_from_es` / `es_search_container_by_path` calls in a local dict cache is redundant — drop the local cache and rely on the existing LRU.
+
+### Testing Philosophy
+- **Test real behavior, not mock wiring** - if a test requires mocking 3+ classes just to verify a method call, it's testing the wrong thing
+- **Prefer integration tests** over heavily-mocked unit tests. This project has full integration test infrastructure (OpenMetadataApplicationTest, Docker containers, real OpenSearch). Use it.
+- **Mocks are for boundaries, not internals** - mock external services (HTTP clients, third-party APIs), not your own classes. If you're mocking static methods left and right to test internal plumbing, write an integration test instead.
+- **A test that mocks everything proves nothing** - it only verifies that your mocks are wired correctly, not that the system works
+- **Ask "what breaks if this test passes but the code is wrong?"** - if the answer is "nothing, because everything real is mocked out", delete the test and write a better one
+- **Test the outcome, not the implementation** - assert on observable results (API responses, database state, stats values) rather than verifying internal method calls with `verify()`
+
+### Response Format
+- Provide clean code blocks without unnecessary explanations
+- Assume readers are experienced developers
+- Focus on functionality over education

CODE_OF_CONDUCT.md

@@ -0,0 +1,91 @@
+# OpenMetadata Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for
+everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity, and expression, level of experience, education, socio-economic status, nationality, personal appearance, race,
+caste, color, religion, or sexual identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our community include:
+
+- Demonstrating empathy and kindness toward other people
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
+- Focusing on what is best not just for us as individuals, but for the overall community
+
+Examples of unacceptable behavior include:<br>
+
+- The use of sexualized language or imagery, and sexual attention or advances of any kind
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing other’s private information, such as a physical or email address, without their explicit permission
+- Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take
+appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits,
+issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for
+moderation decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing
+the community in public spaces. Examples of representing our community include using an official e-mail address, posting
+via an official social media account, or acting as an appointed representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible
+for enforcement at contact@open-metadata.org. All complaints will be reviewed and investigated promptly and fairly. All
+community leaders are obligated to respect the privacy and security of the reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem
+in violation of this Code of Conduct:
+
+### **1. Correction**<br>
+
+Community Impact: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
+
+Consequence: A private, written warning from community leaders, providing clarity around the nature of the violation and
+an explanation of why the behavior was inappropriate. A public apology may be requested.
+
+### **2. Warning**<br>
+
+Community Impact: A violation through a single incident or series of actions.
+
+Consequence: A warning with consequences for continued behavior. No interaction with the people involved, including
+unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding
+interactions in community spaces as well as external channels like social media. Violating these terms may lead to a
+temporary or permanent ban.
+
+### **3. Temporary Ban**<br>
+
+Community Impact: A serious violation of community standards, including sustained inappropriate behavior.
+
+Consequence: A temporary ban from any sort of interaction or public communication with the community for a specified
+period of time. No public or private interaction with the people involved, including unsolicited interaction with those
+enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
+
+### **4. Permanent Ban**<br>
+
+Community Impact: Demonstrating a pattern of violation of community standards, including sustained inappropriate
+behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
+
+Consequence: A permanent ban from any sort of public interaction within the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), version 2.1,
+available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.

CONTRIBUTING.md

@@ -0,0 +1,13 @@
+# Contributors
+
+We ❤️ all contributions, big and small!
+
+Read [Build Code and Run Tests](https://docs.open-metadata.org/developers/contribute/build-code-and-run-tests) for how to setup your local development environment. Get started with our [Good first issues](https://github.com/open-metadata/OpenMetadata/issues?q=is%3Aissue+is%3Aopen+label%3A%22good-first-issue%22).
+
+We also recommend joining the OpenMetadata [Slack Workspace](https://slack.open-metadata.org/) to meet the team, stay up to date with product features, and chat with other contributors! 
+
+Once joining our Slack we recommend Introducing yourself in the #introductions channel. Tell us a bit about yourself and how you are currently using OpenMetadata if you are.
+
+If you need help getting started in contributing, you can reach out in the #support channel and ask if anyone would be available to get you up to speed or just answer some questions.  
+
+If you just have suggestions for how to improve OpenMetadata you can submit your feedback in #feature-requests. If you have a fix or contribution in mind but need some guidance, please post in the #contributor channel with a description of the change before opening a new Github issue. 

DEVELOPER.md

@@ -0,0 +1,582 @@
+# DEVELOPER.md — AI-Assisted Development Guide for OpenMetadata
+
+This guide helps developers (and AI agents like Claude Code, Codex, Copilot) write correct, production-quality code in the OpenMetadata codebase. It covers the preferred workflow for each language, architecture patterns you must understand, and how to use the available skills.
+
+For environment setup, build commands, and coding standards, see [CLAUDE.md](CLAUDE.md).
+For connector-specific development, see [skills/README.md](skills/README.md).
+
+---
+
+## Table of Contents
+
+- [Preferred Workflow by Language](#preferred-workflow-by-language)
+- [Using the Skills](#using-the-skills)
+- [Architecture Deep Dives](#architecture-deep-dives)
+  - [Schema-First Design](#schema-first-design)
+  - [Entity Model and Registry](#entity-model-and-registry)
+  - [REST Resource Pattern](#rest-resource-pattern)
+  - [JDBI3 Data Access Layer](#jdbi3-data-access-layer)
+  - [Database Migrations](#database-migrations)
+  - [Change Events and Audit](#change-events-and-audit)
+  - [Authorization (RBAC)](#authorization-rbac)
+  - [Search Infrastructure](#search-infrastructure)
+  - [Python Ingestion Topology](#python-ingestion-topology)
+  - [Frontend Patterns](#frontend-patterns)
+- [Cross-Cutting Patterns](#cross-cutting-patterns)
+
+---
+
+## Preferred Workflow by Language
+
+### Java Backend
+
+```
+1. /planning          — Design the approach (which entities, endpoints, migrations?)
+2. /tdd               — Write a failing integration test in openmetadata-integration-tests/
+3.                       Implement in openmetadata-service/
+4.                       mvn spotless:apply
+5. /test-enforcement   — Verify 90% coverage, integration tests for all endpoints
+6. /verification       — Show passing test output
+7. /code-review        — Run java-reviewer agent for Kafka-grade quality check
+```
+
+**Key rules:**
+- Start with the JSON schema if adding/modifying an entity (`openmetadata-spec/`)
+- Always write the integration test first — it proves the API contract works
+- Methods must be 15 lines or fewer, no magic strings, no convoluted if/else chains
+- Run `mvn spotless:apply` before every commit
+- Every new REST endpoint needs a corresponding `*IT.java` in `openmetadata-integration-tests/`
+
+### React/TypeScript Frontend
+
+```
+1. /planning          — Identify components, state management, API contracts
+2. /tdd               — Write Jest test for the component
+3.                       Implement the component
+4.                       yarn lint:fix
+5. /test-enforcement   — Verify Jest coverage + Playwright E2E if user-facing
+6. /verification       — Show lint + test output
+7. /code-review        — Run frontend-reviewer agent
+```
+
+**Key rules:**
+- Use components from `openmetadata-ui-core-components`, never MUI
+- All Tailwind classes use `tw:` prefix, all colors use CSS custom properties
+- No string literals in JSX — use `t('label.key-name')` from `useTranslation()`
+- No `any` type — use generated types from `generated/` or define proper interfaces
+- New keys go in `locale/languages/en-us.json` using kebab-case under the appropriate namespace
+- Run `yarn parse-schema` after any connection schema changes
+
+### Python Ingestion
+
+```
+1. /planning          — Choose connector architecture (SQLAlchemy vs REST vs SDK)
+2. /connector-standards — Load the relevant standards
+3. /tdd               — Write pytest tests first
+4.                       Implement using topology pattern
+5.                       make py_format && make py_format_check
+6. /test-enforcement   — Verify 90% coverage
+7. /verification       — Show test + lint output
+8. /connector-review   — Full review against golden standards (for connectors)
+```
+
+**Key rules:**
+- Use pytest style — plain `assert`, no `unittest.TestCase`
+- Use the topology pattern (`ServiceSpec` + `TopologyNode`) for all connectors
+- Keep connector-specific logic in the connector's directory, not in shared files
+- Use generators (`yield`) for streaming entities — never accumulate in memory
+- Implement pagination for all REST API calls
+- Reuse HTTP sessions — create one `requests.Session()` per connector lifetime
+
+---
+
+## Using the Skills
+
+### Slash Commands Quick Reference
+
+| Command | When to use | What it does |
+|---------|-------------|-------------|
+| `/planning` | Starting any non-trivial task | Brainstorm approaches, get approval, create step-by-step plan |
+| `/tdd` | Implementing any feature or fix | Guides RED-GREEN-REFACTOR cycle for Java/Python/TypeScript |
+| `/test-enforcement` | Before creating a PR | Checks 90% coverage, integration tests, Playwright E2E |
+| `/verification` | Before claiming "done" | Requires actual test output as evidence |
+| `/code-review` | Before or during PR review | Two-stage review: spec compliance then code quality |
+| `/systematic-debugging` | Bug with unclear root cause | 4-phase: gather evidence, hypothesize, verify, fix |
+| `/playwright` | Adding E2E tests | Generates zero-flakiness Playwright tests following handbook |
+| `/connector-standards` | Before connector work | Loads all 23 connector development standards |
+| `/connector-review` | Reviewing connector PRs | Multi-agent review against golden standards |
+| `/scaffold-connector` | Building a new connector | Generates JSON Schema, Python boilerplate, AI context |
+| `/test-locally` | Testing in full environment | Builds and deploys local Docker stack |
+
+### Workflow Routing
+
+The `openmetadata-workflow` meta-skill (loaded at session start) routes tasks automatically:
+
+| Task | Skills triggered in order |
+|------|--------------------------|
+| New feature (multi-file) | `/planning` -> `/tdd` -> `/test-enforcement` -> `/verification` |
+| Bug fix | `/systematic-debugging` -> `/tdd` -> `/verification` |
+| New API endpoint | `/planning` -> `/tdd` -> `/test-enforcement` (must include IT) |
+| New connector | `/connector-standards` -> `/scaffold-connector` -> `/test-enforcement` |
+| UI component | `/tdd` -> `/test-enforcement` (Jest + Playwright) |
+| PR review | `/code-review` -> `/test-enforcement` |
+
+---
+
+## Architecture Deep Dives
+
+### Schema-First Design
+
+OpenMetadata uses a single-source-of-truth approach: JSON Schema definitions drive code generation across all languages.
+
+**The pipeline:**
+```
+openmetadata-spec/src/main/resources/json/schema/
+    │
+    ├── entity/          → Entity definitions (table.json, dashboard.json, ...)
+    │   ├── data/        → Data entities (table, topic, dashboard, pipeline, ...)
+    │   ├── services/    → Service entities (databaseService, dashboardService, ...)
+    │   │   └── connections/  → Connection configs per service type
+    │   ├── teams/       → Team/user entities
+    │   ├── policies/    → Governance entities
+    │   └── feed/        → Activity feed entities
+    │
+    ├── api/             → API request/response objects (createTable.json, ...)
+    │
+    └── type/            → Shared type definitions (entityReference.json, tagLabel.json, ...)
+
+                    ↓ Code generation ↓
+
+Java POJOs:     jsonschema2pojo → openmetadata-spec/target/generated-sources/
+Python models:  datamodel-code-generator → ingestion/src/metadata/generated/
+TypeScript:     QuickType → openmetadata-ui/.../ui/src/generated/
+UI forms:       parseSchemas.js → resolved JSON for RJSF auto-rendering
+```
+
+**When you modify a schema:**
+1. Edit the JSON schema in `openmetadata-spec/`
+2. Run `make generate` (Python models)
+3. Run `mvn clean install -pl openmetadata-spec` (Java POJOs)
+4. Run `yarn parse-schema` (UI connection schemas only)
+5. Add corresponding Flyway migration if the change affects the database
+
+**Schema conventions:**
+- `$id` must match the file path
+- `title` is camelCase of the filename
+- `javaType` follows `org.openmetadata.schema.{category}.{ClassName}`
+- Use `$ref` for shared types
+- Set `additionalProperties: false` on connection schemas
+- Feature flags: `supportsMetadataExtraction`, `supportsProfiler`, `supportsDBTExtraction`, `supportsQueryComment`
+
+### Entity Model and Registry
+
+All entities implement `EntityInterface` and are registered in `Entity.java` — the central singleton registry.
+
+**Entity.java key patterns:**
+
+```java
+// Entity type string constants (use these, never raw strings)
+Entity.TABLE           // "table"
+Entity.DATABASE        // "database"
+Entity.DATABASE_SCHEMA // "databaseSchema"
+Entity.DASHBOARD       // "dashboard"
+Entity.PIPELINE        // "pipeline"
+
+// Common field name constants (use these for field references)
+Entity.FIELD_OWNERS
+Entity.FIELD_TAGS
+Entity.FIELD_DESCRIPTION
+Entity.FIELD_DOMAINS
+Entity.FIELD_FULLY_QUALIFIED_NAME
+
+// FQN separator
+Entity.SEPARATOR       // "."
+
+// Access repositories by entity type
+EntityRepository<?> repo = Entity.getRepository(entityType);
+
+// Build href for entity references
+Entity.withHref(uriInfo, entityReference);
+```
+
+**Fully Qualified Names (FQN):**
+
+Entities form a hierarchy with `.` separator:
+```
+databaseService.database.databaseSchema.table
+databaseService.database.databaseSchema.table.column
+dashboardService.dashboard
+pipelineService.pipeline
+```
+
+Each `EntityRepository` must implement `setFullyQualifiedName()` to build the FQN from parent FQN + entity name.
+
+### REST Resource Pattern
+
+All entity REST resources extend `EntityResource<E, R extends EntityRepository<E>>`.
+
+**Creating a new resource:**
+
+```java
+@Path("/v1/myEntities")
+@Tag(name = "MyEntities")
+@Produces(MediaType.APPLICATION_JSON)
+@Consumes(MediaType.APPLICATION_JSON)
+@Collection(name = "myEntities")
+public class MyEntityResource extends EntityResource<MyEntity, MyEntityRepository> {
+
+    public static final String COLLECTION_PATH = "/v1/myEntities/";
+    public static final String FIELDS = "owners,tags,domain";
+
+    public MyEntityResource(Authorizer authorizer, Limits limits) {
+        super(Entity.MY_ENTITY, authorizer, limits);
+    }
+
+    // Inner class for list serialization (required, no body)
+    public static class MyEntityList extends ResultList<MyEntity> {}
+
+    // Standard endpoints are inherited from EntityResource:
+    // GET    /v1/myEntities          — list with pagination
+    // GET    /v1/myEntities/{id}     — get by ID
+    // GET    /v1/myEntities/name/{fqn} — get by FQN
+    // POST   /v1/myEntities          — create
+    // PUT    /v1/myEntities          — create or update
+    // PATCH  /v1/myEntities/{id}     — JSON patch
+    // DELETE /v1/myEntities/{id}     — soft/hard delete
+}
+```
+
+**Conventions:**
+- All methods receive `@Context SecurityContext` and `@Context UriInfo`
+- Cursor-based pagination with `before`/`after` string params + `limit` int
+- Field filtering via `fields` query param (comma-separated, maps to `FIELDS` constant)
+- A dedicated `*Mapper` class handles Create DTO -> Entity mapping
+- Override `getEntitySpecificOperations()` to register field-level view permissions
+
+### JDBI3 Data Access Layer
+
+OpenMetadata uses JDBI3 (not JPA/Hibernate) for database access. All repositories extend `EntityRepository<E>`.
+
+**Creating a new repository:**
+
+```java
+@Slf4j
+public class MyEntityRepository extends EntityRepository<MyEntity> {
+
+    public MyEntityRepository() {
+        super(
+            MyEntityResource.COLLECTION_PATH,
+            Entity.MY_ENTITY,
+            MyEntity.class,
+            Entity.getCollectionDAO().myEntityDAO(),  // DAO interface
+            "",    // patch fields
+            ""     // put fields
+        );
+        supportsSearch = true;  // enable ES indexing
+    }
+
+    // Required overrides:
+
+    @Override
+    public void setFullyQualifiedName(MyEntity entity) {
+        entity.setFullyQualifiedName(
+            FullyQualifiedName.build(entity.getService().getFullyQualifiedName(),
+                                     entity.getName()));
+    }
+
+    @Override
+    public void prepare(MyEntity entity, boolean update) {
+        // Validate references, populate service, resolve owners/tags
+        populateService(entity);
+    }
+
+    @Override
+    public void storeEntity(MyEntity entity, boolean update) {
+        store(entity, update);
+    }
+
+    @Override
+    public void storeRelationships(MyEntity entity) {
+        addServiceRelationship(entity, entity.getService());
+    }
+}
+```
+
+**Key patterns:**
+- `@Transaction` annotation for multi-step writes
+- `Entity.getCollectionDAO()` provides type-safe DAO access
+- Override `getFieldsStrippedFromStorageJson()` to exclude computed fields from JSON storage
+- Bulk operations: override `storeEntities()`, `clearEntitySpecificRelationshipsForMany()`, `storeEntitySpecificRelationshipsForMany()`
+
+### Database Migrations
+
+OpenMetadata uses a **hybrid migration system**: native SQL migrations tracked in `SERVER_CHANGE_LOG`, with Flyway SQL parsers for robust semicolon handling.
+
+**Adding a new migration:**
+
+1. Create the version directory:
+   ```
+   bootstrap/sql/migrations/native/{version}/
+   ├── mysql/schemaChanges.sql
+   └── postgres/schemaChanges.sql
+   ```
+
+2. Write both MySQL and PostgreSQL variants:
+   ```sql
+   -- MySQL
+   ALTER TABLE my_entity ADD COLUMN new_field JSON;
+   ALTER TABLE my_entity ADD INDEX idx_new_field ((new_field->>'$.key'));
+
+   -- PostgreSQL
+   ALTER TABLE my_entity ADD COLUMN new_field JSONB;
+   CREATE INDEX idx_new_field ON my_entity ((new_field->>'key'));
+   ```
+
+**Rules:**
+- Always one `schemaChanges.sql` per database per version — no numbered sub-files
+- Always provide both MySQL and PostgreSQL variants
+- Migrations are tracked in `SERVER_CHANGE_LOG` (keyed by version)
+- Never add new `v0xx` Flyway files — always use the native path
+- Migrations must be idempotent where possible (`IF NOT EXISTS`, etc.)
+- Extension migrations go in `bootstrap/sql/migrations/extensions/{name}/`
+
+### Change Events and Audit
+
+Every non-GET API response triggers the change event system via `ChangeEventHandler` (a JAX-RS `ContainerResponseFilter`).
+
+**Flow:**
+```
+HTTP Response (non-GET)
+    → ChangeEventHandler.process()
+        → Extract ChangeEvent from response
+        → Set userName from SecurityContext
+        → Mask PII in entity JSON
+        → Persist to changeEventDAO (unless ENTITY_NO_CHANGE)
+        → Send to WebsocketNotificationHandler (real-time UI)
+```
+
+**Event types (`EventType` enum):**
+- `ENTITY_CREATED` — new entity
+- `ENTITY_UPDATED` — modified entity
+- `ENTITY_SOFT_DELETED` — soft delete
+- `ENTITY_DELETED` — hard delete
+- `ENTITY_NO_CHANGE` — no-op (not persisted)
+
+**When adding a new entity type:** change events are automatic if your resource extends `EntityResource`. No manual wiring needed. However, `Entity.QUERY` and `Entity.WORKFLOW` events are excluded by default.
+
+### Authorization (RBAC)
+
+Authorization uses the `Authorizer` interface, injected into all resource classes.
+
+**Pattern in resources:**
+```java
+// All mutating operations must authorize:
+authorizer.authorize(securityContext, operationContext, resourceContext);
+
+// OperationContext wraps the MetadataOperation enum:
+new OperationContext(Entity.TABLE, MetadataOperation.CREATE)
+new OperationContext(Entity.TABLE, MetadataOperation.EDIT_TAGS)
+
+// ResourceContextInterface provides the target entity
+```
+
+**Key classes:**
+- `Authorizer` — interface with `authorize()`, `authorizeAdmin()`, `authorizeAdminOrBot()`
+- `DefaultAuthorizer` — production implementation using `PolicyEvaluator`
+- `NoopAuthorizer` — allows everything (for testing)
+- `PolicyEvaluator` — evaluates rules against `SubjectContext` + `ResourceContextInterface` + `OperationContext`
+- `SubjectCache` — caches permission evaluations per user
+
+**When adding new resources:** ensure all mutating methods call `authorizer.authorize()` before execution. Override `getEntitySpecificOperations()` in the resource to register field-level view permissions.
+
+### Search Infrastructure
+
+Entities are indexed in Elasticsearch 7.17+ or OpenSearch 2.6+ for discovery.
+
+**Key patterns:**
+- Set `supportsSearch = true` in the repository constructor to enable indexing
+- Search index classes in `openmetadata-service/src/main/java/org/openmetadata/service/search/indexes/`
+- Each entity type has a corresponding `*Index.java` that defines the search document structure
+- Reindexing triggered via the reindex API or on entity create/update
+
+### Python Ingestion Topology
+
+Connectors use a **declarative topology** that defines the entity traversal order.
+
+**Core concept:**
+```python
+# Topology defines the execution graph:
+class DatabaseServiceTopology(ServiceTopology):
+    root = TopologyNode(
+        producer="get_services",
+        stages=[NodeStage(type_=DatabaseService, ...)],
+        children=["database"],
+    )
+    database = TopologyNode(
+        producer="get_database_names",
+        stages=[NodeStage(type_=Database, processor="yield_database", ...)],
+        children=["database_schema"],
+    )
+    database_schema = TopologyNode(
+        producer="get_database_schema_names",
+        stages=[NodeStage(type_=DatabaseSchema, processor="yield_database_schema", ...)],
+        children=["table"],
+    )
+    table = TopologyNode(
+        producer="get_tables_name_and_type",
+        stages=[NodeStage(type_=Table, processor="yield_table", ...)],
+    )
+```
+
+**Key patterns:**
+- `producer` is a method name on the source class that yields entity names
+- `processor` is a method name that yields the entity objects
+- `TopologyRunnerMixin` drives the depth-first traversal automatically
+- `TopologyContext` tracks the current position in the hierarchy (for FQN building)
+- `Either` monad wraps all results: `Either(right=entity)` or `Either(left=error)`
+- Fingerprinting via `sourceHash`: CREATE if new, PATCH if changed, SKIP if identical
+- Nodes with `threads=True` enable parallel processing
+
+**Source class hierarchy:**
+```
+Source (abstract)
+  → DatabaseServiceSource (defines topology + abstract methods)
+      → CommonDbSourceService (SQL extraction via SQLAlchemy)
+          → PostgresSource, MySQLSource, SnowflakeSource, ...
+```
+
+**ServiceSpec pattern:**
+```python
+ServiceSpec = DefaultDatabaseSpec(
+    metadata_source_class=BigquerySource,
+    lineage_source_class=BigqueryLineageSource,
+    usage_source_class=BigqueryUsageSource,
+    profiler_class=BigQueryProfiler,
+    sampler_class=BigQuerySampler,
+)
+```
+
+### Frontend Patterns
+
+#### i18n Key Structure
+
+Translation keys live in `locale/languages/en-us.json`:
+
+```json
+{
+  "label": {
+    "add-entity": "Add {{entity}}",
+    "activity-feed": "Activity Feed",
+    "activity-feed-plural": "Activity Feeds",
+    "delete-entity": "Delete {{entity}}"
+  },
+  "message": {
+    "entity-deleted-successfully": "{{entity}} deleted successfully!"
+  },
+  "server": {
+    "unexpected-error": "An unexpected error occurred"
+  }
+}
+```
+
+**Conventions:**
+- Keys use kebab-case: `add-data-product`, `activity-feed-and-task-plural`
+- Namespaces: `label` (UI labels), `message` (user-facing messages), `server` (error messages)
+- Interpolation: `{{paramName}}` double-brace mustache
+- Plurals: append `-plural` suffix: `"activity"` / `"activity-plural"`
+- Variants: `-uppercase`, `-lowercase`, `-with-colon`
+
+**Usage:**
+```tsx
+const { t } = useTranslation();
+// Simple
+<span>{t('label.activity-feed')}</span>
+// With parameter
+<span>{t('label.add-entity', { entity: t('label.table') })}</span>
+```
+
+#### Component Library
+
+Use `openmetadata-ui-core-components` for all new UI work:
+- Components: Button, Input, Select, Modal, Table, Tabs, Pagination, Badge, Avatar, Checkbox, Dropdown, Form, Card, Tooltip, Toggle, Slider, Textarea, Tags
+- Source: `openmetadata-ui-core-components/src/main/resources/ui/src/components/`
+- Tailwind CSS v4 with `tw:` prefix for all utility classes
+- CSS custom properties for design tokens (see `globals.css`)
+
+#### State Management
+
+| Scope | Tool | Example |
+|-------|------|---------|
+| Component-local | `useState` | Form inputs, toggle states |
+| Feature-shared | Context providers | `ApplicationsProvider` |
+| Global | Zustand stores | `useLimitStore`, `useWelcomeStore` |
+
+#### Generated Types
+
+TypeScript interfaces are generated from JSON schemas and live in:
+```
+openmetadata-ui/src/main/resources/ui/src/generated/
+```
+
+Always import from `generated/` for API response types. Never hand-write interfaces for schema-defined types.
+
+---
+
+## Cross-Cutting Patterns
+
+### Design Patterns Used Across the Codebase
+
+| Pattern | Where | Purpose |
+|---------|-------|---------|
+| Schema-first | Everywhere | JSON Schema drives all code generation |
+| Topology | Python ingestion | Declarative traversal of entity hierarchies |
+| Either monad | Python ingestion | Unified error handling without exceptions |
+| Singledispatch | `MetadataRestSink` | Type-based routing for entity persistence |
+| Registry | `Entity.java`, `Metrics` enum | Central lookup for entity types and metric implementations |
+| Template method | Validators, repositories | Base class defines skeleton, subclasses fill in steps |
+| Strategy via mixins | Profiler, sampler | SQA vs Pandas implementations composed via mixin |
+| Dynamic import | Connectors, validators | Zero-config discovery by file path convention |
+| Fingerprinting | Ingestion | `sourceHash` for incremental create/patch/skip |
+| Mixin composition | `OpenMetadata` API client | 25+ specialized mixins for different entity operations |
+| Factory | Interface/sampler/profiler | Create the right implementation for the service type |
+| Cascade parsing | Lineage | SqlGlot -> SqlFluff -> SqlParse (each with timeout) |
+
+### Performance Patterns
+
+- **Pagination is mandatory** for all list APIs (REST and database)
+- **Stream, don't accumulate** — use generators in Python, iterators in Java
+- **Reuse HTTP sessions** — one `requests.Session()` per connector lifetime
+- **Bound caches** with `lru_cache(maxsize=N)` or size-limited maps
+- **Build lookup dictionaries in `prepare()`** for O(1) access instead of repeated iteration
+- **Use `computeIfAbsent()`** instead of `containsKey()` + `get()` double lookups
+- **No `Thread.sleep()` in tests** — use condition-based waiting
+
+### Adding a New Entity (End-to-End Checklist)
+
+1. **JSON Schema**: Create `openmetadata-spec/src/main/resources/json/schema/entity/{category}/{entity}.json`
+2. **API Schema**: Create `openmetadata-spec/src/main/resources/json/schema/api/{category}/create{Entity}.json`
+3. **Generate code**: `mvn clean install -pl openmetadata-spec` + `make generate`
+4. **Entity constant**: Add `Entity.MY_ENTITY = "myEntity"` in `Entity.java`
+5. **DAO**: Add `myEntityDAO()` method to `CollectionDAO`
+6. **Repository**: Create `MyEntityRepository extends EntityRepository<MyEntity>`
+7. **Mapper**: Create `MyEntityMapper`
+8. **Resource**: Create `MyEntityResource extends EntityResource<MyEntity, MyEntityRepository>`
+9. **Migration**: Create `bootstrap/sql/migrations/native/{version}/mysql/schemaChanges.sql` + postgres variant
+10. **Search index**: Create `MyEntityIndex.java` if searchable
+11. **Integration test**: Create `MyEntityIT extends BaseEntityIT<MyEntity, CreateMyEntity>` in `openmetadata-integration-tests/`
+12. **Frontend**: Add generated types, API client methods, and UI components
+13. **i18n**: Add labels to `en-us.json`
+
+### Adding a New Connector (End-to-End Checklist)
+
+1. **Connection schema**: `openmetadata-spec/src/main/resources/json/schema/entity/services/connections/{type}/{connector}.json`
+2. **Service type enum**: Add to `{type}Service.json` `oneOf` list
+3. **Generate code**: `make generate` + `mvn clean install -pl openmetadata-spec`
+4. **ClassConverter** (if using `oneOf`): `openmetadata-service/src/main/java/org/openmetadata/service/secrets/converter/`
+5. **Python source class**: `ingestion/src/metadata/ingestion/source/{type}/{connector}/`
+   - `connection.py` — connection handling
+   - `metadata.py` — metadata extraction
+   - `__init__.py` — ServiceSpec definition
+6. **Unit tests**: `ingestion/tests/unit/topology/{type}/test_{connector}.py`
+7. **UI integration**: Update `{type}ServiceUtils.tsx`, add MDX doc file
+8. **Run**: `yarn parse-schema` for UI form generation

INCIDENT_RESPONSE.md

@@ -0,0 +1,305 @@
+# OpenMetadata Incident Response Plan
+
+This document outlines the incident response procedures for security issues in the OpenMetadata project.
+
+## Scope
+
+This incident response plan covers:
+- All code within the OpenMetadata organization repositories
+- Security vulnerabilities in OpenMetadata services
+- Metadata exposure incidents
+- Supply chain security issues
+- Infrastructure compromises
+
+## Incident Lead
+
+**Primary Incident Lead**: @harshach
+- Responsible for coordinating incident response
+- Decision authority for security releases
+- External communication coordination
+
+**Backup Incident Leads**: @pmbrull, @mohityadav766, @tutte
+
+## Reporting Security Issues
+
+### How to Report
+
+All security issues must be reported privately through one of these channels:
+
+1. **GitHub Security Advisories** (Preferred)
+   - Navigate to: https://github.com/open-metadata/OpenMetadata/security/advisories
+   - Click "Report a vulnerability"
+   - Provide detailed information
+
+2. **Email**: security@open-metadata.org
+   - Encrypt sensitive details using our PGP key (available on our website)
+
+### What to Include
+
+- Detailed description of the vulnerability
+- Steps to reproduce
+- Potential impact assessment
+- Affected versions
+- Any proof-of-concept code (if applicable)
+
+### What NOT to Do
+
+- **DO NOT** create public GitHub issues for security vulnerabilities
+- **DO NOT** disclose vulnerabilities on public forums or social media
+- **DO NOT** submit pull requests with security fixes without prior coordination
+
+## Response Timeline
+
+- **Initial Response**: Within 24 hours
+- **Impact Assessment**: Within 72 hours
+- **Resolution Target**: Within 30 days for critical issues
+- **Public Disclosure**: Coordinated after fix is available
+
+## Incident Response Phases
+
+### Phase 1: Detection & Initial Response (0-24 hours)
+
+1. **Acknowledge Receipt**
+   - Send confirmation to reporter
+   - Assign tracking identifier
+   - Engage incident lead (@harshach)
+
+2. **Initial Assessment**
+   - Verify the vulnerability
+   - Determine severity using CVSS scoring
+   - Identify affected versions
+   - Check if actively exploited
+
+3. **Immediate Containment** (if critical)
+   - Disable affected features if possible
+   - Notify cloud service providers if applicable
+   - Revoke compromised credentials
+
+### Phase 2: Investigation & Coordination (24-72 hours)
+
+1. **Form Response Team**
+   - Incident Lead: @harshach
+   - Security Engineer(s)
+   - Affected component maintainer(s)
+   - Communications coordinator
+
+2. **Deep Investigation**
+   - Root cause analysis
+   - Full impact assessment
+   - Check for similar vulnerabilities
+   - Review logs for exploitation attempts
+
+3. **Develop Fix**
+   - Create patches for supported versions
+   - Develop test cases
+   - Security review of proposed fix
+
+### Phase 3: Mitigation & Remediation
+
+1. **Prepare Releases**
+   - Patch all supported versions
+   - Update documentation
+   - Prepare security advisory
+
+2. **Coordinate Disclosure**
+   - Notify major users under embargo (if applicable)
+   - Request CVE assignment
+   - Coordinate disclosure date with reporter
+
+3. **Deploy Fixes**
+   - Release patched versions
+   - Update Docker images
+   - Update Helm charts
+   - Deploy to cloud services
+
+### Phase 4: Communication
+
+1. **Internal Communication**
+   - Update internal teams
+   - Brief support teams
+   - Update runbooks
+
+2. **External Communication**
+   - Publish security advisory
+   - Update security page
+   - Send notifications to mailing list
+   - Social media announcements (if critical)
+
+3. **Credit & Acknowledgment**
+   - Credit reporter (unless anonymity requested)
+   - Update security acknowledgments page
+
+### Phase 5: Post-Incident Review
+
+1. **Incident Review** (Within 1 week)
+   - Timeline review
+   - Response effectiveness
+   - Lessons learned
+   - Process improvements
+
+2. **Security Improvements**
+   - Update security practices
+   - Enhance testing procedures
+   - Update threat model
+   - Security training needs
+
+3. **Documentation Updates**
+   - Update incident response plan
+   - Update security documentation
+   - Share learnings with community
+
+## Severity Levels
+
+### Critical (CVSS 9.0-10.0)
+- Complete system compromise
+- Unauthorized access to all metadata
+- Remote code execution
+- Authentication bypass
+- **Response Time**: Immediate, fix within 24-48 hours
+
+### High (CVSS 7.0-8.9)
+- Significant metadata exposure
+- Privilege escalation
+- Connection credential exposure
+- **Response Time**: Within 7 days
+
+### Medium (CVSS 4.0-6.9)
+- Limited metadata exposure
+- Denial of service
+- Information disclosure
+- **Response Time**: Within 30 days
+
+### Low (CVSS 0.1-3.9)
+- Minor information leakage
+- Difficult to exploit issues
+- **Response Time**: Next regular release
+
+## Special Considerations for OpenMetadata
+
+### Metadata-Specific Incidents
+
+Since OpenMetadata handles only metadata, not actual data:
+
+1. **Metadata Exposure**
+   - Assess what metadata was exposed
+   - Determine business impact (not data breach)
+   - Notify affected teams about architecture exposure
+
+2. **Connection Configuration**
+   - Immediately rotate any exposed credentials
+   - Audit connected system access logs
+   - Verify read-only permissions maintained
+
+3. **Lineage Information**
+   - Assess business process exposure
+   - Review competitive sensitivity
+   - Update access controls
+
+### Supply Chain Incidents
+
+1. **Dependency Vulnerabilities**
+   - Immediate assessment of exposure
+   - Patch or workaround deployment
+   - Update dependency management
+
+2. **Connector Compromises**
+   - Disable affected connectors
+   - Audit ingestion logs
+   - Verify metadata integrity
+
+## Contact Information
+
+### Security Team
+- **Email**: security@open-metadata.org
+- **GitHub Security**: https://github.com/open-metadata/OpenMetadata/security
+- **Incident Lead**: @harshach
+
+### Escalation Path
+1. Security Team
+2. Incident Lead (@harshach)
+3. OpenMetadata Maintainers
+4. Collate (parent organization) if required
+
+## Communication Templates
+
+### Initial Response
+```
+Subject: Security Report Acknowledged - [TRACKING-ID]
+
+Thank you for reporting this security issue. We take all security reports seriously.
+
+We have assigned tracking ID [TRACKING-ID] to your report and will investigate immediately.
+
+Expected timeline:
+- Initial assessment: Within 72 hours
+- Resolution target: Within 30 days
+
+We will keep you updated on our progress. Please keep this issue confidential until we coordinate disclosure.
+
+Thank you for helping keep OpenMetadata secure.
+```
+
+### Security Advisory Template
+```
+# Security Advisory: [CVE-ID]
+
+**Affected Component**: OpenMetadata [Component]
+**Severity**: [Critical/High/Medium/Low]
+**CVSS Score**: [Score]
+**Affected Versions**: [Versions]
+**Fixed Versions**: [Versions]
+
+## Summary
+[Brief description]
+
+## Impact
+[Detailed impact assessment]
+
+## Mitigation
+[Steps to mitigate if patch cannot be immediately applied]
+
+## Fix
+[How to upgrade/patch]
+
+## Credit
+Discovered by [Reporter Name/Organization]
+
+## References
+- [CVE Link]
+- [Additional References]
+```
+
+## Security Disclosure Policy
+
+- We request 90 days to address reported vulnerabilities
+- We coordinate disclosure timing with reporters
+- We credit reporters unless anonymity is requested
+- We maintain a security acknowledgments page
+- We follow responsible disclosure practices
+
+## Training & Preparedness
+
+### Regular Activities
+- Quarterly incident response drills
+- Annual security training for maintainers
+- Regular dependency audits
+- Penetration testing (annually)
+
+### Required Training
+- OWASP Top 10 awareness
+- Secure coding practices
+- Incident response procedures
+- Communication protocols
+
+## References
+
+- [GitHub Security Advisories](https://docs.github.com/en/code-security/security-advisories)
+- [CVE Process](https://cve.mitre.org/)
+- [FIRST PSIRT Framework](https://www.first.org/standards/frameworks/psirts/)
+- [NIST Incident Response Guide](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-61r2.pdf)
+
+---
+
+*Last Updated: 2025-01-28*
+*Version: 1.0*
+*Next Review: Quarterly*

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

Makefile

@@ -0,0 +1,292 @@
+.DEFAULT_GOAL := help
+PY_SOURCE ?= ingestion/src
+include ingestion/Makefile
+
+.PHONY: help
+help:
+	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":"}; {printf "\033[35m%-35s\033[0m %s\n", $$2, $$3}'
+
+.PHONY: prerequisites
+prerequisites:
+	./scripts/check_prerequisites.sh
+
+.PHONY: install_e2e_tests
+install_e2e_tests:  ## Install the ingestion module with e2e test dependencies (playwright)
+	python -m pip install "ingestion[e2e_test]/"
+	playwright install chromium --with-deps
+
+.PHONY: run_e2e_tests
+run_e2e_tests: ## Run e2e tests
+	pytest --screenshot=only-on-failure --output="ingestion/tests/e2e/artifacts" $(ARGS) --slowmo 5 --junitxml=ingestion/junit/test-results-e2e.xml ingestion/tests/e2e
+
+## Yarn
+.PHONY: yarn_install_cache
+yarn_install_cache:  ## Use Yarn to install UI dependencies
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile
+
+.PHONY: yarn_start_dev_ui
+yarn_start_dev_ui:  ## Run the UI locally with Yarn
+	cd openmetadata-ui/src/main/resources/ui && yarn start
+
+.PHONY: yarn_start_e2e
+yarn_start_e2e:  ## Run the e2e tests locally with Yarn
+	cd openmetadata-ui/src/main/resources/ui && yarn playwright:run
+
+.PHONY: yarn_start_e2e_ui
+yarn_start_e2e_ui:  ## Run the e2e tests locally in UI mode with Yarn
+	cd openmetadata-ui/src/main/resources/ui && yarn playwright:open
+
+.PHONY: yarn_start_e2e_codegen
+yarn_start_e2e_codegen:  ## generate playwright code
+	cd openmetadata-ui/src/main/resources/ui && yarn playwright:codegen
+
+.PHONY: py_antlr
+py_antlr:  ## Generate the Python code for parsing FQNs
+	antlr4 -Dlanguage=Python3 -o ingestion/src/metadata/generated/antlr ${PWD}/openmetadata-spec/src/main/antlr4/org/openmetadata/schema/*.g4
+
+.PHONY: js_antlr
+js_antlr:  ## Generate the Python code for parsing FQNs
+	antlr4 -Dlanguage=JavaScript -o openmetadata-ui/src/main/resources/ui/src/generated/antlr ${PWD}/openmetadata-spec/src/main/antlr4/org/openmetadata/schema/*.g4
+
+## Ingestion models generation
+.PHONY: generate
+generate:  ## Generate the pydantic models from the JSON Schemas to the ingestion module
+	@echo "Running Datamodel Code Generator"
+	@echo "Make sure to first run the install_dev recipe"
+	rm -rf ingestion/src/metadata/generated
+	mkdir -p ingestion/src/metadata/generated
+	python scripts/datamodel_generation.py
+	$(MAKE) py_antlr js_antlr
+	$(MAKE) install
+
+.PHONY: install_antlr_cli
+install_antlr_cli:  ## Install antlr CLI locally
+	echo '#!/usr/bin/java -jar' > /usr/local/bin/antlr4
+	curl https://www.antlr.org/download/antlr-4.9.2-complete.jar >> /usr/local/bin/antlr4
+	chmod 755 /usr/local/bin/antlr4
+
+## SNYK
+SNYK_ARGS := --severity-threshold=high
+
+# Drop pip's build/lib tree before scanning so `snyk code test` does not
+# double-report findings (once under src/, once under build/lib/). Same
+# applies to snyk-airflow-apis-report below.
+.PHONY: snyk-ingestion-report
+snyk-ingestion-report:  ## Uses Snyk CLI to validate the ingestion code and container. Don't stop the execution
+	@echo "Validating Ingestion container..."
+	docker build -t openmetadata-ingestion:scan -f ingestion/Dockerfile.ci .
+	snyk container test openmetadata-ingestion:scan --file=ingestion/Dockerfile.ci $(SNYK_ARGS) --json > security-report/ingestion-docker-scan.json | true;
+	@echo "Validating ALL ingestion dependencies. Make sure the venv is activated."
+	cd ingestion; \
+		pip freeze > scan-requirements.txt; \
+		rm -rf build; \
+		snyk test --file=scan-requirements.txt --package-manager=pip --command=python3 $(SNYK_ARGS) --json > ../security-report/ingestion-dep-scan.json | true; \
+		snyk code test $(SNYK_ARGS) --json > ../security-report/ingestion-code-scan.json | true;
+
+.PHONY: snyk-airflow-apis-report
+snyk-airflow-apis-report:  ## Uses Snyk CLI to validate the airflow apis code. Don't stop the execution
+	@echo "Validating airflow dependencies. Make sure the venv is activated."
+	cd openmetadata-airflow-apis; \
+		rm -rf build; \
+		snyk code test $(SNYK_ARGS) --json > ../security-report/airflow-apis-code-scan.json | true;
+
+.PHONY: snyk-catalog-report
+snyk-server-report:  ## Uses Snyk CLI to validate the catalog code and container. Don't stop the execution
+	@echo "Validating catalog container... Make sure the code is built and available under openmetadata-dist"
+	docker build -t openmetadata-server:scan -f docker/development/Dockerfile .
+	snyk container test openmetadata-server:scan --file=docker/development/Dockerfile $(SNYK_ARGS) --json > security-report/server-docker-scan.json | true;
+	snyk test --all-projects $(SNYK_ARGS) --json > security-report/server-dep-scan.json | true;
+	snyk code test --all-projects --severity-threshold=high --json > security-report/server-code-scan.json | true;
+
+.PHONY: snyk-ui-report
+snyk-ui-report:  ## Uses Snyk CLI to validate the UI dependencies. Don't stop the execution
+	snyk test --file=openmetadata-ui/src/main/resources/ui/yarn.lock $(SNYK_ARGS) --json > security-report/ui-dep-scan.json | true;
+
+.PHONY: snyk-dependencies-report
+snyk-dependencies-report:  ## Uses Snyk CLI to validate the project dependencies: MySQL, Postgres and ES. Only local testing.
+	@echo "Validating dependencies images..."
+	snyk container test mysql/mysql-server:latest $(SNYK_ARGS) --json > security-report/mysql-scan.json | true;
+	snyk container test postgres:latest $(SNYK_ARGS) --json > security-report/postgres-scan.json | true;
+	snyk container test docker.elastic.co/elasticsearch/elasticsearch:7.10.2 $(SNYK_ARGS) --json > security-report/es-scan.json | true;
+
+.PHONY: snyk-ingestion-base-slim-report
+snyk-ingestion-base-slim-report:
+	@echo "Validating Ingestion Slim Container"
+	docker build -t openmetadata-ingestion-base-slim:scan -f ingestion/operators/docker/Dockerfile.ci --build-arg INGESTION_DEPENDENCY=slim .
+	snyk container test openmetadata-ingestion-base-slim:scan --file=ingestion/operators/docker/Dockerfile.ci $(SNYK_ARGS) --json > security-report/ingestion-docker-base-slim-scan.json | true;
+
+.PHONY: snyk-report
+snyk-report:  ## Uses Snyk CLI to run a security scan of the different pieces of the code
+	@echo "To run this locally, make sure to install and authenticate using the Snyk CLI: https://docs.snyk.io/snyk-cli/install-the-snyk-cli"
+	rm -rf security-report
+	mkdir -p security-report
+	$(MAKE) snyk-ingestion-report
+	$(MAKE) snyk-ingestion-base-slim-report
+	$(MAKE) snyk-airflow-apis-report
+	$(MAKE) snyk-server-report
+	$(MAKE) snyk-ui-report
+	$(MAKE)	export-snyk-pdf-report
+
+.PHONY: export-snyk-pdf-report
+export-snyk-pdf-report:  ## export json file from security-report/ to HTML
+	@echo "Reading all results"
+	npm install snyk-to-html -g
+	ls security-report | xargs -I % snyk-to-html -i security-report/% -o security-report/%.html
+	pip install pdfkit
+	pip install PyPDF2
+	python scripts/html_to_pdf.py
+
+# Ingestion Operators
+.PHONY: build-ingestion-base-local
+build-ingestion-base-local:  ## Builds the ingestion DEV docker operator with the local ingestion files
+	$(MAKE) install_dev generate
+	docker build -f ingestion/operators/docker/Dockerfile.ci . -t openmetadata/ingestion-base:local
+
+.PHONY: build-ingestion-base-slim-local
+build-ingestion-base-local:  ## Builds the ingestion DEV docker operator with the local ingestion files
+	$(MAKE) install_dev generate
+	docker build -f ingestion/operators/docker/Dockerfile.ci . -t openmetadata/ingestion-base-slim:local --build-arg INGESTION_DEPENDENCY=slim
+
+#Upgrade release automation scripts below
+.PHONY: update_all
+update_all:  ## To update all the release related files run make update_all RELEASE_VERSION=2.2.2
+	@echo "The release version is: $(RELEASE_VERSION)" ; \
+	$(MAKE) update_maven ; \
+	$(MAKE) update_openapi_version ; \
+	$(MAKE) update_pyproject_version ; \
+	$(MAKE) update_dockerfile_version ; \
+	$(MAKE) update_dockerfile_ri_version ; \
+#remove comment and use the below section when want to use this sub module "update_all" independently to update github actions
+#make update_all RELEASE_VERSION=2.2.2
+
+.PHONY: update_maven
+update_maven:  ## To update the common and pom.xml maven version
+	@echo "Updating Maven projects to version $(RELEASE_VERSION)..."; \
+	mvn versions:set -DnewVersion=$(RELEASE_VERSION)
+#remove comment and use the below section when want to use this sub module "update_maven" independently to update github actions
+#make update_maven RELEASE_VERSION=2.2.2
+
+.PHONY: update_openapi_version
+update_openapi_version:  ## To update the OpenAPI version in OpenMetadataApplication.java
+	@echo "Updating OpenAPI version to $(RELEASE_VERSION)..."; \
+	python3 scripts/update_version.py update_openapi_version -v $(RELEASE_VERSION)
+#make update_openapi_version RELEASE_VERSION=2.2.2
+
+.PHONY: update_pyproject_version
+update_pyproject_version:  ## To update the pyproject.toml files
+	file_paths="ingestion/pyproject.toml \
+				openmetadata-airflow-apis/pyproject.toml"; \
+	echo "Updating pyproject.toml versions to $(RELEASE_VERSION)... "; \
+	for file_path in $$file_paths; do \
+	    python3 scripts/update_version.py update_pyproject_version -f $$file_path -v $(RELEASE_VERSION) ; \
+	done
+# Commented section for independent usage of the module update_pyproject_version independently to update github actions
+#make update_pyproject_version RELEASE_VERSION=2.2.2
+
+.PHONY: update_dockerfile_version
+update_dockerfile_version:  ## To update the dockerfiles version
+	@file_paths="docker/docker-compose-ingestion/docker-compose-ingestion.yml \
+		     docker/docker-compose-openmetadata/docker-compose-openmetadata.yml \
+		     docker/docker-compose-quickstart/docker-compose-postgres.yml \
+		     docker/docker-compose-quickstart/docker-compose.yml"; \
+	echo "Updating docker github action release version to $(RELEASE_VERSION)... "; \
+	for file_path in $$file_paths; do \
+	    python3 scripts/update_version.py update_docker_tag -f $$file_path -t $(RELEASE_VERSION) ; \
+	done
+#remove comment and use the below section when want to use this sub module "update_dockerfile_version" independently to update github actions
+#make update_dockerfile_version RELEASE_VERSION=2.2.2
+
+.PHONY: update_dockerfile_ri_version
+update_dockerfile_ri_version:  ## To update the dockerfile RI_VERSION argument
+	@file_paths="ingestion/Dockerfile \
+	             ingestion/operators/docker/Dockerfile"; \
+	echo "Updating ingestion dockerfile release version to $(PY_RELEASE_VERSION)... "; \
+	for file_path in $$file_paths; do \
+	    python3 scripts/update_version.py update_ri_version -f $$file_path -v $(RELEASE_VERSION) --with-python-version ; \
+	done
+	python3 scripts/update_version.py update_ri_version -f docker/docker-compose-quickstart/Dockerfile -v $(RELEASE_VERSION)
+#remove comment and use the below section when want to use this sub module "update_dockerfile_ri_version" independently to update github actions
+#make update_dockerfile_ri_version RELEASE_VERSION=2.2.2
+
+#Upgrade release automation scripts above
+
+.PHONY: update_typescript_types
+update_typescript_types:
+	@echo "Generating JSON to TS files"
+	./openmetadata-ui/src/main/resources/ui/json2ts-generate-all.sh -l true
+	@echo "Generating antlr typescript files"
+	$(MAKE) js_antlr
+
+# Fix license header in all UI files.
+.PHONY: license-header-fix
+license-header-fix:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn license-header-fix
+
+# Run TypeScript type-checking for src files (does not auto-fix errors).
+.PHONY: tsc-src-fix
+tsc-src-fix:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn tsc:check
+
+# Run TypeScript type-checking for Playwright files (does not auto-fix errors).
+.PHONY: tsc-playwright-fix
+tsc-playwright-fix:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn tsc:playwright
+
+# Sync all i18n files to have the same keys and order. This doesn't modify the translation.
+# Just makes sure all files have the same keys in the same order to avoid conflicts and make it easier to maintain.
+.PHONY: i18n-sync-fix
+i18n-sync-fix:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn i18n
+
+# Generate the docs markdown file for all applications.
+.PHONY: generate-app-docs
+generate-app-docs:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn generate:app-docs
+
+# Fix all linting and formatting errors in src folder.
+.PHONY: ui-checkstyle-src
+ui-checkstyle-src:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn ui-checkstyle
+
+# Fix all linting and formatting errors in playwright tests.
+.PHONY: ui-checkstyle-playwright
+ui-checkstyle-playwright:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn ui-checkstyle:playwright
+
+# Fix all linting and formatting errors in core components.
+.PHONY: ui-checkstyle-core-components
+ui-checkstyle-core-components:
+	cd openmetadata-ui-core-components/src/main/resources/ui && yarn install --frozen-lockfile && yarn lint:fix && yarn pretty
+
+# Fix linting and formatting errors in changed files in src folder
+# Changed files are detected based on the current branch against main branch.
+# So make sure to run this after rebasing to main to get the correct list of changed files.
+.PHONY: ui-checkstyle-src-changed
+ui-checkstyle-src-changed:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn ui-checkstyle:changed
+
+# Fix linting and formatting errors in changed playwright test files
+# Changed files are detected based on the current branch against main branch.
+# So make sure to run this after rebasing to main to get the correct list of changed files.
+.PHONY: ui-checkstyle-playwright-changed
+ui-checkstyle-playwright-changed:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn ui-checkstyle:playwright:changed
+
+# Fix linting and formatting errors in changed core components files
+# Changed files are detected based on the current branch against main branch.
+# So make sure to run this after rebasing to main to get the correct list of changed files.
+.PHONY: ui-checkstyle-core-components-changed
+ui-checkstyle-core-components-changed:
+	cd openmetadata-ui-core-components/src/main/resources/ui && yarn install --frozen-lockfile && yarn ui-checkstyle:changed
+
+# Run all full UI checkstyle operations with one command.
+.PHONY: ui-checkstyle-all
+ui-checkstyle-all:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn ui-checkstyle && yarn ui-checkstyle:playwright
+	cd openmetadata-ui-core-components/src/main/resources/ui && yarn install --frozen-lockfile && yarn lint:fix && yarn pretty
+
+# Run all changed-file UI checkstyle operations with one command.
+.PHONY: ui-checkstyle-changed
+ui-checkstyle-changed:
+	cd openmetadata-ui/src/main/resources/ui && yarn install --frozen-lockfile && yarn ui-checkstyle:changed && yarn ui-checkstyle:playwright:changed
+	cd openmetadata-ui-core-components/src/main/resources/ui && yarn install --frozen-lockfile && yarn ui-checkstyle:changed

NOTICE

@@ -0,0 +1,2 @@
+OpenMetadata
+Copyright 2021