wehub-resource-sync
bf2343b7e4
Integration Tests - MySQL + Elasticsearch / Detect Changes (push) Has been cancelled
Integration Tests - MySQL + Elasticsearch / integration-tests-mysql-elasticsearch (push) Has been cancelled
Integration Tests - PostgreSQL + Elasticsearch + Redis / Detect Changes (push) Has been cancelled
Integration Tests - PostgreSQL + Elasticsearch + Redis / integration-tests-postgres-elasticsearch-redis (push) Has been cancelled
Integration Tests - PostgreSQL + OpenSearch / Detect Changes (push) Has been cancelled
Integration Tests - PostgreSQL + OpenSearch / integration-tests-postgres-opensearch (push) Has been cancelled
Java Checkstyle / java-checkstyle (push) Has been cancelled
Maven Collate Tests / maven-collate-ci (push) Has been cancelled
OpenMetadata Service Unit Tests / openmetadata-service-unit-tests-status (push) Has been cancelled
Publish Package to Maven Central Repository / publish-maven-packages (push) Has been cancelled
OpenMetadata Service Unit Tests / Detect Changes (push) Has been cancelled
OpenMetadata Service Unit Tests / openmetadata-service-unit-tests (push) Has been cancelled
OpenMetadata Service Unit Tests / k8s_operator-unit-tests (push) Has been cancelled
31 lines
1.2 KiB
Markdown
31 lines
1.2 KiB
Markdown
---
|
|
This guide will help you setup the Ingestion framework and connectors
|
|
---
|
|
|
|
|
|
|
|
OpenMetadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata.
|
|
**Prerequisites**
|
|
|
|
- Python >= 3.9.x
|
|
|
|
### Docs
|
|
|
|
Please refer to the documentation here https://docs.open-metadata.org/connectors
|
|
|
|
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=c1a30c7c-6dc7-4928-95bf-6ee08ca6aa6a" />
|
|
|
|
### TopologyRunner
|
|
|
|
All the Ingestion Workflows run through the TopologyRunner.
|
|
|
|
The flow is depicted in the images below.
|
|
|
|
**TopologyRunner Standard Flow**
|
|
|
|
|
|
|
|
**TopologyRunner Multithread Flow**
|
|
|
|
|