open-metadata--openmetadata/openmetadata-integration-tests/TEST_MIGRATION_TRACKER.md

Test Migration Tracker: openmetadata-service → openmetadata-integration-tests

Last Updated: 2025-12-26 Status: All Tests Passing - Migration Complete Branch: faster_tests_2

Recent Progress (2025-12-26)

MySQL Deadlock Fix ✅

• Fixed MySQL deadlock issue in TagUsageDAO.applyTagsBatchInternal
• Changed from INSERT IGNORE to INSERT ... ON DUPLICATE KEY UPDATE
• PostgreSQL unchanged (ON CONFLICT DO NOTHING - no deadlock issue)

Flaky Test Fixes ✅

• AppsResourceIT: Changed to SAME_THREAD execution mode
  • Multiple tests trigger SearchIndexingApplication which is a shared resource
  • Added waitForAppJobCompletion() to tests that trigger apps
• TableResourceIT.test_multipleDomainInheritance: Used Awaitility for search index wait
  • Replaced Thread.sleep(2000) with proper Awaitility polling
  • Waits up to 30 seconds for table to appear in search index

Maven Profiles Added ✅

• mysql-elasticsearch (default)
• postgres-opensearch
• postgres-elasticsearch
• mysql-opensearch

GitHub Workflows Created ✅

• integration-tests-mysql-elasticsearch.yml
• integration-tests-postgres-opensearch.yml

Code Cleanup ✅

• Removed wildcard imports from 85+ files
• Added comprehensive README.md documentation

---

Executive Summary

Metric	Count
Total Test Classes	91
Total @Test Methods	~2,100
Base Class Tests (BaseEntityIT)	119

---

Migration Status By Entity Class

Legend

• ✅ Complete: All tests migrated and passing
• 🔄 Partial: Some tests need attention
• ⚠️ Stub: Only BaseEntityIT inherited tests

Entity Resource Tests (Sorted by Test Count)

Target Class	Tests	Status
BaseEntityIT	119	✅
TestCaseResourceIT	95	✅
TableResourceIT	92	✅
DataContractResourceIT	92	✅
GlossaryTermResourceIT	71	✅
UserResourceIT	66	✅
SearchResourceIT	55	✅
NotificationTemplateResourceIT	52	✅
EventSubscriptionResourceIT	48	✅
FeedResourceIT	47	✅
TopicResourceIT	38	✅
MlModelResourceIT	38	✅
PipelineResourceIT	36	✅
GlossaryResourceIT	35	✅
ContainerResourceIT	34	✅
SystemResourceIT	33	✅
TeamResourceIT	32	✅
DashboardResourceIT	29	✅
SpreadsheetResourceIT	27	✅
IngestionPipelineResourceIT	27	✅
ColumnResourceIT	27	✅
AppsResourceIT	27	✅
TestSuiteResourceIT	26	✅
LineageResourceIT	25	✅
ChartResourceIT	25	✅
WorkflowDefinitionResourceIT	24	✅
SuggestionsResourceIT	22	✅
QueryResourceIT	21	✅
OpenLineageResourceIT	21	✅
MetricResourceIT	20	✅
WorksheetResourceIT	19	✅
TagResourceIT	19	✅
PromptTemplateResourceIT	19	✅
TypeResourceIT	17	✅
SearchIndexResourceIT	17	✅
AIApplicationResourceIT	17	✅
UsageResourceIT	16	✅
LLMModelResourceIT	16	✅
DatabaseSchemaResourceIT	16	✅
WorkflowResourceIT	15	✅
DirectoryResourceIT	15	✅
StoredProcedureResourceIT	14	✅
PermissionsResourceIT	14	✅
FileResourceIT	14	✅
EntityProfileResourceIT	14	✅
DatabaseResourceIT	14	✅
PolicyResourceIT	13	✅
DomainResourceIT	13	✅
DashboardDataModelResourceIT	13	✅
AIGovernancePolicyResourceIT	13	✅
DatabaseServiceResourceIT	12	✅
ClassificationResourceIT	12	✅
AppMarketPlaceResourceIT	12	✅
SecurityServiceResourceIT	11	✅
RoleResourceIT	11	✅
PersonaResourceIT	11	✅
MetadataServiceResourceIT	11	✅
LLMServiceResourceIT	11	✅
KpiResourceIT	11	✅
DriveServiceResourceIT	11	✅
DocStoreResourceIT	11	✅
DataProductResourceIT	11	✅
DataInsightChartResourceIT	11	✅
DataContractPermissionIT	11	✅
ConfigResourceIT	11	✅
ChangeEventParserResourceIT	10	✅
APIEndpointResourceIT	10	✅
K8sIngestionPipelineResourceIT	9	✅
IngestionPipelineLogStreamingResourceIT	9	✅
APICollectionResourceIT	9	✅
UserMetricsResourceIT	8	✅
TestConnectionDefinitionResourceIT	8	✅
SearchServiceResourceIT	8	✅
DashboardServiceResourceIT	8	✅
AlertsRuleEvaluatorResourceIT	8	✅
WebAnalyticEventResourceIT	7	✅
MessagingServiceResourceIT	7	✅
APIServiceResourceIT	7	✅
StorageServiceResourceIT	6	✅
ReportDataResourceIT	6	✅
PipelineServiceResourceIT	6	✅
MlModelServiceResourceIT	6	✅
TestDefinitionResourceIT	5	✅
BotResourceIT	5	✅
AgentExecutionResourceIT	5	✅
RdfResourceIT	4	✅
PrometheusResourceIT	3	✅
PaginationIT	1	✅
DatabaseSmokeIT	1	✅
DatabaseHierarchyIT	1	✅
BaseServiceIT	1	✅

---

Test Infrastructure

Available Profiles

Profile	Database	Search Engine
mysql-elasticsearch (default)	MySQL 8.3.0	Elasticsearch 8.11.4
postgres-opensearch	PostgreSQL 15	OpenSearch 2.19.0
postgres-elasticsearch	PostgreSQL 15	Elasticsearch 8.11.4
mysql-opensearch	MySQL 8.3.0	OpenSearch 2.19.0

Running Tests

# Run all tests with MySQL + Elasticsearch (default)
mvn test -pl :openmetadata-integration-tests

# Run with PostgreSQL + OpenSearch
mvn test -pl :openmetadata-integration-tests -Ppostgres-opensearch

# Run a specific test
mvn test -pl :openmetadata-integration-tests -Dtest="TableResourceIT"

---

SDK Fluent APIs Added

The following fluent API classes were added to support the integration tests:

SDK Class	Location
Columns	openmetadata-sdk/.../fluent/Columns.java
DataContracts	openmetadata-sdk/.../fluent/DataContracts.java
TestCases	openmetadata-sdk/.../fluent/TestCases.java
Usage	openmetadata-sdk/.../fluent/Usage.java

SDK Services Added

Service	Purpose
TestCaseResolutionStatusService	Test case resolution status operations
TestCaseResultService	Test case result operations
DataContractService	Data contract CRUD + bulk operations

---

Key Files

• README.md - Comprehensive documentation on writing tests
• BaseEntityIT.java - Base class with 119 inherited tests
• TestSuiteBootstrap.java - Test infrastructure (Testcontainers)
• TestNamespace.java - Test isolation utility
• SdkClients.java - Pre-configured SDK clients

---

CI/CD Integration

GitHub workflows run on every PR:

• integration-tests-mysql-elasticsearch.yml - MySQL + Elasticsearch
• integration-tests-postgres-opensearch.yml - PostgreSQL + OpenSearch

Tests require the "safe to test" label on PRs (uses pull_request_target).

---

Performance

Metric	Value
Full test suite	~20 minutes locally
Parallel execution	Yes (JUnit 5 parallel)
Test isolation	TestNamespace prefixes
Container startup	~30 seconds

---

Recent Fixes

MySQL Deadlock Fix (2025-12-26)

Changed TagUsageDAO.applyTagsBatchInternal from:

INSERT IGNORE INTO tag_usage ...

To:

INSERT INTO tag_usage ... ON DUPLICATE KEY UPDATE ...

This prevents deadlocks from MySQL's gap locking behavior during concurrent tag updates.