chore: import upstream snapshot with attribution
Integration Tests - MySQL + Elasticsearch / Detect Changes (push) Has been cancelled
Integration Tests - MySQL + Elasticsearch / integration-tests-mysql-elasticsearch (push) Has been cancelled
Integration Tests - PostgreSQL + Elasticsearch + Redis / Detect Changes (push) Has been cancelled
Integration Tests - PostgreSQL + Elasticsearch + Redis / integration-tests-postgres-elasticsearch-redis (push) Has been cancelled
Integration Tests - PostgreSQL + OpenSearch / Detect Changes (push) Has been cancelled
Integration Tests - PostgreSQL + OpenSearch / integration-tests-postgres-opensearch (push) Has been cancelled
Java Checkstyle / java-checkstyle (push) Has been cancelled
Maven Collate Tests / maven-collate-ci (push) Has been cancelled
OpenMetadata Service Unit Tests / openmetadata-service-unit-tests-status (push) Has been cancelled
Publish Package to Maven Central Repository / publish-maven-packages (push) Has been cancelled
OpenMetadata Service Unit Tests / Detect Changes (push) Has been cancelled
OpenMetadata Service Unit Tests / openmetadata-service-unit-tests (push) Has been cancelled
OpenMetadata Service Unit Tests / k8s_operator-unit-tests (push) Has been cancelled
Integration Tests - MySQL + Elasticsearch / Detect Changes (push) Has been cancelled
Integration Tests - MySQL + Elasticsearch / integration-tests-mysql-elasticsearch (push) Has been cancelled
Integration Tests - PostgreSQL + Elasticsearch + Redis / Detect Changes (push) Has been cancelled
Integration Tests - PostgreSQL + Elasticsearch + Redis / integration-tests-postgres-elasticsearch-redis (push) Has been cancelled
Integration Tests - PostgreSQL + OpenSearch / Detect Changes (push) Has been cancelled
Integration Tests - PostgreSQL + OpenSearch / integration-tests-postgres-opensearch (push) Has been cancelled
Java Checkstyle / java-checkstyle (push) Has been cancelled
Maven Collate Tests / maven-collate-ci (push) Has been cancelled
OpenMetadata Service Unit Tests / openmetadata-service-unit-tests-status (push) Has been cancelled
Publish Package to Maven Central Repository / publish-maven-packages (push) Has been cancelled
OpenMetadata Service Unit Tests / Detect Changes (push) Has been cancelled
OpenMetadata Service Unit Tests / openmetadata-service-unit-tests (push) Has been cancelled
OpenMetadata Service Unit Tests / k8s_operator-unit-tests (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,306 @@
|
||||
## Context
|
||||
|
||||
The incident manager today is a switch statement in `TestCaseResolutionStatusRepository.storeInternal()` with no extension points. This design moves incident **Thread/task lifecycle** into the governance workflows framework using an event-driven, signal-based architecture where every TCRS event starts a short-lived process that reads state and acts.
|
||||
|
||||
Key files in the current system:
|
||||
|
||||
| Component | File | Role |
|
||||
|---|---|---|
|
||||
| State machine | `TestCaseResolutionStatusRepository.storeInternal()` | Switch on New/Ack/Assigned/Resolved |
|
||||
| Incident creation | `TestCaseResolutionStatusRepository.getOrCreateIncident()` | Creates `New` record, calls `storeInternal()` |
|
||||
| Severity inference | `TestCaseResolutionStatusRepository.inferIncidentSeverity()` | Called inside `storeInternal()` before insert |
|
||||
| Task creation | `TestCaseResolutionStatusRepository.openOrAssignTask()` | Creates Thread on Ack, patches assignee on Assigned |
|
||||
| Resolution | `TestCaseResolutionStatusRepository.resolveTask()` | Closes Thread task via `closeTaskWithoutWorkflow()` |
|
||||
| incidentId linking | `TestCaseResultRepository.setTestCaseResultIncidentId()` | Calls `getOrCreateIncident()` synchronously |
|
||||
| Workflow engine | `WorkflowHandler.java` | Flowable ProcessEngine singleton |
|
||||
| Node registration | `NodeFactory.java` | Switch on `NodeSubType` |
|
||||
| Event routing | `WorkflowEventConsumer.sendMessage()` | Routes ChangeEvents to Flowable signals |
|
||||
| Event dispatcher | `EntityLifecycleEventDispatcher` | Synchronous observer-based event system |
|
||||
| Trigger filter | `FilterEntityImpl.java` | Evaluates JSON Logic against **full entity** (fetched with Include.ALL) |
|
||||
|
||||
---
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Move incident Thread/task lifecycle into a governance workflow
|
||||
- Enable auto-assign on incident creation (configurable, default off)
|
||||
- Ship a default branching workflow that handles open/close
|
||||
- Introduce reusable `openTask` and `closeTask` node types
|
||||
- Extend event pipeline to broadcast TCRS events to workflows
|
||||
- Handle re-open from Resolved to any non-Resolved status
|
||||
|
||||
**Non-Goals:**
|
||||
- Auto-close on test pass (Slice 2)
|
||||
- TTL / stale incident expiration (Slice 3)
|
||||
- Timer subprocess or signal-based timer interruption (Slice 3)
|
||||
- Cleanup timer for orphaned processes (follow-up work)
|
||||
- Changing the Ack/Assigned assignee-patching logic
|
||||
- Custom lifecycle states
|
||||
- UI changes
|
||||
|
||||
---
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1: Event-driven architecture — storeInternal broadcasts, workflow reacts
|
||||
|
||||
**Decision:** `storeInternal()` is decoupled from Flowable. After persisting a TCRS record, it broadcasts an event via `EntityLifecycleEventDispatcher`. A registered handler in `WorkflowHandler` receives the event and broadcasts a Flowable signal. The workflow starts from the signal and handles lifecycle actions.
|
||||
|
||||
**Event flow:**
|
||||
```
|
||||
storeInternal() persists TCRS record
|
||||
→ EntityLifecycleEventDispatcher.postCreate(tcrsRecord)
|
||||
→ WorkflowHandler.onTcrsEvent(tcrsRecord)
|
||||
→ runtimeService.signalEventReceived("tcrs_{testCaseFQN}", variables)
|
||||
→ Flowable starts new process instance(s) in matching workflow(s)
|
||||
```
|
||||
|
||||
**Why EntityLifecycleEventDispatcher (not ChangeEvents):**
|
||||
- TCRS is a time-series entity that does NOT emit ChangeEvents
|
||||
- EntityLifecycleEventDispatcher is synchronous (same thread) — no async delay
|
||||
- The API call returns only after the workflow process completes (short-lived processes reach end event before the signal call returns)
|
||||
- Already used for search indexing; natural extension point
|
||||
|
||||
**Signal variables carried:**
|
||||
- `status`: the TCRS status (New, Ack, Assigned, Resolved)
|
||||
- `testCaseFQN`: the test case fully qualified name
|
||||
- `stateId`: the incident state ID
|
||||
- `entityLink`: the entity link from the TCRS record
|
||||
|
||||
**What stays in `storeInternal()`:**
|
||||
- Record persistence (unchanged)
|
||||
- Severity inference (unchanged, see D6)
|
||||
- `Ack` case: assignee patching on existing Thread (see D5)
|
||||
- `Assigned` case: assignee patching (unchanged)
|
||||
- Event broadcast (new)
|
||||
|
||||
**What moves to the workflow:**
|
||||
- Thread/task creation (currently in `openOrAssignTask()` Ack case)
|
||||
- Thread/task closure (currently in `resolveTask()`)
|
||||
|
||||
### D2: Generic nodes — openTask and closeTask (replaces HumanInterventionTask)
|
||||
|
||||
**Decision:** Instead of a monolithic `HumanInterventionTask`, build two generic, composable nodes following the three-layer pattern:
|
||||
|
||||
**openTask:**
|
||||
|
||||
| Layer | Class | Responsibility | Flowable dependency |
|
||||
|-------|-------|----------------|---------------------|
|
||||
| **Task** | `OpenTask` | Builds ServiceTask BPMN element. References delegate class name. | Yes (BPMN model only) |
|
||||
| **Delegate** | `OpenTaskDelegate implements JavaDelegate` | Receives config via `Expression` fields. Sets `taskCreated` process variable. | Yes (thin adapter) |
|
||||
| **Impl** | `OpenTaskImpl` | Idempotent Thread/task creation. No Flowable imports. | **None** |
|
||||
|
||||
**closeTask:**
|
||||
|
||||
| Layer | Class | Responsibility | Flowable dependency |
|
||||
|-------|-------|----------------|---------------------|
|
||||
| **Task** | `CloseTask` | Builds ServiceTask BPMN element. References delegate class name. | Yes (BPMN model only) |
|
||||
| **Delegate** | `CloseTaskDelegate implements JavaDelegate` | Receives config via `Expression` fields. Sets `taskClosed` process variable. | Yes (thin adapter) |
|
||||
| **Impl** | `CloseTaskImpl` | Thread/task closure. No Flowable imports. | **None** |
|
||||
|
||||
Reference: `CreateAndRunIngestionPipelineTask` → `CreateIngestionPipelineDelegate` → `CreateIngestionPipelineImpl`
|
||||
|
||||
**openTask config:**
|
||||
```json
|
||||
{
|
||||
"template": "incident",
|
||||
"taskType": "RequestTestCaseFailureResolution",
|
||||
"responsibles": { "source": "tableOwner" }
|
||||
}
|
||||
```
|
||||
|
||||
- `template`: identifies the task template (for future extensibility)
|
||||
- `taskType`: the `TaskType` enum value for Thread creation
|
||||
- `responsibles` (optional): auto-assign config. Omitted = unassigned (default)
|
||||
- `{ "source": "tableOwner" }` → resolve table entity owner
|
||||
- `{ "source": "specificUser", "target": "user.fqn" }` → specific user
|
||||
|
||||
**closeTask config:**
|
||||
```json
|
||||
{
|
||||
"template": "incident",
|
||||
"taskType": "RequestTestCaseFailureResolution"
|
||||
}
|
||||
```
|
||||
|
||||
**Idempotency:**
|
||||
- `openTask`: queries for existing open Thread/task for the entity. If found → no-op, sets `taskCreated=false`. If not found → creates Thread, sets `taskCreated=true`.
|
||||
- `closeTask`: queries for open Thread/task. If found → closes it, sets `taskClosed=true`. If not found → no-op, sets `taskClosed=false`.
|
||||
|
||||
**Runs as `governance-bot`** — the `WorkflowEventConsumer` already skips events from governance-bot, preventing infinite loops.
|
||||
|
||||
### D3: Signal architecture — broadcast for fan-out, no queries
|
||||
|
||||
**Decision:** Use Flowable signals (broadcast) for workflow triggering. Signals are the right primitive because:
|
||||
1. Multiple workflows can react to the same TCRS event (fan-out)
|
||||
2. No need to query Flowable for existing processes (signals are fire-and-forget)
|
||||
3. Each workflow's signal start event independently creates a new process instance
|
||||
|
||||
**Slice 1 signal:**
|
||||
- `tcrs_{testCaseFQN}` — broadcast on every TCRS event. Starts new processes.
|
||||
|
||||
**Slice 3 signal (deferred):**
|
||||
- `tcrs_closed_{testCaseFQN}` — broadcast by `closeTask`. Caught by signal boundary events on timer subprocesses to interrupt/cancel timers.
|
||||
|
||||
**Why signals, not messages:**
|
||||
- Messages are unicast (delivered to one specific execution). Would require knowing which process to target → Flowable queries.
|
||||
- Signals are broadcast (all listeners receive). No routing logic needed.
|
||||
- The "double fire" concern (signal catches both start events and intermediate catches) is handled by using different signal names for different purposes.
|
||||
|
||||
**Flowable does NOT enforce business key uniqueness.** Multiple process instances can share the same business key. This is fine — short-lived processes complete quickly and don't accumulate.
|
||||
|
||||
### D4: Short-lived processes — every event is an episode
|
||||
|
||||
**Decision:** Every TCRS event starts a new short-lived process instance. The process reads state, acts, and ends. No long-lived processes in Slice 1.
|
||||
|
||||
**BPMN structure for default workflow:**
|
||||
```
|
||||
[SignalStartEvent: "tcrs_{testCaseFQN}"]
|
||||
→ [ExclusiveGateway: statusGateway]
|
||||
condition: ${status != "Resolved"}
|
||||
→ [ServiceTask: openTask] → [EndEvent]
|
||||
condition: ${status == "Resolved"}
|
||||
→ [ServiceTask: closeTask] → [EndEvent]
|
||||
default:
|
||||
→ [EndEvent: skipEnd]
|
||||
```
|
||||
|
||||
**Process lifecycle per event:**
|
||||
|
||||
| TCRS Event | Gateway route | Node action | Process duration |
|
||||
|---|---|---|---|
|
||||
| New (first failure) | NOT Resolved | openTask creates Thread | Milliseconds |
|
||||
| Ack | NOT Resolved | openTask no-op (Thread exists) | Milliseconds |
|
||||
| Assigned | NOT Resolved | openTask no-op (Thread exists) | Milliseconds |
|
||||
| Resolved | Resolved | closeTask closes Thread | Milliseconds |
|
||||
| Resolved→New (re-open) | NOT Resolved | openTask creates new Thread | Milliseconds |
|
||||
| Resolved→Ack (re-open) | NOT Resolved | openTask creates new Thread | Milliseconds |
|
||||
|
||||
**Why short-lived:**
|
||||
- No Flowable state accumulation (no `ACT_RU_*` rows between events)
|
||||
- No need to correlate messages to running processes
|
||||
- No orphan cleanup needed
|
||||
- Idempotent nodes make repeated execution safe
|
||||
|
||||
**Slice 3 evolution:** The openTask branch gains a timer subprocess:
|
||||
```
|
||||
→ [ServiceTask: openTask]
|
||||
→ [SubProcess: timerChain]
|
||||
[Timer: 24h] → [Notify] → [Timer: 7d] → [AutoClose] → [End]
|
||||
SignalBoundaryEvent (interrupting): "tcrs_closed_{fqn}"
|
||||
→ [EndEvent]
|
||||
```
|
||||
The timer subprocess makes the New/Ack branch long-lived. When Resolved arrives, a new short-lived process runs closeTask AND broadcasts `tcrs_closed_{fqn}`, which interrupts the timer subprocess via its signal boundary event. All timers are cancelled, the old process ends cleanly.
|
||||
|
||||
**Only Resolved kills timers.** Ack/Assigned events don't interrupt the timer subprocess — reminders continue counting from the original failure time. This is a deliberate product choice: "remind the (now assigned) person" is still useful.
|
||||
|
||||
### D5: Ack backward compatibility
|
||||
|
||||
**Decision:** Modify `openOrAssignTask()` for the `Ack` case to check if a Thread/task already exists before creating one.
|
||||
|
||||
**Why:** With the workflow creating the Thread/task on the `New` event (immediately on test failure), by the time a user Acks, the task already exists. Today, `Ack` unconditionally calls `createTask()`, which would create a duplicate.
|
||||
|
||||
**Change:** In `openOrAssignTask()`, the `Ack` case mirrors the `Assigned` pattern:
|
||||
```java
|
||||
case Ack -> {
|
||||
Thread existingTask = getIncidentTask(incidentStatus);
|
||||
if (existingTask == null) {
|
||||
createTask(incidentStatus, Collections.singletonList(incidentStatus.getUpdatedBy()));
|
||||
} else {
|
||||
patchTaskAssignee(existingTask, incidentStatus.getUpdatedBy(),
|
||||
incidentStatus.getUpdatedBy().getName());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This is backward-compatible: if the workflow didn't run (Flowable was down), the Ack fallback creates the task.
|
||||
|
||||
### D6: Severity inference stays in repository
|
||||
|
||||
**Decision:** `inferIncidentSeverity()` is already called inside `storeInternal()` during record creation. No change needed — severity is inferred when the record is created, before the workflow fires.
|
||||
|
||||
**Confirmed by code trace:** `getOrCreateIncident()` → `createNewRecord()` → `storeInternal()` → `inferIncidentSeverity()`.
|
||||
|
||||
### D7: Bootstrap mechanism for default workflow
|
||||
|
||||
**Decision:** Add a JSON file to the existing bootstrap directory at `openmetadata-service/src/main/resources/json/data/governance/workflows/`.
|
||||
|
||||
The server's `WorkflowDefinitionResource.initialize()` calls `initSeedDataFromResources()` on startup, which loads all JSON files matching `.*json/data/workflowDefinition/.*\.json$` and persists them via `createOrUpdate` semantics.
|
||||
|
||||
Two workflows already bootstrap this way:
|
||||
- `GlossaryApprovalWorkflow.json`
|
||||
- `RecognizerFeedbackReviewWorkflow.json`
|
||||
|
||||
We add `IncidentLifecycleWorkflow.json` following the same pattern.
|
||||
|
||||
### D8: Schema changes
|
||||
|
||||
**New files:**
|
||||
- `openmetadata-spec/.../governance/workflows/elements/nodes/automatedTask/openTask.json` — openTask config schema
|
||||
- `openmetadata-spec/.../governance/workflows/elements/nodes/automatedTask/closeTask.json` — closeTask config schema
|
||||
- `openmetadata-service/.../governance/workflows/elements/nodes/automatedTask/OpenTask.java` — Task layer
|
||||
- `openmetadata-service/.../governance/workflows/elements/nodes/automatedTask/CloseTask.java` — Task layer
|
||||
- `openmetadata-service/.../governance/workflows/elements/nodes/automatedTask/impl/OpenTaskDelegate.java` — Delegate
|
||||
- `openmetadata-service/.../governance/workflows/elements/nodes/automatedTask/impl/OpenTaskImpl.java` — Impl
|
||||
- `openmetadata-service/.../governance/workflows/elements/nodes/automatedTask/impl/CloseTaskDelegate.java` — Delegate
|
||||
- `openmetadata-service/.../governance/workflows/elements/nodes/automatedTask/impl/CloseTaskImpl.java` — Impl
|
||||
- `openmetadata-service/src/main/resources/json/data/governance/workflows/IncidentLifecycleWorkflow.json` — Bootstrap data
|
||||
|
||||
**Modified files:**
|
||||
- `openmetadata-spec/.../governance/workflows/elements/nodeSubType.json` — Add `openTask`, `closeTask`
|
||||
- `openmetadata-service/.../governance/workflows/elements/NodeFactory.java` — Add switch cases for openTask, closeTask
|
||||
- `openmetadata-service/.../governance/workflows/WorkflowHandler.java` — Add TCRS event handler, signal broadcasting
|
||||
- `openmetadata-service/.../EntityLifecycleEventDispatcher.java` (or equivalent) — Register TCRS event broadcasting
|
||||
- `openmetadata-service/.../jdbi3/TestCaseResolutionStatusRepository.java` — Add event broadcast in `storeInternal()`, modify `openOrAssignTask()` Ack case, remove Thread creation from `New` path, remove Thread closure from `resolveTask()`
|
||||
- `openmetadata-service/.../jdbi3/TestCaseRepository.java` — Add event broadcast in resolution paths
|
||||
|
||||
### D9: Timer subplot architecture (Slice 3 prep)
|
||||
|
||||
**Decision (deferred to Slice 3, documented here for architectural clarity):**
|
||||
|
||||
When timers are added (reminders, TTL), the openTask branch gains an embedded subprocess:
|
||||
|
||||
```
|
||||
[openTask] → [SubProcess: timerChain]
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ [Timer: 24h] → [Notify] → [Timer: 7d] → [Close] │
|
||||
│ │
|
||||
│ SignalBoundaryEvent (interrupting): │
|
||||
│ "tcrs_closed_{testCaseFQN}" │
|
||||
└──────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
The signal boundary event catches `tcrs_closed_{fqn}` (broadcast by closeTask in the Resolved branch's process). Since Flowable signals are broadcast, the same closeTask execution simultaneously:
|
||||
1. Completes its own short-lived process (Resolved branch)
|
||||
2. Interrupts any running timer subprocess in an older process (via signal boundary)
|
||||
|
||||
No Flowable queries needed. The two-signal pattern (`tcrs_{fqn}` for fan-out, `tcrs_closed_{fqn}` for termination) cleanly separates concerns.
|
||||
|
||||
---
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[EntityLifecycleEventDispatcher is synchronous]**
|
||||
→ The workflow process runs in the same thread as the API call. If the workflow takes too long, the API response is delayed. **Mitigation:** All processes in Slice 1 are short-lived (milliseconds). The idempotent nodes do a DB query + conditional insert — comparable to the current `storeInternal()` logic. For Slice 3, timer subprocesses enter a wait state quickly (the timer setup is fast, only the wait is long).
|
||||
|
||||
**[Multiple processes for same business key]**
|
||||
→ Flowable does not enforce business key uniqueness. Multiple short-lived processes could theoretically run concurrently for the same FQN (e.g., rapid Ack + Assigned). **Mitigation:** Idempotent nodes make this safe — the second process simply no-ops. DB-level row locking on Thread creation prevents duplicates.
|
||||
|
||||
**[TCRS event pipeline is new infrastructure]**
|
||||
→ Extending EntityLifecycleEventDispatcher for TCRS is new code. **Mitigation:** The pattern already exists for other entities (search indexing). The extension is small — one handler registration + one method call.
|
||||
|
||||
**[Ack backward compatibility]**
|
||||
→ If the workflow fails to create the Thread/task (Flowable down, config error), the `Ack` path falls back to creating the task — because `getIncidentTask()` returns null and `createTask()` is called. This is a natural fallback, not a designed dual-path.
|
||||
|
||||
**[Signal name collision]**
|
||||
→ Signal names include the test case FQN, which can be long. **Mitigation:** Flowable stores signal names as strings with no length limit in `ACT_RU_EVENT_SUBSCR`. FQNs are already bounded by entity naming constraints.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **EntityLifecycleEventDispatcher extension**: What's the cleanest integration? A new `TcrsEventHandler` interface, a method on `WorkflowHandler`, or a lambda registration?
|
||||
- **Thread/task creation details**: Does `OpenTaskImpl` create the Thread directly via `FeedRepository.create()`, or does it call `TestCaseResolutionStatusRepository.createTask()`? The latter has coupled logic (severity, metrics) that may not apply.
|
||||
- **Signal payload sufficiency**: Can signal variables carry enough context (status, FQN, stateId, entityLink) to avoid a DB read in openTask/closeTask, or should each node read from DB independently for freshness?
|
||||
- **storeInternal removal scope**: How much of the `New` and `Resolved` cases can we remove vs. keep as fallback? If the dispatcher fails to fire, the old code path could serve as a safety net during rollout.
|
||||
@@ -0,0 +1,90 @@
|
||||
# Incident Lifecycle Workflow
|
||||
|
||||
> **Slice 1 of 3** — Incident Manager → Governance Workflows Migration
|
||||
> **Depends on**: Nothing (first slice)
|
||||
> **Enables**: [incident-auto-close](../incident-auto-close/proposal.md), [incident-ttl](../incident-ttl/proposal.md)
|
||||
> **ADR**: [adr-incident-manager-governance-workflows.md](../../../adr-incident-manager-governance-workflows.md)
|
||||
|
||||
---
|
||||
|
||||
## What Ships
|
||||
|
||||
When a test case's incident status changes, a governance workflow reacts to the event — creating the Thread/task on new incidents and closing it on resolution. The workflow is a single branching definition that users can see and extend in the governance workflows UI.
|
||||
|
||||
**User-visible changes:**
|
||||
- Incident Thread/task created immediately on test failure (no longer deferred to Ack)
|
||||
- Auto-assign to table owner configurable (default: unassigned, matching current behavior)
|
||||
- Incident lifecycle visible in governance workflows UI
|
||||
- Users can customize by adding steps to workflow branches (e.g., notifications, Jira)
|
||||
- Re-open from Resolved to any non-Resolved status creates a new incident lifecycle
|
||||
|
||||
**Behavior preserved:**
|
||||
- REST API surface unchanged
|
||||
- Ack and Assigned transitions unchanged in repository (assignee patching)
|
||||
- TCRS record creation unchanged (synchronous, for incidentId linking)
|
||||
- Severity inference unchanged (in repository)
|
||||
|
||||
---
|
||||
|
||||
## What We Build
|
||||
|
||||
### Generic Task Nodes: `openTask` and `closeTask`
|
||||
|
||||
Two new generic governance workflow nodes, reusable beyond incident management:
|
||||
|
||||
**`openTask`** (`nodeType: automatedTask`, `nodeSubType: openTask`):
|
||||
- Idempotently creates a Thread with configurable `TaskType` and `TaskStatus.Open`
|
||||
- If a Thread/task already exists for the entity, it's a no-op
|
||||
- Optional auto-assign via `responsibles` config (default: unassigned)
|
||||
- Configurable via `template` (e.g., `"incident"`, future: `"review"`)
|
||||
|
||||
**`closeTask`** (`nodeType: automatedTask`, `nodeSubType: closeTask`):
|
||||
- Closes an open Thread/task for the entity
|
||||
- If no open Thread/task exists, it's a no-op
|
||||
|
||||
Both follow the three-layer pattern: Task (BPMN) → Delegate (JavaDelegate) → Impl (pure logic).
|
||||
|
||||
### TCRS Event Broadcasting
|
||||
|
||||
Extend `EntityLifecycleEventDispatcher` to broadcast `TestCaseResolutionStatus` events to registered handlers. TCRS is a time-series entity that does NOT emit ChangeEvents, so this is a new event pipeline.
|
||||
|
||||
Flow: `storeInternal()` → `EntityLifecycleEventDispatcher` → `WorkflowHandler` → Flowable signal
|
||||
|
||||
### Signal-Driven Workflow Triggering
|
||||
|
||||
Every TCRS event broadcasts a Flowable signal `"tcrs_{fqn}"` with the TCRS status as a process variable. The signal starts a new short-lived process instance in every matching workflow. No Flowable queries needed for routing.
|
||||
|
||||
### Default Incident Lifecycle Workflow
|
||||
|
||||
Single branching workflow, ships enabled:
|
||||
```
|
||||
Trigger: Signal "tcrs_{fqn}" (from TCRS event broadcast)
|
||||
|
||||
[Signal Start] → [Gateway: status?]
|
||||
├─ NOT Resolved → [OpenTask] → [End]
|
||||
├─ Resolved → [CloseTask] → [End]
|
||||
└─ Otherwise → [End]
|
||||
```
|
||||
|
||||
All process instances are short-lived in Slice 1 (no timers). OpenTask and CloseTask are idempotent — safe to fire on every event. The gateway routes purely on TCRS status; nodes handle their own edge cases.
|
||||
|
||||
---
|
||||
|
||||
## Out of Scope
|
||||
|
||||
| Feature | Deferred to | Why |
|
||||
|---------|-------------|-----|
|
||||
| Auto-close on test pass | Slice 2 | Independent feature, separate workflow |
|
||||
| TTL / stale incident expiration | Slice 3 | Timer subprocess with signal boundary |
|
||||
| Timer subprocess + signal interruption | Slice 3 | Architecture supports it, no timers yet |
|
||||
| `tcrs_closed_{fqn}` termination signal | Slice 3 | Only needed for timer subprocess interruption |
|
||||
| Cleanup timer for orphaned processes | Follow-up | Batch sweep; idempotent openTask handles most |
|
||||
| Custom lifecycle states | Future | Template-driven state machine evolution |
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **EntityLifecycleEventDispatcher extension**: What's the cleanest way to add TCRS event broadcasting? Observer registration, interface, or direct handler call?
|
||||
- **Thread/task creation**: Does `openTask` create the Thread directly via `FeedRepository.create()`, or reuse `TestCaseResolutionStatusRepository.createTask()`?
|
||||
- **Signal payload**: Can signal variables carry enough context (status, FQN, stateId) to avoid a DB read in the gateway, or should each node read from DB independently?
|
||||
+110
@@ -0,0 +1,110 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: TCRS event broadcasting via EntityLifecycleEventDispatcher
|
||||
After `storeInternal()` persists a `TestCaseResolutionStatus` record, the system SHALL broadcast the event via `EntityLifecycleEventDispatcher` to registered handlers. `WorkflowHandler` SHALL receive the event and broadcast a Flowable signal.
|
||||
|
||||
#### Scenario: TCRS record creation triggers signal broadcast
|
||||
- **WHEN** `storeInternal()` persists a new TCRS record (any status: New, Ack, Assigned, Resolved)
|
||||
- **THEN** `EntityLifecycleEventDispatcher` SHALL notify registered handlers
|
||||
- **AND** `WorkflowHandler` SHALL broadcast a Flowable signal `"tcrs_{testCaseFQN}"` with process variables including `status`, `testCaseFQN`, `stateId`, and `entityLink`
|
||||
|
||||
#### Scenario: Signal starts new process instances
|
||||
- **WHEN** signal `"tcrs_{testCaseFQN}"` is broadcast
|
||||
- **AND** a workflow definition has a SignalStartEvent matching that signal
|
||||
- **THEN** Flowable SHALL create a new process instance with the signal variables as process variables
|
||||
|
||||
#### Scenario: Multiple workflows can react to same event
|
||||
- **WHEN** signal `"tcrs_{testCaseFQN}"` is broadcast
|
||||
- **AND** multiple workflow definitions have matching SignalStartEvents
|
||||
- **THEN** each workflow SHALL independently start a new process instance
|
||||
|
||||
### Requirement: Default incident lifecycle workflow ships enabled
|
||||
A default governance workflow definition (`IncidentLifecycleWorkflow.json`) SHALL be bootstrapped on server startup. It SHALL trigger on TCRS events via signal `"tcrs_{testCaseFQN}"`, and its flow SHALL branch on status: non-Resolved events route to `openTask`, Resolved events route to `closeTask`. Auto-assign is NOT enabled by default.
|
||||
|
||||
#### Scenario: Workflow is loaded on server startup
|
||||
- **WHEN** the OpenMetadata server starts
|
||||
- **THEN** the `IncidentLifecycleWorkflow.json` file in `json/data/governance/workflows/` SHALL be loaded by `initSeedDataFromResources()` with `createOrUpdate` semantics
|
||||
- **AND** the workflow definition SHALL be available via the governance workflows API
|
||||
|
||||
#### Scenario: New failure creates Thread/task via openTask branch
|
||||
- **WHEN** a TCRS record with status `New` is persisted (test case failure)
|
||||
- **THEN** the signal starts a new process instance
|
||||
- **AND** the gateway routes to the `openTask` node (status != Resolved)
|
||||
- **AND** `openTask` creates a Thread/task with `TaskType.RequestTestCaseFailureResolution`
|
||||
- **AND** the process ends (short-lived)
|
||||
|
||||
#### Scenario: Ack/Assigned event is no-op via openTask branch
|
||||
- **WHEN** a TCRS record with status `Ack` or `Assigned` is persisted
|
||||
- **THEN** the signal starts a new process instance
|
||||
- **AND** the gateway routes to the `openTask` node (status != Resolved)
|
||||
- **AND** `openTask` detects the Thread/task already exists and is a no-op
|
||||
- **AND** the process ends (short-lived)
|
||||
|
||||
#### Scenario: Resolved event closes Thread/task via closeTask branch
|
||||
- **WHEN** a TCRS record with status `Resolved` is persisted
|
||||
- **THEN** the signal starts a new process instance
|
||||
- **AND** the gateway routes to the `closeTask` node (status == Resolved)
|
||||
- **AND** `closeTask` closes the open Thread/task
|
||||
- **AND** the process ends (short-lived)
|
||||
|
||||
#### Scenario: Re-open from Resolved creates new lifecycle
|
||||
- **WHEN** a TCRS record with any non-Resolved status is persisted after a previous Resolved record for the same incident
|
||||
- **THEN** the signal starts a new process instance
|
||||
- **AND** the gateway routes to the `openTask` node (status != Resolved)
|
||||
- **AND** `openTask` detects no open Thread/task exists (previous was closed) and creates a new one
|
||||
- **AND** the process ends (short-lived)
|
||||
|
||||
### Requirement: All processes are short-lived (Slice 1)
|
||||
In Slice 1, all workflow process instances SHALL complete within the same synchronous call. No process SHALL enter a wait state or persist Flowable runtime rows between events.
|
||||
|
||||
#### Scenario: Process completes before API returns
|
||||
- **WHEN** a TCRS event triggers a workflow process
|
||||
- **THEN** the process SHALL execute all nodes and reach an end event before the signal broadcast call returns
|
||||
- **AND** no `ACT_RU_EXECUTION` rows SHALL persist for the process after completion
|
||||
|
||||
### Requirement: Ack case backward compatibility
|
||||
The `Ack` case in `openOrAssignTask()` SHALL check for an existing Thread/task before creating one. If the workflow already created a task, the Ack SHALL patch the assignee on the existing task instead of creating a duplicate.
|
||||
|
||||
#### Scenario: Ack with existing workflow-created task patches assignee
|
||||
- **WHEN** a user Acknowledges an incident
|
||||
- **AND** `getIncidentTask()` returns an existing Thread (created by the workflow's openTask)
|
||||
- **THEN** `openOrAssignTask()` SHALL call `patchTaskAssignee()` on the existing Thread with the acknowledging user
|
||||
- **AND** SHALL NOT call `createTask()`
|
||||
|
||||
#### Scenario: Ack without existing task creates task (fallback)
|
||||
- **WHEN** a user Acknowledges an incident
|
||||
- **AND** `getIncidentTask()` returns null (workflow did not run, e.g., Flowable was down)
|
||||
- **THEN** `openOrAssignTask()` SHALL call `createTask()` as before
|
||||
- **AND** incident lifecycle SHALL continue normally
|
||||
|
||||
### Requirement: Severity inference remains in repository
|
||||
Severity inference SHALL continue to be called inside `storeInternal()` during incident record creation. The workflow does NOT duplicate severity inference — it relies on the severity already being set on the record when it reads it.
|
||||
|
||||
#### Scenario: Severity is inferred before workflow sees the record
|
||||
- **WHEN** `getOrCreateIncident()` creates a new `TestCaseResolutionStatus` record
|
||||
- **THEN** `storeInternal()` SHALL call `inferIncidentSeverity()` before persisting
|
||||
- **AND** the record SHALL have severity set when `openTask` reads it to create the Thread/task
|
||||
|
||||
### Requirement: incidentId linking remains synchronous
|
||||
The `TestCaseResolutionStatus` record creation (for `incidentId` linking) SHALL remain synchronous in `getOrCreateIncident()`. The workflow handles Thread/task lifecycle synchronously via the event dispatcher (same thread) after the record is persisted.
|
||||
|
||||
#### Scenario: Test result links to incident synchronously
|
||||
- **WHEN** `setTestCaseResultIncidentId()` is called for a failing test result
|
||||
- **THEN** `getOrCreateIncident()` SHALL create the `TestCaseResolutionStatus` record synchronously
|
||||
- **AND** the `stateId` SHALL be available for linking to the test result within the same transaction
|
||||
- **AND** the Thread/task creation SHALL happen via the workflow triggered by the event broadcast (still within the same synchronous call)
|
||||
|
||||
### Requirement: Thread/task lifecycle moves out of repository
|
||||
The `New` case in `storeInternal()` SHALL no longer create Thread/tasks directly. The `resolveTask()` method SHALL no longer close Thread/tasks directly. These responsibilities move to the workflow's `openTask` and `closeTask` nodes respectively.
|
||||
|
||||
#### Scenario: storeInternal New case does not create Thread
|
||||
- **WHEN** `storeInternal()` handles a `New` TCRS record
|
||||
- **THEN** it SHALL persist the record and infer severity (unchanged)
|
||||
- **AND** it SHALL broadcast the event via EntityLifecycleEventDispatcher
|
||||
- **AND** it SHALL NOT create a Thread/task directly (the workflow's openTask handles this)
|
||||
|
||||
#### Scenario: resolveTask does not close Thread directly
|
||||
- **WHEN** `resolveTask()` resolves an incident
|
||||
- **THEN** it SHALL persist the Resolved record
|
||||
- **AND** it SHALL broadcast the event via EntityLifecycleEventDispatcher
|
||||
- **AND** the workflow's closeTask SHALL handle Thread closure
|
||||
@@ -0,0 +1,90 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: openTask node — idempotent Thread/task creation
|
||||
The `openTask` node (`nodeType: automatedTask`, `nodeSubType: openTask`) SHALL compile into a Flowable ServiceTask referencing `OpenTaskDelegate`. It SHALL idempotently create a Thread with configurable `TaskType` and `TaskStatus.Open`. If an open Thread/task already exists for the entity, it SHALL be a no-op.
|
||||
|
||||
#### Scenario: BPMN structure is compiled correctly
|
||||
- **WHEN** a workflow definition contains an `openTask` node with config `{ "template": "incident", "taskType": "RequestTestCaseFailureResolution" }`
|
||||
- **THEN** the compiled BPMN model SHALL contain a ServiceTask referencing `OpenTaskDelegate.class.getName()`
|
||||
- **AND** the ServiceTask SHALL have Expression fields for `templateExpr`, `taskTypeExpr`, `responsiblesExpr`, and `inputNamespaceMapExpr`
|
||||
|
||||
#### Scenario: Node registered in NodeFactory
|
||||
- **WHEN** `NodeFactory` receives a node definition with `nodeSubType: openTask`
|
||||
- **THEN** it SHALL instantiate `OpenTask` and return a valid node
|
||||
|
||||
### Requirement: OpenTaskDelegate wires Flowable to business logic
|
||||
The `OpenTaskDelegate` SHALL implement `JavaDelegate`, receive configuration via Flowable `Expression` fields, extract values from the `DelegateExecution`, instantiate `OpenTaskImpl`, call it, and set the `taskCreated` process variable from the result. On business logic failure, it SHALL throw a `BpmnError`.
|
||||
|
||||
#### Scenario: Delegate passes config to impl and sets process variable
|
||||
- **WHEN** `OpenTaskDelegate.execute()` is called with a `DelegateExecution` containing valid expression values
|
||||
- **THEN** it SHALL instantiate `OpenTaskImpl` with resolved config values, call the impl, and set `execution.setVariable("taskCreated", result)`
|
||||
|
||||
#### Scenario: Delegate wraps exceptions as BpmnError
|
||||
- **WHEN** `OpenTaskImpl` throws an exception during execution
|
||||
- **THEN** `OpenTaskDelegate` SHALL catch it and throw a `BpmnError`
|
||||
|
||||
### Requirement: OpenTaskImpl creates Thread/task (clean state)
|
||||
When no open Thread/task exists for the entity, `OpenTaskImpl` SHALL create a Thread with the configured `TaskType` and `TaskStatus.Open` and return `taskCreated = true`. Auto-assignment is configurable via the optional `responsibles` config field. When `responsibles` is not set, the task SHALL be created unassigned (matching current behavior).
|
||||
|
||||
#### Scenario: First failure creates unassigned Thread (default, no responsibles config)
|
||||
- **WHEN** no open Thread/task exists for the entity
|
||||
- **AND** no `responsibles` config is set on the node
|
||||
- **THEN** `OpenTaskImpl` SHALL create an unassigned Thread task and return `taskCreated = true`
|
||||
|
||||
#### Scenario: First failure auto-assigns to table owner (responsibles configured)
|
||||
- **WHEN** no open Thread/task exists for the entity
|
||||
- **AND** `responsibles.source` is `"tableOwner"`
|
||||
- **THEN** `OpenTaskImpl` SHALL create a Thread task assigned to the table entity's owner and return `taskCreated = true`
|
||||
|
||||
#### Scenario: First failure auto-assigns to specific user (responsibles configured)
|
||||
- **WHEN** no open Thread/task exists for the entity
|
||||
- **AND** `responsibles.source` is `"specificUser"` with `responsibles.target` set to a user FQN
|
||||
- **THEN** `OpenTaskImpl` SHALL create a Thread task assigned to the specified user and return `taskCreated = true`
|
||||
|
||||
### Requirement: OpenTaskImpl skips when Thread/task already exists
|
||||
When an open Thread/task already exists for the entity, `OpenTaskImpl` SHALL skip creation and return `taskCreated = false`.
|
||||
|
||||
#### Scenario: Duplicate event is idempotently skipped
|
||||
- **WHEN** an open Thread/task already exists for the entity (e.g., Ack or Assigned event after Thread was created on New)
|
||||
- **THEN** `OpenTaskImpl` SHALL return `taskCreated = false`
|
||||
- **AND** no duplicate Thread SHALL be created
|
||||
|
||||
### Requirement: OpenTaskImpl runs as governance-bot
|
||||
All actions performed by `OpenTaskImpl` (Thread creation, task assignment) SHALL execute under the `governance-bot` identity.
|
||||
|
||||
#### Scenario: Governance-bot identity prevents event loops
|
||||
- **WHEN** `OpenTaskImpl` creates a Thread task
|
||||
- **THEN** the ChangeEvent generated by that creation SHALL have `governance-bot` as the updatedBy user
|
||||
- **AND** `WorkflowEventConsumer` SHALL skip processing that event
|
||||
|
||||
### Requirement: closeTask node — Thread/task closure
|
||||
The `closeTask` node (`nodeType: automatedTask`, `nodeSubType: closeTask`) SHALL compile into a Flowable ServiceTask referencing `CloseTaskDelegate`. It SHALL close an open Thread/task for the entity. If no open Thread/task exists, it SHALL be a no-op.
|
||||
|
||||
#### Scenario: BPMN structure is compiled correctly
|
||||
- **WHEN** a workflow definition contains a `closeTask` node with config `{ "template": "incident", "taskType": "RequestTestCaseFailureResolution" }`
|
||||
- **THEN** the compiled BPMN model SHALL contain a ServiceTask referencing `CloseTaskDelegate.class.getName()`
|
||||
|
||||
#### Scenario: Node registered in NodeFactory
|
||||
- **WHEN** `NodeFactory` receives a node definition with `nodeSubType: closeTask`
|
||||
- **THEN** it SHALL instantiate `CloseTask` and return a valid node
|
||||
|
||||
### Requirement: CloseTaskDelegate wires Flowable to business logic
|
||||
The `CloseTaskDelegate` SHALL implement `JavaDelegate`, receive configuration via Flowable `Expression` fields, instantiate `CloseTaskImpl`, call it, and set the `taskClosed` process variable.
|
||||
|
||||
#### Scenario: Delegate passes config to impl and sets process variable
|
||||
- **WHEN** `CloseTaskDelegate.execute()` is called with a `DelegateExecution`
|
||||
- **THEN** it SHALL instantiate `CloseTaskImpl`, call the impl, and set `execution.setVariable("taskClosed", result)`
|
||||
|
||||
### Requirement: CloseTaskImpl closes Thread/task
|
||||
When an open Thread/task exists for the entity, `CloseTaskImpl` SHALL close it and return `taskClosed = true`. When no open Thread/task exists, it SHALL return `taskClosed = false`.
|
||||
|
||||
#### Scenario: Open Thread/task is closed
|
||||
- **WHEN** an open Thread/task exists for the entity
|
||||
- **THEN** `CloseTaskImpl` SHALL close the Thread task and return `taskClosed = true`
|
||||
|
||||
#### Scenario: No-op when no open Thread/task exists
|
||||
- **WHEN** no open Thread/task exists for the entity
|
||||
- **THEN** `CloseTaskImpl` SHALL return `taskClosed = false` without error
|
||||
|
||||
### Requirement: CloseTaskImpl runs as governance-bot
|
||||
All actions performed by `CloseTaskImpl` SHALL execute under the `governance-bot` identity.
|
||||
Reference in New Issue
Block a user