Files
wehub-resource-sync a9cd7750f4
CI / unit-test (push) Has been cancelled
CI / detect-changes (push) Has been cancelled
CI / build (push) Has been cancelled
Publish docs via GitHub Pages / Deploy docs (push) Has been cancelled
CI / test-harness (push) Has been cancelled
CI / generate-e2e-matrix (push) Has been cancelled
CI / e2e (push) Has been cancelled
CI / build-ui (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
UI v2 Integration CI / E2E (Integration) (push) Has been cancelled
UI v2 CI / Lint, Format & Test (push) Has been cancelled
UI v2 CI / E2E (Mocked) (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 12:37:56 +08:00

235 lines
7.1 KiB
Markdown

---
description: "Orchestrate event-driven workflows with Conductor using Kafka, NATS, AMQP (RabbitMQ), and SQS as event buses. Configure event handlers to trigger workflows, complete tasks, or fail tasks on incoming events."
---
# Event Bus Orchestration
Conductor integrates with external messaging systems to enable event-driven workflow orchestration. You can publish events from workflows and react to external events — starting workflows, completing tasks, or failing tasks based on incoming messages.
## Supported event buses
| System | Sink prefix | Module | Use case |
| :--- | :--- | :--- | :--- |
| **Kafka** | `kafka` | `kafka` | High-throughput, durable event streaming |
| **NATS** | `nats` | `nats` | Lightweight, low-latency messaging |
| **NATS Streaming** | `nats-stream` | `nats-streaming` | Durable NATS with replay (legacy) |
| **NATS JetStream** | `nats` | `nats` | Modern durable NATS streaming |
| **AMQP (RabbitMQ)** | `amqp`, `amqp_queue`, `amqp_exchange` | `amqp` | Traditional message queuing with routing |
| **SQS** | `sqs` | `sqs` | AWS-native message queuing |
| **Conductor** | `conductor` | built-in | Internal event routing between workflows |
## How it works
Event bus orchestration has two sides:
1. **Publishing** — Use the [Event task](../../documentation/configuration/workflowdef/systemtasks/event-task.md) or [Kafka Publish task](../../documentation/configuration/workflowdef/systemtasks/kafka-publish-task.md) to send messages from a workflow.
2. **Consuming** — Register [event handlers](../../documentation/configuration/eventhandlers.md) that listen for messages and trigger actions.
```
┌──────────────┐ Event Task ┌──────────────┐ Event Handler ┌──────────────┐
│ Workflow A │ ──────────────────► │ Event Bus │ ──────────────────► │ Workflow B │
│ │ (publish) │ (Kafka/NATS/ │ (start_workflow) │ (triggered) │
│ │ │ AMQP/SQS) │ │ │
└──────────────┘ └──────────────┘ └──────────────┘
```
## Publishing events
### Event task
The [Event task](../../documentation/configuration/workflowdef/systemtasks/event-task.md) publishes a message to any supported event bus. The `sink` parameter determines the target:
```json
{
"name": "notify_downstream",
"taskReferenceName": "notify_ref",
"type": "EVENT",
"sink": "kafka:order-events",
"inputParameters": {
"orderId": "${workflow.input.orderId}",
"status": "PROCESSED"
}
}
```
### Kafka Publish task
For Kafka-specific features (custom headers, key, serializers), use the dedicated [Kafka Publish task](../../documentation/configuration/workflowdef/systemtasks/kafka-publish-task.md):
```json
{
"name": "publish_to_kafka",
"taskReferenceName": "kafka_ref",
"type": "KAFKA_PUBLISH",
"inputParameters": {
"kafka_request": {
"topic": "order-events",
"value": "${workflow.input.orderData}",
"bootStrapServers": "kafka:9092",
"headers": {
"X-Correlation-Id": "${workflow.correlationId}"
}
}
}
}
```
### Sink format
The `sink` parameter follows the format `prefix:queue_name`:
| Example | System |
| :--- | :--- |
| `kafka:order-events` | Kafka topic `order-events` |
| `nats:notifications` | NATS subject `notifications` |
| `amqp:task-queue` | AMQP queue `task-queue` |
| `amqp_exchange:events` | AMQP exchange `events` |
| `sqs:my-queue` | SQS queue `my-queue` |
| `conductor` | Conductor internal queue |
| `conductor:workflow_name:queue_name` | Conductor internal, specific queue |
## Consuming events
### Event handlers
Event handlers listen for messages on an event bus and execute actions when a matching event arrives. Register them via the `/api/event` API.
```json
{
"name": "order_event_handler",
"event": "kafka:order-events",
"condition": "$.status == 'PROCESSED'",
"actions": [
{
"action": "start_workflow",
"start_workflow": {
"name": "fulfillment_workflow",
"input": {
"orderId": "${orderId}"
}
}
}
]
}
```
### Supported actions
| Action | Description |
| :--- | :--- |
| `start_workflow` | Start a new workflow execution with the event payload as input. |
| `complete_task` | Complete a waiting task (e.g., a `WAIT` or `HUMAN` task) in a running workflow. |
| `fail_task` | Fail a task in a running workflow. |
### Conditions
The `condition` field supports JavaScript-like expressions evaluated against the event payload:
| Expression | Result |
| :--- | :--- |
| `$.version > 1` | true if `version` field > 1 |
| `$.metadata.codec == 'aac'` | true if nested field matches |
| `$.status == 'COMPLETED'` | true if status is COMPLETED |
Actions execute only when the condition evaluates to `true`. If no condition is specified, actions execute for every event.
## Patterns
### Event-driven workflow chaining
Decouple workflows using events instead of sub-workflows:
```json
{
"name": "order_pipeline",
"tasks": [
{
"name": "process_order",
"taskReferenceName": "process_ref",
"type": "SIMPLE"
},
{
"name": "notify_fulfillment",
"taskReferenceName": "notify_ref",
"type": "EVENT",
"sink": "kafka:fulfillment-requests",
"inputParameters": {
"orderId": "${workflow.input.orderId}",
"items": "${process_ref.output.items}"
}
}
]
}
```
A separate event handler starts the fulfillment workflow when the event arrives.
### Wait for external event
Combine a `WAIT` task with an event handler to pause a workflow until an external system signals completion:
```json
{
"name": "wait_for_approval",
"taskReferenceName": "approval_ref",
"type": "WAIT"
}
```
Register an event handler that completes the task when an approval event arrives:
```json
{
"name": "approval_handler",
"event": "kafka:approval-events",
"condition": "$.approved == true",
"actions": [
{
"action": "complete_task",
"complete_task": {
"workflowId": "${workflowId}",
"taskRefName": "approval_ref",
"output": {
"approvedBy": "${approvedBy}"
}
}
}
]
}
```
## Configuration
Each event bus module requires its own configuration. Enable the modules you need in your Conductor server configuration:
### Kafka
```properties
conductor.event-queues.kafka.enabled=true
conductor.event-queues.kafka.bootstrap-servers=kafka:9092
```
### NATS
```properties
conductor.event-queues.nats.enabled=true
conductor.event-queues.nats.url=nats://localhost:4222
```
### AMQP (RabbitMQ)
```properties
conductor.event-queues.amqp.enabled=true
conductor.event-queues.amqp.hosts=rabbitmq
conductor.event-queues.amqp.port=5672
conductor.event-queues.amqp.username=guest
conductor.event-queues.amqp.password=guest
```
Refer to the module source code for the full set of configuration properties.