conductor-oss--conductor/docs/documentation/configuration/workflowdef/systemtasks/kafka-publish-task.md

description

description
Kafka Publish Task — send messages to Kafka topics from Conductor workflows with configurable serialization.

Kafka Publish Task

"type" : "KAFKA_PUBLISH"

The Kafka Publish task (KAFKA_PUBLISH) is used to push messages to another microservice via Kafka.

Task parameters

The task expects a field named kafka_request as part of the task's inputParameters.

Use these parameters inside inputParameters in the Kafka Publish task configuration.

Parameter	Type	Description	Required / Optional
kafka_request	KafkaRequest	JSON object containing the bootstrap server, message, and more.	Required.
kafka_request.bootStrapServers	String	The bootstrap server for connecting to the Kafka cluster.	Required.
kafka_request.topic	String	The topic to publish the message to.	Required.
kafka_request.value	Any	The message to publish.	Required.
kafka_request.key	String	The Kafka message key. Messages with the same key will be sent to the same topic partition.	Optional.
kafka_request.keySerializer	String (enum)	The serializer used for serializing the message key. The default is StringSerializer. Supported values: org.apache.kafka.common.serialization.IntegerSerializer org.apache.kafka.common.serialization.LongSerializer org.apache.kafka.common.serialization.StringSerializer	Optional.
kafka_request.headers	Map[String, Any]	Any additional headers to be sent along with the Kafka message.	Optional.
kafka_request.requestTimeoutMs	Integer	The request timeout in milliseconds while awaiting a response.	Optional.
kafka_request.maxBlockMs	Integer	The maximum blocking time while publishing to Kafka.	Optional.

JSON configuration

Here is the task configuration for a Kafka Publish task.

{
  "name": "kafka",
  "taskReferenceName": "kafka_ref",
  "inputParameters": {
    "kafka_request": {
      "topic": "userTopic",
      "value": "Message to publish",
      "bootStrapServers": "localhost:9092",
      "headers": {
        "x-Auth":"Auth-key"    
      },
      "key": "123",
      "keySerializer": "org.apache.kafka.common.serialization.IntegerSerializer"
    }
  },
  "type": "KAFKA_PUBLISH"
}

Output

The task transitions to COMPLETED if the message has been successfully published to the Kafka queue, or marked as FAILED if the message could not be published.