Supertubes automates the deployment of Kafka Connect clusters and the creation of connectors through KafkaConnect custom resource instances and KafkaConnector custom resource instances, respectively.
Imperative Kafka Connect connector management 🔗︎
supertubes install -a
command which will set up a Kafka Connect cluster with default settings, linked to the provisioned demo Apache Kafka cluster and the deployed Schema Registry instance.
supertubes cluster kafka-connector [command]
update sub-commands require KafkaConnector custom resource specifications as descriptors. For details on these commands, see supertubes cluster kafka-connector.
Declarative Kafka Connect connector management 🔗︎
The pattern for managing connectors is similar to that of Kafka Connect. There is a KafkaConnector custom resource that represents a connector type which is used by Supertubes to instantiate connectors on Kafka Connect clusters.
apiVersion: kafka.banzaicloud.io/v1beta1 kind: KafkaConnector metadata: name: my-connector spec: # The connector class name (e.g. io.confluent.connect.s3.S3SinkConnector) class: # The maximum number of tasks the Kafka Connector can create tasksMax: # References the Kafka Connect cluster where this connector is to run clusterRef: name: my-kafka-connect # Defines the config values for Kafka Connector in key - value pairs format config: # Whether to pause or resume the connector execution pause: