Lenses Connectors for Kafka

You can manage multiple Kafka Connect clusters with Lenses as well as build and monitor your streaming flows.

Lenses provides one of the largest collection of Apache Kafka connectors with Lenses SQL support for all major data sources and sinks including certified Cassandra, Elastic, InfluxDB, Azure CosmosDB, MQTT, and JMS connectors.

Lenses is compatible with any Kafka connector plugin.

  1. From the Header Bar Menu, go to the Dashboard panel.
  2. On the side navigation, select Connectors under Flows section.

connectors

Add Connect clusters

Lenses handles multiple connect clusters. You can configure the cluster details via Lenses configuration file. For each cluster, you can give a friendly name to be able to create and filter connectors for this cluster.

Currently, Lenses UI is not supporting adding new clusters without a restart.

Provision Connectors

To provision a connector, the connector plugin must exist on the classpath of each worker in the cluster.

Select the New Connector to provision, and select the plugin type:

connectors

Fill in the configuration required for each connector:

connectors

Manage Connectors

Connectors can be Stopped, Restarted, Edited or Deleted. When an action takes place for a connector, a rebalance happens to the cluster. This causes all connectors to stop and restarted for Kafka versions before 2.3.

Find the connector actions in the options menu:

connectors

Monitor Connectors

Each connector is monitored. You can view the health check and specific metrics for each connector.

Task metrics

For each Task metrics are available via the different worker:

connectors

Use the CLI

See CLI Docs

Use the API

See API Docs

Permissions on Connectors

Application permissions govern connectors. This means that we can access the nodes related to the namespaces you have access too. See more on Application Permissions.

FAQ

Can any connector be plugged-in?

Yes! Lenses connects to the cluster and manages any available plugins.

If you want to have better visibility of a 3rd party connector you can optionally enhance the information about it:

lenses {

  ...

  connectors.info = [
      {
           class.name = "The connector full classpath"
           name = "The name which will be presented in the UI"
           instance = "Details about the instance. Contains the connector configuration field which holds the information. If  a database is involved it would be  the DB connection details, if it is a file it would be the file path, etc"
           sink = true
           extractor.class = "The full classpath for the implementation knowing how to extract the Kafka topics involved. This is only required for a Source"
           icon = "file.png"
           description = "A description for the connector"
           author = "The connector author"
      }
      ...
  ]
}