Kafka Producer monitoring

Lenses continuously monitors all Kafka events produced in a Kafka cluster. It uses the native Kafka client to calculate in real-time metrics around events produced in a Kafka topic per time window.

Monitor Kafka producers

1. From the Header Bar Menu, go to the Dashboard panel.
2. On the side navigation, select Explore under the Data section.
3. Choose one of the topics listed.
4. Click on Metrics.

The above highlights the messages produced on the Kafka topic over the period of the last 30 days. The minimum, average and maximum number of events produced per day are visualised in a way that enables you to easily identify the real behaviour of your producers.

For a real-time view of messages produced per topic, use the data exploration screen.

Kafka producer SLAs

Once you have a better understanding on the SLAs that the producers of your Streaming Data Platform should adhere, you can setup SLA rules:

Visit Admin → Rules under Alerts and click Data Produced

Adding a new Data SLA requires setting up a threshold, a time window and also who (which channel) to notify when the alert is triggered.

Similarly to Infrastructure rules, you can also specify which channel they will be routed to.

JMX

On the bottom of the metrics page, a data table is available. This provides a summary of key topic metrics across your cluster.