Kafka Quotas

Use Quotas to deliver better SLAs, reduced latency and offer an improved multi-tenant experience in your data platform.

Manage Kafka quotas

  1. From the Header bar menu, go to Admin panel.
  2. On the side navigation, select Kafka Quotas under the Kafka Settings section.
Kafka Quotas

Prerequisites

To manage quotas on Kafka, you need to i) have the Kafka Admin permission on your account and ii) have Zookeeper access. See how to configure ZK .

Cloud and managed Kafka services do not offer access to Zookeeper and quota management will become available via KIP-546.

Create a Kafka quota

You can create new Quotas, with the New Quota button and selecting the client or user (or combination) to apply this quota on. You can also set up default quotas.

For example, to apply a Quota on a particular application that can create an aggressive spike on your data platform (i.e. an Apache Spark application), select CLIENT:

Add a new Kafka Quota

And for example, enforce that specific application to not produce and consumer more than 5MBytes/sec:

Create new Kafka Quota

Explore quotas

With quota management enabled, you can create, search, and manage your Quotas:

List Kafka Quotas

Quotas precedence

You can enforce quotas on produce and fetch requests for:

  • A user ( requires an authenticated Kafka )
  • An application ( using the client-id )
  • The application of a specific user ( requires an authenticated Kafka )

The following order of precedence is followed when multiple quotas are in place.

  1. (user) / (client-id)
  2. (user) / (default)
  3. (user)
  4. default / (client-id)
  5. default / clients / (default)
  6. users / default
  7. clients / (client-id)
  8. clients / default

Learn more about Kafka Quotas .

Automation

For automation use the API and CLI