Latest version: 4.1.x
Kafka Quotas can be used to
- deliver better SLAs
- reduce latency
- offer an improved multi-tenant experience and
- provide a charge back model for cost-center allocation of the operation cost
of you data platform, by applying quotas on consume/produce resources.
Manage Kafka quotas
- From the Header bar menu, go to Admin panel.
- On the side navigation, select Kafka Quotas under the Kafka Settings section.
To manage quotas on Kafka, you need to i) have the Kafka Admin permission on your account and ii) have Zookeeper access. See how to configure ZK .
Cloud and managed Kafka services do not currently offer access to Zookeeper and quota management will become available via KIP-546.
Create Kafka quotas
You can create new Quotas, with the New Quota button and selecting the client or user (or combination) to apply this quota on. You can also set up default quotas.
For example, to apply a Quota on a particular application that can create an aggressive spike on your data platform (i.e. an Apache Spark application), select CLIENT:
And for example, enforce that specific application to not produce and consumer more than 5MBytes/sec:
With quota management enabled, you can create, search, and manage your Quotas:
You can enforce quotas on produce and fetch requests for:
- A user ( requires an authenticated Kafka )
- An application ( using the client-id )
- The application of a specific user ( requires an authenticated Kafka )
The following order of precedence is followed when multiple quotas are in place.
- (user) / (client-id)
- (user) / (default)
- default / (client-id)
- default / clients / (default)
- users / default
- clients / (client-id)
- clients / default
Learn more about Kafka Quotas .
API & GitOps
Kafka Quotas permissions
You can use RBAC (Role Based Access) to control users that have visibility and management roles for Kafka Quotas.
Specifically, the permissions
ManageKafkaSettings are needed to view and manage respectively.