Latest version: 4.3.x
Kafka Quotas
Kafka Quotas
can be used to
- deliver better SLAs
- reduce latency
- offer an improved multi-tenant experience and
- provide a charge back model for cost-center allocation of the operation cost
of you data platform, by applying quotas on consume/produce resources.
Manage Kafka quotas
- From the Header bar menu, go to Admin panel.
- On the side navigation, select Kafka Quotas under the Kafka Settings section.
Prerequisites
To manage quotas on Kafka, you need to i) have the Kafka Admin permission on your account and ii) have Zookeeper access. See how to configure ZK .
Cloud and managed Kafka services do not currently offer access to Zookeeper and quota management will become available via KIP-546.
Create Kafka quotas
You can create new Quotas, with the New Quota button and selecting the client or user (or combination) to apply this quota on. You can also set up default quotas.
For example, to apply a Quota on a particular application that can create an aggressive spike on your data platform (i.e. an Apache Spark application), select CLIENT:
And for example, enforce that specific application to not produce and consume more than 5MBytes/sec:
Explore quotas
With quota management enabled, you can create, search, and manage your Quotas:
Quota precedence
You can enforce quotas on produce and fetch requests for:
- A user ( requires an authenticated Kafka )
- An application ( using the client-id )
- The application of a specific user ( requires an authenticated Kafka )
The following order of precedence is followed when multiple quotas are in place.
- (user) / (client-id)
- (user) / (default)
- (user)
- default / (client-id)
- default / clients / (default)
- users / default
- clients / (client-id)
- clients / default
Learn more about Kafka Quotas .
API & GitOps
For automation use the API and CLI
Kafka Quotas permissions
You can use RBAC (Role Based Access) to control users that have visibility and management roles for Kafka Quotas.
Specifically, the permissions ViewKafkaSettings
and ManageKafkaSettings
are needed to view and manage respectively.