Manage your Topics

Lenses allows users to create, edit and delete Kafka topics. Namespace permissions protect each action. Every time an action is taken, Lenses audits the event.

Topic concepts

Topics in Kafka consist of three basic fields and the topic configurations:

  • Name, the unique name of the topic.
  • Partitions, the number of partitions the topic will be divided into.
  • Replication, the replication factor for each partition across the cluster.

All topic configurations have default values. Important configs include: Cleanup policy, Retention size and Retention time

Create a topic

To create a new topic:

  1. From the Header Bar Menu, go to the Dashboard panel.
  2. On the side navigation, select Explore under the data section.
  3. Click the New Topic button and fill in the form.
Topic Create

Edit topic configuration

Topic configurations are Key-Value pairs. Lenses supports all Topic Configs and also provides autocompletion and the documentation for each Key.

To Edit and View the Topic Configuration:

  1. From the Header Bar Menu, go to the Dashboard panel.
  2. On the side navigation, select Topics under the data section.
  3. Click on the Topic Name from the list and navigate to the Config tab.

Topic Config

To edit the topic config, click on the pencil on the rightmost column and fill in the new value.

Edit Topic Configuration

Editing metadata & schemas

Lenses keeps a schema and data types for each topic. When a topic is detected for the first time, Lenses tries to identify the type for the Key and the Value and work out the schema. If the auto-detection is incorrect, you can override the format.

To change the Schena Type:

  1. From the Header Bar Menu, go to the Dashboard panel
  2. On the side navigation, select Explore
  3. Select the topic you wish to edit
  4. Click on the Actions menu on the top right of your screen
  5. Select the Change Type (Key, Value) option

Increase partitions menu

  1. Change the Key or Value types.
Topic Schema

To change the schema:

If schema registry for Avro types exists, Lenses communicates directly with the schema registry to get the Avro schemas.

If the type is not Avro, Lenses tries to detect the schema for the Key and Value. Lenses stores this schema in its storage.

To edit the Lenses Schema:

  1. From the Header Bar Menu, go to the Dashboard panel
  2. On the side navigation, select Explore
  3. Select the topic you wish to edit
  4. Click on the Actions menu on the top right of your screen
  5. Select the Edit Schema option

Increase partitions menu

  1. Edit the schema as you see fit

Topic Schema Edit

Increasing partitions

Lenses provides the ability to increase the number of partitions per topic.

To increase partitions of a topic:

  1. From the Header Bar Menu, go to the Dashboard panel
  2. On the side navigation, select Explore
  3. Select the topic you wish to edit
  4. Click on the Actions menu on the top right of your screen
  5. Select the Add Partitions option

Increase partitions menu

Watch out! Increasing partitions is an irreversible action.

Fill in the new amount of partitions on the modal. Make sure you understand the side effects and confirm by checking the related checkbox underneath.

Initiate the action by clicking the “Save Partitions” button.

Increase partitions

Deleting topics

To Delete a Topic

  1. From the Header Bar Menu, go to the Dashboard panel.
  2. On the side navigation, select Explore under the data section.
  3. Click the trash icon on the topic you want to be deleted.
Kafka deletes topics asynchronously. For a few moments you will be able to observe the topic, indicating that the topics have been marked for deletion.
Indication that Topic is marked for Deletion

Using SQL

Lenses SQL supports managing Topics via SQL statements, CREATE TABLE, DESCRIBE TABLE, DROP TABLE etc. Here is an example:

CREATE TABLE customer (id string, address.line string, address.city string, address.postcode int, email string)
FORMAT (string, json)
PROPERTIES (partitions=1, compacted=true)

From the topic detail page, you can explore data in table browsing mode, or sample a data via live stream.

See more on SQL docs

Promoting topics with the Lenses CLI

Lenses CLI also supports all managing Topics actions. Here is an example:

$ lenses-cli topic create \
    --name="topicName" \
    --replication=1 \
    --partitions=1 \
    --configs="{\"max.message.bytes\": \"1000010\"}"

Lenses CLI can also be used to promote Topics from one cluster to another, using YAML config files and GitOps:

$ lenses-cli topic update ./topic.yaml

Automation

For automation use the API and CLI