Manage your Topics

Lenses allows users to create, edit and delete Topics. Each action is protected by Namespace permissions. Every time an action is taken Lenses audits the event.

Topic Concepts

Topics in Kafka consist of 3 basic fields and the topic configurations:

  • Name, the unique name of the topic.
  • Partitions, the number of partitions the topic will be divided into.
  • Replication, the replication factor for each partition across the cluster.

All topic configurations have default values. Important configs include: Cleanup policy, Retention size and Retention time

Create Topic

To create a new Topic:

  1. From the Header Bar Menu, go to Dashboard panel.
  2. On the side navigation, select Topics under Data section.
  3. Click the New Topic button and fill in the form.

Topic Create

Edit Topic Configuration

Topic configurations are Key-Value pairs. Lenses supports all Topic Configs and also provides autocompletion and the documentation for each Key.

To Edit and View the Topic Configuration:

  1. From the Header Bar Menu, go to Dashboard panel.
  2. On the side navigation, select Topics under Data section.
  3. Click on the Topic Name from the list and navigate to the Config tab.

Topic Config

Edit Metadata & Schema

Lenses keeps a schema and data types for each Topic. When a topic is detected for the first time, Lenses tries to identify the type for the Key and the Value as well as work out the schema. If the auto-detection is incorrect, you can override the format.

Topic Schema

To change the Schena Type:

  • 1. From the Header Bar Menu, go to Dashboard panel
  • 2. On the side navigation, select Topics under Schema tab
  • 3. Select the Change Type option.
  • 4. Change the Key or Valye types.

To change the schema:

If schema registry for Avro types exists, Lenses communicates directly with the schema registry to get the Avro schemas.

If the type is not Avro, Lenses tries to detect the schema for the Key and Value. Lenses stores this schema in its tablestore.

To edit the Lenses Schema:

  • 1. From the Header Bar Menu, go to Dashboard panel
  • 2. On the side navigation, select Topics under Schema tab
  • 3. Select the Edit Schema option.

Topic Schema Edit

Increase Partitions

Lenses provides the ability to increase the amount of partitions per topic.

Partitions can be increased by selecting the relative action from the actions menu.

Increase partitions menu

Watch out! Increasing partitions is an irreversable action.

Fill in the new amount of partitions on the modal. Make sure you understand the side effects and confirm by checking the related checkbox underneath.

Initiate the action, by clicking the “Save Partitions” button.

Increase partitions fprm

Increase Partitions

Lenses provides the ability to increase the amount of partitions per topic.

Partitions can be increased by selecting the relative action from the actions menu.

Increase partitions menu

Watch out! Increasing partitions is an irreversable action.

Fill in the new amount of partitions on the modal. Make sure you understand the side effects and confirm by checking the related checkbox underneath.

Initiate the action, by clicking the “Save Partitions” button.

Increase partitions fprm

Delete Topic

To Delete a Topic

  1. From the Header Bar Menu, go to Dashboard panel.
  2. On the side navigation, select Topics under Data section.
  3. Click the trash icon on the topic you want to be deleted.
Kafka deletes topics asynchronously. For a few moments you will be able to observe the topic, with an indication that the topics has been marked for deletion.

Indication that Topic is marked for Deletion

Use SQL

Lenses SQL supports managing Topics via SQL statements, CREATE TABLE, DESCRIBE TABLE, DROP TABLE etc. Here is an example:

CREATE TABLE customer (id string, address.line string, address.city string, address.postcode int, email string)
FORMAT (string, json)
PROPERTIES (partitions=1, compacted=true)

From the topic detail page you can explore data in table browsing mode, or sample a data via live stream.

See more on SQL docs

Promote Topics and Lenses CLI

Lenses CLI also supports all managing Topics actions. Here is an example:

$ lenses-cli topic create \
    --name="topicName" \
    --replication=1 \
    --partitions=1 \
    --configs="{\"max.message.bytes\": \"1000010\"}"

Lenses CLI can also be used to promote Topics from one cluster to another, using YAML config files and GitOps:

$ lenses-cli topic update ./topic.yaml

Automation

For automation use the API and CLI