Data Types & Schemas

For every Kafka Topic, Lenses tracks and maintains a schema. The Kafka Topic records consist of a Key and a Value and, in some cases, a Header. For Keys and Values, Lenses assigns a type format.

Topic Schemas

Supported types

Lenses auto-detects and keeps a schema and data formats for each topic. When a topic is detected for the first time, Lenses tries to identify the type for the Key and the Value as well as work out the schema.

The supported formats for Key and Value are:

  • JSON, AVRO, XML, CSV, PROTOBUF, STRING, INT, LONG, BYTES (default), Custom pluggable formats.

Update types

To update the schema type and Lenses schema the user will require the appropriate Namespace permission.

To update the Schema Types:

  1. From the Header Bar Menu, go to the Dashboard panel
  2. On the side navigation, select Explore
  3. Select the topic you wish to edit
  4. Click on the Actions menu on the top right of your screen
  5. Select the Change Type (Key, Value) option

Increase partitions menu

  1. Change the Key and / or Value types.
Topic Schema

Update schema

To update the schema:

If schema registry for Avro types exists, Lenses communicates directly with the schema registry to get the Avro schemas.

If the type is not Avro, Lenses is trying to detect the schema for the Key and Value. Lenses stores this schema in its storage.

To edit the Lenses Schema:

  1. From the Header Bar Menu, go to the Dashboard panel
  2. On the side navigation, select Topics under Schema tab
  3. Select the topic you wish to edit
  4. Click on the Actions menu on the top right of your screen
  5. Select the Edit Schema option

Increase partitions menu

  1. Edit the schema as you see fit

Topic Schema Edit

Custom types

Lenses supports custom serializers to support custom data formats.

See Docs for custom formats