Register Custom Apps

Register your applications with Lenses in two ways;

• Via the HTTP endpoint
• Topology Client for JVM applications.

Watch this video to learn how to register your application using the HTTP endpoint and view it in the Topology. Register Your Custom Apps to Lenses Topology

App state and Runners

Each app can be registered with a set of endpoints that return the status of each app’s running instances. These endpoints are called runners and allow Lenses to get the running instances’ status by pinging these endpoints. The app state is being consolidated based on the individual runner’s state, which we periodically check.

Each runner can be only:

• RUNNING - When the health-check endpoints return 200 HTTP Status Code
• UNKNOWN - When the health-check endpoints return anything other than 200 HTTP Status Code

Lenses grabs and consolidates the individual runners’ statuses to a single app state as it follows:

• RUNNING - All Runners are RUNNING
• WARNING - At least 1 Runner is UNKWOWN
• UNKNOWN - No information about Runners, either they are unreachable

To be able to perform any of the actions referenced below, you will need a User with ManageTopology permission. For more information about each API Endpoint, please visit our API Docs

Register Apps with via HTTP

To add your external apps to Lenses, you can currently use the API by doing a POST to api/v1/apps/external endpoint.

curl -X POST \
  <LENSES_HOST_URL>/api/v1/apps/external \
  -H 'Content-Type: application/json' \
  -H 'X-Kafka-Lenses-Token: <LENSES_AUTH_TOKEN>' \
  -d '{
    "name": "Example_Producer_App",
    "metadata": {
        "version": "1.0.0",
        "owner": "Lenses",
        "deployment": "K8s",
        "tags": [
            "fraud",
            "detection",
            "app"
        ]
    },
    "input": [{"name": "fraud_input_topic"}],
    "output": [{"name": "fraud_output_topic"}],
    "runners": [{"url": "<YOUR_HEALTH_CHECK_URL_1>", "name": "Example_Runner"}]
}'

Update Runners with via HTTP

You can increase the number of Runners, an App is attached to, by hitting the appropriate Endpoints. Just do a POST request to api/v1/apps/external endpoint. The body of the new request should be the same as the first one except for the runners key. What needs to be changed is only the array of the runners key

curl -X POST \
  <LENSES_HOST_URL>/api/v1/apps/external \
  -H 'Content-Type: application/json' \
  -H 'X-Kafka-Lenses-Token: <LENSES_AUTH_TOKEN>' \
  -d '{
    "name": "Example_Producer_App",
    "metadata": {
        "version": "1.0.0",
        "owner": "Lenses",
        "deployment": "K8s",
        "tags": [
            "fraud",
            "detection",
            "app"
        ]
    },
    "input": [{"name": "fraud_input_topic"}],
    "output": [{"name": "fraud_output_topic"}],
    "runners": [{"url": "<YOUR_HEALTH_CHECK_URL_2>", "name": "Example_Runner"}]
}'

Remove Runners via HTTP

You can also reduce the number of Runners an App is attached to. Just do a DELETE request to /api/v1/apps/external/{name}/runners.

curl -X DELETE <LENSES_HOST_URL>/api/v1/apps/external/Example_Producer_App/runners \
-H 'Content-Type: application/json' \
-H 'X-Kafka-Lenses-Token: <LENSES_AUTH_TOKEN>' \
-d '{ "runners":["<YOUR_HEALTH_CHECK_URL_1>", "<YOUR_HEALTH_CHECK_URL_2>"]}'

Remove your application via HTTP

You can remove the app by making a DELETE request to /api/v1/apps/external/{name}. By Removing an App from Lenses, you just drop Lenses’ visibility for that specific App.

curl -X DELETE <LENSES_HOST_URL>/api/v1/apps/external/Example_Producer_App \
-H 'Content-Type: application/json' \
-H 'X-Kafka-Lenses-Token: <LENSES_AUTH_TOKEN>'

Alternatively, you can use the Lenses UI to remove the App:

Register applications via the Topology client

Lenses also supports metrics for Kafka Streams, Akka Streams, and Spark Structured Streaming. You can find full working examples on GitHub . To add your external apps to Lenses Topology through the topology client, instrument your code using the Lenses client library.

The open-source topology-client provides a simple builder for topologies, which is then pushed to a Kafka topic to be picked up by Lenses at runtime.

Apps listing

In the application listing page, you can see a table with all of your applications, and a set of information such as State, Runners, Tags and even Deployment information.

In addition to this, the same information can be visualized in the Topology page, which allows you to visualize data movements between applications and input/output topics. More information can be seen about each application by clicking the application node.

Apps details

In the application details page, you can see additional information about each application, and also information about each runner, along with metadata information, such as Description, and auditing information such as Created At, Created By and Modified At, Modified By.

In the example, below, we have registered 2 Apps. One of them with 3 Runners, all of which are in Running state, and another one, with 2 Runners, one of whom are in Unknown State, thus making the state of the application WARNING.

On top of that, we are visualising your input and output data sources, that your application is using. You can see it by clicking the Data Flow tab.

There are two types of nodes, the ‘datasets’ and the ‘apps’. The datasets nodes are representing the input and output sources and provide link to the related dataset. The apps nodes provide information about the health status along with the application type