Upgrading SQL processors to a newer SQL Connector plugin version.

We recommend to match the Lenses version with the SQL connector plugin version. In case you are upgrading Lenses to a newer version, the most probable occurrence is that you will have already some SQL processors (via Connect) running. In such case, we recommend to follow these steps for the upgrade.

Generic Script

  • Identify the Connect clusters that need upgrading.
  • For each of them, stop temporarily all SQL Processors. Please, note down what processors were running to start them back.
  • Update the Connector plugin on each Connect cluster.
  • Proceed with the upgrade to Lenses.
  • Upgrade to the latest CLI.
  • Once Lenses is upgraded and has successfully started, restart all processors that were previously stopped

Example

Let’s see an example of the above script using Lenses CLI considering we are using Lenses box, and the configured paths in such environment:

List existing running processors

lenses-cli processors | grep -iw 'RUNNING\|FAILED\|PENDING' | \
 awk '{print "id=" $1 " cluster-name=" $15}' \
 > processors.bk

Stop each running SQLProcessor

while read processor; \
  do lenses-cli processor stop $(echo "$processor" | \
  awk '{print "--"$1 " --"$2}'); done \
  < processors.bk

Upgrade SQL connector plugin

  1. Plugins are added under the plugin.path of each worker. Each plugin should be in its own subdirectory.

    As an example, the Connect worker’s configuration may include the line below, which sets the plugin.path to /usr/share/connectors. This path contains a folder for the lenses-sql-streaming plugin.

    plugin.path=/usr/share/connectors
    
  2. Remove that /usr/share/connectors/lenses-sql-streaming folder

  3. To install the new SQL connector, download the connector archive (lenses-sql-connect-vX.Y.Z.tar.gz) from the client area , extract it, and copy all files under the connector directory to /usr/share/connectors/lenses-sql-streaming.

    Further instructions to install a new Connector plugin can be found here .

    mkdir -p /usr/share/connectors/lenses-sql-streaming
    tar -xzf lenses-sql-connect.tar.gz \
      -C /usr/share/connectors/lenses-sql-streaming \
      --wildcards */connector/* --strip-components=2
    
  4. Start up the workers.

Proceed with the upgrade to Lenses and latest CLI

Start back previously running SQL Processors

while read processor; \
  do lenses-cli processor start $(echo "$processor" | \
  awk '{print "--"$1 " --"$2}'); done \
  < processors.bk