ssl not enabled in terafoundation_kafka_connector

[jeffmontagna]

• The package terafoundation_kafka_connector installs node-rdkafka, and ssl will not be enabled if the build image does not include libssl-dev.
• The asset will not give a warning that ssl is not enabled in node-rdkafka when registering a job uses ssl.
• The error on the teraslice worker is: SSL handshake failed

message: [thrd:ssl://127.0.0.1:9099/bootstrap]: ssl://127.0.0.1:9099/bootstrap: SSL handshake failed: No further error information available (after 40ms in state SSL_HANDSHAKE, 1 identical error(s) suppressed)

• The error on the kafka broker is:

Failed authentication with /127.0.0.1 (channelId=127.0.0.1:9099-127.0.0.1:5817-5489) (SSL handshake failed)

• To enabled ssl, installation of libssl-dev should added to terascope/workflows used by kafka-assets.

• asset-build-and-publish.yml#L27 add run: sudo apt-get -y install libssl-dev before line 27

jobs:
  asset-build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        # NOTE: Hard Coded Node Version array, should match array in test-asset.yml
        node-version: [14.21.3, 16.20.1, 18.16.1]
    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
      - run: sudo apt-get -y install libssl-dev
      - run: yarn setup
      - run: yarn build
      - run: yarn global add teraslice-cli
      - run: teraslice-cli -v
      - run: teraslice-cli assets build
      - run: ls -l ./build/
      - run: sha256sum ./build/*
      - name: Archive production artifacts
        uses: actions/upload-artifact@v3
        with:
          name: teraslice-asset
          path: ./build

• asset-test.yml#L26 add run: sudo apt-get -y install libssl-dev before line 26

jobs:
 test-linux:
   runs-on: ubuntu-latest
   strategy:
     matrix:
       # NOTE: Hard Coded Node Version array, should match array in build-and-publish-asset.yml
       node-version: [14.21.3, 16.20.1, 18.16.1]
   steps:
     - uses: actions/checkout@v3
     - name: Use Node.js ${{ matrix.node-version }}
       uses: actions/setup-node@v3
       with:
         node-version: ${{ matrix.node-version }}
     # we login to docker to avoid docker pull limit rates
     # secrets are now duplicated for Actions and Dependabot
     - name: Login to Docker Hub
       uses: docker/login-action@v2
       with:
         username: ${{ secrets.DOCKER_USERNAME }}
         password: ${{ secrets.DOCKER_PASSWORD }}
     - run: sudo apt-get -y install libssl-dev
     - run: yarn setup
     - run: yarn lint
     - run: yarn test:all
     - run: yarn global add teraslice-cli
     - run: teraslice-cli -v
     - run: teraslice-cli assets build
     - run: ls -l build/

• Node one liner to check which features were enabled when installing node-rdkafka

$ node -e "const Kafka = require('node-rdkafka');console.log(Kafka.features);"
[
  'gzip',
  'snappy',
  'ssl',
  'sasl',
  'regex',
  'lz4',
  'sasl_gssapi',
  'sasl_plain',
  'sasl_scram',
  'plugins',
  'zstd',
  'sasl_oauthbearer'
]

[godber]

Thanks Jeff, I think the libssl-dev package is present on the ubuntu-latest image:

https://github.com/godber/asg-yarn-npm-test/actions/runs/5981065504/job/16228310557?pr=13

libssl-dev is already the newest version (3.0.2-0ubuntu1.10).

So I suspect there is more to it. Maybe some environment variables.