3.4 Create a Kafka Consumer

In the same way we created our REST service, we can create our Kafka consumer application from git. Then, we’ll add our Kafka connection using our secret just like we did before.

Add an Application from Git

Before you begin, make sure you’re in the developer view and in the right project.

alt text
alt text

  • From the +Add menu, click on the From Git tile:

alt text

Configure the Application

Set the Git Repo

  • In the Git Repo URL field, enter:

https://github.com/blues-man/object-detection-kafka-consumer
alt text

Set the App Name

  • By default, our new deployment will be added to the existing application object-detection we created during our previous app. We can leave that as is. We can change the Name to object-detection-kafka-consumer if desired. Leave the default resource as Deployment.

alt text

Remove the Route

The Kafka consumer does not need to be exposed externally and does not serve any information.

  • Under Advanced Options, uncheck Create a route to the Application.

alt text

Resource Limits

We’ll have to increase the resource limits for the TensorFlow model.

  • Click on Resource Limits

Click resource limits

  • Enter in the following values

    • CPU

      • Request: 500 millicores

      • Limit: 1000 millicores (or 1 cores)

    • Memory

      • Request: 2200 Mi

      • Limit: 3000 Mi

Create app limits form

Create

  • Scroll down and click on Create:

alt text

Your new Kafka consumer should be created and is currently building.

alt text

Add the Secret to the Kafka Consumer

While the application is building, we can add the secret to the deployment. Then it can connect to the Kafka instance and topic and process messages.

  • Navigate to Secrets

  • Find and Select the secret with your Kafka connection information.

alt text

  • Click Add Secret to workload

  • Select object-detection-kafka-consumer deployment as the workload.

  • Click Save

alt text

You’ll be redirected to your application. It will redeploy a new pod to pick up the changes. You can examine the Environment tab to confirm it was added.

Testing your Streaming Application

Once the application is built and the front-end application and kafka consumer are sending and receiving messages, you can test out your application.

  • Navigate to the Topology for your project.

  • Select your front-end application deployment

  • Click on the Route at the bottom of the detail page

alt text

  • Select the video icon on the application.

alt text

Try out your application by using the webcam. You should see the results of the object detection model and the Kafka consumer should be receiving messages in real-time! Pretty neat!