Simplified Kafka streaming to run from Windows command prompt

-
After doing little research from my end, I did simplify the Kafka streaming process to work on Windows command prompt easily. The prerequisite for this simple exercise is like we need Apache Kafka software and a Windows 10 machine with full file/folder access permissions to run the Kafka commands through out the exercise. We can extend this work to next level by adding the security using SSL and we need the jks files to Kafka streaming process by using a properties file (eg:client-ssl.properties) while using producer/consumer commands successfully. Of course, these are not covered in this post but usually that is how most of the organizations used to follow this approach while working on Kafka streaming.

Below are the simple steps to follow on this simple exercise:

 1) Download the Apache Kafka => https://www.apache.org/dyn/closer.cgi?path=/kafka/2.4.0/kafka_2.12-2.4.0.tgz

2) Extract the Kafka into C:\Kafka and the structure would be like this way











3) Update the server.properties and zookeeper.properties to copy all the logs in C:\Kafka\logs and C:\Kafka\Zookeeperlogs directories to troubleshoot









4) Run the below commands from multiple Windows command prompts


Start Zookeeper in one command prompt instance:
D:>C:\Kafka\bin\windows\zookeeper-server-start.bat C:\Kafka\config\zookeeper.properties


Start Kafka Cluster in other command prompt instance:
D:>C:\Kafka\bin\windows\kafka-server-start.bat C:Kafka\config\server.properties 

 Create Kafka Topic in other command prompt instance:
D:>C:\Kafka\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic TEST

Check the configuration of a particular topic:
D:>C:\Kafka\bin\windows\kafka-topics.bat --describe --topic TEST --zookeeper localhost:2181

Alter the configuration of a topic:
D:>C:\Kafka\bin\windows\kafka-topics.bat --zookeeper localhost:2181 --alter --topic TEST --partitions 4

Create a Kafka Console Producer in other command prompt instance:
D:>C:\Kafka\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic TEST

 Create a Kafka Console Consumer in other command prompt instance:
D:>C:\Kafka\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic TEST --from-beginning

References:
1) Google.com
2) apache.org

Popular posts from this blog

Connecting Claude to Pega Infinity 25.1.3 via MCP — Step-by-Step

-
Image
Enable AI-powered access to your Pega Investigative Case Management data directly from Claude prompts using the Model Context Protocol (MCP) and OAuth 2.0. What is MCP and why connect Pega to Claude? The Model Context Protocol (MCP) is an open standard that lets AI assistants like Claude securely call tools and read data from external systems — all from a natural-language prompt. Think of it as a universal adapter between your enterprise systems and AI. Pega Infinity V25.1.3 ships with a first-class MCP service built on its existing REST framework. Once enabled, Claude can: Query and summarize investigative cases without switching apps Retrieve assignment details and case status on demand Trigger Pega workflows from conversational prompts Version requirement: This guide is based on Pega Infinity V25.1.3 (Community Edition) with the Rule-Service-MCP rule type. Earlier versions may not include native MCP support. Architecture overview The connection uses three layers: Claude as the AI c...
Read more

itextpdf API to generate PDF doc from an image file using Pega PE

-
Image
Probably some of the techies might have an idea on an opensource called " itextpdf " (earlier name is lowagie ) to convert an image file(png,jpeg,jpg,tiff,gif etc.) to a PDF file. It is very simple to use this API in Pega function to build the logic using java code. Moreover, this API comes in one single jar file and we need to import it in Pega before we start using it.  I see the conversion comes good in PDF file but not sure if any limitations of this API in terms of conversion is like formatting, font, images etc. So far so good and may be we can keep this API as one of our solutions while working on the requirements accordingly.  Heard that same API can be used to convert the text file to PDF but I didn't explore on it. Those who are interested to work on text -> PDF can take a look at the documentation. Don't hesitate to contact me offline for any further discussion in this context :) :) Prerequisites for doing this exercise: 1) itextpdf 5.4.0 jar file 2) Peg...
Read more

Understanding of Hugging Face platform for AI/ML platform

-
Image
Hugging Face is a platform that provides tools and libraries for natural language processing (NLP), machine learning, and artificial intelligence. It is known for its Transformers   library, which offers pre-trained models for tasks such as text classification, translation, question answering, and more. Hugging Face also hosts a model hub where users can share and discover pre-trained models. Hugging Face is useful in various domains, including: Natural language processing (NLP): Provides tools and pre-trained models for tasks like text classification, translation, sentiment analysis, and question answering. Machine learning research: Offers state-of-the-art models and datasets for researchers to experiment and innovate. Data science and analytics: Helps data scientists analyze and interpret large volumes of text data. AI application development: Enables developers to build AI-powered applications with ease using pre-trained models and APIs. A very simple exercise using the google...
Read more