Free Code

Apache Kafka 2.6 Installation on Ubuntu 20.04

How to install Apache Kafka 2.6 Installation on Ubuntu 20.04

What is Apache Kafka

Apache Kafka is an open-source distributed event streaming platform that lets you read, write, store, and process events (also called records or messages in the documentation) across many machines.

  • Step 1:  Check your java Installation

Java must be installed for the installation of Kafka. if Java is not installed first install java then Kafka.  to check the version and installation of Java, please run the cmd on the terminal java –version

if you got the above message after running the command. Java is installed in your system.

 

  • step 2: Download Kafka

Open a browser and Goto the Kafka Download  link

 

  • Step 3: From the above download link. click on the Source download Link

 

 

if you are downloading in the ubuntu terminal run the  below command

# downloading Apache Kafka 2.6 version

wget https://mirrors.estointernet.in/apache/kafka/2.6.0/kafka-2.6.0-src.tgz

Untar(Unzip ) the download Kafka file

 # untarring the downloaded file
tar -xzvf kafka-2.6.0-src.tgz

 

  • Step 4: Download Zookeeper

Open a browser and Goto the Zookeeper Download  link

 

 

  • Step 5: Download Zookeeper with version 3.5.8 (for Kafka 2.6  )
# Download tar file of zookeeper with 3.5 version
wget https://mirrors.estointernet.in/apache/zookeeper/zookeeper-3.5.8/apache-zookeeper-3.5.8.tar.gz

Untar the zookeeper tar file

tar -xzvf apache-zookeeper-3.5.8.tar.gz
  • Step 6: Create a Zookeeper Conf file
Copy the existing zookeeper sample file to zoo.conf
cp  zoo_sample.cfg zoo.cfg

Add or change the below configuration

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgment
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/var/zookeeper
# the port at which the clients will connect
clientPort=2181
  • Step  7: Configure Kafka Server Properties

Add or update the parameter for the server. properties file

vi server.properties
# The number of threads that the server uses for receiving requests from the network and sending responses to the network
num.network.threads=3

# The number of threads that the server uses for processing requests, which may include disk I/O
num.io.threads=8

# The send buffer (SO_SNDBUF) used by the socket server
socket.send.buffer.bytes=102400

# The receive buffer (SO_RCVBUF) used by the socket server
socket.receive.buffer.bytes=102400

# The maximum size of a request that the socket server will accept (protection against OOM)
socket.request.max.bytes=104857600


############################# Log Basics #############################

# A comma separated list of directories under which to store log files
log.dirs=/var/kafka-logs

# The default number of log partitions per topic. More partitions allow greater
# parallelism for consumption, but this will also result in more files across
# the brokers.
num.partitions=1

 

Start the Zookeeper Server 

#Start the zookeeper server

#Go to the kafka folder and start the server 

bin/zookeeper-server-start.sh

Start the Kafka Server

#Start the Kafka server 

#Go to the kafka folder and start the server 

bin/kafka-server-start.sh config/server.properties