Deploy Apache Kafka

This guide covers launching and using Apache Kafka from AWS Marketplace.

Prerequisites

• An AWS account with permission to subscribe to Marketplace products and launch EC2 instances.
• An EC2 key pair for administrative access.

Step 1 — Subscribe & launch

1. On the AWS Marketplace listing, choose Continue to Subscribe → Continue to Configuration → Continue to Launch.
2. Select an instance type and your key pair.
3. Configure the security group (see the ports below) and launch.

Step 2 — Security group / ports

Open the following ports (restrict SSH/RDP to your own IP):

Port	Protocol	Purpose
2181	TCP	ZooKeeper
9092	TCP	Apache Kafka broker

Step 3 — Connect

ssh -i "your-key.pem" ubuntu@<PUBLIC_IP>

If you get a key-permission error: chmod 400 your-key.pem.

Step 4 — Access the application

Apache Kafka provides:

• Allows data to be distributed across multiple servers, making it scalable beyond what would fit on a single server.
• Kafka decouples data streams so there is very low latency, making it extremely fast
• Partitions are distributed and replicated across many servers, and the data is all written to disk.
• Topics are automatically replicated, but the user can manually configure topics to not be replicated

Step 1. Enable Ports on Instance

Enable the ports 2181 and 9092 port on the instance

Step 2. Start the Apache Kafka Server

Perform the below steps to enable the Apache Kafka on the server,

sudo systemctl status kafka
sudo systemctl status zookeeper

Now run the below command to process the producer in Apache Kafka,

~/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic testTopic

Open another terminal on the same path and run the below command to process the consumer in Apache Kafka

~/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testTopic
~/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testTopic --from-beginning

Screenshots

Day-2 operations

Apply OS and package security updates on your normal cadence, snapshot the EBS volume for recovery, and forward logs/metrics to CloudWatch.

Support

Email support@ioanyt.com for help.