• Linkedin
  • Github
  • Facebook
  • Twitter
  • Google+

About me

Let me introduce myself


A bit about me

Hello friends, I am Dhananjaya Naidu and I’m just like you; a java lover. I feel proud to say that I am a Java Developer and currently located in Bangalore, India.

I enjoy coding in full Java/J2ee stack (Spring, JSF, Hibernate, Struts, Servlets, JSP) and Web Technologies (HTML, CSS, JavaScript, JQuery).


I love to play Cricket, Kabaddi, Valley Ball and any Outdoor Sports. I love the nature and like to travel new places.

Profile

Dhananjaya Naidu

Personal info

Dhananjaya Naidu Reddi

Be good, Do good !!!

Birthday: 19 Jun 1988
Website: www.rdnaidu.com
E-mail: hello@rdnaidu.com

Skills & Things

Productive
90%
Java & J2ee
Creative
70%
HTML & CSS
Progressive
50%
Blogger

Notes

My latest writings


Monday, 3 December 2018

Best tips to be a Better Java Developer

If you are a Java developer in early phases of your career, here are some concepts you should read about and learn to be an outstanding Java developer.

All books have been written about these concepts. We will just summarise them here and encourage you to find and read more about them.

1. Thread Safety
All web-applications are multi-threaded applications. If you didn’t know, read about how application containers e.g. Tomcat start a new thread or pick one from a thread pool, to render a new web request. Thread safety is about ensuring access to shared resources is serializable. The most common thread safety mistake? Declaring a private field on a singleton class in your application which is changed by multiple threads.

2. Functional Programming
This came late to Java, specifically in version 1.8. But you should learn and use Streams & Lambda functions in your programs. They make for readable, elegant & concise code, often with improved performance.

3. Thread Local variables
Thread safety was about accessing shared variables safely. Thread local allows you to do just that. Think of them as a HashMap of variables by thread id. However, handle their initialization and clean up with care, especially when using thread pools or you can get into thread safety issues and memory leaks.

4. Mapped Diagnostic Contexts in Logging
Underlying, this works as a thread local variable. But you don’t have to worry about it because most logging frameworks will support it out of the box. The idea is to start and end contexts when logging messages. For instance, at the entry of a request, you can log the API URL and user id, so the logs inside would always carry that context.

5. Understand ClasspathIt 
seems easy. But can cause problems or be the reason for some weird effects such as – Why does this work on my local but not on production? It may be because of class-path ordering wherein a production application container loaders an older jar from a shared class-path.

6. Class-path & Class Loaders
Class loaders work as a hierarchy. Learn about class-loaders that come with running an application in tomcat. Find if the same class can be loaded twice in memory and how.

7. Hot Loading
Did you know JVM supports hot reloading of Java classes? You change the code and then don’t have to re-compile everything and restart the application. Just the class changed is recompiled and its in-memory byte-code is updated. Find which cases this can happen and configure it in your development environment if you haven’t already.

8. Dependency Injection
You don’t call me. I call you. That’s inversion of control and the principle behind dependency injection, popularised by Spring Framework. You already knew it, but do you know the alternative to dependency injection? Learn and understand why dependency injection is better and leads to more extensible and testable code.

9. Properties
You know about properties files. But what are the principles to decide if something should be a system property, an environment variable or a command line argument? The hint is in runtime i.e. when does the property become available and is provided. Nevertheless, you should know how to accept and use each and override when needed e.g. in a test environment.

10. Fat Jar Deployment
Most application frameworks, spring boot comes to mind, have an embedded container deployment mode. That is a DevOps friendly way to deploy on the cloud, instead of tweaking application container settings. Even if there is an operations team managing deployment for you, learn it as its the way forward.

11. ORM & Persistence
ORM (JPA & Hibernate) give you convenience and power. And often people shoot themselves in their leg with it. Learn to use them responsibly and without performance impact. Transaction boundaries, Lazy vs Eager fetching, different types of caching and how to debug performance. But most importantly, when not to use an ORM and just write a SQL query. Hint – when the amount of data is large or is distributed in too many tables.

12. Fault Tolerance and Isolation
Let’s say one of your application API calls another rest API. If the other rest API becomes slow, will your application go down? It will. Because of the slow API, all threads will be consumed and your application will become unresponsive. So design it to be fault tolerant with proper timeouts. You can make it more robust by using Resilience4j that allows setting up retry & fallback, circuit breakers, rate limiters and more.

13. Deployment and run-time environment
Do you know how your application works in production? Whether its a Paas like Heroku or Iaas like AWS, you should know the memory available, available CPUs, disk latency, network latency within different components to design a performant application.

14. Monitoring
Will you know if some API in your application is slow? What are the key performance indicators you should monitor? You can use an APM provider like New Relic, or build your own health checks and metrics using spring boot actuator.

15. Build Lifecycle and Dependency Management
Can you configure a new java project by hand using Maven or Gradle? Setup development and production environment configuration, create build lifecycles tasks to run checkstyle or find bugs, create build commands to run test-cases and generate coverage and so on.

16. Debugging Setup
Many developers do not know how to set this up.  This is a must when running test-cases, but JVM also allows remote debugging by opening up a port your IDE can connect to. Learn how to set it up and you will be finding bugs 10 times faster than your teammates who don’t have it set up.

17. Know your IDE
Whether its Eclipse or IntelliJ, shame on you if you can’t set up a new or existing project in your IDE. The ideas IDE setup will have static code analyser, run and debug test environment, can connect to debug a running JVM with has libraries source code also downloaded. A good workman knows their tools.

18. Learn to Refactor SensiblyMost code is not perfect. Learn to improve some of it as you go, but not all of it in one go. Unlike the others, this one will take time and experience to master. Start today.

©hashedin.com

Friday, 2 November 2018

Spring RESTful Web service to generate downloadable Excel as response using JExcel API and Apache POI

JExcelApi is a Java library that is dedicated for reading, writing and modifying Excel spreadsheets. It supports Excel 2003 file format and older versions. You can download JExcelApi from the following link:

To work with JExcelApi, you need to add its only jar file: jxl.jar - to your project’s classpath. Download JExcel

Here we are going to create spring API to download Excel file as response using JExcel Api.

import javax.servlet.http.HttpServletResponse;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.ModelAndView;

import com.demo.api.exl.services.ExcelOutputService;

@RestController
@RequestMapping(value="/exceloutput")
public class ExcelOutputServiceController {

    @Autowired
    ExcelOutputService excelOutputService;
   
    @RequestMapping(value="/download", method=RequestMethod.GET)
    public ModelAndView downloadExcelOutputExl(HttpServletResponse response){
       
       excelOutputService.createExcelOutputExcel(response);
       return null;
    }
}


import javax.servlet.http.HttpServletResponse;

import jxl.Workbook;
import jxl.write.Label;
import jxl.write.WritableSheet;
import jxl.write.WritableWorkbook;
import jxl.write.WriteException;
import jxl.write.biff.RowsExceededException;

import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;


@Service("excelOutputService")
public class ExcelOutputServiceImpl implements ExcelOutputService{
    private static final Logger LOGGER = Logger.getLogger(ExcelOutputServiceImpl.class);
   
    @Override
    public WritableWorkbook createExcelOutputExcel(HttpServletResponse response) {
       String fileName = "Excel_Output.xls";
       WritableWorkbook writableWorkbook = null;
       try {
           response.setContentType("application/vnd.ms-excel");

           response.setHeader("Content-Disposition", "attachment; filename=" + fileName);

           writableWorkbook = Workbook.createWorkbook(response.getOutputStream());
           
           WritableSheet excelOutputsheet = writableWorkbook.createSheet("Excel Output", 0);
           addExcelOutputHeader(excelOutputsheet);
           writeExcelOutputData(excelOutputsheet);
            
           writableWorkbook.write();
           writableWorkbook.close();

       } catch (Exception e) {
           LOGGER.error("Error occured while creating Excel file", e); 
       }

       return writableWorkbook;
    }

    private void addExcelOutputHeader(WritableSheet sheet) throws RowsExceededException,  
       WriteException{ 
       // create header row
        sheet.addCell(new Label(0, 0, "Column 1"));
        sheet.addCell(new Label(1, 0, " Column 2"));
        sheet.addCell(new Label(2, 0, " Column 3"));
        sheet.addCell(new Label(3, 0, " Column 4"));
        sheet.addCell(new Label(4, 0, " Column 5"));
        sheet.addCell(new Label(5, 0, " Column 6"));
        sheet.addCell(new Label(6, 0, " Column 7"));
        sheet.addCell(new Label(7, 0, " Column 8"));
        sheet.addCell(new Label(8, 0, " Column 9"));
        sheet.addCell(new Label(9, 0, " Column 10"));
        sheet.addCell(new Label(10, 0, " Column 11"));
    }
   
    private void writeExcelOutputData(WritableSheet sheet) throws RowsExceededException, 
      WriteException{
              
       for(int rowNo = 1; rowNo<=10; rowNo++){
              sheet.addCell(new Label(0, rowNo, “Col Data ”+ (rowNo+0))); 
              sheet.addCell(new Label(1, rowNo, “Col Data ”+ (rowNo+1))); 
              sheet.addCell(new Label(2, rowNo, “Col Data ”+ (rowNo+2))); 
              sheet.addCell(new Label(3, rowNo, “Col Data ”+ (rowNo+3))); 
              sheet.addCell(new Label(4, rowNo, “Col Data ”+ (rowNo+4))); 
              sheet.addCell(new Label(5, rowNo, “Col Data ”+ (rowNo+5))); 
              sheet.addCell(new Label(6, rowNo, “Col Data ”+ (rowNo+6)));  
              sheet.addCell(new Label(7, rowNo, “Col Data ”+ (rowNo+7))); 
              sheet.addCell(new Label(8, rowNo, “Col Data ”+ (rowNo+8))); 
              sheet.addCell(new Label(9, rowNo, “Col Data ”+ (rowNo+9))); 
              sheet.addCell(new Label(10, rowNo, “Col Data ”+ (rowNo+10))); 

       }

    }
}

Now call the service as :
http://localhost:8080/demo/excelOutputService/download
This will download the Excel file to download folder or show download popup window.


Create Excel using Apache POI

POI-XSSF - Java API To Access Microsoft Excel Format Files
Apache POI is a popular API that allows programmers to create, modify, and display MS Office files using Java programs.
XSSF is the POI Project's pure Java implementation of the Excel 2007 OOXML (.xlsx) file format.

Prerequisites
· DK1.5 or later versions
· Apache POlibrary (http://poi.apache.org/download.html )

Add following library to classpath:
“C:\poi-3.9\poi-3.9-20121203.jar;”
“C:\poi-3.9\poi-ooxml-3.9-20121203.jar;”
“C:\poi-3.9\poi-ooxml-schemas-3.9-20121203.jar;”
“C:\poi-3.9\ooxml-lib\dom4j-1.6.1.jar;”
“C:\poi-3.9\ooxml-lib\xmlbeans-2.3.0.jar;.;”


import org.apache.log4j.Logger;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFCell;
import org.apache.poi.xssf.usermodel.XSSFCellStyle;
import org.apache.poi.xssf.usermodel.XSSFRow;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;

public XSSFWorkbook createExcelOutputFile(){
    XSSFWorkbook workbook = null;
    try {           
           
        ClassLoader loader = getClass().getClassLoader();
        File file = new File(loader.getResource("Excel_Output.xlsx").getFile());  //file should be at classpath
        FileInputStream is = new FileInputStream(file);
           
        // Get the workbook instance for XLSX file
        workbook = new XSSFWorkbook(is);
        XSSFSheet rankerSheet1 = workbook.getSheetAt(0);

        writeExcelOutputData(rankerSheet1, workbook);          

        is.close();
           
    } catch (FileNotFoundException e) {
            LOGGER.error(e); 
   } catch (IOException e) {
            LOGGER.error(e); 
   }
        return workbook;
}

private void writeExcelOutputData(XSSFSheet rankerSheet, XSSFWorkbook worksheet){
           
            XSSFRow row1 = rankerSheet.createRow(1);
            row1.createCell(0).setCellValue(1);
            row1.createCell(1).setCellValue(1234
            row1.createCell(2).setCellValue(“Test Excel”);
            row1.createCell(3).setCellValue(“Address”);
           
            XSSFCell cell4 = row1.createCell(4);
               cell4.setCellValue(10.00);
            XSSFCellStyle style = worksheet.createCellStyle();
            style.setDataFormat((short)8);      // this will format cell with $ sign Ex: $10.00
            cell4.setCellStyle(style);  
           
           
          //Creating Data format %
            XSSFCellStyle percentStyle = worksheet.createCellStyle();
            percentStyle.setDataFormat(worksheet.createDataFormat().getFormat("0.0%"));
           
            XSSFCell cell5 = row1.createCell(5);
            cell5.setCellStyle(percentStyle);       
            cell5.setCellValue(20.00);    


    }

References:
https://aboullaite.me/spring-boot-excel-csv-and-pdf-view-example/
http://simplejava2.blogspot.com/

Tuesday, 13 March 2018

SINGLETON Design Pattern: Ways of Implementation

As name says, at any time only one instance of a Singleton class must be present. All classes accessing the singleton class must use the same instance.

To make a class Singleton:

The constructor must be private, so no external class can create an instance using direct constructor.
So are we suppose to create an instance of this class ?
getInstance():

One way to get the instance of the Singleton class is by having a static final object of the class.
public static final MySingleton mInstance = new MySingleton ();

This is called early initialization as the singleton object is initialized when the class is loaded.
The singleton object can be created when actually required, to same resources, as shown below.

public class BadSingleton{
     
    private static BadSingleton mInstance;
     
    private BadSingleton() {
        
    }
     
    public static BadSingleton getInstance(){
 
        if(null == mInstance){
            mInstance = new BadSingleton();
        }
        return mInstance;
    }
}

This is called lazy initialization. The object instance is initialized only when required.

The above snippet works fine as long as multiple threads don’t call the getInstance() method, though we can not control which thread calls the method.The real problem starts when multiple threads request the instance. The above snippet will lead to Multiple Instances being created if called by multiple threads at the same time.

To fix this problem, we can synchronize the getInstance() method

public class BetterSingleton{
 
private static BetterSingleton mInstance;
 
   private BetterSingleton() {
 
   }
 
   public static synchronize  BetterSingleton getInstance(){
 
   if(null == mInstance){
      mInstance = new BetterSingleton();
   }
   return mInstance;
 }
}

But the above approach has performance issues. The time required to finish the getInstance() method increases due to synchronization. And moreover, once the instance is created, synchronization would be of no use, as the already created object can be returned directly. The Object instance will internally handle the synchronization within itself when the object is accessed in different threads.

So only the critical object creation part of code needs to be synchronized, as shown below.

public static BetterSingleton getInstance(){
 
  if(null == mInstance) {
     synchronized (BetterSingleton.class) {
     mInstance = new BetterSingleton();
   }
  }
return mInstance;
}

Above code snippet reduces the getInstance() method access time, since it synchronizes only while creating the object. Once the object is created, it is returned to the caller, without any synchronization.

But the above code can also lead to multiple instances of the Singleton class. If two threads are waiting at the “synchronized (BetterSingleton.class)” line, each will create its own instance of the object and return.

To solve this, we use “double-checked-locking“.

public class BestSingleton{
 
private static BestSingleton mInstance;
 
 private BestSingleton() {
 
 }
 
 public static BestSingleton getInstance(){
 if(null == mInstance) {
    synchronized (BestSingleton.class) {
       if(null == mInstance){
          mInstance = new BestSingleton();
       }
    }
  }
return mInstance;
}

Using double checked locking ensures the Singleton nature of the class.

Eclipse essential 30 Keyboard Shortcuts for Java Programmers

Here is list of 30 chosen Eclipse IDE keyboard shortcuts for Java developers.
It's useful for both core Java developer and Java web application developer using Eclipse IDE for web development.

1)   Ctrl + Shift + T for finding class even from jar
This keyboard shortcut in Eclipse is my most used and favorite shortcut. While working with a high-speed trading system which has a complex code, I often need to find classes with the just blink of the eye and this eclipse keyboard shortcut is just made for that. No matter whether you have class in your application or inside any JAR, this shortcut will find it.

2)   Ctrl + Shift + R for finding any resource (file) including config xml files
This is similar to above Eclipse shortcut with only difference that it can find out not only Java files but any files including XML, configs, and many others, but this eclipse shortcut only finds files from your workspace and doesn’t dig at jar level.

3)    Ctrl + 1 for quick fix
This is another beautiful Eclipse shortcut which can fix up any error for you in Eclipse. Whether it’s missing declaration, missing semi-colon or any import related error this eclipse shortcut will help you to quickly sort that out.

4)    Ctrl + Shift + o for organize imports
Another Eclipse keyboard shortcut for fixing missing imports. Particularly helpful if you copy some code from other file and what to import all dependencies.

Eclipse Shortcut for Quick Navigation

In this section, we will see some eclipse keyboard shortcut which helps to quickly navigate within the file and between file while reading and writing code in Eclipse.

7) Ctrl + o for quick outline going quickly to method
9) Alt + right and Alt + left for going back and forth while editing.
12) Alt + Shift + W for show in package explorer
13) Ctrl + Shift + Up and down for navigating from member to member (variables and methods)
15) Ctrl + k and Ctrl + Shift +K for find next/previous
24) Go to a type declaration: F3, This Eclipse shortcut is very useful to see function definition very quickly.

Eclipse Shortcut for Editing Code

These Eclipse shortcuts are very helpful for editing code in Eclipse.
5) Ctrl + / for commenting, uncommenting lines and blocks, see here for live example.
6) Ctrl + Shift + / for commenting, uncommenting lines with block comment, see here for example.
8) Selecting class and pressing F4 to see its Type hierarchy
10) Ctrl + F4 or Ctrl + w for closing current file
11) Ctrl+Shirt+W for closing all files.
14) Ctrl + l go to line
16) Select text and press Ctrl + Shift + F for formatting.
17) Ctrl + F for find, find/replace
18) Ctrl + D to delete a line
19) Ctrl + Q for going to last edited place

Miscellaneous Eclipse Shortcuts

These are different Eclipse keyboard shortcuts which doesn’t fit on any category but quite helpful and make life very easy while working in Eclipse.

20) Ctrl + T for toggling between supertype and subtype
21) Go to other open editors: Ctrl + E.
22) Move to one problem (i.e.: error, warning) to the next (or previous) in a file: Ctrl +. For next, and Ctrl +, for the previous problem
23) Hop back and forth through the files you have visited: Alt + ← and Alt + →, respectively.
25) CTRL+Shift+G, which searches the work-space for references to the selected method or variable
26) Ctrl+Shift+L to view listing for all Eclipse keyboard shortcuts.
27) Alt + Shift + j to add Javadoc at any place in java source file.
28) CTRL+SHIFT+P to find closing brace. Place the cursor at the opening brace and use this.
29) Alt+Shift+X, Q to run Ant build file using keyboard shortcuts in Eclipse.
30) Ctrl + Shift +F for Auto-formatting.

Here is the nice image to remember these useful Eclipse shortcuts for Java programmers:



Thursday, 21 December 2017

What is Kafka.? What is ZooKeeper.? Kafka Basic Commands...!!!

If you are new to Kafka this notes is a good starter.

What is Apache Kafka?

  • a publisher-based messaging system in the Hadoop ecosystem
  • each node is called a "broker"
  • written in scala and Java 1.6
  • queues are known as "topics"
  • Topics can be partitioned, generally based on the number of consumers of the topic.  
  • Each message in a partition has a unique sequence number associated with it called an offset.  
  • ZooKeeper serves as the coordination interface between the kafka broker and its consumers.  
  • Elects one broker as the leader of a partition, and all the writes and reads must go to the leader partition.
  • Kafka does not have any concept of a master node and treats all the brokers as peers.  


ZooKeeper Basics

  • ZooKeeper is a configuration and coordination manager used in many Hadoop products. 
  • znodes:  shared, hierarchical namespace of data registers much like a filesystem
  • runs, generally, on port 2181
  • config/server.properties
    • lists zookeeper connection info
    • lists the broker.id for the node/namespace
    • lists the logs dir
  • zoo.cfg

Kafka Topics

  • by default a topic (queue) has one partition and a replication factor of 1.  
  • kafka-list-topic.sh actually queries zookeeper for its replication factor, partitions, and leader info.  
  • Leader:  a randomly-selected node for a specific portion of the partitions and is responsible for all reads and writes for the partition.  
  • isr:  in-sync replicas:  replicas that are currently alive and in sync with the leader.  Basically, the non-leader nodes
  • When a new leader needs to be elected this causes significant read-write load in zookeeper and should be avoided.  
  • Kafka tries to ensure that lead replicas for topics are equally distributed among nodes but that breaks down when a node is taken offline. 

Partitioning and Replication

  • The producer determines how the messages are partitioned.  The broker stores the messages in the same order as they arrive.  The number of partitions can be configured for each topic within the broker.  
  • Replication was introduced in Kafka 0.8.
  • ensures better durability of messages and high availability of the cluster.  
  • guarantees that the message will be published and consumed even in case of broker failure 


Useful Kafka Commands

# create a topic

/bin/kafka-create-topic.sh --zookeeper localhost:2181 --replica 1 --partition 1 --topic 

# create a topic with 4 partitions and a replication factor of 2

/bin/kafka-create-topic.sh --zookeeper localhost:2181 --replication-factor 2 --partition 4 --topic 

# have a producer send some messages

/bin/kafka-console-producer.sh --broker-list localhose:9092 --topic 

# starting a consumer

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic  --from-beginning

#list all topics with leader, replication, and partition information
./kafka-list-topic.sh --zookeeper localhost:2181
 
#see information for only one topic
./kafka-list-topic.sh --zookeeper localhost:2181 --topic 
 
#show a topic's "lag"
/opt/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group  --zkconnect :2181
 
#show just the topic's "lag"
/opt/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group  --zkconnect :2181 | awk '{print $6}' | tail -n 3
 
#start Kafka on one node, with its correct configuration (supervisord is the right way to do this)
./kafka-server-start.sh ../config/server.properties
 
#stop Kafka on one node (supervisord is the right way to do this)
./kafka-server-stop.sh
 
#look at Kafka/ZooKeeper supervisord configuration entries
cat /etc/supervisord.conf
 

tail -f /var/log/supervisor/supervisord.log 
Startup/Shutdown

Since Kafka depends on ZooKeeper it's best, IMHO, to ensure each service is managed with something like supervisord.  Having said that, we'll look at the native commands:  

#shutdown a node

bin/kafka-run-class.sh kafka.admin.ShutdownBroker --zookeeper  --broker 

#startup a node

bin/kafka-server-start.sh config/server.properties 

Various Logs

#view supervisord configs
cat /etc/supervisord.conf
 
#supervisord logs
cat /var/log/supervisor/supervisord.log
 
#if only zk is not running, check its logs
tail -f /var/home/user/log/zookeeper.out
 
#if kafka process is not running, check these logs
less /var/home/user/kafka/logs/supervisor-kafka.log
less /var/home/user/log/kafka.out
less /var/home/user/logs/server.log
less /var/home/user/state-change.log

Friday, 15 December 2017

Apache Kafka : Installing and Running on Windows

This notes will provide a step by step guide to run Apache Kafka on a windows OS. This guide will also provide instructions to setup Java & zookeeper. Apache kafka is a fast & scalable messaging queue, capable of handling real heavy loads in context of read & write. You can find more about kafka on kafka. Apache kafka requires a zookeeper instance running which is used for reliable distributed coordination. Please find more about Zookeeper on zookeeper.

Downloading the Required Files


Installation

1. Install & Setup Java Runtime Environment (JRE)

2. Installing & Running Zookeeper

Kafka uses ZooKeeper so you need to first start a ZooKeeper server if you don't already have one. Otherwise Install ZooKeeper by following below steps.
  • Goto your zookeeper config directory. For me its C:\zookeeper-<version>\conf
  • Rename file “zoo_sample.cfg” to “zoo.cfg”
  • Open zoo.cfg in any text editor like notepad but I’ll prefer notepad++.
  • Find & edit dataDir=/tmp/zookeeper to dataDir=C:\zookeeper-<version>\data
  • Add entry in System Environment Variables as we did for java
  • Add in System Variables ZOOKEEPER_HOME = C:\zookeeper-<version>
  • Edit System Variable named “Path” add ;%ZOOKEEPER_HOME%\bin;
  • You can change the default zookeeper port in zoo.cfg file (Default port 2181).
  • Run zookeeper by opening a new cmd & type zkserver.
We can start zookeeper server from here or we can start from Kafka installation folder by using C:\kafka_2.11-0.9.0.0\config\zookeeper.properties.

3. Setting Up Kafka

  • Go to your Kafka config directory. For me its C:\kafka_2.11-0.9.0.0\config
  • Edit file “server.properties”
  • Find & edit line “log.dirs=/tmp/kafka-logs” to “log.dir= C:\kafka_2.11-0.9.0.0\kafka-logs”.
  • If your zookeeper is running on some other machine or cluster you can edit “zookeeper.connect=localhost:2181” to your custom IP & port. For this demo we are using same machine so no need to change. Also Kafka port & broker.id are configurable in this file. Leave other settings as it is.
  • Your Kafka will run on default port 9092 & connect to zookeeper’s default port which is 2181. 

4. Running Kafka Server

Please ensure that your zookeeper is up & running before starting Kafka server.
  • Go to your kafka installation directory C:\kafka_2.11-0.9.0.0\
  • Open command line in your Kafka installation folder
  • Launch Zookeeper with .\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
  • Open a second command line in your Kafka installation folder
  • Launch single Kafka broker: .\bin\windows\kafka-server-start.bat .\config\server.properties
Now your Kafka is up & running, so now you can create topics & produce or consume data from java/scala code & command prompt also.

5. Creating Kafka Topic

  • Now create a topic with name “test” & replication factor 1 as we have only one Kafka server running. If you have a cluster with more than 1 Kafka servers running, you can increase the replication-factor accordingly which will increase the data availability & act like a fault-tolerant system.
  • Open a new command prompt in the location C:\kafka_2.11-0.9.0.0\bin\windows
  • Type following command and hit enter
  • kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

6. Creating a producer & consumer to test server

  • Open a new command prompt in the location C:\kafka_2.11-0.9.0.0\bin\windows
  • To start a producer type command “kafka-console-producer.bat --broker-list localhost:9092 --topic test”.
  • Again open a new command prompt in the same location as C:\kafka_2.11-0.9.0.0\bin\windows
  • Now start a consumer by typing command “kafka-console-consumer.bat --zookeeper localhost:2181 --topic test”.
Now you will have two command prompts like image below...

  • Now type anything in the producer command prompt & press enter and you should be able to see the message in the other consumer command prompt.
  • If you are able to push & see your messages in consumer side, your Kafka setup is ready.


Works

What can I do


Branding

Coming Soon...!!!

Web Design

Coming Soon...!!!

Development

Coming Soon...!!!

Graphic Design

Coming Soon...!!!

Photography

Coming Soon...!!!

User Experience

Coming Soon...!!!

Contact

Get in touch with me


Adress/Street

Bangalore, India