[Kafka]how to build binary of kafka from source

I got a bug of kafka on windows. It caused disk space usage issue because kafka borkers can not delete old log files after the configured time.

So, I have to build binary of kafka with patch from Microsoft/kafka’s version to avoid more disk space usage.

I installed kafka(0.10.2.1) on windows2016 not long ago and everything works fine except delete old log files.

P.S:I installed zookeeper on windows2016 too.

This bug only for windows family. so You will get the same issue with me as below:

I found a ticket of kafka on the Apach jira. The ticket number is KAFKA-1194.

I am so surprise for that because this is a bug around four years ago.

Okay complain done, let’s jump to how to build binary from source.

You can skip below content if you did not install kafka/kafka cluster on windows.

Download Microsoft/kafka

P.S:

I have to use this version of kafka because I got a bug of kafka on windows for delete old log.

 

Install Gradle and Java

Note:

Kafka requires Gradle 2.0 or higher.

Java 7 should be used for building in order to support both Java 7 and Java 8 at runtime.

 

First bootstrap and download the wrapper

cd kafka_source_dir

gradle

 

Building a jar and running it

gradlew jar

 

Building source jar

gradlew srcJar

 

Building aggregated Javadoc

gradlew aggregatedJavadoc

 

Building javadoc and scaladoc

gradlew javadoc

gradlew javadocJar # builds a javadoc jar for each module

gradlew scaladoc

gradlew scaladocJar # builds a scaladoc jar for each module

gradlew docsJar # builds both (if applicable) javadoc and scaladoc jars for each modul

 

Running unit tests

gradlew test

 

Building a binary release gzipped tar ball

gradlew clean

gradlew releaseTarG

note:

you will find binary of kafka in .\kafka_source\core\build\distributions after builded work complete.

Please unzip this file then install kafka on your windows server.

I did not get any issue of kafka for delete old log files on windows2016 after I installed this version.

 

Reference

KAFKA-1194

Microsoft/kafka