自动化安装Kafka集群

软件下载地址

zcbus安装包下载：http://zbomc.com:8888/ZCBUS/2.0/kafka_cluster_install.tar.gz

解压安装包

• tar zxvf kafka_cluster_install.tar.gz

修改安装参数文件及说明

vi conf/kafka_cluster.ini

#ssh_trust
[host1]
name=zcbus_kafka_1
host=192.168.71.150
port=22
user=root
password=1qaz2wsx

[host2]
name=zcbus_kafka_2
host=192.168.71.149
port=22
user=root
password=1qaz2wsx

[host3]
name=zcbus_kafka_3
host=192.168.71.151
port=22
user=root
password=1qaz2wsx #含有特殊字符的密码暂时不支持，ssh_trust配置互信的时候会报错
注：[host]为配置kafka节点的主机信息，name为别名可以自定义，password为明文密码，几个节点的kafka就需要配置几个主机节点(安装介质需要存放在其中一个节点上，此节点也需要配置host)

#kafka_cluster
kafka_ports="12181 12888 13888 19092" #端口(可自定义)
kafka_install_path="/data"  #安装目录

## kafka副本数
FACTOR_NUM=2

## kafka内存，单位：G
KAFKA_MEM=1



ZOOK_PORT_1=`echo "${kafka_ports}"|awk '{print $1}'`
ZOOK_PORT_2=`echo "${kafka_ports}"|awk '{print $2}'`
ZOOK_PORT_3=`echo "${kafka_ports}"|awk '{print $3}'`
KAFKA_PORT=`echo "${kafka_ports}"|awk '{print $4}'`
KAFKA_DIR="${kafka_install_path}/kafka/data/kafka-logs"
ZOOK_DIR="${kafka_install_path}/kafka/data/zookeeper"

安装过程记录

[root@lyzcbus kafka_cluster_install]# sh kafka_cluster_install.sh 

[Step 1]: set shh trust ...
➜ install sshpass
✔ install sshpass successfully ...
➜ shh trust
✔ shh trust successfully ...

[Step 2]: check ports ...

Note: 192.168.71.150
➜ Port 12181 is available!!
➜ Port 12888 is available!!
➜ Port 13888 is available!!
➜ Port 19092 is available!!

Note: 192.168.71.149
➜ Port 12181 is available!!
➜ Port 12888 is available!!
➜ Port 13888 is available!!
➜ Port 19092 is available!!

Note: 192.168.71.151
➜ Port 12181 is available!!
➜ Port 12888 is available!!
➜ Port 13888 is available!!
➜ Port 19092 is available!!

[Step 3]: check install dirs ...

Note: 192.168.71.150
➜ Dir /data is exist!!

Note: 192.168.71.149
➜ Dir /data is exist!!

Note: 192.168.71.151
➜ Dir /data is exist!!

[Step 4]: install kafka cluster ...

Note: 192.168.71.150
➜ config zookeeper
✔ config zookeeper successfully ...
➜ config kafka
✔ config kafka successfully ...
➜ cp kafka
✔ cp kafka successfully ...

Note: 192.168.71.149
➜ config zookeeper
✔ config zookeeper successfully ...
➜ config kafka
✔ config kafka successfully ...
➜ cp kafka
✔ cp kafka successfully ...

Note: 192.168.71.151
➜ config zookeeper
✔ config zookeeper successfully ...
➜ config kafka
✔ config kafka successfully ...
➜ cp kafka
✔ cp kafka successfully ...

[Step 5]: check os ...

Note: 192.168.71.150
➜ check os success !!!

Note: 192.168.71.149
➜ check os success !!!

Note: 192.168.71.151
➜ check os success !!!

[Step 6]: start zookeepre cluster ...

Note: 192.168.71.150
➜ start zookeeper success !!!

Note: 192.168.71.149
➜ start zookeeper success !!!

Note: 192.168.71.151
➜ start zookeeper success !!!

[Step 7]: start kafka cluster ...

Note: 192.168.71.150
➜ start kafka  success !!!

Note: 192.168.71.149
➜ start kafka  success !!!

Note: 192.168.71.151
➜ start kafka  success !!!

[Step 8]: check kafka cluster ...

Note: 192.168.71.150
➜ check kafka  success !!!

Note: 192.168.71.149
➜ check kafka  success !!!

Note: 192.168.71.151
➜ check kafka  success !!!