集群规划
| 系统版本 | 主机名 | IP | 用途 |
|---|---|---|---|
| CentOS-7.7 | hadoop-test-1 | 192.168.233.65 | nn dn hive hmaster resourcemanager |
| CentOS-7.7 | hadoop-test-2 | 192.168.233.94 | nn dn hmaster resourcemanager |
| CentOS-7.7 | hadoop-test-3 | 192.168.233.17 | dn zk |
| CentOS-7.7 | hadoop-test-4 | 192.168.233.238 | dn zk |
| CentOS-7.7 | hadoop-test-5 | 192.168.233.157 | dn zk |
免密登陆
$ for i in {1..5};do ssh hadoop-test-$i "useradd hadoop";done$ for i in {1..5};do ssh hadoop-test-$i "echo 'pwhadoop' | passwd --stdin 'hadoop'";done$ su hadoop$ ssh-keygen $ ssh-copy-id ......zookeeper-3.5.8
安装
wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.5.8/apache-zookeeper-3.5.8-bin.tar.gztar -zxvf apache-zookeeper-3.5.8-bin.tar.gz -C /datamv /data/apache-zookeeper-3.5.8-bin /data/zookeepercp /data/zookeeper/conf/zoo_sample.cfg /data/zookeeper/conf/zoo.cfg同步所有zk节点的配置文件
$ vim /data/zookeeper/conf/zoo.cfgtickTime=2000initLimit=10syncLimit=5dataDir=/data/zookeeper/zkdataclientPort=2181maxClientCnxns=2000server.1=192.168.233.17:2888:3888server.2=192.168.233.238:2888:3888server.3=192.168.233.157:2888:3888autopurge.snapRetainCount=20autopurge.purgeInterval=48authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProviderrequireClientAuthScheme=sasljaasLoginRenew=3600000zookeeper.sasl.client=true在每个zk的data目录内写入myid文件，对应配置文件中的server.1,server.2,server.3:
$ vim /data/zookeeper/zkdata/myid1$ vim /data/zookeeper/zkdata/myid2$ vim /data/zookeeper/zkdata/myid3vim .bashrcexport ZOOKEEPER_HOME=/data/zookeeper export PATH=$PATH:$ZOOKEEPER_HOME/binsource .bashrc启动
$ for i in {3..5};do ssh hadoop-test-$i "zkServer.sh start";done/usr/bin/javaZooKeeper JMX enabled by defaultUsing config: /data/zookeeper/bin/../conf/zoo.cfgStarting zookeeper ... STARTED/usr/bin/javaZooKeeper JMX enabled by defaultUsing config: /data/zookeeper/bin/../conf/zoo.cfgStarting zookeeper ... STARTED/usr/bin/javaZooKeeper JMX enabled by defaultUsing config: /data/zookeeper/bin/../conf/zoo.cfgStarting zookeeper ... STARTED查看状态
$ for i in {3..5};do ssh hadoop-test-$i "zkServer.sh status";done/usr/bin/javaZooKeeper JMX enabled by defaultUsing config: /data/zookeeper/bin/../conf/zoo.cfgClient port found: 2181. Client address: localhost.Mode: follower/usr/bin/javaZooKeeper JMX enabled by defaultUsing config: /data/zookeeper/bin/../conf/zoo.cfgClient port found: 2181. Client address: localhost.Mode: leader/usr/bin/javaZooKeeper JMX enabled by defaultUsing config: /data/zookeeper/bin/../conf/zoo.cfgClient port found: 2181. Client address: localhost.Mode: follower测试状态
$ zkCli.sh -server hadoop-test-4:2181/usr/bin/javaSLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/data/zookeeper/lib/slf4j-log4j12-.........[zk: hadoop-test-4:2181(CONNECTING) 0] WATCHER::WatchedEvent state:SyncConnected type:None path:null[zk: hadoop-test-4:2181(CONNECTED) 0][zk: hadoop-test-4:2181(CONNECTED) 1] ls /[zookeeper][zk: hadoop-test-4:2181(CONNECTED) 2] ls /zookeeper[config, quota][zk: hadoop-test-4:2181(CONNECTED) 3]