集群规划
系统版本 | 主机名 | IP | 用途 |
---|---|---|---|
CentOS-7.7 | hadoop-test-1 | 192.168.233.65 | nn dn hive hmaster resourcemanager |
CentOS-7.7 | hadoop-test-2 | 192.168.233.94 | nn dn hmaster resourcemanager |
CentOS-7.7 | hadoop-test-3 | 192.168.233.17 | dn zk |
CentOS-7.7 | hadoop-test-4 | 192.168.233.238 | dn zk |
CentOS-7.7 | hadoop-test-5 | 192.168.233.157 | dn zk |
免密登陆
$ for i in {1..5};do ssh hadoop-test-$i "useradd hadoop";done
$ for i in {1..5};do ssh hadoop-test-$i "echo'pwhadoop'| passwd --stdin'hadoop'";done
$ su hadoop
$ ssh-keygen
$ ssh-copy-id ......
zookeeper-3.5.8
安装
wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.5.8/apache-zookeeper-3.5.8-bin.tar.gz
tar -zxvf apache-zookeeper-3.5.8-bin.tar.gz -C /data
mv /data/apache-zookeeper-3.5.8-bin /data/zookeeper
cp /data/zookeeper/conf/zoo_sample.cfg /data/zookeeper/conf/zoo.cfg
同步所有 zk 节点的配置文件
$ vim /data/zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/zookeeper/zkdata
clientPort=2181
maxClientCnxns=2000
server.1=192.168.233.17:2888:3888
server.2=192.168.233.238:2888:3888
server.3=192.168.233.157:2888:3888
autopurge.snapRetainCount=20
autopurge.purgeInterval=48
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000
zookeeper.sasl.client=true
在每个 zk 的 data 目录内写入 myid 文件,对应配置文件中的 server.1,server.2,server.3:
$ vim /data/zookeeper/zkdata/myid
1
$ vim /data/zookeeper/zkdata/myid
2
$ vim /data/zookeeper/zkdata/myid
3
vim .bashrc
export ZOOKEEPER_HOME=/data/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin
source .bashrc
启动
$ for i in {3..5};do ssh hadoop-test-$i "zkServer.sh start";done
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
查看状态
$ for i in {3..5};do ssh hadoop-test-$i "zkServer.sh status";done
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: leader
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower
测试状态
$ zkCli.sh -server hadoop-test-4:2181
/usr/bin/java
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/zookeeper/lib/slf4j-log4j12-
...
...
...
[zk: hadoop-test-4:2181(CONNECTING) 0]
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: hadoop-test-4:2181(CONNECTED) 0]
[zk: hadoop-test-4:2181(CONNECTED) 1] ls /
[zookeeper]
[zk: hadoop-test-4:2181(CONNECTED) 2] ls /zookeeper
[config, quota]
[zk: hadoop-test-4:2181(CONNECTED) 3]