hadoop 环境规划 因为是自己的虚拟机所以这里都继续使用hadoop用户, 以及直接安装到了hadoop节点, 减少网络延迟
服务hadoop181hadoop182hadoop183Zookeeper√√√(1) 关闭防火墙 (2) 关闭selinux (3)初始化用户密码(我这里直接用hadoop用户了) (4)分别设置各个主机的ip地址 (5)配置用户sudo 提权 (6)配置hosts文件 (7)配置主机间的免密登陆
上面几个步骤可以参考我的另一篇笔记 hadoop搭建实录
(1) 上传安装包(本次使用apache-zookeeper-3.5.7-bin.tar.gz版本包)
Last login: Wed Sep 2 20:33:53 2020 from 192.168.207.1 [hadoop@hadoop181 ~]$ ll total 520420 -rw-rw-r-- 1 hadoop hadoop 9311744 Mar 10 21:46 apache-zookeeper-3.5.7-bin.tar.gz drwxrwxr-x 2 hadoop hadoop 31 Sep 2 16:24 bin drwxr-xr-x 11 hadoop hadoop 173 Sep 2 18:17 hadoop-3.1.3 -rw-rw-r-- 1 hadoop hadoop 338075860 Aug 4 20:14 hadoop-3.1.3.tar.gz drwxr-xr-x 8 hadoop hadoop 255 Jul 22 2017 jdk1.8.0_144 -rw-rw-r-- 1 hadoop hadoop 185515842 Jan 26 2020 jdk-8u144-linux-x64.tar.gz(2)解压安装包并给文件夹改名
[hadoop@hadoop181 ~]$ tar -zxvf apache-zookeeper-3.5.7-bin.tar.gz [hadoop@hadoop181 ~]$ mv apache-zookeeper-3.5.7-bin apache-zookeeper(3) 生成 zoo.cfg 文件
[hadoop@hadoop181 ~]$ cd apache-zookeeper/conf [hadoop@hadoop181 conf]$ cp zoo_sample.cfg zoo.cfg(4)配置zoo.cfg 文件
[hadoop@hadoop181 conf]$ vim zoo.cfg示例
# 配置数据目录,以及日志目录 dataDir=/home/hadoop/apache-zookeeper/data/ dataLogDir=/home/hadoop/apache-zookeeper/dataLog/ clientPort=2181 # 配置zk 集群选举列表 server.1=hadoop181:2888:3888 server.2=hadoop182:2888:3888 server.3=hadoop183:2888:3888(5)文件分发
[hadoop@hadoop181 ~]$ xsync apache-zookeeper(6)在数据目录下增加myid文件,并修改每个节点的myid值
# 新增加一个生成myid的脚本,追加下面的脚本 [hadoop@hadoop181 bin]$ vim apache-zk-myid.sh #!/bin/bash for i in 1 2 3 do ssh hadoop@hadoop18$i "touch /home/hadoop/apache-zookeeper/data/myid" ssh hadoop@hadoop18$i "echo $i > /home/hadoop/apache-zookeeper/data/myid" done(7)启动集群
[hadoop@hadoop181 ~]$ xssh /home/hadoop/apache-zookeeper/bin/zkServer.sh start(8) 启动状态监控
# 查询下java进程信息 [hadoop@hadoop181 bin]$ xssh jps -l [DEBUG] 1 command is :jps -l [DEBUG] 1 command is :jps -l [DEBUG] ssh to hadoop181 to execute commands [ jps -l] 2065 org.apache.zookeeper.server.quorum.QuorumPeerMain 2122 sun.tools.jps.Jps [DEBUG] ssh to hadoop182 to execute commands [ jps -l] 1969 sun.tools.jps.Jps 1910 org.apache.zookeeper.server.quorum.QuorumPeerMain [DEBUG] ssh to hadoop183 to execute commands [ jps -l] 1907 org.apache.zookeeper.server.quorum.QuorumPeerMain 1959 sun.tools.jps.Jps # 查询下端口占用信息 [hadoop@hadoop181 bin]$ xssh netstat -ntlp [DEBUG] 1 command is :netstat -ntlp [DEBUG] 1 command is :netstat -ntlp [DEBUG] ssh to hadoop181 to execute commands [ netstat -ntlp] (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp6 0 0 ::1:25 :::* LISTEN - tcp6 0 0 :::2181 :::* LISTEN 2065/java tcp6 0 0 :::40077 :::* LISTEN 2065/java tcp6 0 0 192.168.207.181:3888 :::* LISTEN 2065/java tcp6 0 0 :::8080 :::* LISTEN 2065/java tcp6 0 0 :::22 :::* LISTEN - [DEBUG] ssh to hadoop182 to execute commands [ netstat -ntlp] (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp6 0 0 :::45496 :::* LISTEN 1910/java tcp6 0 0 ::1:25 :::* LISTEN - tcp6 0 0 :::2181 :::* LISTEN 1910/java tcp6 0 0 192.168.207.182:2888 :::* LISTEN 1910/java tcp6 0 0 192.168.207.182:3888 :::* LISTEN 1910/java tcp6 0 0 :::8080 :::* LISTEN 1910/java tcp6 0 0 :::22 :::* LISTEN - [DEBUG] ssh to hadoop183 to execute commands [ netstat -ntlp] (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp6 0 0 ::1:25 :::* LISTEN - tcp6 0 0 :::34626 :::* LISTEN 1907/java tcp6 0 0 :::2181 :::* LISTEN 1907/java tcp6 0 0 192.168.207.183:3888 :::* LISTEN 1907/java tcp6 0 0 :::8080 :::* LISTEN 1907/java tcp6 0 0 :::22 :::* LISTEN -