Cloudera Manager 5.14.X 安装部署(上)

tech2026-04-24  4

1. 安装部署CM

注意事项

服务器的IP地址为静态IP; 系统根目录至少50G; 主机名建议统一小写; python版本为2.7.X; 使用root用户安装,或者具有sudo权限的其他用户。

所需软件列表

软件名称版本CentOS7.4或7.5Cloudera Manager5.14.XCDH5.14.X(与上面版本同步)JDK1.8.XMYSQL数据库5.7.16MYSQL的JDBC驱动5.1.46Python2.7.X

注意: CM5.14.X+CDH5.14.X与5.15.X版本搭建过程相同

前置条件:(每个节点都要实现)

1. 修改系统文件句柄数

编辑/etc/security/limits.conf文件

vim /etc/security/limits.conf

添加如下内容

* soft nofile 32768 * hard nofile 65536 * soft nproc 32768 * hard nproc 65536

注意: 上述参数修改应根据实际硬件环境确定,重新登录客户端,即可生效

2. 修改swap交换区空间

临时修改

# 设置 sysctl vm.swappiness=10 # 查看 cat /proc/sys/vm/swappiness

永久修改

vim /etc/sysctl.conf # 添加如下内容 vm.swappiness=10

注意: sysctl -p /etc/sysctl.conf生效

手动关闭swap交换区(可选)

swapoff -a

3. 禁用hugepage透明大页

echo never > /sys/kernel/mm/transparent_hugepage/enabled echo never > /sys/kernel/mm/transparent_hugepage/defrag echo never > /sys/kernel/mm/transparent_hugepage/khugepaged/defrag

设置开机自动关闭,将如下脚本添加到/etc/rc.d/rc.local文件中

if test -f /sys/kernel/mm/transparent_hugepage/enabled then echo never > /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag then echo never > /sys/kernel/mm/transparent_hugepage/defrag fi if test -f /sys/kernel/mm/transparent_hugepage/khugepaged/defrag then echo never > /sys/kernel/mm/transparent_hugepage/khugepaged/defrag fi

赋予执行的权限

chmod 755 /etc/rc.d/rc.local

注意: 重启服务器后生效,/etc/rc.d/rc.local 用于添加开机启动命令;/etc/rc.local是/etc/rc.d/rc.local的软连接

4. 关闭防火墙和selinux

查看防火墙状态

# 方式一:关闭后显示not running,开启后显示running firewall-cmd --state # 方式二:关闭后显Active: inactive (dead),开启后显示Active: active (running) systemctl status firewalld.service

查看防火墙是否开机自启动

systemctl is-enabled firewalld.service

关闭防火墙

# 关闭防火墙 systemctl stop firewalld.service # 禁止防火墙开机自启动 systemctl disable firewalld.service

查看SELINUX

# 方式一:是否显示为Disabled getenforce # 方式二:是否显示为disabled /usr/sbin/sestatus -v

临时关闭SELINUX

setenforce 0

永久关闭SELINUX

# 将SELINUX=enforcing改为SELINUX=disabled,设置后需要重启才能生效 vi /etc/selinux/config

5. 修改主机名和映射文件

修改hostname文件vim /etc/hostname

配置ip到hostname的映射vim /etc/hosts,不用添加域名

注意: 虚拟机有两个网络,一个内部网络和一个外部网络,建议配置内部网络

6. 实现免秘钥登录

在当前用户目录下创建.ssh目录

mkdir .ssh

生成公私密钥对

# 输入以下命令,连续回车 ssh-keygen

拷贝公钥到远程机器的认证列表中

ssh-copy-id -i root@cdh002

ssh-copy-id命令可以把本机的公钥添加到远程主机的authorized_keys文件上,也会给远程主机的用户目录的~/.ssh和~/.ssh/authorized_keys设置合适的权限。

任意两台机器之间实现免秘钥登录

7. 搭建共享源(共享源服务器上操作)

在没有联网的情况下,minimal操作系统需要搭建操作系统ISO共享源(其它服务器同步共享源);

在共享源机器上搭建CDH共享源(其它服务器同步共享源);

前置条件(共享源服务器):

之前将ISO镜像挂载在其它非共享目录下的需要解除挂载

# 挂载 mount 设备名称 挂载点 # 解决挂载 umount 挂载点

安装httpd服务(所有节点)

yum -y install httpd # 安装httpd服务 systemctl start httpd # 开启http服务 systemctl enable httpd # 设置开机启动http服务

注意: 请确认防火墙和selinux已经关闭

安装createrepo(共享源服务器)

yum -y install createrepo

注意: createrepo 用于创建软件仓库,为本地特定位置的rpm包建立索引,描述各包依赖信息,并形成元数据。

创建http共享目录(共享源服务器)

vim /etc/httpd/conf/httpd.conf 修改内容如下: Alias /repo "/var/www/html" <Directory "/var/www/html"> Options Indexes MultiViews FollowSymLinks AllowOverride All Order allow,deny Allow from all </Directory>

在软件包目录创建yum源仓库(共享源服务器)

# centos yum源仓库 mkdir -p /var/www/html/centos_rpm/centos createrepo -p /var/www/html/centos7_rpm/ # CM yum源仓库 mkdir -p /var/www/html/cm_rpm createrepo -p /var/www/html/cm_rpm/

重启httpd服务(共享源服务器)

systemctl restart httpd.service

创建源文件(共享源服务器)

vim /etc/yum.repos.d/centos.repo 修改内容如下: [centosRepo] name=centosRepo baseurl=http://yum服务器主机名/repo/centos7_rpm/centos enabled=1 gpgcheck=0 vim /etc/yum.repos.d/cdh.repo [cmRepo] name=cmRepo baseurl=http://yum服务器主机名/repo/cm_rpm enabled=1 gpgcheck=0

挂载CentOS镜像文件到/var/www/html/centos7_rpm/centos目录

拷贝CM RPM安装包到共享源目录

将下载的CM RPM软件包拷贝到/var/www/html/cm_rpm目录;

centos的镜像文件挂载到/var/www/html/centos7_rpm/centos目录;

更新共享源

createrepo --update -p /var/www/html/cm_rpm createrepo --update -p /var/www/html/centos7_rpm

复制共享源的repo文件到其它节点

可以对其它repo文件进行备份后删除,将创建好的rpm.repo文件复制到其它节点上,并在每一个节点上执行

# 清除yum缓存 yum clean all # 把服务器的包信息下载到本地电脑缓存起来 yum makecache

注意: 安装好后,可以通过浏览器进行访问验证

8. 配置ntp时间同步服务

前提条件

1)服务器之间能ping通,建议优先考虑内部通信网络 2)卸载chrony,命令yum -y remove chrony

8.1 设置NTP服务端

查看ntp状态

systemctl status ntpd.service

出现如下提示,表示未安装ntp服务:

Unit ntpd.service could not be found.

安装ntp服务

yum -y install ntp

设置好时间服务器的时间(与微软公司授时主机同步)

ntpdate time.windows.com

写入硬件时钟(可选)

hwclock -w

修改时钟源的ntp配置文件 vim /etc/ntp.conf

# For more information about this file, see the man pages # ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5). driftfile /var/lib/ntp/drift # Permit time synchronization with our time source, but do not # permit the source to query or modify the service on this system. # 对默认的客户端拒绝一切操作 restrict default nomodify notrap nopeer noquery # Permit all access over the loopback interface. This could # be tightened as well, but to do so would effect some of # the administrative functions. # 允许网段内其它机器同步时间 restrict 10.13.11.0 mask 255.255.255.0 nomodify notrap restrict 127.0.0.1 restrict ::1 # Hosts on local network are less restricted. #restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). server 210.72.145.44 # 中国国家受时中心 server 202.112.10.36 # cn.pool.ntp.org server 59.124.196.83 # asia.pool.ntp.org # 外部时间服务器不可用时,以本地时间作为时间服务 server 127.127.1.0 fudge 127.127.1.0 stratum 10 # 允许上层时间服务器主动修改本机时间 restrict 210.72.145.44 nomodify notrap noquery restrict 202.112.10.36 nomodify notrap noquery restrict 59.124.196.83 nomodify notrap noquery # 注释掉默认的server #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst #broadcast 192.168.1.255 autokey # broadcast server #broadcastclient # broadcast client #broadcast 224.0.1.1 autokey # multicast server #multicastclient 224.0.1.1 # multicast client #manycastserver 239.255.254.254 # manycast server #manycastclient 239.255.254.254 autokey # manycast client # Enable public key cryptography. #crypto includefile /etc/ntp/crypto/pw # Key file containing the keys and key identifiers used when operating # with symmetric key cryptography. keys /etc/ntp/keys # Specify the key identifiers which are trusted. #trustedkey 4 8 42 # Specify the key identifier to use with the ntpdc utility. #requestkey 8 # Specify the key identifier to use with the ntpq utility. #controlkey 8 # Enable writing of statistics records. #statistics clockstats cryptostats loopstats peerstats # Disable the monitoring facility to prevent amplification attacks using ntpdc # monlist command when default restrict does not include the noquery flag. See # CVE-2013-5211 for more details. # Note: Monitoring will not be disabled with the limited restriction flag. disable monitor

启动ntpd服务

systemctl start ntpd.service

设置开机自启动

systemctl enable ntpd.service

8.2 设置NTP客户端

在所有客户端上,修改/etc/ntp.conf文件

# For more information about this file, see the man pages # ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5). driftfile /var/lib/ntp/drift # Permit time synchronization with our time source, but do not # permit the source to query or modify the service on this system. restrict default nomodify notrap nopeer noquery # Permit all access over the loopback interface. This could # be tightened as well, but to do so would effect some of # the administrative functions. restrict 127.0.0.1 restrict ::1 # Hosts on local network are less restricted. #restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap server 10.221.18.114 # 10.221.18.114为时钟服务器 restrict 10.221.18.114 nomodify notrap noquery # server 127.0.0.1 # fudge 127.0.0.1 stratum 10 # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst #broadcast 192.168.1.255 autokey # broadcast server #broadcastclient # broadcast client #broadcast 224.0.1.1 autokey # multicast server #multicastclient 224.0.1.1 # multicast client #manycastserver 239.255.254.254 # manycast server #manycastclient 239.255.254.254 autokey # manycast client # Enable public key cryptography. #crypto includefile /etc/ntp/crypto/pw # Key file containing the keys and key identifiers used when operating # with symmetric key cryptography. keys /etc/ntp/keys # Specify the key identifiers which are trusted. #trustedkey 4 8 42 # Specify the key identifier to use with the ntpdc utility. #requestkey 8 # Specify the key identifier to use with the ntpq utility. #controlkey 8 # Enable writing of statistics records. #statistics clockstats cryptostats loopstats peerstats # Disable the monitoring facility to prevent amplification attacks using ntpdc # monlist command when default restrict does not include the noquery flag. See # CVE-2013-5211 for more details. # Note: Monitoring will not be disabled with the limited restriction flag. disable monitor

启动ntpd服务

systemctl start ntpd.service

设置开机自启动

systemctl enable ntpd.service

ntpq -p 查看网络中的NTP服务器,同时显示客户端和每个服务器的关系

[root@cdh002 softwares]# ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== *cdh001 202.112.10.36 13 u 17 64 377 0.187 -12.303 2.302

注意: 星号表示目前使用的ntp server,其它表示可以使用的ntp server备选

ntpstat 命令查看时间同步状态,这个一般需要5-10分钟后才能成功连接和同步。

未同步显示:

# ntpstat unsynchronised time server re-starting polling server every 64 s

连接并同步后:

# ntpstat synchronised to NTP server (202.112.10.36) at stratum 3 time correct to within 275 ms polling server every 256 s

9. 安装JDK

前提条件: JDK安装在/usr/java,并配置了JDK环境变量

查看系统是否自带jdk

rpm -qa|grep jdk

如果自带jdk,卸载

rpm -e 包名称

上传jdk的rpm包到指定位置

安装jdk

rpm -ivh jdk包名称

检验是否安装成功

java -version

注意: 建议各个主机上的jdk版本一致,版本为1.8系列

最新回复(0)