准备4台虚拟机，具体的虚拟机的配置，根据自己的实际情况分配
角色规划： manager:192.168.0.25 worker node1:192.168.0.53 worker node2:192.168.0.54 worker node3:192.168.0.56
提前安装好docker及相关工具 [root@swarm-manager ~]# yum install docker vim net-tools lsof -y
每台主机修改好主机名
[root@swarm-manager ~]# hostname swarm-manager [root@swarm-manager ~]# cat /etc/hostname localhost.localdomain [root@swarm-manager ~]# hostnamectl set-hostname swarm-manager [root@swarm-manager ~]# cat /etc/hostname swarm-manager [root@swarm-manager ~]# [root@swarm-2 ~]# hostnamectl set-hostname swarm-2 [root@swarm-2 ~]# cat /etc/hostname swarm-2 [root@swarm-3 ~]# hostnamectl set-hostname swarm-3 [root@swarm-3 ~]# cat /etc/hostnam cat /etc/hostname swarm-3 [root@swarm-3 ~]# [root@swarm-4 ~]# hostnamectl set-hostname swarm-4 [root@swarm-4 ~]# cat /etc/hostname swarm-4 [root@swarm-4 ~]#修改hosts文件，添加ip和域名的对应记录
[root@swarm-manager ~]# vim /etc/hosts [root@swarm-manager ~]# ping swarm-2 PING swarm-2 (192.168.0.53) 56(84) bytes of data. 64 bytes from swarm-2 (192.168.0.53): icmp_seq=1 ttl=64 time=1.26 ms 64 bytes from swarm-2 (192.168.0.53): icmp_seq=2 ttl=64 time=0.424 ms ^C --- swarm-2 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1002ms rtt min/avg/max/mdev = 0.424/0.842/1.261/0.419 ms [root@swarm-manager ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.0.25 swarm-manager 192.168.0.53 swarm-2 192.168.0.54 swarm-3 192.168.0.56 swarm-4 将/etc/hosts文件scp到其他的docker node节点上，统一域名解析 [root@swarm-manager ~]# [root@swarm-manager ~]# scp /etc/hosts root@192.168.0.53:/etc/hosts The authenticity of host '192.168.0.53 (192.168.0.53)' can't be established. ECDSA key fingerprint is SHA256:1JtlBdz7WUAWAp7KRG7K0er1N33V/cI2B+I1uIzwxEw. ECDSA key fingerprint is MD5:8c:f6:67:d8:6f:2e:89:81:62:61:54:13:74:69:0d:a4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.0.53' (ECDSA) to the list of known hosts. root@192.168.0.53's password: hosts 100% 249 133.8KB/s 00:00 [root@swarm-manager ~]# scp /etc/hosts root@192.168.0.54:/etc/hosts The authenticity of host '192.168.0.54 (192.168.0.54)' can't be established. ECDSA key fingerprint is SHA256:1JtlBdz7WUAWAp7KRG7K0er1N33V/cI2B+I1uIzwxEw. ECDSA key fingerprint is MD5:8c:f6:67:d8:6f:2e:89:81:62:61:54:13:74:69:0d:a4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.0.54' (ECDSA) to the list of known hosts. root@192.168.0.54's password: hosts 100% 249 125.1KB/s 00:00 [root@swarm-manager ~]# scp /etc/hosts root@192.168.0.56:/etc/hosts The authenticity of host '192.168.0.56 (192.168.0.56)' can't be established. ECDSA key fingerprint is SHA256:1JtlBdz7WUAWAp7KRG7K0er1N33V/cI2B+I1uIzwxEw. ECDSA key fingerprint is MD5:8c:f6:67:d8:6f:2e:89:81:62:61:54:13:74:69:0d:a4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.0.56' (ECDSA) to the list of known hosts. root@192.168.0.56's password: hosts 100% 249 152.2KB/s 00:00所有的主机都禁用firewalld和selinux
[root@swarm-manager ~]# service firewalld stop Redirecting to /bin/systemctl stop firewalld.service [root@swarm-manager ~]# systemctl disable firewalld Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. [root@swarm-manager ~]# [root@swarm-manager ~]# setenforce 0 [root@swarm-manager ~]# vim /etc/sysconfig/selinux在所有的机器上启动docker
[root@swarm-manager ~]# service docker start Redirecting to /bin/systemctl start docker.service [root@swarm-manager ~]# [root@swarm-manager ~]# docker swarm init --advertise-addr 192.168.0.25 Swarm initialized: current node (pcb1438xh9t70tcpasd0ulc9o) is now a manager. To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-2k5qfztq9a6mjjk3q5t0sq3fh9e5jr1tstxrqlfuvi0bush5ok-e9lj7itbtixovdtrrqrys80gt \ 192.168.0.25:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. [root@swarm-manager ~]# 在其他的docker机器上执行命令加入到swarm集群 [root@swarm-4 ~]# docker swarm join \ > --token SWMTKN-1-2k5qfztq9a6mjjk3q5t0sq3fh9e5jr1tstxrqlfuvi0bush5ok-e9lj7itbtixovdtrrqrys80gt \ > 192.168.0.25:2377 This node joined a swarm as a worker. [root@swarm-4 ~]# [root@swarm-3 ~]# docker swarm join \ > --token SWMTKN-1-2k5qfztq9a6mjjk3q5t0sq3fh9e5jr1tstxrqlfuvi0bush5ok-e9lj7itbtixovdtrrqrys80gt \ > 192.168.0.25:2377 This node joined a swarm as a worker. [root@swarm-3 ~]# [root@swarm-2 ~]# docker swarm join \ > --token SWMTKN-1-2k5qfztq9a6mjjk3q5t0sq3fh9e5jr1tstxrqlfuvi0bush5ok-e9lj7itbtixovdtrrqrys80gt \ > 192.168.0.25:2377 This node joined a swarm as a worker. [root@swarm-2 ~]# [root@swarm-manager ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS muwz7i0tnovonc1bok2iilshj swarm-4 Ready Active pcb1438xh9t70tcpasd0ulc9o * swarm-manager Ready Active Leader sr4g9w5qkjpvfasclb3gl4rwc swarm-3 Ready Active yp1vmnjpf96hlhkbupwyh3d03 swarm-2 Ready Active [root@swarm-manager ~]#创建一个overlay类型的网络 sc_nginx
[root@swarm-manager ~]# docker network create -d overlay sc_nginx oqlgdshu2mt3ghsqbgzgi41bg [root@swarm-manager ~]# docker network ls NETWORK ID NAME DRIVER SCOPE a45124458d4a bridge bridge local 305f6c9b2384 docker_gwbridge bridge local 0837f9ee7ac2 host host local opjn30v2uv4q ingress overlay swarm e6a8aeedd772 none null local oqlgdshu2mt3 sc_nginx overlay swarm [root@swarm-manager ~]#创建一个服务并且起四个副本（容器），这里可以看到是逐渐启动的
[root@swarm-manager ~]# docker service create --replicas 4 --network sc_nginx --name sc_nginx1 -p 8000:80 nginx 0sv3k9lndmesdoqwt8y61soi3 [root@swarm-manager ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [root@swarm-manager ~]# docker service ls ID NAME MODE REPLICAS IMAGE 0sv3k9lndmes sc_nginx1 replicated 0/4 nginx:latest [root@swarm-manager ~]# docker service ls ID NAME MODE REPLICAS IMAGE 0sv3k9lndmes sc_nginx1 replicated 2/4 nginx:latest [root@swarm-manager ~]# docker service ls ID NAME MODE REPLICAS IMAGE 0sv3k9lndmes sc_nginx1 replicated 2/4 nginx:latest [root@swarm-manager ~]# docker images; REPOSITORY TAG IMAGE ID CREATED SIZE docker.io/nginx <none> 4bb46517cac3 2 weeks ago 133 MB [root@swarm-manager ~]# docker service ls ID NAME MODE REPLICAS IMAGE 0sv3k9lndmes sc_nginx1 replicated 4/4 nginx:latest容器数量的扩展
[root@swarm-manager ~]# docker service scale sc_nginx1=5 sc_nginx1 scaled to 5 [root@swarm-manager ~]# docker service ls ID NAME MODE REPLICAS IMAGE 0sv3k9lndmes sc_nginx1 replicated 5/5 nginx:latest [root@swarm-manager ~]# docker service ps sc_nginx1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS jldf7sdqifym sc_nginx1.1 nginx:latest swarm-3 Running Running 5 minutes ago qvs44l00en6o sc_nginx1.2 nginx:latest swarm-4 Running Running 5 minutes ago 3xeaf5y0ez6u sc_nginx1.3 nginx:latest swarm-manager Running Running 5 minutes ago mrgn7c02hpmn sc_nginx1.4 nginx:latest swarm-2 Running Running 5 minutes ago pb5g0wzv970h sc_nginx1.5 nginx:latest swarm-manager Running Running 39 seconds ago关闭服务来验证swarm集群的高可用性，一直可用保持固定数量的副本
[root@swarm-2 ~]# service docker stop Redirecting to /bin/systemctl stop docker.service [root@swarm-2 ~]# service docker start Redirecting to /bin/systemctl start docker.service [root@swarm-2 ~]# [root@swarm-manager ~]# docker service ps sc_nginx1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS jldf7sdqifym sc_nginx1.1 nginx:latest swarm-3 Running Running 13 minutes ago qvs44l00en6o sc_nginx1.2 nginx:latest swarm-4 Running Running 13 minutes ago 3xeaf5y0ez6u sc_nginx1.3 nginx:latest swarm-manager Running Running 13 minutes ago di3ogpyhllav sc_nginx1.4 nginx:latest swarm-3 Running Running 4 minutes ago mrgn7c02hpmn \_ sc_nginx1.4 nginx:latest swarm-2 Shutdown Failed 2 minutes ago "No such container: sc_nginx1.…" pb5g0wzv970h sc_nginx1.5 nginx:latest swarm-manager Running Running 8 minutes ago [root@swarm-manager ~]# [root@swarm-manager ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS muwz7i0tnovonc1bok2iilshj swarm-4 Ready Active pcb1438xh9t70tcpasd0ulc9o * swarm-manager Ready Active Leader sr4g9w5qkjpvfasclb3gl4rwc swarm-3 Ready Active yp1vmnjpf96hlhkbupwyh3d03 swarm-2 Down Active模拟镜像升级（关闭一个旧的，马上启动一个新的，慢慢完成）
[root@swarm-manager ~]# docker service create --replicas 6 --network sc_nginx --name sc_redis1 -p 6379:6379 redis:5.0.9 rrd1u8peycl5ibhfmthcw9fj5 [root@swarm-manager ~]# docker service ls ID NAME MODE REPLICAS IMAGE 0sv3k9lndmes sc_nginx1 replicated 1/1 nginx:latest rrd1u8peycl5 sc_redis1 replicated 0/6 redis:5.0.9 [root@swarm-manager ~]# [root@swarm-manager ~]# docker service ls ID NAME MODE REPLICAS IMAGE 0sv3k9lndmes sc_nginx1 replicated 1/1 nginx:latest rrd1u8peycl5 sc_redis1 replicated 5/6 redis:5.0.9 [root@swarm-manager ~]# docker service ls ID NAME MODE REPLICAS IMAGE 0sv3k9lndmes sc_nginx1 replicated 1/1 nginx:latest rrd1u8peycl5 sc_redis1 replicated 6/6 redis:5.0.9 [root@swarm-manager ~]# docker service ps sc_redis1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS zjtdlnpq6vfx sc_redis1.1 redis:5.0.9 swarm-4 Running Running 16 seconds ago 52lfa16bam8x sc_redis1.2 redis:5.0.9 swarm-manager Running Running 18 seconds ago qa06id94z1ko sc_redis1.3 redis:5.0.9 swarm-2 Running Running 17 seconds ago 47tw4671ntrs sc_redis1.4 redis:5.0.9 swarm-3 Running Running 16 seconds ago uzpgfhupys1k sc_redis1.5 redis:5.0.9 swarm-4 Running Running 16 seconds ago mbch1ci13d48 sc_redis1.6 redis:5.0.9 swarm-manager Running Running 18 seconds ago [root@swarm-manager ~]# [root@swarm-manager ~]# docker service update --image redis:latest sc_redis1 [root@swarm-manager ~]# docker service ps sc_redis1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS zjtdlnpq6vfx sc_redis1.1 redis:5.0.9 swarm-4 Running Running about a minute ago 52lfa16bam8x sc_redis1.2 redis:5.0.9 swarm-manager Running Running about a minute ago jl0fq0pfwzd5 sc_redis1.3 redis:latest swarm-2 Running Preparing 2 seconds ago qa06id94z1ko \_ sc_redis1.3 redis:5.0.9 swarm-2 Shutdown Shutdown 1 second ago 47tw4671ntrs sc_redis1.4 redis:5.0.9 swarm-3 Running Running about a minute ago uzpgfhupys1k sc_redis1.5 redis:5.0.9 swarm-4 Running Running about a minute ago mbch1ci13d48 sc_redis1.6 redis:5.0.9 swarm-manager Running Running about a minute ago [root@swarm-manager ~]# docker service ps sc_redis1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS zjtdlnpq6vfx sc_redis1.1 redis:5.0.9 swarm-4 Running Running about a minute ago 52lfa16bam8x sc_redis1.2 redis:5.0.9 swarm-manager Running Running about a minute ago jl0fq0pfwzd5 sc_redis1.3 redis:latest swarm-2 Running Preparing 12 seconds ago qa06id94z1ko \_ sc_redis1.3 redis:5.0.9 swarm-2 Shutdown Shutdown 10 seconds ago 47tw4671ntrs sc_redis1.4 redis:5.0.9 swarm-3 Running Running about a minute ago uzpgfhupys1k sc_redis1.5 redis:5.0.9 swarm-4 Running Running about a minute ago mbch1ci13d48 sc_redis1.6 redis:5.0.9 swarm-manager Running Running about a minute ago [root@swarm-manager ~]# docker service ps sc_redis1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 8sza5lmghuwa sc_redis1.1 redis:latest swarm-4 Running Preparing 2 seconds ago zjtdlnpq6vfx \_ sc_redis1.1 redis:5.0.9 swarm-4 Shutdown Shutdown 1 second ago 52lfa16bam8x sc_redis1.2 redis:5.0.9 swarm-manager Running Running about a minute ago jl0fq0pfwzd5 sc_redis1.3 redis:latest swarm-2 Running Running 5 seconds ago qa06id94z1ko \_ sc_redis1.3 redis:5.0.9 swarm-2 Shutdown Shutdown 21 seconds ago 47tw4671ntrs sc_redis1.4 redis:5.0.9 swarm-3 Running Running about a minute ago msqj349izyeu sc_redis1.5 redis:latest swarm-2 Running Running 3 seconds ago uzpgfhupys1k \_ sc_redis1.5 redis:5.0.9 swarm-4 Shutdown Shutdown 4 seconds ago mbch1ci13d48 sc_redis1.6 redis:5.0.9 swarm-manager Running Running about a minute ago [root@swarm-manager ~]# docker service ps sc_redis1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 8sza5lmghuwa sc_redis1.1 redis:latest swarm-4 Running Running 8 seconds ago zjtdlnpq6vfx \_ sc_redis1.1 redis:5.0.9 swarm-4 Shutdown Shutdown 21 seconds ago 52lfa16bam8x sc_redis1.2 redis:5.0.9 swarm-manager Running Running 2 minutes ago jl0fq0pfwzd5 sc_redis1.3 redis:latest swarm-2 Running Running 26 seconds ago qa06id94z1ko \_ sc_redis1.3 redis:5.0.9 swarm-2 Shutdown Shutdown 41 seconds ago mn7fvw2gtmg9 sc_redis1.4 redis:latest swarm-3 Running Preparing 9 seconds ago 47tw4671ntrs \_ sc_redis1.4 redis:5.0.9 swarm-3 Shutdown Shutdown 8 seconds ago msqj349izyeu sc_redis1.5 redis:latest swarm-2 Running Running 23 seconds ago uzpgfhupys1k \_ sc_redis1.5 redis:5.0.9 swarm-4 Shutdown Shutdown 24 seconds ago mbch1ci13d48 sc_redis1.6 redis:5.0.9 swarm-manager Running Running 2 minutes ago [root@swarm-manager ~]# docker service ps sc_redis1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 8sza5lmghuwa sc_redis1.1 redis:latest swarm-4 Running Running 32 seconds ago zjtdlnpq6vfx \_ sc_redis1.1 redis:5.0.9 swarm-4 Shutdown Shutdown 44 seconds ago wulpde6zyrf3 sc_redis1.2 redis:latest swarm-4 Running Running 16 seconds ago 52lfa16bam8x \_ sc_redis1.2 redis:5.0.9 swarm-manager Shutdown Shutdown 17 seconds ago jl0fq0pfwzd5 sc_redis1.3 redis:latest swarm-2 Running Running 49 seconds ago qa06id94z1ko \_ sc_redis1.3 redis:5.0.9 swarm-2 Shutdown Shutdown about a minute ago mn7fvw2gtmg9 sc_redis1.4 redis:latest swarm-3 Running Running 20 seconds ago 47tw4671ntrs \_ sc_redis1.4 redis:5.0.9 swarm-3 Shutdown Shutdown 31 seconds ago msqj349izyeu sc_redis1.5 redis:latest swarm-2 Running Running 46 seconds ago uzpgfhupys1k \_ sc_redis1.5 redis:5.0.9 swarm-4 Shutdown Shutdown 47 seconds ago 5ptut26o412c sc_redis1.6 redis:latest swarm-manager Running Running 5 seconds ago mbch1ci13d48 \_ sc_redis1.6 redis:5.0.9 swarm-manager Shutdown Shutdown 15 seconds ago [root@swarm-manager ~]# docker service ps sc_redis1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 8sza5lmghuwa sc_redis1.1 redis:latest swarm-4 Running Running 34 seconds ago zjtdlnpq6vfx \_ sc_redis1.1 redis:5.0.9 swarm-4 Shutdown Shutdown 47 seconds ago wulpde6zyrf3 sc_redis1.2 redis:latest swarm-4 Running Running 19 seconds ago 52lfa16bam8x \_ sc_redis1.2 redis:5.0.9 swarm-manager Shutdown Shutdown 20 seconds ago jl0fq0pfwzd5 sc_redis1.3 redis:latest swarm-2 Running Running 52 seconds ago qa06id94z1ko \_ sc_redis1.3 redis:5.0.9 swarm-2 Shutdown Shutdown about a minute ago mn7fvw2gtmg9 sc_redis1.4 redis:latest swarm-3 Running Running 23 seconds ago 47tw4671ntrs \_ sc_redis1.4 redis:5.0.9 swarm-3 Shutdown Shutdown 34 seconds ago msqj349izyeu sc_redis1.5 redis:latest swarm-2 Running Running 49 seconds ago uzpgfhupys1k \_ sc_redis1.5 redis:5.0.9 swarm-4 Shutdown Shutdown 50 seconds ago 5ptut26o412c sc_redis1.6 redis:latest swarm-manager Running Running 8 seconds ago mbch1ci13d48 \_ sc_redis1.6 redis:5.0.9 swarm-manager Shutdown Shutdown 18 seconds ago [root@swarm-manager ~]#再起一台机器安装nginx来实现负载均衡
[root@LB-NGINX ~]# yum install epel-release -y [root@LB-NGINX ~]# yum install nginx -y [root@LB-NGINX ~]# service nginx restart Redirecting to /bin/systemctl restart nginx.service [root@LB-NGINX ~]# lsof -i:80 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME nginx 3186 root 6u IPv4 28224 0t0 TCP *:http (LISTEN) nginx 3186 root 7u IPv6 28225 0t0 TCP *:http (LISTEN) nginx 3187 nginx 6u IPv4 28224 0t0 TCP *:http (LISTEN) nginx 3187 nginx 7u IPv6 28225 0t0 TCP *:http (LISTEN) nginx 3188 nginx 6u IPv4 28224 0t0 TCP *:http (LISTEN) nginx 3188 nginx 7u IPv6 28225 0t0 TCP *:http (LISTEN) [root@LB-NGINX ~]# [root@LB-NGINX nginx]# ls conf.d fastcgi.conf.default koi-utf mime.types.default scgi_params uwsgi_params.default default.d fastcgi_params koi-win nginx.conf scgi_params.default win-utf fastcgi.conf fastcgi_params.default mime.types nginx.conf.default uwsgi_params [root@LB-NGINX nginx]# vim nginx.conf #定义一个负载均衡器 scapp1 upstream scapp1 { server 192.168.0.25; server 192.168.0.53; server 192.168.0.54; server 192.168.0.56; } server { listen 80 default_server; listen [::]:80 default_server; server_name _; root /usr/share/nginx/html; # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; #添加负载均衡的使用 location / { proxy_pass http://scapp1; } [root@LB-NGINX nginx]# service nginx restart Redirecting to /bin/systemctl restart nginx.service [root@LB-NGINX nginx]# 注意在nginx LB服务器上关闭防火墙和selinux扩展：
1、 这里如果我们想实现所有nginx网页内容一致的话，可以再起一台机器充当nfs服务器 2、 这里还可以再搭建一个nginx负载均衡服务器，两台服务器上安装keepalived实现高可用
