devops相关服务搭建

一、cicd工具链搭建1.代码库搭建(gitlab)1.1 下载docker pull镜像启动1.2 支持ssh端口配置1.3 邮件发送配置1.4 并发问题配置1.5 备份到oss配置1.6 支持ssl配置1.7 本地nginx配置并转发到容器 2.jenkins搭建2.1 yum安装2.2 全局插件配置2.3 nginx配置 3.jdk&nexus&maven&node基础环境搭建3.1 jdk环境安装3.2 node环境安装3.3 maven环境安装3.4 nexus安装配置 二、项目管理1.知识库搭建（conference）2.接口文档搭建（yapi）

一、cicd工具链搭建

1.代码库搭建(gitlab)

最早时候公司有一套gogs，轻量级的代码库，功能较少直接没用了，选择代码库前有考虑过阿里云的codeup，后来为了代码安全还是自建gitlab。

现在已经是0202年了，搭建服务首选肯定是docker拉了镜像直接干。

1.1 下载docker pull镜像启动

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo yum install docker-ce-18.06.1.ce-3.el7 systemctl start docker systemctl enable docker

这里可以修改下镜像下载地址配置阿里云加速镜像站 阿里云镜像服务界面找到加速域名 docker配置文件加入

vim /etc/docker/daemon.json { "registry-mirrors": ["https://xxxxxx.mirror.aliyuncs.com"], "live-restore": true } #重启 systemctl restart docker

这里注意如果需要用到ssh免密的需要映射一个2222对应22的端口出来，在配置文件也需要配置2222的ssh

docker pull beginor/gitlab-ce:11.0.1-ce.0 docker run --detach --publish 8443:443 --publish 2222:22--publish 8090:80 --name gitlab --restart always --hostname 10.0.0.0 -v /data/software/gitlab/etc:/etc/gitlab -v /data/software/gitlab/logs:/var/log/gitlab -v /data/software/gitlab/data:/var/opt/gitlab -v /etc/localtime:/etc/localtime:ro --privileged=true beginor/gitlab-ce

1.2 支持ssh端口配置

vim /data/software/gitlab/etc/gitlab.rb gitlab_rails['gitlab_shell_ssh_port'] = 2222 gitlab_rails['gitlab_shell_git_timeout'] = 800

1.3 邮件发送配置

vim /data/software/gitlab/etc/gitlab.rb #腾讯邮箱 nginx['enable'] = true gitlab_rails['smtp_port'] = 465 gitlab_rails['smtp_user_name'] = "a@qq.com" gitlab_rails['smtp_password'] = "fkvgpkocjfatbcee" gitlab_rails['smtp_domain'] = "qq.com" gitlab_rails['smtp_authentication'] = "login" gitlab_rails['smtp_enable_starttls_auto'] = true gitlab_rails['smtp_tls'] = true user['git_user_email'] = "a@qq.com" gitlab_rails['gitlab_email_from'] = 'a@qq.com' #阿里邮箱 gitlab_rails['smtp_enable'] = true gitlab_rails['smtp_address'] = "smtp.qiye.aliyun.com" gitlab_rails['smtp_port'] = 465 gitlab_rails['smtp_user_name'] = "gitlab@.cn" gitlab_rails['smtp_password'] = "@8888" gitlab_rails['smtp_domain'] = "qiye.aliyun.com" gitlab_rails['smtp_authentication'] = "login" gitlab_rails['smtp_enable_starttls_auto'] = true gitlab_rails['smtp_tls'] = true user['git_user_email'] = "gitlab@.cn" gitlab_rails['gitlab_email_from'] = 'gitlab@.cn'

1.4 并发问题配置

在SourceTree升级到2.1.10.0后，默认后台并发查询git仓库更新，项目较多，这个并发查询超了GitLab并发访问的阈值，会封了IP GitLab响应Forbidden

vim /data/software/gitlab/etc/gitlab.rb gitlab_rails['rack_attack_git_basic_auth'] = { 'enabled' => true, 'ip_whitelist' => ["127.0.0.1","0.0.0.0"], 'maxretry' => 200, 'findtime' => 60, 'bantime' => 3600 }

1.5 备份到oss配置

vim /data/software/gitlab/etc/gitlab.rb gitlab_rails['backup_upload_connection'] = { 'provider' => 'aliyun', 'aliyun_accesskey_id' => 'xxxxxxxxxxxxxxxx', 'aliyun_accesskey_secret' => 'xxxxxxxxxxxxmXZz', 'aliyun_oss_endpoint' => 'http://oss-cn-hangzhou-internal.aliyuncs.com', 'aliyun_oss_bucket' => 'gitlab-bk-data', 'aliyun_oss_location' => 'hangzhou', } gitlab_rails['backup_upload_remote_directory'] = 'gitlab'

配置备份计划及定期删除

59 23 * * * /usr/bin/docker exec -t 55f86fb06075 gitlab-rake gitlab:backup:create >> /root/gitbk-nohup.out 2>&1 59 23 * * * /usr/bin/find /data/software/gitlab/data/backups/ -mtime +10 -name '*.tar' -exec rm -rf {} \;

在阿里云oss上配置定期删除

1.6 支持ssl配置

域名转https

vim /data/software/gitlab/etc/gitlab.rb nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.example.com.crt" nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.example.com.key" external_url "https://gitlab.cn" nginx['redirect_http_to_https'] = true

进入容器重新导入配置执行

docker exec -it 55f86fb06075 /bin/bash gitlab-ctl reconfigure

1.7 本地nginx配置并转发到容器

本机nginx配置转发到容器中 https的证书可直接从阿里申请

upstream gitlab{ server localhost:8443; } # 转发到容器 server{ listen 443; server_name gitlab.xxx.cn; client_max_body_size 1000m; ssl_certificate /root/.cert/gitlab.example.com.crt; ssl_certificate_key /root/.cert/gitlab.example.com.key; location / { proxy_pass https://gitlab; proxy_http_version 1.1; proxy_set_header X_FORWARDED_PROTO https; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $host; } } # 转发到443 server { listen 80; server_name gitlab.xxx.cn; rewrite ^(.*)$ https://${server_name}$1 permanent; }

gitlab容器nginx配置

upstream gitlab-workhorse { server unix:/var/opt/gitlab/gitlab-workhorse/socket; } ## Redirects all HTTP traffic to the HTTPS host server { listen *:80; server_name gitlab.xxx.cn;#域名 server_tokens off; ## Don't show the nginx version number, a security best practice location / { return 301 https://gitlab.xxx.cn:443$request_uri; } access_log /var/log/gitlab/nginx/gitlab_access.log gitlab_access; error_log /var/log/gitlab/nginx/gitlab_error.log; } server { listen *:443 ssl http2; server_name gitlab.xxx.cn; server_tokens off; ## Don't show the nginx version number, a security best practice ## Increase this if you want to upload large attachments ## Or if you want to accept large git objects over http client_max_body_size 0;

备注：当你映射的时候忘记加入2222到22的端口时，ssh免密钥是无法使用的，这时候可以通过工具做一个转发

下载rinetd以下链接下载不了可能要重新找下载源了，这个工具个人感觉简单好用，需要的也可以留言

wget https://www.boutell.com/rinetd/http/rinetd.tar.gz mkdir -p /usr/man/man8/ tar xvf rinetd.tar.gz -C /usr/man/man8/ cd /usr/man/man8/rinetd/ make && make install vim /etc/rinetd.conf 0.0.0.0 222 172.17.0.2 22 allow *.*.*.* #执行启动 rinetd -c /etc/rinetd.conf

2.jenkins搭建

由于前面此博客前面有docker的安装方式这里就写yum的安装方式吧

2.1 yum安装

cd /etc/yum.repos.d/ wget http://pkg.jenkins.io/redhat/jenkins.repo rpm --import http://pkg.jenkins.io/redhat/jenkins.io.key yum install -y jenkins #默认安装最新版本。戒者直接安装 jenkins-2.93-1.1.noarch.rpm 包 vim /etc/sysconfig/jenkins JENKINS_PORT="8081" JENKINS_HOME="/var/lib/jenkins" #数据目录，建议用固态磁盘来存数据，可以自己定义 /etc/init.d/jenkins start #启劢 chkconfig jenkins on #设置开机启劢 chkconfig --list jenkins

2.2 全局插件配置

#查看密码文件 cat /var/lib/jenkins/secrets/initialAdminPassword

配置jdk&maven&node

插件下载 NodeJS Plugin == node环境 Maven Integration == maven环境 Localization: Chinese (Simplified) == 中文包

配置全局工具 路径可自定义

2.3 nginx配置

server { listen 80; server_name jenkins.xxx.cn; return 301 https://$server_name$request_uri; } upstream jenkins { server 127.0.0.1:8090; } server { listen 443 ssl; server_name jenkins.xxx.cn; client_max_body_size 100m; charset utf-8; client_body_buffer_size 10M; proxy_redirect off; proxy_set_header Host $host:$server_port; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host:$server_port; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_next_upstream error timeout invalid_header http_502 http_503 http_504; proxy_ignore_client_abort on; proxy_read_timeout 180; proxy_buffering on; proxy_buffer_size 8k; proxy_buffers 8 8M; gzip on; gzip_min_length 1000; gzip_types text/plain text/css application/json text/xml application/xml application/xml+rss text/javascript; ssl_certificate /etc/nginx/conf.d/cert/jenkins.xxx.cn.pem; ssl_certificate_key /etc/nginx/conf.d/cert/jenkins.xxx.cn.key; ssl_session_timeout 5m; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4; #使用此加密套件。 ssl_protocols TLSv1 TLSv1.1 TLSv1.2; #使用该协议进行配置。 ssl_prefer_server_ciphers on; location / { proxy_pass http://jenkins; proxy_redirect default; proxy_set_header Host $host:$server_port; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host:$server_port; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } }

完成

3.jdk&nexus&maven&node基础环境搭建

3.1 jdk环境安装

#下载jdk包 jdk-8u231-linux-x64.tar.gz #建立java目录 mkdir /usr/local/jdk #解压 tar xvf jdk-8u231-linux-x64.tar.gz -C /usr/local/jdk/ #解析环境变量 vim /etc/profile.d/java.sh export JAVA_HOME=/usr/local/jdk/jdk1.8.0_231 export PATH=$PATH:$JAVA_HOME/bin export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

3.2 node环境安装

#下载 wget https://nodejs.org/dist/v12.0.0/node-v12.0.0-linux-x64.tar.gz #创建服务目录 mkdir /usr/local/node #解压到服务目录 tar xvf node-v12.0.0-linux-x64.tar.gz -C /usr/local/node #软链接做全局命令 ln -s /usr/local/node/node-v12.0.0-linux-x64/bin/npm /usr/local/bin/npm ln -s /usr/local/node/node-v12.0.0-linux-x64/bin/node /usr/local/bin/node

3.3 maven环境安装

#下载地址 https://maven.apache.org/download.cgi #创建目录 mkdir /usr/local/maven/ #解压 tar xvf apache-maven-3.6.3-bin.tar.gz -C /usr/local/maven/ #软链接做全局命令 ln -s /usr/local/maven/apache-maven-3.6.3/bin/mvn /usr/bin/mvn

maven安装完成后配置setting文件，加入nexus的仓库 vim conf/settings.xml 加入nexus的账号密码及定义releases和snapshots仓库的id

<servers> <server> <id>maven-releases</id> <username>admin</username> <password>nexusnew</password> </server> <server> <id>maven-snapshots</id> <username>admin</username> <password>nexusnew</password> </server> </servers>

下载地址

<mirrors> <mirror> <id>nexus</id> <mirrorOf>*</mirrorOf> <name>nexus osc</name> <url>https://nexus.xxx.cn/repository/maven-public/</url> </mirror> </mirrors>

到此完成配置

开发上传下载在pom.xml中配置

<repositories> <!-- 配置nexus远程仓库 --> <repository> <id>nexus</id> <name>Nexus Snapshot Repository</name> <url>https://nexus.xxx.cn/repository/maven-public/</url> <releases> <enabled>true</enabled> </releases> <snapshots> <enabled>true</enabled> </snapshots> </repository> </repositories> <distributionManagement> <repository> <id>maven-releases</id>#此id对应maven中配置的id <name>Releases</name> <url>https://nexus.xxx.cn/repository/maven-releases</url> </repository> <snapshotRepository> <id>maven-snapshots</id>#此id对应maven中配置的id <name>Snapshot</name> <url>https://nexus.xxx.cn/repository/maven-snapshots</url> </snapshotRepository> </distributionManagement>

3.4 nexus安装配置

#下载 wget http://download.sonatype.com/nexus/3/nexus-3.14.0-04-unix.tar.gz #解压 tar -zxvf nexus-3.14.0-04-unix.tar.gz #修改启动端口 vim nexus-3.14.0-04/etc/nexus-default.properties #修改启动端口 application-port=8090 #设置打开最大文件数 vim /etc/security/limits.conf nexus soft nofile 65536 nexus hard nofile 65536 #启动 nohup ./nexus run &

页面登录 默认密码 admin admin123 注：记住修改密码在maven中的密码账号对应此的

进入maven-releases maven-snapshots 设置允许上传 nginx配置

upstream nexus-server{ server 127.0.0.1:8085; } server { listen 80; server_name nexus.xxx.cn; location / { return 301 https://$server_name$request_uri; } location ~ /.well-known { root /tmp; } } server { listen 443 ssl; server_name nexus.xxx.cn; client_max_body_size 100m; ssl_certificate /etc/nginx/conf.d/cert/nexus.xxx.cn.pem; #将domain name.pem替换成您证书的文件名。 ssl_certificate_key /etc/nginx/conf.d/cert/nexus.xxx.cn.key; #将domain name.key替换成您证书的密钥文件名。 ssl_session_timeout 5m; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4; #使用此加密套件。 ssl_protocols TLSv1 TLSv1.1 TLSv1.2; #使用该协议进行配置。 ssl_prefer_server_ciphers on; location / { proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto "https"; proxy_pass http://nexus-server; } }

二、项目管理

1.知识库搭建（conference）

注：我们用的付费版的 （🐕） 这里就写下破解版的搭建

一、搭建confluence 下载安装工具所需安装包 confluence_keygen.jar atlassian-confluence-5.4.4-x64.bin Confluence-5.4.4-language-pack-zh_CN.jar 51CTO下载-confluence5.1-crack.zip mysql-connector-java-5.1.32-bin.jar

链接：https://pan.baidu.com/s/1wOP_FB9lRyyGWF6EmEx-Dg 密码：xyuv 链接失效了自己在找找

chmod +x atlassian-confluence-5.4.4-x64.bin ./atlassian-confluence-5.4.4-x64.bin

选择安装 输入o

安装好了可以登录页面 此时需要破解

获取serverid 来破解

将/opt/atlassian/confluence/confluence/WEB-INF/lib路径下，atlassian-extras-2.4.jar下载到本地（windows）

执行破解工具

"C:\Program Files\Java\jdk1.8.0_191\bin\java.exe" -jar E:\BaiduNetdiskDownload\soft\confluence_keygen.jar

.path 上传atlassian-extras-2.4.jar 输入名字和serverid 生成新的atlassian-extras-2.4.jar

将atlassian-extras-2.4.jar覆盖到服务器/opt/atlassian/confluence/confluence/WEB-INF/lib/路径下

将mysql调用包放入mysql-connector-java-5.1.32-bin.jar /opt/atlassian/confluence/confluence/WEB-INF/lib

重启 /etc/init.d/confluence restart

贴入工具中的key

选择production installtion

选择数据库类型

5.4.4版本的confluence，貌似对mysql的存储引擎有要求，需要是InnoDB(我第一次没有改，用的MyISAM，下一步就出错了)。

show variables like '%storage_engine%';//查看默认存储引擎

如果是MyISAM的话，在/etc/my.cnf文件[mysqld]下添加default-storage-engine=InnoDB，重启mysql

/etc/init.d/mysqld restart mysql -uroot -p create database wiki character set UTF8; grant all on wiki.* to wiki_user@"%" identified by "wiki_password";

点击“Direct JDBC”

Driver Class Name ：默认无需更改 Database URL:修改对应IP，port，database, jdbc:mysql://127.0.0.1:3306/wiki?useUnicode=true&characterEncoding=UTF8&sessionVariables=storage_engine%3DInnoDB

UTF8&sessionVariables=storage_engine 这里需要填入数据库的值

show variables like '%storage_engine%'

上一步成功，点击“Empty Site”

下一个页面点击manager配置用户和组

参考 https://www.jianshu.com/p/7aeeb4f9a8c9

nginx配置

server { listen 80; server_name wiki.xxx.cn; return 301 https://$server_name$request_uri; } server { listen 443 ssl; server_name wiki.xxx.cn; ssl_certificate /etc/nginx/conf.d/cert/wiki.xxx.cn.pem; ssl_certificate_key /etc/nginx/conf.d/cert/wiki.xxx.cn.key; ssl_session_timeout 5m; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS'; ssl_prefer_server_ciphers on; location / { client_max_body_size 100m; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://localhost:8090; } location /synchrony { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://localhost:8091/synchrony; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; } location /server-info.action { proxy_pass http://localhost:8090/confluence/server-info.action; } }

2.接口文档搭建（yapi）

安装node 可参考上述的node环境安装

添加mongodb源文件，在/etc/yum.repos.d 创建一个

vim /etc/yum.repos.d/mongodb-org.repo [mongodb-org] name=MongoDB Repository baseurl=http://mirrors.aliyun.com/mongodb/yum/redhat/7Server/mongodb-org/3.2/x86_64/ gpgcheck=0 enabled=1

安装mongodb

yum install -y mongodb-org

启动mongodb

service mongod start

设置开机启动

chkconfig mongod on

配置远程访问，修改mongod.conf配置文件

vim /etc/mongod.conf #bindIp: 127.0.0.1

重启mongod

service mongod restart

安装git

yum -y install git

搭建YApi

npm install -g yapi-cli --registry https://registry.npm.taobao.org

启动

yapi server

启动后可访问127.0.0.1:9090进行安装

nginx配置

server { listen 80; server_name yapi.xxx.cn; return 301 https://$server_name$request_uri; } upstream yapi { server 127.0.0.1:3000 weight=5; } server { listen 443 ssl; server_name yapi.xxx.cn; #charset koi8-r; #access_log logs/host.access.log main; ssl_certificate /etc/nginx/conf.d/cert/all.xxx.cn.pem; ssl_certificate_key /etc/nginx/conf.d/cert/all.xxx.cn.key; ssl_session_timeout 5m; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4; #使用此加密套件。 ssl_protocols TLSv1 TLSv1.1 TLSv1.2; #使用该协议进行配置。 ssl_prefer_server_ciphers on; location / { proxy_pass http://yapi; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-NginX-Proxy true; proxy_set_header Connection "upgrade"; proxy_set_header Upgrade $http_upgrade; proxy_http_version 1.1; } }