azkaban安装

tech2023-05-27  50

安装要求:

1,安装了jdk(1.8版本以上)

2,mysql数据库(azkaban元数据使用)

下载azkaban源码

地址:https://github.com/azkaban/azkaban/releases

选择喜欢的版本下载

wget https://github.com/azkaban/azkaban/archive/3.84.3.tar.gz

编译

cd azkaban-master;   ./gradlew build installDist -x test

因为网络原因编译的时间很长,请耐心等待,如果编译过程中出现缺少依赖的错误,请自行度娘。。。

配置解压

去这四个文件目录下/build/distributions里,取出tar.gz文件

在/opt下创建azkaban目录

mkdir azkaban mv /opt/azkaban-master/azkaban-db/build/distributions/azkaban-db-0.1.0-SNAPSHOT.tar.gz /opt/azkaban/ mv /opt/azkaban-master/azkaban-exec-server/build/distributions/azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz /opt/azkaban/ mv /opt/azkaban-master/azkaban-web-server/build/distributions/azkaban-web-server-0.1.0-SNAPSHOT.tar.gz /opt/azkaban/ mv /opt/azkaban-master/azkaban-solo-server/build/distributions/azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz /opt/azkaban/

cd /opt/azkaban/ tar -zxvf azkaban-db-0.1.0-SNAPSHOT.tar.gz tar -zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz tar -zxvf azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz tar -zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz mv azkaban-db-0.1.0-SNAPSHOT/ azkaban-db mv azkaban-exec-server-0.1.0-SNAPSHOT/ azkaban-exec-server mv azkaban-solo-server-0.1.0-SNAPSHOT/ azkaban-solo-server mv azkaban-web-server-0.1.0-SNAPSHOT/ azkaban-web-server rm -f *.gz

设置元数据库

登录 mysql -uroot -p create database azkaban; use azkaban ; source /opt/azkaban/azkaban-db/create-all-sql-0.1.0-SNAPSHOT.sql ;

修改配置文件

azkaban有3种模式,

solo-server模式:exec进程和web进程为同一个进程,存放元数据的数据库为H2

two-server模式:与之前的单机版本类似,exec进程和web进程分开,存放元数据的数据库为mysql

multiple-executor模式:exec进程和web进程在不同的机器上,存放元数据的数据库为mysql

本次安装使用two-server模式,故需要修改azkaban-exec-server和azkaban-web-server的azkaban.properties里的参数

exec-server:只需修改host和user和password,以及时区default.timezone.id=Asia/Shanghai

# Azkaban Personalization Settings azkaban.name=Test azkaban.label=My Local Azkaban azkaban.color=#FF3601 azkaban.default.servlet.path=/index web.resource.dir=web/ default.timezone.id=America/Los_Angeles # Azkaban UserManager class user.manager.class=azkaban.user.XmlUserManager user.manager.xml.file=conf/azkaban-users.xml # Loader for projects executor.global.properties=conf/global.properties azkaban.project.dir=projects # Velocity dev mode velocity.dev.mode=false # Azkaban Jetty server properties. jetty.use.ssl=false jetty.maxThreads=25 jetty.port=8081 # Where the Azkaban web server is located azkaban.webserver.url=http://localhost:8081 # mail settings mail.sender= mail.host= job.failure.email= job.success.email= lockdown.create.projects=false cache.directory=cache # JMX stats jetty.connector.stats=true executor.connector.stats=true # Azkaban plugin settings azkaban.jobtype.plugin.dir=plugins/jobtypes # Azkaban mysql settings by default. Users should configure their own username and password. database.type=mysql mysql.port=3306 mysql.host=localhost mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100 # Azkaban Executor settings executor.maxThreads=50 executor.flow.threads=30

web-server:修改host和user和password,以及时区default.timezone.id=Asia/Shanghai

# Azkaban Personalization Settings azkaban.name=Test azkaban.label=My Local Azkaban azkaban.color=#FF3601 azkaban.default.servlet.path=/index web.resource.dir=web/ default.timezone.id=Asia/Shanghai # Azkaban UserManager class user.manager.class=azkaban.user.XmlUserManager user.manager.xml.file=conf/azkaban-users.xml # Loader for projects executor.global.properties=conf/global.properties azkaban.project.dir=projects # Velocity dev mode velocity.dev.mode=false # Azkaban Jetty server properties. jetty.use.ssl=false jetty.maxThreads=25 jetty.port=8081 # Azkaban Executor settings # mail settings mail.sender= mail.host= job.failure.email= job.success.email= lockdown.create.projects=false cache.directory=cache # JMX stats jetty.connector.stats=true executor.connector.stats=true # Azkaban mysql settings by default. Users should configure their own username and password. database.type=mysql mysql.port=3306 mysql.host=localhost mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100 #Multiple Executor azkaban.use.multiple.executors=true azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1 azkaban.executorselector.comparator.Memory=1 azkaban.executorselector.comparator.LastDispatched=1 azkaban.executorselector.comparator.CpuUsage=1

添加修改用户/azkaban-web-server/conf/ azkaban-users.xml

<azkaban-users> <user groups="azkaban" password="azkaban123" roles="admin" username="azkaban"/> <user password="metrics" roles="metrics" username="metrics"/> <role name="admin" permissions="ADMIN"/> <role name="metrics" permissions="METRICS"/> </azkaban-users>

启动

一定要先启动exec-server,再启动web-server,不然报错 启动exec-server

cd /opt/azkaban/azkaban-exec-server/ bin/start-exec.sh 手动修改exec-srver状态为激活 cat executor.port获取端口号 curl http://${executorHost}:${executorPort}/executor?action=activate

启动web-server

cd /opt/azkaban/azkaban-web-server/ bin/start-web.sh

登录azkaban web-ui

默认地址是http://localhost:8081

登录成功后,界面如下:

至此azkaban已经安装完成了

最新回复(0)