一、Hadoop 集群背景
1、安装OpenJDK1.8 并查看安装位置
javascript:void(0)
2、Centos7.x 安装部署 Zookeeper 3.6.2 集群
javascript:void(0)
3、Hadoop 3.x 部署 YARN 集群及基本操作
javascript:void(0)
1、启动集群的 HDFS 服务
[root@master ~]# /usr/bigdata/hadoop-3.3.0/sbin/start-dfs.sh
2、编辑文件
[root@master ~]# vim /usr/bigdata/hadoop-3.3.0/etc/hadoop/yarn-site.xml
3、增加 / 编辑内容
<property>
<name>yarn.timeline-service.enabled</name>
<value>true</value>
<description>Indicate to clients whether Timeline service is enabled or not.
If enabled, the TimelineClient library used by end-users will post entities
and events to the Timeline server.</description>
</property>
<property>
<name>yarn.timeline-service.hostname</name>
<value>master</value>
<description>The hostname of the Timeline service web application.</description>
</property>
<property>
<name>yarn.timeline-service.address</name>
<value>master:10200</value>
<description>Address for the Timeline server to start the RPC server.</description>
</property>
<property>
<name>yarn.timeline-service.webapp.address</name>
<value>master:8188</value>
<description>The http address of the Timeline service web application.</description>
</property>
<property>
<name>yarn.timeline-service.webapp.https.address</name>
<value>master:8190</value>
<description>The https address of the Timeline service web application.</description>
</property>
<property>
<name>yarn.timeline-service.handler-thread-count</name>
<value>10</value>
<description>Handler thread count to serve the client RPC requests.</description>
</property>
<property>
<name>yarn.timeline-service.http-cross-origin.enabled</name>
<value>false</value>
<description>Enables cross-origin support (CORS) for web services where
cross-origin web response headers are needed. For example, javascript making
a web services request to the timeline server，是否支持请求头跨域</description>
</property>
<property>
<name>yarn.timeline-service.http-cross-origin.allowed-origins</name>
<value>*</value>
<description>Comma separated list of origins that are allowed for web
services needing cross-origin (CORS) support. Wildcards (*) and patterns
allowed，#需要跨域源支持的web服务所允许的以逗号分隔的列表</description>
</property>
<property>
<name>yarn.timeline-service.http-cross-origin.allowed-methods</name>
<value>GET,POST,HEAD</value>
<description>Comma separated list of methods that are allowed for web
services needing cross-origin (CORS) support.，跨域所允许的请求操作</description>
</property>
<property>
<name>yarn.timeline-service.http-cross-origin.allowed-headers</name>
<value>X-Requested-With,Content-Type,Accept,Origin</value>
<description>Comma separated list of headers that are allowed for web
services needing cross-origin (CORS) support.允许用于web的标题的逗号分隔列表</description>
</property>
<property>
<name>yarn.timeline-service.http-cross-origin.max-age</name>
<value>1800</value>
<description>The number of seconds a pre-flighted request can be cached
for web services needing cross-origin (CORS) support.可以缓存预先传送的请求的秒数</description>
</property>
<property>
<name>yarn.timeline-service.generic-application-history.enabled</name>
<value>true</value>
<description>Indicate to clients whether to query generic application
data from timeline history-service or not. If not enabled then application
data is queried only from Resource Manager.
向资源管理器和客户端指示是否历史记录-服务是否启用。如果启用，资源管理器将启动
记录工时记录服务可以使用历史数据。同样，当应用程序如果启用此选项，请完成.</description>
</property>
<property>
<name>yarn.timeline-service.generic-application-history.store-class</name>
<value>org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore</value>
<description>Store class name for history store, defaulting to file system store</description>
</property>
<property>
<description>Store class name for timeline store.</description>
<name>yarn.timeline-service.store-class</name>
<value>org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore</value>
</property>
<property>
<description>Enable age off of timeline store data.</description>
<name>yarn.timeline-service.ttl-enable</name>
<value>true</value>
</property>
<property>
<description>Time to live for timeline store data in milliseconds.</description>
<name>yarn.timeline-service.ttl-ms</name>
<value>6048000000</value>
</property>
<property>
<name>hadoop.zk.address</name>
<value>slave1:2181,slave2:2181,slave3:2181</value>
</property>
<property>
<name>yarn.resourcemanager.system-metrics-publisher.enabled</name>
<value>true</value>
<description>The setting that controls whether yarn system metrics is published on the timeline server or not by
RM.</description>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
<description>解决不能查看日志的问题</description>
</property>
<property>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/tmp/logs</value>
</property>
4、发送文件至 YARN 集群的各个节点
[root@master ~]# scp /usr/bigdata/hadoop-3.3.0/etc/hadoop/yarn-site.xml slave1:/usr/bigdata/hadoop-3.3.0/etc/hadoop/
[root@master ~]# scp /usr/bigdata/hadoop-3.3.0/etc/hadoop/yarn-site.xml slave2:/usr/bigdata/hadoop-3.3.0/etc/hadoop/
[root@master ~]# scp /usr/bigdata/hadoop-3.3.0/etc/hadoop/yarn-site.xml slave3:/usr/bigdata/hadoop-3.3.0/etc/hadoop/
5、重启 YARN 的相关服务
（1）关闭 yarn：
[root@master ~]#/ usr/bigdata/hadoop-3.3.0/sbin/stop-yarn.sh
（2）启动 yarn：
[root@master ~]# /usr/bigdata/hadoop-3.3.0/sbin/start-yarn.sh
6、启动 timelineserver（ApplicationHistoryServer） 服务
[root@master ~]# /usr/bigdata/hadoop-3.3.0/sbin/yarn-daemon.sh start timelineserver
7、关闭服务
[root@master ~]# /usr/bigdata/hadoop-3.3.0/sbin/yarn-daemon.sh stop timelineserver
1、Hive 3.1.2 分布式集群
javascript:void(0)
2、执行 hive 命令
hive (default)> use hive_test;
hive (hive_test)> select count(1) from table_test where age > 20;
3、浏览器查看 ResourceManager 的 Web 页面
http://master:8088/
4、浏览器查看 ApplicationHistoryServer 的 Web 页面
http://master:8188/
至此，Hadoop 3.x 集群 YARN 启动 timelineserver（ApplicationHistoryServer） 服务操作完毕，希望能够对您有所帮助！
