Ambari介绍
Apache Ambari是一个基于Web的支持Apache Hadoop集群的供应、管理和监控的开源工具， Ambari已支持大多数Hadoop组件，包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeeper、Sqoop和Hcatalog等。提供Web UI进行可视化的集群管理，简化了大数据平台的安装、使用难度。
Ambari体系结构
Ambari 自身也是一个分布式架构的软件，主要由两部分组成：Ambari Server 和 Ambari Agent。
简单来说，用户通过Ambari Server通知 Ambari Agent 安装对应的软件；Agent 会定时地发送各个机器每个软件模块的状态给 Ambari Server，最终这些状态信息会呈现在 Ambari 的 GUI，方便用户了解到集群的各种状态，并进行相应的维护。Ambari Server 从整个集群上收集信息。每个主机上都有 Ambari Agent, Ambari Server 通过 Ambari Agent 控制每部主机。
下面介绍CentOS7系统下利用Ambari搭建HDP大数据平台
参考官方文档
https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/install-ambari-server-rhel7.html https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/hdp_26_repositories.html https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/ambari_repositories.html
一、准备条件
1、jdk安装包 jdk-8u261-linux-x64.tar.gz
2、提前下载好ambari,HDP,HDP-UTILS 本地仓库Tarball文件
http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.2/ambari-2.6.2.2-centos7.tar.gzhttp://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gzhttp://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
二、操作系统基础环境准备
先在ambari机器上配置JDK，准备基础环境
1)、关闭防火墙
systemctl disable firewalldsystemctl stop firewalld2)、关闭SELINUX
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/configsetenforce 0
3)、设置ulimit 文件打开数为10000
在/etc/profile最后一行添加ulimit -n 10000
4)、安装JDK，配置环境变量
cd /opttar -zxvf jdk-8u261-linux-x64.tar.gzmv jdk1.8.0_261 /usr/local/cd /usr/local/mv jdk1.8.0_261 jdk#在/etc/profile中添加如下行vi /etc/profileexport JAVA_HOME=/usr/local/jdkexport CLASSPATH=.:$JAVA_HOME/libPATH=$PATH:$JAVA_HOME/bin5)、source /etc/profile生效
最后检查java，ulimit值
以上配置做好之后，可以poweroff关机做一个虚拟机快照，按这个虚拟机模板克隆4台机器 分别为bigdata1,bigdata2,bigdata3,bigdata4
三、主机名修改与免密钥登录配置
1)5台主机分别修改主机名
hostnamectl set-hostname ambarihostnamectl set-hostname bigdata1hostnamectl set-hostname bigdata2hostnamectl set-hostname bigdata3hostnamectl set-hostname bigdata42)配置ambari-server免密钥登录其他4台节点
ssh-keygen3)配置主机域名解析
vi /etc/hosts添加如下行10.20.90.59 ambari10.20.90.60 bigdata110.20.90.61 bigdata210.20.90.62 bigdata310.20.90.63 bigdata44)拷贝公钥
ssh-copy-id root@bigdata1ssh-copy-id root@bigdata2ssh-copy-id root@bigdata3ssh-copy-id root@bigdata4ssh-copy-id root@ambari
5)主机域名解析文件拷贝到4台节点上
scp /etc/hosts root@bigdata1:/etcscp /etc/hosts root@bigdata2:/etcscp /etc/hosts root@bigdata3:/etcscp /etc/hosts root@bigdata4:/etc
并验证可以正常免密登录
四、配置Ambari HDP以及HDP-UTILS 本地yum源
在ambari服务器上配置Ambari HDP以及HDP-UTILS 本地yum源
yum install -y httpdcd /optmkdir /var/www/html/ambarimkdir /var/www/html/hdptar -zxf ambari-2.6.2.2-centos7.tar.gz -C /var/www/html/ambari/tar -zxf HDP-2.6.5.0-centos7-rpm.tar.gz -C /var/www/html/hdp/tar -zxf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp/ systemctl start httpdsystemctl enable httpdSFTP上传到/opt目录下
验证Ambari HDP以及HDP-UTILS 本地yum源均可以正常访问
http://10.20.90.59/ambari/ambari/centos7/2.6.2.2-1/http://10.20.90.59/hdp/HDP/centos7/2.6.5.0-292/http://10.20.90.59/hdp/HDP-UTILS/centos7/1.1.0.22/
针对下面这个错误的解决办法
删除或者移走html文件
配置ambari YUM源
cat >/etc/yum.repos.d/ambari.repo<#配置hdp YUM源
cat >/etc/yum.repos.d/hdp.repo<
cat /etc/yum.repos.d/ambari.repocat /etc/yum.repos.d/hdp.repo五、安装ambari-server
yum -y install ambari-server
六、安装MariaDB数据库并配置数据库
1、安装MariaDB数据库
yum -y install mariadb-serversystemctl enable mariadb.servicesystemctl start mariadb.servicenetstat -anp | grep 3306
2、设置数据库初始密码
mysqladmin -uroot password "ambari"3、设置可以远程登录
mysql -uroot -pambarigrant all privileges on *.* to 'root'@'ambari' identified by 'ambari' with grant option;grant all privileges on *.* to 'root'@'10.20.90.59' identified by 'ambari' with grant option;flush privileges;systemctl restart mariadb.service4、创建ambari数据库
mysql -uroot -pambari -hambari create database ambari DEFAULT CHARSET utf8;use ambarisource /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
七、ambari-server初始化配置
[root@ambari home]# ambari-server setupUsing python /usr/bin/pythonSetup ambari-serverChecking SELinux...SELinux status is 'disabled'Customize user account for ambari-server daemon [y/n] (n)? yEnter user account for ambari-server daemon (root):rootAdjusting ambari-server permissions and ownership...Checking firewall status...Checking JDK...[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7[3] Custom JDK==============================================================================Enter choice (1): 3WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.Path to JAVA_HOME: /usr/local/jdkValidating JDK on Ambari Server...done.Checking GPL software agreement...GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.htmlEnable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? nCompleting setup...Configuring database...Enter advanced database configuration [y/n] (n)? yConfiguring database...==============================================================================Choose one of the following options:[1] - PostgreSQL (Embedded)[2] - Oracle[3] - MySQL / MariaDB[4] - PostgreSQL[5] - Microsoft SQL Server (Tech Preview)[6] - SQL Anywhere[7] - BDB==============================================================================Enter choice (1): 3Hostname (localhost): Port (3306): Database name (ambari): Username (ambari): rootEnter Database Password (bigdata): ambariRe-enter password: Configuring ambari database...WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java and set property "server.jdbc.driver.path=[path/to/custom_jdbc_driver]" in ambari.properties.Press to continue.Configuring remote database connection properties...WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sqlProceed with configuring remote database connection properties [y/n] (y)? yExtracting system views.......ambari-admin-2.6.2.2.1.jar.......Adjusting ambari-server permissions and ownership...Ambari Server 'setup' completed successfully.拷贝jdbc驱动包，并在ambari.properties配置文件中添加jdbc驱动的配置
mkdir /usr/share/javacp /home/mysql-connector-java-8.0.20.jar /usr/share/java/mysql-connector-java.jarvi /etc/ambari-server/conf/ambari.properties #追加一行server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar
ambari-server start
八、ambari web界面向导搭建集群
注意：这里的URL最后面的/斜杠需要去掉，(图误)
九、排错
1、Confirm Hosts时出现安装ambari-agent的报错的解决方法
修改 /etc/ambari-agent/conf/ambari-agent.ini 文件，在 [security] 配置部分添加[security]...force_https_protocol=PROTOCOL_TLSv1_22、yum.conf配置文件修改
vi /etc/yum.conf installonly_limit=5改为installonly_limit=600
