hive集群搭建
一、hive安装和配置
1、在/opt路径下
[root@hadoop-93 opt]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.4/apache-hive-2.3.4-bin.tar.gz
2、配置环境变量
vi /etc/profile
export HIVE_HOME=/opt/apache-hive-2.3.4-bin/
export HIVE_CONF_DIR=CLASSPATH:
PATH:$HIVE_HOME/bin
source /etc/profile
二、配置mysql（注：切换到root用户）
1、卸载CentOS自带的MySQL
rpm -qa | grep mysql
rpm -e mysql-libs-5.1.66-2.el6_3.i686 --nodeps
2、下载mysql的repo源
wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
3、安装mysql-community-release-el7-5.noarch.rpm包（repo源）
rpm -ivh mysql-community-release-el7-5.noarch.rpm
4、安装MySQL
wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar
yum install mysql-server
#yum install mysql
#yum install mysql-server
#yum install mysql-devel
service mysqld restart
5、初始化MySQL
(1)修改mysql的密码(root权限执行)
cd /usr/bin
./mysql_secure_installation
(2)输入当前MySQL数据库的密码为root, 初始时root是没有密码的, 所以直接回车
Enter current password for root (enter for none):
(3)设置MySQL中root用户的密码(应与下面Hive配置一致,下面设置为123456)
Set root password? [Y/n] Y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables…
… Success!
(4)删除匿名用户
Remove anonymous users? [Y/n] Y
… Success!
(5)是否不允许用户远程连接,选择N
Disallow root login remotely? [Y/n] N
… Success!
(6)删除test数据库
Remove test database and access to it? [Y/n] Y
Dropping test database…
… Success!
Removing privileges on test database…
… Success!
(7)重装
Reload privilege tables now? [Y/n] Y
… Success!
(8)完成
All done! If you’ve completed all of the above steps, your MySQL
installation should now be secure.
Thanks for using MySQL!
(9)登陆mysql
mysql -uroot -p
grant all privileges on . to root@'192.168.32.93’identified by ‘123456’;
三、hive安装和配置
1、下载hive
wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.4/apache-hive-2.3.4-bin.tar.gz
2、将hive解压到/opt下
tar -zxvf apache-hive-2.3.4-bin.tar.gz
3、修改环境变量/etc/profile：
export HIVE_HOME=/opt/apache-hive-2.3.4-bin/
export HIVE_CONF_DIR=CLASSPATH:
PATH:$HIVE_HOME/bin
4、source /etc.profile
5、hive --version
成功or失败
6、修改hive-site.xml：
touch hive-site.xml
vim hive-site.xml
javax.jdo.option.ConnectionURL
jdbc:mysql://192.168.32.93:3306/hive?createDatabaseIfNotExsit=true;characterEncoding=UTF-8
javax.jdo.option.ConnectionDriverName
com.mysql.jdbc.Driver
javax.jdo.option.ConnectionUserName
root
javax.jdo.option.ConnectionPassword
123456
datanucleus.readOnlyDatastore
false
datanucleus.fixedDatastore
false
datanucleus.autoCreateSchema
true
datanucleus.autoCreateTables
true
datanucleus.autoCreateColumns
true
datanucleus.autoCreateColumns
true
hive.metastore.schema.verification
false hive.metastore.warehouse.dir /user/hive/warehouse location of default database for the warehouse
hive.downloaded.resources.dir /opt/apache-hive-2.3.4-bin/tmp/resources Temporary local directory for added resources in the remote file system.
hive.exec.dynamic.partition true hive.exec.dynamic.partition.mode nonstrict
hive.exec.local.scratchdir /opt/apache-hive-2.3.4-bin/tmp/HiveJobsLog Local scratch space for Hive jobs hive.downloaded.resources.dir /opt/apache-hive-2.3.4-bin/tmp/ResourcesLog Temporary local directory for added resources in the remote file system. hive.querylog.location /opt/apache-hive-2.3.4-bin/tmp/HiveRunLog Location of Hive run time structured log file hive.server2.logging.operation.log.location /opt/apache-hive-2.3.4-bin/tmp/OpertitionLog Top level directory where operation tmp are stored if logging functionality is enabled
7、配置hive-env.sh
cp hive-env.sh.template hive-env.sh
export HADOOP_HOME=/opt/hadoop-3.0.2
export HIVE_CONF_DIR=/opt/apache-hive-2.3.4-bin/conf/
export HIVE_AUX_JARS_PATH=/opt/apache-hive-2.3.4-bin/lib/
8、复制mysql的驱动程序到hive/lib下面
wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar9、在mysql中hive的schema（在此之前需要创建mysql下的hive数据库）
[root@s100 bin]# pwd
/usr/local/hive/bin
[root@s100 bin]# schematool -dbType mysql -initSchema10、执行hive命令
成功进入hive界面，hive配置完成11、查询mysql（hive这个库是在 schematool -dbType mysql -initSchema 之前创建的！）
use hive;
那我们做这些事干什么的呢，下面小段测试大家感受一下
hive测试：
备注：这里是第二个配置文件的演示：所以数据库名称是hive数据库！
1.需要知道现在的hadoop中的HDFS存了什么
[root@hadoop-93 bin]# hadoop fs -lsr /
2.进入hive并创建一个测试库和测试表
[root@hadoop-93 bin]# hive
创建库：
hive> create database hive_1;
显示库：
hive> show databases;
OK
default
hive_1
Time taken: 0.19 seconds, Fetched: 2 row(s)
创建库成功！3.查询一下HDFS有什么变化
多了一个库hive_1
那么我们的mysql下的hahive库有什么变化
mysql -uroot -p123456mysql> use hive
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from DBS;
4.在hive_1下创建一个表hive_01
hive> use hive_1;
OK
Time taken: 0.754 seconds
hive> create table hive_01 (id int,name string);
OK
Time taken: 2.447 seconds
hdfs下的情况：
mysql下：
那么在web端是什么样子的呢！
http://hadoop-93:50070 或者 192.168.32.93:50070
