本篇内容介绍了“Hive2的安装配置方法”的有关知识,在实际案例的操作过程中,不少人都会遇到这样的困境,接下来就让小编带领大家学习一下如何处理这些情况吧!希望大家仔细阅读,能够学有所成!
一、环境依赖
Hive必须运行在Hadoop之上,则需要先安装Hadoop环境
http://my.oschina.net/u/204498/blog/519789
二、安装Hive
1.下载Hive
http://apache.mirrors.ionfish.org/hive/
我安装的是apache-hive-1.2.1-bin.tar.gz
[hadoop@hftclclw0001 ~]$ pwd
/home/hadoop
[hadoop@hftclclw0001 ~]$ wget http://apache.mirrors.ionfish.org/hive/hive-1.2.1/apache-hive-1.2.1-bin.tar.gz
[hadoop@hftclclw0001 ~]$ ll
total 637256
drwx------ 10 hadoop root 4096 Oct 27 02:22 apache-hive-1.2.1-bin
-rw------- 1 hadoop root 92834839 Jun 26 18:34 apache-hive-1.2.1-bin.tar.gz
drwx------ 3 hadoop root 4096 Oct 27 09:05 data
drwx------ 11 hadoop root 4096 Oct 21 03:20 hadoop-2.7.1
-rw------- 1 hadoop root 210606807 Oct 20 09:00 hadoop-2.7.1.tar.gz
drwx------ 2 hadoop root 4096 Oct 23 02:08 install-sqoop
drwx------ 13 hadoop root 4096 Oct 20 09:22 spark-1.5.1-bin-hadoop2.6
-rw------- 1 hadoop root 280901736 Oct 20 09:19 spark-1.5.1-bin-hadoop2.6.tgz
drwx------ 22 hadoop root 4096 Oct 23 02:08 sqoop-1.99.6-bin-hadoop200
-rw------- 1 hadoop root 68177818 May 5 22:34 sqoop-1.99.6-bin-hadoop200.tar.gz
[hadoop@hftclclw0001 ~]$ cd apache-hive-1.2.1-bin/conf/
[hadoop@hftclclw0001 conf]$ pwd
/home/hadoop/apache-hive-1.2.1-bin/conf
[hadoop@hftclclw0001 conf]$ vi hive-env.sh
...
...
HADOOP_HOME=/home/hadoop/hadoop-2.7.1 =>配置Hadoop_Home
export HIVE_CONF_DIR=/home/hadoop/apache-hive-1.2.1-bin/conf =>配置HIVE_conf_home
export HIVE_AUX_JARS_PATH=/home/hadoop/apache-hive-1.2.1-bin/lib/
#我使用了mysql作为metastore ,则需要在lib目录下添加mysql的驱动
[hadoop@hftclclw0001 lib]$ pwd
/home/hadoop/apache-hive-1.2.1-bin/lib
[hadoop@hftclclw0001 lib]$ ll | grep mysql
-rw------- 1 hadoop root 848401 Oct 27 01:48 mysql-connector-java-5.1.25-bin.jar
[hadoop@hftclclw0001 conf]$ vi hive-site.xml
[hadoop@hftclclw0001 conf]$ cat hive-site.xml
<?xml version="1.0"?>
<configuration>
<property>
<name>hive.metastore.local</name>
<value>false</value> =>metastore我的mysql不是在该server上
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://{ip:port}/{databases}</value> => mysql服务的ip和端口号
</property>
<property>
<name>javax.jdo.option.ConnectionDriveName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>{username}</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>{password}</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/hive/warehouse</value> =>hive的仓库目录,需要在HDFS上创建,并修改权限
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://{ip}:{port}</value> =>本机ip和端口号,启动metastore服务
</property>
</configuration>
[hadoop@hftclclw0001 conf]$ vi hive-log4j.properties => Log4j的配置,可以修改日志目录
...
2.启动metastore
[hadoop@hftclclw0001 bin]$ pwd
/home/hadoop/apache-hive-1.2.1-bin/bin
[hadoop@hftclclw0001 bin]$ ./hive --service metastore &
[hadoop@hftclclw0001 bin]$ ps ax|grep metastore
...
...
3.启动HiveServer2
[hadoop@hftclclw0001 bin]$ pwd
/home/hadoop/apache-hive-1.2.1-bin/bin
[hadoop@hftclclw0001 bin]$ ./hive --service hiveserver2 &
[hadoop@hftclclw0001 bin]$ ps ax|grep HiveServer2
...
...
4.启动shell 或是 beeline
[hadoop@hftclclw0001 bin]$ ./hive shell
...
...
三、metastore
http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh5ig_topic_18_4.html
1.内置模式:将数据保存在内置的Derby数据库中,这种方式最简单,但是Derby每次只能访问一个数据文件。
Drive ==> Metastore ==> Derby
2.本地模式:将元数据保存在本地的独立数据库(如mysql)等
Driver ===> Metastore
Driver ===> Metastore ===> DB
Driver ===> Metastore
每个server都需要配置metastore,并启动metastore服务
3.远程模式:使用thrift访问metastore
Client1
Client2 ===> Metastore ===> DB
Client3
4.配置:
如上述配置,我们已经启动了metastore服务在上述hftclclw0001机器上,
我们在另一台server上,如hftclcld0001机器上,安装hive,配置如上述配置一直,仅仅修改hive-site.xml如下:
<?xml version="1.0"?>
<configuration>
<property>
<name>hive.metastore.uris</name>
<value>thrift://{ip}:{port}</value> =>hftclclw0001机器的ip和端口号
</property>
</configuration>
即我们通过thrift协议,访问hftclclw0001上面的metastore,并访问hive的元数据
[root@hftclcld0001 apache-hive-1.2.1-bin]# pwd
/home/hadoop/apache-hive-1.2.1-bin
[root@hftclcld0001 apache-hive-1.2.1-bin]# ./bin/hive shell
hive>
hive> show databases;
OK
default
hive =>能访问到hive的metastore,访问到元数据(我们之前创建的)
human_resources
Time taken: 0.388 seconds, Fetched: 3 row(s)
hive>
“Hive2的安装配置方法”的内容就介绍到这里了,感谢大家的阅读。如果想了解更多行业相关的知识可以关注天达云网站,小编将为大家输出更多高质量的实用文章!