hive2.1.0 安装-白红宇

hive2.1.0 安装

阅读量：6702 次

发布时间：2019-06-25

本文共 12383 字，大约阅读时间需要 41 分钟。

参数配置地址

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration

1、配置HIVE_HOME 和PATH

HIVE_HOME=/opt/apache-hive-2.1.0-binPATH=$JAVA_BIN:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$PATHexport JAVA_HOME JAVA_BIN HADOOP_HOME HIVE_HOME  PATH CLASSPATH

2、拷贝hive-default.xml.template 为hive-site.xml


        
     
      javax.jdo.option.ConnectionPassword
         
     
      123456
         
     
      数据库连接密码
       
    
        
     
      javax.jdo.option.ConnectionURL
         
     
      jdbc:mysql://192.168.8.145:3306/hive?createDatabaseIfNotExist=true
         
     
            JDBC connect string for a JDBC metastore.      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.    
       
      
        
     
      javax.jdo.option.ConnectionDriverName
         
     
      com.mysql.jdbc.Driver
         
     
      驱动
       
      
        
     
      javax.jdo.option.ConnectionUserName
         
     
      root
         
     
      用户名
       
      
        
     
      hive.exec.local.scratchdir
         
     
      /home/hive/local
         
     
      Local scratch space for Hive jobs
       
      
        
     
      hive.downloaded.resources.dir
         
     
      /home/hive/resources
         
     
      Temporary local directory for added resources in the remote file system.

3、拷贝hive-env.sh.template 为hive-env.sh

修改

# Set HADOOP_HOME to point to a specific hadoop install directory HADOOP_HOME=/opt/hadoop-2.7.2# Hive Configuration Directory can be controlled by: export HIVE_CONF_DIR=/opt/apache-hive-2.1.0-bin/conf

4、下载 mysql-connector-java-5.1.39.jar 到hive 的lib包中

5 初始化数据库

[root@master opt]# schematool -initSchema -dbType mysqlwhich: no hbase in (/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/opt/apache-hive-2.1.0-bin/bin:/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/opt/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]Metastore connection URL:      	 jdbc:mysql://192.168.8.145:3306/hive?createDatabaseIfNotExist=trueMetastore Connection Driver :  	 com.mysql.jdbc.DriverMetastore connection User:     	 rootTue Aug 23 18:09:10 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Starting metastore schema initialization to 2.1.0Initialization script hive-schema-2.1.0.mysql.sqlTue Aug 23 18:09:11 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Initialization script completedTue Aug 23 18:09:15 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.schemaTool completed

6、出现以下错误

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D       	at org.apache.hadoop.fs.Path.initialize(Path.java:205)       	at org.apache.hadoop.fs.Path.
    
     (Path.java:171)       	at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:631)       	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550)       	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)       	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)       	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)       	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)       	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)       	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)       	at java.lang.reflect.Method.invoke(Method.java:606)       	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)       	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D       	at java.net.URI.checkPath(URI.java:1804)       	at java.net.URI.
     
      (URI.java:752)       	at org.apache.hadoop.fs.Path.initialize(Path.java:202)       	... 12 more

需要修改


        
     
      hive.exec.local.scratchdir
         
     
      ${system:java.io.tmpdir}/${system:user.name}
         
     
      Local scratch space for Hive jobs
       
     
        
     
      hive.downloaded.resources.dir
         
     
      ${system:java.io.tmpdir}/${hive.session.id}_resources
         
     
      Temporary local directory for added resources in the remote file system.

改成


        
     
      hive.exec.local.scratchdir
         
     
      /home/hive/local
         
     
      Local scratch space for Hive jobs
       
      
        
     
      hive.downloaded.resources.dir
         
     
      /home/hive/resources
         
     
      Temporary local directory for added resources in the remote file system.

输入hive

[root@master conf]# hivewhich: no hbase in (/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/opt/apache-hive-2.1.0-bin/bin:/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/opt/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]Logging initialized using configuration in jar:file:/opt/apache-hive-2.1.0-bin/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: trueTue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.hive> show databases;OKdefaultTime taken: 1.16 seconds, Fetched: 1 row(s)hive>

#hadoop fs -mkdir -r user/hive/warehouse

<property>

<name>hive.querylog.location</name>

<value>/usr/hive/log</value>

</property>

这个是用于存放hive相关日志的目录

其余的不用修改。

-------------------------------------------finish hive-site.xml

cp hive-log4j.properties.template  hive-log4j.proprtiesvi hive-log4j.propertieshive.log.dir=

这个是当hive运行时，相应的日志文档存储到什么地方

（mine：hive.log.dir=/usr/hive/log/${user.name}）

hive.log.file=hive.log

转载于:https://my.oschina.net/jywm/blog/736879

你可能感兴趣的文章

PMBOK项目管理PMI主义\IPMA概述

查看>>

解决Azure中COULD NOT LOAD FILE OR ASSEMBLY问题

查看>>

Windows Azure HandBook (8) Azure性能测试(1)

查看>>

优化事务处理

查看>>

iphone：Core Data：Where does a Managed Object Context Come From?

查看>>

spark groupByKey 也是可以filter的

查看>>

sqlmap使用笔记

查看>>

集合类接口IEnumerable,IEnumerator,ICollection,IList,IDictionary理解

查看>>

Visual Studio DSL 入门 8---创建状态机元数据模型

查看>>

LiDAR Textbook & Automated Road Network Extraction

分享20个Android游戏源码，希望大家喜欢哈！

查看>>

Metro Style App开发快速入门之文件选择总结

查看>>

AutoCAD 命令统计魔幻球的实现过程--（2）

查看>>

关于Tool接口--------hadoop接口：extends Configured implements Tool 和 ToolRunner.run

查看>>