테이블에 csv 데이터로드 하이브 SQL 자바 오류 InvocationTargetException

Question

하이브 SQL을 사용하여 테이블에 데이터를로드 할 수 없기 때문에 미쳤습니다. 이 버그를 고치는 방법을 가르쳐주세요.

메타 데이터에 테이블을 만들려면 다음을 수행하십시오. 나는 2 문 사용 :

1 - 테이블 만들기 :

CREATE TABLE db1.test_table_file3(
Col1 INT,Col2 SMALLINT,Col3 SMALLINT) 
COMMENT 'test_table_file2' 
ROW FORMAT DELIMITED 
FIELDS TERMINATED BY '\t' 
LINES TERMINATED BY '\n' 

2 - 테이블에 데이터로드 :

LOAD DATA INPATH '/test/test_test_5.csv' INTO TABLE 
db1.test_table_file3 

코드 아래 :

Connection con = getConnection(); 
Statement stmt = con.createStatement(); 
Statement loadDataStmt = con.createStatement(); 
stmt.execute(createTableQuery); 
loadDataStmt .execute(loadDataSQL); 
closeConnection(con, stmt); 
closeConnection(con, loadDataStmt); 

을 아래 결과 :

Statement 1 create table: Execute successful 
Statement 2 Load data: Execute fail 

java.sql.SQLException: java.lang.reflect.InvocationTargetException 
    at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:296) ~[hive-jdbc-1.2.1.spark2.jar:1.2.1.spark2] 
    at com.toshiba.mwcloud.datalake.griddatalake.repository.impl.MetadataRepositoryImpl.loadDataIntoTable(MetadataRepositoryImpl.java:404) ~[bin/:na] 
    at com.toshiba.mwcloud.datalake.griddatalake.repository.impl.MetadataRepositoryImpl.createMetadata(MetadataRepositoryImpl.java:49) ~[bin/:na] 
    at com.toshiba.mwcloud.datalake.griddatalake.repository.impl.MetadataRepositoryImpl$$FastClassBySpringCGLIB$$10d6b423.invoke(<generated>) ~[bin/:na] 
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:747) ~[spring-aop-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139) ~[spring-tx-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) ~[spring-aop-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at com.toshiba.mwcloud.datalake.griddatalake.repository.impl.MetadataRepositoryImpl$$EnhancerBySpringCGLIB$$bba82441.createMetadata(<generated>) ~[bin/:na] 
    at com.toshiba.mwcloud.datalake.griddatalake.service.impl.MetadataServiceImpl.createMetadata(MetadataServiceImpl.java:85) ~[bin/:na] 
    at com.toshiba.mwcloud.datalake.griddatalake.controller.MetadataController.createTable(MetadataController.java:117) ~[bin/:na] 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_121] 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_121] 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121] 
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121] 
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209) ~[spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) ~[spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102) ~[spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:871) ~[spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:777) ~[spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991) ~[spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925) ~[spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978) [spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:881) [spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:855) [spring-webmvc-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.2.BUILD-SNAPSHOT.jar:5.0.2.BUILD-SNAPSHOT] 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121] 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121] 
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.23.jar:8.5.23] 
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] 

Answer 1

문제점은 두 번째 문장에 있으며, 아래는 설명입니다.

HDFS에 데이터가있는 경우 LOAD DATA INPATH 'hdfs:/test/test_test_5.csv' INTO TABLE db1.test_table_file3이 작동합니다.

로컬에있는 파일이있는 경우 당신은 내가 당신이 실행 제안은 HDFS에서

을 파일을로드하는 경우 파일 경로 전에 네임 노드를 언급 LOCAL INPATH LOAD DATA LOCAL INPATH '/test/test_test_5.csv' INTO TABLE db1.test_table_file3

• 를 사용한다 두 번째 진술은 자바의 첫 번째와 당신이 받고있는 오류를 참조하십시오. 일반적으로 사람들은 외부 테이블에 대한 권한 문제를 겪습니다. HIVE 사용자는 HDFS 경로에 액세스 할 수 있어야합니다.

  매우 쉬운 방법으로 설명 된 this 페이지를 참조 할 수 있습니다.

  질문에서 다른 문제점을 발견했습니다. 테이블에서 '\ t'탭으로 필드 구분 기호를 설정하고 질문 샘플에서 실수인지 확실하지 않은 CSV 파일을 삽입하려고합니다. 아니.

  몇 가지 오류가 발생하여 오류로 의견을 말하면 나는 당신을 도울 수 있습니다.