Sqoop Oracle 하이브 오류

Question

내 테이블의 상한값을 찾는 sqoop에 문제가 있습니다. 이어야하지만 그 결과는 null이며 그 결과 (굵게 표시)이 잘못되었습니다.

hduser@hduser:~$ sudo -u hdfs sqoop job --create testora -- import --connect jdbc:oracle:thin:@192.168.1.6:1521/DB --username SYSTEM -password Welcome123 --table employees --columns "emp_id,emp_name,emp_salary" --check-column "emp_id" --incremental "append" --last-value "200" --hive-table=oraemployees --hive-import --split-by "emp_id" --target-dir /user/hdfs/oraemployees; 

13/04/15 17:35:46 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 

13/04/15 17:35:46 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override 

13/04/15 17:35:46 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc. 
flutura@flutura:~$ sudo -u hdfs sqoop job --exec testora10 

13/04/15 17:35:51 INFO manager.SqlManager: Using default fetchSize of 1000 

13/04/15 17:35:51 INFO tool.CodeGenTool: Beginning code generation 

13/04/15 17:35:52 INFO manager.OracleManager: Time zone has been set to GMT 

13/04/15 17:35:52 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM employees t WHERE 1=0 

13/04/15 17:35:52 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop 

13/04/15 17:35:52 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop/hadoop-core.jar 

13/04/15 17:35:52 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hdfs/compile/a85f9eb2e63c0150ce13f684dfab16ff/employees.jar 

13/04/15 17:35:52 INFO tool.ImportTool: Incremental import based on column emp_id 

13/04/15 17:35:52 INFO tool.ImportTool: **Lower bound value: 200** 

13/04/15 17:35:52 INFO tool.ImportTool: **Upper bound value: null** 

13/04/15 17:35:52 INFO mapreduce.ImportJobBase: Beginning import of employees 

13/04/15 17:35:53 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: **SELECT MIN(emp_id), MAX(emp_id) FROM employees WHERE (emp_id > 200 AND emp_id IS NULL )** 

13/04/15 17:35:53 INFO mapred.JobClient: Running job: job_201304151528_0010 

13/04/15 17:35:54 INFO mapred.JobClient: map 0% reduce 0% 

13/04/15 17:35:59 INFO mapred.JobClient: map 100% reduce 0% 

13/04/15 17:35:59 INFO mapred.JobClient: Job complete: job_201304151528_0010 

13/04/15 17:35:59 INFO mapred.JobClient: Counters: 15 

13/04/15 17:35:59 INFO mapred.JobClient: Job Counters 

13/04/15 17:35:59 INFO mapred.JobClient:  SLOTS_MILLIS_MAPS=3881 

13/04/15 17:35:59 INFO mapred.JobClient:  Total time spent by all reduces waiting after reserving slots (ms)=0 

13/04/15 17:35:59 INFO mapred.JobClient:  Total time spent by all maps waiting after reserving slots (ms)=0 

13/04/15 17:35:59 INFO mapred.JobClient:  Launched map tasks=1 

13/04/15 17:35:59 INFO mapred.JobClient:  SLOTS_MILLIS_REDUCES=0 

13/04/15 17:35:59 INFO mapred.JobClient: FileSystemCounters 

13/04/15 17:35:59 INFO mapred.JobClient:  HDFS_BYTES_READ=109 

13/04/15 17:35:59 INFO mapred.JobClient:  FILE_BYTES_WRITTEN=72408 

13/04/15 17:35:59 INFO mapred.JobClient: Map-Reduce Framework 

13/04/15 17:35:59 INFO mapred.JobClient:  Map input records=0 

13/04/15 17:35:59 INFO mapred.JobClient:  Physical memory (bytes) snapshot=125415424 

13/04/15 17:35:59 INFO mapred.JobClient:  Spilled Records=0 

13/04/15 17:35:59 INFO mapred.JobClient:  CPU time spent (ms)=500 

13/04/15 17:35:59 INFO mapred.JobClient:  Total committed heap usage (bytes)=200998912 

13/04/15 17:35:59 INFO mapred.JobClient:  Virtual memory (bytes) snapshot=1019650048 

13/04/15 17:35:59 INFO mapred.JobClient:  Map output records=0 

13/04/15 17:35:59 INFO mapred.JobClient:  SPLIT_RAW_BYTES=109 

13/04/15 17:35:59 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 7.2782 seconds (0 bytes/sec) 

13/04/15 17:35:59 INFO mapreduce.ImportJobBase: Retrieved 0 records. 

13/04/15 17:36:00 INFO util.AppendUtils: Creating missing output directory - oraemployees10 

13/04/15 17:36:00 INFO manager.OracleManager: Time zone has been set to GMT 

13/04/15 17:36:00 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM employees t WHERE 1=0 

13/04/15 17:36:00 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException 
java.lang.NullPointerException 
    at com.cloudera.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:177) 
    at com.cloudera.sqoop.hive.HiveImport.importTable(HiveImport.java:175) 
    at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:394) 
    at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:455) 
    at com.cloudera.sqoop.tool.JobTool.execJob(JobTool.java:233) 
    at com.cloudera.sqoop.tool.JobTool.run(JobTool.java:288) 
    at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) 
    at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182) 
    at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221) 
    at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230) 
    at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239) 

Answer 1

귀하의 쿼리는 잘못된 것입니다 : 당신이 그래서 오직 NULL을 반환 할 수 있습니다

emp_id IS NULL 

을 요구하고 있기 때문에

SELECT 
    MIN(emp_id), 
    MAX(emp_id) FROM employees WHERE (emp_id > 200 AND emp_id IS NULL ) 

이 어떤 합리적인 값을 반환 할 수 없습니다.