단순한 Hadoop Map에 오류가 발생했습니다.

hadoop 맵을 실행 해 보았습니다. 단어 수를 줄였습니다. HDFS와 모든 것을 올바르게 알고 있습니다. 간단한 단어 수의 문제를 실행하는 동안 단순한 Hadoop Map에 오류가 발생했습니다.

내가 그것을 실행

나는이 오류가 발생합니다. 나는 Hadoop을 처음 사용합니다. 이 문제를 해결하는 방법에 대한 도움을 주시면 감사하겠습니다.

13/06/13 20:21:17 INFO input.FileInputFormat: Total input paths to process : 4 
13/06/13 20:21:17 INFO util.NativeCodeLoader: Loaded the native-hadoop library 
13/06/13 20:21:17 WARN snappy.LoadSnappy: Snappy native library not loaded 
13/06/13 20:21:17 INFO mapred.JobClient: Running job: job_201306131940_0007 
13/06/13 20:21:18 INFO mapred.JobClient: map 0% reduce 0% 
13/06/13 20:21:21 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_0, Status : FAILED 
java.lang.Throwable: Child Error 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) 
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_0 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_0 failed. 
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110) 
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71) 
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316) 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228) 

13/06/13 20:21:21 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_0&filter=stdout 
13/06/13 20:21:21 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_0&filter=stderr 
13/06/13 20:21:24 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_1, Status : FAILED 
java.lang.Throwable: Child Error 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) 
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_1 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_1 failed. 
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110) 
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71) 
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316) 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228) 

13/06/13 20:21:25 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_1&filter=stdout 
13/06/13 20:21:25 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_1&filter=stderr 
13/06/13 20:21:28 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_2, Status : FAILED 
java.lang.Throwable: Child Error 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) 
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_2 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_2 failed. 
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110) 
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71) 
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316) 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228) 

13/06/13 20:21:28 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_2&filter=stdout 
13/06/13 20:21:28 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_2&filter=stderr 
13/06/13 20:21:34 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_0, Status : FAILED 
java.lang.Throwable: Child Error 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) 
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_0 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_0 failed. 
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110) 
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71) 
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316) 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228) 

13/06/13 20:21:34 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_0&filter=stdout 
13/06/13 20:21:34 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_0&filter=stderr 
13/06/13 20:21:37 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_1, Status : FAILED 
java.lang.Throwable: Child Error 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) 
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_1 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_1 failed. 
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110) 
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71) 
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316) 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228) 

13/06/13 20:21:37 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_1&filter=stdout 
13/06/13 20:21:37 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_1&filter=stderr 
13/06/13 20:21:40 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_2, Status : FAILED 
java.lang.Throwable: Child Error 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) 
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_2 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_2 failed. 
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110) 
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71) 
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316) 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228) 
13/06/13 20:21:40 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_2&filter=stdout 
13/06/13 20:21:40 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_2&filter=stderr 
13/06/13 20:21:43 INFO mapred.JobClient: Job complete: job_201306131940_0007 
13/06/13 20:21:43 INFO mapred.JobClient: Counters: 4 
13/06/13 20:21:43 INFO mapred.JobClient: Job Counters 
13/06/13 20:21:43 INFO mapred.JobClient:  SLOTS_MILLIS_MAPS=24100 
13/06/13 20:21:43 INFO mapred.JobClient:  Total time spent by all reduces waiting after reserving slots (ms)=0 
13/06/13 20:21:43 INFO mapred.JobClient:  Total time spent by all maps waiting after reserving slots (ms)=0 
13/06/13 20:21:43 INFO mapred.JobClient:  SLOTS_MILLIS_REDUCES=0

출처

2013-06-13 vidya raju

오류에도 불구하고 작업이 완료되고있는 것으로 보입니다. 13/06/13 20:21:43 INFO mapred.JobClient : 작업 완료 : job_201306131940_0007. 출력 디렉토리를 확인하고 거기에 출력이 있는지 확인하십시오. – Chaos

로컬 컴퓨터에서 실행 중이십니까? –

그러나 map reduce는 실행되지 않습니다. 그래서 일은 끝나지 않았습니다. –

당신이

java.lang.Throwable: Child Error 
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)

을 받고대로 작업 추적기에 할당 된 메모리 노드 실제 메모리보다 더 많은 것 같다.

당신이 얻을 수있는 것이 더 here

출처

2013-06-14 09:19:54

나는 심볼릭 링크의 창조의 작업이 버추얼의 공유 디렉토리에 지원되지 않습니다 온라인 해결책을 찾아 낼 때까지 나는 하루 종일 해결을 보냈다 같은 문제가 있었다. 문제가 해결되면 hadoop 디렉토리를/usr/local로 이동하십시오.

출처

2014-02-19 06:14:29

이 문제가 있었지만 패키지 이름 문제였습니다.

컴파일 할 때 maven을 사용하고있었습니다. pom.xml 파일에서, 우리는 GROUPNAME.newpackage.ClassName가

그런 다음, 작동 예를 들어, 내 모든 패키지 GROUPNAME 안에 있어야 <groupId>GROUPNAME</groupId>

있습니다.

출처

2015-01-20 18:46:15

동일한 작업을 수행하는 동안 동일한 오류가 발생했습니다. 간단한 텍스트에서 단어 수를 계산합니다. 제 경우에는 Hadoop이 의존하는 JDK 버전을 변경하는 문제를 해결했습니다. Hadoop 1.2.1을 사용하고 있는데, Java 기본 버전이 1.6이라는 것을 발견했습니다.

출처

2017-02-20 13:27:20 Calde

단순한 Hadoop Map에 오류가 발생했습니다.

답변

관련 문제