2010-07-24 1 views
7
나는 창에서 하둡을 사용하고

파일을 찾을 수 없습니다와 나는 좋은 모든 것을 구성한 (Cygwin에서 암호없이 SSH를 설치하는 등)하둡가 : 예외

나는 WC.jar의 단어 수 프로그램을 컴파일했습니다 실행하려고했습니다. 완벽하게 독립 실행 형 모드에서 실행 중입니다.하지만 완전 배포 모드에서는 다음과 같이 나타납니다. FileNotFoundException

로그를보고 제게 잘못되었습니다. MACH1에서 dfs 및 mapreduce를 시작했습니다. (내 주인님)

$ bin/hadoop jar WC.jar WordCount words result 

10/07/24 16:57:38 INFO input.FileInputFormat: Total input paths to process : 2 
10/07/24 16:57:39 INFO mapred.JobClient: Running job: job_201007241657_0001 
10/07/24 16:57:40 INFO mapred.JobClient: map 0% reduce 0% 
10/07/24 16:57:50 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00003_0, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_0/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:57:55 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_r_0 
00002_0, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_r_000002_0/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:07 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00003_1, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_1/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:14 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00003_2, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_2/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:26 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00002_0, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_0/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:34 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_r_0 
00001_0, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_r_000001_0/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:41 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00002_1, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_1/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:47 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00002_2, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_2/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:53 INFO mapred.JobClient: Job complete: job_201007241657_0001 
10/07/24 16:58:53 INFO mapred.JobClient: Counters: 0 

[email protected] /usr/local/hadoop-0.20.2 
$` 

고마워요.

+0

우리에게 [DistributedCache.addCacheFile]을 호출하는 [SSCCE] (http://sscce.org/) 코드를 보여줍니다. 가능한 답변도 [here] (http://stackoverflow.com/questions/3219012/filenotfoundexception-when-using-hadoop-distributed-cache)를 참조하십시오. – opyate

답변

1

이전에이 예외가 있었지만 이전 로그에 액세스 할 권한이 없다고 생각합니다. 나는 에 의해 FileNotFoundException을 해결했으며, 이름은으로 재 포맷되었다. namenode 로그에서 "일관성없는 상태"인지 확인하여 다시 포맷하기 전에 원인을 확인할 수 있습니다.

+0

안녕하세요 @wei, 나도 똑같은 문제가있어, 난 너무 재 포맷팅을 시도했지만 여전히 문제가 존재합니다 – jerith

+0

로그에 "일관성없는 상태"가 보이나요? – Wei