Hadoop 0.20.2 reducer가 값을 반복 할 때 ArrayIndexOutOfBoundsException을 throw합니다.

Question

hadoop을 처음 접했지만 "Hadoop : The Definitive Guide"를 읽었으므로 기본 개념을 이해하고 있다고 생각합니다.

나는 아주 간단한 작업을 실행하기 위해 하둡 0.20.2을 사용하지만, 나는 다음과 같은 예외가 얻을 :

java.lang.ArrayIndexOutOfBoundsException: 4096 
     at java.io.ByteArrayInputStream.read(ByteArrayInputStream.java:127) 
     at java.io.DataInputStream.readInt(DataInputStream.java:373) 
     at com.convertro.mapreduce.WritableHit.readFields(Unknown Source) 
     at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeseria 
lizer.deserialize(WritableSerialization.java:67) 
     at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeseria 
lizer.deserialize(WritableSerialization.java:40) 
     at org.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext. 
java:116) 
     at org.apache.hadoop.mapreduce.ReduceContext$ValueIterator.next(ReduceCo 
ntext.java:163) 
     at com.convertro.mapreduce.HitConvertingIterable$HitConvertingIterator.n 
ext(HitConvertingIterable.java:35) 
     at com.convertro.mapreduce.HitConvertingIterable$HitConvertingIterator.n 
ext(HitConvertingIterable.java:1) 
     at com.convertro.naive.NaiveHitReducer.reduce(Unknown Source) 
     at com.convertro.mapreduce.HitReducer.reduce(Unknown Source) 
     at com.convertro.mapreduce.HitReducer.reduce(Unknown Source) 
     at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176) 
     at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566 
) 
     at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) 
     at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:2 

이것은 WritableHit 클래스합니다 (감소 단계로 입력 값)의 읽기 동안 발생합니다. 다음은 WritableHit 클래스 코드입니다.

public class WritableHit implements WritableComparable<WritableHit> { 

    private Hit hit; 

    public WritableHit() { 
     this(null); 
    } 

    public WritableHit(Hit hit) { 
     this.hit = hit; 
    } 

    @Override 
    public void readFields(DataInput input) throws IOException { 
     String clientName = input.readUTF(); 
     String clientSiteId = input.readUTF(); 
     String eventUniqueId = input.readUTF(); 
     String eventValue = input.readUTF(); 
     String pageRequested = input.readUTF(); 
     String refererUrl = input.readUTF(); 
     String uniqueHitId = input.readUTF(); 
     String userAgent = input.readUTF(); 
     String userIdentifier = input.readUTF(); 
     String userIp = input.readUTF(); 
     int timestamp = input.readInt(); 
     int version = input.readInt(); 

     hit = new Hit(version, uniqueHitId, clientName, clientSiteId, timestamp, userIdentifier, 
       userIp, pageRequested, refererUrl, userAgent, eventUniqueId, eventValue); 
    } 

    @Override 
    public void write(DataOutput output) throws IOException { 
     output.writeUTF(hit.getClientName()); 
     output.writeUTF(hit.getClientSiteId()); 
     output.writeUTF(hit.getEventUniqueId()); 
     output.writeUTF(hit.getEventValue()); 
     output.writeUTF(hit.getPageRequested()); 
     output.writeUTF(hit.getRefererUrl()); 
     output.writeUTF(hit.getUniqueHitId()); 
     output.writeUTF(hit.getUserAgent()); 
     output.writeUTF(hit.getUserIdentifier()); 
     output.writeUTF(hit.getUserIp()); 
     output.write(hit.getTimestamp()); 
     output.write(hit.getVersion()); 
    } 

    public Hit getHit() { 
     return hit; 
    } 

    @Override 
    public int compareTo(WritableHit o) { 
     return hit.getUniqueHitId().compareTo(o.getHit().getUniqueHitId()); 
    } 
    } 

어떤 도움을 주시면 감사하겠습니다.

감사합니다.

Answer 1

알아 냈습니다.

은 분명히, 당신이 쓰기 가능한 객체를 구현할 때, 당신은 내의 writeInt 방법을 사용해야 아닌 는 방법을 쓰기.

일단 내가 그렇게하면 매력처럼 작동합니다.