org.apache.hadoop.io.Writable을 org.apache.hadoop.io.IntWritable에 캐스트 할 수없는 이유는 무엇입니까?

Question

내 mapreduce 애플리케이션은 다음과 같습니다. 내가 문자열

public class StockCount { 

public static class MapperClass 
     extends Mapper<Object, Text, Text, IntArrayWritable> { 



    public void map(Object key, Text value, Context context 
    ) throws IOException, InterruptedException { 
     String line[] = value.toString().split(","); 

     //mgrno,rdate,cusip,shares,sole,shared,no 
     // [0], [1], [2], [3], [4], [5],[6] 

     if (line.length > 5){ 


       Text mgrno = new Text(line[0]); 
       IntWritable[] intArray = new IntWritable[3]; 
       intArray[0] = new IntWritable(Integer.parseInt(line[4])); 
       intArray[1] = new IntWritable(Integer.parseInt(line[5])); 
       intArray[2] = new IntWritable(Integer.parseInt(line[6])); 

       int[] pass = new int[3]; 
       pass[0] = Integer.parseInt(line[4]); 
       pass[1] = Integer.parseInt(line[5]); 
       pass[0] = Integer.parseInt(line[6]); 




       IntArrayWritable array = new IntArrayWritable(intArray); 

       context.write(mgrno, array); 
      } 
    } 
} 

public static class IntSumReducer 
     extends Reducer<Text, int[], Text, IntArrayWritable> { 


    public void reduce(Text key, Iterable<IntArrayWritable> values, 
         Context context 
    ) throws IOException, InterruptedException { 
     int sum1 = 0; 
     int sum2 = 0; 
     int sum3 = 0; 
     for (IntArrayWritable val : values) { 

      IntWritable[] temp = new IntWritable[3]; 
      temp = val.get(); 
      sum1 += temp[0].get(); 
      sum2 += temp[1].get(); 
      sum3 += temp[2].get(); 

     } 
     IntWritable[] intArray = new IntWritable[3]; 
     intArray[0] = new IntWritable(sum1); 
     intArray[1] = new IntWritable(sum2); 
     intArray[2] = new IntWritable(sum3); 
     IntArrayWritable result = new IntArrayWritable(intArray); 

     context.write(key, result); 
    } 
} 

내 값의 3 합계 원하는대로

에서 3 개 값을 합계를, 나는 클래스 IntArrayWritable가 ArrayWritable로부터 상속 정의했다. "수익률 (IntWritable []) super.get를();"는 캐스팅 할 수없는 이유의

import org.apache.hadoop.io.ArrayWritable; 
import org.apache.hadoop.io.IntWritable; 

public class IntArrayWritable extends ArrayWritable { 

    public IntArrayWritable(IntWritable[] values) { 
     super(IntWritable.class, values); 
    } 
    public IntArrayWritable() { 
    super(IntWritable.class); 
    } 

    @Override 
    public IntWritable[] get() { 
     return (IntWritable[]) super.get(); 
    } 


    @Override 
    public String toString() { 
     IntWritable[] values = get(); 
     return values[0].toString() + ", " + values[1].toString() + ", " + 
values[2].toString(); 
    } 
} 

정말 볼 수 없습니다 - ArrayWritable는 기록 가능 [] 포함

17/11/21 04:00:26 WARN mapred.LocalJobRunner: job_local1623924180_0001 
java.lang.Exception: java.lang.ClassCastException: [Lorg.apache.hadoop.io.Writable; cannot be cast to [Lorg.apache.hadoop.io.IntWritable; 
     at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) 
     at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529) 
Caused by: java.lang.ClassCastException: [Lorg.apache.hadoop.io.Writable; cannot be cast to [Lorg.apache.hadoop.io.IntWritable; 
     at IntArrayWritable.get(IntArrayWritable.java:15) 
     at IntArrayWritable.toString(IntArrayWritable.java:22) 
     at org.apache.hadoop.mapreduce.lib.output.TextOutputFormat$LineRecordWriter.writeObject(TextOutputFormat.java:85) 
     at org.apache.hadoop.mapreduce.lib.output.TextOutputFormat$LineRecordWriter.write(TextOutputFormat.java:104) 
     at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:558) 
     at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89) 
     at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105) 
     at org.apache.hadoop.mapreduce.Reducer.reduce(Reducer.java:150) 
     at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171) 
     at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) 
     at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) 
     at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319) 
     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
     at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
     at java.lang.Thread.run(Thread.java:748) 

정말 도움을 주시면 감사하겠습니다.

Thans! 모든

Answer 1

첫째, Reducer<Text, int[], 당신은 단지 매퍼에서 쉼표로 구분 된 텍스트 쓰기 가능한 값을 사용할 수 있습니다, 그러나 대신 int[]

의 쓰기 가능한 유형이 있어야합니다.

자신의 Writable 클래스를 작성할 때 배열을 전달할 때만 명확한 이점이 있습니다.

당신은 [문서]에 따르면 감속기