Oozie 및 sqoop 구성 문제

oozie 및 sqoop 워크 플로를 설정하려고합니다 (hdfs에 mySql 데이터를 백업하려고합니다). 하지만 내 일을 시작할 때 막혔습니다.Oozie 및 sqoop 구성 문제

나는 oozie의 마지막 버전 인 hadoop2 (working hdfs node)를 사용하고 있습니다. hozfs config (core-site.xml, hdfs-site.xml, yarn-site.xml, mapred-site.xml)을 oozie conf 파일과 함께 사용하여 컴퓨터에 oozie 서버를 설치했습니다 (배치하기 전에 테스트 해보고 싶습니다)/haddop-conf dir), 그리고 서버에 내 hdfs.

나는 기본 워크 플로우 만든이 같은 (테스트 목적, 난 그냥 Sqoop을가 작동하는지 확인하려면) :

<workflow-app name="Sqoop" xmlns="uri:oozie:workflow:0.4"> 
    <start to="Sqoop"/> 
    <action name="Sqoop"> 
     <sqoop xmlns="uri:oozie:sqoop-action:0.2"> 
     <job-tracker>yarn.resourcemanager.address:8040</job-tracker> 
      <name-node>hdfs://hdfs-server:54310</name-node> 
      <command>job --list</command> 
     </sqoop> 
     <ok to="end"/> 
     <error to="kill"/> 
    </action> 
    <kill name="kill"> 
     <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> 
    </kill> 
    <end name="end"/> 
</workflow-app>

나는 나의 HDFS에이 워크 플로를 넣어. 나는 내 일을 시작하기위한 자바 코드를 만들었 : 나는 내 직업

2013-05-28 12:42:30,004 INFO ActionStartXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] Start action [[email protected]:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2013-05-28 12:42:30,008 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action status=DONE 
2013-05-28 12:42:30,009 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action updated in DB! 
2013-05-28 12:42:30,192 INFO ActionStartXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2013-05-28 12:42:31,389 WARN SqoopActionExecutor:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] credentials is null for the action 
2013-05-28 12:42:42,942 INFO SqoopActionExecutor:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] checking action, external ID [job_1369126414383_0003] status [RUNNING] 
2013-05-28 12:42:42,945 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] [***[email protected]***]Action status=RUNNING 
2013-05-28 12:42:42,946 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB! 
2013-05-28 12:47:43,034 INFO KillXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] STARTED WorkflowKillXCommand for jobId=0000000-130528124140043-oozie-anth-W 
2013-05-28 12:47:43,328 WARN CoordActionUpdateXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100 
2013-05-28 12:47:43,328 INFO KillXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000000-130528124140043-oozie-anth-W

을 실행 그리고 내가 원사 webinterface을 확인할 때, 나는하지만 내 일을 볼 수 있습니다 볼 수 있습니다 Oozie의 webinterface에서

OozieClient wc = new OozieClient("http://localhost:11000/oozie"); 

     Properties conf = wc.createConfiguration(); 
     conf.setProperty(OozieClient.APP_PATH, "hdfs://hdfs_server:54310/hive/testSqoop/sqoop-workflow.xml"); 
     conf.setProperty("queueName", "default"); 

     try { 
      String jobId = wc.run(conf); 
      System.out.println("Workflow job submitted"); 

      while (wc.getJobInfo(jobId).getStatus() == WorkflowJob.Status.RUNNING) { 
       System.out.println("Workflow job running ..."); 
       System.out.println("..." + wc.getJobInfo(jobId).getStatus().toString()); 
       Thread.sleep(10 * 1000); 
      } 
      System.out.println("Workflow job completed ..."); 
      System.out.println(wc.getJobInfo(jobId)); 
     } catch (Exception r) { 
      r.printStackTrace(); 
     }

상태가 FAILED로

Application application_1369126414383_0003 failed 1 times due to AM Container for appattempt_1369126414383_0003_000001 exited with exitCode: 1 due to: .Failing this attempt.. Failing the application.

나는 정말 잘못 알고 있습니다. 귀하의 조언이 필요합니다.

당신은 작업 로그를 검사해야

출처

2013-05-29 AnthonyC

웹 인터페이스에서 실패한 작업 시도의 로그를 찾으면 작업에 실패한 예외가 발생합니다. – SNeumann

~ 감사합니다 :

$ oozie job -log <coord_job_id>

는 무슨 일이 일어나고 있는지 이해하기.

출처

2016-09-13 19:31:45 ozw1z5rd

Oozie 및 sqoop 구성 문제

답변

관련 문제