More Errors Running WordCount



0
Please tell me what this error means and how to correct it.

[hadoop@Base ~]$ hadoop fs -ls /user/hadoop/WordCount
Warning: $HADOOP_HOME is deprecated.

Found 2 items
-rw-r--r-- 1 hadoop supergroup 96 2015-07-20 12:33 /user/hadoop/WordCount/SimpleWordsInput.txt
-rw-r--r-- 1 hadoop supergroup 3470 2015-07-20 12:14 /user/hadoop/WordCount/WordCountInput.txt
[hadoop@Base ~]$ hadoop jar hadoop_projects/WordCount.jar /user/hadoop/WordCount/WordCountInput.txt /user/hadoop/WordCount/Output2
Warning: $HADOOP_HOME is deprecated.

15/07/20 12:34:39 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
15/07/20 12:34:40 INFO util.NativeCodeLoader: Loaded the native-hadoop library
15/07/20 12:34:40 WARN snappy.LoadSnappy: Snappy native library not loaded
15/07/20 12:34:40 INFO mapred.FileInputFormat: Total input paths to process : 1
15/07/20 12:34:41 INFO mapred.JobClient: Running job: job_201507201222_0003
15/07/20 12:34:42 INFO mapred.JobClient: map 0% reduce 0%
15/07/20 12:34:58 INFO mapred.JobClient: Task Id : attempt_201507201222_0003_m_000000_0, Status : FAILED
java.io.IOException: Illegal partition for We (3)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1078)
at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:591)
at com.deb.mapreduce.wordcount$Map.map(wordcount.java:65)
at com.deb.mapreduce.wordcount$Map.map(wordcount.java:1)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)

15/07/20 12:35:58 WARN mapred.JobClient: Error reading task outputconnect timed out
15/07/20 12:36:58 WARN mapred.JobClient: Error reading task outputconnect timed out
15/07/20 12:36:58 INFO mapred.JobClient: Task Id : attempt_201507201222_0003_m_000001_0, Status : FAILED
java.io.IOException: Illegal partition for Counselor's (3)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1078)
at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:591)
at com.deb.mapreduce.wordcount$Map.map(wordcount.java:65)
at com.deb.mapreduce.wordcount$Map.map(wordcount.java:1)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)

15/07/20 12:37:58 WARN mapred.JobClient: Error reading task outputconnect timed out
15/07/20 12:38:58 WARN mapred.JobClient: Error reading task outputconnect timed out
15/07/20 12:38:58 INFO mapred.JobClient: Job complete: job_201507201222_0003
15/07/20 12:38:58 INFO mapred.JobClient: Counters: 7
15/07/20 12:38:58 INFO mapred.JobClient: Job Counters
15/07/20 12:38:58 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=87731
15/07/20 12:38:58 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
15/07/20 12:38:58 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
15/07/20 12:38:58 INFO mapred.JobClient: Rack-local map tasks=8
15/07/20 12:38:58 INFO mapred.JobClient: Launched map tasks=8
15/07/20 12:38:58 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
15/07/20 12:38:58 INFO mapred.JobClient: Failed map tasks=1
15/07/20 12:38:58 INFO mapred.JobClient: Job Failed: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201507201222_0003_m_000000
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1357)
at com.deb.mapreduce.wordcount.main(wordcount.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)

1 Answer(s)


0

hi Michael, there are two issues
1) you are referrring to a hdfs location which is not existing "/user/hadoop" , use "/user/cloudera" which is the path available in hdfs

2) in the command you have not specified the main class name and hence the error (Exception in thread "main" java.io.IOException: Job failed!)

hadoop jar hadoop_projects/WordCount.jar WordCount /user/hadoop/WordCount/WordCountInput.txt /user/hadoop/WordCount/Output2

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password