1-844-696-6465 (US)        +91 77600 44484        help@dezyre.com

"Java Heap Size" error message



0
I ran into this error today, and couldn't figure out how to resolve it. Basically I was trying the 1st assignment (module 3rd), and received the error below. Anyone has any idea? such as where to change heap size conf. parameter, etc.

Thanks!

--------------------
hduser@UBTBOX:~$ hadoop jar /mnt/test/WordCount_InvertIndex.jar test/data test/output-3-1
Warning: $HADOOP_HOME is deprecated.

14/08/30 18:15:18 INFO input.FileInputFormat: Total input paths to process : 1
14/08/30 18:15:18 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/08/30 18:15:18 WARN snappy.LoadSnappy: Snappy native library not loaded
14/08/30 18:15:20 INFO mapred.JobClient: Running job: job_201408301814_0001
14/08/30 18:15:21 INFO mapred.JobClient: map 0% reduce 0%
14/08/30 18:15:40 INFO mapred.JobClient: map 3% reduce 0%
14/08/30 18:15:43 INFO mapred.JobClient: map 7% reduce 0%
14/08/30 18:15:46 INFO mapred.JobClient: map 11% reduce 0%
14/08/30 18:15:49 INFO mapred.JobClient: map 16% reduce 0%
14/08/30 18:15:52 INFO mapred.JobClient: map 19% reduce 0%
14/08/30 18:15:55 INFO mapred.JobClient: map 22% reduce 0%
14/08/30 18:15:58 INFO mapred.JobClient: map 27% reduce 0%
14/08/30 18:16:01 INFO mapred.JobClient: map 32% reduce 0%
14/08/30 18:16:05 INFO mapred.JobClient: map 35% reduce 0%
14/08/30 18:16:08 INFO mapred.JobClient: map 41% reduce 0%
14/08/30 18:16:11 INFO mapred.JobClient: map 45% reduce 0%
14/08/30 18:16:14 INFO mapred.JobClient: map 50% reduce 0%
14/08/30 18:16:17 INFO mapred.JobClient: map 54% reduce 0%
14/08/30 18:16:21 INFO mapred.JobClient: map 60% reduce 0%
14/08/30 18:16:24 INFO mapred.JobClient: map 64% reduce 0%
14/08/30 18:16:27 INFO mapred.JobClient: map 67% reduce 0%
14/08/30 18:16:30 INFO mapred.JobClient: map 73% reduce 0%
14/08/30 18:16:33 INFO mapred.JobClient: map 77% reduce 0%
14/08/30 18:16:36 INFO mapred.JobClient: map 80% reduce 0%
14/08/30 18:16:39 INFO mapred.JobClient: map 86% reduce 0%
14/08/30 18:16:42 INFO mapred.JobClient: map 91% reduce 0%
14/08/30 18:16:45 INFO mapred.JobClient: map 99% reduce 0%
14/08/30 18:16:49 INFO mapred.JobClient: map 100% reduce 0%
14/08/30 18:17:20 INFO mapred.JobClient: map 100% reduce 66%
14/08/30 18:17:26 INFO mapred.JobClient: map 100% reduce 70%
14/08/30 18:17:29 INFO mapred.JobClient: map 100% reduce 79%
14/08/30 18:17:32 INFO mapred.JobClient: map 100% reduce 84%
14/08/30 18:17:35 INFO mapred.JobClient: map 100% reduce 94%
14/08/30 18:17:36 INFO mapred.JobClient: map 100% reduce 0%
14/08/30 18:17:41 INFO mapred.JobClient: Task Id : attempt_201408301814_0001_r_000000_0, Status : FAILED
Error: Java heap space

2 Answer(s)


0

hi Haofeng,

We'd suggest to go to JobTracker, which can be found in the firefox bookmarks tab in Cloudera VM, find the jobid , navigate to the task tracker and look at the logs for the exact issue.

Thanks

0

Thanks. Below is what I found in the task tracker log. I added these property lines in mapred-site.xml, and it's seemingly working ok now.

mapred.child.java.opts
-Xmx1024m

Thanks!

----------- log -------------
2014-08-30 18:17:35,038 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapCharBuffer.(HeapCharBuffer.java:57)
at java.nio.CharBuffer.allocate(CharBuffer.java:331)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:792)
at org.apache.hadoop.io.Text.decode(Text.java:350)
at org.apache.hadoop.io.Text.decode(Text.java:327)
at org.apache.hadoop.io.Text.toString(Text.java:254)
at com.thf.dw.hadoop.WordCount_InvertIndex$WordCountReducer.reduce(WordCount_InvertIndex.java:51)
at com.thf.dw.hadoop.WordCount_InvertIndex$WordCountReducer.reduce(WordCount_InvertIndex.java:1)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password