hi Faruk,

Please check the actual package and class names that were defined in the dezyrewordcount.jar, Once thing you can do to identify them

jar -tvf dezyrewordcount.jar . if the package name is someting like "com.module3" and class name is DeZyreWordCount , invoke the class by something like shown below

hadoop jar dezyrewordcount.jar com.module3.DeZyreWordCount /user/cloudera/wordcount/input/words.txt /user/cloudera/wordcount/output

thanks

Hi Faruk,
The problem is not with the way you were compiling but the way the program was setup. The example java code creates the DeZyreWordCount class in the package com.dezyre.

When the JAR is being created the jar has the corresponding class file in the main directory.

to fix this either :

1) Remove the package name.

2) Fix the JAR that is getting created :
a) Create a directory structure for the packages ( com/dezyre) and move all the classes that were compiled into this directory.
b) your directory structure should be com/dezyre/*.class
c) now in the main directory where your java file was, create the jar with $ jar -cvf wc.jar com
d) now use this jar to be uploaded to hadoop that will work.

(The reason the problem is manifesting is because when hadoop is trying to look for the class file, it is looking in the com/dezyre/ directory for the class com.dezyre.DeZyreWordCount.class Since this is not available it was throwing the Class not found exception. (Your class was in the root directory of the jar))
Doing 2 above puts the class file in the right directory and hence gets it working.



thanks,
Sarath.

Hi Sarath,

Thank you for looking at my issue but it's still not resolved and it's getting very frustrating. I recreated the wordcount project without package name com.dezyre. Moved the new JAR file in /user/cloudera/wordcount and tried to run it a couple of ways:

1. [cloudera@localhost wordcount]$ hadoop jar dezyrewordcount.jar DezYreWordCount /user/cloudera/wordcount/input/words.txt /user/cloudera/wordcount/output

2. 1. [cloudera@localhost wordcount]$ hadoop jar wordcount dezyrewordcount.jar DezYreWordCount /user/cloudera/wordcount/input/words.txt /user/cloudera/wordcount/output

But I am still getting the following error:

Exception in thread "main" java.lang.ClassNotFoundException: DezYreWordCount
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:201)

Seems like hadoop is unable to find the JAR file. First I had the JAR file in:

cloudera@localhost wordcount]$ ls
dezyrewordcount.jar input

Then I copyFromLocal and now I have a copy in:

[cloudera@localhost wordcount]$ hadoop fs -ls /user/cloudera/wordcount
Found 2 items
-rw-r--r-- 3 cloudera cloudera 3902597 2014-08-04 11:13 /user/cloudera/wordcount/dezyrewordcount.jar
drwxr-xr-x - cloudera cloudera 0 2014-08-03 13:22 /user/cloudera/wordcount/input

And my input file words.txt is in:

[cloudera@localhost wordcount]$ hadoop fs -ls /user/cloudera/wordcount/input
Found 1 items
-rw-r--r-- 3 cloudera cloudera 65 2014-08-03 13:22 /user/cloudera/wordcount/input/words.txt

First few lines for my JAR are as follows:

package wordcount;

import java.io.IOException;
import java.util.*;

import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;


public class DeZyreWordCount {

I am very confused at this point, not sure what I am doing wrong?

Any help will be greatly appreciated.

Faruk

I realized that I had DeZyreWordCount spelled DezYreWordCount, so I corrected and reran the following commands:

[cloudera@localhost ~]$ hadoop jar dezyrewordcount.jar wordcount.DeZyreWordCount /user/cloudera/wordcount/input/words.txt /user/cloudera/wordcount/output

[cloudera@localhost ~]$ hadoop jar dezyrewordcount.jar DeZyreWordCount /user/cloudera/wordcount/input/words.txt /user/cloudera/wordcount/output

Still get the error:

Exception in thread "main" java.io.IOException: Error opening job jar: dezyrewordcount.jar
at org.apache.hadoop.util.RunJar.main(RunJar.java:135)
Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:127)
at java.util.jar.JarFile.(JarFile.java:135)
at java.util.jar.JarFile.(JarFile.java:72)
at org.apache.hadoop.util.RunJar.main(RunJar.java:133)

Hi Faruk,
I had the same issues last week wit the execution.

Please make sure you are following every STEP in the video. I can assure you that it will work.
/course/batch-view/course_id/19/batch_id/19#

Note: did you create the folder/file as hadoop fs -mkdir then copy the files from local to hadoop?

Hi Edgar,

Thanks for your response. Yes I did follow the video and yes I created a folder/file but it still gives me an error, I checked it several times. Maybe I need an extra set of eyes. Is it possible to talk to you on the phone or via email? If yes please email me at fansari@vitalsciences.com or call 502-386-2797.

Thank you,

Faruk