1-844-696-6465 (US)        +91 77600 44484        help@dezyre.com

Help - Exception in thread "main" java.io.IOException: Error opening job



0
Please help. I am getting this error:

cloudera@cloudera-vm:~$ jar -tvf /home/cloudera/class3/WordCount.jar
63 Sat Sep 27 22:24:06 PDT 2014 META-INF/MANIFEST.MF
232 Sat Sep 27 22:23:04 PDT 2014 .classpath
385 Sun Sep 07 19:56:20 PDT 2014 .project
2471 Sat Sep 27 22:23:26 PDT 2014 EMS/LIBS/DeZyreWordCount$MyMapper.class
2311 Sat Sep 27 22:23:26 PDT 2014 EMS/LIBS/DeZyreWordCount$MyReducer.class
1661 Sat Sep 27 22:23:26 PDT 2014 EMS/LIBS/DeZyreWordCount.class
4203147 Thu Aug 28 00:34:20 PDT 2014 hadoop-core-1.2.1.jar
cloudera@cloudera-vm:~$ hadoop fs -ls /user/cloudera/class3/emswc
Found 2 items
-rw-r--r-- 1 cloudera supergroup 3902560 2014-09-27 19:33 /user/cloudera/class3/emswc/WordCount.jar
-rw-r--r-- 1 cloudera supergroup 291 2014-09-27 19:32 /user/cloudera/class3/emswc/employee3.csv
cloudera@cloudera-vm:~$ hadoop jar /user/cloudera/class3/emswc/WordCount.jar wordcount /user/cloudera/class3/emswc/employee3.csv /user/cloudera/class3/emswc/o.csv
Exception in thread "main" java.io.IOException: Error opening job jar: /user/cloudera/class3/emswc/WordCount.jar
at org.apache.hadoop.util.RunJar.main(RunJar.java:124)
Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:127)
at java.util.jar.JarFile.(JarFile.java:135)
at java.util.jar.JarFile.(JarFile.java:72)
at org.apache.hadoop.util.RunJar.main(RunJar.java:122)
cloudera@cloudera-vm:~$



Source File:
package EMS.LIBS;

import java.io.IOException;
import java.util.*;

import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;


public class DeZyreWordCount {

public static class MyMapper extends Mapper {

private final static IntWritable one = new IntWritable(1);
private Text word = new Text();

public void map(LongWritable key, Text value, Context context)
throws IOException,InterruptedException {
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()) {
word.set(tokenizer.nextToken());
context.write(word, one);
}


}
}

public static class MyReducer extends Reducer {
public void reduce(Text key, Iterable values, Context context) throws IOException,InterruptedException {
int sum = 0;
for (IntWritable val : values) {
sum += val.get();
}
context.write(key, new IntWritable(sum));
}
}

public static void main(String[] args) throws Exception {

Configuration conf = new Configuration();
Job job = new Job(conf, "wordcount");

job.setJarByClass(DeZyreWordCount.class);


job.setMapperClass(MyMapper.class);
job.setReducerClass(MyReducer.class);

job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(IntWritable.class);

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);

FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));

job.waitForCompletion(true);
}
}

5 Answer(s)


0

hi Senthamarai,

The package name of the WordCount is " EMS.LIBS". While calling the WordCount execute it as below

hadoop jar /user/cloudera/class3/emswc/WordCount.jar EMS.LIBS.wordcount /user/cloudera/class3/emswc/employee3.csv /user/cloudera/class3/emswc/o.csv

Thanks

0

Thanks for your quick reply. Still I am getting the same error:

cloudera@cloudera-vm:~/class3$ hadoop jar /user/cloudera/class3/emswc/WordCount.jar EMS.LIBS.wordcount /user/cloudera/class3/emswc/employee3.csv /user/cloudera/class3/emswc/o.csv
Exception in thread "main" java.io.IOException: Error opening job jar: /user/cloudera/class3/emswc/WordCount.jar
at org.apache.hadoop.util.RunJar.main(RunJar.java:124)
Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:127)
at java.util.jar.JarFile.(JarFile.java:135)
at java.util.jar.JarFile.(JarFile.java:72)
at org.apache.hadoop.util.RunJar.main(RunJar.java:122)


I am getting version conflict error if I run the jar file from local machine:
I tried wordcount as well as EMS.LIBS.wordcount.

cloudera@cloudera-vm:~/class3$ hadoop jar WordCount.jar wordcount /user/cloudera/class3/emswc/employee3.csv /user/cloudera/class3/emswc/o.csv
Exception in thread "main" java.lang.UnsupportedClassVersionError: EMS/LIBS/DeZyreWordCount : Unsupported major.minor version 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:179)



Thanks

0

After compiling the jar file 1.6, I got the following message:

cloudera@cloudera-vm:~/class3$ hadoop jar WordCount.jar EMS.LIBS.wordcount /user/cloudera/class3/emswc/employee3.csv /user/cloudera/class3/emswc/oo.csv
14/09/27 22:01:57 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
14/09/27 22:01:58 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost/var/lib/hadoop-0.20/cache/mapred/mapred/staging/cloudera/.staging/job_201409271648_0007
Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /user/cloudera/class3/emswc/employee3.csv already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:132)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:872)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:476)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:506)
at EMS.LIBS.DeZyreWordCount.main(DeZyreWordCount.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)



0

Hi Getting the same issue .

Help me.

0

I am new to Java. but I have worked in other language and applications like python, oracle, sqlserver, even Mongodb, cassandra but never come across a nasty unfriendly alienating language like java.

Java is the worst language I have ever come across. because it is not user friendly hell lots of configuration, environment setting, case sensitive, path settings, jarfile creation its path, home directory, local directory, class path all bullshit and crapy and makes life hell once you get an issue. I think its still primitive now also as it started decades back. I feel sorry for the millions of developers developing programs in java. 

  

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password