When following Running Mapreduce document



0

I cannot ifnd the the jar files mentioned because of which I am getting  errors.

Select your project in project explorer

File->Properties->Java build path -> Libraries

Remove the missing libraries by clicking on Remove button

Click on “Add External Jars” button to add the following lib from your project/lib folder.

 

  1. hadoop-mapreduce-client-core.jar

  2. hadoop-commons.jar

  3. commons-codec.jar

 

 

Errors -

Exception in thread "main" java.lang.Error: Unresolved compilation problems:
    Configuration cannot be resolved to a type
    Configuration cannot be resolved to a type
    Job cannot be resolved to a type
    Job cannot be resolved
    TextInputFormat cannot be resolved to a type
    FileInputFormat cannot be resolved
    Path cannot be resolved to a type
    Text cannot be resolved to a type
    IntWritable cannot be resolved to a type
    TextOutputFormat cannot be resolved to a type
    FileOutputFormat cannot be resolved
    Path cannot be resolved to a type

 


10 Answer(s)


0

Make sure pom.xml is not showing error , so that the depencies jar files can be downloaded by maven. In my case my pom.xml was having error, I changed the systemPath to point to tools.jar as per my local java installation 

<dependency>
 <groupId>jdk.tools</groupId>
 <artifactId>jdk.tools</artifactId>
 <version>1.8.0_112</version>
 <scope>system</scope>
 <systemPath>C:/Program Files/Java/jdk1.8.0_112/lib/tools.jar</systemPath>
 </dependency>

And I think step 5 in document can be ignored as maven downloads the jar files anyway. 

 


0

Hi Kumaran - I also put the correct part as per my java installation. After i comple the pom.xml I am gitting the following error -

 

 

[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.eng.mr:mapreduce-example:jar:0.0.1-SNAPSHOT
[WARNING] 'dependencies.dependency.systemPath' for jdk.tools:jdk.tools:jar should use a variable instead of a hard-coded path C:\Program Files\Java\jdk1.8.0_131\lib\tools.jar @ line 27, column 14
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE


0

Hi Kan,

These are warning messages . To view error messages try window->showview->markers. If you see errors please post them 

Since we not making any changes other than to pom.xml, I think  its likely an maven issue

Try 

1. Project->clean

2. Rightclick on projectname(mapreduce)->maven->update project

 

 


0

Hi Kumaran - thanks for your update. I am  able to see Windows -> Show View but  Markers is not available. I am new to this so I may have missed it . That is possible.

I have the snapshot  of the errors in the attachment.



0

When you right click on project and see properties->java buildpath->libraries ...

You should see two groups JRE libraries and maven dependencies .... Remove anything else present there .

Now if you expand the maven dependencies group you should get something similar to the screenshot. If you get this , that means maven has downloaded the hadoop jar files needed for the project , and the errors in your screenshot should get resolved because the required jars  are present in the projects build path. 

If you are not getting something similar to my screen, then there might be problem in pom.xml , check if you have any errors there . 

Basically I think only two areas we need to check , 1. Java build Path 2. Pom.xml 

And dont forget to clean the project and update maven project if you make any changes 

Keep me posted , thanks. 

 

 



0

Hi Kumaran - Attached is my screen shot and pom.xml(named as kan_pom.xml). I don't get my jar files updated under Maven dependencies. :(

 

Thanks

Kan


0

Here is my attachment



0

Hi Kan,

screenshot was not attached . And have it named pom.xml not kan_pom.xml. Think maven looks for pom.xml filename by default


0

If you have teamviewer software and want me to take a look at your eclipse , I can do that.

Maven dependency seem fine , if you check dependency hierarchy tab you can see all the jars by expanding the trees  


0

Hi Kan,

Please remove the dependency and add them back using only "Add Jar" option available above "Add External Jar". It will resolve the issue.

Please try and share your workspace folder if you still get the same error.

Hope this helps.

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password