Hi

Like you guessed - this is a permission issue. Let me explain.
The /user/ directory is owned by "hdfs" - hdfs is a superuser

You are logging in as user cloudera - so you will have permissions only to create or write files in the folder /user/cloudera

So a simple fix to your problem would be specify the /user/cloudera folder as shown below

hadoop jar hadoop-examples.jar teragen 1000 /user/cloudera/teragen/out2

Then you can view the output as follows

hadoop fs -cat /user/cloudera/teragen/out2/part-00001

Hope this helps
Thanks

I am having the same issue, please help me to resolve.

I logged in as user cloudera and ran the following command as suggested: hadoop jar hadoop-examples.jar teragen 1000 /user/cloudera/teragen/out2

I received the following error:
Exception in thread "main" java.io.IOException: Error opening job jar: hadoop-examples.jar

at org.apache.hadoop.util.RunJar.main(RunJar.java:135)
Caused by: java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:127)
at java.util.jar.JarFile.(JarFile.java:135)
at java.util.jar.JarFile.(JarFile.java:72)
at org.apache.hadoop.util.RunJar.main(RunJar.java:133)

I found 2 solutions that work as follows but would like to know what is the best solution and why:

Solution 1 - hdfs is the user that owns the folder being accessed. We can run the command "hadoop fs -mkdir /dezyre" as the hdfs user using "sudo -u hdfs" as a prefix to "hadoop fs -mkdir /dezyre" ("sudo -u hdfs hadoop fs -mkdir /dezyre"). Now the command executes without error. If we use this method consistently then it will require a conscious modification to any commands discovered in normal documentation.

Solution 2 - supergroup is the group that owns the folder being accessed, so any users in this group will be able to access the files. Would the ideal solution be to create a user group called supergroup and add all users that should access files under that ownership in HDFS to the group? Since HDFS permissions are tied into the permissions set on the local file system for any users accessing the cluster, this could be performed using the following commands (make sure to su to root):

groupadd supergroup
usermod -a -G supergroup cloudera

Could someone please let me know what the best solution is and why and if you have any other solutions that would work.

hi Robert,

you need to provide complete path of hadoop-examples jar file execute the following in cloudera. There is no need to make an permission changes.

hadoop jar /usr/lib/hadoop-0.20-mapreduce/hadoop-examples.jar teragen 1000 /user/cloudera/teraoutput

check the output using

hadoop dfs -cat /user/cloudera/teraoutput

Thanks

Hi Shobhanc,

Ok, I understand what you are saying regarding the hadoop jar command for the hadoop-examples.jar file.

In regards to creating a directory; if I am within a folder where cloudera owns the folder like the /user/cloudera folder, then there is no need to make permission changes to make a directory/folder. But if we want to make the /dezyre directory like in the page 2 of the HDFS command assignment, then we must change permissions since hdfs is the user that owns that folder.

My question in regards to this situation where we want to make a directory where hdfs is the user that owns the folder; What is the best practice; my Solution 1 or my Solution 2 as stated previously or something different?

Thank you,
Rob

hi Robert,

The user "cloudera" is part of the super group and when you create a directory in hdfs, all the required permissions are set. I've never modified any permissions for any created files/dierctories in hdfs.

Lets see the issue in tomorrows session.

Thanks

Hi Shobhanc,

Sounds good, we will talk tomorrow.

Thank you