Single node cluster setup issue


7 Answer(s)


hi Vijaykanth,

Can you confirm if the SNN, JT, TT on the node. And is it Hadoop 2.x which was installed as you were talking about Resource manager as well.

I recommend to go to the logs location, open the log files for the unseen processes and see what is the exact error. or paste the error in the forum.

Thanks

Hi Vijaykanth,
JPS normally has security privileges setup such that you will not see the daemons/services running in with other user privileges.

You could try running $ sudo jps
This will user your account authentication and run JPS as root and hence should be able to find the services.

On my machine when i run $ jps the output i see is this.
$ jps
xxxx jps
Where as when I run sudo jps I see the following:

$ sudo jps
xxxx JobTracker
xxxx NameNode
xxxx TaskTraker
xxxx DataNode
xxxx RunJar
xxxx Jps
xxxx HMaster


thanks,
Sarath.

I restarted the services and now secondary namenode is running. But JT, TT is not running. And yes i have installed Hadoop 2.x. I do not see any error and in the logs it doesnot say about the services that are not running.

I have pasted the error that i got when i do a hadoop dfs -ls

hduser@ubuntu:~$ hadoop dfs -ls /test
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable



And sarath when i give sudo jps below are the services that are running.

xxxx NameNode
xxxx SecondaryNameNode
xxxx ResourceManager
xxxx NodeManager
xxxx Jps
xxxx DataNode

Also , i checked the hdfs.cmd file where all namenode, datanode etc., script are written. But i was unable to find the script for JT and TT in the file.

Looks like now created directory is being listed out. But the services JT and TT are not running yet. Is it because the script for running JT and TT is not written in hdfs.cmd file ?

hi Vijaykanth,

Every daemons writes out a log file and an error file. if a particular daemon is not listed in the process list, that means the daemons like JT ,TT has died.

One more important point is , you have noted that you had installed hadoop 2.0 and hadoop2.0 doesn't have Job tracker and Task tracker, you have new daemons called Resource manager and node manager. Start these services and see if "jps" is listing these services.

Once more thing , generally all the hadoop daemons are started by a "hadoop" user and not by root. so if the services were started by "hadoop" user, then no need to execute "sudo jps" , you can execute "jps".

Thanks

Thanks a lot.