Ubuntu hadoop 2.5 cluster



0
I set up a hadoop 2.5 cluster on ubuntu 12.04. change all config and was able to start namenode,datanode,jobhistory but unable to start dfs. also when i use command hadoop it says that command could not be found. please see below a snapshot of ./start-all.sh with the error and help me fix the issue.

my .bashrc

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-i386/
export PATH=$JAVA_HOME/bin:$PATH


export HADOOP_HOME=$HOME/hadoop-2.5.2
export HADOOP_CONF_DIR=$HOME/hadoop-2.5.2/etc/hadoop
export HADOOP_MAPRED_HOME=$HOME/hadoop-2.5.2
export HADOOP_COMMON_HOME=$HOME/hadoop-2.5.2
export HADOOP_HDFS_HOME=$HOME/hadoop-2.5.2
export YARN_HOME=$HOME/hadoop-2.5.2
export PATH=$PATH:$HOME/hadoop-2.5.2
export HADOOP_COMMON_LIB_NATIVE_DIR=$HOME/hadoop-2.5.2/lib/native
export HADOOP_OPTS="-Djava.library.path=$HOME/hadoop-2.5.2/lib/native"

==============================Hadoop starts here============================



ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ ls
distribute-exclude.sh hdfs-config.sh slaves.sh start-dfs.cmd start-yarn.sh stop-dfs.cmd stop-yarn.sh
hadoop-daemon.sh httpfs.sh start-all.cmd start-dfs.sh stop-all.cmd stop-dfs.sh yarn-daemon.sh
hadoop-daemons.sh mr-jobhistory-daemon.sh start-all.sh start-secure-dns.sh stop-all.sh stop-secure-dns.sh yarn-daemons.sh
hdfs-config.cmd refresh-namenodes.sh start-balancer.sh start-yarn.cmd stop-balancer.sh stop-yarn.cmd
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ ./hadoop-daemon.sh start hdfs
starting hdfs, logging to /home/ubuntu/hadoop-2.5.2/logs/hadoop-ubuntu-hdfs-ubuntu-VirtualBox.out
Error: Could not find or load main class hdfs
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ ./start-dfs.sh
16/02/13 18:26:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
localhost]
sed: -e expression #1, char 6: unknown option to `s'
-c: Unknown cipher type 'cd'
guard.: ssh: Could not resolve hostname guard.: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
Client: ssh: Could not resolve hostname Client: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
highly: ssh: Could not resolve hostname highly: Name or service not known
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is 03:f4:11:9b:e7:cc:84:c7:c3:87:b5:22:b2:91:c9:3a.
Are you sure you want to continue connecting (yes/no)? to: ssh: connect to host to port 22: Connection refused
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known

localhost: Host key verification failed.
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is 03:f4:11:9b:e7:cc:84:c7:c3:87:b5:22:b2:91:c9:3a.
Are you sure you want to continue connecting (yes/no)? ^Clocalhost: Host key verification failed.
Starting secondary namenodes [OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
0.0.0.0]
sed: -e expression #1, char 6: unknown option to `s'
warning:: ssh: Could not resolve hostname warning:: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
Client: ssh: Could not resolve hostname Client: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
guard.: ssh: Could not resolve hostname guard.: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
-c: Unknown cipher type 'cd'
stack: ssh: Could not resolve hostname stack: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
highly: ssh: Could not resolve hostname highly: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is 03:f4:11:9b:e7:cc:84:c7:c3:87:b5:22:b2:91:c9:3a.
Are you sure you want to continue connecting (yes/no)? with: ssh: Could not resolve hostname with: Name or service not known
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
to: ssh: connect to host to port 22: Connection refused
or: ssh: Could not resolve hostname or: Name or service not known
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
^X
Please type 'yes' or 'no': y
Please type 'yes' or 'no': yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
0.0.0.0: Write failed: Broken pipe
16/02/13 18:34:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ jps
3386 ResourceManager
3763 JobHistoryServer
3206 DataNode
3565 NodeManager
4472 Jps
3291 NameNode
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ cd
ubuntu@ubuntu-VirtualBox:~$ gedit .bashrc
ubuntu@ubuntu-VirtualBox:~$ gedit .bashrc
ubuntu@ubuntu-VirtualBox:~$ gedit .bashrc
ubuntu@ubuntu-VirtualBox:~$ cd hadoop-2.5.2/sbin/
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ ls
distribute-exclude.sh hdfs-config.sh slaves.sh start-dfs.cmd start-yarn.sh stop-dfs.cmd stop-yarn.sh
hadoop-daemon.sh httpfs.sh start-all.cmd start-dfs.sh stop-all.cmd stop-dfs.sh yarn-daemon.sh
hadoop-daemons.sh mr-jobhistory-daemon.sh start-all.sh start-secure-dns.sh stop-all.sh stop-secure-dns.sh yarn-daemons.sh
hdfs-config.cmd refresh-namenodes.sh start-balancer.sh start-yarn.cmd stop-balancer.sh stop-yarn.cmd
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ ./start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
16/02/13 18:57:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
localhost]
sed: -e expression #1, char 6: unknown option to `s'
warning:: ssh: Could not resolve hostname warning:: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
guard.: ssh: Could not resolve hostname guard.: Name or service not known
Client: ssh: Could not resolve hostname Client: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
highly: ssh: Could not resolve hostname highly: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
-c: Unknown cipher type 'cd'
that: ssh: Could not resolve hostname that: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
localhost: namenode running as process 3291. Stop it first.
to: ssh: connect to host to port 22: Connection refused
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
localhost: datanode running as process 3206. Stop it first.
Starting secondary namenodes [OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
0.0.0.0]
sed: -e expression #1, char 6: unknown option to `s'
Client: ssh: Could not resolve hostname Client: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
guard.: ssh: Could not resolve hostname guard.: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
-c: Unknown cipher type 'cd'
guard: ssh: Could not resolve hostname guard: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
highly: ssh: Could not resolve hostname highly: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
0.0.0.0: secondarynamenode running as process 6333. Stop it first.
to: ssh: connect to host to port 22: Connection refused
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
16/02/13 18:58:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
resourcemanager running as process 3386. Stop it first.
localhost: nodemanager running as process 3565. Stop it first.
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ ./stop-all.sh
This script is Deprecated. Instead use stop-dfs.sh and stop-yarn.sh
16/02/13 18:59:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping namenodes on [OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
localhost]
sed: -e expression #1, char 6: unknown option to `s'
VM: ssh: Could not resolve hostname VM: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
Client: ssh: Could not resolve hostname Client: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
guard.: ssh: Could not resolve hostname guard.: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
-c: Unknown cipher type 'cd'
now.: ssh: Could not resolve hostname now.: Name or service not known
highly: ssh: Could not resolve hostname highly: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
localhost: stopping namenode
to: ssh: connect to host to port 22: Connection refused
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
localhost: stopping datanode
Stopping secondary namenodes [OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
0.0.0.0]
sed: -e expression #1, char 6: unknown option to `s'
Client: ssh: Could not resolve hostname Client: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
guard.: ssh: Could not resolve hostname guard.: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
-c: Unknown cipher type 'cd'
highly: ssh: Could not resolve hostname highly: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
0.0.0.0: stopping secondarynamenode
to: ssh: connect to host to port 22: Connection refused
with: ssh: Could not resolve hostname with: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
16/02/13 19:00:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
stopping yarn daemons
stopping resourcemanager
localhost: stopping nodemanager
no proxyserver to stop
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ jps
3763 JobHistoryServer
8384 Jps
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ clear

ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ ./start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
16/02/13 19:01:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
localhost]
sed: -e expression #1, char 6: unknown option to `s'
loaded: ssh: Could not resolve hostname loaded: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
Client: ssh: Could not resolve hostname Client: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
guard.: ssh: Could not resolve hostname guard.: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
-c: Unknown cipher type 'cd'
VM: ssh: Could not resolve hostname VM: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
highly: ssh: Could not resolve hostname highly: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
localhost: starting namenode, logging to /home/ubuntu/hadoop-2.5.2/logs/hadoop-ubuntu-namenode-ubuntu-VirtualBox.out
to: ssh: connect to host to port 22: Connection refused
link: ssh: Could not resolve hostname link: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known
localhost: starting datanode, logging to /home/ubuntu/hadoop-2.5.2/logs/hadoop-ubuntu-datanode-ubuntu-VirtualBox.out
Starting secondary namenodes [OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
0.0.0.0]
sed: -e expression #1, char 6: unknown option to `s'
Client: ssh: Could not resolve hostname Client: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
guard.: ssh: Could not resolve hostname guard.: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
-c: Unknown cipher type 'cd'
guard: ssh: Could not resolve hostname guard: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
highly: ssh: Could not resolve hostname highly: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
0.0.0.0: starting secondarynamenode, logging to /home/ubuntu/hadoop-2.5.2/logs/hadoop-ubuntu-secondarynamenode-ubuntu-VirtualBox.out
to: ssh: connect to host to port 22: Connection refused
with: ssh: Could not resolve hostname with: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
16/02/13 19:01:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
starting resourcemanager, logging to /home/ubuntu/hadoop-2.5.2/logs/yarn-ubuntu-resourcemanager-ubuntu-VirtualBox.out
OpenJDK Client VM warning: You have loaded library /home/ubuntu/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
localhost: starting nodemanager, logging to /home/ubuntu/hadoop-2.5.2/logs/yarn-ubuntu-nodemanager-ubuntu-VirtualBox.out
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ jps
8858 DataNode
3763 JobHistoryServer
8678 NameNode
9185 SecondaryNameNode
9657 Jps
9352 ResourceManager
9538 NodeManager
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ hadoop
hadoop: command not found
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$ haddop fs -mkdir
haddop: command not found
ubuntu@ubuntu-VirtualBox:~/hadoop-2.5.2/sbin$

9 Answer(s)


0

Hi Patrice,
Please make sure, you have ssh installed.
Command to check ssh - sudo apt-get install openssh-server
If the ssh is not installed, then it will install ssh or it will skip the installation.

Please also correct your bashrc:
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-i386
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$JAVA_HOME/bin
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS=”-Djava.library.path=$HADOOP_INSTALL/lib”

export HADOOP_HOME=$HOME/hadoop-2.5.2
export HADOOP_CONF_DIR=$HOME/hadoop-2.5.2/etc/hadoop
export HADOOP_MAPRED_HOME=$HOME/hadoop-2.5.2
export HADOOP_COMMON_HOME=$HOME/hadoop-2.5.2
export HADOOP_HDFS_HOME=$HOME/hadoop-2.5.2
export YARN_HOME=$HOME/hadoop-2.5.2
export PATH=$PATH:$HOME/hadoop-2.5.2
export HADOOP_COMMON_LIB_NATIVE_DIR=$HOME/hadoop-2.5.2/lib/native
export HADOOP_OPTS="-Djava.library.path=$HOME/hadoop-2.5.2/lib/native"

Hope this helps.
Thanks.

0

Thank you Abhijit,
I was able to resolve that issue but i have another one. Connection Refused as you can see below. I am not able to use hadoop comment suhc hadoop fs -ls or -mkdir. anytime i have error localhost connectionRefused


ubuntu@ubuntu-VirtualBox:~$ hadoop fs -mkdir new_dir
OpenJDK Client VM warning: You have loaded library /usr/local/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
16/02/15 11:59:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
mkdir: Call From ubuntu-VirtualBox/127.0.1.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
ubuntu@ubuntu-VirtualBox:~$ hadoop fs -ls
OpenJDK Client VM warning: You have loaded library /usr/local/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
16/02/15 12:00:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
ls: Call From ubuntu-VirtualBox/127.0.1.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
ubuntu@ubuntu-VirtualBox:~$ telnet localhost 9000
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
ubuntu@ubuntu-VirtualBox:~$

0

Hi Patrice,
Please change the host name in your host file:
Commands- sudo vi /etc/hosts
(see the snapshot for how to edit)
--Please made the changes as per attached snapshot.
Restart the hadoop cluster again.

Hope this helps.
Thanks


0

as you can see below, I made the change that you asked me but datanode and namenode are not starting anymore and the connection issue persist.

It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
0.0.0.0]
sed: -e expression #1, char 6: unknown option to `s'
-c: Unknown cipher type 'cd'
guard.: ssh: Could not resolve hostname guard.: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
Client: ssh: Could not resolve hostname Client: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.5.2/logs/hadoop-ubuntu-secondarynamenode-ubuntu-VirtualBox.out
to: ssh: connect to host to port 22: Connection refused
highly: ssh: Could not resolve hostname highly: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known
',: ssh: Could not resolve hostname ',: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
16/02/15 14:33:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop-2.5.2/logs/yarn-ubuntu-resourcemanager-ubuntu-VirtualBox.out
localhost: starting nodemanager, logging to /usr/local/hadoop-2.5.2/logs/yarn-ubuntu-nodemanager-ubuntu-VirtualBox.out
ubuntu@ubuntu-VirtualBox:~$ jps
5179 ResourceManager
5039 SecondaryNameNode
5461 Jps
ubuntu@ubuntu-VirtualBox:~$


0

Hi Abi, Please I have not heard from you. Did you have time to get more help on my question from your team please?

0

Hi Patrice,
The hostname of your system is ubuntu-Virtualbox. Please correct the /etc/hosts file and replace ubuntu with ubuntu-VirtualBox.
Your system is refusing the connection on Port-22.
Please disable the firewall of your system:
- sudo ufw disable

Please share the output of the following command:
- $HADOOP_HOME/sbin/start-dfs.sh
- $HADOOP_HOME/sbin/start-yarn.sh
- jps
You can mail the output at help@dezyre.com
Sorry for inconvenience.
Thanks

0

I tried those again. here is the error: connection still failed because connection is refused. Tried to open the port but nothing.

ubuntu@ubuntu-VirtualBox:~$ sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT[sudo] password for ubuntu:
ubuntu@ubuntu-VirtualBox:~$ hadoop fs -ls
OpenJDK Client VM warning: You have loaded library /usr/local/hadoop-2.5.2/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
16/02/18 00:19:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
ls: Call From ubuntu-VirtualBox/127.0.0.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
ubuntu@ubuntu-VirtualBox:~$ telnet localhost 22
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
SSH-2.0-OpenSSH_6.0p1 Debian-3ubuntu1.2
Connection closed by foreign host.
ubuntu@ubuntu-VirtualBox:~$

Any idea what i am doing wrong? can we connect using teamviewer to get on this please.


0

Hi Patrice,
Please try this, if this doesn't work, we will surely do remote session.
- Stop all the hadoop services
($HADOOP_HOME/sbin/stop-yarn.sh)
($HADOOP_HOME/sbin/stop-dfs.sh)
- Delete the hdfs folder(sudo rm -r )
- Create the hdfs folder again(sudo mkdir )
- Assign the owner ship of the user (sudo chown )
- Format the namenode (hadoop namenode -format)
- Start the hadoop services ($HADOOP_HOME/sbin/start-dfs.sh)
- Start the yarn services ($HADOOP_HOME/sbin/start-yarn.sh)

Please let me know if its works for you.
Thanks.

0

I followed all that but nothing. As you can see on the screenshot below the problem is not folder right (at least from what i think). The issue is that hadoop cannot connect to local host port 22 however i opened this port following all kind of online tutorial including your instruction.


Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password