1-844-696-6465 (US)        +91 77600 44484        help@dezyre.com

How does cloudera environment look in real project?



0
Hi Team,

I would like to know how the cloudera environment in a real project would be. How we would access the cloudera environment? Do we use remote desktop to access or vm player. How we will deploy our mapreduce programs in the cloudera environment?

I am planning to do a POC for my client and would like to know what are the essential things that I need to do start with.

Thanks
Preethi

3 Answer(s)


1

1. In real project following set of machines can be a part of hadoop setup:
a. Cloudera cluster: comprises of several machines behaving as datanodes/tastracker and one or machine behaving as namenode/jobtracker.
b. Client machines: from where cluster can be accessed for firing hadoop commands [client machine itself can be a part of hadoop cluster]
c. User machine: using which user can remotely connect to client machines and perform hadoop specific operations.

2. To deploy map-reduce program on cloudera environment,
a. Prepare jar containing the MapReduce code and related dependency jar files
b. Add the jar in HADOOP_CLASSPATH eg. export HADOOP_CLASSPATH=HADOOP_CLASSPATH:
c. Fire command: hadoop jar

3. To start on a POC, a single [windows/linux] machine would suffice with Cloudera VM installed on it.
Link : http://www.cloudera.com/content/support/en/downloads/download-components/download-products.html?productID=F6mO278Rvo
This VM comes with pre-installed psuedo-distributed hadoop components and eclipse.
Eclipse can be used to write down java based map-reduce code and pre-installed cloudera hadoop can be used to run the program.
Once done with the complete functionality, deploy the program on 5-node hadoop cluster with some good amount of big-data to showcase the time differences.
To deploy 5-node cluster, you can take help from dezyre administrator course material.

0

Hi Neha,
Thanks for your inputs

Regards
Preethi

0

You are welcome Preethi.
If you need any further help, post here.
Requesting to vote-up, if found useful.

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password