What are the daemons running for Distributed Processing using MapReduce? (answers for Hadoop 2.x)

José Miguel

I realized that the answers in the slides are for Hadoop 1.X, but is this correct for Hadoop 2.X?

Resource Manager - Master process for Distributed processing
NodeManager - Slave process for Distributed processing

Container - contains the ApplicationMaster process, Map Tasks and Reduce Tasks

Hadoop 2 Daemon Map Reduce

Dec 18 2017 04:30 AM

1 Answer(s)

Karan

Resource Manager - Master process for YARN,Cluster resource management.
NodeManager - Slave process for YaARN, Cluster resource management.

-ApplicationMaster - It is an on demand process started when users submit MapReduce jobs. AM launches Map Tasks and Reduce Tasks

Jan 06 2018 08:43 PM

Relevant Projects

Design a Hadoop Architecture

Learn to design Hadoop Architecture and understand how to store data using data acquisition tools in Hadoop.

Data Science Project in Python on BigMart Sales Prediction

The goal of this data science project is to build a predictive model and find out the sales of each product at a given Big Mart store.

Predict Credit Default | Give Me Some Credit Kaggle

In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.

Human Activity Recognition Using Smartphones Data Set

In this deep learning project, you will build a classification system where to precisely identify human fitness activities.

Music Recommendation System Project using Python and R

Machine Learning Project - Work with KKBOX's Music Recommendation System dataset to build the best music recommendation engine.

Event Data Analysis using AWS ELK Stack

This Elasticsearch example deploys the AWS ELK stack to analyse streaming event data. Tools used include Nifi, PySpark, Elasticsearch, Logstash and Kibana for visualisation.

Natural language processing Chatbot application using NLTK for text classification

In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.

Finding Unique URL's using Hadoop Hive

Hive Project -Learn to write a Hive program to find the first unique URL, given 'n' number of URL's.

Machine Learning project for Retail Price Optimization

In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Ensemble Machine Learning Project - All State Insurance Claims Severity Prediction

In this ensemble machine learning project, we will predict what kind of claims an insurance company will get. This is implemented in python using ensemble machine learning algorithms.

Learn how you can build Big Data Projects