What are the daemons running for Distributed Processing using MapReduce? (answers for Hadoop 2.x)

José Miguel

I realized that the answers in the slides are for Hadoop 1.X, but is this correct for Hadoop 2.X?

Resource Manager - Master process for Distributed processing
NodeManager - Slave process for Distributed processing

Container - contains the ApplicationMaster process, Map Tasks and Reduce Tasks

Hadoop 2 Daemon Map Reduce

Dec 18 2017 04:30 AM

1 Answer(s)

Karan

Resource Manager - Master process for YARN,Cluster resource management.
NodeManager - Slave process for YaARN, Cluster resource management.

-ApplicationMaster - It is an on demand process started when users submit MapReduce jobs. AM launches Map Tasks and Reduce Tasks

Jan 06 2018 08:43 PM

Relevant Projects

Event Data Analysis using AWS ELK Stack

This Elasticsearch example deploys the AWS ELK stack to analyse streaming event data. Tools used include Nifi, PySpark, Elasticsearch, Logstash and Kibana for visualisation.

Yelp Data Processing using Spark and Hive Part 2

In this spark project, we will continue building the data warehouse from the previous project Yelp Data Processing Using Spark And Hive Part 1 and will do further data processing to develop diverse data products.

Data Science Project in Python on BigMart Sales Prediction

The goal of this data science project is to build a predictive model and find out the sales of each product at a given Big Mart store.

Hadoop Project for Beginners-SQL Analytics with Hive

In this hadoop project, learn about the features in Hive that allow us to perform analytical queries over large datasets.

Data Science Project on Wine Quality Prediction in R

In this R data science project, we will explore wine dataset to assess red wine quality. The objective of this data science project is to explore which chemical properties will influence the quality of red wines.

Machine Learning project for Retail Price Optimization

In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Identifying Product Bundles from Sales Data Using R Language

In this data science project in R, we are going to talk about subjective segmentation which is a clustering technique to find out product bundles in sales data.

Tough engineering choices with large datasets in Hive Part - 1

Explore hive usage efficiently in this hadoop hive project using various file formats such as JSON, CSV, ORC, AVRO and compare their relative performances

Real-Time Log Processing using Spark Streaming Architecture

In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Perform Time series modelling using Facebook Prophet

In this project, we are going to talk about Time Series Forecasting to predict the electricity requirement for a particular house using Prophet.

Learn how you can build Big Data Projects