Splitting output records


2 Answer(s)


All the output from MApper (k2,v2) will be automatically handled and transferred by the MR framework to node where sorting/merging will happen and data will be send to a node where the Reducer is going to execute.

You only need to create logic for map and reducer

Dear DeZyre Support,
Did you understand my question? If not please ask rather giving me an useless answer. I'm receiving similar type of answers to my questions and you are not helping.

Here is my pseudo code for the above question:
public void reduce(Text key, Iterator volumes, OutputCollector output, Reporter report)
{
if (key.toString().equals(key1))
{
// Send output records to DIR-1
}
else
{
// Send output records to DIR-2
}
}

So when I do "hadoop fs -ls /opt/hadoop_dfs/DIR-1" should list all files for the key1 similarly "hadoop fs -ls /opt/hadoop_dfs/DIR-2" should list all files for the key2.

The idea is that I do not want to write two reduce classes or even a function to handle one simple if condition. Let me know if there are any limitations.

If still not clear please ask.
Appreciate your support.
-Lakshman