1-844-696-6465 (US)        +91 77600 44484        help@dezyre.com

Map-reduce deployment related Q



0
Hello,
This was covered in the class but somehow I forgot about it. I think mapper and reducer class files are deployed on each node , where they run in node's jvm. However, I am unable to understand the use case for the following:

Java Library
DistributedCache.addFileToClassPath(new Path("/myapp/mylib.jar"), job);

Would you please share a use case where we may need to send the jar dynamically?

Thanks

Regards,
Karuna

2 Answer(s)


0

There might be cases, where you want to distribute jar files dynamically during runtime. Remember these are not the MapRed specific code. MapReduce jars are distributed before the start of MR program.

0

Right...Would you be able to share one of the cases where we may have to distribute jars dynamically? and how would they be used? Is it that the methods will be overloaded by dynamic jar or dynamic jar methods code will have greater precedence ?

Also, would you please clarify if MR code will be distributed each time we run that mapred program? Or they will be deployed on each of the node as part of s/w deployment?

Thanks

Karuna

Thanks
Karuna

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password