Running hive with oozie taking long time and timing out



0

I have a join accross two hive tables one is of 1 billion and 2 nd table is of the order of 10 million. I have some set properties included in HQL file.

If i run the Hql file in Bee line its taking 7 minutes. If i tried to run the same hql file with oozie workflow it is taking more than 2 hours and finally the job fails.

Can you please help me resolving it? can you please let me know what would have went wrong?


1 Answer(s)


0

HI Manojkumar,

Can you post a portion of the log for the failure if you have access to it?
It will give an insight into what the issue might be.

Also, why I don't expect a difference in behaviour, can you try running the same script with hive1 cli using hive -f <filename> and confirm that the behaviour is the same.

Finally, can you confirm the properties that you are setting in the hql file to be sure that they are not affecting the performance of hive run?


Awaiting your response.


Regards.

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password