Hive load from directory


0

I am trying to load data into hive table from directory. 

LOAD DATA INPATH '/user/cloudera/twitter/*' INTO TABLE tweets_partioned;

I am getting error - FAILED: SemanticException Line 1:17 Invalid path ''/user/cloudera/twitter/*'': source contains directory: hdfs://192.168.219.128:10001/user/cloudera/twitter/2018.

Data lies in - /user/cloudera/twitter/2018/06/10/05. I want to recuservingly load it.

hive 0.11 and hadoop 1.2.

I have set following 3 properties and restarted hadoop and hive both, but no effect



     mapred.input.dir.recursive
     true 


    hive.supports.subdirectories
    true   


    hive.mapred.supports.subdirectories
    true   

How can I,  load data recursively from directories in hive table?

 

 


1 Answer(s)


0

Hi Pankaj,

You can use the Oozie to schedule the job for your hive. Please check the link below: 

http://www.tanzirmusabbir.com/2013/03/oozie-example-hive-actions.html

http://www.thecloudavenue.com/2013/10/executing-oozie-workflow-with-pig-hive.html

https://www.tutorialspoint.com/apache_oozie/apache_oozie_workflow.htm

 

Here is one good example of using Hive and oozie for Twitter data:

https://github.com/cloudera/cdh-twitter-example

Hope this helps.

Regards,

DeZyre Tech Support

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password