sqoop increamental import



0

I donot have log actually i am facing a real time problem. I have to achieve sqoop Increamental Import of 500 table of my database.So i want to write job which run at midnight and do Increamental import on whole table and give update database in the morning. I know how to make job in oozie using sqoop .But How can if fill the parameter required in script for 500 tables.

like –check-column <column name>

–last value <last check column value>

It will be help full if you will demonstrate it with Example


1 Answer(s)


0

Hi Rakesh,

It is not possible to write whole example for you.

Please first fetch the tables name based on the database types.

Once fetched, store it either in file or excel or csv and incrementally load it using programming language like python, pearl or shell script to add column name to your command automatically.

You can add that into command and let it run for night and fetch the results in the morning.

 

Regards,

Dezyre Tech Support