twitter project qn


I was able to do the project , but don't know where to see the results?

I have these qns -

1. Where to see the results?

2. What all needs to be part of submission 

3. Had issues in Oozie workflow setup, it never started and didn't find config file. I did alternate way to complete it as in the PDF. Can you check on this and let me know if steps provided  are correct in PDF.

2 Answer(s)


Hello Veena ,

I dont have answers for all your questions , i do have issues in completing the project , i cant see the data being fetched in the tweets partition table . i have raised it with the team hope they should be able to help us. 

To your second question, please follow the sharepoint link -> module 15 -> solution document template and architecture diagram for the documentation part of the project.




1. Where to see the results.

Please download Twitter Project zip file from Module-15 in LMS. In that zip file, you will find a flume configuration file(flume.conf).

This file contains the hds location where the tweets will be downloaded. 

TwitterAgent.sinks.HDFS.hdfs.path = hdfs://hadoop1:8020/user/flume/tweets/

I am attaching a sample flume configuration file for your reference. Please follow the instructions in Module_Project_Twitter_demo.pdf document to complete the project.

2.I am attaching the solution template to this ticket. Your solution document should include the following

(i) Problem description

(ii)Architecture diagram

(ii) Source code

(iv) snapshot of input and output

(v) commands used to run your programs.


3. You can ignore the steps related to oozie. You can load the data in HDFS directly into Hive tables and complete the project.