flume installation on demo VM, cloudera
1 Answer(s)
DeZyre Support
hi Lambrini,
Please create this directory in hdfs (hadoop dfs -mkdir /user/cloudera/flume/)
and create the flumeSpool in local filesystem using (mkdir /home/cloudera/flumeSpool).
Once the above directories are created take the Module9_Flume-conf and put it in local file system of "/home/cloudera" and start it using the following command
flume-ng agent -n agent -c conf -f flume-conf.properties
Once the flume-ng has started, you should see that everything has started from the info on the terminal. Now open another terminal/tab and copy a text file of lets say 100 lines into the directory /home/cloudera/flumeSpool.
Once the text files are copied , you can see the output on the first terminal where the Flume was executing, you can see that within hdfs location (/user/cloudera/flume/) the data from the copied files will be split across multiple files and each file has 20 lines each.
Hope this helps.
Thanks
Sep 22 2014 10:07 AM
Please create this directory in hdfs (hadoop dfs -mkdir /user/cloudera/flume/)
and create the flumeSpool in local filesystem using (mkdir /home/cloudera/flumeSpool).
Once the above directories are created take the Module9_Flume-conf and put it in local file system of "/home/cloudera" and start it using the following command
flume-ng agent -n agent -c conf -f flume-conf.properties
Once the flume-ng has started, you should see that everything has started from the info on the terminal. Now open another terminal/tab and copy a text file of lets say 100 lines into the directory /home/cloudera/flumeSpool.
Once the text files are copied , you can see the output on the first terminal where the Flume was executing, you can see that within hdfs location (/user/cloudera/flume/) the data from the copied files will be split across multiple files and each file has 20 lines each.
Hope this helps.
Thanks