flume installation on demo VM, cloudera


1 Answer(s)


hi Lambrini,

Please create this directory in hdfs (hadoop dfs -mkdir /user/cloudera/flume/)
and create the flumeSpool in local filesystem using (mkdir /home/cloudera/flumeSpool).

Once the above directories are created take the Module9_Flume-conf and put it in local file system of "/home/cloudera" and start it using the following command

flume-ng agent -n agent -c conf -f flume-conf.properties

Once the flume-ng has started, you should see that everything has started from the info on the terminal. Now open another terminal/tab and copy a text file of lets say 100 lines into the directory /home/cloudera/flumeSpool.

Once the text files are copied , you can see the output on the first terminal where the Flume was executing, you can see that within hdfs location (/user/cloudera/flume/) the data from the copied files will be split across multiple files and each file has 20 lines each.

Hope this helps.
Thanks