How do we start the flume agent? Why isn't the conf code that I have not working?



0

Hi,

I am having trouble with starting the flume agent. And the conf file is'nt working.

P.S. I 've attached the screenshot. It is stuck at this point since half an hour. And, when I try to exit using ctrl+C and try this command

flume-ng agent -n TwitterAgent -c conf -f flume.conf

It gives the same error as the screenshot.

I am trying too continue with the project but not matter what I do. I am stuck. Please help!! Could anyone guide me with the steps. Pllease.

 

This is my code: twitter.conf

TwitterAgent.sources = Twitter

TwitterAgent.channels = MemChannel

TwitterAgent.sinks = HDFS

TwitterAgent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource

TwitterAgent.sources.Twitter.channels = MemChannel

TwitterAgent.sources.Twitter.consumerKey = dazAzGh4R0WD7JYuoaXCOUs6O
TwitterAgent.sources.Twitter.consumerSecret = cCCo696epI2fGjFncWrKNb4td5jAAlvjP9zYcv9wtRTRXms0gE

TwitterAgent.sources.Twitter.accessToken = 767753835112636417-164USPYgbKIHbOTdrLxNksbiL80lLo1
TwitterAgent.sources.Twitter.accessTokenSecret = TGGY6BX8dAmBNgGE7afZ5WrfFbRhslfBZ0ZaJOcKZm2S7

TwitterAgent.sources.Twitter.keywords = hadoop, big data, analytics, bigdata, cloudera, data science, data scientist, business intelligence, mapreduce, data warehouse, mahout, hbase, nosql, newsql, businessintelligence, cloudcomputing, kibana, elastic search, spark, python, R programming, Google, Hue, Yahoo, Doug cutting

TwitterAgent.sinks.HDFS.channel = MemChannel

TwitterAgent.sinks.HDFS.type = hdfs

TwitterAgent.sinks.HDFS.hdfs.path = /user/cloudera/tweets/year=%Y/month=%m/dt=%d/dthour=%H/

TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream

TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text

TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000

TwitterAgent.sinks.HDFS.hdfs.rollSize = 0

TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000


TwitterAgent.channels.MemChannel.type = memory

TwitterAgent.channels.MemChannel.capacity = 10000

TwitterAgent.channels.MemChannel.transactionCapacity = 100

 

P.S. I 've attached the screenshot. It is stuck at this point since half an hour. And, when I try to exit using ctrl+C and try this command

flume-ng agent -n TwitterAgent -c conf -f flume.conf

It gives the same error as the screenshot.

I am trying too continue with the project but not matter what I do. I am stuck. Please help!!

 

Regards

Rashmi



4 Answer(s)


0

This is the continuation of my question:

The error is clearly given: Please help.



0

Hi Rashmi,

Could you please attach your flume.conf file here.

I can run the code and check on my system. Fix the issue and reply you soon.

Thanks


0

Hi Abhijit,

Yes sure. PFA

Regards

Rashmi



0

Hi Rahmi,

I checked, I also getting some error, please give me to get back to you with the solution.

Thanks.

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password