Flume error TwitterSource



0

16/11/17 12:55:26 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: HDFS started
16/11/17 12:55:26 ERROR lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner: { source:com.cloudera.flume.source.TwitterSource{name:Twitter,state:IDLE} } - Exception follows.
java.lang.NoSuchMethodError: twitter4j.FilterQuery.setIncludeEntities(Z)Ltwitter4j/FilterQuery;
    at com.cloudera.flume.source.TwitterSource.start(TwitterSource.java:139)
    at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)

    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
16/11/17 12:55:26 WARN lifecycle.LifecycleSupervisor: Component EventDrivenSourceRunner: { source:com.cloudera.flume.source.TwitterSource{name:Twitter,state:STOP} } stopped, since it could not besuccessfully started due to missing dependencies


14 Answer(s)


0

Hi Jagdeep,

Its seems issue with jar file. Please recreate the jar file again.

If you still get the issue again. Please share the flume.conf with me.

Hope this helps.

Thanks.


0

Hi,

I did create the jar file but it gave me same error.Here is the conf file as attached.Please let me know,when you get a chance.

Thanks!!



0

Hi Jagdeep,

I am looking into it, please give me some time to fix this.

Thanks.


0

Hi Jagdeep,

I tried the code in 2-3 system, it is working fine. I guess you have issue with flume not with flume.conf.

Try this:

1. Delete the flume.

2. Download the new flume setup.

3. Provide the new path in the flume.conf to store the tweets

4. Use the same flume.conf to fetch tweets.

Hope this helps.

Thanks.


0

Hi Abhijit,

I tried to delete the flume but it is not letting me delete the flume.Please guide.

Thanks!!


0

Hi Jagdeep,

Use the following command to delete the flume folder:

$ sudo rm -r path_of_flume_folder

ex. - $ sudo rm -r /home/hadoop/flume

Hope this helps.

Thanks.


0

Hi Abhijit,

I already delete the folder and reinstall flume and now I get diffrerent error as below.I am confused now.

cloudera@localhost /]$ flume-ng agent-n  TwitterAgent -c conf -f /usr/lib/flume-ng/apache-flume-1.7.0-bin/conf/flume.conf
/usr/bin/flume-ng: line 10: /usr/lib/flume-ng/bin/flume-ng: No such file or directory
/usr/bin/flume-ng: line 10: exec: /usr/lib/flume-ng/bin/flume-ng: cannot execute: No such file or directory
[cloudera@localhost /]$ flume-ng agent -n  TwitterAgent -c conf -f /usr/lib/flume-ng/apache-flume-1.7.0-bin/conf/flume.conf
/usr/bin/flume-ng: line 10: /usr/lib/flume-ng/bin/flume-ng: No such file or directory
/usr/bin/flume-ng: line 10: exec: /usr/lib/flume-ng/bin/flume-ng: cannot execute: No such file or directory
[cloudera@localhost /]$ 
 


0

Hi Jagdeep,

It seems, it is not able to find flume-ng maybe because of file permission.

To solve this, copy the flume folder in /home/cloudera/ and it has all the permission required to execute the file.

Thanks.


0

Hi Abhijit,

I checked the folder permission,it has all the permission to execute.


0

I did copied the file to /home/cloudera ,it still same error.The instrcution on LMS are not great to follow.

cloudera@localhost ~]$ flume-ng agent-n TWitterAgent -c conf -f flume.conf/usr/bin/flume-ng: line 10: /usr/lib/flume-ng/bin/flume-ng: No such file or directory
/usr/bin/flume-ng: line 10: exec: /usr/lib/flume-ng/bin/flume-ng: cannot execute: No such file or directory
[cloudera@localhost ~]$ 


0

Hi Abhijit,

The flume is not starting from cludera manager,I tried does not work


0

Hi Jagdeep,

I found the error, /usr/lib/flume-ng/bin/flume-ng: cannot execute: No such file or directory found.

Please confirm you have install the flume successfully by:

$ sudo yum install flume-ng

Please try again, or please come on support chat and provide your contact detail on which I can reach you.

Thanks.


0

Issue has been resolved by Abhijit.He  was very proffessional and very helpful and solved my issue.

Very nice work by him.

Thanks!!

Jagdeep Singh


0

Thank you Jagdeep for appreciation.

Your Answer

Click on this code-snippet-icon icon to add code snippet.

Upload Files (Maximum image file size - 1.5 MB, other file size - 10 MB, total size - not more than 50 MB)

Email
Password