In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security
- Making a case for real time processing of log files
Getting logs at real time using Flume Log4J appenders
Making a case for Kafka for log aggregation.
Storing log event as a time series datasets in HBase
Integrating Hive and HBase for data retrieval using query.
- and more...