DireWolves
- Aditya Bhardwaj (abhardw2)
- Gautam Verma (gverma)
- Samir Jha (sjha4)
Identification and Analysis of popular trends around the world.
In this project, we aim to combine and analyze the real-time data produced by Meetup and Twitter API at a very high speed (120-180 events per minute and ~50 tweets per second). Our analysis include identification of popular trends and choices and the factors on which it is based on a world-level scale.
- The statistical inference can be performed in real time, independent of the velocity of data stream and data size.
- The two streams can be perfectly synchronized in real time and resources are dynamically scaled according to the stream speed and data size.
- Combining two streams of data into a meaningful analysis in real-time considering volume of data received and the stream of data.
- We can use the existing frameworks part of AWS, Kinesis firehose, analytics and S3 to achieve the promised scalability, reliability and reduced latency.
- AWS - S3
- Kinesis Firehose
- Spark
- Hadoop