Institution: PES University
Course Title: Big Data
Course Code: UE23CS343AB2
Team ID: A2_Team_8_RRC
A real-time data streaming system where topics are created, approved, and consumed dynamically.
Built using Kafka, Python, and MySQL, the project demonstrates adaptive topic management with automated creation through the Kafka Admin API, multi-threaded producers, dynamic consumer subscriptions, and a simple web interface for admin control and visualization.
- Programming Language: Python
- Message Broker: Apache Kafka
- Database: MySQL
- Web Framework: Flask
- Libraries / APIs: kafka-python, Kafka Admin API
- Producer Node: Creates topics and publishes messages
- Kafka Broker: Message routing core
- Consumer Node: Subscribes to active topics
- Admin Node: Approves topics and manages metadata
- Database: Central control plane
- Frontend: Displays topics and subscriptions
| Role | Name |
|---|---|
| Producer | Sumedha Karanam |
| Kafka Broker | Khushi Bhupesh |
| Consumer | Tanya Tripathi |
| Admin & Database | Tanviraj Jayam |