An AI-powered behavior-aware smart mobility platform that integrates heterogeneous data sources into a unified analytics engine for real-time monitoring, predictive modeling, and optimization of passenger and vehicle flows.
- Real-time Data Integration: GPS from buses, AIS from ferries, train/metro sensors, ticketing and turnstile logs
- AI-Powered Analytics: LSTM/Transformer forecasting, Graph Neural Networks, Reinforcement Learning optimization
- Anomaly Detection: Multi-method anomaly detection with real-time alerting
- Conversational AI: RAG-powered chatbot for operational queries and insights
- Interactive Dashboard: Real-time visualization with heatmaps, flow forecasts, and recommendations
- Data Engineering: Kafka/MQTT ingestion, Apache Spark processing, PostgreSQL/BigQuery storage
- AI/ML Models: Time-series forecasting, graph-based flow prediction, adaptive scheduling
- API Backend: FastAPI with ML model endpoints and real-time services
- Frontend: Streamlit dashboard with interactive visualizations
- Deployment: Docker containers with Kubernetes orchestration
- Monitoring: Prometheus metrics and Grafana dashboards
- Docker and Docker Compose
- Kubernetes cluster (for production deployment)
- Python 3.11+
- PostgreSQL 15+
- Redis 7+
- Apache Kafka
- MQTT Broker
- Clone the repository
git clone <repository-url>
cd smart-mobility-platform- Install dependencies
pip install -r requirements.txt- Start infrastructure services
docker-compose up -d postgres redis kafka zookeeper mqtt- Initialize the database
psql -h localhost -U mobility_user -d mobility_platform -f data-engineering/storage/init.sql- Start the API backend
cd api-backend
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000- Start the dashboard
cd dashboard
streamlit run app.py --server.port 8501- Access the applications
- Dashboard: http://localhost:8501
- API Documentation: http://localhost:8000/docs
- Health Check: http://localhost:8000/health
- Build all services
docker-compose build- Start the complete platform
docker-compose up -d- Verify deployment
docker-compose ps
curl http://localhost:8000/health- Create namespace
kubectl apply -f deployment/kubernetes/namespace.yaml- Deploy configuration and secrets
kubectl apply -f deployment/kubernetes/configmap.yaml- Deploy infrastructure
kubectl apply -f deployment/kubernetes/database.yaml
kubectl apply -f deployment/kubernetes/redis.yaml
kubectl apply -f deployment/kubernetes/kafka.yaml- Deploy applications
kubectl apply -f deployment/kubernetes/api-backend.yaml
kubectl apply -f deployment/kubernetes/dashboard.yaml- Set up monitoring
kubectl apply -f deployment/kubernetes/monitoring.yaml- Configure ingress
kubectl apply -f deployment/kubernetes/ingress.yamlβββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
β Data Sources β β Data Ingestion β β Processing β
β β β β β β
β β’ GPS Data βββββΆβ β’ Kafka/MQTT ββββΆβ β’ Apache Spark β
β β’ AIS Data β β β’ Real-time β β β’ Feature Eng. β
β β’ Sensor Data β β β’ Batch ETL β β β’ Data Quality β
β β’ Ticketing β β β β β
β β’ Weather β β β β β
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
β
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
β Frontend β β API Backend β β AI/ML Models β
β β β β β β
β β’ Dashboard ββββββ β’ FastAPI βββββ β’ LSTM/GNN β
β β’ Visualizationsβ β β’ Authentication β β β’ Anomaly Det. β
β β’ AI Assistant β β β’ WebSocket β β β’ RL Optimizer β
β β β β β β’ RAG System β
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
β
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
β Monitoring β β Storage β β Deployment β
β β β β β β
β β’ Prometheus β β β’ PostgreSQL β β β’ Docker β
β β’ Grafana β β β’ Redis Cache β β β’ Kubernetes β
β β’ Alerting β β β’ Vector DB β β β’ Auto-scaling β
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
- LSTM Models: Passenger demand prediction with 24-hour horizon
- Transformer Models: Advanced sequence modeling for complex patterns
- Features: Temporal encoding, weather integration, event impact modeling
- Network Representation: Stops as nodes, routes as edges with dynamic weights
- Flow Prediction: GNN-based passenger flow forecasting between locations
- Centrality Analysis: Identification of critical network nodes
- Adaptive Scheduling: PPO/DQN agents for vehicle dispatch optimization
- Dynamic Pricing: RL-based demand distribution strategies
- Route Optimization: Real-time route adjustment based on conditions
- Multi-Method Approach: Isolation Forest, Autoencoders, LSTM-AE
- Real-time Detection: Streaming anomaly identification
- Classification: Automatic categorization of anomaly types
- RAG System: Retrieval-Augmented Generation for operational queries
- Knowledge Base: Dynamic integration of system documentation and real-time data
- Multi-Modal: Support for text, voice, and visual interactions
- Vehicle GPS: Real-time location, speed, occupancy
- AIS Data: Ferry positions, destinations, ETAs
- Sensor Data: Station occupancy, environmental conditions
- Ticketing Systems: Passenger flows, payment data
- External APIs: Weather, traffic, special events
- Ingestion: Kafka/MQTT for real-time streaming
- Validation: Data quality checks and cleansing
- Transformation: Feature engineering and normalization
- Storage: Time-series data in PostgreSQL, caching in Redis
- Analysis: Batch and streaming analytics with Spark
- Operational: Real-time system state and performance
- Analytical: Historical patterns and trend analysis
- Predictive: Future state forecasting and scenario modeling
# Database
DB_HOST=localhost
DB_PORT=5432
DB_NAME=mobility_platform
DB_USER=mobility_user
DB_PASSWORD=mobility_pass
# Redis
REDIS_HOST=localhost
REDIS_PORT=6379
# Kafka
KAFKA_BOOTSTRAP_SERVERS=localhost:9092
KAFKA_GROUP_ID=mobility-platform-consumer
# API
API_BASE_URL=http://localhost:8000
JWT_SECRET=your-jwt-secret
# AI/ML
OPENAI_API_KEY=your-openai-key
MODEL_PATH=/app/models- API Backend: Horizontal Pod Autoscaler (2-10 replicas)
- Database: Read replicas for analytics workloads
- Cache: Redis cluster for high availability
- Processing: Spark cluster auto-scaling
- System Health: Uptime, response times, error rates
- ML Performance: Model accuracy, prediction latency
- Business KPIs: Passenger satisfaction, operational efficiency
- Infrastructure: Resource utilization, scaling events
- Operational: Real-time system status and alerts
- Analytical: Performance trends and insights
- Technical: Infrastructure metrics and health
- Critical: System outages, data pipeline failures
- Warning: Performance degradation, capacity issues
- Info: Deployment events, scaling activities
pytest tests/unit/pytest tests/integration/locust -f tests/load/locustfile.pypython -m ai-models.validation.model_validator- Docker Compose for local development
- Hot reloading for rapid iteration
- Mock data generators for testing
- Kubernetes deployment with reduced resources
- Production-like data pipeline
- Automated testing and validation
- Multi-zone Kubernetes deployment
- Auto-scaling and load balancing
- Comprehensive monitoring and alerting
- Blue-green deployments
- Response caching with Redis
- Database connection pooling
- Async processing for ML inference
- Request rate limiting
- Model quantization and pruning
- Batch inference for efficiency
- GPU acceleration where applicable
- Model versioning and A/B testing
- Stream processing for real-time data
- Partitioning strategies for large datasets
- Compression and serialization optimization
- Incremental processing patterns
- JWT-based API authentication
- Role-based access control (RBAC)
- API key management for external integrations
- Session management and timeout policies
- Encryption at rest and in transit
- PII data anonymization
- Audit logging for sensitive operations
- Secure credential management with Kubernetes secrets
- TLS/SSL for all communications
- Network policies in Kubernetes
- API rate limiting and DDoS protection
- Regular security scanning and updates
- Follow PEP 8 style guidelines
- Write comprehensive tests
- Update documentation
- Ensure backward compatibility
This project is licensed under the MIT License - see the LICENSE file for details.
- OpenAI for GPT models and embeddings
- Apache Foundation for Kafka and Spark
- PostgreSQL Global Development Group
- Redis Labs
- Kubernetes community
- Streamlit team
- FastAPI developers
For support and questions:
- Create an issue in the repository
- Contact the development team
- Check the documentation wiki
- Join our community discussions
Smart Mobility Platform - Transforming urban transportation through AI and data science.