Blueprint Proposal for AI-Powered In-Vehicle Companion (CarMate)
Goal
This blueprint demonstrates an end-to-end Software Defined Vehicle (SDV) architecture for an AI-powered in-vehicle companion, named CarMate, that enhances driver well-being, safety, and interaction.
CarMate integrates real-time vehicle data, AI-driven conversational interaction, and safe human-machine interfaces into a reproducible SDV reference architecture.
The blueprint connects:
- In-vehicle data sources, including MCU sensors, simulation data from CARLA, and vehicle signals
- Vehicle middleware and orchestration components, including VSS-based data models, MQTT providers, Eclipse Kuksa Databroker, and digital.auto
- AI interaction components, including speech processing, conversational AI, and LLM integration
- User interface and visualization components, including cluster display and Web UI
The goal is to provide a reusable and extensible SDV blueprint based on open-source components, aligned with the COVESA Vehicle Signal Specification (VSS) semantics, and suitable for future integration with broader SDV capabilities, such as service mesh, orchestration, authentication, and authorization.
Use Case
This blueprint focuses on a driver companion scenario for long-distance drivers, such as truck drivers and commuters.
The AI companion combines emotional engagement, contextual awareness, and intelligent vehicle interaction.
Example use cases include:
- Capturing driver voice input and converting it into structured commands via speech-to-text
- Mapping vehicle and environmental data, such as temperature, weather, and simulation data, into VSS signals
- Enabling conversational AI to:
- Respond socially and reduce driver loneliness
- Provide contextual driving insights, such as weather and road condition information
- Control selected vehicle features, such as ambient lighting
- Supporting bidirectional interaction:
- Driver → AI → Vehicle systems
- Vehicle data → AI → Driver feedback
- Visualizing key vehicle data and AI-generated insights in the cluster display
Projects
This blueprint builds upon the following open source projects and technologies:
- Eclipse Kuksa
- Eclipse Mosquitto
- Eclipse Zenoh
- CARLA Simulator
- COVESA VSS
- Eclipse AutoWRX
- Docker
- Local or external LLM providers
CarMate WIP Repo: link
Future Work
1. Emotion & Driver State Awareness
Enhance CarMate with driver monitoring capabilities (e.g., voice tone analysis, camera-based detection) to infer fatigue, stress, or mood. This would allow the AI to adapt conversations, suggest breaks, or reduce interaction intensity for safer driving.
2. AI Optimization
Move from reactive responses to proactive intelligence. CarMate could anticipate driver needs based on context (route, time, habits), such as suggesting rest stops, warning about upcoming hazards, or preparing vehicle settings in advance.
3. Integration with Advanced Vehicle Systems
Expand beyond basic signals to integrate with ADAS and infotainment systems (e.g., navigation, driver assistance alerts). CarMate could proactively explain warnings, assist in decision-making, or provide contextual driving guidance.
4. Multi-User & Personalization Profiles
Introduce driver profiles with memory and preferences (e.g., language, tone, favorite settings). CarMate could recognize different drivers and adapt behavior, making the experience more personalized and consistent across trips.
Acknowledgement
This blueprint is built upon and further developed from the ArBytesMoral project, which was created during the Eclipse SDV Hackathon Chapter Three.
We would like to sincerely thank all contributors and team members of ArBytesMoral for their foundational work that made this blueprint possible.
Blueprint Proposal for AI-Powered In-Vehicle Companion (CarMate)
Goal
This blueprint demonstrates an end-to-end Software Defined Vehicle (SDV) architecture for an AI-powered in-vehicle companion, named CarMate, that enhances driver well-being, safety, and interaction.
CarMate integrates real-time vehicle data, AI-driven conversational interaction, and safe human-machine interfaces into a reproducible SDV reference architecture.
The blueprint connects:
The goal is to provide a reusable and extensible SDV blueprint based on open-source components, aligned with the COVESA Vehicle Signal Specification (VSS) semantics, and suitable for future integration with broader SDV capabilities, such as service mesh, orchestration, authentication, and authorization.
Use Case
This blueprint focuses on a driver companion scenario for long-distance drivers, such as truck drivers and commuters.
The AI companion combines emotional engagement, contextual awareness, and intelligent vehicle interaction.
Example use cases include:
Projects
This blueprint builds upon the following open source projects and technologies:
CarMate WIP Repo: link
Future Work
1. Emotion & Driver State Awareness
Enhance CarMate with driver monitoring capabilities (e.g., voice tone analysis, camera-based detection) to infer fatigue, stress, or mood. This would allow the AI to adapt conversations, suggest breaks, or reduce interaction intensity for safer driving.
2. AI Optimization
Move from reactive responses to proactive intelligence. CarMate could anticipate driver needs based on context (route, time, habits), such as suggesting rest stops, warning about upcoming hazards, or preparing vehicle settings in advance.
3. Integration with Advanced Vehicle Systems
Expand beyond basic signals to integrate with ADAS and infotainment systems (e.g., navigation, driver assistance alerts). CarMate could proactively explain warnings, assist in decision-making, or provide contextual driving guidance.
4. Multi-User & Personalization Profiles
Introduce driver profiles with memory and preferences (e.g., language, tone, favorite settings). CarMate could recognize different drivers and adapt behavior, making the experience more personalized and consistent across trips.
Acknowledgement
This blueprint is built upon and further developed from the ArBytesMoral project, which was created during the Eclipse SDV Hackathon Chapter Three.
We would like to sincerely thank all contributors and team members of ArBytesMoral for their foundational work that made this blueprint possible.