AI-Powered Pet Communication System
PawTalk is a modular, AI-driven ecosystem that revolutionizes human-pet communication using button-based "speech," bark analysis, and smart home integration. Built for multi-pet households (starting with dogs LeBron & Kobe, cats Cobweb & Punkin).
Current Phase: Prototyping
Target: Ship in 1-2 weeks
Cost: <$100 total
pawtalk/
├── hardware/ # ESP32 firmware, circuits, 3D models
├── mobile/ # Flutter app (iOS/Android)
├── ai/ # ML models & training pipelines
├── backend/ # Node.js API for data sync
├── fvr-integration/ # .FVR adapters for session tracking
├── docs/ # Comprehensive documentation
└── tools/ # Development utilities
- 🔘 Multi-function buttons - Short press vs. hold (e.g., "outside" vs. "urgent outside")
- 🎨 Visual feedback - RGB LEDs + audio confirmation
- 🤖 AI intent prediction - "Dog LLM" for context-aware responses
- 📱 Mobile app - Training mode, pet profiles, analytics
- 🐕 Bark analysis - Integrate vocal patterns for richer context
- 💾 Anti-drift tracking - Full session replay via .FVR files
- 🔄 Wireless sync - BLE + WiFi connectivity
- Hardware: ESP32 board, arcade buttons, RGB LEDs, DFPlayer Mini
- Software:
- PlatformIO (ESP32 development)
- Flutter SDK 3.24+ (mobile app)
- Python 3.11+ with Poetry (AI pipeline)
- Node.js 20+ with npm (backend)
# Clone and enter directory
cd pawtalk
# 1. Hardware Setup
cd hardware/firmware
pio run --target upload
# 2. Backend Setup
cd ../../backend
npm install
npm run dev
# 3. AI Pipeline Setup
cd ../ai
poetry install
poetry run python train_intent_model.py
# 4. Mobile App Setup
cd ../mobile
flutter pub get
flutter runSee GETTING_STARTED.md for detailed instructions.
- ✅ V1 (Weeks 1-2): Button tech + basic AI + mobile app
- 🔄 V2 (Week 3): HomeKit integration & automation
- 🎯 V3 (Weeks 4-6): Gemini ER spatial awareness + agent hooks
- 🌟 Future: Multi-species support, community .FVR sharing, VR/AR modes
- LeBron (Bernese Mountain Dog) - Trained, modeling for Kobe
- Kobe (Bernese Mountain Dog) - Learner
- Cobweb (Cat) - Indoor only
- Punkin (Cat) - Outdoor privileges
| Component | Technology |
|---|---|
| Firmware | C++ (PlatformIO) |
| Mobile | Flutter 3.24, Dart 3.5 |
| Backend | Node.js 20, TypeScript 5.5, Express |
| AI/ML | Python 3.11, PyTorch 2.4, Hugging Face Transformers |
| Database | SQLite (local), Firebase (optional cloud sync) |
| Communication | BLE (via flutter_blue_plus), WiFi (HTTP/WebSocket) |
| FVR | TFCG-FVR-SUITE adapters |
- Getting Started Guide
- Hardware Assembly
- Firmware Development
- Mobile App Architecture
- AI Model Training
- FVR Integration
- API Reference
This is an in-house project for now. See CONTRIBUTING.md for development guidelines.
MIT License - see LICENSE for details.
- Inspired by Zoolingua, FluentPet, UMich bark research
- Built on TFCG-FVR-SUITE for anti-drift provenance
- Powered by Hugging Face, Gemini ER, HomeKit
Built with ❤️ for LeBron, Kobe, Cobweb & Punkin