中文 | English
Introduction • Key Features • Technical Architecture • Quick Start • Performance Metrics • Acknowledgments • Citation
DigiBoard is a real-time, on-device mobile system designed to eliminate presenter occlusion in whiteboard demonstrations while reconstructing whiteboard content with low latency and high visual fidelity. Whether for remote education, video conferencing, or in-person presentations, DigiBoard ensures whiteboard content remains clearly visible, as if the presenter never obstructed it.
Specifically engineered for mobile and edge devices, DigiBoard achieves real-time processing through lightweight models and efficient algorithms while maintaining high image quality.
- 🚀 Real-time On-device Inference: Deeply optimized for mobile devices, achieving processing speeds of 30FPS+ on mainstream smartphones.
- 🖼️ Precise Semantic Segmentation: Lightweight segmentation model integrated with EASPP (Enhanced Atrous Spatial Pyramid Pooling) module, accurately extracting presenter silhouettes and precisely locating occluded areas.
- ⏱️ Temporal Background Reconstruction: Efficiently maintains a historical frame buffer to retrieve and fill previously exposed pixel information, achieving seamless background restoration with temporal stability and eliminating flickering artifacts.
- 🌈 Adaptive Image Enhancement: Built-in pipeline featuring automatic white balance correction, adaptive binarization, and color restoration, eliminating uneven lighting and environmental noise to output whiteboard content as clear as digital documents.
DigiBoard's workflow consists of three core modules, forming an efficient real-time processing pipeline:
Figure: DigiBoard End-to-End Processing Pipeline
- Input: Raw camera video frames
- Core: Lightweight segmentation network + EASPP module (Enhanced Feature Pyramid)
- Output: Precise presenter silhouette mask, accurately locating occluded whiteboard areas
- Core: Efficient historical frame buffer + pixel retrieval strategy
- Mechanism: When the presenter moves, the system retrieves "clean" pixel information of occluded areas from the historical buffer for filling
- Advantages: Ensures natural transition of background restoration, avoiding artificial artifacts and temporal flickering
- Processing Pipeline:
- Automatic White Balance: Corrects color shifts under different lighting conditions
- Adaptive Binarization: Enhances stroke contrast, eliminates shadows
- Color Restoration: Restores whiteboard's original appearance, outputs high-definition visuals
- Final Output: Clear, occlusion-free, visually-friendly digital whiteboard content
- Android 10.0+ (API Level 29+)
- Paddle Lite 2.10+
- OpenCV 4.5.3+ (Android SDK)
- Mobile devices with OpenGL ES 3.0+ support
git clone https://github.com/Me106y/DigiBoard.git
cd DigiBoardOpen the project folder with Android Studio and wait for Gradle synchronization to complete.
Connect your Android device and click the Run button to experience DigiBoard on your phone.
| Metric Type | Parameters/Results |
|---|---|
| Segmentation Model Size | 1.97 M |
| Computational Cost | 0.102 G FLOPs |
| Average Latency | 31.1 ms (Tested on Mi 10) |
| Segmentation Accuracy | 0.9096 (mIoU) |
DigiBoard's development would not have been possible without the wisdom and support of the open-source community. We extend our sincere gratitude to:
Thanks to PaddleSeg for providing efficient semantic segmentation tools and rich pre-trained models, which laid a solid foundation for the portrait segmentation module of this project.
Thanks to CS-23-SW-6-21 for providing valuable insights in architectural design, which provided critical support for the smooth progress of the project.
DigiBoard is released under the Apache 2.0 License. See the LICENSE file for more details.