AI Robotic Infant and Toddlers Alert In-Home System

[Paper]
Authors: Minchae Kim, Seojin Lee, Taein Kim, Hyung Seok Kim

Abstract: Babies require continuous observation to ensure their safety, yet most existing action recognition systems are ill-suited for recognizing baby-specific behaviors, especially in real-world home environments with low illumination and visual clutter. In this study, we propose a unified vision-based baby behavior recognition system that supports both infant and toddler stages. Our model is trained on a custom video dataset collected from in-the-wild YouTube footage, capturing diverse lighting conditions and baby-specific actions, including high-risk behaviors such as face-down posture and chewing toys. To enhance robustness under poor lighting, we simulate low-light conditions and apply a lightweight enhancement model to recover visual clarity. We adopt a transformer-based VideoMAE V2 backbone to learn spatiotemporal features and perform behavior classification. The proposed system achieved an overall accuracy of 83.46 % across 8 behavior classes under low-light conditions. We further validated the system’s applicability by connecting it to a mobile robot in a simulated environment using a baby doll, confirming its feasibility for real-time robotic monitoring. These results suggest that the system can serve as a foundation for autonomous baby monitoring in real-world settings.