FamilyLog: Monitoring Family Mealtime Activities by Mobile Devices

Abstract

By learning from the existing family mealtime activities, family members can be motivated to make the positive changes towards better relationships, which are important for the physical and mental health of children. Moreover, the details of family mealtime activities provide rich information for study in sociology and culture. This paper presents FamilyLog - a practical system to log family mealtime activities using smartphones and smartwatches. FamilyLog automatically detects and logs details of activities during the mealtime, including occurrence and duration of meal, conversations, participants, TV viewing, etc., in an unobtrusive manner. Based on the sensor data collected from real families, we carefully design robust yet lightweight signal features from a set of complex activities during the meal, including clattering sound, arm gestures of eating, human voice, TV sound, etc. Moreover, FamilyLog opportunistically fuses data from built-in sensors of multiple mobile devices available in a family with a CRFs-based classifier. To evaluate the real-world performance of FamilyLog, we perform extensive experiments that consist of 77 days of sensor data from 37 subjects in 8 families with children. FamilyLog can detect those events with high accuracy across different families and home environments.