About this srcd poster session

---

Panel information
Panel 12. Methods, History, Theory

Abstract

---

Unpredictability of maternal sensory signals refers to the degree of randomness vs. predictability of maternal micro-level patterns of auditory, visual, and tactile signals in interaction with the child (Davis et al., 2017). Such unpredictability is nowadays considered an important vulnerability marker of the mother–infant relationship, with negative consequences on later child well-being and development (Davis & Glynn, 2024). The typically used method for analyzing maternal unpredictability relies on video observations of mother–infant interaction and manual coding by trained psychologists; a method that is accurate but labor-intensive and costly. Recent advances in the Machine Learning (ML) and Deep Learning (DL) technologies provide possibilities for more efficient and dynamic methods of assessing mother-infant interaction. Therefore, the present study reports an interdisciplinary pursuit to develop an automated ML- and DL-based model to detect the level of unpredictability of maternal sensory signals. We aim at reaching high correspondence between manually and ML analyzed unpredictability.

The study sample consisted of 63 Finnish voluntary mothers with 5–8 month-old infants (infant age M = 6.6 months, SD = 0.8 months; 54% females), participating in the “Machine understanding of mother-infant interaction” -study at Tampere University. The mother-infant dyads made a laboratory visit, where a 12-minute free-play interaction was recorded using a a high-quality multi-camera setup. Unpredictability of the maternal auditory, visual, and tactile signals was manually coded with the Maternal Sensory Behavior Coding Scheme observation method (Davis et al., 2017), using the Noldus Observer XT program. For automatic assessment of maternal unpredictability, we employed a transformer-based model to predict tactile stimuli, Voice Activity Detection (VAD) for auditory stimuli, and Computer Vision techniques to analyze visual stimuli, focusing on gaze direction detection. The preliminary results indicate successfulness of our model in detecting maternal unpredictability, achieving 74.8% accuracy in identifying and categorizing tactile stimuli. Comparable quantitative results for auditory and visual stimuli are still pending calculation and will be addressed in the next phase of the analysis.

To conclude, our study proposes the usefulness of integrating advanced ML and DL techniques with traditional observational methods to gain deeper insights into the dynamics of mother–infant relationship. Yet, although utilizing new automatic techiniques can enhance the efficiency and accuracy of traditional methods, it cannot yet replicate results obtained by them. In our study this was particularly due to limited data and visibility of the participants’ faces and gaze directions in the videos. These obstacles can be overcome by collecting data in even more controlled environments with using e.g. face-tracking cameras and employing ML methods tailored for dealing with smaller datasets.

Author information

Author	Role
Eveliina Mykkänen, Faculty of Social Sciences, Department of Psychology and Logopedics, Tampere University	Presenting author
Fareeda Mohammad, Faculty of Information Technology and Communication Sciences, Tampere University	Non-presenting author
Sari Peltonen, Faculty of Information Technology and Communication Sciences, Tampere University	Non-presenting author
Laura Perasto, FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Department of Clinical Medicine, University of Turku; Centre for Population Health Research, University of Turku and Turku University Hospital	Non-presenting author
Riikka Korja, FinnBrain Birth Cohort Study, Turku Brain and Mind Center, Department of Clinical Medicine, University of Turku; Department of Psychology, University of Turku	Non-presenting author
Jani Käpylä, Centre for Immersive Visual Technologies, Tampere University	Non-presenting author
Mervi Vänskä, Faculty of Social Sciences, Department of Psychology and Logopedics, Tampere University	Non-presenting author

⇦ Back to session