Early detection of autism provides an opportunity for early intervention, which can improve developmental trajectories and strengthen social, language, cognitive, and motor competencies during a period of heightened brain plasticity. Motor impairments are often one of the earliest reported signs associated with autism, and thus early assessment of motor skills could be an important component of a screening battery for autism.

Studies have found that autistic people often struggle with tasks that involve visual-motor integration, which can have an impact on the development of social skills. This study analyzed the use of a tablet-based bubble-popping game to assess such early visual-motor skills, as it requires the coordination of a dynamic visual stimulus with a motor response involving touch. 

The development of miniaturized inertial sensors and wearable sensors, and the ubiquity of mobile devices such as tablets and smartphones have allowed unprecedented access to massive multimodal data that have been used to characterize motor behavior. These data provide promising ways to identify and quantify an autism motor signature and characterize the nature of motor impairments in autism.

The bubble-popping game developed by our team, co-led by Guillermo Sapiro and me, is one part of a mobile application (app) - SenseToKnow - that is designed to detect early signs of autism. The app displays developmentally appropriate and strategically designed movies on a smartphone or tablet while the camera in the device records the child’s behavioral responses to the stimuli. In previously published work with the app, we showed that computer vision analysis and machine learning can be used to automatically quantify a wide range of autism signs, including differences in gaze, attention, facial expressions and dynamics, and head movements. In this study, we hypothesized that autistic children would have a distinct performance on the bubble-popping game compared to neurotypical children. We also examined whether having co-occurring ADHD affected motor performance. Finally, we examined whether motor digital phenotypes are correlated with standardized measures of cognitive, language, and motor abilities, as well as the level of autism-related behaviors. Sam Perochon led the analysis of the data for this study. 

The bubble-popping game was delivered at a clinic following a well-child visit with a pediatrician. The game was presented on an iPad placed on a tripod around 50 cm from the participant. The game is composed of 5 vertical tracks with bubbles appearing from the bottom and moving upwards. Any time a bubble is touched, the bubble pops, making a distinct popping sound releasing a cartoon animal character inside the bubble. When the bubble is popped, it appears again (same cartoon character and color) from the bottom of the same lane, otherwise, a random one appears after the bubble exits the screen from the top. The repeated bubble is designed to assess the tendency to return to the same bubble (repetitive behavior) versus exploring other bubbles.

Caregivers were asked to hold their child on their lap, and the child was encouraged to pop the bubbles independently before the analyzed data was recorded for 20 seconds. Using the touch data collected and the tablet kinetic information provided by the device sensors, we computed a set of features representing the participant’s motor behavior.  For example, we computed the number of touches, the ratio of popped bubbles over the total number of touches, the touch error variation as the standard deviation of the distance between the child's finger position when hitting the screen and the center of the closest bubble, and the average estimated force applied on the screen when touching it.

Our study showed that toddlers as young as 18 months old and children up to 10 years old found the game engaging. We observed correlations between age and several motor variables, such as the number of touches, bubble popping rate, median distance to the center, average touch duration, and average touch length, suggesting that these features could be used to assess children's visual motor skills development.

In a younger sample, comprised of 151 children between 18 and 36 months of age, of which 23 were diagnosed with autism, autistic children popped the bubbles at a lower rate despite an equal number of touches, and their ability to touch the center of the bubble was less accurate. When they popped a bubble, their finger lingered for a longer period, and they showed more variability in their performance.

An older sample was also studied and was comprised of 82 children between 36 and 120 months of age, of which 63 were diagnosed with autism, 32 of whom had co-occurring attention-deficit/hyperactivity disorder (ADHD), and 19 were neurotypical. Compared to neurotypical children, we found that autistic children spent a longer duration on a targeted bubble rather than moving quickly from one bubble to another. Consistent with previous research, the presence of co-occurring ADHD was associated with lower visual motor skills. We found that autistic children with ADHD had lower accuracy, lower number of pops despite an equal number of touches, higher number of touches per target, and overall, more variability in their motor behavior.

We hypothesized that combining multiple features would improve the discrimination of (i) autistic and neurotypical toddlers and of (ii) autistic children with or without co-occurring ADHD, respectively. To this end, we trained logistic regression models to infer from the touch-based features the participant’s clinical diagnosis and performed leave-one-out cross-validation to assess the generalization performances of these models. The resulting areas under the Receiving Operating Characteristic (ROC) curves were AUC=.73 [95% CI 0.63-0.83], and AUC=.74 [0.62-0.86], respectively, suggesting that the use of a combination of the extracted motor features improved diagnostic group discrimination.

This work is an important step toward characterizing early motor abilities in autistic children. Additional research is needed to understand how co-occurring psychiatric conditions further influence motor abilities.

Future work includes evaluating the features proposed here in combination with the other autism-related behaviors measured using the SenseToKnow app. We are working toward a multi-modal solution that objectively describes the rich and diverse realm of developmental variation precisely and quantitatively. We believe that the availability of objective, quantitative, and scalable digital phenotyping tools will greatly improve the field of behavioral assessment. 

Perochon, S., Di Martino, J.M., Carpenter, K.L.H., Compton, S., Davis, N., Espinosa, E., Franz, L., Rieder, A., Sullivan, C., Sapiro, G. and Dawson, G. A tablet-based game for the assessment of visual motor skills in autistic children. npj Digital Medicine. 2023;6,17.

🧠 Link to gallery