For patients with chronic neurological conditions, having access to trained neurologists who can properly diagnose and manage their conditions is incredibly important to patient outcomes. However, trained neurologists are becoming increasingly short in supply¹. As the US population becomes older, which comes with a higher burden of chronic neurological conditions, the gap of supply and demand widens even further. These factors shorten the amount of time that neurologists and trained clinicians have to see patients, causing providers to perform non-comprehensive neurological exams, leading them to miss patient’s deficits, draw faulty conclusions on disease severity, and prescribe the patient the wrong course of treatment.

Here at the NIH, we are fortunate to be buffered from the issue of having insufficient time for performing neurological exams. All MS patients in our study receive a comprehensive, 40 to 60 minutes neurological exam, at every visit. We understand that this is a privilege not provided to many patients who see clinicians in outpatient settings, rural areas, and countries with an even more limited number of neurologists.

To address this gap in neurological care, we created the Neurological Functional Test Suite (NeuFun), an Android app that captures the entire neurological exam through patient-autonomous mobile games. When patients use the NeuFun from home, the game results can be streamed to clinicians and help them easily identify the deficits that patients experience. During patient visits, clinicians can focus their exams on the deficits identified by NeuFun, which would help them perform quality exams in spite of the time pressure in their care settings.

Prior to implementing NeuFun for clinicians and patients everywhere, we must show that each game in the app does measure the right neurological functions. We demonstrate this validation process on the NeuFun version of the Symbol-Digit Modalities Test (SDMT) in our paper².

In the SDMT, patients are asked to match a series of symbols with their corresponding digits under a time constraint. The SDMT is classically administered as an oral exam, where patients are shown the symbols and then say aloud the digits matching those symbols. Taking the test as an oral exam, instead of a written one, is meant to prevent motoric disabilities from affecting the measurement of cognitive functions.

We developed an oral version in the NeuFun with voice recognition to emulate this usual method of administering the SDMT. The order of symbols being shown and the symbol-digit key are randomized with every trial to discourage memorization of the test. Initial training with patients showed us that voice recognition largely failed to pick up patients’ answers accurately. Additionally, most patients in our study preferred to use a keypad for inputting the correct answer. These observations led us to use the keypad version and analyze its results for the remainder of the study.  

After accounting for the influence of motoric disabilities, we showed that NeuFun SDMT scores are associated with brain lesions and atrophy – signs of slowing cognitive processes – at levels comparable to existing oral SDMT literature³. Looking at the longitudinal data from patients who use NeuFun SDMT from home, we observed that patients have artificial score improvements due to repeated testing, also known as practice effects. The duration of the practice effects can be mathematically identified for each patient, which allows clinicians to know when to start tracking the NeuFun SDMT results for true cognitive changes.

Our process for analyzing the utility of the NeuFun SDMT demonstrates that digital health applications must always be developed with and validated against clinical knowledge. We have previously applied this framework towards the analyses of other tests^4,5 in the NeuFun that can measure hand mobility, coordination, and proprioception. We will continue to validate the remaining tests within NeuFun using this framework and hope to make neurological care more accessible through our refinement of this application.