Some considerations on digital health validation

Adoption of digital health technologies is hindered by lack of robust systems for validation. We respond to feedback on our previous framework article in this reply.

Like Comment
Read the paper

The motivation for designing a validation framework for digital health was inspired by the real world challenge of not being able to reliably give patients and colleagues recommendations for which health technologies to use. This limitation is due to difficulty in identifying quality in a crowded marketplace of solutions. In traditional clinical medicine, there are generally agreed upon conventions on what constitutes high quality evidence which are used to discriminate traditional devices and therapies. This clarity is what ultimately changes clinical practice. In the world of consumer goods there are also number of trusted entities that help convey quality through rigorous testing. Digital health represents an overlap of these two worlds where products are marketed directly to patients and physicians with clinical intent, but unfortunately without any meaningful oversight. Our approach was to develop a pragmatic and transparent scorecard to evaluate health technology to help all stakeholders in medicine identify the solutions that can make a difference. We designed our framework of evaluation along four principal domains: clinical validation; technical validation; usability; and cost. In addition, an evaluation of whether the technology satisfied its end user's functional requirements was also included. 

Our initial framework generated significant interest from different stakeholder groups. We welcomed the feedback and in this edition of npj Digital Medicine we respond to several points raised. Notably, we emphasize the flexibility of our approach to different technology types and suggest a stakeholder centric approach to adoption. We also acknowledge the importance of learning from the experience of low and middle income countries as well as the need to demonstrate tangible value which is currently underway. Ultimately, digital health needs improved validation to facilitate widespread adoption. We believe our framework is one practical approach to meeting this goal and look forward to sharing the results of our pilot study in the future.

Simon C. Mathews

Assistant Professor of Medicine, Johns Hopkins Medicine