Ensemble Learning Predicts Multiple Sclerosis Disease Course in the SUMMIT Study

The rate of disability accumulation varies across multiple sclerosis (MS) patients. Machine learning techniques may offer more powerful means to predict disease course in MS patients.

Multiple Sclerosis (MS) affects approximately 1 million persons in the United States and is the number one cause of non-traumatic medical disability in young persons. An emerging paradigm is that MS patients experience varied disease trajectories with a significant proportion of patients experiencing mild disability accrual even 10-15 years after diagnosis, to early and severe disability accrual experienced by approximately 15-20% of patients [1, 2]. These different disease courses have taken on a variety of terms over the years including mild/benign [3, 4] on the low end of the spectrum, and malignant/severe/aggressive MS [1, 2] on the higher end of the spectrum [5].

 The early identification of patients who are more likely to accrue disability would allow clinicians to institute more rigorous MS treatments and management strategies. However, there are currently no predictive algorithms that identify MS patients at risk of severe disability.

In our research, we apply machine learning techniques to predict the disability level of MS patients measured by the expanded disability status score (EDSS) at the five-year mark using the first two years of longitudinal data. Our goal is to predict which patients will accumulate disability ("worsening") and which are likely to remain without disability accumulation ("non-worsening") in their disease course. We define "worsening" as an increase of 1.5 or more from baseline EDSS to the five-year EDSS and "non-worsening" as all other cases, based on the fact that an EDSS increase of 1.0 or 1.5 is clinically significant and generally sustained.

We found that two ensemble models, XGBoost [6] and LightGBM [7] were superior to the other four models evaluated in our study. Ensemble learning [8] is a family of algorithms that seek to create a "strong" classifier based on a group of "weak" classifiers, where "strong" and "weak" refer to how accurately the classifiers can predict the target variable. L1, L2, … Ln are independent learners trained on the entire training data D. The stacked generalized [9] is a logistic regression model trained to produce a final prediction P based on the decisions from individual classifiers. Model performance is measured using the final predictions.

An additional motivation for our research is to study risk factors affecting MS patients’ disease progressions. To this end, we ranked the top predictors in our models and identified the most predictive factors. Of variables evaluated, EDSS, Pyramidal Function, and Ambulatory Index were the top common predictors in forecasting the MS disease course.

As the most widespread disabling neurological condition of young adults around the world, MS is associated with lifelong challenges that are not only debilitating for those afflicted, but also represents an ever-rising social and economic burden in the US. For future work, we plan to assess whether recently identified biomarkers, genetic markers, clinical features, and medications as covariates will improve the predictability of our machine learning models. We hope that utilizing artificial intelligence paired with large comprehensive longitudinal MS datasets incorporating clinical, MRI, biological markers, and genetic data can form the basis for optimizing precision medicine approaches for treating this complex disease [10].

  1. Menon S, Shirani A, Zhao Y, Oger J, Traboulsee A, Freedman MS, Tremlett H. Characterising aggressive multiple sclerosis. J Neurol Neurosurg Psychiatry. 2013;84(11):1192-8. Epub 2013/06/08. doi: 10.1136/jnnp-2013-304951. PubMed PMID: 23744892.
  2. Gholipour T, Healy B, Baruch NF, Weiner HL, Chitnis T. Demographic and clinical characteristics of malignant multiple sclerosis. Neurology. 2011;76(23):1996-2001. Epub 2011/06/08. doi: 10.1212/WNL.0b013e31821e559d. PubMed PMID: 21646626.
  3. Sayao AL, Bueno AM, Devonshire V, Tremlett H, Neurologists UMC. The psychosocial and cognitive impact of longstanding 'benign' multiple sclerosis. Multiple sclerosis. 2011;17(11):1375-83. doi: 10.1177/1352458511410343. PubMed PMID: 21677023.
  4. Sayao AL, Devonshire V, Tremlett H. Longitudinal follow-up of "benign" multiple sclerosis at 20 years. Neurology. 2007;68(7):496-500. doi: 10.1212/01.wnl.0000253185.03943.66. PubMed PMID: 17296915.
  5. Roxburgh RH, Seaman SR, Masterman T, Hensiek AE, Sawcer SJ, Vukusic S, Achiti I, Confavreux C, Coustans M, le Page E, Edan G, McDonnell GV, Hawkins S, Trojano M, Liguori M, Cocco E, Marrosu MG, Tesser F, Leone MA, Weber A, Zipp F, Miterski B, Epplen JT, Oturai A, Sorensen PS, Celius EG, Lara NT, Montalban X, Villoslada P, Silva AM, Marta M, Leite I, Dubois B, Rubio J, Butzkueven H, Kilpatrick T, Mycko MP, Selmaj KW, Rio ME, Sa M, Salemi G, Savettieri G, Hillert J, Compston DA. Multiple Sclerosis Severity Score: using disability and disease duration to rate disease severity. Neurology. 2005;64(7):1144-51. doi: 10.1212/01.WNL.0000156155.19270.F8. PubMed PMID: 15824338.
  6. Chen, T. & Guestrin, Xgboost: A scalable tree boosting system. Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, (KDD 2016).
  7. Ke, et al. Lightgbm: A highly efficient gradient boosting decision tree. Proceedings of the 31st Conference on Neural Information Processing Systems, (NIPS 2017).
  8. Dietterich, G. Ensemble methods in machine learning. Proceedings of the International workshop on multiple classifier systems, (2000).
  9. Wolpert, H. Stacked generalization. Neural networks 5, 241-259 (1992).
  10. Chitnis T, Prat A. A roadmap to precision medicine for multiple sclerosis. Mult Scler. 2020;26(5):522-32. Epub 2020/01/23. doi: 10.1177/1352458519881558. PubMed PMID: 31965904