Artificial intelligence to detect malignant eyelid tumors from photographic images

This study developed an AI system, which employed a faster region-based convolutional neural network and deep learning classification networks, to automatically locate eyelid tumors and then distinguish malignant tumors from benign ones in photographic images captured by ordinary digital cameras.
Artificial intelligence to detect malignant eyelid tumors from photographic images

Eyelid tumors are the most common neoplasm encountered in daily ophthalmology practice1,2. Malignant eyelid tumors can invade adjacent structures (e.g., eyeballs, brain, and paranasal sinuses), which may cause vision loss, cosmetic disfigurement, and severe morbidity, and even pose a threat to life. Early detection and appropriate treatment can reduce the risk of cosmetic disfigurement and morbidity induced by these tumors. However, due to the relatively small size and variability in presentation, distinguishing malignant eyelid tumors from benign ones can be challenging for primary care physicians who often first evaluate eyelid tumor cases in clinics and for ophthalmologists without sufficient experience in eye cancer3,4. Besides, although over 200,000 ophthalmologists worldwide, there is a present and expected future shortfall in the number of ophthalmologists in both developing and developed countries5. The shortage of experienced ophthalmologists may hinder the early detection of malignant eyelid tumors, especially in underdeveloped countries and remote regions.

 To address this issue, we developed an artificial intelligence (AI) system that could automatically detect various types of malignant eyelid tumors from photographic images captured by ordinary digital cameras. Specifically, we first established an eyelid tumor detection system (ETDS) using the Faster-RCNN, an object detection network depending on region proposal algorithms6, to automatically locate and crop eyelid tumors from photographic images. This step can also remove the background noise around tumors in photographic images for better training the subsequent deep learning-based classification networks. The pipeline of the ETDS is described in Figure 1. In the second step, we developed a deep learning classification system using four state-of-the-art CNN architectures (DenseNet121, ResNet50, Inception-v3, and VGG16) to distinguish malignant eyelid tumors from benign ones based on the cropped images created by the ETDS.


 Figure 1. Diagram showing an overview of the proposed eyelid tumor detection system.

Our results showed that the average precision (AP) scores of the ETDS for locating eyelid tumors were 0.801 in the internal test set and 0.762 in the external test set. The representative detection results of the Faster-RCNN for eyelid tumors were shown in Figure 2.

Figure 2. Representative detection results of the Faster-RCNN for eyelid tumors.

The findings shown in Figure 3 demonstrated that deep learning classification algorithms performed well in discerning malignant eyelid tumors and the algorithm DenseNet121 had better performance than the other three algorithms. The generalizability of our system was confirmed on the basis of its good performance (AUC 0.899, sensitivity 91.5%, specificity 79.2%) in the external test set, of which images were collected from two other hospitals. When compared to the ophthalmologists of different levels, the system’s sensitivity was higher than that of the junior and senior ophthalmologists and comparable to that of the expert, while the system’s specificity is lower than that of the expert.

 Figure 3. Performance of four deep learning algorithms in discerning malignant eyelid tumors.

Due to the reliable performance of our system, it has the potential to be applied to ordinary digital cameras, which would be a convenient and cost-effective procedure for assisting medical practitioners and suspected patients to proactively track eyelid tumors and identify malignant ones earlier.

 Our team took over two years working on developing this system, wishing it could be utilized both at the screening stage before patients visit the physician and at the disease confirmation stage after the consultation, promoting the early detection and treatment of malignant eyelid tumors. For more details about this work, please refer to our paper ( published in npj Digital Medicine.


  1. Yu, S. S., Zhao, Y., Zhao, H., Lin, J. Y. & Tang, X. A retrospective study of 2228 cases with eyelid tumors. Int J Ophthalmol. 11, 1835-1841 (2018).
  2. Deprez, M. & Uffer, S. Clinicopathological features of eyelid skin tumors. A retrospective study of 5504 cases and review of literature. Am J Dermatopathol. 31, 256-262 (2009).
  3. Huang, Y. Y. et al. Comparison of the clinical characteristics and outcome of benign and malignant eyelid tumors: An analysis of 4521 eyelid tumors in a tertiary medical center. Biomed Res. Int. 2015, 453091 (2015).
  4. Leung, C., Johnson, D., Pang, R. & Kratky, V. Identifying predictive morphologic features of malignancy in eyelid lesions. Can. Fam. Physician. 61, e43-e49 (2015).
  5. Resnikoff, S., Felch, W., Gauthier, T. M. & Spivey, B. The number of ophthalmologists in practice and training worldwide: A growing gap despite more than 200,000 practitioners. Br J Ophthalmol. 96, 783-787 (2012).
  6. Ren, S., He, K., Girshick, R. & Sun, J. Faster R-CNN: Towards Real-Time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 39, 1137-1149 (2017).

Please sign in or register for FREE

If you are a registered user on Nature Portfolio Health Community, please sign in