PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging

Like Comment
Read the paper

The need for automated PE diagnosis models

Pulmonary Embolism (PE) is responsible for 180,000 deaths per year in the US alone. The gold standard diagnostic modality for PE is Computed Tomography Pulmonary Angiography (CTPA) which is interpreted by radiologists. Studies have shown that prompt diagnosis and treatment can greatly reduce morbidity and mortality, yet PE remains among the diagnoses most frequently missed or delayed. Therefore, there is an urgency in automating accurate interpretation and timely reporting of CTPA examinations to the immediate attention of physicians. 

The challenges in building automated PE diagnosis models

Although many studies have reported promising results in applying deep learning models to automate diagnosis in medical imaging, building deep learning models for PE classification on CTPA studies is significantly more challenging. For example, CTPA examinations are orders of magnitudes larger than most common medical imaging examinations (i.e., chest X-rays or head CT) and PE findings represent only a small fraction of the pixel data relative to the 3D CTPA volume. Further exacerbating this signal-to-noise problem is the extreme inter-image and interclass variance unique to CTPA studies caused by reliance on timing of intravenous contrast injection protocol and patient compliance with breath-holding instructions. Lastly, generalization across institutions particularly in the setting of varying CT scanner models and reconstruction methods present another difficult problem in generalization for automated diagnosis.

Our solution: PENet

To address this issue, we have curated CTPAs imaging examination from two separate institutions (Stanford and Intermountain) and developed an end-to-end deep learning model, PENet, capable of detecting PE. Some notable implementation details of PENet include 1) pretraining the model with a video dataset (Kinetics-600) for transfer learning and 2) using a sliding window of CT slices as inputs to increase the proportion of the target PE relative to the input. Our model also highlights regions in the original CT scans that contributed most to the model’s prediction, which can potentially help draw Radiologists’ attention to the most relevant parts of the imaging volume for more efficient and accurate diagnosis (see example). 

Axial contrast-enhanced CT pulmonary angiogram and Class Activation Map  inferred by PENet
CT pulmonary angiogram of a patient with acute pulmonary embolism. Left: CT-scan demonstrates a non-occlusive filling defect in the right posterior basal segmental artery. Right: Class Activation Map is centered on the acute pulmonary embolism.

Our results demonstrate robust and interpretable diagnoses including sustained cross-institutional performance on an external dataset. Thus, this work supports that successful application of deep learning to the diagnosis of a difficult radiologic finding such as PE on volumetric imaging in CTPA is possible, and can generalize on data from an external institution despite that the external institution used a different CT scanner and imaging protocol settings. Ultimately, clinical integration may aid in prioritizing positive studies by sorting CTPA studies for timely diagnosis of this important disease including in settings where radiological expertise is limited.

For more information, please see the full manuscript “PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging” at this link

Mars Huang

Ph.D. Student , Stanford University