State of the art deep learning methods such as convolutional neural networks (CNNs) can accurately classify patient disease status from imaging data. Researchers at Google recently applied such an algorithm to detect diabetic retinopathy from retinal fundus images with high sensitivity and specificity. However, training these algorithms requires mass amounts of high-quality images that are difficult to obtain and often unavailable for rare diseases. To address this issue, we assessed whether classification accuracy on a given number of images could be improved by performing a separate image segmentation task on a subset of data prior to classifier training. Diabetic retinopathy is a leading cause of blindness in the U.S. that presents with retinal lesions indicative of severity of disease. Using a fully convolutional neural network (FCN), we segmented ophthalmologist-labeled retinal fundus photographs to locate microaneurysms (MAs) and neovascularization (NV), two types of lesions seen in diabetic retinopathy. We then used the trained FCN to segment, or predict lesion locations, for images in the classification training, validation and test sets. Final classification was run on these predicted segmentations. In preliminary tests, segmentation-assisted classification achieved approximately 8% and 5% greater accuracy on average for MAs and NV respectively, compared to classification with CNN alone. Although further work is needed to identify parameters that optimize improvement, these results show that segmentation-assisted classification holds promise as a method for increasing conventional CNN classification accuracy. If applied to rare diseases, it has the potential to help train deep learning classifiers with smaller data requirements even in settings of limited data availability.



Author: Emma Zhao

Coauthor(s): Carson Lam Darvin Yi Daniel Rubin

Status: Work In Progress