Automatic Detection and Classification of Diabetic Retinopathy

using Retinal Using Image Processing by Smartphone

Preprocessing
All images were converted to a hierarchical data format for preprocessing, data augmentation,
and training. Preprocessing involved several steps: images were cropped using Otsu’s method
to isolate the circular colored image of the retina. Images were normalized by subtracting the
minimum pixel intensity from each channel and dividing by the mean pixel intensity to represent
pixels in the range 0 to 1. Contrast adjustment was performed using the contrast limited
adaptive histogram equalization (CLAHE) filtering algorithm

Data Augmentation
We augmented the number of images in real-time to improve network localization capability
and reduce overfitting. During each epoch, a random augmentation of images that preserve
collinearity and distance ratios was performed. We implemented random padding with zeros,
zoom, rolling and rotation. These affine transformations are particularly effective when applied
to disease class R1 which are the most difficult to grade and fewest in number.

Training and Testing Models

A Deep Learning GPU Training System (DIGITS) with prebuilt convolutional neural networks for
image classification facilitated data management, model prototyping and real-time performance
monitoring. DIGITS is an interactive system and was first used to build a classification dataset
by splitting the Messidor and MildDR fundus folder into training and validation subsets of 1077
and 269 images respectively. The images were cropped to area size 256x256 and used as input
data by Imagenet models previously trained for generic classification tasks. The test subset
folder contained 400 images from the Lariboisiere hospital Messidor partition and was disjoint
from training data. This training system, which offered extensive hyperparameter selections,
was then used to build model prototypes over 100 epochs requiring approximately 20 minutes
each to complete. A Tesla K80 GPU hardware device powered the training and a form of early
stopping determined the optimal test set model epoch. Advanced visualization monitoring and
confusion matrix statistical analysis provided important insights. The knowledge gained guided
construction of more complex model architectures finely tuned for improved interpretation of
our datasets see Appendix.
Digital image processing improves sensitivity for mild class detection
The dataset contained images from a disparate patient population with extremely varied levels
of lighting in the fundus photography. The lighting affects pixel intensity values within the
images and creates unnecessary variation unrelated to classification levels. A contrast limited
adaptive histogram equalization filtering algorithm, using the OpenCV (http://opencv.org/)
package was applied to address this artifact. Results from this preprocessing step are visually
depicted in Fig. 2. We discovered that 3-ary classifier sensitivity for the mild case increased
from 0 to 29.4%, while this measure was approximately the same for the remaining two
classes Fig. 3. Our digital image preprocessing technique enabled improved detection of
pinpoint subtle features and microaneurysms via convolutional filters, which were previously
imperceptible by the CNN. We hypothesize this change is attributable to the channel wise
contrast enhancing effect of histogram equalization.

Multi-class training sensitivities is highly dependent on dataset fidelity

However, when we trained 3-ary and 4-ary classifiers with a GoogLeNet model on the Kaggle
dataset, we were unable to achieve significant sensitivity levels for the mild class. As shown in
the confusion matrix (Table 2), the sensitivity of the no DR and severe DR classes were 98%
and 93% respectively; however the sensitivity for the mild class was only 7%. Thus, we find
that our performance is limited by the inability of CNNs to detect very subtle features. We
hypothesize that this can be explained by the considerable noise and camera artifacts as well as
poor fidelity labeling (misleading or incorrect) of the Kaggle images.

Transfer learning as a parallel means of exploring optimal CNN models

Our previous models relied on a Tensorflow4 implementation without fixed weights. The
practicality of transfer learning was investigated by using a baseline prototype consisting of
pretrained GoogLeNet model obtained from the ImageNet visual object recognition database.
The prototype was trained on the Messidor dataset for 30 epochs using stochastic gradient
descent optimization with step decay learning rate initialized at 0.002. The classification model
validation achieved 66.03% as the best accuracy.
Introducing loss (and accuracy layers) to intermediate representations of the deep network
allowed for faster propagation during training and avoided the vanishing gradients issue. We
found that training converged much faster using the preprocessed images—in a matter of 25
epochs for the transfer learning scenario rather than 90 epochs when training on the raw data.
References
[1] http://cs231n.github.io/convolutional-networks/
[2] http://cs231n.github.io/neural-networks-1/
[3] http://cs231n.github.io/neural-networks-2/
[4] https://github.com/yidarvin/firstaid.
[5] Abràmoff M. D., Reinhardt J. M., Russell S. R., Folk J. C., Mahajan V. B., Niemeijer M.,
Quellec G. Automated early detection of diabetic
retinopathy. Ophthalmology, 2010;117(6):1147–1154.[PMC free article] [PubMed] [Google
Scholar]
[6] Antal B., Hajdu A. An ensemble-based system for microaneurysm detection and diabetic
retinopathy grading. IEEE transactions on biomedical engineering, 2012;59(6):1720–
1726. [PubMed] [Google Scholar]
[7] Chan J. C., Malik V., Jia W., Kadowaki T., Yajnik C. S., Yoon K.-H., Hu F. B. Diabetes in
asia: epidemiology, risk factors, and pathophysiology. JAMA, 2009;301(20):2129–
2140. [PubMed] [Google Scholar]
[8] Congdon N. G., Friedman D. S., Lietman T. Important causes of visual impairment in the
world today. Jama, 2003;290(15):2057–2060. [PubMed] [Google Scholar]
[9] Decenciere E., Zhang X., Cazuguel G., Lay B., Cochener B., Trone C., Gain P., Ordonez R.,
Massin P., Erginay A., et al. Feedback on a publicly distributed image database: the messidor
database. 2014;33:231–234. [Google Scholar]
[10] Gardner G., Keating D., Williamson T., Elliott A. Automatic detection of diabetic
retinopathy using an artificial neural network: a screening tool. British journal of
Ophthalmology. 1996;80(11):940–944.[PMC free article] [PubMed] [Google Scholar]
[11] Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep
learning. Elsevier. 2017 [PubMed] [Google Scholar]
[12] Goh J. K. H, Cheung C. Y., Sim S. S., Tan P. C., Tan G. S. W, Wong T. Y. Retinal imaging
techniques for diabetic retinopathy screening. Journal of diabetes science and
technology, 2016;10(2):282–294.[PMC free article] [PubMed] [Google Scholar]
[13] Graham B. Kaggle diabetic retinopathy detection competition report. 2015 [Google Scholar]
[14] Gulshan V., Peng L., Coram M., Stumpe M. C., Wu D., Narayanaswamy A., Venugopalan
S., Widner K., Madams T., Cuadros J., et al. Development and validation of a deep learning
algorithm for detection of diabetic retinopathy in retinal fundus
photographs. JAMA, 2016;316(22):2402–2410. [PubMed] [Google Scholar]
[15] Huang L. C., Yu C., Kleinman R., Smith R., Shields R., Yi D., Lam C., Rubin D. Opening
the black box: Visualization of deep neural network for detection of disease in retinal fundus
photographs. The Association for Research in Vision and Ophthalmology. 2017 [Google
Scholar]
[16] Mookiah M., Acharya U., Chua C., Lim C., Ng E., Laude A. Computer-aided diagnosis of
diabetic retinopathy: A review. In Computers in Biology and Medicine. 2013:2136–
2155. [PubMed] [Google Scholar]
[17] Niemeijer M., Van Ginneken B., Cree M. J., Mizutani A., Quellec G., Sanchez C. I., Zhang
B., Hornero R., Lamard M., Muramatsu C., et al. Retinopathy online challenge: automatic
detection of microaneurysms in digital color fundus photographs. IEEE transactions on medical
imaging, 2010;29(1):185–195. [PubMed] [Google Scholar]
[18] Oquab M., Bottou L., Laptev I., Sivic J. Learning and transferring mid-level image
representations using convolutional neural networks. In Proceedings of the IEEE conference on
computer vision and pattern recognition, 2014:1717–1724. [Google Scholar]
[19] Philip S., Fleming A., Goatman K., Mcnamee P., Scotland G. The efficacy of automated
disease/no disease grading for diabetic retinopathy in a systematic screening programme. In
British Journal of Ophthalmology, 2007:1512–1517. [PMC free article] [PubMed] [Google
Scholar]
[20] Quellec G., Lamard M., Josselin P. M., Cazuguel G., Cochener B., Roux C. Optimal
wavelet transform for the detection of microaneurysms in retina photographs. IEEE Transactions
on Medical Imaging, 2008;27(9):1230–1241. [PMC free article] [PubMed] [Google Scholar]
[21] Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the inception
architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, 2016:2818–2826. [Google Scholar]
[22] UR A. Decision support system for diabetic retinopathy using discrete wavelet
transform. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of
Engineering in Medicine, 2013;227(3):251–261. [PubMed] [Google Scholar]
[23] Wang S., Tang H. L., Hu Y., Sanei S., Saleh G. M., Peto T., et al. Localizing
microaneurysms in fundus images through singular spectrum analysis. IEEE Transactions on
Biomedical Engineering, 2017;64(5):990–1002. [PubMed] [Google Scholar]
[24] WHO. Global report on diabetes. 2016.
[25] Xu Y., Xiao T., Zhang J., Yang K., Zhang Z. Scale-invariant convolutional neural
networks. arXiv preprint arXiv:1411.6369, 2014 [Google Scholar]
[26] Yosinski J., Clune J., Nguyen A., Fuchs T., Lipson H. Understanding neural networks
through deep visualization. arXiv preprint arXiv:1506.06579, 2015 [Google Scholar]