58 American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) ISSN (Print) 2313-4410, ISSN (Online) 2313-4402 © Global Society of Scientific Research and Researchers http://asrjetsjournal.org/ Recent Applications of Deep Learning Algorithms in Medical Image Analysis Arman Sarraf a *, Ali Esmaeilnia Jalali b , Javad Ghaffari c a,b,c Department of Electrical and Computer Engineering Islamic Azad University North Tehran Branch a Email: armanhs13@gmail.com Abstract Advances in deep learning have enabled researchers in the field of medical imaging to employ such techniques for various applications, including early diagnosis of different diseases. Deep learning techniques such as convolutional neural networks offer the capability of extracting invariant features from images that can improve the performance of most predictive models in medical and diagnostic imaging. This work concentrates on reviewing deep learning architectures along with medical imaging modalities where the crucial applications of such algorithms, including image classification and segmentation, are discussed. Also, brain imaging as a branch of medical imaging which allows scientists to explore the structure and function of the brain is explored, and the applications of deep learning to early diagnose Alzheimer‟s Disease, and Autism as the most critical brain disorders are studied. Moreover, the recent research findings revealed that employing deep learning-based semantic segmentation techniques could significantly improve the accuracy of models developed for brain tumor detection. Such advances in early diagnosis of disorders and tumors encourage medical imaging practitioners to implement software applications assisting them to improve their decision-making process. Keywords: Deep Learning; Convolutional Neural Network; Image Classification; Medical Science. 1. Introduction Deep learning, which is currently one of the most essential and practical machine learning techniques, has the ability to learn to detect and classify objects by extracting features from input samples. It should be mentioned that the human brain pattern first inspired the creation and invention of deep neural networks. ------------------------------------------------------------------------ * Corresponding author. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 72, No 1, pp 58-66 59 Deep learning also has shown high-level performance in various fields, and applications such as Image Classification, Object Detection, Video Processing, Natural Language Processing, Speech Recognition, therefore, for improving the accuracy rate and pace in these fields, variety of architectures have been developed to reduce the processing problems [1]. Convolutional neural network (CNN), which is famous for its powerfulness, has been the most successful and efficient algorithm was introduced in deep learning science. It consists of a large number of hidden layers such as a Conv layer, pooling layer, and a fully-connected layer, which each one composes of variants of neural neurons having the ability to recognize and classifying particular features from an input. Its architecture is composed of a considerable number of convolutional cores to reduce the size of the input and highlight the critical features making the system able to extract the exact needed information in higher layers [2]. The image classification task is reputable for its intensity and the high-level ability for predicting the sample‟s class. This technique works via trainable layers of the network which firstly, all the layers (including conv, pooling, fully connected) are trained by the training samples gathered for the specific purposes, then after, through passing the testing samples through the layers, the CNN model predicts and distinguishes each one into predefined classes [3]. Besides image classification, also semantic segmentation contributes a vital role in the deep learning field. The main aim of segmentation is locating the exact location and counting the existing number of objects in one specific sample rather than classifying them into different classes. Bearing in mind, determining the exact and accurate dimensions of any object is an elaborate task and requires a large number of advanced algorithms. However, scientists have designed and introduced a sophisticated and powerful network named “Fast R-CNN” for addressing and facilitating this problem [4]. As mentioned earlier, beside processing and classifying images, deep learning algorithms have shown a satisfactory and low error-rate performance in video processing, natural language processing, and the fields related to sequence data. Due to the differences in processing sequence data except for standard ones, models with different algorithms and architectures are required with the ability to maintain and passing the continuing samples through the layers without any interruption. Fortunately, several high-level deep learning models such as Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), are introduced to accommodate such these needs [5]. Forasmuch as training the models is through collected samples, detecting the best dataset is one of the essential parts of each procedure [6]. For instance, there are different image modalities with different approaches that can be used in medical imaging. CT scan images have a proper resolution and high efficiency, capturing the specific area of a body but emits X-Radiation which is not the best option for the patients. PET scan is a powerful imaging technique that uses nuclear substances for visualizing and measuring metabolic processes usually used for diagnosing cancers, which for its radioactive radiation, is considered a dangerous method. Magnetic resonance imaging (MRI) is known as the most straightforward technique, uses strong magnetic fields, gradients, and radio waves to generate images of the different organs in the body and is the most common method considerably used for researches and medical purposes [7, 8]. Therefore, achieving high performance and accuracy from using deep learning techniques in medical science, especially for diagnosing brain disorders such as Alzheimer‟s, Autism, and brain stroke, requires having a bright and high-resolution brain imaging samples [9]. Concluding, from imaging techniques mentioned above, researches has shown MRI and fMRI samples are the most efficient and useful data is commonly considered and used for training deep learning models and diagnosing brain diseases [10]. Also, it should be noted that PET scan brain images have also been used in some different researches and medical American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 72, No 1, pp 58-66 60 purposes, which could bring a satisfactory result. 2. Deep learning-based semantic segmentation Semantic segmentation is known as a process for distributing an image into variants of pixels due to the aim of making the sample more understandable and easier for analyzing purposes. Bearing in mind that segmentation is significantly different from image classification tasks. The main difference is in semantic segmentation, each specific pixel in the sample attributes to a class, although during the image classification process, the whole sample is assigned to a class. Deep Learning, which is considered as a branch of Artificial neural network and has the intensity to be trained by the variety of different data samples, showed a very satisfactory efficiency in image classification, object detection, localization, which in some cases the error rate was less than human being miscalculation. Today‟s advanced deep learning algorithms have the ability to facilitate semantic segmentation tasks. There are entirely four main prominent models of deep learning named Stacked Auto Encoder (SAE), Deep Belief Network (DBN), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) [11]. Due to the success and high ability of the CNN model, it has been significantly valuable for researchers and variants of architectures have been introduced to improve the performance and efficiency for extracting features. With the help of convolutional neural networks, semantic segmentation has made great strides. Today, VGGnet and Resnet are regarded as the best architectures with the highest efficiency for CNNs. In brief, VGG architecture is consisting of thirteen Conv layers; each is followed by a max-pooling layer and three FC layers. And Resnet, although it is more many times deeper than other models, has less computational complexity, which increases its accuracy and pace [12]. Researchers have discovered by determining the convolutional layer rather than fully connected layer, the performance and accuracy of semantic segmentation are substantially increased. Therefore, by transforming mentioned famous models to Fully Convolutional Network (FCN), we can accomplish higher and better performance in results. The reason behind semantic segmentation‟s complexity and needing more advanced models‟ attributes to processing three difficult tasks simultaneously. Firstly, Image classification, secondly, objects localization, and at the end, semantic segmentation. A process of segmentation is shown in the figures below as an instance. Figure 1: Classification Figure 2: Localization American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 72, No 1, pp 58-66 61 Figure 3: Semantic Segmentation Figure 4: Full Instance The intensity and efficiency of the fusion of deep learning and semantic segmentation raised the expectation of using these techniques for medical purposes and helped to diagnose diseases, which were approximately a problematic and time-consuming task for human beings [13]. Researches have shown that semantic segmentation could achieve a convincing accuracy and authenticity efficiency invariant fields in medical science, including diagnosing and predicting stroke, tumor, Alzheimer‟s disease. Cognition and diagnosing procedures for the two most essential diseases, „stroke‟ and „tumor‟ is discussed below [14]. It is a time consuming, and tedious task and also might be with a high error-rate to localize the exact location of infarcts manually. Therefore, today‟s advanced segmentation methods are spontaneously able to be trained for diagnosing and recognizing the exact location of any stroke. Researches have shown that the most efficient model which can understand and learn the features from 2D MRI images is a fully convolutional Resnet network (Res-FCN). Therefore, after training the model with a large-scale dataset of MRI clinical images, the model reached a high performance that presented a very low false negatives equals to a mean number of 1.515 per subject, which is regarded as a satisfactory result as misdiagnosing [15]. Due to the differences in size, location, contrast, and shape of each lesion, it is roughly a difficult task to detect the exact diagnosis of any tumor. However, today‟s sophisticated deep learning algorithms have also shown acceptable results for detecting any tumor, whether benign, premalignant, or malignant. By improving fully convolutional networks, scientists have designed a 3D-semantic segmentation deep-learning-based algorithm for diagnosing and treatment plans, especially for brain and lung tumors. The end to end designed algorithm with encoder-decoder is trained with PET an CT scan samples individually with a fully automatic approach. A graph cut based- segmentation is also implemented for facilitating the progress [16]. 3. Brain disorder diagnosis 3.1 Alzheimer’s and MCI Today‟s the two most important and common diseases between usually older adults are Alzheimer‟s and Mild Cognitive Impairment (MCI). Alzheimer‟s disease is an irretrievable progressive brain disorder that causes brain cells to stop working and gradually die. This disease, which mainly appears in the middle-60s, advances steadily that leads to memory loss, anger, making patients unable to do the simplest human tasks [17]. Alzheimer‟s has seven different stages in which each higher stage exhibits more symptoms in both the person and their MRI American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 72, No 1, pp 58-66 62 image, and the only way for diagnosing is through MRI /fMRI/PET scans. Mild Cognitive Impairment (MCI) is known as a similar disease to Alzheimer‟s, which the patient starts being led to dementia. Considering, on rare occasions, MCI leads to Alzheimer‟s, and the stages start from losing memory, debilitating verbal ability, and in high stages, misjudgment adds to mentioned problems. However, patients usually can survive and live longer than those who are diagnosed with having Alzheimer‟s [18]. 3.2 Autism Another insidious and painful disease for a human being is Autism (ASD). Same as the two mentioned diseases earlier, Autism also impacts on brain cells; however, the symptoms are shown from mostly third year except ripe ages. Autism is combined with environmental and genetic factors, which causes a heavy toll on social communication, repetitive use of language, and inherent tendency with unusual objects. Although, a groundbreaking research has shown there is not any specific pattern of getting better or worse by becoming older. Because no medical or blood test could show the disease, ASD needs to be diagnosed by MRI images and brain scans, which with today‟s advanced deep learning algorithm, is not a complicated task [19]. 4. Deep Learning in detecting Alzheimer’s disease As mentioned, Alzheimer‟s disease ultimately causes brain cells to die, and its procedure impacts on brain patterns. Therefore, it might seem to diagnose Alzheimer‟s from MRI, and fMRI images are utterly an uncomplicated task; however, due to the resemblance of an adult‟s brain pattern and patient‟s brain pattern, it is profoundly an elaborate task. Hierarchical deep learning, which was first inspired by the human brain, is based on complex algorithms and heavy processing, having the ability to extract high-level features from samples [20]. With today‟s advanced GPUs and variety of cutting-edge deep learning algorithms, predicting and diagnosing lots of diseases is approximately achievable, and also variants of researches have shown the beneficial performance of deep learning implementation in Medical Science. Diagnosing Alzheimer‟s is through advanced trainable convolutional neural networks (CNN). One of the essential parts of training each neural network is data. Due to the lack of MRI samples with Alzheimer‟s brain patterns, fortunately, a few big data platforms have been created, including 3D and 4D MRI and fMRI images for medical purposes to facilitate the process [21]. However, choosing the most efficient CNN model incredibly causes positive or negative effects on accuracy and results. Figure 5: LeNet architecture consisting of 2 Conv layer, two pool layer, and two fully connected layer, and the output is in binary form American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 72, No 1, pp 58-66 63 Researches have shown that LeNet and GoogleNet architecture are the two most successful model for predicting and diagnosing this disease. Scientists, by training a LeNet and a GoogleNet, which included dimensional reduction, through MRI samples, could accomplish diagnosing Alzheimer‟s with 99.9% accuracy via the LeNet model and almost 100% accuracy via GoogleNet. The figures below demonstrate LeNet and GoogleNet architecture used in their research briefly [22]. Figure 6: GoogleNet architecture with dimensional reduction Figure 7: After training the model by MCI, AD, and Normal Control brain scan so-called NC MRI samples, results showed a satisfactory accuracy rate of classifying test samples into three categories. The blue frame in the left refers to NC brain samples. The green frame in the middle shows the MCI brain patterns. And the red frame in the right shows the AD brain samples. As mentioned before, each cognitive brain impairment has different stages, and due to the importance of the exact stage for each patient, predicting and diagnosing the disease doesn‟t restrict to just classifying the sample as a normal control or diagnosed. Although scientists could hardly achieve diagnosing Alzheimer‟s in a few past years, which was a significant transformation and caused starting a new era for medical science, they are American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 72, No 1, pp 58-66 64 ambitiously interested in designing convolutional network models being able to predict the exact stages, and also develop therapies to slow down its process. One of the most successful convolutional neural network was designed for classifying the exact stage of Alzheimer‟s disease (AD), and mild cognitive impairment (MCI) simultaneously is MCADNNet. This deep learning algorithm is based on an optimized convolutional neural network, including a multi-classification in the last layer applying a softmax layer. Also, for improving accuracy, a decision-making algorithm was designed to simplify the output of the model [23]. Ultimately the scientists who designed and developed MCADNNet could respectively achieve 99.77% accuracy for classifying MCI, AD, and normally aging brain samples in adults over the age of 75 years [24]. Figure 6 demonstrates the function of MCADNNet. 5. Deep Learning in detecting Autism disease In the beginning, it is essential to mention the human brain, and its function has been a mystery for all scientific fields. There are approximately more than 100 billion neurons with more than 100 trillion connections in the brain of any human being and although, with the help of today‟s sophisticated computers and deep learning algorithms, scientists are not undoubtedly able to diagnose or prevent brain diseases like Autism Spectrum Disorder (ADS), Alzheimer‟s (AD) beforehand. However, a variety of researches are going on investigating to find the best methods for predicting and diagnosing such these diseases. Fortunately, scientists in new research explored one of the most significant deep learning models known as LeNet-5 could accomplish a very high accuracy for classifying Autism brain MRI samples from normal controls. In brief, LeNet composed of 5 layers, accepting gray-scale images with a size of 32 * 32 * 1 as input which is transferred to two convolutional layers each following with one subsampling layer, and at the end, a fully connected layer classifies the sample into existing classes [25]. Therefore, after running and training the LeNet-5 by both Autism (ADS) and normal control (NC), brain MRI samples on the NVIDIA GPU training system, the accuracy rate reached a peak of about 99.99% which certainly was a satisfactory achievement [19]. A simple LeNet-5 architecture is illustrated in figure 7. Figure 8: A LeNet model, which is one of the most straightforward CNN architectures composed of 5 layers considering conv and pooling and then, a fully connected layer American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 72, No 1, pp 58-66 65 6. Conclusion This work summarized the recent applications of deep learning algorithms in medical imaging, where early diagnosis of brain disorders and tumors were discussed. The invariant feature extraction capability of deep learning techniques, especially CNN models, could significantly improve the performance of medical imaging classification and segmentation models. Such importance was achieved by using convolutional operators in deep learning architectures as an invariant feature extractor, which led the scientist to develop models with a performance close to 99% such as DeepAD and MCADNNet. Also, recent advances in early diagnosis of Autism using deep learning techniques have opened new avenues for researchers to discover new biomarkers to trace structure and functional changes in the brain. To conclude, medical image analysis using deep learning has advanced over the past decade, which encourages to productize such techniques rather than research applications. References [1]. Sarraf, S., Binary Image Segmentation Using Classification Methods: Support Vector Machines, Artificial Neural Networks and K th Nearest Neighbours. International Journal of Computer (IJC), 2017. 24(1): p. 56-79. [2]. Lawrence, S., et al., Face recognition: A convolutional neural-network approach. IEEE transactions on neural networks, 1997. 8(1): p. 98-113. [3]. Sarraf, S., Hair color classification in face recognition using machine learning algorithms. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS), 2016. 26(3): p. 317-334. [4]. Wang, X., A. Shrivastava, and A. Gupta. A-fast-rcnn: Hard positive generation via adversary for object detection. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. [5]. Sarraf, S., French Word Recognition through a Quick Survey on Recurrent Neural Networks Using Long-Short Term Memory RNN-LSTM. arXiv preprint arXiv:1804.03683, 2018. [6]. Sarraf, S., EEG-based movement imagery classification using machine learning techniques and Welch‟s power spectral density estimation. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS), 2017. 33(1): p. 124-145. [7]. Sarraf, S. and J. Sun, Advances in functional brain imaging: a comprehensive survey for engineers and physical scientists. International Journal of Advanced Research, 2016. 4(8): p. 640-660. [8]. Strother, S.C., S. Sarraf, and C. Grady. A hierarchy of cognitive brain networks revealed by multivariate performance metrics. in 2014 48th Asilomar Conference on Signals, Systems and Computers. 2014. IEEE. [9]. Sarraf, S., et al., Resting-State fMRI Data Classification of Exercise-Induced Brain Changes in Healthy Subjects Using Probabilistic Independent Component Analysis (PICA). 2013, Researchgate. [10]. Saverino, C., et al., The associative memory deficit in aging is related to reduced selectivity of brain activity during encoding. Journal of cognitive neuroscience, 2016. 28(9): p. 1331-1344. [11]. Lateef, F. and Y. Ruichek, Survey on semantic segmentation using deep learning techniques. Neurocomputing, 2019. 338: p. 321-348. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS) (2020) Volume 72, No 1, pp 58-66 66 [12]. Garcia-Garcia, A., et al., A review on deep learning techniques applied to semantic segmentation. arXiv preprint arXiv:1704.06857, 2017. [13]. Zhao, B., et al., A survey on deep learning-based fine-grained object classification and semantic segmentation. International Journal of Automation and Computing, 2017. 14(2): p. 119-135. [14]. Anderson, J.A., et al., Task-linked diurnal brain network reorganization in older adults: A graph theoretical approach. Journal of Cognitive Neuroscience, 2017. 29(3): p. 560-572. [15]. Liu, Z., et al., Towards clinical diagnosis: Automated stroke lesion segmentation on multi-spectral MR image using convolutional neural network. IEEE Access, 2018. 6: p. 57006-57016. [16]. Zhong, Z., et al. 3D fully convolutional networks for co-segmentation of tumors on PET-CT images. in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). 2018. IEEE. [17]. Grady, C., et al., Age differences in the functional interactions among the default, frontoparietal control, and dorsal attention networks. Neurobiology of aging, 2016. 41: p. 159-172. [18]. Machulda, M.M., et al., Comparison of memory fMRI response among normal, MCI, and Alzheimer‟s patients. Neurology, 2003. 61(4): p. 500-506. [19]. Yang, X., S. Sarraf, and N. Zhang, Deep learning-based framework for Autism functional MRI image classification. Journal of the Arkansas Academy of Science, 2018. 72(1): p. 47-52. [20]. Sarraf, S. and G. Tofighi. Deep learning-based pipeline to recognize Alzheimer‟s disease using fMRI data. in 2016 Future Technologies Conference (FTC). 2016. IEEE. [21]. Sarraf, S. and M. Ostadhashem. Big data application in functional magnetic resonance imaging using apache spark. in 2016 Future Technologies Conference (FTC). 2016. IEEE. [22]. Sarraf, S., et al., DeepAD: Alzheimer‟s Disease Classification via Deep Convolutional Neural Networks using MRI and fMRI. bioRxiv, 2017: p. 070441. [23]. Sarraf, S., C. Saverino, and A.M. Golestani. A robust and adaptive decision-making algorithm for detecting brain networks using functional mri within the spatial and frequency domain. in 2016 IEEE- EMBS International Conference on Biomedical and Health Informatics (BHI). 2016. IEEE. [24]. Sarraf, S., et al., MCADNNet: Recognizing stages of cognitive impairment through efficient convolutional fMRI and MRI neural network topology models. IEEE Access, 2019. 7: p. 155584- 155600. [25]. LeCun, Y., et al., Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998. 86(11): p. 2278-2324.