 Kurdistan Journal of Applied Research (KJAR) Print-ISSN: 2411-7684 | Electronic-ISSN: 2411-7706 Website: Kjar.spu.edu.iq | Email: kjar@spu.edu.iq Modified AlexNet Convolution Neural Network For Covid-19 Detection Using Chest X-ray Images Shadman Q. Salih Hawre Kh. Abdulla Database Technology Department Database Technology Department Technical College of Informatics Technical College of Informatics Sulaimani Polytechnic University Sulaimani Polytechnic University Sulaimani, Iraq Sulaimani, Iraq shadman.salih@spu.edu.iq hawre.abdulla@spu.edu.iq Zanear Sh. Ahmed Nigar M. Shafiq Surameery Information Technology Department Building and Construction Engineering Department Erbil Technical institute College of Engineering Erbil Polytechnic University University of Garmian Erbil, Iraq Kalar, Sulaimani, Iraq zanear.ahmad@epu.edu.iq nigar.mahmoud@garmian.edu.krd Rasber Dh. Rashid Software Engineering Department Faculty of Engineering Koya University Koya, Erbil, Iraq rasber.rashid@koyauniversity.org Article Info ABSTRACT Special Issue on Coronavirus (COVID-19) DOI: 10.24017/covid.14 Article history: Received: 29 May 2020 Accepted 03 June 2020 First outbreak of COVID-19 was in the city of Wuhan in China in Dec.2019 and then it becomes a pandemic disease all around the world. World Health Organization (WHO) confirmed more than 5.5 million cases and 341,155 deaths from the disease till the time of writing this paper. This new worldwide disease forced researchers to make more precise way to diagnose COVID-19. In the last decade, medical imaging techniques show its efficiency in helping radiologists to detect and diagnose the diseases. Deep learning and transfer learning algorithms are good techniques to detect disease from different image source types such as X-Ray and CT scan images. In this work we used a deep learning technique based on Convolution Neural Network (CNN) to detect and diagnose COVID-19 disease using Chest X-ray images. Moreover, the modified AlexNet architecture is proposed in different scenarios were differing from each other in terms of the type of the pooling layers and/or the number of the neurons that have used in the second fully connected layer. The used chest X- ray images are gathered from two COVID-19 X-ray image datasets and one dataset includes large number of normal and pneumonia X-ray images. With the proposed models we obtained Keywords: COVID-19, Chest X-Ray Images, CNN, AlexNet, Deep Learning. Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 120 the same or even better result than the original AlexNet with having a smaller number of neurons in the second fully connected layer. Copyright © 2020 Kurdistan Journal of Applied Research. All rights reserved. 1. INTRODUCTION At the end of 2019 an emerged novel Corona Virus has declared in Wuhan, Hubei Province, China after investigating the unidentified pneumonia cases which caused by a disease officially is called COVID-19 [1, 2]. The virus is rapidly spread in other countries all over the world and millions of people have infected by the disease. Consequently, World Health Organization announced COVID-19 as a global health emergency [3]. In addition, a report categorizes the cases of COVID-19 into two parts which are active and closed, under the active cases 98 percent infected patients in mild condition [3]. The prodrome of the virus includes high temperature, cough, dyspnea also it causes middle east respiratory syndrome (MERS-COV) and sever acute respiratory syndrome (SARS) that is known as (SARS-COV) [4, 5]. As the virus is transmitted human-to-human, accordingly, it is strongly recommended to halt direct contact with others and put social distancing to prevent the distribution the disease from one person to another [6]. Hence, identifying those who are infected by the virus in early stage is very important to save their lives. The diagnoses of the disease become an issue for the hospitals due to the shortage of the test kits, limited finding rate and it takes time as well as using reverse transcription polymerase chain reaction (RT-PCR) to determine the infected patient is not accurate as per the rate is from 30 to 60 percent, which means a large number of COVID-19 patients have not detected and will not take proper treatment and isolation. Moreover, inaccurate test helps the disease to speared faster as it transmits person to person [7]. Furthermore, the X-ray and (CT) scan of chest are also used to establish the MERS- COV and SARS-COV in early stage of the disease. In this technology era, enhancing the CT and X-ray based on artificial intelligence (AI) tools may help clinicians to detect and monitor the virus faster and more efficiently [7]. Additionally, there are a bunch of researches [1, 7, 8, 9] used deep learning-based techniques to detect the infected patients which help to reduce the load on doctors and hospitals to identify the disease and make the process faster in compare with RT-PCR. Accordingly, we have studied available Convolution Neural Network (CNN) in the literature named AlexNet. Based on this CNN architecture, we proposed a modified version of AlexNet to train and test the available X-ray images for the study. The aim is to determine and identify the COVID-19 cases from other two class cases named Normal and Abnormal cases. 2. RELATED WORK Nowadays, due to rapidly spreading the COVID-19 disease, more attention goes to the field of detecting the disease by the researchers. In [10] the researchers proposed two different techniques to evaluate lung pneumonia and cancer namely modified AlexNet (MAN), moreover, in the study, the principle component analysis (PCA) used by combination with MAN. In the first proposed technique, the classification result improved by using five blocks from 1-5 with two fully connected layers, so, each initial block consists of Convolution, max pooling, ReLU and normalization. At the end, the SoftMax layer has been replaced by Support Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 121 Vector Machin (SVM). The proposed technique was tested on two different chest image types X-ray and lung CT scans with including only two types of classes Normal and Abnormal for each of them. The accuracy obtained was 96% and 97.27% for X-ray and CT scans respectively. According to [11] the new CNN proposed including 16 layers only, the method designed to distinguish the Normal and COVID-19 classes only. The algorithm tested on both X-ray and CT scan images. It is worth to mention that the proposed technique was obtained good accuracy performance with declaring that they used a very small number of cases. In [12]  three different scenarios have been tested with using three different available CNN architecture that are: AlexNet, googlenet, and resnet18. The idea of transferring learning (TL) is used for all of the three scenarios. The scenarios are different from each other in the number of classes that have used. In the first one, four different classes (Normal, COVID-19, Viral, and Bacteria) have been used with googlenet that resulting in having the accuracy rate of 80.6 %. In the second scenario, three different classes have been used (Normal, COVID -19, and Pneumonia) and obtaining the accuracy rate about 85.2 with using AlexNet CNN architecture, while in the third scenario, only two classes have been used (Normal and COVID-19) and the algorithm guaranteed 100% of accuracy with using googlenet architecture. Jianping Zhang et al. [13] presents a new deep learning model for detecting COVID-19 that contain three components, they used 100 X-Ray images as a Covid-19 from 70 patients and 1431 X-Ray images as non covid-19. The result shows 96% for detecting covid-19 and 70.65% for detecting non-covid-19. Wang et al. [14] proposed a new CNN architecture named as COVID-Net. They generate the COVIDx dataset by combining and modifying two different datasets, “covid-19 image data collection” and “RSNA Pneumonia   Detection Challenge” dataset, and it contain 16,756 chest radiography across 13,645 patient cases from the two different dataset that have been mentioned above. The accuracy of testing COVIDx is 92.4%. From the proposed works available in the literature, it is clear that a good accuracy performance can be obtained when they tested and applied over only two classes (Normal and COVID-19). The challenge is to have more than two classes, therefore in this work, we aimed to design and test a new CNN architecture to be suitable for more than two classes. Moreover, we aim to design an architecture to distinguish three different classes (Normal, COVID-19, and Pneumonia (Bacteria and other (Non COVID-19) Viral). 3. AlexNet Architecture Analysis The history of deep CNN starts with the LeNet which originally designed for the purpose of handwritten digit recognition. The LeNet not performed well due to its limitation to the handwritten digit recognition tasks [15]. Thus, the AlexNet is supposed to be the first deep CNN architecture because of its’ outstanding results for the classifications and recognition tasks performed on images [16]. In early 2000, the learning capacity of the deep CNN architectures was restricted to small size because of the hardware limitations. Thus, in order to overcome the limitations of the hardware and getting the whole capacity of deep CNN, AlexNet was trained on two “NVIDIA GTX 580” GPUs in parallel. In order to make the CNN applicable for different types of images, in AlexNet, the CNN depth was extended from only (5) layers in the LetNet CNN to (8) layers. The layers are: five convolutional layers, two fully connected hidden layers, and one fully connected output layer, as it shown in Fig1. Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 122 Figure 1: AlexNet Architecture Design Although increasing the depth of the architecture will improve the generalization for different image resolutions, on the other hand, it results in the overfitting issue. To fix this issue, the dropout algorithm has been used, in which the algorithm randomly removes some transformational units during the training process to motivate the model to learn more better features [17, 18]. In addition, the local response normalization and overlapping subsampling have been applied to reduce the overfitting in order to improve the generalization. Moreover, in AlexNet, the sigmoid activation function has been replaced by the ReLU function in order to improve the convergence rate by reducing the problem of vanishing gradient [19, 20]. As it is clear in Fig1, in AlexNet, the input image must be augmented to a fix size of 227x227x3, in the first layer the 96 convolution filter with window shape of size 11 × 11 applied, whereas in the second layer 256 convolution filter with window shape size of 5×5 applied, followed by 384, 384, and 256 convolution filter with window size of 3×3 for the rest of the other three layers applied. Moreover, after the first, second, and the last convolutional layers, the network contains 3 × 3 maximum pooling layers with stride of 2. In addition to this five convolution layers, there are two fully-connected layers with 4096 neuron outputs after the fifth convolutional layer. This is followed by one fully connected output layer placed at the end of the network which is originally having 1000 output classes. Finally, Dropout, ReLU and preprocessing represent the important key steps in achieving excellent performance in computer vision tasks. In this paper we used the original AlexNet as a transfer learning mode by modifying the last output layer to be suitable to the number of the X-ray classes that have been used in these experiments. We replaced the 1000 classes that the original AlexNet have, with only 3 classes because we tested only three classes (COVID-19, Normal, and Pneumonia ) in this study. 4. Proposed Models (Modified AlexNet) In this research, we proposed a new CNN model by modifying the architecture layers of the AlexNet Convolution Neural Network to see how the classification result will be affected and improved. In fact, we selected the AlexNet architecture because the structure of the AlexNet is the simplest among other CNN systems and need suitable time and efficiency for learning phase especially we are using the normal personal computer with normal CPU based systems. The proposed model(s) Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 123 has been developed using software package in MATLAB program. Another reason of using this architecture is high performance and fast training with very few epochs. We propose four different modified models based on the original AlexNet architecture. In all of the four models, the structure of the original AlexNet remain with having modifications on some layers. The four modified models can be described as follows: Model 1: All of the 3x3 Max Pooling layers that are available in the original AlexNet have been replaced with corresponding 3x3 Average Pooling layers with remaining the stride value as its which is 2. See figure 2. Figure 2: Model 1 of Modified AlexNet Architecture Model 2: The 3x3 Max Pooling layers that are available after the first and second convolution layers are replaced with corresponding 3x3 Average Pooling layers with remaining the Max Pooling after the fifth convolution without any changes (i.e Max Pooling), again the stride value remain as it is which is 2 in all of the three Pooling layers. See figure 3. Figure 3: Model 2 of Modified AlexNet Architecture Model 3: As shown in Figure 4, the number of the neurons in the second fully connected layer has been reduced form 4096 neurons to 2096 neurons. All other available layers are remained as the original AlexNet. This model designed to see how reducing the number of neurons affects the results besides increasing the efficiency. Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 124 Figure 4: Model 3 of Modified AlexNet Architecture Model 4: In this model, we combined the two previously proposed models’ number 2 and 3 as shown in figure 5. This means that we replaced the Max Pooling layers after the first and second convolution layers with Average Pooling layers and the Max Pooling after the fifth convolution remain as it is, in addition, we reduced the number of the neurons in the second fully connected layer form 4096 neurons to 2096 neurons. Figure 5: Model 4 of Modified AlexNet Architecture 5. Experimental Data and Setup Recently, Covid-19 virus has affected a global population. The virus continues to have a huge influence on the welfare and health of the people around the world. The essential step in confrontation this risky virus is radiology examination. This could help the doctors to diagnose the disease and gave the rapid treatment to the patient in a precise way [21]. These radiographs are then stored as X-ray or CT-scan images in various public available sources such as COVIDx dataset by Cohen [22], a set of 6290 chest X-ray images of non-Covid-19 viral cases, bacterial infections and confirmed normal images by Kermany et al authors [23]. Another source of dataset is a collection of 79 Covid-19 chest X-ray confirmed radiographs that was recently made public by some websites such as Radiopedia, “SIRM (Italian Society of Medical and Interventional Radiology)” and “RSNA (Radiological Society of North America)” [24]. The main reason of these huge-data collection and made them publicly available sources is to provide a free access mechanism for the researchers to be able to use them in their research papers as well as helping them to contribute and share their de-identified COVID-19 chest X-ray data to the project. Hence, in this section, we will explain the details of the datasets that have been used for the train and test Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 125 process in this paper for purpose of Covid-19 and other cases detections such as Normal, Bacterial and Viral. 5.1 First Dataset (COVIDx) The COVIDx dataset recently made available at [22] by the authors of Covid-Net, which is one of the first open source network. It is a deep CNN, which is made publicly available to detect corona virus from CT-Scan or X-ray images. The dataset is consists of 13,975 X-ray and CT scans images taken from 13,870 patient cases. In the recent version of the COVIDx dataset, there was a total of 111 Covid-19 X-ray images from approved corona virus patients. In this paper work, we used all 111 chest X-ray positive Covid-19 images. 5.2 Second Dataset This dataset was collected and made publicly available by Kermany and et. al. It contains 6290 chest X-ray images from numerous patients with normal infections, bacterial and other viral (non-Covid-19) cases [23]. In this huge data set images, only 1108 chest radiography images has been chosen to train, validation and test. More especially, we used 554 images from normal cases, 277 images from bacterial cases and 277 images from viral (non-Covid-19) cases. 5.3 Third Dataset Third dataset contains only 79 positive chest X-ray images. It was recently composed from Radiological Society of North America (RSNA), Radiopedia, and Italian Society of Medical and Interventional Radiology (SIRM) websites. This collection of confirmed positive images are publicly available in [24]. In this dataset only 43 chest X-ray positive Covid-19 images are nominated to train, validation and test sets. Figure 6 shows random samples of the images for all classes. Figure 6: Chest X-ray samples As a result, from the three above mentioned databases, we used in total 154 (100 for train and 54 for test) X-ray images of confirmed Covid-19 cases, 554 (500 for train and 54 for test) X-ray images of Normal cases, and 554 (500 for train and 54 for test) X-ray images of Abnormal (pneumonia) cases. The 554 cases of Abnormal (pneumonia) cases originally taken from 277 cases of bacteria and 277 cases of viral (Non-Covid-19) cases. It is valuable to be mentioned that the number of Covid-19 X-ray images is considerably lower than other classes that lead to imbalance classification issue. However, to reduce the impact of imbalance classification we tried to not use all available cases of normal and pneumonia and only the above mentioned number of cases have been used. To investigate and test the mentioned proposed models in the previous section as well as the original AlexNet architecture, we used the above-classified dataset. The same train and test sets have been used with the original AlexNet architecture as well as the proposed modified models. See the next section for the experimental results obtained. Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 126 6. Experimental Results The process of utilizing learning weights or using obtained data from determining and solving one issue to another correlated issue is known as Transfer Learning (TL). Weights enhanced from the AlexNet architecture on X-ray image dataset used in TL model. In this paper, TL approach has been adopted to investigate the performance discussion of original AlexNet CNN architecture and then compared with Modified AlexNet architectures. Accordingly, in all of the layers, the weights are re-trained on our chest X-ray images. The resizing process to the appropriate dimensions of the original AlexNet and Modified AlexNet architectures has been done on the entire set of the images from the training and the testing sets. According to the former researches in literature review, no preprocessing applied to input images, hence the same norm has been followed. For all CNN architectures, the training parameters in TL are as follows: Mini Batch Size =15, Number of Epochs =10, Initial Learn Rate = 3e-4. The software package MATAB is used for conducting all experimental results. Two metrics were recorded in order to measure the performance of CNN classification, which are specificity and sensitivity. For calculating the mentioned metrics, we recorded the following classifications test measurements: TP – True Positive: represents the number of the correctly recognized positive cases. TN – True Negative: represents the number of the correctly recognized negative cases. FP – False Positive: represents the number of the incorrectly identified negative cases. FN – False Negative: represents the number of the incorrect classification of disease. The amount of the unhealthy cases appropriately identified by CNN models is known as Sensitivity while the Specificity measures the amount of negative cases properly detected as healthy by CNNs. For the purpose of comparisons as well as testing the performance, we calculate the sensitivity and specificity of the original and all of the four modified proposed models. Table 1 shows the obtained results from all experiments conducted. Moreover, the table shows the sensitivity and specificity of all of the three individual classes Covid-19, Normal, and Abnormal (pneumonia). To see individual classified cases for every single experiment, we calculated the confusion matrix among the classes, figure 7 shows the confusion matrix of the original AlexNet while figure 8 shows the confusion matrix of obtained results from all of the four proposed models. Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 127 Table 1: Testing result for Original and Modified AlexNet architecture Figure 7: Confusion matrix of original AlexNet CNN Architecture Class Sensitivity Specificity Original AlexNet Covid-19 98.14 99.04 Normal 72.22 87.60 Abnormal 90.74 94.89 Proposed (Model 1) Covid-19 79.62 90.67 Normal 83.33 91.42 Abnormal 94.44 97.02 Proposed (Model 2) Covid-19 90.74 95.57 Normal 81.48 90.90 Abnormal 94.44 97.02 Proposed (Model 3) Covid-19 90.74 95.32 Normal 83.33 91.74 Abnormal 90.74 95.37 Proposed (Model 4) Covid-19 94.44 97.27 Normal 81.48 90.99 Abnormal 90.74 95.14 Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 128 (a) Model 1 (b) Model 2 (c) Model 3 (d) Model 4 Figure 8: Confusion matrix of Proposed Models From table1 and figure7, we can notice that the obtained results from the original AlexNet for recognising Covid-19 cases is promising but there is a problem with the results of detecting the normal case. There are 15 cases out of 54 cases are misclassified. Although our aim is to detect the Covid-19 cases accurately, there is a problem when the normal cases are classified as affected cases especially in the medical side. Therefore, the proposed models are designed to decrease incorrectly classified cases. For example, when we compare the obtained results from model1 with the original AlexNet results, we can see that the sensitivity and the specificity of both of the normal and abnormal classes are increased with decreasing the sensitivity and the specificity of the Covid-19 class. In model2, after we mixed the average and max pooling, we obtained higher detection degree for Covid- 19 classes compared to model1. Again having better detection degree for both of the normal and abnormal classes comparing to the original AlexNet. Although reducing the number of neurons of the original AlexNet such as in model3 increased the efficiency of executing the training phase, the same conclusion as model2 can be obtained when the results are compared with the original AlexNet. Finally when we combine both of the ideas in model2 and model3, we can guaranteed the efficiency as well as the better detection rate of Covid-19 classes if compared with all other three proposed models and losing a small degree of detection if compared with the original AlexNet, taken in account that model4 have better detection for both other classes normal and abnormal when compared with the original AlexNet. Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 129 If we take overall, the detection rate and counting the corrected number of cases for all of the three classes in each architecture, we can say that out of 162 cases the original AlexNet classified 141 cases correctly while the correctly classified cases are 139, 144, 143, and 144 for our four proposed model1, model2, model 3, and model 4 respectively. 7. Conclusion and Future Work Many researchers have exhibited that the already existing convolutional neural network architectures have a significant ability of detecting COVID-19 from X-ray or CT-Scan images in an accurate manner and provide excellent results [11, 25, 26]. However, their results can be improved by modifying their structure in one or more of their layers or their other features. In this paper we used original AlexNet architecture for Covid-19 detection, Pneumonia, and Normal cases. This architecture consists of eight layers: five convolutional layers and three fully-connected layers with some of the features that are new approaches to convolutional neural networks and have made the AlexNet special [27]. The aim of this paper was to introduce four effectives Modified AlexNet CNN models that can outperform pre-trained AlexNet for the purpose of helping radiologists to recognize and detect correctly the classes from chest X-ray images. These models were differing from each other in terms of the type of the pooling layers and/or the number of the neurons that have used in the second fully connected layer. Moreover, the images that have been used in this study are collected from three different sources. So, we collectively make the use of these X- ray images to be publicly available for the researchers. From the obtained results, it became obvious that in case of a small dataset and unbalanced number of the X-ray images, we can conclude that with replacing all of the max pooling layers with average Pooling layers, the result has been enhanced even though we have a noticeable reduction in the sensitivity of the Covid-19 detection result. This issue was reduced by replacing just the first two max pooling layers with average Pooling layers, which results in the improvement of the Covid-19 case sensitivity from 79.6 to 81.46. On the other hand, with reducing the number of the neurons by 2000, we have got the same or even better result than the original AlexNet with a little bit reduction in the Covid-19 case sensitivity. Finally, one open door for any experiments further is to apply the same models with a large-scale dataset where its images can be collected from different countries. In addition, more than three classes can be tested with keeping in mind the balancing issue. Moreover, trying to apply the same modifications on deeper transfer networks such as Densenet, and Inceptionresnet can be another future research direction. REFERENCE [1] Y. Song, S. Zheng, X. Zhang, X. Zhang, Z. Huang,J. et al., "Deep learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) with CT images," p. 10, 25 February 2020. [2] W. H. Organization, Health, [Online]. Available: https://www.who.int/dg/speeches/detail/who-director- general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-may-2020. [Accessed 12 May 2020]. [3] "Worldometers," [Online]. Available: https://www.worldometers.info/coronavirus/. [Accessed 27 May 2020]. [4] Q. Li, M. Med, et al, "Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus–Infected Pneumonia," The new england journal of medicine, vol. 382, p. 13, 26-March-2020. [5] S. Stoecklin, P. Rolland, et al, "First cases of coronavirus disease 2019 (COVID-19) in France: surveillance, investigations and control measures," Euro surveillance, vol. 2000094, January 2020. [6] "How to Protect Yourself & Others," Centers for Disease Control and Prevention (CDC), [Online]. Available: https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting- sick/prevention.html?CDC_AA_refVal=https%3A%2F%2Fwww.cdc.gov%2Fcoronavirus%2F2019 - ncov%2Fprepare%2Fprevention.html. [Accessed 13 May 2020]. [7] S. Khobahi, Ch. Agarwal and M. Soltanalian, "CoroNet: A Deep Network Architecture for Semi-Supervised Task-Based Identification of COVID-19 from Chest X-ray Images," medRxiv , 2020. [8] A. Narin, C. Kaya and Z. Pamuk, "Automatic Detection of Coronavirus Disease (COVID-19) Using X-ray Images and Deep Convolutional Neural Networks," eprint arXiv:2003.10849, p. 17, March 2020. [9] X. Xu, X. Jiang, et al., "Deep Learning System to Screen Coronavirus Disease 2019 Pneu monia," arXiv, vol. 2002.09334, p. 29, 2020. [10] A. Bhandary, G. Prabhu, et al, "Deep-Learning Framework to Detect Lung Abnormality – A study with Chest X-Ray and Lung CT Scan Images," elsevier Pattern Recogn Lett , vol. 129, pp. 271-278, 2020. Kurdistan Journal of Applied Research | Special Issue on Coronavirus (COVID-19)| 130 [11] H. S. Maghdid, A. T. Asaad, et al., "Diagnosing COVID-19 Pneumonia from X-Ray and CT Images using Deep Learning and Transfer Learning Algorithms," arxiv, vol. 2004.00038, p. 8, 2020. [12] M. Loey, F. Smarandache, et al, "Within the Lack of Chest COVID-19 X-ray Dataset: A Novel Detection Model Based on GAN and Deep Transfer Learning," Symmetry, vol. 12, no. 651, p. 19, 2020. [13] J. Zhang, Y. Xie, et al., "COVID-19 Screening on Chest X-ray Images Using Deep Learning based Anomaly Detection," arXiv, vol. 2003.12338, p. 6, 2020. [14] K. Hammoudi, H. Benhabiles, et al., "Deep Learning on Chest X-ray Images to Detect and Evaluate Pneumonia Cases at the Era of COVID-19," arXiv, vol. 2004.03399, p. 6, 2020. [15] Y. LeCun, L. D. Jackel, et al, "Learning algorithms for classification: A comparison on handwritten digit recognition," Neural networks: the statistical mechanics perspective, vol. 261, p. 16, 1995. [16] A. Krizhevsky, I. Sutskever, et al., "Imagenet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems 25 (NIPS 2012), 2012. [17] N. Srivastava, G. Hinton, et al., "Dropout: A Simple Way to Prevent Neural Networks from Overfitting," Journal of Machine Learning Research 15, Vols. 1929-1958, p. 30, 2014. [18] G. E. Dahl, T. N. Sainath, G. E. Hinton, "IMPROVING DEEP NEURAL NETWORKS FOR LVCSR USING RECTIFIED LINEAR UNITS AND DROPOUT," in 2013 IEEE international conference on acoustics, speech and signal processing, 2013. [19] S. Hochreiter, "The vanishing gradient problem during learning recurrent neural nets and problem solutions," International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 06, no. 02, pp. 107-116, 1998. [20] V. Nair, G. E. Hinton, "Rectified Linear Units Improve Restricted Boltzmann Machines," in Proceedings of the 27th international conference on machine learning (ICML-10), 2010. [21] L. Wang, Z. Q. Lin, A. Wong, "COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images," arXiv, vol. 2003.09871, p. 12, 2020. [22] J. P.Cohen, P. Morrison, L. Dao, "An open database of COVID-19 cases with chest X-ray or CT images.," arXiv, vol. 2003.11597, p. 4, Mar 2020. [23] D. S.Kermany, M. Goldbaum, W. Cai , et al., "Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning," Cellpress, vol. 172, no. 5, pp. 1122-1131, 2018. [24] Larxel, "COVID-19 X rays," Kaggle, [Online]. Available: https://www.kaggle.com/andrewmvd/convid19- X-rays. [Accessed 27 May 2020]. [25] M. Farooq, A. Hafeez, "COVID-ResNet: A Deep Learning Framework for Screening of COVID19 from Radiographs," arXiv:, vol. 2003.14395 , p. 6, 2020. [26] T. Majeed, R. Rashid, et al., "Covid-19 Detection using CNN Transfer Learning from X-ray Images," medRxiv, vol. 20098954, p. 10, 2020. [27] J. Wei, "AlexNet: The Architecture that Challenged CNNs," Towards Data Science, 3 Jul 2019. [Online]. Available: https://towardsdatascience.com/alexnet-the-architecture-that-challenged-cnns-e406d5297951. [Accessed 25 May 2020]. [28] L. Liao, Y. Zhao, et al., "Finding the Secret of CNN Parameter Layout under Strict Size Constraint," in Proceedings of the 25th ACM international conference on Multimedia, 2017.