INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL Online ISSN 1841-9844, ISSN-L 1841-9836, Volume: 17, Issue: 3, Month: June, Year: 2022 Article Number: 4697, https://doi.org/10.15837/ijccc.2022.3.4697 CCC Publications Covid-19 Patients’ Hospital Occupancy Prediction During the Recent Omicron Wave via some Recurrent Deep Learning Architectures H. Bouhamed, M. Hamdi, R. Gargouri Heni Bouhamed* Advanced Technologies for Image and Signal Processing Unit (ATISP) Sfax University, Tunisia *Corresponding author: heni.bouhamed@fsegs.usf.tn Monia Hamdi Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O.Box 84428, Riyadh 11671, Saudi Arabia mshamdi@pnu.edu.sa Rahma Gargouri Pneumology Department, Hedi Chaker Hospital, Sfax, Tunisia gargouri.bouhamed.rahma@gmail.com Abstract This paper described a suggested model to predict bed occupancy for Covid-19 patients by country during the rapid spread of the Omicron variant. This model can be used to make decisions on the introduction or alleviation of restrictive measures and on the prediction of oxygen and health human resource requirements. To predict Covid-19 hospital occupancy, we tested some recurrent deep learning architectures. To train the model, we referred to Covid-19 hospital occupancy data from 15 countries whose curves started their regressions during January 2022. The studied period covers the month of December 2021 and the beginning of January 2022, which represents the period of strong contagion of the omicron variant around the world. The evolution sequences of hospital occupancy, vaccination percentages and median ages of populations were used to train our model. The results are very promising which could help to better manage the current pandemic peak. Keywords: prediction, COVID-19 hospital occupancy levels, Deep Learning, GRU, vaccination. 1 Introduction The coronavirus has had a huge impact on health systems and the global economy, so far and the World Health Organization has officially labeled Covid-19 a pandemic since March 2020. By the end of December 2021, the number of people diagnosed with Covid-19 has surpassed 340 million worldwide, putting hospitals and health systems in general in a bind. Indeed, the pandemic is placing a burden on clinical bed capacity with oxygen sources and intensive care units that has far exceeded https://doi.org/10.15837/ijccc.2022.3.4697 2 available capabilities over the past two years. The chaotic situation in Lombardy, Italy in March 2020 tragically demonstrated the impact of the lack of health care capacity (Rosenbaum, 2020), and the need to manage hospital bed capacity as best as possible. The authors in (Phua et al., 2020, Bekker et al., 2022) called on practitioners, hospital administrators, governments, and policy makers to quickly prepare for a substantial increase in oxygen bed capacity, especially with the virus variants that further complicated the situation. Their recommendations include hospital capacity and staffing. With the emergence of the Omicron variant in November 2021, being reported as extremely rapid contagious (Le Page, 2021), it has become strongly recommended to plan for increased capacity due to the rapid increase in patients in a relatively short time compared to other waves. Another less studied but even more important issue is the impact on other types of care that are delayed due to Covid-19 patients occupying many beds and using various forms of capacity that would normally be used for non-COVID care. A 2020 study of the impact of the first wave resulted in an estimated loss of 400,000 healthy life years in the Netherlands (Gupta Strategists, 2020). The objective of this work was to develop a predictive model that helps decision makers in any country to reserve the right number of beds, adequate human resources and quantities of oxygen needed for COVID-19 patients without deteriorating the quality of service for other patients. Many studies have tackled this issue over the last two years (Bouhamed 2020a, Bouhamed 2020b, Bouhamed 2020c, Findel and Suarez 2021, Caro et al 2021, Bekker et al 2022). They have focused mainly on the prediction of the number of cases, cures, deaths and even the use of health resources (Caro et al., 2021) according to the contagion rate and the old covid-19 strains severity of the old covid-19 strains. In this work we proposed to develop a model to predict the evolution of the use of hospital beds by COVID-19 patients according to the high rate of contagion of the Omicron variant (Le page 2021) taking into consideration the massive vaccination campaign initiated in most countries and the doubts about the dangerousness of the Omicron variant in comparison with the old strains. Several studies invoke the supremacy of deep neural networks (DNNs) that exceed the performance of classical Machine Learning formalisms in various applications (Bouhamed and Ruichek, 2018; Hinton et al., 2012; Mohamed et al., 2012; Ciresan et al., 2010; Yu et al., 2011). Deep Learning, considered part of the Big Data domain, is a set of machine learning methods for modeling high abstraction level data. It is based on articulated architectures of various transformations in nonlinear space (Bouhamed and Ruichek, 2018; Bengio, 2009). The current interest in Deep Learning stems from both its conceptual advances and its technological advances allowing to run learning processes in parallel. Indeed, all available solutions can exploit the huge potential of computational power, provided by current modern computers in single or multi-use (cluster of machines), by soliciting the main processor(s) (CPU) and dedicated graphics processors (GPU) (Bouhamed and Ruichek, 2018). A Big Data model can adapt to a huge volume of data and a huge amount of sequential processing beyond the capabilities of the most powerful servers (Bouhamed and Ruichek, 2018; Zikopoulos and Eaton, 2011). Since prediction, in our context, depends on observations obtained at previous timings, our scope was more about temporal sequence prediction. Sequence prediction poses different problems form the other types of supervised learning. The sequence imposes an order on the observations that must be preserved when training models and predictions. In general, prediction problems that involve sequence data are called sequence prediction problems, despite the existence of several other problems that differ based on input and output sequences. Many artificial recurrent neural network (RNN) architectures (Brownlee, 2017; Hochreiter and Schmid- huber, 1997) are used in the field of deep learning, including the simple recurrent neural network (SRNN), long-term memory (LSTM), and the grid recurrent unit (GRU). Unlike standard feedfor- ward neural networks, the SRNN has feedback connections. It is capable of processing not only single data points, but also entire sequences of data (such as speech or video). For example, the RNN is applicable to tasks such as non-segmented and connected handwriting recognition (Graves et al., 2009), speech recognition (Xiangang and Xihong, 2014) and anomaly detection in network traffic or intrusion detection systems. Therefore, we proposed to test these three RNN architectures (SRNN, LSTM, GRU) for this study. The rest of this paper is organized as follows: In Section 2, we introduced the RNNs and their three main architectures. In Section 3, we detailed the data as well as the model features and hyper- https://doi.org/10.15837/ijccc.2022.3.4697 3 parameters. In section 4, we proceeded to the methodological and epidemiological experimentation and discussion. Finally, in the last section, we drew our conclusions and suggested some perspectives. 2 RNN, SRNN, LSTM and GRU The RNN is an artificial neural network with recurrent connections (Figure 1). It consists of intercon- nected neuronal units that interact in a non-linear way having at least, one cycle in the structure. The neuronal units are connected by arcs with weights. The output of a neuron is a non-linear combination of its inputs. The simplest architecture is noted by SRNN; it is simple and adapted to input data of different sizes. The network learning techniques are the same as for the classical neural networks, exploiting gradient backpropagation for learning. Nevertheless, the SRNNs can be plagued by the gradient fading problem when they try to keep too many past elements in memory. Other architectures deal with this problem, notably the GRU and the LSTM. Figure 1: RNN The LSTM is a widely used architecture to deal with the problem of gradient vanishing. This archi- tecture was proposed by Hochreiter and Schmidhuber (Hochreiter and Schmidhuber, 1997). The basic idea underlying this architecture is that each computational unit is linked not only to a hidden state h, but also to a state c of the cell which acts as a memory. The transition results in the transfer of a constant gain equal to 1, so that errors propagate to previous steps, up to 1000 steps in the past, without any instance of gradient vanishing (Gers et al., 2002). GRUs, which were introduced in 2014 by Cho et al. (Cho et al., 2014), have a simpler architecture than the LSTM. They have a much smaller number of parameters, and therefore can train faster with a constant hidden layer size. 3 Data and methods This is a multicenter study on the evolution of the number of hospitalized covid-19 patients for 15 countries (Table 1) from December 01, 2021 to January 27, 2022 (supplementary material named data19012022bis.txt present the evolution of the number of hospitalized covid-19 patients during 58 day for each of the 15 countries studies depicted on Our World in Data1). The study focused on this period because of the prevalence of Omicron in these countries from December 2021 (World Health Organization2). As for the countries, their choice was not arbitrary but rather justified by the fact that 1Our World in Data (OWID) is a scientific online publication that focuses on large global problems such as poverty, disease, hunger, climate change, war, existential risks, and inequality. Web site: https://ourworldindata.org/ 2The World Health Organization (WHO) is a specialized agency of the United Nations responsible for international public health. Web site: https://www.who.int/ https://doi.org/10.15837/ijccc.2022.3.4697 4 the curve peak of the number of hospitalized patients has already been exceeded for these 15 countries. This will allow our model to approximately predict the descent slope of this peak for the remainder of the countries in full Omicron wave. Owing to the information on the use of hospital beds during this period, we also introduced two other variables into the learning base, namely the vaccination rate and the national median age. The choice of the variables was inspired by data from the literature during the old virus variants waves and by the preliminary data related to the Omicron variant: the median age of hospitalized patients seems to be among the parameters that are likely to vary with waves and variants. For the original Alpha variant, hospitalizations involved people above 65 years (World Health Organization, 2021) but decreased with the delta variant. The outbreak in India proved that viral mutations cause increased pathogenicity in young, apparently even in healthy people. Shocking US data showed an increasing hospitalization and death rates among the young. As time progresses, the advent of COVID vaccination and its relative efficacy skews studies regarding the impact of age on the severity and morbidity of the infection. Indeed, in the beginning, access to vaccines was limited to the oldest people and those with chronic diseases. And even after the generalization of vaccination, the young population was more reluctant about this issue because of doubts about the safety and efficacy of vaccines. The result of such behavior was an increase in the number of COVID19 infections among unvaccinated young people, with sometimes a severe disease, especially for the Delta variant. Currently, with the mandatory health passport worldwide, vaccination of the young has become more common. This has coincided with the appearance of the new Omicron variant. The result is currently in favor of mild forms that affect more such young people. In the same perspective, vaccination has not yet included the pediatric population, which is apparently the most vulnerable to the Omicron variant. The question that needs to be addressed is on the effect of vaccination, or the extent of the virus virulence in a specific age group due to new mutations (World Health Organization, 2021). Table 1: Countries included in our study Number of people Median age Vaccination Covid-19 hospital bed (million) rate at occupancy at the the beginning beginning of 2022 per of 2022 million population Croatia 4.04 42.6 52.83 417 Cyprus 1.2 37.3 67.84 169 Luxembourg 0.63 40.4 68 110 USA 329.5 38 61.93 292 United Kingdom 67.22 40.5 69.79 210 Lithuania 2.79 45.1 68.36 435 Bulgaria 6.92 44.6 28 592 Czech Republic 10.7 42.5 62.43 300 Malaysia 32.37 30.26 78.24 81 Hungary 9.75 43.3 62.34 358 Netherlands 17.44 43.8 71.32 68 Norway 5.37 39.8 71.99 58 Poland 37.95 41.7 55.97 511 Slovakia 5.45 42.7 44.64 388 Switzerland 8.63 43.5 67.11 184 The three most popular RNN architectures namely SRNN, LSTM and GRU were tested. The ex- perimental codes were written with the Python language. The TensorFlow and Keras packages were mainly used for the Deep Learning (LSTM, SRNN, GRU) and the matplotlib package for visualizing the results. All the codes, including the database itself, are available on GitHub3. K-fold cross- validation was used to evaluate the model. The learning process was repeated 10 times under the three architectures LSTM, GRU, and SRNN (Table 2). For the optimization of the model hyper- parameters, we first tested the use of layers in a bottom-up manner (one layer then two etc.), the 3https://github.com/henibouhamed/COVID-19-omicron https://doi.org/10.15837/ijccc.2022.3.4697 5 results of the cross-validation and mainly the average R2 score (Pearson’s linear coefficient of determi- nation) continuously evolved up to 4 layers before stagnating and this is the reason why we adopted four layers for the model parameterization. A dropout was used after each layer to avoid overfitting. We also tested the four most used activation functions, namely the Relu, Sigmoid and Tangent, and the results in terms of average R2 score were largely favorable to the last. In order to validate the model, we initiated a large forecast of the occupancy of hospital beds by covid- 19 patients for two countries, Tunisia and France between January 7 and February 23 for Tunisia and between the above-mentioned dates for France during this period (period when the true occupancy values for these two countries are known), on the one hand, and detect the approximate regression date of the covid-19 hospital occupancy, on the other. We extended our investigations in order to evaluate the impact of vaccination on the overall evolution of hospital occupancy for these two countries. For Tunisia, which has a vaccination rate against covid-19 of around 50%, we predicted the hospital occupancy if it had the same vaccination rate as France (around 77%). For France we reversed the case; i.e. we have made the forecast of the hospital occupancy if it had the same rate of vaccination as Tunisia (around 50%). 4 Results and discussion In this context, many studies have tackled this issue over the last two years (Bouhamed 2020c, Findel and Suarez 2021, Caro et al 2021, Bekker et al 2022). They have focused mainly on the prediction of the number of cases, cures, deaths and even the use of health resources (Caro et al., 2021) according to the contagion rate and the old covid-19 strains severity of the old covid-19 strains. However, our model predict the evolution of the use of hospital beds by COVID-19 patients according to the high rate of contagion of the Omicron variant (Le page 2021) taking into consideration the massive vaccination campaign initiated in most countries and the doubts about the dangerousness of the Omicron variant in comparison with the old strains. To our knowledge, our work is the only one to propose this conduct. The test results of our model are very promising despite the short time period that generated the data. The average R2 score of the results of the 10 cross-validation iterations is very close to 1 with the GRU architecture (Table 2). The predictions of the test data for each iteration and the true values are very close (Figure 2). During training, the loss function that evaluates the deviation between the predictions made by the model and the true values of the observations used during training approaches 0 quickly in less than 400 epochs (Figure 3). Table 2: R2 average scores of our model for 10 iterations of training Architecture Average LSTM R2 0.979 GRU R2 0.996 SRNN R2 0.959 The prediction curve shape is very compact with that of the true values over the study period for Tunisia and France (Figures 4 and 5). Our model predicts the beginning of the regression of the use of hospital beds for covid-19 patients around February 18 for Tunisia and February 27 for France. Figure 6 shows and compares the evolution of hospital occupancy for Tunisia if it had the vaccination rate of France, i.e. 77% instead of 50%. Figure 7 shows and compares the evolution of hospital occupancy for France if it had the vaccination rate of Tunisia, i.e. 50% instead of 77%. In general, and regardless of the country, age, type of vaccine, number of doses received, and the state of herd immunity, the impact of vaccination is evident. This observation was verified through this comparison of Tunisia, which is a low-vaccination country but with considerable herd immunity due to lack of resources and adherence to barrier measures, and France, which is a well-vaccinated country. Moreover, the model prediction accuracy on hospitalizations for France is the best witness: Theoretically, in Tunisia, vaccination, in particular by the doses of immune reinforcement targeted the old people, with multiplicity of the types of received vaccines, whereas collective immunity concerns the young people, socially active workers, https://doi.org/10.15837/ijccc.2022.3.4697 6 Figure 2: Test data predictions for the first two cross-validation iterations with the true values in red and respective predicted one in blue with the coexistence of the two variants Delta and Omicron. In France, vaccination is always more advanced among the elderly and also concerns young people and even children under 18. So this proves that finally, the vaccination rate allows predicting the fate of each country in terms of severe forms, whatever the age, viral variant, and the other interfering factors are. Figure 3: Evolution of the loss function (of train data in yellow and of test data in blue) with GRU, LSTM and SRNN architectures Figure 4: Tunisia Covid19 Hospitalization prediction and real sub-values (07 to 18 January 2022) 5 Conclusion and perspectives We have developed a model to predict bed occupancy of COVID-19 patients by country during the rapid spread period of the Omicron variant. This model can be used to make decisions, first, on the introduction or alleviation of restrictive measures and, second, the prediction of oxygen and health human resources needs without deteriorating the quality of service for other patients. We tested some https://doi.org/10.15837/ijccc.2022.3.4697 7 Figure 5: France Covid19 Hospitalization prediction and real sub-values (07 to 18 janary 2022) Figure 6: Tunisia Hospitalization prediction: real vaccination rate Vs 77% vaccination rate (France vaccination rate) recurrent deep learning architectures. For the model training, we relied on hospitalization data from 15 countries whose curves started regressing. The studied period covers the month of December 2021 and the beginning of January 2022 which represents the period of strong contagion of the omicron variant in the world. The sequences of changes in hospital occupancy, vaccination rates and median ages of the populations were used to train our model. The results were very interesting: first and foremost, they allowed us to predict the dates of the approximate descent of bed occupancy, and second, they enabled us to feel the positive impact of vaccination on the covid-19 hospital occupancy in France and Tunisia which could help better manage the pandemic peak caused by the Omicron variant. In the future, we plan to develop a real-time data streaming application that will allow us to retrieve Figure 7: France Hospitalization prediction: real vaccination rate Vs 50% vaccination rate (Tunisia vaccination rate) https://doi.org/10.15837/ijccc.2022.3.4697 8 data and automatically integrate them into the learning base in order to make the model evolve according to the latest developments. Acknowledgment This research was funded by Princess Nourah bint Abdulrahman University Researchers Supporting Project number (PNURSP2022R125),Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia. References [1] Rosenbaum, L. (2020). Facing Covid-19 in Italy-ethics, logistics, and therapeutics on the epi- demic’s front line, New England Journal of Medicine, 382(20), 1873-1875, 2020. [2] Jason, P.; Li, W.; Lowell, L.; Moritoki, E.; Chae-Man, L.; Jigeeshu, D.; Babu, S.; Yaseen, A.; Jensen, N.; Charles G.; Masaji, N.; Younsuck, K.; Bin, D. (2020). Asian Critical Care Clinical Trials Group, Intensive care management of coronavirus disease 2019 (COVID-19): challenges and recommendations, The lancet respiratory medicine, 8(5), 506-517, 2020. [3] Bekker, R.; Michiel, B.; Ger, K. (2022). Modeling COVID-19 hospital admissions and occupancy in the Netherlands, European journal of operational research, In press, 2022. [4] Le Page, M. (2021). The omicron wave, NewScientist, 252(3364), 7, 2021. [5] Strategists, G. (2020). COVID goes Cuckoo. How the March-April 2020 COVID-19 surge over- whelmed Dutch hospitals and undermined regular care, gupta-strategists.nl, 2020. [6] Bouhamed, H. (2020). Covid-19 cases and recovery previsions with deep learning nested sequence prediction models with long short-term memory (LSTM) architecture, International Journal of Scientific Research in Computer Science and Engineering, 8(2), 10-15, 2020. [7] Bouhamed, H. (2020). COVID-19, Bacille Calmette-Guerin (BCG) and tuberculosis: cases and re- covery previsions with deep learning sequence prediction, Ingenierie des Systemes d’Information, 25(2), 165-172, 2020. [8] Bouhamed, H. (2020). COVID-19 Deaths Previsions With Deep Learning Sequence Prediction: Bacille Calmette-Guérin (BCG) and Tuberculosis Track, International Journal of Big Data and Analytics in Healthcare (IJBDAH), 5(2), 65-77, 2020. [9] Findl, J.; Javier, S. (2021). Descriptive understanding and prediction in COVID-19 modelling, History and philosophy of the life sciences, 43(4), 1-31, 2021. [10] Jaime, C.; Jörgen, M.; Vatshalan, S.; Harpreet, G.; Jessica, J.; Kariem, E.; Ramai, S.; Paul, K.; Alistair, M. (2021). Predicting hospital resource use during COVID-19 surges: a simple but flexible discretely integrated condition event simulation of individual patient-hospital trajectories, Value in Health, 24(11), 1570-1577, 2021. [11] Bekker, R.; Michiel uit het, B.; Ger, K. (2022). Modeling COVID-19 hospital admissions and occupancy in the Netherlands, European journal of operational research, In press, 2022. [12] Bouhamed, H.; Ruichek, Y. (2018). Deep feedforward neural network learning using Local Binary Patterns histograms for outdoor object categorization, Advances In Modelling And Analyses B, 61(3), 158-162, 2018. [13] Hinton, G.; Deng, L.; Yu, D.; Dahl, G. E.; Mohamed, A.; Jaitly, N.; Senior, A.; Vanhoucke, V.; Nguyen, P.; Sainath, T. N.; Kingsbury, B. (2012). Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, IEEE Signal processing magazine, 29(6), 82-97, 2012. https://doi.org/10.15837/ijccc.2022.3.4697 9 [14] Mohamed, A.; George, E. D.; Geoffrey, H. (2011). Acoustic modeling using deep belief networks, IEEE transactions on audio, speech, and language processing, 20(1), 14-22, 2011. [15] Cires, D. C.; Meier, U.; Gambardella, L. M.; Schmidhuber, J. (2010). Deep, big, simple neural nets for handwritten digit recognition, Neural computation, 22(12), 3207-3220, 2010. [16] Yu, D.; Li, D. (2010). Deep learning and its applications to signal and information processing [exploratory dsp], IEEE Signal Processing Magazine, 28(1), 145-154, 2010. [17] Bengio, Y. (2009). Learning deep architectures for AI, Now Publishers Inc, 2(1), 1-56, 2009. [18] Zikopoulos, P.; Chris, E. (2011). Understanding big data: Analytics for enterprise class hadoop and streaming data, McGraw-Hill Osborne Media, 2011. [19] Brownlee, J. (2017). Long short-term memory networks with python: develop sequence prediction models with deep learning, Machine Learning Mastery, 2017. [20] Hochreiter, S.; Schmidhuber, J. (1997). Long short-term memory, Neural computation, 9(8), 1735- 1780, 1997. [21] Fernandez, A.; Bertolami, R.; Bunke, H.; Schmiduber, J. (2009). A novel connectionist system for improved unconstrained handwriting recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5), 855-868, 2009. [22] Li, X.; Xihong, W. (2015). Constructing long short-term memory based deep recurrent neural net- works for large vocabulary speech recognition, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4520-4524, 2015. [23] Gers, F. A.; Schraudolph, N.; Schmidhuber, J. (2002). Learning precise timing with LSTM re- current networks, Journal of machine learning research, 3(1), 115-143, 2002. [24] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio. (2014). Learning phrase representations using RNN encoder- decoder for statistical machine translation, arXiv preprint arXiv:1406.1078, 2014. Copyright ©2022 by the authors. Licensee Agora University, Oradea, Romania. This is an open access article distributed under the terms and conditions of the Creative Commons Attribution-NonCommercial 4.0 International License. Journal’s webpage: http://univagora.ro/jour/index.php/ijccc/ This journal is a member of, and subscribes to the principles of, the Committee on Publication Ethics (COPE). https://publicationethics.org/members/international-journal-computers-communications-and-control Cite this paper as: Bouhamed, H.; Hamdi, M.; Gargouri, R. (2022). Covid-19 Patients’ Hospital Occupancy Prediction During the Recent Omicron Wave via some Recurrent Deep Learning Architectures, International Journal of Computers Communications & Control, 17(3), 4697, 2022. https://doi.org/10.15837/ijccc.2022.3.4697 Introduction RNN, SRNN, LSTM and GRU Data and methods Results and discussion Conclusion and perspectives