International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 22, 2021 Paper—A New Covid-19 Tracing Approach using Machine Learning and Drones Enabled… A New Covid-19 Tracing Approach using Machine Learning and Drones Enabled Wireless Network https://doi.org/10.3991/ijim.v15i22.22623 Ayoub Alsarhan(), Islam T. Almalkawi, Yousef Kilani Hashemite University, Zarqa, Jordan ayoubm@hu.edu.jo Abstract—The continuous advancements in wireless network systems have reshaped the healthcare systems towards using emerging communication technol- ogies at different levels. This paper makes two major contributions. Firstly, a new monitoring and tracking wireless system is developed to handle the COVID-19 spread problem. Unmanned aerial vehicles (UAVs), i.e., drones, are used as base stations as well as data collection points from Internet of Things (IoT) devices on the ground. These UAVs are also able to exchange data with other UAVs and cloud servers. Secondly, this paper introduces a new reinforcement learn- ing (RL) framework for learning the optimal signal-aware UAV trajectories under quality of service constraints. The proposed RL algorithm is instrumental in making the UAV movement decisions that maximize the signal power at the receiver and the data collected from the ground agents. Simulation experiments confirm that the system overcomes conventional wireless monitoring systems and demonstrates efficiency especially in terms of flexible continues connectiv- ity, line-of sight visibility, and collision avoidance. The results show that the proposed healthcare system is feasible and innovative. Furthermore, the system shows good performance under different conditions. Keywords—contact tracing, UAVs, Covid-19, wireless monitoring system, wireless mesh networks, reinforcement learning 1 Introduction The most important step in the fight against COVID-19 pandemic is to prevent the spread of the disease [1]. Wireless communication technology has been instrumental in healthcare system at different levels [3–5]. Wireless technology enables real-time mon- itoring of patients and connect different units in the healthcare system in unprecedented fashions. In our work, the aim of the proposed wireless monitoring system (WMS) is monitoring persons who are infected with COVID-19 to prevent transferring the dis- ease. Contact tracing (CT) is defined as the process of specifying the persons (contacts) who come close to an infected person [1, 2]. Owing to the possibility of transmission, contacts should be tested for infection. In order to reduce the likelihood of transmis- sion, each person should be isolated if the test result is negative [1, 2]. However, the person should be treated and isolated if his/her test result is positive [1, 2]. iJIM ‒ Vol. 15, No. 22, 2021 111 https://doi.org/10.3991/ijim.v15i22.22623 mailto:ayoubm@hu.edu.jo Paper—A New Covid-19 Tracing Approach using Machine Learning and Drones Enabled… IoT devices are traced in our system to identify persons who contact infected person. Furthermore, they are used for collecting data and exchanging information with mesh routers (i.e. MRs). For areas with insufficient communication infrastructure the UAVs are used to conduct CT. Moreover, UAVs can be used to provide continuous wireless internet communication in such areas [5]. UAVs can be used for many assistance roles in complex communication scenarios and play as the aerial relays to support terrestrial communications. Dynamic deployment of UAVs is a key challenge that needs to be tackled in order to improve the autonomy of UAVs and enabling them for taking part in more complex mission. In our work, we propose RL to enable UAVs to discover knowledge for extracting efficient UAV trajectory. However, intelligent UAV trajectory maximizes the received signal power and satisfies the MR’s minimum data rate without requiring outside instructions. Our wireless contact tracing (WCT) scheme attempts to extract the opti- mal UAVs’ locations for given zone area and given transmitted signal power that maxi- mizes the received power. CT is carried out on the ground and the collected data is sent to UAVs. 5G networks allow integration of different wireless networks such as cellular net- work, Bluetooth, and ad hoc networks [6, 7]. 5G provides a range of different commu- nication technology such as wired network, wireless network, infrared, and optical. For example, short-range wireless network (SRWN) can be built using various IoT devices such as mobile phones, laptops, sensors… etc. [7]. Besides extending the mobility pro- vided by traditional network, these SRWNs enable contact tracing system to obtain wide-range coverage of tracing, high accuracy of tracing results, and high level of trac- ing details. In order to prevent the transmission of COVID-19, an alert message should be sent to all persons who may have come into contact with an infected person at the right time [1, 2]. In order to cope with the limitations of the conventional wireless monitoring system, our contact tracing system made up of ground sensors and UAVs. Ground sensors (i.e. IoT devices) monitor the people in crowded places and acquiring information in an intelligent manner. These sensors are combined with UAVs which are used as data col- lector. UAVs are used to provide long distance communication. Furthermore, they are used to collect data over large area within short time and disseminating the situation of the target area to MRs in other zones. There are several methods which can be used to establish wireless network with UAVs. These methods include cellular network, Bluetooth, and Wi-Fi. Although cel- lular network provides communication over wide area, it has some drawbacks such as interference, noise, and handoff. Bluetooth can be used efficiently specially for short distance communication [7]. However, Bluetooth is not suitable for transmitting large volume of data such as multimedia files [7]. It is hard to avoid interference in Wi-Fi since the medium is shared between all nodes. Recently, wireless mesh networks (WMNs) have emerged [8] to extend internet access and other networking services in public transportation. Furthermore, the network provide network access to the remote clients and other networking functions such as routing, and packet forwarding. In our work, MR searches the database to identify the 112 http://www.i-jim.org Paper—A New Covid-19 Tracing Approach using Machine Learning and Drones Enabled… infected persons. After that, it sends a list of patients to all users. The system broadcasts alert messages for users to leave the place if infected person is detected. The attributes for each close physical contact (CPC) are recorded in the database if physical distance between persons is less than certain threshold (i.e. 1.5 m). These attributes include: the telephone number for each person, time of CPC, date of CPC, and distance between the persons. Furthermore, the database contains appropriate actions such as calling via phone an emergency number, and informing infected persons to quarantine themselves. Each action taken is preferably responsive to the particular individual uniquely iden- tified by the identifying signal. The major contributions of this paper are summarized as follows: 1. A new architecture is proposed for WMS where an intelligent effective wireless mesh technology and UAVs are integrated to build a wide-range wireless network for CT in wide area. The proposed WMS is IoT networks that consists of a massive number of heterogeneous end user devices. These devices include: smart phones, sensors, electronic gadgets and wearables, household electrical, and anymore. The system offers an efficient and reliable connectivity that enables decision-makers to take preventative action for breaking the chains of COVID-19 transmission by exchanging warning messages to inform the contacts of infected cases as quickly as possible about the risk of infection, in order to take the right steps in a timely manner. 2. Developing an automated, and reliable method to select the values of the configura- tion parameters where new intelligent RL based algorithm is suggested for optimal configuration of UAVs. The main concern of the proposed algorithm is enabling UAVs to provide wireless coverage for mobile users by maximizing the total received power while satisfying WMS constraints. 3. Analyzing the performance of the RL configuration model under different network conditions. The rest of the paper is organized as follows: Section 2 briefly surveys the rele- vant work; our assumptions and work environment are shown in Section 3; Section 4 describes our scheme for contact tracing and configuring the monitoring wireless net- work. The performance results of our scheme are evaluated in Section 5. Finally, the paper is concluded and future research directions are given. 2 Related work Due to a lack of surveillance system for controlling Covid-19 pandemic, many patients are dying. A system that provides continuous health monitoring and emer- gency reporting is required to assist them. WMS is a reliable technology for monitor- ing and ensuring the safety especially for dealing with COVID-19 pandemic. Besides providing accurate, rapid, and low-cost, WMS supports real-time data gathering from multiple sensors. Drones have been used in WMS to guarantee connectivity and to sup- port long-distance peer-to-peer communication. They are routinely used in healthcare iJIM ‒ Vol. 15, No. 22, 2021 113 Paper—A New Covid-19 Tracing Approach using Machine Learning and Drones Enabled… monitoring systems in many countries. In [9], new monitoring system for patient at home has been proposed to reduce the cost of monitoring patients and to allow a patient to get a full range of services at home. Networks of IoT were used to monitor patients remotely. Firstly, the network collects healthcare data from patients using sensors and actuators. After that, it sends data to a Cloud-based Hospital Information System (HIS) for processing. In [9], new system was proposed for a remote health monitor- ing system based on IoT. Several technologies have integrated in this system to pro- vide up-to-date health monitoring systems in smart environment. These technologies include: IoT, cloud computing, smart environment, and virtual machines. In [10], new healthcare system was proposed to overcome the delay with traditional WMS. A model for monitoring and processing the received data was used. Further- more, the patients’ health data are analyzed to identify anomalies. In [11], telecommu- nication techniques were used to develop new device that collect the patients’ health data and transfer the data to a remote device wirelessly. These data includes: a patient’s body temperature, heart rate, and electrocardiography. In [12], new healthcare system was proposed to collect patients’ data such as: weight, height, and pulse rate. The data are transferred to a device that analyzes the data to generate final report of the patient health status. Many wireless devices are used in this system. These devices include: microcontroller, height sensor, weight sensor, and pulse sensor. Three sensors were used in the proposed system in [13]. These sensors include: temperature sensor, and heartbeat sensor. The temperature of patient is measured using temperature sensor. The system sends alert to the processor if the heartbeat of the patient changes from normal rate. Authors proposed a new framework in [14] to collect patients’ data in real time. Furthermore, the proposed system suggests medical whenever needed. The framework integrates different technologies such as cloud tech- nology, sensors, and mobile devices. The collected data are stored in cloud to make the data available for physicians, paramedics, or any other authorized entity. New framework was proposed in [15] for developing health monitoring system. The main concern of the system is providing extensible and usable services for perva- sive patient care. Sensors are used to collect patient’s physiological parameters in this system. Beside analyzing patient’s data, smart phones are used to send the results to a medical center system. A home-based wireless monitoring system was considered in [16] for monitoring patient in their own home. Zigbee technology was used to gather data. These systems can continuously collect patient’s physiological parameters and offer further analysis and interpretation. Authors proposed a novel, IoT-aware, smart architecture for automatic healthcare monitoring system in [17]. The system can track patients anywhere and anytime. Besides collecting patients’ physiological parameters in real time, the system can gather environmental conditions. The collected data is fed into a control center for processing the data. In [18], new scheme was proposed to optimize the transmission rate, the transmis- sion power, and the allocated time slots for each sensor in the healthcare monitoring system. Furthermore, authors proposed new sub-optimal resource allocation to min- imize the time complexity of the resource allocation. In order to support active and assisted living health care services and applications, authors proposed new technique that integrates IoT and non-interoperable IoT platforms for monitoring patients in [19]. 114 http://www.i-jim.org Paper—A New Covid-19 Tracing Approach using Machine Learning and Drones Enabled… The main concern of the proposed technique is detecting and correcting wrong life- styles or critical situations quickly. New IoT enabled wearable device was designed in [26] to send notifications when any of the monitored parameters falls outside of the typical range. In [27], authors proposed new application-based temperature monitoring system. The system displays real-time temperature data. In [28], authors proposed new nonprescription drugs mobile health application (NMMHA) for helping patients in the initial medication. However, most of these studies assumed that patients are located indoor. More specifically, all previous studies neglect monitoring patients in outdoor environment. Therefore, in our work, we study integrating UAVs with wireless mesh network for the COVID-19 remote patient monitoring service and enable developing alerting system to inform at-risk people (i.e. contacts) about what to do for contact. Therefore, we propose using UAV to enhance coverage, reliability, and connectivity for monitoring system. The problem is formulated as maximization of signal power at receiver to cover the entire zone. 3 System model We present our assumption in this Section. Each MR is responsible for health and safety at certain subarea in the context of COVID-19. The dimension of each subarea is 250 m X 250 m. The network consists of three types of nodes: MRs, mesh clients (MCs), and UAVs. While MRs have fixed locations, MCs and UAVs are moving and changing their places arbitrarily. On the ground, MCs (i.e. IoT devices) send the col- lected data about CT to MRs. Each node is equipped with a single IEEE 802.11b based transceiver. The spectrum is partitioned into non-overlapping channels (16 channels with 5 MHZ spacing with transmission and power mask restrictions similar to the ISM band). MC could be referred to as the integration of sensors, actuators, smart phones, vehicles, and any wireless system which can gather data and fed it to MR. The main principle behind MCs in our work is that the objects connected via the Internet can col- lect data with the help of existing technologies and the collected data would be sent to MRs. Each UAV searches for MR in its range to make WMS. UAVs can move dynam- ically to collect data from the MRs using uplink communication links. Basically, UAVs are used as base stations. Assume (xi, yi, zi) is 3D location of the i th UAV. We assume that MRs are distributed in 3D space and (xj, yj, zj) is the location of j th MR. The distance between ith UAV and jth MR is computed as follows: d x x y y z zi j i j i j= − + − + −( ) ( ) ( ) 2 2 2 (1) The path loss model in [18] is used in our work and it is computed as follows: P P P PF B I= + + (2) where PF is the free space path loss, PB is the building penetration loss, and PI is the indoor loss. Figure 1 illustrates the architecture of the proposed WMS. We assume that each MR has a unique ID, which can be the MAC address of the node. iJIM ‒ Vol. 15, No. 22, 2021 115 Paper—A New Covid-19 Tracing Approach using Machine Learning and Drones Enabled… Fig. 1. WMS architecture 4 Covid-19 drone-based wireless monitoring system In this section, we propose a drone-based wireless monitoring system. In this sys- tem, drones configure a wireless mesh network to enable monitoring CT in wide area. The proposed system is shown in Figure 1 where set of drones are distributed over regular zones to establish wireless network with drones serving as wireless relays for improving connectivity and coverage of ground IoT devices. 4.1 Signaling protocol for CT In this section, we introduce the signaling protocol for WCT. Contact tracing is con- ducted in two levels: local (zone), and global (the whole area) level. In our scheme, tracing is managed as follows: Step 1: Every MC (i.e. IoT device) collects physical distancing measures and users’ data. Step 2: All MCs send their distancing measures and data to MR. Step 3: MR combines results from all MCs and generates a final contact tracing’s status. Step 4: MRs exchange zone’s status using UAVs and then a final status of area is extracted at each MR. Step 5: A new status for each zone is broadcasted to all MRs using UAVs. In our system, Markov Decision Process (MDP) is used to extract optimal policy for controlling UAV’s location. This problem can be formulated a follows: find the optimal UAV place, which maximizes the signal power at receiver. The received signal power is defined as follows [20]: 116 http://www.i-jim.org Paper—A New Covid-19 Tracing Approach using Machine Learning and Drones Enabled… S S Pd r t= − α (3) where Sd r is the received power at distance d, α is the cost of increment of path loss P, and St is the transmitted power. The main concern of our policy is determining the optimal location of UAV such that the transmitted signal can be detected clearly by MR. This problem can be formulated as follows: max , ,x y z d rS (4) s.t. ω ρ≤