Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 5, No. 3, 2023 150 SafeCross: Helping Elder People Cross the Road Timely Using Thermal Imaging and Image Recognition Leyan Yang * School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China * Corresponding author Email: yangleyanchengdu@gmail.com Abstract: Some older adults face challenges of cross the road in time due to reduced mobility and declining sensory abilities. Existing technologies have limitations in accurately identifying pedestrians. We propose SafeCross, a novel system that combines image recognition and thermal imaging technology based on the FLIR TrafiOne 156 sensor, which accurately detects elderly individuals at road intersections in low visibility or obstructed conditions and implement traffic light control based on the algorithm provided by SafeCross. Our findings demonstrate the potential of SafeCross system ensures safety for older adults at intersections, Making a significant contribution to reducing the accident rate among the elderly. Keywords: Thermal Imaging; Image Recognition; Traffic Safety; Identify the Elders. 1. Introduction By the year 2050, aging populations are expected to cover 27% and 15% of developed and developing nations, respectively, resulting in an increased need for specialized care and support for the elders [1]. According to World Health Organization (WHO) data, approximately 1.3 million people's lives are claimed by road traffic accidents worldwide each year. Reduced mobility and declining sensory abilities making simple tasks like crossing the road in time a potential hazard to the elders. Relying solely on simple pedestrian crossings and traffic lights may not provide sufficient assistance for elderly individuals, especially those with mobility challenges, Therefore, the issue of elderly people's travel has attracted considerable attention from researchers. Existing technologies, such as pedestrian recognition based on monocular cameras Convolutional Neural Network (CNN), have limitations in terms of accuracy and stability [2]. These methods do not take into account the impact of constantly changing traffic conditions and unstable weather on sensors or are limited to data collection without considering practical solutions to address the problem. Therefore, there is an urgent need for innovative solutions to ensure the safety of older adults when crossing the road. In order to address the issue of stable and accurate identification of elderly individuals and provide them with protection. This research combines image recognition with thermal imaging technology using the FLIR TrafiOne 156 sensor to create a system that accurately detects the presence of elderly individuals at road intersections. By analyzing real- time images and thermal data, SafeCross identifies if there are elderly individuals who have not been able to cross the road in time when the pedestrian green light ends. It also takes into consideration factors such as traffic volume, vehicle speed, and weather conditions to appropriately adjust the duration of the traffic signal, aiding older adults in safely crossing the road. Additionally, SafeCross provides real-time guidance and warnings during traffic light transitions to ensure that older adults are aware of the changing traffic signals. It offers several advantages over existing methods. The integration of image recognition and thermal imaging technology provides a more comprehensive and accurate detection system. By combining visual data with thermal information, SafeCross achieves a recognition accuracy of 98.52% under normal visibility conditions. It also detects older adults with an accuracy of over 82% in low visibility or occluded situations caused by other objects. In addition, the article proposes a real-time algorithm which can calculate time adjustment strategies within 0.3 seconds based on the position of elderly people who are unable to cross the road in time and the traffic congestion situation. The primary contribution of this research includes the following: 1. SafeCross achieves higher accuracy in recognizing older adults. 2. Proposes a real-time algorithm which ensures the safety of the elderly while minimizing disruptions to vehicular traffic on the road segment. 3. Detection experiments were conducted on over 200 elderly individuals at 6 traffic light intersections, followed by data analysis. 2. Related Works 2.1. Assistive Technologies for the Elders With the continuous development of cities and the deepening issue of aging, scholars have given sufficient attention to the safety, health, and assistance problems of the elderly. Social accessible facilities and services have also been continuously improved. Scholars such as Athanasios Bamis and Zongfang Lin utilize intelligent sensors to detect the activities of the elderly, ensuring the safe monitoring of their indoor activities and potentially assisting in their independent living [3,4]. Additionally, researchers have developed assistive devices like smart wheelchairs to enhance the daily life quality of the elderly [5]. The SafeCross system, designed in this study, also provides services for the elderly in terms of traffic safety through intelligent technological means. 2.2. Traffic Safety Technique for the Elders Based on existing technologies, we can utilize network surveillance to identify pedestrians, analyze their positions and surroundings, thereby addressing pedestrian safety concerns [2]. Additionally, researchers have proposed the use 151 of wearable display devices attached to clothing, to enhance visibility for vulnerable groups in traffic and improve overall traffic safety [6]. It is evident that researchers place a significant emphasis on traffic safety, particularly for vulnerable groups on the road. This study aims to build upon previous research by specifically identifying and safeguarding the traffic safety of the elderly when crossing the road without relying on wearable devices which are not elderly-friendly. 2.3. Thermal Imaging and Image Recognition Device The feasibility of thermal imaging in user recognition has been extensively validated through studies on foot-based user recognition, human pose recognition and detection using drones, and automatic detection of human bodies in thermal imaging video images [7-9]. These technical explanations provide ample evidence for implementing recognition of the elderly population in this research. The algorithms for recognizing human motion from image recognition videos can also assist in accurately identifying elderly individuals from a crowd [10,11]. This study has successfully integrated thermal imaging recognition and image recognition, thereby enhancing the system's recognition accuracy and stability under extreme weather conditions. 3. Concept and Methodology 3.1. Integration of Thermal Imaging and Image Recognition Image recognition technology is a technique that extracts meaningful features from captured images using specific algorithms and matches them with known samples for identification. The performance of image recognition relies on the quality of the images and the design of the algorithms. Thermal imaging is a technology that utilizes infrared detectors to detect the infrared radiation emitted from the surface of objects and convert it into corresponding electrical signals, which are then processed into visual images. This study integrates image recognition and thermal imaging technology using sensors to overcome their respective limitations. The specific integration approach in this study is as follows: Data collection and training: Firstly, a large amount of thermal imaging and image data needs to be collected, including various scenarios and poses of elderly pedestrians crossing the road. This data will be used to train models for image recognition and thermal imaging analysis. Fusion of image recognition and thermal imaging: Utilizing deep learning and computer vision techniques, the collected image data can be analyzed and processed. With trained models, elderly pedestrians can be accurately detected and recognized, and their position and motion information can be extracted. Additionally, thermal imaging technology can capture the heat distribution of their bodies, further confirming their presence and location. Traffic signal control: Based on the results of image recognition and thermal imaging, SafeCross can continuously monitor the number and positions of elderly pedestrians on the road. When an elderly pedestrian is ready to cross, the system can automatically adjust the traffic signals to extend the green light duration, ensuring they have enough time to safely cross. Figure 1. Thermal image of pedestrians and cars Figure 2. Comparison of simple image recognition and combination of image recognition and thermal imaging Real-time monitoring and alerts: The SafeCross system can emit sound. When the traffic signal changes, the system will emit real-time audio signals to alert elderly individuals. When detecting an elderly pedestrian crossing the street, it can send an alarm signal to traffic participants, reminding them to be cautious of pedestrians. 152 Continuous improvement and optimization: By continuously collecting and analyzing data, the system can undergo ongoing improvements and optimizations. For example, by analyzing the behavior patterns and habits of elderly pedestrians, the system can predict their intentions and accordingly adjust the traffic signals to provide better safety measures. By combining thermal imaging and image recognition technology, the SafeCross system can offer accurate detection and recognition of elderly pedestrians, ensuring their safety while crossing the road. 3.2. Algorithms Design The system requires extensive algorithm support for detection, computation, and real-time decision-making. The algorithms we need are as follow: Object Detection Algorithms: Object detection algorithms, such as the Faster R-CNN or YOLO (You Only Look Once) algorithms, can be used to detect and localize objects of interest in images. These algorithms can identify the presence and location of elderly pedestrians in real-time. Heatmap Analysis Algorithms: Thermal imaging data can be processed using various algorithms to generate heatmaps. These heatmaps represent the distribution of heat energy in the captured scene. By analyzing the heatmaps, it is possible to identify areas with high heat signatures corresponding to the presence of human bodies, including elderly pedestrians. Tracking Algorithms: To track the movements of elderly pedestrians, tracking algorithms Kalman filters or particle filters are used. These algorithms can predict the trajectory and position of pedestrians over time, allowing the system to adjust traffic signals accordingly. Reinforcement Learning Algorithms: Reinforcement learning algorithms can be employed to continuously optimize the system's performance based on feedback and rewards. The system can learn to make better decisions, such as adjusting traffic signals, by maximizing the safety and efficiency of elderly pedestrians crossing the road. Delay Time Algorithm based on Real-time Location of Elderly Individuals and Traffic Congestion: SafeCross will calculate the real-time delay time needed for elderly individuals to cross the road based on their location at the end of the green light and real-time traffic flow data. For example, if the elderly person has a short distance to travel, a longer delay time will be given, and the duration of the next vehicle movement will be increased appropriately based on the traffic conditions to avoid causing road congestion. 4. Data Analysis 4.1. Experimental Design and Procedure A one-month experiment was conducted in six intersections with different traffic volumes in the city to set up the SafeCross system. During this period, data information on pedestrian conditions, traffic congestion, signal light delay, and other factors were collected. 4.2. Result and Analysis During the experimental period from August 20th 2023 to September 21st 2023. In this period of A total of 238,229 pedestrian information has been collected, identifying 2,329 elderly people who have difficulties crossing the road in a timely manner. Signal light delays have been implemented to address this issue. Among them, 92.8% of the delays can assist the elderly in crossing the road promptly without causing traffic congestion. 5. Conclusion, Limitation and Future Work By combining real-time image analysis and thermal data interpretation, the SafeCross system demonstrates a remarkable recognition accuracy of 98.52% under normal visibility conditions, while maintaining an impressive 82% accuracy in challenging scenarios marked by low visibility or occlusions. Furthermore, the proposed real-time algorithm, capable of making swift adjustments to traffic signal durations based on elderly individuals' positions and traffic conditions, showcases an effective means to ensure pedestrian safety without unduly disrupting vehicular flow. Through comprehensive detection experiments involving over 200 elderly participants across six different traffic light intersections, the system's performance was meticulously evaluated and subsequently analyzed. This extensive empirical validation underlines the viability and effectiveness of the SafeCross system in addressing the critical safety concerns associated with elderly road crossing. However, due to the utilization of sophisticated sensor equipment, our system carries a higher cost compared to conventional devices, posing challenges for its widespread implementation. Furthermore, ethical issues such as the responsibility associated with potential traffic safety incidents arising from errors in real-time adjustments of traffic signal durations need to be further deliberated and contemplated before the system can be effectively deployed. But this research not only fills a significant gap in existing pedestrian recognition technologies but also provides a viable pathway toward fostering safer and more inclusive urban mobility for the elders. As societies around the world continue to grapple with the implications of aging populations, the contributions of this study hold great promise for enhancing the quality of life and overall safety of seniors, redefining the landscape of road intersection safety for the years to come. References [1] Sarcar, S., Joklnen, J., Oulasvirta, A., et al. (2016) Towards Ability-Based Optimization for Aging Users. In: Proceedings of the International Symposium on Interactive Technology and Ageing Populations (ITAP '16). New York. pp. 77–86. [2] Xu, W.J., Ruiz-Juri, N., Huang, R.Z., et al. (2018) Automated pedestrian safety analysis using data from traffic monitoring cameras. In: Proceedings of the 1st ACM/EIGSCC Symposium on Smart Cities and Communities (SCC '18). New York. pp. 1- 8. [3] Bamis, A., Lymberopoulos, D., Teixeira, T., et al. (2008) Towards precision monitoring of elders for providing assistive services. In: Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments (PETRA '08). New York. pp. 1-8. [4] Lin, Z.F., Hanson, A.R., Osterweil, L.J., et al. (2011) Precise process definitions for activities of daily living: a basis for real- time monitoring and hazard detection. In: Proceedings of the 3rd Workshop on Software Engineering in Health Care (SEHC '11). New York. pp. 13-16. [5] Wong, S.F., Hoi, W.K., Wan, I.K. (2018) Advanced Smart Wheelchair Design in Enhancing Quality of Life for Elder and Handicapped People. In: Proceedings of the 4th International 153 Conference on Industrial and Business Engineering (ICIBE' 18). New York. pp. 203-210. [6] Grosse-Puppendahl, T., Bechtold, O., Strassel, L., et al. (2015) Enhancing traffic safety with wearable low-resolution displays. In: Proceedings of the 2nd international Workshop on Sensor- based Activity Recognition and Interaction (iWOAR '15). New York. pp. 1-10. [7] Saad, A., Izadi, K., Khan, A.A., et al. (2023) HotFoot: Foot- Based User Identification Using Thermal Imaging. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). Hamburg. pp. 1-13. [8] Park, D.M., Yeom, S. (2022) Human Detection and Posture Estimation with IR Thermal Images by a Drone. In: Proceedings of the 2021 4th Artificial Intelligence and Cloud Computing Conference (AICCC '21). New York. pp. 115-121. [9] Ivašić-Kos, M., Krišto, M., Pobar, M. (2019) Human Detection in Thermal Imaging Using YOLO. In: Proceedings of the 2019 5th International Conference on Computer and Technology Applications (ICCTA '19). New York. pp. 20-24. [10] Sethi, R.J., Roy-Chowdhury, A.K. (2010) The human action image and its application to motion recognition. In: Proceedings of the Seventh Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP '10). New York. pp. 1-8. [11] liu, S., Liu, J., Zhang, T.Z., et al. (2009) Human action recognition in videos using motion impression image. In: Proceedings of the First International Conference on Internet Multimedia Computing and Service (ICIMCS '09). New York. pp. 174-178.