Academic Journal of Science and Technology ISSN: 2771-3032 | Vol. 1, No. 1, 2022 1 Research on Human Skeleton Teaching System Based on Leap Motion Yehua Feng1, a 1Majoring in Forensic Medicine, School of Basic Medicine, Hebei North University, Zhangjiakou, Hebei, 075000, Chain afengyehuaz@163.com Abstract: With the development of virtual reality technology, virtual teaching technology continues to improve, virtual reality technology is more and more widely used in medicine. Virtual reality technology can improve teaching efficiency by simulating real scenes with high reality. Human learning is very necessary for a medical professional, however due to the limitation of teaching resources, teaching things make teaching process insurmountable problems, therefore, in this paper, based on the Leap Motion of human body skeleton virtual teaching system, through computer modeling to simulate human body skeleton, the human-computer interaction technology based on gesture recognition algorithm, Realize human skeleton simulation teaching. Finally, the experiment proves that the virtual assistant teaching system can obviously improve the learning efficiency of learners. Keywords: Virtual Reality, Human skeleton, Teaching, Medicine. 1. Introduction Virtual Reality, also known as spiritual environment, is the combination of simulation technology, computer graphics, human-computer interaction technology and other research directions, including computer, electronic information, simulation technology, its basic realization method is computer simulation of Virtual environment to give people a sense of environmental addiction. The information interaction between human and the real environment is simulated by multi-sensory feedback human-computer interaction technology. Virtual reality technology has been widely used in industrial manufacturing, military, education, medicine and other fields, especially in the medical field, it provides a new method for training interns. Virtual reality technology has played a driving role in surgery, pathological analysis, tissue and organ analysis and other aspects. David Weilk et al. [1] proposed the application of Virtual Reality technology in the training of hip surgery, developed a tactile reamer that can simulate vibration by integrating electronic components, and integrated tactile feedback into the acetabulum reamer through vibration and sound to simulate hip surgery and improve the training process of surgeons. Lizeth Vega-Medina et al. [2] proposed the Neonatal venous access simulation system, which integrates the execution program, the neonatal model based on image projection and the real tactile feedback module, so as to guide surgeons in surgery. Jonas Schild et al. [3] proposed a multi- user virtual reality collaborative medical training system, which builds a dual-user full-size VR environment, and the head-mounted display can be used by two participants at the same time, providing novel training tools for medical personnel and enhancing the current learning methods. Mingwei Cai et al. [4] proposed the virtual medical support system, which demonstrated and explained the operation state in the process of surgical teaching through VR support system, making it easier for students to understand and reduce the fatigue and distraction of long-term learning. Philipp Stefan et al. [5] proposed a medical augmented reality system for bone anatomy learning to improve learning efficiency through intuitive 3D imaging and visual learning. Maximilian Kaluschke et al. [6] proposed a novel haptic rendering method to simulate the material removal process in medical simulation with haptic rate, and a stable and flexible 6-dOF collision response scheme based on force calculation of punishment and constraint to achieve large-scale parallel continuous collision detection algorithm. Chen Youzhi [7] implementation based on virtual reality technology of lower limb fracture surgery, emergency training system design, due to the sudden and the complexity of surgery, may lead to the results may not be able to succeed, so using the Unreal Engine 4 software made a virtual operating room scene, surgical instruments, etc, and the actual operation data uploaded to the system by the software, Users can enter the system for repeated training by wearing VR peripherals. The emergency ability of doctors is well trained and the success rate of surgery is improved to a certain extent. Wang Ping 'an et al. [8] realized the application of the whole system of augmented reality system in liver puncture surgery. When the tumor is located inside the liver, it cannot be observed directly. He realized the flexible body puncture augmented reality navigation system through Hololens, and realized the dynamic tracking and registration of scenes in surgery through deformation prediction in the process of model puncture. Wang Ruiyu et al. [9] applied digital 3D surgical simulation and 3D printing in orthognathic surgeon training, respectively applied digital software and 3D printing in theoretical knowledge teaching, surgical operation teaching and other teaching links, combined practical teaching with theoretical teaching, and actively played the subjective initiative of students. In human skeleton teaching, not only a large number of teaching sites and bone models are needed, but also it is difficult to achieve the expected teaching effect when there are too many students. The virtual reality human skeleton teaching system can greatly relieve the training and teaching resources, and provide multiple users with learning at the same time, which plays a great role in promoting human skeleton teaching. 2. System Builds In this paper, 3DS MAX is used as a 3D model 2 development tool to build a human skeleton model according to the relative proportion of each part in the actual human skeleton system, and show the actual human skeleton as realistically as possible, as shown in Figure 1. Figure 1. Build a 3d human skeleton model Unity 3D engine is highly compatible and can be developed across platforms. Therefore, Unity 3D is used as the scene building tool in the virtual reality human skeleton teaching system in this paper, and the virtual teaching scene is built according to the real laboratory environment, as shown in Figure 2. Figure 2. Virtual teaching scene The 3DS MAX human skeleton model is imported into Unity development platform, and combined with OpenGl and Shader, lighting setting and shadow calculation are carried out on the virtual teaching environment and human skeleton model as a whole, and the actual physical environment is restored with high sense of reality. In order to improve the light reflection effect and realism of the model surface, the characteristics of the real bone surface are simulated through Material editing and texture setting, as shown in Figure 3. Figure 3. Human skeleton simulation model 3. Human-Computer Interaction Leap Motion based on image recognition is adopted as a human-computer interaction tool in this paper. Leap Motion is widely used in virtual reality due to its small size, low price and good recognition effect, as shown in Figure 4. Figure 4. Leap Motion Gesture recognition device Currently, Kinect, a commonly used motion-sensing interaction device, has the characteristics of wide recognition range and high accuracy, and is suitable for face capture and human movement recognition. Leap Motion is more oriented to "desktop" operation. Its main function is to identify finger movements with high precision and low delay. It has the advantages of small size, low price and high recognition accuracy, making it more suitable for virtual experiments and teaching demonstrations with high usage rate and universality. Therefore, Leap Motion interactive device, which is more suitable for hand Leap Motion capture, is used as the input of external data in the research algorithm in this paper. In this paper, Leap Motion interactive device is used as human-computer interaction device. Leap Motion device collects real-time movement data and position information of users' hands through three data acquisition cameras to further simulate the movements of hands. The Leap Motion data interaction interface is created in Unity to quantify the collected hand data, that is, the measurement of hand joint bending is converted into a specific value of 0-10, so as to reproduce the gesture state in the virtual system, as shown in Figure. 5. Figure 5. Leap Motion Gesture recognition By collecting the relative physical coordinate position of the user's hand and Leap Motion device in the real environment, the hand position information is transmitted to the computer, and the relative position of the virtual hand model in the virtual space is calculated through coordinate transformation matrix operation. In order to facilitate the separation and integration of all parts of the skeleton, the head skeleton is taken as the parent object, and all other bone objects are taken as child, and their local position relative to the head bone object are calculated. By recognizing hand movements, the operation of picking, moving and assembling the skeleton model can be realized. When the skeleton object is judged to be in the pick state, the 3 relative position of the object with respect to the virtual hand model are calculated and kept unchanged. When the virtual hand model moves, the picked bone object model moves with it to realize the "grab" operation, as shown in Figure 6. Figure 6. Picking up bone model 4. Comparative Experiment After the completion of the system, 60 medical students were recruited from freshmen in medical colleges as experiment participants, and the participants were divided into two groups, A and B, with 30 students in each group. The participants in group A were explained the traditional human skeleton course respectively, while the students in group B participated in the virtual decomposition and assembly of human skeleton in the system after learning the traditional human skeleton course. After the experiment, students in group A and GROUP B were tested on their knowledge of human skeleton, and the test scores were statistically analyzed, as shown in Figure. 7. a) Pie chart of score of Group A b) Pie chart of score of Group B c) Bar chart of scores of Group A d) Bar chart of scores of Group B Figure 7. Comparative experimental data The experiment verifies that the system in this paper greatly improves students' learning interest and learning efficiency, and also proves that the new teaching form and virtual reality technology have great value in the field of medical teaching. 5. Conclusion In this paper, a Leap Motion-based virtual teaching system for human skeleton is completed, which provides auxiliary teaching and scientific research functions for medical researchers and solves the contradiction between the lack of teaching resources and the increasing number of medical students. The human-computer interaction system based on gesture recognition is realized, which improves students' interest in learning. The experiment proves that the system plays a significant role in improving students' learning efficiency and interest. In the future research, we will explore in the field of virtual reality and medical science, so as to restore realistic scenes more realistically. References [1] D. Weik et al., "Integrating Tactile Feedback in an Acetabular Reamer for Surgical VR-Training," 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2019, pp. 1227- 1228, doi: 10.1109/VR.2019.8798287. [2] L. Vega-Medina, B. Perez-Gutierrez, G. Tibamoso, A. Uribe- Quevedo and N. Jaimes, "VR central venous access simulation system for newborns," 2014 IEEE Virtual Reality (VR), 2014, pp. 121-122, doi: 10.1109/VR.2014.6802081. [3] J. Schild et al., "Applying Multi-User Virtual Reality to Collaborative Medical Training," 2018 IEEE Conference on 4 Virtual Reality and 3D User Interfaces (VR), 2018, pp. 775- 776, doi: 10.1109/VR.2018.8446160. [4] M. Cai, R. Sato, K. Yoshida and K. Takayasu, "Development and Application of VR Support System for Medical Students," 2019 Nicograph International (NicoInt), 2019, pp. 78-81, doi: 10.1109/NICOInt.2019.00023. [5] P. Stefan et al., "An AR edutainment system supporting bone anatomy learning," 2014 IEEE Virtual Reality (VR), 2014, pp. 113-114, doi: 10.1109/VR.2014.6802077. [6] M. Kaluschke, R. Weller, G. Zachmann and M. Lorenz, "A Continuous Material Cutting Model with Haptic Feedback for Medical Simulations," 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2019, pp. 1002-1003, doi: 10.1109/VR.2019.8798268. [7] Chen Youzhi, Fan Xiaozhi. Design of emergency training system for lower limb fracture surgery based on Virtual Reality Technology[J/OL]. Packaging Engineering:1-10[2021-12-03]. https://kns-cnki- net.webvpn.ncepu.edu.cn/kcms/detail/50.1094.tb.20210518.1 648.002.html. [8] Wang Ping 'an. Medical image analysis and surgical simulation: Medical Image Analysis and Surgical Simulation: AI and VR Application for Medicine[J].Optics & Optoelectronic Technology ,2021,19(06):1-10.DOI:10.19519/j.cnki.1672- 3392.2021.06.001. [9] Wang Ruiyu, Zhao Wenli, Liu Yao, Zhu Songsong, BI Ruiye. Application of Digital 3D Surgery Simulation Combined with 3D Printing in Orthognathic Surgery Training[J]. Chinese Journal of Medical Education Research,2021,20(08):963-966