©2022 Published by LUMEN Publishing. This is an open access article under the CC BY-NC-ND license BRAIN. Broad Research in Artificial Intelligence and Neuroscience ISSN: 2068-0473 | e-ISSN: 2067-3957 Covered in: Web of Science (WOS); PubMed.gov; IndexCopernicus; The Linguist List; Google Academic; Ulrichs; getCITED; Genamics JournalSeek; J-Gate; SHERPA/RoMEO; Dayang Journal System; Public Knowledge Project; BIUM; NewJour; ArticleReach Direct; Link+; CSB; CiteSeerX; Socolar; KVK; WorldCat; CrossRef; Ideas RePeC; Econpapers; Socionet. 2022, Volume 13, Issue 3, pages: 252-276 | https://doi.org/10.18662/brain/13.3/366 Submitted: August 10th, 2022 | Accepted for publication: September 19th, 2022 SABAS: A Smartphone-Aided Training Simulator based on Virtual and Augmented Reality for Brain Anatomy Assessment Emre DANDIL 1 , Zafer SERIN 2 , Yeşim ŞENOL 3 1 Assoc. Prof. PhD Department of Computer Engineering, Faculty of Engineering, Bilecik Seyh Edebali University, Bilecik, Turkey. emre.dandil@bilecik.edu.tr 2 MSc Student Department of Computer Engineering, Institute of Science, Bilecik Seyh Edebali University, Bilecik, Turkey. 3 Prof. Dr. MD Department of Medical Education, Faculty of Medicine, Akdeniz University, Turkey. Abstract: Many application areas for augmented reality (AR) and virtual reality (VR) emerged with the technological advances. These technologies, which initially appeared in sectors such as entertainment and games, are now widely used in the field of health care. In this study, a traditional simulator named SABAS is designed with its all components to be used in the training of brain anatomy. The designed simulator is equipped with AR and VR supported innovative e-learning technologies in order to examine and learn the structure of the human brain, whose anatomical structure and functioning is complex, using 3D models in anatomy education. This smartphone-aided application is achieved a high level of success in examination of brain anatomy with the additional features such as interface design and application usability. After the cornerstones of this designed prototype application are presented, the required suggestions are obtained from experts and healthcare professionals and it is observed that the application worked with maximum efficiency. In the study, the effectiveness of the VR and AR aided SABAS mobile application simulator, developed to teach the anatomical structure of the brain, is evaluated based on the experiences of 30 participants who wanted to voluntarily participate in the study. Keywords: Brain anatomy; anatomy education; training simulator; virtual reality; augmented reality; mobile application. How to cite: Dandıl, E., Serın, Z., & Şenol, Y. (2022). SABAS: A Smartphone-Aided Training Simulator based on Virtual and Augmented Reality for Brain Anatomy Assessment. BRAIN. Broad Research in Artificial Intelligence and Neuroscience, 13(3), 252-276. https://doi.org/10.18662/brain/13.3/366 https://doi.org/10.18662/brain/13.3/366 mailto:emre.dandil@bilecik.edu.tr https://doi.org/10.18662/brain/13.3/366 SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 253 Introduction Today, many organs in the human body are modeled in 3D for anatomy education. In addition, the organs modeled as 3D can be transferred to the computer and presented or staged in integration with the real world due to augmented and virtual reality technologies (Tatar et al., 2019). One of the cornerstones of learning the anatomical structures in our body is to create mental models of anatomical structures and to understand the spatial relationships between each other (Aziz et al., 2002). Today, 2D and static images printed in books lack a better representation of scientific concepts and they may cause limitation to meaningful learning. Therefore, computer-aided new technologies and different learning materials are required to promote a better understanding. From this perspective, interactive digital graphics applications help the teaching-learning and knowledge discovery process of concepts, contents and skills that can be gained through practice, due to the fact that they encourage self-learning. These processes are generally complemented by training resources, auxiliary equipment and training simulators supported by AR and VR technologies. AR is defined as the concept of digitally superimposing virtual objects over physical objects in the real world. In the display of virtual objects created with AR, it is possible to display them using a marker, a projection, or without using any markers (Gannis, 2017; Mekni & Lemieux, 2014). VR is an effective way of visualizing, managing and interacting with extremely complex data in a virtual scene created with the help of computers (Boas, 2013; Isdale, 1998; Wexelblat, 2014). Simulation-based trainings are mostly provided with the help of VR, AR technologies and interactive solutions that provide tactile feedback (Ruthenbeck & Reynolds, 2015). VR and AR-based medical training simulators have many advantages in terms of teaching styles as they simulate the real environment. The success of education can be increased with these technologies. For example, with a medical training simulator, the motion of the surgical equipment can be recorded and imitated, more precise measurements can be made, and the damages that can be given to the similar anatomical structures can be foreseen. In addition, simulation-based training has significant economic benefits. For example, trainee surgeons can practice numerous exercises on simulated patients without the need for a cadaver or a very expensive and limited animal model (Coles et al., 2010). Moreover, simulation-based trainings are not affected by the limitations of the instructor‟s capacity in traditional teaching methods due to their contribution such as hardware quality and software capacity. Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 254 Learning through “trial and error” or “seeing, hearing, doing” in medical education is replaced by innovative approaches that enable the development of critical thinking and decision-making in clinical care, effective communication and problem solving skills. Medical, biomedical engineering, and healthcare students who want to learn about human anatomy often use auxiliary equipment such as textbooks, solid 3D organ models, texts, visual images, and sculptures. However, these auxiliary tools are generally insufficient, do not allow making more practices on them and are highly costly. In addition, students also benefit from cadavers, which is a proven and effective learning method that enables learning by experience with the help of surgeries for practical applications. However, learning on cadavers is more costly and more complex than learning on a 3D model. In addition, due to the lack of sufficient number of cadavers, students often practice in groups and cannot find the opportunity to practice on their own. In this case, it becomes inevitable to avoid such complexity and to use a less costly AR, VR supported mobile application and to practice and experience on the developed application. An application offering these supports will shorten the learning time of students and provide practical learning with the use of an application that can be accessed from anywhere instead of cadavers that are difficult to access and examine (Kurniawan & Witjaksono, 2018). In addition to being frequently used in areas such as communication, socializing, gaming and obtaining information, mobile devices have been widely used in the medical field in recent years as an auxiliary learning resource and practice tool with interactive applications for the presentation of learning content. Since new technologies also target mobile platforms, the use of mobile devices in academic education has been increasing. Especially in recent years, the use of mobile devices in content sharing has increased rapidly (Boruff & Storie, 2014). Since especially mobile phones are accessible from everywhere regardless of country, city and social and economic differences, it is clear that practicing the teaching using the developed applications is preferred as an effective teaching method (Jones et al., 2010). For example, Popovic et al. (Popović et al., 2016) developed a testing tool for a school and analyzed this mobile educational application. As a result of the tests carried out, they revealed that the mobile application was a very effective auxiliary resource. In addition, the results showed that the total success rate in the term in which this method was used was higher compared to the term in which this method was not used. In some similar studies (Huang & Chiu, 2015; Khachan & Özmen, 2019), it was concluded that the developed mobile applications should have a simple interface, provide SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 255 interaction between students, provide a problem environment that can be solved, and receive feedback in real time in order to be appealing for a wider target group. In a study conducted by Wang (Wang, 2017) the researcher added the mobile support function to the flipped learning system, and demonstrated the benefits it provided in subjects such as engagement in the lesson, interaction with the teacher, problem solving ability, and critical thinking. Mobile applications facilitate teaching. Besides, it is seen that AR and VR supported studies have been also proposed in the field of health care. For example, in their study Safi et al. (Safi et al., 2010) proposed that AR could be used to treat skin cancer or to prevent disease progression. In their study, they followed the changing lesions based on professional images and texts previously obtained with AR systems. On the other hand, Parkes et al. (Parkes et al., 2009) developed a mixed reality simulator to be used in the training of veterinarians. In the simulator, haptic devices were positioned both side of a modified toy cat, and virtual models of the chest and some abdominal contents were superimposed on the physical model. When the literature is examined, it is seen that there are some studies supported by AR and VR to facilitate the teaching of the anatomical structure of the human body. Blum et al. (Blum et al., 2012) developed an AR application and by using it, they tracked the pose of a user standing in front of a large display using a depth camera and facilitated the teaching of human anatomical structure. In a similar study, Jamali et al. (Jamali et al., 2015) developed mobile prototype learning environment that utilizes Augmented Reality for learning human anatomy. This mobile-based application run on the Android tablet platform and cannot provide cross- platform support. Besides these studies, there are systems that are used for liver resection and suggest using the vein tree and hepatic vein structure for liver resection (Bourquain et al., 2002; Reitinger et al., 2006). In other anatomy-based studies, Goswami et al. (Goswami et al., 2010) and Rhienmora et al. (Rhienmora et al., 2010) developed a simulator for dental anatomy, dental extraction and operational processes. There are not many studies in the literature on teaching the anatomical structure of the central nervous system and surgical organs, especially of the brain, using mobile application, auxiliary educational tools supported by AR and VR technologies. Lemole et al. (Lemole Jr et al., 2007), developed a VR-assisted simulator that provides feedback with haptic devices on central nervous system surgery. In the anatomical environment created on computed tomography images, students carried out practical applications with realistic, visual and haptic features. In another study Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 256 conducted on the brain, Rose et al. (Rose et al., 2005) proposed a VR- assisted method for the treatment of brain damage. In their studies, they found that using VR would be beneficial in cases of brain damage such as memory deficits, attention deficits and executive dysfunction. Although anatomy education should be given in a practical way, traditional methods are used in teaching brain diseases and teaching the anatomical structure of brain. Cadavers are the most important teaching materials used in traditional anatomy education; however this process is difficult and expensive. In addition, using cadavers for learning the specific anatomical regions such as the brain may be insufficient in some cases. With the development of three-dimensional (3D) medical imaging techniques after the 2000s, computer aided alternative methods have been used in anatomy education. In recent years, technological advances in imaging techniques have encouraged the simulation-based anatomy training. In addition, simulations have been widely used in medical education courses. However, the anatomical structure and examination of the human brain is more difficult than in other regions. In addition, it is very difficult to reach these parts for diagnosis and treatment because the skull is a bony structure. Although it does not weigh much, there are many anatomical formations in the human brain. On the other hand, the pathways connecting both cerebral hemispheres and different areas of the brain, and nerve fibers of the peripheral nervous system further complicate this region. For these reasons, a holistic training simulator that presents the anatomy of the human brain and brain pathways with appropriate scenarios on 3D images is required in order to be used in medical education. When the literature on training simulators supported by virtual learning environment proposed for the brain is examined, it is seen that the studies on this issue are very limited and there is not a holistic simulator that includes scenarios of all stages and surgical interventions in the brain. In addition, many of the proposed systems can be described as theoretical studies since they are not tested in clinical settings. Most of the training simulators that were previously proposed and tested under clinical setting are not preferred due to reasons such as difficulty of use, lack of innovative technologies, and not being integrated. In addition, although the use of AR and VR is still inadequate, especially in the field of health, it is seen that there have been developments and their use has increased. Based on these studies, it can be argued that the use of AR and VR will be an integral part of anatomy, treatment, rehabilitation, and education, in particular. As can be understood from the previous studies, interactive education is costly. It is also observed that the number of applications developed on this subject is SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 257 inadequate. It has been found that most of the applications developed lack cross-platform support. In this study, it was aimed to eliminate such deficiencies with the proposed application. In this study, a training simulator named SABAS was designed with its all components to be used in the anatomy education. The designed simulator was equipped with AR and VR supported innovative e-learning technologies in order to examine and learn the structure of the human brain, whose anatomical structure and functioning is complex, using 3D models in anatomy education. This smartphone-aided application is achieved a high level of success in examination of brain anatomy with the additional features such as interface design and application usability. After the cornerstones of this designed prototype application were presented, the required suggestions were obtained from experts and healthcare professionals and it was observed that the application worked with maximum efficiency. SABAS‟s contributions to the literature with its innovative and original aspect can be listed as follows: • In this study, a holistic medical training simulator is developed in order to teach the complex structure of the brain in a top-down manner. Thus, the learners learned the basic anatomical structures of the mass brain on 3D models and the learning process is aided with the prepared gamification and educational mobile software based on AR and VR. • 3D brain modules and AR and VR aided applications fulfill an important function of the medical training simulator when considered in a holistic way so that the learner can better understand the whole learning process on the brain. • In the proposed study, simulator training software compatible with mobile environments that can offer an interactive learning-by-doing simulation environment supported by AR and VR are developed for modular brain structures. • The developed medical training simulator offers detailed supportive trainings for learning the basic anatomical structures on the brain. An effective learning environment is provided by integrating the educational software that simulates the anatomical and vascular structure of the human brain in detail with VR hardware. • With this study, it is also provided to simulate the scenarios that are complex and difficult to learn on the cadaver due to the structure of the brain on the training simulator. • The simulator software also includes the application of the learned information in a realistic way and the design of e-learning infrastructures with feedback. Thus, an e-learning system is created and this system is in an Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 258 infrastructure appropriate for use in medical education and postgraduate medical education. • The simulator-based medical education processes developed in this study are implemented as a pilot study on a group of doctors and students, and a holistic general evaluation is provided. Thus, the standardization, adaptation and widespread use of the developed medical education simulator in medical education are realized and the errors and deficiencies are eliminated. Additionally, with this study, innovative, interactive and animated educational contents presenting the usage and operating structure of the medical education simulator are prepared. The next sections of the study are organized as follows: In Section II, the software architecture, components and usage features of the SABAS application are explained in detail. In Section III, the survey results that provide feedback for user experiences are evaluated. In the final section, the results obtained are emphasized and the effectiveness of SABAS is discussed. Methodology The methodology of the study is discussed in two parts. The first part is the development of the system and the second part is the evaluation of the system by the users. System overview In this study, a VR and AR supported mobile application simulator named SABAS was developed to teach the anatomical structure of the brain. Although there are many software and applications that have been developed as a desktop application that show the anatomical structure of the brain, the smartphone application, which has been supported by current technologies, is quite limited. For this reason, a mobile application that teaches the anatomical structure of the brain in an easy way has been designed to increase the level of readiness using SABAS. In addition, the developed application simulator SABAS contains pedagogical information similar to the textbooks on the anatomical structure of the brain in terms of terminology, representation, design and technology. The main form user interface showing the basic components of the mobile application SABAS developed is presented in Figure 1. SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 259 Figure 1. Mobile interface form showing the basic components of the mobile application (SABAS). Source: acquired from SABAS Software architecture The anatomical structure of the brain can be examined on this model using SABAS, and all central nervous system organs can be seen in detail and simulated. Due to its infrastructure, SABAS provides learners to simulate and learn the anatomical structure of the brain in a fast and accurate way. SABAS realizes the application of content with AR and VR technologies both in real time in the classroom and as offline in any environment. Figure 2. 3D anatomical brain model used in the development of SABAS infrastructure. Source: https://sketchfab.com/3d-models/eyeball-extension-neuroanatomy- 7d1ff51c69c04455b093bf03c67a0d40 https://sketchfab.com/3d-models/eyeball-extension-neuroanatomy-7d1ff51c69c04455b093bf03c67a0d40 https://sketchfab.com/3d-models/eyeball-extension-neuroanatomy-7d1ff51c69c04455b093bf03c67a0d40 Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 260 In the developed application, the model containing the anatomical structure of the brain is presented in Figure 2 and this model was obtained from Sketchfab (Sketchfab, 2019) platform, which offers 3D anatomical models in the web environment. The accuracy of these models of the anatomical structure of the brain has been approved by experts. The SABAS mobile application has been developed with the Unity3D 2018 3.5f1 version of the Unity3D (Unity3D, 2019) game engine. Unity3D offers an environment that can be coded using C# and JavaScript languages. Models of the anatomical structure of the 3D brain were then transferred to the Unity3D platform. Applications developed by Unity3D have direct cross-platform support. Applications developed with this tool can be output to nine different platforms, including many mobile operating systems. Thus, applications developed without encountering any problems with general coding can be used on every platform, as long as libraries working on a specific platform are not used. SABAS was designed in a way to run on Android based mobile operating system platforms. In addition, SABAS has an infrastructure that can be easily transferred to other cross- platform environments when desired. SABAS includes components for both AR-supported and VR-supported simulations for the anatomical structure of the brain. Vuforia libraries (Vuforia, 2019) that can work as integrated with Unity3D were used to create AR-supported simulations in the SABAS infrastructure. Besides, Google VR extension was used to create VR supported simulations for SABAS. Figure 3. Marker model of the brain developed for AR-aided simulations in SABAS application. Source: https://commons.wikimedia.org/wiki/File:Brain_(PNG).png https://commons.wikimedia.org/wiki/File:Brain_(PNG).png SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 261 Components of the SABAS SABAS, the smartphone-aided mobile application developed to understand the anatomical structure of the brain, was designed to be simulated in two different environments, as AR and VR environments. Therefore, it was aimed to eliminate the difficulties experienced in learning the structure of the brain, which has a complex anatomical structure, by the current technological innovations. In the part of SABAS which was supported with AR, the brain model that acts as a marker as seen in Figure 3 was used in order to carry out detailed simulation applications in the anatomical structure of the brain. Figure 4. Initiation of menus with the help of the marker brain model for AR-based applications in SABAS. Source: acquired from SABAS Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 262 Figure 5. Imaging all parts within the anatomical structures of the brain with the “show the brain parts” function. Source: acquired from SABAS In the SABAS application, functions that enable anatomical simulations to be initiated in the brain were operated by providing a simple menu as shown in Figure 4 in the lower right corner of the screen when the marker is seen. In this section, three functions are operated as “move mode"”, “part animation” and “show the brain parts”. As shown in Figure 5, with “show the brain parts” function, all parts within the anatomical structures of the brain are shown and opened with a menu. Users can select the anatomical structure of the brain that they want to examine. In addition, in this section, more than one part of the brain can be selected and interacted with these structures. One of the important features aimed with the SABAS application is the visualization of the 3D views of the parts in the brain in a virtual environment for educational purposes, enabling planning before surgical procedures and reducing the training time. By the “examine the part” option presented in Figure 5, any part in the brain model is examined specially and a separate scene is loaded for this part as in Figure 6. In this loaded scene, there is information about the functions of the part, the 3D model of the part and its functions. In this scene, the “cutting mode” is also activated and the part is cut on a linear plane. In this part, it is aimed to make a cut similar to surgical cutting procedures. At this stage of SABAS application, since the cutting process is planar, it is not sufficient in terms of surgical cutting. In this respect, the application should be developed. SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 263 Figure 6. Application scene and cutting mode where the anatomical structures of the brain are examined in detail. Source: acquired from SABAS In the SABAS application, when “move mode” seen in Figure 4 is selected, the rotation, location and size of the relevant anatomical structure in the brain can be changed. These functions are provided with the help of some buttons and toggles in Figure 7. In this part, the anatomical structure of the brain is brought to different angles, different sizes and to a different position on the screen, providing optimum view and interaction for the user. As a result of these processes, not only the outer part of the head structure (skull), but also the parts inside it are affected by the same processes. If the head is rotated about 45°, the other parts in the head will also rotate. Thus, SABAS provides ease of use. These operations are conducted by taking the middle point of the head as the basis of the pivot point. The reason for this is to prevent the head structure from turning from any different point and to provide vision from the right angles. In the “move mode” environment in SABAS, as shown in Figure 8, when the toggle is activated, all parts displayed with the head are changed (with the help of mouse in computer environment, by touching in mobile environment). The object control here has been realized with the help of collider. Thus, objects can be easily moved, resized or rotated to their next position individually or collectively. Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 264 Figure 7. Changing the rotation, location and size of the anatomical structures of the brain in the “move mode” stage. Source: acquired from SABAS Figure 8. Activating toggle mode in “move mode” environment and changing the anatomical structures in SABAS. Source: acquired from SABAS In SABAS, when the “part animation” function presented in Figure 4 is selected, an animation is activated. This animation provides the distribution of all anatomical structures of the brain shown in Figure 9 in a top-down manner and in a scattered position towards the screen. In this scene, all the parts inside the head are displayed separately. SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 265 Figure 9. Distribution of all anatomical structures of the brain in a scattered way with the “part animation” function in SABAS. Source: acquired from SABAS In the SABAS, Google VR libraries were used to support the anatomical structure of the brain with VR supported technologies. When the “VR Assisted” is selected in the first main form of SABAS, a transition is provided to the environment shown in Figure 10. In this section, anatomical examinations can be made on the 3D anatomical model of the brain. On the VR supported stage, the anatomical parts of the brain can be examined from different angles with an auxiliary virtual reality glasses to the extent that the model allows. In this section, each part can be displayed in different positions due to the animation that functions similar to the distribution animation in the AR-supported part, which enables the anatomical structures in the brain to be opened. When any part is viewed from a maximum distance of 5 units from the VR supported stage, a panel displayed on the screen opens and information about the anatomical part is given. In addition, the VR-supported part developed in the SABAS application was tested at different resolutions (maximum 1440x720 pixels) and on different devices, and it was ensured to be operated smoothly in different environments. Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 266 Figure 10. Brain model and anatomical structures developed for VR supported simulations in SABAS application. Source: acquired from SABAS Results Study participants In this study, the effectiveness of the SABAS mobile application simulator, which was developed to teach the anatomical structure of the brain and supported by VR and AR, was evaluated based on the experiences of 30 participants who were wanted to voluntarily participate in the study. Of these 30 participants, 6 were medical doctors, 16 were medical faculty students, and 8 were health school students. For this study, it was approved by Clinical Research Ethics Committee at the Faculty of Medicine of Akdeniz University with the code 2012-KAEK-20 that there is no scientific and ethical inconvenience. The SABAS application developed for smartphones running on the Android platform was shared online with the participants. In line with the guideline prepared for the installation of SABAS on smart phones, the application was installed on the participants‟ phones. In addition, the process was initiated for the free download of SABAS from Google Play, and it is estimated that it will take its place in the market in a short time. All data in questionnaire were imported into SPSS software package. Item data distribution for questionnaire was denoted by percentages of approval. Therefore, mean values with standard deviation were additionally. SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 267 Assessments of the user feedbacks A questionnaire was prepared for the participants who used the SABAS application and it was sent to the participants through Google Forms. By doing so, it was aimed to obtain the participants‟ opinions on the application. The questionnaire was prepared by the researchers by scanning the relevant literature. The questions prepared to obtain the opinions of the participants are as follows: • The design of SABAS is very efficient and robust • The SABAS is very helpful to simulate human brain anatomy • In my opinion the SABAS can be very useable in other fields • The SABAS using markers is more remarkable to use than static images • The SABAS using markers are more flexible in visualizing the anatomy of the human brain • The SABAS using markers is faster in understanding the human brain anatomy • The SABAS is very helpful and interesting in description of each brain part • The SABAS is very practical in rotating of 3D brain anatomical model • The SABAS is very practical in seeing the details of brain anatomy • Learning using SABAS can be very practical in anatomy classes • I am very satisfied with SABAS to learn human brain anatomy • Overall SABAS greatly aided the brain anatomy learning process Table 1. Percentage distribution of the responses given to the items in the questionnaire Source: Our own conception QID Question (1) No (%) (2) Rather no (%) (3) Do not know(%) (4) Rather yes (%) (5) Yes (%) Q1 The design of SABAS is very efficient and robust 0 3.3 20 43.3 33.3 Q2 The SABAS is very helpful to simulate human brain anatomy 0 0 26.7 36.7 36.7 Q3 In my opinion the SABAS can be very useable in other fields 0 0 13.3 46.7 40.0 Q4 The SABAS using markers is more remarkable to use than static images 0 3.3 16.7 36.7 43.3 Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 268 Q5 The SABAS using markers are more flexible in visualizing the anatomy of the human brain 0 0 13.3 40.0 46.7 Q6 The SABAS using markers is faster in understanding the human brain anatomy 0 0 13.3 50.0 36.7 Q7 The SABAS is very helpful and interesting in description of each brain part 0 0 16.7 26.7 56.7 Q8 The SABAS is very practical in rotating of 3D brain anatomical model 0 0 23.3 53.3 23.3 Q9 The SABAS is very practical in seeing the details of brain anatomy 0 0 10.0 43.3 46.7 Q10 Learning using SABAS can be very practical in anatomy classes 0 0 6.7 43.3 50.0 Q11 I am very satisfied with SABAS to learn human brain anatomy 0 0 10.0 56.7 33.3 Q12 Overall SABAS greatly aided the brain anatomy learning process 0 0 10.0 36.7 53.3 The questionnaire consists of twelve items that range between 1 and 5 as (1) No, (2) Rather no, (3) Do not know, (4) Rather yes and (5) Yes. Table 1 presents the percentage distribution of the responses given to each question in the questionnaire. As can be seen in the table, the participants did not score the (1) No option for the questions in the questionnaire. On the other hand, it was obtained that only 3.3% of the participants answered as (2) Rather no for the Q1 and Q3 items. When the total scores of the participants for the questions in the questionnaire are examined, it is seen that the participants‟ scores are mostly grouped under (4) Rather yes and (5) Yes options. Besides, it was found that most of the participants (56.7%) answered as (5) Yes to the Q7 item stating that; "The SABAS is very helpful and interesting in description of each brain part”. In Table 2, average scores and standard deviation values for the items in the questionnaire are presented. The highest mean scores were obtained from Q10, Q12, Q7 and Q9 items. The mean values of these items SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 269 were found as 4.43, 4.43, 4.40 and 4.37, respectively. The lowest standard deviation value was obtained as 0.626 for Q10 and Q11 items. In addition, the total standard deviation and total mean score were calculated as 0.719 and 4.26, respectively. Table 2. Mean scores and standard deviation scores obtained from the questionnaire Source: Our own conception QID Question Standard deviation Mean score (1-5) Q1 The design of SABAS is very efficient and robust .828 4.07 Q2 The SABAS is very helpful to simulate human brain anatomy .803 4.10 Q3 In my opinion the SABAS can be very useable in other fields .691 4.27 Q4 The SABAS using markers is more remarkable to use than static images .847 4.20 Q5 The SABAS using markers are more flexible in visualizing the anatomy of the human brain .711 4.33 Q6 The SABAS using markers is faster in understanding the human brain anatomy .679 4.23 Q7 The SABAS is very helpful and interesting in description of each brain part .770 4.40 Q8 The SABAS is very practical in rotating of 3D brain anatomical model .695 4.00 Q9 The SABAS is very practical in seeing the details of brain anatomy .669 4.37 Q10 Learning using SABAS can be very practical in anatomy classes .626 4.43 Q11 I am very satisfied with SABAS to learn human brain anatomy .626 4.23 Q12 Overall SABAS greatly aided the brain anatomy learning process .679 4.43 In the questionnaire, the highest mean score was obtained as 4.43 for Q10 and Q12 items. Figure 11 shows the scores and percentage distribution of the answers given to the options by the participants for the item Q10 stating that; “Learning using SABAS can be very practical in anatomy classes”. The findings showed that for this item, 50% of the participants Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 270 responded as (5) Yes, 43.3% of them responded as (4) Rather yes and 6.7% of them responded as (3) Do not know. On the other hand, it was obtained that none of the participants score the options (2) Rather no and (1) no for this item. The second highest mean score was obtained for Q12 item stating that „Overall SABAS greatly aided the brain anatomy learning processes. As seen in Figure 12, the percentage of the (5) Yes, (4) Rather yes, (3) Do not know, (2) Rather no options were found as 53.3% (16 participants), 36.7% (11 participants), 10% (3 participants), 0% (no participant) and 0% (no participant), respectively. Figure 11. The percentage distributions of the scores for the Q10 item Source: Our own conception Figure 12. The percentage distributions of the scores for the Q12 item Source: Our own conception In the questionnaire, the third and fourth highest scores were obtained as 4.40 and 4.37 from Q7 and Q9 items, respectively. As presented in Figure 13, the percentage of the (5) Yes, (4) Rather yes, (3) Do not know, (2) Rather no and (1) No options were found as 56.7% (17 participants), 26.7% (8 participants), 16.7% (5 participants), 0% (0 participant) and 0% (0 SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 271 participant), respectively for Q7 item. On the other hand, in Figure 14, the percentage of the (5) Yes, (4) Rather yes, (3) Do not know, (2) Rather no and (1) No options were found 46.7% (14 participants), 43.3% (13 participants), 10% (3 participants), 0% (0 participant) and 0% (0 participant), respectively for Q9 item. Figure 13. The percentage distributions of the scores for the Q7 item Source: Our own conception Figure 14. The percentage distributions of the scores for the Q9 item Source: Our own conception Discussion Mobile devices give highly successful and satisfying results in researches in the field of medicine and health. In our pilot application, most of them state that they are satisfied and benefited from the developed system (Chase et al., 2018; Walsh, 2015). It has also been shown that these tools help students learn better with less cognitive effort, provide learning satisfaction, and enable students to structure their knowledge to complete learning tasks (Küçük et al., 2016). Considering the results of studies and our study, the usage of this mobile device will contribute to managing Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 272 information for students, teaching innovation, and increasing the quality of knowledge about brain anatomy and ensuring permanence. The SABAS training simulator was designed to be used in pre/post- graduation and continuing medical education. A supportive learning management system was created using appropriate innovative mobile technology in the SABAS infrastructure. Due to the developed SABAS application, mobile teaching infrastructures were established and therefore significant contributions were made to the development and use of innovative learning strategies. Optimization is of particular importance when developing a SABAS type application. Especially in the VR section, frame per second (fps) value should be captured. It is generally accepted to achieve the maximum fps value, but in most applications, the design is achieved based on 30 fps or 60 fps values. Optimization methods to be used while developing Unity3D applications also contain some differences in this respect. The simplest optimization method in such an application is to create low-poly models. Although this method does not provide an optimum solution, it is one of the most important methods to increase the fps value when required. Although low-poly models have the advantage of having a low number of polygons, they have fewer details. In this situation, the knowledge of the expert who has developed the model becomes prominent. While it is possible to develop models that look very good with fewer polygons, the opposite is also possible. The expert who develops the application should know these situations and choose the most suitable model. In addition, optimization can be achieved by using static objects in Unity3D. Objects marked as static should not be moved even if they contain an animation or script. Another important optimization method that can be used in this situation is occlusion culling. With occlusion culling method, in Unity3D, the objects that only camera render display on the screen and therefore many models are not displayed on the screen, which reduces the load on the program and increases the fps value. Another limitation encountered in SABAS application is the application of cutting process on anatomical structures. The use of dynamic mesh cut to simulate surgical cuts on anatomical structures rather than on a linear plane can eliminate this problem. In this way, solid models are degraded into their vertices (points) and are not processed as an entire model. As a result, a realistic cut appearance can be obtained. In addition, although a previously designed anatomical model will be used, this model must first be made deformable. SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 273 Conclusion Although there have many technological developments in the field of smart phones recently, these developments have not been fully transferred to education context using mobile devices. In this study, a training simulator named SABAS was designed with its all components to be used in the anatomy education. The designed simulator was equipped with AR and VR supported innovative e-learning technologies in order to examine and learn the structure of the human brain, whose anatomical structure and functioning is complex, using 3D models in anatomy education. In this proposed study, a method that allows investigation of the anatomic structure of the brain was presented to be used in different fields, especially medicine field. For the simulator, anatomical structures showing the 3D brain structure were used in an integrated manner, and the brain structure was examined on SABAS, a smartphone-aided mobile application. Unlike traditional methods, AR technology can combine real and virtual objects, and VR technology can offer various interactions in a virtual world. AR and VR supported innovative, interactive, original applications that appropriate for learning was developed on the created 3D anatomical images. By the supportive educational simulation applications developed on these 3D models of the brain structure, the educational experiences that are difficult to obtain in the real environment were transferred by virtualization. Although traditional methods (writing, books, cadavers, etc.) used in faculties in the field of health are effective methods, the AR and VR technologies can be good alternatives to these methods. These applications promote learning for students by enabling them to learn through interactive interactions instead of books and complex trainings. After the developed application was completed, the application was used and evaluated by a total of 30 participants, 6 of whom were medical doctors, 16 of whom were medical faculty students, and 8 of whom were students of health schools. The opinions of the participants regarding their experiences were obtained via an online questionnaire. Acknowledgement The data used to support the findings of this study are available from the corresponding author on request. The authors declare that they have no conflicts of interest. The authors received no funding for this work. The authors of the study would like to thank Prof. Dr. A. Utku Şenol from Faculty of Medicine of Akdeniz University for confirming the accuracy of 3D models of the anatomical structure of the brain. In addition, we thank Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 274 Clinical Research Ethics Committee at the Faculty of Medicine of Akdeniz University for approving the study in terms of ethical. References Aziz, M. A., Mckenzie, J. C., Wilson, J. S., Cowie, R. J., Ayeni, S. A., & Dunn, B. K. (2002). The human cadaver in the age of biomedical informatics. The Anatomical Record: An Official Publication of the American Association of Anatomists, 269(1), 20-32. https://doi.org/10.1002/ar.10046 Blum, T., Kleeberger, V., Bichlmeier, C., & Navab, N. (2012). Mirracle: An augmented reality magic mirror system for anatomy education. In Proceedings of the 2012 IEEE Virtual Reality Workshops (VRW) (pp. 115-116). http://dx.doi.org/10.1109/VR.2012.6180909 Boas, Y. (2013). Overview of virtual reality technologies. In Proceedings of the Interactive Multimedia Conference. https://www.coursehero.com/file/47603440/Overview-of-VR-techpdf/ Boruff, J. T., & Storie, D. (2014). Mobile devices in medicine: a survey of how medical students, residents, and faculty use smartphones and other mobile devices to find information. Journal of the Medical Library Association: JMLA, 102(1), 22-30. https://doi.org/10.3163%2F1536-5050.102.1.006 Bourquain, H., Schenk, A., Link, F., Preim, B., Prause, G., & Peitgen, H. O. (2002). HepaVision2 - A software assistant for preoperative planning in living- related liver transplantation and oncologic liver surgery. In CARS 2002 Computer Assisted Radiology and Surgery (pp. 341-346). Berlin: Springer. Chase, T. J., Julius, A., Chandan, J. S., Powell, E., Hall, C. S., Phillips, B. L., Burnett, R., Gill, D., & Fernando, B. (2018). Mobile learning in medicine: an evaluation of attitudes and behaviours of medical students. BMC Medical Education, 18(1), 1-8. https://doi.org/10.1186/s12909-018-1264-5 Coles, T. R., Meglan, D., & John, N. W. (2010). The role of haptics in medical training simulators: A survey of the state of the art. In IEEE Transactions on haptics, 4(1), 51-66. https://doi.org/10.1109/toh.2010.19 Gannis, C. (2017). The Augmented selfie. In Proceedings of the Electronic Visualisation and the Arts (EVA 2017). (pp. 319-326). http://dx.doi.org/10.14236/ewic/EVA2017.66 Goswami, P., Schlegel, P., Solenthaler, B., & Pajarola, R. (2010). Interactive SPH simulation and rendering on the GPU. In Proceedings of the Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (pp. 55-64). http://dx.doi.org/10.2312/SCA/SCA10/055-064 Huang, Y. M., & Chiu, P. S. (2015). The effectiveness of a meaningful learning‐ based evaluation model for context‐aware mobile learning. British Journal of Educational Technology, 46(2), 437-447. https://doi.org/10.1111/bjet.12147 https://doi.org/10.1002/ar.10046 http://dx.doi.org/10.1109/VR.2012.6180909 https://www.coursehero.com/file/47603440/Overview-of-VR-techpdf/ https://doi.org/10.3163%2F1536-5050.102.1.006 https://doi.org/10.1186/s12909-018-1264-5 https://doi.org/10.1109/toh.2010.19 http://dx.doi.org/10.14236/ewic/EVA2017.66 http://dx.doi.org/10.2312/SCA/SCA10/055-064 https://doi.org/10.1111/bjet.12147 SABAS: A Smartphone-Aided Training Simulator based on Virtual and … Emre DANDIL et al. 275 Isdale, J. (1998). What is virtual reality. http://isdale.com/jerry/VR/WhatIsVR/noframes/WhatIsVR4.1.html Jamali, S. S., Shiratuddin, M. F., Wong, K. W., & Oskam, C. L. (2015). Utilising mobile-augmented reality for learning human anatomy. Procedia-Social and Behavioral Sciences, 197, 659-668. https://doi.org/10.1016/j.sbspro.2015.07.054 Jones, N., Blackey, H., Fitzgibbon, K., & Chew, E. (2010). Get out of MySpace! Computers & Education, 54(3), 776-782. http://dx.doi.org/10.1016/j.compedu.2009.07.008 Khachan, A. M., & Özmen, A. (2019). IMSSAP: After‐school interactive mobile learning student support application. Computer Applications in Engineering Education, 27(3), 543-552. https://doi.org/10.1002/cae.22096 Küçük, S., Kapakin, S., & Göktaş, Y. (2016). Learning anatomy via mobile augmented reality: Effects on achievement and cognitive load. Anatomical Sciences Education, 9(5), 411-421. https://doi.org/10.1002/ase.1603 Kurniawan, M. H., & Witjaksono, G. (2018). Human anatomy learning systems using augmented reality on mobile application. Procedia Computer Science, 135, 80-88. https://doi.org/10.1016/j.procs.2018.08.152 Lemole Jr, G. M., Banerjee, P. P., Luciano, C., Neckrysh, S., & Charbel, F. T. (2007). Virtual reality in neurosurgical education: Part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery, 61(1), 142-149. https://doi.org/10.1227/01.neu.0000279734.22931.21 Mekni, M., & Lemieux, A. (2014). Augmented reality: Applications, challenges and future trends. Applied Computational Science, 205-214. http://www.cs.ucf.edu/courses/cap6121/spr2020/readings/Mekni2014.p df Parkes, R., Forrest, N., & Baillie, S. (2009). A mixed reality simulator for feline abdominal palpation training in veterinary medicine. In Proceedings of the MMVR (pp. 244-246). https://pubmed.ncbi.nlm.nih.gov/19377159/ Popović, O., Marković, D. S., & Popović, R. (2016). mTester - Mobile learning system. Computer Applications in Engineering Education, 24(3), 412-420. https://doi.org/10.1002/cae.21719 Reitinger, B., Bornik, A., Beichel, R., & Schmalstieg, D. (2006). Liver surgery planning using virtual reality. In IEEE Computer Graphics and Applications, 26(6), 36-47. https://doi.org/10.1109/mcg.2006.131 Rhienmora, P., Gajananan, K., Haddawy, P., Suebnukarn, S., Dailey, M. N., Supataratarn, E., & Shrestha, P. (2010). Haptic augmented reality dental trainer with automatic performance assessment. In Proceedings of the Proceedings of the 15th international Conference on Intelligent User Interfaces (pp. 425-426). http://dx.doi.org/10.1145/1719970.1720054 http://isdale.com/jerry/VR/WhatIsVR/noframes/WhatIsVR4.1.html https://doi.org/10.1016/j.sbspro.2015.07.054 http://dx.doi.org/10.1016/j.compedu.2009.07.008 https://doi.org/10.1002/cae.22096 https://doi.org/10.1002/ase.1603 https://doi.org/10.1016/j.procs.2018.08.152 https://doi.org/10.1227/01.neu.0000279734.22931.21 http://www.cs.ucf.edu/courses/cap6121/spr2020/readings/Mekni2014.pdf http://www.cs.ucf.edu/courses/cap6121/spr2020/readings/Mekni2014.pdf https://pubmed.ncbi.nlm.nih.gov/19377159/ https://doi.org/10.1002/cae.21719 https://doi.org/10.1109/mcg.2006.131 http://dx.doi.org/10.1145/1719970.1720054 Broad Research in Artificial Intelligence and Neuroscience September 2022 Volume 13, Issue 3 276 Rose, F. D., Brooks, B. M., & Rizzo, A. A. (2005). Virtual reality in brain damage rehabilitation. Cyber Psychology & Behavior, 8(3), 241-262. https://doi.org/10.1089/cpb.2005.8.241 Ruthenbeck, G. S., & Reynolds, K. J. (2015). Virtual reality for medical training: the state-of-the-art. Journal of Simulation, 9(1), 16-26. https://doi.org/10.1057/jos.2014.14 Safi, A., Castaneda, V., Lasser, T., & Navab, N. (2010). Skin lesions classification with optical spectroscopy. In H. Liao, P. J. Edwards, X. Pan, Y. Fan & G. Z. Yang (Eds.), Proceedings of the International Workshop on Medical Imaging and Virtual Reality (pp. 411-418). https://doi.org/10.1007/978-3-642-15699- 1_43 Sketchfab. (2019). Home page. https://sketchfab.com/ Tatar, I., Huri, E., Selcuk, I., Moon, Y. L., Paoluzzi, A., & Skolarikos, A. (2019). Review of the effect of 3D medical printing and virtual reality on urology training with MedTRain3DModsim'Erasmus + European Union Project. Turkish Journal of Medical Sciences, 49(5), 1257-1270. https://doi.org/10.3906/sag-1905-73 Unity3D. (2019). Home page. https://unity.com/ Vuforia. (2019). Home page. https://developer.vuforia.com/ Walsh, K. (2015). Mobile learning in medical education. Ethiopian Journal of Health Sciences, 25(4), 363-366. https://doi.org/10.4314/ejhs.v25i4.10 Wang, F. H. (2017). An exploration of online behaviour engagement and achievement in flipped classroom supported by learning management system. Computers & Education, 114, 79-91. https://doi.org/10.1016/j.compedu.2017.06.012 Wexelblat, A. (2014). Virtual reality: Applications and explorations. Cambridge: Academic Press. https://doi.org/10.1089/cpb.2005.8.241 https://doi.org/10.1057/jos.2014.14 https://doi.org/10.1007/978-3-642-15699-1_43 https://doi.org/10.1007/978-3-642-15699-1_43 https://sketchfab.com/ https://doi.org/10.3906/sag-1905-73 https://unity.com/ https://developer.vuforia.com/ https://doi.org/10.4314/ejhs.v25i4.10 https://doi.org/10.1016/j.compedu.2017.06.012